An MDP Approach for Intervention Planning of Partially Observable Systems Prone to Failures

Önder Bulut, Department of Industrial Engineering, Yaşar University, Izmir

Abstract

This talk will be an overview of our Markov Decision Process (MDP) approach developed for the intervention/maintenance planning of the systems where the deterioration levels of the components are partially observable.

In the setting we consider in this talk, each component deteriorates over time by making transitions among a finite number of levels from 0 to K where 0 represents the level at which the component works perfectly and K represents the level at which the component has failed and requires replacement. An intervention is required whenever at least one of the components is at level K. At other degradation levels, intervention is based on the decision of the controller. An intervention always restores all components to perfect working conditions. The full information on the exact deterioration levels of components can only be observed through on-site inspection. However, the system has a sensor that provides partial information via a system-wide, three-level signal. The sensor transmits the signals periodically. Intervention decisions are based on the partial information by these signals. We assume that the system operates with jidoka (autonomation) principle where all components automatically stop working upon an intervention decision. The system-wide stoppage creates an opportunity for all components for restoration.

Relevant literature proposes Partially Observable Markov Decision Process (POMDP) models with continuous state variables. Our MDP model, on the other hand, is based on a single discrete state-variable. We use the time index of our regenerative discrete-time model to keep track of the other necessary system information.

Önder Bulut is currently an associate professor in the Department of Industrial Engineering, Yaşar University, Izmir. Dr. Bulut received his BS, MS and PhD degrees from Bilkent University, Department of Industrial Engineering. His main research area is stochastic modeling with production, inventory, energy, reliability and revenue management applications.

Venue

Friday, May 5, 2023, 4.00 pm - Zoom