Continuous-Observation Partially Observable Semi-Markov Decision Processes for Machine Maintenance
Item Type:Journal Article
Citation:Zhang, M. & Revie, M. Continuous-Observation Partially Observable Semi-Markov Decision Processes for Machine Maintenance, IEEE Transactions on Reliability, 66 (1), 2017, 202 - 218
Partially observable semi-Markov decision processes (POSMDPs) provide a rich framework for planning under both state transition uncertainty and observation uncertainty. In this paper, we widen the literature on POSMDP by studying discrete-state discrete-action yet continuous-observation POSMDPs. We prove that the resultant α-vector set is continuous and, therefore, propose a point-based value iteration algorithm. This paper also bridges the gap between POSMDP and machine maintenance by incorporating various types of maintenance actions, such as actions changing machine state, actions changing degradation rate, and the temporally extended action “do nothing.” Both finite and infinite planning horizons are reviewed, and the solution methodology for each type of planning horizon is given. We illustrate the maintenance decision process via a real industrial problem and demonstrate that the developed framework can be readily applied to solve relevant maintenance problems.
Type of material:Journal Article
Series/Report no:IEEE Transactions on Reliability;
Availability:Full text available