Time-Based Maintenance Models Under Uncertainty

Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10740)


Model based computation of optimal maintenance strategies is one of the classical applications of Markov Decision Processes. Unfortunately, a Markov Decision Process often does not capture the behavior of a component or system of components correctly because the duration of different operational phases is not exponentially distributed and the status of component is often only partially observable during operational times. The paper presents a general model for components with partially observable states and non-exponential failure, maintenance and repair times which are modeled by phase type distributions. Optimal maintenance strategies are computed using Markov decision theory. However, since the internal state of a component is not completely known, only bounds for the parameters of a Markov decision process can be computed resulting in a bounded parameters Markov decision process. For this kind of process optimal strategies can be computed assuming best, worst or average case behavior.


Maintenance models Markov decision processes Stochastic dynamic programming Numerical methods 


  1. 1.
    Markov Maintenance Models Under Uncertainty Online Companion.
  2. 2.
    Barron, Y., Frostig, E., Levikson, B.: Analysis of R out of N systems with several repairmen, exponential life times and phase type repair times: an algorithmic approach. Eur. J. Oper. Res. 169(1), 202–225 (2006)MathSciNetCrossRefzbMATHGoogle Scholar
  3. 3.
    Bayle, T.: Preventive maintenance strategy for data centers. White Paper 124, American Power Conversation (2009)Google Scholar
  4. 4.
    Buchholz, P., Dohndorf, I., Scheftelowitsch, D.: Analysis of Markov decision processes under parameter uncertainty. In: Reinecke, P., Di Marco, A. (eds.) EPEW 2017. LNCS, vol. 10497, pp. 3–18. Springer, Cham (2017). CrossRefGoogle Scholar
  5. 5.
    Buchholz, P., Kriege, J.: Markov modeling of availability and unavailability data. In: 2014 Tenth European Dependable Computing Conference, Newcastle, United Kingdom, 13–16 May 2014, pp. 94–105. IEEE Computer Society (2014)Google Scholar
  6. 6.
    Chan, G.K., Asgarpoor, S.: Optimum maintenance policy with Markov processes. Electr. Power Syst. Res. 76(67), 452–456 (2006)CrossRefGoogle Scholar
  7. 7.
    Chen, D., Cao, Y., Trivedi, K.S., Hong, Y.: Preventive maintenance of multi-state system with phase-type failure time distribution and non-zero inspection time. Int. J. Reliab. Qual. Saf. Eng. 10(3), 323–344 (2003)CrossRefGoogle Scholar
  8. 8.
    Chen, D., Trivedi, K.S.: Optimization for condition-based maintenance with semi-Markov decision process. Rel. Eng. Syst. Saf. 90(1), 25–29 (2005)CrossRefGoogle Scholar
  9. 9.
    Crouzen, P., Pulungan, R.: Acyclic phase-type distributions in fault trees. In: Proceedings of the International Workshop on Performability Modeling of Computer and Communication Systems (2009)Google Scholar
  10. 10.
    Cumani, A.: On the canonical representation of homogeneous Markov processes modeling failure-time distributions. Micorelectronics Reliab. 22(3), 583–602 (1982)MathSciNetCrossRefGoogle Scholar
  11. 11.
    Dekker, R., Nicolai, R.P., Kallenberg, L.: Maintenance and Markov decision models. In: Encyclopedia of Statistics in Quality and Reliability. Wiley, Chichester (2007)Google Scholar
  12. 12.
    Endrenyi, J., et al.: The present status of maintenance strategies and the impact of maintenance on reliability. IEEE Trans. Power Syst. 16(4), 638–646 (2001)CrossRefGoogle Scholar
  13. 13.
    Faddy, M.J.: Phase-type distributions for failure times. Math. Comput. Model. 22(10–12), 63–70 (1995)CrossRefzbMATHGoogle Scholar
  14. 14.
    Givan, R., Leach, S.M., Dean, T.L.: Bounded-parameter Markov decision processes. Artif. Intell. 122(1–2), 71–109 (2000)MathSciNetCrossRefzbMATHGoogle Scholar
  15. 15.
    Hatoyama, Y.: Markov maintenance models with repair. Technical Report 175, Stanford University Department of Statistics (1976)Google Scholar
  16. 16.
    Ivy, J.S., Nembhard, H.B.: A modeling approach to maintenance decisions using statistical quality control and optimization. Qual. Reliab. Eng. Int. 21(4), 355–366 (2005)CrossRefGoogle Scholar
  17. 17.
    Kline, M.B.: Suitability of the lognormal distribution for corrective maintenance repair times. Reliab. Eng. 9(2), 65–80 (1984)CrossRefGoogle Scholar
  18. 18.
    Kuhn, K.D., Madanat, S.M.: Robust maintenance policies for Markovian systems under model uncertainty. Comput. Aid. Civil Infrastruct. Eng. 21(3), 171–178 (2006)CrossRefGoogle Scholar
  19. 19.
    Puterman, M.L.: Markov Decision Processes. Wiley, Boca Raton (2005)zbMATHGoogle Scholar
  20. 20.
    Sennott, L.I.: Average reward optimization theory for denumerable state spaces. In: Feinberg, E.A., Schwartz, A. (eds.) Handbook of Markov Decision Processes, pp. 153–172. Kluwer, Boston (2002)Google Scholar
  21. 21.
    Thümmler, A., Buchholz, P., Telek, M.: A novel approach for phase-type fitting with the EM algorithm. IEEE Trans. Dependable Sec. Comput. 3(3), 245–258 (2006)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2018

Authors and Affiliations

  1. 1.Informatik IVTU DortmundDortmundGermany

Personalised recommendations