Skip to main content

Time-Based Maintenance Models Under Uncertainty

Part of the Lecture Notes in Computer Science book series (LNPSE,volume 10740)


Model based computation of optimal maintenance strategies is one of the classical applications of Markov Decision Processes. Unfortunately, a Markov Decision Process often does not capture the behavior of a component or system of components correctly because the duration of different operational phases is not exponentially distributed and the status of component is often only partially observable during operational times. The paper presents a general model for components with partially observable states and non-exponential failure, maintenance and repair times which are modeled by phase type distributions. Optimal maintenance strategies are computed using Markov decision theory. However, since the internal state of a component is not completely known, only bounds for the parameters of a Markov decision process can be computed resulting in a bounded parameters Markov decision process. For this kind of process optimal strategies can be computed assuming best, worst or average case behavior.


  • Maintenance models
  • Markov decision processes
  • Stochastic dynamic programming
  • Numerical methods

This is a preview of subscription content, access via your institution.

Buying options

USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
USD   39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions


  1. 1.

    We denote a non-negative matrix \(\varvec{A}\) as substochastic, iff a stochastic matrix \(\varvec{P}\) with \(\varvec{A} \le \varvec{P}\) and \(\varvec{A} \ne \varvec{P}\) exists. Similarly we denote \(\varvec{A}\) as superstochastic, iff a stochastic matrix \(\varvec{P}\) with \(\varvec{A} \ge \varvec{P}\) and \(\varvec{A} \ne \varvec{P}\) exists.


  1. Markov Maintenance Models Under Uncertainty Online Companion.

  2. Barron, Y., Frostig, E., Levikson, B.: Analysis of R out of N systems with several repairmen, exponential life times and phase type repair times: an algorithmic approach. Eur. J. Oper. Res. 169(1), 202–225 (2006)

    CrossRef  MathSciNet  MATH  Google Scholar 

  3. Bayle, T.: Preventive maintenance strategy for data centers. White Paper 124, American Power Conversation (2009)

    Google Scholar 

  4. Buchholz, P., Dohndorf, I., Scheftelowitsch, D.: Analysis of Markov decision processes under parameter uncertainty. In: Reinecke, P., Di Marco, A. (eds.) EPEW 2017. LNCS, vol. 10497, pp. 3–18. Springer, Cham (2017).

    CrossRef  Google Scholar 

  5. Buchholz, P., Kriege, J.: Markov modeling of availability and unavailability data. In: 2014 Tenth European Dependable Computing Conference, Newcastle, United Kingdom, 13–16 May 2014, pp. 94–105. IEEE Computer Society (2014)

    Google Scholar 

  6. Chan, G.K., Asgarpoor, S.: Optimum maintenance policy with Markov processes. Electr. Power Syst. Res. 76(67), 452–456 (2006)

    CrossRef  Google Scholar 

  7. Chen, D., Cao, Y., Trivedi, K.S., Hong, Y.: Preventive maintenance of multi-state system with phase-type failure time distribution and non-zero inspection time. Int. J. Reliab. Qual. Saf. Eng. 10(3), 323–344 (2003)

    CrossRef  Google Scholar 

  8. Chen, D., Trivedi, K.S.: Optimization for condition-based maintenance with semi-Markov decision process. Rel. Eng. Syst. Saf. 90(1), 25–29 (2005)

    CrossRef  Google Scholar 

  9. Crouzen, P., Pulungan, R.: Acyclic phase-type distributions in fault trees. In: Proceedings of the International Workshop on Performability Modeling of Computer and Communication Systems (2009)

    Google Scholar 

  10. Cumani, A.: On the canonical representation of homogeneous Markov processes modeling failure-time distributions. Micorelectronics Reliab. 22(3), 583–602 (1982)

    CrossRef  MathSciNet  Google Scholar 

  11. Dekker, R., Nicolai, R.P., Kallenberg, L.: Maintenance and Markov decision models. In: Encyclopedia of Statistics in Quality and Reliability. Wiley, Chichester (2007)

    Google Scholar 

  12. Endrenyi, J., et al.: The present status of maintenance strategies and the impact of maintenance on reliability. IEEE Trans. Power Syst. 16(4), 638–646 (2001)

    CrossRef  Google Scholar 

  13. Faddy, M.J.: Phase-type distributions for failure times. Math. Comput. Model. 22(10–12), 63–70 (1995)

    CrossRef  MATH  Google Scholar 

  14. Givan, R., Leach, S.M., Dean, T.L.: Bounded-parameter Markov decision processes. Artif. Intell. 122(1–2), 71–109 (2000)

    CrossRef  MathSciNet  MATH  Google Scholar 

  15. Hatoyama, Y.: Markov maintenance models with repair. Technical Report 175, Stanford University Department of Statistics (1976)

    Google Scholar 

  16. Ivy, J.S., Nembhard, H.B.: A modeling approach to maintenance decisions using statistical quality control and optimization. Qual. Reliab. Eng. Int. 21(4), 355–366 (2005)

    CrossRef  Google Scholar 

  17. Kline, M.B.: Suitability of the lognormal distribution for corrective maintenance repair times. Reliab. Eng. 9(2), 65–80 (1984)

    CrossRef  Google Scholar 

  18. Kuhn, K.D., Madanat, S.M.: Robust maintenance policies for Markovian systems under model uncertainty. Comput. Aid. Civil Infrastruct. Eng. 21(3), 171–178 (2006)

    CrossRef  Google Scholar 

  19. Puterman, M.L.: Markov Decision Processes. Wiley, Boca Raton (2005)

    MATH  Google Scholar 

  20. Sennott, L.I.: Average reward optimization theory for denumerable state spaces. In: Feinberg, E.A., Schwartz, A. (eds.) Handbook of Markov Decision Processes, pp. 153–172. Kluwer, Boston (2002)

    Google Scholar 

  21. Thümmler, A., Buchholz, P., Telek, M.: A novel approach for phase-type fitting with the EM algorithm. IEEE Trans. Dependable Sec. Comput. 3(3), 245–258 (2006)

    CrossRef  Google Scholar 

Download references

Author information

Authors and Affiliations


Corresponding author

Correspondence to Iryna Dohndorf .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and Permissions

Copyright information

© 2018 Springer International Publishing AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Buchholz, P., Dohndorf, I., Scheftelowitsch, D. (2018). Time-Based Maintenance Models Under Uncertainty. In: German, R., Hielscher, KS., Krieger, U. (eds) Measurement, Modelling and Evaluation of Computing Systems. MMB 2018. Lecture Notes in Computer Science(), vol 10740. Springer, Cham.

Download citation

  • DOI:

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-74946-4

  • Online ISBN: 978-3-319-74947-1

  • eBook Packages: Computer ScienceComputer Science (R0)