Skip to main content

Time-Based Maintenance Models Under Uncertainty

Part of the Lecture Notes in Computer Science book series (LNPSE,volume 10740)

Abstract

Model based computation of optimal maintenance strategies is one of the classical applications of Markov Decision Processes. Unfortunately, a Markov Decision Process often does not capture the behavior of a component or system of components correctly because the duration of different operational phases is not exponentially distributed and the status of component is often only partially observable during operational times. The paper presents a general model for components with partially observable states and non-exponential failure, maintenance and repair times which are modeled by phase type distributions. Optimal maintenance strategies are computed using Markov decision theory. However, since the internal state of a component is not completely known, only bounds for the parameters of a Markov decision process can be computed resulting in a bounded parameters Markov decision process. For this kind of process optimal strategies can be computed assuming best, worst or average case behavior.

Keywords

  • Maintenance models
  • Markov decision processes
  • Stochastic dynamic programming
  • Numerical methods

This is a preview of subscription content, access via your institution.

Buying options

Chapter
USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-3-319-74947-1_1
  • Chapter length: 16 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
eBook
USD   64.99
Price excludes VAT (USA)
  • ISBN: 978-3-319-74947-1
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Softcover Book
USD   84.00
Price excludes VAT (USA)
Fig. 1.
Fig. 2.
Fig. 3.
Fig. 4.
Fig. 5.

Notes

  1. 1.

    We denote a non-negative matrix \(\varvec{A}\) as substochastic, iff a stochastic matrix \(\varvec{P}\) with \(\varvec{A} \le \varvec{P}\) and \(\varvec{A} \ne \varvec{P}\) exists. Similarly we denote \(\varvec{A}\) as superstochastic, iff a stochastic matrix \(\varvec{P}\) with \(\varvec{A} \ge \varvec{P}\) and \(\varvec{A} \ne \varvec{P}\) exists.

References

  1. Markov Maintenance Models Under Uncertainty Online Companion. http://ls4-www.cs.tu-dortmund.de/cms/de/home/dohndorf/publications/

  2. Barron, Y., Frostig, E., Levikson, B.: Analysis of R out of N systems with several repairmen, exponential life times and phase type repair times: an algorithmic approach. Eur. J. Oper. Res. 169(1), 202–225 (2006)

    MathSciNet  CrossRef  MATH  Google Scholar 

  3. Bayle, T.: Preventive maintenance strategy for data centers. White Paper 124, American Power Conversation (2009)

    Google Scholar 

  4. Buchholz, P., Dohndorf, I., Scheftelowitsch, D.: Analysis of Markov decision processes under parameter uncertainty. In: Reinecke, P., Di Marco, A. (eds.) EPEW 2017. LNCS, vol. 10497, pp. 3–18. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-66583-2_1

    CrossRef  Google Scholar 

  5. Buchholz, P., Kriege, J.: Markov modeling of availability and unavailability data. In: 2014 Tenth European Dependable Computing Conference, Newcastle, United Kingdom, 13–16 May 2014, pp. 94–105. IEEE Computer Society (2014)

    Google Scholar 

  6. Chan, G.K., Asgarpoor, S.: Optimum maintenance policy with Markov processes. Electr. Power Syst. Res. 76(67), 452–456 (2006)

    CrossRef  Google Scholar 

  7. Chen, D., Cao, Y., Trivedi, K.S., Hong, Y.: Preventive maintenance of multi-state system with phase-type failure time distribution and non-zero inspection time. Int. J. Reliab. Qual. Saf. Eng. 10(3), 323–344 (2003)

    CrossRef  Google Scholar 

  8. Chen, D., Trivedi, K.S.: Optimization for condition-based maintenance with semi-Markov decision process. Rel. Eng. Syst. Saf. 90(1), 25–29 (2005)

    CrossRef  Google Scholar 

  9. Crouzen, P., Pulungan, R.: Acyclic phase-type distributions in fault trees. In: Proceedings of the International Workshop on Performability Modeling of Computer and Communication Systems (2009)

    Google Scholar 

  10. Cumani, A.: On the canonical representation of homogeneous Markov processes modeling failure-time distributions. Micorelectronics Reliab. 22(3), 583–602 (1982)

    MathSciNet  CrossRef  Google Scholar 

  11. Dekker, R., Nicolai, R.P., Kallenberg, L.: Maintenance and Markov decision models. In: Encyclopedia of Statistics in Quality and Reliability. Wiley, Chichester (2007)

    Google Scholar 

  12. Endrenyi, J., et al.: The present status of maintenance strategies and the impact of maintenance on reliability. IEEE Trans. Power Syst. 16(4), 638–646 (2001)

    CrossRef  Google Scholar 

  13. Faddy, M.J.: Phase-type distributions for failure times. Math. Comput. Model. 22(10–12), 63–70 (1995)

    CrossRef  MATH  Google Scholar 

  14. Givan, R., Leach, S.M., Dean, T.L.: Bounded-parameter Markov decision processes. Artif. Intell. 122(1–2), 71–109 (2000)

    MathSciNet  CrossRef  MATH  Google Scholar 

  15. Hatoyama, Y.: Markov maintenance models with repair. Technical Report 175, Stanford University Department of Statistics (1976)

    Google Scholar 

  16. Ivy, J.S., Nembhard, H.B.: A modeling approach to maintenance decisions using statistical quality control and optimization. Qual. Reliab. Eng. Int. 21(4), 355–366 (2005)

    CrossRef  Google Scholar 

  17. Kline, M.B.: Suitability of the lognormal distribution for corrective maintenance repair times. Reliab. Eng. 9(2), 65–80 (1984)

    CrossRef  Google Scholar 

  18. Kuhn, K.D., Madanat, S.M.: Robust maintenance policies for Markovian systems under model uncertainty. Comput. Aid. Civil Infrastruct. Eng. 21(3), 171–178 (2006)

    CrossRef  Google Scholar 

  19. Puterman, M.L.: Markov Decision Processes. Wiley, Boca Raton (2005)

    MATH  Google Scholar 

  20. Sennott, L.I.: Average reward optimization theory for denumerable state spaces. In: Feinberg, E.A., Schwartz, A. (eds.) Handbook of Markov Decision Processes, pp. 153–172. Kluwer, Boston (2002)

    Google Scholar 

  21. Thümmler, A., Buchholz, P., Telek, M.: A novel approach for phase-type fitting with the EM algorithm. IEEE Trans. Dependable Sec. Comput. 3(3), 245–258 (2006)

    CrossRef  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Iryna Dohndorf .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and Permissions

Copyright information

© 2018 Springer International Publishing AG

About this paper

Verify currency and authenticity via CrossMark

Cite this paper

Buchholz, P., Dohndorf, I., Scheftelowitsch, D. (2018). Time-Based Maintenance Models Under Uncertainty. In: German, R., Hielscher, KS., Krieger, U. (eds) Measurement, Modelling and Evaluation of Computing Systems. MMB 2018. Lecture Notes in Computer Science(), vol 10740. Springer, Cham. https://doi.org/10.1007/978-3-319-74947-1_1

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-74947-1_1

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-74946-4

  • Online ISBN: 978-3-319-74947-1

  • eBook Packages: Computer ScienceComputer Science (R0)