Abstract
Model based computation of optimal maintenance strategies is one of the classical applications of Markov Decision Processes. Unfortunately, a Markov Decision Process often does not capture the behavior of a component or system of components correctly because the duration of different operational phases is not exponentially distributed and the status of component is often only partially observable during operational times. The paper presents a general model for components with partially observable states and non-exponential failure, maintenance and repair times which are modeled by phase type distributions. Optimal maintenance strategies are computed using Markov decision theory. However, since the internal state of a component is not completely known, only bounds for the parameters of a Markov decision process can be computed resulting in a bounded parameters Markov decision process. For this kind of process optimal strategies can be computed assuming best, worst or average case behavior.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsNotes
- 1.
We denote a non-negative matrix \(\varvec{A}\) as substochastic, iff a stochastic matrix \(\varvec{P}\) with \(\varvec{A} \le \varvec{P}\) and \(\varvec{A} \ne \varvec{P}\) exists. Similarly we denote \(\varvec{A}\) as superstochastic, iff a stochastic matrix \(\varvec{P}\) with \(\varvec{A} \ge \varvec{P}\) and \(\varvec{A} \ne \varvec{P}\) exists.
References
Markov Maintenance Models Under Uncertainty Online Companion. http://ls4-www.cs.tu-dortmund.de/cms/de/home/dohndorf/publications/
Barron, Y., Frostig, E., Levikson, B.: Analysis of R out of N systems with several repairmen, exponential life times and phase type repair times: an algorithmic approach. Eur. J. Oper. Res. 169(1), 202–225 (2006)
Bayle, T.: Preventive maintenance strategy for data centers. White Paper 124, American Power Conversation (2009)
Buchholz, P., Dohndorf, I., Scheftelowitsch, D.: Analysis of Markov decision processes under parameter uncertainty. In: Reinecke, P., Di Marco, A. (eds.) EPEW 2017. LNCS, vol. 10497, pp. 3–18. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-66583-2_1
Buchholz, P., Kriege, J.: Markov modeling of availability and unavailability data. In: 2014 Tenth European Dependable Computing Conference, Newcastle, United Kingdom, 13–16 May 2014, pp. 94–105. IEEE Computer Society (2014)
Chan, G.K., Asgarpoor, S.: Optimum maintenance policy with Markov processes. Electr. Power Syst. Res. 76(67), 452–456 (2006)
Chen, D., Cao, Y., Trivedi, K.S., Hong, Y.: Preventive maintenance of multi-state system with phase-type failure time distribution and non-zero inspection time. Int. J. Reliab. Qual. Saf. Eng. 10(3), 323–344 (2003)
Chen, D., Trivedi, K.S.: Optimization for condition-based maintenance with semi-Markov decision process. Rel. Eng. Syst. Saf. 90(1), 25–29 (2005)
Crouzen, P., Pulungan, R.: Acyclic phase-type distributions in fault trees. In: Proceedings of the International Workshop on Performability Modeling of Computer and Communication Systems (2009)
Cumani, A.: On the canonical representation of homogeneous Markov processes modeling failure-time distributions. Micorelectronics Reliab. 22(3), 583–602 (1982)
Dekker, R., Nicolai, R.P., Kallenberg, L.: Maintenance and Markov decision models. In: Encyclopedia of Statistics in Quality and Reliability. Wiley, Chichester (2007)
Endrenyi, J., et al.: The present status of maintenance strategies and the impact of maintenance on reliability. IEEE Trans. Power Syst. 16(4), 638–646 (2001)
Faddy, M.J.: Phase-type distributions for failure times. Math. Comput. Model. 22(10–12), 63–70 (1995)
Givan, R., Leach, S.M., Dean, T.L.: Bounded-parameter Markov decision processes. Artif. Intell. 122(1–2), 71–109 (2000)
Hatoyama, Y.: Markov maintenance models with repair. Technical Report 175, Stanford University Department of Statistics (1976)
Ivy, J.S., Nembhard, H.B.: A modeling approach to maintenance decisions using statistical quality control and optimization. Qual. Reliab. Eng. Int. 21(4), 355–366 (2005)
Kline, M.B.: Suitability of the lognormal distribution for corrective maintenance repair times. Reliab. Eng. 9(2), 65–80 (1984)
Kuhn, K.D., Madanat, S.M.: Robust maintenance policies for Markovian systems under model uncertainty. Comput. Aid. Civil Infrastruct. Eng. 21(3), 171–178 (2006)
Puterman, M.L.: Markov Decision Processes. Wiley, Boca Raton (2005)
Sennott, L.I.: Average reward optimization theory for denumerable state spaces. In: Feinberg, E.A., Schwartz, A. (eds.) Handbook of Markov Decision Processes, pp. 153–172. Kluwer, Boston (2002)
Thümmler, A., Buchholz, P., Telek, M.: A novel approach for phase-type fitting with the EM algorithm. IEEE Trans. Dependable Sec. Comput. 3(3), 245–258 (2006)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer International Publishing AG
About this paper
Cite this paper
Buchholz, P., Dohndorf, I., Scheftelowitsch, D. (2018). Time-Based Maintenance Models Under Uncertainty. In: German, R., Hielscher, KS., Krieger, U. (eds) Measurement, Modelling and Evaluation of Computing Systems. MMB 2018. Lecture Notes in Computer Science(), vol 10740. Springer, Cham. https://doi.org/10.1007/978-3-319-74947-1_1
Download citation
DOI: https://doi.org/10.1007/978-3-319-74947-1_1
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-74946-4
Online ISBN: 978-3-319-74947-1
eBook Packages: Computer ScienceComputer Science (R0)