Abstract
We analyze the moments of the accumulated reward over the interval (0,t) in a continuous-time Markov chain. We develop a numerical procedure to compute efficiently the normalized moments using the uniformization technique. Our algorithm involves auxiliary quantities whose convergence is analyzed, and for which we provide a probabilistic interpretation.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Bowerman PN, Nolty RG, Scheuer EM (1990) Calculation of the Poisson cumulative distribution function. IEEE Trans Reliab 39:158–161
Ciardo G, Marie R, Sericola B, Trivedi KS (1990) Performability analysis using semi-Markov reward processes. IEEE Trans Comput C-39:1251–1264, October
Cloth L, Haverkort BR (2006) Five performability algorithms: a comparison. In: Markov anniversary meeting, Charleston, SC, USA, Boson Books, pp 39–54
de Souza e Silva E, Gail HR (1998) An algorithm to calculate transient distributions of cumulative rate and impluse based reward. Commun Stat, Stoch Models 14(3)
de Souza e Silva E, Gail HR (2000) Transient solutions for Markov chains. In: Grassmann WK (ed) Computational probability, chapter 3. Kluwer
Donatiello L, Grassi V (1991) On evaluating the cumulative performance distribution of fault-tolerant computer systems. IEEE Trans Comput 45(11)
Iyer BR, Donatiello L, Heidelberger P (1986) Analysis of performability for stochastic models of fault-tolerant systems. IEEE Trans Comput, C-35:902–907, October
Kulkarni VG, Nicola VF, Smith RM, Trivedi KS (1986) Numerical evaluation of performability and job completion time in repairable fault-tolerant systems. In: Proceedings IEEE 16-th fault-tolerant computing symposium, Vienna, Austria, pp 252–257, July
Meyer JF (1980) On evaluating the performability of degradable computing systems. IEEE Trans Comput C-29:720–731, August
Meyer JF (1982) Closed-form solutions for performability. IEEE Trans Comput C-31:648–657, July
Nabli H, Sericola B (1996) Performability analysis: a new algorithm. IEEE Trans Comput C-45:491–494, April
Rácz S, Telek M (1999) Numerical analysis of large Markovian reward models. Perform Eval 36–37
Ross SM (1983) Stochastic processes. John Wiley and Sons
Sericola B (1999) Availability analysis of repairable computer systems and stationarity detection. IEEE Trans Comput C-48:1166–1172, November
Smith RM, Trivedi KS, Ramesh AV (1988) Performability analysis: measures, an algorithm, and a case study. IEEE Trans Comput C-37:406–417, April
Stenberg F, Manca R, Silvestrov D (2007) An algorithmic approach to discrete time non-homogeneous backward semi-Markov reward processes with an application to disability insurance. Methodol Comput Appl Probab 9(4):497–519
Telek M, Horváth A, Horváth G (2004) Analysis of inhomogeneous Markov reward models. Linear Algebra Appl 386:383–405
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Castella, F., Dujardin, G. & Sericola, B. Moments’ Analysis in Homogeneous Markov Reward Models. Methodol Comput Appl Probab 11, 583–601 (2009). https://doi.org/10.1007/s11009-008-9075-5
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11009-008-9075-5