Computing the Expected Markov Reward Rates with Stationarity Detection and Relative Error Control
By combining in a novel way the randomization method with the stationary detection technique, we develop two new algorithms for the computation of the expected reward rates of finite, irreducible Markov reward models, with control of the relative error. The first algorithm computes the expected transient reward rate and the second one computes the expected averaged reward rate. The algorithms are numerically stable. Further, it is argued that, from the point of view of run-time computational cost, for medium-sized and large Markov reward models, we can expect the algorithms to be better than the only variant of the randomization method that allows to control the relative error and better than the approach that consists in employing iteratively the currently existing algorithms that use the randomization method with stationarity detection but allow to control the absolute error. The performance of the new algorithms is illustrated by means of examples, showing that the algorithms can be not only faster but also more efficient than the alternatives in terms of run-time computational cost in relation to accuracy.
KeywordsMarkov reward model Markov chain Expected reward rate Relative error Randomization Stationarity detection
Mathematics Subject Classification (2010)60J28 60J22 65C40
Unable to display preview. Download preview PDF.
- Abramowitz M, Stegun IA (1964) Handbook of mathematical functions: with formulas, graphs, and mathematical tables. Number 55 in Applied Mathematics Series. Courier Corporation. 10th printing (December 1972), with correctionsGoogle Scholar
- Carrasco JA (2003a) Solving dependability/performability irreducible Markov models using regenerative randomization. IEEE Trans Reliab 52(3):319–329Google Scholar
- Carrasco JA (2003b) Transient analysis of rewarded continuous time Markov models by regenerative randomization with Laplace transform inversion. Comput J 46(1):84–99Google Scholar
- Carrasco JA (2015) Numerically stable methods for the computation of exit rates in Markov chains. Methodol Comput Appl Probab. To appearGoogle Scholar
- IEEE754 (1985) IEEE standard for binary floating-point arithmetic. ANSI/IEEE Std 754-1985Google Scholar
- IEEE754-2008 (2008) IEEE standard for floating-point arithmetic - redline. IEEE Std 754-2008 (Revision of IEEE Std 754-1985) - RedlineGoogle Scholar
- Stallman RM, et al. (2012) Using the GNU compiler collection. For GCC version 4.7.2. Free Software FoundationGoogle Scholar