Some Improvements for the Computation of the Steady-State Distribution of a Markov Chain by Monotone Sequences of Vectors
We present several new improvements for a recently published algorithm  for computing the steady-state distribution of a finite ergodic Markov chain, which has a proved monotone convergence under some structural constraints on the matrix. We show how to accommodate infinite state space and that the structural constraints of the algorithm are consistent with Pagerank matrix. We present how to combine this algorithm with stochastic comparison theory to numerically obtain bounds and we prove a pre-processing of the matrix which allows to alleviate the structural constraints. The approaches are illustrated through several small examples.
KeywordsMarkov Chain State Space Structural Constraint Stochastic Matrix Stochastic Order
Unable to display preview. Download preview PDF.
- 1.Abu-Amsha, O., Vincent, J.M.: An algorithm to bound functionals on Markov chains with large state space. In: 4th INFORMS Conference on Telecommunications, Boca Raton, Floride, E.U, Boca Raton, Floride, E.U. INFORMS (1998)Google Scholar
- 3.Busic, A., Fourneau, J.-M.: A matrix pattern compliant strong stochastic bound. In: 2005 IEEE/IPSJ International Symposium on Applications and the Internet Workshops (SAINT 2005 Workshops), Trento, Italy, pp. 260–263. IEEE Computer Society (2005)Google Scholar
- 4.Busic, A., Fourneau, J.-M.: A toolbox for component-wise bounds for steady-state distribution of a DTMC. In: QEST 2010, Seventh International Conference on the Quantitative Evaluation of Systems, W. sburg, Viginia, USA, pp. 81–82. IEEE Computer Society (2010)Google Scholar
- 7.Dayar, T., Fourneau, J.-M., Pekergin, N., Vincent, J.-M.: Polynomials of a stochastic matrix and strong stochastic bounds. In: Markov Anniversary Meeting, Charleston, pp. 211–228. Boson Books, Raleigh, North Carolina (2006)Google Scholar
- 8.Fourneau, J.-M., Le Coz, M., Pekergin, N., Quessette, F.: An open tool to compute stochastic bounds on steady-state distributions and rewards. In: 11th International Workshop on Modeling, Analysis, and Simulation of Computer and Telecommunication Systems (MASCOTS 2003), Orlando, FL. IEEE Computer Society (2003)Google Scholar
- 10.Langville, A.N., Meyer, C.D.: Google’s PageRank and beyond - the science of search engine rankings. Princeton University Press (2006)Google Scholar
- 14.Stoyan, D.: Comparaison Methods for Queues and Other Stochastic Models. John Wiley and Sons, Berlin (1983)Google Scholar