Some Improvements for the Computation of the Steady-State Distribution of a Markov Chain by Monotone Sequences of Vectors

  • Jean-Michel Fourneau
  • Franck Quessette
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7314)


We present several new improvements for a recently published algorithm [5] for computing the steady-state distribution of a finite ergodic Markov chain, which has a proved monotone convergence under some structural constraints on the matrix. We show how to accommodate infinite state space and that the structural constraints of the algorithm are consistent with Pagerank matrix. We present how to combine this algorithm with stochastic comparison theory to numerically obtain bounds and we prove a pre-processing of the matrix which allows to alleviate the structural constraints. The approaches are illustrated through several small examples.


Markov Chain State Space Structural Constraint Stochastic Matrix Stochastic Order 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Abu-Amsha, O., Vincent, J.M.: An algorithm to bound functionals on Markov chains with large state space. In: 4th INFORMS Conference on Telecommunications, Boca Raton, Floride, E.U, Boca Raton, Floride, E.U. INFORMS (1998)Google Scholar
  2. 2.
    Berman, A., Plemmons, R.J.: Nonnegative Matrices in the Mathematical Sciences. Academic Press, NewYork (1994)zbMATHCrossRefGoogle Scholar
  3. 3.
    Busic, A., Fourneau, J.-M.: A matrix pattern compliant strong stochastic bound. In: 2005 IEEE/IPSJ International Symposium on Applications and the Internet Workshops (SAINT 2005 Workshops), Trento, Italy, pp. 260–263. IEEE Computer Society (2005)Google Scholar
  4. 4.
    Busic, A., Fourneau, J.-M.: A toolbox for component-wise bounds for steady-state distribution of a DTMC. In: QEST 2010, Seventh International Conference on the Quantitative Evaluation of Systems, W. sburg, Viginia, USA, pp. 81–82. IEEE Computer Society (2010)Google Scholar
  5. 5.
    Busic, A., Fourneau, J.-M.: Iterative component-wise bounds for the steady-state distribution of a markov chain. Numerical Linear Algebra with Applications 18(6), 1031–1049 (2011)MathSciNetCrossRefGoogle Scholar
  6. 6.
    Dayar, T., Fourneau, J.-M., Pekergin, N.: Transforming stochastic matrices for stochastic comparison with the st-order. RAIRO Operations Research 37, 85–97 (2003)MathSciNetzbMATHCrossRefGoogle Scholar
  7. 7.
    Dayar, T., Fourneau, J.-M., Pekergin, N., Vincent, J.-M.: Polynomials of a stochastic matrix and strong stochastic bounds. In: Markov Anniversary Meeting, Charleston, pp. 211–228. Boson Books, Raleigh, North Carolina (2006)Google Scholar
  8. 8.
    Fourneau, J.-M., Le Coz, M., Pekergin, N., Quessette, F.: An open tool to compute stochastic bounds on steady-state distributions and rewards. In: 11th International Workshop on Modeling, Analysis, and Simulation of Computer and Telecommunication Systems (MASCOTS 2003), Orlando, FL. IEEE Computer Society (2003)Google Scholar
  9. 9.
    Fourneau, J.-M., Pekergin, N.: An Algorithmic Approach to Stochastic Bounds. In: Calzarossa, M.C., Tucci, S. (eds.) Performance 2002. LNCS, vol. 2459, pp. 64–88. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  10. 10.
    Langville, A.N., Meyer, C.D.: Google’s PageRank and beyond - the science of search engine rankings. Princeton University Press (2006)Google Scholar
  11. 11.
    Semal, P.: Refinable bounds for large Markov chains. IEEE Trans. on Computers 44(10), 1216–1222 (1995)zbMATHCrossRefGoogle Scholar
  12. 12.
    Shaked, M., Shantikumar, J.G.: Stochastic Orders and their Applications. Academic Press, San Diego (1994)zbMATHGoogle Scholar
  13. 13.
    Song, Y.: Monotone convergence of iterative methods for singular linear systems. BIT Numerical Mathematics 42, 611–624 (2002)MathSciNetzbMATHCrossRefGoogle Scholar
  14. 14.
    Stoyan, D.: Comparaison Methods for Queues and Other Stochastic Models. John Wiley and Sons, Berlin (1983)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Jean-Michel Fourneau
    • 1
  • Franck Quessette
    • 1
  1. 1.PRiSMUniversité de Versailles-Saint-Quentin, CNRS UMR 8144, UniverSudVersaillesFrance

Personalised recommendations