Computing Entry-Wise Bounds of the Steady-State Distribution of a Set of Markov Chains

  • F. Ait Salaht
  • J. M. Fourneau
  • N. Pekergin
Conference paper


We present two algorithms to find the component-wise upper and lower bounds of the steady-state distribution of an ergodic Markov chain. whose transition matrix\(\mathbf{M}\) is entry-wise larger than matrix \(\mathbf{L}\). The algorithms are faster than Muntz’s approach. They are based on the polyhedral theory developed by Courtois and Semal and on a new iterative algorithm which gives bounds of the steady-state distribution at each iteration.



This work was partially supported by a grant from CNRS GdR RO 2011.


  1. 1.
    Buchholz, P.: An improved method for bounding stationary measures of finite Markov processes. Perform. Eval. 62, 349–365 (2005)CrossRefGoogle Scholar
  2. 2.
    Buchholz, P.: Bounding reward measures of markov models using the Markov decision processes. Numer. Linear Algebra Appl. 18(6), 919–930 (2011)MathSciNetCrossRefGoogle Scholar
  3. 3.
    Busic, A., Fourneau, J.-M.: Iterative component-wise bounds for the steady-state distribution of a Markov chain. Numer. Linear Algebra Appl. 18(6), 1031–1049 (2011)MathSciNetCrossRefGoogle Scholar
  4. 4.
    Courtois, P.-J., Semal, P.: Bounds for the positive eigenvectors of nonnegative matrices and for their approximations by decomposition. J. ACM 31(4), 804–825 (1984)Google Scholar
  5. 5.
    Fourneau, J.-M., Le Coz, M., Pekergin, N., Quessette, F.: An open tool to compute stochastic bounds on steady-state distributions and rewards. In: 11th International Workshop on Modeling, Analysis, and Simulation of Computer and Telecommunication Systems (MASCOTS 2003). IEEE Computer Society, Orlando, FL (2003)Google Scholar
  6. 6.
    Fourneau, J.-M., Quessette, F.: Some improvements for the computation of the steady-state distribution of a Markov chain by monotone sequences of vectors. In: ASMTA, volume 7314 of Lecture Notes in Computer Science, pp. 178–192. Springer (2012)Google Scholar
  7. 7.
    Franceschinis, G., Muntz, R.R.: Bounds for quasi-lumpable Markov chains. In: Performance ’93, pp. 223–243. Elsevier Science Publishers B. V. (1994)Google Scholar
  8. 8.
    Grassman, W., Taksar, M., Heyman, D.: Regenerative analysis and steady state distributions for Markov chains. Oper. Res. 33(5), 1107–1116 (1985)MathSciNetCrossRefGoogle Scholar
  9. 9.
    Stewart, W.: Introduction to the Numerical Solution of Markov Chains. Princeton University Press, New Jersey (1995)Google Scholar

Copyright information

© Springer-Verlag London 2013

Authors and Affiliations

  1. 1.PRiSMUniversité de Versailles-Saint-QuentinVersaillesFrance
  2. 2.LACLUniversité Paris EstCréteilFrance

Personalised recommendations