Recurrence and Transience of Markov Chains

  • Philippe Robert
Part of the Applications of Mathematics book series (SMAP, volume 52)


The general problem of convergence in distribution of Markov chains with a countable state space is investigated in this chapter. For the GI/GI/1 queue in Chapter 2, the convergence in distribution of the Markov chain (W n ) has been obtained by using an explicit representation of the random variable W n as a functional of the random walk associated with the interarrival times and the service times. Markov chains describing the behavior of most queueing systems cannot, in general, be represented in such a simple way. In this chapter, simple criteria are given to determine whether a given Markov chain is ergodic or transient. The main results are Theorem 8.6 for ergodicity and Theorem 8.10 for transience. In practice, these results can be used in many applications. These criteria can be seen as an extension, in a probabilistic setting, of a classical stability result of ordinary differential equations due to Lyapunov[Lia07] in 1892. In a stochastic context, the first results of this type are apparently due to Khasminskii for diffusions (see Khasminskii [26]). The stability criterion by Lyapunov is recalled at the end of the chapter (see Hirsch and Smale [27] for a detailed presentation of these questions). In Chapter 9, an important scaling method is introduced that will give a more precise picture of the relation between the stability of ordinary differential equations and the stability properties of Markov


Markov Chain Markov Process Lyapunov Function Markov Property Strong Markov Property 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. [Lia07]
    A.M. Liapunov, Problème général de la stabilité du mouvement, Annales de la Faculté des Sciences de lUniversité de Toulouse 9 (1907), 203–475.Google Scholar
  2. [Fil89]
    Y. Filonov, A criterion for the ergodicity of discrete homogeneous Markov chains, Akademiya nauk Ukrainskoi SSR. Institut Matematiki. Ukrainskii Matematicheskii Zhurual 41 (1989), no. 10, 1421–1422.Google Scholar
  3. [Lam60]
    J. Lamperti, Criteria foe the recurrence or transience of stochastic process I., Journal of Mathematical Analysis and Applications 1 (1960), 314–330.Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2003

Authors and Affiliations

  • Philippe Robert
    • 1
  1. 1.Domaine de VoluceauINRIALe ChesnayFrance

Personalised recommendations