Skip to main content
Download PDF

Geometric ergodicity and the spectral gap of non-reversible Markov chains

Probability Theory and Related Fields volume 154, pages 327–339 (2012)Cite this article

Abstract

We argue that the spectral theory of non-reversible Markov chains may often be more effectively cast within the framework of the naturally associated weighted-L ∞ space \({L_\infty^V}\) , instead of the usual Hilbert space L 2 = L 2(π), where π is the invariant measure of the chain. This observation is, in part, based on the following results. A discrete-time Markov chain with values in a general state space is geometrically ergodic if and only if its transition kernel admits a spectral gap in \({L_\infty^V}\) . If the chain is reversible, the same equivalence holds with L 2 in place of \({L_\infty^V}\) . In the absence of reversibility it fails: There are (necessarily non-reversible, geometrically ergodic) chains that admit a spectral gap in \({L_\infty^V}\) but not in L 2. Moreover, if a chain admits a spectral gap in L 2, then for any \({h\in L_2}\) there exists a Lyapunov function \({V_h\in L_1}\) such that V h dominates h and the chain admits a spectral gap in \({L_\infty^{V_h}}\) . The relationship between the size of the spectral gap in \({L_\infty^V}\) or L 2, and the rate at which the chain converges to equilibrium is also briefly discussed.

References

  1. Bradley R.C. Jr.: Information regularity and the central limit question. Rocky Mountain J. Math. 13(1), 77–97 (1983)

    MathSciNet  Article  MATH  Google Scholar 

  2. Brémaud P.: Markov chains: Gibbs fields, Monte Carlo simulation, and queues, volume 31 of Texts in Applied Mathematics. Springer-Verlag, New York (1999)

    Google Scholar 

  3. Brooks S.P., Roberts G.O.: Convergence assessment techniques for Markov chain Monte Carlo. Statistics and Computing 8, 319–335 (1998)

    Article  Google Scholar 

  4. Chen Mu-Fa.: Eigenvalues, inequalities, and ergodic theory, Probability and its Applications (New York). Springer-Verlag London Ltd., London (2005)

    Google Scholar 

  5. Conway J.B.: A course in functional analysis 2nd edn. Springer-Verlag, New York (1990)

    MATH  Google Scholar 

  6. Dellnitz M., Junge O.: On the approximation of complicated dynamical behavior. SIAM J. Numer. Anal. 36(2), 491–515 (1999)

    MathSciNet  Article  Google Scholar 

  7. Diaconis, P.: Mathematical developments from the analysis of riffle shuffling. In: Groups, combinatorics & geometry (Durham, 2001), pp. 73–97. World Sci. Publ., River Edge NJ, (2003)

  8. Diaconis P.: The Markov chain Monte Carlo revolution. Bull. Amer. Math. Soc. (N.S.) 46(2), 179–205 (2009)

    MathSciNet  Article  MATH  Google Scholar 

  9. Diaconis P., Holmes S., Neal R.M.: Analysis of a nonreversible Markov chain sampler. Ann. Appl. Probab. 10(3), 726–752 (2000)

    MathSciNet  Article  MATH  Google Scholar 

  10. Diaconis, P., Khare, K., Saloff-Coste, L.: Gibbs sampling, exponential families and orthogonal polynomials. Statist. Sci., 23(2), pp. 151–200 (2008). (With comments and a rejoinder by the authors)

    Google Scholar 

  11. Diaconis P., Saloff-Coste L.: Comparison theorems for reversible Markov chains. Ann. Appl. Probab. 3(3), 696–730 (1993)

    MathSciNet  Article  MATH  Google Scholar 

  12. Diaconis, P., Saloff-Coste, L.: What do we know about the Metropolis algorithm? J. Comput. System Sci., 57(1), pp. 20–36 (1998). 27th Annual ACM Symposium on the Theory of Computing (STOC’95) (Las Vegas, NV)

    Google Scholar 

  13. Diaconis P., Stroock D.: Geometric bounds for eigenvalues of Markov chains. Ann. Appl. Probab. 1(1), 36–61 (1991)

    MathSciNet  Article  MATH  Google Scholar 

  14. Dobrow, R.P., Fill, J.A.: The move-to-front rule for self-organizing lists with Markov dependent requests. In: Discrete probability and algorithms (Minneapolis, MN, 1993), volume 72 of IMA Vol. Math. Appl., pp. 57–80. Springer, New York, (1995)

  15. Evans W., Kenyon C., Peres Y., Schulman L.J.: Broadcasting on trees and the Ising model. Ann. Appl. Probab. 10(2), 410–433 (2000)

    MathSciNet  Article  MATH  Google Scholar 

  16. Fill J.A.: Eigenvalue bounds on convergence to stationarity for nonreversible Markov chains, with an application to the exclusion process. Ann. Appl. Probab. 1(1), 62–87 (1991)

    MathSciNet  Article  MATH  Google Scholar 

  17. Häggström O.: On the central limit theorem for geometrically ergodic Markov chains. Probab. Theory Relat. Fields 132(1), 74–82 (2005)

    Article  MATH  Google Scholar 

  18. Häggström, O.: Acknowledgement of priority concerning “On the central limit theorem for geometrically ergodic Markov chains” [Probab. Theory Relat. Fields 132(1), 74–82 (2005)]. Probab. Theory Relat. Fields 135(3):470 (2006)

  19. Huisinga, W., Meyn, S.P., Schuette, C.: Phase transitions and metastability in Markovian and molecular systems. Ann. Appl. Probab. (2001). (in press)

  20. Kontoyiannis I., Meyn S.P.: Spectral theory and limit theorems for geometrically ergodic Markov processes. Ann. Appl. Probab. 13, 304–362 (2003)

    MathSciNet  Article  MATH  Google Scholar 

  21. Kontoyiannis I., Meyn S.P.: Large deviation asymptotics and the spectral theory of multiplicatively regular Markov processes. Electron. J. Probab. 10(3), 61–123 (2005)

    MathSciNet  Google Scholar 

  22. Lawler G.F., Sokal A.D.: Bounds on the L 2 spectrum for Markov chains and Markov processes: a generalization of Cheeger’s inequality. Trans. Amer. Math. Soc. 309(2), 557–580 (1988)

    MathSciNet  MATH  Google Scholar 

  23. Lebeau, G., Diaconis, P.: Métropolis: le jour où l’étoile probabilité entra dans le champ gravitationnel de la galaxie microlocale. In: Séminaire: Équations aux D érivées Partielles. 2006–2007, Sémin. Équ. Dériv. Partielles, pages Exp. No. XIV, 13. École Polytech., Palaiseau (2007)

  24. Meyn, S.P., Tweedie, R.L.: Markov Chains and Stochastic Stability 2nd edn. Cambridge University Press, London (2009). Published in the Cambridge Mathematical Library. 1993 edition online: http://black.csl.uiuc.edu/~meyn/pages/book.html

  25. Meyn S.P., Tweedie R.L.: Computable bounds for geometric convergence rates of Markov chains. Ann. Appl. Probab. 4(4), 981–1011 (1994)

    MathSciNet  Article  MATH  Google Scholar 

  26. Montenegro R., Tetali P.: Mathematical aspects of mixing times in Markov chains. Found. Trends Theor. Comput. Sci. 1(3), 1–121 (2006)

    MathSciNet  Google Scholar 

  27. Nummelin E.: General Irreducible Markov Chains and Nonnegative Operators. Cambridge University Press, Cambridge (1984)

    Book  Google Scholar 

  28. Phatarfod R.M., Pryde A.J., Dyte D.: On the move-to-front scheme with Markov dependent requests. J. Appl. Probab. 34(3), 790–794 (1997)

    MathSciNet  Article  MATH  Google Scholar 

  29. Roberts G.O., Rosenthal J.S.: Geometric ergodicity and hybrid Markov chains. Electron. Comm. Probab. 2(2), 13–25 (1997) (electronic)

    MathSciNet  MATH  Google Scholar 

  30. Roberts, G.O., Tweedie, R.L.: Geometric L 2 and L 1 convergence are equivalent for reversible Markov chains. J. Appl. Probab. 38A, 37–41 (2001). Probability, statistics and seismology

  31. Rosenthal J.S.: Correction: Minorization conditions and convergence rates for Markov chain Monte Carlo. J. Amer. Statist. Assoc. 90(431), 1136 (1995)

    MathSciNet  Article  Google Scholar 

  32. Rosenthal J.S.: Minorization conditions and convergence rates for Markov chain Monte Carlo. J. Amer. Statist. Assoc. 90(430), 558–566 (1995)

    MathSciNet  Article  MATH  Google Scholar 

  33. Winkler G.: Image Analysis, Random Fields and Dynamic Monte Carlo Methods: A Mathematical Introduction. Springer-Verlag, Berlin (1995)

    Book  MATH  Google Scholar 

  34. Wübker, A.: L 2-spectral gaps for time discrete reversible Markov chains. Preprint (2009)

  35. Wübker, A., Kabluchko, Z.: L 2-spectral gaps, weak-reversible and very weak-reversible Markov chains. Preprint (2009)

Download references

Author information

Authors and Affiliations

  1. Department of Informatics, Athens University of Economics and Business, Patission 76, Athens, 10434, Greece

    I. Kontoyiannis

  2. Department of Electrical and Computer Engineering and the Coordinated Sciences Laboratory, University of Illinois at Urbana-Champaign, Urbana, IL, 61801, USA

    S. P. Meyn

Authors
  1. I. Kontoyiannis
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. S. P. Meyn
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to I. Kontoyiannis.

Additional information

I. Kontoyiannis was supported in part by a Sloan Foundation Research Fellowship, and by a Marie Curie International Outgoing Fellowship, PIOF-GA-2009-235837. S.P. Meyn was supported in part by the National Science Foundation ECS-0523620, and by AFOSR grant FA9550-09-1-0190. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the National Science Foundation.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Kontoyiannis, I., Meyn, S.P. Geometric ergodicity and the spectral gap of non-reversible Markov chains. Probab. Theory Relat. Fields 154, 327–339 (2012). https://doi.org/10.1007/s00440-011-0373-4

Download citation

  • Received:

  • Revised:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00440-011-0373-4

Keywords

  • Markov chain
  • Geometric ergodicity
  • Spectral theory
  • Stochastic Lyapunov function
  • Reversibility
  • Spectral gap

Mathematics Subject Classification (2000)

  • 60J05
  • 60J10
  • 37A30
  • 37A25