Skip to main content
SpringerLink
Log in

Metastable Distributions of Markov Chains with Rare Transitions

  • Published:
Journal of Statistical Physics Aims and scope Submit manuscript

Abstract

In this paper we consider Markov chains \(X^\varepsilon _t\) with transition rates that depend on a small parameter \(\varepsilon \). We are interested in the long time behavior of \(X^\varepsilon _t\) at various \(\varepsilon \)-dependent time scales \(t = t(\varepsilon )\). The asymptotic behavior depends on how the point \((1/\varepsilon , t(\varepsilon ))\) approaches infinity. We introduce a general notion of complete asymptotic regularity (a certain asymptotic relation between the ratios of transition rates), which ensures the existence of the metastable distribution for each initial point and a given time scale \(t(\varepsilon )\). The technique of i-graphs allows one to describe the metastable distribution explicitly. The result may be viewed as a generalization of the ergodic theorem to the case of parameter-dependent Markov chains.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

Notes

  1. The main result can be obtained even if the positivity assumption is relaxed, as mentioned in Sect. 6.

  2. Formula (3.3) can be improved to \({\mathrm {P}_i}({X}^\varepsilon _{t(\varepsilon )} = j ) \sim \mu (j, \varepsilon )~\mathrm{as}~\varepsilon \downarrow 0\), but we don’t need it here.

References

  1. Aldous, D., Brown, M.: Inequalities for rare events in time-reversible Markov chains. I. Stochastic inequalities (Seattle, WA, 1991), 1–16, IMS Lecture Notes Monogr. Ser., 22, Inst. Math. Statist., Hayward, CA (1992)

  2. Aldous, D., Brown, M.: Inequalities for rare events in time-reversible Markov chains II. Stoch. Process. Appl. 44(1), 15–25 (1993)

    Article  MathSciNet  MATH  Google Scholar 

  3. Bakhtin, Y., Pajor-Gyulai, Z.: Metastability and cycle structure in strictly attracting noisy heteroclinic networks (in preparation)

  4. Beltran, J., Landim, C.: Tunneling and metastability of continuous time Markov chains. J. Stat. Phys. 140(6), 1065–1114 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  5. Beltran, J., Landim, C.: A martingale approach to metastability. Probab. Theory Relat. Fields 161(1–2), 267–307 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  6. Berglund, N., Gentz, B.: The Eyring–Kramers law for potentials with nonquadratic saddles. Markov Process. Relat. Fields 16(3), 549–598 (2010)

    MathSciNet  MATH  Google Scholar 

  7. Betz, V., Le Roux, S.: Multi-scale metastable dynamics and the asymptotic stationary distribution of perturbed Markov chains. Stoch. Process. Appl. 126(11), 3499–3526 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  8. Bianchi, A., Gaudilliere, A.: Metastable states, quasi-stationary distributions and soft measures. Stoch. Process. Appl. 126(6), 1622–1680 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  9. Bovier, A., Eckhoff, M., Gayrard, V., Klein, M.: Metastability in stochastic dynamics of disordered meanfield models. Probab. Theory Relat. Fields 119(1), 99–161 (2001)

    Article  MATH  Google Scholar 

  10. Bovier, A., Eckhoff, M., Gayrard, V., Klein, M.: Metastability in reversible diffusion processes. I. Sharp asymptotics for capacities and exit times. J. Eur. Math. Soc. 6(4), 399–424 (2004)

    Article  MathSciNet  MATH  Google Scholar 

  11. Bovier, A., Eckhoff, M., Gayrard, V., Klein, M.: Metastability and low lying spectra in reversible Markov chains. Commun. Math. Phys. 228, 219–255 (2002)

    Article  ADS  MathSciNet  MATH  Google Scholar 

  12. Bovier, A., den Hollander, F.: Metastability. A Potential-Theoretical Approach. Springer, New York (2015)

    MATH  Google Scholar 

  13. Catoni O.: Simulated annealing algorithms and Markov chains with rare transitions. In: Seminaire de Probabilites, XXXIII, volume 1709 of Lecture Notes in Math., pp. 69–119. Springer, Berlin (1999)

  14. Catoni, O., Cerf, R.: The exit path of a Markov chain with rare transitions. ESAIM Probab. Stat. 1: 95–144 (1995/1997)

  15. Cirillo, E., Nardi, F.: Metastability for a stochastic dynamics with a parallel heat bath updating rule. J. Stat. Phys. 110(1–2), 183–217 (2003)

    Article  MathSciNet  MATH  Google Scholar 

  16. Cirillo, E., Nardi, F.: Relaxation height in energy landscapes: an application to multiple metastable states. J. Stat. Phys. 150(6), 1080–1114 (2013)

    Article  ADS  MathSciNet  MATH  Google Scholar 

  17. Cirillo, E., Nardi, F., Sohier, J.: Metastability for general dynamics with rare transitions: escape time and critical configurations. J. Stat. Phys. 161, 365–403 (2015)

    Article  ADS  MathSciNet  MATH  Google Scholar 

  18. Doob, J.L.: Stochastic Processes. Wiley-Interscience. Revised edition (January 25, 1990)

  19. Eckhoff, M.: Precise asymptotics of small eigenvalues of reversible diffusions in the metastable regime. Ann. Prob. 33(1), 244–299 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  20. Fernandez, R., Manzo, F., Nardi, F., Scoppola, E.: Asymptotically exponential hitting times and metastability: a pathwise approach without reversibility. Electron. J. Probab. 20(122), 1–37 (2015)

    MathSciNet  MATH  Google Scholar 

  21. Fernandez, R., Manzo, F., Nardi, F., Scoppola, E., Sohier, J.: Conditioned, quasi-stationary restricted measures and escape from metastable states. Ann. Appl. Prob. 26, 760–793 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  22. Freidlin, M.I.: Sublimiting distributions and stabilization of solutions of parabolic equations with a small parameter. Soviet Math. Dokl. 18(4), 1114–1118 (1977)

    MATH  Google Scholar 

  23. Freidlin, M.I.: On stochastic perturbations of systems with rough symmetry. Hierarchy of Markov chains. J. Stat. Phys. 157(6), 1031–1045 (2014)

    Article  ADS  MathSciNet  MATH  Google Scholar 

  24. Freidlin, M.I., Wentzell, A.D.: Random Perturbations of Dynamical Systems, 3rd edn. Springer, Berlin (2012)

    Book  MATH  Google Scholar 

  25. Freidlin, M.I., Koralov, L., Wentzell, A.D.: On the behavior of diffusion processes with traps. To appear in Annals of Probability

  26. Hennion, H., Herve, L.: Limit Theorems for Markov Chains and Stochastic Properties of Dynamical Systems by Quasi-Compactness. Springer, Berlin (2001)

    Book  MATH  Google Scholar 

  27. Holmes-Cerfon, M., Gortler, S.J., Brenner, M.P.: A geometrical approach to computing free-energy landscapes from short-ranged potentials. Proc. Natl. Acad. Sci. 110(1), E5–E14 (2013)

    Article  ADS  Google Scholar 

  28. Huang, C., Sheu, S.: Singular perturbed Markov chains and exact behaviors of simulated annealing processes. J. Theor. Probab. 5(2), 223–249 (1992)

    Article  MathSciNet  MATH  Google Scholar 

  29. Manzo, F., Nardi, F.R., Olivieri, E., Scoppola, E.: On the essential features of metastability: tunneling time and critical configurations. J. Stat. Phys. 115(1/2), 591–642 (2004)

    Article  ADS  MATH  Google Scholar 

  30. Olivieri, E., Scoppola, E.: Markov chains with exponentially small transition probabilities: first exit problem from a general domain. I. The reversible case. J. Stat. Phys. 79(3/4), 613–647 (1995)

    Article  ADS  MathSciNet  MATH  Google Scholar 

  31. Olivieri, E., Scoppola, E.: Markov chains with exponentially small transition probabilities: first exit problem from a general domain. II. The general case. J. Stat. Phys. 84(5/6), 987–1041 (1996)

    Article  ADS  MathSciNet  MATH  Google Scholar 

  32. Oliveiri, E., Vares, M.E.: Large Deviations and Metastability. Cambridge University Press, Cambridge (2005)

    Book  Google Scholar 

  33. Scoppola, E.: Renormalization group for Markov chains and application to metastability. JSP 73, 83–121 (1993)

    ADS  MathSciNet  MATH  Google Scholar 

  34. Trouve, A.: Cycle decomposition and simulated annealing. SIAM J. Control. Optim. 34, 966–986 (1996)

    Article  MathSciNet  MATH  Google Scholar 

  35. Trouve, A.: Rough large deviation estimates for the optimal convergence speed exponent of generalized simulated annealing algorithms. Ann. Inst. H. Poinc. 32, 299–348 (1996)

    MathSciNet  MATH  Google Scholar 

  36. Wentzell, A.D.: On the asymptotics of eigenvalues of matrices with elements of order \(\exp (-V_{ij}/(2 \varepsilon ^2))\). Soviet Math. Dokl. 13(1), 65–68 (1972)

    Google Scholar 

Download references

Acknowledgements

We are grateful to three anonymous referees who read the paper very thoroughly and suggested many important improvements. While working on this article, M. Freidlin was supported by NSF Grant DMS-1411866 and L. Koralov was supported by NSF Grant DMS-1309084.

Author information

Authors and Affiliations

  1. Department of Mathematics, University of Maryland, College Park, MD, 20742, USA

    M. Freidlin & L. Koralov

Authors
  1. M. Freidlin
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. L. Koralov
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to L. Koralov.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Freidlin, M., Koralov, L. Metastable Distributions of Markov Chains with Rare Transitions. J Stat Phys 167, 1355–1375 (2017). https://doi.org/10.1007/s10955-017-1777-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10955-017-1777-z

Keywords

Mathematics Subject Classification

Search

Navigation