# A Graph-Algorithmic Approach for the Study of Metastability in Markov Chains

- 131 Downloads

## Abstract

Large continuous-time Markov chains with exponentially small transition rates arise in modeling complex systems in physics, chemistry, and biology. We propose a constructive graph-algorithmic approach to determine the sequence of critical timescales at which the qualitative behavior of a given Markov chain changes, and give an effective description of the dynamics on each of them. This approach is valid for both time-reversible and time-irreversible Markov processes, with or without symmetry. Central to this approach are two graph algorithms, Algorithm 1 and Algorithm 2, for obtaining the sequences of the critical timescales and the hierarchies of Typical Transition Graphs or T-graphs indicating the most likely transitions in the system without and with symmetry, respectively. The sequence of critical timescales includes the subsequence of the reciprocals of the real parts of eigenvalues. Under a certain assumption, we prove sharp asymptotic estimates for eigenvalues (including pre-factors) and show how one can extract them from the output of Algorithm 1. We discuss the relationship between Algorithms 1 and 2 and explain how one needs to interpret the output of Algorithm 1 if it is applied in the case with symmetry instead of Algorithm 2. Finally, we analyze an example motivated by R. D. Astumian’s model of the dynamics of kinesin, a molecular motor, by means of Algorithm 2.

## Keywords

Continuous-time Markov chain Optimal W-graphs Typical Transition Graphs Graph algorithms Symmetry Asymptotic estimates for eigenvalues Freidlin’s cycles Molecular motor## Notes

### Acknowledgements

We would like to thank Mr. Weilin Li for valuable discussions in the early stage of this work. We are grateful to Prof. C. Jarzynski and Prof. M. Fisher for suggesting us to consider molecular motors as an example of a natural time-irreversible system with transition rates in the exponential form. We also thank the anonymous reviewers for their valuable feedback and comments. This work was partially supported by NSF Grants 1217118 and 1554907.

## References

- Astumian, R.D.: Biasing the random walk of a molecular motor. J. Phys.: Condens. Matter
**17**, S3753 (2005)Google Scholar - Arkus, N., Manoharan, V., Brenner, M.P.: Minimal energy clusters of hard spheres with short ranged attractions. Phys. Rev. Lett.
**103**, 118303 (2009)CrossRefGoogle Scholar - Arkus, N., Manoharan, V., Brenner, M.P.: Deriving finite sphere packings. SIAM J. Discret. Math.
**25**, 1860–1901 (2011)MathSciNetCrossRefMATHGoogle Scholar - Berglund, N., Dutercq, S.: The Eyring—Kramers Law for Markovian Jump Processes with Symmetries. J. Theory Prob. First online: 21 May 2015, doi: 10.1007/s10959-015-0617-9
- Bouchet, F., Reygner, J.: Generalization of the Eyring-Kramers transition rate formula to irreversible diffusion processes, Annales Henri Poincare, First online: 11 June 2016, pp. 1–34 arXiv:1507.02104v1
- Bovier, A., Eckhoff, M., Gayrard, V., Klein, M.: Metastability and low lying spectra in reversible Markov chains. Comm. Math. Phys.
**228**, 219–255 (2002)MathSciNetCrossRefMATHGoogle Scholar - Bovier, A., Eckhoff, M., Gayrard, V., Klein, M.: Metastability in reversible diffusion processes 1. Sharp estimates for capacities and exit times. J. Eur. Math. Soc.
**6**, 399–424 (2004)CrossRefMATHGoogle Scholar - Bovier, A., Gayrard, V., Klein, M.: Metastability in reversible diffusion processes 2. Precise estimates for small eigenvalues. J. Eur. Math. Soc.
**7**, 69–99 (2005)CrossRefMATHGoogle Scholar - Bovier, A., den Hollander, F.: Metastability: A Potential-Theoretic Approach. Springer, Berlin (2016)MATHGoogle Scholar
- Cameron, M.K.: Computing Freidlin’s cycles for the overdamped Langevin dynamics. J. Stat. Phys.
**152**(3), 493–518 (2013)MathSciNetCrossRefMATHGoogle Scholar - Cameron, M.K.: Computing the asymptotic spectrum for networks representing energy landscapes using the minimal spanning tree, M. Cameron. Netw. Heterog. Media
**9**, 3 (2014)MathSciNetCrossRefGoogle Scholar - Cameron, M.K., Gan, T.: Spectral analysis and clustering of large stochastic networks. Application to the Lennard-Jones-75 cluster. Mol. Simul.
**42**(16), 1410–1428 (2016)CrossRefGoogle Scholar - Chu, Y.J., Liu, T.H.: On the shortest arborescence of a directed graph. Sci. Sin.
**14**, 1396–1400 (1965)MathSciNetMATHGoogle Scholar - Edmonds, J.: Optimum branchings. J. Res. Natl. Bur. Stand.: B Math. Math. Phys.
**71 B**, 4 (1967)MathSciNetMATHGoogle Scholar - Freidlin, M.I.: Sublimiting distributions and stabilization of solutions of parabolic equations with small parameter. Sov. Math. Dokl.
**18**(4), 1114–1118 (1977)MATHGoogle Scholar - Freidlin, M.I.: Quasi-deterministic approximation, metastability and stochastic resonance. Phys. D
**137**, 333–352 (2000)MathSciNetCrossRefMATHGoogle Scholar - Freidlin, M.: On stochastic perturbations of dynamical systems with a “rough” symmetry. Hierarchy of Markov chains. J. Stat. Phys.
**157**(6), 1031–1045 (2014)MathSciNetCrossRefMATHGoogle Scholar - Freidlin, M.I., Wentzell, A.D.: Random Perturbations of Dynamical Systems, 3rd edn. Springer, Berlin (2012)CrossRefMATHGoogle Scholar
- Gaveau, B., Schulman, L.S.: Theory of nonequilibrium first-order phase transitions for stochastic dynamics. J. Phys.
**33**, 4837–4850 (2000)MathSciNetGoogle Scholar - Gerschgorin, S.: Uber die abgrenzung der eigenwerte einer matrix. Izv. Akad. Nauk. USSR Otd. Fiz.-Mat. Nauk
**7**, 749–754 (1931)MATHGoogle Scholar - Holmes-Cerfon, M., Gortler, S.J., Brenner, M.P.: A geometrical approach to computing free-energy landscapes from short-ranged potentials. Proc. Natl. Acad. Sci.
**110**(1), E5–E14 (2013)CrossRefGoogle Scholar - Meng, G., Arkus, N., Brenner, M.P., Manoharan, V.: The free energy landscape of hard sphere clusters. Science
**327**, 560 (2010)CrossRefGoogle Scholar - Metzner, P., Schuette, Ch., Vanden-Eijnden, E.: Transition path theory for Markov jump processes. SIAM Multiscale Model Simul.
**7**, 1192–1219 (2009)MathSciNetCrossRefMATHGoogle Scholar - Norris, J.P.: Markov Chains. Cambridge University Press, Cambridge (1997)CrossRefMATHGoogle Scholar
- Wales, D.J.: Discrete path sampling. Mol. Phys.
**100**, 3285–3306 (2002)CrossRefGoogle Scholar - Wales, D.J.: Energy Landscapes: Applications to Clusters, Biomolecules and Glasses. Cambridge University Press, Cambridge (2003)Google Scholar
- Wales, D.J.: Energy landscapes: calculating pathways and rates. Int. Rev. Chem. Phys.
**25**(1–2), 237–282 (2006)CrossRefGoogle Scholar - Wentzell, A.D.: On the asymptotics of eigenvalues of matrices with elements of order \(exp \{ -V_{ij} / 2 (\varepsilon ^2) \} \), (in Russian) Dokl. Akad. Nauk SSSR
**202**, 263–265 (1972)Google Scholar