Skip to main content

Markov Model Theory

  • Chapter

Part of the Advances in Experimental Medicine and Biology book series (AEMB,volume 797)

Abstract

This section reviews the relation between the continuous dynamics of a molecular system in thermal equilibrium and the kinetics given by a Markov State Model (MSM). We will introduce the dynamical propagator, an error-less, alternative description of the continuous dynamics, and show how MSMs result from its discretization. This allows for an precise understanding of the approximation quality of MSMs in comparison to the continuous dynamics. The results on the approximation quality are key for the design of good MSMs. While this section is important for understanding the theory of discretization and related systematic errors, practitioners wishing only to learn how to construct MSMs may skip directly to the discussion of Markov model estimation.

Part of this chapter is reprinted with permission from Prinz et al. (Markov models of molecular kinetics: Generation and validation. J. Chem. Phys. 134:174,105, 2011). Copyright 2011, American Institute of Physics.

A more detailed text book version is to be found in Schütte and Sarich (Metastability and Markov State Models in Molecular Dynamics: Modeling, Analysis, Algorithmic Approaches. Courant Lecture Notes, 24. American Mathematical Society, 2013).

Keywords

  • State Markov Model (MSM)
  • Crisp Partition
  • Dynamic Markov Model
  • Discretization Error
  • Discretization Basis

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

This is a preview of subscription content, access via your institution.

Buying options

Chapter
USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-94-007-7606-7_3
  • Chapter length: 22 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
eBook
USD   149.00
Price excludes VAT (USA)
  • ISBN: 978-94-007-7606-7
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Hardcover Book
USD   199.99
Price excludes VAT (USA)
Fig. 3.1
Fig. 3.2
Fig. 3.3
Fig. 3.4
Fig. 3.5
Fig. 3.6
Fig. 3.7
Fig. 3.8
Fig. 3.9
Fig. 3.10
Fig. 3.11
Fig. 3.12
Fig. 3.13
Fig. 3.14

References

  1. Andersen HC (1980) Molecular dynamics simulations at constant pressure and/or temperature. J Chem Phys 72(4):2384–2393. doi:10.1063/1.439486

    CrossRef  CAS  Google Scholar 

  2. Bieri O, Wirz J, Hellrung B, Schutkowski M, Drewello M, Kiefhaber T (1999) The speed limit for protein folding measured by triplet-triplet energy transfer. Proc Natl Acad Sci USA 96(17):9597–9601. http://www.pnas.org/content/96/17/9597.abstract

    PubMed  CrossRef  CAS  Google Scholar 

  3. Bowman GR, Beauchamp KA, Boxer G, Pande VS (2009) Progress and challenges in the automated construction of Markov state models for full protein systems. J Chem Phys 131(12):124,101+. doi:10.1063/1.3216567

    CrossRef  Google Scholar 

  4. Buchete NV, Hummer G (2008) Coarse Master Equations for Peptide Folding Dynamics. J Phys Chem B 112:6057–6069

    PubMed  CrossRef  CAS  Google Scholar 

  5. Chan CK, Hu Y, Takahashi S, Rousseau DL, Eaton WA, Hofrichter J (1997) Submillisecond protein folding kinetics studied by ultrarapid mixing. Proc Natl Acad Sci USA 94(5):1779–1784. http://www.pnas.org/content/94/5/1779.abstract

    PubMed  CrossRef  CAS  Google Scholar 

  6. Chodera JD, Dill KA, Singhal N, Pande VS, Swope WC, Pitera JW (2007) Automatic discovery of metastable states for the construction of Markov models of macromolecular conformational dynamics. J Chem Phys 126:155,101

    CrossRef  Google Scholar 

  7. Chodera JD, Noé F (2010) Probability distributions of molecular observables computed from Markov models, II: uncertainties in observables and their time-evolution. J Chem Phys 133:105,102

    CrossRef  Google Scholar 

  8. Chodera JD, Swope WC, Noé F, Prinz JH, Pande VS (2010) Dynamical reweighting: improved estimates of dynamical properties from simulations at multiple temperatures. J Phys Chem. doi:10.1063/1.3592152

    Google Scholar 

  9. Chodera JD, Swope WC, Pitera JW, Dill KA (2006) Long-time protein folding dynamics from short-time molecular dynamics simulations. Multiscale Model Simul 5:1214–1226

    CrossRef  Google Scholar 

  10. Cooke B, Schmidler SC (2008) Preserving the Boltzmann ensemble in replica-exchange molecular dynamics. J Chem Phys 129:164,112

    CrossRef  Google Scholar 

  11. Deuflhard P, Huisinga W, Fischer A, Schütte C (2000) Identification of almost invariant aggregates in reversible nearly uncoupled Markov chains. Linear Algebra Appl 315:39–59

    CrossRef  Google Scholar 

  12. Deuflhard P, Weber M (2005) Robust Perron cluster analysis in conformation dynamics. Linear Algebra Appl 398:161–184

    CrossRef  Google Scholar 

  13. Djurdjevac N, Sarich M, Schütte C (2012) Estimating the eigenvalue error of Markov state models. Multiscale Model Simul 10(1):61–81

    CrossRef  Google Scholar 

  14. Duane S (1987) Hybrid Monte Carlo. Phys Lett B 195(2):216–222. doi:10.1016/0370-2693(87)91197-X

    CrossRef  CAS  Google Scholar 

  15. Ermak DL (1975) A computer simulation of charged particles in solution, I: technique and equilibrium properties. J Chem Phys 62(10):4189–4196

    CrossRef  CAS  Google Scholar 

  16. Ermak DL, Yeh Y (1974) Equilibrium electrostatic effects on the behavior of polyions in solution: polyion-mobile ion interaction. Chem Phys Lett 24(2):243–248

    CrossRef  CAS  Google Scholar 

  17. Golub GH, van Loan CF (1996) Matrix computations, 3rd edn. John Hopkins University Press, Baltimore

    Google Scholar 

  18. Herau R, Hitrik M, Sjoestrand J (2010) Tunnel effect and symmetries for Kramers Fokker-Planck type operators. arXiv:1007.0838v1

  19. Huisinga W (2001) Metastability of Markovian systems: a transfer operator based approach in application to molecular dynamics. PhD thesis, Fachbereich Mathematik und Informatik, FU Berlin

    Google Scholar 

  20. Huisinga W, Meyn S, Schütte C (2004) Phase transitions and metastability for Markovian and molecular systems. Ann Appl Probab 14:419–458

    CrossRef  Google Scholar 

  21. Jäger M, Nguyen H, Crane JC, Kelly JW, Gruebele M (2001) The folding mechanism of a beta-sheet: the WW domain. J Mol Biol 311(2):373–393. doi:10.1006/jmbi.2001.4873

    PubMed  CrossRef  Google Scholar 

  22. van Kampen NG (2006) Stochastic processes in physics and chemistry, 4th edn. Elsevier, Amsterdam

    Google Scholar 

  23. Keller B, Hünenberger P, van Gunsteren W (2010) An analysis of the validity of Markov state models for emulating the dynamics of classical molecular systems and ensembles. J Chem Theor Comput. doi:10.1021/ct200069c

    Google Scholar 

  24. Kube S, Weber M (2007) A coarse graining method for the identification of transition rates between molecular conformations. J Chem Phys 126:024,103–024,113

    CrossRef  Google Scholar 

  25. Meerbach E, Schütte C, Horenko I, Schmidt B (2007) Metastable conformational structure and dynamics: peptides between gas phase and aqueous solution. Series in chemical physics, vol 87. Springer, Berlin, pp 796–806

    Google Scholar 

  26. Metzner P, Horenko I, Schütte C (2007) Generator estimation of Markov jump processes based on incomplete observations nonequidistant in time. Phys Rev E 76(6):066,702+. doi:10.1103/PhysRevE.76.066702

    CrossRef  Google Scholar 

  27. Metzner P, Schütte C, Vanden-Eijnden E (2009) Transition path theory for Markov jump processes. Multiscale Model Simul 7(3):1192–1219

    CrossRef  CAS  Google Scholar 

  28. Neuweiler H, Doose S, Sauer M (2005) A microscopic view of miniprotein folding: enhanced folding efficiency through formation of an intermediate. Proc Natl Acad Sci USA 102(46):16,650–16,655. doi:10.1073/pnas.0507351102

    CrossRef  CAS  Google Scholar 

  29. Noé F, Horenko I, Schütte C, Smith JC (2007) Hierarchical analysis of conformational dynamics in biomolecules: transition networks of metastable states. J Chem Phys 126:155,102

    CrossRef  Google Scholar 

  30. Noé F, Schütte C, Vanden-Eijnden E, Reich L, Weikl TR (2009) Constructing the full ensemble of folding pathways from short off-equilibrium simulations. Proc Natl Acad Sci USA 106:19,011–19,016

    CrossRef  Google Scholar 

  31. Prinz JH et al. (2011) Markov models of molecular kinetics: generation and validation. J Chem Phys 134:174,105

    CrossRef  Google Scholar 

  32. Sarich M (2011) Projected transfer operators. PhD thesis, Freie Universität Berlin

    Google Scholar 

  33. Sarich M, Noé F, Schütte C (2010) On the approximation error of Markov state models. Multiscale Model Simul 8:1154–1177

    CrossRef  Google Scholar 

  34. Sarich M, Schütte C (2012) Approximating selected non-dominant timescales by Markov state models. Comm Math Sci (in press)

    Google Scholar 

  35. Schütte C (1998) Conformational dynamics: modelling, theory, algorithm, and applications to biomolecules. Habilitation thesis, Fachbereich Mathematik und Informatik, FU Berlin

    Google Scholar 

  36. Schütte C, Huisinga W (2003) Biomolecular conformations can be identified as metastable sets of molecular dynamics. In: Handbook of numerical analysis. Elsevier, Amsterdam, pp 699–744

    Google Scholar 

  37. Schütte C, Sarich M (2013) Metastability and Markov state models in molecular dynamics: modeling, analysis, algorithmic approaches. Courant lecture notes, vol 24. American Mathematical Society, Providence

    Google Scholar 

  38. Schütte C, Fischer A, Huisinga W, Deuflhard P (1999) A direct approach to conformational dynamics based on hybrid Monte Carlo. J Comput Phys 151:146–168

    CrossRef  Google Scholar 

  39. Schütte C, Noé F, Meerbach E, Metzner P, Hartmann C (2009) Conformation dynamics. In: Jeltsch RGW (ed) Proceedings of the international congress on industrial and applied mathematics (ICIAM). EMS Publishing House, New York, pp 297–336

    Google Scholar 

  40. Schütte C, Noé F, Lu J, Sarich M, Vanden-Eijnden E (2011) Markov state models based on milestoning. J Chem Phys 134(19)

    Google Scholar 

  41. Sriraman S, Kevrekidis IG, Hummer G (2005) Coarse master equation from Bayesian analysis of replica molecular dynamics simulations. J Phys Chem B 109:6479–6484

    PubMed  CrossRef  CAS  Google Scholar 

  42. Swope WC, Andersen HC, Berens PH, Wilson KR (1982) A computer simulation method for the calculation of equilibrium constants for the formation of physical clusters of molecules: application to small water clusters. J Chem Phys 76(1):637–649

    CrossRef  CAS  Google Scholar 

  43. Swope WC, Pitera JW, Suits F (2004) Describing protein folding kinetics by molecular dynamics simulations, 1: theory. J Phys Chem B 108:6571–6581

    CrossRef  CAS  Google Scholar 

  44. Swope WC, Pitera JW, Suits F, Pitman M, Eleftheriou M (2004) Describing protein folding kinetics by molecular dynamics simulations, 2: example applications to alanine dipeptide and beta-hairpin peptide. J Phys Chem B 108:6582–6594

    CrossRef  CAS  Google Scholar 

  45. Voronoi MG (1908) Nouvelles applications des parametres continus a la theorie des formes quadratiques. J Reine Angew Math 134:198–287

    Google Scholar 

  46. Weber M (2006) Meshless methods in conformation dynamics. PhD thesis, Freie Universität Berlin

    Google Scholar 

  47. Weber M (2010) A subspace approach to molecular Markov state models via an infinitesimal generator. ZIB report 09-27-rev

    Google Scholar 

  48. Weber M (2011) A subspace approach to molecular Markov state models via a new infinitesimal generator. Habilitation thesis, Fachbereich Mathematik und Informatik, Freie Universität Berlin

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Christof Schütte .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and Permissions

Copyright information

© 2014 Springer Science+Business Media Dordrecht

About this chapter

Cite this chapter

Sarich, M., Prinz, JH., Schütte, C. (2014). Markov Model Theory. In: Bowman, G., Pande, V., Noé, F. (eds) An Introduction to Markov State Models and Their Application to Long Timescale Molecular Simulation. Advances in Experimental Medicine and Biology, vol 797. Springer, Dordrecht. https://doi.org/10.1007/978-94-007-7606-7_3

Download citation