Skip to main content
SpringerLink
Log in
Book cover

Current Challenges in Stability Issues for Numerical Differential Equations pp 39–88Cite as

Markov Chain Monte Carlo and Numerical Differential Equations

  • Chapter
  • First Online:

Part of the book series: Lecture Notes in Mathematics ((LNMCIME,volume 2082))

Abstract

The aim of this contribution is to provide a readable account of Markov Chain Monte Carlo methods, with particular emphasis on their relations with the numerical integration of deterministic and stochastic differential equations. The exposition is largely based on numerical experiments and avoids mathematical technicalities. The presentation is largely self-contained and includes tutorial sections on stochastic processes, Markov chains, stochastic differential equations and Hamiltonian dynamics. The Metropolis Random-Walk algorithm, Metropolis adjusted Langevin algorithm and Hybrid Monte Carlo are discussed in detail, including some recent results.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   44.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   59.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Notes

  1. 1.

    We assume notions such as discrete and continuous random variables, expectation, variance, conditional probability and independence.

  2. 2.

    Recall that (1) Ω is a set and each point ω ∈ Ω corresponds to a possible outcome of a random experiment, (2) \(\mathcal{A}\) (a σ-algebra) is the family of those subsets A ⊆ Ω called events to which a probability \(\mathbb{P}(A)\) is assigned, (3) \(\mathbb{P}\) is a probability measure, \(\mathbb{P}: \mathcal{A}\rightarrow [0, 1]\). The probability space plays very little explicit role in the study of the process; this is carried out in terms of the distributions of the X t (see the examples in this section).

  3. 3.

    Often the dependence of X t on ω is not incorporated explicitly to the notation.

  4. 4.

    In general, a process \(\{X_{n}\}_{n\geq 0}\) is a random walk if \(X_{n+1} = X_{n} + Z_{n}\), where Z n is independent of X n ,…, X 0.

  5. 5.

    Note that e.g. \(\mathbb{P}(X_{n+1} = j\mid X_{n} = i)\) does not make sense if \(\mathbb{P}(X_{n} = i) = 0\). Here we shall not pay attention to the difficulties created by probabilities conditioned to events \(\{X_{n} = i\}\) of 0 probability. These difficulties are easily avoided if, as in [9], the Markov chain is defined in the first place by means of the transition probabilities rather than in terms of the variables X n .

  6. 6.

    If E comprises an infinite number of states, this “matrix” will of course have infinitely many rows/columns. Sums like \(\sum _{j}p_{\mathit{ij}}p_{\mathit{jk}}\) that we shall find below have a finite value if P is stochastic.

  7. 7.

    See [5], Theorem 8.1 for the construction of the X n and the underlying probability space.

  8. 8.

    If all the rows of P are equal, X n+1 is independent of X n .

  9. 9.

    This should not lead to the conclusion that period 2 is the rule for MCs. The three examples in the last section are not typical in this respect and were chosen in view of the fact that are very easily described—in each of them transitions may only occur between state i and states i ± 1. Any chain where the diagonal elements of P are all ≠ 0 only contains aperiodic states.

  10. 10.

    If λ, ν are distributions the notation \({\mid \lambda }^{T} {-\nu }^{T}\mid\) means \(\sum _{i}\vert \lambda _{i} -\nu _{i}\vert\).

  11. 11.

    Note that there is no invariant probability distribution, since the chain is null recurrent.

  12. 12.

    Of course computing the probability of an event A is equivalent to computing the expectation of its indicator, i.e. of the random variable that takes the value 1 if ω ∈ A and 0 if \(\omega \notin A\).

  13. 13.

    It is also necessary that the chain constructed be positive recurrent. Also not all positive recurrent chains having the target as equilibrium measure are equally efficient, as the velocity of the convergence to the limit in (7) is of course chain-dependent.

  14. 14.

    ∧ means min.

  15. 15.

    The symbol ∝ means proportional to. To obtain a probability density it is necessary to divide exp(−β V (x)) by the normalizing constant \(\int _{\mathbb{R}}V (x)\,\mathit{dx}\). As pointed out before the Metropolis algorithm does not require the knowledge of the normalizing constant.

  16. 16.

    For further details of the statistical analysis of the sequence of samples x i the reader is referred to [13].

  17. 17.

     ∼ means “has a distribution.”

  18. 18.

    These references also show that (4) is essentially a consequence of (1), (2) and (3).

  19. 19.

    The trajectories of the Wiener process are in fact complex objects. For instance, with probability 1, the set Z(ω) of values t for which a trajectory B t (ω) vanishes is closed, unbounded, without isolated points and of Lebesgue measure 0, [5], Theorem 37.4.

  20. 20.

    The terminology Fokker-Panck is used in physics; in probability the equation is known as Kolmogorov’s forward equation, see e.g. [10], Chap. X, Sect. 5.

  21. 21.

    More precisely it is customary to insist in the integrator being symplectic [35], Chap. 6; symplecticness implies conservation of volume and satisfactory—but not exact—conservation of energy, [35], Sect. 10.3.3. The Verlet scheme is symplectic.

  22. 22.

    In this connection it may be worth noting that the proof of Theorem 9 demands that the mapping Ψ T is time reversible and volume preserving, but would work even if Ψ T were not an approximation to the true Φ T . However if Ψ T is not close to Φ T , the acceptance probability will be low.

  23. 23.

    Recall that when the MALA proposal is seen as an Euler-Maruyama step for an SDE, the MALA parameter h coincides with the square of the time-step Δ t. However in the relation of MALA with HMC studied in this section, h = Δ t as we have just pointed out.

References

  1. E. Akhmatskaya, S. Reich, GSHMC: An efficient method for molecular simulations. J. Comput. Phys. 227, 4934–4954 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  2. V.I. Arnold, Mathematical Methods of Classical Mechanics, 2nd edn. (Springer, New York, 1989)

    Book  Google Scholar 

  3. A. Beskos, F.J. Pinski, J.M. Sanz-Serna, A.M. Stuart, Hybrid Monte-Carlo on Hilbert spaces. Stoch. Process. Appl. 121, 2201–2230 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  4. A. Beskos, N. Pillai, G.O. Roberts, J.M. Sanz-Serna, A.M. Stuart, Optimal tuning of the Hybrid Monte-Carlo algorithm. Bernoulli (to appear)

    Google Scholar 

  5. P. Billingsley, Probability and Measure, 3rd edn. (Wiley, New York, 1995)

    MATH  Google Scholar 

  6. P. Brémaud, Markov Chains, Gibbs Fields, Monte Carlo Simulation, and Queues (Springer, Berlin, 1999)

    MATH  Google Scholar 

  7. E. Cancès, F. Legoll, G. Stoltz, Theoretical and numerical comparison of some sampling methods for molecular dynamics. Esaim Math. Model. Numer. Anal. 41, 351–389 (2007)

    Article  MATH  Google Scholar 

  8. S. Duane, A.D. Kennedy, B. Pendleton, R. Roweth, Hybrid Monte Carlo. Phys. Lett. B 195, 216–222 (1987)

    Google Scholar 

  9. W. Feller, An Introduction to Probability Theory and Its Applications, vol. 1, 3rd edn. (Wiley, New York, 1968)

    Google Scholar 

  10. W. Feller, An Introduction to Probability Theory and Its Applications, vol. 1, 2nd edn. (Wiley, New York, 1971)

    Google Scholar 

  11. R.P. Feynman, R.B. Leighton, M. Sands, The Feynman Lectures on Physics, vol. 1 (Addison-Wesley, Reading, 1963)

    Google Scholar 

  12. A. Friedman, Stochastic Differential Equations and Applications (Dover, Mineola, 2006)

    MATH  Google Scholar 

  13. C.J. Geyer, Practical Markov Chain Monte Carlo. Stat. Sci. 7, 473–483 (1992)

    Article  Google Scholar 

  14. M. Girolami, B. Calderhead, Riemann manifold Langevin and Hamiltonian Monte Carlo methods. J. R. Stat. Soc. B 73, 123–214 (2011)

    Article  MathSciNet  Google Scholar 

  15. G. Grimmett, D. Stirzaker, Probability and Random Processes, 3rd edn. (Oxford University Press, Oxford, 2001)

    Google Scholar 

  16. E. Hairer, C. Lubich, G. Wanner, Geometric Numerical Integration, 2nd edn. (Springer, Berlin, 2006)

    MATH  Google Scholar 

  17. W. Hastings, Monte Carlo sampling methods using Markov chains and their application. Biometrika 57, 97–109 (1970)

    Article  MATH  Google Scholar 

  18. D. Higham, An algorithmic introduction to numerical simulation of stochastic differential equations. SIAM Rev. 43, 525–546 (2001)

    Article  MathSciNet  MATH  Google Scholar 

  19. M.D. Hoffman, A. Gelman, The No-U-Turn sampler: Adaptively setting path lengths in Hamiltonian Monte Carlo, preprint

    Google Scholar 

  20. J.A. Izaguirre, S.S. Hampton, Shadow hybrid Monte Carlo: An efficient propagator in phase space of macromolecules. J. Comput. Phys. 200, 581–604 (2004)

    Article  MATH  Google Scholar 

  21. I.D. Lawrie, A Unified Grand Tour of Theoretical Physics (Institute of Physics Publishing, Bristol, 1990)

    Book  MATH  Google Scholar 

  22. B. Leimkuhler, S. Reich, Simulating Hamiltonian Dynamics (Cambridge University Press, Cambridge, 2004)

    MATH  Google Scholar 

  23. T. Lelievre, M. Rousset, G. Stoltz, Free Energy Computations: A Mathematical Perspective (Imperial College Press, London, 2010)

    Book  Google Scholar 

  24. X. Mao, Stochastic Differential Equations and Applications, 2nd edn. (Horwood, Chichester, 2008)

    Book  Google Scholar 

  25. J.C. Mattingly, N.S. Pillai, A.M. Stuart, Diffusion limits of the random walk Metropolis algorithm in high dimensions authors. Ann. Appl. Probab. 22, 881–930 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  26. N. Metropolis, A. Rosenbluth, M. Rosenbluth, A. Teller, E. Teller, Equations of state calculations by fast computing machines. J. Chem. Phys. 21, 1087–1092 (1953)

    Article  Google Scholar 

  27. S. Meyn, R. Tweedie, Markov Chains and Stochastic Stability (Springer, New York, 1993)

    Book  MATH  Google Scholar 

  28. R. Neal, Probabilistic Inference Using Markov Chain Monte Carlo Methods. Technical Report CRG-TR-93-1, Department of Computer Science, University of Toronto, 1993

    Google Scholar 

  29. N.S. Pillai, A.M. Stuart, A.H. Thiery, Optimal scaling and diffusion limits for the Langevin algorithm in high dimensions. Ann. Appl. Probab. 22, 2320–2356 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  30. C.P. Robert, G. Casella, Monte Carlo Statistical Methods, 2nd edn. (Springer, Berlin, 2004)

    Book  MATH  Google Scholar 

  31. G.O. Roberts, J.S. Rosenthal, Optimal scaling of discrete approximations to Langevin diffusions. J. R. Stat. Soc. B 60, 255–268 (1998)

    Article  MathSciNet  MATH  Google Scholar 

  32. G.O. Roberts, R.L. Tweedie, Exponential convergence of Langevin diffusions and their discrete approximations. Bernoulli 2, 341–263 (1996)

    Article  MathSciNet  MATH  Google Scholar 

  33. G.O. Roberts, A. Gelman, W.R. Gilks, Weak convergence and optimal scaling of random walk Metropolis algorithms. Ann. Appl. Probab. 7, 110–120 (1997)

    Article  MathSciNet  MATH  Google Scholar 

  34. J.M. Sanz-Serna, Geometric integration, in The State of the Art in Numerical Analysis, ed. by I.S. Duff, A.G. Watson (Clarendon, Oxford, 1997), pp. 121–143

    Google Scholar 

  35. J.M. Sanz-Serna, M.P. Calvo, Numerical Hamiltonian Problems (Chapman & Hall, London, 1994)

    MATH  Google Scholar 

  36. T. Schilick, Molecular Modeling and Simulation: An Interdisciplinary Guide, 2nd edn. (Springer, New York, 2010)

    Book  Google Scholar 

  37. C. Schütte, Conformational Dynamics: Modelling, Theory, Algorithmm and Application to Biomolecules. Habilitation Thesis, Free University Berlin, 1999

    Google Scholar 

  38. B. Shahbaba, S. Lan, W.O. Johnson, R.M. Neal, Split Hamiltonian Monte Carlo, preprint

    Google Scholar 

Download references

Acknowledgements

This work has been supported by Project MTM2010-18246-C03-01, Ministerio de Ciencia e Innovación, Spain.

Author information

Authors and Affiliations

  1. Departamento de Matemática Aplicada e IMUVA, Facultad de Ciencias, Universidad de Valladolid, Valladolid, Spain

    J. M. Sanz-Serna

Authors
  1. J. M. Sanz-Serna
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to J. M. Sanz-Serna .

Rights and permissions

Reprints and permissions

Copyright information

© 2014 Springer International Publishing Switzerland

About this chapter

Cite this chapter

Sanz-Serna, J.M. (2014). Markov Chain Monte Carlo and Numerical Differential Equations. In: Current Challenges in Stability Issues for Numerical Differential Equations. Lecture Notes in Mathematics(), vol 2082. Springer, Cham. https://doi.org/10.1007/978-3-319-01300-8_2

Download citation

Publish with us

Policies and ethics

Search

Navigation