Skip to main content

Adaptive Sampling of Large Deviations

Journal of Statistical Physics volume 172, pages 1525–1544 (2018)Cite this article

A Correction to this article was published on 11 September 2018

This article has been updated

Abstract

We introduce and test an algorithm that adaptively estimates large deviation functions characterizing the fluctuations of additive functionals of Markov processes in the long-time limit. These functions play an important role for predicting the probability and pathways of rare events in stochastic processes, as well as for understanding the physics of nonequilibrium systems driven in steady states by external forces and reservoirs. The algorithm uses methods from risk-sensitive and feedback control to estimate from a single trajectory a new process, called the driven process, known to be efficient for importance sampling. Its advantages compared to other simulation techniques, such as splitting or cloning, are discussed and illustrated with simple equilibrium and nonequilibrium diffusion models.

This is a preview of subscription content, access via your institution.

Access options

Buy single article

Instant access to the full article PDF.

USD 39.95

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Change history

  • 11 September 2018

    The original version of this article unfortunately contained an error. The authors would like to correct the error with this erratum.

Notes

  1. See [6] for a treatment of diffusions with multiplicative noise.

References

  1. Shwartz, A., Weiss, A.: Large Deviations for Performance Analysis. Stochastic Modeling Series. Chapman and Hall, London (1995)

    MATH  Google Scholar 

  2. Dembo, A., Zeitouni, O.: Large Deviations Techniques and Applications, 2nd edn. Springer, New York (1998)

    Book  Google Scholar 

  3. den Hollander, F.: Large Deviations. AMS, Providence, RI (2000). Fields Institute Monograph

    MATH  Google Scholar 

  4. Jack, R.L., Sollich, P.: Large deviations and ensembles of trajectories in stochastic models. Prog. Theor. Phys. Suppl. 184, 304–317 (2010)

    Article  ADS  Google Scholar 

  5. Chetrite, R., Touchette, H.: Nonequilibrium microcanonical and canonical ensembles and their equivalence. Phys. Rev. Lett. 111, 120601 (2013)

    Article  ADS  Google Scholar 

  6. Chetrite, R., Touchette, H.: Nonequilibrium Markov processes conditioned on large deviations. Ann. Henri Poincaré 16(9), 2005–2057 (2015)

    Article  ADS  MathSciNet  Google Scholar 

  7. Touchette, H.: The large deviation approach to statistical mechanics. Phys. Rep. 478(1–3), 1–69 (2009)

    Article  ADS  MathSciNet  Google Scholar 

  8. Derrida, B.: Non-equilibrium steady states: fluctuations and large deviations of the density and of the current. J. Stat. Mech. 2007(07), P07023 (2007)

    Article  MathSciNet  Google Scholar 

  9. Harris, R.J., Touchette, H.: Large deviation approach to nonequilibrium systems. In: Klages, R., Just, W., Jarzynski, C. (eds.) Nonequilibrium Statistical Physics of Small Systems: Fluctuation Relations and Beyond, Reviews of Nonlinear Dynamics and Complexity, vol. 6, pp. 335–360. Wiley-VCH, Weinheim (2013)

    Google Scholar 

  10. Harris, R.J., Schütz, G.M.: Fluctuation theorems for stochastic dynamics. J. Stat. Mech. 2007(07), P07020 (2007)

    Article  MathSciNet  Google Scholar 

  11. Garrahan, J.P., Jack, R.L., Lecomte, V., Pitard, E., van Duijvendijk, K., van Wijland, F.: Dynamical first-order phase transition in kinetically constrained models of glasses. Phys. Rev. Lett. 98(19), 195702 (2007)

    Article  ADS  Google Scholar 

  12. Hedges, L.O., Jack, R.L., Garrahan, J.P., Chandler, D.: Dynamic order-disorder in atomistic models of structural glass formers. Science 323, 1309–1313 (2009)

    Article  ADS  Google Scholar 

  13. Espigares, C.P., Garrido, P.L., Hurtado, P.I.: Dynamical phase transition for current statistics in a simple driven diffusive system. Phys. Rev. E 87, 032115 (2013)

    Article  ADS  Google Scholar 

  14. Aminov, A., Bunin, G., Kafri, Y.: Singularities in large deviation functionals of bulk-driven transport models. J. Stat. Mech. 2014(8), P08017 (2014)

    Article  MathSciNet  Google Scholar 

  15. Dean, T., Dupuis, P.: Splitting for rare event simulation: a large deviation approach to design and analysis. Stoch. Proc. Appl. 119(2), 562–587 (2009)

    Article  MathSciNet  Google Scholar 

  16. Cérou, F., Guyader, A.: Adaptive multilevel splitting for rare event analysis. Stoch. Anal. Appl. 25(2), 417–443 (2007)

    Article  MathSciNet  Google Scholar 

  17. Aristoff, D., Lelièvre, T., Mayne, C.G., Teo, I.: Adaptive multilevel splitting in molecular dynamics simulations. ESAIM Proc 48, 215–225 (2015)

    Article  MathSciNet  Google Scholar 

  18. Grassberger, P.: Go with the winners: a general Monte Carlo strategy. Comp. Phys. Comm. 147(1–2), 64–70 (2002)

    Article  ADS  MathSciNet  Google Scholar 

  19. Giardina, C., Kurchan, J., Peliti, L.: Direct evaluation of large-deviation functions. Phys. Rev. Lett. 96(12), 120603 (2006)

    Article  ADS  Google Scholar 

  20. Lecomte, V., Tailleur, J.: A numerical approach to large deviations in continuous time. J. Stat. Mech. 2007(03), P03004 (2007)

    Article  Google Scholar 

  21. Bucklew, J.A.: Introduction to Rare Event Simulation. Springer, New York (2004)

    Book  Google Scholar 

  22. Juneja, S., Shahabuddin, P.: Rare-Event Simulation Techniques: An Introduction and Recent Advances, vol. 13, pp. 291–350. Elsevier, Amsterdam (2006)

    Google Scholar 

  23. Asmussen, S., Glynn, P.W.: Stochastic Simulation: Algorithms and Analysis. Stochastic Modelling and Applied Probability. Springer, New York (2007)

    MATH  Google Scholar 

  24. Bolhuis, P.G., Chandler, D., Dellago, C., Geissler, P.L.: Transition path sampling: throwing ropes over rough mountain passes, in the dark. Ann. Rev. Phys. Chem. 53(1), 291–318 (2002)

    Article  ADS  Google Scholar 

  25. Heymann, M., Vanden Eijnden, E.: Pathways of maximum likelihood for rare events in nonequilibrium systems: application to nucleation in the presence of shear. Phys. Rev. Lett. 100, 140601 (2008)

    Article  ADS  Google Scholar 

  26. Vanden-Eijnden, E., Weare, J.: Rare event simulation of small noise diffusions. Commun. Pure Appl. Math. 65(12), 1770–1803 (2012)

    Article  MathSciNet  Google Scholar 

  27. Grafke, T., Grauer, R., Schäfer, T.: The instanton method and its numerical implementation in fluid mechanics. J. Phys. A 48(33), 333001 (2015)

    Article  MathSciNet  Google Scholar 

  28. Borkar, V.S., Juneja, S., Kherani, A.A.: Peformance analysis conditioned on rare events: an adaptive simulation scheme. Commun. Info. Syst. 3(4), 259–278 (2004)

    MATH  Google Scholar 

  29. Ahamed, T.P.I., Borkar, V.S., Juneja, S.: Adaptive importance sampling technique for Markov chains using stochastic approximation. Oper. Res. 54(3), 489–504 (2006)

    Article  MathSciNet  Google Scholar 

  30. Basu, A., Bhattacharyya, T., Borkar, V.S.: A learning algorithm for risk-sensitive cost. Math. Oper. Res. 33(4), 880–898 (2008)

    Article  MathSciNet  Google Scholar 

  31. Borkar, V.S.: Learning algorithms for risk-sensitive control. In: Proc. 19th Int. Symp. Math. Theory Networks and Systems, pp. 1327–1332 (2010)

  32. Chetrite, R., Touchette, H.: Variational and optimal control representations of conditioned and driven processes. J. Stat. Mech. 2015(12), P12001 (2015)

    Article  MathSciNet  Google Scholar 

  33. Chauveau, D., Diebolt, J.: Estimation of the asymptotic variance in the CLT for Markov chains. Stoch. Models 19(4), 449–465 (2003)

    Article  MathSciNet  Google Scholar 

  34. Roberts, G.O., Rosenthal, J.S.: General state space Markov chains and MCMC algorithms. Prob. Surv. 1, 20–71 (2004)

    Article  MathSciNet  Google Scholar 

  35. Benveniste, A., Métivier, M., Priouret, P.: Adaptive Algorithms and Stochastic Approximations. Stochastic Modelling and Applied Probability, vol. 22. Springer, New York (2012)

    MATH  Google Scholar 

  36. Ferré, G., Stoltz, G.: Error estimates on ergodic properties of discretized Feynman–Kac semigroups (2017). arXiv:1712.04013

  37. Pavliotis, G.A.: Stochastic Processes and Applications. Springer, New York (2014)

    Book  Google Scholar 

  38. Chernyak, V.Y., Chertkov, M., Bierkens, J., Kappen, H.J.: Stochastic optimal control as non-equilibrium statistical mechanics: calculus of variations over density and current. J. Phys. A 47(2), 022001 (2014)

    Article  ADS  Google Scholar 

  39. Sekimoto, K.: Stochastic Energetics. Lect. Notes. Phys., vol. 799. Springer, New York (2010)

    MATH  Google Scholar 

  40. Bierkens, J., Chernyak, V.Y., Chertkov, M., Kappen, H.J.: Linear PDEs and eigenvalue problems corresponding to ergodic stochastic optimization problems on compact manifolds. J. Stat. Mech. 2016, 013206 (2016)

    Article  MathSciNet  Google Scholar 

  41. Lelièvre, T., Stoltz, G.: Partial differential equations and stochastic methods in molecular dynamics. Acta Numer. 25, 681–880 (2016)

    Article  MathSciNet  Google Scholar 

  42. Chatelin, F.: Spectral Approximation of Linear Operators. Classics in Applied Mathematics. SIAM, Philadelphia (2011)

    Book  Google Scholar 

  43. Gorissen, M., Vanderzande, C.: Finite size scaling of current fluctuations in the totally asymmetric exclusion process. J. Phys. A 44(11), 115005 (2011)

    Article  ADS  MathSciNet  Google Scholar 

  44. Fleming, W.H., Soner, H.M.: Controlled Markov Processes and Viscosity Solutions, Stochastic Modelling and Applied Probability, vol. 25. Springer, New York (2006)

    MATH  Google Scholar 

  45. Rousset, M.: On the control of an interacting particle estimation of Schrödinger ground states. SIAM J. Math. Anal. 38(3), 824–844 (2006)

    Article  MathSciNet  Google Scholar 

  46. Nemoto, T., Bouchet, F., Jack, R.L., Lecomte, V.: Population-dynamics method with a multicanonical feedback control. Phys. Rev. E 93, 062123 (2016)

    Article  ADS  MathSciNet  Google Scholar 

  47. Nemoto, T., Hidalgo, E.G., Lecomte, V.: Finite-time and finite-size scalings in the evaluation of large-deviation functions: analytical study using a birth-death process. Phys. Rev. E 95, 012102 (2017)

    Article  ADS  MathSciNet  Google Scholar 

  48. Kloeden, P.E., Platen, E.: Numerical Solution of Stochastic Differential Equations. Springer, Berlin (1992)

    Book  Google Scholar 

  49. Demmel, J.: Applied Numerical Linear Algebra. SIAM, Philadelphia (1997)

    Book  Google Scholar 

  50. Polyak, B.T., Juditsky, A.B.: Acceleration of stochastic approximation by averaging. SIAM J. Cont. Opt. 30(4), 838–855 (1992)

    Article  MathSciNet  Google Scholar 

  51. Hartmann, C., Schütte, C.: Efficient rare event simulation by optimal nonequilibrium forcing. J. Stat. Mech. 2012(11), P11004 (2012)

    Article  Google Scholar 

  52. Hartmann, C., Banisch, R., Sarich, M., Badowski, T., Schütte, C.: Characterization of rare events in molecular dynamics. Entropy 16(1), 350 (2014)

    Article  ADS  MathSciNet  Google Scholar 

  53. Zhang, W., Wang, H., Hartmann, C., Weber, M., Schütte, C.: Applications of the cross-entropy method to importance sampling and optimal control of diffusions. SIAM J. Sci. Compd. 36(6), A2654–A2672 (2014)

    Article  MathSciNet  Google Scholar 

  54. Rohwer, C.M., Angeletti, F., Touchette, H.: Convergence of large deviation estimators. Phys. Rev. E 92, 052104 (2015)

    Article  ADS  Google Scholar 

  55. Nemoto, T., Sasa, S.I.: Computation of large deviation statistics via iterative measurement-and-feedback procedure. Phys. Rev. Lett. 112, 090602 (2014)

    Article  ADS  Google Scholar 

  56. Risken, H.: The Fokker-Planck Equation: Methods of Solution and Applications, 3rd edn. Springer, Berlin (1996)

    MATH  Google Scholar 

  57. Reimann, P.: Brownian motors: noisy transport far from equilibrium. Phys. Rep. 361, 57–265 (2002)

    Article  ADS  MathSciNet  Google Scholar 

  58. Ciliberto, S., Joubaud, S., Petrosyan, A.: Fluctuations in out-of-equilibrium systems: from theory to experiment. J. Stat. Mech. 2010(12), P12003 (2010)

    Article  Google Scholar 

  59. Tsobgni Nyawo, P., Touchette, H.: Large deviations of the current for driven periodic diffusions. Phys. Rev. E 94, 032101 (2016)

    Article  ADS  Google Scholar 

  60. Mehl, J., Speck, T., Seifert, U.: Large deviation function for entropy production in driven one-dimensional systems. Phys. Rev. E 78, 011123 (2008)

    Article  ADS  Google Scholar 

  61. Nemoto, T., Sasa, S.I.: Variational formula for experimental determination of high-order correlations of current fluctuations in driven systems. Phys. Rev. E 83, 030105 (2011)

    Article  ADS  Google Scholar 

  62. Dupuis, P., Wang, H.: Dynamic importance sampling for uniformly recurrent Markov chains. Ann. Appl. Prob. 15(1A), 1–38 (2005)

    Article  MathSciNet  Google Scholar 

  63. Nemoto, T., Jack, R.L., Lecomte, V.: Finite-size scaling of a first-order dynamical phase transition: adaptive population dynamics and an effective model. Phys. Rev. Lett. 118, 115702 (2017)

    Article  ADS  Google Scholar 

  64. Blanchet, J., Lam, H.: State-dependent importance sampling for rare-event simulation: an overview and recent advances. Surv. Oper. Res. Manag. Sci. 17(1), 38–59 (2012)

    MathSciNet  Google Scholar 

  65. Kappen, H.J., Ruiz, H.C.: Adaptive importance sampling for control and inference. J. Stat. Phys. 162(5), 1244–1266 (2016)

    Article  ADS  MathSciNet  Google Scholar 

  66. Foulkes, W.M.C., Mitas, L., Needs, R.J., Rajagopal, G.: Quantum Monte Carlo simulations of solids. Rev. Mod. Phys. 73(1), 33 (2001)

    Article  ADS  Google Scholar 

  67. Lim, L.H., Weare, J.: Fast randomized iteration: diffusion Monte Carlo through the lens of numerical linear algebra. SIAM Rev. 59(3), 547–587 (2017)

    Article  MathSciNet  Google Scholar 

Download references

Acknowledgements

We are grateful to Florian Angeletti for useful discussions in the initial phase of this work. G.F. is supported by the Labex Bezout. H.T. was supported by the National Research Foundation of South Africa (Grant Nos. 90322 and 96199) and Stellenbosch University (Project Funding for New Appointee). This research was also supported in part by the International Centre for Theoretical Sciences (ICTS) during a visit for participating in the program “Large deviation theory in statistical physics: Recent advances and future challenges” (Code: ICTS/Prog-ldt/2017/8).

Author information

Authors and Affiliations

  1. CERMICS, École des Ponts ParisTech, Université Paris-Est, 77455, Marne-la-Vallée, France

    Grégoire Ferré

  2. National Institute for Theoretical Physics (NITheP), Stellenbosch, 7600, South Africa

    Hugo Touchette

  3. Institute of Theoretical Physics and Department of Physics, University of Stellenbosch, Stellenbosch, 7600, South Africa

    Hugo Touchette

Authors
  1. Grégoire Ferré
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hugo Touchette
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hugo Touchette.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Ferré, G., Touchette, H. Adaptive Sampling of Large Deviations. J Stat Phys 172, 1525–1544 (2018). https://doi.org/10.1007/s10955-018-2108-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10955-018-2108-8

Keywords

  • Large deviations
  • Rare event simulation
  • Diffusions
  • Nonequilibrium processes