Skip to main content

Adaptive Sampling of Large Deviations

A Correction to this article was published on 11 September 2018

This article has been updated

Abstract

We introduce and test an algorithm that adaptively estimates large deviation functions characterizing the fluctuations of additive functionals of Markov processes in the long-time limit. These functions play an important role for predicting the probability and pathways of rare events in stochastic processes, as well as for understanding the physics of nonequilibrium systems driven in steady states by external forces and reservoirs. The algorithm uses methods from risk-sensitive and feedback control to estimate from a single trajectory a new process, called the driven process, known to be efficient for importance sampling. Its advantages compared to other simulation techniques, such as splitting or cloning, are discussed and illustrated with simple equilibrium and nonequilibrium diffusion models.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Change history

  • 11 September 2018

    The original version of this article unfortunately contained an error. The authors would like to correct the error with this erratum.

Notes

  1. 1.

    See [6] for a treatment of diffusions with multiplicative noise.

References

  1. 1.

    Shwartz, A., Weiss, A.: Large Deviations for Performance Analysis. Stochastic Modeling Series. Chapman and Hall, London (1995)

    MATH  Google Scholar 

  2. 2.

    Dembo, A., Zeitouni, O.: Large Deviations Techniques and Applications, 2nd edn. Springer, New York (1998)

    Book  Google Scholar 

  3. 3.

    den Hollander, F.: Large Deviations. AMS, Providence, RI (2000). Fields Institute Monograph

    MATH  Google Scholar 

  4. 4.

    Jack, R.L., Sollich, P.: Large deviations and ensembles of trajectories in stochastic models. Prog. Theor. Phys. Suppl. 184, 304–317 (2010)

    ADS  Article  Google Scholar 

  5. 5.

    Chetrite, R., Touchette, H.: Nonequilibrium microcanonical and canonical ensembles and their equivalence. Phys. Rev. Lett. 111, 120601 (2013)

    ADS  Article  Google Scholar 

  6. 6.

    Chetrite, R., Touchette, H.: Nonequilibrium Markov processes conditioned on large deviations. Ann. Henri Poincaré 16(9), 2005–2057 (2015)

    ADS  MathSciNet  Article  Google Scholar 

  7. 7.

    Touchette, H.: The large deviation approach to statistical mechanics. Phys. Rep. 478(1–3), 1–69 (2009)

    ADS  MathSciNet  Article  Google Scholar 

  8. 8.

    Derrida, B.: Non-equilibrium steady states: fluctuations and large deviations of the density and of the current. J. Stat. Mech. 2007(07), P07023 (2007)

    MathSciNet  Article  Google Scholar 

  9. 9.

    Harris, R.J., Touchette, H.: Large deviation approach to nonequilibrium systems. In: Klages, R., Just, W., Jarzynski, C. (eds.) Nonequilibrium Statistical Physics of Small Systems: Fluctuation Relations and Beyond, Reviews of Nonlinear Dynamics and Complexity, vol. 6, pp. 335–360. Wiley-VCH, Weinheim (2013)

    Google Scholar 

  10. 10.

    Harris, R.J., Schütz, G.M.: Fluctuation theorems for stochastic dynamics. J. Stat. Mech. 2007(07), P07020 (2007)

    MathSciNet  Article  Google Scholar 

  11. 11.

    Garrahan, J.P., Jack, R.L., Lecomte, V., Pitard, E., van Duijvendijk, K., van Wijland, F.: Dynamical first-order phase transition in kinetically constrained models of glasses. Phys. Rev. Lett. 98(19), 195702 (2007)

    ADS  Article  Google Scholar 

  12. 12.

    Hedges, L.O., Jack, R.L., Garrahan, J.P., Chandler, D.: Dynamic order-disorder in atomistic models of structural glass formers. Science 323, 1309–1313 (2009)

    ADS  Article  Google Scholar 

  13. 13.

    Espigares, C.P., Garrido, P.L., Hurtado, P.I.: Dynamical phase transition for current statistics in a simple driven diffusive system. Phys. Rev. E 87, 032115 (2013)

    ADS  Article  Google Scholar 

  14. 14.

    Aminov, A., Bunin, G., Kafri, Y.: Singularities in large deviation functionals of bulk-driven transport models. J. Stat. Mech. 2014(8), P08017 (2014)

    MathSciNet  Article  Google Scholar 

  15. 15.

    Dean, T., Dupuis, P.: Splitting for rare event simulation: a large deviation approach to design and analysis. Stoch. Proc. Appl. 119(2), 562–587 (2009)

    MathSciNet  Article  Google Scholar 

  16. 16.

    Cérou, F., Guyader, A.: Adaptive multilevel splitting for rare event analysis. Stoch. Anal. Appl. 25(2), 417–443 (2007)

    MathSciNet  Article  Google Scholar 

  17. 17.

    Aristoff, D., Lelièvre, T., Mayne, C.G., Teo, I.: Adaptive multilevel splitting in molecular dynamics simulations. ESAIM Proc 48, 215–225 (2015)

    MathSciNet  Article  Google Scholar 

  18. 18.

    Grassberger, P.: Go with the winners: a general Monte Carlo strategy. Comp. Phys. Comm. 147(1–2), 64–70 (2002)

    ADS  MathSciNet  Article  Google Scholar 

  19. 19.

    Giardina, C., Kurchan, J., Peliti, L.: Direct evaluation of large-deviation functions. Phys. Rev. Lett. 96(12), 120603 (2006)

    ADS  Article  Google Scholar 

  20. 20.

    Lecomte, V., Tailleur, J.: A numerical approach to large deviations in continuous time. J. Stat. Mech. 2007(03), P03004 (2007)

    Article  Google Scholar 

  21. 21.

    Bucklew, J.A.: Introduction to Rare Event Simulation. Springer, New York (2004)

    Book  Google Scholar 

  22. 22.

    Juneja, S., Shahabuddin, P.: Rare-Event Simulation Techniques: An Introduction and Recent Advances, vol. 13, pp. 291–350. Elsevier, Amsterdam (2006)

    Google Scholar 

  23. 23.

    Asmussen, S., Glynn, P.W.: Stochastic Simulation: Algorithms and Analysis. Stochastic Modelling and Applied Probability. Springer, New York (2007)

    MATH  Google Scholar 

  24. 24.

    Bolhuis, P.G., Chandler, D., Dellago, C., Geissler, P.L.: Transition path sampling: throwing ropes over rough mountain passes, in the dark. Ann. Rev. Phys. Chem. 53(1), 291–318 (2002)

    ADS  Article  Google Scholar 

  25. 25.

    Heymann, M., Vanden Eijnden, E.: Pathways of maximum likelihood for rare events in nonequilibrium systems: application to nucleation in the presence of shear. Phys. Rev. Lett. 100, 140601 (2008)

    ADS  Article  Google Scholar 

  26. 26.

    Vanden-Eijnden, E., Weare, J.: Rare event simulation of small noise diffusions. Commun. Pure Appl. Math. 65(12), 1770–1803 (2012)

    MathSciNet  Article  Google Scholar 

  27. 27.

    Grafke, T., Grauer, R., Schäfer, T.: The instanton method and its numerical implementation in fluid mechanics. J. Phys. A 48(33), 333001 (2015)

    MathSciNet  Article  Google Scholar 

  28. 28.

    Borkar, V.S., Juneja, S., Kherani, A.A.: Peformance analysis conditioned on rare events: an adaptive simulation scheme. Commun. Info. Syst. 3(4), 259–278 (2004)

    MATH  Google Scholar 

  29. 29.

    Ahamed, T.P.I., Borkar, V.S., Juneja, S.: Adaptive importance sampling technique for Markov chains using stochastic approximation. Oper. Res. 54(3), 489–504 (2006)

    MathSciNet  Article  Google Scholar 

  30. 30.

    Basu, A., Bhattacharyya, T., Borkar, V.S.: A learning algorithm for risk-sensitive cost. Math. Oper. Res. 33(4), 880–898 (2008)

    MathSciNet  Article  Google Scholar 

  31. 31.

    Borkar, V.S.: Learning algorithms for risk-sensitive control. In: Proc. 19th Int. Symp. Math. Theory Networks and Systems, pp. 1327–1332 (2010)

  32. 32.

    Chetrite, R., Touchette, H.: Variational and optimal control representations of conditioned and driven processes. J. Stat. Mech. 2015(12), P12001 (2015)

    MathSciNet  Article  Google Scholar 

  33. 33.

    Chauveau, D., Diebolt, J.: Estimation of the asymptotic variance in the CLT for Markov chains. Stoch. Models 19(4), 449–465 (2003)

    MathSciNet  Article  Google Scholar 

  34. 34.

    Roberts, G.O., Rosenthal, J.S.: General state space Markov chains and MCMC algorithms. Prob. Surv. 1, 20–71 (2004)

    MathSciNet  Article  Google Scholar 

  35. 35.

    Benveniste, A., Métivier, M., Priouret, P.: Adaptive Algorithms and Stochastic Approximations. Stochastic Modelling and Applied Probability, vol. 22. Springer, New York (2012)

    MATH  Google Scholar 

  36. 36.

    Ferré, G., Stoltz, G.: Error estimates on ergodic properties of discretized Feynman–Kac semigroups (2017). arXiv:1712.04013

  37. 37.

    Pavliotis, G.A.: Stochastic Processes and Applications. Springer, New York (2014)

    Book  Google Scholar 

  38. 38.

    Chernyak, V.Y., Chertkov, M., Bierkens, J., Kappen, H.J.: Stochastic optimal control as non-equilibrium statistical mechanics: calculus of variations over density and current. J. Phys. A 47(2), 022001 (2014)

    ADS  Article  Google Scholar 

  39. 39.

    Sekimoto, K.: Stochastic Energetics. Lect. Notes. Phys., vol. 799. Springer, New York (2010)

    MATH  Google Scholar 

  40. 40.

    Bierkens, J., Chernyak, V.Y., Chertkov, M., Kappen, H.J.: Linear PDEs and eigenvalue problems corresponding to ergodic stochastic optimization problems on compact manifolds. J. Stat. Mech. 2016, 013206 (2016)

    MathSciNet  Article  Google Scholar 

  41. 41.

    Lelièvre, T., Stoltz, G.: Partial differential equations and stochastic methods in molecular dynamics. Acta Numer. 25, 681–880 (2016)

    MathSciNet  Article  Google Scholar 

  42. 42.

    Chatelin, F.: Spectral Approximation of Linear Operators. Classics in Applied Mathematics. SIAM, Philadelphia (2011)

    Book  Google Scholar 

  43. 43.

    Gorissen, M., Vanderzande, C.: Finite size scaling of current fluctuations in the totally asymmetric exclusion process. J. Phys. A 44(11), 115005 (2011)

    ADS  MathSciNet  Article  Google Scholar 

  44. 44.

    Fleming, W.H., Soner, H.M.: Controlled Markov Processes and Viscosity Solutions, Stochastic Modelling and Applied Probability, vol. 25. Springer, New York (2006)

    MATH  Google Scholar 

  45. 45.

    Rousset, M.: On the control of an interacting particle estimation of Schrödinger ground states. SIAM J. Math. Anal. 38(3), 824–844 (2006)

    MathSciNet  Article  Google Scholar 

  46. 46.

    Nemoto, T., Bouchet, F., Jack, R.L., Lecomte, V.: Population-dynamics method with a multicanonical feedback control. Phys. Rev. E 93, 062123 (2016)

    ADS  MathSciNet  Article  Google Scholar 

  47. 47.

    Nemoto, T., Hidalgo, E.G., Lecomte, V.: Finite-time and finite-size scalings in the evaluation of large-deviation functions: analytical study using a birth-death process. Phys. Rev. E 95, 012102 (2017)

    ADS  MathSciNet  Article  Google Scholar 

  48. 48.

    Kloeden, P.E., Platen, E.: Numerical Solution of Stochastic Differential Equations. Springer, Berlin (1992)

    Book  Google Scholar 

  49. 49.

    Demmel, J.: Applied Numerical Linear Algebra. SIAM, Philadelphia (1997)

    Book  Google Scholar 

  50. 50.

    Polyak, B.T., Juditsky, A.B.: Acceleration of stochastic approximation by averaging. SIAM J. Cont. Opt. 30(4), 838–855 (1992)

    MathSciNet  Article  Google Scholar 

  51. 51.

    Hartmann, C., Schütte, C.: Efficient rare event simulation by optimal nonequilibrium forcing. J. Stat. Mech. 2012(11), P11004 (2012)

    Article  Google Scholar 

  52. 52.

    Hartmann, C., Banisch, R., Sarich, M., Badowski, T., Schütte, C.: Characterization of rare events in molecular dynamics. Entropy 16(1), 350 (2014)

    ADS  MathSciNet  Article  Google Scholar 

  53. 53.

    Zhang, W., Wang, H., Hartmann, C., Weber, M., Schütte, C.: Applications of the cross-entropy method to importance sampling and optimal control of diffusions. SIAM J. Sci. Compd. 36(6), A2654–A2672 (2014)

    MathSciNet  Article  Google Scholar 

  54. 54.

    Rohwer, C.M., Angeletti, F., Touchette, H.: Convergence of large deviation estimators. Phys. Rev. E 92, 052104 (2015)

    ADS  Article  Google Scholar 

  55. 55.

    Nemoto, T., Sasa, S.I.: Computation of large deviation statistics via iterative measurement-and-feedback procedure. Phys. Rev. Lett. 112, 090602 (2014)

    ADS  Article  Google Scholar 

  56. 56.

    Risken, H.: The Fokker-Planck Equation: Methods of Solution and Applications, 3rd edn. Springer, Berlin (1996)

    MATH  Google Scholar 

  57. 57.

    Reimann, P.: Brownian motors: noisy transport far from equilibrium. Phys. Rep. 361, 57–265 (2002)

    ADS  MathSciNet  Article  Google Scholar 

  58. 58.

    Ciliberto, S., Joubaud, S., Petrosyan, A.: Fluctuations in out-of-equilibrium systems: from theory to experiment. J. Stat. Mech. 2010(12), P12003 (2010)

    Article  Google Scholar 

  59. 59.

    Tsobgni Nyawo, P., Touchette, H.: Large deviations of the current for driven periodic diffusions. Phys. Rev. E 94, 032101 (2016)

    ADS  Article  Google Scholar 

  60. 60.

    Mehl, J., Speck, T., Seifert, U.: Large deviation function for entropy production in driven one-dimensional systems. Phys. Rev. E 78, 011123 (2008)

    ADS  Article  Google Scholar 

  61. 61.

    Nemoto, T., Sasa, S.I.: Variational formula for experimental determination of high-order correlations of current fluctuations in driven systems. Phys. Rev. E 83, 030105 (2011)

    ADS  Article  Google Scholar 

  62. 62.

    Dupuis, P., Wang, H.: Dynamic importance sampling for uniformly recurrent Markov chains. Ann. Appl. Prob. 15(1A), 1–38 (2005)

    MathSciNet  Article  Google Scholar 

  63. 63.

    Nemoto, T., Jack, R.L., Lecomte, V.: Finite-size scaling of a first-order dynamical phase transition: adaptive population dynamics and an effective model. Phys. Rev. Lett. 118, 115702 (2017)

    ADS  Article  Google Scholar 

  64. 64.

    Blanchet, J., Lam, H.: State-dependent importance sampling for rare-event simulation: an overview and recent advances. Surv. Oper. Res. Manag. Sci. 17(1), 38–59 (2012)

    MathSciNet  Google Scholar 

  65. 65.

    Kappen, H.J., Ruiz, H.C.: Adaptive importance sampling for control and inference. J. Stat. Phys. 162(5), 1244–1266 (2016)

    ADS  MathSciNet  Article  Google Scholar 

  66. 66.

    Foulkes, W.M.C., Mitas, L., Needs, R.J., Rajagopal, G.: Quantum Monte Carlo simulations of solids. Rev. Mod. Phys. 73(1), 33 (2001)

    ADS  Article  Google Scholar 

  67. 67.

    Lim, L.H., Weare, J.: Fast randomized iteration: diffusion Monte Carlo through the lens of numerical linear algebra. SIAM Rev. 59(3), 547–587 (2017)

    MathSciNet  Article  Google Scholar 

Download references

Acknowledgements

We are grateful to Florian Angeletti for useful discussions in the initial phase of this work. G.F. is supported by the Labex Bezout. H.T. was supported by the National Research Foundation of South Africa (Grant Nos. 90322 and 96199) and Stellenbosch University (Project Funding for New Appointee). This research was also supported in part by the International Centre for Theoretical Sciences (ICTS) during a visit for participating in the program “Large deviation theory in statistical physics: Recent advances and future challenges” (Code: ICTS/Prog-ldt/2017/8).

Author information

Affiliations

Authors

Corresponding author

Correspondence to Hugo Touchette.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Ferré, G., Touchette, H. Adaptive Sampling of Large Deviations. J Stat Phys 172, 1525–1544 (2018). https://doi.org/10.1007/s10955-018-2108-8

Download citation

Keywords

  • Large deviations
  • Rare event simulation
  • Diffusions
  • Nonequilibrium processes