Journal of Statistical Physics

, Volume 171, Issue 2, pp 220–268 | Cite as

Convergence and Efficiency of Adaptive Importance Sampling Techniques with Partial Biasing

  • G. Fort
  • B. Jourdain
  • T. Lelièvre
  • G. Stoltz


We propose a new Monte Carlo method to efficiently sample a multimodal distribution (known up to a normalization constant). We consider a generalization of the discrete-time Self Healing Umbrella Sampling method, which can also be seen as a generalization of well-tempered metadynamics. The dynamics is based on an adaptive importance technique. The importance function relies on the weights (namely the relative probabilities) of disjoint sets which form a partition of the space. These weights are unknown but are learnt on the fly yielding an adaptive algorithm. In the context of computational statistical physics, the logarithm of these weights is, up to an additive constant, the free-energy, and the discrete valued function defining the partition is called the collective variable. The algorithm falls into the general class of Wang–Landau type methods, and is a generalization of the original Self Healing Umbrella Sampling method in two ways: (i) the updating strategy leads to a larger penalization strength of already visited sets in order to escape more quickly from metastable states, and (ii) the target distribution is biased using only a fraction of the free-energy, in order to increase the effective sample size and reduce the variance of importance sampling estimators. We prove the convergence of the algorithm and analyze numerically its efficiency on a toy example.


Wang Landau Self Healing Umbrella Sampling Metadynamics Convergence of stochastic approximation algorithms 



We would like to thank Brad Dickson, Alessandro Laio and Michele Parrinello for useful discussions. This work is supported by the European Research Council under the European Union’s Seventh Framework Programme (FP/2007-2013) / ERC Grant Agreement number 614492 and by the French National Research Agency under the grant ANR-14-CE23-0012 (COSMOS). We also benefited from the scientific environment of the Laboratoire International Associé between the Centre National de la Recherche Scientifique and the University of Illinois at Urbana-Champaign.


  1. 1.
    Andrieu, C., Moulines, E., Priouret, P.: Stability of stochastic approximation under verifiable conditions. SIAM J. Control Optim. 44, 283–312 (2005)MathSciNetCrossRefzbMATHGoogle Scholar
  2. 2.
    Atchadé, Y.F., Liu, J.S.: The Wang-Landau algorithm for Monte Carlo computation in general state spaces. Stat. Sin. 20(1), 209–233 (2010)zbMATHGoogle Scholar
  3. 3.
    Barducci, A., Bussi, G., Parrinello, M.: Well-tempered metadynamics: a smoothly converging and tunable free-energy method. Phys. Rev. Lett. 100, 020603 (2008)ADSCrossRefGoogle Scholar
  4. 4.
    Benveniste, A., Metivier, M., Priouret, P.: Adaptive Algorithms and Stochastic Approximations. Springer, Berlin (1987)zbMATHGoogle Scholar
  5. 5.
    Borkar, V.S.: Stochastic Approximation: A Dynamical Systems Viewpoint. Cambridge University Press, Cambridge (2008)CrossRefzbMATHGoogle Scholar
  6. 6.
    Brooks, S., Gelman, A., Jones, G.L., Meng, X.-L.: Handbook of Markov Chain Monte Carlo. Chapman & Hall, Boca Raton (2011)CrossRefzbMATHGoogle Scholar
  7. 7.
    Bussi, G., Laio, A., Parrinello, M.: Equilibrium free energies from nonequilibrium metadynamics. Phys. Rev. Lett. 96, 090601 (2006)ADSCrossRefGoogle Scholar
  8. 8.
    Chen, H.: Stochastic Approximation and Its Applications. Kluwer Academic Publishers, Boston (2002)zbMATHGoogle Scholar
  9. 9.
    Chipot, C., Pohorille, A. (eds): Free-Energy Calculations. Springer Series in Chemical Physics, vol. 86. Springer, Berlin (2007)Google Scholar
  10. 10.
    Chopin, N., Lelièvre, T., Stoltz, G.: Free-energy methods for Bayesian inference: efficient exploration of univariate Gaussian mixture posteriors. Stat. Comput. 22(4), 897–916 (2012)MathSciNetCrossRefzbMATHGoogle Scholar
  11. 11.
    Crespo, Y., Marinelli, F., Pietrucci, F., Laio, A.: Metadynamics convergence law in a multidimensional system. Phys. Rev. E 81(5), 055701 (2010)ADSCrossRefGoogle Scholar
  12. 12.
    Dama, J.F., Hocky, G.M., Sun, R., Voth, G.A.: Exploring valleys without climbing every peak: more efficient and forgiving metabasin metadynamics via robust on-the-fly bias domain restriction. J. Chem. Theory Comput. 11(12), 5638–5650 (2015)CrossRefGoogle Scholar
  13. 13.
    Dama, J.F., Parrinello, M., Voth, G.A.: Well-tempered metadynamics converges asymptotically. Phys. Rev. Lett. 112, 240602 (2014)ADSCrossRefGoogle Scholar
  14. 14.
    Darve, E., Pohorille, A.: Calculating free energies using average force. J. Chem. Phys. 115(20), 9169–9183 (2001)ADSCrossRefGoogle Scholar
  15. 15.
    Dickson, B.: \(\mu \)-tempered metadynamics: artifact independent convergence times for wide hills. J. Chem. Phys. 143(23), 234109 (2015)ADSCrossRefGoogle Scholar
  16. 16.
    Fort, G., Jourdain, B., Kuhn, E., Lelièvre, T., Stoltz, G.: Efficiency of the Wang-Landau algorithm: a simple test case. Appl. Math. Res. Express 2014(2), 275–311 (2014)MathSciNetzbMATHGoogle Scholar
  17. 17.
    Fort, G.: Central limit theorems for stochastic approximation with controlled Markov chain dynamics. ESAIM: PS 19, 60–80 (2015)MathSciNetCrossRefzbMATHGoogle Scholar
  18. 18.
    Fort, G., Jourdain, B., Kuhn, E., Lelièvre, T., Stoltz, G.: Convergence of the Wang-Landau algorithm. Math. Comput. 84(295), 2297–2327 (2015)MathSciNetCrossRefzbMATHGoogle Scholar
  19. 19.
    Fort, G., Jourdain, B., Lelièvre, T., Stoltz, G.: Self-healing umbrella sampling: convergence and efficiency. Stat. Comput. 27(1), 147–168 (2017)MathSciNetCrossRefzbMATHGoogle Scholar
  20. 20.
    Fort, G., Moulines, E., Priouret, P.: Convergence of adaptive and interacting Markov chain Monte Carlo algorithms. Ann. Stat. 39(6), 3262–3289 (2012)MathSciNetCrossRefzbMATHGoogle Scholar
  21. 21.
    Fort, G., Moulines, E., Schreck, A., Vihola, M.: Convergence of Markovian stochastic approximation with discontinuous dynamics. SIAM J. Control Optim. 54(2), 866–893 (2016)MathSciNetCrossRefzbMATHGoogle Scholar
  22. 22.
    Hall, P., Heyde, P.P.: Martingale Limit Theory and Its Application. Academic Press, New York (1980)zbMATHGoogle Scholar
  23. 23.
    Hastings, W.K.: Monte Carlo sampling methods using Markov chains and their applications. Biometrika 57, 97–109 (1970)MathSciNetCrossRefzbMATHGoogle Scholar
  24. 24.
    Hénin, J., Chipot, C.: Overcoming free-energy barriers using unconstrained molecular dynamics simulations. J. Chem. Phys. 121(7), 2904–2914 (2004)ADSCrossRefGoogle Scholar
  25. 25.
    Jacob, P.E., Ryder, R.J.: The Wang-Landau algorithm reaches the flat histogram criterion in finite time. Ann. Appl. Probab. 24(1), 34–53 (2014)MathSciNetCrossRefzbMATHGoogle Scholar
  26. 26.
    Jourdain, B., Lelièvre, T., Roux, R.: Existence, uniqueness and convergence of a particle approximation for the adaptive biasing force process. ESAIM: M2AN 44(5), 831–865 (2010)MathSciNetCrossRefzbMATHGoogle Scholar
  27. 27.
    Kong, A., Liu, J.S., Wong, W.H.: Sequential imputation and Bayesian missing data problems. J. Am. Statist. Assoc. 89, 278–288 (1994)CrossRefzbMATHGoogle Scholar
  28. 28.
    Kushner, H.: Stochastic approximation: a survey. Wiley Interdiscip. Rev. 2(1), 87–96 (2010)CrossRefGoogle Scholar
  29. 29.
    Kushner, H.J., Yin, G.G.: Stochastic Approximation and Recursive Algorithms and Applications. Springer, New York (2003)zbMATHGoogle Scholar
  30. 30.
    Laio, A., Parrinello, M.: Escaping free-energy minima. Proc. Natl. Acad. Sci. USA 99, 12562–12566 (2002)ADSCrossRefGoogle Scholar
  31. 31.
    Lelièvre, T., Minoukadeh, K.: Long-time convergence of an adaptive biasing force method: the bi-channel case. Arch. Ration. Mech. Anal. 202(1), 1–34 (2011)MathSciNetCrossRefzbMATHGoogle Scholar
  32. 32.
    Lelièvre, T., Rousset, M., Stoltz, G.: Long-time convergence of an adaptive biasing force method. Nonlinearity 21, 1155–1181 (2008)MathSciNetCrossRefzbMATHGoogle Scholar
  33. 33.
    Lelièvre, T., Rousset, M., Stoltz, G.: Free Energy Computations: A Mathematical Perspective. Imperial College Press, London (2010)CrossRefzbMATHGoogle Scholar
  34. 34.
    Marsili, S., Barducci, A., Chelli, R., Procacci, P., Schettino, V.: Self-healing Umbrella Sampling: a non-equilibrium approach for quantitative free-energy calculations. J. Phys. Chem. B 110(29), 14011–14013 (2006)CrossRefGoogle Scholar
  35. 35.
    McCarty, J., Valsson, O., Tiwary, P., Parrinello, M.: Variationally optimized free-energy flooding for rate calculation. Phys. Rev. Lett. 115(7), 070601 (2015)ADSCrossRefGoogle Scholar
  36. 36.
    McGovern, M., de Pablo, J.: A boundary correction algorithm for metadynamics in multiple dimensions. J. Chem. Phys. 139(8), 084102 (2013)ADSCrossRefGoogle Scholar
  37. 37.
    Metropolis, N., Rosenbluth, A.W., Rosenbluth, M.N., Teller, A.H., Teller, E.: Equations of state calculations by fast computing machines. J. Chem. Phys. 21(6), 1087–1091 (1953)ADSCrossRefGoogle Scholar
  38. 38.
    Metzner, P., Schütte, C., Vanden-Eijnden, E.: Illustration of transition path theory on a collection of simple examples. J. Chem. Phys. 125(1), 084110 (2006)ADSCrossRefGoogle Scholar
  39. 39.
    Meyn, S., Tweedie, R.L.: Markov Chains and Stochastic Stability. Springer, London (2009)CrossRefzbMATHGoogle Scholar
  40. 40.
    Minoukadeh, K., Chipot, C., Lelièvre, T.: Potential of mean force calculations: a multiple-walker adaptive biasing force approach. J. Chem. Theory. Comput. 6(4), 1008–1017 (2010)CrossRefGoogle Scholar
  41. 41.
    Park, S., Sener, M.K., Lu, D., Schulten, K.: Reaction paths based on mean first-passage times. J. Chem. Phys. 119(3), 1313–1319 (2003)ADSCrossRefGoogle Scholar
  42. 42.
    Polyak, B.T., Juditsky, A.B.: Acceleration of stochastic approximation by averaging. SIAM J. Control Optim. 30(4), 838–855 (1992)MathSciNetCrossRefzbMATHGoogle Scholar
  43. 43.
    Ruppert, D.: Efficient estimations from a slowly convergent Robbins-Monro process. Technical Report 781, Cornell University Operations Research and Industrial Engineering (1988)Google Scholar
  44. 44.
    Wang, F., Landau, D.P.: Determining the density of states for classical statistical models: a random walk algorithm to produce a flat histogram. Phys. Rev. E 64, 056101 (2001)ADSCrossRefGoogle Scholar
  45. 45.
    Wang, F., Landau, D.P.: Efficient, multiple-range random walk algorithm to calculate the density of states. Phys. Rev. Lett. 86(10), 2050–2053 (2001)ADSCrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  • G. Fort
    • 1
    • 2
  • B. Jourdain
    • 3
  • T. Lelièvre
    • 3
  • G. Stoltz
    • 3
  1. 1.LTCI, CNRS, Télécom ParisTechUniversité Paris-SaclayParisFrance
  2. 2.IMTUniversité de Toulouse et CNRSToulouse Cedex 9France
  3. 3.Université Paris-Est, CERMICS (ENPC), INRIAMarne-la-ValléeFrance

Personalised recommendations