Skip to main content

A simulated annealing approach to approximate Bayes computations

Abstract

Approximate Bayes computations (ABC) are used for parameter inference when the likelihood function of the model is expensive to evaluate but relatively cheap to sample from. In particle ABC, an ensemble of particles in the product space of model outputs and parameters is propagated in such a way that its output marginal approaches a delta function at the data and its parameter marginal approaches the posterior distribution. Inspired by Simulated Annealing, we present a new class of particle algorithms for ABC, based on a sequence of Metropolis kernels, associated with a decreasing sequence of tolerances w.r.t. the data. Unlike other algorithms, our class of algorithms is not based on importance sampling. Hence, it does not suffer from a loss of effective sample size due to re-sampling. We prove convergence under a condition on the speed at which the tolerance is decreased. Furthermore, we present a scheme that adapts the tolerance and the jump distribution in parameter space according to some mean-fields of the ensemble, which preserves the statistical independence of the particles, in the limit of infinite sample size. This adaptive scheme aims at converging as close as possible to the correct result with as few system updates as possible via minimizing the entropy production of the process. The performance of this new class of algorithms is compared against two other recent algorithms on two toy examples as well as on a real-world example from genetics.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3

References

  1. Andresen, B., Hoffmann, K.H., Mosegaard, K., Nulton, J., Pedersen, J.M., Salamon, P.: On lumped models for thermodynamic properties of simulated annealing problems. J. Phys. 49(9), 1485–1492 (1988)

    Article  Google Scholar 

  2. Beaumont, M.A., Cornuet, J.M., Marin, J.M., Robert, C.P.: Adaptive approximate Bayesian computation. Biometrika 96(4), 983–990 (2009)

    MATH  MathSciNet  Article  Google Scholar 

  3. Beskos, A., Crisan, D., Jasra, A.: On the Stability of Sequential Monte Carlo Methods in High Dimensions. arXiv: 1103.3965v2, (2012)

  4. Burkholder, D., Pardoux, E., Sznitman, A.: Topics in propagation of chaos. In Ecole d’Ete de Probabilites de Saint-Flour XIX—1989, volume 1464 of Lecture Notes in Mathematics, pp. 165–251. Springer, Berlin/Heidelberg, (1991). doi:10.1007/BFb0085169

  5. Del Moral, P., Doucet, A., Jasra, A.: An adaptive sequential Monte Carlo method for approximate Bayesian computation. Stat. Comput. 22(5), 1009–1020 (2012)

  6. Douc, R., Moulines, E., Rosenthal, J.S.: Quantitative bounds on convergence of time-inhomogeneous Markov chains. Ann. Appl. Probab. 14(4), 1643–1665 (2004)

  7. Fearnhead, P., Prangle, D.: Constructing summary statistics for approximate Bayesian computation: semi-automatic approximate Bayesian computation. J. R. Stat. Soc. B 74(3), 419–474 (2012)

    MathSciNet  Article  Google Scholar 

  8. Föllmer, H.: Random fields and diffusion processes. Ecole d’Ete de Probabilites de Saint-Flour XV–XVII. 1985–87, volume 1362 of Lecture Notes in Mathematics, pp. 101–203. Springer, Berlin/Heidelberg (1988)

  9. Jabot, F., Faure, T., Dumoullin, N.: EasyABC: EasyABC: performing efficient approximate Bayesian computation sampling schemes (2013). R package version 1.2.2

  10. Lee, A.: On the choice of MCMC kernels for approximate Bayesian computation with SMC samplers. In Proceedings of the 2012 Winter Simulation Conference (WSC 2012), page 12 pp. IEEE Syst., Man, Cybernetics Soc., 2012 2012. 2012 Winter Simulation Conference (WSC 2012), 9–12 Dec (2012), Berlin

  11. Lenormand, M., Jabot, F.: Adaptive approximate Bayesian computation for complex models. Stat. Comput. 28(6), 2777–2796 (2013)

  12. Leuenberger, C., Wegmann, D.: Bayesian computation and model selection without likelihoods. Genetics 184(2), 243–252 (2010)

    Article  Google Scholar 

  13. Marin, J.M., Pudlo, P., Robert, C.P., Ryder, R.J.: Approximate Bayesian computational methods. Stat. Comput. 22(6, SI), 1167–1180 (2012)

    MATH  MathSciNet  Article  Google Scholar 

  14. Marjoram, P., Molitor, J., Plagnol, V., Tavaré, S.: Markov chain Monte Carlo without likelihoods. Proc. Natl. Acad. Sci. USA. 100(2), 15324–15328 (2003)

    Article  Google Scholar 

  15. Metropolis, N., Rosenbluth, A.W., Rosenbluth, M.N., Teller, A.H., Teller, E.: Equation of state calculations by fast computing machines. J. Chem. Phys. 21(6), 1087–1092 (1953)

    Article  Google Scholar 

  16. Onsager, L.: Reciprocal relations in irreversible processes. I. Phys. Rev. 37(4), 405–426 (1931)

    Article  Google Scholar 

  17. Rubin, M.H.: Optimal configuration of a class of irreversible heat engines I. Phys. Rev. A 19(3), 1272–1276 (1979)

    Article  Google Scholar 

  18. Ruppeiner, G., Pedersen, J.M., Salamon, P.: Ensemble approach to simulated annealing. J. Phys. I 1, 455–470 (1991)

    Google Scholar 

  19. Salamon, P., Nitzan, A., Andresen, B., Berry, R.S.: Minimum entropy production and the optimization of heat engines. Phys. Rev. A 21(6), 2115–2129 (1980)

    MathSciNet  Article  Google Scholar 

  20. Sedki, M., Pudlo, P., Marin J.M., Robert, C.P., Cornuet, J.M.: Efficient learning in ABC algorithms. arXiv: 1210.1388v2 [stat.CO] (2013)

  21. Seifert, U.: Entropy production along a stochastic trajectory and an integral fluctuation theorem. Phys. Rev. Lett. 95, 040602 (2005)

    Article  Google Scholar 

  22. Spirkl, W., Ries, H.: Optimal finite-time endoreversible processes. Phys. Rev. E 52(4, A), 3485–3489 (1995)

    Article  Google Scholar 

  23. Tanaka, M.M., Francis, A.R., Luciani, F., Sisson, S.A.: Using approximate Bayesian computation to estimate tuberculosis transmission parameters from genotype data. Genetics 173(3), 1511–1520 (2006)

    Article  Google Scholar 

  24. Tavaré, S., Balding, D.J., Griffiths, R.C., Donnelly, P.: Inferring coalescence times from DNA sequence data. Genetics 145, 505–518 (1997)

    Google Scholar 

  25. Toni, T., Welch, D., Strelkowa, N., Ipsen, A., Stumpf, M.P.H.: Approximate Bayesian computation scheme for parameter inference and model selection in dynamical systems. J. R. Soc. Interface 6(31), 187–202 (2009)

    Article  Google Scholar 

  26. Weiss, G., Haeseler, A.: Inference of population history using a likelihood approach. Genetics 149, 1539–1546 (1998)

    Google Scholar 

  27. Wilkinson, R.D.: Approximate Bayesian computation (ABC) gives exact results under the assumption of model error. Stat. Appl. Genet. Mol. Biol. 12(2), 129–141 (2013)

    MathSciNet  Google Scholar 

Download references

Acknowledgments

The first author is indebted to Bjarne Andresen for valuable comments on the adaptive algorithm.

Author information

Affiliations

Authors

Corresponding author

Correspondence to Carlo Albert.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Albert, C., Künsch, H.R. & Scheidegger, A. A simulated annealing approach to approximate Bayes computations. Stat Comput 25, 1217–1232 (2015). https://doi.org/10.1007/s11222-014-9507-8

Download citation

Keywords

  • Approximate Bayes computations
  • Simulated annealing
  • Non-equilibrium thermodynamics
  • Entropy