Statistics and Computing

, Volume 25, Issue 6, pp 1217–1232 | Cite as

A simulated annealing approach to approximate Bayes computations

  • Carlo Albert
  • Hans R. Künsch
  • Andreas Scheidegger
Article

Abstract

Approximate Bayes computations (ABC) are used for parameter inference when the likelihood function of the model is expensive to evaluate but relatively cheap to sample from. In particle ABC, an ensemble of particles in the product space of model outputs and parameters is propagated in such a way that its output marginal approaches a delta function at the data and its parameter marginal approaches the posterior distribution. Inspired by Simulated Annealing, we present a new class of particle algorithms for ABC, based on a sequence of Metropolis kernels, associated with a decreasing sequence of tolerances w.r.t. the data. Unlike other algorithms, our class of algorithms is not based on importance sampling. Hence, it does not suffer from a loss of effective sample size due to re-sampling. We prove convergence under a condition on the speed at which the tolerance is decreased. Furthermore, we present a scheme that adapts the tolerance and the jump distribution in parameter space according to some mean-fields of the ensemble, which preserves the statistical independence of the particles, in the limit of infinite sample size. This adaptive scheme aims at converging as close as possible to the correct result with as few system updates as possible via minimizing the entropy production of the process. The performance of this new class of algorithms is compared against two other recent algorithms on two toy examples as well as on a real-world example from genetics.

Keywords

Approximate Bayes computations Simulated annealing Non-equilibrium thermodynamics Entropy 

References

  1. Andresen, B., Hoffmann, K.H., Mosegaard, K., Nulton, J., Pedersen, J.M., Salamon, P.: On lumped models for thermodynamic properties of simulated annealing problems. J. Phys. 49(9), 1485–1492 (1988)CrossRefGoogle Scholar
  2. Beaumont, M.A., Cornuet, J.M., Marin, J.M., Robert, C.P.: Adaptive approximate Bayesian computation. Biometrika 96(4), 983–990 (2009)MATHMathSciNetCrossRefGoogle Scholar
  3. Beskos, A., Crisan, D., Jasra, A.: On the Stability of Sequential Monte Carlo Methods in High Dimensions. arXiv: 1103.3965v2, (2012)
  4. Burkholder, D., Pardoux, E., Sznitman, A.: Topics in propagation of chaos. In Ecole d’Ete de Probabilites de Saint-Flour XIX—1989, volume 1464 of Lecture Notes in Mathematics, pp. 165–251. Springer, Berlin/Heidelberg, (1991). doi:10.1007/BFb0085169
  5. Del Moral, P., Doucet, A., Jasra, A.: An adaptive sequential Monte Carlo method for approximate Bayesian computation. Stat. Comput. 22(5), 1009–1020 (2012) Google Scholar
  6. Douc, R., Moulines, E., Rosenthal, J.S.: Quantitative bounds on convergence of time-inhomogeneous Markov chains. Ann. Appl. Probab. 14(4), 1643–1665 (2004)Google Scholar
  7. Fearnhead, P., Prangle, D.: Constructing summary statistics for approximate Bayesian computation: semi-automatic approximate Bayesian computation. J. R. Stat. Soc. B 74(3), 419–474 (2012)MathSciNetCrossRefGoogle Scholar
  8. Föllmer, H.: Random fields and diffusion processes. Ecole d’Ete de Probabilites de Saint-Flour XV–XVII. 1985–87, volume 1362 of Lecture Notes in Mathematics, pp. 101–203. Springer, Berlin/Heidelberg (1988)Google Scholar
  9. Jabot, F., Faure, T., Dumoullin, N.: EasyABC: EasyABC: performing efficient approximate Bayesian computation sampling schemes (2013). R package version 1.2.2Google Scholar
  10. Lee, A.: On the choice of MCMC kernels for approximate Bayesian computation with SMC samplers. In Proceedings of the 2012 Winter Simulation Conference (WSC 2012), page 12 pp. IEEE Syst., Man, Cybernetics Soc., 2012 2012. 2012 Winter Simulation Conference (WSC 2012), 9–12 Dec (2012), BerlinGoogle Scholar
  11. Lenormand, M., Jabot, F.: Adaptive approximate Bayesian computation for complex models. Stat. Comput. 28(6), 2777–2796 (2013)Google Scholar
  12. Leuenberger, C., Wegmann, D.: Bayesian computation and model selection without likelihoods. Genetics 184(2), 243–252 (2010)CrossRefGoogle Scholar
  13. Marin, J.M., Pudlo, P., Robert, C.P., Ryder, R.J.: Approximate Bayesian computational methods. Stat. Comput. 22(6, SI), 1167–1180 (2012)MATHMathSciNetCrossRefGoogle Scholar
  14. Marjoram, P., Molitor, J., Plagnol, V., Tavaré, S.: Markov chain Monte Carlo without likelihoods. Proc. Natl. Acad. Sci. USA. 100(2), 15324–15328 (2003)CrossRefGoogle Scholar
  15. Metropolis, N., Rosenbluth, A.W., Rosenbluth, M.N., Teller, A.H., Teller, E.: Equation of state calculations by fast computing machines. J. Chem. Phys. 21(6), 1087–1092 (1953)CrossRefGoogle Scholar
  16. Onsager, L.: Reciprocal relations in irreversible processes. I. Phys. Rev. 37(4), 405–426 (1931)CrossRefGoogle Scholar
  17. Rubin, M.H.: Optimal configuration of a class of irreversible heat engines I. Phys. Rev. A 19(3), 1272–1276 (1979)CrossRefGoogle Scholar
  18. Ruppeiner, G., Pedersen, J.M., Salamon, P.: Ensemble approach to simulated annealing. J. Phys. I 1, 455–470 (1991)Google Scholar
  19. Salamon, P., Nitzan, A., Andresen, B., Berry, R.S.: Minimum entropy production and the optimization of heat engines. Phys. Rev. A 21(6), 2115–2129 (1980)MathSciNetCrossRefGoogle Scholar
  20. Sedki, M., Pudlo, P., Marin J.M., Robert, C.P., Cornuet, J.M.: Efficient learning in ABC algorithms. arXiv: 1210.1388v2 [stat.CO] (2013)
  21. Seifert, U.: Entropy production along a stochastic trajectory and an integral fluctuation theorem. Phys. Rev. Lett. 95, 040602 (2005)CrossRefGoogle Scholar
  22. Spirkl, W., Ries, H.: Optimal finite-time endoreversible processes. Phys. Rev. E 52(4, A), 3485–3489 (1995)CrossRefGoogle Scholar
  23. Tanaka, M.M., Francis, A.R., Luciani, F., Sisson, S.A.: Using approximate Bayesian computation to estimate tuberculosis transmission parameters from genotype data. Genetics 173(3), 1511–1520 (2006)CrossRefGoogle Scholar
  24. Tavaré, S., Balding, D.J., Griffiths, R.C., Donnelly, P.: Inferring coalescence times from DNA sequence data. Genetics 145, 505–518 (1997)Google Scholar
  25. Toni, T., Welch, D., Strelkowa, N., Ipsen, A., Stumpf, M.P.H.: Approximate Bayesian computation scheme for parameter inference and model selection in dynamical systems. J. R. Soc. Interface 6(31), 187–202 (2009)CrossRefGoogle Scholar
  26. Weiss, G., Haeseler, A.: Inference of population history using a likelihood approach. Genetics 149, 1539–1546 (1998)Google Scholar
  27. Wilkinson, R.D.: Approximate Bayesian computation (ABC) gives exact results under the assumption of model error. Stat. Appl. Genet. Mol. Biol. 12(2), 129–141 (2013)MathSciNetGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2014

Authors and Affiliations

  • Carlo Albert
    • 1
  • Hans R. Künsch
    • 2
  • Andreas Scheidegger
    • 1
  1. 1.Eawag, Aquatic ResearchDübendorfSwitzerland
  2. 2.Seminar für StatistikETH ZürichZürichSwitzerland

Personalised recommendations