Abstract
Approximate Bayes computations (ABC) are used for parameter inference when the likelihood function of the model is expensive to evaluate but relatively cheap to sample from. In particle ABC, an ensemble of particles in the product space of model outputs and parameters is propagated in such a way that its output marginal approaches a delta function at the data and its parameter marginal approaches the posterior distribution. Inspired by Simulated Annealing, we present a new class of particle algorithms for ABC, based on a sequence of Metropolis kernels, associated with a decreasing sequence of tolerances w.r.t. the data. Unlike other algorithms, our class of algorithms is not based on importance sampling. Hence, it does not suffer from a loss of effective sample size due to re-sampling. We prove convergence under a condition on the speed at which the tolerance is decreased. Furthermore, we present a scheme that adapts the tolerance and the jump distribution in parameter space according to some mean-fields of the ensemble, which preserves the statistical independence of the particles, in the limit of infinite sample size. This adaptive scheme aims at converging as close as possible to the correct result with as few system updates as possible via minimizing the entropy production of the process. The performance of this new class of algorithms is compared against two other recent algorithms on two toy examples as well as on a real-world example from genetics.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Andresen, B., Hoffmann, K.H., Mosegaard, K., Nulton, J., Pedersen, J.M., Salamon, P.: On lumped models for thermodynamic properties of simulated annealing problems. J. Phys. 49(9), 1485–1492 (1988)
Beaumont, M.A., Cornuet, J.M., Marin, J.M., Robert, C.P.: Adaptive approximate Bayesian computation. Biometrika 96(4), 983–990 (2009)
Beskos, A., Crisan, D., Jasra, A.: On the Stability of Sequential Monte Carlo Methods in High Dimensions. arXiv: 1103.3965v2, (2012)
Burkholder, D., Pardoux, E., Sznitman, A.: Topics in propagation of chaos. In Ecole d’Ete de Probabilites de Saint-Flour XIX—1989, volume 1464 of Lecture Notes in Mathematics, pp. 165–251. Springer, Berlin/Heidelberg, (1991). doi:10.1007/BFb0085169
Del Moral, P., Doucet, A., Jasra, A.: An adaptive sequential Monte Carlo method for approximate Bayesian computation. Stat. Comput. 22(5), 1009–1020 (2012)
Douc, R., Moulines, E., Rosenthal, J.S.: Quantitative bounds on convergence of time-inhomogeneous Markov chains. Ann. Appl. Probab. 14(4), 1643–1665 (2004)
Fearnhead, P., Prangle, D.: Constructing summary statistics for approximate Bayesian computation: semi-automatic approximate Bayesian computation. J. R. Stat. Soc. B 74(3), 419–474 (2012)
Föllmer, H.: Random fields and diffusion processes. Ecole d’Ete de Probabilites de Saint-Flour XV–XVII. 1985–87, volume 1362 of Lecture Notes in Mathematics, pp. 101–203. Springer, Berlin/Heidelberg (1988)
Jabot, F., Faure, T., Dumoullin, N.: EasyABC: EasyABC: performing efficient approximate Bayesian computation sampling schemes (2013). R package version 1.2.2
Lee, A.: On the choice of MCMC kernels for approximate Bayesian computation with SMC samplers. In Proceedings of the 2012 Winter Simulation Conference (WSC 2012), page 12 pp. IEEE Syst., Man, Cybernetics Soc., 2012 2012. 2012 Winter Simulation Conference (WSC 2012), 9–12 Dec (2012), Berlin
Lenormand, M., Jabot, F.: Adaptive approximate Bayesian computation for complex models. Stat. Comput. 28(6), 2777–2796 (2013)
Leuenberger, C., Wegmann, D.: Bayesian computation and model selection without likelihoods. Genetics 184(2), 243–252 (2010)
Marin, J.M., Pudlo, P., Robert, C.P., Ryder, R.J.: Approximate Bayesian computational methods. Stat. Comput. 22(6, SI), 1167–1180 (2012)
Marjoram, P., Molitor, J., Plagnol, V., Tavaré, S.: Markov chain Monte Carlo without likelihoods. Proc. Natl. Acad. Sci. USA. 100(2), 15324–15328 (2003)
Metropolis, N., Rosenbluth, A.W., Rosenbluth, M.N., Teller, A.H., Teller, E.: Equation of state calculations by fast computing machines. J. Chem. Phys. 21(6), 1087–1092 (1953)
Onsager, L.: Reciprocal relations in irreversible processes. I. Phys. Rev. 37(4), 405–426 (1931)
Rubin, M.H.: Optimal configuration of a class of irreversible heat engines I. Phys. Rev. A 19(3), 1272–1276 (1979)
Ruppeiner, G., Pedersen, J.M., Salamon, P.: Ensemble approach to simulated annealing. J. Phys. I 1, 455–470 (1991)
Salamon, P., Nitzan, A., Andresen, B., Berry, R.S.: Minimum entropy production and the optimization of heat engines. Phys. Rev. A 21(6), 2115–2129 (1980)
Sedki, M., Pudlo, P., Marin J.M., Robert, C.P., Cornuet, J.M.: Efficient learning in ABC algorithms. arXiv: 1210.1388v2 [stat.CO] (2013)
Seifert, U.: Entropy production along a stochastic trajectory and an integral fluctuation theorem. Phys. Rev. Lett. 95, 040602 (2005)
Spirkl, W., Ries, H.: Optimal finite-time endoreversible processes. Phys. Rev. E 52(4, A), 3485–3489 (1995)
Tanaka, M.M., Francis, A.R., Luciani, F., Sisson, S.A.: Using approximate Bayesian computation to estimate tuberculosis transmission parameters from genotype data. Genetics 173(3), 1511–1520 (2006)
Tavaré, S., Balding, D.J., Griffiths, R.C., Donnelly, P.: Inferring coalescence times from DNA sequence data. Genetics 145, 505–518 (1997)
Toni, T., Welch, D., Strelkowa, N., Ipsen, A., Stumpf, M.P.H.: Approximate Bayesian computation scheme for parameter inference and model selection in dynamical systems. J. R. Soc. Interface 6(31), 187–202 (2009)
Weiss, G., Haeseler, A.: Inference of population history using a likelihood approach. Genetics 149, 1539–1546 (1998)
Wilkinson, R.D.: Approximate Bayesian computation (ABC) gives exact results under the assumption of model error. Stat. Appl. Genet. Mol. Biol. 12(2), 129–141 (2013)
Acknowledgments
The first author is indebted to Bjarne Andresen for valuable comments on the adaptive algorithm.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Albert, C., Künsch, H.R. & Scheidegger, A. A simulated annealing approach to approximate Bayes computations. Stat Comput 25, 1217–1232 (2015). https://doi.org/10.1007/s11222-014-9507-8
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11222-014-9507-8