Skip to main content
Log in

Likelihood-free parallel tempering

  • Published:
Statistics and Computing Aims and scope Submit manuscript

Abstract

Approximate Bayesian Computational (ABC) methods, or likelihood-free methods, have appeared in the past fifteen years as useful methods to perform Bayesian analysis when the likelihood is analytically or computationally intractable. Several ABC methods have been proposed: MCMC methods have been developed by Marjoram et al. (2003) and by Bortot et al. (2007) for instance, and sequential methods have been proposed among others by Sisson et al. (2007), Beaumont et al. (2009) and Del Moral et al. (2012). Recently, sequential ABC methods have appeared as an alternative to ABC-PMC methods (see for instance McKinley et al., 2009; Sisson et al., 2007). In this paper a new algorithm combining population-based MCMC methods with ABC requirements is proposed, using an analogy with the parallel tempering algorithm (Geyer 1991). Performance is compared with existing ABC algorithms on simulations and on a real example.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

References

  • Atchadé, Y., Roberts, G., Rosenthal, S.: Towards optimal scaling of metropolis-coupled Markov chain Monte Carlo. Stat. Comput. 21(4), 555–568 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  • Beaumont, M., Zhang, W., Balding, D.: Approximate Bayesian computation in population genetics. Genetics 162, 2025–2035 (2002)

    Google Scholar 

  • Beaumont, M., Cornuet, J., Marin, J., Robert, C.: Adaptive approximate Bayesian computation. Biometrika 96(4), 983–990 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  • Beskos, A., Crisan, D., Jasra, A.: On the stability of sequential Monte Carlo methods in high dimensions. Tech. rep. Imperial College, London (2011)

  • Blum, M.: Approximate Bayesian computational: a non-parametric perspective. J. Am. Stat. Assoc. 491, 1178–1187 (2010)

    Article  Google Scholar 

  • Blum, M., François, O.: Non-linear regression models for approximate Bayesian computation. Stat. Comput. 20(1), 63–73 (2010)

    Article  MathSciNet  Google Scholar 

  • Bortot, P., Coles, S., Sisson, S.: Inference for stereological extremes. J. Am. Stat. Assoc. 102, 84–92 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  • Del Moral, P., Doucet, A., Jasra, A.: An adaptive sequential Monte Carlo method for approximate Bayesian computation. Stat. Comput. (2012). doi:10.1007/s11222-011-9271-y

  • Drovandi, C., Pettitt, A.: Estimation of parameters for macroparasite population evolution using approximated Bayesian computation. Biometrics 67, 225–233 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  • Filippi, S., Barnes, C., Stumpf, M.: On optimal kernels for ABC SMC (2011). arXiv:1106.6280v2

  • Gelfand, A., Smith, A.: Sampling-based approaches to calculating marginal densities. J. Am. Stat. Assoc. 85(410), 398–409 (1990)

    Article  MathSciNet  MATH  Google Scholar 

  • Geyer, C.: Markov chain Monte Carlo maximum likelihood. In: Computing Science and Statistics: Proceedings of the 23rd Symposium on the Interface, pp 156–163 (1991)

    Google Scholar 

  • Geyer, C., Thompson, E.: Annealing Markov chain Monte Carlo with applications to ancestral nference. J. Am. Stat. Assoc. 90, 909–920 (1995)

    Article  MATH  Google Scholar 

  • Green, P., Mira, A.: Delayed rejection in reversible jump metropolis-hastings. Biometrika 88, 1035–1053 (2001)

    Article  MathSciNet  MATH  Google Scholar 

  • Grelaud, A., Marin, J., Robert, C., Rodolphe, F., Tally, F.: Likelihood-free methods for model choice in Gibbs random fields. Bayesian Anal. 3(2), 427–442 (2009)

    Google Scholar 

  • Hastings, W.: Monte Carlo sampling methods using Markov chains and their applications. Biometrika 88, 1035–1053 (1970)

    Google Scholar 

  • Jasra, A., Stephens, D., Holmes, C.: Population-based reversible jump Markov chain Monte Carlo. Biometrika 94, 787–807 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  • Kou, S., Zhou, Q., Wong, W.: Equi-energy sampler with application in statistical inference and statistical mechanics. Ann. Stat. 34(4), 1581–1619 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  • Leuenberger, C., Wegmann, D., Excoffier, L.: Bayesian computation and model selection in population genetics. Genetics 184, 243–252 (2010)

    Article  Google Scholar 

  • Liang, F., Wong, W.: Real-parameter evolutionary Monte Carlo with applications to Bayesian mixture models. J. Am. Stat. Assoc. 96, 653–666 (2001)

    Article  MathSciNet  MATH  Google Scholar 

  • Marjoram, P., Molitor, J., Plagnol, V., Tavaré, S.: Markov chain Monte Carlo without likelihoods. Proc. Natl. Acad. Sci. USA 100(26), 15,324–15,328 (2003).

    Article  Google Scholar 

  • McKinley, T., Cook, A., Deardon, R.: Inference in epidemic models without likelihoods. Int. J. Biostat. 5(1) (2009)

  • Metropolis, N., Rosenbluth, A., Rosenbluth, M., Teller, A., Teller, E.: Equations of state calculations by fast computing machines. J. Chem. Phys. 21(6), 1087–1092 (1953)

    Article  Google Scholar 

  • Müller, P., Sanso, B., De Iorio, M.: Optimal Bayesian design by inhomogeneous Markov chain simulation. J. Am. Stat. Assoc. 99(467), 788–798 (2004)

    Article  MATH  Google Scholar 

  • Nagata, K., Watanabe, S.: Asymptotic behavior of exchange ratio in exchange Monte Carlo method. Neural Netw. 21, 980–988 (2008)

    Article  MATH  Google Scholar 

  • Neal, R.: Sampling from multimodal distributions using tempered transitions. Stat. Comput. 6, 353–366 (1996)

    Article  Google Scholar 

  • Pritchard, J., Seielstad, M., Perez-Lezaun, A., Feldman, M.: Population growth of human y chromosomes: a study of y chromosome microsatellites. Mol. Biol. Evol. 16, 1791–1798 (1999)

    Article  Google Scholar 

  • Ratmann, O., Jorgensen, O., Hinkley, T., Stumpf, M., Richardson, S., Wiuf, C.: Using likelihood-free inference to compare evolutionary dynamics of the protein networks of H. pylori and P. falciparum. PLoS Comput. Biol. 3(11), 2266–2278 (2007)

    Article  MathSciNet  Google Scholar 

  • Sisson, S., Fan, Y., Tanaka, M.: Sequential Monte Carlo without likelihood. Proc. Natl. Acad. Sci. USA 104, 1760–1765 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  • Sisson, S., Fan, Y., Tanaka, M.: Sequential Monte Carlo without likelihood: Errata. P Natl. Acad. Sci. USA 106 (2009)

  • Small, P., Hopewell, P., Singh, S., Paz, A., Parsonnet, J., Ruston, D., Schecter, G., Daley, C., Schoolnik, G.: The epidemiology of tuberculosis in San Francisco: a population-based study using conventional and molecular methods. N. Engl. J. Med. 330, 1703–1709 (1994)

    Article  Google Scholar 

  • Tanaka, M., Francis, A., Luciani, F., Sisson, S.: Using approximate Bayesian computation to estimate tuberculosis transmission parameters form genotype data. Genetics 173, 1511–1520 (2006)

    Article  Google Scholar 

  • Tavaré, S., Balding, D., Griffith, R., Donnelly, P.: Inferring coalescence times from DNA sequence data. Genetics 145, 505–518 (1997)

    Google Scholar 

  • Toni, T., Welch, D., Strelkowa, N., Ipsen, A., Stumpf, M.: Approximate Bayesian computation scheme for parameter inference and model selection in dynamical systems. J. R. Soc. Interface 6, 187–202 (2009)

    Article  Google Scholar 

Download references

Acknowledgements

The authors are very grateful to the reviewers and to the Associate Editor for useful comments which enabled us to greatly improve the manuscript.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Meïli Baragatti.

Appendices

Appendix A: Matrices of exchange rates for ABC-PT with and without rings

Tables 7 and 8 compare accepted exchange rates between chains based on the modified toy example, using ABC-PT algorithms with or without rings. The rate between chains i and j is equal to the number of accepted exchange moves between these chains divided by the total number of iterations. It is observed that the use of rings allows more accepted exchange moves, especially between high tempered chains.

Table 7 ABC-PT with 3 rings for the modified toy example: the diagonal gives local acceptance rates, and outside the diagonal are given the proportions of accepted exchange moves
Table 8 ABC-PT for the modified toy example: the diagonal gives local acceptance rates, and outside the diagonal are given the proportions of accepted exchange moves

Appendix B: Formula for the tuberculosis example

We used the same notations than Tanaka et al. (2006). The number of cases of genotype i at time t is denoted by X i (t), G(t) is the number of distinct genotypes that have existed in the population up to and including time t, and N(t) is the total number of cases at time t.

$$N(t)=\sum_{i=1}^{G(t)} X_i(t).$$

The genotypes are labeled 1,2,3,… for convenience, although the ordering has no meaning, except that i=1 represents the parental type from which others are descended (directly or indirectly). The three rates of the system are the birth rate per case per year α, the death rate per case per year δ, and the mutation rate per case per year θ. Tanaka et al. (2006) define the following probabilities:

$$P_{i,x}(t) = P\bigl(X_i(t)=x\bigr),$$
$$\bar{P}_n(t) = P\bigl(N(t)=n\bigr)\quad \hbox{and}\quad \tilde{P}_g(t)= P\bigl(G(t)=g\bigr).$$

The time evolution of P i,x (t) is described by the following differential equations:

Initially there is only one copy of the ancestral genotype, hence the initial conditions are: P i,x (0)=0 for all (i,x), except P 1,1(0)=1, and for i=2,3,4,…,P i,0(0)=1. To take into account the creation of new genotypes, the probability \(\tilde{P}_{g}(t)\) is described by the following differential equations (only a mutation can create a new genotype):

The initial condition is G(0)=1. Let t g be the time when a new genotype g is created, we have P g,1(t g )=P(X g (t g )=1)=1, and P g,x (t g )=P(X g (t g )=x)=0 for x≠1.

The total number of cases N(t) is described by the following differential equations (only a birth or a death influence changes in this number):

The initial conditions are \(\bar{P}_{1}(0)=1\) and \(\bar{P}_{n}(0)=0\) for n≠1.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Baragatti, M., Grimaud, A. & Pommeret, D. Likelihood-free parallel tempering. Stat Comput 23, 535–549 (2013). https://doi.org/10.1007/s11222-012-9328-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11222-012-9328-6

Keywords

Navigation