Advertisement

Statistics and Computing

, Volume 23, Issue 2, pp 185–200 | Cite as

Interacting multiple try algorithms with different proposal distributions

  • Roberto Casarin
  • Radu Craiu
  • Fabrizio Leisen
Article

Abstract

We introduce a new class of interacting Markov chain Monte Carlo (MCMC) algorithms which is designed to increase the efficiency of a modified multiple-try Metropolis (MTM) sampler. The extension with respect to the existing MCMC literature is twofold. First, the sampler proposed extends the basic MTM algorithm by allowing for different proposal distributions in the multiple-try generation step. Second, we exploit the different proposal distributions to naturally introduce an interacting MTM mechanism (IMTM) that expands the class of population Monte Carlo methods and builds connections with the rapidly expanding world of adaptive MCMC. We show the validity of the algorithm and discuss the choice of the selection weights and of the different proposals. The numerical studies show that the interaction mechanism allows the IMTM to efficiently explore the state space leading to higher efficiency than other competing algorithms.

Keywords

Interacting Monte Carlo Markov chain Monte Carlo Multiple-try Metropolis Population Monte Carlo Simulated annealing 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Atchadé, Y., Roberts, G.O., Rosenthal, J.S.: Towards optimal scaling of Metropolis-coupled Markov chain Monte Carlo. Stat. Comput. 21, 555–568 (2011) MathSciNetzbMATHCrossRefGoogle Scholar
  2. Barrett, M., Galipeau, P., Sanchez, C., Emond, M., Reid, B.: Determination of the frequency of loss of heterozygosity in esophageal adenocarcinoma nu cell sorting, whole genome amplification and microsatellite polymorphisms. Oncogene 12 (1996) Google Scholar
  3. Bédard, M., Douc, R., Moulines, E.: Scaling analysis of multiple-try MCMC methods. Technical report, Université de Montréal (2010) Google Scholar
  4. Campillo, F., Rakotozafy, R., Rossi, V.: Parallel and interacting Markov chain Monte Carlo algorithm. Math. Comput. Simul. 79, 3424–3433 (2009) MathSciNetzbMATHCrossRefGoogle Scholar
  5. Cappé, O., Gullin, A., Marin, J., Robert, C.P.: Population Monte Carlo. J. Comput. Graph. Stat. 13, 907–927 (2004) CrossRefGoogle Scholar
  6. Casarin, R., Marin, J.-M.: Online data processing: Comparison of Bayesian regularized particle filters. Electron. J. Stat. 3, 239–258 (2009) MathSciNetCrossRefGoogle Scholar
  7. Casarin, R., Marin, J.-M., Robert, C.: A discussion on: approximate Bayesian inference for latent Gaussian models by using integrated nested Laplace approximations by Rue, H., Martino, S. and Chopin, N. J. R. Stat. Soc. B 71, 360–362 (2009) Google Scholar
  8. Celeux, G., Marin, J.-M., Robert, C.: Iterated importance sampling in missing data problems. Comput. Stat. Data Anal. 50, 3386–3404 (2006) MathSciNetzbMATHCrossRefGoogle Scholar
  9. Chauveau, D., Vandekerkhove, P.: Improving convergence of the Hastings-Metropolis algorithm with an adaptive proposal. Scand. J. Stat. 29, 13 (2002) MathSciNetzbMATHCrossRefGoogle Scholar
  10. Craiu, R.V., Lemieux, C.: Acceleration of the multiple-try Metropolis algorithm using antithetic and stratified sampling. Stat. Comput. 17, 109–120 (2007) MathSciNetCrossRefGoogle Scholar
  11. Craiu, R.V., Meng, X.L.: Multi-process parallel antithetic coupling for forward and backward MCMC. Ann. Stat. 33, 661–697 (2005) MathSciNetzbMATHCrossRefGoogle Scholar
  12. Craiu, R.V., Rosenthal, J.S., Yang, C.: Learn from thy neighbor: parallel-chain adaptive and regional MCMC. J. Am. Stat. Assoc. 104, 1454–1466 (2009) MathSciNetzbMATHCrossRefGoogle Scholar
  13. Del Moral, P.: Feynman-Kac Formulae. Genealogical and Interacting Particle Systems with Applications. Springer, Berlin (2004) zbMATHGoogle Scholar
  14. Del Moral, P., Doucet, A., Jasra, A.: Sequential Monte Carlo samplers. J. R. Stat. Soc. B 68, 411–436 (2006) zbMATHCrossRefGoogle Scholar
  15. Desai, M.: Mixture models for genetic changes in cancer cells. Ph.D. thesis, University of Washington (2000) Google Scholar
  16. Früwirth-Schnatter, S.: Finite Mixture and Markov Switching Models. Springer, Berlin (2006) Google Scholar
  17. Gelman, A., Rubin, D.B.: Inference from iterative simulation using multiple sequences (with discussion). Stat. Sci. 457–511 (1992) Google Scholar
  18. Geyer, C.J., Thompson, E.A.: Annealing Markov chain Monte Carlo with applications to ancestral inference. Tech. rep. 589, University of Minnesota (1994) Google Scholar
  19. Hastings, W.K.: Monte Carlo sampling methods using Markov chains and their applications. Biometrika 57, 97–109 (1970) zbMATHCrossRefGoogle Scholar
  20. Heard, N.A., Holmes, C., Stephens, D.: A quantitative study of gene regulation involved in the immune response od anophelinemosquitoes: an application of Bayesian hierarchical clustering of curves. J. Am. Stat. Assoc. 101, 18–29 (2006) MathSciNetzbMATHCrossRefGoogle Scholar
  21. Jasra, A., Stephens, D.A., Holmes, C.: Markov chain Monte Carlo methods and the label switching problem in Bayesian mixture modelling. Stat. Sci. 20, 50–67 (2005) MathSciNetzbMATHCrossRefGoogle Scholar
  22. Jasra, A., Stephens, D., Holmes, C.: On population-based simulation for static inference. Stat. Comput. 17, 263–279 (2007) MathSciNetCrossRefGoogle Scholar
  23. Jennison, C.: Discussion of Bayesian computation via the Gibbs sampler and related Markov chain Monte Carlo methods, by A.F.M. Smith and G.O. Roberts. J. R. Stat. Soc. B 55, 54–56 (1993) Google Scholar
  24. Liu, J., Liang, F., Wong, W.: The multiple-try method and local optimization in Metropolis sampling. J. Am. Stat. Assoc. 95, 121–134 (2000) MathSciNetzbMATHCrossRefGoogle Scholar
  25. Liang, F., Wong, W.: Real parameter evolutionary Monte Carlo with applications to Bayesian mixture models. J. Am. Stat. Assoc. 96, 653–666 (2001) MathSciNetzbMATHCrossRefGoogle Scholar
  26. Marinari, E., Parisi, G.: Simulated tempering: a new Monte Carlo scheme. Europhys. Lett. 19, 451–458 (1992) CrossRefGoogle Scholar
  27. Mengersen, K., Robert, C.: The pinball sampler. In: Bernardo, J., Dawid, A., Berger, J., West, M. (eds.) Bayesian Statistics 7. Springer, Berlin (2003) Google Scholar
  28. Metropolis, N., Rosenbluth, A., Rosenbluth, M., Teller, A., Teller, E.: Equations of state calculations by fast computing machines. J. Chem. Phys. 21, 1087–1092 (1953) CrossRefGoogle Scholar
  29. Neal, R.M.: Sampling from multimodal distributions using tempered transitions. Tech. rep. 9421, University of Toronto (1994) Google Scholar
  30. Pandolfi, S., Bartolucci, F., Friel, N.: A generalization of the multiple-try Metropolis algorithm for Bayesian estimation and model selection. In: Proceedings of the 13th International Conference on Artificial Intelligence and Statistics (AISTATS), Chia Laguna Resort, Sardinia, Italy, pp. 581–588 (2010a) Google Scholar
  31. Pandolfi, S., Bartolucci, F., Friel, N.: A generalized Multiple-try Metropolis version of the Reversible Jump algorithm. Tech. rep. (2010b). http://arxiv.org/pdf/1006.0621
  32. Pritchard, J.K., Stephens, M., Donnelly, P.: Inference of population structure using multilocus genotype data. Genetics 155, 945–959 (2000) Google Scholar
  33. Richardson, S., Green, P.J.: On Bayesian analysis of mixtures with an unknown number of components (with discussion). J. R. Stat. Soc. B 4(59), 731–792 (1997) MathSciNetCrossRefGoogle Scholar
  34. Shephard, N., Pitt, M.: Likelihood analysis of non-Gaussian measurement time series. Biometrika 84, 653–667 (1997) MathSciNetzbMATHCrossRefGoogle Scholar
  35. So, M.K.P.: Bayesian analysis of nonlinear and non-Gaussian state space models via multiple-try sampling methods. Stat. Comput. 16, 125–141 (2006) MathSciNetCrossRefGoogle Scholar
  36. Taylor, S.: Modelling stochastic volatility. Math. Finance 4, 183–204 (1994) zbMATHCrossRefGoogle Scholar
  37. Warnes, G.: The Normal kernel coupler: an adaptive Markov chain Monte Carlo method for efficiently sampling from multi-modal distributions. Technical report, George Washington University (2001) Google Scholar

Copyright information

© Springer Science+Business Media, LLC 2011

Authors and Affiliations

  1. 1.Advanced School of EconomicsVeniceItaly
  2. 2.University Ca’ Foscari of VeniceVeniceItaly
  3. 3.Universidad Carlos III de MadridMadridSpain
  4. 4.University of TorontoTorontoCanada

Personalised recommendations