Advertisement

Statistics and Computing

, Volume 18, Issue 4, pp 447–459 | Cite as

Adaptive importance sampling in general mixture classes

  • Olivier Cappé
  • Randal Douc
  • Arnaud Guillin
  • Jean-Michel Marin
  • Christian P. Robert
Article

Abstract

In this paper, we propose an adaptive algorithm that iteratively updates both the weights and component parameters of a mixture importance sampling density so as to optimise the performance of importance sampling, as measured by an entropy criterion. The method, called M-PMC, is shown to be applicable to a wide class of importance sampling densities, which includes in particular mixtures of multivariate Student t distributions. The performance of the proposed scheme is studied on both artificial and real examples, highlighting in particular the benefit of a novel Rao-Blackwellisation device which can be easily incorporated in the updating scheme.

Keywords

Importance sampling Adaptive Monte Carlo Mixture model Entropy Kullback-Leibler divergence EM algorithm Population Monte Carlo 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Cappé, O., Guillin, A., Marin, J., Robert, C.: Population Monte Carlo. J. Comput. Graph. Stat. 13(4), 907–929 (2004) CrossRefGoogle Scholar
  2. Cappé, O., Moulines, E., Rydén, T.: Inference in Hidden Markov Models. Springer, New York (2005) zbMATHGoogle Scholar
  3. Chen, R., Liu, J.S.: Predictive updating method and Bayesian classification. J. R. Stat. Soc. Ser. B 58(2), 397–415 (1996) zbMATHGoogle Scholar
  4. Douc, R., Guillin, A., Marin, J.-M., Robert, C.: Convergence of adaptive mixtures of importance sampling schemes. Ann. Stat. 35(1), 420–448 (2007a) zbMATHCrossRefMathSciNetGoogle Scholar
  5. Douc, R., Guillin, A., Marin, J.-M., Robert, C.: Minimum variance importance sampling via population Monte Carlo. ESAIM: Probab. Stat. 11, 427–447 (2007b) zbMATHCrossRefMathSciNetGoogle Scholar
  6. Doucet, A., de Freitas, N., Gordon, N.: Sequential Monte Carlo Methods in Practice. Springer, New York (2001) zbMATHGoogle Scholar
  7. Geweke, J.: Bayesian inference in econometric models using Monte Carlo integration. Econometrica 57, 1317–1340 (1989) zbMATHCrossRefMathSciNetGoogle Scholar
  8. Hesterberg, T.: Weighted average importance sampling and defensive mixture distributions. Technometrics 37(2), 185–194 (1995) zbMATHCrossRefMathSciNetGoogle Scholar
  9. Oh, M., Berger, J.: Integration of multimodal functions by Monte Carlo importance sampling. J. Am. Stat. Assoc. 88, 450–456 (1993) zbMATHCrossRefMathSciNetGoogle Scholar
  10. Peel, D., McLachlan, G.: Robust mixture modelling using the t distribution. Stat. Comput. 10, 339–348 (2000) CrossRefGoogle Scholar
  11. R Development Core Team: R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna (2006) Google Scholar
  12. Robert, C., Casella, G.: Monte Carlo Statistical Methods, 2nd edn. Springer, New York (2004) zbMATHGoogle Scholar
  13. Rubinstein, R.Y., Kroese, D.P.: The Cross-Entropy Method. Springer, New York (2004) zbMATHGoogle Scholar
  14. West, M.: Modelling with mixtures. In: Berger, J., Bernardo, J., Dawid, A., Smith, A. (eds.) Bayesian Statistics, vol. 4, pp. 503–525. Oxford University Press, Oxford (1992) Google Scholar

Copyright information

© Springer Science+Business Media, LLC 2008

Authors and Affiliations

  • Olivier Cappé
    • 1
  • Randal Douc
    • 2
  • Arnaud Guillin
    • 3
  • Jean-Michel Marin
    • 4
    • 5
  • Christian P. Robert
    • 6
    • 7
  1. 1.LTCITELECOM ParisTech, CNRSParisFrance
  2. 2.TELECOM SudParisÉvryFrance
  3. 3.LATPEcole Centrale Marseille, CNRSMarseilleFrance
  4. 4.Project select, INRIA SaclayOrsayFrance
  5. 5.CREST, INSEEParisFrance
  6. 6.CEREMADEUniversité Paris Dauphine, CNRSParisFrance
  7. 7.CREST, INSEEParisFrance

Personalised recommendations