Statistics and Computing

, Volume 18, Issue 4, pp 447–459 | Cite as

Adaptive importance sampling in general mixture classes

  • Olivier Cappé
  • Randal Douc
  • Arnaud Guillin
  • Jean-Michel Marin
  • Christian P. Robert
Article

Abstract

In this paper, we propose an adaptive algorithm that iteratively updates both the weights and component parameters of a mixture importance sampling density so as to optimise the performance of importance sampling, as measured by an entropy criterion. The method, called M-PMC, is shown to be applicable to a wide class of importance sampling densities, which includes in particular mixtures of multivariate Student t distributions. The performance of the proposed scheme is studied on both artificial and real examples, highlighting in particular the benefit of a novel Rao-Blackwellisation device which can be easily incorporated in the updating scheme.

Keywords

Importance sampling Adaptive Monte Carlo Mixture model Entropy Kullback-Leibler divergence EM algorithm Population Monte Carlo 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Cappé, O., Guillin, A., Marin, J., Robert, C.: Population Monte Carlo. J. Comput. Graph. Stat. 13(4), 907–929 (2004) CrossRefGoogle Scholar
  2. Cappé, O., Moulines, E., Rydén, T.: Inference in Hidden Markov Models. Springer, New York (2005) MATHGoogle Scholar
  3. Chen, R., Liu, J.S.: Predictive updating method and Bayesian classification. J. R. Stat. Soc. Ser. B 58(2), 397–415 (1996) MATHGoogle Scholar
  4. Douc, R., Guillin, A., Marin, J.-M., Robert, C.: Convergence of adaptive mixtures of importance sampling schemes. Ann. Stat. 35(1), 420–448 (2007a) MATHCrossRefMathSciNetGoogle Scholar
  5. Douc, R., Guillin, A., Marin, J.-M., Robert, C.: Minimum variance importance sampling via population Monte Carlo. ESAIM: Probab. Stat. 11, 427–447 (2007b) MATHCrossRefMathSciNetGoogle Scholar
  6. Doucet, A., de Freitas, N., Gordon, N.: Sequential Monte Carlo Methods in Practice. Springer, New York (2001) MATHGoogle Scholar
  7. Geweke, J.: Bayesian inference in econometric models using Monte Carlo integration. Econometrica 57, 1317–1340 (1989) MATHCrossRefMathSciNetGoogle Scholar
  8. Hesterberg, T.: Weighted average importance sampling and defensive mixture distributions. Technometrics 37(2), 185–194 (1995) MATHCrossRefMathSciNetGoogle Scholar
  9. Oh, M., Berger, J.: Integration of multimodal functions by Monte Carlo importance sampling. J. Am. Stat. Assoc. 88, 450–456 (1993) MATHCrossRefMathSciNetGoogle Scholar
  10. Peel, D., McLachlan, G.: Robust mixture modelling using the t distribution. Stat. Comput. 10, 339–348 (2000) CrossRefGoogle Scholar
  11. R Development Core Team: R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna (2006) Google Scholar
  12. Robert, C., Casella, G.: Monte Carlo Statistical Methods, 2nd edn. Springer, New York (2004) MATHGoogle Scholar
  13. Rubinstein, R.Y., Kroese, D.P.: The Cross-Entropy Method. Springer, New York (2004) MATHGoogle Scholar
  14. West, M.: Modelling with mixtures. In: Berger, J., Bernardo, J., Dawid, A., Smith, A. (eds.) Bayesian Statistics, vol. 4, pp. 503–525. Oxford University Press, Oxford (1992) Google Scholar

Copyright information

© Springer Science+Business Media, LLC 2008

Authors and Affiliations

  • Olivier Cappé
    • 1
  • Randal Douc
    • 2
  • Arnaud Guillin
    • 3
  • Jean-Michel Marin
    • 4
    • 5
  • Christian P. Robert
    • 6
    • 7
  1. 1.LTCITELECOM ParisTech, CNRSParisFrance
  2. 2.TELECOM SudParisÉvryFrance
  3. 3.LATPEcole Centrale Marseille, CNRSMarseilleFrance
  4. 4.Project select, INRIA SaclayOrsayFrance
  5. 5.CREST, INSEEParisFrance
  6. 6.CEREMADEUniversité Paris Dauphine, CNRSParisFrance
  7. 7.CREST, INSEEParisFrance

Personalised recommendations