Statistics and Computing

, Volume 18, Issue 4, pp 447–459 | Cite as

Adaptive importance sampling in general mixture classes

  • Olivier Cappé
  • Randal Douc
  • Arnaud Guillin
  • Jean-Michel MarinEmail author
  • Christian P. Robert


In this paper, we propose an adaptive algorithm that iteratively updates both the weights and component parameters of a mixture importance sampling density so as to optimise the performance of importance sampling, as measured by an entropy criterion. The method, called M-PMC, is shown to be applicable to a wide class of importance sampling densities, which includes in particular mixtures of multivariate Student t distributions. The performance of the proposed scheme is studied on both artificial and real examples, highlighting in particular the benefit of a novel Rao-Blackwellisation device which can be easily incorporated in the updating scheme.


Importance sampling Adaptive Monte Carlo Mixture model Entropy Kullback-Leibler divergence EM algorithm Population Monte Carlo 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Cappé, O., Guillin, A., Marin, J., Robert, C.: Population Monte Carlo. J. Comput. Graph. Stat. 13(4), 907–929 (2004) CrossRefGoogle Scholar
  2. Cappé, O., Moulines, E., Rydén, T.: Inference in Hidden Markov Models. Springer, New York (2005) zbMATHGoogle Scholar
  3. Chen, R., Liu, J.S.: Predictive updating method and Bayesian classification. J. R. Stat. Soc. Ser. B 58(2), 397–415 (1996) zbMATHGoogle Scholar
  4. Douc, R., Guillin, A., Marin, J.-M., Robert, C.: Convergence of adaptive mixtures of importance sampling schemes. Ann. Stat. 35(1), 420–448 (2007a) zbMATHCrossRefMathSciNetGoogle Scholar
  5. Douc, R., Guillin, A., Marin, J.-M., Robert, C.: Minimum variance importance sampling via population Monte Carlo. ESAIM: Probab. Stat. 11, 427–447 (2007b) zbMATHCrossRefMathSciNetGoogle Scholar
  6. Doucet, A., de Freitas, N., Gordon, N.: Sequential Monte Carlo Methods in Practice. Springer, New York (2001) zbMATHGoogle Scholar
  7. Geweke, J.: Bayesian inference in econometric models using Monte Carlo integration. Econometrica 57, 1317–1340 (1989) zbMATHCrossRefMathSciNetGoogle Scholar
  8. Hesterberg, T.: Weighted average importance sampling and defensive mixture distributions. Technometrics 37(2), 185–194 (1995) zbMATHCrossRefMathSciNetGoogle Scholar
  9. Oh, M., Berger, J.: Integration of multimodal functions by Monte Carlo importance sampling. J. Am. Stat. Assoc. 88, 450–456 (1993) zbMATHCrossRefMathSciNetGoogle Scholar
  10. Peel, D., McLachlan, G.: Robust mixture modelling using the t distribution. Stat. Comput. 10, 339–348 (2000) CrossRefGoogle Scholar
  11. R Development Core Team: R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna (2006) Google Scholar
  12. Robert, C., Casella, G.: Monte Carlo Statistical Methods, 2nd edn. Springer, New York (2004) zbMATHGoogle Scholar
  13. Rubinstein, R.Y., Kroese, D.P.: The Cross-Entropy Method. Springer, New York (2004) zbMATHGoogle Scholar
  14. West, M.: Modelling with mixtures. In: Berger, J., Bernardo, J., Dawid, A., Smith, A. (eds.) Bayesian Statistics, vol. 4, pp. 503–525. Oxford University Press, Oxford (1992) Google Scholar

Copyright information

© Springer Science+Business Media, LLC 2008

Authors and Affiliations

  • Olivier Cappé
    • 1
  • Randal Douc
    • 2
  • Arnaud Guillin
    • 3
  • Jean-Michel Marin
    • 4
    • 5
    Email author
  • Christian P. Robert
    • 6
    • 7
  1. 1.LTCITELECOM ParisTech, CNRSParisFrance
  2. 2.TELECOM SudParisÉvryFrance
  3. 3.LATPEcole Centrale Marseille, CNRSMarseilleFrance
  4. 4.Project select, INRIA SaclayOrsayFrance
  5. 5.CREST, INSEEParisFrance
  6. 6.CEREMADEUniversité Paris Dauphine, CNRSParisFrance
  7. 7.CREST, INSEEParisFrance

Personalised recommendations