Statistics and Computing

, Volume 18, Issue 1, pp 47–57

Particle methods for maximum likelihood estimation in latent variable models

Article

Abstract

Standard methods for maximum likelihood parameter estimation in latent variable models rely on the Expectation-Maximization algorithm and its Monte Carlo variants. Our approach is different and motivated by similar considerations to simulated annealing; that is we build a sequence of artificial distributions whose support concentrates itself on the set of maximum likelihood estimates. We sample from these distributions using a sequential Monte Carlo approach. We demonstrate state-of-the-art performance for several applications of the proposed approach.

Keywords

Latent variable models Markov chain Monte Carlo Maximum likelihood Sequential Monte Carlo Simulated annealing 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Amzal, B., Bois, F.Y., Parent, E., Robert, C.P.: Bayesian optimal design via interacting particle systems. J. Am. Stat. Assoc. 101(474), 773–785 (2006) MATHCrossRefMathSciNetGoogle Scholar
  2. Chopin, N.: Central limit theorem for sequential Monte Carlo methods and its applications to Bayesian inference. Ann. Stat. 32(6), 2385–2411 (2004) MATHCrossRefMathSciNetGoogle Scholar
  3. Del Moral, P.: Feynman-Kac Formulae: Genealogical and Interacting Particle Systems with Applications. Probability and its Applications. Springer, New York (2004) MATHGoogle Scholar
  4. Del Moral, P., Doucet, A., Jasra, A.: Sequential Monte Carlo samplers. J. R. Stat. Soc. B 63(3), 411–436 (2006) CrossRefMathSciNetGoogle Scholar
  5. Dempster, A.P., Laird, N.M., Rubin, D.B.: Maximum likelihood from incomplete data via the EM Algorithm. J. R. Stat. Soc. B 39, 2–38 (1977) MathSciNetGoogle Scholar
  6. Doucet, A., de Freitas, N., Gordon, N. (eds.): Sequential Monte Carlo Methods in Practice. Statistics for Engineering and Information Science. Springer, New York (2001) MATHGoogle Scholar
  7. Doucet, A., Godsill, S.J., Robert, C.P.: Marginal maximum a posteriori estimation using Markov chain Monte Carlo. Stat. Comput. 12, 77–84 (2002) CrossRefMathSciNetGoogle Scholar
  8. Doucet, A., Briers, M., Sénécal, S.: Efficient block sampling strategies for sequential Monte Carlo methods. J. Comput. Graph. Stat. 15(3), 693–711 (2006) CrossRefGoogle Scholar
  9. Escobar, M.D., West, M.: Bayesian density estimation and inference using mixtures. J. Am. Stat. Assoc. 90(430), 577–588 (1995) MATHCrossRefMathSciNetGoogle Scholar
  10. Gaetan, C., Yao, J.-F.: A multiple-imputation Metropolis version of the EM algorithm. Biometrika 90(3), 643–654 (2003) CrossRefMathSciNetGoogle Scholar
  11. Hwang, C.-R.: Laplace’s method revisited: weak convergence of probability measures. Ann. Probab. 8(6), 1177–1182 (1980) MATHMathSciNetGoogle Scholar
  12. Jacquier, E., Johannes, M., Polson, N.: MCMC maximum likelihood for latent state models. J. Econom. 137(2), 615–640 (2007) CrossRefGoogle Scholar
  13. Johansen, A.M.: Some non-standard sequential Monte Carlo methods with applications, Ph.D. thesis. University of Cambridge, Department of Engineering (2006) Google Scholar
  14. Liu, J.S., Chen, R.: Sequential Monte Carlo methods for dynamic systems. J. Am. Stat. Assoc. 93(443), 1032–1044 (1998) MATHCrossRefGoogle Scholar
  15. Müller, P., Sansó, B., de Iorio, M.: Optimum Bayesian design by inhomogeneous Markov chain simulation. J. Am. Stat. Assoc. 99(467), 788–798 (2004) MATHCrossRefGoogle Scholar
  16. Robert, C.P., Casella, G.: Monte Carlo Statistical Methods, 2nd edn. Springer, New York (2004) MATHGoogle Scholar
  17. Robert, C.P., Titterington, D.M.: Reparameterization strategies for hidden Markov models and Bayesian approaches to maximum likelihood estimation. Stat. Comput. 8, 145–158 (1998) CrossRefGoogle Scholar
  18. Roeder, K.: Density estimation with cofidence sets exemplified by superclusters and voids in galaxies. J. Am. Stat. Assoc. 85(411), 617–624 (1990) MATHCrossRefGoogle Scholar
  19. Roeder, K., Wasserman, L.: Practical Bayesian density estimation using mixtures of normals. J. Am. Stat. Assoc. 92(439), 894–902 (1997) MATHCrossRefMathSciNetGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2007

Authors and Affiliations

  1. 1.Department of Mathematics, University WalkUniversity of BristolBristolUK
  2. 2.Department of Statistics & Department of Computer ScienceUniversity of British ColumbiaVancouverCanada
  3. 3.LAGIS UMR 8146Villeneuve d’Ascq CedexFrance

Personalised recommendations