Statistics and Computing

, Volume 17, Issue 2, pp 121–130 | Cite as

A parameter expansion version of the SAEM algorithm

  • Marc LavielleEmail author
  • Cristian Meza


The EM algorithm and its extensions are very popular tools for maximum likelihood estimation in incomplete data setting. However, one of the limitations of these methods is their slow convergence. The PX-EM (parameter-expanded EM) algorithm was proposed by Liu, Rubin and Wu to make EM much faster. On the other hand, stochastic versions of EM are powerful alternatives of EM when the E-step is untractable in a closed form. In this paper we propose the PX-SAEM which is a parameter expansion version of the so-called SAEM (Stochastic Approximation version of EM). PX-SAEM is shown to accelerate SAEM and improve convergence toward the maximum likelihood estimate in a parametric framework. Numerical examples illustrate the behavior of PX-SAEM in linear and nonlinear mixed effects models.


EM PX-EM SAEM Nonlinear mixed effects models 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Celeux G. and Diebolt J. 1985. The SEM algorithm: A probabilistic teacher algorithm derived from the EM algorithm for the mixture problem. Computational. Statistics Quaterly 2: 73–82.Google Scholar
  2. Concordet D. and Nunez O. 2002. A simulated pseudo–maximum likelihood estimator for nonlinear mixed models. Comput. Statist. Data Anal. 39: 187–201.zbMATHCrossRefMathSciNetGoogle Scholar
  3. Delyon B., Lavielle M., and Moulines E. 1999. Convergence of a stochastic approximation version of the EM algorithm. Ann. Statist. 27: 94–128.zbMATHCrossRefMathSciNetGoogle Scholar
  4. Dempster A., Laird N., and Rubin D. 1977. Maximum likelihood from incomplete data via the EM algorithm. J. Roy. Statist. Soc. Ser. B 39: 1–38. With discussion.zbMATHMathSciNetGoogle Scholar
  5. Fesslern J. and Hero A. 1994. Space-alternating generalized expectation-maximization algorithm. IEE Trans. Sig. Proc. 42: 2664–2677.CrossRefGoogle Scholar
  6. Jank W. 2004. Quasi–Monte Carlo sampling to improve the efficiency of Monte Carlo EM. CSDA 48: 685–701.MathSciNetGoogle Scholar
  7. Kuhn E. and Lavielle M. 2004. Coupling a stochastic approximation version of EM with a MCMC procedure. ESAIM P&S 8: 115–131.zbMATHCrossRefMathSciNetGoogle Scholar
  8. Kuhn E. and Lavielle M. 2005. Maximum likelihood estimation in nonlinear mixed effects models. CSDA 49(4), 20–38.MathSciNetGoogle Scholar
  9. Liu C. and Rubin D. 1994. The ECME algorithm: a simple extension of em and ecm with faster monotone convergence. Biometrika 81: 633–648.zbMATHCrossRefMathSciNetGoogle Scholar
  10. Liu C., Rubin D., and Wu Y. 1998. Parameter expansion to accelerate EM: The PX–EM algorithm. Biometrika 85: 755–770.zbMATHCrossRefMathSciNetGoogle Scholar
  11. McLachlan G. and Krishnan T. 1997. The EM Algorithm and Extensions. New York: J. Wiley and Sons.zbMATHGoogle Scholar
  12. Meng X.-L. and Rubin D.B. 1993. Maximum likelihood estimation via the ECM algorithm: a general framework. Biometrika 80(2): 267–278.zbMATHCrossRefMathSciNetGoogle Scholar
  13. Wei G. and Tanner M. 1990. A Monte Carlo implementation of the EM algorithm and the poor man’s data augmentation algorithms. J. Amer. Statist. Assoc. 85: 699–704.CrossRefGoogle Scholar
  14. Wu C. 1983. On the convergence properties of the EM algorithm. Ann. Statist. 11: 95–103.zbMATHMathSciNetGoogle Scholar
  15. Wu L. 2004. Exact and approximate inferences for nonlinear mixed-effects models with missing covariates. J. Amer. Statist. Assoc. 99: 700–709.CrossRefMathSciNetGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2007

Authors and Affiliations

  1. 1.INRIA Futurs and Université Paris-Sud, Equipe de ProbabilitésStatistique et ModélisationOrsayFrance
  2. 2.Laboratoire de MathématiquesUniversité Paris-SudOrsay CedexFrance

Personalised recommendations