Statistics and Computing

, Volume 24, Issue 5, pp 693–707 | Cite as

An improved SAEM algorithm for maximum likelihood estimation in mixtures of non linear mixed effects models

  • Marc LavielleEmail author
  • Cyprien Mbogning


We propose a new methodology for maximum likelihood estimation in mixtures of non linear mixed effects models (NLMEM). Such mixtures of models include mixtures of distributions, mixtures of structural models and mixtures of residual error models. Since the individual parameters inside the NLMEM are not observed, we propose to combine the EM algorithm usually used for mixtures models when the mixture structure concerns an observed variable, with the Stochastic Approximation EM (SAEM) algorithm, which is known to be suitable for maximum likelihood estimation in NLMEM and also has nice theoretical properties. The main advantage of this hybrid procedure is to avoid a simulation step of unknown group labels required by a “full” version of SAEM. The resulting MSAEM (Mixture SAEM) algorithm is now implemented in the Monolix software. Several criteria for classification of subjects and estimation of individual parameters are also proposed. Numerical experiments on simulated data show that MSAEM performs well in a general framework of mixtures of NLMEM. Indeed, MSAEM provides an estimator close to the maximum likelihood estimator in very few iterations and is robust with regard to initialization. An application to pharmacokinetic (PK) data demonstrates the potential of the method for practical applications.


SAEM algorithm Maximum likelihood estimation Mixture models Non linear mixed effects model Monolix 



The research leading to these results has received support from the Innovative Medicines Initiative Joint Undertaking under grant agreement n. 115156, resources of which are composed of financial contributions from the European Union’s Seventh Framework Programme (FP7/2007-2013) and EFPIA companies in kind contribution. The DDMoRe project is also financially supported by contributions from Academic and SME partners.

Supplementary material

11222_2013_9396_MOESM1_ESM.pdf (295 kb)
An improved SAEM algorithm for maximum likelihood estimation in mixtures of non linear mixed effects models (PDF 295 kB)


  1. Allassonnière, S., Kuhn, E., Trouvé, A.: Construction of Bayesian deformable models via a stochastic approximation algorithm: a convergence study. Bernoulli 16, 641–678 (2010) CrossRefzbMATHMathSciNetGoogle Scholar
  2. Banfield, J.D., Raftery, A.E.: Model-based Gaussian and non Gaussian clustering. Biometrics 49, 803–821 (1993) CrossRefzbMATHMathSciNetGoogle Scholar
  3. Celeux, G., Govaert, G.: Gaussian parsimonious clustering models. Pattern Recognit. 28, 781–793 (1995) CrossRefGoogle Scholar
  4. Celeux, G., Hurn, M., Robert, C.: Computational and inferential difficulties with mixtures posterior distribution. J. Am. Stat. Assoc. 95(3), 957–979 (2000) CrossRefzbMATHMathSciNetGoogle Scholar
  5. Davidian, M., Giltinan, D.M.: Some simple methods for estimating intraindividual variability in nonlinear random effects models. Biometrics 49, 59–73 (1993) CrossRefGoogle Scholar
  6. De la Cruz, R., Quintana, F., Marshall, G.: Model based clustering for longitudinal data. Comput. Stat. Data Anal. 52(3), 1441–1457 (2008) CrossRefzbMATHGoogle Scholar
  7. Delyon, B., Lavielle, M., Moulines, E.: Convergence of a stochastic approximation version of the EM algorithm. Ann. Stat. 27, 94–128 (1999) CrossRefzbMATHMathSciNetGoogle Scholar
  8. Dempster, A.P., Laird, N.M., Rubin, D.B.: Maximum likelihood from incomplete data via the EM algorithm. Suppl. J. R. Stat. Soc. 39(1), 1–38 (1977) zbMATHMathSciNetGoogle Scholar
  9. Frühwirth-Schnatter, S.: Finite Mixture and Markov Switching Models. Springer, New York (2006) zbMATHGoogle Scholar
  10. Hou, W., Li, H., Zhang, B., Huang, M., Wu, R.: A nonlinear mixed-effect mixture model for functional mapping of dynamic traits. Heredity 101, 321–328 (2008) CrossRefGoogle Scholar
  11. Ketchum, J., Best, A., Ramakrishnan, V.: A within-subject normal-mixture model with mixed-effects for analyzing heart rate variability. J. Biometr. Biostat. S7, 013 (2012) Google Scholar
  12. Kuhn, E., Lavielle, M.: Coupling a stochastic approximation version of EM with a MCMC procedure. ESAIM, Probab. Stat. 8, 115–131 (2004) CrossRefzbMATHMathSciNetGoogle Scholar
  13. Kuhn, E., Lavielle, M.: Maximum likelihood estimation in nonlinear mixed effects models. Comput. Stat. Data Anal. 49, 1020–1038 (2005) CrossRefzbMATHMathSciNetGoogle Scholar
  14. Lavielle, M., Mesa, H., Chatel, K., Vermeulen, A.: Mixture models and model mixtures with Monolix. In: Abstracts of the Annual Meeting of the Population Approach Group in Europe, Berlin (2010) Google Scholar
  15. Lindstrom, M., Bates, D.: Nonlinear mixed effects models for repeated measures data. Biometrics 46, 673–687 (1990) CrossRefMathSciNetGoogle Scholar
  16. McLachland, G.J., Peel, D.: Finite Mixture Models. Wiley-Interscience, New York (2000) CrossRefGoogle Scholar
  17. Ng, S., McLachlan, G., Wang, K., Ben-Tovim, L., Ng, S.: A mixture model with mixed effects components for clustering correlated gene-expression profiles. Bioinformatics 22, 1745–1752 (2006) CrossRefGoogle Scholar
  18. Proust, C., Jacqmin-Gadda, H.: Estimation of linear mixed models with a mixture of distribution for the random effects. Comput. Methods Programs Biomed. 78(2), 165–173 (2005) CrossRefGoogle Scholar
  19. Roeder, K., Wasserman, L.: Practical Bayesian density estimation using mixtures of normals. J. Am. Stat. Assoc. 92, 894–902 (1997) CrossRefzbMATHMathSciNetGoogle Scholar
  20. Verbeke, G., Lesaffre, E.: A linear mixed-effects model with heterogeneity in the random-effects population. J. Am. Stat. Assoc. 91(433), 217–221 (1996) CrossRefzbMATHGoogle Scholar
  21. Wang, X., Schumitzky, A., D’Argenio, D.Z.: Non linear random effects mixture models: maximum likelihood estimation via the EM algorithm. Comput. Stat. Data Anal. 51, 6614–6623 (2007) CrossRefzbMATHMathSciNetGoogle Scholar
  22. Wang, X., Schumitzky, A., D’Argenio, D.Z.: Population pharmacokinetic/pharmacodynamic mixture models via maximum a posteriori estimation. Comput. Stat. Data Anal. 53, 3907–3915 (2009) CrossRefzbMATHMathSciNetGoogle Scholar
  23. Wei, G., Tanner, M.: A Monte Carlo implementation of the EM algorithm and the poor man’s data augmentation algorithms. J. Am. Stat. Assoc. 85, 699–704 (1990) CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2013

Authors and Affiliations

  1. 1.Laboratoire de Mathématiques d’Orsay (LMO)Orsay CedexFrance
  2. 2.POPIX TeamInriaSaclayFrance
  3. 3.LIMSSEcole Nationale Supérieure Polytechnique (ENSP)YaoundéCameroon

Personalised recommendations