Statistics and Computing

, Volume 22, Issue 1, pp 121–139 | Cite as

Estimation in nonlinear mixed-effects models using heavy-tailed distributions

  • Cristian MezaEmail author
  • Felipe Osorio
  • Rolando De la Cruz


Nonlinear mixed-effects models are very useful to analyze repeated measures data and are used in a variety of applications. Normal distributions for random effects and residual errors are usually assumed, but such assumptions make inferences vulnerable to the presence of outliers. In this work, we introduce an extension of a normal nonlinear mixed-effects model considering a subclass of elliptical contoured distributions for both random effects and residual errors. This elliptical subclass, the scale mixtures of normal (SMN) distributions, includes heavy-tailed multivariate distributions, such as Student-t, the contaminated normal and slash, among others, and represents an interesting alternative to outliers accommodation maintaining the elegance and simplicity of the maximum likelihood theory. We propose an exact estimation procedure to obtain the maximum likelihood estimates of the fixed-effects and variance components, using a stochastic approximation of the EM algorithm. We compare the performance of the normal and the SMN models with two real data sets.


Mixed-effects model Outliers Scale mixtures of normal distributions SAEM algorithm Random effects 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Andrews, D.F., Mallows, C.L.: Scale mixtures of normal distributions. J. R. Stat. Soc. Ser. B 36, 99–102 (1974) MathSciNetzbMATHGoogle Scholar
  2. Beal, S.L., Sheiner, L.B.: NONMEN User’s Guide. Nonlinear Mixed-Effects Models for Repeated Measures Data. University of California, San Francisco (1992) Google Scholar
  3. Branco, M.D., Dey, D.K.: A general class of multivariate skew-elliptical distributions. J. Multivar. Anal. 79, 99–113 (2001) MathSciNetzbMATHCrossRefGoogle Scholar
  4. Cai, L.: High-dimensional exploratory item factor analysis by a Metropolis-Hastings Robbins-Monro algorithm. Psychometrika 75, 33–57 (2010) MathSciNetzbMATHCrossRefGoogle Scholar
  5. Choy, S.T.B., Smith, A.F.M.: Hierarchical models with scale mixtures of normal distributions. Test 6, 205–221 (1997) zbMATHCrossRefGoogle Scholar
  6. Cook, R.D.: Assessment of local influence (with discussion). J. R. Stat. Soc. Ser. B 48, 133–169 (1986) zbMATHGoogle Scholar
  7. Cook, R.D.: Local Influence. In: Kotz, S., Read, C.B., Banks, D.L. (eds.) Encyclopedia of Statistical Sciences, Update, vol. 1, pp. 380–385. Wiley, New York (1997) Google Scholar
  8. Cook, R.D., Weisberg, S.: Residuals and Influence in Regression. Chapman & Hall, London (1982) zbMATHGoogle Scholar
  9. Copt, S., Victoria-Feser, M.: High breakdown inference in the mixed linear model. J. Am. Stat. Assoc. 101, 292–300 (2006) MathSciNetzbMATHCrossRefGoogle Scholar
  10. Davidian, M., Giltinan, D.M.: Nonlinear Models for Repeated Measurements Data. Chapman & Hall, New York (1995) Google Scholar
  11. Davidian, M., Giltinan, D.M.: Nonlinear models for repeated measurements: An overview and update. J. Agric. Biol. Environ. Stat. 8, 387–419 (2003) CrossRefGoogle Scholar
  12. De la Cruz, R.: Bayesian non-linear regression models with skew-elliptical errors: Applications to the classification of longitudinal profiles. Comput. Stat. Data Anal. 53, 436–229 (2008) zbMATHCrossRefGoogle Scholar
  13. De la Cruz, R., Branco, M.D.: Bayesian analysis for nonlinear regression model under skewed errors, with application in growth curves. Biometric. J. 51(4), 588609 (2009) Google Scholar
  14. Demidenko, E.: Mixed Models: Theory and Applications. Wiley, New York (2004) zbMATHCrossRefGoogle Scholar
  15. Delyon, B., Lavielle, M., Moulines, E.: Convergence of a stochastic approximation version of the EM algorithm. Ann. Stat. 27, 94–128 (1999) MathSciNetzbMATHCrossRefGoogle Scholar
  16. Dempster, A.P., Laird, N.M., Rubin, D.B.: Maximum likelihood from incomplete data via the EM algorithm (with discussion). J. R. Stat. Soc. Ser. B 39, 1–38 (1977) MathSciNetzbMATHGoogle Scholar
  17. Fernández, C., Steel, M.F.J.: Multivariate Student-t regression models: pitfalls and inference. Biometrika 86, 153–167 (1999) MathSciNetzbMATHCrossRefGoogle Scholar
  18. Gu, M.G., Kong, F.H.: A stochastic approximation algorithm with Markov chain Monte-Carlo method for incomplete data estimation problems. Proc. Natl. Acad. Sci. USA 95, 7270–7274 (1998) MathSciNetzbMATHCrossRefGoogle Scholar
  19. Jank, W.: Implementing and diagnosing the stochastic approximation EM algorithm. J. Comput. Graph. Stat. 15, 803–829 (2006) MathSciNetCrossRefGoogle Scholar
  20. Jamshidian, M.: Adaptive robust regression by using a nonlinear regression program. J. Stat. Softw. (1999)
  21. Jara, A., Quintana, F., San Martin, E.: Linear mixed models with skew-elliptical distributions: A Bayesian approach. Comput. Stat. Data Anal. 52, 5033–5045 (2008) MathSciNetzbMATHCrossRefGoogle Scholar
  22. Johansen, S.: Functional Relations, Random Coefficients, and Nonlinear Regression with Application to Kinectic Data. Springer, New York (1984) CrossRefGoogle Scholar
  23. Kent, J.T., Tyler, D.E., Vardi, Y.: A curious likelihood identity for the multivariate t distribution. Commun. Stat. Simul. C. 23, 441–453 (1994) MathSciNetzbMATHCrossRefGoogle Scholar
  24. Kuhn, E., Lavielle, M.: Coupling a stochastic approximation version of EM with a MCMC procedure. ESAIM P&S 8, 115–131 (2004) MathSciNetzbMATHCrossRefGoogle Scholar
  25. Kuhn, E., Lavielle, M.: Maximum likelihood estimation in nonlinear mixed effects models. Comput. Stat. Data Anal. 49, 1020–1038 (2005) MathSciNetzbMATHCrossRefGoogle Scholar
  26. Lange, K., Sinsheimer, J.: Normal/independent distributions and their applications in robust regression. J. Comput. Graph. Stat. 2, 175–198 (1993) MathSciNetCrossRefGoogle Scholar
  27. Lange, K.L., Little, R.J.A., Taylor, J.M.G.: Robust statistical modeling using the t distribution. J. Am. Stat. Assoc. 84, 881–896 (1989) MathSciNetCrossRefGoogle Scholar
  28. Lavielle, M.: Monolix User Guide Manual. (2005)
  29. Lavielle, M., Meza, C.: A parameter expansion version of the SAEM algorithm. Stat. Comput. 17, 121–130 (2007) MathSciNetCrossRefGoogle Scholar
  30. Lee, S., Xu, L.: Influence analyses of nonlinear mixed-effects models. Comput. Stat. Data Anal. 45, 321–341 (2004) MathSciNetzbMATHCrossRefGoogle Scholar
  31. Lin, T.I.: Longitudinal data analysis using t linear mixed models with autoregressive dependence structures. J. Data Sci. 6, 333–355 (2008) Google Scholar
  32. Lin, T.I., Lee, J.C.: A robust approach to t linear mixed models applied to multiple sclerosis data. Stat. Med. 25, 1397–1412 (2006) MathSciNetCrossRefGoogle Scholar
  33. Lin, T.I., Lee, J.C.: Estimation and prediction in linear mixed models with skew-normal random effects for longitudinal data. Stat. Med. 27, 1490–1507 (2007) MathSciNetCrossRefGoogle Scholar
  34. Lindstrom, M.J., Bates, D.M.: Nonlinear mixed-effects models for repeated measures data. Biometrics 46, 673–787 (1990) MathSciNetCrossRefGoogle Scholar
  35. Liu, C.: Bayesian robust multivariate linear regression with incomplete data. J. Am. Stat. Assoc. 91, 1219–1227 (1996) zbMATHCrossRefGoogle Scholar
  36. Liu, C., Rubin, D., Wu, Y.: Parameter expansion to accelerate EM: The PX-EM algorithm. Biometrika 85, 755–770 (1998) MathSciNetzbMATHCrossRefGoogle Scholar
  37. Louis, T.A.: Finding the observed information matrix when using the EM algorithm. J. R. Stat. Soc. Ser. B 44, 226–233 (1982) MathSciNetzbMATHGoogle Scholar
  38. Lucas, A.: Robustness of the Student t based-M-estimator. Commun. Stat. Theor. M 26, 1165–1182 (1997) MathSciNetzbMATHCrossRefGoogle Scholar
  39. McCulloch, C.E.: Maximum likelihood algorithms for generalized linear mixed models. J. Am. Stat. Assoc. 92, 162–170 (1997) MathSciNetzbMATHCrossRefGoogle Scholar
  40. Meng, X.L., van Dyk, D.A.: The EM algorithm—an old folk song sung to a fast new tune (with discussion). J. R. Stat. Soc. Ser. B 59, 511–567 (1997) zbMATHCrossRefGoogle Scholar
  41. Meza, C., Jaffrézic, F., Foulley, J.-L.: REML estimation of variance parameters in nonlinear mixed effects models using SAEM algorithm. Biometric. J. 49, 876–888 (2007) CrossRefGoogle Scholar
  42. Meza, C., Jaffrézic, F., Foulley, J.-L.: Estimation in the probit normal model for binary outcomes using the SAEM algorithm. Comput. Stat. Data Anal. 53, 1350–1360 (2009) zbMATHCrossRefGoogle Scholar
  43. Osorio, F., Paula, G.A., Galea, M.: Assessment of local influence in elliptical linear models with longitudinal structure. Comput. Stat. Data Anal. 51, 4354–4368 (2007) MathSciNetzbMATHCrossRefGoogle Scholar
  44. Philippe, A.: Simulation of right and left truncated gamma distributions by mixtures. Stat. Comput. 7, 173–181 (1997) CrossRefGoogle Scholar
  45. Pinheiro, J., Bates, D.M.: Approximations to the log-likelihood function in the nonlinear mixed-effects model. J. Comput. Graph. Stat. 4, 12–35 (1995) CrossRefGoogle Scholar
  46. Pinheiro, J., Bates, D.M.: Mixed-Effects Models in S and S-PLUS. Springer, New York (2000) zbMATHCrossRefGoogle Scholar
  47. Pinheiro, J., Liu, C., Wu, Y.: Efficient algorithms for robust estimation in linear mixed-effects models using the multivariate t distribution. J. Comput. Graph. Stat. 10, 249–276 (2001) MathSciNetCrossRefGoogle Scholar
  48. Roberts, G.O., Rosenthal, J.S.: Optimal scaling of various metropolis-hastings algorithms. Stat. Sci. 16, 351–367 (2001) MathSciNetzbMATHCrossRefGoogle Scholar
  49. Roberts, G.O., Gelman, A., Gilks, W.: Weak convergence and optimal scaling of random walk metropolis algorithm. Ann. Appl. Prob. 7, 110–120 (1997) MathSciNetzbMATHCrossRefGoogle Scholar
  50. Rogers, W.H., Tuckey, J.W.: Understanding some long-tailed distributions. Stat. Neerl. 26, 211–226 (1972) CrossRefGoogle Scholar
  51. Rosa, G.J.M., Padovani, C.R., Gianola, D.: Robust linear mixed models with Normal/Independent distributions and Bayesian MCMC implementation. Biometric. J. 45, 573–590 (2003) MathSciNetCrossRefGoogle Scholar
  52. Rosa, G.J.M., Gianola, D., Padovani, C.R.: Bayesian longitudinal data analysis with mixed models and thick-tailed distributions using MCMC. J. Appl. Stat. 31, 855–873 (2004) MathSciNetzbMATHCrossRefGoogle Scholar
  53. Russo, C.M., Paula, G.A., Aoki, R.: Influence diagnostics in nonlinear mixed-effects elliptical models. Comput. Stat. Data Anal. 53, 4143–4156 (2009) MathSciNetzbMATHCrossRefGoogle Scholar
  54. Shi, L., Chen, G.: Detection of outliers in multilevel models. J. Stat. Plan. Inference 138, 3189–3199 (2008) MathSciNetzbMATHCrossRefGoogle Scholar
  55. Staudenmayer, J., Lake, E.E., Wand, M.P.: Robustness for general design mixed models using the t-distribution. Stat. Model. 9, 235–255 (2009) MathSciNetCrossRefGoogle Scholar
  56. Spiegelhalter, D.J., Thomas, A., Best, N.G.: Winbugs version 1.2 user manual. MRC Biostatistics Unit (1999) Google Scholar
  57. Vaida, F.: Parameter convergence for EM and MM algorithms. Stat. Sin. 15, 831–840 (2005) MathSciNetzbMATHGoogle Scholar
  58. Vonesh, E.F.: A note on the use of Laplace’s approximation for nonlinear mixed-effects models. Biometrika 83, 447–452 (1996) MathSciNetzbMATHCrossRefGoogle Scholar
  59. Vonesh, E.F., Chinchilli, V.M.: Linear and Nonlinear Models for the Analysis of Repeated Measurements. Marcel Dekker, New York (1997) zbMATHGoogle Scholar
  60. Walker, S.: An EM algorithm for nonlinear random effects models. Biometrics 52, 934–944 (1996) MathSciNetzbMATHCrossRefGoogle Scholar
  61. Wang, J.: EM algorithms for nonlinear mixed effects models. Comput. Stat. Data Anal. 51, 3244–3256 (2007) zbMATHCrossRefGoogle Scholar
  62. Wei, B., Shih, J.: On statistical models for regression diagnostics. Ann. Inst. Stat. Math. 46, 267–278 (1994) MathSciNetzbMATHCrossRefGoogle Scholar
  63. Wei, G., Tanner, M.: A Monte Carlo implementation of the EM algorithm and the poor man’s data augmentation algorithms. J. Am. Stat. Assoc. 85, 699–704 (1990) CrossRefGoogle Scholar
  64. Wei, W.H., Fung, W.K.: The mean-shift outlier model in general weighted regression and its applications. Comput. Stat. Data Anal. 30, 429–441 (1999) MathSciNetzbMATHCrossRefGoogle Scholar
  65. Welsh, A.H., Richardson, A.M.: Approaches to the robust estimation of mixed models. In: Maddala, G.S., Rao, C.R. (eds.) Handbook of Statistics, vol. 15, pp. 343–384. Elsevier Science, Amsterdam (1997) Google Scholar
  66. Wolfinger, R.: Laplace’s approximation for nonlinear mixed models. Biometrika 80, 791–795 (1993) MathSciNetzbMATHCrossRefGoogle Scholar
  67. Wolfinger, R.D., Lin, X.: Two Taylor-series approximation methods for nonlinear mixed models. Comput. Stat. Data Anal. 25, 465490 (1997) CrossRefGoogle Scholar
  68. Wu, C.-F.J.: On the convergence properties of the EM algorithm. Ann. Stat. 11, 95–103 (1983) zbMATHCrossRefGoogle Scholar
  69. Yeap, B.Y., Davidian, M.: Robust two-stage estimation in hierarchical nonlinear models. Biometrics 57, 266–272 (2001) MathSciNetzbMATHCrossRefGoogle Scholar
  70. Yeap, B.Y., Catalano, P.J., Ryan, L.M., Davidian, M.: Robust two stage approach to repeated measurements analysis of chronic ozone exposure in rats. J. Agric. Biol. Environ. Stat. 8, 438–454 (2003) CrossRefGoogle Scholar
  71. Zhu, H., Lee, S.: Analysis of generalized linear mixed models via a stochastic approximation algorithm with Markov chain Monte-Carlo method. Stat. Comput. 12, 175–183 (2002) MathSciNetCrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2010

Authors and Affiliations

  • Cristian Meza
    • 1
    Email author
  • Felipe Osorio
    • 1
  • Rolando De la Cruz
    • 2
  1. 1.Departamento de Estadística, CIMFAVUniversidad de ValparaísoValparaísoChile
  2. 2.Departamento de Salud Pública, Escuela de Medicina, and Departamento de Estadística, Facultad de MatemáticasPontificia Universidad Católica de ChileSantiagoChile

Personalised recommendations