Statistics and Computing

, Volume 22, Issue 1, pp 79–92 | Cite as

Multivariate generalized linear mixed models with semi-nonparametric and smooth nonparametric random effects densities



We extend the family of multivariate generalized linear mixed models to include random effects that are generated by smooth densities. We consider two such families of densities, the so-called semi-nonparametric (SNP) and smooth nonparametric (SMNP) densities. Maximum likelihood estimation, under either the SNP or the SMNP densities, is carried out using a Monte Carlo EM algorithm. This algorithm uses rejection sampling and automatically increases the MC sample size as it approaches convergence. In a simulation study we investigate the performance of these two densities in capturing the true underlying shape of the random effects distribution. We also examine the implications of misspecification of the random effects distribution on the estimation of the fixed effects and their standard errors. The impact of the assumed random effects density on the estimation of the random effects themselves is investigated in a simulation study and also in an application to a real data set.


Longitudinal data Mixed models Multinomial responses Random effects Semi-nonparametric densities Smooth nonparametric densities 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Agresti, A., Caffo, B., Ohman-Strickland, P.: Examples in which misspecification of a random effects distribution reduces efficiency, and possible remedies. Comput. Stat. Data Anal. 47(3), 639–653 (2004) MathSciNetMATHCrossRefGoogle Scholar
  2. Aitkin, M.: A general maximum likelihood analysis of variance components in generalized linear models. Biometrics 55(1), 117–128 (1999) MathSciNetMATHCrossRefGoogle Scholar
  3. Booth, J.G., Hobert, J.P.: Maximizing generalized linear mixed model likelihoods with an automated Monte Carlo EM algorithm. J. R. Stat. Soc., Ser. B Stat. Methodol. 61, 265–285 (1999) MATHCrossRefGoogle Scholar
  4. Breslow, N.E., Clayton, D.G.: Approximate inference in generalized linear mixed models. J. Am. Stat. Assoc. 88, 9–25 (1993) MATHCrossRefGoogle Scholar
  5. Carroll, R.J., Hall, P.: Optimal rates of convergence for deconvolving a density. J. Am. Stat. Assoc. 83, 1184–1186 (1988) MathSciNetMATHCrossRefGoogle Scholar
  6. Chen, J., Zhang, D., Davidian, M.: A Monte Carlo EM algorithm for generalized linear mixed models with flexible random effects distribution. Biostatistics 3(3), 347–360 (2002) MATHCrossRefGoogle Scholar
  7. Davidian, M., Gallant, AR: The nonlinear mixed effects model with a smooth random effects density. Biometrika 80, 475–488 (1993) MathSciNetMATHCrossRefGoogle Scholar
  8. Dempster, A.P., Laird, N.M., Rubin, D.B.: Maximum likelihood from incomplete data via the EM algorithm. J. R. Stat. Soc., Ser. B, Methodol. 39, 1–22 (1977) MathSciNetMATHGoogle Scholar
  9. Eilers, P.H.C., Marx, B.D.: Flexible smoothing with B-splines and penalties. Stat. Sci. 11(2), 89–121 (1996) MathSciNetMATHCrossRefGoogle Scholar
  10. Fahrmeir, L., Kaufmann, H.: Consistency and asymptotic normality of the maximum likelihood estimator in generalized linear models. Ann. Stat. 13, 342–368 (1985) MathSciNetMATHCrossRefGoogle Scholar
  11. Fahrmeir, L., Tutz, G.: Multivariate Statistical Modelling Based on Generalized Linear Models. Springer, Berlin (2001) MATHGoogle Scholar
  12. Follmann, D.A., Lambert, D.: Generalizing logistic regression by nonparametric mixing. J. Am. Stat. Assoc. 84, 295–300 (1989) CrossRefGoogle Scholar
  13. Gallant, AR, Nychka, D.W.: Semi-nonparametric maximum likelihood estimation. Econometrica 55, 363–390 (1987) MathSciNetMATHCrossRefGoogle Scholar
  14. Gallant, R., Tauchen, G.: A nonparametric approach to nonlinear time series analysis: estimation and simulation. In: Brillinger, D., Caines, P., Geweke, J., Parzen, E., Rosenblatt, M., Taqqu, M. (eds.) New Directions in Time Series Analysis, Part II, pp. 71–92. Springer, Berlin (1992) Google Scholar
  15. Geweke, J.: Monte Carlo simulation and numerical integration. In: Amman, H.M., Kendrick, D.A., Rust, J. (eds.) Handbook of Computational Economics, pp. 731–800. Elsevier, Amsterdam (1996) Google Scholar
  16. Ghidey, W., Lesaffre, E., Eilers, P.: Smooth random effects distribution in a linear mixed model. Biometrics 60(4), 945–953 (2004) MathSciNetMATHCrossRefGoogle Scholar
  17. Hartzel, J., Agresti, A., Caffo, B.: Multinomial logit random effects models. Stat. Model. 1(2), 81–102 (2001) MATHCrossRefGoogle Scholar
  18. Heagerty, P.J., Kurland, B.F.: Misspecified maximum likelihood estimates and generalised linear mixed models. Biometrika 88(4), 973–985 (2001) MathSciNetMATHCrossRefGoogle Scholar
  19. Hedeker, D., Gibbons, R.: Longitudinal Data Analysis. Wiley, Palo Alto (2006) MATHGoogle Scholar
  20. Laird, N.: Nonparametric maximum likelihood estimation of a mixing distribution. J. Am. Stat. Assoc. 73, 805–811 (1978) MathSciNetMATHCrossRefGoogle Scholar
  21. Lesperance, M.L., Kalbfleisch, J.D.: An algorithm for computing the nonparametric MLE of a mixing distribution. J. Am. Stat. Assoc. 87, 120–126 (1992) MATHCrossRefGoogle Scholar
  22. Lindsay, B.G.: The geometry of mixture likelihoods, part II: the exponential family. Ann. Stat. 11, 783–792 (1983) MathSciNetMATHCrossRefGoogle Scholar
  23. Litière, S., Alonso, A., Molenberghs, G.: The impact of a misspecified random-effects distribution on the estimation and the performance of inferential procedures in generalized linear mixed models. Stat. Med. 27(16), 3125–3144 (2008) MathSciNetCrossRefGoogle Scholar
  24. Magder, L.S., Zeger, S.L.: A smooth nonparametric estimate of a mixing distribution using mixtures of Gaussians. J. Am. Stat. Assoc. 91, 1141–1151 (1996) MathSciNetMATHCrossRefGoogle Scholar
  25. McCulloch, C.E.: Maximum likelihood algorithms for generalized linear mixed models. J. Am. Stat. Assoc. 92, 162–170 (1997) MathSciNetMATHCrossRefGoogle Scholar
  26. Monahan, J.F.: An algorithm for generating chi random variables. ACM Trans. Math. Softw. 13, 168–172 (1987) MathSciNetMATHCrossRefGoogle Scholar
  27. Neuhaus, J.M., Hauck, W.W., Kalbfleisch, J.D.: The effects of mixture distribution misspecification when fitting mixed-effects logistic models. Biometrika 79, 755–762 (1992) CrossRefGoogle Scholar
  28. Tutz, G., Hennevogl, W.: Random effects in ordinal regression models. Comput. Stat. Data Anal. 22, 537–557 (1996) MATHCrossRefGoogle Scholar
  29. Verbeke, G., Lesaffre, E.: A linear mixed-effects model with heterogeneity in the random-effects population. J. Am. Stat. Assoc. 91, 217–221 (1996) MATHCrossRefGoogle Scholar
  30. Wei, G.C.G., Tanner, M.A.: A Monte Carlo implementation of the EM algorithm and the poor man’s data augmentation algorithms. J. Am. Stat. Assoc. 85, 699–704 (1990) CrossRefGoogle Scholar
  31. Zhang, D., Davidian, M.: Linear mixed models with flexible distribution of random effects for longitudinal data. Biometrics 57(3), 795–802 (2001) MathSciNetMATHCrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2010

Authors and Affiliations

  1. 1.School of Mathematics, Statistics and Applied MathematicsNational University of IrelandGalwayIreland

Personalised recommendations