Statistics and Computing

, Volume 20, Issue 1, pp 75–86 | Cite as

Maximum likelihood computation for fitting semiparametric mixture models

  • Yong WangEmail author


Three general algorithms that use different strategies are proposed for computing the maximum likelihood estimate of a semiparametric mixture model. They seek to maximize the likelihood function by, respectively, alternating the parameters, profiling the likelihood and modifying the support set. All three algorithms make a direct use of the recently proposed fast and stable constrained Newton method for computing the nonparametric maximum likelihood of a mixing distribution and employ additionally an optimization algorithm for unconstrained problems. The performance of the algorithms is numerically investigated and compared for solving the Neyman-Scott problem, overcoming overdispersion in logistic regression models and fitting two-level mixed effects logistic regression models. Satisfactory results have been obtained.


Constrained optimization Maximum likelihood computation Mixed effects Neyman-Scott problem Profile likelihood Semiparametric mixture 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Aitkin, M.: A general maximum likelihood analysis of overdispersion in generalised linear models. Stat. Comput. 6, 251–262 (1996) CrossRefGoogle Scholar
  2. Aitkin, M.: A general maximum likelihood analysis of variance components in generalized linear models. Biometrics 55, 117–128 (1999) zbMATHCrossRefMathSciNetGoogle Scholar
  3. Böhning, D.: A review of reliable algorithms for the semi-parametric maximum likelihood estimator of a mixture distribution. J. Stat. Plan. Inference 47, 5–28 (1995) zbMATHCrossRefGoogle Scholar
  4. Böhning, D.: The EM algorithm with gradient function update for discrete mixtures with known (fixed) number of components. Stat. Comput. 13, 257–265 (2003) CrossRefMathSciNetGoogle Scholar
  5. Dax, A.: The smallest point of a polytope. J. Optim. Theory Appl. 64, 429–432 (1990) zbMATHCrossRefMathSciNetGoogle Scholar
  6. DerSimonian, R.: Maximum likelihood estimation of a mixing distribution. J. R. Stat. Soc., Ser. C 35, 302–309 (1986) zbMATHGoogle Scholar
  7. Einbeck, J., Darnell, R., Hinde, J.: NPMLREG: Nonparametric maximum likelihood estimation for random effect models. R Package Version 0.43 (2007) Google Scholar
  8. Fletcher, R.: Practical Methods of Optimization, 2nd edn. Wiley, New York (1987) zbMATHGoogle Scholar
  9. Follmann, D.A., Lambert, D.: Generalizing logistic regression by nonparametric mixing. J. Am. Stat. Assoc. 84, 295–300 (1989) CrossRefGoogle Scholar
  10. Haskell, K.H., Hanson, R.J.: An algorithm for linear least squares problems with equality and nonnegativity constraints. Math. Program. 21, 98–118 (1981) zbMATHCrossRefMathSciNetGoogle Scholar
  11. Heckman, J., Singer, B.: A method for minimizing the impact of distributions assumptions in econometric models for duration data. Econometrica 52, 271–320 (1984) zbMATHCrossRefMathSciNetGoogle Scholar
  12. Jamshidian, M., Jennrich, R.I.: Acceleration of the EM algorithm by using quasi-Newton methods. J. R. Stat. Soc., Ser. B 59, 569–587 (1997) zbMATHCrossRefMathSciNetGoogle Scholar
  13. Kiefer, J., Wolfowitz, J.: Consistency of the maximum likelihood estimator in the presence of infinitely many incidental parameters. Ann. Math. Stat. 27, 886–906 (1956) Google Scholar
  14. Laird, N.M.: Nonparametric maximum likelihood estimation of a mixing distribution. J. Am. Stat. Assoc. 73, 805–811 (1978) zbMATHCrossRefMathSciNetGoogle Scholar
  15. Lawson, C.L., Hanson, R.J.: Solving Least Squares Problems. Prentice-Hall, New York (1974) zbMATHGoogle Scholar
  16. Lindsay, B.G.: The geometry of mixture likelihoods: A general theory. Ann. Stat. 11, 86–94 (1983) zbMATHCrossRefMathSciNetGoogle Scholar
  17. Lindsay, B.G.: Mixture Models: Theory, Geometry and Applications. NSF-CBMS Regional Conference Series in Probability and Statistics, vol. 5. Institute for Mathematical Statistics, Hayward (1995) zbMATHGoogle Scholar
  18. Lindsay, B.G., Lesperance, M.L.: A review of semiparametric mixture models. J. Stat. Plan. Inference 47, 29–39 (1995) zbMATHCrossRefMathSciNetGoogle Scholar
  19. Murphy, S.A., van der Vaart, A.W.: On profile likelihood. J. Am. Stat. Assoc. 95, 449–465 (2000) zbMATHCrossRefMathSciNetGoogle Scholar
  20. Neyman, J., Scott, E.L.: Consistent estimates based on partially consistent observations. Econometrica 16, 1–32 (1948) CrossRefMathSciNetGoogle Scholar
  21. Powell, M.J.D.: On search directions for minimization algorithms. Math. Program. 4, 193–201 (1973) zbMATHCrossRefMathSciNetGoogle Scholar
  22. R Development Core Team. R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria (2006) Google Scholar
  23. Tsodikov, A.: Semiparametric models: a generalized self-consistency approach. J. R. Stat. Soc., Ser. B 65, 759–774 (2003) zbMATHCrossRefMathSciNetGoogle Scholar
  24. Wang, Y.: On fast computation of the non-parametric maximum likelihood estimate of a mixing distribution. J. R. Stat. Soc., Ser. B 69, 185–198 (2007) zbMATHCrossRefGoogle Scholar
  25. Wang, Y.: Dimension-reduced nonparametric maximum likelihood computation for interval-censored data. Comput. Stat. Data Anal. 52, 2388–2402 (2008) zbMATHCrossRefGoogle Scholar
  26. Zeng, D., Lin, D.Y.: Maximum likelihood estimation in semiparametric regression models with censored data. J. R. Stat. Soc., Ser. B 69, 507–564 (2007) CrossRefMathSciNetGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2009

Authors and Affiliations

  1. 1.Department of StatisticsThe University of AucklandAucklandNew Zealand

Personalised recommendations