Maximum likelihood computation for fitting semiparametric mixture models

Statistics and Computing volume 20, pages75–86(2010)Cite this article

Abstract

Three general algorithms that use different strategies are proposed for computing the maximum likelihood estimate of a semiparametric mixture model. They seek to maximize the likelihood function by, respectively, alternating the parameters, profiling the likelihood and modifying the support set. All three algorithms make a direct use of the recently proposed fast and stable constrained Newton method for computing the nonparametric maximum likelihood of a mixing distribution and employ additionally an optimization algorithm for unconstrained problems. The performance of the algorithms is numerically investigated and compared for solving the Neyman-Scott problem, overcoming overdispersion in logistic regression models and fitting two-level mixed effects logistic regression models. Satisfactory results have been obtained.

This is a preview of subscription content, log in to check access.

References

  1. Aitkin, M.: A general maximum likelihood analysis of overdispersion in generalised linear models. Stat. Comput. 6, 251–262 (1996)

    Article  Google Scholar 

  2. Aitkin, M.: A general maximum likelihood analysis of variance components in generalized linear models. Biometrics 55, 117–128 (1999)

    MATH  Article  MathSciNet  Google Scholar 

  3. Böhning, D.: A review of reliable algorithms for the semi-parametric maximum likelihood estimator of a mixture distribution. J. Stat. Plan. Inference 47, 5–28 (1995)

    MATH  Article  Google Scholar 

  4. Böhning, D.: The EM algorithm with gradient function update for discrete mixtures with known (fixed) number of components. Stat. Comput. 13, 257–265 (2003)

    Article  MathSciNet  Google Scholar 

  5. Dax, A.: The smallest point of a polytope. J. Optim. Theory Appl. 64, 429–432 (1990)

    MATH  Article  MathSciNet  Google Scholar 

  6. DerSimonian, R.: Maximum likelihood estimation of a mixing distribution. J. R. Stat. Soc., Ser. C 35, 302–309 (1986)

    MATH  Google Scholar 

  7. Einbeck, J., Darnell, R., Hinde, J.: NPMLREG: Nonparametric maximum likelihood estimation for random effect models. R Package Version 0.43 (2007)

  8. Fletcher, R.: Practical Methods of Optimization, 2nd edn. Wiley, New York (1987)

    Google Scholar 

  9. Follmann, D.A., Lambert, D.: Generalizing logistic regression by nonparametric mixing. J. Am. Stat. Assoc. 84, 295–300 (1989)

    Article  Google Scholar 

  10. Haskell, K.H., Hanson, R.J.: An algorithm for linear least squares problems with equality and nonnegativity constraints. Math. Program. 21, 98–118 (1981)

    MATH  Article  MathSciNet  Google Scholar 

  11. Heckman, J., Singer, B.: A method for minimizing the impact of distributions assumptions in econometric models for duration data. Econometrica 52, 271–320 (1984)

    MATH  Article  MathSciNet  Google Scholar 

  12. Jamshidian, M., Jennrich, R.I.: Acceleration of the EM algorithm by using quasi-Newton methods. J. R. Stat. Soc., Ser. B 59, 569–587 (1997)

    MATH  Article  MathSciNet  Google Scholar 

  13. Kiefer, J., Wolfowitz, J.: Consistency of the maximum likelihood estimator in the presence of infinitely many incidental parameters. Ann. Math. Stat. 27, 886–906 (1956)

    Google Scholar 

  14. Laird, N.M.: Nonparametric maximum likelihood estimation of a mixing distribution. J. Am. Stat. Assoc. 73, 805–811 (1978)

    MATH  Article  MathSciNet  Google Scholar 

  15. Lawson, C.L., Hanson, R.J.: Solving Least Squares Problems. Prentice-Hall, New York (1974)

    Google Scholar 

  16. Lindsay, B.G.: The geometry of mixture likelihoods: A general theory. Ann. Stat. 11, 86–94 (1983)

    MATH  Article  MathSciNet  Google Scholar 

  17. Lindsay, B.G.: Mixture Models: Theory, Geometry and Applications. NSF-CBMS Regional Conference Series in Probability and Statistics, vol. 5. Institute for Mathematical Statistics, Hayward (1995)

    Google Scholar 

  18. Lindsay, B.G., Lesperance, M.L.: A review of semiparametric mixture models. J. Stat. Plan. Inference 47, 29–39 (1995)

    MATH  Article  MathSciNet  Google Scholar 

  19. Murphy, S.A., van der Vaart, A.W.: On profile likelihood. J. Am. Stat. Assoc. 95, 449–465 (2000)

    MATH  Article  MathSciNet  Google Scholar 

  20. Neyman, J., Scott, E.L.: Consistent estimates based on partially consistent observations. Econometrica 16, 1–32 (1948)

    Article  MathSciNet  Google Scholar 

  21. Powell, M.J.D.: On search directions for minimization algorithms. Math. Program. 4, 193–201 (1973)

    MATH  Article  MathSciNet  Google Scholar 

  22. R Development Core Team. R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria (2006)

  23. Tsodikov, A.: Semiparametric models: a generalized self-consistency approach. J. R. Stat. Soc., Ser. B 65, 759–774 (2003)

    MATH  Article  MathSciNet  Google Scholar 

  24. Wang, Y.: On fast computation of the non-parametric maximum likelihood estimate of a mixing distribution. J. R. Stat. Soc., Ser. B 69, 185–198 (2007)

    MATH  Article  Google Scholar 

  25. Wang, Y.: Dimension-reduced nonparametric maximum likelihood computation for interval-censored data. Comput. Stat. Data Anal. 52, 2388–2402 (2008)

    MATH  Article  Google Scholar 

  26. Zeng, D., Lin, D.Y.: Maximum likelihood estimation in semiparametric regression models with censored data. J. R. Stat. Soc., Ser. B 69, 507–564 (2007)

    Article  MathSciNet  Google Scholar 

Download references

Author information

Affiliations

  1. Department of Statistics, The University of Auckland, Private Bag 92019, Auckland, 1142, New Zealand

    Yong Wang

Authors
  1. Yong Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yong Wang.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Wang, Y. Maximum likelihood computation for fitting semiparametric mixture models. Stat Comput 20, 75–86 (2010). https://doi.org/10.1007/s11222-009-9117-z

Download citation

Keywords

  • Constrained optimization
  • Maximum likelihood computation
  • Mixed effects
  • Neyman-Scott problem
  • Profile likelihood
  • Semiparametric mixture