Statistics and Computing

, Volume 9, Issue 2, pp 91–99 | Cite as

Pointwise and functional approximations in Monte Carlo maximum likelihood estimation

  • Anthony Y. C. Kuk
  • Yuk W. Cheng


We consider the use of Monte Carlo methods to obtain maximum likelihood estimates for random effects models and distinguish between the pointwise and functional approaches. We explore the relationship between the two approaches and compare them with the EM algorithm. The functional approach is more ambitious but the approximation is local in nature which we demonstrate graphically using two simple examples. A remedy is to obtain successively better approximations of the relative likelihood function near the true maximum likelihood estimate. To save computing time, we use only one Newton iteration to approximate the maximiser of each Monte Carlo likelihood and show that this is equivalent to the pointwise approach. The procedure is applied to fit a latent process model to a set of polio incidence data. The paper ends by a comparison between the marginal likelihood and the recently proposed hierarchical likelihood which avoids integration altogether.

EM algorithm Gibbs sampling hierarchical likelihood importance sampling marginal likelihood Newton Raphson procedure random effects 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Anderson, D. A. and Aitkin, M. A. (1985) Variance component models with binary response: interviewer variability. J. R. Statist. Soc. B, 47, 203–210.Google Scholar
  2. Breslow, N. E. and Clayton, D. G. (1993) Approximate inference in generalized linear mixed models. J. Am. Statist. Ass., 88, 9–25.Google Scholar
  3. Chan, K. S. and Ledolter, J. (1995) Monte Carlo EM estimation for time series models involving counts. J. Amer. Statist. Assoc., 90, 242–252.Google Scholar
  4. Crouch, E. A. C. and Spiegelman, D. (1990) The evaluation of integrals of the from ∫ f(t)exp(–t 2)dt: application to logistic-normal models. J. Am. Statist. Ass., 85, 464–469.Google Scholar
  5. Dempster, A. P., Laird, N. M. and Rubin, D. B. (1977) Maxi-mum likelihood from incomplete data via the EM algorithm (with discussion). J. R. Statist. Soc. B, 39, 1–38.Google Scholar
  6. Geyer, C. J. (1994) On the convergence of Monte Carlo maxi-mum likelihood calculations. J. R. Statist. Soc. B., 56, 261–274.Google Scholar
  7. Geyer, C. J. and Thompson, E. A. (1992) Constrained maximum likelihood for dependent data (with discussion). J. R. Statist. Soc. B., 54, 657–699.Google Scholar
  8. Karim, M. R. and Zeger, S. L. (1992) Generalized linear models with random effects; salamander mating revisited. Biometrics, 48, 631–644.Google Scholar
  9. Kleinman, J. C. (1973) Proportions with extraneous variance: single and independent samples. J. Amer. Statist. Assoc., 68, 46–54.Google Scholar
  10. Kuk, A. Y. C. and Cheng, Y. W. (1997) The Monte Carlo Newton-Raphson algorithm. J. Statist. Comput. Simul., 59, 233–250.Google Scholar
  11. Lange, K. (1995) A gradient algorithm locally equivalent to the EM algorithm. J. R. Statist. Soc. B, 57, 425–437.Google Scholar
  12. Lee, Y. and Nelder, J. A. (1996) Hierarchical generalized linear models (with discussion). J. R. Statist. Soc. B, 58, 619–678.Google Scholar
  13. Louis, T. A. (1982) Finding the observed information matrix when using the EM algorithm. J. R. Statist. Soc. B, 44, 226–233.Google Scholar
  14. McCulloch, C. E. (1997) Maximum likelihood algorithms for generalized linear mixed models. J. Am. Statist. Assoc., 92, 162–170.Google Scholar
  15. McGilchrist, C. A. (1994) Estimation in generalized mixed mod-els. J. R. Statist. Soc. B, 56, 61–69.Google Scholar
  16. McGilchrist, C. A. and Aisbett, C. W. (1991a) Restricted BLUP for mixed linear models. Biometr. J., 32, 545–550.Google Scholar
  17. McGilchrist, C. A. and Aisbett, C. W. (1991b) Regression with frailty in survival analysis. Biometrics, 47, 461–466.Google Scholar
  18. Penttinen, A. (1984) Modelling interaction in spatial point pat-terns: parameter estimation by the maximum likelihood method. Jy. Stud. Comput. Sci. Econ. Statist., 7, 1–105Google Scholar
  19. Schall, R. (1991) Estimation in generalised linear models with random effects. Biometrika, 78, 719–727.Google Scholar
  20. Smith, D. M. (1983) Maximum likelihood estimation of the pa-rameters of the beta binomial distribution. Applied Statist., 32, 196–204.Google Scholar
  21. Wei, G. C. G. and Tanner, M. A. (1990) A Monte Carlo imple-mentation of the EM algorithm and the poor man's data augmentation algorithms. J. Amer. Statist. Assoc., 85, 699–704.Google Scholar
  22. Weil, C. S. (1970) Selection of the valid number of sampling units and consideration of their combination in toxicological studies involving reproduction, teratogenesis or carcinogen-esis. Food and Cosmetic Toxicology, 8, 177–182.Google Scholar
  23. Zeger, S. L. (1988) A regression model for time series of counts. Biometrika, 75, 621–629.Google Scholar

Copyright information

© Kluwer Academic Publishers 1999

Authors and Affiliations

  • Anthony Y. C. Kuk
  • Yuk W. Cheng

There are no affiliations available

Personalised recommendations