Statistics and Computing

, Volume 7, Issue 2, pp 145–151 | Cite as

An EM-type algorithm for multivariate mixture models

  • G. R. Oskrochi
  • R. B. Davies


This paper introduces a new approach, based on dependent univariate GLMs, for fitting multivariate mixture models. This approach is a multivariate generalization of the method for univariate mixtures presented by Hinde (1982). Its accuracy and efficiency are compared with direct maximization of the log-likelihood. Using a simulation study, we also compare the efficiency of Monte Carlo and Gaussian quadrature methods for approximating the mixture distribution. The new approach with Gaussian quadrature outperforms the alternative methods considered. The work is motivated by the multivariate mixture models which have been proposed for modelling changes of employment states at an individual level. Similar formulations are of interest for modelling movement between other social and economic states and multivariate mixture models also occur in biostatistics and epidemiology.

Multivariate generalized linear models Markov model EM algorithm random effect models Monte Carlo simulation Cholesky decomposition 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Anderson, D. A. and Hinde, J. (1988) Random effects in generalised linear models and the EM algorithm. Communications in Statistics, 17, 3847–56.Google Scholar
  2. Clayton, D. (1994) Generalised linear mixed models. In Markov Chain Monte Carlo in Practice. W. Gilks, S. Richardson, and D Spiegelhalter (eds). Chapman & Hall, London.Google Scholar
  3. Davies, R. B. (1984) A generalised beta-logistic model for longitudinal data with an application to residential mobility. Environment and Planning, 16, 1375–86.Google Scholar
  4. Davies, R. B.(1993) Nonparametric control for residual heterogeneity in modelling recurrent behaviour. Computational Statistics and Data Analysis, 16, 143–60Google Scholar
  5. Davies, R. B. and Pickles, A. R. (1987) A joint trip timing, store choice model including feedback effects and nonparametric control for omitted variables. Transportation Research A, 21, 345–61.Google Scholar
  6. Fahrmeir, L. and Tutz, G. (1994) Multivariate Statistical Modelling Based on Generalised Linear Models. Springer-Verlag, New York.Google Scholar
  7. Flinn, C. and Heckman, J. J. (1982) New methods for analysing individual event histories. In Sociological Methodology 1982, S. Leinhardt (ed.), pp. 99–140, Jossey Bass, San Francisco.Google Scholar
  8. Fotouhi, A. and Davies, R. B. (1995) The initial condition problem for discrete-time, discrete-response data. Working paper of the Centre for Applied Statistics, Lancaster University.Google Scholar
  9. Gaus 3.0 Applications (1992) Maximum likelihood. Aptech Systems, Maple Valley, WA.Google Scholar
  10. Heckman, J. J. and Singer, B. (1984) A method for minimising the impact of distributional assumptions in econometric models for duration data. Econometrica, 52, 271–320.Google Scholar
  11. Hinde, J. (1982) Compound Poisson regression models. In GLIM 82, Proceedings of the International Conference on Generalised Linear Models, R. Gilchrist (ed.), pp. 109–121. Springer-Verlag, Berlin.Google Scholar
  12. Laird, N. (1978) Nonparametric maximum likelihood estimation of a mixing distribution. Journal of the American Statistical Association, 73, 805–11.Google Scholar
  13. Pickles, A. R. and Davies R. B., (1985) The longitudinal analysis of housing careers. Journal of Regional Science, 25, 85–101.Google Scholar
  14. Stiratelli, R., Laird, N. and Ware, J. H. (1984) Random-effects models for serial observation with binary response. Biometrics, 40, 961–71.PubMedGoogle Scholar
  15. Wood, A. and Hinde, J. (1987) Binomial variance component models with nonparametric mixing distributions: a GLIM approach. In Longitudinal Data Analysis, R. Crouchley (ed.), pp. 110–28, Avebury, Aldershot, UK.Google Scholar

Copyright information

© Chapman and Hall 1997

Authors and Affiliations

  • G. R. Oskrochi
    • 1
  • R. B. Davies
    • 1
  1. 1.Centre for Applied StatisticsLancaster UniversityLancasterUK

Personalised recommendations