, Volume 64, Issue 4, pp 435–450 | Cite as

Constrained maximum likelihood estimation of two-level covariance structure model via EM type algorithms

  • Sik-Yum Lee
  • Sin-Yu Tsang


In this paper, the constrained maximum likelihood estimation of a two-level covariance structure model with unbalanced designs is considered. The two-level model is reformulated as a single-level model by treating the group level latent random vectors as hypothetical missing-data. Then, the popular EM algorithm is extended to obtain the constrained maximum likelihood estimates. For general nonlinear constraints, the multiplier method is used at theM-step to find the constrained minimum of the conditional expectation. An accelerated EM gradient procedure is derived to handle linear constraints. The empirical performance of the proposed EM type algorithms is illustrated by some artifical and real examples.

Key words

Two-level covariance structure model missing data EM algorithm multiplier method scoring iteration linear and nonlinear constraints 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Bentler, P.M. (1992).EQS Structural Equations Program Manual. Los Angeles: BMDP Statistical Software.Google Scholar
  2. Bentler, P.M., & Weeks, D.G. (1980). Linear structural equations with latent variables,Psychometrika, 45, 289–308.Google Scholar
  3. Bertsekas, D.P. (1976). Multiplier method: A survey.Automatica, 12, 133–145.Google Scholar
  4. Dempster, A.P., Laird, N., & Rubin, D.B. (1977). Maximum likelihood from incomplete data via the EM algorithm.Journal of the Royal Statistical Society, Series B, 39, 1–38.Google Scholar
  5. Dempster, A.P., Rubin, D.B., & Tsutakawa, J.R. (1981). Estimation in covariance components models.Journal of the American Statistical Association, 76, 341–353.Google Scholar
  6. Goldstein, H., & McDonald, R.P. (1988). A general model for the analysis of multilevel data.Psychometrika, 53, 455–467.Google Scholar
  7. Jamshidian, M., & Jennrich, R.I. (1993). Conjugate gradient acceleration of the EM algorithm.Journal of the American Statistical Association, 88, 1459–1471.Google Scholar
  8. Jöreskog, K.G., & Sörbom, D. (1996).LISREL 8: Structural equation modeling with the SIMPLIS command language. Hove and London: Scientific Software International.Google Scholar
  9. Kim, D.G., & Taylor, J.M.G. (1995). The restricted EM algorithm for maximum likelihood estimation under linear restrictions on the parameters.Journal of the American Statistical Association, 90, 708–716.Google Scholar
  10. Lange, K. (1995). A gradient algorithm locally equivalent to the EM algorithm.Journal of the Royal Statistical Society, Series B, 57, 425–437.Google Scholar
  11. Lee, S.Y. (1990). Multilevel analysis of structural equation models.Biometrika, 77, 763–772.Google Scholar
  12. Lee, S.Y., & Bentler, P.M. (1980). Some asymptotic properties of constrained generalized least squares estimation in covariance structure models.South African Statistical Journal, 14, 121–136.Google Scholar
  13. Lee, S.Y., & Poon, W.Y. (1992). Two level analysis of covariance structures for unbalanced design with small level one samples.British Journal of Mathematical and Statistical Psychology, 45, 109–123.Google Scholar
  14. Lee, S.Y., & Poon, W.Y. (1998). Analysis of two-level structural equation models via EM type algorithms.Statistica Sinica, 8, 749–766.Google Scholar
  15. Lee, S.Y., & Tsui, K.L. (1982). Covariance structure analysis in several populations.Psychometrika, 47, 297–308.Google Scholar
  16. Lindley, D.V. & Smith, A.F.M. (1972). Bayes estimation for the linear model.Journal of the Royal Statistical Society, Series B, 34, 1–42.Google Scholar
  17. Longford, N.T. & Muthén, B. (1992). Factor analysis for clustered observations.Psychometrika, 57, 581–597.Google Scholar
  18. Louis, T.A. (1982). Finding the observed informationmatrix when using the EM algorithm.Journal of the Royal Statistical Society, Series B, 44, 226–232.Google Scholar
  19. McDonald, R.P., & Goldstein, H. (1989). Balanced versus unbalanced designs for linear structural relations in two-level data.British Journal of Mathematical and Statistical Psychology, 42, 215–232.Google Scholar
  20. Meng Xiao-Li, & Rubin, D.B. (1991). Using EM to obtain asymptotic variance-covariance matrices: The SEM algorithm.Journal of the American Statistical Association, 86, 899–909.Google Scholar
  21. Meng Xiao-Li, & Van Dyk, D. (1997). The EM Algorithm-an old folk-song sung to a fast new tune. (with discussion)Journal of the Royal Statistical Society, Series B, 59, 511–567.Google Scholar
  22. Morisky, D.E., Tiglao, T.V., Sneed, C.D., Tempongko, S.B., Baltazar, J.C., Detels, R., & Stein, J.A. (1998). The effects of establishment practices, knowledge and attitudes on condom use among Filipina sex workers.AIDS Care, 10, 213–220.Google Scholar
  23. Muthén, B. (1989). Latent variable modeling in heterogeneous populations.Psychometrika, 54, 557–556.Google Scholar
  24. Raudenbush, S.W. (1995). Maximum likelihood estimation for unbalanced multilevel covariance structure models via the EM algorithm.British Journal of Mathematical and Statistical Psychology, 48, 359–370.Google Scholar
  25. Rubin, D.B. (1991). EM and beyond.Psychometrika, 56, 241–254.Google Scholar

Copyright information

© The Psychometric Society 1999

Authors and Affiliations

  • Sik-Yum Lee
    • 1
  • Sin-Yu Tsang
    • 1
  1. 1.Department of StatisticsChinese University of Hong Kong

Personalised recommendations