Skip to main content
SpringerLink
Log in

Maximum likelihood estimation of nonlinear structural equation models

  • Articles
  • Published:
Psychometrika Aims and scope Submit manuscript

Abstract

The existing maximum likelihood theory and its computer software in structural equation modeling are established based on linear relationships among manifest variables and latent variables. However, models with nonlinear relationships are often encountered in social and behavioral sciences. In this article, an EM type algorithm is developed for maximum likelihood estimation of a general nonlinear structural equation model. To avoid computation of the complicated multiple integrals involved, the E-step is completed by a Metropolis-Hastings algorithm. It is shown that the M-step can be completed efficiently by simple conditional maximization. Standard errors of the maximum likelihood estimates are obtained via Louis's formula. The methodology is illustrated with results from a simulation study and two real examples.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Anderson, T.W. (1989). Linear latent variable models and covariance structures.Journal of Econometrics, 41, 91–119.

    Google Scholar 

  • Arminger, G., & Muthén, B.O. (1998). A Bayesian approach to nonlinear latent variable models using the Gibbs sampler and the Metropolis-Hastings algorithm.Psychometrika, 63, 271–300.

    Google Scholar 

  • Bagozzi, R.P., Baumgartner, H., & Yi, Y. (1992). State versus action orientation and the theory of reasoned action: An application to coupon usage.Journal of Consumer Research, 18, 505–517.

    Google Scholar 

  • Bentler, P.M. (1983). Some contributions to efficient statistics for structural models: Specification and estimation of moment structures.Psychometrika, 48, 493–517.

    Google Scholar 

  • Bentler, P.M. (1992).EQS: Structural equation program manual. Los Angeles, CA: BMDP Statistical Software.

    Google Scholar 

  • Bentler, P.M., & Dudgeon, P. (1996). Covariance structure analysis: Statistical practice, theory, and directions.Annual Review of Psychology, 47, 541–570.

    Google Scholar 

  • Berger, J.O., & Perrichi, L.R. (1996). The intrinsic Bayes factor for model selection and prediction.Journal of the American Statistical Association, 91, 109–122.

    Google Scholar 

  • Bollen, K.A., & Paxton, P. (1998). Two-stage least squares estimation of interaction effects. In R.E. Schumacker & G.A. Marcoulides Eds.),Interaction and nonlinear effects in structural equation models (pp. 125–151). Mahwah, NJ: Lawrence Erlbaum Associates.

    Google Scholar 

  • Booth, J.G., & Hobert, J.P. (1999). Maximizing generalized linear mixed model likelihoods with an automated Monte Carlo EM algorithm.Journal of the Royal Statistical Society, Series B, 61, 265–285.

    Google Scholar 

  • Browne, M.W. (1984). Asymptotically distribution-free methods in the analysis of covariance structures.British Journal of Mathematical and Statistical Psychology, 37, 62–83.

    Google Scholar 

  • Browne, M.W. (1987). Robustness of statistical inference in factor analysis and related models.Biometrika, 74, 375–384.

    Google Scholar 

  • Busemeyer, J.R., & Jones, L.E. (1983). Analysis of multiplicative combination rules when the causal variables are measured with error.Psychological Bulletin, 93, 549–562.

    Google Scholar 

  • Celeux, G., & Diebolt, J. (1989). The SEM algorithm: A probabilistic teacher algorithm derived from the EM algorithm for the mixture problem.Computational Statistics Quarterly, 2, 73–82.

    Google Scholar 

  • Dempster, A.P., Laird, N.M., & Rubin, D.B. (1977). Maximum likelihood from incomplete data via the EM algorithm (with discussion).Journal of the Royal Statistical Society, Series B, 39, 1–38.

    Google Scholar 

  • Etezadi-Amoli, J., & McDonald, R.P. (1983). A second generation nonlinear factor analysis.Psychometrika, 48, 315–342.

    Google Scholar 

  • Fraser, C. (1980).COSAN user's guide. Toronto, Canada: The Ontario Institute for Studies in Education.

    Google Scholar 

  • Gelman, A., & Meng, X.L. (1998). Simulating normalizing constants: From importance sampling to bridge sampling to path sampling.Statistical Science, 13, 163–185.

    Google Scholar 

  • Gelman, A., Roberts, G.O., & Gilks, W.R. (1995). Efficient Metropolis humping rules. In J.M. Bernardo, J.O. Berger, A.P. Dawid, & A.F.M. Smith (Eds.),Bayesian statistics 5 (pp. 599–607). Oxford, England: Oxford University Press.

    Google Scholar 

  • Geman, S., & Geman, D. (1984). Stochastic relaxation, Gibbs distribution, and the Bayesian restoration of images.IEEE Transactions on Pattern Analysis and Machine Intelligence, 6, 721–741.

    Google Scholar 

  • Hastings, W.K. (1970). Monte Carlo sampling methods using Markov chains and their application.Biometrika, 57, 97–109.

    Google Scholar 

  • Hu, L., Bentler, P.M., & Kano, Y. (1992). Can test statistics in covariance structure analysis be trusted.Psychological Bulletin, 112, 351–362.

    Google Scholar 

  • Jaccard, J., & Wan, C.K. (1995). Measurement error in the analysis of interaction effects between continuous predictors using multiple regression: Multiple indicator and structural equation approaches.Psychological Bulletin, 117, 348–357.

    Google Scholar 

  • Jamshidian, M., & Jennrich, R.I. (1993). Conjugate gradient acceleration of the EM algorithm.Journal of the American Statistical Association, 88, 221–228.

    Google Scholar 

  • Jonsson, F.Y. (1998). Modeling interaction and non-linear effects: A step by step LISREL example. In R.E. Schumacker & G.A. Marcoulides (Eds.),Interaction and nonlinear effects in structural equation models (pp. 17–42). Mahwah, NJ: Lawrence Erlbaum Associates.

    Google Scholar 

  • Jöreskog, K.G., & Sörbom, D. (1996).LISREL 8: Structural equation modeling with the SIMPLIS command language. Hove and London, England: Scientific Software International.

    Google Scholar 

  • Jöreskog, K.G., & Yang, F. (1996). Nonlinear structural equation models: The Kenny-Judd model with interaction effects. In G.A. Marcoulides & R.E. Schumacker (Eds.),Advanced structural equation modeling techniques (pp. 57–88). Hillsdale, NJ: Lawrence Erlbaum Associates.

    Google Scholar 

  • Kass, R.E., & Raftery, A.E. (1995). Bayes Factors.Journal of the American Statistical Association, 90, 773–795.

    Google Scholar 

  • Kenny, D.A., & Judd, C.M. (1984). Estimating the nonlinear and interactive effects of latent variables.Psychological Bulletin, 96, 201–210.

    Google Scholar 

  • Klein, A., Moosbrugger, H., Schermelleh-Engel, K., & Frandk, D (1997). A new approach to the estimation of latent interaction effects in structural equation models. In W. Bandilla & F. Fanlbaum (Eds.),SOFTSTAT '97—Advances in statistical software (pp. 479–488). Stuttgart, Germany: Lucius & Lucius.

    Google Scholar 

  • Lange, K. (1995). A gradient algorithm locally equivalent to the EM algorithm.Journal of the Royal Statistical Association, Series B, 57, 425–437.

    Google Scholar 

  • Lee, S.Y., Poon, W.Y., & Bentler, P.M. (1995). A two-stage estimation of structural equation models with continuous and polytomous variables.British Journal of Mathematical and Statistical Psychology, 48, 339–358.

    Google Scholar 

  • Lee, S.Y., & Song, X.Y. (2001). Hypothesis testing and model comparison in two-level structural equation models.Multivariate Behavioral Research, 36, 639–655.

    Google Scholar 

  • Lee, S.Y., & Tsang, S.Y. (1999). Constrained maximum likelihood estimation of two-level covariance structure model via EM type algorithms.Psychometrika, 64, 435–450.

    Google Scholar 

  • Lee, S.Y., & Zhu, H.T. (2000). Statistical analysis of nonlinear structural equation model with continuous and polytomous data.British Journal of Mathematical and Statistical Psychology, 53, 209–232.

    Google Scholar 

  • Li, F., Harmer, P., Duncan, T.E., Duncan, S.C., Acock, A., & Boles, S. (1998). Approaches to testing interaction effects using structural equation modeling methodology.Multivariate Behavioral Research, 33, 1–39.

    Google Scholar 

  • Liu, C., & Rubin, D.B. (1998). Maximum likelihood estimation of factor analysis using the ECME algorithm with complete and incomplete data.Statistica Sinica, 8, 729–747.

    Google Scholar 

  • Liu, J.S., Liang, F.M., & Wong, W.H. (2000). The use of multiple-try method and local optimization in metropolis sampling.Journal of the American Statistical Association, 95, 121–134.

    Google Scholar 

  • Louis, T.A. (1982). Finding the observed information matrix when using EM algorithm.Journal of the Royal Statistical Society, Series B, 44, 226–233.

    Google Scholar 

  • Marschner, I.C. (2001). On stochastic version of the EM algorithm.Biometrika, 88, 281–286.

    Google Scholar 

  • McDonald, R.P. (1962). A general approach to nonlinear factor analysis.Psychometrika, 27, 123–157.

    Google Scholar 

  • McDonald, R.P. (1967a). Numerical methods for polynomial models in nonlinear factor analysis.Psychometrika, 32, 77–112.

    Google Scholar 

  • McDonald, R.P. (1967b). Factor interaction in nonlinear factor analysis.British Journal of Mathematical and Statistical Psychology, 20, 205–215.

    Google Scholar 

  • Meng, X.L., & Rubin, D.B. (1993). Maximum likelihood estimation via the ECM algorithm: A general framework.Biometrika, 80, 267–278.

    Google Scholar 

  • Meng, X.L., & Schilling, S. (1996). Fitting full-information item factor models and an empirical investigation of bridge sampling.Journal of the American Statistical Association, 91, 1254–1267.

    Google Scholar 

  • Meng, X.L., & van Dyk, D. (1997). The EM algorithm—An old folk song sung to a fast new tune (with discussion).Journal of the Royal Statistical Society, Series B, 59, 511–567.

    Google Scholar 

  • Meng, X.L., & Wong, W.H. (1996). Simulating ratios of normalizing constants via a simple identity: A theoretical exploration.Statistic Sinica, 6, 831–860.

    Google Scholar 

  • Metropolis, N., Rosenbluth, A.W., Rosenbluth, M.N., Teller, A.H., & Teller, E. (1953). Equations of state calculations by fast computing machine.Journal of Chemical Physics, 21, 1087–1091.

    Google Scholar 

  • Mooijaart, A., & Bentler, P. (1986). Random polynomial factor analysis. In E. Diday, M. Jambu, L. Lebart, J. Pages, & R. Tomassone (Eds.),Data analysis and informatics, IV (pp. 241–250). North-Holland: Elsevier Science Publishers.

    Google Scholar 

  • Ping, R.A. (1996a). Interaction and quadratic effect estimation: A two step technique using structural equation analysis.Psychological Bulletin, 119, 166–175.

    Google Scholar 

  • Ping, R.A. (1996b). Latent variable regression: A technique for estimating interaction and quadratic coefficients.Multivariate Behavioral Research, 31, 95–120.

    Google Scholar 

  • Ping, R.A. (1996c). Estimating latent variable interactions and quadratics: The state of this art.Journal of Management, 22, 163–183.

    Google Scholar 

  • Raftery, A.E. (1993). Bayesian model selection in structural equation models. In K.A. Bollen & J.S. Long (Eds.),Testing structural equation models (pp. 163–180). Beverly Hills, CA: Sage.

    Google Scholar 

  • Roberts, G.O. (1996). Markov Chain concepts related to sampling algorithms. In W.R. Gilks, S. Richardson, & D.J. Spiegelhalter (Eds.),Markov chain Monte Carlo in practice (pp. 45–57). London, England: Chapman and Hall.

    Google Scholar 

  • Rubin, D.B. (1991). EM and beyond.Psychometrika, 56, 241–254.

    Google Scholar 

  • Rubin, D.B., & Thayer, D.T. (1982). EM algorithm for ML factor analysis.Psychometrika, 47, 69–76.

    Google Scholar 

  • Schumacker, R.E., & Marcoulides, G.A. (Eds.). (1998).Interaction and nonlinear effects in structural equation models. Mahwah, NJ: Lawrence Erlbaum Associates.

    Google Scholar 

  • Shi, J.Q., & Lee, S.Y. (1998). Bayesian sampling-based approach for factor analysis model with continuous and polytomous data.British Journal of Mathematical and Statistical Psychology, 51, 233–252.

    Google Scholar 

  • Shi, J.Q., & Lee, S.Y. (2000). Latent variable models with mixed continuous and polytomous data.Journal of the Royal Statistical Society, Series B, 62, 77–87.

    Google Scholar 

  • Wei, G.C.G., & Tanner, M.A. (1990). A Monte Carlo implementation of the EM algorithm and the Poor man's data augmentation algorithm.Journal of the American Statistical Association, 85, 699–704.

    Google Scholar 

  • World Values Survey: 1981–1984 & 1990–1993. (1994). Ann Arbor, MI: Inter-University Consortium of Political and Social Research. (For the I.C.P.S.R. version the Institute for Social Research is the producer, and the Inter-University Consortium of Political and Social Research is the distributor.)

Download references

Author information

Authors and Affiliations

  1. Department of Statistics, The Chinese University of Hong Kong, Shatin, HONG KONG

    Sik-Yum Lee

  2. Department of Mathematics and Statistics, University of Victoria, USA

    Hong-Tu Zhu

Authors
  1. Sik-Yum Lee
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hong-Tu Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sik-Yum Lee.

Additional information

The order of the authorship is alphabetical. This research is fully supported by a Hong Kong UGC Earmarked grant CUHK 4088/99H. The authors are indebted to the Editor, Associate Editor and anonymous reviewers for valuable comments for improving the paper; and also to ICPSR and the relevant funding agency for allowing use of the data. We thank Xin-Yuan Song, N.H. Tang and Liang Xu for helpful discussions. Assistance of Xin-Yuan Song and Michael K. H. Leung in analyzing the real examples and Esther L.S. Tam in preparing the manuscript is also acknowledged.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Lee, SY., Zhu, HT. Maximum likelihood estimation of nonlinear structural equation models. Psychometrika 67, 189–210 (2002). https://doi.org/10.1007/BF02294842

Download citation

  • Received:

  • Revised:

  • Issue Date:

  • DOI: https://doi.org/10.1007/BF02294842

Key words

Search

Navigation