Abstract
A maximum likelihood estimation routine for two-level structural equation models with random slopes for latent covariates is presented. Because the likelihood function does not typically have a closed-form solution, numerical integration over the random effects is required. The routine relies upon a method proposed by du Toit and Cudeck (Psychometrika 74(1):65–82, 2009) for reformulating the likelihood function so that an often large subset of the random effects can be integrated analytically, reducing the computational burden of high-dimensional numerical integration. The method is demonstrated and assessed using a small-scale simulation study and an empirical example.
Similar content being viewed by others
References
Asparouhov, T., & Muthén, B. (2019a). Latent variable centering of predictors and mediators in multilevel and time-series models. Structural Equation Modeling: A Multidisciplinary Journal, 26(1), 119–142.
Asparouhov, T., & Muthén, B. (2019b). Latent variable interactions using maximum-likelihood and Bayesian estimation for single- and two-level models. Mplus Web Notes: No. 23.
Bentler, P. M. (2004). Eqs 6: Structural equations program manual. Encino: Multivariate software.
Bollen, K. A. (1989). Structural equations with latent variables. Hoboken: Wiley.
Cai, L. (2010). A two-tier full-information item factor analysis model with applications. Psychometrika, 75(4), 581–612.
Carpenter, B., Hoffman, M. D., Brubaker, M., Lee, D., Li, P., & Betancourt, M. (2015). The stan math library: Reverse-mode automatic differentiation in c++. arXiv preprint arXiv:1509.07164.
Cronbach, L. J. (1976). Research on classrooms and schools: Formulation of questions, design and analysis. Stanford, CA: Stanford University Evaluation Consortium.
Cudeck, R. (2005). Fitting psychometric models with methods based on automatic differentiation. Psychometrika, 70(4), 599–617.
Cudeck, R., Harring, J. R., & du Toit, S. H. (2009). Marginal maximum likelihood estimation of a latent variable model with interaction. Journal of Educational and Behavioral Statistics, 34(1), 131–144.
du Toit, S. H., & Cudeck, R. (2009). Estimation of the nonlinear random coefficient model when some random effects are separable. Psychometrika, 74(1), 65–82.
du Toit, S. H., & du Toit, M. (2008). Multilevel structural equation modeling. In Handbook of multilevel analysis (pp. 435–478). Springer.
Eddelbuettel, D. (2013). Seamless R and C++ integration with Rcpp. New York: Springer. ISBN 978-1-4614-6867-7. https://doi.org/10.1007/978-1-4614-6868-4.
Enders, C. K., & Tofighi, D. (2007). Centering predictor variables in cross-sectional multilevel models: A new look at an old issue. Psychological Methods, 12(2), 121–138.
Gibbons, R. D., & Hedeker, D. R. (1992). Full-information item bi-factor analysis. Psychometrika, 57(3), 423–436.
Goldstein, H., & McDonald, R. P. (1988). A general model for the analysis of multilevel data. Psychometrika, 53(4), 455–467.
Griewank, A., & Walther, A. (2008). Evaluating derivatives: Principles and techniques of algorithmic differentiation (Vol. 105). Philadelphia: SIAM.
Hallquist, M. N., & Wiley, J. F. (2018). MplusAutomation: An R package for facilitating large-scale latent variable analyses in Mplus. Structural Equation Modeling: A Multidisciplinary Journal, 25(4), 621–638.
Jöreskog, K. G., & Sörbom, D. (1996). Lisrel 8: User’sreference guide. Lincolnwood: Scientific Software International.
Lee, S.-Y. (1990). Multilevel analysis of structural equation models. Biometrika, 77(4), 763–772.
Liang, J., & Bentler, P. M. (2004). An EM algorithm for fitting two-level structural equation models. Psychometrika, 69(1), 101–122.
Lüdtke, O., Marsh, H. W., Robitzsch, A., Trautwein, U., Asparouhov, T., & Muthén, B. (2008). The multilevel latent covariate model: A new, more reliable approach to group-level effects in contextual studies. Psychological Methods, 13(3), 203–229.
Maas, C. J., & Hox, J. J. (2005). Sufficient sample sizes for multilevel modeling. Methodology, 1(3), 86–92.
Marsh, H. W., Lüdtke, O., Nagengast, B., Trautwein, U., Morin, A. J., Abduljabbar, A. S., et al. (2012). Classroom climate and contextual effects: Conceptual and methodological issues in the evaluation of group-level effects. Educational Psychologist, 47(2), 106–124.
McDonald, R. P. (1993). A general model for two-level data with responses missing at random. Psychometrika, 58(4), 575–585.
McDonald, R. P., & Goldstein, H. (1989). Balanced versus unbalanced designs for linear structural relations in two-level data. British Journal of Mathematical and Statistical Psychology, 42(2), 215–232.
Mehta, P. D., & Neale, M. C. (2005). People are variables too: Multilevel structural equations modeling. Psychological Methods, 10(3), 259–284.
Muthén, B. (1984). A general structural equation model with dichotomous, ordered categorical, and continuous latent variable indicators. Psychometrika, 49(1), 115–132.
Muthén, B. O. (1989). Latent variable modeling in heterogeneous populations. Psychometrika, 54(4), 557–585.
Muthén, B. O., & Satorra, A. (1989). Multilevel aspects of varying parameters in structural models. In Multilevel analysis of educational data (pp. 87–99). Elsevier.
Muthén, L. K., & Muthén, B. (2017). Mplus version 8 [Computer software manual]. Los Angeles, CA: Muthén & Muthén.
Nash, J. C. (2014). On best practice optimization methods in R. Journal of Statistical Software, 60(2), 1–14.
Neale, M. C., Hunter, M. D., Pritikin, J. N., Zahery, M., Brick, T. R., Kirkpatrick, R. M., et al. (2016). OpenMx 2.0: Extended structural equation and statistical modeling. Psychometrika, 81(2), 535–549.
OECD. (2003). Programme for International Student Assessment 2003. Retrieved February 1, 2019 from http://www.oecd.org/pisa/data/database-pisa2003.htm.
Pinheiro, J. C., & Bates, D. M. (1995). Approximations to the log-likelihood function in the nonlinear mixed-effects model. Journal of Computational and Graphical Statistics, 4(1), 12–35.
Preacher, K. J., Zhang, Z., & Zyphur, M. J. (2016). Multilevel structural equation models for assessing moderation within and across levels of analysis. Psychological Methods, 21(2), 189–205.
Preacher, K. J., Zyphur, M. J., & Zhang, Z. (2010). A general multilevel SEM framework for assessing multilevel mediation. Psychological Methods, 15(3), 209–233.
R Core Team. (2019). R: A language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing. https://urldefense.proofpoint.com/v2/url?u=https-3A__www.r2Dproject.org&d=DwIGaQ&c=vh6FgFnduejNhPPD0fl_yRaSfZy8CWbWnIf4XJhSqx8&r=cijxKIUfIjh6xB35XSxKelnSNfz2185wGO_qFr-DFH8&m=2tlyHFkIA11Yzt64XP7lrKUV1F_N4EqYjSlNwvys8zE&s=yIxHEeyUO9whFYdd4zujlILP0486_iGL1mIlAxlRsc&e=.
Rabe-Hesketh, S., Skrondal, A., & Pickles, A. (2002). Reliable estimation of generalized linear mixed models using adaptive quadrature. The Stata Journal, 2(1), 1–21.
Rabe-Hesketh, S., Skrondal, A., & Pickles, A. (2004). Generalized multilevel structural equation modeling. Psychometrika, 69(2), 167–190. https://doi.org/10.1007/bf02295939.
Rabe-Hesketh, S., Skrondal, A., & Pickles, A. (2005). Maximum likelihood estimation of limited and discrete dependent variable models with nested random effects. Journal of Econometrics, 128(2), 301–323.
Rijmen, F. (2009). An efficient EM algorithm for multidimensional IRT models: Full information maximum likelihood estimation in limited time (Tech. Rep.). Princeton, NJ: ETS Research Report (RR0903).
Rijmen, F. (2010). Formal relations and an empirical comparison among the bi-factor, the testlet, and a second-order multidimensional irt model. Journal of Educational Measurement, 47(3), 361–372.
Rosseel, Y. (2012). Lavaan: An R package for structural equation modeling and more. version 0.5–12 (BETA). Journal of Statistical Software, 48(2), 1–36.
Schmidt, W. H. (1969). Covariance structure analysis of the multivariate random effects model (Unpublished doctoral dissertation). Department of Education: University of Chicago.
Shin, Y., & Raudenbush, S. W. (2010). A latent cluster-mean approach to the contextual effects model with missing data. Journal of Educational and Behavioral Statistics, 35(1), 26–53.
Stan Development Team. (2020). RStan: The R interface to Stan. R package version 2.19.3. https://urldefense.proofpoint.com/v2/url?u=https-3A__mc-2Dstan.org&d=DwIGaQ&c=vh6FgFnduejNhPPD0fl_yRaSfZy8CWbWnIf4XJhSqx8&r=cijxKIUfIjh6xB35XSxKelnSNfz2185wGO_qFr-DFH8&m=2tlyHFkIA11Yzt64XP7lrKUV1F_N4EqYjSlNwvys8zE&s=w3Alv-F1vraydtUj39bGIhlI9AU2loF1hvQZLAAB16w&e=.
Stapleton, L. M., & Johnson, T. L. (2019). Models to examine the validity of cluster-level factor structure using individual-level data. Advances in Methods and Practices in Psychological Science, 2(3), 312–329.
Stapleton, L. M., Yang, J. S., & Hancock, G. R. (2016). Construct meaning in multilevel settings. Journal of Educational and Behavioral Statistics, 41(5), 481–520.
StataCorp. (2005). Stata statistical software: Release 15. College Station, TX: StataCorp LLC.
Acknowledgments
I thank the editor, associate editors, and reviewers, as well as Drs. Andrew Hayes, Paul De Boeck, Jolynn Pek, and Robert Cudeck for helpful comments and discussions that led to the improvement of this manuscript. A portion of this research was conducted at The Ohio State University.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Appendix
Appendix
In this section, methods for adapting the estimation routine to allow for data missing at random are sketched. As in Sect. 2, suppose there are k level-2 variables \({\mathbf {z}}_j\) and p level-1 variables \({\mathbf {y}}_{ij}\). However, now consider that one or more elements within these vectors for a given j or ij may be missing. Suppose cluster j has \(k_j\) non-missing elements of \({\mathbf {z}}_j\) and individual i in cluster j has \(p_{ij}\) non-missing elements in \({\mathbf {y}}_{ij}\).
Define \({\mathbf {K}}_j\) (\(k_j \times k\)) and \({\mathbf {M}}_{ij}\) (\(p_{ij} \times p\)) to be zero-one matrices that select the non-missing elements of \({\mathbf {z}}_{j}\) and \({\mathbf {y}}_{ij}\), respectively. For example, suppose k is 4 and cluster \(j'\) is missing the third element of \({\mathbf {z}}_{j'}\), so that
can be used to select the non-missing subset of \({\mathbf {z}}_{j'}\):
The matrix \({\mathbf {M}}_{ij}\) performs the same role as \({\mathbf {K}}_j\), except it is used to select non-missing elements of \({\mathbf {y}}_{ij}\) rather than \({\mathbf {z}}_j\). Thus, \({\mathbf {z}}_j^* = {\mathbf {K}}_{j}{\mathbf {z}}_{j}\) will be used in place of \({\mathbf {z}}_j\) and \({\mathbf {y}}_j^* = {\mathbf {M}}_{ij}{\mathbf {y}}_{ij}\) will be used in place of \({\mathbf {y}}_{ij}\).
By premultiplying some of the other model matrices within the likelihood calculation by \({\mathbf {K}}_j\) or \({\mathbf {M}}_{ij}\), the estimation routine can be adapted to account for the missing elements within \({\mathbf {z}}_j\) and \({\mathbf {y}}_{ij}\). Specifically, for Eqs. (24)–(25), replace \({\varvec{\mu }}_{{\mathbf {z}}_j}\) and \({\varvec{\mu }}_{{\mathbf {y}}_{ij}}\) with \({\mathbf {K}}_j{\varvec{\mu }}_{{\mathbf {z}}_j}\) and \({\mathbf {M}}_{ij}{\varvec{\mu }}_{{\mathbf {y}}_{ij}}\), respectively. Within Eqs. (26)–(30) replace \(\tilde{{\mathbf {G}}}\) and \(\tilde{{\mathbf {Q}}}^*_{ij}\) with \({\mathbf {K}}_j\tilde{{\mathbf {G}}}\) and \({\mathbf {M}}_{ij}\tilde{{\mathbf {Q}}}^*_{ij}\), and replace \({\varvec{\Sigma }}_{W}^*\) with \({\varvec{\Sigma }}_{Wij}^* = {\mathbf {M}}_{ij}{\varvec{\Sigma }}_{W}^*{\mathbf {M}}_{ij}'\). Lastly, replace \({\mathbf {I}}_{n_j} \otimes {\varvec{\Sigma }}_W^*\) in Eq. (26) with
where \(\oplus \) is the direct sum. Using these replacements, the simplified expressions for \(|{\varvec{\Sigma }}_{{\mathbf {d}}_j}|\) and \({\varvec{\epsilon }}_{{\mathbf {d}}_j}'{\varvec{\Sigma }}_{{\mathbf {d}}_j}^{-1}{\varvec{\epsilon }}_{{\mathbf {d}}_j}\) in the new conditional log-likelihood
are
and
Rights and permissions
About this article
Cite this article
Rockwood, N.J. Maximum Likelihood Estimation of Multilevel Structural Equation Models with Random Slopes for Latent Covariates. Psychometrika 85, 275–300 (2020). https://doi.org/10.1007/s11336-020-09702-9
Received:
Revised:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11336-020-09702-9