Two-Step Estimation of Models Between Latent Classes and External Variables

Abstract

We consider models which combine latent class measurement models for categorical latent variables with structural regression models for the relationships between the latent classes and observed explanatory and response variables. We propose a two-step method of estimating such models. In its first step, the measurement model is estimated alone, and in the second step the parameters of this measurement model are held fixed when the structural model is estimated. Simulation studies and applied examples suggest that the two-step method is an attractive alternative to existing one-step and three-step methods. We derive estimated standard errors for the two-step estimates of the structural model which account for the uncertainty from both steps of the estimation, and show how the method can be implemented in existing software for latent variable modelling.

This is a preview of subscription content, access via your institution.

Fig. 1

References

  1. Anderson, J. C., & Gerbing, D. W. (1988). Structural equation modeling in practice: A review and recommended two-step approach. Psychological Bulletin, 103, 411–423.

    Article  Google Scholar 

  2. Asparouhov, T., & Muthén, B. (2014). Auxiliary variables in mixture modeling: Three-step approaches using Mplus. Structural Equation Modeling, 21, 329–341.

    Article  Google Scholar 

  3. Asparouhov, T., & Muthén, B. (2015). Auxiliary variables in mixture modeling: Using the BCH method in Mplus to estimate a distal outcome model and an arbitrary secondary model (Mplus Web Notes No. 21).

  4. Bakk, Z., Oberski, D., & Vermunt, J. (2014a). Relating latent class assignments to external variables: Standard errors for correct inference. Political Analysis, 22, 520–540.

    Article  Google Scholar 

  5. Bakk, Z., Oberski, D. L., & Vermunt, J. K. (2014b). Replication data for: Relating latent class assignments to external variables: Standard errors for correct inference. Harvard Dataverse. Retrieved from https://doi.org/10.7910/DVN/24497.

  6. Bakk, Z., Tekle, F. T., & Vermunt, J. K. (2013). Estimating the association between latent class membership and external variables using bias-adjusted three-step approaches. Sociological Methodology, 43, 272–311.

    Article  Google Scholar 

  7. Bakk, Z., & Vermunt, J. K. (2016). Robustness of stepwise latent class modeling with continuous distal outcomes. Structural Equation Modeling, 23, 20–31.

    Article  Google Scholar 

  8. Bandeen-Roche, K., Miglioretti, D. L., Zeger, S. L., & Rathouz, P. J. (1997). Latent variable regression for multiple discrete outcomes. Journal of the American Statistical Association, 92, 1375–1386.

    Article  Google Scholar 

  9. Bartolucci, F. , Montanari, G. E., & Pandolfi, S. (2014). A comparison of some estimation methods for latent Markov models with covariates. In Proceedings of COMPSTAT 2014—21st International Conference on Computational Statistics (pp. 531–538).

  10. Bolck, A., Croon, M., & Hagenaars, J. (2004). Estimating latent structure models with categorical variables: One-step versus three-step estimators. Political Analysis, 12, 3–27.

    Article  Google Scholar 

  11. Bollen, K. A. (1989). Structural equations with latent variables. New York: Wiley.

    Book  Google Scholar 

  12. Bollen, K. A. (1996). An alternative two stage least squares (2SLS) estimator for latent variable equations. Psychometrika, 61, 109–121.

    Article  Google Scholar 

  13. Burt, R. S. (1973). Confirmatory factor-analytic structures and the theory construction process. Sociological Methods & Research, 2, 131–190.

    Article  Google Scholar 

  14. Burt, R. S. (1976). Interpretational confounding of unobserved variables in structural equation models. Sociological Methods & Research, 5, 3–52.

    Article  Google Scholar 

  15. Cameron, A. C., & Trivedi, P. K. (2005). Microeconometrics: Methods and applications. Cambridge: Cambridge University Press.

    Book  Google Scholar 

  16. Chan, T. W., & Goldthorpe, J. H. (2007). European social stratification and cultural consumption: Music in England. Sociological Review, 23, 11–19.

    Google Scholar 

  17. Clogg, C. C. (1981). New developments in latent structure analysis. In D. J. Jackson & E. F. Borgotta (Eds.), Factor analysis and measurement in sociological research. London: Sage.

    Google Scholar 

  18. Croon, M. (2002). Using predicted latent scores in general latent structure models. In G. A. Marcoulides & I. Moustaki (Eds.), Latent variable and latent structure models (pp. 195–223). Mahwah, NJ: Lawrence Erlbaum.

    Google Scholar 

  19. Dayton, C. M., & Macready, G. B. (1988). Concomitant-variable latent class models. Journal of the American Statistical Association, 83, 173–178.

    Article  Google Scholar 

  20. De Cuyper, N., Rigotti, T., Witte, H. D., & Mohr, G. (2008). Balancing psychological contracts: Validation of a typology. International Journal of Human Resource Management, 19, 543–561.

    Article  Google Scholar 

  21. Dempster, A. P., Laird, N. W., & Rubin, D. B. (1977). Maximum likelihood from incomplete data via the EM algorithm (with discussion). Journal of the Royal Statistical Society B, 39, 1–38.

    Google Scholar 

  22. Devlieger, I., Mayer, A., & Rosseel, Y. (2016). Hypothesis testing using factor score regression: A comparison of four methods. Educational and Psychological Measurement, 76, 741–770.

    Article  Google Scholar 

  23. De Witte, H. (2000). Arbeidsethos en jobonzekerheid: Meting en gevolgen voor welzijn, tevredenheid en inzet op het werk [Work ethic and job insecurity: Measurement and consequences for well-being, satisfaction and performance]. In R. Bouwen, K. De Witte, H. De Witte, & T. Taillieu (Eds.), Van groep naar gemeenschap. Liber Amicorum Prof. Dr. Leo Lagrou (pp. 325–350). Leuven: Garant.

    Google Scholar 

  24. Dias, J. G., & Vermunt, J. K. (2008). A bootstrap-based aggregate classifier for model-based clustering. Computational Statistics, 23, 643–59.

    Article  Google Scholar 

  25. Einarsen, S., Hoel, H., & Notelaers, G. (2009). Measuring exposure to bullying and harassment at work: Validity, factor structure and psychometric properties of the Negative Acts Questionnaire—Revised. Work & Stress, 23, 24–44.

    Article  Google Scholar 

  26. Gong, G., & Samaniego, F. J. (1981). Pseudo maximum likelihood estimation: Theory and applications. The Annals of Statistics, 76, 861–889.

    Article  Google Scholar 

  27. Goodman, L. A. (1974). The analysis of systems of qualitative variables when some of the variables are unobservable. Part I: A modified latent structure approach. American Journal of Sociology, 79, 1179–1259.

    Article  Google Scholar 

  28. Gourieroux, C., & Monfort, A. (1995). Statistics and econometric models (Vol. 2). Cambridge: Cambridge University Press.

    Google Scholar 

  29. Haberman, S. (1979). Analysis of qualitative data. vol. 2: New developments. New York: Academic Press.

    Google Scholar 

  30. Hagenaars, J. A. (1993). Loglinear models with latent variables. Newbury Park, CA: Sage.

    Book  Google Scholar 

  31. Jöreskog, K. G., & Sörbom, D. (1986). LISREL VI: Analysis of linear structural relationships by maximum likelihood and least squares methods. Mooresville, IN: Scientific Software Inc.

    Google Scholar 

  32. Kam, J. A. (2011). Identifying changes in youth’s subgroup membership over time based on their targeted communication about substance use with parents and friends. Human Communication Research, 37, 324–349.

    Article  Google Scholar 

  33. Lance, C. E., Cornwell, J. M., & Mulaik, S. A. (1988). Limited information parameter estimates for latent or mixed manifest and latent variable models. Multivariate Behavioral Research, 23, 171–187.

    Article  Google Scholar 

  34. Lanza, T. S., Tan, X., & Bray, C. B. (2013). Latent class analysis with distal outcomes: A flexible model-based approach. Structural Equation Modeling, 20(1), 1–26.

    Article  Google Scholar 

  35. Lazarsfeld, P. F., & Henry, N. W. (1968). Latent structure analysis latent structure analysis. Boston: Houghton-Mifflin.

    Google Scholar 

  36. Lu, I. R. R., & Thomas, D. R. (2008). Avoiding and correcting bias in score-based latent variable regression with discrete manifest items. Structural Equation Modeling, 15, 462–490.

    Article  Google Scholar 

  37. Magidson, J. (1981). Qualitative variance, entropy, and correlation ratios for nominal dependent variables. Social Science Research, 10, 177–194.

    Article  Google Scholar 

  38. McCutcheon, A. L. (1985). A latent class analysis of tolerance for nonconformity in the American public. Public Opinion Quarterly, 494, 474–488.

    Article  Google Scholar 

  39. McCutcheon, A. L. (1987). Latent class analysis. Newbury Park, CA: Sage.

    Book  Google Scholar 

  40. Muthén, L. K., & Muthén, B. O. (2017). Mplus user’s guide [Computer software manual] (8th ed.). Los Angeles, CA: Muthen and Muthen.

    Google Scholar 

  41. Petersen, J., Bandeen-Roche, K., Budtz-Jørgensen, E., & Groes Larsen, K. (2012). Predicting latent class scores for subsequent analysis. Psychometrika, 77, 244–262.

    Article  Google Scholar 

  42. Ping, R. A. (1996). Latent variable interaction and quadratic effect estimation: A two-step technique using structural equation analysis. Psychological Bulletin, 119, 166–175.

    Article  Google Scholar 

  43. PSYCONES. (2006). Psychological contracts across employment situations, final report. DG Research, European Commission. Retrieved from http://cordis.europa.eu/documents/documentlibrary/100123961EN6.pdf

  44. R Core Team. (2016). R: A language and environment for statistical computing [Computer software manual]. Vienna, Austria. https://www.R-project.org.

  45. Skrondal, A., & Kuha, J. (2012). Improved regression calibration. Psychometrika, 77, 649–669.

    Article  Google Scholar 

  46. Skrondal, A., & Laake, P. (2001). Regression among factor scores. Psychometrika, 66, 563–576.

    Article  Google Scholar 

  47. Vermunt, J. K. (2010). Latent class modeling with covariates: Two improved three-step approaches. Political Analysis, 18, 450–469.

    Article  Google Scholar 

  48. Vermunt, J. K., & Magidson, J. (2005). Latent GOLD 4.0 user’s guide. Belmont, MA: Statistical Innovations.

    Google Scholar 

  49. Vermunt, J. K., & Magidson, J. (2016). Technical guide for Latent GOLD 5.1: Basic, Advanced and Syntax. Belmont, MA: Statistical Innovations.

    Google Scholar 

  50. Xue, Q. L., & Bandeen-Roche, K. (2002). Combining complete multivariate outcomes with incomplete covariate information: A latent class approach. Biometrics, 58, 110–120.

    Article  Google Scholar 

Download references

Author information

Affiliations

Authors

Corresponding author

Correspondence to Zsuzsa Bakk.

Electronic supplementary material

Below is the link to the electronic supplementary material.

11336_2017_9592_MOESM1_ESM.zip

Supplementary material 1 (zip 81 KB)

Appendix: Score Functions and Information Matrices for Latent Class Models

Appendix: Score Functions and Information Matrices for Latent Class Models

Consider first in general terms a model which involves a latent class variable X with a total of C latent classes (here X may also represent all combinations of the classes of several latent class variables). Suppose that the model depends on parameters \(\varvec{\theta }\) (in this Appendix, we take all vectors to be column vectors, the opposite of the practice in Sect. 2 where they were row vectors for simplicity of notation). The log likelihood contribution for a single unit i is then \(l_{i}= \log L_{i} = \log \sum _{c=1}^{C} L_{ic}\), where \(L_{ic}=\exp (l_{ic})\) is the term in \(L_{i}\) which refers to latent class \(c=1,\ldots ,C\). The contribution of unit i to the score function is then \(\mathbf {u}_{i}=\partial l_{i}/\partial \varvec{\theta }=\mathbf {h}_{i}/L_{i}\) where \(\mathbf {h}_{i} = \partial L_{i}/\partial \varvec{\theta }=\sum _{c} L_{ic} \mathbf {u}_{ic}\) and \(\mathbf {u}_{ic}=\partial l_{ic}/\partial \varvec{\theta }\), and the contribution to the observed information matrix is

$$\begin{aligned} \mathbf {J}_{i}=-\frac{\partial ^{2} l_{i}}{\partial \varvec{\theta } \partial \varvec{\theta }'} = -\frac{1}{L_{i}^{2}} \, \left\{ L_{i} \, \left[ \sum _{c} L_{ic} \left( \mathbf {u}_{ic}\mathbf {u}_{ic}'-\mathbf {J}_{ic} \right) \right] - \mathbf {h}_{i}\mathbf {h}_{i}' \right\} , \end{aligned}$$

where \(\mathbf {J}_{ic}=-\,\partial ^{2} l_{ic}/(\partial \varvec{\theta } \partial \varvec{\theta }')\).

Suppose that observations for different units \(i=1,\ldots ,n\) are independent. Point estimation of \(\varvec{\theta }\) is easiest with the EM algorithm (Dempster, Laird, & Rubin, 1977). For this, let \(l_{i}^{*}\) be the same expression as \(l_{ic}\) but now regarded as a function of c. At the E step of the \((t+1)\)th iteration of EM, we calculate \(Q^{(t+1)}=\sum _{i} \text {E}[l^{*}_{i}|\mathbf {D},\varvec{\theta }^{(t)}]= \sum _{i}\left( \sum _{c} \pi _{ic}^{(t)} l_{ic}\right) \) where \(\pi _{ic}^{(t)}=p(X_{i}=c|\mathbf {D},\varvec{\theta }^{(t)})\), \(\mathbf {D}\) denotes all the observed data, and \(\varvec{\theta }^{(t)}\) is the estimate of \(\varvec{\theta }\) from the tth iteration. At the M step, \(Q^{(t+1)}\) is maximized with respect to \(\varvec{\theta }\) to produce an updated estimate \(\varvec{\theta }^{(t+1)}\). This is relatively straightforward because \(\partial Q^{(t+1)}/\partial \varvec{\theta }= \sum _{i}\sum _{c} \left( \pi _{ic}^{(t)} \mathbf {u}_{ic}\right) \) and \(-\,\partial ^{2} Q^{(t+1)}/\partial \varvec{\theta }\partial \varvec{\theta }'= \sum _{i}\sum _{c} \left( \pi _{ic}^{(t)} \mathbf {J}_{ic}\right) \), i.e. these are the score function and observed information matrix for a model where X is known, fitted to pseudo-data of \(n\times C\) observations with fractional weights \(\pi _{ic}^{(t)}\).

The information matrix \(\varvec{\mathcal {I}}\) for the model can be estimated by \(n^{-1}\sum _{i} \mathbf {J}_{i}\) or \(n^{-1}\sum _{i} \mathbf {u}_{i}\mathbf {u}_{i}'\). Together with \(\mathbf {u}_{i}\), these could also be used to implement other estimation algorithms than EM. When evaluated at the final estimate of \(\varvec{\theta }\), they give estimates of the \(\varvec{\mathcal{I}}_{22}\) and \(\varvec{\mathcal {I}}_{12}\) which are needed for the two-step variance matrix (5). An estimate of the \(\mathbf {\Sigma }_{11}\) that is also needed there is obtained similarly from the estimated information matrix for step 1 latent class model.

What then remains to be done for any specific model is to evaluate \(l_{ic}\), \(\mathbf {u}_{ic}\) and (if used) \(\mathbf {J}_{ic}\) for it. As an example, consider the model with covariates \(\mathbf {Z}_{p}\), one latent class variable X, and a response variable \(Z_{o}\) which is considered in Sect. 2.1. The \(L_{i}\) for it is given by (2) with \(p(\mathbf {Z}_{pi})\) omitted. Then \(l_{ic}=\log p(X_{i}=c|\mathbf {Z}_{pi}) + \log p(Z_{oi}|\mathbf {Z}_{pi},X_{i}=c) + \sum _{k} \log p(Y_{ik}|X_{i}=c)\) \(\equiv l^{(x)}_{ic} + l^{(z)}_{ic} + \sum _{k} l^{(yk)}_{ic}\). If the parameters for the different components of the model are distinct, which should be the case for most sensible models, we only need to consider the separate derivatives of the terms in this sum. Suppose that \(X_{i}\) given \(\mathbf {Z}_{pi}\) is given by the multinomial logistic model (3), writing it now as \(p(X_{i}=c\vert \mathbf Z _{pi})\equiv \pi ^{(x)}_{ic}= \exp (\varvec{\alpha }_{c}'\mathbf {Z}^{*}_{i})/ \sum \limits _{s=1}^{C}\exp (\varvec{\alpha }_{s}'\mathbf {Z}^{*}_{i})\) for \(c=1,\ldots ,C\), with \(\varvec{\alpha }_{1}=\mathbf {0}\), \(\varvec{\alpha }_{c}=(\beta _{0c},\varvec{\beta }_{c}')'\) for \(c\ne 1\), and \(\mathbf {Z}_{i}^{*}=(1,\mathbf {Z}_{pi}')'\). Then \(l^{(x)}_{ic}=\log \pi ^{(x)}_{ic}\), \(\partial l^{(x)}_{ic}/\partial \varvec{\alpha }_{r}=(I(r=c)-\pi ^{(x)}_{ir})\mathbf {Z}_{i}^{*}\), and \(\partial ^{2} l^{(x)}_{ic}/\partial \varvec{\alpha }_{r}\partial \varvec{\alpha }'_{s}=-\,(I(s=r)\pi ^{(x)}_{ir}-\pi ^{(x)}_{ir}\pi ^{(x)}_{is})\mathbf {Z}_{i}^{*}\mathbf {Z}_{i}^{*\prime }\) for \(r,s=2,\ldots ,C\). The measurement models for the items \(Y_{ik}\) can also be formulated as multinomial logistic models, by writing them as \(p(Y_{ik}=r|X_{i}=c)\equiv \pi ^{(yk)}_{icr}= \exp (\varvec{\gamma }_{kr}'\mathbf {X}_{ic}^{*})/ \sum _{s=1}^{R_{k}}\exp (\varvec{\gamma }_{ks}'\mathbf {X}_{ic}^{*})\) for \(r=1,\ldots ,R_{k}\), with \(\mathbf {X}_{ic}^{*}=(I(c=1),\ldots ,I(c=C))'\) and \(\varvec{\gamma }_{k1}=\mathbf {0}\). Then \(l^{(yk)}_{ic}= \sum _{r=1}^{R_{k}} I(Y_{ik}=r)\log \pi ^{(yk)}_{icr}\), and if the parameters for different items k are also distinct, the terms in \(\sum _{k} l^{(yk)}_{ic}\) can be differentiated separately. Their derivatives are \(\partial l^{(yk)}_{ic}/\partial \varvec{\gamma }_{r}= (I(Y_{ik}=r)-\pi ^{(yk)}_{icr})\mathbf {X}_{ic}^{*}\) and \(\partial ^{2} l^{(yk)}_{ic}/\partial \varvec{\gamma }_{r}\partial \varvec{\gamma }'_{s}=-\,(I(s=r)\pi ^{(yk)}_{icr}-\pi ^{(yk)}_{icr}\pi ^{(yk)}_{ics})\mathbf {X}_{ic}^{*}\mathbf {X}_{ic}^{*\prime }\) for \(r,s=2,\ldots ,R_{k}\). For \(l^{(z)}_{ic}\), suppose, for example, that \(Z_{oi}\) is normally distributed with mean \(\mu _{i}=\varvec{\delta }'\mathbf {Z}_{i}^{**}\) and variance \(\tau ^{-1}\), where \(\mathbf {Z}_{i}^{**}=(\mathbf {X}_{ic}',\mathbf {Z}_{pi}')'\). Defining \(e_{i}=Z_{oi}-\mu _{i}\), then \(l^{(z)}_{ic}=(\log \tau -\tau e_{i}^{2})/2\), \(\partial l^{(z)}_{ic}/\partial \varvec{\delta }=\tau e_{i}\mathbf {Z}_{i}^{**}\), \(\partial l^{(z)}_{ic}/\partial \tau =(1/\tau -e_{i}^{2})/2\), \(\partial ^{2} l^{(z)}_{ic}/\partial \varvec{\delta } \partial \varvec{\delta }'=-\tau (\mathbf {Z}_{i}^{**})(\mathbf {Z}_{i}^{**})'\), \(\partial ^{2} l^{(z)}_{ic}/\partial ^{2} \tau =-1/(2\tau ^{2})\), and \(\partial ^{2} l^{(z)}_{ic}/\partial \varvec{\delta }\partial \tau =e_{i}\mathbf {Z}_{i}^{**}\). The formulas for the situations considered in our simulations and examples are obtained from these results by setting \(\mathbf {Z}_{i}^{*}=1\) for the case with no \(Z_{pi}\), and omitting \(l^{(z)}_{ic}\) for the case with no \(Z_{oi}\). Finally, doing both of these things gives the formulas for the basic latent class model which is estimated in step 1 of the two-step method.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Bakk, Z., Kuha, J. Two-Step Estimation of Models Between Latent Classes and External Variables. Psychometrika 83, 871–892 (2018). https://doi.org/10.1007/s11336-017-9592-7

Download citation

Keywords

  • latent variables
  • mixture models
  • structural equation models
  • pseudo-maximum likelihood estimation