Identification of Confirmatory Factor Analysis Models of Different Levels of Invariance for Ordered Categorical Outcomes

Abstract

This article considers the identification conditions of confirmatory factor analysis (CFA) models for ordered categorical outcomes with invariance of different types of parameters across groups. The current practice of invariance testing is to first identify a model with only configural invariance and then test the invariance of parameters based on this identified baseline model. This approach is not optimal because different identification conditions on this baseline model identify the scales of latent continuous responses in different ways. Once an invariance condition is imposed on a parameter, these identification conditions may become restrictions and define statistically non-equivalent models, leading to different conclusions. By analyzing the transformation that leaves the model-implied probabilities of response patterns unchanged, we give identification conditions for models with invariance of different types of parameters without referring to a specific parametrization of the baseline model. Tests based on this approach have the advantage that they do not depend on the specific identification condition chosen for the baseline model.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Notes

  1. 1.

    This includes the same number of factors, the same loading patterns, and that all nonzero loadings are positive. This assumption is generally satisfied for confirmatory analyses.

  2. 2.

    Many estimation methods in factor analysis do not assume normality, but because they only fit the covariance and mean structures, the identification conditions mentioned here are still valid.

  3. 3.

    It is usually good enough to identify a subset of the parameter space as long as its complement has lower dimensions.

  4. 4.

    Millsap and Yun-Tein (2004) used slightly different notations: the thresholds are \(\nu \) (we use \(\tau \)), the intercepts are \(\tau \) (we use \(\nu \)), groups are indexed in subscript by k (we use superscript (g)), thresholds indexed as \(m=0, 1, \ldots , c+1\) (we use \(k=0, 1, \ldots , K+1\)).

  5. 5.

    We choose ML because it produces a \(\chi ^2\) distributed statistic if regularity conditions are met.

References

  1. Babakus, E., Ferguson, C. E., & Jöreskog, K. G. (1987). The sensitivity of confirmatory maximum likelihood factor analysis to violations of measurement scale and distributional assumptions. Journal of Marketing Research, 24(2), 222–228.

    Article  Google Scholar 

  2. Baker, F. B. (1992). Item response theory parameter estimation techniques. New York: Marcel Dekker.

    Google Scholar 

  3. Bernstein, I. H., & Teng, G. (1989). Factoring items and factoring scales are different: Spurious evidence for multidimensionality due to item categorization. Psychological Bulletin, 105, 467–477.

    Article  Google Scholar 

  4. Boker, S., Neale, M., Maes, H., Wilde, M., Spiegel, M., Brick, T., et al. (2011). OpenMx: An open source extended structural equation modeling framework. Psychometrika, 76, 306–317.

    Article  PubMed  PubMed Central  Google Scholar 

  5. Carey, G. (2005). Cholesky problems. Behavioral Genetics, 35, 653–665.

    Article  Google Scholar 

  6. Cheung, G. W., & Lau, R. S. (2012). A direct comparison approach for testing measurement invariance. Organizational Research Methods, 15(2), 167–198.

    Article  Google Scholar 

  7. Cheung, G. W., & Rensvold, R. (1998). Cross cultural comparisons using non-invariant measurement items. Applied Behavioral Science Review, 6, 93–110.

    Article  Google Scholar 

  8. Cheung, G. W., & Rensvold, R. (1999). Testing factorial invariance across groups: A reconceptualization and proposed new method. Journal of Management, 25, 1–27.

    Article  Google Scholar 

  9. Christoffersson, A. (1975). Factor analysis of dichotomized variables. Psychometrika, 40, 5–32.

    Article  Google Scholar 

  10. Davies, R. B. (1977). Hypothesis testing when a nuisance parameter is present only under the alternative. Biometrika, 64, 247–254.

    Article  Google Scholar 

  11. Davies, R. B. (1987). Hypothesis testing when a nuisance parameter is present only under the alternative. Biometrika, 74, 33–43.

    Google Scholar 

  12. Drton, M. (2009). Likelihood ratio tests and singularities. The Annals of Statistics, 37(2), 979–1012.

    Article  Google Scholar 

  13. Estabrook, R. (2012). Factorial invariance: Tools and concepts for strengthening research. In G. Tenenbaum, R. Eklund, & A. Kamata (Eds.), Measurement in sport and exercise psychology. Champaign, IL: Human Kinetics.

    Google Scholar 

  14. Jeffries, N. O. (2003). A note on ’Testing the number of components in a normal mixture’. Biometrika, 90(4), 991–994.

    Article  Google Scholar 

  15. Jöreskog, K. G., & Moustaki, I. (2001). Factor analysis of ordinal variables: A comparison of three approaches. Multivariate Behaviorial Research, 36, 347–387.

    Article  Google Scholar 

  16. Lubke, G. H., & Muthén, B. O. (2004). Applying multiple group confirmatory factor models for continuous outcomes to Likert scale data complicates meaningful group comparisons. Structural Equation Modeling, 11(4), 514–534.

    Article  Google Scholar 

  17. Mehta, P. D., Neale, M. C., & Flay, B. R. (2004). Squeezing interval change from ordinal panel data: Latent growth curves with ordinal outcomes. Psychological Methods, 9(3), 301–333.

    Article  PubMed  Google Scholar 

  18. Meredith, W. (1964a). Notes on factorial invariance. Psychometrika, 29, 177–185.

    Article  Google Scholar 

  19. Meredith, W. (1964b). Rotation to achieve factorial invariance. Psychometrika, 29, 186–206.

    Google Scholar 

  20. Meredith, W. (1993). Measurement invariance, factor analysis and factor invariance. Psychometrika, 58, 525–543.

    Article  Google Scholar 

  21. Millsap, R. E., & Meredith, W. (2007). Factorial invariance: Historical perspectives and new problems. In R. Cudeck & R. C. MacCallum (Eds.), Factor analysis at 100: Historical developments and future directions (pp. 131–152). Mahwah, NJ: Lawrence Erlbaum Associates.

    Google Scholar 

  22. Millsap, R. E., & Yun-Tein, J. (2004). Assessing factorial invariance in ordered categorical measures. Multivariate Behavioral Research, 39(3), 479–515.

    Article  Google Scholar 

  23. Mislevy, R. J. (1986). Recent developments in the factor analysis of categorical variables. Journal of Educational Statistics, 11, 3–31.

    Article  Google Scholar 

  24. Muthén, B., & Christofferson, A. (1981). Simultaneous factor analysis of dichotomous variables in several groups. Psychometrika, 46, 407–419.

    Article  Google Scholar 

  25. Muthén, B. O. (1984). A general structural equation model for dichotomous, ordered categorical and continuous latent variable indicators. Psychometrika, 49, 115–132.

    Article  Google Scholar 

  26. Muthén, B. O., & Kaplan, D. (1985). A comparison of some methodologies for the factor analysis of non-normal Likert variables. British Journal of Mathematical and Statistical Psychology, 38, 171–189.

  27. Muthén, L. K., & Muthén, B. O. (1998–2012). Mplus Users Guide (7th ed.). Los Angeles, CA: Muthén & Muthén.

  28. Neale, M. C., Hunter, M. D., Pritkin, J., Zahery, M., Brick, T. R., Kirkpatrick, R. M., et al. (2016). OpenMx 2.0: Extended structural equation and statistical modeling. Psychometrika, 81(2), 535–549.

    Article  PubMed  Google Scholar 

  29. Oort, F. J. (1998). Simulation study of item bias detection with restricted factor analysis. Structural Equation Modeling, 5, 107–124.

    Article  Google Scholar 

  30. R Development Core Team. (2013). R: A language and environment for statistical computing. http://www.R-project.org.

  31. Rensvold, R. B., & Cheung, G. W. (2001). Testing for metric invariance using structural equation models, solving the standardization problem. Research in Management, 1, 25–50.

    Google Scholar 

  32. Strom, D. O., & Lee, J. W. (1994). Variance components testing in the longitudinal mixed effects model. Biometrics, 50, 1171–1177 (Corrected in. Biometrics, 51, 1196.)

  33. van der Linden, W. J. and Barrett, M. D. (2015). Linking item response model parameters. Psychometrika. doi:10.1007/s11336-015-9469-6.

  34. Vandenberg, R. J., & Lance, C. E. (2000). A review and synthesis of the measurement invariance literature: Suggestions, practices, and recommendations for organizational research. Organizational Research Methods, 3(1), 4–69.

    Article  Google Scholar 

  35. Widaman, K. F., & Reise, S. P. (1997). Exploring the measurement invariance of psychological instruments: Applications in the substance use domain. In K. J. Bryant, M. Windle, & S. G. West (Eds.), The science of prevention: Methodological advances from alcohol and substance abuse research (pp. 281–324). Washington, DC: American Psychological Association.

    Google Scholar 

  36. Wu, H., & Neale, M. C. (2013). On the likelihood ratio tests in bivariate ACDE models. Psychometrika, 78(3), 441–463.

    Article  PubMed  Google Scholar 

  37. Wu, H. (accepted) A note on the identifiability of fixed effect 3PL models. Psychometrika.

  38. Yoon, M., & Millsap, R. E. (2007). Detecting violations of factorial invariance using data-based specification searches: A Monte-Carlo study. Structural Equation Modeling, 14(3), 435–463.

    Article  Google Scholar 

Download references

Acknowledgments

Work on this research by the second author was partially supported by the National Institute of Drug Abuse research education program R25DA026-119 (Director: Michael C. Neale) and by grant R01 AG18436 (20112016, Director: Daniel K. Mroczek) from National Institute on Aging, National Institute on Mental Health.

Author information

Affiliations

Authors

Corresponding author

Correspondence to Hao Wu.

Additional information

Roger Millsap, whose work inspired this paper, unexpectedly passed away when we were preparing this manuscript. We would like to honor him for his pioneering work in measurement invariance.

Appendix 1: Mplus and OpenMx Codes

Appendix 1: Mplus and OpenMx Codes

Model Specification in Mplus

figurea

Model Specification in OpenMx

figureb

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Wu, H., Estabrook, R. Identification of Confirmatory Factor Analysis Models of Different Levels of Invariance for Ordered Categorical Outcomes. Psychometrika 81, 1014–1045 (2016). https://doi.org/10.1007/s11336-016-9506-0

Download citation

Keywords

  • ordered categorical data
  • invariance testing
  • model identification