Skip to main content

Identification of Confirmatory Factor Analysis Models of Different Levels of Invariance for Ordered Categorical Outcomes

Psychometrika volume 81, pages 1014–1045 (2016)Cite this article

Abstract

This article considers the identification conditions of confirmatory factor analysis (CFA) models for ordered categorical outcomes with invariance of different types of parameters across groups. The current practice of invariance testing is to first identify a model with only configural invariance and then test the invariance of parameters based on this identified baseline model. This approach is not optimal because different identification conditions on this baseline model identify the scales of latent continuous responses in different ways. Once an invariance condition is imposed on a parameter, these identification conditions may become restrictions and define statistically non-equivalent models, leading to different conclusions. By analyzing the transformation that leaves the model-implied probabilities of response patterns unchanged, we give identification conditions for models with invariance of different types of parameters without referring to a specific parametrization of the baseline model. Tests based on this approach have the advantage that they do not depend on the specific identification condition chosen for the baseline model.

This is a preview of subscription content, access via your institution.

Access options

Buy single article

Instant access to the full article PDF.

43,34 €

Price includes VAT (Finland)
Tax calculation will be finalised during checkout.

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Notes

  1. This includes the same number of factors, the same loading patterns, and that all nonzero loadings are positive. This assumption is generally satisfied for confirmatory analyses.

  2. Many estimation methods in factor analysis do not assume normality, but because they only fit the covariance and mean structures, the identification conditions mentioned here are still valid.

  3. It is usually good enough to identify a subset of the parameter space as long as its complement has lower dimensions.

  4. Millsap and Yun-Tein (2004) used slightly different notations: the thresholds are \(\nu \) (we use \(\tau \)), the intercepts are \(\tau \) (we use \(\nu \)), groups are indexed in subscript by k (we use superscript (g)), thresholds indexed as \(m=0, 1, \ldots , c+1\) (we use \(k=0, 1, \ldots , K+1\)).

  5. We choose ML because it produces a \(\chi ^2\) distributed statistic if regularity conditions are met.

References

  • Babakus, E., Ferguson, C. E., & Jöreskog, K. G. (1987). The sensitivity of confirmatory maximum likelihood factor analysis to violations of measurement scale and distributional assumptions. Journal of Marketing Research, 24(2), 222–228.

    Article  Google Scholar 

  • Baker, F. B. (1992). Item response theory parameter estimation techniques. New York: Marcel Dekker.

    Google Scholar 

  • Bernstein, I. H., & Teng, G. (1989). Factoring items and factoring scales are different: Spurious evidence for multidimensionality due to item categorization. Psychological Bulletin, 105, 467–477.

    Article  Google Scholar 

  • Boker, S., Neale, M., Maes, H., Wilde, M., Spiegel, M., Brick, T., et al. (2011). OpenMx: An open source extended structural equation modeling framework. Psychometrika, 76, 306–317.

    Article  PubMed  PubMed Central  Google Scholar 

  • Carey, G. (2005). Cholesky problems. Behavioral Genetics, 35, 653–665.

    Article  Google Scholar 

  • Cheung, G. W., & Lau, R. S. (2012). A direct comparison approach for testing measurement invariance. Organizational Research Methods, 15(2), 167–198.

    Article  Google Scholar 

  • Cheung, G. W., & Rensvold, R. (1998). Cross cultural comparisons using non-invariant measurement items. Applied Behavioral Science Review, 6, 93–110.

    Article  Google Scholar 

  • Cheung, G. W., & Rensvold, R. (1999). Testing factorial invariance across groups: A reconceptualization and proposed new method. Journal of Management, 25, 1–27.

    Article  Google Scholar 

  • Christoffersson, A. (1975). Factor analysis of dichotomized variables. Psychometrika, 40, 5–32.

    Article  Google Scholar 

  • Davies, R. B. (1977). Hypothesis testing when a nuisance parameter is present only under the alternative. Biometrika, 64, 247–254.

    Article  Google Scholar 

  • Davies, R. B. (1987). Hypothesis testing when a nuisance parameter is present only under the alternative. Biometrika, 74, 33–43.

    Google Scholar 

  • Drton, M. (2009). Likelihood ratio tests and singularities. The Annals of Statistics, 37(2), 979–1012.

    Article  Google Scholar 

  • Estabrook, R. (2012). Factorial invariance: Tools and concepts for strengthening research. In G. Tenenbaum, R. Eklund, & A. Kamata (Eds.), Measurement in sport and exercise psychology. Champaign, IL: Human Kinetics.

    Google Scholar 

  • Jeffries, N. O. (2003). A note on ’Testing the number of components in a normal mixture’. Biometrika, 90(4), 991–994.

    Article  Google Scholar 

  • Jöreskog, K. G., & Moustaki, I. (2001). Factor analysis of ordinal variables: A comparison of three approaches. Multivariate Behaviorial Research, 36, 347–387.

    Article  Google Scholar 

  • Lubke, G. H., & Muthén, B. O. (2004). Applying multiple group confirmatory factor models for continuous outcomes to Likert scale data complicates meaningful group comparisons. Structural Equation Modeling, 11(4), 514–534.

    Article  Google Scholar 

  • Mehta, P. D., Neale, M. C., & Flay, B. R. (2004). Squeezing interval change from ordinal panel data: Latent growth curves with ordinal outcomes. Psychological Methods, 9(3), 301–333.

    Article  PubMed  Google Scholar 

  • Meredith, W. (1964a). Notes on factorial invariance. Psychometrika, 29, 177–185.

    Article  Google Scholar 

  • Meredith, W. (1964b). Rotation to achieve factorial invariance. Psychometrika, 29, 186–206.

    Google Scholar 

  • Meredith, W. (1993). Measurement invariance, factor analysis and factor invariance. Psychometrika, 58, 525–543.

    Article  Google Scholar 

  • Millsap, R. E., & Meredith, W. (2007). Factorial invariance: Historical perspectives and new problems. In R. Cudeck & R. C. MacCallum (Eds.), Factor analysis at 100: Historical developments and future directions (pp. 131–152). Mahwah, NJ: Lawrence Erlbaum Associates.

    Google Scholar 

  • Millsap, R. E., & Yun-Tein, J. (2004). Assessing factorial invariance in ordered categorical measures. Multivariate Behavioral Research, 39(3), 479–515.

    Article  Google Scholar 

  • Mislevy, R. J. (1986). Recent developments in the factor analysis of categorical variables. Journal of Educational Statistics, 11, 3–31.

    Article  Google Scholar 

  • Muthén, B., & Christofferson, A. (1981). Simultaneous factor analysis of dichotomous variables in several groups. Psychometrika, 46, 407–419.

    Article  Google Scholar 

  • Muthén, B. O. (1984). A general structural equation model for dichotomous, ordered categorical and continuous latent variable indicators. Psychometrika, 49, 115–132.

    Article  Google Scholar 

  • Muthén, B. O., & Kaplan, D. (1985). A comparison of some methodologies for the factor analysis of non-normal Likert variables. British Journal of Mathematical and Statistical Psychology, 38, 171–189.

  • Muthén, L. K., & Muthén, B. O. (1998–2012). Mplus Users Guide (7th ed.). Los Angeles, CA: Muthén & Muthén.

  • Neale, M. C., Hunter, M. D., Pritkin, J., Zahery, M., Brick, T. R., Kirkpatrick, R. M., et al. (2016). OpenMx 2.0: Extended structural equation and statistical modeling. Psychometrika, 81(2), 535–549.

    Article  PubMed  Google Scholar 

  • Oort, F. J. (1998). Simulation study of item bias detection with restricted factor analysis. Structural Equation Modeling, 5, 107–124.

    Article  Google Scholar 

  • R Development Core Team. (2013). R: A language and environment for statistical computing. http://www.R-project.org.

  • Rensvold, R. B., & Cheung, G. W. (2001). Testing for metric invariance using structural equation models, solving the standardization problem. Research in Management, 1, 25–50.

    Google Scholar 

  • Strom, D. O., & Lee, J. W. (1994). Variance components testing in the longitudinal mixed effects model. Biometrics, 50, 1171–1177 (Corrected in. Biometrics, 51, 1196.)

  • van der Linden, W. J. and Barrett, M. D. (2015). Linking item response model parameters. Psychometrika. doi:10.1007/s11336-015-9469-6.

  • Vandenberg, R. J., & Lance, C. E. (2000). A review and synthesis of the measurement invariance literature: Suggestions, practices, and recommendations for organizational research. Organizational Research Methods, 3(1), 4–69.

    Article  Google Scholar 

  • Widaman, K. F., & Reise, S. P. (1997). Exploring the measurement invariance of psychological instruments: Applications in the substance use domain. In K. J. Bryant, M. Windle, & S. G. West (Eds.), The science of prevention: Methodological advances from alcohol and substance abuse research (pp. 281–324). Washington, DC: American Psychological Association.

    Chapter  Google Scholar 

  • Wu, H., & Neale, M. C. (2013). On the likelihood ratio tests in bivariate ACDE models. Psychometrika, 78(3), 441–463.

    Article  PubMed  Google Scholar 

  • Wu, H. (accepted) A note on the identifiability of fixed effect 3PL models. Psychometrika.

  • Yoon, M., & Millsap, R. E. (2007). Detecting violations of factorial invariance using data-based specification searches: A Monte-Carlo study. Structural Equation Modeling, 14(3), 435–463.

    Article  Google Scholar 

Download references

Acknowledgments

Work on this research by the second author was partially supported by the National Institute of Drug Abuse research education program R25DA026-119 (Director: Michael C. Neale) and by grant R01 AG18436 (20112016, Director: Daniel K. Mroczek) from National Institute on Aging, National Institute on Mental Health.

Author information

Affiliations

  1. Boston College, Chestnut Hill, USA

    Hao Wu

  2. Northwestern University, Evanston, USA

    Ryne Estabrook

Authors
  1. Hao Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ryne Estabrook
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hao Wu.

Additional information

Roger Millsap, whose work inspired this paper, unexpectedly passed away when we were preparing this manuscript. We would like to honor him for his pioneering work in measurement invariance.

Appendix 1: Mplus and OpenMx Codes

Appendix 1: Mplus and OpenMx Codes

Model Specification in Mplus

figure a

Model Specification in OpenMx

figure b

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Wu, H., Estabrook, R. Identification of Confirmatory Factor Analysis Models of Different Levels of Invariance for Ordered Categorical Outcomes. Psychometrika 81, 1014–1045 (2016). https://doi.org/10.1007/s11336-016-9506-0

Download citation

  • Received:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11336-016-9506-0

Keywords

  • ordered categorical data
  • invariance testing
  • model identification