Abstract
In multigroup factor analysis, configural measurement invariance is accepted as tenable when researchers either (a) fail to reject the null hypothesis of exact fit using a χ2 test or (b) conclude that a model fits approximately well enough, according to one or more alternative fit indices (AFIs). These criteria fail for two reasons. First, the test of perfect fit confounds model fit with group equivalence, so rejecting the null hypothesis of perfect fit does not imply that the null hypothesis of configural invariance should be rejected. Second, treating common rules of thumb as critical values for judging approximate fit yields inconsistent results across conditions because fixed cutoffs ignore sampling variability of AFIs. As a solution, we propose replacing χ2 and fixed AFI cutoffs with permutation tests. Iterative permutation of group assignment yields an empirical distribution of any fit measure under the null hypothesis of invariance. Simulations show the permutation test of configural invariance controls Type I error rates better than χ2 or AFIs when a model has parsimony error (i.e., negligible misspecification) but the factor structure is equivalent across groups (i.e., the null hypothesis is true).
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
A notable exception is RMSEA. See an excellent discussion by Kenny et al. (2015).
- 2.
Population-level AFIs can be obtained by fitting the analysis model to the population moments or can be estimated from the average AFI across Monte Carlo samples.
- 3.
The exchangeability assumption might be violated for natural groups (Hayes 1996), which we bring up in the Discussion.
References
P.M. Bentler, Comparative fit indexes in structural models. Psychol. Bull. 107(2), 238–246 (1990). doi:10.1037/0033-2909.107.2.238
P.M. Bentler, D.G. Bonett, Significance tests and goodness of fit in the analysis of covariance structures. Psychol. Bull. 88(3), 588–606 (1980). doi:10.1037/0033-2909.88.3.588
M.W. Browne, R. Cudeck, Alternative ways of assessing model fit. Sociol. Methods Res. 21, 230–258 (1992). doi:10.1177/0049124192021002005
B.M. Byrne, R.J. Shavelson, B. Muthén, Testing for the equivalence of factor co-variance and mean structures: The issue of partial measurement invariance. Psychol. Bull. 105(3), 456–466 (1989). doi:10.1037/0033-2909.105.3.456
R. Cudeck, S.J. Henly, Model selection in covariance structures analysis and the “problem” of sample size: a clarification. Psychol. Bull. 109(3), 512–519 (1991). doi:10.1037//0033-2909.109.3.512
A.F. Hayes, Permutation test is not distribution-free: Testing H0: ρ = 0. Psychol. Methods 1(2), 184–198 (1996). doi:10.1037/1082-989X.1.2.184
L.-T. Hu, P.M. Bentler, Cutoff criteria for fit indexes in covariance structure analysis: conventional criteria versus new alternatives. Struct. Equ. Model. 6(1), 1–55 (1999). doi:10.1080/10705519909540118
D.A. Kenny, B. Kaniskan, D.B. McCoach, The performance of RMSEA in models with small degrees of freedom. Sociol. Methods Res. 44(3), 486–507 (2015). doi:10.1177/0049124114543236
R.C. MacCallum, 2001 presidential address: working with imperfect models. Multivar. Behav. Res. 38(1), 113–139 (2003). doi:10.1207/S15327906MBR3801_5
R.C. MacCallum, M.W. Browne, H.M. Sugawara, Power analysis and determination of sample size for covariance structure modeling. Psychol. Methods 1(2), 130–149 (1996). doi:10.1037//1082-989X.1.2.130
H.W. Marsh, K.-T. Hau, Z. Wen, In search of golden rules: comment on hypothesis-testing approaches to setting cutoff values for fit indexes and dangers in overgeneralizing Hu and Bentler’s (1999) findings. Struct. Equ. Model. 11(3), 320–341 (2004). doi:10.1207/s15328007sem1103_2
A.W. Meade, E.C. Johnson, P.W. Braddy, Power and sensitivity of alternative fit indices in tests of measurement invariance. J. Appl. Psychol. 93(3), 568–592 (2008). doi:10.1037/0021-9010.93.3.568
J. Nevitt, G.R. Hancock, Evaluating small sample approaches for model test statistics in structural equation modeling. Multivar. Behav. Res. 39(3), 439–478 (2004). doi:10.1207/S15327906MBR3903_3
D.L. Putnick, M.H. Bornstein, Measurement invariance conventions and reporting: The state of the art and future directions for psychological research. Dev. Rev. 41, 71–90 (2016). doi:10.1016/j.dr.2016.06.004
R Core Team, R: a language and environment for statistical computing (version 3.3.0). R Foundation for Statistical Computing (2016). Available via CRAN, https://www.R-project.org/
J.L. Rodgers, The bootstrap, the jackknife, and the randomization test: a sampling taxonomy. Multivar. Behav. Res. 34(4), 441–456 (1999). doi:10.1207/S15327906MBR3404_2
Y. Rosseel, Lavaan: an R package for structural equation modeling. J. Stat. Softw. 48(2), 1–36 (2012.) http://www.jstatsoft.org/v48/i02/
A. Satorra, W.E. Saris, Power of the likelihood ratio test in covariance structure analysis. Psychometrika 50, 83–90 (1985). doi:10.1007/BF02294150
J.H. Steiger, J.C. Lind, Statistically-based tests for the number of common factors. Paper presented at the annual meeting of the Psychometric Society, Iowa City (1980)
semTools Contributors, semTools: useful tools for structural equation modeling (version 0.4–12) (2016). Available via CRAN. https://www.R-project.org/
R.J. Vandenberg, C.E. Lance, A review and synthesis of the measurement invariance literature: Suggestions, practices, and recommendations for organizational research. Organ. Res. Methods 3(1), 4–70 (2000). doi:10.1177/109442810031002
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this paper
Cite this paper
Jorgensen, T.D., Kite, B.A., Chen, PY., Short, S.D. (2017). Finally! A Valid Test of Configural Invariance Using Permutation in Multigroup CFA. In: van der Ark, L.A., Wiberg, M., Culpepper, S.A., Douglas, J.A., Wang, WC. (eds) Quantitative Psychology. IMPS 2016. Springer Proceedings in Mathematics & Statistics, vol 196. Springer, Cham. https://doi.org/10.1007/978-3-319-56294-0_9
Download citation
DOI: https://doi.org/10.1007/978-3-319-56294-0_9
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-56293-3
Online ISBN: 978-3-319-56294-0
eBook Packages: Mathematics and StatisticsMathematics and Statistics (R0)