, Volume 79, Issue 4, pp 569–584 | Cite as

Testing for Measurement Invariance with Respect to an Ordinal Variable

  • Edgar C. MerkleEmail author
  • Jinyan Fan
  • Achim Zeileis


Researchers are often interested in testing for measurement invariance with respect to an ordinal auxiliary variable such as age group, income class, or school grade. In a factor-analytic context, these tests are traditionally carried out via a likelihood ratio test statistic comparing a model where parameters differ across groups to a model where parameters are equal across groups. This test neglects the fact that the auxiliary variable is ordinal, and it is also known to be overly sensitive at large sample sizes. In this paper, we propose test statistics that explicitly account for the ordinality of the auxiliary variable, resulting in higher power against “monotonic” violations of measurement invariance and lower power against “non-monotonic” ones. The statistics are derived from a family of tests based on stochastic processes that have recently received attention in the psychometric literature. The statistics are illustrated via an application involving real data, and their performance is studied via simulation.

Key words

measurement invariance stochastic process factor analysis ordinal data 



This work was supported by National Science Foundation grant SES-1061334.


  1. Akaike, H. (1974). A new look at the statistical model identification. IEEE Transactions on Automatic Control, 19(6), 716–723. Available from CrossRefGoogle Scholar
  2. Bauer, D.J., & Hussong, A.M. (2009). Psychometric approaches for developing commensurate measures across independent studies: traditional and new models. Psychological Methods, 14, 101–125. PubMedCentralPubMedCrossRefGoogle Scholar
  3. Bentler, P.M., & Bonett, D.G. (1980). Significance tests and goodness of fit in the analysis of covariance structures. Psychological Bulletin, 88, 588–606. CrossRefGoogle Scholar
  4. Browne, M.W., & Cudeck, R. (1993). Alternative ways of assessing model fit. In K.A. Bollen & J.S. Long (Eds.), Testing structural equation models (pp. 136–162). Newbury Park: Sage. Google Scholar
  5. Dolan, C.V., & van der Maas, H.L.J. (1998). Fitting multivariate normal finite mixtures subject to structural equation modeling. Psychometrika, 63, 227–253. CrossRefGoogle Scholar
  6. Emmons, R.A. & McCullough, M.E. (Eds.) (2004). The psychology of gratitude. New York: Oxford University Press. Google Scholar
  7. Ferguson, T.S. (1996). A course in large sample theory. London: Chapman & Hall. CrossRefGoogle Scholar
  8. Froh, J.J., Fan, J., Emmons, R.A., Bono, G., Huebner, E.S., & Watkins, P. (2011). Measuring gratitude in youth: assessing the psychometric properties of adult gratitude scales in children and adolescents. Psychological Assessment, 23, 311–324. PubMedCrossRefGoogle Scholar
  9. Genz, A., Bretz, F., Miwa, T., Mi, X., Leisch, F., Scheipl, F., et al. (2012). mvtnorm: multivariate normal and t distributions [Computer software manual]. Available from (R package version 0.9-9992).
  10. Hjort, N.L., & Koning, A. (2002). Tests for constancy of model parameters over time. Nonparametric Statistics, 14, 113–132. CrossRefGoogle Scholar
  11. Hothorn, T., & Zeileis, A. (2008). Generalized maximally selected statistics. Biometrics, 64(4), 1263–1269. PubMedCrossRefGoogle Scholar
  12. Huber, P.J. (1967). The behavior of maximum likelihood estimation under nonstandard conditions. In L.M. LeCam & J. Neyman (Eds.), Proceedings of the fifth Berkeley symposium on mathematical statistics and probability. Berkeley: University of California Press. Google Scholar
  13. Lubke, G.H., & Muthén, B. (2005). Investigating population heterogeneity with factor mixture models. Psychological Methods, 10, 21–39. PubMedCrossRefGoogle Scholar
  14. McCullough, M.E., Emmons, R.A., & Tsang, J.-A. (2002). The grateful disposition: a conceptual and empirical topography. Journal of Personality and Social Psychology, 82, 112–127. PubMedCrossRefGoogle Scholar
  15. Mellenbergh, G.J. (1989). Item bias and item response theory. International Journal of Educational Research, 13, 127–143. CrossRefGoogle Scholar
  16. Merkle, E.C., & Zeileis, A. (2013). Tests of measurement invariance without subgroups: a generalization of classical methods. Psychometrika, 78, 59–82. PubMedCrossRefGoogle Scholar
  17. Millsap, R.E. (2011). Statistical approaches to measurement invariance. New York: Routledge. Google Scholar
  18. Molenaar, D., Dolan, C.V., Wicherts, J.M., & van der Mass, H.L.J. (2010). Modeling differentiation of cognitive abilities within the higher-order factor model using moderated factor analysis. Intelligence, 38, 611–624. CrossRefGoogle Scholar
  19. Purcell, S. (2002). Variance components models for gene-environment interaction in twin analysis. Twin Research, 5, 554–571. PubMedCrossRefGoogle Scholar
  20. R Development Core Team. (2012). R: a language and environment for statistical computing [Computer software manual]. URL Vienna, Austria. (ISBN 3-900051-07-0)
  21. Rosseel, Y. (2012). lavaan: an R package for structural equation modeling. Journal of Statistical Software, 48(2), 136. Available from Google Scholar
  22. Satorra, A. (1989). Alternative test criteria in covariance structure analysis: a unified approach. Psychometrika, 54, 131–151. CrossRefGoogle Scholar
  23. Satorra, A., & Bentler, P.M. (2001). A scaled difference chi-square test statistic for moment structure analysis. Psychometrika, 66, 507–514. CrossRefGoogle Scholar
  24. Sörbom, D. (1989). Model modification. Psychometrika, 54, 371–384. CrossRefGoogle Scholar
  25. Strobl, C., Kopf, J., & Zeileis, A. (2013). A new method for detecting differential item functioning in the Rasch model. Psychometrika. doi: 10.1007/s11336-013-9388-3. PubMedGoogle Scholar
  26. Thomas, M., & Watkins, P. (2003). Measuring the grateful trait: development of the revised GRAT. Poster presented at the Annual Convention of the Western Psychological Association, Vancouver, BC. Google Scholar
  27. Vandenberg, R.J., & Lance, C.E. (2000). A review and synthesis of the measurement invariance literature: suggestions, practices, and recommendations for organizational research. Organizational Research Methods, 3, 4–70. CrossRefGoogle Scholar
  28. Yuan, K.-H., & Bentler, P.M. (1997). Mean and covariance structure analysis: theoretical and practical improvements. Journal of the American Statistical Association, 92, 767–774. CrossRefGoogle Scholar
  29. Zeileis, A. (2006a). Implementing a class of structural change tests: an econometric computing approach. Computational Statistics & Data Analysis, 50(11), 2987–3008. CrossRefGoogle Scholar
  30. Zeileis, A. (2006b). Object-oriented computation of sandwich estimators. Journal of Statistical Software, 16(9), 1–16. Available from Google Scholar
  31. Zeileis, A., & Hornik, K. (2007). Generalized M-fluctuation tests for parameter instability. Statistica Neerlandica, 61, 488–508. CrossRefGoogle Scholar
  32. Zeileis, A., Leisch, F., Hornik, K., & Kleiber, C. (2002). strucchange: an R package for testing structural change in linear regression models. Journal of Statistical Software, 7, 1–38. Available from Google Scholar

Copyright information

© The Psychometric Society 2013

Authors and Affiliations

  1. 1.Department of Psychological SciencesUniversity of MissouriColumbiaUSA
  2. 2.Auburn UniversityAuburnUSA
  3. 3.Universität InnsbruckInnsbruckAustria

Personalised recommendations