, Volume 84, Issue 1, pp 65–83 | Cite as

Tests of Matrix Structure for Construct Validation

  • Brian D. SegalEmail author
  • Thomas Braun
  • Richard Gonzalez
  • Michael R. Elliott


Psychologists and other behavioral scientists are frequently interested in whether a questionnaire measures a latent construct. Attempts to address this issue are referred to as construct validation. We describe and extend nonparametric hypothesis testing procedures to assess matrix structures, which can be used for construct validation. These methods are based on a quadratic assignment framework and can be used either by themselves or to check the robustness of other methods. We investigate the performance of these matrix structure tests through simulations and demonstrate their use by analyzing a big five personality traits questionnaire administered as part of the Health and Retirement Study. We also derive rates of convergence for our overall test to better understand its behavior.


permutation testing hubert’s gamma quadratic assignment 



We thank Jacqui Smith, Philippa Clarke, and Trivellore Raghunathan for helpful discussion and feedback.

Supplementary material

11336_2018_9647_MOESM1_ESM.pdf (1.1 mb)
Supplementary material 1 (pdf 1112 KB)


  1. Barrett, P. (2007). Structural equation modeling: Adjudging model fit. Personality and Individual Differences, 42, 815–824.Google Scholar
  2. Beauducel, A., & Wittmann, W. (2005). Simulation study on fit indices in CFA based on data with slightly distorted simple structure. Structural Equation Modeling, 12(1), 41–75.Google Scholar
  3. Bentler, P. M. (1990). Comparative fit indexes in structural models. Psychological Bulletin, 107(2), 238–246.Google Scholar
  4. Bock, R. D., & Bargmann, R. E. (1966). Analysis of covariance structures. Psychometrika, 31(4), 507–534.Google Scholar
  5. Browne, M. W., & Cudeck, R. (1992). Alternative ways of assessing model fit. Sociological Methods & Research, 21(2), 230–258.Google Scholar
  6. Chung, E., & Romano, J. P. (2013). Exact and asymptotically robust permutation tests. The Annals of Statistics, 41(2), 484–507.Google Scholar
  7. Fan, X., & Sivo, S. A. (2005). Sensitivity of fit indices to misspecified structural or measurement model components: Rationale of two-index strategy revisited. Structural Equation Modeling, 12(3), 343–367.Google Scholar
  8. Fisher, R. A. (1921). On the “probable error” of a coefficient of correlation deduced from a small sample. Metron, 1, 3–32.Google Scholar
  9. Good, P. (2000). Permutation tests: A practical guide to resampling methods for testing hypotheses (2nd ed.). New York: Springer.Google Scholar
  10. Halkidi, M., Batistakis, Y., & Vazirgiannis, M. (2001). On clustering validation techniques. Journal of Intelligent Information Systems, 17(2/3), 107–145.Google Scholar
  11. Hawkins, D. L. (1989). Using U statistics to derive the asymptotic distribution of Fisher’s Z statistic. The American Statistician, 43(4), 235–237.Google Scholar
  12. Hooper, D., Coughlan, J., & Mullen, M. (2008). Structural equation modelling: Guidelines for determining model fit. The Electronic Journal of Business Research Methods, 6(1), 53–60.Google Scholar
  13. HRS (2016). Health and retirement study (core data release) public use dataset.Google Scholar
  14. Hu, L.-T., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6(1), 1–55.Google Scholar
  15. Hubert, L., & Schultz, J. (1976). Quadratic assignment as a general data analysis strategy. British Journal of Mathematical and Statistical Psychology, 29(2), 190–241.Google Scholar
  16. Jain, A. K., & Dubes, R. C. (1988). Algorithms for clustering data. Englewood Cliffs: Prentice Hall.Google Scholar
  17. Jöreskog, K. G. (1978). Structural analysis of covariance and correlation matrices. Psychometrika, 43(4), 443–477.Google Scholar
  18. Kline, R. B. (2011). Principles and practice of structural equation modeling (3rd ed.). New York: Guilford Press.Google Scholar
  19. Lehmann, E. L., & Romano, J. P. (2005). Testing statistical hypotheses (3rd ed.). New York: Springer.Google Scholar
  20. Mantel, N. (1967). The detection of disease clustering and a generalized regression approach. Cancer Research, 27(2), 209–220.Google Scholar
  21. Marsh, H. W., Hau, K.-T., & Wen, Z. (2004). In search of golden rules: Comment on hypothesis-testing approaches to setting cutoff values for fit indexes and dangers in overgeneralizing Hu and Bentlers (1999) findings. Structural Equation Modeling, 11(3), 320–341.Google Scholar
  22. McDonald, R. P. (1974). Testing pattern hypotheses for covariance matrices. Psychometrika, 39(2), 189–201.Google Scholar
  23. R Core Team. (2017). R: A language and environment for statistical computing. Vienna: R Foundation for Statistical Computing.Google Scholar
  24. Rosseel, Y. (2012). lavaan: An R package for structural equation modeling. Journal of Statistical Software, 48(2), 1–36.Google Scholar
  25. Srivastava, J. N. (1966). On testing hypotheses regarding a class of covariance structures. Psychometrika, 31(2), 147–164.Google Scholar
  26. Steiger, J. H. (1980a). Testing pattern hypotheses on correlation matrices: Alternative statistics and some empirical results. Multivariate Behavioral Research, 15(3), 335–352.Google Scholar
  27. Steiger, J. H. (1980b). Tests for comparing elements of a correlation matrix. Psychological Bulletin, 87(2), 245–251.Google Scholar
  28. Steiger, J. H. (1990). Structural model evaluation and modification: An interval estimation approach. Multivariate Behavioral Research, 25, 173–180.Google Scholar
  29. Steiger, J. H. (2007). Understanding the limitations of global fit assessment in structural equation modeling. Personality and Individual Differences, 42(5), 893–898.Google Scholar
  30. Steiger, J. H., & Lind, J. (1980). Statistically-based tests for the number of common factors. Iowa City: Paper presented at the annual spring meeting of the Psychometric Society.Google Scholar
  31. Tucker, L. R., & Lewis, C. (1973). A reliability coefficient for maximum likelihood factor analysis. Psychometrika, 38(1), 1–10.Google Scholar
  32. Wasserstein, R. L., & Lazar, N. A. (2016). The ASA’s statement on p values: Context, process, and purpose. The American Statistician, 70(2), 193–242.Google Scholar
  33. Westfall, P. H., & Young, S. S. (1993). Resampling-based multiple testing: Examples and methods for p-value adjustment (Vol. 279). New York, NY: Wiley.Google Scholar
  34. Yuan, K.-H. (2005). Fit indices versus test statistics. Multivariate Behavioral Research, 40(1), 115–148.Google Scholar
  35. Zaki, M. J., & Meira, W, Jr. (2014). Data mining and analysis: Fundamental concepts and algorithms. New York, NY: Cambridge University Press.Google Scholar

Copyright information

© The Psychometric Society 2018

Authors and Affiliations

  • Brian D. Segal
    • 1
    Email author
  • Thomas Braun
    • 1
  • Richard Gonzalez
    • 1
  • Michael R. Elliott
    • 1
  1. 1.University of MichiganAnn ArborUSA

Personalised recommendations