Tests of Matrix Structure for Construct Validation

Psychometrika volume 84, pages65–83(2019)Cite this article

Abstract

Psychologists and other behavioral scientists are frequently interested in whether a questionnaire measures a latent construct. Attempts to address this issue are referred to as construct validation. We describe and extend nonparametric hypothesis testing procedures to assess matrix structures, which can be used for construct validation. These methods are based on a quadratic assignment framework and can be used either by themselves or to check the robustness of other methods. We investigate the performance of these matrix structure tests through simulations and demonstrate their use by analyzing a big five personality traits questionnaire administered as part of the Health and Retirement Study. We also derive rates of convergence for our overall test to better understand its behavior.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

References

  1. Barrett, P. (2007). Structural equation modeling: Adjudging model fit. Personality and Individual Differences, 42, 815–824.

    Article  Google Scholar 

  2. Beauducel, A., & Wittmann, W. (2005). Simulation study on fit indices in CFA based on data with slightly distorted simple structure. Structural Equation Modeling, 12(1), 41–75.

    Article  Google Scholar 

  3. Bentler, P. M. (1990). Comparative fit indexes in structural models. Psychological Bulletin, 107(2), 238–246.

    Article  PubMed  Google Scholar 

  4. Bock, R. D., & Bargmann, R. E. (1966). Analysis of covariance structures. Psychometrika, 31(4), 507–534.

    Article  PubMed  Google Scholar 

  5. Browne, M. W., & Cudeck, R. (1992). Alternative ways of assessing model fit. Sociological Methods & Research, 21(2), 230–258.

    Article  Google Scholar 

  6. Chung, E., & Romano, J. P. (2013). Exact and asymptotically robust permutation tests. The Annals of Statistics, 41(2), 484–507.

    Article  Google Scholar 

  7. Fan, X., & Sivo, S. A. (2005). Sensitivity of fit indices to misspecified structural or measurement model components: Rationale of two-index strategy revisited. Structural Equation Modeling, 12(3), 343–367.

    Article  Google Scholar 

  8. Fisher, R. A. (1921). On the “probable error” of a coefficient of correlation deduced from a small sample. Metron, 1, 3–32.

    Google Scholar 

  9. Good, P. (2000). Permutation tests: A practical guide to resampling methods for testing hypotheses (2nd ed.). New York: Springer.

    Google Scholar 

  10. Halkidi, M., Batistakis, Y., & Vazirgiannis, M. (2001). On clustering validation techniques. Journal of Intelligent Information Systems, 17(2/3), 107–145.

    Article  Google Scholar 

  11. Hawkins, D. L. (1989). Using U statistics to derive the asymptotic distribution of Fisher’s Z statistic. The American Statistician, 43(4), 235–237.

    Google Scholar 

  12. Hooper, D., Coughlan, J., & Mullen, M. (2008). Structural equation modelling: Guidelines for determining model fit. The Electronic Journal of Business Research Methods, 6(1), 53–60.

    Google Scholar 

  13. HRS (2016). Health and retirement study (core data release) public use dataset.

  14. Hu, L.-T., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6(1), 1–55.

    Article  Google Scholar 

  15. Hubert, L., & Schultz, J. (1976). Quadratic assignment as a general data analysis strategy. British Journal of Mathematical and Statistical Psychology, 29(2), 190–241.

    Article  Google Scholar 

  16. Jain, A. K., & Dubes, R. C. (1988). Algorithms for clustering data. Englewood Cliffs: Prentice Hall.

    Google Scholar 

  17. Jöreskog, K. G. (1978). Structural analysis of covariance and correlation matrices. Psychometrika, 43(4), 443–477.

    Article  Google Scholar 

  18. Kline, R. B. (2011). Principles and practice of structural equation modeling (3rd ed.). New York: Guilford Press.

    Google Scholar 

  19. Lehmann, E. L., & Romano, J. P. (2005). Testing statistical hypotheses (3rd ed.). New York: Springer.

    Google Scholar 

  20. Mantel, N. (1967). The detection of disease clustering and a generalized regression approach. Cancer Research, 27(2), 209–220.

    PubMed  Google Scholar 

  21. Marsh, H. W., Hau, K.-T., & Wen, Z. (2004). In search of golden rules: Comment on hypothesis-testing approaches to setting cutoff values for fit indexes and dangers in overgeneralizing Hu and Bentlers (1999) findings. Structural Equation Modeling, 11(3), 320–341.

    Article  Google Scholar 

  22. McDonald, R. P. (1974). Testing pattern hypotheses for covariance matrices. Psychometrika, 39(2), 189–201.

    Article  Google Scholar 

  23. R Core Team. (2017). R: A language and environment for statistical computing. Vienna: R Foundation for Statistical Computing.

    Google Scholar 

  24. Rosseel, Y. (2012). lavaan: An R package for structural equation modeling. Journal of Statistical Software, 48(2), 1–36.

    Article  Google Scholar 

  25. Srivastava, J. N. (1966). On testing hypotheses regarding a class of covariance structures. Psychometrika, 31(2), 147–164.

    Article  PubMed  Google Scholar 

  26. Steiger, J. H. (1980a). Testing pattern hypotheses on correlation matrices: Alternative statistics and some empirical results. Multivariate Behavioral Research, 15(3), 335–352.

    Article  PubMed  Google Scholar 

  27. Steiger, J. H. (1980b). Tests for comparing elements of a correlation matrix. Psychological Bulletin, 87(2), 245–251.

    Article  Google Scholar 

  28. Steiger, J. H. (1990). Structural model evaluation and modification: An interval estimation approach. Multivariate Behavioral Research, 25, 173–180.

    Article  PubMed  Google Scholar 

  29. Steiger, J. H. (2007). Understanding the limitations of global fit assessment in structural equation modeling. Personality and Individual Differences, 42(5), 893–898.

    Article  Google Scholar 

  30. Steiger, J. H., & Lind, J. (1980). Statistically-based tests for the number of common factors. Iowa City: Paper presented at the annual spring meeting of the Psychometric Society.

  31. Tucker, L. R., & Lewis, C. (1973). A reliability coefficient for maximum likelihood factor analysis. Psychometrika, 38(1), 1–10.

    Article  Google Scholar 

  32. Wasserstein, R. L., & Lazar, N. A. (2016). The ASA’s statement on p values: Context, process, and purpose. The American Statistician, 70(2), 193–242.

    Article  Google Scholar 

  33. Westfall, P. H., & Young, S. S. (1993). Resampling-based multiple testing: Examples and methods for p-value adjustment (Vol. 279). New York, NY: Wiley.

    Google Scholar 

  34. Yuan, K.-H. (2005). Fit indices versus test statistics. Multivariate Behavioral Research, 40(1), 115–148.

    Article  PubMed  Google Scholar 

  35. Zaki, M. J., & Meira, W, Jr. (2014). Data mining and analysis: Fundamental concepts and algorithms. New York, NY: Cambridge University Press.

    Google Scholar 

Download references

Acknowledgements

We thank Jacqui Smith, Philippa Clarke, and Trivellore Raghunathan for helpful discussion and feedback.

Author information

Affiliations

  1. University of Michigan, Ann Arbor, MI, USA

    Brian D. Segal, Thomas Braun, Richard Gonzalez & Michael R. Elliott

Authors
  1. Brian D. Segal
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Thomas Braun
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Richard Gonzalez
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Michael R. Elliott
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Brian D. Segal.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (pdf 1112 KB)

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Segal, B.D., Braun, T., Gonzalez, R. et al. Tests of Matrix Structure for Construct Validation. Psychometrika 84, 65–83 (2019). https://doi.org/10.1007/s11336-018-9647-4

Download citation

Keywords

  • permutation testing
  • hubert’s gamma
  • quadratic assignment