Skip to main content

Modified Distribution-Free Goodness-of-Fit Test Statistic

Abstract

Covariance structure analysis and its structural equation modeling extensions have become one of the most widely used methodologies in social sciences such as psychology, education, and economics. An important issue in such analysis is to assess the goodness of fit of a model under analysis. One of the most popular test statistics used in covariance structure analysis is the asymptotically distribution-free (ADF) test statistic introduced by Browne (Br J Math Stat Psychol 37:62–83, 1984). The ADF statistic can be used to test models without any specific distribution assumption (e.g., multivariate normal distribution) of the observed data. Despite its advantage, it has been shown in various empirical studies that unless sample sizes are extremely large, this ADF statistic could perform very poorly in practice. In this paper, we provide a theoretical explanation for this phenomenon and further propose a modified test statistic that improves the performance in samples of realistic size. The proposed statistic deals with the possible ill-conditioning of the involved large-scale covariance matrices.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

References

  1. Amemiya, Y., & Anderson, T. W. (1990). Asymptotic chi-square tests for a large class of factor analysis models. Annals of Statistics, 18, 1453–1463.

    Article  Google Scholar 

  2. Bellman, R. E. (1960). Introduction to matrix analysis. New York: McGraw-Hill Book Company.

    Google Scholar 

  3. Bollen, K. A. (1989). Structural equations with latent variables. New York: Wiley.

    Book  Google Scholar 

  4. Boomsma, A., & Hoogland, J. J. (2001). The robustness of lisrel modeling revisited. In Structural equation modeling: Present and future: A Festschrift in Honor of Karl Jöreskog (pp. 139–168). Chicago: Scientific Software International.

  5. Browne, M. W. (1982). Covariance structures. In D. M. Hawkins (Ed.), Topics in applied multivariate analysis (pp. 72–141). Cambridge: Cambridge University Press.

    Chapter  Google Scholar 

  6. Browne, M. W. (1984). Asymptotically distribution-free methods for the analysis of covariance structures. British Journal of Mathematical and Statistical Psychology, 37, 62–83.

    Article  PubMed  Google Scholar 

  7. Browne, M. W., & Shapiro, A. (1987). Robustness of normal theory methods in the analysis of linear latent variate models. British Journal of Mathematical and Statistical Psychology, 41, 193–208.

    Article  Google Scholar 

  8. Browne, M. W., & Shapiro, A. (2015). Comments on the asymptotics of a distribution-free goodness of fit test statistic. Psychometrika, 80, 196–199.

    Article  PubMed  Google Scholar 

  9. Byrne, B. M. (2012). Choosing structural equation modeling computer software: Snapshots of lisrel, eqs, amos, and mplus. In R. H. Hoyle (Ed.), Handbook of structural equation modeling, chap. 19 (pp. 307–324). New York: Guilford Press.

    Google Scholar 

  10. Chun, S. Y., & Shapiro, A. (2009). Normal versus noncentral chi-square asymptotics of misspecified models. Multivariate Behavioral Research, 44(6), 803–827.

    Article  PubMed  Google Scholar 

  11. Hoogland, J. J., & Boomsma, A. (1998). Robustness studies in covariance structure modeling: An overview and a meta-analysis. Sociological Methods and Research, 26(3), 329–367.

    Article  Google Scholar 

  12. Hu, L., Bentler, P. M., & Kano, Y. (1992). Can test statistics in covariance structure analysis be trusted? Psychological Bulletin, 112, 351–362.

    Article  PubMed  Google Scholar 

  13. Huang, Y., & Bentler, P. M. (2015). Behavior of asymptotically distribution free test statistics in covariance versus correlation structure analysis. Structural Equation Modeling, 22, 489–503.

    Article  Google Scholar 

  14. Jennrich, R., & Satorra, A. (2013). The nonsingularity of \(\gamma \) in covariance structure analysis of nonnormal data. Psychometrika, 79, 51–59.

    Article  PubMed  Google Scholar 

  15. Kano, Y. (2002). Variable selection for structural models. Journal of Statistical Planning and Inference, 108(1–2), 173–187.

    Article  Google Scholar 

  16. Lee, S.-Y. (1990). Multilevel analysis of structural equation models. Biometrika, 77(4), 763–772.

    Article  Google Scholar 

  17. McManus, D. A. (1991). Who invented local power analysis? Econometric Theory, 7, 265–268.

    Article  Google Scholar 

  18. Shapiro, A. (1986). Asymptotic theory of overparametrized structural models. Journal of the American Statistical Association, 81, 142–149.

    Article  Google Scholar 

  19. Tomarken, A., & Waller, N. G. (2005). Structural equation modeling as a data-analytic framework for clinical science: Strengths, limitations, and misconceptions. Annual Review of Clinical Psychology, 1, 31–65.

    Article  PubMed  Google Scholar 

  20. Wu, Hao, & Browne, M. W. (2015). Quantifying adventitious error in a covariance structure as a random effect. Psychometrika, 80(3), 571–600.

    Article  PubMed  PubMed Central  Google Scholar 

  21. Xu, J., & Mackenzie, G. (2012). Modelling covariance structure in bivariate marginal models for longitudinal data. Biometrika, 99(3), 649–662.

    Article  Google Scholar 

  22. Yuan, K. H., & Bentler, P. M. (1998). Normal theory based test statistics in structural equation modelling. British Journal of Mathematical and Statistical Psychology, 51, 289–309.

    Article  PubMed  Google Scholar 

  23. Yuan, K. H., & Bentler, P. M. (1999). F tests for mean and covariance structure analysis. Journal of Educational and Behavioral Statistics, 24(3), 225–243.

    Article  Google Scholar 

  24. Yuan, K. H., Hayashi, K., & Bentler, P. M. (2007). Normal theory likelihood ratio statistic for mean and covariance structure analysis under alternative hypotheses. Journal of Multivariate Analysis, 98(6), 1262–1282.

    Article  Google Scholar 

Download references

Acknowledgements

Funding was provided for the third author by National Science Foundation (Grant No. CMMI1232623).

Author information

Affiliations

Authors

Corresponding author

Correspondence to So Yeon Chun.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Chun, S.Y., Browne, M.W. & Shapiro, A. Modified Distribution-Free Goodness-of-Fit Test Statistic. Psychometrika 83, 48–66 (2018). https://doi.org/10.1007/s11336-017-9574-9

Download citation

Keywords

  • covariance structures
  • distribution-free test statistic
  • asymptotics
  • Chi-square distribution
  • ill-conditioned problem