, Volume 83, Issue 1, pp 48–66 | Cite as

Modified Distribution-Free Goodness-of-Fit Test Statistic

  • So Yeon ChunEmail author
  • Michael W. Browne
  • Alexander Shapiro


Covariance structure analysis and its structural equation modeling extensions have become one of the most widely used methodologies in social sciences such as psychology, education, and economics. An important issue in such analysis is to assess the goodness of fit of a model under analysis. One of the most popular test statistics used in covariance structure analysis is the asymptotically distribution-free (ADF) test statistic introduced by Browne (Br J Math Stat Psychol 37:62–83, 1984). The ADF statistic can be used to test models without any specific distribution assumption (e.g., multivariate normal distribution) of the observed data. Despite its advantage, it has been shown in various empirical studies that unless sample sizes are extremely large, this ADF statistic could perform very poorly in practice. In this paper, we provide a theoretical explanation for this phenomenon and further propose a modified test statistic that improves the performance in samples of realistic size. The proposed statistic deals with the possible ill-conditioning of the involved large-scale covariance matrices.


covariance structures distribution-free test statistic asymptotics Chi-square distribution ill-conditioned problem 



Funding was provided for the third author by National Science Foundation (Grant No. CMMI1232623).


  1. Amemiya, Y., & Anderson, T. W. (1990). Asymptotic chi-square tests for a large class of factor analysis models. Annals of Statistics, 18, 1453–1463.CrossRefGoogle Scholar
  2. Bellman, R. E. (1960). Introduction to matrix analysis. New York: McGraw-Hill Book Company.Google Scholar
  3. Bollen, K. A. (1989). Structural equations with latent variables. New York: Wiley.CrossRefGoogle Scholar
  4. Boomsma, A., & Hoogland, J. J. (2001). The robustness of lisrel modeling revisited. In Structural equation modeling: Present and future: A Festschrift in Honor of Karl Jöreskog (pp. 139–168). Chicago: Scientific Software International.Google Scholar
  5. Browne, M. W. (1982). Covariance structures. In D. M. Hawkins (Ed.), Topics in applied multivariate analysis (pp. 72–141). Cambridge: Cambridge University Press.CrossRefGoogle Scholar
  6. Browne, M. W. (1984). Asymptotically distribution-free methods for the analysis of covariance structures. British Journal of Mathematical and Statistical Psychology, 37, 62–83.CrossRefPubMedGoogle Scholar
  7. Browne, M. W., & Shapiro, A. (1987). Robustness of normal theory methods in the analysis of linear latent variate models. British Journal of Mathematical and Statistical Psychology, 41, 193–208.CrossRefGoogle Scholar
  8. Browne, M. W., & Shapiro, A. (2015). Comments on the asymptotics of a distribution-free goodness of fit test statistic. Psychometrika, 80, 196–199.CrossRefPubMedGoogle Scholar
  9. Byrne, B. M. (2012). Choosing structural equation modeling computer software: Snapshots of lisrel, eqs, amos, and mplus. In R. H. Hoyle (Ed.), Handbook of structural equation modeling, chap. 19 (pp. 307–324). New York: Guilford Press.Google Scholar
  10. Chun, S. Y., & Shapiro, A. (2009). Normal versus noncentral chi-square asymptotics of misspecified models. Multivariate Behavioral Research, 44(6), 803–827.CrossRefPubMedGoogle Scholar
  11. Hoogland, J. J., & Boomsma, A. (1998). Robustness studies in covariance structure modeling: An overview and a meta-analysis. Sociological Methods and Research, 26(3), 329–367.CrossRefGoogle Scholar
  12. Hu, L., Bentler, P. M., & Kano, Y. (1992). Can test statistics in covariance structure analysis be trusted? Psychological Bulletin, 112, 351–362.CrossRefPubMedGoogle Scholar
  13. Huang, Y., & Bentler, P. M. (2015). Behavior of asymptotically distribution free test statistics in covariance versus correlation structure analysis. Structural Equation Modeling, 22, 489–503.CrossRefGoogle Scholar
  14. Jennrich, R., & Satorra, A. (2013). The nonsingularity of \(\gamma \) in covariance structure analysis of nonnormal data. Psychometrika, 79, 51–59.CrossRefPubMedGoogle Scholar
  15. Kano, Y. (2002). Variable selection for structural models. Journal of Statistical Planning and Inference, 108(1–2), 173–187.CrossRefGoogle Scholar
  16. Lee, S.-Y. (1990). Multilevel analysis of structural equation models. Biometrika, 77(4), 763–772.CrossRefGoogle Scholar
  17. McManus, D. A. (1991). Who invented local power analysis? Econometric Theory, 7, 265–268.CrossRefGoogle Scholar
  18. Shapiro, A. (1986). Asymptotic theory of overparametrized structural models. Journal of the American Statistical Association, 81, 142–149.CrossRefGoogle Scholar
  19. Tomarken, A., & Waller, N. G. (2005). Structural equation modeling as a data-analytic framework for clinical science: Strengths, limitations, and misconceptions. Annual Review of Clinical Psychology, 1, 31–65.CrossRefPubMedGoogle Scholar
  20. Wu, Hao, & Browne, M. W. (2015). Quantifying adventitious error in a covariance structure as a random effect. Psychometrika, 80(3), 571–600.CrossRefPubMedPubMedCentralGoogle Scholar
  21. Xu, J., & Mackenzie, G. (2012). Modelling covariance structure in bivariate marginal models for longitudinal data. Biometrika, 99(3), 649–662.CrossRefGoogle Scholar
  22. Yuan, K. H., & Bentler, P. M. (1998). Normal theory based test statistics in structural equation modelling. British Journal of Mathematical and Statistical Psychology, 51, 289–309.CrossRefPubMedGoogle Scholar
  23. Yuan, K. H., & Bentler, P. M. (1999). F tests for mean and covariance structure analysis. Journal of Educational and Behavioral Statistics, 24(3), 225–243.CrossRefGoogle Scholar
  24. Yuan, K. H., Hayashi, K., & Bentler, P. M. (2007). Normal theory likelihood ratio statistic for mean and covariance structure analysis under alternative hypotheses. Journal of Multivariate Analysis, 98(6), 1262–1282.CrossRefGoogle Scholar

Copyright information

© The Psychometric Society 2017

Authors and Affiliations

  • So Yeon Chun
    • 1
    Email author
  • Michael W. Browne
    • 2
  • Alexander Shapiro
    • 3
  1. 1.McDonough School of BusinessGeorgetown UniversityWashingtonUSA
  2. 2.Department of PsychologyOhio State UniversityColumbusUSA
  3. 3.School of Industrial and Systems EngineeringGeorgia Institute of TechnologyAtlantaUSA

Personalised recommendations