Advertisement

Springer Nature is making SARS-CoV-2 and COVID-19 research free. View research | View latest news | Sign up for updates

The Normal-Theory and Asymptotic Distribution-Free (ADF) Covariance Matrix of Standardized Regression Coefficients: Theoretical Extensions and Finite Sample Behavior

Abstract

Yuan and Chan (Psychometrika, 76, 670–690, 2011) recently showed how to compute the covariance matrix of standardized regression coefficients from covariances. In this paper, we describe a method for computing this covariance matrix from correlations. Next, we describe an asymptotic distribution-free (ADF; Browne in British Journal of Mathematical and Statistical Psychology, 37, 62–83, 1984) method for computing the covariance matrix of standardized regression coefficients. We show that the ADF method works well with nonnormal data in moderate-to-large samples using both simulated and real-data examples. R code (R Development Core Team, 2012) is available from the authors or through the Psychometrika online repository for supplementary materials.

This is a preview of subscription content, log in to check access.

Notes

  1. 1.

    Specifically, the “textbook” standard error for \(\hat{\beta}_{j}\) is incorrect unless the population regression coefficient for variable j equals zero.

  2. 2.

    For a review of these works, see Jones and Waller (2013).

  3. 3.

    Although not pursued in this paper, it is also possible to derive our results using the Jacobian from Section 3 (see Equation (27)) and an ADF estimator of \(\operatorname{acov}(\hat{\boldsymbol{r}}_{vp})\) using population standardized fourth-order central moments; see Browne and Shapiro (1986), Hsu (1949), Isserlis (1916), and Steiger and Hakstian (1982) for details.

  4. 4.

    Note that RAMONA estimates standard errors using n−1 rather than n. We rescaled our estimates accordingly before making the comparisons.

References

  1. Abadir, K.M., & Magnus, J.R. (2005). Matrix algebra. New York: Cambridge University Press.

  2. Algina, J., & Moulder, B.C. (2001). Sample sizes for confidence intervals on the increase in the squared multiple correlation coefficient. Educational and Psychological Measurement, 61, 633–649.

  3. Astin, H.S. (1967). Career development during the high school years. Journal of Counseling Psychology, 14, 94–98.

  4. Austin, J.T., & Hanisch, K.A. (1990). Occupational attainment as a function of abilities and interests: a longitudinal analysis using project talent data. Journal of Applied Psychology, 75(1), 77–86.

  5. Becker, B., & Wu, M. (2007). The synthesis of regression slopes in meta-analysis. Statistical Science, 22, 414–429.

  6. Bentler, P., & Lee, S.Y. (1983). Covariance structures under polynomial constraints: applications to correlation and alpha-type structural models. Journal of Educational and Behavioral Statistics, 8(3), 207.

  7. Bollen, K.A., & Stine, R. (1990). Direct and indirect effects: classical and bootstrap estimates of variability. Sociological Methodology, 20, 115–140.

  8. Boomsma, A., & Hoogland, J.J. (2001). The robustness of LISREL modeling revisited. In R. Cudeck, S. Du Toit, & D. Sorbom (Eds.), Structural equation modeling: present and future (pp. 139–168). Chicago: Scientific Software International.

  9. Bring, J. (1994). How to standardize regression coefficients. American Statistician, 48, 209–213.

  10. Browne, M.W. (1974). Generalized least squares estimators in the analysis of covariance structures. South African Statistical Journal, 8, 1–24.

  11. Browne, M. (1982). Covariance structures. In D.M. Hawkins (Ed.), Topics in applied multivariate analysis (pp. 72–141). Cambridge: Cambridge University Press.

  12. Browne, M.W. (1984). Asymptotically distribution-free methods for the analysis of covariance structures. British Journal of Mathematical & Statistical Psychology, 37, 62–83.

  13. Browne, M.W., Mels, G., & Cowan, M. (1994). Path analysis: Ramona: Systat for DOS advanced applications. SOFTWARE, Version 6:167–224.

  14. Browne, M.W., & Shapiro, A. (1986). The asymptotic covariance matrix of sample correlation coefficients under general conditions. Linear Algebra and Its Applications, 82, 169–176.

  15. Card, J.J. (1987). Epidemiology of post-traumatic stress disorder in a national cohort of Vietnam veterans. Journal of Clinical Psychology, 43(1), 6–17.

  16. Casella, G., & Berger, R.L. (2001). Statistical inference. Belmont: Wadsworth.

  17. Cohen, J., Cohen, P., West, S.G., & Aiken, L.S. (2003). Applied multiple regression/correlation analysis for the behavioral sciences (3rd ed.). Mahwah: Lawrence Erlbaum Associates, Inc.

  18. Cudeck, R. (1989). Analysis of correlation matrices using covariance structure models. Psychological Bulletin, 105, 317–327.

  19. Curran, P.J. (1994). The robustness of confirmatory factor analysis to model misspecification and violations of normality. Unpublished Doctoral Dissertation, Arizona State University.

  20. Efron, B. (1982). The jackknife, the bootstrap and other resampling plans. In CBMS-NSF regional conference series in applied mathematics (Vol. 38). Philadelphia: SIAM.

  21. Efron, B. (1987). Better bootstrap confidence intervals. Journal of the American Statistical Association, 82(397), 171–185.

  22. Efron, B., & Tibshirani, R.J. (1994). Monographs on statistics & applied probability. An introduction to the bootstrap. Boca Raton: Chapman & Hall/CRC.

  23. Ferguson, T.S. (1996). A course in large sample theory. New York: Chapman & Hall.

  24. Fleishman, A.I. (1978). A method for simulating non-normal distributions. Psychometrika, 43, 521–532.

  25. Greene, W.H. (2003). Econometric analysis (5th ed.). Upper Saddle River: Prentice-Hall.

  26. Greenland, S., Maclure, M., Schlesselman, J.J., Poole, C., & Morgenstern, H. (1991). Standardized regression coefficients: a further critique and review of some alternatives. Epidemiology, 2, 387–392.

  27. Harris, R.J. (2001). A primer on multivariate statistics (3rd ed.). Mahwah: Lawrence Erlbaum Associates.

  28. Hays, W.L. (1994). Statistics (5th ed.). Worth Fort: Harcourt Brace College Publisher.

  29. Hsu, P.L. (1949). The limiting distribution of functions of sample means and application to testing hypotheses. In J. Neyman (Ed.), Proceedings of the first Berkeley symposium on mathematical statistics and probability (pp. 359–402). Berkeley: Univ. of California Press.

  30. Isserlis, L. (1916). On certain probable errors and correlation coefficients of multiple frequency distributions with skew regression. Biometrika, 11, 185–190.

  31. Jamshidian, M., & Bentler, P.M. (2000). Improved standard errors of standardized parameters in covariance structure models: implications for construction explication. In R.D. Goffin & E. Helmes (Eds.), Problems and solutions in human assessment (pp. 73–94). Dordrecht: Kluwer Academic.

  32. Jones, J.A., & Waller, N.G. (in press). Computing confidence intervals for standardized regression coefficients. Psychological Methods. doi:10.1037/a0033269.

  33. Jones, J.A., & Waller, N.G. (2013). The normal-theory and asymptotic distribution-free (ADF) covariance matrix of standardized regression coefficients: theoretical extensions and finite sample behavior. Minneapolis: University of Minnesota. Retrieved from http://www.psych.umn.edu/faculty/waller/downloads/techreports/TR052913.pdf.

  34. Kano, Y., Berkane, M., & Bentler, P.M. (1993). Statistical inference based on pseudo-maximum likelihood estimators in elliptical populations. Journal of the American Statistical Association, 88(421), 135–143.

  35. Kelley, K. (2007). Confidence intervals for standardized effect sizes: theory, application, and implementation. Journal of Statistical Software, 20(8), 1–24.

  36. Kelley, K., & Maxwell, S.E. (2003). Sample size for multiple regression: obtaining regression coefficients that are accurate, not simply significant. Psychological Methods, 8, 305–321.

  37. Kim, R.S. (2011). Standardized regression coefficients as indices of effect sizes in meta analysis. Unpublished Doctoral Dissertation, Florida State University, Tallahassee, Florida.

  38. Kwan, J.L., & Chan, W. (2011). Comparing standardized coefficients in structural equation modeling: a model reparameterization approach. Behavior Research Methods, 43(3), 730–745.

  39. King, G. (1985). How not to lie with statistics: avoiding common mistakes in quantitative political science. American Journal of Political Science, 30, 666–687.

  40. Mardia, K.V. (1970). Measures of multivariate skewness and kurtosis with applications. Biometrika, 57, 519–530.

  41. Micceri, T. (1989). The unicorn, the normal curve, and other improbable creatures. Psychological Bulletin, 105, 156–166.

  42. Muthén, B., & Muthén, L. (2012). Mplus users guide (7th ed.). Los Angeles: Muthen & Muthen.

  43. Nel, D.G. (1985). A matrix derivation of the asymptotic covariance matrix of sample correlation coefficients. Linear Algebra and Its Applications, 67, 137–145.

  44. Oehlert, G.W. (1992). A note on the delta method. American Statistician, 46, 27–29.

  45. Ogasawara, H. (2007). Asymptotic expansion and asymptotic robustness of the normal-theory estimators in the random regression model. Journal of Statistical Computation and Simulation, 77(10), 821–838.

  46. Ogasawara, H. (2008). Asymptotic expansion in reduced rank regression under normality and nonnormality. Communications in Statistics. Theory and Methods, 37, 1051–1070.

  47. Olkin, I., & Finn, J.D. (1995). Correlations redux. Psychological Bulletin, 118, 155–164.

  48. Olkin, I., & Siotani, M. (1976). Asymptotic distribution of functions of a correlation matrix. In S. Ideka (Ed.), Essays in probability and statistics (pp. 235–251). Tokyo: Shinko, Tsusho Co., Ltd.

  49. Pearson, K., & Filon, L. (1898). On the probable errors of frequency constants and on the influence of random selection on variation and correlation. Philosophical Transactions of the Royal Society of London, 191, 229–311.

  50. Peterson, R., & Brown, S. (2005). On the use of beta coefficients in meta-analysis. Journal of Applied Psychology, 90, 175–181.

  51. R Development Core Team (2012). R: a language and environment for statistical computing. Vienna: R Foundation for Statistical Computing. ISBN 3-900051-07-0. http://www.R-project.org/.

  52. Rencher, A.C. (2008). Linear models in statistics (2nd ed.). Hoboken: Wiley.

  53. Steiger, J.H. (1995). Structural equation modeling (SEPATH). Tulsa: Statsoft Inc.

  54. Steiger, J.H., & Fouladi, R.T. (1997). Noncentrality interval estimation and the evaluation of statistical models. In L.L. Harlow, S.A. Mulaik, & J.H. Steiger (Eds.), What if there were no significance tests? (pp. 221–257). London: Taylor and Francis.

  55. Steiger, J.H., & Hakstian, A. (1980). The asymptotic distribution of elements of a correlation matrix (Technical Report No. 80 3 (May)). Institute of Applied Mathematics and Statistics, University of British Columbia.

  56. Steiger, J., & Hakstian, A. (1982). The asymptotic distribution of elements of a correlation matrix: theory and application. British Journal of Mathematical & Statistical Psychology, 35, 208–215.

  57. Tukey, J.W. (1954). Causation, regression, and path analysis. In O. Kempthorne, T.A. Bancroft, J.W. Gowen, & J.L. Lush (Eds.), Statistics and mathematics in biology (pp. 35–66). Ames: Iowa State College Press.

  58. Vale, C.D., & Maurelli, V.A. (1983). Simulating multivariate nonnormal distributions. Psychometrika, 48, 465–471.

  59. Waller, N.G. (2011). The geometry of enhancement in multiple regression. Psychometrika, 76, 634–649.

  60. Waller, N.G., & Jones, J.A. (2010). Correlation weights in multiple regression. Psychometrika, 75(1), 58–69.

  61. Waller, N.G., & Jones, J.A. (2011). Investigating the performance of alternate regression weights by studying all possible criteria in regression models with a fixed set of predictors. Psychometrika, 76, 410–439.

  62. Waller, N.G., Underhill, M., & Kaiser, H.A. (1999). A method for generating simulated plasmodes and artificial test clusters with user-defined shape, size, and orientation. Multivariate Behavioral Research, 34, 123–142.

  63. West, S.G., Aiken, L.S., Wu, W., & Taylor, A.B. (2007). Multiple regression: applications of the basics and beyond in personality research. In R. Robins, R.C. Fraley, & R.F. Krueger (Eds.), Handbook of research methods in personality psychology (pp. 573–601). New York: Guilford.

  64. Wise, L.L., McLaughlin, D.H., & Steel, L. (1979). The project TALENT data bank handbook. Palo Alto: American Institutes for Research.

  65. Yuan, K.H., & Chan, W. (2011). Biases and standard errors of standardized regression coefficients. Psychometrika, 76, 670–690.

Download references

Author information

Correspondence to Niels G. Waller.

Electronic Supplementary Material

Below are the links to the electronic supplementary material.

(PDF 44 kB)

(PDF 133 kB)

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Jones, J.A., Waller, N.G. The Normal-Theory and Asymptotic Distribution-Free (ADF) Covariance Matrix of Standardized Regression Coefficients: Theoretical Extensions and Finite Sample Behavior. Psychometrika 80, 365–378 (2015). https://doi.org/10.1007/s11336-013-9380-y

Download citation

Key words

  • standardized regression coefficients
  • multiple regression
  • ADF, confidence intervals