Advertisement

Behavior Research Methods

, Volume 39, Issue 4, pp 709–722 | Cite as

Using heteroskedasticity-consistent standard error estimators in OLS regression: An introduction and software implementation

  • Andrew F. HayesEmail author
  • Li Cai
Article

Abstract

Homoskedasticity is an important assumption in ordinary least squares (OLS) regression. Although the estimator of the regression parameters in OLS regression is unbiased when the homoskedasticity assumption is violated, the estimator of the covariance matrix of the parameter estimates can be biased and inconsistent under heteroskedasticity, which can produce significance tests and confidence intervals that can be liberal or conservative. After a brief description of heteroskedasticity and its effects on inference in OLS regression, we discuss a family of heteroskedasticity-consistent standard error estimators for OLS regression and argue investigators should routinely use one of these estimators when conducting hypothesis tests using OLS regression. To facilitate the adoption of this recommendation, we provide easy-to-use SPSS and SAS macros to implement the procedures discussed here.

Keywords

Ordinary Little Square Grade Point Average Weight Little Square Ordinary Little Square Regression Generalize Little Square 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

References

  1. Bera, A. K., Suprayitno, T., &Premaratne, G. (2002). On some heteroskedasticity-robust estimators of variance-covariance matrix of the least-squares estimators.Journal of Statistical Planning & Inference,108, 121–136.CrossRefGoogle Scholar
  2. Berry, W. D. (1993).Understanding regression assumptions. Newbury Park, CA: Sage.Google Scholar
  3. Box, G. E. P., &Cox, D. R. (1964). An analysis of transformations.Journal of the Royal Statistical Society B,26, 211–243.Google Scholar
  4. Breusch, T. S., &Pagan, A. R. (1979). A simple test for heteroskedasticity and random coefficient variation.Econometrica,47, 1287–1294.CrossRefGoogle Scholar
  5. Cai, L., & Hayes, A. F. (in press). A new test of linear hypotheses in OLS regression under heteroskedasticity of unknown form.Journal of Educational & Behavioral Statistics.Google Scholar
  6. Carroll, R. J. (2003). Variances are not always nuisance parameters.Biometrics,59, 211–220.CrossRefPubMedGoogle Scholar
  7. Carroll, R. J., &Ruppert, D. (1988).Transformation and weighting in regression. New York: Chapman and Hall.Google Scholar
  8. Chesher, A., &Jewitt, I. (1987). The bias of a heteroskedasticity consistent covariance matrix estimator.Econometrica,55, 1217–1222.CrossRefGoogle Scholar
  9. Cook, R. D., &Weisberg, S. (1983). Diagnostics for heteroskedasticity in regression.Biometrika,70, 1–10.CrossRefGoogle Scholar
  10. Cook, R. D., &Weisberg, S. (1999).Applied regression including computing and graphics. New York: Wiley.CrossRefGoogle Scholar
  11. Cribari-Neto, F. (2004). Asymptotic inference under heteroskedasticity of unknown form.Computational Statistics & Data Analysis,45, 215–233.CrossRefGoogle Scholar
  12. Cribari-Neto, F., Ferrari, S. L. P., &Cordeiro, G. M. (2000). Improved heteroskedasticity-consistent covariance matrix estimators.Biometrika,87, 907–918.CrossRefGoogle Scholar
  13. Cribari-Neto, F., Ferrari, S. L. P., &Oliveira, W. A. S. C. (2005). Numerical evaluation of tests based on different heteroskedasticity-consistent covariance matrix estimators.Journal of Statistical Computation & Simulation,75, 611–628.CrossRefGoogle Scholar
  14. Cribari-Neto, F., &Zarkos, S. G. (2001). Heteroskedasticity-consistent covariance matrix estimation: White’s estimator and the bootstrap.Journal of Statistical Computation & Simulation,68, 391–411.CrossRefGoogle Scholar
  15. Darlington, R. B. (1990).Regression and linear models. New York: McGraw-Hill.Google Scholar
  16. Davidson, R., &MacKinnon, J. G. (1993).Estimation and inference in econometrics. Oxford: Oxford University Press.Google Scholar
  17. Downs, G. W., &Rocke, D. M. (1979). Interpreting heteroskedasticity.American Journal of Political Science,23, 816–828.CrossRefGoogle Scholar
  18. Draper, N. R., &Smith, H. (1981).Applied regression analysis (2nd ed.). New York: Wiley.Google Scholar
  19. Duncan, G. T., &Layard, W. M. J. (1973). A Monte-Carlo study of asymptotically robust tests for correlation coefficients.Biometrika,60, 551–558.CrossRefGoogle Scholar
  20. Edgell, S. E., &Noon, S. M. (1984). Effect of violation of normality on the t test of the correlation coefficient.Psychological Bulletin,95, 576–583.CrossRefGoogle Scholar
  21. Eicker, F. (1963). Asymptotic normality and consistency of the least squares estimator for families of linear regression.Annals of Mathematical Statistics,34, 447–456.CrossRefGoogle Scholar
  22. Eicker, F. (1967). Limit theorems for regression with unequal and dependent errors. In L. M. Le Cam & J. Neyman (Eds.),Proceedings of the fifth Berkeley symposium on mathematical statistics and probability. Berkeley, CA: University of California Press.Google Scholar
  23. Furno, M. (1996). Small sample behavior of a robust heteroskedasticity consistent covariance matrix estimator.Journal of Statistical Computation & Simulation,54, 115–128.CrossRefGoogle Scholar
  24. Godfrey, L. G. (2006). Tests for regression models with heteroskedasticity of unknown form.Computational Statistics & Data Analysis,50, 2715–2733.CrossRefGoogle Scholar
  25. Godfrey, L. G., &Orne, C. D. (2004). Controlling the finite sample significance levels of heteroskedasticity-robust tests of several linear restrictions on regression coefficients.Economics Letters,82, 281–287.CrossRefGoogle Scholar
  26. Goldfeld, S. M., &Quandt, R. E. (1965). Some tests for homoskedasticity.Journal of the American Statistical Association,60, 539–547.CrossRefGoogle Scholar
  27. Hayes, A. F. (1996). Permutation test is not distribution-free: TestingHo: π=0.Psychological Methods,1, 184–198.CrossRefGoogle Scholar
  28. Hinkley, D. V. (1977). Jackknifing in unbalanced situations.Technometrics,19, 285–292.CrossRefGoogle Scholar
  29. Huber, P. J. (1967). The behavior of maximum likelihood estimation under nonstandard conditions. In L. M. Le Cam & J. Neyman (Eds.),Proceedings of the fifth Berkeley symposium on mathematical statistics and probability. Berkeley, CA: University of California Press.Google Scholar
  30. Kauermann, G., &Carroll, R. J. (2001). A note on the efficiency of sandwich covariance matrix estimation.Journal of the American Statistical Association,96, 1387–1396.CrossRefGoogle Scholar
  31. Kowalski, C. J. (1973). On the effects of non-normality on the distribution of the sample product-moment correlation coefficient.Applied Statistics,21, 1–12.CrossRefGoogle Scholar
  32. Long, J. S., &Ervin, L. H. (2000). Using heteroskedasticity consistent standard errors in the linear regression model.American Statistician,54, 217–224.CrossRefGoogle Scholar
  33. MacCallum, R. C. (2003). Working with imperfect models.Multivariate Behavioral Research,38, 113–139.CrossRefGoogle Scholar
  34. MacKinnon, J. G., &White, H. (1985). Some heteroskedasticity-consistent covariance matrix estimators with improved finite sample properties.Journal of Econometrics,29, 305–325.CrossRefGoogle Scholar
  35. Mardia, K. V., Kent, J. T., &Bibby, J. M. (1979).Multivariate analysis. New York: Academic Press.Google Scholar
  36. Moore, D. S., &McCabe, G. P. (2003).Introduction to the practice of statistics (4th ed.). New York: Freeman.Google Scholar
  37. Olejnik, S. F. (1988). Variance heterogeneity: An outcome to explain or a nuisance factor to control?Journal of Experimental Education,56, 193–197.Google Scholar
  38. Perry, D. K. (1986). Looking for heteroskedasticity: A means of searching for neglected conditional relationships. In M. L. McLaughlin (Ed.),Communication yearbook 9 (pp. 658–670). Beverly Hills, CA: Sage.Google Scholar
  39. Rasmussen, J. L. (1989). Computer-intensive correlational analysis: Bootstrap and approximate randomization techniques.British Journal of Mathematical & Statistical Psychology,42, 103–111.Google Scholar
  40. Sudmant, W., &Kennedy, P. (1990). On inference in the presence of heteroskedasticity without replicated observations.Communication in Statistics-Simulation & Computation,19, 491–504.CrossRefGoogle Scholar
  41. Weisberg, S. (1980).Applied linear regression. New York: Wiley.Google Scholar
  42. White, H. (1980). A heteroskedasticity-consistent covariance matrix estimator and a direct test for heteroskedasticity.Econometrica,48, 817–838.CrossRefGoogle Scholar
  43. Wilcox, R. R. (2001). Comment.The American Statistician,55, 374–375.Google Scholar
  44. Wilcox, R. R. (2005).Introduction to robust estimation and hypothesis testing. New York: Academic Press.Google Scholar
  45. Wooldridge, J. M. (2000).Introductory econometrics: A modern approach. Cincinnati, OH: South-Western College Publishing.Google Scholar
  46. Wu, C. F. J. (1986). Jackknife bootstrap and other resampling methods in regression analysis.Annals of Statistics,14, 1261–1295.CrossRefGoogle Scholar

Copyright information

© Psychonomic Society, Inc. 2007

Authors and Affiliations

  1. 1.School of CommunicationOhio State UniversityColumbus
  2. 2.University of North CarolinaChapel Hill

Personalised recommendations