Skip to main content
Download PDF

Using heteroskedasticity-consistent standard error estimators in OLS regression: An introduction and software implementation

Behavior Research Methods volume 39, pages 709–722 (2007)Cite this article

Abstract

Homoskedasticity is an important assumption in ordinary least squares (OLS) regression. Although the estimator of the regression parameters in OLS regression is unbiased when the homoskedasticity assumption is violated, the estimator of the covariance matrix of the parameter estimates can be biased and inconsistent under heteroskedasticity, which can produce significance tests and confidence intervals that can be liberal or conservative. After a brief description of heteroskedasticity and its effects on inference in OLS regression, we discuss a family of heteroskedasticity-consistent standard error estimators for OLS regression and argue investigators should routinely use one of these estimators when conducting hypothesis tests using OLS regression. To facilitate the adoption of this recommendation, we provide easy-to-use SPSS and SAS macros to implement the procedures discussed here.

References

  • Bera, A. K., Suprayitno, T., &Premaratne, G. (2002). On some heteroskedasticity-robust estimators of variance-covariance matrix of the least-squares estimators.Journal of Statistical Planning & Inference,108, 121–136.

    Article  Google Scholar 

  • Berry, W. D. (1993).Understanding regression assumptions. Newbury Park, CA: Sage.

    Google Scholar 

  • Box, G. E. P., &Cox, D. R. (1964). An analysis of transformations.Journal of the Royal Statistical Society B,26, 211–243.

    Google Scholar 

  • Breusch, T. S., &Pagan, A. R. (1979). A simple test for heteroskedasticity and random coefficient variation.Econometrica,47, 1287–1294.

    Article  Google Scholar 

  • Cai, L., & Hayes, A. F. (in press). A new test of linear hypotheses in OLS regression under heteroskedasticity of unknown form.Journal of Educational & Behavioral Statistics.

  • Carroll, R. J. (2003). Variances are not always nuisance parameters.Biometrics,59, 211–220.

    Article  PubMed  Google Scholar 

  • Carroll, R. J., &Ruppert, D. (1988).Transformation and weighting in regression. New York: Chapman and Hall.

    Google Scholar 

  • Chesher, A., &Jewitt, I. (1987). The bias of a heteroskedasticity consistent covariance matrix estimator.Econometrica,55, 1217–1222.

    Article  Google Scholar 

  • Cook, R. D., &Weisberg, S. (1983). Diagnostics for heteroskedasticity in regression.Biometrika,70, 1–10.

    Article  Google Scholar 

  • Cook, R. D., &Weisberg, S. (1999).Applied regression including computing and graphics. New York: Wiley.

    Book  Google Scholar 

  • Cribari-Neto, F. (2004). Asymptotic inference under heteroskedasticity of unknown form.Computational Statistics & Data Analysis,45, 215–233.

    Article  Google Scholar 

  • Cribari-Neto, F., Ferrari, S. L. P., &Cordeiro, G. M. (2000). Improved heteroskedasticity-consistent covariance matrix estimators.Biometrika,87, 907–918.

    Article  Google Scholar 

  • Cribari-Neto, F., Ferrari, S. L. P., &Oliveira, W. A. S. C. (2005). Numerical evaluation of tests based on different heteroskedasticity-consistent covariance matrix estimators.Journal of Statistical Computation & Simulation,75, 611–628.

    Article  Google Scholar 

  • Cribari-Neto, F., &Zarkos, S. G. (2001). Heteroskedasticity-consistent covariance matrix estimation: White’s estimator and the bootstrap.Journal of Statistical Computation & Simulation,68, 391–411.

    Article  Google Scholar 

  • Darlington, R. B. (1990).Regression and linear models. New York: McGraw-Hill.

    Google Scholar 

  • Davidson, R., &MacKinnon, J. G. (1993).Estimation and inference in econometrics. Oxford: Oxford University Press.

    Google Scholar 

  • Downs, G. W., &Rocke, D. M. (1979). Interpreting heteroskedasticity.American Journal of Political Science,23, 816–828.

    Article  Google Scholar 

  • Draper, N. R., &Smith, H. (1981).Applied regression analysis (2nd ed.). New York: Wiley.

    Google Scholar 

  • Duncan, G. T., &Layard, W. M. J. (1973). A Monte-Carlo study of asymptotically robust tests for correlation coefficients.Biometrika,60, 551–558.

    Article  Google Scholar 

  • Edgell, S. E., &Noon, S. M. (1984). Effect of violation of normality on the t test of the correlation coefficient.Psychological Bulletin,95, 576–583.

    Article  Google Scholar 

  • Eicker, F. (1963). Asymptotic normality and consistency of the least squares estimator for families of linear regression.Annals of Mathematical Statistics,34, 447–456.

    Article  Google Scholar 

  • Eicker, F. (1967). Limit theorems for regression with unequal and dependent errors. In L. M. Le Cam & J. Neyman (Eds.),Proceedings of the fifth Berkeley symposium on mathematical statistics and probability. Berkeley, CA: University of California Press.

    Google Scholar 

  • Furno, M. (1996). Small sample behavior of a robust heteroskedasticity consistent covariance matrix estimator.Journal of Statistical Computation & Simulation,54, 115–128.

    Article  Google Scholar 

  • Godfrey, L. G. (2006). Tests for regression models with heteroskedasticity of unknown form.Computational Statistics & Data Analysis,50, 2715–2733.

    Article  Google Scholar 

  • Godfrey, L. G., &Orne, C. D. (2004). Controlling the finite sample significance levels of heteroskedasticity-robust tests of several linear restrictions on regression coefficients.Economics Letters,82, 281–287.

    Article  Google Scholar 

  • Goldfeld, S. M., &Quandt, R. E. (1965). Some tests for homoskedasticity.Journal of the American Statistical Association,60, 539–547.

    Article  Google Scholar 

  • Hayes, A. F. (1996). Permutation test is not distribution-free: TestingHo: π=0.Psychological Methods,1, 184–198.

    Article  Google Scholar 

  • Hinkley, D. V. (1977). Jackknifing in unbalanced situations.Technometrics,19, 285–292.

    Article  Google Scholar 

  • Huber, P. J. (1967). The behavior of maximum likelihood estimation under nonstandard conditions. In L. M. Le Cam & J. Neyman (Eds.),Proceedings of the fifth Berkeley symposium on mathematical statistics and probability. Berkeley, CA: University of California Press.

    Google Scholar 

  • Kauermann, G., &Carroll, R. J. (2001). A note on the efficiency of sandwich covariance matrix estimation.Journal of the American Statistical Association,96, 1387–1396.

    Article  Google Scholar 

  • Kowalski, C. J. (1973). On the effects of non-normality on the distribution of the sample product-moment correlation coefficient.Applied Statistics,21, 1–12.

    Article  Google Scholar 

  • Long, J. S., &Ervin, L. H. (2000). Using heteroskedasticity consistent standard errors in the linear regression model.American Statistician,54, 217–224.

    Article  Google Scholar 

  • MacCallum, R. C. (2003). Working with imperfect models.Multivariate Behavioral Research,38, 113–139.

    Article  Google Scholar 

  • MacKinnon, J. G., &White, H. (1985). Some heteroskedasticity-consistent covariance matrix estimators with improved finite sample properties.Journal of Econometrics,29, 305–325.

    Article  Google Scholar 

  • Mardia, K. V., Kent, J. T., &Bibby, J. M. (1979).Multivariate analysis. New York: Academic Press.

    Google Scholar 

  • Moore, D. S., &McCabe, G. P. (2003).Introduction to the practice of statistics (4th ed.). New York: Freeman.

    Google Scholar 

  • Olejnik, S. F. (1988). Variance heterogeneity: An outcome to explain or a nuisance factor to control?Journal of Experimental Education,56, 193–197.

    Google Scholar 

  • Perry, D. K. (1986). Looking for heteroskedasticity: A means of searching for neglected conditional relationships. In M. L. McLaughlin (Ed.),Communication yearbook 9 (pp. 658–670). Beverly Hills, CA: Sage.

    Google Scholar 

  • Rasmussen, J. L. (1989). Computer-intensive correlational analysis: Bootstrap and approximate randomization techniques.British Journal of Mathematical & Statistical Psychology,42, 103–111.

    Google Scholar 

  • Sudmant, W., &Kennedy, P. (1990). On inference in the presence of heteroskedasticity without replicated observations.Communication in Statistics-Simulation & Computation,19, 491–504.

    Article  Google Scholar 

  • Weisberg, S. (1980).Applied linear regression. New York: Wiley.

    Google Scholar 

  • White, H. (1980). A heteroskedasticity-consistent covariance matrix estimator and a direct test for heteroskedasticity.Econometrica,48, 817–838.

    Article  Google Scholar 

  • Wilcox, R. R. (2001). Comment.The American Statistician,55, 374–375.

    Google Scholar 

  • Wilcox, R. R. (2005).Introduction to robust estimation and hypothesis testing. New York: Academic Press.

    Google Scholar 

  • Wooldridge, J. M. (2000).Introductory econometrics: A modern approach. Cincinnati, OH: South-Western College Publishing.

    Google Scholar 

  • Wu, C. F. J. (1986). Jackknife bootstrap and other resampling methods in regression analysis.Annals of Statistics,14, 1261–1295.

    Article  Google Scholar 

Download references

Author information

Affiliations

  1. School of Communication, Ohio State University, 154 N. Oval Mall, 43210, Columbus, OH

    Andrew F. Hayes

  2. University of North Carolina, Chapel Hill, North Carolina

    Li Cai

Authors
  1. Andrew F. Hayes
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Li Cai
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Andrew F. Hayes.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Hayes, A.F., Cai, L. Using heteroskedasticity-consistent standard error estimators in OLS regression: An introduction and software implementation. Behavior Research Methods 39, 709–722 (2007). https://doi.org/10.3758/BF03192961

Download citation

  • Received:

  • Accepted:

  • Issue Date:

  • DOI: https://doi.org/10.3758/BF03192961

Keywords

  • Ordinary Little Square
  • Grade Point Average
  • Weight Little Square
  • Ordinary Little Square Regression
  • Generalize Little Square