Clustering and Standard Error Bias in Fixed Effects Panel Data Regressions

Abstract

Objective

We address several issues concerning standard error bias in pooled time-series cross-section regressions. These include autocorrelation, problems with unit root tests, nonstationarity in levels regressions, and problems with clustered standard errors.

Methods

We conduct unit root test for crimes and other variables. We use Monte Carlo procedures to illustrate the standard error biases caused by the above issues in pooled studies. We replicate prior research that uses clustered standard errors with difference-in-differences regressions and only a small number of policy changes.

Results

Standard error biases in the presence of autocorrelation are substantial when standard errors are not clustered. Importantly, clustering greatly mitigates bias resulting from the use of nonstationary variables in levels regressions, although in some circumstances clustering can fail to correct for standard error biases due to other reasons. The “small number of policy changes” problem can cause extreme standard error bias, but this can be corrected using “placebo laws”. Other biases are caused by weighting regressions, having too few units, and having dissimilar autocorrelation coefficients across units.

Conclusions

With clustering, researchers can usually conduct regressions in levels even with nonstationary variables. They should, however, be leery of pitfalls caused by clustering, especially when conducting difference-in-differences analyses.

Appendix: Fixed Effects Regression and Monte Carlo

The fixed effects (FE) regression model has the advantage of being unbiased in the presence of unobserved heterogeneity. That is, if each state has long-run, permanent features that are correlated both with the dependent variable and the independent variables in the model, then any regression procedure, such as the random effects model or the pooled ordinary least squares model, that uses variation across states will be biased and inconsistent. This is very likely to be the case in criminology since Massachusetts, for example, is permanently different from Louisiana because of history, culture, climate, and a number of other dimensions. The same can be said for Arizona and Vermont, Hawaii and any other state.

The FE model has the form²⁷

$$y_{it} = \alpha_{i} + \sum\limits_{k = 1}^{K} {\beta_{k} } x_{k,it} + u_{it}$$

(6)

where i = 1, …, N, t = 1, …, T, X_k,it is the value of the kth regressor for state i in year t, α_i are state-specific fixed effects, and γ_t are year-specific fixed effects. This model requires four assumptions (assuming one regressor for simplicity):

$$E(u_{it} |x_{i1} , \ldots ,x_{iT} ,\alpha_{i} ) = 0$$

(7)

that is, the conditional value of the error term is zero, given the value of the regressor(s);

$$(x_{i1} , \ldots ,x_{iT} ,y_{i1} , \ldots ,y_{iT} ),\quad i = 1, \ldots ,N\,{\text{are}}\,{\text{iid}}\,{\text{draws}}\,{\text{from}}\,{\text{their}}\,{\text{joint}}\,{\text{distribution}};$$

(8)

that is, the variable(s) over all the years in one state are distributed identically but independent of the same variable(s) over the same time span in other states;

$$\left( {x_{it} ,u_{it} } \right)$$

(9)

have nonzero finite fourth moments; this assumption is important for asymptotic results, it limits the probability of observing extreme values of the regressor(s) or the errors;

$$\text{cov} (u_{it} ,u_{is} |x_{it} ,x_{is} ) = 0\quad {\text{for}}\quad s \ne t;$$

(10)

the errors are uncorrelated over time, conditional on the regressor(s).

With these assumptions we can estimate the fixed-effects model, generating unbiased and consistent estimates. (The FE model is not efficient because it ignores cross-section variation.) The model is estimated by applying ordinary least squares to the demeaned variable(s), e.g. $\bar{y}_{i} = (1/T)\sum\nolimits_{t = 1}^{T} {y_{it} } ,\;\bar{x}_{i} = (1/T)\sum\nolimits_{t = 1}^{T} {x_{it} } ,\bar{u}_{i} = (1/T)\sum\nolimits_{t = 1}^{T} {u_{it} }$. The cross-section equation is

$$\bar{y}_{i} = \alpha_{i} + \beta \bar{x}_{i} + \bar{u}_{i}$$

(11)

Subtracting the cross-section equation from (6), still assuming only one regressor, yields

$$y_{it} - \bar{y}_{i} = \alpha_{i} + \beta x_{it} + u_{it} - \alpha_{i} - \beta \bar{x}_{i} - \bar{u}_{i} = \beta (x_{it} - \bar{x}_{i} ) + u_{it} - \bar{u}_{i}$$

(12)

The fixed effects have been “swept out” or “absorbed.” We can write this equation as

$$\tilde{y}_{it} = \beta \tilde{x}_{it} + \tilde{u}_{it}$$

(13)

The fixed-effects estimator is just OLS applied to (13). This estimator is also known as the “within” estimator because it only uses variation within each state.

$$\hat{\beta }_{FE} = {{\sum\limits_{i = 1}^{N} {\sum\limits_{t = 1}^{T} {\tilde{x}_{it} \tilde{y}_{it} } } } \mathord{\left/ {\vphantom {{\sum\limits_{i = 1}^{N} {\sum\limits_{t = 1}^{T} {\tilde{x}_{it} \tilde{y}_{it} } } } {\sum\limits_{i = 1}^{N} {\sum\limits_{t = 1}^{T} {\tilde{x}_{it}^{2} } } }}} \right. \kern-0pt} {\sum\limits_{i = 1}^{N} {\sum\limits_{t = 1}^{T} {\tilde{x}_{it}^{2} } } }}$$

(14)

Identical results can be achieved by regressing y on x and a set of dummy variables, one for each state such that the dummy for state i is one if state = state i, otherwise zero. This is the “least squares dummy variable” or LSDV model.

Generalizing to multiple regression, define the matrix of observations on the demeaned.

Regressor for state i as $\tilde{X}_{i}$ so that

$$\hat{\beta }_{FE} = \left( {\sum\limits_{i = 1}^{N} {\tilde{X}_{i}^{{\prime }} \tilde{X}_{i} } } \right)^{ - 1} \left( {\sum\limits_{i = 1}^{N} {\tilde{X}_{ii}^{{\prime }} \tilde{y}_{i} } } \right)$$

The asymptotic variance–covariance matrix is

$$AVAR(\hat{\beta }_{FE} ) = \hat{\sigma }_{u}^{2} \left( {\sum\limits_{i = 1}^{N} {\tilde{X}_{i}^{{\prime }} \tilde{X}_{i} } } \right)^{ - 1}$$

where $\hat{\sigma }_{u}^{2} = {{\sum\nolimits_{i = 1}^{N} {\sum\nolimits_{t = 1}^{T} {\hat{u}_{it}^{2} } } } \mathord{\left/ {\vphantom {{\sum\nolimits_{i = 1}^{N} {\sum\nolimits_{t = 1}^{T} {\hat{u}_{it}^{2} } } } {(N(T - 1) - K)}}} \right. \kern-0pt} {(N(T - 1) - K)}}$ is a consistent estimator of the error variance and

$$\hat{u}_{i} = \tilde{y}_{i} - \tilde{X}_{i} \hat{\beta }_{FE} ,\quad {\text{i}} = 1, \ldots ,{\text{N}}.$$

The square roots of the principal diagonal of the AVAR matrix are the standard errors.

Clustered Standard Errors

The clustered asymptotic variance–covariance matrix (Arellano 1987) is a modified sandwich estimator (White 1984, Chapter 6):

$$\hat{V}(\hat{\beta }_{FE} ) = \left( {\tilde{X}^{\prime}\tilde{X}} \right)^{ - 1} \left( {\sum\limits_{i = 1}^{N} {\tilde{X}^{\prime}_{i} } \hat{u}_{i} \hat{u}_{i}^{\prime } \tilde{X}_{i} } \right)\left( {\tilde{X}^{\prime}\tilde{X}} \right)^{ - 1}$$

The “meat” of the sandwich contains the estimated covariances among the error terms. The residuals are “clustered” in the sense that only covariances from state i are used, covariances from other states are ignored. The formula therefore corrects for heteroskedasticity (using the squared terms) and autocorrelation using the remaining terms.

Bias with Lagged Dependent Variables

A problem particular to TSCS regression is potential bias when a lagged dependent variable is included in the list of regressors. This variable is correlated with the error term, which biases its coefficient (Nickell 1981). The order of the bias is 1/T, and the bias declines relatively rapidly with more years. Judson and Owen (1999) show that, for T of 30 or more, fixed effects TSCS performs as well or better than other methods, such as generalized methods of moments. For T = 20 the bias is roughly 20% for the coefficient on the lagged dependent variable. The impact on the lagged dependent variable affects the coefficients and standard errors on other independent variables if these are correlated with the lagged dependent variable. In practice, the researcher does not know the direction and extent of this bias. We do not encounter the Nichols problem in our simulations because we set the coefficient on “x” to be zero, thereby dropping it from the regression and assuring that its coefficient remains unbiased.

DFGLS Unit Root Test

Elliott et al. (1996) propose the following two-step procedure.

If the data has a trend:

• 1. Detrend by estimating the following regression using OLS.

  $$(y_{t} - ay_{t - 1} ) = \alpha (1 - a) + \delta (t - a(t - 1)) + v_{t}$$

ERS do Monte Carlo experiments to determine the optimal value of a:

$$a = 1 - \frac{13.5}{T}$$

Note that $a \to 1$ as T goes to infinity (local to unity).

• 2. Compute the (detrended) residuals $e = \hat{v}_{t}$ and estimate the usual ADF test equation.

  $$\Delta e_{t} = (\rho - 1)e_{t - 1} + \sum\limits_{j = 1}^{p} {\gamma_{j} } \Delta e_{t - j} + v_{t}$$

Use the modified AIC, to choose the lag length, p. Test using the standard t-ratio on the lagged level, taking the critical values from the ERS tables.

If the data does not have a trend:

• 1. Estimate the constant only model (demeaned not detrended).

  $$(y_{t} - ay_{t - 1} ) = \alpha (1 - a) + v_{t} \;{\text{where}}\;a = 1 - \frac{7}{T}$$

• 2. Compute the residuals, $\tilde{y}_{t}$ and estimate the same ADF test.

  $$\Delta \tilde{y}_{t} = (\rho - 1)\tilde{y}_{t - 1} + \sum\limits_{j = 1}^{p} {\gamma_{j} } \Delta \tilde{y}_{t - j} + v_{t}$$

Monte Carlo Program

The model is generated as follows. The researcher specifies the number of states with nonstationary dependent variables, n_y, and the number of states with nonstationary independent variables, n_x. In each case, the remaining states series are stationary with an autocorrelation coefficient randomly chosen with values between 0.80 and 0.99.

The state fixed effects ($a_{i}$,$b_{i}$) are drawn from a uniform distribution with range [0, 100] and $\sigma_{\varepsilon }^{2}$ = $\sigma_{\nu }^{2}$ = 1. The estimated fixed-effects model is

$$y_{it} = \alpha_{i} + \beta x_{it} + u_{it}$$

We ran 10,000 regressions of this model for each variation of years and numbers of nonstationary dependent and independent variables. The results are presented in Table A in the online appendix, along with the Stata do-files. We also prepared a lengthy table with critical values for clustered regressions comparable to Table A, but giving the critical values from 0.10 to 0.01 as well as 0.005 (Table B).