Problems with Explanatory Variables: Random Variables, Collinearity, and Instability

Abstract

The multiple regression model supposes that the explanatory variables are (i) independent of the error term and (ii) are linearly independent. This chapter looks at what happens when these assumptions do not hold. If the first assumption is violated, the implication is that the explanatory variables are dependent on the error term. Under these conditions, the ordinary least squares estimators are no longer consistent, and it is necessary to use another estimator called the instrumental variables estimator. The consequence of violating the second assumption is that the explanatory variables are not linearly independent. In other words, they are collinear. Finally, the chapter concentrates on the third problem related to the explanatory variables, namely, the question of the stability of the estimated model.

Appendix: Demonstration of the Formula for Constrained Least Squares Estimators

In order to determine the constrained least squares estimator, we need to solve a minimization program of sum of squared residuals:

$$\displaystyle \begin{aligned} Min\left( \boldsymbol{Y}-\boldsymbol{X\hat{\beta}}_{0}\right)^{\prime}\left( \boldsymbol{Y}-\boldsymbol{X\hat{\beta}}_{0}\right) =Min\left( \boldsymbol{e}^{\prime }\boldsymbol{e}\right)\end{aligned} $$

(5.119)

under the constraint: $\boldsymbol {R\hat {\beta }}_{0}=\boldsymbol {r}$.

We define the Lagrange function:

$$\displaystyle \begin{aligned} \mathcal{L} =\left( \boldsymbol{Y}-\boldsymbol{X\hat{\beta}}_{0}\right)^{\prime}\left( \boldsymbol{Y}-\boldsymbol{X\hat{\beta}}_{0}\right) -2\boldsymbol{\lambda}^{\prime }\left( \boldsymbol{R\hat{\beta}}_{0}-\boldsymbol{r}\right)\end{aligned} $$

(5.120)

where $\boldsymbol {\lambda }$ is a column vector formed by the q Lagrange multipliers. We calculate the partial derivatives:

$$\displaystyle \begin{aligned} \frac{\partial \mathcal{L} }{\partial\hat{\boldsymbol{\beta}}_{0}}=-2\boldsymbol{X}^{\prime}\boldsymbol{Y} +2\boldsymbol{X}^{\prime}\boldsymbol{X\hat{\beta}}_{0}-2\boldsymbol{R}^{\prime }\boldsymbol{\lambda} \end{aligned} $$

(5.121)

and:

$$\displaystyle \begin{aligned} \frac{\partial \mathcal{L} }{\partial\boldsymbol{\lambda}}=-2\left( \boldsymbol{R\hat{\beta}}_{0}-\boldsymbol{r} \right)\end{aligned} $$

(5.122)

Canceling these partial derivatives, we have:

$$\displaystyle \begin{aligned} \boldsymbol{X}^{\prime}\boldsymbol{X\hat{\beta}}_{0}-\boldsymbol{X}^{\prime} \boldsymbol{Y}-\boldsymbol{R}^{\prime}\boldsymbol{\lambda}=0{} \end{aligned} $$

(5.123)

and:

$$\displaystyle \begin{aligned} \boldsymbol{R\hat{\beta}}_{0}-\boldsymbol{r}=0\end{aligned} $$

(5.124)

Let us multiply each member of (5.123) by $\boldsymbol {R}\left ( \boldsymbol {X}^{\prime }\boldsymbol {X}\right )^{-1}$:

$$\displaystyle \begin{aligned} \boldsymbol{R\hat{\beta}}_{0}-\boldsymbol{R}\left( \boldsymbol{X}^{\prime}\boldsymbol{X} \right)^{-1}\boldsymbol{X}^{\prime}\boldsymbol{Y}-\boldsymbol{R}\left( \boldsymbol{X} ^{\prime}\boldsymbol{X}\right)^{-1}\boldsymbol{R}^{\prime}\boldsymbol{\lambda}=0\end{aligned} $$

(5.125)

Hence:

$$\displaystyle \begin{aligned} \boldsymbol{\lambda}=\left( \boldsymbol{R}\left( \boldsymbol{X}^{\prime}\boldsymbol{X} \right)^{-1}\boldsymbol{R}^{\prime}\right)^{-1}\left( \boldsymbol{r} -\boldsymbol{R\hat{\beta}}\right)\end{aligned} $$

(5.126)

with $\hat {\boldsymbol {\beta }}=\left ( \boldsymbol {X}^{\prime }\boldsymbol {X}\right )^{-1}\boldsymbol {X}^{\prime }\boldsymbol {Y}$ denoting the OLS estimator of the unconstrained model. It is then sufficient to replace $\boldsymbol {\lambda }$ by its value in (5.123):

$$\displaystyle \begin{aligned} \hat{\boldsymbol{\beta}}_{0}=\left( \boldsymbol{X}^{\prime}\boldsymbol{X}\right)^{-1}\boldsymbol{X}^{\prime}\boldsymbol{Y}+\left( \boldsymbol{X}^{\prime}\boldsymbol{X} \right)^{-1}\boldsymbol{R}^{\prime}\left( \boldsymbol{R}\left( \boldsymbol{X}^{\prime }\boldsymbol{X}\right)^{-1}\boldsymbol{R}^{\prime}\right)^{-1}\left( \boldsymbol{r}-\boldsymbol{R\hat{\beta}}\right)\end{aligned} $$

(5.127)

Hence:

$$\displaystyle \begin{aligned} \hat{\boldsymbol{\beta}}_{0}=\hat{\boldsymbol{\beta}}+\left( \boldsymbol{X}^{\prime }\boldsymbol{X}\right)^{-1}\boldsymbol{R}^{\prime}\left( \boldsymbol{R}\left( \boldsymbol{X}^{\prime}\boldsymbol{X}\right)^{-1}\boldsymbol{R}^{\prime}\right)^{-1}\left( \boldsymbol{r}-\boldsymbol{R\hat{\beta}}\right)\end{aligned} $$

(5.128)

which defines the constrained least squares estimator.