Simple Linear Regression

Abstract

In this chapter, we study extensively the estimation of a linear relationship between two variables, Y_i and X_i, of the form:

$$\displaystyle Y_{i}=\alpha +\beta X_{i}+u_{i}\quad i=1,2,\ldots ,n $$

where Y_i denotes the i-th observation on the dependent variable Y  which could be consumption, investment, or output, and X_i denotes the i -th observation on the independent variable X which could be disposable income, the interest rate, or an input.

Appendix: Centered and Uncentered R2

From the OLS regression on (3.1) we get

$$\displaystyle \begin{aligned} Y_{i}=\widehat{Y}_{i}+e_{i}\quad i=1,2,\ldots ,n {} \end{aligned} $$

(A.1)

where $\widehat {Y}_{i}=\widehat {\alpha } _{OLS}+X_{i}\widehat {\beta }_{OLS}$. Squaring and summing the above equation, we get

(A.2)

since . The uncentered R² is given by

(A.3)

Note that the total sum of squares for Y_i is not expressed in deviation from the sample mean $\bar {Y}$.

In other words, the uncentered R² is the proportion of variation of that is explained by the regression Y  on X. Regression packages usually report the centered R² which was defined in Sect. 3.6 as where $y_{i}=Y_{i}-\bar {Y}.$ The latter measure focuses on explaining the variation in Y_i after fitting the constant.

From Sect. 3.6, we have seen that a naive model with only a constant in it gives $\bar {Y}$ as the estimate of the constant, see also Problem 2. The variation in Y_i that is not explained by this naive model is . Subtracting $n \bar {Y}^{2}$ from both sides of (A.2) we get

and the centered R² is

(A.4)

If there is a constant in the model $\bar {Y}= \overline {\widehat {Y}}$, see Sect. 3.6, and . Therefore, the centered which is the R² reported by regression packages. If there is no constant in the model, some regression packages give you the option of (no constant) and the R² reported is usually the uncentered R². Check your regression package documentation to verify what you are getting. We will encounter uncentered R² again in constructing test statistics using regressions, see for example Chap. 11.