On the estimation of prediction errors in linear regression models

Abstract

Estimating the prediction error is a common practice in the statistical literature. Under a linear regression model, lete be the conditional prediction error andê be its estimate. We use ρ(ê, e), the correlation coefficient betweene andê, to measure the performance of a particular estimation method. Reasons are given why correlation is chosen over the more popular mean squared error loss. The main results of this paper conclude that it is generally not possible to obtain good estimates of the prediction error. In particular, we show that ρ(ê, e)=O(n ^−1/2) whenn → ∞. When the sample size is small, we argue that high values of ρ(ê, e) can be achieved only when the residual error distribution has very heavy tails and when no outlier presents in the data. Finally, we show that in order for ρ(ê, e) to be bounded away from zero asymptotically,ê has to be biased.