Bias-Variance Trade-offs: Novel Applications

Definition

Consider a given random variable $\underline{F}$ and a random variable that we can modify, $\hat{\underline{F}}$. We wish to use a sample of $\hat{\underline{F}}$ as an estimate of a sample of $\underline{F}$. The mean squared error (MSE) between such a pair of samples is a sum of four terms. The first term reflects the statistical coupling between $\underline{F}$ and $\hat{\underline{F}}$ and is conventionally ignored in bias-variance analysis. The second term reflects the inherent noise in $\underline{F}$ and is independent of the estimator $\hat{\underline{F}}$. Accordingly, we cannot affect this term. In contrast, the third and fourth terms depend on $\hat{\underline{F}}$. The third term, called the bias, is independent of the precise samples of both $\underline{F}$ and $\hat{\underline{F}}$ and reflects the difference between the means of $\underline{F}$ and $\hat{\underline{F}}$. The fourth term, called the variance, is independent of the...