Forecast combination when outcomes are difficult to predict

Abstract

We show that when outcomes are difficult to forecast in the sense that forecast errors have a large common component that (a) optimal weights are not affected by this common component, and may well be far from equal to each other but (b) the relative mean square error loss from averaging over optimal combination can be small. Hence, researchers could well estimate combining weights that indicate that correlations could be exploited for better forecasts only to find that the difference in terms of loss is negligible. The results then provide an additional explanation for the commonly encountered practical situation of the averaging of forecasts being difficult to improve upon.

Appendix

Proof

(Proposition 1) We generalize slightly from the proposition, where above $b=1.$ From matrix inverse laws

$$\begin{aligned} \Sigma ^{-1}= & {} \left( a\iota \iota ^{\prime }+b\tilde{\Sigma }\right) ^{-1}\\= & {} b^{-1}\left( \tilde{\Sigma }^{-1}-\frac{ab^{-1}}{1+ab^{-1}(\iota ^{\prime } \tilde{\Sigma }^{-1}\iota )}\tilde{\Sigma }^{-1}\iota \iota ^{\prime }\tilde{ \Sigma }^{-1}\right) . \end{aligned}$$

By direct calculation, we have

$$\begin{aligned} \iota ^{\prime }\Sigma ^{-1}\iota =b^{-1}(\iota ^{\prime }\tilde{\Sigma }^{-1}\iota )\left( \frac{1}{1+ab^{-1}(\iota ^{\prime }\tilde{\Sigma }^{-1}\iota )}\right) \end{aligned}$$

and

$$\begin{aligned} \Sigma ^{-1}\iota =b^{-1}(\tilde{\Sigma }^{-1}\iota )\left( \frac{1}{ 1+ab^{-1}(\iota ^{\prime }\tilde{\Sigma }^{-1}\iota )}\right) \end{aligned}$$

so

$$\begin{aligned} \omega ^{opt}= & {} (\iota ^{\prime }\Sigma ^{-1}\iota )^{-1}\Sigma ^{-1}\iota \\= & {} \frac{b^{-1}(\tilde{\Sigma }^{-1}\iota )\left( \frac{1}{1+ab^{-1}(\iota ^{\prime }\tilde{\Sigma }^{-1}\iota )}\right) }{b^{-1}(\iota ^{\prime }\tilde{ \Sigma }^{-1}\iota )\left( \frac{1}{1+ab^{-1}(\iota ^{\prime }\tilde{\Sigma } ^{-1}\iota )}\right) } \\= & {} (\iota ^{\prime }\tilde{\Sigma }^{-1}\iota )^{-1}\tilde{\Sigma }^{-1}\iota \end{aligned}$$

yielding the result. $\square$

Proof

(Proposition 2). When the optimal weights are equivalent to the averaging weights, i.e. $(\iota ^{\prime }\Sigma \iota )^{-1}\iota ^{\prime }\Sigma =m^{-1}\iota ^{\prime }$, then $rl=0$ and is independent of $\sigma _{\varepsilon }^{2}.$ If not, we have by taking derivatives with respect to $\sigma _{\varepsilon }^{2}$ that

$$\begin{aligned} \frac{\partial rl}{\partial \sigma _{\varepsilon }^{2}}= & {} \frac{1}{\sigma _{\varepsilon }^{2}+\left( \iota ^{\prime }\tilde{\Sigma }^{-1}\iota \right) ^{-1}}-\frac{\sigma _{\varepsilon }^{2}+m^{-2}(\iota ^{\prime }\tilde{\Sigma } \iota )}{\left[ \sigma _{\varepsilon }^{2}+\left( \iota ^{\prime }\tilde{ \Sigma }^{-1}\iota \right) ^{-1}\right] ^{2}} \\= & {} \frac{\left( \iota ^{\prime }\tilde{\Sigma }^{-1}\iota \right) ^{-1}-m^{-2}(\iota ^{\prime }\tilde{\Sigma }\iota )}{\left[ \sigma _{\varepsilon }^{2}+\left( \iota ^{\prime }\tilde{\Sigma }^{-1}\iota \right) ^{-1}\right] ^{2}} \\\le & {} 0 \end{aligned}$$

and the equality sign only holds in the case we are in the situation of (a). $\square$