Abstract
The aim of this paper is to assess whether three well-known commodity-specific variables (basis, hedging pressure, and momentum) may improve the predictive power for commodity futures returns of models otherwise based on macroeconomic factors. We compute recursive, out-of-sample forecasts for the monthly returns of fifteen commodity futures, when estimation is based on a stepwise model selection approach under a probability-weighted regime-switching regression that identifies different volatility regimes. We systematically compare these forecasts with those produced by a simple AR(1) model that we use as a benchmark and we find that the inclusion of commodity-specific factors does not improve the forecasting power. We perform a back-testing exercise of a mean–variance investment strategy that exploits any predictability of the conditional risk premium of commodities, stocks, and bond returns, also consider transaction costs caused by portfolio rebalancing. The risk-adjusted performance of this strategy does not allow us to conclude that any forecasting approach outperforms the others. However, there is evidence that investment strategies based on commodity-specific predictors outperform the remaining strategies in the high-volatility state.
Similar content being viewed by others
Notes
Jensen et al. (2000) provide evidence on the role of commodity futures in mean–variance portfolios. They find that in periods of restrictive monetary policy, commodity futures carry an important weight and yield a considerable performance enhancement. However, since their paper, it has become common to classify financial market regimes on the basis of the level of volatility.
Yamashita et al. (2007) compare the stepwise AIC selection method with other stepwise methods for variable selection and show that this practical criterion leads to the same results as partial F tests.
As recently shown by Aslan et al. (2018), with reference to commodity returns, it may be possible to group different commodity returns series on the basis of commonalities in the estimated linear autoregressive and non-linear threshold autoregressive features to further reduce the dimension of the cross-section.
More precisely, the procedure is as follows: at any time, t, we sort the commodities according to their past 12‐month performances and create an equally‐weighted portfolio that is long on the first 5 commodities in the ranking and short on the last 5 commodities in the ranking.
The fact that the number and nature of the principal components included in the “optimized” predictive regressions is highly sensitive to whether the data are drawn from a low- versus a high-volatility regime provides indirect confirmation of the presence of regime switching dynamics in the data.
While the upper portion of the Table relies only on whether the state probability of a low regime exceed 0.5 or not, the bottom parts of the Table also rely on the predictions from the estimated two-state MS model for the VIX. Although one may argue that this way of proceeding is more elegant and consistent with the framework of our paper, note that at this point we find ourselves jointly assessing the forecasting power of the predictive regressions that include or not commodity-specific factors and the forecasting accuracy of a simple MS model for the VIX. The latter model, as simple and compelling as it may appear, does not represent the main object of our analysis.
We have also experimented with 5-year rolling estimation windows, obtaining qualitatively similar results.
In panel C, the average allocations implied by the benchmark are even more biased towards long positions in government bonds, now exceeding 100%. The long positions in commodities are modest and now concentrated in silver, Brent crude oil, and gasoline; gold is instead massively shorted, which represents the most visible difference versus the allocations in panels A and B.
Also in this case, the effect can be noted only when the predictions are computed using a forward stepwise algorithm that starts out with a null model without any predictability, and progressively expands the set of predictors if and when these lower the AIC of the resulting model.
In Table 8 and also as a way to check the robustness of our results, we have extended the exercise to include more values of the risk aversion coefficient \( \gamma \), also exceeding 1.
We have also performed this robustness check for the case without transaction costs and it gave insights qualitatively similar to those reported in the main text.
We also performed the exercises accounting for transaction costs, similarly to Sect. 5.1. The results, which are not reported for the sake of brevity, are comparable to those discussed for the case of no transaction costs.
References
Aslan, S., Yozgatligil, C., & Iyigun, C. (2018). Temporal clustering of time series via threshold autoregressive models: Application to commodity prices. Annals of Operations Research, 260, 51–77.
Bakshi, G., Gao, X., & Rossi, A. G. (2019). Understanding the sources of risk underlying the cross section of commodity returns. Management Science, 65(2), 619–641.
Bessembinder, H., & Chan, K. (1992). Time-varying risk premia and forecastable returns in futures markets. Journal of Financial Economics, 32, 169–193.
Bossaerts, P., & Hillion, P. (1999). Implementing statistical criteria to select return forecasting models: What do we learn? Review of Financial Studies, 12, 405–428.
Brennan, M. J. (1958). The supply of storage. American Economic Review, 48, 50–72.
Chong, J., & Miffre, J. (2010). Conditional correlation and volatility in commodity futures and traditional asset markets. Journal of Alternative Investments, 12, 61–75.
Dal Pra, G., Guidolin, M., Pedio, M., & Vasile, F. (2018). Regime shifts in excess stock return predictability: An out-of-sample portfolio analysis. Journal of Portfolio Management, 40, 10–24.
Daskalaki, C., Kostakis, A., & Skiadopoulos, G. (2014). Are there common factors in individual commodity futures returns? Journal of Banking & Finance, 40, 346–363.
Daskalaki, C., & Skiadopoulos, G. (2011). Should investors include commodities in their portfolios after all? New evidence. Journal of Banking & Finance, 35, 2606–2626.
de Roon, F. A., Nijman, T. E., & Veld, C. (2000). Hedging pressure effects in futures markets. Journal of Finance, 55, 1437–1456.
DeMiguel, V., Garlappi, L., & Uppal, R. (2007). Optimal versus naive diversification: How inefficient is the 1/N portfolio strategy? Review of Financial studies, 22, 1915–1953.
Erb, C. B., & Harvey, C. R. (2006). The tactical and strategic value of commodity futures. Financial Analysts Journal, 62, 69–97.
Fuertes, A. M., Miffre, J., & Rallis, G. (2010). Tactical allocation in commodity futures markets. Journal of Banking & Finance, 34, 2530–2540.
Gargano, A., & Timmermann, A. (2014). Forecasting commodity price indexes using macroeconomic and financial predictors. International Journal of Forecasting, 30, 825–843.
Giampietro, M., Guidolin, M., & Pedio, M. (2018). Estimating stochastic discount factor models with hidden regimes: Applications to commodity pricing. European Journal of Operational Research, 265, 685–702.
Gorton, G. B., Hayashi, F., & Rouwenhorst, K. G. (2012). The fundamentals of commodity futures returns. Review of Finance, 17, 35–105.
Guidolin, M., & Timmermann, A. (2007). Asset allocation under multivariate regime switching. Journal of Economic Dynamics and Control, 31(11), 3503–3544.
Hamilton, J. D., & Wu, J. C. (2015). Effects of index-fund investing on commodity futures prices. International Economic Review, 56, 187–205.
Henriksen, T. E. S., Pichler, A., Westgaard, S., & Frydenberg, S. (2019). Can commodities dominate stock and bond portfolios? Annals of Operations Research, 282(1–2), 155–177.
Hicks, J. R. (1939). Value and capital. London: Oxford Clarendon Press.
Hirshleifer, D. (1988). Residual risk, trading costs, and commodity futures risk premia. Review of Financial Studies, 1, 173–193.
Hong, H., & Yogo, M. (2012). What does futures market interest tell us about the macroeconomy and asset prices? Journal of Financial Economics, 105, 473–490.
Irwin, S. H., & Sanders, D. R. (2011). Index funds, financialization, and commodity futures markets. Applied Economic Perspectives and Policy, 33, 1–31.
Jensen, G., Johnson, R., & Mercer, J. (2000). Efficient use of commodity futures in diversified portfolios. Journal of Futures Markets, 20, 489–506.
Kaldor, N. (1939). Speculation and economic stability. Review of Economic Studies, 7, 1–27.
Keynes, J. M. (1930). A Treatise on Money. London: Macmillan Press.
Lean, H., H., Nguyen, D., K., & Uddin, G. (2018). On the Role of Commodity Futures in Portfolio Diversification, Working paper, Indiana University.
Ludvigson, S. C., & Ng, S. (2009). Macro factors in bond risk premia. Review of Financial Studies, 22, 5027–5067.
Narayan, P. K., Ahmed, H. A., & Narayan, S. (2015). Do momentum-based trading strategies work in the commodity futures markets? Journal of Futures Markets, 35, 868–891.
Rapach, D. E., & Wohar, M. E. (2006). In-sample vs. out-of-sample tests of stock return predictability in the context of data mining. Journal of Empirical Finance, 13, 231–247.
Rapach, D., E., & Zhou, G. (2013). Forecasting stock returns. In Handbook of Economic Forecasting Vol. 2, Part A, pp. 328–383.
Sharma, M. J., & Yu, S. J. (2015). Stepwise regression data envelopment analysis for variable reduction. Applied Mathematics and Computation, 253, 126–134.
Shen, Q., Szakmary, A. C., & Sharma, S. C. (2007). An examination of momentum strategies in commodity futures markets. Journal of Futures Markets, 27, 227–256.
Stock, J. H., & Watson, M. W. (2002). Forecasting using principal components from a large number of predictors. Journal of the American Statistical Association, 97, 1167–1179.
Stock, J. H., & Watson, M. W. (2006). Forecasting with many predictors. In G. Elliott, C. Granger, & A. Timmermann (Eds.), Handbook of economic forecasting (Vol. 1, pp. 515–554). Amsterdam: Elsevier.
Stoll, H. R. (1979). Commodity futures and spot price determination and hedging in capital market equilibrium. Journal of Financial and Quantitative Analysis, 14, 873–894.
Tang, K., & Xiong, W. (2012). Index investment and the financialization of commodities. Financial Analysts Journal, 68, 54–74.
Welch, I., & Goyal, A. (2008). A comprehensive look at the empirical performance of equity premium prediction. Review of Financial Studies, 21, 1455–1508.
Yamashita, T., Yamashita, K., & Kamimura, R. (2007). A stepwise AIC method for variable selection in linear regression. Communications in Statistics-Theory and Methods, 36(13), 2395–2403.
You, L., & Daigler, R. T. (2013). A Markowitz optimization of commodity futures portfolios. Journal of Futures Markets, 33, 343–368.
Zheng, H., Kulkarni, S. R., & Poor, H. V. (2013). A sequential predictor retraining algorithm and its application to market prediction. Annals of Operations Research, 208, 209–225.
Acknowledgements
We would like to thank the editors of the special issue on “Recent Developments in Financial Modeling and Risk Management” and three anonymous referees for insightful comments and encouragement to improve this paper.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Appendix
Appendix
The first group of macroeconomic variables includes output and income series, such as personal income, retail sales, and real consumption, taken from The Conference Board’s Indicator (TCB), and total industrial production, extrapolated from the Global Insights Basic Economics Database (GIBED). The second group contains labour market series. The series of average weekly hours of production or non-support workers on private non-farm payrolls, total employees in civilian labour force, and unemployment average duration in weeks are taken from the database GIBED, while the source of average weekly initial claims for unemployment insurance is TCB. The third group contains housing series. The series of total new private housing units are taken from GIBED, while the manufacturing and trade inventories are taken from TCB. The fourth group includes consumption, orders, and inventories series, such as manufacturers’ new orders series and National Association Of Purchasing Managers (NAPM) new orders index, National Association Of Purchasing Managers (NAPM) vendor deliveries index, and National Association Of Purchasing Managers (NAPM) inventories index. The fifth group includes money and credit variables. The series of monetary base, S&P 500, commercial and industrial loans, and average effective exchange rates of this group are taken from GIBED. To this series, we add the historical news-based policy index and the economic policy uncertainty index. The historical news-based policy index is a proxy of the economic policy uncertainty for the US, based on three types of underlying components: the newspaper coverage of policy-related economic uncertainty, the number of federal tax code provisions set to expire in future years, and the disagreement among economic forecasters. The economic policy uncertainty index represents the US movements in policy-related economic uncertainty. This index indicates that when the uncertainty increases, stock prices rise and the level of investment, employment, and country’s outcome go down.
The sixth group includes stock market series. They are S&P 500 Index and S&P 500 industrials series. We also include dividend yield (DY), earning price ratio (EP), and dividend pay-out ratio (DP) of S&P 500 index. The dividends used here are the 12-month moving sums of dividends paid on the S&P 500 Index. Earning price ratio is obtained as the log of earnings minus the log of stock prices where the earnings are the 12-month moving sums on the S&P 500 Index. The dividend pay-out ratio is computed as the log of dividends minus the log of earnings. We also consider stock return volatility (SVOL), which is the monthly sum of the squared daily stock returns on the S&P 500 index, and book-to-market ratio (BM), that is the ratio of the book value to market value for the Dow Jones Industrial Average. We also take into account the net equity expansion (NTIS), which is the ratio of the 12-month moving sums of net issues by New York Stock Exchange (NYSE) listed stocks to the total market capitalization of NYSE stocks.
The seventh group includes bond and exchange rate series. We additionally consider Treasury Bill Rate (TBL), which relates to the interest rate on a three-month Treasury bill, the long-term government bond yield (LTY), the long-term return government bond (LTR) returns, the term spread (TMS), the default yield spread (DFY), calculated as the difference between Moody’s BAA- and AAA-rated bond yields, and the default return spread (DFR), calculated as the difference between long-term corporate and government bond returns.
The eighth group includes prices series. They represent producer price index series, National Association of Purchasing Managers (NAPM) commodity price index, consumer of price index series, personal consumption expenditures series, and average hourly earnings of production and nonsupervisory employees series. To these, we add the crude oil price (COP), computed using the logarithmic changes in the nominal price of West Texas Intermediate crude oil, provided by the Federal Reserve Bank of St. Louis.
Rights and permissions
About this article
Cite this article
Guidolin, M., Pedio, M. Forecasting commodity futures returns with stepwise regressions: Do commodity-specific factors help?. Ann Oper Res 299, 1317–1356 (2021). https://doi.org/10.1007/s10479-020-03515-w
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10479-020-03515-w