Abstract
We propose an imperfect information model for the expectations of macroeconomic forecasters that explains differences in average disagreement levels across forecasters by means of cross-sectional heterogeneity in the variance of private noise signals. We show that the forecaster-specific signal-to-noise ratios determine both the average individual disagreement level and an individuals’ forecast performance: Forecasters with very noisy signals deviate strongly from the average forecasts and report forecasts with low accuracy. We take the model to the data by empirically testing for this implied correlation. Evidence based on data from the Surveys of Professional Forecasters for the USA and for the Euro Area supports the model for short- and medium-run forecasts but rejects it based on its implications for long-run forecasts.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
The alternative “sticky” information model of Mankiw and Reis (2002) has been empirically rejected as an appropriate model to describe the behavior of professional forecasters mainly due to the fact that the observed frequency of forecast updates is much higher than implied by this model (Dovern 2013; Andrade and Le Bihan 2013; Dovern et al. 2015).
We thank an anonymous referee for pointing this out.
Note that, in addition, they also allow for a forecaster-specific bias term that influences forecast performances and individual disagreement levels.
For simplicity, we ignore anticipated shocks and bias terms at this point.
In the simulation, \(\alpha \) takes values from 0 to 0.95 and \(\sigma ^2_{\varepsilon }/\sigma ^2_{\eta }\) ranges from 0.1 to 2.
By ensuring that the factors are on average equal to 1, we can simulate cases which are comparable to the homogeneous case in the sense that the average signal-to-noise ratio is equal to that in the model with symmetric forecasters.
Note that the scaling ensures that the mean is equal to 1. Results are the same qualitatively for other parameterizations of the beta distribution or other distributional assumptions.
The simulation is based on \(T=5000\) to ensure a good approximation of the expected moments.
We set \(\sigma ^2_\mu =0.5\).
Survey waves before 1992q1 refer to gross national product (GNP) rather than GDP.
Strictly speaking, the 5-years-ahead forecasts are fixed-event forecasts made for the annual average of the forecast variables in a particular target year. This target year is changing in such a way that the forecast horizon varies between 21 and 18 quarters. Given the very long forecast horizon, it is unlikely that forecasts are affected by changes in the target year or small variations in the forecast horizon.
Results are robust against selecting a higher required number of observations.
To limit the influence of outlier observations, we use the square root of \(\widehat{ MSFE}_{i,h}\) as the dependent variable in (4.1). The results based directly on \(\widehat{ MSFE}_{i,h}\) are qualitatively equivalent to those reported in the paper and are available from the authors upon request.
References
Andrade P, Crump RK, Eusepi S, Moench E (2014) Fundamental disagreement. Working papers 524, Banque de France
Andrade P, Le Bihan H (2013) Inattentive professional forecasters. J Monet Econ 60(8):967–982
Barron OE, Kim O, Lim S, Stevens DE (1998) Using analysts’ forecasts to measure properties of analysts’ information environment. Account Rev 73(4):421–433
Branch WA (2004) The theory of rationally heterogeneous expectations: evidence from survey data on inflation expectations. Econ J 114(497):592–621
Coibion O, Gorodnichenko Y (2012) What can survey forecasts tell us about information rigidities? J Polit Econ 120(1):116–159
Davies A, Lahiri K (1995) A new framework for analysing survey forecasts using three-dimensional panel data. J Econ 68:205–227
Davies A, Lahiri K, Sheng X (2011) Analyzing three-dimensional panel data of forecasts. In: Clements MP, Hendry DF (eds) The Oxford handbook of economic forecasting, chapter 17. Oxford University Press, Oxford, pp 473–498
Dovern J (2013) When are GDP forecasts updated? Evidence from a large international panel. Econ Lett 120(3):521–524
Dovern J (2015) A multivariate analysis of forecast disagreement: confronting models of disagreement with survey data. Eur Econ Rev 80((C)):16–35
Dovern J, Fritsche U, Loungani P, Tamirisa NT (2015) Information rigidities: comparing average and individual forecasts for a large international panel. Int J Forecast 31(1):144–154
Harris M, Raviv A (1993) Differences of opinion make a horse race. Rev Financ Stud 6(3):473–506
Kalman RE (1960) A new approach to linear filtering and prediction problems. J Basic Eng 82(1):35–45
Lahiri K, Liu F (2006) Modelling multi-period inflation uncertainty using a panel of density forecasts. J Appl Econom 21(8):1199–1219
Lahiri K, Sheng X (2008) Evolution of forecast disagreement in a Bayesian learning model. J Econom 144(2):325–340
Lahiri K, Sheng X (2010a) Learning and heterogeneity in GDP and inflation forecasts. Int J Forecast 26(2):265–292
Lahiri K, Sheng X (2010b) Measuring forecast uncertainty by disagreement: the missing link. J Appl Econom 25(4):514–538
Lahiri K, Peng H, Sheng X (2015) Measuring uncertainty of a combined forecast and some tests for forecaster heterogeneity. CESifo working paper series 5468, CESifo Group Munich
Laster D, Bennett P, Geoum IS (1999) Rational bias in macroeconomic forecasts. Quart J Econ 114(1):293–318
Lorenzoni G (2009) A theory of demand shocks. Am Econ Rev 99(5):2050–2084
Malmendier U, Nagel S (2011) Depression babies: do macroeconomic experiences affect risk taking? Q J Econ 126(1):373–416
Mankiw NG, Reis R (2002) Sticky information versus sticky prices: a proposal to replace the new Keynesian Phillips curve. Quart J Econ 117:1295–1328
Mankiw NG, Reis R, Wolfers J (2003) Disagreement on inflation expectations. NBER macroeconomics annual, pp 209–248
Melosi L (2014) Estimating models with dispersed information. Am Econ J Macroecon 6(1):1–31
Nimark KP (2014) Man-bites-dog business cycles. Am Econ Rev 104(8):2320–2367
Sims C (2003) Implications of rational inattention. J Monet Econ 50(3):665–690
Author information
Authors and Affiliations
Corresponding author
Additional information
Matthias Hartmann gratefully acknowledges financial support by the Fritz-Thyssen-Foundation.
Rights and permissions
About this article
Cite this article
Dovern, J., Hartmann, M. Forecast performance, disagreement, and heterogeneous signal-to-noise ratios. Empir Econ 53, 63–77 (2017). https://doi.org/10.1007/s00181-016-1137-x
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00181-016-1137-x