Macroeconomic Survey Expectations pp 37-63 | Cite as

# Assessing the Point Predictions

## Abstract

Point forecasts can typically be evaluated in terms of their accuracy relative to a rival set of forecasts, or in terms of whether they possess certain desirable characteristics, such as the forecast and forecast error being uncorrelated. There are differences between the evaluation of survey and model forecasts. An obvious one is that survey forecasts cannot be compared to their expected accuracy given the in-sample fit (past track record) of the model, since their provenance is unknown. Comparisons of forecasts from rival models typically seek to compare the models (in population). The evaluation of rival sets of survey forecasts rests squarely on the forecasts themselves. Tests of equal accuracy or forecast encompassing, as well as tests of whether the forecasts possess desirable characteristics, such as forecast efficiency, can be applied globally over the whole sample of forecasts, or locally on sub-samples, allowing for instabilities in performance over time. Also considered is the appropriate way of testing for forecast rationality or efficiency in a panel context, where the options include testing the aggregate, pooling the observations over individuals and time periods, or running separate regressions for each individual. A key assumption is that individuals weigh over- and under-prediction equally that the loss function is symmetric. This assumption is testable.

## References

- Bonham, C., & Cohen, R. (2001). To aggregate, pool, or neither: Testing the rational expectations hypothesis using survey data.
*Journal of Business and Economic Statistics, 190*, 278–291.CrossRefGoogle Scholar - Capistrán, C., & Timmermann, A. (2009). Disagreement and biases in inflation expectations.
*Journal of Money, Credit and Banking, 41*, 365–396.CrossRefGoogle Scholar - Chong, Y. Y., & Hendry, D. F. (1986). Econometric evaluation of linear macro-economic models.
*Review of Economic Studies, 53*, 671–690. Reprinted in Granger, C. W. J. (Ed.). (1990).*Modelling economic series*. Oxford: Clarendon Press.Google Scholar - Christoffersen, P. F., & Diebold, F. X. (1997). Optimal prediction under asymmetric loss.
*Econometric Theory, 13*, 808–817.CrossRefGoogle Scholar - Clements, M. P. (1995). Rationality and the role of judgement in macroeconomic forecasting.
*Economic Journal, 105*, 410–420.CrossRefGoogle Scholar - Clements, M. P. (1997). Evaluating the rationality of fixed-event forecasts.
*Journal of Forecasting, 16*, 225–239.CrossRefGoogle Scholar - Clements, M. P. (2009). Internal consistency of survey respondents’ forecasts: Evidence based on the Survey of Professional Forecasters. In J. L. Castle & N. Shephard (Eds.),
*The methodology and practice of econometrics. A Festschrift in Honour of David F. Hendry*(Chap. 8, pp. 206–226). Oxford: Oxford University Press.Google Scholar - Clements, M. P. (2014). US inflation expectations and heterogeneous loss functions, 1968–2010.
*Journal of Forecasting, 33*(1), 1–14.CrossRefGoogle Scholar - Clements, M. P., & Harvey, D. I. (2009). Forecasting combination and encompassing. In T. C. Mills & K. Patterson (Eds.),
*Palgrave handbook of econometrics, volume 2: Applied econometrics*(pp. 169–198). New York: Palgrave MacMillan.CrossRefGoogle Scholar - Clements, M. P., & Harvey, D. I. (2010). Forecast encompassing tests and probability forecasts.
*Journal of Applied Econometrics, 25*, 1028–1062.CrossRefGoogle Scholar - Croushore, D. (2011a). Forecasting with real-time data vintages. In M. P. Clements & D. F. Hendry (Eds.),
*The Oxford handbook of economic forecasting*(Chap. 9, pp. 247–267). Oxford: Oxford University Press.Google Scholar - Croushore, D. (2011b). Frontiers of real-time data analysis.
*Journal of Economic Literature, 49*, 72–100.CrossRefGoogle Scholar - Diebold, F. X., & Mariano, R. S. (1995). Comparing predictive accuracy.
*Journal of Business and Economic Statistics, 13*, 253–263.Google Scholar - Elliott, G., Komunjer, I., & Timmermann, A. (2005). Estimation and testing of forecast rationality under flexible loss.
*Review of Economic Studies, 72*, 1107–1125.CrossRefGoogle Scholar - Elliott, G., Komunjer, I., & Timmermann, A. (2008). Biases in macroeconomic forecasts: Irrationality or asymmetric loss.
*Journal of the European Economic Association, 6*, 122–157.CrossRefGoogle Scholar - Fair, R. C., & Shiller, R. J. (1989). The informational content of
*ex ante*forecasts.*Review of Economics and Statistics, 71*, 325–331.CrossRefGoogle Scholar - Faust, J., & Wright, J. H. (2009). Comparing Greenbook and reduced form forecasts using a large realtime dataset.
*Journal of Business and Economic Statistics, 27*(4), 468–479.CrossRefGoogle Scholar - Figlewski, S., & Wachtel, P. (1981). The formation of inflationary expectations.
*Review of Economics and Statistics, 63*, 1–10.CrossRefGoogle Scholar - Figlewski, S., & Wachtel, P. (1983). Rational expectations, informational efficiency, and tests using survey data: A reply.
*The Review of Economics and Statistics, 65*(3), 529–531.CrossRefGoogle Scholar - Fixler, D. J., Greenaway-McGrevy, R., & Grimm, B. T. (2014, August). The revisions to GDP, GDI, and their major components.
*Survey of Current Business*, 1–23.Google Scholar - Giacomini, R. (2011). Testing conditional predictive ability. In M. P. Clements & D. F. Hendry (Eds.),
*The Oxford handbook of economic*(pp. 441–455). Oxford: Oxford University Press.Google Scholar - Giacomini, R., & White, H. (2006). Tests of conditional predictive ability.
*Econometrica, 74*, 1545–1578.CrossRefGoogle Scholar - Giordani, P., & Söderlind, P. (2003). Inflation forecast uncertainty.
*European Economic Review, 47*(6), 1037–1059.CrossRefGoogle Scholar - Granger, C. W. (1969). Prediction with a generalized cost of error function.
*Operations Research Quarterly, 20*, 199–207.CrossRefGoogle Scholar - Granger, C. W. J., & Newbold, P. (1973). Some comments on the evaluation of economic forecasts.
*Applied Economics, 5*, 35–47. Reprinted in Mills, T. C. (Ed.). (1999).*Economic forecasting. The International Library of Critical Writings in Economics*. Cheltenham: Edward Elgar.Google Scholar - Granger, C. W. J., & Newbold, P. (1977).
*Forecasting economic time series*. New York: Academic Press.Google Scholar - Harvey, D. I., Leybourne, S., & Newbold, P. (1998). Tests for forecast encompassing.
*Journal of Business and Economic Statistics, 16*, 254–259. Reprinted in Mills, T. C. (Ed.). (1999).*Economic forecasting*. Cheltenham: Edward Elgar.Google Scholar - Hendry, D. F., & Richard, J.-F. (1989). Recent developments in the theory of encompassing. In B. Cornet & H. Tulkens (Eds.),
*Contributions to operations research and economics. The XXth anniversary of CORE*(pp. 393–440). Cambridge, MA: MIT Press. Reprinted in Campos, J., Ericsson, N. R., & Hendry, D. F. (Eds.). (2005).*General to specific modelling*. Cheltenham: Edward Elgar.Google Scholar - Keane, M. P., & Runkle, D. E. (1990). Testing the rationality of price forecasts: New evidence from panel data.
*American Economic Review, 80*(4), 714–735.Google Scholar - Lahiri, K., & Liu, F. (2009). On the use of density forecasts to identify asymmetry in forecasters’ loss function.
*Business and Economic Statistics Section - JSM*, 2396–2408.Google Scholar - Landefeld, J. S., Seskin, E. P., & Fraumeni, B. M. (2008). Taking the pulse of the economy.
*Journal of Economic Perspectives, 22*, 193–216.CrossRefGoogle Scholar - Mincer, J., & Zarnowitz, V. (1969). The evaluation of economic forecasts. In J. Mincer (Ed.),
*Economic forecasts and expectations*(pp. 3–46). New York: National Bureau of Economic Research.Google Scholar - Mizon, G. E., & Richard, J.-F. (1986). The encompassing principle and its application to non-nested hypothesis tests.
*Econometrica, 54*, 657–678.CrossRefGoogle Scholar - Morgan, W. A. (1939–1940). A test for significance of the difference between the two variances in a sample form a normal bivariate population.
*Biometrika, 31*, 13–19.Google Scholar - Muth, J. F. (1961). Rational expectations and the theory of price movements.
*Econometrica, 29*, 315–335.CrossRefGoogle Scholar - Nelson, C. R. (1972). The prediction performance of the FRB-MIT-PENN model of the US economy.
*American Economic Review, 62*, 902–917. Reprinted in Mills, T. C. (Ed.). (1999).*Economic forecasting. The International Library of Critical Writings in Economics*. Cheltenham: Edward Elgar.Google Scholar - Nordhaus, W. D. (1987). Forecasting efficiency: Concepts and applications.
*Review of Economics and Statistics, 69*, 667–674.CrossRefGoogle Scholar - Patton, A. J., & Timmermann, A. (2007). Testing forecast optimality under unknown loss.
*Journal of the American Statistical Association, 102*, 1172–1184.CrossRefGoogle Scholar - Patton, A. J., & Timmermann, A. (2012). Forecast rationality tests based on multi-horizon bounds.
*Journal of Business & Economic Statistics*,*30*(1), 1–17.CrossRefGoogle Scholar - Pesaran, M. H., & Weale, M. (2006). Survey expectations. In G. Elliott, C. Granger, & A. Timmermann (Eds.),
*Handbook of economic forecasting, volume 1. Handbook of economics 24*(pp. 715–776). Amsterdam: Elsevier, Horth-Holland.CrossRefGoogle Scholar - Romer, C. D., & Romer, D. H. (2000). Federal Reserve information and the behaviour of interest rates.
*American Economic Review, 90*(3), 429–457.CrossRefGoogle Scholar - Rossi, B., & Sekhposyan, T. (2016). Forecast rationality tests in the presence of instabilities, with applications to Federal Reserve and Survey Forecasts.
*Journal of Applied Econometrics*,*31*(3), 507–532.CrossRefGoogle Scholar - Varian, H. R. (1975). A Bayesian approach to real estate assessment. In S. E. Fienberg & A. Zellner (Eds.),
*Studies in Bayesian econometrics and statistics in honor of Leonard J. Savage*(pp. 195–208). Amsterdam: North Holland.Google Scholar - West, K. D. (1996). Asymptotic inference about predictive ability.
*Econometrica, 64*, 1067–1084.CrossRefGoogle Scholar - West, K. D. (2001). Tests for forecast encompassing when forecasts depend on estimated regression parameters.
*Journal of Business and Economic Statistics, 19*, 29–33.CrossRefGoogle Scholar - West, K. D., & McCracken, M. W. (1998). Regression-based tests of predictive ability.
*International Economic Review, 39*, 817–840.CrossRefGoogle Scholar - Zarnowitz, V. (1985). Rational expectations and macroeconomic forecasts.
*Journal of Business and Economic Statistics, 3*(4), 293–311.Google Scholar - Zellner, A. (1986). Biased predictors, rationality and the evaluation of forecasts.
*Economics Letters, 21*, 45–48.CrossRefGoogle Scholar