Abstract
This paper investigates the impact of the timeliness of information releases and data vintage variation on economic forecast quality. Specifically, using a set of 63 key US economic series, we provide a concise measure of the forecast accuracy associated with use of economic activity indices with different publication lags. A forecasting model based on an economic activity index that is subject to a short publication lag (viz. the Aruoba-Diebold-Scotti index) is more efficient than competing models. Moreover, if this publication lag advantage is removed (by artificially imposing a publication lag restriction comparable to that of a competing indicator) this efficiency largely disappears. The final part of the analysis employs a novel (simulation-based) method of assessing the impact of data vintage variation on forecast accuracy, and finds that the results are somewhat sensitive to such variation.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
From 1978 onwards, the final value of the UMCSI is released at 10.00 a.m. ET on the last Friday of each calendar month. For instance, the October 2011 UMCSI is released on October 28, 2011. Similarly, the PMI is released at 10:00 a.m. ET on the first business day of every month. Thus, the October 2011 PMI is released on November 1, 2011.
Fig. 1 
Indicators of economic activity. This figure contains quarterly frequency time series plots of the UMCSI, PMI, CFNAI and ADS index over the period 1968–2010. The values of the UMCSI and PMI are standardised to have a mean of zero and standard deviation of unity, while CFNAI and ADS index are given in their original form. The start and end points of recessions are given by the vertical lines
The CFNAI is released at 8:30 a.m. ET toward the end of each calendar month and pertain to the preceding month. For instance, the October 2011 CFNAI level is released on November 21, 2011. As we use quarterly data and the CFNAI has a monthly frequency this means that the CFNAI value used will contain a 1 month lag. By contrast, the ADS index is released at 10:00 a.m. ET approximately seven times per month and is subject to a minimum publication lag of just 5 days (corresponding to the publication lag associated with the highest frequency data input, viz. weekly initial jobless claims).
A literature exists devoted to the examination of the impact of data vintage on forecast quality; see, e.g. Koenig et al. (2003) for evidence supporting the use of real-time vintage data. To isolate the impact of timeliness, we follow Stock and Watson (2003b) and initially use data of a recent vintage (2011 quarter 3 vintage). However, we subsequently undo this assumption and also present results based on using synthetic real-time vintage data.
Alternatively, an unrestricted AR could be estimated, but the appropriate forecast would be a two-step ahead forecast pertaining to the period \(T+1\). In this case, forecast quality will deteriorate because of a lengthening of the forecast horizon.
The latter effect is likely to be of second-order importance, especially in the presence of moderate to large sample sizes.
The 1968 starting point of the sample is used as this represents the first year in which a joint observation of the UMCSI, PMI, CFNAI and ADS index series is available.
The following series are augmented to bring them in line with series used in previous studies (e.g. Stock and Watson 2003b): private non-residential fixed investment (PNFI), private residential fixed investment (PRFI) and fixed private investment (FPI) are converted to real measures by deflating by the gross private domestic investment chain-type price index (GPDICTPI); net exports (NETEXP), federal government receipts (FGRECPT) and federal government expenditure (FGEXPND) are divided by the gross domestic product implicit price deflator (GDPDEF); personal consumption expenditures on durable goods (PCDG), personal consumption expenditures on non-durable goods (PCND), personal consumption expenditures on services (PCESV) and personal savings (PSAVE) are divided by the personal consumption expenditures chain-type price index (PCECTPI); and changes in real private inventories (CBIC96) is divided by real gross domestic product (GDPC96).
In addition to these predictors, we follow Rudebusch and Williams (2009) and Wheelock and Wohar (2009) and consider use of the difference between the 3-month and 10-year treasury interest rates as a predictor variable. However, this particular predictor was found to be inferior to the other predictors in the current context and hence the results are not presented. Results pertaining to this predictor are available upon request.
Where possible the logarithm of each series is considered.
To ensure that the data used in this t test are symmetric, we take the natural logarithm of the efficiency measures when conducting this test. See Pearson and Please (1975) for evidence regarding the robustness of the univariate t test in the presence of non-normal data when using small sample sizes.
Only if the month/quarter end falls on a Thursday will the actual publication lag be 5 days. Otherwise, a longer publication lag will hold. For instance, the final day of the third quarter of 2011 (that is, September 30, 2011) was a Friday. Consequently, the latest set of ADS index levels would have been released on Thursday September 29, 2011 and would pertain to the week leading up to Saturday September 24, 2011. Thus a 6-day publication lag occurs in this instance. Over the sample that we consider the average actual publication lag was 8.11 days.
Individual macroeconomic vintage data are available over a slightly longer period from the archival federal reserve economic data (ALFRED) website: http://research.stlouisfed.org/alfred/. However, for most series the earliest start point typically occurs during the 1990s. For this reason, we do not consider end-of-sample vintages for individual macroeconomic series. Rather, we consider performance with respect to forecasts of final vintage data. This choice is highly unlikely to bias performance in favour of a particular model.
Note that as the UMCSI is not subject to data revision it follows that \(\sigma \) equals zero in this instance.
Vintage data for the PMI and CFNAI were obtained from the ALFRED database, while the corresponding data for the ADS index were obtained from the Federal Reserve Bank of Philadelphia website.
As the PMI is based on a different measurement scale (it can range from 0 to 100) to the CFNAI and ADS index (these tend to have a range from \(-4\) to \(4\)), the \(\widehat{\sigma }\) values should not be used to judge the extent of data revision.
Sampling error considerations aside, the lower correlation associated with the ADS index may be due to three reasons. First, the ADS index is constructed using a dynamic factor model that may generate large amounts of index revision if the model is periodically re-estimated (though it does appear that the model was re-estimated over the period used to construct the correlations). Second, it may be the case that data revisions cancel each other out if a large number of series are used to construct an index. As the CFNAI is based on 85 series, while the ADS index is based on only six, this may explain the differences in data vintage noise. Third, real activity-type series are more prone to data revision than inflation-type series (see, e.g. Corradi et al. 2009). Consequently, as the ADS index is composed exclusively of the former type of series, while the CFNAI is based on both types of series, this may also explain the observed differences in index revision.
Croushore and Stark (2003) find that the conclusions of a number of previously published econometric studies are qualitatively robust to data vintage variation.
References
Aruoba B, Diebold F (2010) Real-time macroeconomic monitoring: real activity, inflation, and interactions. Am Econ Rev 100:20–24
Aruoba B, Diebold F, Scotti C (2009) Real-time measurement of business conditions. J Bus Econ Stat 27:417–427
Bańbura M, Giannone D, Reichlin L (2011) Nowcasting. In: Clements M, Hendry D (eds) Oxford handbook of economic forecasting. Oxford University Press, Oxford
Brave S (2009) The Chicago fed national activity index and business cycles. Chicago Fed Lett 268:1
Clements M, Galvão A (2012) Real-time forecasting of inflation and output growth in the presence of data revisions. J Appl Econ (forthcoming)
Corradi V, Fernandez A, Swanson N (2009) Information in the revision process of real-time datasets. J Bus Econ Stat 27:455–467
Croushore D, Stark T (2003) A real-time data set for macroeconomists: does the data vintage matter? Rev Econ Stat 85:605–617
Evans M (2005) Where are we now? Real-time estimates of the macroeconomy. Int J Cent Bank 1:127–175
Evans C, Liu C, Pham-Kanter G (2002) The 2001 recession and the Chicago fed national activity index: identifying business cycle turning points. Econ Perspect 3rd Q 26:26–43
Faust J, Rogers J, Wright J (2005) News and noise in G-7 GDP announcements. J Money Credit Bank 37:403–417
Fildes R, Stekler H (2002) The state of macroeconomic forecasting. J Macroecon 24:435–468
Giannone D, Reichlin L, Small L (2008) Nowcasting: the real-time information content of macroeconomic data. J Monet Econ 55:665–676
Harding D, Pagan A (2010) Can we predict recessions? Working paper \(\#\)69, NCER Working Paper Series:1–31
Koenig E (2002) Using the purchasing managers index to assess the economy strength and the likely direction of monetary policy. Federal Reserve Bank Dallas Econ Finan Policy Rev 4:1–14
Koenig E, Dolmas S, Piger J (2003) The use and abuse of real-time data in economic forecasting. Rev Econ Stat 85:618–628
Land D, Lansing K (2010) Forecasting growth over the next year with a business cycle index. FRBSF Econ Lett 16:1–5
Ludvigson S (2004) Consumer confidence and consumer spending. J Econ Perspect 18:29–50
Marcellino M, Stock J, Watson M (2006) A comparison of direct and iterated multistep AR methods for forecasting macroeconomic time series. J Econ 135:499–526
Pearson E, Please N (1975) Relation between the shape of population distribution and the robustness of four simple test statistics. Biometrika 62:223–241
Phillips P (1987) Time series regressions with a unit root. Econometrica 55:277–301
Phillips P, Perron P (1988) Testing for a unit root in time series regression. Biometrika 75:335–346
Rogers R (1992) Forecasting industrial production: purchasing managers versus production-worker hours data. Federal Reserve Bank Atlanta Econ Rev 77:25–36
Rudebusch G, Williams J (2009) Forecasting recessions: the puzzle of the enduring power of the yield curve. J Bus Econ Stat 27:492–503
Stock J, Watson M (1999) Forecasting inflation. J Monet Econ 44:293–335
Stock J, Watson M (2003a) How did leading indicator forecasts perform during the 2001 recession? Federal Reserve Bank Richmond Econ Q 89:71–90
Stock J, Watson M (2003b) Has the business cycle changes and why? Natl Bur Econ Res 17:159–230
Wheelock D, Wohar M (2009) Can the term spread predict output growth and recessions? A survey of the literature. Federal Reserve Bank St. Louis Rev, September/October Part 1:419–440
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Taylor, N. Economic forecast quality: information timeliness and data vintage effects. Empir Econ 46, 145–174 (2014). https://doi.org/10.1007/s00181-012-0672-3
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00181-012-0672-3