Macroeconometric forecasting using a cluster of dynamic factor models

Abstract

We propose a modeling approach based on a set of small-scale factor models linked together in a cluster with linkages derived from Granger causality tests. GDP forecasts are produced using a disaggregated approach across production, expenditure and income accounts. The method combines the advantages of large structural macroeconomic models and small factor models, making our cluster of dynamic factor models (CDFM) useful for large-scale model-consistent forecasting. The CDFM has a simple structure, and its forecasts outperform those of a variety of competing models and professional forecasters. In addition, the CDFM allows forecasters to use their own judgment to produce conditional forecasts.

Appendices

Appendix A: Alternative Granger causality tests

We check the links uncovered by the conventional Granger causality test using two alternative tests. The Granger is a bivariate test that cannot account for the possibility of a causal relationship between a pair of variables being induced by a third variable acting as a common cause. The omission of common causes can lead to spurious causality. The second issue is that the Granger test involves model selection as a first step, which invalidates the subsequent asymptotic and finite sample inference (Leeb and Pötscher 2005).

The generalization by Hecq et al. (2019) rectifies the deficiency of a bivariate causality test. This multivariate test is based on a high-dimensional VAR that includes the entire dataset used to estimate the CDFM. The estimation procedure uses a sparsity-seeking regularization that uncovers the key dynamic interactions among the variables while discarding the rest. Model selection and testing follow the Belloni et al. (2014) post-double-selection approach that partially mitigates the problem associated with the validity of post-model-selection estimators. The LM test then accounts for multiple joint cause for each pair of variables of interest. We use test statistic with a final sample correction that has been shown to improve the size of the test.

The conventional Granger causality test and its multivariate generalization involve a VAR and thus assume a linear relationship between the variables. Our second alternative is based on a highly nonlinear view adaptive recurrent neural network (VA-RNN) model, which is a multilayer feed-forward neural network (Hmamouche 2020). The nonlinear test is bivariate. The test statistic being equivalent to that of the conventional Granger test in that it relates the residual sum of squares of errors of the restricted and the unrestricted models and follows an F-distribution under the null hypothesis of no causality.

Table 8 Alternative causality tests

Table 8 summarizes the test results for the final selection of links. The underlying models have been tested using quarterly data up to lag two, as suggested by the BIC criterion. Due to a sequential structure of the cluster, we are only interested in unidirectional causality relationships, by which past realizations of the variable listed in the first column improves forecasts of the corresponding variable in the second column, but not the other way around. We call unidirectional causality weak (W) if the test is significant at the 10% level, and strong (S) if it is significant at the 5% level. Note that the multivariate test tends to uncover fewer causal links that the two bivariate tests, which is expected due to the existence of multiple common causes in large dataset of highly interdependent time series. All the links except those between the exports of services and the value added of the service sector, and between construction investment and the value added in the construction sector have been validated by at least one of the three tests. These two links have been retained because their inclusion has significantly improved the forecasting performance of the respective value-added models when entered contemporaneously. Contemporaneous links could not be validated by causality tests that seek an intertemporal relationship between a pair of variables.

Appendix B: An alternative view on the production account: tradable and nontradable goods (TNT)

The approach proposed in Sect. 2 for modeling production is frequently used by international organizations, central banks and economic research institutes. It follows the sectoral composition of the National Accounts, but does not reflect the macroeconomic theory, which usually groups the sectors in producers of tradable goods (T) and nontradable goods (NT).¹⁹ While the TNT classification is clear for some sectors, it can be ambiguous for others. Moreover, structural change might turn a previously tradable sector into a nontradable one, and vice versa. We follow the approach to the sectoral classification in Friesenbichler and Glocker (2019), determine the nominal value added for tradable goods and nontradable goods and calculate the corresponding deflators, which allows us to determine the real value added of these two categories.

The next step involves specification of separate behavioral models for tradable goods and nontradable goods, and an aggregator model for the GDP. The aggregator model features an error term, as the tradable and nontradable goods do not sum up to GDP, the difference being product taxes and subsidies. Another reason for including an error term is that we again consider a log-linearized representation of a weighted sum, in which the weights of the components can change over time. These changes in the weights are addressed by an autoregressive error term, as shown in Eq. (5).

Table 3 provides the normalized values of the RMSE for a GDP forecast based on the TNT approach. The forecasting accuracy of the TNT approach is similar to that of the standard production-side GDP approach considered in Sect. 3. Although the normalized RMSE of GDP is slightly smaller than in the conventional three-sector approach (manufacturing, construction, services), this difference is not statistically significant. The forecasts of the value added of tradable goods and nontradable goods seem to be comparatively precise. The values for the normalized RMSE for these two components are smaller than those for the sectors in the conventional approach for all forecasting horizons.

Subplot (d) in Fig. 3 shows the annual GDP-forecast for the year 2009 obtained using the TNT approach over a period of 2008 and 2009. The TNT-based forecasts indicate negative annual growth relatively early, somewhat overestimating the extent of the recession at the end of 2008, but nonetheless approaching the realized value quickly.

Table 9 Tradables (T) and nontradables (NT) (2007–2018)

Finally, the deflators for tradable and nontradable goods allow for an alternative approach to modeling and forecasting the GDP deflator. We specify a behavioral model for each of the two deflators and link them to the GDP deflator in an aggregator model. The forecast evaluation in Table 9 shows the normalized RMSE of the deflators for tradable and nontradable goods to be comparatively small. The TNT-based forecasts of the GDP deflator are thus comparable to the baseline approach.

Appendix C: The small-scale dynamic factor model

We consider a small-dynamic factor model (small DFM), as popularized by Mariano and Murasawa (2003), Camacho and Pérez-Quirós (2010), Camacho and Pérez-Quirós (2011), Arnoštová et al. (2011), Aastveit and Trovik (2012) as a competing model. This approach comprises a small-scale and hence simple factor model applied directly to GDP growth. Following Mariano and Murasawa (2003), we combine monthly and quarterly data, expressing the quarterly data as a function of monthly data. If the sample mean of the three monthly observations in a given quarter can be approximated by the geometric mean, then the quarterly growth rates can be decomposed as weighted averages of monthly growth rates. We follow the outline put forward in Sect. 3 and utilize the approach motivated by Glocker and Wegmüller (2020) to select an appropriate set of variables. This approach explicitly takes into account the fact that additional variables do not necessarily improve the model’s forecast. The available set of variables contains around sixty variables.

The principal criterion for variable selection is out-of-sample forecasting ability, producing a set of variables geared toward economic expectations. We have already seen in Sect. 4.6 that the resulting model performs well in forecasting the 2009 economic downturn. The final specification of the small-scale factor model (SDFM) for GDP includes: (1) expectations in the construction sector, (2) expectations in the manufacturing sector, (3) expectations in the service sector, (4) Purchasing Managers Index (PMI), (6) order backlog (manufacturing sector), (7) employment (all sectors), (8) vacancies (all sectors), (9) retail sales (total) and (10) truck mileage. We add further variables only if they improve the out-of-sample forecasting performance of the model. We find that some additional variables could be included; however, they do not improve the forecast (e.g., Economic Sentiment Index (ESI) from the European Commission, ATX/Austrian Traded Index volatility, the financial market stress indicator as considered in Glocker and Kaniovski (2014), term-structure—i.e., the difference between 10-year and 2-year government bond yield, industrial production—excluding the construction sector, and retail sales). Other variables worsened the out-of-sample forecasts and were subsequently discarded from the model. The final selection proved robust to enlargements of the model in various directions. We tested our model using disaggregated versions of the variables already included in the model. For instance, we used retail sales without oil-related products instead of total retail sales. We also checked for the employment of different sectors (manufacturing sector, construction sector) instead of the aggregate measure, failing to improve the model in all cases.

Since the variables considered address both the outlook and the current situation, we allow for a temporal displacement between GDP as the target variable and the additional variables, for which we follow Camacho and García-Serrador (2014). This setup follows Camacho and Pérez-Quirós (2010) with a dynamic factor structure involving one factor with two lags (we omit the elements concerning data revisions from the model). The number of factors is selected by using Bai and Ng (2002) information criteria (BG) modified to take into account that the parameters are estimated using maximum likelihood. The number of lags in the factor equation and for the error terms was chosen by relying on the Bayesian Information Criterion (BIC).

Appendix D: The large-scale dynamic factor model

In addition to the small-scale dynamic factor model as competing model, we also consider a large-scale dynamic factor model (large DFM). Large factor models use a small number of factors to capture the co-movement of a high-dimensional set of time-series. High dimensionality poses challenges. The first concerns the data frequency and missing observations in general. The second concerns the identification of the latent factors.

In order to allow for a decent comparison of the predictive accuracy of the CDFM to the large DFM, we use all variables of the CDFM in the large DFM. We extract the latent factors by relying on principal component analysis. This approach is easy to implement, and, given that the cross-section and time dimension are large, provides consistent estimates under quite general assumptions. It suffers, however, from one main drawback: the dataset must be balanced, that is, the start and end points have to be the same across all observable variables and all data-series must have the same frequency so that missing observations do not arise. To this purpose, we consider a quarterly frequency of the data and of the large DFM alike. We transform all monthly series into quarterly series by considering a three-months average. The sample used for the estimation starts in 2007 as missing observations prior to this year would otherwise impede the estimation. All series enter the large DFM contemporaneously; hence, in contrast to the small DFM, we do not allow for temporal displacements within the data series. We rely on Bai and Ng (2002) to determine the number of factors, and specify a finite-order VAR model to approximate the dynamics of the latent factors.

We consider the principal component methods and maximum likelihood methods to estimate the large DFM (see Bai and Wang 2016, for further details). To this purpose, we again standardize all data-series prior to the estimation. The estimation relies on a two-step procedure in which the latent factors are estimated in a first step, and the VAR model in the second step. As highlighted by Doz et al. (2011), this procedure yields consistent estimates even when the static factor model is misspecified with respect to some of its dynamic elements. We establish forecasts from the large DFM in the same form as done for the other models. This allows for an adequate comparison of the large DFM’s forecasts with those of the other models, keeping in mind, though, the different underlying frequency.

Appendix E: The mixed-data-sampling (MIDAS) regression model

We complete the set of competing models with a mixed-data-sampling (MIDAS) regression model for GDP using the set of indicators from the small-scale dynamic factor model detailed in Sect. 3. MIDAS regression was developed by Ghysels et al. (2006) as a means of predicting a single low-frequency time series (quarterly GDP growth) with multiple high-frequency indicators (monthly indicators). The key feature of MIDAS regression is the distribution of the current and past values of the high-frequency indicators that effectively yields different forecasting models for each forecast horizon. We found that a MIDAS specification with simple uniform distribution (fixed weighting scheme) produces the most accurate GDP forecasts at all frequencies. To facilitate comparison with the CDFM and the other competing models, the performance of MIDAS regression was measured using the NRMSE and MAE measures of forecast error. Forecasting quarterly GDP using MIDAS regression requires future values of monthly indicators as input. The forecasts of the monthly indicators were determined using optimally selected ARMA models, with model selection based on BIC.

There are several important conceptual differences between a MIDAS regression and a DFM used as a building block of the CDFM, or as direct competing model in Sect. 3. First, a DFM is based on a system of equations, whereas a MIDAS regression involves a single equation in reduced form. Bai et al. (2013) argue that a MIDAS model can be less efficient and more prone to specification errors than a DFM estimated using the Kalman filter. Second, because a MIDAS regression requires forecasts for the indicators as input, it must be supplemented by auxiliary models for the indicators. This makes the choice of a MIDAS regression less practical than that of a DFM for the purpose of designing a cluster of such models such as the CDFM. Finally, a DFM estimated using the Kalman filter can easily cope with ragged edges and missing observations at different frequencies, as well as changing frequencies within a time series, which poses a problem to MIDAS.

Appendix F: Additional figures and tables

See Tables 10, 11, 12, 13, 14, 15 and 16.

Table 10 Summary statistics

Table 11 Ordering of behavioral models in the CDFM

Table 12 Aggregator models in the CDFM

Table 13 Factor correlation and MAE by core DFM (2007–2018)

Table 14 MAE by aggregator DFM (2007–2018)

Table 15 Diebold–Mariano test for GDP forecasts (2007–2018)

Table 16 Coroneo–Iacone test for GDP forecasts (2007–2018)