Evaluating FOMC forecast ranges: an interval data approach

Abstract

The Federal Open Market Committee (FOMC) of the U.S. Federal Reserve publishes the range of members’ forecasts for key macroeconomic variables, but not the distribution of forecasts within this range. To evaluate these projections, previous papers compare the midpoint of the range with the realized outcome. This paper proposes an alternative approach to forecast evaluation that takes account of the interval nature of projections. It is shown that using the conventional Mincer–Zarnowitz approach to evaluate FOMC forecasts misses important information contained in the width of the forecast interval. This additional information plays a minor role at short forecast horizons but turns out to be of sometimes crucial importance for longer-horizon forecasts. For 18-month-ahead forecasts, the variation of members’ projections contains information that is more relevant for explaining future inflation than information embodied in the midpoint. Likewise, when longer-range forecasts for real GDP growth and the unemployment rate are considered, the width of the forecast interval comprises information over and above the one given by the midpoint alone.

Appendices

Data appendix: measuring the outcomes

We measure the outcomes for the three variables being forecast using real-time data, closely following the procedure employed by Romer and Romer (2008) in a related study that comprises the period 1979–2001. The exact computations and data sources are described in the following.

1.1 Real output growth rate

Real-time actuals are 4Q-on-4Q growth rates calculated using BEA’s “final (third)” Q4 estimates of real GNP/GDP, typically released in March and first published in the March or April issue of the Survey of Current Business (SCB). The data were downloaded as monthly vintages from the Federal Reserve Bank of Philadelphia’s Real-Time Data Research Center (RTDSM) at http://www.philadelphiafed.org/research-and-data/real-time-center/real-time-data/data-files/ROUTPUT/. We compute percentage changes using numbers from the same SCB issue (i.e., the same mid-April vintage from the real-time data set). For instance, our figure for real GDP growth in 1999 is computed as the percentage change in the estimates of real GDP from 1998Q4 to 1999Q4 that are contained in the NIPA tables of the April 2000 SCB issue.

The formerly mentioned real-time actuals are used throughout for all years from 1983 until 2011 for all three forecast horizons under study, except for the years 1991 and 1992. This is due to the fact that the MPRs until (including) July 1991 contained forecasts for growth in real GNP, not real GDP, whereas the real-time data series as available in the RTDSM report values for real GDP only starting with the 1991M12 vintage. We follow Romer and Romer (2008) in solving this issue as follows: Because the 6-, 12-, and 18-month projections for 1991Q4 all forecasted growth in real GNP, but the actual time series mentioned above gives real-time values for real GDP only, the actual for all three forecast horizons for 1991Q4 is calculated using BEA’s “final (third)” Q4 estimates for real GNP in 1990Q4 and 1991Q4 taken from the respective SBC issue 3/92. Similarly, the actual for the 18-month projections for 1992Q4 is calculated using BEA’s “final (third)” Q4 estimates for real GNP in 1991Q4 and 1992Q4 as given in the respective SBC issue 3/93. Archived digital copies of hardcover SCB issues can be found online on the BEA homepage at http://www.bea.gov/scb/date_guide.asp from 1921 till today.

Our outcome measures match the measures being forecast as closely as possible. One issue remaining where we cannot completely reconcile the actual with the forecast concerns the change in base years. This involves the switches in reporting standards from 1990 to 1991 and from 1994 to 1995, as described in detail in the data appendix of Romer and Romer (2008) available online at http://www.aeaweb.org/articles.php?doi=10.1257/aer.98.2.230.

1.2 Inflation rate

Real-time actuals are 4Q-on-4Q growth rates calculated using estimates from several sources, depending on whether the measure of inflation is based on a variable in the NIPA tables or not.

For NIPA variables (i.e., inflation measured by the implicit GNP deflator, the PCE chain-type price index, or the PCE core chain-type price index), we use BEA’s “final (third)” Q4 estimates typically released in March and first published in the March or April issue of the SCB. We compute percentage changes using numbers from the same SCB issue. For instance, we compute our figure for the inflation rate in 1986 as the percentage change in the implicit GNP deflator using the estimates of real and nominal GDP from 1985Q4 to 1986Q4 that are all contained in the NIPA tables of the March 1987 SCB issue. These “final (third)” estimates can be downloaded as monthly vintages from the Philadelphia Fed’s RTDSM at http://www.philadelphiafed.org/research-and-data/real-time-center/real-time-data/data-files/ROUTPUT/ and http://www.philadelphiafed.org/research-and-data/real-time-center/real-time-data/data-files/NOUTPUT/, respectively. Our figure for the inflation rate in 2002 is computed as the percentage change in the estimates of the PCE chain-type price index from 2001Q4 to 2002Q4, which are contained in the NIPA tables of the April 2003 SCB issue. These “final (third)” Q4 estimates can be downloaded as monthly vintages from the Federal Reserve Bank of St. Louis’s ALFRED database at http://alfred.stlouisfed.org/series?seid=PCECTPI&cid=21. As regards the real-time actuals for (percentage changes in) the PCE core chain-type price index, “final (third)” Q4 estimates can be downloaded as vintages either from the Philadelphia Fed RTDSM at http://www.philadelphiafed.org/research-and-data/real-time-center/real-time-data/data-files/PCONX/ or from the ALFRED database at http://alfred.stlouisfed.org/series?seid=JCXFE&cid=21. Both sources in general provide the same quarterly estimates taken from the respective SCB issues, the RTDSM offering monthly vintages starting in February 1996 and the ALFRED database offering monthly vintages starting in July 1999. Note that all real-time actuals can also be found in the original SCB issues available online as scans or digital issues via the BEA homepage at http://www.bea.gov/scb/date_guide.asp.

For the CPI as a non-NIPA variable that is not subject to immediate revisions, we use the 4Q-on-4Q percentage changes as first reported by the Bureau of Labor Statistics (BLS) in January. Since the BLS publishes monthly figures, but does not construct its own quarterly averages, following Romer and Romer (2008), we use the figures for actual Q4-to-Q4 CPI inflation from the first Greenbook prepared after the release of the December data. This is always the Greenbook prepared in late January or the very beginning of February. Historical Greenbooks are published online by the Board of Governors of the Federal Reserve System at http://www.federalreserve.gov/monetarypolicy/fomc_historical.htm. The 5-year publication lag is not relevant for our study, since we need actual CPI data for the period 1989–2000 only. In contrast, the Philadelphia Fed’s RTDSM provides real-time CPI data in monthly frequency as either monthly vintages starting in November 1998 or quarterly vintages starting in 1994Q3 only.

1.3 Unemployment rate

Real-time actuals for the civilian unemployment rate are averages for the fourth quarter of the relevant year. We obtain real-time data from the Philadelphia Fed’s RTDSM available as quarterly vintages at http://www.philadelphiafed.org/research-and-data/real-time-center/real-time-data/data-files/RUC/. This data represents the values from the BLS’s February, May, August, and November issues of Employment and Earnings. We use the monthly estimates available in mid-February of each year for the previous October, November, and December and calculate the arithmetic mean of these three values. The unemployment rate is hardly revised; for the time period considered here, in particular, the estimates from the February vintages employed by us do not differ at all from those of the May vintages available three months later. Comparisons with other potential real-time unemployment rates reveal that the mean values computed by us for the fourth quarter of each year are virtually identical to the Q4 estimates from the first Greenbook prepared after the release of the BLS December data (i.e., the Greenbook available in late January or the very beginning of February). The monthly real-time figures used by us do not differ from the respective monthly values contained in the January issue of the BLS’s Monthly Labor Review, either, the latter being archived at http://www.bls.gov/opub/mlr/archive.htm for editions starting in 2000.

Technical appendix: estimating the interval data model

This appendix is intended to provide details on the estimation of the interval model “$\mathrm{M}_\mathrm{G}$” as given by Eq. (2) for the general case in which the response variable $Y_t$ is not necessarily an interval degenerated into a real number. More profound information on the model itself and the estimation process can be found in Blanco-Fernández et al. (2012), who also present the model’s cognate predecessors being the simple, less flexible linear regression models for interval data of Blanco-Fernández et al. (2011) and González-Rodríguez et al. (2007). The reader may also refer to these papers for some preliminary concepts of the interval data framework including the basics of interval arithmetic.

The interval data model “$\mathrm{M}_\mathrm{G}$” as given in Eq. (2) can also be written as

$$\begin{aligned} Y_{t} = \varvec{x}_{t}^{\prime } \cdot \varvec{\gamma } + \mathcal E _{t}, \end{aligned}$$

(8)

where $\varvec{x}_{t}^{\prime } = (X^{\mathrm{Mid}}_{t}, \, X^\mathrm{Spr}_{t}, \, X^{\mathrm{Spr}_2}_{t}, \, X^{\mathrm{Mid_2}}_{t})$, and $\varvec{\gamma } = (\gamma _1, \gamma _2, \gamma _3, \gamma _4)'$; hence, the equivalent matrix expression—which consolidates all $T$ random interval pairs $\{(X_t, Y_t)\}_{t=1,\,\ldots \,,T}$ of the sample considered—is given by

$$\begin{aligned} \varvec{y} = \varvec{X} \cdot \varvec{\gamma } + \varvec{\varepsilon }, \end{aligned}$$

(9)

where $\varvec{y} = (Y_1,Y_2,\,\ldots \,,{}Y_T)'$ and $\varvec{\varepsilon } = (\mathcal E _1,\mathcal E _2,\,\ldots \,,{}\mathcal E _T)'$ are $(T\!\times \!1)$ column vectors collecting the $T$ interval-valued response variables and interval-valued errors, respectively, and $\varvec{X} = (\varvec{x}^{\mathrm{Mid}}, \varvec{x}^\mathrm{Spr}, \varvec{x}^{\mathrm{Spr}_2}, \varvec{x}^{\mathrm{Mid_2}})$ is a $(T \times 4)$ matrix containing the four $(T \times 1)$ interval-valued column vectors $\varvec{x}^{\mathrm{Mid}} = (X^{Mid}_1,X^{\mathrm{Mid}}_2,{}\,\ldots \,,{}X^{\mathrm{Mid}}_T)'$, $\varvec{x}^\mathrm{Spr} = (X^\mathrm{Spr}_1,X^\mathrm{Spr}_2,{}\,\ldots \,,{}X^\mathrm{Spr}_T)'$, $\varvec{x}^{\mathrm{Spr}_2} = (X^{\mathrm{Spr}_2}_1,X^{\mathrm{Spr}_2}_2,\,\ldots \,,{}X^{\mathrm{Spr}_2}_T)'$, and $\varvec{x}^{\mathrm{Mid_2}} = (X^{\mathrm{Mid_2}}_1,X^{\mathrm{Mid_2}}_2,\,\ldots \,,{}X^{\mathrm{Mid_2}}_T)'$.

In order to estimate the coefficient vector $\varvec{\gamma }$ in (9), a least-squared-error-type minimization problem is solved, which features two important characteristics setting it apart from the “standard” unrestricted least-squares method. The first distinctive feature worth mentioning is one restricting the search space for two of the four model parameters of interest. Note that for the two explanatory intervals of positive width given in Eq. (2), by definition, the identities $X^\mathrm{Spr}_{t}=-X^\mathrm{Spr}_{t}$ and $X^{\mathrm{Mid_2}}_{t}=-X^{\mathrm{Mid_2}}_{t}$ hold, which leads to the interval model as specified in Eq. (2) being not unique. Hence, there are four equivalent representations for model “$\mathrm{M}_\mathrm{G}$”, which allows—without loss of generality—to consider the parameters $\gamma _2$ and $\gamma _4$ to be nonnegative. The second peculiarity to be noted regards the class of nonempty, closed, and bound intervals in $\mathbb R$ employed in the linear regression, denoted by $\mathcal K _c(\mathbb R )$. It is a semilinear space as, in general, the existence of a symmetric element with respect to the addition is not guaranteed. For this reason, it is useful to consider the so-called Hukuhara difference (Hukuhara 1967), which is the most common difference when dealing with intervals, even though it does not always exist.¹² Taking into account that the model-implied estimated interval errors given by the corresponding Hukuhara differences $Y_{t} -_{\small _H} \varvec{x}_{t}' \cdot \varvec{\widehat{\gamma }}$ have to exist (i.e., the residual has to be a well-defined interval whose supremum is not smaller than its infimum, for each observation $t=1,2,\ldots ,T$), the minimization problem becomes a constrained one, where the set of constraints is devoted to assure the existence of the Hukuhara differences and hence the residuals.

Thus, the constrained minimization problem at hand is

$$\begin{aligned} \min _{\varvec{\gamma }} \Big (\varvec{y}^{*} - \varvec{X}^{*} \cdot \varvec{\gamma }\Big )'\Big (\varvec{y}^{*} - \varvec{X}^{*} \cdot \varvec{\gamma }\Big ) \quad {\mathrm{s.t.}} \quad \varvec{X}_{s} \cdot \varvec{\gamma } \le \varvec{\mathrm{spr} Y}, \end{aligned}$$

(10)

for which the real-valued vectors and matrices are defined as

$$\begin{aligned} \varvec{X}^{*}&= \begin{pmatrix} \varvec{X}_{m}^{*} \\ \varvec{X}_{s}^{*} \end{pmatrix} \quad \mathrm{is}\; \mathrm{a}\, (2T\times 4)\, \mathrm{matrix},\; \mathrm{wherein}\\ \varvec{X}_{m}^{*}&= \varvec{X}_{m} - \varvec{1} \cdot \overline{\varvec{x}_{m}}',\, \mathrm{wherein}\\ \varvec{X}_{m}&= \begin{pmatrix} \varvec{{\mathrm{mid}} X}, \varvec{0}, \varvec{{\mathrm{spr}} X}, \varvec{0} \end{pmatrix},\, \mathrm{with} \\ \varvec{{\mathrm{mid}} X}&= \Big (\mathrm{mid}\,X_1, \mathrm{mid}\,X_2, \,\ldots \, , \mathrm{mid}\,X_T\Big )', \\ \varvec{{\mathrm{spr}} X}&= \Big (\mathrm{spr}\,X_1, \mathrm{spr}\,X_2, \,\ldots \, , \mathrm{spr}\,X_T\Big )',\\&\varvec{0} \qquad \mathrm{being}\; \mathrm{a}\, (T\times 1)\, \mathrm{vector}\; \mathrm{of}\; \mathrm{zeros}, \\ \overline{{\varvec{x}}_{m}}&= \Big (\overline{\mathrm{mid}\,X}, 0, \overline{\mathrm{spr}\,X}, 0\Big )',\, \mathrm{with} \\ \overline{\mathrm{mid}\,X}&= \displaystyle \frac{1}{T} \sum \limits _{t=1}^{T} \mathrm{mid}\,X_t, \\ \overline{\mathrm{spr}\,X}&= \displaystyle \frac{1}{T} \sum \limits _{t=1}^{T} \mathrm{spr}\,X_t, \\&\varvec{1} \qquad \mathrm{being}\; \mathrm{a}\, (T\times 1)\, \mathrm{vector}\; \mathrm{of}\; \mathrm{ones},\\ \varvec{X}_{s}^{*}&= \varvec{X}_{s} - \varvec{1} \cdot \overline{\varvec{x}_{s}}',\, \mathrm{wherein} \\ \varvec{X}_{s}&= \begin{pmatrix} \varvec{0},&\varvec{{\mathrm{spr}} X},&\varvec{0},&|\varvec{\mathrm{mid}\,X}| \end{pmatrix},\, \mathrm{with} \\ \varvec{{\mathrm{spr}} X}&= \Big (\mathrm{spr}\,X_1, \mathrm{spr}\,X_2, \ldots , \mathrm{spr}\,X_T\Big )', \\ |\varvec{\mathrm{mid}\,X}|&= \Big (|\mathrm{mid}\,X_1|, |\mathrm{mid}\,X_2|,\ldots , |\mathrm{mid}\,X_T|\Big )', \\&\varvec{0} \quad \mathrm{being}\; \mathrm{a}\, (T\times 1)\, \mathrm{vector}\; \mathrm{of}\; \mathrm{zeros}, \\ \overline{\varvec{x}_{s}}&= \Big (0, \overline{\mathrm{spr}\,X}, 0, \overline{|\mathrm{mid}\,X|}\Big )',\, \mathrm{with} \\ \overline{\mathrm{spr}\,X}&= \displaystyle \frac{1}{T} \sum \limits _{t=1}^{T} \mathrm{spr}\,X_t, \\ \overline{|\mathrm{mid}\,X|}&= \displaystyle \frac{1}{T} \sum \limits _{t=1}^{T} |\mathrm{mid}\,X_t|, \\&\varvec{1} \qquad \mathrm{being}\; \mathrm{a}\, (T\times 1)\, \mathrm{vector}\; \mathrm{of}\; \mathrm{ones}, \end{aligned}$$

$\varvec{\gamma } = \Big (\gamma _1,\, \gamma _2,\, \gamma _3,\, \gamma _4\Big )'$, with $\gamma _1,\gamma _3 \in \mathbb R$ and $\gamma _2,\gamma _4 \in \mathbb R _{0}^{+}$,

$$\begin{aligned} \varvec{y}^{*}&= \begin{pmatrix} \varvec{y}_{m}^{*} \\ \varvec{y}_{s}^{*} \end{pmatrix} \qquad \mathrm{is}\; \mathrm{a}\, (2T\times 1)\, \mathrm{vector},\; \mathrm{wherein} \\ \varvec{y}_{m}^{*}&= \varvec{\mathrm{mid} Y} - \overline{\mathrm{mid}\,Y} \cdot \varvec{1},\, \mathrm{with} \\ \varvec{\mathrm{mid} Y}&= \Big (\mathrm{mid}\,Y_1, \mathrm{mid}\,Y_2, \,\ldots \, , \mathrm{mid}\,Y_T\Big )', \\ \overline{\mathrm{mid}\,Y}&= \displaystyle \frac{1}{T} \sum \limits _{t=1}^{T} \mathrm{mid}\,Y_t, \\&\varvec{1} \qquad \mathrm{being}\; \mathrm{a}\, (T\times 1)\, \mathrm{vector}\; \mathrm{of}\; \mathrm{ones}, \\ \varvec{y}_{s}^{*}&= \varvec{\mathrm{spr} Y} - \overline{\mathrm{spr}\,Y} \cdot \varvec{1},\, \mathrm{with} \\ \varvec{\mathrm{spr} Y}&= \Big (\mathrm{spr}\,Y_1, \mathrm{spr}\,Y_2, \ldots , \mathrm{spr}\,Y_T\Big )', \\ \overline{\mathrm{spr}\,Y}&= \displaystyle \frac{1}{T} \sum \limits _{t=1}^{T} \mathrm{spr}\,Y_t, \\&\varvec{1} \qquad \mathrm{being}\; \mathrm{a}\, (T\times 1)\, \mathrm{vector}\; \mathrm{of}\; \mathrm{ones}. \end{aligned}$$

As the objective function has quadratic shape and the inequality constraints are linear, some standard routines from numerical analysis can be used to solve the minimization problem. We implement a Matlab code based on the Karush–Kuhn–Tucker conditions to obtain the regression coefficients.