Forecasting US Unemployment with Radial Basis Neural Networks, Kalman Filters and Support Vector Regressions

Abstract

This study investigates the efficiency of the radial basis function neural networks in forecasting the US unemployment and explores the utility of Kalman filter and support vector regression as forecast combination techniques. On one hand, an autoregressive moving average model, a smooth transition autoregressive model and three different neural networks architectures, namely a multi-layer perceptron, recurrent neural network and a psi sigma network are used as benchmarks for our radial basis function neural network. On the other hand, our forecast combination methods are benchmarked with a simple average and a least absolute shrinkage and selection operator. The statistical performance of our models is estimated throughout the period of 1972–2012, using the last 7 years for out-of-sample testing. The results show that the radial basis function neural network statistically outperforms all models’ individual performances. The forecast combinations are successful, since both Kalman filter and support vector regression techniques improve the statistical accuracy. Finally, support vector regression is found to be the superior model of the forecasting competition. The empirical evidence of this application are further validated by the use of the modified Diebold–Mariano test.

Appendices

Appendix 1: NNs’ Structure and Training Characteristics

This appendix section briefly describes the structure of the three traditional NNs used to benchmark the RBFNN. It also includes a summary of the training characteristics of all four NNs.

Firstly, a typical MLP model is shown in Fig. 4.

Fig. 4

A single output, fully connected MLP model (bias nodes are not shown for simplicity) where $x_t ^{[n]}\left( {n=1,2,\ldots ,k+1} \right)$ are the inputs; (including the input bias node) at time $t$; $h_t ^{[m]}\left( {m=1,2,\ldots ,j+1} \right)$ are the hidden nodes outputs; $\hat{{Y}}_t$ is the MLP output (target value); $u_{jk}, w_{j }$ are the network weights;

is the transfer sigmoid function $S\left( x \right) =\frac{1}{1+e^{-x}}$;

is a linear function $F\left( x \right) =\sum _i {x_i}$

The error function to be minimized is:

$$\begin{aligned} E\left( {u_{jk} ,w_j } \right) =\frac{1}{T}\sum _{t=1}^T {\left( {Y_t -\hat{{Y}}_t \left( {u_{jk} ,w_j } \right) } \right) ^{2}} \end{aligned}$$

(18)

Secondly, the simple architecture of an RNN is presented in Fig. 5.

Fig. 5

RNN with two nodes in the hidden layer where $x_t^{[n]} (n=1,2,\ldots ,k+1),u_t^{[1]} ,u_t^{[2]}$ are the RNN inputs at time $t$ (including bias node); $\tilde{y}_t$is the output of the RNN; $d_t^{[f]} (f=1,2)$ and $w_t^{[n]} (n=1,2,\ldots ,k+1)$ are the weights of the network; $U_t^{[f]} ,f=(1,2)$ is the output of the hidden nodes at time $t$;

is the transfer sigmoid function $S\left( x \right) =\frac{1}{1+e^{-x}}$;

is a linear function $F\left( x \right) =\sum _i {x_i }$

The error function to be minimized is:

$$\begin{aligned} E\left( {d_t ,w_t } \right) =\frac{1}{T}\sum _{t=1}^T {\left( {y_t -\tilde{y}_t \left( {d_t ,w_t } \right) } \right) ^{2}} \end{aligned}$$

(19)

Thirdly, Fig. 6 describes the PSN architecture.

Fig. 6

A PSN with one output layer, where $x_{t} (n=1,2,{\ldots },k+1)$ are the model inputs; $y_t ,\tilde{y}_t$ are the PSN input and output respectively; $w_{j} (j=1,2\ldots ,k)$ are the adjustable weights ($k$ is the desired order of the network); the hidden layer activation function: $h\left( x \right) =\sum _i {x_i}$; the output sigmoid activation function ($c$ the adjustable term) $\sigma (x)=\frac{1}{1+e^{-xc}}$

The error function minimized in this case:

$$\begin{aligned} E\left( {c,w_j } \right) =\frac{1}{T}\sum _{t=1}^T {\left( {y_t -\tilde{y}_t \left( {w_k ,c} \right) } \right) ^{2}} \end{aligned}$$

(20)

Finally, Table 6 summarizes the training characteristics of the four NN architectures used in this forecasting competition.

Table 6 The NNs training characteristics

Appendix 2: Statistical Performance Measures

The statistical performance measures are calculated as shown in Table 7.

Table 7 Statistical Performance Measures and Calculation