The robustness of neural networks for modelling and trading the EUR/USD exchange rate at the ECB fixing
Abstract
The objective of this study is to investigate the use, the stability and the robustness of alternative novel neural network (NN) architectures when applied to the task of forecasting and trading the Euro/Dollar (EUR/USD) exchange rate using the European Central Bank (ECB) fixing series with only autoregressive terms as inputs. This is achieved by benchmarking the forecasting performance of three different NN designs representing a Higher Order Neural Network (HONN), a Recurrent Neural Network (RNN) and the classic Multilayer Perceptron (MLP) with some traditional techniques, either statistical, such as an autoregressive moving average model, or technical, such as a moving average convergence/divergence model, plus a naïve strategy. More specifically, the trading performance of all models is investigated in a forecast and trading simulation on the EUR/USD ECB fixing time series over the period January 1999 – August 2008 using the last 8 months for out-of-sample testing. Our results in terms of their robustness and stability are compared with a previous study by the authors, who apply the same models and follow the same methodology forecasting the same series, using as out-of-sample the period from July 2006 to December 2007. As it turns out, the HONN and MLP networks present a robust performance and do remarkably well in outperforming all other models in a simple trading simulation exercise in both studies. Moreover, when transaction costs are considered and leverage is applied, the same networks continue to outperform all other NN and traditional statistical models in terms of annualised return – a robust and stable result as it is identical to that obtained by the authors in their previous study, examining a different period for the series.
Keywords
higher order neural networks recurrent neural networks leverage multilayer perceptron networks quantitative trading strategiesINTRODUCTION
Neural networks (NNs) are an emergent technology with an increasing number of real-world applications, including finance.^{1} However, their numerous limitations and the contradicting empirical evident around their forecasting power often create scepticism with regard to their use among practitioners.
The objective of this study is to investigate not only the use of several new NNs techniques that try to overcome these limitations, but also the stability and robustness of their performance. This is achieved by benchmarking three different NN architectures representing a Multilayer Perceptron (MLP), a Higher Order Neural Network (HONN) and a Recurrent Neural Network (RNN). Their trading performance on the Euro/Dollar (EUR/USD) time series is investigated, and is compared with some traditional statistical or technical methods, such as an autoregressive moving average (ARMA) model or a moving average convergence/divergence (MACD) model and a naïve strategy. In terms of the stability and robustness of our findings, we compare them with the conclusions of Dunis et al,^{2} who apply the same models and follow the same methodology to forecast the same series, but use a different out-of-sample period. With regard to the inputs of our NNs, we use the same selection of inputs and lags as those of Dunis et al.^{2} Similarly, our MACD model is identical to that of of Dunis et al;^{2} our ARMA model, however, is different, as we need all coefficients to be significant in the new in-sample period.
Moreover, our conclusions can supplement not only those of Dunis et al,^{2} but also those of Dunis and Chen^{3} and Dunis and Williams,^{4} who conduct similar forecasting comparisons between the EUR/USD and the Dollar/Japanese yen (USD/JPY) foreign exchange rates using similar networks but with multivariate series as inputs.
With regard to our data, the EUR/USD daily fixing is published by the European Central Bank (ECB) and is a tradable quantity, as it is possible to leave orders with a bank and trade on that basis.
As it turns out, the MLP and HONN demonstrate a remarkable performance and outperform the other models in a simple trading-simulation exercise. Moreover, when transaction costs are considered and leverage is applied, the MLP and HONN models continue to outperform all other NN and traditional statistical models in terms of annualised return. As these results are identical to those of Dunis et al,^{2} who follow the same methodology but for a different period of the EUR/USD ECB fixing series, we can argue that the forecasting superiority of the HONN and MLP is stable and robust over time. In terms of the RNNs, their poor performance in this research may be a result of their inability to provide adequate results when only autoregressive terms are used as inputs.
The remainder of the article is organised as follows. In the next section, we present the literature relevant to the RNN and HONN. The section after that describes the data set used for this research and its characteristics. An overview of the different NN models and statistical techniques is provided in the subsequent section. The penultimate section presents the empirical results of all the models considered and investigates the possibility of improving their performance with the application of leverage. In that section we also test the robustness of our models. The final section provides some concluding remarks.
LITERATURE REVIEW
The objective of this study is to apply some of the most promising new NNs architectures that have been developed recently with the purposes of overcoming the numerous limitations of the more classic neural architectures and assessing whether they can achieve a higher performance in a trading simulation using only autoregressive series as inputs.
RNNs have an activation feedback that embodies short-term memory, allowing them to learn extremely complex temporal patterns. Their superiority against feedfoward networks when performing nonlinear time series prediction is well documented in studies by Connor and Atlas^{5} and Adam et al.^{6} In financial applications, Kamijo and Tanigawa^{7} apply them successfully to the recognition of stock patterns of the Tokyo Stock Exchange, while Tenti^{8} achieve good results using RNNs to forecast the exchange rate of the Deutsche Mark. Tino et al^{9} use them to successfully trade the volatility of the DAX and FTSE 100 using straddles, while Dunis and Huang,^{10} using continuous implied volatility data from the currency options market, obtain remarkably good results from their GBP/USD and USD/JPY exchange rate volatility trading simulation.
HONNs were first introduced by Giles and Maxwell^{11} as a fast learning network with increased learning capabilities. Although their function approximation superiority over the more traditional architectures is well documented in the literature (see, among others, studies by Redding et al,^{12} Kosmatopoulos et al^{13} and Psaltis et al^{14}), their use in finance thus far has been limited. This changed when scientists started to investigate not only the benefits of NNs against the more traditional statistical techniques, but also the differences between the different NN model architectures. Practical applications have now verified the theoretical advantages of HONNs by demonstrating their superior forecasting ability, and have put them at the front line of applied research in financial forecasting. For example, Dunis et al^{15} use them to successfully forecast the gasoline crack spread, while Fulcher et al^{16} apply HONNs to forecast the AUD/USD exchange rate, achieving 90 per cent accuracy. However, Dunis et al^{17} show that, in the case of the futures spreads and for the period under review, the MLPs performed better compared with HONNs and RNNs. Moreover, Dunis et al,^{18} who also studied the EUR/USD series for a period of 10 years, demonstrate that when multivariate series are used as inputs, the HONNs, RNNs and MLP networks have a similar forecasting power. Finally, Dunis et al,^{2} in a study with a methodology identical to that used in this study, demonstrate that the HONNs and MLP network are superior in forecasting the EUR/USD ECB fixing until the end of 2007, compared to the RNNs, an ARMA model, an MACD model and a naïve strategy.
THE EUR/USD EXCHANGE RATE AND RELATED FINANCIAL DATA
The ECB publishes a daily fixing for selected EUR exchange rates: these reference mid-rates are based on a daily concentration procedure between central banks within and outside the European System of Central Banks, which normally takes place at 14:15 hours ECB time. The reference exchange rates are published both by electronic market information providers and on the ECB's website shortly after the concentration procedure has ended. Although only a reference rate, many financial institutions are ready to trade at the EUR fixing, and it is therefore possible to leave orders with a bank for business to be transacted at this level.
The EUR/USD data set
Name of period | Trading days | Beginning | End |
---|---|---|---|
Total data set | 2474 | 4 January 1999 | 29 August 2008 |
In-sample data set | 2304 | 4 January 1999 | 31 December 2007 |
Out-of-sample data set (Validation set) | 170 | 2 January 2008 | 29 August 2008 |
The observed EUR/USD time series is non-normal (Jarque-Bera statistics confirm this at the 99 per cent confidence interval), containing slight skewness and high kurtosis. It is also non-stationary, and hence we decided to transform the EUR/USD series into a stationary daily series of rates of return,^{20} using the following formula:
Explanatory variables
Number | Variable | Lag |
---|---|---|
1 | EUR/USD exchange rate return | 1 |
2 | EUR/USD exchange rate return | 2 |
3 | EUR/USD exchange rate return | 3 |
4 | EUR/USD exchange rate return | 7 |
5 | EUR/USD exchange rate return | 11 |
6 | EUR/USD exchange rate return | 12 |
7 | EUR/USD exchange rate return | 14 |
8 | EUR/USD exchange rate return | 15 |
9 | EUR/USD exchange rate return | 16 |
10 | Moving average of the EUR/USD exchange rate return | 15 |
11 | Moving average of the EUR/USD exchange rate return | 20 |
12 | 1-day Riskmetrics volatility | 1 |
The neural networks data sets
Name of period | Trading days | Beginning | End |
---|---|---|---|
Total data set | 2474 | 4 January 1999 | 29 August 2008 |
Training data set | 1794 | 4 January 1999 | 31 December 2005 |
Test data set | 510 | 2 January 2006 | 31 December 2007 |
Out-of-sample data set (Validation set) | 170 | 2 January 2008 | 29 August 2008 |
FORECASTING MODELS
Benchmark models
In this study, we benchmark our NN models with three traditional strategies, namely, an ARMA model, an MACD model and a naïve strategy.
Naïve strategy
The naïve strategy simply takes the most recent period change as the best prediction of the future change, that is, a simple random walk. The model is defined by:
The performance of the strategy is evaluated in terms of trading performance via a simulated trading strategy.
Moving average
The moving average model is defined as:
The MACD strategy used is quite simple. Two moving average series are created with different moving average lengths. The decision rule for taking positions in the market is straightforward. Positions are taken if the moving averages intersect. If the short-term moving average intersects, the long-term moving average from below a ‘long’ position is taken. Conversely, if the long-term moving average is intersected from above, a ‘short’ position is taken.^{21}
The forecaster must use judgement when determining the number of periods n on which to base the moving averages. The combination that performed best over the in-sample subperiod was retained for out-of-sample evaluation. The model selected was a combination of the EUR/USD and its 24-day moving average, namely, n=1 and 24, respectively, or a (1, 24) combination. The performance of this strategy is evaluated solely in terms of trading performance.
ARMA model
ARMA models assume that the value of a time series depends on its previous values (the autoregressive component) and on previous residual values (the moving average component). For a full discussion of the procedure, refer to the studies by Box et al^{22} or Pindyck and Rubinfield.^{23}
The ARMA model takes the following form:
The output of the autoregressive moving average model used in this study is presented below
Variable | Coefficient | SE | t-Statistic | Prob. |
---|---|---|---|---|
C | 0.000128 | 0.000128 | 1.002067 | 0.3164 |
AR(1) | −1.216599 | 0.042416 | −28.68222 | 0.0000 |
AR(2) | −0.475940 | 0.081259 | −5.857069 | 0.0000 |
AR(7) | −0.139565 | 0.046558 | −2.997679 | 0.0027 |
AR(11) | 0.197421 | 0.055779 | 3.539363 | 0.0004 |
MA(1) | 1.213517 | 0.039315 | 30.86659 | 0.0000 |
MA(2) | 0.473609 | 0.077725 | 6.093399 | 0.0000 |
MA(7) | 0.152391 | 0.044815 | 3.400464 | 0.0007 |
MA(11) | −0.217830 | 0.054607 | −3.989048 | 0.0001 |
R^{2} | 0.008541 | Mean dependent var | 0.000122 | |
Adjusted R^{2} | 0.005067 | SD dependent var | 0.006169 | |
SE of regression | 0.006153 | Akaike info criterion | −7.339687 | |
Sum squared resid | 0.086446 | Schwarz criterion | −7.317159 | |
Log likelihood | 8420.282 | F-statistic | 2.458312 | |
Durbin–Watson stat | 1.997623 | Prob(F-statistic) | 0.011925 | |
Inverted AR roots | 0.75 | 0.65+0.46i | 0.65−0.46i | 0.23+0.77i |
0.23−0.77i | −0.21−0.82i | −0.21+0.82i | −0.73−0.67i | |
−0.73+0.67i | −0.94−0.20i | −0.94+0.20i | — | |
Inverted MA roots | 0.76 | 0.66+0.46i | 0.66−0.46i | 0.24+0.78i |
0.24−0.78i | −0.21+0.83i | −0.21−0.83i | −0.73−0.68i | |
−0.73+0.68i | −0.94+0.20i | −0.94−0.20i | — |
The selected ARMA model takes the following form:
Empirical results for the benchmark models
Trading performance of the benchmark models out-of-sample
NAIVE | MACD | ARMA | |
---|---|---|---|
Sharpe ratio (excluding costs) | 1.28 | −0.02 | −0.45 |
Annualised volatility (excluding costs) | 9.51% | 9.54% | 9.54% |
Annualised return (excluding costs) | 12.14% | −0.19% | −4.32% |
Maximum drawdown (excluding costs) | −3.69% | −9.85% | −11.39% |
Positions taken (annualised) | 108 | 28 | 185 |
Training subperiod trading performance
NAIVE | MACD | ARMA | MLP | RNN | HONN | |
---|---|---|---|---|---|---|
Sharpe ratio (excluding costs) | −0.10 | 0.51 | 1.17 | 0.46 | 0.16 | 0.48 |
Annualised volatility (excluding costs) | 10.36% | 10.44% | 10.42% | 9.97% | 10.38% | 10.40% |
Annualised return (excluding costs) | −1.07% | 5.31% | 12.19% | 4.62% | 1.67% | 4.95% |
Maximum drawdown (excluding costs) | −29.39% | −15.35% | −12.91% | −17.63% | −23.82% | −18.20% |
Positions taken (annualised) | 79 | 11 | 113 | 75 | 63 | 106 |
Trading simulation performance measures
Performance measure | Description | |
---|---|---|
Annualised return | R^{A}=252(1/N)Σ_{t=1}^{N}R_{t} | (A1) |
With R_{t} being the daily return | ||
Cumulative return | R^{C}=Σ_{t=1}^{N}R_{t} | (A2) |
Annualised volatility | (A3) | |
Sharpe Ratio | SR=(R^{A})/(σ_{A}) | (A4) |
Maximum negative value of Σ(R_{t}) over the period | ||
Maximum drawdown | (A5) |
As can be seen, the naïve strategy outperforms all other models by far. These results are surprising not only because of the simplicity of the naïve model, but also based on the training subperiod results (see Table A3). There the naïve strategy presents an annualised return of −1.07 per cent and a Sharpe ratio of −0.10. Moreover, with a closer look at the returns in our out-of-sample period, we observe that the positive and negative returns are clustered. In order to verify that the sequence of signs of the returns in the validation period is not random, we conduct the Wald–Wolfowitz or runs test for randomness.^{24} The test confirms that the sequence of signs is not random at 99 per cent and 95 per cent confidence intervals. Therefore, as the naïve strategy is only using today's return as a forecast for tomorrow, it is able to exploit this phenomenon and present a remarkable performance. This anomaly was not present in previous years, and this is why the performance of the naïve strategy in-sample is so much worse. In the circumstances, this phenomenon is accidental, and we have no reason to believe that it should continue in the future. We thus discard the results of the naïve strategy from our conclusions.
Neural networks
Neural networks exist in several forms in the literature. The most popular architecture is the MLP.
A standard NN has at least three layers. The first layer is called the input layer (the number of its nodes corresponds to the number of explanatory variables). The last layer is called the output layer (the number of its nodes corresponds to the number of response variables). An intermediary layer of nodes, the hidden layer, separates the input from the output layer. Its number of nodes defines the amount of complexity the model is capable of fitting. In addition, the input and hidden layer contain an extra node, called the bias node. This node has a fixed value of 1 and has the same function as the intercept in traditional regression models. Normally, each node of one layer has connections to all the other nodes of the next layer.
The network processes information as follows: the input nodes contain the value of the explanatory variables. As each node connection represents a weight factor, the information reaches a single hidden layer node as the weighted sum of its inputs. Each node of the hidden layer passes the information through a nonlinear activation function and passes it on to the output layer if the calculated value is above a threshold.
The training of the network (which is the adjustment of its weights in the way that the network maps the input value of the training data to the corresponding output value) starts with randomly chosen weights and proceeds by applying a learning algorithm called backpropagation of errors (for a more detailed discussion see the study by Shapiro^{25}). Backpropagation networks are the most common multilayer networks and are the most commonly used type in financial time series forecasting (see Kaastra and Boyd^{26}). The learning algorithm simply tries to find those weights that minimise an error function (normally the sum of all squared differences between target and actual values). As networks with sufficient hidden nodes are able to learn the training data (as well as their outliers and their noise) by heart, it is crucial to stop the training procedure at the right time to prevent overfitting (this is called ‘early stopping’). This can be achieved by dividing the data set into three subsets, respectively called the training and test sets, used for simulating the data currently available to fit and tune the model, and the validation set, used for simulating future values. The network parameters are then estimated by fitting the training data using the above-mentioned iterative procedure (backpropagation of errors). The iteration length is optimised by maximising the forecasting accuracy for the test data set. Our networks, which are specially designed for financial purposes, will stop training when the profit of our forecasts in the test subperiod is maximised. Then the predictive value of the model is evaluated applying it to the validation data set (out-of-sample data set).
The MLP model
The MLP network architecture
Empirical results of the MLP model
Out-of-sample trading performance of the Multilayer Perceptron (MLP)
MACD | ARMA | MLP | |
---|---|---|---|
Sharpe ratio (excluding costs) | −0.02 | −0.45 | 0.74 |
Annualised volatility (excluding costs) | 9.54% | 9.54% | 9.18% |
Annualised return (excluding costs) | −0.19% | −4.32% | 6.82% |
Maximum drawdown (excluding costs) | −9.85% | −11.39% | −5.78% |
Positions taken (annualised) | 28 | 185 | 107 |
Network characteristics
Parameters | MLP | RNN | HONN |
---|---|---|---|
Learning algorithm | Gradient descent | Gradient descent | Gradient descent |
Learning rate | 0.001 | 0.001 | 0.001 |
Momentum | 0.003 | 0.003 | 0.003 |
Iteration steps | 20 000 | 20 000 | 10 000 |
Initialisation of weights | N(0,1) | N(0, 1) | N(0, 1) |
Input nodes | 12 | 12 | 12 |
Hidden nodes (one layer) | 7 | 5 | 0 |
Output node | 1 | 1 | 1 |
As can be seen, the MLP outperforms our benchmark statistical models.
The RNN
Our next model is the RNN. While a complete explanation of RNN models is beyond the scope of this study, we present below a brief explanation of the significant differences between RNN and MLP architectures. For an exact specification of the RNN, see the study by Elman.^{28}
A simple RNN has activation feedback, which embodies short-term memory. The advantages of using RNNs over feedforward networks, for modelling nonlinear time series, have been well documented in the past. However, as described in by Tenti,^{8} ‘the main disadvantage of RNNs is that they require substantially more connections, and more memory in simulation, than standard backpropagation networks’ (p. 569), thus resulting in a substantial increase in computational time. However, having said this, RNNs can yield better results in comparison to simple MLPs owing to the additional memory inputs.
The RNN architecture
In short, the RNN architecture can provide more accurate outputs because the inputs are potentially taken from all previous values (see inputs U_{j−1}^{ [1]} and U_{j−1}^{ [2]} in Figure 4).
Empirical results of the RNN model
The RNNs are trained with gradient descent, as are the MLPs. However, the increase in the number of weights, as mentioned before, makes the training process extremely slow, taking 10 times as long as the MLP.
We follow the same methodology as we did for the MLPs for the selection of our optimal network. The characteristics of the network that we use are shown in Table A4, while a summary of the performance of the network in the training subperiod is presented in Table A3.
Out-of-sample trading performance results of the Recurrent Neural Network (RNN)
MACD | ARMA | MLP | RNN | |
---|---|---|---|---|
Sharpe ratio (excluding costs) | −0.02 | −0.45 | 0.74 | 0.40 |
Annualised volatility (excluding costs) | 9.54% | 9.54% | 9.18% | 9.54% |
Annualised return (excluding costs) | −0.19% | −4.32% | 6.82% | 3.83% |
Maximum drawdown (excluding costs) | −9.85% | −11.39% | −5.78% | −5.91% |
Positions taken (annualised) | 28 | 185 | 107 | 93 |
The HONN
HONNs were first introduced by Giles and Maxwell,^{11} and were called ‘Tensor Networks’. Although the extent of their use in finance has so far been limited, Knowles et al^{29} show that, with shorter computational times and limited input variables, ‘the best HONN models show a profit increase over the MLP of around 8 per cent’ on the EUR/USD time series (p. 7). For Zhang et al,^{30} a significant advantage of HONNs is that ‘HONN models are able to provide some rationale for the simulations they produce and thus can be regarded as “open box” rather then “black box”. Moreover, HONNs are able to simulate higher frequency, higher order non-linear data, and consequently provide superior simulations compared to those produced by ANNs (Artificial Neural Networks)’ (p. 188).
The HONN architecture
HONNs use joint activation functions; this technique reduces the need to establish the relationships among inputs when training. Furthermore, this reduces the number of free weights and means that HONNs are faster to train than even MLPs. However, because the number of inputs can be very large for higher-order architectures, orders of four and over are rarely used.
Another advantage of the reduction of free weights is that the problems of overfitting and local optima affecting the results of NNs can be largely avoided. For a complete description of HONNs, see the study by Knowles et al^{29}.
Empirical results of the HONN model
Out-of-sample trading performance results of the Higher Order Neural Network (HONN)
MACD | ARMA | MLP | RNN | HONN | |
---|---|---|---|---|---|
Sharpe ratio (excluding costs) | −0.02 | −0.45 | 0.74 | 0.40 | 0.86 |
Annualised volatility (excluding costs) | 9.54% | 9.54% | 9.18% | 9.54% | 9.49% |
Annualised return (excluding costs) | −0.19% | −4.32% | 6.82% | 3.83% | 8.16% |
Maximum drawdown (excluding costs) | −9.85% | −11.39% | −5.78% | −5.91% | −7.21% |
Positions taken (annualised) | 28 | 185 | 107 | 93 | 170 |
We can see that the HONN performs significantly better than the RNN and the traditional MLP models.
TRADING COSTS, LEVERAGE AND ROBUSTNESS
Thus far, we have presented the trading results of all our models without considering transaction costs. As some of our models trade quite often, taking transaction costs into account might change the whole picture.
We therefore introduce transaction costs as well as a leverage for each of our models. Moreover, we examine the robustness of our models by examining and comparing not only the results of our current research, but also the conclusions of a previous study in which we examined the same series with the same models but over a different period. For the purposes of comparability, we use the same selection of inputs for our NNs in both studies.^{32} The only difference is the out-of-sample period: here it is 2 January 2008 – 29 August 2008, while in the previous study it was 3 July 2006 – 31 December 2007. The purposes of this test are to validate the robustness of our models over time and to provide concrete empirical evidence of the forecasting power of our models.
Transaction costs
The transaction costs for a tradable amount, say USD 5–10 million, are approximately 1 pip (0.0001 EUR/USD) per trade (one way) between market makers. But as the EUR/USD time series considered here is a series of middle rates, the transaction cost is one spread per round trip.
With an average exchange rate of EUR/USD 1.532 for the out-of-sample period, a cost of 1 pip is equivalent to an average cost of 0.007 per cent per position.
Out-of-sample trading performance results with transaction costs (2 January 2008 – 29 August 2008)
MACD | ARMA | MLP | RNN | HONN | |
---|---|---|---|---|---|
Sharpe ratio (excluding costs) | −0.02 | −0.45 | 0.74 | 0.40 | 0.86 |
Annualised volatility (excluding costs) | 9.54% | 9.54% | 9.18% | 9.54% | 9.49% |
Annualised return (excluding costs) | −0.19% | −4.32% | 6.82% | 3.83% | 8.16% |
Maximum drawdown (excluding costs) | −9.85% | −11.39% | −5.78% | −5.91% | −7.21% |
Positions taken (annualised) | 28 | 185 | 107 | 93 | 170 |
Transaction costs | 0.20% | 1.30% | 0.75% | 0.65% | 1.19% |
Annualised return (including costs) | −0.39% | −5.62% | 6.07% | 3.18% | 6.97% |
We observe that although the HONN model presents higher transaction costs, it continues to outperform the other models in terms of annualised return. The MLP comes second, while the RNN demonstrates the third best performance. In contrast, the ARMA and MACD models have rather disappointing performances, as they both present negative annualised returns.
Out-of-sample trading performance results with transaction costs (3 July 2006 – 31 December 2007)
MACD | ARMA | MLP | RNN | HONNs | |
---|---|---|---|---|---|
Sharpe ratio (excluding costs) | 0.70 | −0.40 | 1.88 | 0.60 | 0.99 |
Annualised volatility (excluding costs) | 6.38% | 6.38% | 6.34% | 6.34% | 6.37% |
Annualised return (excluding costs) | 4.44% | −2.53% | 11.91% | 3.82% | 6.29% |
Maximum drawdown (excluding costs) | −4.73% | −10.12% | −5.05% | −4.64% | −4.37% |
Positions taken (annualised) | 20 | 189 | 109 | 167 | 80 |
Transaction costs | 0.16% | 1.51% | 0.87% | 1.33% | 0.64% |
Annualised return (including costs) | 4.28% | −4.04% | 11.04% | 2.48% | 5.65% |
We observe that in both periods the MLP and HONN models clearly outperform the other strategies. In the latest period, the HONN model has a better performance, while 1½ year before the MLP presented better results. In contrast, the ARMA model in both periods presents a rather disappointing trading performance. This empirical evidence allows us to argue that HONNs and MLPs have a consistent and better performance than the RNN, MACD and ARMA models in forecasting the ECB daily fixing of the EUR/USD.
Leverage to exploit low volatility
In order to further improve the trading performance of our models, we introduce a ‘level of confidence’ to our forecasts, that is, a leverage based on the test subperiod that takes into account the low volatility of the trading performance of our models. For the ARMA and MACD models, which show a negative return, we do not apply leverage. The leverage factors applied are calculated in such a way that each model has a common volatility of 10 per cent^{33} on the test data set.
Trading performance – Final results (2 January 2008 – 29 August 2008)^{35}
MACD | ARMA | MLP | RNN | HONN | |
---|---|---|---|---|---|
Sharpe ratio (excluding costs) | −0.02 | −0.45 | 0.74 | 0.40 | 0.86 |
Annualised volatility (excluding costs) | 9.54% | 9.54% | 10.01% | 10.02% | 9.96% |
Annualised return (excluding costs) | −0.19% | −4.32% | 7.43 % | 4.02% | 8.56% |
Maximum drawdown (excluding costs) | −9.85% | −11.39% | −6.30% | −6.21% | −7.57% |
Positions taken (annualised) | 28 | 185 | 107 | 93 | 170 |
Leverage | — | — | 1.09 | 1.05 | 1.05 |
Transaction costs | 0.20% | 1.30% | 0.99% | 0.79% | 1.33% |
Annualised return (including costs) | −0.39% | −5.62% | 6.44% | 3.23% | 7.23% |
As can be seen from the last row in Table 10, the HONN model continues to demonstrate a superior trading performance. Similarly, the MLP and RNN continue to perform well and present the second- and the third-highest annualised return, respectively. In general, we observe that all models in which leverage was applied were able to exploit it and increase their trading performance despite the higher transaction costs.
Trading performance – Final results (3 July 2006 – 31 December 2007)^{35}
MACD | ARMA | MLP | RNN | HONN | |
---|---|---|---|---|---|
Sharpe ratio (excluding costs) | 0.70 | −0.40 | 1.88 | 0.60 | 0.99 |
Annualised volatility (excluding costs) | 7.27% | 6.38% | 7.16% | 7.23% | 7.26% |
Annualised return (excluding costs) | 5.06% | −2.53% | 13.45% | 4.35% | 7.17% |
Maximum drawdown (excluding costs) | −5.39% | −10.12% | −5.70% | −5.30% | −4.98% |
Positions taken (annualised) | 20 | 189 | 109 | 167 | 80 |
Leverage | 1.14 | — | 1.13 | 1.14 | 1.14 |
Transaction costs | 1.02% | 1.51% | 1.67% | 2.19% | 1.50% |
Annualised return (including costs) | 4.04% | −4.04% | 11.78% | 2.16% | 5.67% |
We note that in both periods, the MLP and HONN models continue to outperform the other models, as they are able to exploit the leverage and present an increased trading performance in both out-of-sample periods. So even if the ranking of these two models is different in the two out-of-sample forecasting periods retained, they clearly outperform all other models in all cases, a fact that allows us to argue with confidence their forecasting superiority and their stability and robustness over time. In contrast, the RNN model was not able to exploit the extra memory inputs in its architecture and presents rather disappointing results. Moreover, the time spent to derive the RNN results is 10 times longer than the time required with the HONN and MLP models. Similarly, the MACD and ARMA models present a very weak forecasting power, even though their training subperiod performance was promising (see Table A3).
CONCLUDING REMARKS
In this study, we apply MLP, RNNs and HONNs to a 1-day-ahead forecasting and trading task of the EUR/USD exchange rate using the ECB fixing series with only autoregressive terms as inputs. We use a naïve, an MACD and an ARMA model as benchmarks. Our aim is not only to examine the forecasting and trading performance of our models, but also to see whether this performance is stable and robust over time. In order to do so, we develop these different prediction models over the period January 1999 – December 2007, and validate their out-of-sample trading efficiency over the following period from January 2008 to August 2008. To examine the robustness and stability of our models, we compare our results with those from a previous study using the same models and the same selection of autoregressive terms as inputs to the NNs, but with an out-of-sample period between July 2006 and December 2007.
As it turns out, the MLP and HONN models clearly outperform the other models in both out-of-sample periods in terms of annualised return. Our conclusions are the same even after we introduced transaction costs and a leverage to exploit the low volatility of the trading performance of those models. This enables us to argue with confidence their forecasting superiority and their stability and robustness over time. In contrast, the RNN model seems to have difficulty in providing good forecasts when only autoregressive series are used as inputs. Similarly, the ARMA and MACD models present low or even negative annualised returns in this application, despite their satisfactory training subperiod performance.
REFERENCES AND NOTES
- Lisboa, P. and Vellido, A. (2000) Business Applications of Neural Networks: The State-of-the-Art of Real-World Applications. Singapore: World Scientific.CrossRefGoogle Scholar
- Dunis, C., Laws, J. and Sermpinis, G. (2008) Modelling and Trading the EUR/USD Exchange Rate at the ECB Fixing. CIBEF Working Papers, http://www.cibef.com.
- Dunis, C. and Chen, Y. (2005) Alternative volatility models for risk management and trading: Application to the EUR/USD and USD/JPY rates. Derivatives Use, Trading & Regulation 11 (2): 126–156.CrossRefGoogle Scholar
- Dunis, C. and Williams, M. (2002) Modelling and trading the EUR/USD exchange rate: Do neural network models perform better? Derivatives Use, Trading & Regulation 8 (3): 211–239.Google Scholar
- Connor, J. and Atlas, L. (1991) Recurrent neural networks and time series prediction. Proceedings of the International Joint Conference on Neural Networks, Vol. 1, Seattle, pp. 301–306.Google Scholar
- Adam, O., Zarader, L. and Milgram, M. (1994) Identification and prediction of non-linear models with recurrent neural networks. Laboratoire de Robotique de Paris, New Trends in Neural Computation. Berlin, Heidelberg: Springer, pp. 531–535.Google Scholar
- Kamijo, K. and Tanigawa, T. (1990) Stock price pattern recognition: A recurrent neural network approach. In: Proceedings of the International Joint Conference on Neural Networks, Vol. 1, San Diago, pp. 215–221.Google Scholar
- Tenti, P. (1996) Forecasting foreign exchange rates using recurrent neural networks. Applied Artificial Intelligence 10: 567–581.CrossRefGoogle Scholar
- Tino, P., Schittenkopf, C. and Doffner, G. (2001) Financial volatility trading using recurrent networks. IEEE Transactions in Neural Networks 12 (4): 865–874.CrossRefGoogle Scholar
- Dunis, C. and Huang, X. (2002) Forecasting and trading currency volatility: An application of recurrent neural regression and model combination. Journal of Forecasting 21 (5): 317–354.CrossRefGoogle Scholar
- Giles, L. and Maxwell, T. (1987) Learning, invariance and generalization in higher order neural networks. Applied Optics 26: 4972–4978.CrossRefGoogle Scholar
- Redding, N., Kowalczyk, A. and Downs, T. (1993) Constructive higher-order network algorithm that is polynomial time. Neural Networks 6: 997–1010.CrossRefGoogle Scholar
- Kosmatopoulos, E., Polycarpou, M., Christodoulou, M. and Ioannou, P. (1995) High-order neural network structures for identification of dynamical systems. IEEE Transactions on Neural Networks 6: 422–431.CrossRefGoogle Scholar
- Psaltis, D., Park, C. and Hong, J. (1988) Higher order associative memories and their optical implementations. Neural Networks 1: 149–163.CrossRefGoogle Scholar
- Dunis, C., Laws, J. and Evans, B. (2006) Modelling and trading the gasoline crack spread: A non-linear story. Derivatives Use, Trading & Regulation 12: 126–145.CrossRefGoogle Scholar
- Fulcher, J., Zhang, M. and Xu, S. (2006) The application of higher-order neural networks to financial time series. In: J. Kamruzzaman, R. Begg and R. Sarker (eds.). Artificial Neural Networks in Finance and Manufacturing. Hershey, PA: Idea Group, London.Google Scholar
- Dunis, C., Laws, J. and Evans, B. (2006) Trading futures spreads: An application of correlation and threshold filters. Applied Financial Economics 16: 1–12.CrossRefGoogle Scholar
- Dunis, C., Laws, J. and Sermpinis, G. (2008) Higher Order and Recurrent Neural Architectures for Trading the EUR/USD Exchange Rate. CIBEF Working Papers, http://www.cibef.com.
- The EUR/USD is quoted as the number of USD per euro: for example, a value of 1.2657 is USD1.2657 per 1 euro. We examine the EUR/USD since its first trading day on 4 January 1999, and until 29 August 2008.Google Scholar
- Confirmation of its stationary property is obtained at the 1 per cent significance level by both the Augmented Dickey Fuller (ADF) and Phillips-Perron (PP) test statistics.Google Scholar
- A ‘long’ EUR/USD position means buying Euros at the current price, while a ‘short’ position means selling Euros at the current price.Google Scholar
- Box, G., Jenkins, G. and Gregory, G. (1994) Time Series Analysis: Forecasting and Control. New Jersey: Prentice-Hall.Google Scholar
- Pindyck, R. and Rubinfeld, D. (1998) Econometric Models and Economic Forecasts. New York: McGraw-Hill.Google Scholar
- Wald, A. and Wolfowitz, J. (1940) On a test whether two samples are from the same population. Annals of Mathematical Statistics 11: 147–162.CrossRefGoogle Scholar
- Shapiro, A.F. (2000) A hitchhiker's guide to the techniques of adaptive nonlinear models. Insurance, Mathematics and Economics 26: 119–132.CrossRefGoogle Scholar
- Kaastra, I. and Boyd, M. (1996) Designing a neural network for forecasting financial and economic time series. Neurocomputing 10: 215–236.CrossRefGoogle Scholar
- The bias nodes are not shown here for the sake of simplicity.Google Scholar
- Elman, J.L. (1990) Finding structure in time. Cognitive Science 14: 179–211.CrossRefGoogle Scholar
- Knowles, A., Hussein, A., Deredy, W., Lisboa, P. and Dunis, C.L. (2005) Higher-Order Neural Networks with Bayesian Confidence Measure for Prediction of EUR/USD Exchange Rate. CIBEF Working Papers, http://www.cibef.com.
- Zhang, M., Xu, S.X. and Fulcher, J. (2002) Neuron-adaptive higher order neural-network models for automated financial data modelling. IEEE Transactions on Neural Networks 13 (1): 188–204.CrossRefGoogle Scholar
- Karayiannis, N. and Venetsanopoulos, A. (1994) On the training and performance of high-order neural networks. Mathematical Biosciences 129: 143–168.CrossRefGoogle Scholar
- The complete paper, Dunis et al ^{2}, is available at http://www.cibef.com.
- Since most of the models have a volatility of about 10 per cent, we have chosen this level as our basis. The leverage factors retained are given in Table 11.Google Scholar
- The interest costs are calculated by considering a 4 per cent interest rate p.a. divided by 252 trading days. In reality, leverage costs also apply during non-trading days so that we should calculate the interest costs using 360 days per year. But for the sake of simplicity, we use the approximation of 252 trading days to spread the leverage costs of non-trading days equally over the trading days. This approximation prevents us from keeping track of how many non-trading days we hold a position.Google Scholar
- Not taken into account the interest that could be earned during times where the capital is not traded (non-trading days) and could therefore be invested.Google Scholar