Threshold Recurrent Reinforcement Learning Model for Automated Trading
This paper presents the threshold recurrent reinforcement learning (TRRL) model and describes its application in a simple automated trading system. The TRRL is a regime-switching extension of the recurrent reinforcement learning (RRL) algorithm. The basic RRL model was proposed by Moody and Wu (1997) and used for uncovering trading strategies. We argue that the RRL is not sufficiently equipped to capture the non-linearities and structural breaks present in financial data, and propose the TRRL model as a more suitable algorithm for such environments. This paper gives a detailed description of the TRRL and compares its performance with that of the basic RRL model in a simple automated trading framework using daily data from four well-known European indices. We assume a frictionless setting and use volatility as an indicator variable for switching between regimes. We find that the TRRL produces better trading strategies in all the cases studied, and demonstrate that it is more apt at finding structure in non-linear financial time series than the standard RRL.
Keywordsregime-switching automated trading reinforcement learning differential Sharpe ratio
Unable to display preview. Download preview PDF.
- 1.Moody, J., Wu, L.: Optimization of trading systems and portfolios. In: Proceedings of the IEEE/IAFE Computational Intelligence for Financial Engineering (CIFEr), pp. 300–307 (1997)Google Scholar
- 4.Gold, C.: FX trading via recurrent reinforcement learning. In: IEEE International Conference on Computational Intelligence for Financial Engineering, pp. 363–370 (2003)Google Scholar
- 5.Dempster, M., Leemans, V.: An automated FX trading system using adaptive reinforcement learning. Expert Systems with Applications 30(3), 543–552 (2006); Intelligent Information Systems for Financial EngineeringGoogle Scholar
- 7.Hamilton, J.: Regime-switching models. In: The New Palgrave Dictionary of Economics. Palgrave Macmillan, Basingstoke (2008)Google Scholar
- 8.Tong, H.: On a threshold model. In: Chen, C. (ed.) Pattern Recognition and Signal Processing, pp. 101–141. Sijthoff & Noordhoff (1978)Google Scholar
- 10.White, H.: Some asymptotic results for learning in single hidden-layer feedforward network models. Journal of the American Statistical Association, 1003–1013 (1989)Google Scholar
- 11.Teräsvirta, T.: Specification, estimation, and evaluation of smooth transition autoregressive models. Journal of the American Statistical Association, 208–218 (1994)Google Scholar
- 12.Maringer, D., Ramtohul, T.: Regime-switching recurrent reinforcement learning. Mimeo University of Basel (2009)Google Scholar
- 14.Sharpe, W.: Mutual fund performance. Journal of Business, 119–138 (1966)Google Scholar