Reinforcement Learning for Automated Financial Trading: Basics and Applications
The construction of automated financial trading systems (FTSs) is a subject of high interest for both the academic environment and the financial one due to the potential promises by self-learning methodologies. In this paper we consider Reinforcement Learning (RL) type algorithms, that is algorithms that real-time optimize their behavior in relation to the responses they get from the environment in which they operate, without the need for a supervisor. In particular, first we introduce the essential aspects of RL which are of interest for our purposes, second we present some original automatic FTSs based on differently configured RL-based algorithms, then we apply such FTSs to artificial and real time series of daily stock prices. Finally, we compare our FTSs with a classical one based on Technical Analysis indicators. All the results we achieve are generally quite satisfactory.
KeywordsFinancial trading system Reinforcement Learning stochastic control Q-learning algorithm Kernel-based Reinforcement Learning algorithm financial time series Technical Analysis
Unable to display preview. Download preview PDF.
- 1.Barto, A.G., Sutton, R.S.: Reinforcement Learning: An Introduction. Adaptive Computation and Machine Learning. The MIT Press (1998)Google Scholar
- 4.Bertsekas, D.P., Tsitsiklis, J.N.: Neuro-Dynamic Programming. Athena Scientific (1996)Google Scholar
- 5.Brent, R.P.: Algorithms for Minimization without Derivatives. Prentice-Hall (1973)Google Scholar
- 6.Bosq, D.: Nonparametric Statistics for Stochastic Processes. Estimation and Prediction, vol. 110. Springer (1996)Google Scholar
- 7.Cuthbertson, K., Nitzsche, D.: Quantitative Financial Economics. Wiley (2004)Google Scholar
- 9.Gold, C.: FX trading via recurrent Reinforcement Learning. In: Proceedings of the IEEE International Conference on Computational Intelligence in Financial Engineering, pp. 363–370 (2003)Google Scholar
- 10.Li, H., Dagli, C.H., Enke, D.: Short-term stock market timing prediction under reinforcement learning schemes. In: Proceedings of the 2007 IEEE Symposium on Approximate Dynamic Programming and Reinforcement Learning, pp. 233–240 (2007)Google Scholar
- 11.Lo, A.W., Mamaysky, H., Wang, J.: Foundations of technical analysis: Computational algorithms, statistical inference, and empirical (2000)Google Scholar
- 15.Murphy, J.J.: Technical Analysis of the Financial Markets. A Comprehensive Guide to Trading Methods and Applications. New York Institute of Finance (1999)Google Scholar
- 18.Smart, W.D., Kaelbling, L.P.: Practical Reinforcement Learning in continuous spaces. In: Proceedings of the 17th International Conference on Machine Learning, pp. 903–910 (2000)Google Scholar