Reinforcement Learning for Automated Financial Trading: Basics and Applications

Part of the Smart Innovation, Systems and Technologies book series (SIST, volume 26)

Abstract

The construction of automated financial trading systems (FTSs) is a subject of high interest for both the academic environment and the financial one due to the potential promises by self-learning methodologies. In this paper we consider Reinforcement Learning (RL) type algorithms, that is algorithms that real-time optimize their behavior in relation to the responses they get from the environment in which they operate, without the need for a supervisor. In particular, first we introduce the essential aspects of RL which are of interest for our purposes, second we present some original automatic FTSs based on differently configured RL-based algorithms, then we apply such FTSs to artificial and real time series of daily stock prices. Finally, we compare our FTSs with a classical one based on Technical Analysis indicators. All the results we achieve are generally quite satisfactory.

Keywords

Financial trading system Reinforcement Learning stochastic control Q-learning algorithm Kernel-based Reinforcement Learning algorithm financial time series Technical Analysis 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Barto, A.G., Sutton, R.S.: Reinforcement Learning: An Introduction. Adaptive Computation and Machine Learning. The MIT Press (1998)Google Scholar
  2. 2.
    Bekiros, S.D.: Heterogeneous trading strategies with adaptive fuzzy Actor-Critic reinforcement learning: A behavioral approach. Journal of Economic Dynamics & Control 34, 1153–1170 (2010)CrossRefMATHMathSciNetGoogle Scholar
  3. 3.
    Bertoluzzo, F., Corazza, M.: Making financial trading by recurrent reinforcement learning. In: Apolloni, B., Howlett, R.J., Jain, L. (eds.) KES 2007/WIRN 2007, , Part II. LNCS (LNAI), vol. 4693, pp. 619–626. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  4. 4.
    Bertsekas, D.P., Tsitsiklis, J.N.: Neuro-Dynamic Programming. Athena Scientific (1996)Google Scholar
  5. 5.
    Brent, R.P.: Algorithms for Minimization without Derivatives. Prentice-Hall (1973)Google Scholar
  6. 6.
    Bosq, D.: Nonparametric Statistics for Stochastic Processes. Estimation and Prediction, vol. 110. Springer (1996)Google Scholar
  7. 7.
    Cuthbertson, K., Nitzsche, D.: Quantitative Financial Economics. Wiley (2004)Google Scholar
  8. 8.
    Farmer, D., Lo, A.W.: Market force, ecology and evolution. Industrial and Corporate Change 11, 895–953 (2002)CrossRefGoogle Scholar
  9. 9.
    Gold, C.: FX trading via recurrent Reinforcement Learning. In: Proceedings of the IEEE International Conference on Computational Intelligence in Financial Engineering, pp. 363–370 (2003)Google Scholar
  10. 10.
    Li, H., Dagli, C.H., Enke, D.: Short-term stock market timing prediction under reinforcement learning schemes. In: Proceedings of the 2007 IEEE Symposium on Approximate Dynamic Programming and Reinforcement Learning, pp. 233–240 (2007)Google Scholar
  11. 11.
    Lo, A.W., Mamaysky, H., Wang, J.: Foundations of technical analysis: Computational algorithms, statistical inference, and empirical (2000)Google Scholar
  12. 12.
    Lo, A.W.: The Adaptive Markets Hypothesis. Market efficiency from an evolutionary perspective. The Journal of Portfolio Management 30, 15–29 (2004)CrossRefGoogle Scholar
  13. 13.
    Moody, J., Wu, L., Liao, Y., Saffel, M.: Performance functions and Reinforcement Learning for trading systems and portfolios. Journal of Forecasting 17, 441–470 (1998)CrossRefGoogle Scholar
  14. 14.
    Moody, J., Saffel, M.: Learning to trade via Direct Reinforcement. IEEE Transactions on Neural Network 12, 875–889 (2001)CrossRefGoogle Scholar
  15. 15.
    Murphy, J.J.: Technical Analysis of the Financial Markets. A Comprehensive Guide to Trading Methods and Applications. New York Institute of Finance (1999)Google Scholar
  16. 16.
    Jangmin, O., Lee, J., Lee, J.W., Zhang, B.-T.: Adaptive stock trading with dynamic asset allocation using reinforcemnt learning. Information Sciences 176, 2121–2147 (2006)CrossRefMATHGoogle Scholar
  17. 17.
    Ormonet, D.: Kernel-Based Reinforcement Learning. Machine Learning 49, 161–178 (2002)CrossRefGoogle Scholar
  18. 18.
    Smart, W.D., Kaelbling, L.P.: Practical Reinforcement Learning in continuous spaces. In: Proceedings of the 17th International Conference on Machine Learning, pp. 903–910 (2000)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  1. 1.Department of EconomicsCa’ Foscari University of VeniceVeniceItaly
  2. 2.Advanced School of Economics of VeniceCa’ Foscari University of VeniceVeniceItaly

Personalised recommendations