Reinforcement Learning for Automated Financial Trading: Basics and Applications

Conference paper
Part of the Smart Innovation, Systems and Technologies book series (SIST, volume 26)


The construction of automated financial trading systems (FTSs) is a subject of high interest for both the academic environment and the financial one due to the potential promises by self-learning methodologies. In this paper we consider Reinforcement Learning (RL) type algorithms, that is algorithms that real-time optimize their behavior in relation to the responses they get from the environment in which they operate, without the need for a supervisor. In particular, first we introduce the essential aspects of RL which are of interest for our purposes, second we present some original automatic FTSs based on differently configured RL-based algorithms, then we apply such FTSs to artificial and real time series of daily stock prices. Finally, we compare our FTSs with a classical one based on Technical Analysis indicators. All the results we achieve are generally quite satisfactory.


Financial trading system Reinforcement Learning stochastic control Q-learning algorithm Kernel-based Reinforcement Learning algorithm financial time series Technical Analysis 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Barto, A.G., Sutton, R.S.: Reinforcement Learning: An Introduction. Adaptive Computation and Machine Learning. The MIT Press (1998)Google Scholar
  2. 2.
    Bekiros, S.D.: Heterogeneous trading strategies with adaptive fuzzy Actor-Critic reinforcement learning: A behavioral approach. Journal of Economic Dynamics & Control 34, 1153–1170 (2010)CrossRefzbMATHMathSciNetGoogle Scholar
  3. 3.
    Bertoluzzo, F., Corazza, M.: Making financial trading by recurrent reinforcement learning. In: Apolloni, B., Howlett, R.J., Jain, L. (eds.) KES 2007/WIRN 2007, , Part II. LNCS (LNAI), vol. 4693, pp. 619–626. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  4. 4.
    Bertsekas, D.P., Tsitsiklis, J.N.: Neuro-Dynamic Programming. Athena Scientific (1996)Google Scholar
  5. 5.
    Brent, R.P.: Algorithms for Minimization without Derivatives. Prentice-Hall (1973)Google Scholar
  6. 6.
    Bosq, D.: Nonparametric Statistics for Stochastic Processes. Estimation and Prediction, vol. 110. Springer (1996)Google Scholar
  7. 7.
    Cuthbertson, K., Nitzsche, D.: Quantitative Financial Economics. Wiley (2004)Google Scholar
  8. 8.
    Farmer, D., Lo, A.W.: Market force, ecology and evolution. Industrial and Corporate Change 11, 895–953 (2002)CrossRefGoogle Scholar
  9. 9.
    Gold, C.: FX trading via recurrent Reinforcement Learning. In: Proceedings of the IEEE International Conference on Computational Intelligence in Financial Engineering, pp. 363–370 (2003)Google Scholar
  10. 10.
    Li, H., Dagli, C.H., Enke, D.: Short-term stock market timing prediction under reinforcement learning schemes. In: Proceedings of the 2007 IEEE Symposium on Approximate Dynamic Programming and Reinforcement Learning, pp. 233–240 (2007)Google Scholar
  11. 11.
    Lo, A.W., Mamaysky, H., Wang, J.: Foundations of technical analysis: Computational algorithms, statistical inference, and empirical (2000)Google Scholar
  12. 12.
    Lo, A.W.: The Adaptive Markets Hypothesis. Market efficiency from an evolutionary perspective. The Journal of Portfolio Management 30, 15–29 (2004)CrossRefGoogle Scholar
  13. 13.
    Moody, J., Wu, L., Liao, Y., Saffel, M.: Performance functions and Reinforcement Learning for trading systems and portfolios. Journal of Forecasting 17, 441–470 (1998)CrossRefGoogle Scholar
  14. 14.
    Moody, J., Saffel, M.: Learning to trade via Direct Reinforcement. IEEE Transactions on Neural Network 12, 875–889 (2001)CrossRefGoogle Scholar
  15. 15.
    Murphy, J.J.: Technical Analysis of the Financial Markets. A Comprehensive Guide to Trading Methods and Applications. New York Institute of Finance (1999)Google Scholar
  16. 16.
    Jangmin, O., Lee, J., Lee, J.W., Zhang, B.-T.: Adaptive stock trading with dynamic asset allocation using reinforcemnt learning. Information Sciences 176, 2121–2147 (2006)CrossRefzbMATHGoogle Scholar
  17. 17.
    Ormonet, D.: Kernel-Based Reinforcement Learning. Machine Learning 49, 161–178 (2002)CrossRefGoogle Scholar
  18. 18.
    Smart, W.D., Kaelbling, L.P.: Practical Reinforcement Learning in continuous spaces. In: Proceedings of the 17th International Conference on Machine Learning, pp. 903–910 (2000)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  1. 1.Department of EconomicsCa’ Foscari University of VeniceVeniceItaly
  2. 2.Advanced School of Economics of VeniceCa’ Foscari University of VeniceVeniceItaly

Personalised recommendations