Computational Economics

, Volume 47, Issue 4, pp 551–567 | Cite as

Using a Genetic Algorithm to Improve Recurrent Reinforcement Learning for Equity Trading

  • Jin ZhangEmail author
  • Dietmar Maringer


Recurrent reinforcement learning (RRL) has been found to be a successful machine learning technique for building financial trading systems. In this paper, we use a genetic algorithm (GA) to improve the trading results of a RRL-type equity trading system. The proposed trading system takes the advantage of GA’s capability to select an optimal combination of technical indicators, fundamental indicators and volatility indicators for improving out-of-sample trading performance. In our experiment, we use the daily data of 180 S&P stocks (from the period January 2009 to April 2014) to examine the profitability and the stability of the proposed GA-RRL trading system. We find that, after feeding the indicators selected by the GA into the RRL trading system, the out-of-sample trading performance improves as the number of companies with a significantly positive Sharpe ratio increases.


Artificial intelligence Algorithmic trading Recurrent reinforcement learning Genetic algorithm Indicator selection Sharpe ratio 



The authors acknowledge the support provided by the Swiss National Science Foundation (SNSF) grant program. The authors would like to thank three reviewers from the Artificial Intelligence Applications and Innovations Conference (AIAI) 2013, two anonymous referees, and editors of Computational Economics for their valuable suggestions.


  1. Achelis, S. B. (2001). Technical analysis from A to Z. New York: McGraw Hill.Google Scholar
  2. Allen, F., & Karjalainen, R. (1999). Using genetic algorithms to find technical trading rules. Journal of Financial Economics, 51, 245–271.CrossRefGoogle Scholar
  3. Baillie, R. T., & DeGennaro, R. P. (1990). Stock returns and volatility. The Journal of Financial and Quantitative Analysis, 25, 203–214.CrossRefGoogle Scholar
  4. Bertoluzzo, F., & Corazza, M. (2007). Making financial trading by recurrent reinforcement learning. In B. Apolloni, R. Howlett, & L. Jain (Eds.), Lecture notes in computer science (Vol. 4693, pp. 619–626). Berlin: Springer.Google Scholar
  5. Bollerslev, T. (1986). Generalized autoregressive conditional heteroskedasticity. Journal of Econometrics, 31, 307–327.CrossRefGoogle Scholar
  6. Chen, G., Firth, M., & Rui, O. M. (2001). The dynamic relation between stock returns, trading volume, and volatility. Financial Review, 36, 153–174.CrossRefGoogle Scholar
  7. Creamer, G. (2012). Model calibration and automated trading agent for euro futures. Quantitative Finance, 12, 531–545.CrossRefGoogle Scholar
  8. Dempster, M. A. H., & Leemans, V. (2006). An automated FX trading system using adaptive reinforcement learning. Expert Systems with Applications, 30, 543–552.CrossRefGoogle Scholar
  9. Dempster, M. A. H., Payne, T. W., Romahi, Y., & Thompson, G. W. P. (2001). Computational learning techniques for intraday FX trading using popular technical indicators. IEEE Transactions on Neural Networks, 12(4), 744–754.CrossRefGoogle Scholar
  10. Efron, B. (1981). Nonparametric estimates of standard error: The jackknife, the bootstrap and other methods. Biometrika, 68, 589–599.CrossRefGoogle Scholar
  11. Fama, E. F. (1970). Efficient capital markets: A review of theory and empirical work. Journal of Finance, 25, 383–417.CrossRefGoogle Scholar
  12. Fosback, N. G. (1991). Stock market logic: A sophisticated approach to profits on wall street. Fort Lauderdale: Dearborn Financial Publishing.Google Scholar
  13. Gilli, M., Këllezi, E., & Hysi, H. (2006). A data-driven optimization heuristic for downside risk minimization. Journal of Risk, 8, 1–18.Google Scholar
  14. Gold, C. (2003). FX trading via recurrent reinforcement learning. Proceedings of IEEE international conference on computational intelligence in financial engineering (pp. 363–370). Los Alamitos: IEEE Computer Society Press.Google Scholar
  15. Gorse, D. (2011). Application of stochastic recurrent reinforcement learning to index trading. In European symposium on artificial neural networks.Google Scholar
  16. Graham, B., & Dodd, D. (2004). Security analysis. Burlington: McGraw Hill.Google Scholar
  17. Graham, B., & Zweig, J. (2003). The intelligent investor: The definitive book on value investing. New York: HarperCollins.Google Scholar
  18. Holland, J. (1975). Adaptation in natural and artificial systems: An introductory analysis with applications to biology, control, and artificial intelligence. Ann Arbor: University of Michigan Press.Google Scholar
  19. Kirkpatrick, C. D., & Dahlquist, J. (2006). Technical analysis: The complete resource for financial market technicians. Upper Saddle River: Financial Times Press.Google Scholar
  20. Lev, B., & Thiagarajan, S. R. (1993). Fundamental information analysis. Journal of Accounting Research, 31, 190–215.CrossRefGoogle Scholar
  21. Maringer, D., & Ramtohul, T. (2012). Regime-switching recurrent reinforcement learning for investment decision making. Computational Management Science, 9, 89–107.CrossRefGoogle Scholar
  22. Maringer, D., & Zhang, J. (2014). Transition variable selection for regime switching recurrent reinforcement learning. In 2014 IEEE conference on computational intelligence for financial engineering and economics, (pp. 407–412).Google Scholar
  23. Modigliani, F., & Modigliani, L. (1997). Risk-adjusted performance. The Journal of Portfolio Management, 23, 45–54.CrossRefGoogle Scholar
  24. Moody, J., Wu, L., Liao, Y., & Saffell, M. (1998). Performance functions and reinforcement learning for trading systems and portfolios. Journal of Forecasting, 17, 441–470.CrossRefGoogle Scholar
  25. Sharpe, W. (1994). The Sharpe ratio. Journal of Portfolio Management, 21, 49–58.CrossRefGoogle Scholar
  26. Wilder, J. W. (1978). New concepts in technical trading systems. Greensboro: Trend Research.Google Scholar
  27. Zhang, J., & Maringer, D. (2013). Indicator selection for daily equity trading with recurrent reinforcement learning. In the genetic and evolutionary computation conference (GECCO), 2013, 1757–1758.Google Scholar

Copyright information

© Springer Science+Business Media New York 2015

Authors and Affiliations

  1. 1.Faculty of Economics and Business AdministrationUniversity of BaselBaselSwitzerland

Personalised recommendations