Skip to main content
SpringerLink
Log in

Reinforcement Learning in Stock Trading

  • Conference paper
  • First Online:
Advanced Computational Methods for Knowledge Engineering (ICCSAMA 2019)

Part of the book series: Advances in Intelligent Systems and Computing ((AISC,volume 1121))

Abstract

Using machine learning techniques in financial markets, particularly in stock trading, attracts a lot of attention from both academia and practitioners in recent years. Researchers have studied different supervised and unsupervised learning techniques to either predict stock price movement or make decisions in the market.

In this paper we study the usage of reinforcement learning techniques in stock trading. We evaluate the approach on real-world stock dataset. We compare the deep reinforcement learning approach with state-of-the-art supervised deep learning prediction in real-world data. Given the nature of the market where the true parameters will never be revealed, we believe that the reinforcement learning has a lot of potential in decision-making for stock trading.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 129.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 169.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    According to NASDAQ standard, recommendation from analysts can be Strong Buy, Buy, Hold, Underperform or Sell. Reference: https://www.nasdaq.com/quotes/analyst-recommendations.aspx. Accessed on 07-September-2019.

  2. 2.

    https://www.kaggle.com/borismarjanovic/price-volume-data-for-all-us-stocks-etfs.

  3. 3.

    https://www.isods.org/.

References

  1. Azhikodan, A.R., Bhat, A.G., Jadhav, M.V.: Stock trading bot using deep reinforcement learning. In: Innovations in Computer Science and Engineering, pp. 41–49. Springer, Singapore (2019)

    Google Scholar 

  2. Bisoi, R., Dash, P.K.: A hybrid evolutionary dynamic neural network for stock market trend analysis and prediction using unscented Kalman filter. Appl. Soft Comput. 19, 41–56 (2014)

    Article  Google Scholar 

  3. Bradley, D.A.: Stock Market Prediction: The Planetary Barometer and how to Use it. Llewellyn Publications, Woodbury (1948)

    Google Scholar 

  4. Chong, E., Han, C., Park, F.C.: Deep learning networks for stock market analysis and prediction: methodology, data representations, and case studies. Expert Syst. Appl. 83, 187–205 (2017)

    Article  Google Scholar 

  5. Dang, Q., Ignat, C.: Computational trust model for repeated trust games. In: Trustcom/BigDataSE/ISPA, pp. 34–41. IEEE (2016)

    Google Scholar 

  6. Deng, Y., Bao, F., Kong, Y., Ren, Z., Dai, Q.: Deep direct reinforcement learning for financial signal representation and trading. IEEE Trans. Neural Netw. Learn. Syst. 28(3), 653–664 (2017)

    Article  Google Scholar 

  7. Elton, E.J., Gruber, M.J., Brown, S.J., Goetzmann, W.N.: Modern Portfolio Theory and Investment Analysis, 9th edn. Wiley, Hoboken (2014)

    Google Scholar 

  8. Florescu, I., Mariani, M.C., Stanley, H.E., Viens, F.G.: Handbook of High-frequency Trading and Modeling in Finance, vol. 9. Wiley, Hoboken (2016)

    Book  Google Scholar 

  9. Föllmer, H., Schied, A.: Stochastic Finance: An Introduction in Discrete Time, 4th edn. Walter de Gruyter, Berlin (2016)

    Book  Google Scholar 

  10. Göçken, M., Özçalici, M., Boru, A., Dosdogru, A.T.: Stock price prediction using hybrid soft computing models incorporating parameter tuning and input variable selection. Neural Comput. Appl. 31(2), 577–592 (2019)

    Article  Google Scholar 

  11. Granger, C.W.J., Morgenstern, O.: Predictability of Stock Market Prices. Heath Lexington Books, Lexington (1970)

    Google Scholar 

  12. van Hasselt, H., Guez, A., Silver, D.: Deep reinforcement learning with double q-learning. In: AAAI, pp. 2094–2100. AAAI Press (2016)

    Google Scholar 

  13. Henrique, B.M., Sobreiro, V.A., Kimura, H.: Literature review: machine learning techniques applied to financial market prediction. Expert Syst. Appl. 124, 226–251 (2019)

    Article  Google Scholar 

  14. Hester, T., Vecerík, M., Pietquin, O., Lanctot, M., Schaul, T., Piot, B., Horgan, D., Quan, J., Sendonaris, A., Osband, I., Dulac-Arnold, G., Agapiou, J., Leibo, J.Z., Gruslys, A.: Deep q-learning from demonstrations. In: AAAI, pp. 3223–3230. AAAI Press (2018)

    Google Scholar 

  15. Hiransha, M., Gopalakrishnan, E.A., Menon, V.K., Soman, K.: NSE stock market prediction using deep-learning models. Procedia Comput. Sci. 132, 1351–1362 (2018)

    Article  Google Scholar 

  16. Ignat, C.L., Dang, Q.V., Shalin, V.L.: The influence of trust score on cooperative behavior. ACM Trans. Internet Technol. (TOIT) 19(4), 46 (2019)

    Article  Google Scholar 

  17. Jacobs, R.A., Jordan, M.I., Nowlan, S.J., Hinton, G.E., et al.: Adaptive mixtures of local experts. Neural Comput. 3(1), 79–87 (1991)

    Article  Google Scholar 

  18. Jangmin, O., Lee, J., Lee, J.W., Zhang, B.T.: Adaptive stock trading with dynamic asset allocation using reinforcement learning. Inf. Sci. 176(15), 2121–2147 (2006)

    Article  Google Scholar 

  19. Jiang, X., Pan, S., Jiang, J., Long, G.: Cross-domain deep learning approach for multiple financial market prediction. In: IJCNN, pp. 1–8. IEEE (2018)

    Google Scholar 

  20. Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: NIPS, pp. 1106–1114 (2012)

    Google Scholar 

  21. Längkvist, M., Karlsson, L., Loutfi, A.: A review of unsupervised feature learning and deep learning for time-series modeling. Pattern Recogn. Lett. 42, 11–24 (2014)

    Article  Google Scholar 

  22. Lee, J.W.: Stock price prediction using reinforcement learning. In: ISIE 2001. 2001 IEEE International Symposium on Industrial Electronics Proceedings (Cat. No. 01TH8570), vol. 1, pp. 690–695. IEEE (2001)

    Google Scholar 

  23. Lo, A.W., Mamaysky, H., Wang, J.: Foundations of technical analysis: computational algorithms, statistical inference, and empirical implementation. J. Financ. 55(4), 1705–1765 (2000)

    Article  Google Scholar 

  24. Long, W., Lu, Z., Cui, L.: Deep learning-based feature engineering for stock price movement prediction. Knowl. Based Syst. 164, 163–173 (2019)

    Article  Google Scholar 

  25. Malkiel, B.G., Fama, E.F.: Efficient capital markets: a review of theory and empirical work. J. Financ. 25(2), 383–417 (1970)

    Article  Google Scholar 

  26. Menon, V.K., Vasireddy, N.C., Jami, S.A., Pedamallu, V.T.N., Sureshkumar, V., Soman, K.: Bulk price forecasting using spark over NSE data set. In: International Conference on Data Mining and Big Data, pp. 137–146. Springer, Cham (2016)

    Chapter  Google Scholar 

  27. Mitra, S.K.: How rewarding is technical analysis in the indian stock market? Quant. Financ. 11(2), 287–297 (2011)

    Article  MathSciNet  Google Scholar 

  28. Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D., Riedmiller, M.A.: Playing Atari with deep reinforcement learning. CoRR abs/1312.5602 (2013)

    Google Scholar 

  29. Nazário, R.T.F., e Silva, J.L., Sobreiro, V.A., Kimura, H.: A literature review of technical analysis on stock markets. Q. Rev. Econ. Financ. 66, 115–126 (2017)

    Article  Google Scholar 

  30. Nison, S.: Japanese Candlestick Charting Techniques: A Contemporary Guide to the Ancient Investment Techniques of the Far East. Penguin, New York (2001)

    Google Scholar 

  31. Park, C.H., Irwin, S.H.: What do we know about the profitability of technical analysis? J. Econ. Surv. 21(4), 786–826 (2007)

    Article  Google Scholar 

  32. Patel, J., Shah, S., Thakkar, P., Kotecha, K.: Predicting stock and stock price index movement using trend deterministic data preparation and machine learning techniques. Expert Syst. Appl. 42(1), 259–268 (2015)

    Article  Google Scholar 

  33. Preis, T., Moat, H.S., Stanley, H.E.: Quantifying trading behavior in financial markets using Google trends. Sci. Rep. 3, 1684 (2013)

    Article  Google Scholar 

  34. Rilling, G., Flandrin, P., Goncalves, P., et al.: On empirical mode decomposition and its algorithms. In: IEEE-EURASIP Workshop on Nonlinear Signal and Image Processing, vol. 3, pp. 8–11. NSIP 2003, Grado (I) (2003)

    Google Scholar 

  35. Schulmeister, S.: Profitability of technical stock trading: has it moved from daily to intraday data? Rev. Financ. Econ. 18(4), 190–201 (2009)

    Article  Google Scholar 

  36. Sewak, M.: Deep Reinforcement Learning - Frontiers of Artificial Intelligence. Springer, Singapore (2019)

    Book  Google Scholar 

  37. Simon, H.A.: A behavioral model of rational choice. Q. J. Econ. 69(1), 99–118 (1955)

    Article  Google Scholar 

  38. Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction. MIT Press, Cambridge (2018)

    MATH  Google Scholar 

  39. Thomsett, M.C.: Getting Started in Fundamental Analysis. Wiley, Hoboken (2006)

    Google Scholar 

  40. Wang, J., Leu, J.: Stock market trend prediction using ARIMA-based neural networks. In: ICNN, pp. 2160–2165. IEEE (1996)

    Google Scholar 

  41. Wang, Z., Schaul, T., Hessel, M., van Hasselt, H., Lanctot, M., de Freitas, N.: Dueling network architectures for deep reinforcement learning. In: ICML. JMLR Workshop and Conference Proceedings, vol. 48, pp. 1995–2003. JMLR.org (2016)

    Google Scholar 

  42. Zhai, Y.Z., Hsu, A.L., Halgamuge, S.K.: Combining news and technical indicators in daily stock price trends prediction. In: ISNN (3). Lecture Notes in Computer Science, vol. 4493, pp. 1087–1096. Springer, Heidelberg (2007)

    Google Scholar 

  43. Zhang, G.P.: Time series forecasting using a hybrid ARIMA and neural network model. Neurocomputing 50, 159–175 (2003)

    Article  Google Scholar 

  44. Zhang, X., Li, A., Pan, R.: Stock trend prediction based on a new status box method and adaboost probabilistic support vector machine. Appl. Soft Comput. 49, 385–398 (2016)

    Article  Google Scholar 

  45. Zhou, F., Zhou, H., Yang, Z., Yang, L.: EMD2FNN: a strategy combining empirical mode decomposition and factorization machine based neural network for stock market trend prediction. Expert Syst. Appl. 115, 136–151 (2019)

    Article  MathSciNet  Google Scholar 

Download references

Acknowledgment

We would like to thank the anonymous reviewer for valuable comments.

Author information

Authors and Affiliations

  1. Industrial University of Ho Chi Minh city, Ho Chi Minh City, Vietnam

    Quang-Vinh Dang

Authors
  1. Quang-Vinh Dang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Quang-Vinh Dang .

Editor information

Editors and Affiliations

  1. Computer Science and Applications Department LGIPM, University of Lorraine, Metz Cedex 03, France

    Hoai An Le Thi

  2. Computer Science and Applications Department LGIPM, University of Lorraine, Metz Cedex 03, France

    Hoai Minh Le

  3. Laboratory of Mathematics, National Institute for Applied Sciences, Saint-Étienne-du-Rouvray Cedex, France

    Tao Pham Dinh

  4. Department of Information Systems, Wroclaw University of Science and Technology, Wrocław, Poland

    Ngoc Thanh Nguyen

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Dang, QV. (2020). Reinforcement Learning in Stock Trading. In: Le Thi, H., Le, H., Pham Dinh, T., Nguyen, N. (eds) Advanced Computational Methods for Knowledge Engineering. ICCSAMA 2019. Advances in Intelligent Systems and Computing, vol 1121. Springer, Cham. https://doi.org/10.1007/978-3-030-38364-0_28

Download citation

Publish with us

Policies and ethics

Search

Navigation