Skip to main content
SpringerLink
Log in

Stock closing price prediction based on sentiment analysis and LSTM

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

Stock market prediction has been identified as a very important practical problem in the economic field. However, the timely prediction of the market is generally regarded as one of the most challenging problems due to the stock market’s characteristics of noise and volatility. To address these challenges, we propose a deep learning-based stock market prediction model that considers investors’ emotional tendency. First, we propose to involve investors’ sentiment for stock prediction, which can effectively improve the model prediction accuracy. Second, the stock pricing sequence is a complex time sequence with different scales of fluctuations, making the accurate prediction very challenging. We propose to gradually decompose the complex sequence of stock price by adopting empirical modal decomposition (EMD), which yields better prediction accuracy. Third, we adopt LSTM due to its advantages of analyzing relationships among time-series data through its memory function. We further revised it by adopting attention mechanism to focus more on the more critical information. Experiment results show that the revised LSTM model can not only improve prediction accuracy, but also reduce time delay. It is confirmed that investors’ emotional tendency is effective to improve the predicted results; the introduction of EMD can improve the predictability of inventory sequences; and the attention mechanism can help LSTM to efficiently extract specific information and current mission objectives from the information ocean.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

References

  1. Fama EF (1998) Market efficiency, long-term returns, and behavioral finance. J Financ Econ 49(3):283–306

    Article  Google Scholar 

  2. Wang B, Huang H, Wang X (2012) A novel text mining approach to financial time series forecasting. Neuro Comput 83(6):136–45

    Google Scholar 

  3. De Gooijer JG, Hyndman RJ (2005) 25 years of IIF time series forecasting: a selective review. Soc Sci Electron Publ 22(3):443–473

    Google Scholar 

  4. Neild S (2003) A review of time-frequency methods for structural vibration analysis. Eng Struct 25(6):713–728

    Article  Google Scholar 

  5. Krunz MM, Makowski AM (2002) Modeling video traffic using M/G//splinfin/ input processes: a compromise between Markovian and LRD models. IEEE J Sel Areas Commun 16(5):733–748

    Article  Google Scholar 

  6. Farina L, Rinaldi S (2000) Positive linear systems. Theory and applications. J Vet Med Sci 63(9):945–8

    MATH  Google Scholar 

  7. Contreras J, Espinola R, Nogales FJ et al (2002) ARIMA models to predict next-day electricity prices. IEEE Power Eng Rev 22(9):57–57

    Article  Google Scholar 

  8. Smola AJ, Schölkopf B (2004) A tutorial on support vector regression. Stat Comput 14(3):199–222

    Article  MathSciNet  Google Scholar 

  9. Judith EDPD, Deleo JM (2001) Artificial neural networks. Cancer 91(S8):1615–1635

    Article  Google Scholar 

  10. Prasaddas S, Padhy S (2012) Support vector machines for prediction of futures prices in Indian stock market. Int J Comput Appl 41(3):22–6

    Google Scholar 

  11. Cavalcante RC, Brasileiro RC, Souza VLF, Nobrega JP, Oliveira ALI (2016) Computational intelligence and financial markets: a survey and future directions. Expert Syst Appl 55:194–211

    Article  Google Scholar 

  12. Antweiler W, Frank MZ (2004) Is all that talk just noise? The information content of internet stock message boards. J Finance 59(3):1259–1294

    Article  Google Scholar 

  13. Baker M, Wurgler J (2006) Investor sentiment and the cross-section of returns. J Finance 61:1645e1680

    Google Scholar 

  14. Dragomiretskiy K, Zosso D (2014) Variational mode decomposition. IEEE Trans Signal Process 62(3):531–544

    Article  MathSciNet  Google Scholar 

  15. Daubechies I, Lu J, Wu HT (2011) Synchrosqueezed wavelet transforms: an empirical mode decomposition-like tool. Appl Comput Harmon Anal 30(2):243–261

    Article  MathSciNet  Google Scholar 

  16. Tang H, Chiu KC, Lei X (2003) In: Proceedings of 3rd international workshop on computational intelligence in economics and finance (CIEF’2003), North Carolina, USA, September 26–30, pp 1112–1119

  17. Duan T (2016) Auto regressive dynamic Bayesian network and its application in stock market inference. In: IFIP international conference on artificial intelligence applications and innovations. Springer, Berlin

  18. Hinton G, Deng L, Yu D et al (2012) Deep neural networks for acoustic modeling in speech recognition: the shared views of four research groups. IEEE Signal Process Mag 29(6):82–97

    Article  Google Scholar 

  19. Zhang DZD, Song HSH, Chen PCP (2008) Stock market forecasting model based on a hybrid ARMA and support vector machines. In: International conference on management science and engineering. IEEE

  20. Hochreiter S, Schmidhuber J (1997) Long short-term memory. Neural Comput 9(8):1735–1780

    Article  Google Scholar 

  21. Xingjian SH, Chen Z, Wang H, Yeung DY, Wong WK, Woo WC (2015) Convolutional LSTM network: A machine learning approach for precipitation nowcasting. In: Advances in neural information processing systems 2015, pp. 802–810

  22. Ma X, Tao Z, Wang Y et al (2015) Long short-term memory neural network for traffic speed prediction using remote microwave sensor data. Transp Res Part C Emerg Technol 54:187–197

    Article  Google Scholar 

  23. Liu H, Mi X, Li Y (2018) Smart multi-step deep learning model for wind speed forecasting based on variational mode decomposition, singular spectrum analysis, LSTM network and ELM. Energy Convers Manag 159:54–64

    Article  Google Scholar 

  24. Ding X, Zhang Y, Liu T, Duan J (eds) (2015) Deep learning for event-driven stock prediction. In: International conference on artificial intelligence

  25. Fischer T, Krauss C (2017) Deep learning with long short-term memory networks for financial market predictions. Eur J Oper Res. S0377221717310652

  26. Mao Y, Wang B, Wei W, Liu B (2012) Correlating S&P 500 stocks with twitter data. In: Proceedings of the first ACM international workshop on hot topics on interdisciplinary social networks research, New York, vol 12e16, pp 69–72

  27. Mittal A, Goel A (2009) Stock prediction using twitter sentiment analysis. https://pdfs.semanticscholar.org/4ecc/55e1c3ff1cee41f21e5b0a3b22c58d04c9d6.pdf. Accessed 9 May 2016

  28. Kaminski J, Gloor PA (2014) Nowcasting the bitcoin market with twitter signals. Accessed 9 May 2016

  29. Baker M, Wurgler J (2006) Investor sentiment and the cross-section of stock returns. J. Finance 61(4):1645–1680

    Article  Google Scholar 

  30. Baker M, Wurgler J (2007) Investor sentiment in the stock market. J Econ Perspect 21:129–152

    Article  Google Scholar 

  31. Connor B (2010) In: Fourth international AAAI conference on weblogs and social media. Aaai Publications

  32. Guo K, Sun Y, Qian X (2017) Can investor sentiment be used to predict the stock price? Dynamic analysis based on China stock market. Physica A Stat Mech Appl 469(C):390–396

    Article  Google Scholar 

  33. Zhou Z, Zhao J, Xu K, (2016) can online emotions predict the stock market in China?. In: Cellary W, Mokbel M, Wang J, Wang H, Zhou R, Zhang Y (eds) Web information systems engineering—WISE. WISE 2016. Lecture Notes in Computer Science, vol 10041. Springer, Cham

  34. Hu Z, Liu W, Bian J, Liu X, Liu T-Y (2018) Listening to chaotic whispers: a deep learning framework for news-oriented stock trend prediction. In: Proceedings of the eleventh ACM international conference on web search and data mining, pp 261–269. ACM

  35. Kim Y (2014) Convolutional Neural Networks for Sentence Classification. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), Doha, Qatar, pp 1746–1751

  36. Huang NE, Shen Z, Long SR et al (1971) The empirical mode decomposition and the Hilbert spectrum for non-linear and non-stationary time series analysis. Proc A 1998(454):903–995

    MATH  Google Scholar 

  37. Sharma R, Pachori R, Acharya U (2015) Application of entropy measures on intrinsic mode functions for the automated identification of focal electroencephalogram signals. Entropy 17(2):669–691

    Article  Google Scholar 

  38. Wang S, Zhang N, Wu L et al (2016) Wind speed forecasting based on the hybrid ensemble empirical mode decomposition and GA-BP neural network method. Renew Energy 94:629–636

    Article  Google Scholar 

  39. Ben Ali J, Fnaiech N, Saidi L et al (2015) Application of empirical mode decomposition and artificial neural network for automatic bearing fault diagnosis based on vibration signals. Appl Acoust 89:16–27

    Article  Google Scholar 

  40. Antweiler W, Frank MZ (2004) Is all that talk just noise? The news content of internet stock message boards. J Finance 59:1259–1294

    Article  Google Scholar 

  41. Ding M, Chen Y, Bressler SL (2006) Granger causality: basic theory and application to neuroscience. Quant Biol 437:826–831

    MATH  Google Scholar 

  42. Sharma N, Juneja A (2017) Combining of random forest estimates using LSboost for stock market index prediction. In: 2nd international conference for convergence in technology (I2CT), Mumbai, pp 1199–1202. https://doi.org/10.1109/I2CT.2017.8226316

Download references

Acknowledgements

This work was supported by National Nature Science Foundation of China (NSFC) under Project 71502125.

Author information

Authors and Affiliations

  1. School of Electrical and Information Engineering, Tianjin University, Tianjin, People’s Republic of China

    Zhigang Jin & Yang Yang

  2. Department of Computer Science and Engineering, Santa Clara University, Santa Clara, USA

    Yuhong Liu

Authors
  1. Zhigang Jin
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yang Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yuhong Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yuhong Liu.

Ethics declarations

Conflict of Interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Jin, Z., Yang, Y. & Liu, Y. Stock closing price prediction based on sentiment analysis and LSTM. Neural Comput & Applic 32, 9713–9729 (2020). https://doi.org/10.1007/s00521-019-04504-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-019-04504-2

Keywords

Search

Navigation