Skip to main content
SpringerLink
Log in

An optimal deep learning-based LSTM for stock price prediction using twitter sentiment analysis

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Stock Price Prediction is one of the hot research topics in financial engineering, influenced by economic, social, and political factors. In the present stock market, the positive and negative opinions are the important indicators for the forthcoming stock prices. At the same time, the growth of the internet and social network enables the clients to express their opinions and shares their views on future stock processes. Therefore, sentiment analysis of the social media data of stock prices helps to predict future stock prices effectively. With this motivation, this research presents a new novel Teaching and Learning Based Optimization (TLBO) model with Long Short-Term Memory (LSTM) based sentiment analysis for stock price prediction using Twitter data. The tweets are generally short, having unusual grammatical structures, and hence the data pre-processing is essential to remove the unwanted data and transform it into a meaningful format. Besides, the LSTM model is applied to classify tweets into positive and negative sentiments related to stock prices. They help investigate how the tweets correlate with the nature of the stock market prices. To improve the predictive outcome of the LSTM model, the Adam optimizer is used to determine the learning rate. Furthermore, the TLBO model is applied to tune the output unit of the LSTM model optimally. Experiments are carried out on the Twitter data to ensure the better stock price predictive performance of the TLBO-LSTM model. The experimental findings of the TLBO-LSTM model show promising results over the state of art methods in terms of diverse aspects. The TLBO-LSTM model produced a superior outcome, with a maximum precision of 95.33%, a recall of 85.28%, and an F-score of 90%. By achieving a greater accuracy of 94.73%, the TLBO-LSTM model surpassed the other techniques.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. Pagolu VS, Reddy KN, Panda G, Majhi B (2016) Sentiment analysis of Twitter data for predicting stock market movements. In 2016 international conference on signal processing, communication, power and embedded system (SCOPES), pp 1345-1350. IEEE

  2. Paulraj D (2020) A gradient boosted decision tree-based sentiment classification of twitter data. International Journal of Wavelets, Multiresolution and Information Processing, World Scientific, ISSN 1793-690X (online), 18:(4):205027 1–21

  3. Pak A, Paroubek P (2010) Twitter as a corpus for sentiment analysis and opinion mining. In: Proceedings of the Seventh International Conference on Language Resources and Evaluation, pp 13201326

  4. Neelakandan S (2020) An automated learning model of conventional neural network based sentiment analysis on Twitter data. Journal of Computational and Theoretical Nano Science 17(5):2230–2236

    Article  Google Scholar 

  5. Teti E, Dallocchio M, Aniasi A (2019) The relationship between twitter and stock prices. Evidence from the US technology industry. Technological Forecasting and Social Change 149:119747

    Article  Google Scholar 

  6. Ambeth Kumar VD, Malathi S, Kumar A, Prakash M, Veluvolu KC (2020) Active volume control in smart phones based on user activity and ambient noise. Sensors 20(15):4117. https://doi.org/10.3390/s20154117

    Article  Google Scholar 

  7. Paulraj D (2020) An automated exploring and learning model for data prediction using balanced CA-Svm. Journal of Ambient Intelligence and Humanized Computing. Springer, pp 1-12, ISSN 1868-5137 (online), Published Online: April 2020

  8. Derakhshan A, Beigy H (2019) Sentiment analysis on stock social media for stock price movement prediction. Eng Appl Artif Intell 85:569–578

    Article  Google Scholar 

  9. Zou F, Chen D, Wang J (2016) An improved teaching-learning-based optimization with the social character of PSO for global optimization. Computational Intelligence and Neuroscience:2016

  10. Annamalai R, Rayen SJ, Arunajsmine J (2020) Social media networks owing to disruptions for effective learning. Procedia Computer Science 172:145–151. https://doi.org/10.1016/j.procs.2020.05.022

    Article  Google Scholar 

  11. Gokul Anand J (2011) "trust based optimal routing in MANET's," 2011 international conference on emerging trends in electrical and computer technology. Nagercoil, India, pp 1150–1156. https://doi.org/10.1109/ICETECT.2011.5760293

    Book  Google Scholar 

  12. Divyabharathi S (2016) Large scale optimization to minimize network traffic using MapReduce in big data applications. International Conference on Computation of Power, Energy Information and Communication (ICCPEIC), pp. 193–199, April 2016. 10.1109/ICCPEIC.2016.7557196

  13. Anand R, Singh H (2021) Interpretable filter based convolutional neural network (IF-CNN) for glucose prediction and classification using PD-SS algorithm. Measurement 183. https://doi.org/10.1016/j.measurement.2021.109804

  14. Bhukya RR, Hardas BM, Ch T et al (2022) An automated word embedding with parameter tuned model for web crawling. Intelligent Automation & Soft Computing 32(3):1617–1632

    Article  Google Scholar 

  15. Vargas DL, Evsukoff Vargas MR, De Lima BS, Evsukoff AG (2017) Deep learning for stock market prediction from financial news articles. 2017 IEEE international conference on computational intelligence and virtual environments for measurement systems and applications (CIVEMSA); Piscataway: IEEE; 2017

  16. Tripathi S et al (2021) IoT-based traffic prediction and traffic signal control system for smart city. Soft Comput. https://doi.org/10.1007/s00500-021-05896-x

  17. Zhang X, Tan Y (2018) Deep stock ranker: a LSTM neural network model for stock selection. International conference on data mining and big data. Springer, pp 614–623

  18. Li X, Cao J, Pan Z (2019) Market impact analysis via deep learned architectures. Neural Comput & Applic 31:5989–6000

    Article  Google Scholar 

  19. Chen K, Zhou Y, Dai F (2015) A LSTM-based method for stock returns prediction: a case study of China stock market. 2015 IEEE international conference on big data. IEEE2823–2824

  20. Ding X, Zhang Y, Liu T, Duan J (2015) Deep learning for event-driven stock prediction. Proceedings of the 24th international conference on artificial intelligenceIJCAI’15AAAI Press2327–2333

  21. Akita R, Yoshihara A, Matsubara T, Uehara K (2016) Deep learning for stock prediction using numerical and textual information. 2016IEEE/ACIS 15th international conference on computer and information science. IEEE1–6

  22. Nelson DM, Pereira AC, De Oliveira RA (2017) Stock market’s price movement prediction with LSTM neural networks. Proceedings of the international joint conference on neural networks2017-May. Proceedings of the international joint conference on neural networks IEEE1419–1426

  23. Li W., Liao J (2018). A comparative study on trend forecasting approach for stock price time series. Proceedings of the international conference on anti-counterfeiting, security and identification, asid2017-Octob. Proceedings of the international conference on anti-counterfeiting, security and identification, asid 74–78

  24. Hu Z, Liu W, Bian J, Liu X, Liu T-Y (2018) Listening to chaotic whispers: a deep learning framework for news-oriented stock trend prediction. Proceedings of the eleventh ACM international conference on web search and data miningWSDM ‘18New York, NY, USA: ACM261–269

  25. Shi L, Member S, Teng Z, Wang L, Zhang Y, Binder A (2019) DeepClue : visual interpretation of text-based deep stock prediction. IEEE Trans Knowl Data Eng 31:1094–1108

    Article  Google Scholar 

  26. Kumar BS, Ravi V (2016) A survey of the applications of text mining in financial domain. Knowl-Based Syst 114:128–147

    Article  Google Scholar 

  27. Xing FZ, Cambria E, Welsch RE (2018) Natural language based financial forecasting: a survey. Artif Intell Rev 50(1):49–73

    Article  Google Scholar 

  28. Yoshihara A, Seki K, Uehara K (2015) Leveraging temporal properties of news events for stock market prediction. Artif Intell Res 5(1):103

    Article  Google Scholar 

  29. Rajaraman PV (2020) A Survey on Text Question Responsive Systems in English and Indian Languages. Soft Computing and Signal Processing. Advances in Intelligent Systems and Computing 1118. https://doi.org/10.1007/978-981-15-2475-2_25

  30. Jia Y, Wu Z, Xu Y, Ke D, Su K (2017) Long short-term memory projection recurrent neural network architectures for piano’s continuous note recognition. Journal of Robotics 2017

  31. Yaqub M, Jinchao F, Zia MS, Arshid K, Jia K, Rehman ZU, Mehmood A (2020) State-of-the-art CNN optimizer for brain tumor segmentation in magnetic resonance images. Brain Sciences 10(7):427

    Article  Google Scholar 

  32. Ding X, Zhang Y, Liu T, Duan J (2016) Knowledge-driven event embedding for stock prediction. Proceedings of Coling 2016, the 26th International Conference on Computational Linguistics: Technical Papers.2016

  33. Rene Beulah J, Sumathy R, Varalakshmi G, Neelakandan S (2022) IoT enabled environmental toxicology for air pollution monitoring using AI techniques. Environ Res 205:112574. https://doi.org/10.1016/j.envres.2021.112574

    Article  Google Scholar 

  34. Mathai PP, Karthikeyan C et al (2021) Deep learning based capsule neural network model for breast cancer diagnosis using mammogram images. Interdiscip Sci Comput Life Sci. https://doi.org/10.1007/s12539-021-00467-y

  35. Nofer M, Hinz O (2015) Using twitter to predict the stock market. Bus Inf Syst Eng 57(4):229–242

    Article  Google Scholar 

  36. Cyril CPD, Beulah JR, Mohan P, Harshavardhan A, Sivabalaselvamani D An automated learning model for sentiment analysis and data classification of Twitter data using balanced CA-SVM. https://doi.org/10.1177/1063293X211031485

  37. Deng S, Zhang N, Zhang W, Chen J, Pan JZ, Chen H (2019) Knowledge-driven stock trend prediction and explanation via temporal convolutional network. Companion Proceedings of the 2019. World Wide Web Conference

  38. Jin Y, Liu Jin Z, Yang Y, Liu Y (2019) Stock closing price prediction based on sentiment analysis and LSTM. Neural Computing and Applications 32:9713–9729

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of CSE, Jawaharlal Nehru Technological University Anantapur (JNTUA), Ananthapuramu, 515002, India

    T. Swathi

  2. Department of CSE, G Pulla Reddy Engineering College Kurnool, Kurnool, 518001, India

    N. Kasiviswanath

  3. Vice Chancellor Rayalaseema University, Kurnool, 518001, India

    A. Ananda Rao

Authors
  1. T. Swathi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. N. Kasiviswanath
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. A. Ananda Rao
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to T. Swathi.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendix

Appendix

figure 11

The above figures show the confusion matrix produced by the LSTM-TLBO algorithm on the Twitter dataset. The figure clarifies that the LSTM-TLBO model has improved results by classifying a set of 1900 instances under the Negative class and 3586 instances under the Positive class.

figure 12

A ROC analysis of the LSTM model against the Twitter dataset is shown in Fig. 7, that the LSTM model has effectively classified the stock prices using Twitter data with a higher ROC of 0.98. Besides, ROC analysis of the TLBO-LSTM model demonstrated in Fig. 8 portrays the enhanced classification outcome with a maximum ROC of 0.99.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Swathi, T., Kasiviswanath, N. & Rao, A.A. An optimal deep learning-based LSTM for stock price prediction using twitter sentiment analysis. Appl Intell 52, 13675–13688 (2022). https://doi.org/10.1007/s10489-022-03175-2

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-022-03175-2

Keywords

Search

Navigation