Advertisement

A Comparison of Neural Network Methods for Accurate Sentiment Analysis of Stock Market Tweets

  • Narges TabariEmail author
  • Armin Seyeditabari
  • Tanya Peddi
  • Mirsad Hadzikadic
  • Wlodek Zadrozny
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11054)

Abstract

Sentiment analysis of Twitter messages is a challenging task because they contain limited contextual information. Despite the popularity and significance of this task for financial institutions, models being used still lack high accuracy. Also, most of these models are not built specifically on stock market data. Therefore, there is still a need for a highly accurate model of sentiment classification that is specifically tuned and trained for stock market data.

Facing the lack of a publicly available Twitter dataset that is labeled with positive or negative sentiments, in this paper, we first introduce a dataset of 11,000 stock market tweets. This dataset was labeled manually using Amazon Mechanical Turk. Then, we report a thorough comparison of various neural network models against different baselines. We find that when using a balanced dataset of positive and negative tweets, and a unique pre-processing technique, a shallow CNN achieves the best error rate, while a shallow LSTM, with a higher number of cells, achieves the highest accuracy of 92.7% compared to baseline of 79.9% using SVM. Building on this substantial improvement in the sentiment analysis of stock market tweets, we expect to see a similar improvement in any research that investigates the relationship between social media and various aspects of finance, such as stock market prices, perceived trust in companies, and the assessment of brand value. The dataset and the software are publicly available. In our final analysis, we used the LSTM model to assign sentiment to three years of stock market tweets. Then, we applied Granger Causality in different intervals to sentiments and stock market returns to analyze the impact of social media on stock market and visa versa.

Keywords

Sentiment analysis Neural networks Social media Stock market 

References

  1. 1.
    Antenucci, D., Cafarella, M., Levenstein, M.C., Ré, C., Shapiro, M.D.: Using social media to measure labor market flows. NBER (2014)Google Scholar
  2. 2.
    Bollen, J., Pepe, A.: modeling public mood and emotion: twitter sentiment and socio-economic phenomena, pp. 450–453 (2011)Google Scholar
  3. 3.
    Du, S., Xi, Z.: SemEval17.pdf (39), 120–125 (2016)Google Scholar
  4. 4.
    Geweke, J., Geweke, J.: Measurement of linear dependence and feedback between multiple time series measurement of linear dependence and feedback between multiple time Series 77(378), 304–313 (2018)MathSciNetCrossRefGoogle Scholar
  5. 5.
    Graves, A., Mohamed, A., Hinton, G.E.: Speech recognition with deep recurrent neural networks. CoRR abs/1303.5778 (2013). http://arxiv.org/abs/1303.5778
  6. 6.
    Hitchcock, C.: Probabilistic causation. In: Zalta, E.N. (ed.) The Stanford Encyclopedia of Philosophy. Metaphysics Research Lab, Stanford University, Winter 2016 (edn.) (2016)Google Scholar
  7. 7.
    Jiang, M., Lan, M., Wu, Y.: ECNU at SemEval-2017 Task 5: an ensemble of regression algorithms with effective features for fine-grained sentiment analysis in financial domain. In: Proceedings of the 11th International Workshop on Semantic Evaluation (SemEval-2017), pp. 885–890 (2017).  https://doi.org/10.18653/v1/S17-2152, http://www.aclweb.org/anthology/S17-2152
  8. 8.
    Johnson, R., Zhang, T.: Deep pyramid convolutional neural networks for text categorization. In: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 562–570 (2017).  https://doi.org/10.18653/v1/P17-1052, http://aclweb.org/anthology/P17-1052
  9. 9.
    Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. CoRR abs/1412.6980 (2014). http://arxiv.org/abs/1412.6980
  10. 10.
    Kolchyna, O., Souza, T.T.P., Treleaven, P., Aste, T.: Twitter sentiment analysis: lexicon method, machine learning method and their combination, p. 32 (2015). http://arxiv.org/abs/1507.00955
  11. 11.
    Kouloumpis, E., Wilson, T., Moore, J.: Twitter sentiment analysis: the good the bad and the OMG! In: Proceedings of the Fifth International AAAI Conference on Weblogs and Social Media (ICWSM 2011), pp. 538–541 (2011). http://www.aaai.org/ocs/index.php/ICWSM/ICWSM11/paper/download/2857/3251?iframe=true&width=90%25&height=90%25
  12. 12.
    Lillo, F., Miccichè, S., Tumminello, M., Piilo, J.: How news affect the trading behavior of different categories of investors in a financial market. Papers.Ssrn.Com (April), 30 (2012).  https://doi.org/10.1080/14697688.2014.931593, http://papers.ssrn.com/sol3/papers.cfm?abstract_id=2109337MathSciNetCrossRefGoogle Scholar
  13. 13.
    Loughran, T.I.M., Mcdonald, B.: When is a liability not a liability? Textual analysis, dictionaries, and 10-Ks. J. Financ. (2010, forthcoming)Google Scholar
  14. 14.
    Mikolov, T., Chen, K., Corrado, G., Dean, J.: Efficient estimation of word representations in vector space. CoRR abs/1301.3781 (2013). http://arxiv.org/abs/1301.3781
  15. 15.
    Mikolov, T., Chen, K., Corrado, G., Dean, J.: Efficient estimation of word representations in vector space, pp. 1–12 (2013).  https://doi.org/10.1162/153244303322533223, http://arxiv.org/abs/1301.3781
  16. 16.
    Pennington, J., Socher, R., Manning, C.D.: GloVe: global vectors for word representation. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, pp. 1532–1543 (2014).  https://doi.org/10.3115/v1/D14-1162
  17. 17.
    Ranco, G., Aleksovski, D., Caldarelli, G., Grčar, M., Mozetič, I.: The effects of twitter sentiment on stock price returns. PLoS One 10(9), 1–21 (2015).  https://doi.org/10.1371/journal.pone.0138441CrossRefGoogle Scholar
  18. 18.
    Ruan, Y., Durresi, A., Alfantoukh, L.: Using twitter trust network for stock market analysis. Knowl.-Based Syst. 145, 207–218 (2018).  https://doi.org/10.1016/j.knosys.2018.01.016, http://www.sciencedirect.com/science/article/pii/S0950705118300248CrossRefGoogle Scholar
  19. 19.
    dos Santos, C.N., Gatti, M.: Deep Convolutional Neural Networks for Sentiment Analysis of Short Texts. Coling-2014 pp. 69–78 (2014)Google Scholar
  20. 20.
    Severyn, A., Moschitti, A.: Twitter sentiment analysis with deep convolutional neural networks. In: Proceedings of the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval - SIGIR 2015, pp. 959–962 (2015).  https://doi.org/10.1145/2766462.2767830, http://dl.acm.org/citation.cfm?doid=2766462.2767830
  21. 21.
    Snow, R., Connor, B.O., Jurafsky, D., Ng, A.Y., Labs, D., St, C.: Cheap and fast - but is it good? Evaluating non-expert annotations for natural language tasks, pp. 254–263, October 2008Google Scholar
  22. 22.
    Sohangir, S., Wang, D., Pomeranets, A., Khoshgoftaar, T.M.: Big data: deep learning for financial sentiment analysis. J. Big Data 5(1) (2018).  https://doi.org/10.1186/s40537-017-0111-6
  23. 23.
    Sutskever, I., Vinyals, O., Le, Q.V.: Sequence to sequence learning with neural networks. CoRR abs/1409.3215 (2014). http://arxiv.org/abs/1409.3215
  24. 24.
    Tabari, N., Seyeditabari, A., Zadrozny, W., Tabari, N.: SentiHeros at SemEval-2017 task 5: an application of sentiment analysis on financial tweets, pp. 857–860 (2017)Google Scholar
  25. 25.
    Zhao, X., Wang, C., Yang, Z., Zhang, Y., Yuan, X.: Online news emotion prediction with bidirectional LSTM, pp. 238–250 (2016)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Narges Tabari
    • 1
    Email author
  • Armin Seyeditabari
    • 1
  • Tanya Peddi
    • 1
  • Mirsad Hadzikadic
    • 1
  • Wlodek Zadrozny
    • 1
  1. 1.University of North Carolina at CharlotteCharlotteUSA

Personalised recommendations