Multimedia Tools and Applications

, Volume 77, Issue 24, pp 32213–32242 | Cite as

Deep neural network architecture for sentiment analysis and emotion identification of Twitter messages

  • Dario StojanovskiEmail author
  • Gjorgji Strezoski
  • Gjorgji Madjarov
  • Ivica Dimitrovski
  • Ivan Chorbev


In the work presented in this paper, we showcase a deep learning system for sentiment analysis and emotion identification in Twitter messages. The system consists of a convolutional neural network used for extracting features from textual data and a classifier for which we experiment with several different classifying algorithms. We train the network using pre-trained word embeddings obtained by unsupervised learning on large text corpora and compare the effectiveness of the different word vectors for this task. We evaluate our system on 3-class sentiment analysis with datasets provided by the Sentiment analysis in Twitter task from the SemEval competition. Additionally, we explore the effectiveness of our approach for emotion identification, by using an automatically annotated dataset with 7 distinct emotions. Our architecture achieves comparable performances to state-of-the-art techniques in the field of sentiment analysis and improves results in the field of emotion identification on the test we use in our evaluation. Moreover, the paper presents several use case scenarios, depicting real-world usage of our architecture.


Twitter Convolutional neural networks Word embeddings Sentiment analysis Emotion identification 



We would like to acknowledge the support of the European Commission through the project MAESTRA Learning from Massive, Incompletely annotated, and Structured Data (Grant number ICT-2013-612944). Also, this work was partially financed by the Faculty of Computer Science and Engineering at the Ss. Cyril and Methodius University.


  1. 1.
    Agarwal A, Xie B, Vovsha I, Rambow O, Passonneau R (2011) Sentiment analysis of twitter data. In: Proceedings of the workshop on languages in social media, association for computational linguistics, Stroudsburg, PA, USA, LSM ’11, pp 30–38.
  2. 2.
    Balabantaray R, Mohammad M, Sharma N (2012) Multi-class twitter emotion classification: a new approach. Int J Appl Inform Syst 4(1):48–53Google Scholar
  3. 3.
    Bastien F, Lamblin P, Pascanu R, Bergstra J, Goodfellow IJ, Bergeron A, Bouchard N, Bengio Y (2012) Theano: new features and speed improvements. Deep Learning and Unsupervised Feature Learning NIPS 2012 WorkshopGoogle Scholar
  4. 4.
    Bergstra J, Breuleux O, Bastien F, Lamblin P, Pascanu R, Desjardins G, Turian J, Warde-Farley D, Bengio Y (2010) Theano: a CPU and GPU math expression compiler. In: Proceedings of the python for scientific computing conference (SciPy), oral presentationGoogle Scholar
  5. 5.
    Chintala S (2012) Sentiment analysis using neural architectures. New York UniversityGoogle Scholar
  6. 6.
    Collobert R, Weston J, Bottou L, Karlen M, Kavukcuoglu K, Kuksa P (2011) Natural language processing (almost) from scratch. J Mach Learn Res 12:2493–2537. zbMATHGoogle Scholar
  7. 7.
    Corney D, Martin C, Göker A (2014) Spot the ball: Detecting sports events on Twitter. In: Advances in information retrieval. Springer, pp 449–454Google Scholar
  8. 8.
    Demirsoz O, Ozcan R (2017) Classification of news-related tweets. J Inf Sci 43(4):509–524CrossRefGoogle Scholar
  9. 9.
    Dong L, Wei F, Yin Y, Zhou M, Xu K (2015) Splusplus: a feature-rich two-stage classifier for sentiment analysis of tweets. In: Proceedings of the 9th international workshop on semantic evaluation (SemEval 2015), pp 515–519Google Scholar
  10. 10.
    dos Santos C, Gatti M (2014) Deep convolutional neural networks for sentiment analysis of short texts. In: Proceedings of COLING 2014, the 25th international conference on computational linguistics: technical papers. Dublin City University and Association for Computational Linguistics, pp 69–78.
  11. 11.
    Freitas J, Ji H (2016) Identifying news from tweets. In: Proceedings of the first workshop on NLP and computational social science, pp 11–16Google Scholar
  12. 12.
    Ghazi D, Inkpen D, Szpakowicz S (2010) Hierarchical approach to emotion recognition and classification in texts. In: Advances in artificial intelligence. Springer, pp 40–50Google Scholar
  13. 13.
    Go A, Bhayani R, Huang L (2009) Twitter sentiment classification using distant supervision. CS224N Project Report. Stanford, pp 1–12Google Scholar
  14. 14.
    Guo W, Li H, Ji H, Diab MT (2013) Linking tweets to news: a framework to enrich short text data in social media. In: ACL (1). Citeseer, pp 239–249Google Scholar
  15. 15.
    Hagen M, Potthast M, Büchner M, Stein B (2015) Webis: an ensemble for twitter sentiment detection. In: Proceedings of the 9th international workshop on semantic evaluation (SemEval 2015), pp 582–589Google Scholar
  16. 16.
    Hinton GE, Srivastava N, Krizhevsky A, Sutskever I, Salakhutdinov RR (2012) Improving neural networks by preventing co-adaptation of feature detectors. arXiv:12070580
  17. 17.
    Hoang-Vu TA, Bessa A, Barbosa L, Freire J (2014) Bridging vocabularies to link tweets and news. In: Seventeenth International workshop on the web and databases (WebDB 2014)Google Scholar
  18. 18.
    Hochreiter S, Schmidhuber J (1997) Long short-term memory. Neural Comput 9(8):1735–1780CrossRefGoogle Scholar
  19. 19.
    Huang FJ, LeCun Y (2006) Large-scale learning with svm and convolutional for generic object categorization. In: 2006 IEEE Computer society conference on computer vision and pattern recognition, vol 1. IEEE, pp 284–291Google Scholar
  20. 20.
    Ifrim G, Shi B, Brigadir I (2014) Event detection in twitter using aggressive filtering and hierarchical tweet clustering. In: SNOW-DC@ WWW, pp 33–40Google Scholar
  21. 21.
    Kalchbrenner N, Grefenstette E, Blunsom P (2014) A convolutional neural network for modelling sentences. arXiv:14042188
  22. 22.
    Kim Y (2014) Convolutional neural networks for sentence classification. arXiv:14085882
  23. 23.
    Kokkinos F, Potamianos A (2017) Structural attention neural networks for improved sentiment analysis. arXiv:170101811
  24. 24.
    Kolchyna O, Souza TT, Treleaven P, Aste T (2015) Twitter sentiment analysis: Lexicon method, machine learning method and their combination. arXiv:150700955
  25. 25.
    Kouloumpis E, Wilson T, Moore J (2011) Twitter sentiment analysis: the good the bad and the omg!. Icwsm 11:538–541Google Scholar
  26. 26.
    Lakkaraju H, Socher R, Manning C (2014) Aspect specific sentiment analysis using hierarchical deep learning. In: NIPS Workshop on deep learning and representation learningGoogle Scholar
  27. 27.
    Le P, Zuidema W (2015) Compositional distributional semantics with long short term memory. arXiv:150302510
  28. 28.
    Lin X, Gu Y, Zhang R, Fan J (2016) Linking news and tweets. In: Australasian database conference. Springer, pp 467–470Google Scholar
  29. 29.
    Lu B, Ott M, Cardie C, Tsou BK (2011) Multi-aspect sentiment analysis with topic models. In: 2011 IEEE 11th International conference on data mining workshops (ICDMW). IEEE, pp 81–88Google Scholar
  30. 30.
    Mikolov T, Sutskever I, Chen K, Corrado GS, Dean J (2013) Distributed representations of words and phrases and their compositionality. In: Burges C, Bottou L, Welling M, Ghahramani Z, Weinberger K (eds) Advances in neural information processing systems., vol 26. Curran Associates, Inc., pp 3111–3119
  31. 31.
    Mohammad S, Kiritchenko S, Zhu X (2013) Nrc-canada: building the state-of-the-art in sentiment analysis of tweets. In: Proceedings of the seventh international workshop on semantic evaluation exercises (SemEval-2013). AtlantaGoogle Scholar
  32. 32.
    Nichols J, Mahmud J, Drews C (2012) Summarizing sporting events using twitter. In: Proceedings of the 2012 ACM international conference on intelligent user interfaces. ACM, pp 189–198Google Scholar
  33. 33.
    Pak A, Paroubek P (2010) Twitter as a corpus for sentiment analysis and opinion mining. In: LREC, vol 10, pp 1320–1326Google Scholar
  34. 34.
    Pang B, Lee L (2008) Opinion mining and sentiment analysis. Found Trends Inform Retriev 2(1–2):1–135CrossRefGoogle Scholar
  35. 35.
    Pennington J, Socher R, Manning C (2014) Glove: global vectors for word representation. In: Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP). Association for Computational Linguistics, Doha, pp 1532–1543
  36. 36.
    Purver M, Battersby S (2012) Experimenting with distant supervision for emotion classification. In: Proceedings of the 13th conference of the European chapter of the association for computational linguistics. Association for Computational Linguistics, pp 482–491Google Scholar
  37. 37.
    Qian Q, Huang M, Lei J, Zhu X (2016) Linguistically regularized lstms for sentiment classification. arXiv:161103949
  38. 38.
    Radford A, Jozefowicz R, Sutskever I (2017) Learning to generate reviews and discovering sentiment. arXiv:170401444
  39. 39.
    Roberts K, Roach MA, Johnson J, Guthrie J, Harabagiu SM (2012) Empatweet: annotating and detecting emotions on Twitter. In: LREC, pp 3806–3813Google Scholar
  40. 40.
    Sankaranarayanan J, Samet H, Teitler BE, Lieberman MD, Sperling J (2009) Twitterstand: news in tweets. In: Proceedings of the 17th acm sigspatial international conference on advances in geographic information systems. ACM, pp 42–51Google Scholar
  41. 41.
    Severyn A, Moschitti A (2015) Twitter sentiment analysis with deep convolutional neural networks. In: Proceedings of the 38th International ACM SIGIR conference on research and development in information retrieval. ACM, pp 959–962Google Scholar
  42. 42.
    Shi B, Ifrim G, Hurley N (2014) Insight4news: connecting news to relevant social conversations. In: Machine Learning and knowledge discovery in databases. Springer, pp 473–476Google Scholar
  43. 43.
    Sintsova V, Musat CC, Pu Faltings P (2013) Fine-grained emotion recognition in olympic tweets based on human computation. In: 4th Workshop on computational approaches to subjectivity, sentiment and social media analysis, EPFL-CONF-197185Google Scholar
  44. 44.
    Socher R, Perelygin A, Wu JY, Chuang J, Manning CD, Ng AY, Potts C (2013) Recursive deep models for semantic compositionality over a sentiment treebank. In: Proceedings of the conference on empirical methods in natural language processing (EMNLP). Citeseer, pp 1631–1642Google Scholar
  45. 45.
    Srivastava N, Hinton G, Krizhevsky A, Sutskever I, Salakhutdinov R (2014) Dropout: a simple way to prevent neural networks from overfitting. J Mach Learn Res 15(1):1929–1958MathSciNetzbMATHGoogle Scholar
  46. 46.
    Štajner T, Thomee B, Popescu AM, Pennacchiotti M, Jaimes A (2013) Automatic selection of social media responses to news. In: Proceedings of the 19th ACM SIGKDD international conference on knowledge discovery and data mining. ACM, pp 50–58Google Scholar
  47. 47.
    Stojanovski D, Strezoski G, Madjarov G, Dimitrovski I (2015) Emotion identification in fifa world cup tweets using convolutional neural network. In: 11th International conference on innovations in information technology (IIT), pp 52–57Google Scholar
  48. 48.
    Stojanovski D, Strezoski G, Madjarov G, Dimitrovski I (2015) Twitter sentiment analysis using deep convolutional neural network. In: Hybrid artificial intelligent systems. Springer, pp 726–737Google Scholar
  49. 49.
    Stojanovski D, Chorbev I, Dimitrovski I, Madjarov G (2016) Social networks vgi: Twitter sentiment analysis of social hotspots. In: European Handbook of crowdsourced geographic information, pp 223–235CrossRefGoogle Scholar
  50. 50.
    Stojanovski D, Strezoski G, Madjarov G, Dimitrovski I (2016) Finki at semeval-2016 task 4: deep learning architecture for twitter sentiment analysis. In: Proceedings of the 10th International workshop on semantic evaluation (SemEval-2016), pp 149–154Google Scholar
  51. 51.
    Strezoski G, Stojanovski D, Dimitrovski I, Madjarov G (2015) Deep learning and support vector machine for effective plant identification. ICT Innovations 2015. Web Proceedings ISSN null, pp 41–50Google Scholar
  52. 52.
    Taboada M, Brooke J, Tofiloski M, Voll K, Stede M (2011) Lexicon-based methods for sentiment analysis. Comput Linguist 37(2):267–307CrossRefGoogle Scholar
  53. 53.
    Tang Y (2013) Deep learning using linear support vector machines. arXiv:13060239
  54. 54.
    Tang D, Wei F, Qin B, Liu T, Zhou M (2014) Coooolll: a deep learning system for twitter sentiment classification. In: Proceedings of the 8th international workshop on semantic evaluation (SemEval 2014), pp 208–212Google Scholar
  55. 55.
    Tang D, Wei F, Yang N, Zhou M, Liu T, Qin B (2014) Learning sentiment-specific word embedding for twitter sentiment classification. In: Proceedings of the 52nd Annual meeting of the association for computational linguistics (volume 1: long papers). Association for Computational Linguistics, pp 1555–1565.
  56. 56.
    Tang D, Qin B, Feng X, Liu T (2015) Effective lstms for target-dependent sentiment classification. arXiv:151201100
  57. 57.
    Wang W, Chen L, Thirunarayan K, Sheth AP (2012) Harnessing twitter “big data” for automatic emotion identification. In: 2012 International Conference on and 2012 international confernece on social computing (SocialCom) privacy, security, risk and trust (PASSAT). IEEE, pp 587–592Google Scholar
  58. 58.
    Wang J, Yu LC, Lai KR, Zhang X (2016) Dimensional sentiment analysis using a regional cnn-lstm model. In: Proceedings of the 54th annual meeting of the association for computational linguistics (volume 2: short papers), vol 2, pp 225–230Google Scholar
  59. 59.
    Wang Y, Huang M, Zhao L et al (2016) Attention-based lstm for aspect-level sentiment classification. In: Proceedings of the 2016 conference on empirical methods in natural language processing, pp 606–615Google Scholar
  60. 60.
    Wilson T, Wiebe J, Hoffmann P (2005) Recognizing contextual polarity in phrase-level sentiment analysis. In: Proceedings of the conference on human language technology and empirical methods in natural language processing. Association for Computational Linguistics, pp 347–354Google Scholar
  61. 61.
    Yessenalina A, Yue Y, Cardie C (2010) Multi-level structured models for document-level sentiment classification. In: Proceedings of the 2010 conference on empirical methods in natural language processing. Association for Computational Linguistics, pp 1046–1056Google Scholar
  62. 62.
    Yu Y, Wang X (2015) World cup 2014 in the Twitter world: a big data analysis of sentiments in us sports fans’ tweets. Comput Hum Behav 48:392–400CrossRefGoogle Scholar
  63. 63.
    Zeiler MD (2012) Adadelta: An adaptive learning rate method. arXiv:12125701
  64. 64.
    Zhou S, Chen Q, Wang X (2010) Active deep networks for semi-supervised sentiment classification. In: Proceedings of the 23rd international conference on computational linguistics: posters. Association for Computational Linguistics, pp 1515–1523Google Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Faculty of Computer Science and EngineeringSs. Cyril and Methodius UniversitySkopjeRepublic of Macedonia

Personalised recommendations