Textual Emotion Classification: An Interoperability Study on Cross-Genre Data Sets

  • Bahadorreza OfoghiEmail author
  • Karin Verspoor
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10400)


This paper describes the application and analysis of a previously developed textual emotion classification system (READ-BioMed-EC) on a different data set in the same language with different textual properties. The classifier makes use of a number of lexicon-based and text-based features. The data set originally used to develop this classifier consisted of English-language Twitter microblogs with mentions of Ebola disease. The data was manually labelled with one of six emotion classes, plus sarcasm, news-related, or neutral. In this new work, we applied the READ-BioMed-EC emotion classifier without retraining to an independently collected set of Web blog posts, also annotated with emotion classes, to understand how well the Twitter-trained disease-focused emotion classifier might generalise to an entirely different collection of open-domain sentences. The results of our study show that direct cross-genre application of the classifier does not achieve meaningful results, but when re-trained on the open-domain data set, the READ-BioMed-EC system outperforms the previously published results. The study has implications for cross-genre applicability of emotion classifiers, demonstrating that emotion is expressed differently in different text types.


Emotion classification Lexicon-based Cross-genre 



We thank Saima Aman and Stan Szpakowicz for sharing their Web blog data set with us for the purpose of this study.


  1. 1.
    Agrawal, A., An, A.: Unsupervised emotion detection from text using semantic and syntactic relations. In: Proceedings of the 2012 IEEE/WIC/ACM International Joint Conferences on Web Intelligence and Intelligent Agent Technology, vol. 01, pp. 346–353 (2012)Google Scholar
  2. 2.
    Alm, C.O., Roth, D., Sproat, R.: Emotions from text: machine learning for text-based emotion prediction. In: Proceedings of the Conference on Human Language Technology and Empirical Methods in Natural Language Processing, HLT 2005, pp. 579–586. Association for Computational Linguistics, Stroudsburg, PA, USA (2005).
  3. 3.
    Aman, S., Szpakowicz, S.: Identifying expressions of emotion in text. In: Matoušek, V., Mautner, P. (eds.) TSD 2007. LNCS, vol. 4629, pp. 196–205. Springer, Heidelberg (2007). doi: 10.1007/978-3-540-74628-7_27 CrossRefGoogle Scholar
  4. 4.
    Aman, S., Szpakowicz, S.: Using roget’s thesaurus for fine-grained emotion recognition. In: Proceedings of the Third International Joint Conference on Natural Language Processing, pp. 296–302 (2008)Google Scholar
  5. 5.
    Balabantaray, R., Mohammad, M., Sharma, N.: Multi-class Twitter emotion classification: a new approach. Int. J. Appl. Inform. Syst. 4, 48–53 (2012)CrossRefGoogle Scholar
  6. 6.
    Buhrmester, M., Kwang, T., Gosling, S.D.: Amazon’s Mechanical Turk: a new source of inexpensive, yet high-quality, data? Perspect. Psychol. Sci. 6(1), 3–5 (2011)CrossRefGoogle Scholar
  7. 7.
    Danisman, T., Alpkocak, A.: Feeler: emotion classification of text using vector space model. In: Proceedings of the AISB 2008 Convention, Communication, Interaction and Social Intelligence, Scotland (2008)Google Scholar
  8. 8.
    Ekman, P.: Universals and cultural differences in facial expression of emotion. In: Nebraska Symposium on Motivation, Lincoln, Nebraska, pp. 207–283 (1972)Google Scholar
  9. 9.
    Jain, M.C., Kulkarni, V.: TexEmo: conveying emotion from text - the study. Int. J. Comput. Appl. 86, 43–50 (2014)Google Scholar
  10. 10.
    Li, S., Huang, L., Wang, R., Zhou, G.: Sentence-level emotion classification with label and context dependence. In: Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing, pp. 1045–1053 (2015)Google Scholar
  11. 11.
    Li, W., Xu, H.: Text-based emotion classification using emotion cause extraction. Expert Syst. Appl. 41, 1742–1749 (2014)CrossRefGoogle Scholar
  12. 12.
    Liu, B.: Sentiment Analysis and Opinion Mining, Synthesis Lectures on Human Language Technologies. Morgan & Claypool Publishers, San Rafael (2012)Google Scholar
  13. 13.
    Masum, S.A., Prendinger, H., Ishizuka, M.: Emotion sensitive news agent: an approach towards user centric emotion sensing from the news. In: Proceedings of the International Conference on Web Intelligence, IEEE/WIC/ACM, pp. 614–620 (2007)Google Scholar
  14. 14.
    McCallum, A.K.: MALLET: A Machine Learning for Language Toolkit (2002).
  15. 15.
    Milne, D., Paris, C., Christensen, H., Batterham, P., O’Dea, B.: We feel: taking the emotional pulse of the world. In: Proceedings of the 19th Triennial Congress of the International Ergonomics Association, Melbourne, Australia (2015)Google Scholar
  16. 16.
    Ofoghi, B., Mann, M., Verspoor, K.: Towards early discovery of salient health threats: a social media emotion classification technique. In: Proceedings of Pacific Symposium on Biocomputing (PSB), Hawaii, US, pp. 504–515 (2016)Google Scholar
  17. 17.
    Ofoghi, B., Siddiqui, S., Verspoor, K.: READ-BioMed-SS: adverse drug reaction classification of microblogs using emotional and conceptual enrichment. In: Proceedings of the Social Media Mining Shared Task Workshop at the Pacific Symposium on Biocomputing, Hawaii, US (2016)Google Scholar
  18. 18.
    Pang, B., Lee, L., Vaithyanathan, S.: Thumbs up?: Sentiment classification using machine learning techniques. In: Proceedings of the ACL 2002 Conference on Empirical Methods in Natural Language Processing, Philadelphia, PA, US, vol. 10, pp. 79–86 (2002)Google Scholar
  19. 19.
    Schulz, A., Thanh, T.D., Paulheim, H., Schweizer, I.: A fine-grained sentiment analysis approach for detecting crisis related microposts. In: Geldermann, J., Muller, T., Fortier, S., Comes, F. (eds.) Proceedings of the 10th International Conference on Information Systems for Crisis Response and Management, pp. 846–851 (2013)Google Scholar
  20. 20.
    Socher, R., Perelygin, A., Wu, J.Y., Chuang, J., Manning, C.D., Ng, A.Y., Potts, C.: Recursive deep models for semantic compositionality over a sentiment Treebank. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP 2013), Seattle, USA (2013)Google Scholar
  21. 21.
    Stone, P.J., Dunphy, D.C., Smith, M.S., Ogilvie, D.M.: The general inquirer: a computer approach to content analysis. Am. J. Sociol. 73(5), 634–635 (1968)CrossRefGoogle Scholar
  22. 22.
    Valitutti, R.: WordNet-Affect: an affective extension of WordNet. In: Proceedings of the 4th International Conference on Language Resources and Evaluation, pp. 1083–1086 (2004)Google Scholar
  23. 23.
    Wan, S., Paris, C.: Understanding public emotional reactions on Twitter. In: Proceedings of the Ninth International AAAI Conference on Web and Social Media, Oxford, UK (2015)Google Scholar
  24. 24.
    Wang, W., Chen, L., Thirunarayan, K., Sheth, A.P.: Harnessing Twitter “big data” for automatic emotion identification. In: Proceedings of the International Conference on Privacy, Security, Risk and Trust and the 2012 International Confernece on Social Computing (SocialCom), Amsterdam, pp. 587–592 (2012)Google Scholar
  25. 25.
    Wang, X., Zheng, Q.: Text emotion classification research based on improved latent semantic analysis algorithm. In: Proceedings of the 2nd International Conference on Computer Science and Electronics Engineering, pp. 210–213 (2013)Google Scholar
  26. 26.
    Wen, S., Wan, X.: Emotion classification in microblog texts using class sequential rules. In: Proceedings of the Twenty-Eighth AAAI Conference on Artificial Intelligence, Quebec City, Canada, pp. 187–193 (2014)Google Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  1. 1.School of Computing and Information SystemsThe University of MelbourneMelbourneAustralia
  2. 2.Health and Biomedical Informatics CentreThe University of MelbourneMelbourneAustralia

Personalised recommendations