Multi-emotion Detection in User-Generated Reviews

  • Lars Buitinck
  • Jesse van Amerongen
  • Ed Tan
  • Maarten de Rijke
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9022)


Expressions of emotion abound in user-generated content, whether it be in blogs, reviews, or on social media. Much work has been devoted to detecting and classifying these emotions, but little of it has acknowledged the fact that emotionally charged text may express multiple emotions at the same time. We describe a new dataset of user-generated movie reviews annotated for emotional expressions, and experimentally validate two algorithms that can detect multiple emotions in each sentence of these reviews.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Alm, C.O., Roth, D., Sproat, R.: Emotions from text: machine learning for text-based emotion prediction. In: Proc. HLT–EMNLP, pp. 579–586 (2005)Google Scholar
  2. 2.
    Aman, S., Szpakowicz, S.: Identifying expressions of emotion in text. In: Matoušek, V., Mautner, P. (eds.) TSD 2007. LNCS (LNAI), vol. 4629, pp. 196–205. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  3. 3.
    Buitinck, L., Louppe, G., Blondel, M., Pedregosa, F., Müller, A., Grisel, O., Niculae, V., Prettenhofer, P., Gramfort, A., Grobler, J., Layton, R., Vanderplas, J., Joly, A., Holt, B., Varoquaux, G.: API design for machine learning software: experiences from the scikit-learn project. In: ECML PKDD Workshop on Languages for Machine Learning (2013)Google Scholar
  4. 4.
    Calvo, R.A., D’Mello, S.K.: Affect detection: an interdisciplinary review of models, methods, and their applications. IEEE Trans. on Affective Computing 1(1), 18–37 (2010)CrossRefGoogle Scholar
  5. 5.
    Danisman, T., Alpkocak, A.: Feeler: emotion classification of text using vector space model. In: Proc. AISB Convention (2008)Google Scholar
  6. 6.
    D’Mello, S.K., Craig, S.D., Sullins, J., Graesser, A.C.: Predicting affective states expressed through an emote-aloud procedure from AutoTutor’s mixed-initiative dialogue. Int’l J. AI in Education 16, 3–28 (2006)Google Scholar
  7. 7.
    Fan, R.E., Chang, K.W., Hsieh, C.J., Wang, X.R., Lin, C.J.: Liblinear: A library for large linear classification. JMLR 9, 1871–1874 (2008)zbMATHGoogle Scholar
  8. 8.
    Godbole, S., Sarawagi, S.: Discriminative methods for multi-labeled classification. In: Dai, H., Srikant, R., Zhang, C. (eds.) PAKDD 2004. LNCS (LNAI), vol. 3056, pp. 22–30. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  9. 9.
    Pang, B., Lee, L.: A sentimental education: Sentiment analysis using subjectivity. In: Proc. ACL (2004)Google Scholar
  10. 10.
    Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, V., Vanderplas, J., Passos, A., Cournapeau, D., Brucher, M., Perrot, M., Duchesnay, E.: Scikit-learn: Machine learning in Python. JMLR 12 (2011)Google Scholar
  11. 11.
    Sechidis, K., Tsoumakas, G., Vlahavas, I.: On the stratification of multi-label data. In: Gunopulos, D., Hofmann, T., Malerba, D., Vazirgiannis, M. (eds.) ECML PKDD 2011, Part III. LNCS, vol. 6913, pp. 145–158. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  12. 12.
    Shaver, P., Schwartz, J., Kirson, D., O’Connor, C.: Emotion knowledge: further exploration of a prototype approach. J. Personality and Social Psychology 52(6) (1987)Google Scholar
  13. 13.
    Stenetorp, P., Pyysalo, S., Topić, G., Ohta, T., Ananiadou, S., Tsujii, J.: BRAT: a web-based tool for NLP-assisted text annotation. In: Demos at 13th Conf. EACL, pp. 102–107 (2012)Google Scholar
  14. 14.
    Strapparava, C., Mihalcea, R.: SemEval-2007 task 14: Affective text. In: Proc. 4th Int’l Workshop on Semantic Evaluations, pp. 70–74 (2007)Google Scholar
  15. 15.
    Strapparava, C., Mihalcea, R.: Learning to identify emotions in text. In: Proc. ACM Symp. Applied Computing, pp. 1556–1560 (2008)Google Scholar
  16. 16.
    Tan, E.: Emotion and the structure of narrative film. Erlbaum, Mahwah (1996)Google Scholar
  17. 17.
    Trohidis, K., Tsoumakas, G., Kalliris, G., Vlahavas, I.: Multi-label classification of music into emotions. In: Proc. Int’l Conf. on Music IR, pp. 325–330 (2008)Google Scholar
  18. 18.
    Tsoumakas, G., Katakis, I.: Multi-label classification: An overview. Int’l J. Data Warehousing and Mining 3(3), 1–13 (2007)CrossRefGoogle Scholar
  19. 19.
    Tsoumakas, G., Vlahavas, I.: Random k-labelsets: An ensemble method for multilabel classification. In: Kok, J.N., Koronacki, J., Lopez de Mantaras, R., Matwin, S., Mladenič, D., Skowron, A. (eds.) ECML 2007. LNCS (LNAI), vol. 4701, pp. 406–417. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  20. 20.
    Yang, C., Lin, K.H.Y., Chen, H.H.: Emotion classification using web blog corpora. In: IEEE/WIC/ACM Int’l Conf. on Web Intelligence, pp. 275–278 (2007)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  • Lars Buitinck
    • 1
    • 2
  • Jesse van Amerongen
    • 2
  • Ed Tan
    • 2
  • Maarten de Rijke
    • 2
  1. 1.Netherlands eScience CenterAmsterdamThe Netherlands
  2. 2.University of AmsterdamAmsterdamThe Netherlands

Personalised recommendations