Advertisement

An Empirical, Quantitative Analysis of the Differences Between Sarcasm and Irony

  • Jennifer Ling
  • Roman Klinger
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9989)

Abstract

A variety of classification approaches for the detection of ironic or sarcastic messages has been proposed in the last decade to improve sentiment classification. However, despite the availability of psychologically and linguistically motivated theories regarding the difference between irony and sarcasm, these typically do not carry over to a use in predictive models; one reason might be that these concepts are often considered very similar. In this paper, we contribute an empirical analysis of Tweets and how authors label them as irony or sarcasm. We use this distantly labeled corpus to estimate a model to distinguish between39 both classes of figurative language with the aim to, ultimately, improve the semantically correct interpretation of opinionated statements. Our model separates irony from sarcasm with 79 % accuracy on a balanced set. This result suggests that the task is harder than separating irony or sarcasm from regular texts with 89 % and 90 % accuracy, respectively. A feature analysis shows that ironic Tweets have on average a lower number of sentences than sarcastic Tweets. Sarcastic Tweets contain more positive words than ironic Tweets. Sarcastic Tweets are more often messages to a specific recipient than ironic Tweets. The analysis of bag-of-words features suggests that the comparably high classification performance to distinguish irony from sarcasm is supported by specific, reoccurring topics.

Keywords

Sentiment Analysis Regular Language Negative Word Concept Drift Literal Meaning 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

References

  1. 1.
    Amazon Review: “worst book i have ever read” (2010). http://www.amazon.com/review/R86RAMEBZSB11. Accessed 29 Feb 2016
  2. 2.
    Maynard, D., Greenwood, M.: Who cares about sarcastic tweets? investigating the impact of sarcasm on sentiment analysis. In: Calzolari, N., Choukri, K., Declerck, T., Loftsson, H., Maegaard, B., Mariani, J., Moreno, A., Odijk, J., Piperidis, S. (eds.) Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC’14), Reykjavik, Iceland, European Language Resources Association (ELRA). pp. 4238–4243. ACL Anthology Identifier: L14–1527, May 2014Google Scholar
  3. 3.
    Kreuz, R.J., Glucksberg, S.: How to be sarcastic: the echoic reminder theory of verbal irony. J. Exp. Psychol. Gen. 118(4), 374 (1989)CrossRefGoogle Scholar
  4. 4.
    Go, A., Bhayani, R., Huang, L.: Twitter sentiment classification using distant supervision. Technical report, Stanford University (2009). http://www.stanford.edu/alecmgo/papers/TwitterDistantSupervision09.pdf
  5. 5.
    Abrams, M.H.: A Glossary of Literary Terms, 7th edn. Heinle & Heinle, Thomson Learning, Boston (1999)Google Scholar
  6. 6.
    Nakassis, C., Snedeker, J.: Beyond sarcasm: intonation and context as relational cues in children’s recognition of irony. In: Proceedings of the Twenty-sixth Boston University Conference on Language Development. Cascadilla Press (2002)Google Scholar
  7. 7.
    Creusere, M.A.: A developmental test of theoretical perspective on the understanding of verbal irony: children’s recognition of allusion and pragmatic insincerity. In: Raymond W Gibbs, J., Colston, H.L. (eds.) Irony in Language and Thought: A Cognitive Science Reader, 1st edn, pp. 409–424. Lawrence Erlbaum Associates, New York (2007)Google Scholar
  8. 8.
    Glenwright, M.H., Pexman, P.M.: Children’s perceptions of the social functions of verbal irony. In: Raymond W Gibbs, J., Colston, H.L. (eds.) Irony in Language and Thought: A Cognitive Science Reader, 1st edn, pp. 447–464. Lawrence Erlbaum Associates, New York (2007)Google Scholar
  9. 9.
    Kreuz, R.J.: The use of verbal irony: cues and constraints. In: Mio, J.S., Katz, A.N. (eds.) Metaphor: Implications and Applications. Lawrence Erlbaum Associates, Mahwah (1996)Google Scholar
  10. 10.
    Gibbs, R.W.J.: Irony in talk among friends. In: Raymond W Gibbs, J., Colston, H.L. (eds.) Irony in Language and Thought: A Cognitive Science Reader, 1st edn. Lawrence Erlbaum Associates, New York (2007)Google Scholar
  11. 11.
    Utsumi, A.: A unified theory of irony and its computational formalization. In: Proceedings of the 16th Conference on Computational Linguistics - vol. 2. COLING 1996, pp. 962–967. Association for Computational Linguistics, Stroudsburg (1996)Google Scholar
  12. 12.
    Wilson, D., Sperber, D.: On verbal irony. Lingua 87, 53–76 (1992)CrossRefGoogle Scholar
  13. 13.
    Clark, H.H., Gerrig, R.J.: On the pretense theory of irony. J. Exp. Psychol. Gen. 113(1), 121–126 (1984)CrossRefGoogle Scholar
  14. 14.
    Kumon-Nakamura, S., Glucksberg, S., Brown, M.: How about another piece of pie: the allusional pretense theory of discourse irony. J. Exp. Psychol. Gen. 124(1), 3–21 (1995)CrossRefGoogle Scholar
  15. 15.
    Wilson, D., Sperber, D.: Explaining Irony. Cambridge University Press, Cambridge (2012)CrossRefGoogle Scholar
  16. 16.
    Utsumi, A.: Verbal irony as implicit display of ironic environment: distinguishing ironic utterances from nonirony. J. Pragmatics 32(12), 1777–1806 (2000)CrossRefGoogle Scholar
  17. 17.
    Clift, R.: Irony in conversation. Lang. Soc. 28(4), 523–553 (1999)CrossRefGoogle Scholar
  18. 18.
    Ptáček, T., Habernal, I., Hong, J.: Sarcasm detection on czech and english twitter. In: Proceedings of COLING 2014, the 25th International Conference on Computational Linguistics: Technical Papers, Dublin, Ireland, Dublin City University and Association for Computational Linguistics, pp. 213–223, August 2014Google Scholar
  19. 19.
    Filatova, E.: Irony and sarcasm: Corpus generation and analysis using crowdsourcing. In: Chair, N.C.C., Choukri, K., Declerck, T., Doan, M.U., Maegaard, B., Mariani, J., Moreno, A., Odijk, J., Piperidis, S. (eds.) Proceedings of the Eight International Conference on Language Resources and Evaluation (LREC 2012), Istanbul, Turkey, European Language Resources Association (ELRA), May 2012Google Scholar
  20. 20.
    Reyes, A., Rosso, P., Veale, T.: A multidimensional approach for detecting irony in Twitter. Lang. Resour. Eval. 47(1), 239–268 (2013)CrossRefGoogle Scholar
  21. 21.
    Rajadesingan, A., Zafarani, R., Liu, H.: Sarcasm detection on twitter: a behavioral modeling approach. In: Proceedings of the Eighth ACM International Conference on Web Search and Data Mining, WSDM 2015, 97–106. ACM, New York (2015)Google Scholar
  22. 22.
    Tepperman, J., Traum, D., Narayanan, S.S.: “Yeah right”: sarcasm recognition for spoken dialogue systems. In: Proceedings of InterSpeech, Pittsburgh, PA, pp. 1838–1841, September 2006Google Scholar
  23. 23.
    Tsur, O., Davidov, D., Rappoport, A.: ICWSM - a great catchy name: semi-supervised recognition of sarcastic sentences in online product reviews. In: International AAAI Conference on Web and Social Media (ICWSM), Washington D.C., USA (2010)Google Scholar
  24. 24.
    Barbieri, F., Saggion, H.: Modelling irony in twitter. In: Proceedings of the Student Research Workshop at the 14th Conference of the European Chapter of the Association for Computational Linguistics, Gothenburg, Sweden, pp. 56–64. Association for Computational Linguistics, April 2014Google Scholar
  25. 25.
    Riloff, E., Qadir, A., Surve, P., De Silva, L., Gilbert, N., Huang, R.: Sarcasm as contrast between a positive sentiment and negative situation. In: Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pp. 704–714. Association for Computational Linguistics, Seattle, October 2013Google Scholar
  26. 26.
    Ghosh, A., Li, G., Veale, T., Rosso, P., Shutova, E., Barnden, J., Reyes, A.: SemEval-2015 task 11: sentiment analysis of figurative language in twitter. In: Proceedings of the 9th International Workshop on Semantic Evaluation (SemEval 2015), pp. 470–478. Association for Computational Linguistics, Denver, June 2015Google Scholar
  27. 27.
    Van Hee, C., Lefever, E., Hoste, V.: LT3: sentiment analysis of figurative tweets: piece of cake #NotReally. In: Proceedings of the 9th International Workshop on Semantic Evaluation (SemEval 2015), pp. 684–688. Association for Computational Linguistics, Denver, June 2015Google Scholar
  28. 28.
    Wang, P.Y.A.: #Irony or #Sarcasm - a quantitative and qualitative study based on twitter. In: 27th Pacific Asia Conference on Language, Information, and Computation (PACLIC), Taiwan, Taipei, pp. 349–356 (2013)Google Scholar
  29. 29.
    Sulis, E., Farías, D.I.H., Rosso, P., Patti, V., Ruffo, G.: Figurative messages and affect in Twitter: differences between #irony, #sarcasm and #not. Knowledge-Based Systems, May 2016. (in Press)Google Scholar
  30. 30.
    Jaccard, P.: Étude comparative de la distribution florale dans une portion des Alpes et des Jura. Bull. Soc. Vaudoise des Sci. Nat. 37, 547–579 (1901)Google Scholar
  31. 31.
    Potts, C.: Twitter-aware tokenizer (2011). http://sentiment.christopherpotts.net/code-data/happyfuntokenizing.py. Accessed 03 Mar 2016
  32. 32.
    Derczynski, L., Ritter, A., Clark, S., Bontcheva, K.: Twitter part-of-speech tagging for all: Overcoming sparse and noisy data. In: Proceedings of the International Conference Recent Advances in Natural Language Processing RANLP 2013, Hissar, Bulgaria, pp. 198–206. Incoma Ltd., Shoumen, September 2013Google Scholar
  33. 33.
    Buschmeier, K., Cimiano, P., Klinger, R.: An impact analysis of features in a classification approach to irony detection in product reviews. In: Proceedings of the 5th Workshop on Computational Approaches to Subjectivity, Sentiment and Social Media Analysis, pp. 42–49. Association for Computational Linguistics, Baltimore, June 2014Google Scholar
  34. 34.
    Davidov, D., Tsur, O., Rappoport, A.: Semi-Supervised Recognition of Sarcasm in Twitter and Amazon. In: Proceedings of the Fourteenth Conference on Computational Natural Language Learning, pp. 107–116. Association for Computational Linguistics, Uppsala, July 2010Google Scholar
  35. 35.
    Whitlock, T.: Emoji unicode tables (2015). http://apps.timwhitlock.info/emoji/tables/unicode. Accessed 26 Feb 2016
  36. 36.
    González-Ibáñez, R., Muresan, S., Wacholder, N.: Identifying sarcasm in twitter: A closer look. In: Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, pp. 581–586. Association for Computational Linguistics, Portland, June 2011Google Scholar
  37. 37.
    Kreuz, R., Caucci, G.: Lexical influences on the perception of sarcasm. In: Proceedings of the Workshop on Computational Approaches to Figurative Language, pp. 1–4. Association for Computational Linguistics, Rochester, April 2007Google Scholar
  38. 38.
    Holen, V.: Dictionary of interjections (2016). http://www.vidarholen.net/contents/interjections/. Accessed 03 Mar 2016
  39. 39.
    Nielsen, F.: Afinn, March 2011. http://www2.imm.dtu.dk/pubdb/p.php?6010. Accessed 18 Mar 2016
  40. 40.
    Barbieri, F., Saggion, H.: Modelling irony in twitter: feature analysis and evaluation. In: Calzolari, N., Choukri, K., Declerck, T., Loftsson, H., Maegaard, B., Mariani, J., Moreno, A., Odijk, J., Piperidis, S. (eds.) Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC 2014), pp. 4258–4264. European Language Resources Association (ELRA), Reykjavik, May 2014Google Scholar
  41. 41.
    Fan, R.E., Chang, K.W., Hsieh, C.J., Wang, X.R., Lin, C.J.: Liblinear: a library for large linear classification. J. Mach. Learn. Res. 9, 1871–1874 (2008)zbMATHGoogle Scholar
  42. 42.
    Cristianini, N., Shawe-Taylor, J.: An Introduction to Support Vector Machines: And Other Kernel-based Learning Methods. Cambridge University Press, Cambridge (2000)CrossRefzbMATHGoogle Scholar
  43. 43.
    Utgoff, P.E.: Incremental induction of decision trees. Mach. Learn. 4(2), 161–186 (1989)CrossRefGoogle Scholar
  44. 44.
    Witten, I.H., Frank, E., Hall, M.H.: Data Mining: Practical Machine Learning Tools and Techniques. Morgan Kaufmann, San Francisco (2010)Google Scholar
  45. 45.
    Nigam, K., Lafferty, J., McCallum, A.: Using maximum entropy for text classification. In: IJCAI Workshop on Machine Learning for Information Filtering, Stockholm, Sweden (1999)Google Scholar
  46. 46.
    @Katie_Blair_: Mom signed me up... Twitter (2015). https://twitter.com/Katie_Blair_status/624580420668104704
  47. 47.
    @Bellastar12597: You can smile... Twitter (2015). https://twitter.com/Bellastar12597/status/630851771599069184
  48. 48.
    @MrHoffman9: Another wonderful day... Twitter (2015). https://twitter.com/MrHoffman9/status/635539576426262529
  49. 49.
    @CidHialeah: Another wonderful day... Twitter (2015). https://twitter.com/CidHialeah/status/623120702221103104
  50. 50.
    @Deadstitch: Another wonderful day... Twitter (2015). https://twitter.com/Deadstitch/status/622931551064453120

Copyright information

© Springer International Publishing AG 2016

Authors and Affiliations

  1. 1.Institut für Maschinelle SprachverarbeitungUniversität StuttgartStuttgartGermany

Personalised recommendations