Skip to main content
SpringerLink
Log in

On the difficulty of automatically detecting irony: beyond a simple case of negation

  • Regular Paper
  • Published:
Knowledge and Information Systems Aims and scope Submit manuscript

Abstract

It is well known that irony is one of the most subtle devices used to, in a refined way and without a negation marker, deny what is literally said. As such, its automatic detection would represent valuable knowledge regarding tasks as diverse as sentiment analysis, information extraction, or decision making. The research described in this article is focused on identifying key values of components to represent underlying characteristics of this linguistic phenomenon. In the absence of a negation marker, we focus on representing the core of irony by means of three conceptual layers. These layers involve 8 different textual features. By representing four available data sets with these features, we try to find hints about how to deal with this unexplored task from a computational point of view. Our findings are assessed by human annotators in two strata: isolated sentences and entire documents. The results show how complex and subjective the task of automatically detecting irony could be.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

Notes

  1. We do not make a fine-grained distinction among these figurative devices due to the difficulty to clearly separate their uses, in particular when the focus is on user-generated contents, and due to the general points of view (cited previously) that take them as subtypes of irony. Nevertheless, there are others approaches which are focused on specific figurative devices (cf. the research described by [12, 37] with respect to sarcasm, as well as the one described by [6] regarding satire).

  2. Cf. [25] for a detailed explanation about the concepts employed in Cognitive Grammar.

  3. Like most tasks that involve information beyond grammar; i.e., subjective, social, or cultural knowledge (e.g. machine translation), we believe that the irony detection task implies a human assessment to validate the results as well as to learn from errors. Further improvements of the model would suppose a less human involvement (such as in the research work reported by [5]).

  4. This is probably due to the difficulty and subjectivity to obtain \(ad\) \(hoc\) corpora.

  5. Available at http://www.cs.cornell.edu/People/pabo/movie-review-data.

  6. Ibid.

  7. Available at http://www.informatics.sussex.ac.uk/users/tz21.

  8. Available at http://www.csse.unimelb.edu.au/research/lt/resources/satire.

  9. This dictionary scores over 8,000 English words along its three categories. The range of scores goes from 1.0 (most passive, or most difficult to form a mental picture, or most unpleasant) to 3.0 (most active, or easiest to form a mental picture, or most pleasant). For instance, the item \(flower\) is passive (activation = 1.0), easily representable (imagery = 3.0), and generally produces a pleasant affect (pleasantness = 2.75); whereas, \(crazy\) is more active (1.33), moderately representable (2.16), and quite unpleasant (1.6).

  10. A recent interesting approach considering semantic information is reported by [26], in which authors are focused on mining fine-grained information from queries by suggesting a semantic similarity-based clustering approach.

  11. This content must be understood on the basis of the definition given in Sect. 2.1.

  12. Note that we do not consider the whole document to be completely ironic. Instead, we highlight the possibility to have fragments or sentences that can be considered to be ironic.

  13. No matter if two or more sentences belong to a same document.

  14. Unlike the others sets which are only labeled with positive or negative polarity.

  15. Both annotators are bilingual and they work as English–Spanish translators.

  16. It is unlikely to expect more ironic documents in these data sets because they were not compiled with the purpose of finding irony.

  17. Cf. [34] regarding the presence of irony to produce funny effects in user-generated contents.

  18. Specifically, note that most sentences of the set \(books\), which were considered to be ironic by the annotators, when re-analyzing within their contexts acquire their real meaning: They are only positive or negative utterances without any figurative sense.

References

  1. Artstein R, Poesio M (2008) Inter-coder agreement for computational linguistics. Comput Linguistics 34(4):555–596

    Article  Google Scholar 

  2. Atserias J, Casas B, Comelles E, González M, Padró L, Padró M (2006) Freeling 1.3: syntactic and semantic services in an open-source nlp library. In: Proceedings of the 5th international conference on language resources and evaluation, pp 48–55

  3. Attardo S (2007) Irony as relevant inappropriateness. In: Gibbs R, Colston H (eds) Irony in language and thought. Taylor and Francis Group, London, pp 135–174

    Google Scholar 

  4. Banerjee S, Agarwal N (2012) Analyzing collective behavior from blogs using swarm intelligence. Knowl Inf Syst. doi:10.1007/s10115-012-0512-y

  5. Beydoun G, Hoffmann A (2012) Dynamic evaluation of the development process of knowledge-based information systems. Knowl Inf Syst. doi:10.1007/s10115-012-0491-z

  6. Burfoot C, Baldwin T (2009) Automatic satire detection: are you having a laugh? In: ACL-IJCNLP ’09: proceedings of the ACL-IJCNLP 2009 conference short papers, pp 161–164

  7. Carvalho P, Sarmento L, Silva M, de Oliveira E (2009) Clues for detecting irony in user-generated contents: oh...!! It’s “so easy”; -). In: TSA ’09: proceeding of the 1st international CIKM workshop on topic-sentiment analysis for mass opinion. ACM, Hong Kong, China, pp 53–56

  8. Clark H, Gerrig R (1984) On the pretense theory of irony. J Exp Psychol Gen 113(1):121–126

    Article  Google Scholar 

  9. Colston H (2007) On necessary conditions for verbal irony comprehension. In: Gibbs R, Colston H (eds) Irony in language and thought. Taylor and Francis Group, London, pp 97–134

    Google Scholar 

  10. Colston H, Gibbs R (2007) A brief history of irony. In: Gibbs R, Colston H (eds) Irony in language and thought. Taylor and Francis Group, London, pp 3–24

    Google Scholar 

  11. Curcó C (2007) Irony: negation, echo, and metarepresentation. In: Gibbs R, Colston H (eds) Irony in language and thought. Taylor and Francis Group, London, pp 269–296

    Google Scholar 

  12. Davidov D, Tsur O, Rappoport A (2010) Semi-supervised recognition of sarcastic sentences in Twitter and Amazon. In: Proceedings of the 14th conference on computational natural language learning, CoNLL ’10. Association for Computational Linguistics, Stroudsburg, PA, USA, pp 107–116

  13. Francisco V, Gervás P, Peinado F (2010) Ontological reasoning for improving the treatment of emotions in text. Knowl Inf Syst 24(2):23

    Google Scholar 

  14. Gibbs R (2007) Irony in talk among friends. In: Gibbs R, Colston H (eds) Irony in language and thought. Taylor and Francis Group, London, pp 339–360

    Google Scholar 

  15. Gibbs R, Colston H (2007) The future of irony studies. In: Gibbs R, Colston H (eds) Irony in language and thought. Taylor and Francis Group, London

    Google Scholar 

  16. Giora R (1995) On irony and negation. Discourse Process 19(2):239–264

    Article  Google Scholar 

  17. Giora R, Balaban N, Fein O, Alkabets I (2005) Negation as positivity in disguise. In: Colston H, Katz A (eds) Figurative language comprehension: social and cultural influences. Erlbaum, Hillsdale, pp 233–258

    Google Scholar 

  18. Giora R, Federman S, Kehat A, Fein O, Sabah H (2005) Irony aptness. Humor 18:23–39

    Article  Google Scholar 

  19. Grice H (1975) Logic and conversation. In: Cole P, Morgan JL (eds) Syntax and semantics, vol 3. Academic Press, New York, pp 41–58

    Google Scholar 

  20. Horn L, Kato Y (2000) Introduction: negation and polarity at the millennium. In: Horn L, Kato Y (eds) Studies in negation and polarity. Oxford University Press, Oxford, pp 1–19

    Google Scholar 

  21. Kaup B, Lüdtke J, Zwaan R (2006) Processing negated sentences with contradictory predicates: is a door that is not open mentally closed? J Pragmat 38:1033–1050

    Article  Google Scholar 

  22. Kisilevich S, Ang CS, Last M (2011) Large-scale analysis of self-disclosure patterns among online social networks users: A Russian context. Knowl Inf Syst. doi:10.1007/s10115-011-0443-z

  23. Kreuz R (2001) Using figurative language to increase advertising effectiveness. In: Office of Naval Research Military Personnel Research Science Workshop. University of Memphis, Memphis, TN

  24. Kumon-Nakamura S, Glucksberg S, Brown M (2007) How about another piece of pie: the allusional pretense theory of discourse irony. In: Gibbs R, Colston H (eds) Irony in language and thought. Taylor and Francis Group, London

    Google Scholar 

  25. Langacker R (1991) Concept, image and symbol, the cognitive basis of grammar. Mounton de Gruyter, Berlin

    Book  Google Scholar 

  26. Liu J, Wang K (2012) Anonymizing bag-valued sparse data by semantic similarity-based clustering. Knowl Inf Syst. doi:10.1007/s10115-012-0515-8

  27. Lucariello J (2007) Situational irony: a concept of events gone away. In: Gibbs R, Colston H (eds) Irony in language and thought. Taylor and Francis Group, London, pp 467–498

    Google Scholar 

  28. Miller G (1995) Wordnet: a lexical database for english. Commun ACM 38(11):39–41

    Article  Google Scholar 

  29. Pang B, Lee L (2004) A sentimental education: sentiment analysis using subjectivity summarization based on minimum cuts. In: Proceedings of the ACL, pp 271–278

  30. Pang B, Lee L, Vaithyanathan S (2002) Thumbs up? Sentiment classification using machine learning techniques. In: Proceedings of the 2002 conference on empirical methods in natural language processing (EMNLP). Association for Computational Linguistics, Morristown, NJ, USA, pp 79–86

  31. Pedersen T, Patwardhan S, Michelizzi J (2004) Wordnet:similarity—measuring the relatedness of concepts. In: Proceeding of the 9th national conference on artificial intelligence (AAAI-04). Association for Computational Linguistics, Morristown, NJ, USA, pp 1024–1025

  32. Reyes A, Rosso P (2011) Mining subjective knowledge from customer reviews: a specific case of irony detection. In: Proceedings of the 2nd workshop on computational approaches to subjectivity and sentiment analysis (WASSA 2.011). Association for Computational Linguistics, pp 118–124

  33. Reyes A, Rosso P (2012) Making objective decisions from subjective data: detecting irony in customers reviews. Decis Support Syst 53(4):754–760. doi:10.1016/j.dss.2012.05.027

  34. Reyes A, Rosso P, Buscaldi D (2012) From humor recognition to irony detection: the figurative language of social media. Data Knowl Eng 74:1–12. doi:10.1016/j.datak.2012.02.005

  35. Sarmento L, Carvalho P, Silva M, de Oliveira E (2009) Automatic creation of a reference corpus for political opinion mining in user-generated content, In: TSA ’09: proceeding of the 1st international CIKM workshop on topic-sentiment analysis for mass opinion. ACM, Hong Kong, China, pp 29–36

  36. Sperber D, Wilson D (1992) On verbal irony. Lingua 87:53–76

    Article  Google Scholar 

  37. Tsur O, Davidov D, Rappoport A (2010) ICWSM—a great catchy name: semi-supervised recognition of sarcastic sentences in online product reviews. In: Cohen WW, Gosling S (eds) Proceedings of the 4t international AAAI conference on weblogs and social media. The AAAI Press, Washington, DC, pp 162–169

  38. Utsumi A (1996) A unified theory of irony and its computational formalization. In: Proceedings of the 16th conference on computational linguistics. Association for Computational Linguistics, Morristown, NJ, USA, pp 962–967

  39. Veale T, Hao Y (2009) Support structures for linguistic creativity: a computational analysis of creative irony in similes. In: Proceedings of CogSci 2009, the 31st annual meeting of the cognitive science society, pp 1376–1381

  40. Veale T, Hao Y (2010) Detecting ironic intent in creative comparisons. In: Proceedings of 19th European conference on artificial intelligence—ECAI 2010. IOS Press, Amsterdam, The Netherlands, pp 765–770

  41. Whissell C (2009) Using the revised dictionary of affect in language to quantify the emotional undertones of samples of natural language. Psychol Rep 105(2):509–521

    Article  Google Scholar 

  42. Wilson D, Sperber D (2007) On verbal irony. In: Gibbs R, Colston H (eds) Irony in language and thought. Taylor and Francis Group, London, pp 35–56

    Google Scholar 

  43. Zagibalov T, Belyatskaya K, Carroll J (2010) Comparable English-Russian book review corpora for sentiment analysis. In: Proceedings of the 1st workshop on computational approaches to subjectivity and sentiment analysis. Lisbon, Portugal, pp 67–72

Download references

Acknowledgments

The research work of Paolo Rosso was done in the framework of the European Commission WIQ-EI Web Information Quality Evaluation Initiative (IRSES grant no. 269180) project within the FP 7 Marie Curie People, the DIANA-APPLICATIONS - Finding Hidden Knowledge in Texts: Applications (TIN2012-38603-C02-01) project, and the VLC/CAMPUS Microcluster on Multimodal Interaction in Intelligent Systems.

Author information

Authors and Affiliations

  1. Laboratorio de Tecnologías Lingüísticas, Instituto Superior de Intérpretes y Traductores, Mexico City, 06500, Mexico

    Antonio Reyes

  2. Natural Language Engineering Lab, ELIRF, DSIC, Universitat Politècnica de València, Valencia, Spain

    Paolo Rosso

Authors
  1. Antonio Reyes
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Paolo Rosso
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Antonio Reyes.

Appendix A: Sample of the most ironic sentences

Appendix A: Sample of the most ironic sentences

According to our model predictions, here are presented some of the most ironic sentences. Each sentence has a document identifier. Such identifiers were kept as in the original data sets in order to facilitate their location.

  1. 1.

    \(Movies2\)

    • “Expecting them to give the viewer insights into the human condition is like expecting your car to vacuum your house” (doc. id. cv116_28942.txt).

    • “That degree of complexity combined with those very realistic looking dinosaur effects is just about as much as I require” (doc. id. cv116_28942.txt).

    • “Moulin Rogue is an original, and an original, even a flawed one, is a thing to be cherished” (doc. id. cv275_28887.txt).

    • “In some respects, Rush Hour is the ultimate exercise in cliched filmmaking. The hero is the renegade cop that prefers to work alone. The cop in question cannot solve the case until he gets in trouble. All Chinese people are somehow involved in the criminal element. The duo must always be completely mismatched. The hero has to say some smart-assed comment before (and after) shooting someone. However, that doesn’t necessarily make for a bad film” (doc. id. cv402_14425.txt).

    • “Making her dramatic debut, after appearing in over 300 triple X adult films, porn star Nina Hartley takes command of her role with considerable assurance and a screen presence which puts many other contemporary ’straight’ actresses to shame” (doc. id. cv422_9381.txt).

    • “If I’d laughed any more, I might have needed an iron lung” (doc. id. cv507_9220.txt).

    • “I never believed love at first site was possible until I saw this film” (doc. id. cv513_6923.txt).

    • “Usually a movie is about something more than a soiled rug” (doc. id. cv718_11434.txt).

    • “I remember really enjoying this movie when I saw it years ago. I guess my memory really sucks” (doc. id. cv982_22209.txt).

    • “It’s not that there isn’t anything positive to say about the film. There is. After 92 minutes, it ends.” (doc. id. cv123_12165.txt).

    • “There’s an enormous woman (played by transvestite porn star)” (doc. id. cv142_23657.txt).

    • “However, isn’t bad at all. The actors do the best they can with the bad material” (doc. id. cv733_9891.txt).

  1. 2.

    \(Movies1\)

    • “The only actor in the movie with any demonstrable talent is a cute little prairie dog named Petey” (doc. id. cv039_tok-11790.txt).

    • “This film needed that whole theater-shaking: they needed to wake everybody up because they were so bored” (doc. id. cv229_tok-9484.txt).

    • “Appreciate this movie for the few weeks it will be in theaters folks” (doc. id. cv342_tok-24681.txt).

    • “I hated this movie for every second that I sat watching it, and I actively hate it now, days later, with the simpering, superficial, nauseatingly sentimental images forever plaguing my memories” (doc. id. cv352_tok-15921.txt).

    • “It’s too trashy to be good drama, but too dramatic to be good trash” (doc. id. cv494_tok-11693.txt).

    • “I only wish that I could make that one hour and forty-five minutes of my life re-appear” (doc. id. cv495_tok-18551.txt).

    • “In order to make the film a success, all they had to do was cast two extremely popular and attractive stars, have them share the screen for about two hours and then collect the profits” (doc. id. cv176_tok-15918.txt).

    • “(Why, oh why, couldn’t Lucas use computers to substitute better performers in the lead roles?)” (doc. id. cv228_tok-8817.txt).

    • “Nostalgia appears to have a great appeal, but don’t you think we could have more than 14 years before we yearn for the past?” (doc. id. cv173_tok-11316.txt).

    • “The weak scenes could have been cut, but then there wouldn’t have been much left” (doc. id. cv198_tok-11090.txt).

    • “It’s not a silent movie; there is lots of atmospheric music, occasional screams and weird sound effects, but nobody ever utters an audible word; unfortunately, is so bad that it’s really bad” (doc. id. cv524_tok-20616.txt).

    • “It seems that comedy is the main motive, and the violence is only intended to punctuate the laughs. Unfortunately, there are no laughs” (doc. id. cv680_tok-12227.txt).

  1. 3.

    \(Books\)

    • “Essentially the entire plot can be summarized in a sentence of two, girl falls in love with boy, girl becomes damsel in distress, boy saves girl, end of.....” (doc. id. document 017 Negative).

    • “Yes that literally is the entire plot, but far worse than this is the complete lack of intelligent character design” (doc. id. document 017 Negative).

    • “Harry goes to Hogwarts, bad guys try to kill Harry, battle with the bad guys, Harry triumphs - hurrah!” (doc. id. document 043 Negative).

    • “In fact I could see myself possibly enjoying this book ten years ago. Than again, maybe not” (doc. id. document 108 Negative).

  1. 4.

    \(Articles\)

    • “As we examine the passengers’ cell-phone calls and flight recordings, we get a sense of the incredible courage displayed by these ordinary men and women” (doc. id. 014-test-0153.satire).

    • “Despite years of diplomatic stalemate in the Mideast crisis, Syrian officials appeared eager to mend troubled Arab–Israeli relations this week by participating in a second round of U.S.-led peace talks, which feature representatives from every country in the region, as well as a complimentary continental breakfast in the hotel lobby” (doc. id. 016-test-0165.satire).

    • “Unfortunately, most of the men and women who passed by seemed to speak only a bizarre Asian dialect unknown to me, and those who could communicate were more interested in selling me exotic cologne out of a duffel bag” (doc. id. 022-test-0294.satire).

    • “This is merely about improving liquidity, said King” (doc. id. 095-test-1483.satire).

    • “Virtually free, except for digging, pumping, processing, storage, by-product-disposal and shipping costs” (doc. id. 179-training-1407.satire).

    • “Maybe the one person who allowed Bush to ignore the opinions of 45 percent of America has a busy schedule” (doc. id. 144-training-0769.satire).

Rights and permissions

Reprints and permissions

About this article

Cite this article

Reyes, A., Rosso, P. On the difficulty of automatically detecting irony: beyond a simple case of negation. Knowl Inf Syst 40, 595–614 (2014). https://doi.org/10.1007/s10115-013-0652-8

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10115-013-0652-8

Keywords

Search

Navigation