Skip to main content
SpringerLink
Log in

Sarcasm Annotation and Detection in Tweets

  • Conference paper
  • First Online:
Book cover Computational Linguistics and Intelligent Text Processing (CICLing 2017)

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 10762))

Abstract

Identifying sarcasm in text is a challenging task which can be difficult also for humans, in particular in very short texts with little explicit context, such as tweets (Twitter messages). The paper presents a comparison of three sets of tweets marked for sarcasm, two annotated manually and one annotated using the common strategy of relying on the authors correctly using hashtags to mark sarcasm. To evaluate the difficulty of the datasets, a state-of-the-art system for automatic sarcasm detection in tweets was implemented. Experiments on the two manually annotated datasets show comparable results, while deviating considerably from results on automatically annotated data, indicating that using hashtags is not a reliable approach to creating Twitter sarcasm corpora.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    https://about.twitter.com/company, http://newsroom.fb.com/company-info/.

  2. 2.

    www.ling.upenn.edu/courses/Fall_2003/ling001/penn_treebank_pos.html.

References

  1. Bamman, D., Smith, N.A.: Contextualized sarcasm detection on Twitter. In: Proceedings of the 9th International Conference on Web and Social Media, pp. 574–577. Oxford, United Kingdom (2015)

    Google Scholar 

  2. Barbieri, F., Ronzano, F., Saggion, H.: UPF-taln: SemEval 2015 Task 10 and 11 sentiment analysis of literal and figurative language in Twitter. In: Proceedings of the 9th International Workshop on Semantic Evaluation (SemEval 2015), pp. 704–708. Denver, Colorado (2015)

    Google Scholar 

  3. Carvalho, P., Sarmento, L., Silva, M.J., de Oliveira, E.: Clues for detecting irony in user-generated contents: Oh...!! it’s “so easy”; -). In: Proceedings of the 1st International CIKM Workshop on Topic-Sentiment Analysis for Mass Opinion, pp. 53–56. Hong Kong, China (2009)

    Google Scholar 

  4. Cliche, M.: The Sarcasm Detector. http://www.thesarcasmdetector.com/about/ (2014)

  5. Daily Mail: Frustrated air passenger arrested under Terrorism Act after Twitter joke about bombing airport. http://www.dailymail.co.uk/news/article-1244091/Man-arrested-Twitter-joke-bombing-airport-Terrorism-Act.html (2010)

  6. Davidov, D., Tsur, O., Rappoport, A.: Semi-supervised recognition of sarcastic sentences in Twitter and Amazon. In: Proceedings of the 14th Conference on Computational Natural Language Learning, pp. 107–116. Uppsala, Sweden (2010)

    Google Scholar 

  7. Department of Homeland Security: computer based annual social media analytics subscription. https://www.fbo.gov/?s=opportunity&mode=form&id=8aaf9a50dd4558899b0df22abc31d30e&tab=core&_cview=0 (2014)

  8. Forslid, E., Wikén, N.: Automatic irony- and sarcasm detection in Social media. Master’s thesis, Uppsala University, Uppsala, Sweden (2015)

    Google Scholar 

  9. Fredriksen, V., Jahren, B.E.: Twitter sentiment analysis: exploring automatic creation of sentiment lexica. Master’s thesis, Norwegian University of Science and Technology, Trondheim, Norway (2016)

    Google Scholar 

  10. Ghosh, A., Li, G., Veale, T., Rosso, P., Shutova, E., Reyes, A., Barnden, J.: SemEval-2015 Task 11: sentiment analysis of figurative language in Twitter. In: Proceedings of the 9th International Workshop on Semantic Evaluation (SemEval 2015), pp. 470–478. Denver, Colorado (2015a)

    Google Scholar 

  11. Ghosh, D., Guo, W., Muresan, S.: Sarcastic or not: word embeddings to predict the literal or sarcastic meaning of words. In: Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pp. 1003–1012. Lisbon, Portugal (2015b)

    Google Scholar 

  12. González-Ibáñez, R., Muresan, S., Wacholder, N.: Identifying sarcasm in Twitter: a closer look. In: Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics:shortpapers, pp. 581–586. Portland, Oregon (2011)

    Google Scholar 

  13. Hao, Y., Veale, T.: An ironic fist in a velvet glove: creative mis-representation in the construction of ironic similes. Minds Mach. 20(4), 635–650 (2010)

    Article  Google Scholar 

  14. Hee, C.V., Lefever, E., Hoste, V.: LT3: sentiment analysis of figurative tweets: piece of cake #NotReally. In: Proceedings of the 9th International Workshop on Semantic Evaluation (SemEval 2015), pp. 684–688. Denver, Colorado (2015)

    Google Scholar 

  15. Joshi, A., Sharma, V., Bhattacharyya, P.: Harnessing context incongruity for sarcasm detection. In: Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing, pp. 757–762. Beijing, China (2015)

    Google Scholar 

  16. Karanasou, M., Doulkeridis, C., Halkidi, M.: DsUniPi: an SVM-based approach for sentiment analysis of figurative language on Twitter. In: Proceedings of the 9th International Workshop on Semantic Evaluation (SemEval 2015), pp. 709–713. Denver, Colorado (2015)

    Google Scholar 

  17. Khattri, A., Joshi, A., Bhattacharyya, P., Carman, M.J.: Your Sentiment Precedes You: using an author’s historical tweets to predict sarcasm. In: Proceedings of the 6th Workshop on Computational Approaches to Subjectivity, Sentiment and Social Media Analysis (WASSA 2015), pp. 25–30. Lisbon, Portugal (2015)

    Google Scholar 

  18. Kunneman, F., Liebrecht, C., van Mulken, M., van den Bosch, A.: Signaling sarcasm: from hyperbole to hashtag. Inf. Process. Manag. 51(4), 500–509 (2015)

    Article  Google Scholar 

  19. Liebrecht, C., Kunneman, F., van den Bosch, A.: The perfect solution for detecting sarcasm in tweets #not. In: Proceedings of the 4th Workshop on Computational Approaches to Subjectivity, Sentiment and Social Media Analysis (WASSA 2013), pp. 29–37. Atlanta, Georgia (USA) (2013)

    Google Scholar 

  20. Loper, E., Bird, S.: NLTK: the natural language toolkit. In: Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics. Workshop on Effective Tools and Methodologies for Teaching Natural Language Processing and Computational Linguistics, pp. 63–70. ACL, Philadelphia, Pennsylvania (2002)

    Google Scholar 

  21. McGillion, S., Alonso, H.M., Plank, B.: CPH: sentiment analysis of figurative language on Twitter #easypeasy #not. In: Proceedings of the 9th International Workshop on Semantic Evaluation (SemEval 2015), pp. 699–703. Denver, Colorado (2015)

    Google Scholar 

  22. Mohammad, S.M., Kiritchenko, S., Zhu, X.: NRC-Canada: building the state-of-the-art in sentiment analysis of tweets. In: Proceedings of the 7th International Workshop on Semantic Evaluation (SemEval-2013), pp. 321–327. Atlanta, Georgia (USA) (2013)

    Google Scholar 

  23. Årup Nielsen, F.: A new ANEW: evaluation of a word list for sentiment analysis in microblogs. In: Proceedings of the ESWC2011 Workshop on ‘Making Sense of Microposts’: Big Things come in Small Packages, pp. 93–98. Heraklion, Crete (2011)

    Google Scholar 

  24. Özdemir, C., Bergler, S.: ClaC-SentiPipe: SemEval2015 Subtasks 10 B, E, and Task 11. In: Proceedings of the 9th International Workshop on Semantic Evaluation (SemEval 2015), pp. 479–485. Denver, Colorado (2015)

    Google Scholar 

  25. Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, V., Vanderplas, J., Passos, A., Cournapeau, D., Brucher, M., Perrot, M., Duchesnay, E.: Scikit-learn: machine learning in Python. J. Mach. Learn. Res. 12, 2825–2830 (2011)

    MathSciNet  MATH  Google Scholar 

  26. Ptáček, T., Habernal, I., Hong, J.: Sarcasm detection on Czech and English Twitter. In: Proceedings of COLING 2014, the 25th International Conference on Computational Linguistics: Technical Papers, pp. 213–223. Dublin, Ireland (2014)

    Google Scholar 

  27. Riloff, E., Qadir, A., Surve, P., Silva, L.D., Gilbert, N., Huang, R.: Sarcasm as contrast between a positive sentiment and negative situation. In: Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing (EMNLP 2013). Seattle, Washington (2013)

    Google Scholar 

  28. Wang, H., Castanon, J.A.: Sentiment expression via emoticons on social media. In: IEEE International Conference on Big Data (Big Data 2015), pp. 2404–2408. Santa Clara, California (2015)

    Google Scholar 

  29. Xu, H., Santus, E., Laszlo, A., Huang, C.R.: LLT-PolyU: identifying sentiment intensity in ironic tweets. In: Proceedings of the 9th International Workshop on Semantic Evaluation (SemEval 2015), pp. 673–678. Denver, Colorado (2015)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science, Norwegian University of Science and Technology, NO–7491, Trondheim, Norway

    Johan G. Cyrus M. Ræder & Björn Gambäck

Authors
  1. Johan G. Cyrus M. Ræder
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Björn Gambäck
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Björn Gambäck .

Editor information

Editors and Affiliations

  1. CIC, Instituto Politécnico Nacional, Mexico City, Mexico

    Alexander Gelbukh

Rights and permissions

Reprints and permissions

Copyright information

© 2018 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Ræder, J.G.C.M., Gambäck, B. (2018). Sarcasm Annotation and Detection in Tweets. In: Gelbukh, A. (eds) Computational Linguistics and Intelligent Text Processing. CICLing 2017. Lecture Notes in Computer Science(), vol 10762. Springer, Cham. https://doi.org/10.1007/978-3-319-77116-8_5

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-77116-8_5

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-77115-1

  • Online ISBN: 978-3-319-77116-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation