Skip to main content
SpringerLink
Log in

Transformer-Based Word Embedding With CNN Model to Detect Sarcasm and Irony

  • Research Article-Computer Engineering and Computer Science
  • Published:
Arabian Journal for Science and Engineering Aims and scope Submit manuscript

Abstract

Accurate semantic illustrations of text data and conclusive information extraction are major strides towards correct computation of sentence meaning, particularly for figurative languages like humor, irony, and sarcasm. We propose an encoder model called LMTweets, trained on 500 k tweets scraped from Twitter and social media. LMTweets are used to extract the dataset's features, namely SemEval 2018 Task 3. An (Irony), SARC (Sarcasm), and Riloff (Sarcasm). The extracted features are passed as input to the convolution neural network model to classify the text as sarcastic/non-sarcastic and irony/non-irony. We also apply five classification algorithms for the detection of sarcasm/irony, namely Naive Bayes (NB), Logistic Regression (LR), Support Vector Machine (SVM), Random Forest (RF), K-Nearest Neighbors (KNN), six deep learning algorithms, namely Convolutional Neural Network (CNN), Long Short Term Memory (LSTM), Gated Recurrent Unit (GRU), GRU-Pooling, LSTM-Attention Mechanism (AM), GRU-AM and six transformer models namely BERT, RoBERTa, ELECTRA, XLNet, XLM-RoBERTa, and ULMFIT. For the implementation purpose, Keras API is used with Tensorflow as the backend with Python. The performance parameters considered are precision, recall, accuracy, AUC, and f1-score. Experimental results show that LMTweets + CNN model performs better among all models used and gives around 6% better performance on SemEval 2018 Task 3. A dataset, 2–3% on Rillof and SARC datasets shows the results obtained by applying various models are statistically different. The results are validated by applying the ANOVA one-way statistical test.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

Notes

  1. https://www.omnicoreagency.com/twitter-statistics/

Abbreviations

AM:

Attention Mechanism

ANOVA:

Analysis of Variance

BERT:

Bi-directional Encoder Decoder Representation

Bi-LSTM:

Bi-directional Long Short Term Memory

Bi-GRU:

Bi-directional Gated Recurrent Unit

ELMo:

Embeddings from Language Models

HAN:

Hierarchical Attention Network

KNN:

K-Nearest Neighbor

MNB:

Multinomial Naïve Bayes

NB:

Naïve Bayes

GRU:

Gated Recurrent Unit

RF:

Random Forest

SVM:

Support Vector Machine

TF-IDF:

Term Frequency-Inverse Document Frequency

CNN:

Convolutional Neural Network

LSTM:

Long Short Term Memory

RNN:

Recurrent Neural Network

SARC:

Self Annotated Reddit Corpus

NLP:

Natural Language Processing

References

  1. Abulaish, M.; Kamal, A.; Zaki, M.J.: A survey of figurative language and its computational detection in online social networks. ACM Trans. WEB (TWEB) 14(1), 1–52 (2020)

    Article  Google Scholar 

  2. Hepburn, A.D.: Manual of English rhetoric. Wilson Hinkle Company, Durham (1875)

    Google Scholar 

  3. Wintner, S.; Elliott, D.; Garoufi, K., Kiela, D.; & Vulić, I. (2014, April). Proceedings of the Student Research Workshop at the 14th Conference of the European Chapter of the Association for Computational Linguistics. In Proceedings of the Student Research Workshop at the 14th Conference of the European Chapter of the Association for Computational Linguistics.

  4. Bharti, S. K.; Babu, K. S.; & Jena, S. K. (2015, August). Parsing-based sarcasm sentiment recognition in twitter data. In 2015 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM) (pp. 1373–1380). IEEE.

  5. Joshi, A.; Bhattacharyya, P.; Carman, M.J.: Automatic sarcasm detection: A survey. ACM Comput Surv. (CSUR) 50(5), 1–22 (2017)

    Article  Google Scholar 

  6. Qadir, A.; Riloff, E.; & Walker, M. (2015). Learning to recognize affective polarity in similes. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing (pp. 190–200).

  7. Gibbs, R.W.; Gibbs, R.W.: Intentions in the Experience of Meaning. Cambridge University Press (1999)

    Book  Google Scholar 

  8. Stringfellow, F. Jr.: Meaning of Irony, The: A Psychoanalytic Investigation. SUNY Press, Albany (1994)

    Google Scholar 

  9. Gibbs, R.W.; Jr.; Gibbs, R.W.; Colston, H.L. (eds.): Irony in language and thought: A cognitive science reader. Psychology Press, Hove (2007)

    Google Scholar 

  10. Potamias, R. A.; Siolas, G.; & Stafylopatis, A. (2019, May). A robust deep ensemble classifier for figurative language detection. In International Conference on Engineering Applications of Neural Networks (pp. 164–175). Springer, Cham.

  11. Sulis, E.; Farías, D.I.H.; Rosso, P.; Patti, V.; Ruffo, G.: Figurative messages and affect in Twitter: Differences between# irony,# sarcasm and# not. Knowl.-Based Syst. 108, 132–143 (2016)

    Article  Google Scholar 

  12. Jones, K. S. (1972). A statistical interpretation of term specificity and its application in retrieval. J. Doc.

  13. Azam, M. S.; Raihan, M. A.; & Rana, H. K. An Experimental Study of Various Machine Learning Approaches in Heart Disease Prediction. Int. J. Comput. Appl., 975, 8887

  14. Liaw, A.; Wiener, M.: Classification and regression by random forest. R news 2(3), 18–22 (2002)

    Google Scholar 

  15. Gibbs, R.W.: On the psycholinguistics of sarcasm. J. Exp. Psychol. Gen. 115(1), 3 (1986)

    Article  Google Scholar 

  16. Rosenthal, S.; Farra, N.; & Nakov, P. (2017, August). SemEval-2017 task 4: Sentiment analysis in Twitter. In Proceedings of the 11th international workshop on semantic evaluation (SemEval-2017) (pp. 502–518).

  17. Kumar, A.; Sangwan, S.R.; Arora, A.; Nayyar, A.; Abdel-Basset, M.: Sarcasm detection using soft attention-based bidirectional long short-term memory model with convolution network. IEEE access 7, 23319–23328 (2019)

    Article  Google Scholar 

  18. Ling, J.; & Klinger, R. (2016, May). An empirical, quantitative analysis of the differences between sarcasm and irony. In European semantic web conference (pp. 203–216). Springer, Cham.

  19. Van Hee, C.; Lefever, E.; Hoste, V.: Exploring the fine-grained analysis and automatic detection of irony on Twitter. Lang. Resour. Eval. 52(3), 707–731 (2018)

    Article  Google Scholar 

  20. Bouazizi, M.; Ohtsuki, T.O.: A pattern-based approach for sarcasm detection on twitter. IEEE Access 4, 5477–5488 (2016)

    Article  Google Scholar 

  21. Saha, S.; Yadav, J.; Ranjan, P.: Proposed approach for sarcasm detection in twitter. Indian J. Sci. Technol. 10(25), 1–8 (2017)

    Article  Google Scholar 

  22. Sarsam, S.M.; Al-Samarraie, H.; Alzahrani, A.I.; Wright, B.: Sarcasm detection using machine learning algorithms in Twitter: A systematic review. Int. J. Mark. Res. 62(5), 578–598 (2020)

    Article  Google Scholar 

  23. Farías, D. I. H.; Montes-y-Gómez, M.; Escalante, H. J.; Rosso, P.; & Patti, V. (2018, October). A knowledge-based weighted KNN for detecting irony in Twitter. In Mexican International Conference on Artificial Intelligence (pp. 194–206). Springer, Cham.

  24. CHIA, Z. L.; Ptaszynski, M.; & Masui, F. (2019). Exploring machine learning techniques for irony detection. In Proceedings of the Annual Conference of JSAI 33rd Annual Conference, 2019 (pp. 2A4E203–2A4E203). The Japanese Society for Artificial Intelligence.

  25. Pawar, N.; & Bhingarkar, S. (2020, June). Machine Learning based Sarcasm Detection on Twitter Data. In 2020 5th International Conference on Communication and Electronics Systems (ICCES) (pp. 957–961). IEEE.

  26. Ghosh, A.; & Sarkar, K. (2020, February). Irony Detection in Bengali Tweets: A New Dataset, Experimentation and Results. In International Conference on Computational Intelligence in Data Science (pp. 112–127). Springer, Cham.

  27. Mehndiratta, P.; Soni, D.: Identification of sarcasm using word embeddings and hyperparameters tuning. J. Discrete Math. Sci. Cryptogr. 22(4), 465–489 (2019)

    Article  Google Scholar 

  28. Ghosh, A.; & Veale, T. (2016, June). Fracking sarcasm using neural network. In Proceedings of the 7th workshop on computational approaches to subjectivity, sentiment and social media analysis (pp. 161–169).

  29. Ilić, S.; Marrese-Taylor, E.; Balazs, J. A.; & Matsuo, Y. (2018). Deep contextualized word representations for detecting sarcasm and irony. arXiv preprint arXiv:1809.09795.

  30. Wu, C.; Wu, F.; Wu, S.; Liu, J.; Yuan, Z.; & Huang, Y. (2018, June). Thu_ngn at semeval-2018 task 3: Tweet irony detection with densely connected lstm and multi-task learning. In Proceedings of The 12th International Workshop on Semantic Evaluation (pp. 51–56).

  31. Baziotis, C.; Athanasiou, N.; Papalampidi, P.; Kolovou, A.; Paraskevopoulos, G.; Ellinas, N.; & Potamianos, A. (2018). Ntua-slp at semeval-2018 task 3: Tracking ironic tweets using ensembles of word and character level attentive rnns. arXiv preprint arXiv:1804.06659.

  32. Zhang, S.; Zhang, X.; Chan, J.; Rosso, P.: Irony detection via sentiment-based transfer learning. Inf. Process. Manage. 56(5), 1633–1644 (2019)

    Article  Google Scholar 

  33. Kumar, A.; Garg, G.: Empirical study of shallow and deep learning models for sarcasm detection using context in benchmark datasets. J. Ambient Intell. Humaniz. Comput. (2019). https://doi.org/10.1007/s12652-019-01419-7

    Article  Google Scholar 

  34. Shrivastava, M.; Kumar, S.: A pragmatic and intelligent model for sarcasm detection in social media text. Technol. Soc. 64, 101489 (2021)

    Article  Google Scholar 

  35. Potamias, R.A.; Siolas, G.; Stafylopatis, A.G.: A transformer-based approach to irony and sarcasm detection. Neural Comput. Appl. 32(23), 17309–17320 (2020)

    Article  Google Scholar 

  36. Avvaru, A.; Vobilisetty, S.; & Mamidi, R. (2020, July). Detecting Sarcasm in Conversation Context Using Transformer-Based Models. In Proceedings of the Second Workshop on Figurative Language Processing (pp. 98–103).

  37. Gregory, H.; Li, S.; Mohammadi, P.; Tarn, N.; Draelos, R.; & Rudin, C. (2020, July). A Transformer approach to contextual sarcasm detection in twitter. In Proceedings of the Second Workshop on Figurative Language Processing (pp. 270–275).

  38. Srivastava, H.; Varshney, V.; Kumari, S.; & Srivastava, S. (2020, July). A novel hierarchical BERT architecture for sarcasm detection. In Proceedings of the Second Workshop on Figurative Language Processing (pp. 93–97).

  39. Agrawal, A.; Jha, A. K.; Jaiswal, A.; & Kumar, V. (2020, August). Irony Detection Using Transformers. In 2020 International Conference on Computing and Data Science (CDS) (pp. 165–168). IEEE.

  40. Van Hee, C.; Lefever, E.; & Hoste, V. (2018, June). Semeval-2018 task 3: Irony detection in english tweets. In Proceedings of The 12th International Workshop on Semantic Evaluation (pp. 39–50).

  41. Riloff, E.; Qadir, A.; Surve, P.; De Silva, L.; Gilbert, N.; & Huang, R. (2013, October). Sarcasm as contrast between a positive sentiment and negative situation. In Proceedings of the 2013 conference on empirical methods in natural language processing (pp. 704–714).

  42. Jianqiang, Z. (2015, December). Pre-processing boosting Twitter sentiment analysis?. In 2015 IEEE International Conference on Smart City/SocialCom/SustainCom (SmartCity) (pp. 748–753). IEEE.

  43. Jianqiang, Z.; Xiaolin, G.: Comparison research on text pre-processing methods on twitter sentiment analysis. IEEE Access 5, 2870–2879 (2017)

    Article  Google Scholar 

  44. Tay, Y.; Tuan, L. A.; Hui, S. C.; & Su, J. (2018). Reasoning with sarcasm by reading in-between. arXiv preprint arXiv:1805.02856.

  45. Aha, D.W.; Kibler, D.; Albert, M.K.: Instance-based learning algorithms. Mach. Learn. 6(1), 37–66 (1991)

    Google Scholar 

  46. Tabaei, B.P.; Herman, W.H.: A multivariate logistic regression equation to screen for diabetes: development and validation. Diabetes Care 25(11), 1999–2003 (2002)

    Article  Google Scholar 

  47. Rish, I. (2001, August). An empirical study of the naive Bayes classifier. In IJCAI 2001 workshop on empirical methods in artificial intelligence (Vol. 3, No. 22, pp. 41–46).

  48. Kalchbrenner, N.; Grefenstette, E.; & Blunsom, P. (2014). A convolutional neural network for modelling sentences. arXiv preprint arXiv:1404.2188.

  49. Hochreiter, S.; & Schmidhuber, J. (1997). Long short-term memory. Neural computation, 9(8), 1735–1780. .

  50. Cho, K.; Van Merriënboer, B.; Bahdanau, D.; & Bengio, Y. (2014). On the properties of neural machine translation: Encoder-decoder approaches. arXiv preprint arXiv:1409.1259

  51. Howard, J.; & Ruder, S. (2018). Universal language model fine-tuning for text classification. arXiv preprint arXiv:1801.06146.

  52. Zhou, X.; Wan, X.; & Xiao, J. (2016, November). Attention-based LSTM network for cross-lingual sentiment classification. In Proceedings of the 2016 conference on empirical methods in natural language processing (pp. 247–256).

  53. Du, C.; Huang, L.: Text classification research with attention-based recurrent neural networks. Int. J. Comput. Commun. Control 13(1), 50–61 (2018)

    Article  Google Scholar 

  54. Pennington, J.; Socher, R.; & Manning, C. D. (2014, October). Glove: Global vectors for word representation. In Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP) (pp. 1532–1543).

  55. Devlin, J.; Chang, M. W.; Lee, K.; & Toutanova, K. (2018). Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805.

  56. Liu, Y.; Ott, M.; Goyal, N.; Du, J.; Joshi, M.; Chen, D.; ... & Stoyanov, V. (2019). Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692.

  57. Clark, K.; Luong, M. T.; Le, Q. V.; & Manning, C. D. (2020). Electra: Pre-training text encoders as discriminators rather than generators. arXiv preprint arXiv:2003.10555.

  58. Yang, Z.; Dai, Z.; Yang, Y.; Carbonell, J.; Salakhutdinov, R.; & Le, Q. V. (2019). Xlnet: Generalized autoregressive pretraining for language understanding. arXiv preprint arXiv:1906.08237.

  59. Conneau, A.; Khandelwal, K.; Goyal, N.; Chaudhary, V.; Wenzek, G.; Guzmán, F.; ... & Stoyanov, V. (2019). Unsupervised cross-lingual representation learning at scale. arXiv preprint arXiv:1911.02116.

  60. Hao, J.; Ho, T.K.: Machine learning made easy: A review of scikit-learn package in Python programming language. J. Educ. Behav. Stat. 44(3), 348–361 (2019)

    Article  Google Scholar 

  61. Onan, A.; Korukoğlu, S.; Bulut, H.: A hybrid ensemble pruning approach based on consensus clustering and multi-objective evolutionary algorithm for sentiment classification. Inf. Process. Manage. 53(4), 814–833 (2017)

    Article  Google Scholar 

  62. Maslej-Krešňáková, V.; Sarnovský, M.; Butka, P.; Machová, K.: Comparison of deep learning models and various text pre-processing techniques for the toxic comments classification. Appl. Sci. 10(23), 8631 (2020)

    Article  Google Scholar 

  63. Yüksel, A. E.; Türkmen, Y. A.; Özgür, A.; & Altınel, B. (2019, September). Turkish tweet classification with transformer encoder. In Proceedings of the International Conference on Recent Advances in Natural Language Processing (RANLP 2019) (pp. 1380–1387).

  64. Khodak, M.; Saunshi, N.; & Vodrahalli, K. (2017). A large self-annotated corpus for sarcasm. arXiv preprint arXiv:1704.05579.

  65. Hazarika, D.; Poria, S.; Gorantla, S.; Cambria, E.; Zimmermann, R.; & Mihalcea, R. (2018). Cascade: Contextual sarcasm detection in online discussion forums. arXiv preprint arXiv:1805.06413.

Download references

Author information

Authors and Affiliations

  1. IIT Roorkee Saharanpur Campus, Saharanpur, Uttar Pradesh, India

    Ravinder Ahuja & S. C. Sharma

Authors
  1. Ravinder Ahuja
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. S. C. Sharma
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ravinder Ahuja.

Appendix A

Appendix A

The configuration of the machine and packages used in this study are as follows:

Architecture: x86_64, CPU op-mode(s): 32-bit, 64-bit, CPU(s): 16, Model name: Intel(R) Xeon(R) CPU @ 2.30 GHz, CUDA Version: 11.2, GPU: V100, Number of GPUs:4, VRAM/per gpu: 32, Tensor Flow Version: 2.5.0, Keras Version: 2.5.0, Python 3.7.10, Pandas 1.1.5, Scikit-Learn 0.22.2.post1.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ahuja, R., Sharma, S.C. Transformer-Based Word Embedding With CNN Model to Detect Sarcasm and Irony. Arab J Sci Eng 47, 9379–9392 (2022). https://doi.org/10.1007/s13369-021-06193-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13369-021-06193-3

Keywords

Search

Navigation