Skip to main content
Springer Nature Link
Log in

Identifying suicidal emotions on social media through transformer-based deep learning

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Suicide-related cases throughout the world are increasing on a day-to-day basis. This is owing to lack of control on negative human emotions which lead one to take one’s life. Due to rapid development of social media, users are posting their views and daily activities including texts related to suicide. This research identifies negative emotions in suicidal postings which describe an individual’s mental health. We proposed two models for detecting negative emotions like anger, anxiety, depression, guilt, fear, sadness, and stress on social media. Our first model is a context-based bidirectional gated recurrent unit with multi-head attention and a convolutional neural network (C-BiGRU-MHA-CNN) that is meant for preserving contextual data and for maintaining long-term dependencies. This paper proposes the masked language modeling (MLM) and self-attention (SA) mechanism procedures for model training and to detect contextual features over context-free models. We also suggested the lexicon-based bidirectional long short-term memory with multi-head attention and convolutional neural network (L-BiLSTM-MHA-CNN) model. It is a single channel-based model that performs better when it comes to dealing with the lexicon-based approach against erstwhile methods. It combines input features with parts-of-speech (POS) tagging can train on embedding representations for identifying emotions, while recognizing those which are the most dominant in suicide-related texts. We compared the performance of our models with various word embeddings. We also conducted an ablation study in order to highlight the contribution of the most essential components in our models for achieving better performance. Our proposed models with the bidirectional encoder representations from transformers (BERT) mechanism have resulted in an outstanding performance against the state-of-the-art methods meant to identify emotions on text sequence data.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+
from $39.99 /Month
  • Starting from 10 chapters or articles per month
  • Access and download chapters and articles from more than 300k books and 2,500 journals
  • Cancel anytime
View plans

Buy Now

Price includes VAT (Canada)

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Algorithm 1
Fig. 3
Fig. 4
Algorithm 2
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18

Similar content being viewed by others

Explore related subjects

Discover the latest articles, books and news in related subjects, suggested using machine learning.

Data Availability

Data will be made available on reasonable request.

References

  1. Venek V, Scherer S, Morency L-P, Rizzo A, Pestian J (2017) Adolescent suicidal risk assessment in clinician-patient interaction. IEEE Trans Affect Com 8(2):204–215. https://doi.org/10.1109/TAFFC.2016.2518665

    Google Scholar 

  2. Caicedo R, Gómez J, Sasieta H (2020) Assessment of supervised classifiers for the task of detecting messages with suicidal ideation. Heliyon 6:04412. https://doi.org/10.1016/j.heliyon.2020.e04412

    Google Scholar 

  3. Platt S, Arensman E, Rezaeian M (2019) National suicide prevention strategies – progress and challenges. Crisis 40:75–82. https://doi.org/10.1027/0227-5910/a000587

    Google Scholar 

  4. Ji S, Yu C, Fung S-f, Pan S, Long G (2018) Supervised learning for suicidal ideation detection in online user content. Complexity 2018:1–10. https://doi.org/10.1155/2018/6157249

    Google Scholar 

  5. Sarsam S, Al-Samarraie H, Alzahrani A, Alnumay W, Smith A (2021) A lexicon-based approach to detecting suicide-related messages on twitter. Biomed Signal Process 65:102355. https://doi.org/10.1016/j.bspc.2020.102355

    Google Scholar 

  6. Kumar R, Rao K, Nayak S.R, Chandra R (2020) Suicidal ideation prediction in twitter data using machine learning techniques. J Interdiscip Math 23:117–125. https://doi.org/10.1080/09720502.2020.1721674

    Google Scholar 

  7. Liu D, Fu Q, Wan C, Liu X, Jiang T, Liao G, Qiu X, Liu R (2020) Suicidal ideation cause extraction from social texts. IEEE Access 8:169333–169351. https://doi.org/10.1109/ACCESS.2020.3019491https://doi.org/10.1109/ACCESS.2020.3019491

    Google Scholar 

  8. Wu J, He Y, Yu L, Lai KR (2020) Identifying emotion labels from psychiatric social texts using a bi-directional lstm-cnn model. IEEE Access 8:66638–66646

    Google Scholar 

  9. Dheeraj K, Ramakrishnudu T (2021) Negative emotions detection on online mental-health related patients texts using the deep learning with mha-bcnn model. Expert Syst Appl 182:115–265. https://doi.org/10.1016/j.eswa.2021.115265

    Google Scholar 

  10. Pennington J, Socher R, Manning C (2014) Glove: Global vectors for word representation. In: Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), vol 14, pp 1532–1543. https://doi.org/10.3115/v1/D14-1162

  11. Aladağ A.E, Muderrisoglu S, Akbas N, Zahmacioglu O, Bingol H (2018) Detecting suicidal ideation on forums and blogs: Proof-of-concept study. J Med Internet Res 20:215. https://doi.org/10.2196/jmir.9840

    Google Scholar 

  12. Tadesse M, Lin H, Xu B, Yang L (2019) Detection of suicide ideation in social media forums using deep learning. Algorithms 13:7. https://doi.org/10.3390/a13010007

    Google Scholar 

  13. Jashinsky J, Burton S, Hanson C, West J, Giraud-Carrier C, Barnes M, Argyle T (2013) Tracking suicide risk factors through twitter in the us. Crisis 35:1–9. https://doi.org/10.1027/0227-5910/a000234

    Google Scholar 

  14. Desmet B, Hoste V (2013) Emotion detection in suicide notes. Expert Syst Appl 40:6351–6358. https://doi.org/10.1016/j.eswa.2013.05.050

    Google Scholar 

  15. Li Y, Mihalcea R, Wilson S (2018) Text-Based detection and understanding of changes in mental health: 10th International conference, SocInfo 2018, St. Petersburg, Russia, September 25-28, 2018, Proceedings, Part II, pp 176–188. https://doi.org/10.1007/978-3-030-01159-8_17

  16. Calvo R, Milne D, Hussain S, Christensen H (2017) Natural language processing in mental health applications using non-clinical texts. Nat Lang Eng 23:1–37. https://doi.org/10.1017/S1351324916000383

    Google Scholar 

  17. Amir S, Coppersmith G, Carvalho P, Silva M, Wallace B (2017) Quantifying mental health from social media with neural user embeddings

  18. Gkotsis G, Oellrich A, Velupillai S, Liakata M, Hubbard T, Dobson R, Dutta R (2017) Characterisation of mental health conditions in social media using informed deep learning. Sci Rep 7:45141. https://doi.org/10.1038/srep45141

    Google Scholar 

  19. Huang X, Paul M, Burke R, Dernoncourt F, Dredze M (2021) User factor adaptation for user embedding via multitask learning

  20. Bahgat M, Wilson S, Magdy W (2020) Towards using word embedding vector space for better cohort analysis. blueProceedings of the International AAAI Conference on Web and Social Media 14(1):919–923

    Google Scholar 

  21. Xia C, Zhao D, Wang J, Jing L, Ma J (2018) ICSH 2018: LSTM based sentiment analysis for patient experience narratives in e-survey tools: international conference, ICSH 2018, Wuhan, China, July 1–3, 2018, Proceedings, pp 231–239. https://doi.org/10.1007/978-3-030-03649-2_23

  22. Bagroy S, Kumaraguru P, Choudhury M (2017) A social media based index of mental well-being in college campuses. In: Proceedings of the 2017 CHI conference on human factors in computing systems vol 2017, pp 1634–1646. https://doi.org/10.1145/3025453.3025909

  23. Chatterjee A, Gupta U, Chinnakotla M, Srikanth R, Galley M, Agrawal P (2018) Understanding emotions in text using deep learning and big data. Comput Hum Behav 93:309–317. https://doi.org/10.1016/j.chb.2018.12.029

    Google Scholar 

  24. Ren F, Kang X, Quan C (2016) Examining accumulated emotional traits in suicide blogs with an emotion topic model. IEEE J Biomed Health Inform 20(5):1384–1396. https://doi.org/10.1109/JBHI.2015.2459683

    Google Scholar 

  25. Benton A, Mitchell M, Hovy D (2017) Multi-task learning for mental health using social media text

  26. Ghosh S, Ekbal A, Bhattacharyya P (2021) A multitask framework to detect depression, sentiment and multi-label emotion from suicide notes, Cognit Comput. https://doi.org/10.1007/s12559-021-09828-7

  27. Allen N, Nelson B, Brent D, Auerbach R (2019) Short-term prediction of suicidal thoughts and behaviors in adolescents: Can recent developments in technology and computational science provide a breakthrough? J Affect Disord, vol 250. https://doi.org/10.1016/j.jad.2019.03.044

  28. Glenn JJ, Nobles AL, Barnes LE, Teachman BA (2020) Can text messages identify suicide risk in real time? a within-subjects pilot examination of temporally sensitive markers of suicide risk. Clin Psychol Sci 8(4):704–722. https://doi.org/10.1177/2167702620906146

    Google Scholar 

  29. Ophir Y, Tikochinski R, Asterhan C, Sisso I, Reichart R (2020) Deep neural networks detect suicide risk from textual facebook posts. Sci Rep 10(1):1–10. https://doi.org/10.1038/s41598-020-73917-0

    Google Scholar 

  30. Canales L, Strapparava C, Boldrini E, Martínez-Barco P (2020) Intensional learning to efficiently build up automatically annotated emotion corpora. IEEE Trans Affect Comput 11(2):335–347. https://doi.org/10.1109/TAFFC.2017.2764470

    Google Scholar 

  31. Larsen ME, Boonstra TW, Batterham PJ, O’Dea B, Paris C, Christensen H (2015) We feel: Mapping emotion on twitter. IEEE J Biomed Health Inform 19(4):1246–1252. https://doi.org/10.1109/JBHI.2015.2403839

    Google Scholar 

  32. Hassan SB, Hassan SB, Zakia U (2020) Recognizing suicidal intent in depressed population using nlp: A pilot study. In: 2020 11th IEEE Annual information technology, electronics and mobile communication conference (IEMCON), pp 0121–0128. https://doi.org/10.1109/IEMCON51383.2020.9284832

  33. Chen Y, Yisheng L, Wang X, Li L (2018) Detecting traffic information from social media texts with deep learning approaches, IEEE trans Intell Transp Syst, pp 1–10. https://doi.org/10.1109/TITS.2018.2871269

  34. Devlin J, Chang M-W, Lee K, Toutanova K (2019) Bert: Pre-training of deep bidirectional transformers for language understanding. In: NAACL-HLT. https://doi.org/10.18653/v1/N19-1423https://doi.org/10.18653/v1/N19-1423

  35. Rhanoui M, Mikram M, Yousfi S, Barzali S (2019) A cnn-bilstm model for document-level sentiment analysis. Mach Learn Knowl Extr 1:832–847

    Google Scholar 

  36. Batbaatar E, Li M, Ryu KH (2019) Semantic-emotion neural network for emotion recognition from text. IEEE Access 7:111866–111878. https://doi.org/10.1109/ACCESS.2019.2934529

    Google Scholar 

  37. Lin S-C, Su W-Y, Chien P-C, Tsai M-F, Wang C-J (2020) Self-attentive sentimental sentence embedding for sentiment analysis. In: ICASSP 2020 - 2020 IEEE International conference on acoustics, speech and signal processing (ICASSP), pp 1678–1682. https://doi.org/10.1109/ICASSP40776.2020.9054274

  38. Xie J, Chen B, Gu X, Liang F, Xu X (2019) Self-attention-based bilstm model for short text fine-grained sentiment classification. IEEE Access 7:180558–180570

    Google Scholar 

  39. Song P, Geng C, Li Z (2019) Research on text classification based on convolutional neural network. In: 2019 International conference on computer network, electronic and automation (ICCNEA), pp 229–232. https://doi.org/10.1109/ICCNEA.2019.00052

  40. Gao K, Xu H, Gao C, Hao H, Deng J, Sun X (2018) Attention-based bilstm network with lexical feature for emotion classification. In: 2018 International joint conference on neural networks (IJCNN), pp 1–2. https://doi.org/10.1109/IJCNN.2018.8489577

  41. Bandhakavi AS, Wiratunga NPD, Massie S (2016) Lexicon based feature extraction for emotion text classification. Pattern Recognit Lett 93:133–142. https://doi.org/10.1016/j.patrec.2016.12.009

    Google Scholar 

  42. Singh V, Mukherjee M, Mehta G (2011) Sentiment and mood analysis of weblogs using pos tagging based approach. In: International conference on contemporary computing, vol 168, pp 313–324. https://doi.org/10.1007/978-3-642-22606-9_33

  43. Almeida A MG, Cerri R, Paraiso EC, Mantovani RG, Junior SB (2018) Applying multi-label techniques in emotion identification of short texts. Neurocomputing 320:35–46

    Google Scholar 

  44. Li X, Feng S, Wang D, Zhang Y (2019) Context-aware emotion cause analysis with multi-attention-based neural network. Knowl Based Syst 174:205–218. https://doi.org/10.1016/j.knosys.2019.03.008

    Google Scholar 

  45. Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser L, Polosukhin I (2017) Attention is all you need. arXiv:1706.03762 [cs.CL]

  46. Park S, Lee W, Moon IC (2015) Efficient extraction of domain specific sentiment lexicon with active learning. Pattern Recognit Lett 56:38–44. https://doi.org/10.1016/j.patrec.2015.01.004

    Google Scholar 

  47. Rao Y, Lei J, Liu W, Li Q, Chen M (2014) Building emotional dictionary for sentiment analysis of online news. World Wide Web 17(4):723–742. https://doi.org/10.1007/s11280-013-0221-9

    Google Scholar 

  48. Zhang Q, Lu R (2019) A multi-attention network for aspect-level sentiment analysis. Future Internet 11:157. https://doi.org/10.3390/fi11070157

    Google Scholar 

  49. Kingma PD, Ba LJ (2015) Adam: a method for stochastic optimization international conference on learning representations

  50. Plank B, Søgaard A, Goldberg Y (2016) Multilingual part-of-speech tagging with bidirectional long short-term memory models and auxiliary loss, pp 412–418. https://doi.org/10.18653/v1/P16-2067

  51. Ling W, Luís T, Marujo L, Astudillo R, Amir S, Dyer C, Black A, Trancoso I. (2015) Finding function in form: Compositional character models for open vocabulary word representation. https://doi.org/10.18653/v1/D15-1176

  52. Navarrete A, Martinez-Araneda C, Manzano C (2021) A novel approach to the creation of a labelling lexicon for improving emotion analysis in text. the electronic library ahead-of-print. https://doi.org/10.1108/EL-04-2020-0110

  53. Zhang J, Liu F, Xu W, Yu H (2019) Feature fusion text classification model combining cnn and bigru with multi-attention mechanism. Future Internet 11:2–37. https://doi.org/10.3390/fi11110237

    Google Scholar 

  54. Srivastava N, Hinton G, Krizhevsky A, Sutskever I, Salakhutdinov R (2014) Dropout: A simple way to prevent neural networks from overfitting. J Mach Learn Res 15(56):1929–1958

    MathSciNet  MATH  Google Scholar 

  55. Kumar M, Dredze M, Coppersmith G, Choudhury M. (2015) Detecting changes in suicide content manifested in social media following celebrity suicides. https://doi.org/10.1145/2700171.2791026

  56. Renjith S, Abraham A, Jyothi SB, Chandran L, Thomson J. (2021) An ensemble deep learning technique for detecting suicidal ideation from posts in social media platforms. J King Saud Univ- Comput Inf Sci. https://doi.org/10.1016/j.jksuci.2021.11.010

  57. Zhang T, Schoene AM, Ananiadou S (2021) Automatic identification of suicide notes with a transformer-based deep learning model. Internet Interv 25:100–422. https://doi.org/10.1016/j.invent.2021.100422

    Google Scholar 

  58. Ghosh S, Ekbal A, Bhattacharyya P (2020) Cease, a corpus of emotion annotated suicide notes in english

  59. Ghosh S, Varshney D, Ekbal A, Bhattacharyya P (2021) Context and knowledge enriched transformer framework for emotion recognition in conversations. In: 2021 International joint conference on neural networks (IJCNN), pp 1–8. https://doi.org/10.1109/IJCNN52387.2021.9533452

  60. Wang N, Luo F, Shivtare Y, Badal VD, Subbalakshmi K, Chandramouli R, Lee E (2021) Learning models for suicide prediction from social media posts. arXiv:2105.03315

  61. Mikolov T, Grave E, Bojanowski P, Puhrsch C (2017) Joulin, A Advances in pre-training distributed word representations

  62. Ishaq A, Asghar S, Gillani SA (2020) Aspect-based sentiment analysis using a hybridized approach based on cnn and ga. IEEE Access 8:135499–135512. https://doi.org/10.1109/ACCESS.2020.3011802

    Google Scholar 

  63. Li X, Cui M, Li J, Bai R, Lu Z., Aickelin U (2021) A hybrid medical text classification framework: Integrating attentive rule construction and neural network. Neurocomputing 443:345–355. https://doi.org/10.1016/j.neucom.2021.02.069

    Google Scholar 

  64. Yousaf A, Umer M, Sadiq S, Ullah S, Mirjalili S, Rupapara V, Nappi M (2021) Emotion recognition by textual tweets classification using voting classifier (lr-sgd). IEEE Access 9:6286–6295. https://doi.org/10.1109/ACCESS.2020.3047831

    Google Scholar 

  65. Dong M, Li Y, Tang X, Xu J, Bi S, Cai Y (2020) Variable convolution and pooling convolutional neural network for text sentiment classification. IEEE Access 8:16174–16186. https://doi.org/10.1109/ACCESS.2020.2966726

    Google Scholar 

  66. Yang G, He H, Chen Q (2019) Emotion-semantic-enhanced neural network. IEEE/ACM Transactions on Audio, Speech, and Language Processing 27(3):531–543. https://doi.org/10.1109/TASLP.2018.2885775

    Google Scholar 

  67. Zhang Y, Xu B, Zhao T (2020) Convolutional multi-head self-attention on memory for aspect sentiment classification. IEEE/CAA Journal of Automatica Sinica 7(4):1038–1044. https://doi.org/10.1109/JAS.2020.1003243

    Google Scholar 

  68. Cheng K, Yue Y, Song Z (2020) Sentiment classification based on part-of-speech and self-attention mechanism. IEEE Access 8:16387–16396. https://doi.org/10.1109/ACCESS.2020.2967103

    Google Scholar 

  69. Xu D, Tian Z, Lai R, Kong X, Tan Z, Shi W (2020) Deep learning based emotion analysis of microblog texts. Inf Fusion 64:1–11

    Google Scholar 

  70. Ahmad S, Asghar MZ, Alotaibi FM, Khan S (2020) Classification of poetry text into the emotional states using deep learning technique. IEEE Access 8:73865–73878. https://doi.org/10.1109/ACCESS.2020.2987842

    Google Scholar 

  71. Xiao X, Wei P, Mao W, Wang L (2019) Context-aware multi-view attention networks for emotion cause extraction. In: 2019 IEEE International conference on intelligence and security informatics (ISI), pp 128–133. https://doi.org/10.1109/ISI.2019.8823225

  72. Hameed Z, Garcia-Zapirain B (2020) Sentiment classification using a single-layered bilstm model. IEEE Access 8:73992–74001. https://doi.org/10.1109/ACCESS.2020.2988550

    Google Scholar 

  73. Kumar P, Raman B (2022) A bert based dual-channel explainable text emotion recognition system. Neural Netw 150:392–407. https://doi.org/10.1016/j.neunet.2022.03.017

    Google Scholar 

  74. Xu G, Zhang Z, Zhang T, Yu S, Meng Y, Chen S (2022) Aspect-level sentiment classification based on attention-bilstm model and transfer learning. Knowl Based Syst 245:108–586. https://doi.org/10.1016/j.knosys.2022.108586

    Google Scholar 

  75. Uban A-S, Chulvi B, Rosso P (2021) An emotion and cognitive based analysis of mental health disorders from social media data. Future Gener Comput Syst 124:480–494. https://doi.org/10.1016/j.future.2021.05.032

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science and Engineering, National Institute of Technology, Warangal, India

    Dheeraj Kodati & Ramakrishnudu Tene

Authors
  1. Dheeraj Kodati
    View author publications

    Search author on:PubMed Google Scholar

  2. Ramakrishnudu Tene
    View author publications

    Search author on:PubMed Google Scholar

Corresponding author

Correspondence to Dheeraj Kodati.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kodati, D., Tene, R. Identifying suicidal emotions on social media through transformer-based deep learning. Appl Intell 53, 11885–11917 (2023). https://doi.org/10.1007/s10489-022-04060-8

Download citation

  • Accepted:

  • Published:

  • Issue date:

  • DOI: https://doi.org/10.1007/s10489-022-04060-8

Keywords

Profiles

  1. Ramakrishnudu Tene View author profile