Skip to main content

Early Risk Detection of Self-Harm Using BERT-Based Transformers

  • Chapter
  • First Online:
Early Detection of Mental Health Disorders by Social Media Monitoring

Part of the book series: Studies in Computational Intelligence ((SCI,volume 1018))

  • 713 Accesses

Abstract

In the past few years, individuals choose increasingly to share online about themselves, which reflect greatly about their personality and self-image. In particular, individuals with mental health issues tend to discuss or mention such issues online, in an obvious or concealed way. Thus, analysing individuals’ online posts for mental health, became critical for an early detection and fast intervention. This chapter presents our efforts towards detecting mental health issues by focusing on the task: Early Detection of Signs of Self-Harm. Core to how we approached this problem was the use of BERT-based classifiers which were trained specifically for each task. Additional investigations of the efficiency of this method in detecting mental health issues are also presented with the following tasks: Early Detection of Signs of Anorexia and Early Detection of Signs of Depression. Our results indicate that this approach delivers high performance across a series of measures, particularly for self-harm and anorexia, where our classifiers obtained the best performance for precision, F1, latency-weighted F1, and ERDE at 5 and 50. This work suggests that BERT-based classifiers, when trained appropriately, can accurately infer which social media users are at risk of self-harming, with precision up to 91.3 and 83.9% for anorexia. Given these promising results, it will be interesting to further refine the training regime, classifier and early detection scoring mechanism, as well as apply the same approach to other related tasks (e.g., pathological gambling, suicide).

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 129.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 169.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 169.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    An emotional disorder characterised by an obsessive desire to lose weight by refusing to eat.

  2. 2.

    https://early.irlab.org/.

  3. 3.

    http://www.clef-initiative.eu/.

  4. 4.

    Github: https://github.com/labteral/ernie/.

  5. 5.

    Github: https://github.com/huggingface/transformers/.

  6. 6.

    https://reddit.com/.

  7. 7.

    https://pushshift.io/api-parameters/.

References

  1. Baumgartner, J., Zannettou, S., Keegan, B., Squire, M., & Blackburn, J. (2020). The pushshift reddit dataset. In Proceedings of the International AAAI Conference on Web and Social Media, 14, 830–839.

    Google Scholar 

  2. Burdisso, S. G., Errecalde, M., & Montes-y Gómez, M. (2019). A text classification framework for simple and effective early depression detection over social media streams. Expert Systems with Applications, 133, 182–197.

    Article  Google Scholar 

  3. Burdisso, S. G., Errecalde, M., & Montes-y Gómez, M. (2019). Unsl at erisk 2019: a unified approach for anorexia, self-harm and depression detection in social media. In CLEF (Working Notes).

    Google Scholar 

  4. Chancellor, S., & De Choudhury, M. (2020). Methods in predictive techniques for mental health status on social media: A critical review. NPJ Digital Medicine, 3(1), 1–11.

    Article  Google Scholar 

  5. Cohan, A., Young, S., & Goharian, N. (2016). Triaging mental health forum posts. In Proceedings of the Third Workshop on Computational Linguistics and Clinical Psychology (pp. 143–147).

    Google Scholar 

  6. Conneau, A., Khandelwal, K., Goyal, N., Chaudhary, V., Wenzek, G., Guzmán, F., Grave, É., Ott, M., Zettlemoyer, L., & Stoyanov, V. (2020). Unsupervised cross-lingual representation learning at scale. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (pp. 8440–8451).

    Google Scholar 

  7. Devlin, J., & Chang, M.-W. (2018). Open sourcing bert: State-of-the-art pre-training for natural language processing. Retrieved April 22, 2021, from http://aiweb.techfak.uni-bielefeld.de/content/bworld-robot-control-software/.

  8. Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018). Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv:1810.04805.

  9. Gao, Z., Feng, A., Song, X., & Wu, X. (2019). Target-dependent sentiment classification with bert. IEEE Access, 7, 154290–154299.

    Article  Google Scholar 

  10. Lample, G., & Conneau, A. (2019). Cross-lingual language model pretraining. arXiv:1901.07291.

  11. Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., & Stoyanov, V. (2019). Roberta: A robustly optimized bert pretraining approach. arXiv:1907.11692.

  12. Losada, D. E., & Crestani, F. (2016). A test collection for research on depression and language use. In International Conference of the Cross-Language Evaluation Forum for European Languages (pp. 28–39). Springer.

    Google Scholar 

  13. Losada, D. E., Crestani, F., & Parapar, J. (2018). Overview of eRisk 2018: Early risk prediction on the internet (extended lab overview). CEUR Workshop Proceedings, 2125.

    Google Scholar 

  14. Losada, D. E., Crestani, F., & Parapar, J. (2019). Overview of eRisk 2019 early risk prediction on the internet. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) 11696 LNCS, September 2019 (pp. 340–357).

    Google Scholar 

  15. Losada, D. E., Crestani, F., & Parapar, J. (2020). Overview of eRisk 2020: early risk prediction on the internet. In Experimental IR Meets Multilinguality, Multimodality, and Interaction Proceedings of the Eleventh International Conference of the CLEF Association (CLEF 2020).

    Google Scholar 

  16. Martínez-Castaño, R., Htait, A., Azzopardi, L., & Moshfeghi, Y. (2020). Early risk detection of self-harm and depression severity using bert-based transformers: Ilab at clef erisk 2020. Early Risk Prediction on the Internet.

    Google Scholar 

  17. Nikolov, A., & Radivchev, V. (2019). Nikolov-radivchev at semeval-2019 task 6: Offensive tweet classification with bert and ensembles. In Proceedings of the 13th International Workshop on Semantic Evaluation (pp. 691–695).

    Google Scholar 

  18. Obeid, J. S., Dahne, J., Christensen, S., Howard, S., Crawford, T., Frey, L. J., Stecker, T., & Bunnell, B. E. (2020). Identifying and predicting intentional self-harm in electronic health record clinical notes: Deep learning approach. JMIR Medical Informatics, 8(7), e17784.

    Google Scholar 

  19. Parikh, P., Abburi, H., Badjatiya, P., Krishnan, R., Chhaya, N., Gupta, M., & Varma, V. Multi-label categorization of accounts of sexism using a neural framework. In EMNLP/IJCNLP, (1) (2019).

    Google Scholar 

  20. Pater, J., & Mynatt, E. (2017). Defining digital self-harm. In Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing (pp. 1501–1513).

    Google Scholar 

  21. Peters, M., Neumann, M., Iyyer, M., Gardner, M., Clark, C., Lee, K., & Zettlemoyer, L. (2018). Deep contextualized word representations. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) (pp. 2227–2237).

    Google Scholar 

  22. Radford, A., Narasimhan, K., Salimans, T., & Sutskever, I. (2018). Improving language understanding by generative pre-training. Retrieved April 22, 2021, from https://s3-us-west-2.amazonaws.com/openai-assets/research-covers/language-unsupervised/language_understanding_paper.pdf.

  23. Satcher, D. (2000). Mental health: A report of the surgeon general-executive summary. Professional Psychology: Research and Practice, 31(1), 5.

    Article  Google Scholar 

  24. Skaik, R., & Inkpen, D. (2020). Using social media for mental health surveillance: A review. ACM Computing Surveys (CSUR), 53(6), 1–31.

    Article  Google Scholar 

  25. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, Ł., & Polosukhin, I. (2017). Attention is all you need. In Advances in Neural Information Processing Systems (pp. 5998–6008).

    Google Scholar 

  26. Yates, A., Cohan, A., & Goharian, N. (2017). Depression and self-harm risk assessment in online forums. arxiv:1709.01848.

Download references

Acknowledgements

The first author would like to thank the following funding bodies for their support: FEDER/Ministerio de Ciencia, Innovación y Universidades, Agencia Estatal de Investigación/Project (RTI2018-093336-B-C21), Consellería de Educación, Universidade e Formación Profesional and the European Regional Development Fund (ERDF) (accreditation 2019-2022 ED431G-2019/04, ED431C 2018/29, ED431C 2018/19).

The second and third authors would like to thank the UKRI’s EPSRC Project Cumulative Revelations in Personal Data (Grant Number: EP/R033897/1) for their support. We would also like to thank David Losada for arranging this collaboration.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Rodrigo Martínez-Castaño .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Martínez-Castaño, R., Htait, A., Azzopardi, L., Moshfeghi, Y. (2022). Early Risk Detection of Self-Harm Using BERT-Based Transformers. In: Crestani, F., Losada, D.E., Parapar, J. (eds) Early Detection of Mental Health Disorders by Social Media Monitoring. Studies in Computational Intelligence, vol 1018. Springer, Cham. https://doi.org/10.1007/978-3-031-04431-1_8

Download citation

Publish with us

Policies and ethics