Abstract
The COVID-19 pandemic is causing monumental effects on mental wellbeing worldwide. Literature calls for action to avert an impending global mental health crisis. This chapter presents the challenge, and presents a new approach to avert the crisis in mental health. The objective is to design a solution by applying advances in Deep Learning literature. The advances in Transformer neural network enable pandemic scale screening, mental health diagnostics and counseling. A synthesis of Multimodal Deep Learning architecture to analyze thinking patterns is presented. The use of BERT for diagnosis is well established in the literature. By approaching the modeling of Cognitive Triad and Cognitive Behaviour Therapy with an Encoder-Decoder Transformer architecture, this chapter initiates future research by interested communities to avert the global mental health crisis.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Galea, S., Merchant, R.M., Lurie, N.: The mental health consequences of COVID-19 and physical distancing: the need for prevention and early intervention. JAMA Intern. Med. (2020). https://doi.org/10.1001/jamainternmed.2020.1562
UNSDG Policy Brief: COVID-19 and the need for action on mental health (2020).https://unsdg.un.org/resources/policy-brief-covid-19-and-need-action-mental-health. Accessed 28 2020
WHO Mental Health Facts. https://www.who.int/news-room/facts-in-pictures/detail/mental-health
Holmes, E.A., et al.: Multidisciplinary research priorities for the COVID-19 pandemic: a call for action for mental health science. Lancet Psychiatry (2020). https://doi.org/10.1016/s2215-0366(20)30168-1
[NHS website]: Mental Health of Children and Young People in England 2021, 30 Sep 2021, NHS, https://digital.nhs.uk/data-and-information/publications/statistical/mental-health-of-children-and-young-people-in-england/2021-follow-up-to-the-2017-survey
Keeley, B.: The state of the world’s children 2021: on my mind-promoting. Protecting and Caring for Children’s Mental Health, UNICEF (2021)
WHO team: The impact of COVID-19 on mental, neurological and substance use services, WHO Publication, ISBN: 978-92-4-001245-5 (2020). https://www.who.int/publications/i/item/978924012455
[Editorial]: The intersection of COVID-19 and mental health. The Lancet Infectious Diseases (2020). https://doi.org/10.1016/S1473-3099(20)30797-0
[Editorial]: Keep mental health in mind. Nat. Med. 26, 631 (2020). https://doi.org/10.1038/s41591-020-0914-4
[Editorial]: Mental health matters. Lancet Glob. Health 8(11) (2020). https://doi.org/10.1016/S2214-109X(20)30432-0
McGorry, P.D., Ratheesh, A., O’Donoghue, B.: Early intervention–an implementation challenge for 21st century mental health care. JAMA Psychiatry 75(6), 545–546 (2018). https://doi.org/10.1001/jamapsychiatry.2018.0621
Ford, T., John, A., Gunnell, D.: Mental health of children and young people during pandemic. BMJ 372 (2021). https://doi.org/10.1136/bmj.n614
Durstewitz, D., Koppe, G., Meyer-Lindenberg, A.: Deep neural networks in psychiatry. Nat. Mol. Psychiatry 24, 1583–1598 (2019). https://www.nature.com/articles/s41380-019-0365-9, https://doi.org/10.1038/s41380-019-0365-9
Denecke, K., Vaaheesan, S., Arulnathan, A.: A Mental Health Chatbot for Regulating Emotions (SERMO)—concept and usability test. IEEE Trans. Emerg. Top. Comput. 1 (2020). https://doi.org/10.1109/tetc.2020.2974478
Doan, S. et al.: Extracting health-related causality from twitter messages using natural language processing. BMC Med. Inform. Decis. Mak. 19 (2019). https://doi.org/10.1186/s12911-019-0785-0
Gkotsis, G., Oellrich, A., Velupillai, S. et al.: Characterisation of mental health conditions in social media using Informed Deep Learning. (Nature) Sci. Rep. 7, 45141 (2017). https://doi.org/10.1038/srep45141
Chancellor, S., De Choudhury, M.: Methods in predictive techniques for mental health status on social media: a critical review. NPJ Digit. Med. 3 (2020). https://www.nature.com/articles/s41746-020-0233-7, https://doi.org/10.1038/s41746-020-0233-7
Ranti, D. et al.: The Utility of General Domain Transfer Learning for Medical Language Tasks (2020). arXiv:2002.06670
Dobson, K.S.: Handbook of Cognitive-Behavioral Therapies. Guilford Press (2019)
Oh, K., Lee, D., Ko, B., Choi, H.: A chatbot for psychiatric counseling in mental healthcare service based on emotional dialogue analysis and sentence generation. In: 18th IEEE International Conference on Mobile Data Management (MDM), Daejeon, pp. 371–375 (2017). https://doi.org/10.1109/MDM.2017.64
Yang, T. et al.: Applied Federated Learning: Improving Google Keyboard Query Suggestions (2018). arXiv:1812.02903
Li, X. et al.: Multi-site fMRI Analysis Using Privacy-preserving Federated Learning and Domain Adaptation: ABIDE Results (2020). arXiv:2001.05647
Devlin, J., Chang, M.-W., Lee, K., Toutanova, K.: BERT: Pre-Training of Deep Bidirectional Transformers for Language Understanding (2018). arXiv:1810.04805
Sanches, P. et al.: HCI and affective health: taking stock of a decade of studies and charting future research directions. In: Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (CHI ‘19). ACM (2019). https://doi.org/10.1145/3290605.3300475
Wolf, T. et al.: HuggingFace’s Transformers: State-of-the-art Natural Language Processing (2020). arXiv:1910.03771
Keser, E., Kahya, Y., Akn, B.: Stress generation hypothesis of depressive symptoms in interpersonal stressful life events: the roles of cognitive triad and coping styles via structural equation modeling. Curr. Psychol. (2017). https://doi.org/10.1007/s12144-017-9744-z
Ungar, M., Theron, L., Murphy, K., Jefferies, P.: Researching multisystemic resilience: a sample methodology. Front. Psychol. 11, 3808 (2021). https://doi.org/10.3389/fpsyg.2020.607994
Manji, H., Moore, G., Rajkowska, G. et al.: Neuroplasticity and cellular resilience in mood disorders. Nat. Mol. Psychiatry 5, 578–593 (2000). https://doi.org/10.1038/sj.mp.4000811
Holt-Quick, C. et al.: A chatbot architecture for promoting youth resilience (2020). arXiv:2005.07355
Dobson, K.S., Shaw, B.F.: The effects of self-correction on cognitive distortions in depression. Cogn. Ther. Res. 5, 391–403 (1981)
Ehlers, A., et al.: A randomized controlled trial of cognitive therapy, a self-help booklet, and repeated assessments as early interventions for posttraumatic stress disorder. Arch. Gen. Psychiatry 60, 1024 (2003)
Beck, J.S.: Cognitive Behavior Therapy Basics and Beyond. Guilford Press (1995)
Vaswani, A. et al.: Attention is all you need. In: Advances in Neural Information Processing Systems (NIPS), pp. 5998-6008 (2017)
Li, X., Li, P., Bi, W., Liu, X., Lam, W.: Relevance-Promoting Language Model for Short-Text Conversation (2019). arXiv:1911.11489
Vig, J.: A multiscale visualization of attention in the transformer model (2019). arXiv:1906.05714
Fedus, W., Zoph, B., Shazeer, N.: Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity (2021). arXiv:2101.03961
Radford, A. et al.: Language models are unsupervised multitask learners. OpenAI Blog 1(8), 9 (2019)
Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A. et al.: Language models are few-shot learners 4 (2020). arXiv:2005.14165
Liao, Y., Wang, Y., Liu, Q., Jiang, X.: GPT-Based Generation for Classical Chinese Poetry (2019). arXiv:1907.00151
Zellers, R. et al.: Defending Against Neural Fake News (2019). arXiv:1905.12616
Peng, X., Li, S., Frazier, S., Riedl, M.: Fine-Tuning a Transformer-Based Language Model to Avoid Generating Non-Normative Text (2020). arXiv:2001.08764
Sharma, A., Lin, I. W., Miner, A.S., Atkins, D.C., Althoff, T.: Towards facilitating empathic conversations in online mental health support: a reinforcement learning approach. In: Proceedings of the Web Conference 2021, pp. 194–205 (2021). https://doi.org/10.1145/3442381.3450097
Lim, W.Y.B. et al.: Federated learning in mobile edge networks: a comprehensive survey. IEEE Commun. Surv. Tutor. 1 (2020). https://doi.org/10.1109/comst.2020.2986024
Hendricks, L.A. et al.: Decoupling the role of data, attention, and losses in multimodal transformers (2021). arXiv:2102.00529
Yang, L., Jiang, D., Han, W., Sahli, H.: DCNN and DNN based multi-modal depression recognition. In: Proceedings of 2017 7th International Conference on Affective Computing and Intelligent Interaction, pp. 484–489 (2017). https://doi.org/10.1109/ACII.2017.8273643
Gratzer, D., Goldbloom, D.: Therapy and E-therapy—preparing future psychiatrists in the era of apps and chatbots. Acad. Psychiatry 44, 231–234 (2020). https://doi.org/10.1007/s40596-019-01170-3
Zhu, Y., Shang, Y., Shao, Z., Guo, G.: Automated depression diagnosis based on deep networks to encode facial appearance and dynamics. IEEE Trans. Affect. Comput. 9, 578–584 (2018). https://doi.org/10.1109/TAFFC.2017.2650899
Su, C., Xu, Z., Pathak, J. et al.: Deep learning in mental health outcome research: a scoping review. Nat. Transl. Psychiatry 10, 116 (2020). https://doi.org/10.1038/s41398-020-0780-3
DeSouza, D.D., Robin, J., Gumus, M., Yeung, A.: Natural language processing as an emerging tool to detect late-life depression. Front. Psychiatry 1525 (2021). https://doi.org/10.3389/fpsyt.2021.719125
Eisenschlos, J. et al.: MultiFiT: Efficient Multi-lingual Language Model Fine-Tuning (2019). arXiv:1909.04761
Lu, J.et al.: Vilbert: Pretraining task-agnostic visiolinguistic representations for vision-and-language tasks (2019). arXiv:1908.02265
Maiya, A.S.: ktrain: A Low-Code Library for Augmented Machine Learning (2020). arXiv:2004.10703
Clark, K., Luong, M.-T., Khandelwal, U., Manning, C.D., Le, Q.V.: BAM! Born-Again Multi-Task Networks for Natural Language Understanding (2019). arXiv:1907.04829
Sharma, A., Miner, A.S., Atkins, D.C., Althoff, T.: A Computational Approach to Understanding Empathy Expressed in Text-Based Mental Health Support (2020). arXiv:2009.08441
Xia, Q. et al.: XGPT: Cross-Modal Generative Pre-Training for Image Captioning (2020). arXiv:2003.01473
Floridi, L., Chiriatti, M.: GPT-3: Its nature, scope, limits, and consequences. Minds Mach 30(4) (2020). https://doi.org/10.1007/s11023-020-09548-1
Korngiebel, D.M., Mooney, S.D.: Considering the possibilities and pitfalls of generative pre-trained transformer 3 (GPT-3) in healthcare delivery. NPJ Digit. Med. (2021). https://doi.org/10.1038/s41746-021-00464-x
Melas-Kyriazi, L., Han, G., Liang, C.: Generation-Distillation for Efficient Natural Language Understanding in Low-Data Settings (2020). arXiv:2002.00733
Dalglish, S.L., Costello, A., Clark, H., Coll-Seck, A.: Children in all policies 2030: a new initiative to implement the recommendations of the WHO-UNICEF-lancet commission. Lancet 397(10285), 1605–1607 (2021)
[Editorial]: Mental health of children and young people during pandemic. BMJ (2021). https://doi.org/10.1136/bmj.n614
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this chapter
Cite this chapter
Rajagopal, A., Nirmala, V., Andrew, J., Arun, M.V., Piush, A. (2023). Prevention of Global Mental Health Crisis with Transformer Neural Networks. In: Biswas, A., Semwal, V.B., Singh, D. (eds) Artificial Intelligence for Societal Issues. Intelligent Systems Reference Library, vol 231. Springer, Cham. https://doi.org/10.1007/978-3-031-12419-8_11
Download citation
DOI: https://doi.org/10.1007/978-3-031-12419-8_11
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-12418-1
Online ISBN: 978-3-031-12419-8
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)