Skip to main content

Advertisement

SpringerLink
Log in

Screening for Psychological Distress in Healthcare Workers Using Machine Learning: A Proof of Concept

  • Original Paper
  • Published:
Journal of Medical Systems Aims and scope Submit manuscript

Abstract

The purpose of this study was to train and test preliminary models using two machine learning algorithms to identify healthcare workers at risk of developing anxiety, depression, and post-traumatic stress disorder. The study included data from a prospective cohort study of 816 healthcare workers collected using a mobile application during the first two waves of COVID-19. Each week, the participants responded to 11 questions and completed three screening questionnaires (one for anxiety, one for depression, and one for post-traumatic stress disorder). Then, the research team selected two questions (out of the 11), which were used with biological sex to identify whether scores on each screening questionnaire would be positive or negative. The analyses involved a fivefold cross-validation to test the accuracy of models based on logistic regression and support vector machines using cross-sectional and cumulative measures. The findings indicated that the models derived from the two questions and biological sex accurately identified screening scores for anxiety, depression, and post-traumatic stress disorders in 70% to 80% of cases. However, the positive predictive value never exceeded 50%, underlining the importance of collecting more data to train better models. Our proof of concept demonstrates the feasibility of using machine learning to develop novel models to screen for psychological distress in at-risk healthcare workers. Developing models with fewer questions may reduce burdens of active monitoring in practical settings by decreasing the weekly assessment duration.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

Data Availability

Raw data and code are available freely in an online repository, the URL is provided in the methods section of the manuscript.

References

  1. S. Johnson, C. Cooper, S. Cartwright, I. Donald, P. Taylor, and C. Millet, “The experience of work‐related stress across occupations,” J. Manag. Psychol., vol. 20, no. 2, pp. 178–187, Jan. 2005, https://doi.org/10.1108/02683940510579803.

    Article  Google Scholar 

  2. K. Santarone, M. McKenney, and A. Elkbuli, “Preserving mental health and resilience in frontline healthcare workers during COVID-19,” Am. J. Emerg. Med., vol. 38, no. 7, pp. 1530–1531, Jul. 2020, https://doi.org/10.1016/j.ajem.2020.04.030.

    Article  PubMed  PubMed Central  Google Scholar 

  3. P. Trivellas, P. Reklitis, and C. Platis, “The Effect of Job Related Stress on Employees’ Satisfaction: A Survey in Health Care,” Proc. 2nd Int. Conf. Integr. Inf. IC-ININFO 2012 Bp. Hung. August 30 – Sept. 3 2012, vol. 73, pp. 718–726, Feb. 2013, https://doi.org/10.1016/j.sbspro.2013.02.110.

  4. Ε. Moustaka and T. C. Constantinidis, “Sources and effects of Work-related stress in nursing,” Health Sci. J., vol. 4, no. 4, p. 7, 2010.

    Google Scholar 

  5. J. Lai et al., “Factors Associated With Mental Health Outcomes Among Health Care Workers Exposed to Coronavirus Disease 2019,” JAMA Netw. Open, vol. 3, no. 3, pp. e203976–e203976, Mar. 2020, https://doi.org/10.1001/jamanetworkopen.2020.3976.

    Article  PubMed  PubMed Central  Google Scholar 

  6. A. Lasalvia et al., “The Sustained Psychological Impact of the COVID-19 Pandemic on Health Care Workers One Year after the Outbreak-A Repeated Cross-Sectional Survey in a Tertiary Hospital of North-East Italy,” Int. J. Environ. Res. Public. Health, vol. 18, no. 24, 2021, https://doi.org/10.3390/ijerph182413374.

  7. F. C. Troglio da Silva and M. L. R. Neto, “Psychiatric disorders in health professionals during the COVID-19 pandemic: A systematic review with meta-analysis,” J. Psychiatr. Res., vol. 140, pp. 474–487, Aug. 2021, https://doi.org/10.1016/j.jpsychires.2021.03.044.

    Article  PubMed  PubMed Central  Google Scholar 

  8. L. N. Dyrbye et al., “Burnout Among Health Care Professionals: A Call to Explore and Address This Underrecognized Threat to Safe, High-Quality Care,” NAM Perspect., Jul. 2017, https://doi.org/10.31478/201707b.

    Article  Google Scholar 

  9. A. Zaka, S. E. Shamloo, P. Fiorente, and A. Tafuri, “COVID-19 pandemic as a watershed moment: A call for systematic psychological health care for frontline medical staff,” J. Health Psychol., vol. 25, no. 7, pp. 883–887, Jun. 2020, https://doi.org/10.1177/1359105320925148.

    Article  PubMed  Google Scholar 

  10. H. Blake, F. Bermingham, G. Johnson, and A. Tabner, “Mitigating the Psychological Impact of COVID-19 on Healthcare Workers: A Digital Learning Package,” Int. J. Environ. Res. Public. Health, vol. 17, no. 9, Art. no. 9, Jan. 2020, https://doi.org/10.3390/ijerph17092997.

  11. W. El-Hage et al., “Les professionnels de santé face à la pandémie de la maladie à coronavirus (COVID-19) : quels risques pour leur santé mentale ?,” L’Encéphale, vol. 46, no. 3, Supplement, pp. S73–S80, Jun. 2020, https://doi.org/10.1016/j.encep.2020.04.008.

  12. P. Galanis, I. Vraka, D. Fragkou, A. Bilali, and D. Kaitelidou, “Nurses’ burnout and associated risk factors during the COVID‐19 pandemic: A systematic review and meta‐analysis,” J. Adv. Nurs., vol. 77, no. 8, pp. 3286–3302, Aug. 2021, https://doi.org/10.1111/jan.14839.

    Article  PubMed  PubMed Central  Google Scholar 

  13. A. E. Muller et al., “The mental health impact of the covid-19 pandemic on healthcare workers, and interventions to help them: A rapid systematic review,” Psychiatry Res., vol. 293, p. 113441, 2020, https://doi.org/10.1016/j.psychres.2020.113441.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  14. W. Wilson et al., “Prevalence and Predictors of Stress, anxiety, and Depression among Healthcare Workers Managing COVID-19 Pandemic in India: A Nationwide Observational Study,” Indian J. Psychol. Med., vol. 42, no. 4, pp. 353–358, 2020, https://doi.org/10.1177/0253717620933992.

    Article  PubMed  PubMed Central  Google Scholar 

  15. D. Zaçe, I. Hoxhaj, A. Orfino, A. M. Viteritti, L. Janiri, and M. L. D. Pietro, “Interventions to address mental health issues in healthcare workers during infectious disease outbreaks: A systematic review,” J. Psychiatr. Res., vol. 136, pp. 319–333, 2021, https://doi.org/10.1016/j.jpsychires.2021.02.019.

    Article  PubMed  PubMed Central  Google Scholar 

  16. J. Maben 1 2, 3, ✉ and J. Bridges, “Covid-19: Supporting nurses’ psychological and mental health,” J. Clin. Nurs., vol. 29, no. 15–16, pp. 2742–2750, 2020, https://doi.org/10.1111/jocn.15307.

  17. A. C. McFarlane, P. Williamson, and C. A. Barton, “The impact of traumatic stressors in civilian occupational settings,” J. Public Health Policy, vol. 30, no. 3, pp. 311–327, Sep. 2009, https://doi.org/10.1057/jphp.2009.21.

    Article  PubMed  Google Scholar 

  18. M. Luo, L. Guo, M. Yu, W. Jiang, and H. Wang, “The psychological and mental impact of coronavirus disease 2019 (COVID-19) on medical staff and general public – A systematic review and meta-analysis,” Psychiatry Res., vol. 291, p. 113190, Sep. 2020, https://doi.org/10.1016/j.psychres.2020.113190.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  19. T. Halms, M. Strasser, M. Kunz, and A. Hasan, “How to Reduce Mental Health Burden in Health Care Workers During COVID-19?–A Scoping Review of Guideline Recommendations,” Front. Psychiatry, vol. 12, 2022, Accessed: Jun. 27, 2022. [Online]. Available: https://www.frontiersin.org/article/10.3389/fpsyt.2021.770193

  20. S. Rose, J. Hartnett, and S. Pillai, “Healthcare worker’s emotions, perceived stressors and coping mechanisms during the COVID-19 pandemic,” PLOS ONE, vol. 16, no. 7, p. e0254252, Jul. 2021, https://doi.org/10.1371/journal.pone.0254252.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  21. National Institute for Health and Care Excellence, “Post-traumatic stress disorder,” p. 54, Dec. 2018, [Online]. Available: https://www.nice.org.uk/guidance/ng116/resources/posttraumatic-stress-disorder-pdf-66141601777861

  22. A. Ehlers et al., “A Randomized Controlled Trial of Cognitive Therapy, a Self-help Booklet, and Repeated Assessments as Early Interventions for Posttraumatic Stress Disorder,” Arch. Gen. Psychiatry, vol. 60, no. 10, pp. 1024–1032, Oct. 2003, https://doi.org/10.1001/archpsyc.60.10.1024.

    Article  PubMed  Google Scholar 

  23. D. S. Krantz and A. S. Baum, Technology and Methods in Behavioral Medicine. Psychology Press, 1998.

  24. S. Shiffman and M. Hufford, “Ecological momentary assessment,” Appl. Clin. Trials, vol. 10, Jan. 2008.

  25. J. J. McArdle and G. Ritschard, Contemporary issues in exploratory data mining in the behavioral sciences. in Contemporary issues in exploratory data mining in the behavioral sciences. New York, NY, US: Routledge/Taylor & Francis Group, 2014, pp. xxii, 473.

  26. J. Chung and J. Teo, “Mental Health Prediction Using Machine Learning: Taxonomy, Applications, and Challenges,” Appl. Comput. Intell. Soft Comput., vol. 2022, p. e9970363, Jan. 2022, https://doi.org/10.1155/2022/9970363.

    Article  Google Scholar 

  27. K. Le Nguyen, H. Thi Trinh, T. T. Nguyen, and H. D. Nguyen, “Comparative study on the performance of different machine learning techniques to predict the shear strength of RC deep beams: Model selection and industry implications,” Expert Syst. Appl., vol. 230, p. 120649, Nov. 2023, https://doi.org/10.1016/j.eswa.2023.120649.

  28. D. Bzdok and A. Meyer-Lindenberg, “Machine Learning for Precision Psychiatry: Opportunities and Challenges,” Biol. Psychiatry Cogn. Neurosci. Neuroimaging, vol. 3, no. 3, pp. 223–230, Mar. 2018, https://doi.org/10.1016/j.bpsc.2017.11.007.

    Article  PubMed  Google Scholar 

  29. T. Ching et al., “Opportunities and obstacles for deep learning in biology and medicine,” J. R. Soc. Interface, vol. 15, no. 141, p. 20170387, Apr. 2018, https://doi.org/10.1098/rsif.2017.0387.

    Article  PubMed  PubMed Central  Google Scholar 

  30. M.-M. Dufour, N. Bergeron, A. Rabasa, S. Guay, and S. Geoffrion, “Assessment of Psychological Distress in Health-care Workers during and after the First Wave of COVID-19: A Canadian Longitudinal Study: Évaluation de la Détresse Psychologique Chez Les Travailleurs de la Santé Durant et Après la Première Vague de la COVID-19: une étude longitudinale canadienne,” Can. J. Psychiatry, vol. 66, no. 9, pp. 807–814, Sep. 2021, https://doi.org/10.1177/07067437211025217.

    Article  PubMed  PubMed Central  Google Scholar 

  31. E. von Elm et al., “The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: guidelines for reporting observational studies,” J. Clin. Epidemiol., vol. 61, no. 4, pp. 344–349, Apr. 2008, https://doi.org/10.1016/j.jclinepi.2007.11.008.

    Article  Google Scholar 

  32. R. L. Spitzer, K. Kroenke, J. B. W. Williams, and B. Löwe, “A Brief Measure for Assessing Generalized Anxiety Disorder: The GAD-7,” Arch. Intern. Med., vol. 166, no. 10, pp. 1092–1097, May 2006, https://doi.org/10.1001/archinte.166.10.1092.

    Article  PubMed  Google Scholar 

  33. K. Kroenke, R. L. Spitzer, and J. B. Williams, “The PHQ-9: validity of a brief depression severity measure,” J. Gen. Intern. Med., vol. 16, no. 9, pp. 606–613, Sep. 2001, https://doi.org/10.1046/j.1525-1497.2001.016009606.x.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  34. M. Price, D. D. Szafranski, K. van Stolk-Cooke, and D. F. Gros, “Investigation of abbreviated 4 and 8 item versions of the PTSD Checklist 5,” Psychiatry Res., vol. 239, pp. 124–130, May 2016, https://doi.org/10.1016/j.psychres.2016.03.014.

    Article  PubMed  Google Scholar 

  35. C. Beard, K. J. Hsu, L. S. Rifkin, A. B. Busch, and T. Björgvinsson, “Validation of the PHQ-9 in a psychiatric sample,” J. Affect. Disord., vol. 193, pp. 267–273, Mar. 2016, https://doi.org/10.1016/j.jad.2015.12.075.

    Article  CAS  PubMed  Google Scholar 

  36. A. S. Moriarty, S. Gilbody, D. McMillan, and L. Manea, “Screening and case finding for major depressive disorder using the Patient Health Questionnaire (PHQ-9): a meta-analysis,” Gen. Hosp. Psychiatry, vol. 37, no. 6, pp. 567–576, Nov. 2015, https://doi.org/10.1016/j.genhosppsych.2015.06.012.

    Article  PubMed  Google Scholar 

  37. N. H. Kalin, “The Critical Relationship Between Anxiety and Depression,” Am. J. Psychiatry, vol. 177, no. 5, pp. 365–367, May 2020, https://doi.org/10.1176/appi.ajp.2020.20030305.

    Article  PubMed  Google Scholar 

  38. B. N. Rusli, B. A. Edimansyah, and L. Naing, “Working conditions, self-perceived stress, anxiety, depression and quality of life: A structural equation modelling approach,” BMC Public Health, vol. 8, p. 48, Feb. 2008, https://doi.org/10.1186/1471-2458-8-48.

    Article  PubMed  PubMed Central  Google Scholar 

  39. Mardi J. Horowitz, Stress Response Syndromes: PTSD, Grief, Adjustment, and Dissociative Disorders, 5th Edition. Accessed: Oct. 03, 2022. [Online]. Available: https://universitypressofamerica.com/ISBN/9780765710079/Stress-Response-Syndromes-PTSD-Grief-Adjustment-and-Dissociative-Disorders-5th-Edition

  40. S. M. Horowitz, R. T. Blackburn, D. W. Edington, and D. M. Kloss, “Association between Job Stress and Perceived Quality of Life,” J. Am. Coll. Health, vol. 37, no. 1, pp. 29–35, Jul. 1988, https://doi.org/10.1080/07448481.1988.9939038.

    Article  CAS  PubMed  Google Scholar 

  41. E. Kabir-Mokamelkhah, A. Bahrami-Ahmadi, and N. Aghili, “Work-related stress and quality of life among Iranian blue-collar workers with self-reported low back pain,” Med. J. Islam. Repub. Iran, vol. 30, p. 474, Dec. 2016, Accessed: Oct. 03, 2022. [Online]. Available: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5419238/

  42. N. Breslau, R. C. Kessler, H. D. Chilcoat, L. R. Schultz, G. C. Davis, and P. Andreski, “Trauma and posttraumatic stress disorder in the community: the 1996 Detroit Area Survey of Trauma,” Arch. Gen. Psychiatry, vol. 55, no. 7, pp. 626–632, Jul. 1998, https://doi.org/10.1001/archpsyc.55.7.626.

    Article  CAS  PubMed  Google Scholar 

  43. R. A. Bryant, A. C. McFarlane, D. Silove, M. L. O’Donnell, D. Forbes, and M. Creamer, “The Lingering Impact of Resolved PTSD on Subsequent Functioning,” Clin. Psychol. Sci., vol. 4, no. 3, pp. 493–498, May 2016, https://doi.org/10.1177/2167702615598756.

    Article  Google Scholar 

  44. T. A. Prout, “Identifying Predictors of Psychological Distress During COVID-19: A Machine Learning Approach,” Front. Psychol., vol. 11, p. 14, 2020.

    Article  Google Scholar 

  45. D. A. Adler, F. Wang, D. C. Mohr, and T. Choudhury, “Machine learning for passive mental health symptom prediction: Generalization across different longitudinal mobile sensing studies,” PLOS ONE, p. 20.

  46. K. Aafjes-van Doorn, C. Kamsteeg, J. Bate, and M. Aafjes, “A scoping review of machine learning in psychotherapy research,” Psychother. Res., vol. 31, no. 1, pp. 92–116, Jan. 2021, https://doi.org/10.1080/10503307.2020.1808729.

Download references

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research project was supported by a grant (#2020–2023-COVID19-PSOv2a-51476) from the Fonds de Recherche du Québec—Ministère de l’Économie et Innovation as well as a salary award (#268274) from the Fonds de Recherche du Québec— Institut de Recherche Robert Sauvé en Santé et Sécurité au Travail to the last author.

Author information

Authors and Affiliations

  1. Research center of l’Institut universitaire en santé mentale de Montréal (CR-IUSMM), Montréal, Québec, Canada

    Steve Geoffrion, Catherine Morse, Stéphane Guay & Marc J. Lanovaz

  2. School of Psychoeducation, University of Montreal, Université de Montréal Pavillon Marie-Victorin École de psychoéducation, C. P. 6128, succursale Centre-ville, Montréal, Québec, H3C 3J7, Canada

    Steve Geoffrion, Catherine Morse, Marie-Michèle Dufour & Marc J. Lanovaz

  3. Department of Psychiatry, Université de Montréal, Montréal, Québec, Canada

    Nicolas Bergeron & Stéphane Guay

  4. Research Center of Centre Hospitalier de l’Université de Montréal (CRCHUM), Montréal, Québec, Canada

    Nicolas Bergeron

  5. School of Criminology, Université de Montréal, Montréal, Québec, Canada

    Stéphane Guay

Authors
  1. Steve Geoffrion
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Catherine Morse
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Marie-Michèle Dufour
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Nicolas Bergeron
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Stéphane Guay
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Marc J. Lanovaz
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

S.G. and M.L supervised the project, and collaborated with C.M. to write the main manuscript text. M.-M.B. contributed in terms of data management and analysis. N.B. and S.G. were responsible for conceptualization and received funding for the project. All authors reviewed the manuscript.

Corresponding author

Correspondence to Steve Geoffrion.

Ethics declarations

Ethical Approval

The research ethics board of the CRCHUM approved the research project. The original study followed the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) reporting guidelines [31].

Consent to Participate

Each participant provided informed consent before their participation.

Consent to Publish

Each participant provided informed consent to publish results.

Competing Interests

The Authors declare that there is no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Geoffrion, S., Morse, C., Dufour, MM. et al. Screening for Psychological Distress in Healthcare Workers Using Machine Learning: A Proof of Concept. J Med Syst 47, 120 (2023). https://doi.org/10.1007/s10916-023-02011-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10916-023-02011-5

Keywords

Search

Navigation