Skip to main content

The Role of AI in Mental Health Applications and Liability

  • Chapter
  • First Online:
YSEC Yearbook of Socio-Economic Constitutions 2023

Part of the book series: YSEC Yearbook of Socio-Economic Constitutions ((YSEC,volume 2023))

Abstract

The COVID-19 pandemic has affected the entire area of health care, including care provided to patients with mental health problems. Due to the stressful nature of the pandemic, the number of patients experiencing mental health problems, especially depression or anxiety, has increased. Even well-before the pandemic, Europe struggled with a lack of mental health care, which was especially caused by the long waiting times. The problem seems to have been solved by the plethora of mental health applications that are freely available on the market. Given the user’s accessibility to these applications, I decided to scrutinise the safety of using AI in these health apps, with a particular focus on chatbots. I examined whether existing European legislation may protect users from possible harm to their health and require these mental health applications to be certified as medical devices.

After analysing the Product Liability Directive and the upcoming legislation focused on liability associated with AI, I must state that there is insufficient transparency and protection for users of these applications. Based on experience from the user’s perspective, I have identified the lack of (1) scheduling an appointment with a healthcare professional, (2) human oversight, and (3) transparency as regards the type of AI used. Due to the ‘black box problem’, it is likely that the user who was harmed will not be able to get compensation because of the difficulty of proving causality between the defect and the damage.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 139.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Hardcover Book
USD 179.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    Worldometer (2020).

  2. 2.

    Germany disposes of a special system called DIGA, which is controlled by the German Federal Institute for Drugs and Medical Devices. Based on this system, the manufacturers can register the Digital Health Applications as medical devices. More information can be found at this link: https://www.bfarm.de/EN/Medical-devices/Tasks/DiGA-and-DiPA/Digital-Health-Applications/_node.html.

  3. 3.

    Müllerová (2020), p. 139.

  4. 4.

    The UK has a special organisation that checks the security of digital health applications.

    The Organization for the Review of Care and Health Apps (ORCHA) provides the core infrastructure to overcome these barriers and introduce digital health safely. More information can be found at this link: https://orchahealth.com/.

  5. 5.

    Nahy and Williams (2021), p. 11.

  6. 6.

    IQVIA (2021), p. 4.

  7. 7.

    Ming et al. (2020).

  8. 8.

    ORCHA (2021), p. 3.

  9. 9.

    IQVIA (2021), p. 8.

  10. 10.

    Miller and Polson (2019), p. 213.

  11. 11.

    Buchholz (2020).

  12. 12.

    ORCHA (2021).

  13. 13.

    Statista (2022).

  14. 14.

    Ministry of Health and Social Affairs (2016), p. 3.

  15. 15.

    Statista (2021).

  16. 16.

    Eurostat (2020).

  17. 17.

    Ernsting et al. (2017); Wildenbos et al. (2019), p. 76.

  18. 18.

    World Health Organisation (2017).

  19. 19.

    Bernardo et al. (2021).

  20. 20.

    Bohr and Memarzadeh (2020), p. 34.

  21. 21.

    Eurostat (2020).

  22. 22.

    The organ named by the Swedish government has the task of analysing medical-ethical issues from an overall societal perspective and stimulating the public debate on medical-ethical issues.

  23. 23.

    SMER (2017), p. 4.

  24. 24.

    Sawrikar and Mote (2022), p. 5.

  25. 25.

    ORCHA (2022), p. 3.

  26. 26.

    Orăştean et al. (2022), p. 68.

  27. 27.

    Davenport et al. (2022), p. 4.

  28. 28.

    ORCHA (2022), p. 6.

  29. 29.

    Byambasuren et al. (2019).

  30. 30.

    ORCHA (2021).

  31. 31.

    Regulation (EU) 2016/679. The protection of natural persons with regard to the processing of personal data and on the free movement of such data. European Parliament and Council. http://data.europa.eu/eli/reg/2016/679/oj.

  32. 32.

    Deloitte (2017), p. 8.

  33. 33.

    E-hälsomyndigheten (2022), p. 21.

  34. 34.

    SMER (2017), p. 2.

  35. 35.

    Zhang and Koch (2015).

  36. 36.

    Kleinman (2021).

  37. 37.

    IQVIA (2021), p. 10.

  38. 38.

    ORCHA (2022), p. 6.

  39. 39.

    Bhaskar and Rao (2022), p. 51.

  40. 40.

    Hälsoappar förutsättningar och användning (2022), p. 4.

  41. 41.

    Haute Autorité de Santé (2021), p. 9.

  42. 42.

    Haute Autorité de Santé (2021), p. 10.

  43. 43.

    Directive 93/42/EEC of 14 June 1993 concerning medical devices. Council of the European Communities. http://data.europa.eu/eli/dir/1993/42/oj.

  44. 44.

    Regulation (EU) 2017/745 of the European Parliament and of the Council of 5 April 2017 on medical devices, amending Directive 2001/83/EC, Regulation (EC) No 178/2002 and Regulation (EC) No 1223/2009 and repealing Council Directives 90/385/EEC and 93/42/EEC. European Parliament and Council. http://data.europa.eu/eli/reg/2017/745/oj.

  45. 45.

    Regulation (EU) 2017/746 of the European Parliament and of the Council of 5 April 2017 on in vitro diagnostic medical devices and repealing Directive 98/79/EC and Commission Decision 2010/227/EU. European Parliament and Council. http://data.europa.eu/eli/reg/2017/746/oj.

  46. 46.

    Gordon et al. (2020).

  47. 47.

    Parmar (2022), p. 8.

  48. 48.

    E-hälsomyndigheten (2022), p. 19.

  49. 49.

    CJEU, Case C-219/115, Brain Products GmbH contre BioSemi VOF e.a, ECLI :EU :C :2012 :742. point 11.

  50. 50.

    De Grove-Valdeyron (2019).

  51. 51.

    Davenport et al. (2022), p. 51.

  52. 52.

    Point 5.

  53. 53.

    Martelli et al. (2019).

  54. 54.

    Zednik (2021), p. 265.

  55. 55.

    Morley et al. (2020).

  56. 56.

    The search in the library Best for you is available at this link: https://bestforyou.orcha.co.uk/defaultsearch/?search=Depression.

  57. 57.

    Turk (2018).

  58. 58.

    Singh et al. (2016).

  59. 59.

    Wachter et al. (2018).

  60. 60.

    Murphy (2021).

  61. 61.

    Markowetz et al. (2014).

  62. 62.

    Crawford and Whittaker (2016).

  63. 63.

    European Union Agency for Fundamental Rights (2022), p. 17.

  64. 64.

    Shusted et al. (2021).

  65. 65.

    Last (2007).

  66. 66.

    Rajkomar et al. (2018).

  67. 67.

    Brault and Saxena (2021).

  68. 68.

    Tat et al. (2020).

  69. 69.

    SMER (2020), p. 5.

  70. 70.

    Brault and Saxena (2021).

  71. 71.

    Tegen (2022).

  72. 72.

    Paul Mason (2016).

  73. 73.

    Roser (2023).

  74. 74.

    Schneeberger et al. (2020), p. 220.

  75. 75.

    Wagner (2017), p. 59.

  76. 76.

    Schönberger (2019), p. 199.

  77. 77.

    Pesapane (2018).

  78. 78.

    European Commission (2020), p. 13.

  79. 79.

    Lohsse et al. (2019)

  80. 80.

    Ordish (2018), p. 3.

  81. 81.

    Schneeberger et al. (2020).

  82. 82.

    Wendehorst (2020), p. 160.

  83. 83.

    Schneeberger et al. (2020), p. 222.

  84. 84.

    Navas (2020), p. 77.

  85. 85.

    European Commission (2022).

  86. 86.

    Article 9 para 4 of the Proposal for a Directive of the European Parliament and of the Council on liability for defective products.

  87. 87.

    European Commission (2016).

References

Journal Articles

  • Brault N, Saxena M (2021) For a critical appraisal of artificial intelligence in healthcare: the problem of bias in mHealth. Eval Clin Pract 27(3):513–519. https://doi.org/10.1111/jep.13528

    Article  Google Scholar 

  • Byambasuren O, Beller E, Glasziou P (2019) Current knowledge and adoption of mobile health apps among Australian general practitioners: survey study. JMIR Mhealth Uhealth 7(6). https://doi.org/10.2196/13199

  • Crawford K, Whittaker M (2016) The AI now report the social and economic implications of artificial intelligence technologies in the near-term. In AI now public symposium 2016, White House and New York University’s Information Law Institute

    Google Scholar 

  • Davenport TH, Glaser J, Gardner E (2022) Artificial intelligence in healthcare. Edward Elgar, Cheltenham

    Google Scholar 

  • De Grove-Valdeyron N (2019) Lecture du droit européen numérique de la santé : conséquences sur les patients, l’accès aux soins, la circulation des patients, pp 79–105. https://doi.org/10.4000/books.putc.4133

  • Ernsting C, Dombrowski SU, Oedekoven M, O’Sullivan JL, Kanzler M, Kuhlmey A, Gellert P (2017) Using smartphones and health apps to change and manage health behaviors: a population-based survey. J Med Internet 19(4). https://doi.org/10.2196/jmir.6838

  • Gordon WJ, Landman A, Zhang H, Bates DW (2020) Beyond validation: getting health apps into clinical practice. Digital Med 3(14). https://doi.org/10.1038/s41746-019-0212-z

  • Lohsse S, Schulze R, Staudenmayer D (2019) New approaches: form of liability. Münster Colloquia on EU law and the digital economy IV, pp 185–228. https://doi.org/10.5771/9783845294797

  • Markowetz A, Błaszkiewicz K, Montag C, Switala C, Schlaepfer TE (2014) Psycho-informatics: big data shaping modern psychometrics. Med Hypotheses 82(4):405–411

    Article  Google Scholar 

  • Martelli N, Eskenazy D, Déan C et al (2019) New European regulation for medical devices: what is changing? Cardiovasc Intervent Radiol 42:1272–1278. https://doi.org/10.1007/s00270-019-02247-0

    Article  Google Scholar 

  • Miller E, Polson D (2019) Apps, avatars, and robots: the future of mental healthcare. Issues Mental Health Nurs 40(3):208–2014

    Article  Google Scholar 

  • Ming LC, Untong N, Aliudin NA, Osili N, Kifli N, Tan CS, Goh KW, Al-Worafi YM, Lee KS, Goh HP (2020) Mobile health apps on COVID-19 launched in the early days of the pandemic: content analysis and review. JMIR Mhealth Uhealth 8(9). https://doi.org/10.2196/19796

  • Morley J, Machado CV, Burr C, Cowls J, Joshi I, Taddeo M, Floridi L (2020) The ethics of AI in health care: a mapping review. Soc Sci Med 260. https://doi.org/10.1016/j.socscimed.2020.113172

  • Murphy K, Di Ruggiero E, Upshur R et al (2021) Artificial intelligence for good health: a scoping review of the ethics literature. BMC Med Ethics 22(14). https://doi.org/10.1186/s12910-021-00577-8

  • Nahy F, Williams AG (2021) Use of digital health tools in Europe - before, during and after COVID-19. World Health Organization. ISSN 1997-8073

    Google Scholar 

  • Navas S (2020) Producer liability for AI-based technologies in the European Union. Int Law Res 9(1):77–84. https://doi.org/10.5539/ilr.v9n1p77

    Article  Google Scholar 

  • Orăştean R, Sava R, Mărginean S (2022) Measuring healthcare digitalisation in the European Union: trends and challenges. J Econ Financ Theory Pract 74(4):64–74. https://doi.org/10.56043/reveco-2022-0038

    Article  Google Scholar 

  • Parmar P, Ryu J, Pandya S et al (2022) Health-focused conversational agents in person-centered care: a review of apps. npj Digit Med 5(21). https://doi.org/10.1038/s41746-022-00560-6

  • Pesapane F, Volonté C, Codari M et al (2018) Artificial intelligence as a medical device in radiology: ethical and regulatory issues in Europe and the United States. Insights Imaging 9:745–753. https://doi.org/10.1007/s13244-018-0645-y

    Article  Google Scholar 

  • Rajkomar A, Hardt M, Howell MD, Corrado G, Chin MH (2018) Ensuring fairness in machine learning to advance health equity. Ann Intern Med 169(12):866–872. https://doi.org/10.7326/M18-1990

    Article  Google Scholar 

  • Sawrikar V, Mote K (2022) Technology acceptance and trust: overlooked considerations in young people’s use of digital mental health interventions. Health Policy Technol 11(4). https://doi.org/10.1016/j.hlpt.2022.100686

  • Schneeberger D, Stöger K, Holzinger A (2020) The European Legal Framework for medical AI. In: 4th international cross-domain conference for machine learning and knowledge extraction. Lecture notes in computer science, vol 12279. Springer, pp 209–226. https://doi.org/10.1007/978-3-030-57321-8_12

  • Schönberger D (2019) Artificial intelligence in healthcare: a critical analysis of the legal and ethical implications. Int J Law Inf Technol 27(2):171–203. https://doi.org/10.1093/ijlit/eaz004

    Article  Google Scholar 

  • Shusted CS, Evans NR, Juon H, Kane GC, Barta JA (2021) Association of race with lung cancer risk among adults undergoing lung cancer screening. JAMA Netw Open 4(4). https://doi.org/10.1001/jamanetworkopen.2021.4509

  • Singh A, Thakur N, Sharma A (2016) A review of supervised machine learning algorithms. In 3rd international conference on computing for sustainable global development, New Delhi, pp 1310–1315

    Google Scholar 

  • Tat E, Bhatt DL, Rabbat MG (2020) Addressing bias: artificial intelligence in cardiovascular medicine. Lancet Digital Health 2(12). https://doi.org/10.1016/S2589-7500(20)30249-1

  • Turk M (2018) Explainable Artificial Intelligence (XAI). Defense Advanced Research Projects Agency (Darpa); Local Interpretable Model Agnostic Explanations (LIME): an introduction

    Google Scholar 

  • Wachter S, Mittelstadt B, Russell C (2018) Counterfactual explanations without opening the black box: automated decisions and the GDPR. Harv J Law Technol 31:841–887

    Google Scholar 

  • Wagner G (2017) Produkthaftung für Autonome Systeme. Archiv für die civilistische Praxis 217(6):707–765

    Article  Google Scholar 

  • Wendehorst C (2020) Strict liability for AI and other emerging technologies. J Eur Tort Law 11(2):150–180. https://doi.org/10.1515/jetl-2020-0140

    Article  Google Scholar 

  • Wildenbos GA, Jaspers MWM, Schijven MP, Dusseljee-Peute LW (2019) Mobile health for older adult patients: using an aging barriers framework to classify usability problems. Int J Med Inf 124:68–77. https://doi.org/10.1016/j.ijmedinf.2019.01.006

    Article  Google Scholar 

  • Zednik C (2021) Solving the Black Box problem: a normative framework for explainable artificial intelligence. Philos Technol 34:265–288. https://doi.org/10.1007/s13347-019-00382-7

    Article  Google Scholar 

  • Zhang Y, Koch S (2015) Mobile health apps in Sweden: what do physicians recommend? Stud Health Technol Inform 210:793–797

    Google Scholar 

Books

  • Bhaskar P, Rao S (2022) Role of mobile health in the situation of COVID-19 pandemics: pros and cons. In: Poonia RCH (ed) Cyber-physical systems, pp 37–54

    Google Scholar 

  • Bohr A, Memarzadeh K (2020) Artificial intelligence in healthcare. In: The rise of artificial intelligence in healthcare applications. Academic Press, pp 25–60. https://doi.org/10.1016/C2018-0-04097-9

Thesis

  • Müllerová P (2020) The development of telemedicine in the context of cross-border healthcare services in the European Union: comparative approach France/Czech Republic. Dissertation, L’Université Toulouse 1 Capitole

    Google Scholar 

  • Tegen A (2022) Interactive online machine learning. Dissertation, Malmö University

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Petra Müllerová .

Editor information

Editors and Affiliations

Appendices

Legislation

  • Council Directive 85/374/EEC of 25 July 1985 on the approximation of the laws, regulations and administrative provisions of the Member States concerning liability for defective products. European Parliament and Council. http://data.europa.eu/eli/dir/1985/374/oj.

  • European Commission (2022) Commission Staff Working Document: Impact Assessment Report Accompanying the document Proposal for a Directive of the European Parliament and of the Council on liability for defective products, Annex 5. https://data.consilium.europa.eu/doc/document/ST-13134-2022-ADD-2/en/pdf.

  • European Commission (2016) Draft Code of Conduct on privacy for mobile health applications. https://digital-strategy.ec.europa.eu/en/library/code-conduct-privacy-mhealth-apps-has-been-finalised. Accessed 30 May 2023.

  • European Commission (2022) Commission Staff Working Document: Impact Assessment Report Accompanying the document Proposal for a Directive of the European Parliament and of the Council on liability for defective products, Annex 5. https://data.consilium.europa.eu/doc/document/ST-13134-2022-ADD-2/en/pdf.

  • European Commission Proposal for a Directive of the European Parliament and of the Council on adapting non-contractual civil liability rules to artificial intelligence. COM/2022/496 final.

  • European Commission Proposal for a Directive of the European Parliament and of the Council on liability for defective products. COM/2022/495 final.

  • European Commission Proposal for a Regulation of the European Parliament and of the Council laying down harmonised rules on Artificial intelligence and amending certain union legislative acts. COM/2021/206 final.

  • Report from the Commission to the European Parliament, the Council, and the European Economic and Social Committee. Report on the safety and liability implications of Artificial Intelligence, the Internet of Things and robotics. COM/2020/64 final.

  • European Commission Report from the Commission to the European Parliament, the Council and the European Economic and Social Committee: Report on the safety and liability implications of Artificial Intelligence, the Internet of Things and robotics. COM/2020/64 final.

  • European Commission (2020) White paper - On Artificial Intelligence - A European approach to excellence and trust. commission-white-paper-artificial-intelligence-feb2020_en.pdf (europa.eu).

  • European Union Agency for Fundamental Rights (2022) Bias in Algorithms-Artificial Intelligence and Discrimination. https://fra.europa.eu/sites/default/files/fra_uploads/fra-2022-bias-in-algorithms_en.pdf.

  • Directive 93/42/EEC of 14 June 1993 concerning medical devices. Council of the European Communities. http://data.europa.eu/eli/dir/1993/42/oj.

  • Regulation (EU) 2016/679. The protection of natural persons with regard to the processing of personal data and on the free movement of such data. European Parliament and Council. http://data.europa.eu/eli/reg/2016/679/oj.

  • Regulation (EU) 2017/745 of the European Parliament and of the Council of 5 April 2017 on medical devices, amending Directive 2001/83/EC, Regulation (EC) No 178/2002 and Regulation (EC) No 1223/2009 and repealing Council Directives 90/385/EEC and 93/42/EEC. European Parliament and Council. http://data.europa.eu/eli/reg/2017/745/oj.

  • Regulation (EU) 2017/746 of the European Parliament and of the Council of 5 April 2017 on in vitro diagnostic medical devices and repealing Directive 98/79/EC and Commission Decision 2010/227/EU. European Parliament and Council. http://data.europa.eu/eli/reg/2017/746/oj.

  • CJEU:CJEU Case C-219/115, Brain Products GmbH contre BioSemi VOF e.a, ECLI:EU:C:2012:742.

Online Publications

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Müllerová, P. (2023). The Role of AI in Mental Health Applications and Liability. In: Gill-Pedro, E., Moberg, A. (eds) YSEC Yearbook of Socio-Economic Constitutions 2023. YSEC Yearbook of Socio-Economic Constitutions, vol 2023. Springer, Cham. https://doi.org/10.1007/16495_2023_60

Download citation

  • DOI: https://doi.org/10.1007/16495_2023_60

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-55831-3

  • Online ISBN: 978-3-031-55832-0

  • eBook Packages: Law and CriminologyLaw and Criminology (R0)

Publish with us

Policies and ethics