Skip to main content
SpringerLink
Log in

The European Legal Framework for Medical AI

  • Conference paper
  • First Online:
Machine Learning and Knowledge Extraction (CD-MAKE 2020)

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 12279))

Abstract

In late February 2020, the European Commission published a White Paper on Artificial Intelligence (AI) and an accompanying report on the safety and liability implications of AI, the Internet of Things (IoT) and robotics. In its White Paper, the Commission highlighted the “European Approach” to AI, stressing that “it is vital that European AI is grounded in our values and fundamental rights such as human dignity and privacy protection”. It also announced its intention to propose EU legislation for “high risk” AI applications in the nearer future which will include the majority of medical AI applications.

Based on this “European Approach” to AI, this paper analyses the current European framework regulating medical AI. Starting with the fundamental rights framework as clear guidelines, subsequently a more in-depth look will be taken at specific areas of law, focusing on data protection, product approval procedures and liability law. This analysis of the current state of law, including its problems and ambiguities regarding AI, is complemented by an outlook at the proposed amendments to product approval procedures and liability law, which, by endorsing a human-centric approach, will fundamentally influence how medical AI and AI in general will be used in Europe in the future.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Article 29 Data Protection Working Group: Guidelines on Data Protection Impact Assessment (DPIA) and determining whether processing is “likely to result in a high risk” for the purposes of Regulation 2016/679, WP248rev.01 (2017). https://ec.europa.eu/newsroom/article29/item-detail.cfm?item_id=611236

  2. Article 29 Data Protection Working Group: Guidelines on Automated individual decision-making and Profiling for the purposes of Regulation 2016/679, WP251rev.01 (2018). https://ec.europa.eu/newsroom/article29/item-detail.cfm?item_id=612053

  3. Article 29 Data Protection Working Group: Guidelines on transparency under Regulation 2016/679, WP260.rev.01 (2018). https://ec.europa.eu/newsroom/article29/item-detail.cfm?item_id=622227

  4. Bambauer, J.R.: Dr. Robot. UC Davis Law Rev. 51, 383–398 (2017)

    Google Scholar 

  5. Bathaee, Y.: The artificial intelligence black box and the failure of intent and causation. Harv. J. Law Technol. 31, 889–938 (2018)

    Google Scholar 

  6. Brkan, M.: Do algorithms rule the world? Algorithmic decision-making and data protection in the framework of the GDPR and beyond. Int. J. Law Inf. Technol. 27, 91–121 (2019). https://doi.org/10.1093/ijlit/eay017

    Article  Google Scholar 

  7. Brkan, M., Bonnet, G.: Legal and technical feasibility of the GDPR’s quest for explanation of algorithmic decisions: of black boxes, white boxes and Fata Morganas. Eur. J. Risk Regul. 11, 18–50 (2020). https://doi.org/10.1017/err.2020.10

    Article  Google Scholar 

  8. Bygrave, L.: Data protection by design and by default: deciphering the EU’s legislative requirements. Oslo Law Rev. 4, 105–120 (2017). https://doi.org/10.18261/issn.2387-3299-2017-02-03

    Article  Google Scholar 

  9. Bygrave, L.: Minding the machine v2.0. The EU general data protection regulation and automated decision-making. In: Yeung, K., Lodge, M. (eds.) Algorithmic Regulation, pp. 248–262. Oxford University Press, Oxford (2019). https://doi.org/10.1093/oso/9780198838494.001.0001

  10. Bygrave, L.: Article 22. In: Kuner, C., Bygrave, L., Docksey, C., Drechsler, L. (eds.) The EU General Data Protection Regulation (GDPR). A Commentary. Oxford University Press, Oxford (2020)

    Google Scholar 

  11. Casey, B., Farhangi, A., Vogl, R.: Rethinking explainable machines: the GDPR’s ‘right to explanation’ debate and the rise of algorithmic audits in enterprise. Berkeley Technol. Law J. 34, 143–188 (2019)

    Google Scholar 

  12. Cohen, I.G.: Informed consent and medical artificial intelligence: what to tell the patient? Georgetown Law J. (2020). https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3529576

  13. Datenethikkommission: Gutachten der Datenethikkommission (2019). https://www.bmi.bund.de/SharedDocs/downloads/DE/publikationen/themen/it-digitalpolitik/gutachten-datenethikkommission.html

  14. Denga, M.: Deliktische Haftung für künstliche Intelligenz. Computer und Recht 34, 69–78 (2018). https://doi.org/10.9785/cr-2018-0203

    Article  Google Scholar 

  15. Dupré, C.: Article 1. In: Peers, S., Hervey, T., Kenner, J., Ward, A. (eds.) The EU Charter of Fundamental Rights. A Commentary. C.H. Beck - Hart - Nomos, Baden-Baden - München - Oxford (2014). https://doi.org/10.5771/9783845259055

  16. Eberbach, W.: Wird die ärztliche Aufklärung zur Fiktion? (Teil 1). Medizinrecht 37, 1–10 (2019). https://doi.org/10.1007/s00350-018-5120-8

    Article  Google Scholar 

  17. Eberbach, W.: Wird die ärztliche Aufklärung zur Fiktion? (Teil 2). Medizinrecht 37, 111–117 (2019). https://doi.org/10.1007/s00350-019-5147-5

    Article  Google Scholar 

  18. Edwards, L., Veale, M.: Slave to the algorithm? Why a ‘right to explanation’ is probably not the remedy you are looking for. Duke Law Technol. Rev. 16, 18–84 (2017)

    Google Scholar 

  19. Edwards, L., Veale, M.: Enslaving the algorithm: from a “right to an explanation” to a “right to better decisions”? IEEE Secur. Priv. 16, 46–54 (2018). https://doi.org/10.1109/MSP.2018.2701152

    Article  Google Scholar 

  20. Etzioni, A., Etzioni, O.: Designing AI systems that obey our laws and values. Commun. ACM 59, 29–31 (2016). https://doi.org/10.1145/2955091

    Article  Google Scholar 

  21. European Commission: Report on the safety and liability implications of Artificial Intelligence, the Internet of Things and robotics (2020). https://ec.europa.eu/info/sites/info/files/report-safety-liability-artificial-intelligence-feb2020_en_1.pdf

  22. European Commission: White Paper On Artificial Intelligence - A European approach to excellence and trust (2020). https://ec.europa.eu/info/sites/info/files/commission-white-paper-artificial-intelligence-feb2020_en.pdf

  23. European Data Protection Board: Guidelines 05/2020 on consent under Regulation 2016/679, Version 1.1 (2020). https://edpb.europa.eu/sites/edpb/files/files/file1/edpb_guidelines_202005_consent_en.pdf

  24. Expert Group on Liability and New Technologies - New Technologies Formation: Liability for artificial intelligence and other emerging digital technologies (2019). https://ec.europa.eu/transparency/regexpert/index.cfm?do=groupDetail.groupMeetingDoc&docid=36608

  25. FDA: Proposed Regulatory Framework for Modifications to Artificial Intelligence/Machine Learning (AI/ML)-Based Software as a Medical Device (SaMD) - Discussion Paper and Request for Feedback (2019). https://www.fda.gov/medical-devices/software-medical-device-samd/artificial-intelligence-and-machine-learning-software-medical-device

  26. Fosch Villaronga, E., Kieseberg, P., Li, T.: Humans forget, machines remember: artificial intelligence and the right to be forgotten. Comput. Law Secur. Rev. 34, 304–313 (2018). https://doi.org/10.1016/j.clsr.2017.08.007

    Article  Google Scholar 

  27. FRA: Data quality and artificial intelligence - mitigating bias and error to protect fundamental rights (2019). https://fra.europa.eu/sites/default/files/fra_uploads/fra-2019-data-quality-and-ai_en.pdf

  28. Goodman, P., Flaxman, S.: European Union regulations on algorithmic decision-making and a “right to explanation”. AI Mag. 38, 50–57 (2017). https://doi.org/10.1609/aimag.v38i3.2741

    Article  Google Scholar 

  29. Hacker, P.: Teaching fairness to artificial intelligence: existing and novel strategies against algorithmic discrimination under EU law. Common Market Law Rev. 55, 1143–1186 (2018). https://kluwerlawonline.com/JournalArticle/Common+Market+Law+Review/55.4/COLA2018095

  30. Hacker, P., Krestel, R., Grundmann, S., Naumann, F.: Explainable AI under contract and tort law: legal incentives and technical challenges. Artif. Intell. Law 16 (2020). https://doi.org/10.1007/s10506-020-09260-6

  31. Haidinger, V.: Art 22 DSGVO. In: Knyrim, R. (ed.) Der DatKomm Praxiskommentar zum Datenschutzrecht - DSGVO und DSG. Manz, Wien, rdb.at (2018)

    Google Scholar 

  32. Harned, Z., Lungren, M.P., Rajpurkar, P.: Machine vision, medical AI, and malpractice. Harv. J. Law Technol. Digest (2019). https://jolt.law.harvard.edu/digest/machine-vision-medical-ai-and-malpractice

  33. Harris, D., O’Boyle, M., Bates, E., Buckley, C.: Law of the European Convention on Human Rights, 4th edn. Oxford University Press, Oxford (2018)

    Google Scholar 

  34. High-Level Expert Group on Artificial Intelligence: Ethics Guidelines for trustworthy AI (2019). https://ec.europa.eu/digital-single-market/en/news/ethics-guidelines-trustworthy-ai

  35. Holzinger, A.: Interactive machine learning for health informatics: when do we need the human-in-the-loop? Brain Inform. 3, 119–131 (2016). https://doi.org/10.1007/s40708-016-0042-6

    Article  Google Scholar 

  36. Holzinger, A., Langs, G., Denk, H., Zatloukal, K., Müller, H.: Causability and explainability of artificial intelligence in medicine. Wiley Interdiscip. Rev. Data Min. Knowl. Discov. 59, 29–31 (2019). https://doi.org/10.1002/widm.1312

    Article  Google Scholar 

  37. Jabri, S.: Artificial intelligence and healthcare: products and procedures. In: Wischmeyer, T., Rademacher, T. (eds.) Regulating Artificial Intelligence, pp. 307–335. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-32361-5_14

    Chapter  Google Scholar 

  38. Kaminski, M.E.: The right to explanation, explained. Berkeley Technol. Law J. 34, 189–218 (2019). https://doi.org/10.15779/Z38TD9N83H

    Article  Google Scholar 

  39. Koziol, H.: Comparative conclusions. In: Koziol, H. (ed.) Basic Questions of Tort Law from a Comparative Perspective, pp. 685–838. Jan Sramek Verlag, Vienna (2015)

    Google Scholar 

  40. Lapuschkin, S., Wäldchen, S., Binder, A., Montavon, G., Samek, W., Müller, K.R.: Unmasking Clever Hans predictors and assessing what machines really learn. Nat. Commun. 10(1) (2019). https://doi.org/10.1038/s41467-019-08987-4

  41. Lipton, Z.C.: The mythos of model interpretability. ACM Queue 16, 1–27 (2018). https://doi.org/10.1145/3236386.3241340

    Article  Google Scholar 

  42. Malgieri, G., Comandé, G.: Why a right to legibility of automated decision-making exists in the general data protection regulation. Int. Data Priv. Law 7, 243–265 (2017). https://doi.org/10.1093/idpl/ipx019

    Article  Google Scholar 

  43. Mendoza, I., Bygrave, L.: The right not to be subject to automated decisions based on profiling. In: Synodinou, T.E., Jougleux, P., Markou, C., Prastitou, T. (eds.) EU Internet Law. Regulation and Enforcement, pp. 77–98. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-64955-9_4

    Chapter  Google Scholar 

  44. Miller, T.: Explanation in artificial intelligence: insights from the social sciences. Artif. Intell. 267, 1–38 (2019). https://doi.org/10.1016/j.artint.2018.07.007

    Article  MathSciNet  MATH  Google Scholar 

  45. Minssen, T., Gerke, S., Aboy, M., Price, N., Cohen, G.: Regulatory responses to medical machine learning. J. Law Biosci. 1–18 (2020). https://doi.org/10.1093/jlb/lsaa002

  46. Mittelstadt, B., Russell, C., Wachter, S.: Explaining explanations in AI. In: FAT* 2019: Proceedings of the Conference on Fairness, Accountability, and Transparency, January 2019. pp. 279–288. ACM (2019). https://doi.org/10.1145/3287560.3287574

  47. Molnár-Gábor, F.: Artificial intelligence in healthcare: doctors, patients and liabilities. In: Wischmeyer, T., Rademacher, T. (eds.) Regulating Artificial Intelligence, pp. 337–360. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-32361-5_15

    Chapter  Google Scholar 

  48. O’Sullivan, S., et al.: Legal, regulatory, and ethical frameworks for development of standards in artificial intelligence (AI) and autonomous robotic surgery. Int. J. Med. Robot. Comput. Assist. Surg. 15, 1–12 (2019). https://doi.org/10.1002/rcs.1968

    Article  Google Scholar 

  49. PHG Foundation: Legal liability for machine learning in healthcare (2018). https://www.phgfoundation.org/briefing/legal-liability-machine-learning-in-healthcare

  50. PHG Foundation: Algorithms as medical devices (2019). https://www.phgfoundation.org/documents/algorithms-as-medical-devices.pdf

  51. Price, N.W.: Medical malpractice and black box medicine. In: Cohen, G., Fernandez Lynch, H., Vayena, E., Gasser, U. (eds.) Big Data, Health Law and Bioethics, pp. 295–306. Cambridge University Press, Cambridge (2018). https://doi.org/10.1017/9781108147972

    Chapter  Google Scholar 

  52. Reinisch, F.: Künstliche Intelligenz - Haftungsfragen 4.0. Österreichische Juristen-Zeitung, pp. 298–305 (2019)

    Google Scholar 

  53. Schönberger, D.: Artificial intelligence in healthcare: a critical analysis of the legal and ethical implications. Int. J. Law Inf. Technol. 27, 171–203 (2019). https://doi.org/10.1093/ijlit/eaz004

    Article  Google Scholar 

  54. Seehafer, A., Kohler, J.: Künstliche Intelligenz: Updates für das Produkthaftungsrecht? Europäische Zeitschrift für Wirtschaftsrecht 31, 213–218 (2020)

    Google Scholar 

  55. Selbst, A.D., Powles, J.: Meaningful information and the right to explanation. Int. Data Priv. Law 7, 233–242 (2017). https://doi.org/10.1093/idpl/ipx022

    Article  Google Scholar 

  56. Spindler, G.: Roboter, Automation, künstliche Intelligenz, selbst-steuernde Kfz - Braucht das Recht neue Haftungskategorien? Computer und Recht 31, 766–776 (2015). https://doi.org/10.9785/cr-2015-1205

    Article  Google Scholar 

  57. Topol, E.: Deep Medicine. Basic Books, New York (2019)

    Google Scholar 

  58. Wachter, S., Mittelstadt, B., Floridi, L.: Why a right to explanation of automated decision-making does not exist in the general data protection regulation. Int. Data Priv. Law 7, 76–99 (2017). https://doi.org/10.1093/idpl/ipx005

    Article  Google Scholar 

  59. Wachter, S., Mittelstadt, B., Russell, C.: Counterfactual explanations without opening the black box: automated decisions and the GDPR. Harv. J. Law Technol. 31, 841–887 (2018)

    Google Scholar 

  60. Zech, H.: Künstliche Intelligenz und Haftungsfragen. Zeitschrift für die gesamte Privatrechtswissenschaft 5, 198–219 (2019)

    Google Scholar 

  61. Zweig, K.A.: Wo Maschinen irren können (2018). https://www.bertelsmann-stiftung.de/fileadmin/files/BSt/Publikationen/GrauePublikationen/WoMaschinenIrrenKoennen.pdf

Download references

Acknowledgements

The authors declare that there are no conflicts of interests and the work does not raise any ethical issues. Parts of this work have been funded by the Austrian Science Fund (FWF), Project: P-32554 “A reference model of explainable Artificial Intelligence for the Medical Domain”.

Author information

Authors and Affiliations

  1. University of Graz, Universitätsstrasse 15, 8010, Graz, Austria

    David Schneeberger & Karl Stöger

  2. Medical University of Graz, Auenbruggerplatz 2, 8036, Graz, Austria

    David Schneeberger & Andreas Holzinger

Authors
  1. David Schneeberger
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Karl Stöger
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Andreas Holzinger
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to David Schneeberger .

Editor information

Editors and Affiliations

  1. Human-Centered AI Lab, Institute for Medical Informatics, Statistics and Doumentation, Medical University Graz, Graz, Austria

    Andreas Holzinger

  2. UAS St. Pölten, St. Pölten, Austria

    Peter Kieseberg

  3. Institute of Software Technology and Interactive Systems, Technical University of Vienna, Vienna, Austria

    A Min Tjoa

  4. SBA Research, Vienna, Austria

    Edgar Weippl

Rights and permissions

Reprints and permissions

Copyright information

© 2020 IFIP International Federation for Information Processing

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Schneeberger, D., Stöger, K., Holzinger, A. (2020). The European Legal Framework for Medical AI. In: Holzinger, A., Kieseberg, P., Tjoa, A., Weippl, E. (eds) Machine Learning and Knowledge Extraction. CD-MAKE 2020. Lecture Notes in Computer Science(), vol 12279. Springer, Cham. https://doi.org/10.1007/978-3-030-57321-8_12

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-57321-8_12

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-57320-1

  • Online ISBN: 978-3-030-57321-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Societies and partnerships

Search

Navigation