Skip to main content
SpringerLink
Log in

Digging into the Accountability Gap: Operator’s Civil Liability in Healthcare AI-systems

  • Chapter
  • First Online:
Law and Artificial Intelligence

Part of the book series: Information Technology and Law Series ((ITLS,volume 35))

Abstract

The increasing autonomy of artificial intelligence systems (AI-systems) has put the debate about a possible ‘accountability gap’ in liability law center stage. The debate is about a possible failure of incumbent liability regimes to pinpoint the accountable agent, if in the wrongdoing an AI-system is involved. A recent attempt to address this ‘accountability gap’ is a proposal of the European Parliament, which advances laws on civil liability for the entities that control AI-systems. These newly created entities, which have no blueprint yet in liability law, are called ‘Operators’. By branching out on the healthcare applications of AI-systems, this chapter analyzes the concept of operator’s civil liability. It starts with a description of the liability concepts presented in the proposed legislation, and how they fit doctrinally with the laws and regulations of the current medical liability regime. Complementing the doctrinal analysis, this chapter employs a law and economics analysis, which showcases that the accountability gap is a serious challenge also from a consequentialist point of view. Lastly, this chapter proposes a few legal alternatives that depart from the incumbent concept of strict liability.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 79.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 99.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 99.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    European Parliament Resolution 2020b. This proposal can be seen as a continuation and manifestation of other related documents issued by expert bodies of EU institutions, such as the “Liability for artificial intelligence and other emerging digital technologies” by the Expert Group on Liability and New Technologies—New Technologies Formation and the “Ethics Guidelines for Trustworthy Artificial Intelligence” by the High-Level Expert Group on AI. This chapter was basically concluded before publication of the draft AI Act by the European Commission, COM (2021) 206 final, but we could conclude that this latter proposal does not relate to civil liability.

  2. 2.

    Article 3 of the proposal.

  3. 3.

    Teubner 2018, p. 107.

  4. 4.

    Karnow 1996, p. 191.

  5. 5.

    Ibid.

  6. 6.

    Matthias 2004, p. 175.

  7. 7.

    Floridi and Sanders 2004, p. 364.

  8. 8.

    Recital 3 of the proposal.

  9. 9.

    Article 3 of the proposal.

  10. 10.

    Ibid.

  11. 11.

    Council Directive 85/374/EEC of 25 July 1985 on the approximation of the laws, regulations and administrative provisions of the Member States concerning liability for defective products OJ L 210, 7.8.1985, pp. 29–33.

  12. 12.

    See Point 8 of the Introduction to the proposal.

  13. 13.

    Shademan et al. 2016, p. 341.

  14. 14.

    Shishhegar et al. 2018, p. 1.

  15. 15.

    It is however found in the “Ethics Guidelines for Trustworthy Artificial Intelligence” by the High-Level Expert Group on AI.

  16. 16.

    Recital 11 of the proposal.

  17. 17.

    Recital 14 of the proposal.

  18. 18.

    European Parliament Resolution 2020a and European Commission 2020.

  19. 19.

    Regulation (EU) 2016/679 of the European Parliament and of the Council of 27 April 2016 on the protection of natural persons with regard to the processing of personal data and on the free movement of such data, and repealing Directive 95/46/EC, OJ L 119, 4.5.2016, pp. 1–88, Article 9.

  20. 20.

    Koch 2011, pp. 1–39.

  21. 21.

    Article 7 of the PLD.

  22. 22.

    Proposal for a Council Directive on the liability of suppliers of services, COM (90) 482 final—SYN 308 of 20 December 1990.

  23. 23.

    For a systematic analysis of member state laws on medical liability, see Koch 2011.

  24. 24.

    Koch 2011, pp. 10–21.

  25. 25.

    Koch 2011, pp. 8–10.

  26. 26.

    Koch 2011, pp. 10–21.

  27. 27.

    Van Dam 2013, p. 300.

  28. 28.

    Ibid.

  29. 29.

    Ficuciello et al. 2019, p. 30.

  30. 30.

    Recital 17 of the proposal.

  31. 31.

    See Article 8 of the proposal for negligence-based liability rules.

  32. 32.

    Besides defect, harm, and causal link, the negligence-based regime requires a higher threshold for liability, specifically a breach of duty of care, which in turn offers less protection for the victim compared to the PLD regime which would allocate liability regardless of any specified breach in the duty of care.

  33. 33.

    Karnow 1996, p. 191.

  34. 34.

    Matthias 2004 and Karnow 1996. The role of control and foreseeability is also evident in the PLD, respectively Article 7(e).

  35. 35.

    O’Quin 2000, p. 287 and Karnow 1996, p. 191.

  36. 36.

    Matthias 2004, p. 175.

  37. 37.

    Article 3 of the proposal.

  38. 38.

    Ibid.

  39. 39.

    See Matthias 2004 and the discussion on the accountability gap in Sect. 15.1.

  40. 40.

    Alpaydin 2016, pp. 1–28.

  41. 41.

    Annany and Crawford 2018.

  42. 42.

    Recital 3 of the proposal.

  43. 43.

    Posner 1972 and Shavell 2007.

  44. 44.

    See, e.g., van Dam 2013.

  45. 45.

    Shavell 2007. For an overview, see Cooter and Ulen 2012 and Posner 2011.

  46. 46.

    Heine and Grabovets 2015, p. 44. For a discussion see also Arlen 1994.

  47. 47.

    Engstrom 2013.

  48. 48.

    Teubner 2018, pp. 129–149.

  49. 49.

    Ibid.

  50. 50.

    Simon 1960, pp. 17–55.

  51. 51.

    See, e.g., Abraham and Weiler 1994, and Cebul et al. 2008.

  52. 52.

    Galasso and Luo 2019, pp. 493–504.

  53. 53.

    A collection of alternative regulations added to liability law can be found in Rachum-Twaig 2020. Similarly, Galasso and Luo 2019, pp. 493–504.

  54. 54.

    In 2017 the European Parliament was open to this attribute of electronic entities; Res. P8_TA(2017)0051. The sometimes-furious responses (e.g. Robotics, Open letter to the European Commission Artificial Intelligence and robotics, http://www.robotics-openletter.eu/) appear to have wiped the whole idea off the table. The Commission’s White Paper of February 2020 remains silent; EC White Paper of 19 February 2020, COM (2020) 65 final.

  55. 55.

    The basic idea of conceiving the firm as a “nexus of contracts” has been put forward by Jensen and Meckling 1976. For a legal discussion, see also Easterbrook and Fischel 1989.

  56. 56.

    For the facilitative effect of corporate law for industrialization in a comprehensive political setting see, for example, Pistor et al. 2002.

  57. 57.

    Design regulations are also discussed for 3D-printing, which is another disruptive digital technology. See Heine and Li 2019.

  58. 58.

    See Fosch-Villaronga 2019 for a risk-based approach to regulation of healthcare robots.

  59. 59.

    The economic effects of granting limited liability are yet not fully understood and are dependent on specific contexts. A recent study by Koudijs and Salisbury 2020 on the effect of limited liability in marital property law sheds more light on the “social innovation” of limited liability.

  60. 60.

    For a similar argument, see Galasso and Luo 2019, pp. 97–98.

  61. 61.

    Here it might be interesting to learn from the literature on catastrophe insurance by smart public-private insurance schemes. This literature puts less the deterrence aspect into focus (a catastrophe is to a large degree an unavoidable random shock), but the compensation of victims. See, e.g., Bruggeman et al. 2010.

  62. 62.

    For a recent discussion on AI and legal personality, see Chesterman 2020.

References

  • Abraham K S, Weiler P C (1994) Enterprise Medical Liability and the Evolution of the American Health Care System. Harvard Law Review 108:381–436.

    Google Scholar 

  • Alpaydin E (2016) Machine Learning. MIT Press, Cambridge MA.

    Google Scholar 

  • Annany M, Crawford K (2018) Seeing without knowing: Limitations of the transparency ideal and its application to algorithmic accountability. New Media & Society 20: 973–989.

    Google Scholar 

  • Arlen J (1994) The Potentially Perverse Effects of Corporate Criminal Liability. Journal of Legal Studies 23:832–867.

    Google Scholar 

  • Bruggeman V et al (2010) The Government as Reinsurer of Catastrophic Risks? Geneva Papers on Risk and Insurance Theory 35:369–390.

    Google Scholar 

  • Cebul R D et al (2008) Organizational Fragmentation and Care Quality in the U.S. Health Care System. Journal of Economic Perspectives 22:93–113.

    Google Scholar 

  • Chesterman S (2020) Artificial intelligence and the limits of legal personality. International and Comparative Law Quarterly 69:819–844.

    Google Scholar 

  • Cooter R, Ulen Th (2012) Law and Economics, 6th edn. Addison-Wesley, Boston.

    Google Scholar 

  • Easterbrook F H, Fischel D R (1989) The Corporate Contract. Columbia Law Review 89:1416–1448.

    Google Scholar 

  • Engstrom N F (2013) 3-D Printing and Product Liability: Identifying the Obstacles. University of Pennsylvania Law Review Online 162:35–42.

    Google Scholar 

  • European Commission (1990) Proposal for a Council Directive on the liability of suppliers of services COM/90/482FINAL - SYN 308. https://eur-lex.europa.eu/legal-content/EN/TXT/PDF/?uri=CELEX:51990PC0482&from=EN Accessed 16 August 2021.

  • European Commission (2020) White Paper ‘On Artificial Intelligence - A European approach to excellence and trust’ COM(2020) 65 final. https://ec.europa.eu/info/publications/white-paper-artificial-intelligence-european-approach-excellence-and-trust_en 25 February 2021 Accessed 16 August 2021.

  • European Parliament Resolution (2020a) A framework of ethical aspects of artificial intelligence, robotics and related technologies. https://oeil.secure.europarl.europa.eu/oeil/popups/ficheprocedure.do?lang=en&reference=2020/2012(INL) Accessed 16 August 2021.

  • European Parliament Resolution (2020b) Civil liability regime for artificial intelligence. https://oeil.secure.europarl.europa.eu/oeil/popups/ficheprocedure.do?lang=en&reference=2020/2014(INL) Accessed 16 August 2021.

  • Ficuciello F et al (2019) Autonomy in surgical robots and its meaningful human control. https://doi.org/10.1515/pjbr-2019-0002.

  • Floridi L, Sanders J (2004) On the morality of artificial agents. Minds and Machines 14.3:349–379.

    Google Scholar 

  • Fosch-Villaronga E (2019) Robots, Healthcare and the Law: Regulating Automation in Personal Care. Routledge, New York.

    Google Scholar 

  • Galasso A, Luo H (2019) Punishing Robots: Issues in the Economics of Tort Liability and Innovation in Artificial Intelligence. In: Ajay A et al (eds) The Economics of Artificial Intelligence: An Agenda. University of Chicago Press, Chicago, pp. 493–504.

    Google Scholar 

  • Heine K, Grabovets K (2015) From individuals to organizations: The puzzle of organizational liability in tort law. Dovenschmidt Quarterly 2:44–53.

    Google Scholar 

  • Heine K, Li S (2019) What Shall we do with the Drunken Sailor? Product Safety in the Aftermath of 3D Printing. European Journal of Risk Regulation 10:23–40.

    Google Scholar 

  • Jensen M C, Meckling W H (1976) Theory of the firm: Managerial behavior, agency costs and ownership structure. Journal of Financial Economics 3:305–360.

    Google Scholar 

  • Karnow C EA (1996) Liability for distributed artificial intelligences. Berkeley Technology Law Journal 11:147–204.

    Google Scholar 

  • Koch B (2011) Medical Liability in Europe: A Comparison of Selected Jurisdictions. De Gruyter, Berlin.

    Google Scholar 

  • Koudijs P, Salisbury L (2020) Limited liability and investment: Evidence from changes in marital property laws in the US South, 1840–1850. Journal of Financial Economics 138:1–26.

    Google Scholar 

  • Matthias A (2004) The responsibility gap: Ascribing responsibility for the actions of learning automata. Ethics and Information Technology 6:175–183.

    Google Scholar 

  • O’Quinn J C (2000) Not-So-Strict Liability: A Foreseeability Test for Rylands v. Fletcher and Other Lessons from Cambridge Water Co. v. Eastern Counties Leather PLC. Harvard Environmental Law Review 24:287–314.

    Google Scholar 

  • Pistor K et al (2002) Evolution of Corporate Law: A Cross-Country Comparison. University of Pennsylvania Journal of International Economic Law 23:791–872.

    Google Scholar 

  • Posner R A (1972) A Theory of Negligence. Journal of Legal Studies 1:29–96.

    Google Scholar 

  • Posner R A (2011) Economic Analysis of Law, 8th edn. Wolters-Kluwer, New York.

    Google Scholar 

  • Rachum-Twaig O (2020) Whose Robot Is It Anyway?: Liability for Artificial-Intelligence-Based Robots. University of Illinois Law Review 4:1141–1175.

    Google Scholar 

  • Shademan A et al (2016) Supervised autonomous robotic soft tissue surgery. Science Translational Medicine 8:337–364.

    Google Scholar 

  • Shavell S (2007) Economic Analysis of Accident Law. Harvard University Press, Cambridge MA.

    Google Scholar 

  • Shishehgar M et al (2018) A systematic review of research into how robotic technology can help older people. Smart Health 7:1–18.

    Google Scholar 

  • Simon H (1960) The Corporation: Will it be Managed by Machines? In: Ansehn M, Bach G L (eds) Management and Corporations. McGraw Hill, New York, pp. 17–55.

    Google Scholar 

  • Teubner G (2018) Digital Personhood? The Status of Autonomous Software Agents in Private Law. Ancilla Iuris 106:107–149.

    Google Scholar 

  • van Dam C (2013) European Tort Law. Oxford University Press, Oxford.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Erasmus School of Law, Erasmus University Rotterdam, Burgemeester Oudlaan 50, 3062 PA, Rotterdam, The Netherlands

    Kostina Prifti, Evert Stamhuis & Klaus Heine

  2. Court of Appeal’s Hertogenbosch, Den Bosch, The Netherlands

    Evert Stamhuis

  3. University of Aruba, Oranjestad, Aruba

    Evert Stamhuis

Authors
  1. Kostina Prifti
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Evert Stamhuis
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Klaus Heine
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Kostina Prifti .

Editor information

Editors and Affiliations

  1. eLaw - Center for Law and Digital Technologies, Leiden University, Leiden, The Netherlands

    Bart Custers

  2. eLaw - Center for Law and Digital Technologies, Leiden University, Leiden, The Netherlands

    Eduard Fosch-Villaronga

Rights and permissions

Reprints and permissions

Copyright information

© 2022 T.M.C. Asser Press and the authors

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Prifti, K., Stamhuis, E., Heine, K. (2022). Digging into the Accountability Gap: Operator’s Civil Liability in Healthcare AI-systems. In: Custers, B., Fosch-Villaronga, E. (eds) Law and Artificial Intelligence. Information Technology and Law Series, vol 35. T.M.C. Asser Press, The Hague. https://doi.org/10.1007/978-94-6265-523-2_15

Download citation

  • DOI: https://doi.org/10.1007/978-94-6265-523-2_15

  • Published:

  • Publisher Name: T.M.C. Asser Press, The Hague

  • Print ISBN: 978-94-6265-522-5

  • Online ISBN: 978-94-6265-523-2

  • eBook Packages: Law and CriminologyLaw and Criminology (R0)

Publish with us

Policies and ethics

Search

Navigation