Skip to main content
SpringerLink
Log in
Book cover

Use and Misuse of New Technologies pp 211–232Cite as

Who Is to Blame for Autonomous Weapons Systems’ Misdoings?

  • Chapter
  • First Online:

Abstract

This Chapter analyses who (or what legal entity) should be held responsible for behaviours by Autonomous Weapons Systems (AWS) that, were they enacted by a human agent, would qualify as internationally wrongful acts. After illustrating the structural problems which make ascription of responsibility for AWS’ activities particularly difficult, when not impossible, the alternative routes proposed to solve the ensuing responsibility gap will be assessed. The analysis will focus, in the first place, on the international criminal responsibility of the individuals who, in one way or another, are involved in the process of production, deployment and activation of the AWS. The possibility to hold the deploying State accountable for AWS’ wrongdoings will then be gauged. Subsequently, attention will be paid to the responsibility of the corporations manufacturing and/or programming the AWS. It will be observed that these options may solve some responsibility problems more effectively than critics of AWS are ready to admit. At the same time, it will be shown that, unless a no-fault liability regime is adopted, autonomy in weapons systems is bound to magnify the risk that no one may be held to answer for acts which are objectively in contrast with international legal prescriptions. Also, it will be argued that, given the complementary relationship among the various forms of responsibility under international law, proposals aimed at focusing solely on one of these at the expense of others are incapable of leading to satisfying results.

Paper submitted on 23 February 2018. Although the Authors equally share the responsibility for the entire work, just for evaluation purposes, Sects. 2, 3, 4 and 6 should be attributed to Daniele Amoroso, while the remaining sections should be attributed to Benedetta Giordano.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   149.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Hardcover Book
USD   199.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Notes

  1. 1.

    Academic discussion on this topic was kicked off by Sparrow (2007). Since then, this problem has been dealt with in several articles, speeches and reports, of uneven quality indeed, which we will recall—as comprehensively as possible—throughout the analysis.

  2. 2.

    See, e.g., Crootof (2016), pp. 1373–1375; NATO JAPCC (2016), p. 27.

  3. 3.

    The GGE was established after a three-year cycle of informal meetings, with the mandate of exploring possible recommendations on Lethal AWS to be submitted to CCW State parties. See the Final Document of the Fifth Review Conference, 23 December 2016, UN Doc. CCW/CONF.V/10, Decision 1.

  4. 4.

    Report of the 2017 Group of Governmental Experts on Lethal Autonomous Weapons Systems, 20 November 2017, UN Doc. CCW/GGE.1/2017/CRP.1, para. 16(c).

  5. 5.

    Compare, e.g., NATO JAPCC (2016), pp. 28–31, with HRW and IHRC (2015).

  6. 6.

    Amoroso and Tamburrini (2017), pp. 3–4.

  7. 7.

    US Department of Defense, “Autonomy in Weapons Systems” Directive 3000.09 (21 November 2012), p. 2.

  8. 8.

    ICRC (2016), p. 8.

  9. 9.

    Remarkably enough, by adopting a definition of this kind, it is possible to qualify some presently operating weapons systems as autonomous, their limited target baskets notwithstanding, including stationary robotic sentinels, loitering munitions and fire-and-forget systems. For an overview of existing weapons captured by this notion of autonomy, see Amoroso and Tamburrini (2017), pp. 3–4.

  10. 10.

    Matthias (2004).

  11. 11.

    Ibid., p. 182.

  12. 12.

    Sparrow (2007).

  13. 13.

    Ibid., p. 70.

  14. 14.

    For an overview, see Johnson (2014), pp. 709–710.

  15. 15.

    Ibid., p. 714.

  16. 16.

    See, e.g., Reitinger (2015–2016), pp. 116–118.

  17. 17.

    Arkin (2009), p. 108.

  18. 18.

    Tamburrini (2016), pp. 127–128.

  19. 19.

    Ibid., p. 129. The most notable exception in this respect is provided by autonomous anti-materiel defensive systems, like the Israeli Iron Dome, which operates in sufficiently structured areas pre-set by military officers on the basis of IHL-informed judgments. See Amoroso and Tamburrini (2017), p. 14.

  20. 20.

    Thompson (1980), p. 905.

  21. 21.

    Nissenbaum (1996), p. 29.

  22. 22.

    Ibid., pp. 29–32.

  23. 23.

    Report of the Special Rapporteur on extrajudicial, summary or arbitrary executions, Christof Heyns, 9 April 2013, UN Doc. A/HRC/23/47, para. 77.

  24. 24.

    Bovens (1998), pp. 46–47.

  25. 25.

    Ibid., pp. 50–52.

  26. 26.

    See below Sect. 4.

  27. 27.

    These are the “easy cases” (Crootof 2016, pp. 1376–1377), which are often relied upon by supporters of AWS to argue that no responsibility gap would ensue from their deployment (Schmitt 2013, pp. 33–34).

  28. 28.

    McFarland and McCormack (2014).

  29. 29.

    See, e.g., Article 25(c) of the Statute of the International Criminal Court (ICC).

  30. 30.

    British Military Court, Trial of Bruno Tesch et al., 1–8 March 1946, The UN War Crimes Commission “Law reports of trials of war criminals”, Vol. I (1947), p. 93.

  31. 31.

    McFarland and McCormack (2014), p. 370.

  32. 32.

    Article 25(a) of the ICC Statute.

  33. 33.

    ICC (2011), pp. 13–43.

  34. 34.

    McFarland and McCormack (2014), pp. 376–378.

  35. 35.

    Ibid., p. 384.

  36. 36.

    van Sliedregt (2012), p. 1174.

  37. 37.

    ICTY, Prosecutor v. Tadić, IT-94-1, Appeals Judgment, 15 July 1999, paras. 185–229. The JCE doctrine has been subsequently endorsed by the International Criminal Tribunal for Rwanda (see below footnote 43), the Special Court for Sierra Leone (Prosecutor v. Kondewa, No. SCSL-03-12-PT, Decision and Order on Defence Preliminary Motion for Defects in the Form of the Indictment, 27 November 2003, para. 9) and the East Timorese Special Panel for Serious Crimes (Prosecutor v. Perreira, No. 34/2003, Judgment, 27 April 2005, pp. 19–20).

  38. 38.

    International Criminal Tribunal for Rwanda, Prosecutor v. Ntakirutimana et al., ICTR-96-17, Appeals Judgment, 13 December 2004, para. 465.

  39. 39.

    See, e.g., Badar (2006), who wittingly dubbed the doctrine as “Just Convict Everyone”.

  40. 40.

    Article 30 of the ICC Statute.

  41. 41.

    Prosecutor v. Galić, IT-98-29-T, Trial Judgment, 5 December 2003, paras. 57–58. The Trial Chamber uses, quite interchangeably, the civil law notion of dolus eventualis and the common law one of “recklessness”, although only the former can be properly characterized as a form of intent.

  42. 42.

    ICRC (1987), p. 994.

  43. 43.

    Gaeta (2016), p. 45.

  44. 44.

    Ohlin (2013).

  45. 45.

    Ibid., p. 113.

  46. 46.

    See, e.g., Reitinger (2015–2016), pp. 110–115; Schmitt (2013), p. 33; Margulies (2017).

  47. 47.

    ICRC (2005), p. 558.

  48. 48.

    Chengeta (2016), p. 31; Geiss and Lahmann (2017), p. 393.

  49. 49.

    HRW and IHRC (2015), pp. 21–22.

  50. 50.

    US Military Court, Trial of Wilhelm von Leeb et al. (The German High Command Trial), 30 December 1947–28 October 1948, The UN War Crimes Commission “Law reports of trials of war criminals”, Vol. XII, p. 76.

  51. 51.

    See, generally, Ambos (2013), pp. 197–228.

  52. 52.

    Corn (2016), p. 221.

  53. 53.

    See, e.g., HRW and IHRC (2015), p. 13.

  54. 54.

    Gaeta (2016), p. 45.

  55. 55.

    Melzer (2013), p. 40.

  56. 56.

    Conforti (2018), pp. 410–413.

  57. 57.

    For a similar conclusion, see Geiss and Lahmann (2017), pp. 386–387.

  58. 58.

    Hammond (2015) and Crootof (2016).

  59. 59.

    Conforti (2018), pp. 412–413.

  60. 60.

    In this sense, see Geiss and Lahmann (2017), pp. 390–391.

  61. 61.

    ICRC (2005), p. 56.

  62. 62.

    Ronen (2009), pp. 185–186.

  63. 63.

    Ohlin (2013), pp. 115–116.

  64. 64.

    See, e.g., ICRC (2014), pp. 8, 23 and 47.

  65. 65.

    For an overview, see Kaeb (2016).

  66. 66.

    See, above all, Article 21 of the ICC Statute, which rules out the Court’s jurisdiction over “legal persons”, including corporations. It has been carefully demonstrated (Clapham 2000), however, that this limitation was due to reasons other than the their (alleged) lack of legal personality under international criminal law.

  67. 67.

    See, also for further references, Kaeb (2016), p. 396.

  68. 68.

    Ibid., p. 385.

  69. 69.

    Stephens (2002).

  70. 70.

    Lin (2012), p. 8.

  71. 71.

    Krishnan (2009), pp. 103–104; NATO JAPCC (2016), pp. 29–30.

  72. 72.

    Quite remarkably, the 1977 European Convention on Products Liability in regard to Personal Injury and Death was signed by only 4 States and ratified by none.

  73. 73.

    See, for instance, American Law Institute 1998, Restatement of the Law, Third, Torts: Products Liability, para. 3 (“Circumstantial Evidence Supporting Inference of Product Defect”).

  74. 74.

    See, e.g., ibid., para. 2(b) (“[A product] is defective in design when the foreseeable risks of harm posed by the product could have been reduced or avoided by the adoption of a reasonable alternative design”); Directive 85/374/EEC, Article 7 (“The producer shall not be liable […] if he proves: […] e) that the state of scientific and technical knowledge at the time when he put the product into circulation was not such as to enable the existence of the defect to be discovered”).

  75. 75.

    Beck (2015), p. 475.

  76. 76.

    487 U.S. 500 (1988).

  77. 77.

    976 F.2d 1328 (1992), 1337.

  78. 78.

    See, also for further references, HRW and IHRC (2015), p. 35; Crootof (2016), p. 1395.

  79. 79.

    Kiss and Shelton (2007).

  80. 80.

    Ibid., p. 1148.

  81. 81.

    Crootof (2016), p. 1395.

  82. 82.

    Prosecutor v. Furundžija, IT-95-17/1-T, Trial Judgment, 10 December 1998, para. 146.

  83. 83.

    Bianchi (2009), p. 24. See also Chengeta (2016), pp. 49–50.

  84. 84.

    See, for references, Amoroso and Tamburrini (2017), pp. 1–2.

  85. 85.

    On a more optimistic note, it is worth adding that Germany has recently joined the list of countries that will not use AWS. see “German military has no plans to acquire robot weapons”, Reuters (15 February 2018).

  86. 86.

    Article 36, “Killer Robots: UK Government Policy on Fully Autonomous Weapons” (April 2013).

  87. 87.

    United Nations Institute for Disarmament Research, The Weaponization of Increasingly Autonomous Technologies: Considering how Meaningful Human Control Might Move the Discussion Forward, UNIDIR Resources No. 2, 2014.

  88. 88.

    For a first (and admittedly embryonic) attempt to define the MHC requirement on a principled ethical and legal basis, see Amoroso and Tamburrini (2017), pp. 13–14.

References

  • Ambos, Kai. 2013. Treatise on International Criminal Law. Volume I: Foundations and General Part. Oxford: Oxford University Press.

    Google Scholar 

  • Amoroso, Daniele, and Guglielmo Tamburrini. 2017. The Ethical and Legal Case Against Autonomy in Weapons Systems. Global Jurist (on-line): 1–20.

    Google Scholar 

  • Arkin, Ronald. 2009. Governing Lethal Behavior in Autonomous Robots. Boca Raton: CRC Press.

    Book  Google Scholar 

  • Badar, Mohamed. 2006. “Just Convict Everyone!” – Joint Perpetration: From Tadić to Stakić and Back Again. International Criminal Law Review 6: 293–302.

    Article  Google Scholar 

  • Beck, Susanne. 2015. The Problem of Ascribing Legal Responsibility in the Case of Robotics. AI & Society 31: 473–481.

    Article  Google Scholar 

  • Bianchi, Andrea. 2009. State Responsibility and Criminal Liability of Individuals. In The Oxford Companion to International Criminal Justice, ed. Antonio Cassese, 16–24. Oxford: Oxford University Press.

    Google Scholar 

  • Bovens, Mark. 1998. The Quest for Responsibility. Accountability and Citizenship in Complex Organisations. Cambridge: Cambridge University Press.

    Google Scholar 

  • Chengeta, Thompson. 2016. Accountability Gap: Autonomous Weapon Systems and Modes of Responsibility in International Law. Denver Journal of International Law and Policy 45: 1–50.

    Google Scholar 

  • Clapham, Andrew. 2000. The Question of Jurisdiction Under International Criminal Law Over Legal Persons: Lessons from the Rome Conference on the International Criminal Court. In Liability of Multinational Corporations Under International Law, ed. Menno Y. Kamminga and Saman Zia-Zarifi, 139–195. The Hague: Kluwer Law International.

    Google Scholar 

  • Conforti, Benedetto. 2018. In Diritto internazionale, ed. Massimo Iovane. Napoli: Editoriale Scientifica.

    Google Scholar 

  • Corn, Geoffrey S. 2016. Autonomous Weapons Systems: Managing the Inevitability of “Taking the Man Out of the Loop”. In Autonomous Weapons Systems: Law, Ethics, Policy, ed. Nehal Bhuta et al., 209–242. Cambridge: Cambridge University Press.

    Chapter  Google Scholar 

  • Crootof, Rebecca. 2016. War Torts: Accountability for Autonomous Weapons. University of Pennsylvania Law Review 164: 1347–1402.

    Google Scholar 

  • Gaeta, Paola. 2016. Autonomous Weapon Systems and the Alleged Responsibility Gap. Speaker’s Summary. In Expert Meeting, Autonomous Weapon Systems Implications of Increasing Autonomy in the Critical Functions of Weapons (Versoix, Switzerland, 15–16 March 2016), ed. ICRC, 44–45. Geneva: ICRC.

    Google Scholar 

  • Geiss, Robin, and Henning Lahmann. 2017. Autonomous Weapons Systems: A Paradigm Shift for the Law of Armed Conflict? In Research Handbook on Remote Warfare, ed. Jens D. Ohlin, 371–404. Cheltenham: Edward Elgar Press.

    Chapter  Google Scholar 

  • Hammond, Daniel M. 2015. Autonomous Weapons and the Problem of State Accountability. Chicago Journal of International Law 15: 652–687.

    Google Scholar 

  • Human Rights Watch (HRW) and Harvard’s International Human Rights Clinic (IHRC). 2015. Mind the Gap. The Lack of Accountability for Killer Robots.

    Google Scholar 

  • ICC. 2011. Elements of Crimes.

    Google Scholar 

  • ICRC. 1987. In Commentary on the Additional Protocols of 8 June 1977 to the Geneva Conventions of 12 August 1949, ed. Yves Sandoz, Christophe Swinarski, and Bruno Zimmermann. Geneva: Martinus Nijhoff.

    Google Scholar 

  • ———. 2005. In Customary International Humanitarian Law, ed. Jean-Marie Henckaerts and Louise Doswald-Beck. Cambridge: Cambridge University Press.

    Google Scholar 

  • ———, ed. 2014. Expert Meeting. Autonomous Weapon Systems. Technical, Military, Legal and Humanitarian Aspects (Geneva, Switzerland, 26 to 28 March 2014). Geneva: ICRC.

    Google Scholar 

  • ———. 2016. Summary Report. In Expert Meeting, Autonomous Weapon Systems Implications of Increasing Autonomy in the Critical Functions of Weapons (Versoix, Switzerland, 15–16 March 2016), ed. ICRC, 7-22. Geneva: ICRC.

    Google Scholar 

  • Johnson, Deborah G. 2014. Technology with No Human Responsibility? Journal of Business Ethics 127: 707–715.

    Article  Google Scholar 

  • Kaeb, Caroline. 2016. The Shifting Sands of Corporate Liability Under International Criminal Law. The George Washington International Law Review 49: 351–403.

    Google Scholar 

  • Kiss, Alex, and Dinah L. Shelton. 2007. Strict Liability in International Environmental Law. In Law of the Sea, Environmental Law and Settlement of Disputes: Liber Amicorum Judge Thomas A. Mensah, ed. Tafsir Malick Ndiaye and Wolfrum Rüdiger. The Hague: Brill.

    Google Scholar 

  • Krishnan, Armin. 2009. Killer Robots: Legality and Ethicality of Autonomous Weapons. Farnham: Ashgate.

    Google Scholar 

  • Lin, Patrick. 2012. Introduction to Robot Ethics. In Robot Ethics. The Ethical and Social Implications of Robotics, ed. Patrick Lin, Keith Abney, and George A. Bekey, 3–16. Cambridge, MA: MIT Press.

    Google Scholar 

  • Margulies, Peter. 2017. Making Autonomous Weapons Accountable: Command Responsibility for Computer-Guided Lethal Force in Armed Conflicts. In Research Handbook on Remote Warfare, ed. Jens D. Ohlin, 405–442. Cheltenham: Edward Elgar Press.

    Chapter  Google Scholar 

  • Matthias, Andreas. 2004. The Responsibility Gap: Ascribing Responsibility for the Actions of Learning Automata. Ethics and Information Technology 6: 175–183.

    Article  Google Scholar 

  • McFarland, Tim, and Tim McCormack. 2014. Mind the Gap: Can Developers of Autonomous Weapons Systems Be Liable for War Crimes? International Law Studies 90: 361–385.

    Google Scholar 

  • Melzer, Nils. 2013. Human Rights Implications of the Usage of Drones and Unmanned Robots in Warfare. Bruxelles: EU Directorate-General for External Policies.

    Google Scholar 

  • NATO Joint Air Power Competence Centre (JAPCC). 2016. Future Unmanned System Technologies. Legal and Ethical Implications of Increasing Automation. Kalkar (Germany): JAPCC.

    Google Scholar 

  • Nissenbaum, Helen. 1996. Accountability in a Computerized Society. Science and Engineering Ethics 2: 25–42.

    Article  Google Scholar 

  • Ohlin, Jens D. 2013. Targeting and the Concept of Intent. Michigan Journal of International Law 35: 79–130.

    Google Scholar 

  • Reitinger, Nathan. 2015–2016. Algorithmic Choice and Superior Responsibility: Closing the Gap Between Liability and Lethal Autonomy by Defining the Line Between Actors and Tools. Gonzaga Law Review 51: 79–119.

    Google Scholar 

  • Ronen, Yaël. 2009. Avoid or Compensate? Liability for Incidental Injury to Civilians Inflicted During Armed Conflict. Vanderbilt Journal of Transnational Law 42: 181–225.

    Google Scholar 

  • Schmitt, Michael N. 2013. Autonomous Weapon Systems and International Humanitarian Law: A Reply to the Critics. Harvard National Security Journal Features 1: 1–37.

    Google Scholar 

  • Sparrow, Robert. 2007. Killer Robots. Journal of Applied Philosophy 24: 62–77.

    Article  Google Scholar 

  • Stephens, Beth. 2002. Translating Filártiga: A Comparative and International Law Analysis of Domestic Remedies for International Human Rights Violations. Yale Journal of International Law 27: 1–57.

    Google Scholar 

  • Tamburrini, Guglielmo. 2016. On Banning Autonomous Weapon Systems: From Deontological to Wide Consequentialist Reasons. In Autonomous Weapons Systems: Law, Ethics, Policy, ed. Nehal Bhuta et al., 122–142. Cambridge: Cambridge University Press.

    Chapter  Google Scholar 

  • Thompson, Dennis F. 1980. Moral Responsibility of Public Officials: The Problem of Many Hands. The American Political Science Review 74: 95–106.

    Google Scholar 

  • van Sliedregt, Elies. 2012. The Curious Case of International Criminal Liability. Journal of International Criminal Justice 10: 1171–1188.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. University of Cagliari, Cagliari, Italy

    Daniele Amoroso

  2. Juvenile Court of Salerno, Salerno, Italy

    Benedetta Giordano

Authors
  1. Daniele Amoroso
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Benedetta Giordano
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Center for Studies in European and International Affairs, University of Parma, Parma, Italy

    Elena Carpanelli

  2. Department of Legal Sciences, University of Florence, Florence, Italy

    Nicole Lazzerini

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Amoroso, D., Giordano, B. (2019). Who Is to Blame for Autonomous Weapons Systems’ Misdoings?. In: Carpanelli, E., Lazzerini, N. (eds) Use and Misuse of New Technologies. Springer, Cham. https://doi.org/10.1007/978-3-030-05648-3_11

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-05648-3_11

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-05647-6

  • Online ISBN: 978-3-030-05648-3

  • eBook Packages: Law and CriminologyLaw and Criminology (R0)

Publish with us

Policies and ethics

Search

Navigation