Building Moral Robots: Ethical Pitfalls and Challenges

Abstract

This paper examines the ethical pitfalls and challenges that non-ethicists, such as researchers and programmers in the fields of computer science, artificial intelligence and robotics, face when building moral machines. Whether ethics is “computable” depends on how programmers understand ethics in the first place and on the adequacy of their understanding of the ethical problems and methodological challenges in these fields. Researchers and programmers face at least two types of problems due to their general lack of ethical knowledge or expertise. The first type is so-called rookie mistakes, which could be addressed by providing these people with the necessary ethical knowledge. The second, more difficult methodological issue concerns areas of peer disagreement in ethics, where no easy solutions are currently available. This paper examines several existing approaches to highlight the ethical pitfalls and challenges involved. Familiarity with these and similar problems can help programmers to avoid pitfalls and build better moral machines. The paper concludes that ethical decisions regarding moral robots should be based on avoiding what is immoral (i.e. prohibiting certain immoral actions) in combination with a pluralistic ethical method of solving moral problems, rather than relying on a particular ethical approach, so as to avoid a normative bias.

This is a preview of subscription content, log in to check access.

Notes

  1. 1.

    In this paper, the concept of computability is used in a rather broad sense. It concerns the way in which a machine is built to make ethical decisions that generally seek to resemble the moral reasoning and decision making of human beings.

  2. 2.

    For example, Anderson and Anderson (2007: 15) observe, “Using Moor’s terminology, most of those working on machine ethics would say that the ultimate goal is to create a machine that is an explicit ethical agent.” This might be true, but we should be prepared for the advent of IRs who are full ethical agents, regardless of how long it may take for such IRs to become a reality.

  3. 3.

    See Allen et al. (2000: 251–261) for the interesting idea of the so-called Moral Turing Test (MTT) that robots would pass if they (1) give correct answers to moral dilemmas and (2) offer the appropriate justification for their claims. This would amount to understanding ethics.

  4. 4.

    It is not claimed that the approaches analysed in the table are the only important ones, but they are a representative sample of the existing range of accounts that have been extensively discussed in recent years.

  5. 5.

    The model presented by Wallach et al. is not necessarily inadequate with respect to how moral decision making works in an empirical sense, but their approach is descriptive rather than normative by nature. Therefore, their empirical model does not solve the normative problem of how moral machines should actually act. Descriptive ethics and normative ethics are two different things. The first tells us how human beings make moral decisions; the latter is concerned with how we should act.

  6. 6.

    For a brief but good overview of the philosophical concerns, see Anderson (2011b: 162–167) and Gips (2011: 244–253).

  7. 7.

    Hall (2011: 523) claims, “In the long run, AIs will run everything, and should, because they can do a better job than humans. Not only will their intellectual prowess exceed ours, but their moral judgment as well.”

  8. 8.

    Dietrich argues that human beings, as biological entities, might have a genetic predisposition for selfish behaviour as a survival mechanism with respect to competition with others. Machines lack this predisposition and could be made in such a way that they both possess the good features of human beings and follow ethical principles. They could then be seen as “Homo sapiens 2.0”—enhanced or improved human beings.

  9. 9.

    One might also claim here that the ability of moral machines to perform “better” than humans in the realm of ethics relates to the fact that they do not suffer from moral imperfections—e.g. prejudice, selfishness, partiality, weakness of will, and being carried away by emotions—in their ethical reasoning and decision making.

  10. 10.

    See Pereira and Saptawijaya (2011) for a detailed description of how their system, ACORDA, uses prospective logic programming to solve various dilemmas like the trolley case.

  11. 11.

    See, for example, the interesting programmable “moral machine” developed by Scalable Cooperation at MIT Media Lab concerning the question “Between whom is the self-driving car deciding?” in a case where the car’s brakes fail. It is a “platform for gathering a human perspective on moral decisions made by machine intelligence, such as self-driving cars … where a driverless car must choose the lesser of two evils, such as killing two passengers or five pedestrians. As an outside observer, you judge which outcome you think is more acceptable” (www.moralmachine.mit.edu, accessed 2 July 2018). However, the MIT trial does not involve cars working in concert, but only in isolation or in a way that involves human–robot interaction. Self-driving cars may eventually be part of a larger system in which machines will communicate with each other and thereby solve upcoming problems more fully than can occur when each machine functions in isolation, particularly in the context of automated driving (Borenstein et al. 2017).

  12. 12.

    The idea that machines might come up with new or unexpressed ethical principles has been advanced by Anderson and Anderson (2011b: 476–492, esp. 479).

References

  1. Abney, K. (2014). Robotics, ethical theory, and metaethics: A guide for the perplexed. In P. Lin, K. Abney, & G. Bekey (Eds.), Robot ethics: The ethical and social implications of robotics (pp. 35–52). Cambridge, MA: MIT Press.

    Google Scholar 

  2. Aletras, N., Tsarapatsanis, D., Preoţiuc-Pietro, D., & Lampos, V. (2016). Predicting judicial decisions of the European Court of Human Rights: A natural language processing perspective. PeerJ Computer Science,2(e93), 1–19.

    Google Scholar 

  3. Allen, C., Varner, G., & Zinser, J. (2000). Prolegomena to any future artificial moral agent. Journal of Experimental & Theoretical Artificial Intelligence,12(3), 251–261.

    Google Scholar 

  4. Allen, C., Wallach, W., & Smit, I. (2011). Why machine ethics? In M. Anderson & S. Anderson (Eds.), Machine ethics (pp. 51–61). Cambridge, MA: Cambridge University Press.

    Google Scholar 

  5. Anderson, S. L. (2011a). How machines might help us achieve breakthroughs in ethical theory and inspire us to behave better. In M. Anderson & S. L. Anderson (Eds.), Machine ethics (pp. 524–5309). Cambridge, MA: Cambridge University Press.

    Google Scholar 

  6. Anderson, S. L. (2011b). Philosophical concerns with machine ethics. In M. Anderson & S. L. Anderson (Eds.), Machine ethics (pp. 162–167). Cambridge, MA: Cambridge University Press.

    Google Scholar 

  7. Anderson, S. L. (2011c). Machine metaethics. In M. Anderson & S. L. Anderson (Eds.), Machine ethics (pp. 21–27). Cambridge, MA: Cambridge University Press.

    Google Scholar 

  8. Anderson, M., & Anderson, S. L. (2007). Machine ethics: Creating an ethical intelligent agent. AI Magazine,28(4), 15–26.

    Google Scholar 

  9. Anderson, M., & Anderson, S. L. (2011a). Machine ethics. Cambridge, MA: Cambridge University Press.

    Google Scholar 

  10. Anderson, M., & Anderson, S. L. (2011b). A prima facie duty approach to machine ethics. In M. Anderson & S. L. Anderson (Eds.), Machine ethics (pp. 476–492). Cambridge, MA: Cambridge University Press.

    Google Scholar 

  11. Angwin, J., Larson, J., Mattu, S., & Kirchner, L. (2016). Machine bias. ProPublica. https://www.propublica.org/article/machine-bias-risk-assessments-in-criminal-sentencing. Accessed 8 March 2018.

  12. Bayertz, K. (1991). Praktische Philosophie als angewandte Ethik”. In K. Bayertz (Ed.), Praktische Philosophie: Grundorientierungen angewandter Ethik (pp. 7–47). Hamburg: Rowohlt Taschenbuch Verlag.

    Google Scholar 

  13. Bekey, G. A. (2014). Current trends in robotics: Technology and ethics. In P. Lin, K. Abney, & G. Bekey (Eds.), Robot ethics: The ethical and social implications of robotics (pp. 17–34). Cambridge, MA: MIT Press.

    Google Scholar 

  14. Borenstein, J., Herkert, J. R., & Miller, K. W. (2017). Self-driving cars and engineering ethics: The need for a system level analysis. Science and Engineering Ethics. https://doi.org/10.1007/s11948-017-0006-0.

    Article  Google Scholar 

  15. Bringsjord, S., & Taylor, J. (2014). The divine-command approach to robot ethics. In P. Lin, K. Abney, & G. Bekey (Eds.), Robot ethics: The ethical and social implications of robotics (pp. 85–108). Cambridge, MA: MIT Press.

    Google Scholar 

  16. Brody, B. (1988). Life and death decision making. Oxford: Oxford University Press.

    Google Scholar 

  17. Clouser, K. D., & Gert, B. (1990). A critique of principlism. Journal of Medicine and Philosophy,15(2), 219–236.

    Google Scholar 

  18. Coeckelbergh, M. (2014). The moral standing of machines: Towards a relational and non-Cartesian moral hermeneutics. Philosophy & Technology,27(1), 61–77.

    Google Scholar 

  19. Copeland, J. (1993). Artificial intelligence: A philosophical introduction. Oxford: Wiley.

    Google Scholar 

  20. Dehghani, M., Forbus, K., Tomai, E., & Klenk, M. (2011). An integrated reasoning approach to moral decision making. In M. Anderson & S. L. Anderson (Eds.), Machine ethics (pp. 422–441). Cambridge, MA: Cambridge University Press.

    Google Scholar 

  21. Dietrich, E. (2011). Homo sapiens 2.0: Building the better robots of our nature. In M. Anderson & S. L. Anderson (Eds.), Machine ethics (pp. 531–538). Cambridge, MA: Cambridge University Press.

    Google Scholar 

  22. Floridi, L. (2011). On the morality of artificial agents. In M. Anderson & S. L. Anderson (Eds.), Machine ethics (pp. 184–212). Cambridge: Cambridge University Press.

    Google Scholar 

  23. Floridi, L., & Sanders, J. W. (2004). On the morality of artificial agents. Minds and Machines,14(3), 349–379.

    Google Scholar 

  24. Foot, P. (1967). The problem of abortion and the doctrine of double effect. Oxford Review,5, 5–15.

    Google Scholar 

  25. Gips, J. (2011). Towards the ethical robot. In M. Anderson & S. L. Anderson (Eds.), Machine ethics (pp. 244–253). Cambridge, MA: Cambridge University Press.

    Google Scholar 

  26. Giubilini, A., & Savulescu, J. (2018). The artificial moral advisor. The “Ideal Observer” meets artificial intelligence. Philosophy & Technology,31(2), 169–188.

    Google Scholar 

  27. Gordon, J.-S. (2011). Global ethics and principlism. Kennedy Institute of Ethics Journal,21(3), 251–276.

    Google Scholar 

  28. Gordon, J.-S. (2015). Human rights and cultural identity. Baltic Journal of Law & Politics,8(2), 112–135.

    Google Scholar 

  29. Gordon, J. S. (2016a). Ethics as a method. In J. Schildmann, J. S. Gordon, & J. Vollmann (Eds.), Clinical ethics consultation: Theories and methods, implementation, evaluation (pp. 11–20). London: Routledge.

    Google Scholar 

  30. Gordon, J.-S. (2016b). Should moral enhancement be compulsory? Ethical Perspectives,23(2), 277–305.

    Google Scholar 

  31. Gordon, J.-S. (2018). What do we owe to intelligent robots? AI & Society. https://doi.org/10.1007/s00146-018-0844-6.

    Article  Google Scholar 

  32. Gordon, J.-S., Rauprich, O., & Vollmann, J. (2011). Applying the four-principle approach. Bioethics,25(6), 293–300.

    Google Scholar 

  33. Guarini, M. (2006). Particularism and the classification and reclassification of moral cases. IEEE Intelligent Systems,21(4), 22–28.

    Google Scholar 

  34. Gunkel, D. J., & Bryson, J. (2014). The machine as moral agent and patient. Philosophy & Technology,27(1), 5–142.

    Google Scholar 

  35. Hall, J. S. (2011). Ethics for self-improving machines. In M. Anderson & S. L. Anderson (Eds.), Machine ethics (pp. 512–523). Cambridge, MA: Cambridge University Press.

    Google Scholar 

  36. Hughes, J. (2014). Compassionate AI and selfless robots: A Buddhist approach. In P. Lin, K. Abney, & G. Bekey (Eds.), Robot ethics: The ethical and social implications of robotics (pp. 69–84). Cambridge, MA: MIT Press.

    Google Scholar 

  37. Johnson, A. M., & Axinn, S. (2014). Acting versus being moral: The limits of technological moral actors. In Proceedings of the IEEE 2014 international symposium on ethics in engineering, science, and technology (pp. 30(1)–30(4)). Piscataway, NJ: IEEE Press.

  38. Kant, I., & Paton, H. J. (2009). Groundwork of the metaphysic of morals. New York: Harper Perennial Modern Classics.

    Google Scholar 

  39. Katz, D. M., Bommarito, M. J., II, & Blackman, J. (2017). A general approach for predicting the behavior of the Supreme Court of the United States. PLoS ONE,12(4), 1–18.

    Google Scholar 

  40. Lin, P., Abney, K., & Bekey, G. A. (Eds.). (2014). Robot ethics: The ethical and social implications of robotics. Intelligent robotics and autonomous agents. Cambridge, MA: MIT Press.

    Google Scholar 

  41. Macklin, R. (1999). Against relativism. Cultural diversity and the search for ethical universals in medicine. New York: Oxford University Press.

    Google Scholar 

  42. McLaren, B. M. (2011). Computational models of ethical reasoning: Challenges, initial steps, and future directions. In M. Anderson & S. L. Anderson (Eds.), Machine ethics (pp. 297–315). Cambridge, MA: Cambridge University Press.

    Google Scholar 

  43. Moor, J. H. (2006). The nature, importance, and difficulty of machine ethics. Research Gate,21(4), 18–21.

    Google Scholar 

  44. Nadeau, J. E. (2006). Only androids can be ethical. In K. Ford & C. Glymour (Eds.), Thinking about android epistemology (pp. 241–248). Cambridge, MA: MIT Press.

    Google Scholar 

  45. Pereira, L. M., & Saptawijaya, A. (2011). Modeling morality with prospective logic. In M. Anderson & S. L. Anderson (Eds.), Machine ethics (pp. 398–421). Cambridge, MA: Cambridge University Press.

    Google Scholar 

  46. Picard, R. (1997). Affective computing. Cambridge: The MIT Press.

    Google Scholar 

  47. Pontier, M. A., & Hoorn, J. F. (2012). Toward machines that behave ethically better than humans do. In N. Miyake, B. Peebles, & R. P. Cooper (Eds.), Proceedings of the 34th international annual conference of the cognitive science society (pp. 2198–2203). CogSci’12.

  48. Powers, T. (2011). Prospectives for a Kantian machine. In M. Anderson & S. L. Anderson (Eds.), Machine ethics (pp. 464–475). Cambridge, MA: Cambridge University Press.

    Google Scholar 

  49. Rodogno, R. (2016). Robots and the limits of morality. In M. Norskov (Ed.), Social robots: Boundaries, potential, challenges. London: Routledge. http://pure.au.dk/portal/files/90856828/Robots_and_the_Limits_of_Morality.pdf. Accessed 12 March 2016.

    Google Scholar 

  50. Ross, W. D. (1930). The right and the good. Oxford: Clarendon Press.

    Google Scholar 

  51. Rzepka, R., & Araki, K. (2005). What statistics could do for ethics? The idea of common sense processing based safety valve. In AAAI fall symposium on machine ethics, Technical Report FS-05-06: 85-87.

  52. Sullins, J. P. (2011). When is a robot a moral agent? In M. Anderson & S. L. Anderson (Eds.), Machine ethics (pp. 151–161). Cambridge, MA: Cambridge University Press.

    Google Scholar 

  53. Thomson, J. J. (1976). Killing, letting die, and the trolley problem. The Monist,59(2), 204–217.

    Google Scholar 

  54. Torrance, S. (2008). Ethics and consciousness in artificial agents. AI & Society,22(4), 495–521.

    Google Scholar 

  55. Wallach, W., & Allen, C. (2010). Moral machines: Teaching robots right from wrong. Oxford: Oxford University Press.

    Google Scholar 

  56. Wallach, W., Franklin, S., & Allen, C. (2010). A conceptual and computational model of moral decision making in human and artificial agents. Topics in Cognitive Science,2(3), 454–485.

    Google Scholar 

Download references

Acknowledgements

I would like to thank the anonymous reviewers for their valuable comments. This research is funded by the European Social Fund according to the activity ‘Improvement of researchers’ qualification by implementing world-class R&D projects of Measure No. 09.3.3-LMT-K-712.

Author information

Affiliations

Authors

Corresponding author

Correspondence to John-Stewart Gordon.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Gordon, J. Building Moral Robots: Ethical Pitfalls and Challenges. Sci Eng Ethics 26, 141–157 (2020). https://doi.org/10.1007/s11948-019-00084-5

Download citation

Keywords

  • Moral machines
  • Full ethical agents
  • Ethical expertise
  • Programming ethics
  • Moral pluralism