Skip to main content
SpringerLink
Log in

Building Moral Robots: Ethical Pitfalls and Challenges

  • Original Research/Scholarship
  • Published:
Science and Engineering Ethics Aims and scope Submit manuscript

Abstract

This paper examines the ethical pitfalls and challenges that non-ethicists, such as researchers and programmers in the fields of computer science, artificial intelligence and robotics, face when building moral machines. Whether ethics is “computable” depends on how programmers understand ethics in the first place and on the adequacy of their understanding of the ethical problems and methodological challenges in these fields. Researchers and programmers face at least two types of problems due to their general lack of ethical knowledge or expertise. The first type is so-called rookie mistakes, which could be addressed by providing these people with the necessary ethical knowledge. The second, more difficult methodological issue concerns areas of peer disagreement in ethics, where no easy solutions are currently available. This paper examines several existing approaches to highlight the ethical pitfalls and challenges involved. Familiarity with these and similar problems can help programmers to avoid pitfalls and build better moral machines. The paper concludes that ethical decisions regarding moral robots should be based on avoiding what is immoral (i.e. prohibiting certain immoral actions) in combination with a pluralistic ethical method of solving moral problems, rather than relying on a particular ethical approach, so as to avoid a normative bias.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

Notes

  1. In this paper, the concept of computability is used in a rather broad sense. It concerns the way in which a machine is built to make ethical decisions that generally seek to resemble the moral reasoning and decision making of human beings.

  2. For example, Anderson and Anderson (2007: 15) observe, “Using Moor’s terminology, most of those working on machine ethics would say that the ultimate goal is to create a machine that is an explicit ethical agent.” This might be true, but we should be prepared for the advent of IRs who are full ethical agents, regardless of how long it may take for such IRs to become a reality.

  3. See Allen et al. (2000: 251–261) for the interesting idea of the so-called Moral Turing Test (MTT) that robots would pass if they (1) give correct answers to moral dilemmas and (2) offer the appropriate justification for their claims. This would amount to understanding ethics.

  4. It is not claimed that the approaches analysed in the table are the only important ones, but they are a representative sample of the existing range of accounts that have been extensively discussed in recent years.

  5. The model presented by Wallach et al. is not necessarily inadequate with respect to how moral decision making works in an empirical sense, but their approach is descriptive rather than normative by nature. Therefore, their empirical model does not solve the normative problem of how moral machines should actually act. Descriptive ethics and normative ethics are two different things. The first tells us how human beings make moral decisions; the latter is concerned with how we should act.

  6. For a brief but good overview of the philosophical concerns, see Anderson (2011b: 162–167) and Gips (2011: 244–253).

  7. Hall (2011: 523) claims, “In the long run, AIs will run everything, and should, because they can do a better job than humans. Not only will their intellectual prowess exceed ours, but their moral judgment as well.”

  8. Dietrich argues that human beings, as biological entities, might have a genetic predisposition for selfish behaviour as a survival mechanism with respect to competition with others. Machines lack this predisposition and could be made in such a way that they both possess the good features of human beings and follow ethical principles. They could then be seen as “Homo sapiens 2.0”—enhanced or improved human beings.

  9. One might also claim here that the ability of moral machines to perform “better” than humans in the realm of ethics relates to the fact that they do not suffer from moral imperfections—e.g. prejudice, selfishness, partiality, weakness of will, and being carried away by emotions—in their ethical reasoning and decision making.

  10. See Pereira and Saptawijaya (2011) for a detailed description of how their system, ACORDA, uses prospective logic programming to solve various dilemmas like the trolley case.

  11. See, for example, the interesting programmable “moral machine” developed by Scalable Cooperation at MIT Media Lab concerning the question “Between whom is the self-driving car deciding?” in a case where the car’s brakes fail. It is a “platform for gathering a human perspective on moral decisions made by machine intelligence, such as self-driving cars … where a driverless car must choose the lesser of two evils, such as killing two passengers or five pedestrians. As an outside observer, you judge which outcome you think is more acceptable” (www.moralmachine.mit.edu, accessed 2 July 2018). However, the MIT trial does not involve cars working in concert, but only in isolation or in a way that involves human–robot interaction. Self-driving cars may eventually be part of a larger system in which machines will communicate with each other and thereby solve upcoming problems more fully than can occur when each machine functions in isolation, particularly in the context of automated driving (Borenstein et al. 2017).

  12. The idea that machines might come up with new or unexpressed ethical principles has been advanced by Anderson and Anderson (2011b: 476–492, esp. 479).

References

  • Abney, K. (2014). Robotics, ethical theory, and metaethics: A guide for the perplexed. In P. Lin, K. Abney, & G. Bekey (Eds.), Robot ethics: The ethical and social implications of robotics (pp. 35–52). Cambridge, MA: MIT Press.

    Google Scholar 

  • Aletras, N., Tsarapatsanis, D., Preoţiuc-Pietro, D., & Lampos, V. (2016). Predicting judicial decisions of the European Court of Human Rights: A natural language processing perspective. PeerJ Computer Science,2(e93), 1–19.

    Google Scholar 

  • Allen, C., Varner, G., & Zinser, J. (2000). Prolegomena to any future artificial moral agent. Journal of Experimental & Theoretical Artificial Intelligence,12(3), 251–261.

    Google Scholar 

  • Allen, C., Wallach, W., & Smit, I. (2011). Why machine ethics? In M. Anderson & S. Anderson (Eds.), Machine ethics (pp. 51–61). Cambridge, MA: Cambridge University Press.

    Google Scholar 

  • Anderson, S. L. (2011a). How machines might help us achieve breakthroughs in ethical theory and inspire us to behave better. In M. Anderson & S. L. Anderson (Eds.), Machine ethics (pp. 524–5309). Cambridge, MA: Cambridge University Press.

    Google Scholar 

  • Anderson, S. L. (2011b). Philosophical concerns with machine ethics. In M. Anderson & S. L. Anderson (Eds.), Machine ethics (pp. 162–167). Cambridge, MA: Cambridge University Press.

    Google Scholar 

  • Anderson, S. L. (2011c). Machine metaethics. In M. Anderson & S. L. Anderson (Eds.), Machine ethics (pp. 21–27). Cambridge, MA: Cambridge University Press.

    Google Scholar 

  • Anderson, M., & Anderson, S. L. (2007). Machine ethics: Creating an ethical intelligent agent. AI Magazine,28(4), 15–26.

    Google Scholar 

  • Anderson, M., & Anderson, S. L. (2011a). Machine ethics. Cambridge, MA: Cambridge University Press.

    Google Scholar 

  • Anderson, M., & Anderson, S. L. (2011b). A prima facie duty approach to machine ethics. In M. Anderson & S. L. Anderson (Eds.), Machine ethics (pp. 476–492). Cambridge, MA: Cambridge University Press.

    Google Scholar 

  • Angwin, J., Larson, J., Mattu, S., & Kirchner, L. (2016). Machine bias. ProPublica. https://www.propublica.org/article/machine-bias-risk-assessments-in-criminal-sentencing. Accessed 8 March 2018.

  • Bayertz, K. (1991). Praktische Philosophie als angewandte Ethik”. In K. Bayertz (Ed.), Praktische Philosophie: Grundorientierungen angewandter Ethik (pp. 7–47). Hamburg: Rowohlt Taschenbuch Verlag.

    Google Scholar 

  • Bekey, G. A. (2014). Current trends in robotics: Technology and ethics. In P. Lin, K. Abney, & G. Bekey (Eds.), Robot ethics: The ethical and social implications of robotics (pp. 17–34). Cambridge, MA: MIT Press.

    Google Scholar 

  • Borenstein, J., Herkert, J. R., & Miller, K. W. (2017). Self-driving cars and engineering ethics: The need for a system level analysis. Science and Engineering Ethics. https://doi.org/10.1007/s11948-017-0006-0.

    Article  Google Scholar 

  • Bringsjord, S., & Taylor, J. (2014). The divine-command approach to robot ethics. In P. Lin, K. Abney, & G. Bekey (Eds.), Robot ethics: The ethical and social implications of robotics (pp. 85–108). Cambridge, MA: MIT Press.

    Google Scholar 

  • Brody, B. (1988). Life and death decision making. Oxford: Oxford University Press.

    Google Scholar 

  • Clouser, K. D., & Gert, B. (1990). A critique of principlism. Journal of Medicine and Philosophy,15(2), 219–236.

    Google Scholar 

  • Coeckelbergh, M. (2014). The moral standing of machines: Towards a relational and non-Cartesian moral hermeneutics. Philosophy & Technology,27(1), 61–77.

    Google Scholar 

  • Copeland, J. (1993). Artificial intelligence: A philosophical introduction. Oxford: Wiley.

    Google Scholar 

  • Dehghani, M., Forbus, K., Tomai, E., & Klenk, M. (2011). An integrated reasoning approach to moral decision making. In M. Anderson & S. L. Anderson (Eds.), Machine ethics (pp. 422–441). Cambridge, MA: Cambridge University Press.

    Google Scholar 

  • Dietrich, E. (2011). Homo sapiens 2.0: Building the better robots of our nature. In M. Anderson & S. L. Anderson (Eds.), Machine ethics (pp. 531–538). Cambridge, MA: Cambridge University Press.

    Google Scholar 

  • Floridi, L. (2011). On the morality of artificial agents. In M. Anderson & S. L. Anderson (Eds.), Machine ethics (pp. 184–212). Cambridge: Cambridge University Press.

    Google Scholar 

  • Floridi, L., & Sanders, J. W. (2004). On the morality of artificial agents. Minds and Machines,14(3), 349–379.

    Google Scholar 

  • Foot, P. (1967). The problem of abortion and the doctrine of double effect. Oxford Review,5, 5–15.

    Google Scholar 

  • Gips, J. (2011). Towards the ethical robot. In M. Anderson & S. L. Anderson (Eds.), Machine ethics (pp. 244–253). Cambridge, MA: Cambridge University Press.

    Google Scholar 

  • Giubilini, A., & Savulescu, J. (2018). The artificial moral advisor. The “Ideal Observer” meets artificial intelligence. Philosophy & Technology,31(2), 169–188.

    Google Scholar 

  • Gordon, J.-S. (2011). Global ethics and principlism. Kennedy Institute of Ethics Journal,21(3), 251–276.

    Google Scholar 

  • Gordon, J.-S. (2015). Human rights and cultural identity. Baltic Journal of Law & Politics,8(2), 112–135.

    Google Scholar 

  • Gordon, J. S. (2016a). Ethics as a method. In J. Schildmann, J. S. Gordon, & J. Vollmann (Eds.), Clinical ethics consultation: Theories and methods, implementation, evaluation (pp. 11–20). London: Routledge.

    Google Scholar 

  • Gordon, J.-S. (2016b). Should moral enhancement be compulsory? Ethical Perspectives,23(2), 277–305.

    Google Scholar 

  • Gordon, J.-S. (2018). What do we owe to intelligent robots? AI & Society. https://doi.org/10.1007/s00146-018-0844-6.

    Article  Google Scholar 

  • Gordon, J.-S., Rauprich, O., & Vollmann, J. (2011). Applying the four-principle approach. Bioethics,25(6), 293–300.

    Google Scholar 

  • Guarini, M. (2006). Particularism and the classification and reclassification of moral cases. IEEE Intelligent Systems,21(4), 22–28.

    Google Scholar 

  • Gunkel, D. J., & Bryson, J. (2014). The machine as moral agent and patient. Philosophy & Technology,27(1), 5–142.

    Google Scholar 

  • Hall, J. S. (2011). Ethics for self-improving machines. In M. Anderson & S. L. Anderson (Eds.), Machine ethics (pp. 512–523). Cambridge, MA: Cambridge University Press.

    Google Scholar 

  • Hughes, J. (2014). Compassionate AI and selfless robots: A Buddhist approach. In P. Lin, K. Abney, & G. Bekey (Eds.), Robot ethics: The ethical and social implications of robotics (pp. 69–84). Cambridge, MA: MIT Press.

    Google Scholar 

  • Johnson, A. M., & Axinn, S. (2014). Acting versus being moral: The limits of technological moral actors. In Proceedings of the IEEE 2014 international symposium on ethics in engineering, science, and technology (pp. 30(1)–30(4)). Piscataway, NJ: IEEE Press.

  • Kant, I., & Paton, H. J. (2009). Groundwork of the metaphysic of morals. New York: Harper Perennial Modern Classics.

    Google Scholar 

  • Katz, D. M., Bommarito, M. J., II, & Blackman, J. (2017). A general approach for predicting the behavior of the Supreme Court of the United States. PLoS ONE,12(4), 1–18.

    Google Scholar 

  • Lin, P., Abney, K., & Bekey, G. A. (Eds.). (2014). Robot ethics: The ethical and social implications of robotics. Intelligent robotics and autonomous agents. Cambridge, MA: MIT Press.

    Google Scholar 

  • Macklin, R. (1999). Against relativism. Cultural diversity and the search for ethical universals in medicine. New York: Oxford University Press.

    Google Scholar 

  • McLaren, B. M. (2011). Computational models of ethical reasoning: Challenges, initial steps, and future directions. In M. Anderson & S. L. Anderson (Eds.), Machine ethics (pp. 297–315). Cambridge, MA: Cambridge University Press.

    Google Scholar 

  • Moor, J. H. (2006). The nature, importance, and difficulty of machine ethics. Research Gate,21(4), 18–21.

    Google Scholar 

  • Nadeau, J. E. (2006). Only androids can be ethical. In K. Ford & C. Glymour (Eds.), Thinking about android epistemology (pp. 241–248). Cambridge, MA: MIT Press.

    Google Scholar 

  • Pereira, L. M., & Saptawijaya, A. (2011). Modeling morality with prospective logic. In M. Anderson & S. L. Anderson (Eds.), Machine ethics (pp. 398–421). Cambridge, MA: Cambridge University Press.

    Google Scholar 

  • Picard, R. (1997). Affective computing. Cambridge: The MIT Press.

    Google Scholar 

  • Pontier, M. A., & Hoorn, J. F. (2012). Toward machines that behave ethically better than humans do. In N. Miyake, B. Peebles, & R. P. Cooper (Eds.), Proceedings of the 34th international annual conference of the cognitive science society (pp. 2198–2203). CogSci’12.

  • Powers, T. (2011). Prospectives for a Kantian machine. In M. Anderson & S. L. Anderson (Eds.), Machine ethics (pp. 464–475). Cambridge, MA: Cambridge University Press.

    Google Scholar 

  • Rodogno, R. (2016). Robots and the limits of morality. In M. Norskov (Ed.), Social robots: Boundaries, potential, challenges. London: Routledge. http://pure.au.dk/portal/files/90856828/Robots_and_the_Limits_of_Morality.pdf. Accessed 12 March 2016.

    Google Scholar 

  • Ross, W. D. (1930). The right and the good. Oxford: Clarendon Press.

    Google Scholar 

  • Rzepka, R., & Araki, K. (2005). What statistics could do for ethics? The idea of common sense processing based safety valve. In AAAI fall symposium on machine ethics, Technical Report FS-05-06: 85-87.

  • Sullins, J. P. (2011). When is a robot a moral agent? In M. Anderson & S. L. Anderson (Eds.), Machine ethics (pp. 151–161). Cambridge, MA: Cambridge University Press.

    Google Scholar 

  • Thomson, J. J. (1976). Killing, letting die, and the trolley problem. The Monist,59(2), 204–217.

    Google Scholar 

  • Torrance, S. (2008). Ethics and consciousness in artificial agents. AI & Society,22(4), 495–521.

    Google Scholar 

  • Wallach, W., & Allen, C. (2010). Moral machines: Teaching robots right from wrong. Oxford: Oxford University Press.

    Google Scholar 

  • Wallach, W., Franklin, S., & Allen, C. (2010). A conceptual and computational model of moral decision making in human and artificial agents. Topics in Cognitive Science,2(3), 454–485.

    Google Scholar 

Download references

Acknowledgements

I would like to thank the anonymous reviewers for their valuable comments. This research is funded by the European Social Fund according to the activity ‘Improvement of researchers’ qualification by implementing world-class R&D projects of Measure No. 09.3.3-LMT-K-712.

Author information

Authors and Affiliations

  1. Department of Philosophy and Social Critique, Faculty of Political Science and Diplomacy, Vytautas Magnus University, V. Putvinskio g. 23 (R 403), 44243, Kaunas, Lithuania

    John-Stewart Gordon

  2. Research Cluster for Applied Ethics, Faculty of Law, Vytautas Magnus University, V. Putvinskio g. 23 (R 403), 44243, Kaunas, Lithuania

    John-Stewart Gordon

Authors
  1. John-Stewart Gordon
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to John-Stewart Gordon.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Gordon, JS. Building Moral Robots: Ethical Pitfalls and Challenges. Sci Eng Ethics 26, 141–157 (2020). https://doi.org/10.1007/s11948-019-00084-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11948-019-00084-5

Keywords

Search

Navigation