Skip to main content

Integrating robot ethics and machine morality: the study and design of moral competence in robots

Ethics and Information Technology volume 18, pages 243–256 (2016)Cite this article

Abstract

Robot ethics encompasses ethical questions about how humans should design, deploy, and treat robots; machine morality encompasses questions about what moral capacities a robot should have and how these capacities could be computationally implemented. Publications on both of these topics have doubled twice in the past 10 years but have often remained separate from one another. In an attempt to better integrate the two, I offer a framework for what a morally competent robot would look like (normally considered machine morality) and discuss a number of ethical questions about the design, use, and treatment of such moral robots in society (normally considered robot ethics). Instead of searching for a fixed set of criteria of a robot’s moral competence I identify the multiple elements that make up human moral competence and probe the possibility of designing robots that have one or more of these human elements, which include: moral vocabulary; a system of norms; moral cognition and affect; moral decision making and action; moral communication. Juxtaposing empirical research, philosophical debates, and computational challenges, this article adopts an optimistic perspective: if robotic design truly commits to building morally competent robots, then those robots could be trustworthy and productive partners, caretakers, educators, and members of the human community. Moral competence does not resolve all ethical concerns over robots in society, but it may be a prerequisite to resolve at least some of them.

This is a preview of subscription content, access via your institution.

Access options

Buy single article

Instant access to the full article PDF.

USD 39.95

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Fig. 1

Notes

  1. Numbers are derived from an EBSCO database search using “robot* and ethic*” as subject search terms and restricting to contributions in journals and books.

  2. This volume appeared as part of MIT Press’s series on “Intelligent robotics and autonomous agents.” Notably, ethics was addressed as the 19th topic in the series, 15 years after the series commenced.

  3. Strictly speaking, this requirement holds only for truthful explanations, which I hope will be the default for social robots.

References

  • Alicke, M. D. (2000). Culpable control and the psychology of blame. Psychological Bulletin, 126, 556–574. doi:10.1037//0033-2909.126.4.556.

    Article  Google Scholar 

  • Allen, C. (2011). The future of moral machines. The New York Times: Opinionator. Retrieved December 29, 2014, from http://opinionator.blogs.nytimes.com/2011/12/25/the-future-of-moral-machines/.

  • Anderson, M., & Anderson, S. (2011). Machine ethics. Cambridge: Cambridge University Press.

    Book  Google Scholar 

  • Antaki, C. (1994). Explaining and arguing: The social organization of accounts. London: Sage.

    Google Scholar 

  • Arkin, R. C. (2009). Governing lethal behavior in autonomous robots. Boca Raton, FL: CRC Press.

    Book  Google Scholar 

  • Asaro, P. M. (2006). What should we want from a robot ethic? International Review of Information Ethics, 6, 9–16.

    Google Scholar 

  • Avramova, Y. R., & Inbar, Y. (2013). Emotion and moral judgment. Wiley Interdisciplinary Reviews Cognitive Science, 4, 169–178. doi:10.1002/wcs.1216.

    Article  Google Scholar 

  • Baumeister, R. F., & Leary, M. R. (1995). The need to belong: Desire for interpersonal attachments as a fundamental human motivation. Psychological Bulletin, 117, 497–529. doi:10.1037/0033-2909.117.3.497.

    Article  Google Scholar 

  • Bello, P. (2012). Cognitive foundations for a computational theory of mindreading. Advances in Cognitive Systems, 1, 59–72.

    Google Scholar 

  • Bicchieri, C. (2006). The grammar of society: The nature and dynamics of social norms. New York, NY: Cambridge University Press.

    Google Scholar 

  • Blomkamp, N., Kinberg, S. (Producers), & Blomkamp, N. (Director). (2015).  Chappie [Motion picture]. USA: Sony Pictures Home Entertainment.  

  • Brachman, R. J. (2002). Systems that know what they’re doing. IEEE Intelligent Systems, 17, 67–71. doi:10.1109/MIS.2002.1134363.

    Article  Google Scholar 

  • Breazeal, C. L. (2002). Designing sociable robots. Cambridge, MA: MIT Press.

    MATH  Google Scholar 

  • Bringsjord, S. (2009). But perhaps robots are essentially non-persons. Erwägen Wissen Ethik, 20, 193–195.

    Google Scholar 

  • Bringsjord, S., Arkoudas, K., & Bello, P. (2006). Toward a general logicist methodology for engineering ethically correct robots. Intelligent Systems, IEEE, 21, 38–44.

    Article  Google Scholar 

  • Calverley, D. J. (2006). Android science and animal rights, does an analogy exist? Connection Science, 18, 403–417. doi:10.1080/09540090600879711.

    Article  Google Scholar 

  • Coates, D. J., & Tognazzini, N. A. (2012). The contours of blame. In D. J. Coates & N. A. Tognazzini (Eds.), Blame: Its nature and norms (pp. 3–26). New York, NY: Oxford University Press.

    Chapter  Google Scholar 

  • Coeckelbergh, M. (2010). Robot rights? Towards a social-relational justification of moral consideration. Ethics and Information Technology, 12, 209–221. doi:10.1007/s10676-010-9235-5.

    Article  Google Scholar 

  • Cox, M. T. (2011). Metareasoning, monitoring, and self-explanation. In M. T. Cox & A. Raja (Eds.), Metareasoning (pp. 131–149). Cambridge, MA: The MIT Press.

    Chapter  Google Scholar 

  • Csibra, G., & Gergely, G. (2009). Natural pedagogy. Trends in Cognitive Sciences, 13, 148–153. doi:10.1016/j.tics.2009.01.005.

    Article  Google Scholar 

  • Cushman, F. (2008). Crime and punishment: Distinguishing the roles of causal and intentional analyses in moral judgment. Cognition, 108, 353–380. doi:10.1016/j.cognition.2008.03.006.

    Article  Google Scholar 

  • Cushman, F., & Young, L. (2011). Patterns of moral judgment derive from nonmoral psychological representations. Cognitive Science, 35, 1052–1075. doi:10.1111/j.1551-6709.2010.01167.x.

    Article  Google Scholar 

  • DeBaets, A. M. (2014). Can a robot pursue the good? Exploring artificial moral agency. Journal of Evolution and Technology, 24, 76–86.

    MathSciNet  Google Scholar 

  • Dersley, I., & Wootton, A. (2000). Complaint sequences within antagonistic argument. Research on Language and Social Interaction, 33, 375–406. doi:10.1207/S15327973RLSI3304_02.

    Article  Google Scholar 

  • Eisenberg, N. (2000). Emotion, regulation, and moral development. Annual Review of Psychology, 51, 665–697.

    Article  Google Scholar 

  • Emde, R. N. (1992). Social referencing research: Uncertainty, self, and the search for meaning. In S. Feinman (Ed.), Social referencing and the social construction of reality in infancy (pp. 79–94). New York, NY: Plenum Press.

    Chapter  Google Scholar 

  • Fehr, E., & Fischbacher, U. (2004). Third-party punishment and social norms. Evolution and Human Behavior, 25, 63–87. doi:10.1016/S1090-5138(04)00005-4.

    Article  Google Scholar 

  • Fisher, M., Spielberg, S., & Weaver, B. (2014). Extant [Television series]. Los Angeles: CBS.

    Google Scholar 

  • Fiske, S. T., & Taylor, S. E. (2008). Social cognition: From brains to culture (1st ed.). Boston, MA: McGraw-Hill.

    Google Scholar 

  • Flack, J. C., & de Waal, F. B. M. (2000). “Any animal whatever”. Darwinian building blocks of morality in monkeys and apes. Journal of Consciousness Studies, 7, 1–29.

    Google Scholar 

  • Floridi, L., & Sanders, J. W. (2004). On the morality of artificial agents. Minds and Machines, 14, 349–379. doi:10.1023/B:MIND.0000035461.63578.9d.

    Article  Google Scholar 

  • Ford, K. M., & Hayes, P. J. (1991). Reasoning agents in a dynamic world: The frame problem. Greenwich, CT: JAI Press.

    Google Scholar 

  • Fridin, M. (2014). Kindergarten social assistive robot: First meeting and ethical issues. Computers in Human Behavior, 30, 262–272. doi:10.1016/j.chb.2013.09.005.

    Article  Google Scholar 

  • Garcia, E., Jimenez, M. A., De Santos, P. G., & Armada, M. (2007). The evolution of robotics research. IEEE Robotics Automation Magazine, 14, 90–103. doi:10.1109/MRA.2007.339608.

    Article  Google Scholar 

  • Gilovich, T., Keltner, D., & Nisbett, R. E. (2013). Social psychology (3rd ed.). New, NY: W.W. Norton & Co.

    Google Scholar 

  • Grau, C. (2011). There is no “I” in “Robot”: Robots and utilitarianism. In M. Anderson & S. L. Anderson (Eds.), Machine ethics (pp. 451–463). Cambridge: Cambridge University Press.

    Chapter  Google Scholar 

  • Gray, K., Young, L., & Waytz, A. (2012). Mind perception is the essence of morality. Psychological Inquiry, 23, 101–124. doi:10.1080/1047840X.2012.651387.

    Article  Google Scholar 

  • Greene, J. D., Nystrom, L. E., Engell, A. D., Darley, J. M., & Cohen, J. D. (2004). The neural bases of cognitive conflict and control in moral judgment. Neuron, 44, 389–400. doi:10.1016/j.neuron.2004.09.027.

    Article  Google Scholar 

  • Guglielmo, S., Monroe, A. E., & Malle, B. F. (2009). At the heart of morality lies folk psychology. Inquiry: An Interdisciplinary Journal of Philosophy, 52, 449–466. doi:10.1080/00201740903302600.

    Article  Google Scholar 

  • Gunkel, D. J. (2014). A vindication of the rights of machines. Philosophy & Technology, 27, 113–132. doi:10.1007/s13347-013-0121-z.

    Article  Google Scholar 

  • Haidt, J. (2001). The emotional dog and its rational tail: A social intuitionist approach to moral judgment. Psychological Review, 108, 814–834. doi:10.1037/0033-295X.108.4.814.

    Article  Google Scholar 

  • Hamlin, J. K. (2013). Moral judgment and action in preverbal infants and toddlers: Evidence for an innate moral core. Current Directions in Psychological Science, 22, 186–193. doi:10.1177/0963721412470687.

    Article  Google Scholar 

  • Harenski, C. L., Harenski, K. A., Shane, M. S., & Kiehl, K. A. (2010). Aberrant neural processing of moral violations in criminal psychopaths. Journal of Abnormal Psychology, 119, 863–874.

    Article  Google Scholar 

  • Heath, J. (2001). Communicative action and rational choice. Studies in contemporary German social thought. Cambridge, MA: MIT Press.

    Google Scholar 

  • Hilton, D. J. (2007). Causal explanation: From social perception to knowledge-based causal attribution. In A. W. Kruglanski & E. T. Higgins (Eds.), Social psychology: Handbook of basic principles (2nd ed., pp. 232–253). New York, NY: Guilford Press.

    Google Scholar 

  • Hoffman, M. L. (2008). Empathy and prosocial behavior. In M. Lewis, J. M. Haviland-Jones, & L. F. Barrett (Eds.), Handbook of emotions (3rd ed., pp. 440–455). New York, NY: Guilford Press.

    Google Scholar 

  • Hofmann, B. (2013). Ethical challenges with welfare technology: A review of the literature. Science and Engineering Ethics, 19, 389–406.

    Article  Google Scholar 

  • Huebner, B., Dwyer, S., & Hauser, M. (2009). The role of emotion in moral psychology. Trends in Cognitive Sciences, 13, 1–6. doi:10.1016/j.tics.2008.09.006.

    Article  Google Scholar 

  • Hutcherson, C. A., & Gross, J. J. (2011). The moral emotions: A social—functionalist account of anger, disgust, and contempt. Journal of Personality and Social Psychology, 100, 719–737. doi:10.1037/a0022408.

    Article  Google Scholar 

  • Johnson, A. M., & Axinn, S. (2013). The morality of autonomous robots. Journal of Military Ethics, 12, 129–141. doi:10.1080/15027570.2013.818399.

    Article  Google Scholar 

  • Kahn, Jr., P. H., Kanda, T., Ishiguro, H., Gill, B. T., Ruckert, J. H., Shen, S., Gary, H. E., et al. (2012). Do people hold a humanoid robot morally accountable for the harm it causes?. Proceedings of the Seventh Annual ACM/IEEE International Conference on Human-Robot Interaction (pp. 33–40). New York, NY: ACM. doi:10.1145/2157689.2157696.

  • Kibble, R. (2012). Can an unmanned drone be a moral agent? Ethics and accountability in military robotics. In D. J. Gunkel, J. J. Bryson, & S. Torrance (Eds.), The machine question: AI, ethics and moral responsibility (Proceedings of symposium “Machine Question: AI, Ethics, and Moral Responsibility” AISB/IACAP 2012) (pp. 62–67). The Society for the Study of Artificial Intelligence and Simulation of Behaviour.

  • Knobe, J. (2010). Person as scientist, person as moralist. Behavioral and Brain Sciences, 33, 315–329. doi:10.1017/S0140525X10000907.

    Article  Google Scholar 

  • Knobe, J., & Fraser, B. (2008). Causal judgment and moral judgment: Two experiments. Moral psychology (Vol. 2): The cognitive science of morality: Intuition and diversity (Vol. 2, pp. 441–447). Cambridge, MA: MIT Press.

  • Kohlberg, L. (1984). The psychology of moral development: The nature and validity of moral stages. San Francisco, CA: Harper & Row.

    Google Scholar 

  • Lin, P. (2013). The ethics of autonomous cars. The Atlantic. Retrieved September 30, 2014, from http://www.theatlantic.com/technology/archive/2013/10/the-ethics-of-autonomous-cars/280360/.

  • Lin, P., Abney, K., & Bekey, G. A. (Eds.). (2012). Robot ethics: The ethical and social implications of robotics. Cambridge, MA: MIT Press.

    Google Scholar 

  • Littman, M. L. (2001). Value-function reinforcement learning in Markov games. Cognitive Systems Research, 2, 55–66. doi:10.1016/S1389-0417(01)00015-8.

    Article  Google Scholar 

  • Lomas, M., Chevalier, R., Cross, E. V., Garrett, R. C., Hoare, J., & Kopack, M. (2012). Explaining robot actions. Proceedings of the 7th ACM/IEEE International Conference on Human-Robot Interaction (HRI) (pp. 187–188). Boston, MA.

  • Luo, Q., Nakic, M., Wheatley, T., Richell, R., Martin, A., & Blair, R. J. R. (2006). The neural basis of implicit moral attitude—An IAT study using event-related fMRI. NeuroImage, 30, 1449–1457. doi:10.1016/j.neuroimage.2005.11.005.

    Article  Google Scholar 

  • Malle, B. F. (1999). How people explain behavior: A new theoretical framework. Personality and Social Psychology Review, 3, 23–48. doi:10.1207/s15327957pspr0301_2.

    Article  Google Scholar 

  • Malle, B. F. (2004). How the mind explains behavior: Folk explanations, meaning, and social interaction. Cambridge, MA: MIT Press.

    Google Scholar 

  • Malle, B. F. (2011). Time to give up the dogmas of attribution: A new theory of behavior explanation. In M. P. Zanna & J. M. Olson (Eds.), Advances of experimental social psychology (Vol. 44, pp. 297–352). San Diego, CA: Academic Press.

    Google Scholar 

  • Malle, B. F., & Dickert, S. (2007). Values. In R. F. Baumeister & K. D. Vohs (Eds.), The encyclopedia of social psychology. Thousand Oaks, CA: Sage.

    Google Scholar 

  • Malle, B. F., Guglielmo, S., & Monroe, A. E. (2014). A theory of blame. Psychological Inquiry, 25, 147–186. doi:10.1080/1047840X.2014.877340.

    Article  Google Scholar 

  • Malle, B. F., & Scheutz, M. (2014). Moral competence in social robots. IEEE International Symposium on Ethics in Engineering, Science, and Technology (pp. 30–35). Presented at the IEEE International Symposium on Ethics in Engineering, Science, and Technology, June, Chicago, IL: IEEE.

  • Malle, B. F., Scheutz, M., Arnold, T., Voiklis, J., & Cusimano, C. (2015). Sacrifice one for the good of many? People apply different moral norms to human and robot agents. HRI’15: Proceedings of the Tenth Annual ACM/IEEE International Conference on Human-Robot Interaction (pp. 117–124). New York, NY: ACM.

  • McCullough, M. E., Kurzban, R., & Tabak, B. A. (2013). Putting revenge and forgiveness in an evolutionary context. Behavioral and Brain Sciences, 36, 41–58. doi:10.1017/S0140525X12001513.

    Article  Google Scholar 

  • McKenna, M. (2012). Directed blame and conversation. In D. J. Coates & N. A. Tognazzini (Eds.), Blame: Its nature and norms (pp. 119–140). New York, NY: Oxford University Press.

    Chapter  Google Scholar 

  • MHAT-IV. (2006). Mental Health Advisory Team (MHAT) IV: Operation Iraqi Freedom 05-07 Final report. Washington, DC: Office of the Surgeon, Multinational Force-Iraq; Office of the Surgeon General, United States Army Medical Command.

  • Mikhail, J. (2007). Universal moral grammar: Theory, evidence and the future. Trends in Cognitive Sciences, 11, 143–152. doi:10.1016/j.tics.2006.12.007.

    Article  Google Scholar 

  • Milgram, S. (1963). Behavioral study of obedience. Journal of Abnormal and Social Psychology, 67, 371–378.

    Article  Google Scholar 

  • Millar, J. (2014). An ethical dilemma: When robot cars must kill, who should pick the victim? Robohub. Robohub.org. Retrieved September 28, 2014, from http://robohub.org/an-ethical-dilemma-when-robot-cars-must-kill-who-should-pick-the-victim/.

  • Mithen, S. (Ed.). (1998). Creativity in human evolution and prehistory. New York, NY: Taylor & Francis.

    Google Scholar 

  • Monroe, A. E., Dillon, K. D., & Malle, B. F. (2014). Bringing free will down to earth: People’s psychological concept of free will and its role in moral judgment. Consciousness and Cognition, 27, 100–108. doi:10.1016/j.concog.2014.04.011.

    Article  Google Scholar 

  • Monroe, A. E., & Malle, B. F. (2010). From uncaused will to conscious choice: The need to study, not speculate about people’s folk concept of free will. Review of Philosophy and Psychology, 1, 211–224.

    Article  Google Scholar 

  • Monroe, A. E., & Malle, B. F. (2014). Free will without metaphysics. In A. R. Mele (Ed.), Surrounding free will (pp. 25–48). New York, NY: Oxford University Press.

    Chapter  Google Scholar 

  • Moor, J. H. (2006). The nature, importance, and difficulty of machine ethics. IEEE Intelligent Systems, 21, 18–21. doi:10.1109/MIS.2006.80.

    Article  Google Scholar 

  • Nisbett, R. E., & Wilson, T. D. (1977). Telling more than we know: Verbal reports on mental processes. Psychological Review, 84, 231–259.

    Article  Google Scholar 

  • Nourbakhsh, I. R. (2013). Robot futures. Cambridge, MA: MIT Press.

    Google Scholar 

  • Open Roboethics Initiative. (2014a). If death by autonomous car is unavoidable, who should die? Reader poll results.

  • Open Roboethics Initiative. (2014b). My (autonomous) car, my safety: Results from our reader poll.

  • Parthemore, J., & Whitby, B. (2013). What makes any agent a moral agent? Reflections on machine consciousness and moral agency. International Journal of Machine Consciousness, 4, 105–129.

    Article  Google Scholar 

  • Paxton, J. M., Ungar, L., & Greene, J. D. (2012). Reflection and reasoning in moral judgment. Cognitive Science, 36, 163–177. doi:10.1111/j.1551-6709.2011.01210.x.

    Article  Google Scholar 

  • Petersen, S. (2007). The ethics of robot servitude. Journal of Experimental & Theoretical Artificial Intelligence, 19, 43–54. doi:10.1080/09528130601116139.

    Article  Google Scholar 

  • Powell, N. L., Derbyshire, S. W. G., & Guttentag, R. E. (2012). Biases in children’s and adults’ moral judgments. Journal of Experimental Child Psychology, 113, 186–193. doi:10.1016/j.jecp.2012.03.006.

    Article  Google Scholar 

  • Powers, T. M. (2006). Prospects for a Kantian machine. IEEE Intelligent Systems, 21, 46–51. doi:10.1109/MIS.2006.77.

    Article  Google Scholar 

  • Powers, T. M. (2011). Incremental machine ethics. Robotics & Automation Magazine, IEEE, 18, 51–58. doi:10.1109/MRA.2010.940152.

    Article  Google Scholar 

  • Pylyshyn, Z. W. (Ed.). (1987). The Robot’s dilemma: The frame problem in artificial intelligence. Norwood, NJ: Ablex.

    Google Scholar 

  • Ryle, G. (1949). The concept of mind. London: Penguin Press.

    Google Scholar 

  • Schank, R. C., & Abelson, R. P. (1977). Scripts, plans, goals, and understanding: An inquiry into human knowledge structures. Hillsdale, NJ: Erlbaum.

    MATH  Google Scholar 

  • Scheutz, M. (2012). The affect dilemma for artificial agents: Should we develop affective artificial agents? IEEE Transactions on Affective Computing, 3, 424–433.

    Article  Google Scholar 

  • Scheutz, M., & Crowell, C. R. (2007). The burden of embodied autonomy: Some reflections on the social and ethical implications of autonomous robots. Proceedings of Workshop on Roboethics at ICRA 2007. Rome, Italy.

  • Scheutz, M., Malle, B. F., & Briggs, G. (2015). Towards morally sensitive action selection for autonomous social robots. The 24th IEEE international symposium on robot and human interactive communication, 2015 RO-MAN. Presented at the 24th IEEE International Symposium on Robot and Human Interactive Communication. (2015). RO-MAN. Japan: Kobe.

    Google Scholar 

  • Scheutz, M., & Malle, B. F. (2014). “Think and do the right thing”: A plea for morally competent autonomous robots. Presented at the 2014 IEEE Ethics conference, Chicago, IL.

  • Semin, G. R., & Manstead, A. S. R. (1983). The accountability of conduct: A social psychological analysis. London: Academic Press.

    Google Scholar 

  • Shaver, K. G. (1985). The attribution of blame: Causality, responsibility, and blameworthiness. New York: Springer.

    Book  Google Scholar 

  • Sullins, J. P. (2011). Introduction: Open questions in roboethics. Philosophy & Technology, 24, 233. doi:10.1007/s13347-011-0043-6.

    Article  Google Scholar 

  • Talamadupula, K., Schermerhorn, P., Benton, J., Kambhampati, S., & Scheutz, M. (2011). Planning for agents with changing goals. ICAPS 2011 System Demonstration. Germany: Freiburg.

    Google Scholar 

  • Tanaka, F., Cicourel, A., & Movellan, J. R. (2007). Socialization between toddlers and robots at an early childhood education center. Proceedings of the National Academy of Sciences, 104, 17954–17958. doi:10.1073/pnas.0707769104.

    Article  Google Scholar 

  • Tedeschi, J. T., & Reiss, M. (1981). Verbal strategies as impression management. In C. Antaki (Ed.), The psychology of ordinary social behaviour (pp. 271–309). London: Academic Press.

    Google Scholar 

  • Thiessen, E. D., Kronstein, A. T., & Hufnagle, D. G. (2013). The extraction and integration framework: A two-process account of statistical learning. Psychological Bulletin, 139, 792–814. doi:10.1037/a0030801.

    Article  Google Scholar 

  • Tomasello, M., & Vaish, A. (2013). Origins of human cooperation and morality. Annual Review of Psychology, 64, 231–255. doi:10.1146/annurev-psych-113011-143812.

    Article  Google Scholar 

  • Traverso, V. (2009). The dilemmas of third-party complaints in conversation between friends. Journal of Pragmatics, 41, 2385–2399. doi:10.1016/j.pragma.2008.09.047.

    Article  Google Scholar 

  • Turiel, E. (1983). The development of social knowledge: Morality and convention. Cambridge, England: Cambridge University Press.

    Google Scholar 

  • Van Berkum, J. J. A., Holleman, B., Nieuwland, M., Otten, M., & Murre, J. (2009). Right or wrong? The brain’s fast response to morally objectionable statements. Psychological Science, 20, 1092–1099. doi:10.1111/j.1467-9280.2009.02411.x.

    Article  Google Scholar 

  • van Wynsberghe, A. (2013). Designing robots for care: Care centered value-sensitive design. Science and Engineering Ethics, 19, 407–433. doi:10.1007/s11948-011-9343-6.

    Article  Google Scholar 

  • Veloso, M., Aisen, M., Howard, A., Jenkins, O. C., Mutlu, B., & Scassellati, B. (2012). Human-robot interaction: Japan, South Korea, and China. WTEC Panel Report. Arlington, VA: World Technology Evaluation Center, Inc.

  • Veruggio, G., Solis, J., & Van der Loos, M. (2011). Roboethics: Ethics applied to robotics. IEEE Robotics Automation Magazine, 18, 21–22. doi:10.1109/MRA.2010.940149.

    Article  Google Scholar 

  • Voiklis, J., Cusimano, C., & Malle, B. F. (2014). A social-conceptual map of moral criticism. In P. Bello, M. Guarini, M. McShane, & B. Scassellati (Eds.), Proceedings of the 36th annual conference of the cognitive science society (pp. 1700–1705). Austin, TX: Cognitive Science Society.

    Google Scholar 

  • Walker, M. U. (2006). Moral repair: Reconstructing moral relations after wrongdoing. New York, NY: Cambridge University Press.

    Book  Google Scholar 

  • Wallach, W. (2010). Robot minds and human ethics: The need for a comprehensive model of moral decision making. Ethics and Information Technology, 12, 243–250. doi:10.1007/s10676-010-9232-8.

    Article  Google Scholar 

  • Wallach, W., & Allen, C. (2008). Moral machines: Teaching robots right from wrong. New York, NY: Oxford University Press.

    Google Scholar 

  • Warneken, F., Lohse, K., Melis, A. P., & Tomasello, M. (2011). Young children share the spoils after collaboration. Psychological Science, 22, 267–273. doi:10.1177/0956797610395392.

    Article  Google Scholar 

  • Weiner, B. (1995). Judgments of responsibility: A foundation for a theory of social conduct. New York, NY: Guilford Press.

    Google Scholar 

  • Williams, K. D. (2009). Ostracism: A temporal need-threat model. In M. P. Zanna (Ed.), Advances in experimental social psychology (Vol. 41, pp. 275–314). San Diego, CA: Elsevier Academic Press.

    Google Scholar 

  • Wolpert, D. M., & Flanagan, J. R. (2001). Motor prediction. Current Biology, 11, R729–R732. doi:10.1016/S0960-9822(01)00432-8.

    Article  Google Scholar 

  • Wright, J. C., & Bartsch, K. (2008). Portraits of early moral sensibility in two children’s everyday conversations. Merrill-Palmer Quarterly, 54, 56–85. doi:10.2307/23096079.

    Article  Google Scholar 

  • Wyman, E., Rakoczy, H., & Tomasello, M. (2009). Normativity and context in young children’s pretend play. Cognitive Development, 24, 146–155. doi:10.1016/j.cogdev.2009.01.003.

    Article  Google Scholar 

Download references

Acknowledgments

This project was partially supported by a grant from the Office of Naval Research (ONR), No. N00014-13-1-0269. The opinions expressed here are my own and do not necessarily reflect the views of ONR. The ideas on moral competence featured in this article have been developed jointly with Matthias Scheutz, Tufts University.

Author information

Authors and Affiliations

  1. Department of Cognitive, Linguistic, and Psychological Sciences, Brown University, 190 Thayer St., Providence, RI, 02912, USA

    Bertram F. Malle

Authors
  1. Bertram F. Malle
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Bertram F. Malle.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Malle, B.F. Integrating robot ethics and machine morality: the study and design of moral competence in robots. Ethics Inf Technol 18, 243–256 (2016). https://doi.org/10.1007/s10676-015-9367-8

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10676-015-9367-8

Keywords

  • Social cognition
  • Moral cognition
  • Human-robot interaction
  • Moral psychology
  • Social robotics