Advertisement

Ethics and Information Technology

, Volume 18, Issue 4, pp 243–256 | Cite as

Integrating robot ethics and machine morality: the study and design of moral competence in robots

  • Bertram F. Malle
Original Paper

Abstract

Robot ethics encompasses ethical questions about how humans should design, deploy, and treat robots; machine morality encompasses questions about what moral capacities a robot should have and how these capacities could be computationally implemented. Publications on both of these topics have doubled twice in the past 10 years but have often remained separate from one another. In an attempt to better integrate the two, I offer a framework for what a morally competent robot would look like (normally considered machine morality) and discuss a number of ethical questions about the design, use, and treatment of such moral robots in society (normally considered robot ethics). Instead of searching for a fixed set of criteria of a robot’s moral competence I identify the multiple elements that make up human moral competence and probe the possibility of designing robots that have one or more of these human elements, which include: moral vocabulary; a system of norms; moral cognition and affect; moral decision making and action; moral communication. Juxtaposing empirical research, philosophical debates, and computational challenges, this article adopts an optimistic perspective: if robotic design truly commits to building morally competent robots, then those robots could be trustworthy and productive partners, caretakers, educators, and members of the human community. Moral competence does not resolve all ethical concerns over robots in society, but it may be a prerequisite to resolve at least some of them.

Keywords

Social cognition Moral cognition Human-robot interaction Moral psychology Social robotics 

Notes

Acknowledgments

This project was partially supported by a grant from the Office of Naval Research (ONR), No. N00014-13-1-0269. The opinions expressed here are my own and do not necessarily reflect the views of ONR. The ideas on moral competence featured in this article have been developed jointly with Matthias Scheutz, Tufts University.

References

  1. Alicke, M. D. (2000). Culpable control and the psychology of blame. Psychological Bulletin, 126, 556–574. doi: 10.1037//0033-2909.126.4.556.CrossRefGoogle Scholar
  2. Allen, C. (2011). The future of moral machines. The New York Times: Opinionator. Retrieved December 29, 2014, from http://opinionator.blogs.nytimes.com/2011/12/25/the-future-of-moral-machines/.
  3. Anderson, M., & Anderson, S. (2011). Machine ethics. Cambridge: Cambridge University Press.CrossRefGoogle Scholar
  4. Antaki, C. (1994). Explaining and arguing: The social organization of accounts. London: Sage.Google Scholar
  5. Arkin, R. C. (2009). Governing lethal behavior in autonomous robots. Boca Raton, FL: CRC Press.CrossRefGoogle Scholar
  6. Asaro, P. M. (2006). What should we want from a robot ethic? International Review of Information Ethics, 6, 9–16.Google Scholar
  7. Avramova, Y. R., & Inbar, Y. (2013). Emotion and moral judgment. Wiley Interdisciplinary Reviews Cognitive Science, 4, 169–178. doi: 10.1002/wcs.1216.CrossRefGoogle Scholar
  8. Baumeister, R. F., & Leary, M. R. (1995). The need to belong: Desire for interpersonal attachments as a fundamental human motivation. Psychological Bulletin, 117, 497–529. doi: 10.1037/0033-2909.117.3.497.CrossRefGoogle Scholar
  9. Bello, P. (2012). Cognitive foundations for a computational theory of mindreading. Advances in Cognitive Systems, 1, 59–72.Google Scholar
  10. Bicchieri, C. (2006). The grammar of society: The nature and dynamics of social norms. New York, NY: Cambridge University Press.Google Scholar
  11. Blomkamp, N., Kinberg, S. (Producers), & Blomkamp, N. (Director). (2015).  Chappie [Motion picture]. USA: Sony Pictures Home Entertainment.  Google Scholar
  12. Brachman, R. J. (2002). Systems that know what they’re doing. IEEE Intelligent Systems, 17, 67–71. doi: 10.1109/MIS.2002.1134363.CrossRefGoogle Scholar
  13. Breazeal, C. L. (2002). Designing sociable robots. Cambridge, MA: MIT Press.zbMATHGoogle Scholar
  14. Bringsjord, S. (2009). But perhaps robots are essentially non-persons. Erwägen Wissen Ethik, 20, 193–195.Google Scholar
  15. Bringsjord, S., Arkoudas, K., & Bello, P. (2006). Toward a general logicist methodology for engineering ethically correct robots. Intelligent Systems, IEEE, 21, 38–44.CrossRefGoogle Scholar
  16. Calverley, D. J. (2006). Android science and animal rights, does an analogy exist? Connection Science, 18, 403–417. doi: 10.1080/09540090600879711.CrossRefGoogle Scholar
  17. Coates, D. J., & Tognazzini, N. A. (2012). The contours of blame. In D. J. Coates & N. A. Tognazzini (Eds.), Blame: Its nature and norms (pp. 3–26). New York, NY: Oxford University Press.CrossRefGoogle Scholar
  18. Coeckelbergh, M. (2010). Robot rights? Towards a social-relational justification of moral consideration. Ethics and Information Technology, 12, 209–221. doi: 10.1007/s10676-010-9235-5.CrossRefGoogle Scholar
  19. Cox, M. T. (2011). Metareasoning, monitoring, and self-explanation. In M. T. Cox & A. Raja (Eds.), Metareasoning (pp. 131–149). Cambridge, MA: The MIT Press.CrossRefGoogle Scholar
  20. Csibra, G., & Gergely, G. (2009). Natural pedagogy. Trends in Cognitive Sciences, 13, 148–153. doi: 10.1016/j.tics.2009.01.005.CrossRefGoogle Scholar
  21. Cushman, F. (2008). Crime and punishment: Distinguishing the roles of causal and intentional analyses in moral judgment. Cognition, 108, 353–380. doi: 10.1016/j.cognition.2008.03.006.CrossRefGoogle Scholar
  22. Cushman, F., & Young, L. (2011). Patterns of moral judgment derive from nonmoral psychological representations. Cognitive Science, 35, 1052–1075. doi: 10.1111/j.1551-6709.2010.01167.x.CrossRefGoogle Scholar
  23. DeBaets, A. M. (2014). Can a robot pursue the good? Exploring artificial moral agency. Journal of Evolution and Technology, 24, 76–86.MathSciNetGoogle Scholar
  24. Dersley, I., & Wootton, A. (2000). Complaint sequences within antagonistic argument. Research on Language and Social Interaction, 33, 375–406. doi: 10.1207/S15327973RLSI3304_02.CrossRefGoogle Scholar
  25. Eisenberg, N. (2000). Emotion, regulation, and moral development. Annual Review of Psychology, 51, 665–697.CrossRefGoogle Scholar
  26. Emde, R. N. (1992). Social referencing research: Uncertainty, self, and the search for meaning. In S. Feinman (Ed.), Social referencing and the social construction of reality in infancy (pp. 79–94). New York, NY: Plenum Press.CrossRefGoogle Scholar
  27. Fehr, E., & Fischbacher, U. (2004). Third-party punishment and social norms. Evolution and Human Behavior, 25, 63–87. doi: 10.1016/S1090-5138(04)00005-4.CrossRefGoogle Scholar
  28. Fisher, M., Spielberg, S., & Weaver, B. (2014). Extant [Television series]. Los Angeles: CBS.Google Scholar
  29. Fiske, S. T., & Taylor, S. E. (2008). Social cognition: From brains to culture (1st ed.). Boston, MA: McGraw-Hill.Google Scholar
  30. Flack, J. C., & de Waal, F. B. M. (2000). “Any animal whatever”. Darwinian building blocks of morality in monkeys and apes. Journal of Consciousness Studies, 7, 1–29.Google Scholar
  31. Floridi, L., & Sanders, J. W. (2004). On the morality of artificial agents. Minds and Machines, 14, 349–379. doi: 10.1023/B:MIND.0000035461.63578.9d.CrossRefGoogle Scholar
  32. Ford, K. M., & Hayes, P. J. (1991). Reasoning agents in a dynamic world: The frame problem. Greenwich, CT: JAI Press.Google Scholar
  33. Fridin, M. (2014). Kindergarten social assistive robot: First meeting and ethical issues. Computers in Human Behavior, 30, 262–272. doi: 10.1016/j.chb.2013.09.005.CrossRefGoogle Scholar
  34. Garcia, E., Jimenez, M. A., De Santos, P. G., & Armada, M. (2007). The evolution of robotics research. IEEE Robotics Automation Magazine, 14, 90–103. doi: 10.1109/MRA.2007.339608.CrossRefGoogle Scholar
  35. Gilovich, T., Keltner, D., & Nisbett, R. E. (2013). Social psychology (3rd ed.). New, NY: W.W. Norton & Co.Google Scholar
  36. Grau, C. (2011). There is no “I” in “Robot”: Robots and utilitarianism. In M. Anderson & S. L. Anderson (Eds.), Machine ethics (pp. 451–463). Cambridge: Cambridge University Press.CrossRefGoogle Scholar
  37. Gray, K., Young, L., & Waytz, A. (2012). Mind perception is the essence of morality. Psychological Inquiry, 23, 101–124. doi: 10.1080/1047840X.2012.651387.CrossRefGoogle Scholar
  38. Greene, J. D., Nystrom, L. E., Engell, A. D., Darley, J. M., & Cohen, J. D. (2004). The neural bases of cognitive conflict and control in moral judgment. Neuron, 44, 389–400. doi: 10.1016/j.neuron.2004.09.027.CrossRefGoogle Scholar
  39. Guglielmo, S., Monroe, A. E., & Malle, B. F. (2009). At the heart of morality lies folk psychology. Inquiry: An Interdisciplinary Journal of Philosophy, 52, 449–466. doi: 10.1080/00201740903302600.CrossRefGoogle Scholar
  40. Gunkel, D. J. (2014). A vindication of the rights of machines. Philosophy & Technology, 27, 113–132. doi: 10.1007/s13347-013-0121-z.CrossRefGoogle Scholar
  41. Haidt, J. (2001). The emotional dog and its rational tail: A social intuitionist approach to moral judgment. Psychological Review, 108, 814–834. doi: 10.1037/0033-295X.108.4.814.CrossRefGoogle Scholar
  42. Hamlin, J. K. (2013). Moral judgment and action in preverbal infants and toddlers: Evidence for an innate moral core. Current Directions in Psychological Science, 22, 186–193. doi: 10.1177/0963721412470687.CrossRefGoogle Scholar
  43. Harenski, C. L., Harenski, K. A., Shane, M. S., & Kiehl, K. A. (2010). Aberrant neural processing of moral violations in criminal psychopaths. Journal of Abnormal Psychology, 119, 863–874.CrossRefGoogle Scholar
  44. Heath, J. (2001). Communicative action and rational choice. Studies in contemporary German social thought. Cambridge, MA: MIT Press.Google Scholar
  45. Hilton, D. J. (2007). Causal explanation: From social perception to knowledge-based causal attribution. In A. W. Kruglanski & E. T. Higgins (Eds.), Social psychology: Handbook of basic principles (2nd ed., pp. 232–253). New York, NY: Guilford Press.Google Scholar
  46. Hoffman, M. L. (2008). Empathy and prosocial behavior. In M. Lewis, J. M. Haviland-Jones, & L. F. Barrett (Eds.), Handbook of emotions (3rd ed., pp. 440–455). New York, NY: Guilford Press.Google Scholar
  47. Hofmann, B. (2013). Ethical challenges with welfare technology: A review of the literature. Science and Engineering Ethics, 19, 389–406.CrossRefGoogle Scholar
  48. Huebner, B., Dwyer, S., & Hauser, M. (2009). The role of emotion in moral psychology. Trends in Cognitive Sciences, 13, 1–6. doi: 10.1016/j.tics.2008.09.006.CrossRefGoogle Scholar
  49. Hutcherson, C. A., & Gross, J. J. (2011). The moral emotions: A social—functionalist account of anger, disgust, and contempt. Journal of Personality and Social Psychology, 100, 719–737. doi: 10.1037/a0022408.CrossRefGoogle Scholar
  50. Johnson, A. M., & Axinn, S. (2013). The morality of autonomous robots. Journal of Military Ethics, 12, 129–141. doi: 10.1080/15027570.2013.818399.CrossRefGoogle Scholar
  51. Kahn, Jr., P. H., Kanda, T., Ishiguro, H., Gill, B. T., Ruckert, J. H., Shen, S., Gary, H. E., et al. (2012). Do people hold a humanoid robot morally accountable for the harm it causes?. Proceedings of the Seventh Annual ACM/IEEE International Conference on Human-Robot Interaction (pp. 33–40). New York, NY: ACM. doi: 10.1145/2157689.2157696.
  52. Kibble, R. (2012). Can an unmanned drone be a moral agent? Ethics and accountability in military robotics. In D. J. Gunkel, J. J. Bryson, & S. Torrance (Eds.), The machine question: AI, ethics and moral responsibility (Proceedings of symposium “Machine Question: AI, Ethics, and Moral Responsibility” AISB/IACAP 2012) (pp. 62–67). The Society for the Study of Artificial Intelligence and Simulation of Behaviour.Google Scholar
  53. Knobe, J. (2010). Person as scientist, person as moralist. Behavioral and Brain Sciences, 33, 315–329. doi: 10.1017/S0140525X10000907.CrossRefGoogle Scholar
  54. Knobe, J., & Fraser, B. (2008). Causal judgment and moral judgment: Two experiments. Moral psychology (Vol. 2): The cognitive science of morality: Intuition and diversity (Vol. 2, pp. 441–447). Cambridge, MA: MIT Press.Google Scholar
  55. Kohlberg, L. (1984). The psychology of moral development: The nature and validity of moral stages. San Francisco, CA: Harper & Row.Google Scholar
  56. Lin, P. (2013). The ethics of autonomous cars. The Atlantic. Retrieved September 30, 2014, from http://www.theatlantic.com/technology/archive/2013/10/the-ethics-of-autonomous-cars/280360/.
  57. Lin, P., Abney, K., & Bekey, G. A. (Eds.). (2012). Robot ethics: The ethical and social implications of robotics. Cambridge, MA: MIT Press.Google Scholar
  58. Littman, M. L. (2001). Value-function reinforcement learning in Markov games. Cognitive Systems Research, 2, 55–66. doi: 10.1016/S1389-0417(01)00015-8.CrossRefGoogle Scholar
  59. Lomas, M., Chevalier, R., Cross, E. V., Garrett, R. C., Hoare, J., & Kopack, M. (2012). Explaining robot actions. Proceedings of the 7th ACM/IEEE International Conference on Human-Robot Interaction (HRI) (pp. 187–188). Boston, MA.Google Scholar
  60. Luo, Q., Nakic, M., Wheatley, T., Richell, R., Martin, A., & Blair, R. J. R. (2006). The neural basis of implicit moral attitude—An IAT study using event-related fMRI. NeuroImage, 30, 1449–1457. doi: 10.1016/j.neuroimage.2005.11.005.CrossRefGoogle Scholar
  61. Malle, B. F. (1999). How people explain behavior: A new theoretical framework. Personality and Social Psychology Review, 3, 23–48. doi: 10.1207/s15327957pspr0301_2.CrossRefGoogle Scholar
  62. Malle, B. F. (2004). How the mind explains behavior: Folk explanations, meaning, and social interaction. Cambridge, MA: MIT Press.Google Scholar
  63. Malle, B. F. (2011). Time to give up the dogmas of attribution: A new theory of behavior explanation. In M. P. Zanna & J. M. Olson (Eds.), Advances of experimental social psychology (Vol. 44, pp. 297–352). San Diego, CA: Academic Press.Google Scholar
  64. Malle, B. F., & Dickert, S. (2007). Values. In R. F. Baumeister & K. D. Vohs (Eds.), The encyclopedia of social psychology. Thousand Oaks, CA: Sage.Google Scholar
  65. Malle, B. F., Guglielmo, S., & Monroe, A. E. (2014). A theory of blame. Psychological Inquiry, 25, 147–186. doi: 10.1080/1047840X.2014.877340.CrossRefGoogle Scholar
  66. Malle, B. F., & Scheutz, M. (2014). Moral competence in social robots. IEEE International Symposium on Ethics in Engineering, Science, and Technology (pp. 30–35). Presented at the IEEE International Symposium on Ethics in Engineering, Science, and Technology, June, Chicago, IL: IEEE.Google Scholar
  67. Malle, B. F., Scheutz, M., Arnold, T., Voiklis, J., & Cusimano, C. (2015). Sacrifice one for the good of many? People apply different moral norms to human and robot agents. HRI’15: Proceedings of the Tenth Annual ACM/IEEE International Conference on Human-Robot Interaction (pp. 117–124). New York, NY: ACM.Google Scholar
  68. McCullough, M. E., Kurzban, R., & Tabak, B. A. (2013). Putting revenge and forgiveness in an evolutionary context. Behavioral and Brain Sciences, 36, 41–58. doi: 10.1017/S0140525X12001513.CrossRefGoogle Scholar
  69. McKenna, M. (2012). Directed blame and conversation. In D. J. Coates & N. A. Tognazzini (Eds.), Blame: Its nature and norms (pp. 119–140). New York, NY: Oxford University Press.CrossRefGoogle Scholar
  70. MHAT-IV. (2006). Mental Health Advisory Team (MHAT) IV: Operation Iraqi Freedom 05-07 Final report. Washington, DC: Office of the Surgeon, Multinational Force-Iraq; Office of the Surgeon General, United States Army Medical Command.Google Scholar
  71. Mikhail, J. (2007). Universal moral grammar: Theory, evidence and the future. Trends in Cognitive Sciences, 11, 143–152. doi: 10.1016/j.tics.2006.12.007.CrossRefGoogle Scholar
  72. Milgram, S. (1963). Behavioral study of obedience. Journal of Abnormal and Social Psychology, 67, 371–378.CrossRefGoogle Scholar
  73. Millar, J. (2014). An ethical dilemma: When robot cars must kill, who should pick the victim? Robohub. Robohub.org. Retrieved September 28, 2014, from http://robohub.org/an-ethical-dilemma-when-robot-cars-must-kill-who-should-pick-the-victim/.
  74. Mithen, S. (Ed.). (1998). Creativity in human evolution and prehistory. New York, NY: Taylor & Francis.Google Scholar
  75. Monroe, A. E., Dillon, K. D., & Malle, B. F. (2014). Bringing free will down to earth: People’s psychological concept of free will and its role in moral judgment. Consciousness and Cognition, 27, 100–108. doi: 10.1016/j.concog.2014.04.011.CrossRefGoogle Scholar
  76. Monroe, A. E., & Malle, B. F. (2010). From uncaused will to conscious choice: The need to study, not speculate about people’s folk concept of free will. Review of Philosophy and Psychology, 1, 211–224.CrossRefGoogle Scholar
  77. Monroe, A. E., & Malle, B. F. (2014). Free will without metaphysics. In A. R. Mele (Ed.), Surrounding free will (pp. 25–48). New York, NY: Oxford University Press.CrossRefGoogle Scholar
  78. Moor, J. H. (2006). The nature, importance, and difficulty of machine ethics. IEEE Intelligent Systems, 21, 18–21. doi: 10.1109/MIS.2006.80.CrossRefGoogle Scholar
  79. Nisbett, R. E., & Wilson, T. D. (1977). Telling more than we know: Verbal reports on mental processes. Psychological Review, 84, 231–259.CrossRefGoogle Scholar
  80. Nourbakhsh, I. R. (2013). Robot futures. Cambridge, MA: MIT Press.Google Scholar
  81. Open Roboethics Initiative. (2014a). If death by autonomous car is unavoidable, who should die? Reader poll results.Google Scholar
  82. Open Roboethics Initiative. (2014b). My (autonomous) car, my safety: Results from our reader poll.Google Scholar
  83. Parthemore, J., & Whitby, B. (2013). What makes any agent a moral agent? Reflections on machine consciousness and moral agency. International Journal of Machine Consciousness, 4, 105–129.CrossRefGoogle Scholar
  84. Paxton, J. M., Ungar, L., & Greene, J. D. (2012). Reflection and reasoning in moral judgment. Cognitive Science, 36, 163–177. doi: 10.1111/j.1551-6709.2011.01210.x.CrossRefGoogle Scholar
  85. Petersen, S. (2007). The ethics of robot servitude. Journal of Experimental & Theoretical Artificial Intelligence, 19, 43–54. doi: 10.1080/09528130601116139.CrossRefGoogle Scholar
  86. Powell, N. L., Derbyshire, S. W. G., & Guttentag, R. E. (2012). Biases in children’s and adults’ moral judgments. Journal of Experimental Child Psychology, 113, 186–193. doi: 10.1016/j.jecp.2012.03.006.CrossRefGoogle Scholar
  87. Powers, T. M. (2006). Prospects for a Kantian machine. IEEE Intelligent Systems, 21, 46–51. doi: 10.1109/MIS.2006.77.CrossRefGoogle Scholar
  88. Powers, T. M. (2011). Incremental machine ethics. Robotics & Automation Magazine, IEEE, 18, 51–58. doi: 10.1109/MRA.2010.940152.CrossRefGoogle Scholar
  89. Pylyshyn, Z. W. (Ed.). (1987). The Robot’s dilemma: The frame problem in artificial intelligence. Norwood, NJ: Ablex.Google Scholar
  90. Ryle, G. (1949). The concept of mind. London: Penguin Press.Google Scholar
  91. Schank, R. C., & Abelson, R. P. (1977). Scripts, plans, goals, and understanding: An inquiry into human knowledge structures. Hillsdale, NJ: Erlbaum.zbMATHGoogle Scholar
  92. Scheutz, M. (2012). The affect dilemma for artificial agents: Should we develop affective artificial agents? IEEE Transactions on Affective Computing, 3, 424–433.CrossRefGoogle Scholar
  93. Scheutz, M., & Crowell, C. R. (2007). The burden of embodied autonomy: Some reflections on the social and ethical implications of autonomous robots. Proceedings of Workshop on Roboethics at ICRA 2007. Rome, Italy.Google Scholar
  94. Scheutz, M., Malle, B. F., & Briggs, G. (2015). Towards morally sensitive action selection for autonomous social robots. The 24th IEEE international symposium on robot and human interactive communication, 2015 RO-MAN. Presented at the 24th IEEE International Symposium on Robot and Human Interactive Communication. (2015). RO-MAN. Japan: Kobe.Google Scholar
  95. Scheutz, M., & Malle, B. F. (2014). “Think and do the right thing”: A plea for morally competent autonomous robots. Presented at the 2014 IEEE Ethics conference, Chicago, IL.Google Scholar
  96. Semin, G. R., & Manstead, A. S. R. (1983). The accountability of conduct: A social psychological analysis. London: Academic Press.Google Scholar
  97. Shaver, K. G. (1985). The attribution of blame: Causality, responsibility, and blameworthiness. New York: Springer.CrossRefGoogle Scholar
  98. Sullins, J. P. (2011). Introduction: Open questions in roboethics. Philosophy & Technology, 24, 233. doi: 10.1007/s13347-011-0043-6.CrossRefGoogle Scholar
  99. Talamadupula, K., Schermerhorn, P., Benton, J., Kambhampati, S., & Scheutz, M. (2011). Planning for agents with changing goals. ICAPS 2011 System Demonstration. Germany: Freiburg.Google Scholar
  100. Tanaka, F., Cicourel, A., & Movellan, J. R. (2007). Socialization between toddlers and robots at an early childhood education center. Proceedings of the National Academy of Sciences, 104, 17954–17958. doi: 10.1073/pnas.0707769104.CrossRefGoogle Scholar
  101. Tedeschi, J. T., & Reiss, M. (1981). Verbal strategies as impression management. In C. Antaki (Ed.), The psychology of ordinary social behaviour (pp. 271–309). London: Academic Press.Google Scholar
  102. Thiessen, E. D., Kronstein, A. T., & Hufnagle, D. G. (2013). The extraction and integration framework: A two-process account of statistical learning. Psychological Bulletin, 139, 792–814. doi: 10.1037/a0030801.CrossRefGoogle Scholar
  103. Tomasello, M., & Vaish, A. (2013). Origins of human cooperation and morality. Annual Review of Psychology, 64, 231–255. doi: 10.1146/annurev-psych-113011-143812.CrossRefGoogle Scholar
  104. Traverso, V. (2009). The dilemmas of third-party complaints in conversation between friends. Journal of Pragmatics, 41, 2385–2399. doi: 10.1016/j.pragma.2008.09.047.CrossRefGoogle Scholar
  105. Turiel, E. (1983). The development of social knowledge: Morality and convention. Cambridge, England: Cambridge University Press.Google Scholar
  106. Van Berkum, J. J. A., Holleman, B., Nieuwland, M., Otten, M., & Murre, J. (2009). Right or wrong? The brain’s fast response to morally objectionable statements. Psychological Science, 20, 1092–1099. doi: 10.1111/j.1467-9280.2009.02411.x.CrossRefGoogle Scholar
  107. van Wynsberghe, A. (2013). Designing robots for care: Care centered value-sensitive design. Science and Engineering Ethics, 19, 407–433. doi: 10.1007/s11948-011-9343-6.CrossRefGoogle Scholar
  108. Veloso, M., Aisen, M., Howard, A., Jenkins, O. C., Mutlu, B., & Scassellati, B. (2012). Human-robot interaction: Japan, South Korea, and China. WTEC Panel Report. Arlington, VA: World Technology Evaluation Center, Inc.Google Scholar
  109. Veruggio, G., Solis, J., & Van der Loos, M. (2011). Roboethics: Ethics applied to robotics. IEEE Robotics Automation Magazine, 18, 21–22. doi: 10.1109/MRA.2010.940149.CrossRefGoogle Scholar
  110. Voiklis, J., Cusimano, C., & Malle, B. F. (2014). A social-conceptual map of moral criticism. In P. Bello, M. Guarini, M. McShane, & B. Scassellati (Eds.), Proceedings of the 36th annual conference of the cognitive science society (pp. 1700–1705). Austin, TX: Cognitive Science Society.Google Scholar
  111. Walker, M. U. (2006). Moral repair: Reconstructing moral relations after wrongdoing. New York, NY: Cambridge University Press.CrossRefGoogle Scholar
  112. Wallach, W. (2010). Robot minds and human ethics: The need for a comprehensive model of moral decision making. Ethics and Information Technology, 12, 243–250. doi: 10.1007/s10676-010-9232-8.CrossRefGoogle Scholar
  113. Wallach, W., & Allen, C. (2008). Moral machines: Teaching robots right from wrong. New York, NY: Oxford University Press.Google Scholar
  114. Warneken, F., Lohse, K., Melis, A. P., & Tomasello, M. (2011). Young children share the spoils after collaboration. Psychological Science, 22, 267–273. doi: 10.1177/0956797610395392.CrossRefGoogle Scholar
  115. Weiner, B. (1995). Judgments of responsibility: A foundation for a theory of social conduct. New York, NY: Guilford Press.Google Scholar
  116. Williams, K. D. (2009). Ostracism: A temporal need-threat model. In M. P. Zanna (Ed.), Advances in experimental social psychology (Vol. 41, pp. 275–314). San Diego, CA: Elsevier Academic Press.Google Scholar
  117. Wolpert, D. M., & Flanagan, J. R. (2001). Motor prediction. Current Biology, 11, R729–R732. doi: 10.1016/S0960-9822(01)00432-8.CrossRefGoogle Scholar
  118. Wright, J. C., & Bartsch, K. (2008). Portraits of early moral sensibility in two children’s everyday conversations. Merrill-Palmer Quarterly, 54, 56–85. doi: 10.2307/23096079.CrossRefGoogle Scholar
  119. Wyman, E., Rakoczy, H., & Tomasello, M. (2009). Normativity and context in young children’s pretend play. Cognitive Development, 24, 146–155. doi: 10.1016/j.cogdev.2009.01.003.CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media Dordrecht 2015

Authors and Affiliations

  1. 1.Department of Cognitive, Linguistic, and Psychological SciencesBrown UniversityProvidenceUSA

Personalised recommendations