Autonomous Driving and Perverse Incentives
This paper discusses the ethical implications of perverse incentives with regard to autonomous driving. We define perverse incentives as a feature of an action, technology, or social policy that invites behavior which negates the primary goal of the actors initiating the action, introducing a certain technology, or implementing a social policy. As a special form of means-end-irrationality, perverse incentives are to be avoided from a prudential standpoint, as they prove to be directly self-defeating: They are not just a form of unintended side effect that must be balanced against the main goal or value to be realized by an action, technology, or policy. Instead, they directly cause the primary goals of the actors—i.e., the goals that they ultimately pursue with the action, technology, or policy—to be “worse achieved” (Parfit). In this paper, we elaborate on this definition and distinguish three ideal-typical phases of adverse incentives, where only in the last one the threshold for a perverse incentive is crossed. In addition, we discuss different possible relevant actors and their goals in implementing autonomous vehicles. We conclude that even if some actors do not pursue traffic safety as their primary goal, as part of a responsibility network they incur the responsibility to act on the common primary goal of the network, which we argue to be traffic safety.
KeywordsAutonomous driving Perverse incentives Robot ethics Responsibility networks
- Borenstein, J., Howard, A., & Wagner, A. (2017). Pediatric robotics and ethics: The robot is ready to see you now, but should it be trusted? In P. Lin, K. Abney, & R. Jenkins (Eds.), Robot ethics 2.0: New challenges in philosophy, law, and society (pp. 127–141). Oxford: Oxford Univ. Press.Google Scholar
- Boudette, N. (2017). Tesla's self-driving system cleared in deadly crash. The new Yorck times. https://www.nytimes.com/2017/01/19/business/tesla-model-s-autopilot-fatal-crash.html?_r=0. Accessed 27 February 2017.
- Dennett, D. (1989). The intentional stance. Cambridge Mass: MIT Press.Google Scholar
- ERTRAC. (2017). Automated Driving Roadmap: Version 7.0. http://www.ertrac.org/uploads/documentsearch/id48/ERTRAC_Automated_Driving_2017.pdf. Accessed 25 November 2017.
- Gibson, J. J. (1986). The ecological approach to visual perception. New York: Psychology Press.Google Scholar
- Goddard, T., Dill, J., & Monsere, C. (2016). Driver Attitudes About Bicyclists: Negative Evaluations of Rule-Following and Predictability. Transportation Research Board 95th Annual Meeting.Google Scholar
- Golson, J. (2017). Driver in fatal tesla autopilot crash had seven seconds to take action. The Verge. http://www.theverge.com/2017/1/19/14326604/tesla-autopilot-crash-driver-seven-seconds-inattentive-nhtsa. Accessed 23 March 2017.
- Gosepath, S. (1992). Aufgeklärtes Eigeninteresse: Eine Theorie theoretischer und praktischer Rationalität. Frankfurt/Main: Suhrkamp.Google Scholar
- Kirkpatrick, J., Hahn, E., & Haufler, A. (2017). Trust and human-robot interactions. In P. Lin, K. Abney, & R. Jenkins (Eds.), Robot ethics 2.0: New challenges in philosophy, law, and society (pp. 142–156). Oxford: Oxford Univ. Press.Google Scholar
- Kyriakidis, M., Happee, R., & de Winter, J. C. F. (2015). Public opinion on automated driving: Results of an international questionnaire among 5000 respondents. Transportation Research Part F: Traffic Psychology and Behaviour, 32, 127–140. https://doi.org/10.1016/j.trf.2015.04.014.CrossRefGoogle Scholar
- Levin, S., & Woolf, N. (2016, July 1). Tesla driver killed while using autopilot was watching Harry potter, witness says. The Guardian. https://www.theguardian.com/technology/2016/jul/01/tesla-driver-killed-autopilot-self-driving-car-harry-potter. Accessed 2 June 2018.
- Lin, P. (2013). The ethics of saving lives with autonomous cars is far murkier than you think. Wired. https://www.wired.com/2013/07/the-surprising-ethics-of-robot-cars/. Accessed 16 August 2017.
- Lin, P. (2014, August 18). Here’s a Terrible Idea: Robot Cars with Adjustable Ethics Settings. Wired. http://www.wired.com/2014/ 8/heres-a-terrible-idea-robotcars-with-adjustable-ethics-settings/. Accessed 30 November 2017.
- Lin, P. (2016). Is tesla responsible for the deadly crash on auto-pilot? Maybe. Forbes. https://www.forbes.com/sites/patricklin/2016/07/01/is-tesla-responsible-for-the-deadly-crash-on-auto-pilot-maybe/#23ec768b1c07. Accessed 27 February 2017.
- Lin, P., Abney, K., & Jenkins, R. (Eds.). (2017). Robot Ethics 2.0: New Challenges in Philosophy, Law, and Society. Oxford: Oxford Univ. Press.Google Scholar
- Loh, J., & Loh, W. (2017). Autonomy and responsibility in hybrid systems: The example of autonomous cars. In P. Lin, K. Abney, & R. Jenkins (Eds.), Robot ethics 2.0: New challenges in philosophy, law, and society. Oxford UK: Oxford Univ. Press.Google Scholar
- Madigan, R., Louw, T., Dziennus, M., Graindorge, T., Ortega, E., Graindorge, M., et al. (2016). Acceptance of automated road transport systems (ARTS): An adaptation of the UTAUT model. Transportation Research Procedia, 14, 2217–2226. https://doi.org/10.1016/j.trpro.2016.05.237.CrossRefGoogle Scholar
- Meyer, G., & Beiker, S. (Eds.). (2016). Road Vehicle Automation 3, Lecture notes in mobility. Cham: Springer International Publishing; Imprint; Springer.Google Scholar
- Misselhorn, C. (2013). Robots as Moral Agents. In F. Rövekamp & B. Friederike (Eds.), Ethics in Science and Society: German and Japanese Views (pp. 30–42). München: Iudicum.Google Scholar
- NHTSA. (2014). Fatalities by State and Road Function Class. https://www-fars.nhtsa.dot.gov/States/StatesCrashesAndAllVictims.aspx. Accessed 23 April 2017.
- Norman, D. A. (1988). The psychology of everyday things. New York, NY: Basic Books.Google Scholar
- NTMVSA. (1966). National Traffic and motor vehicle safety act: Public law 89–563.Google Scholar
- Nyholm, S. (2017). Attributing agency to automated systems: Reflections on human–robot collaborations and responsibility-loci. Science and Engineering Ethics.Google Scholar
- Nyholm, S., & Smids, J. (2016). The ethics of accident-algorithms for self-driving cars: An applied trolley problem? Ethical Theory and Moral Practice, 19, 1275–1289n.d.. https://doi.org/10.1007/s10677-016-9745-2.
- Parfit, D. (1984). Reasons and persons. Oxford: Oxford Univ. Press.Google Scholar
- Peden, M., Scurfield, R., Sleet, D., Mohan, D., Hyder, A., Jarawan, E., et al. (2004). World report on road traffic injury prevention. Geneva. http://cdrwww.who.int/violence_injury_prevention/publications/road_traffic/world_report/intro.pdf. Accessed 30 November 2017.
- Robinette, P., Li, W., Allen, R., Howard, A. M., & Wagner, A. R. (2016). Overtrust of Robots in Emergency Evacuation Scenarios (pp. 101–108). Piscataway NJ: IEEE Press.Google Scholar
- Roeser, S., Hillerbrand, R., Sandin, P., & Peterson, M. (Eds.). (2012). Handbook of risk theory. Dordrecht: Springer Netherlands.Google Scholar
- Rogers, E. M. (1962). Diffusion of innovations. New York: Free Press of Glencoe.Google Scholar
- SAE International. (2016). Taxonomy and definitions for terms related to driving automation systems for on-road motor vehicles. Washington DC: SAE International.Google Scholar
- Santoni de Sio, F. (2016). Ethics and self-driving cars: A white paper on responsible innovation in automated driving systems. Dutch Ministry of Infrastructure and Environment Rijkswaterstaat.Google Scholar
- Scanlon, T. (1998). What we owe to each other. Cambridge MA: Belknap Press of Harvard Univ. Press.Google Scholar
- Scanlon, T. (2007). Structural irrationality. In G. Brennan, R. Goodin, F. Jackson, & M. Smith (Eds.), Common minds: Themes from the philosophy of Philip Pettit (pp. 84–103). Oxford: Clarendon.Google Scholar
- Searle, J. (1990). Collective intentions and actions. In P. Cohen, J. Morgan, & M. Pollack (Eds.), Intentions in communication (pp. 401–415). Cambridge MA: MIT Press.Google Scholar
- Siebert, H. (2001). Der Kobra-Effekt: Wie man Irrwege der Wirtschaftspolitik vermeidet. München: Deutsche Verlags-Anstalt.Google Scholar
- Smith, M. (1994). The moral problem, Philosophical theory. Oxford: Blackwell.Google Scholar
- Stilgoe, J. (2017). Tesla crash report blames human error - this is a missed opportunity. The Guardian. https://www.theguardian.com/science/political-science/2017/jan/21/tesla-crash-report-blames-human-error-this-is-a-missed-opportunity. Accessed 23 March 2017.
- Taurek, J. (1977). Should the numbers count? Philosophy and Public Affairs, 6(4), 293–316.Google Scholar
- Tennant, C., Howard, S., Franks, B., Bauer, M., Stares, S., Pansegrau, P., et al. (2016). Autnonomous Vehicles - Negotiating a Place on the Road: A study on how drivers feel about Interacting with Autonomous Vehicles on the road. http://www.lse.ac.uk/website-archive/newsAndMedia/PDF/AVs-negociating-a-place-on-the-road-1110.pdf. Accessed 16 August 2017.
- Tesla Motors. (2016). A Tragic Loss. https://www.tesla.com/blog/tragic-loss. Accessed 23 March 2017.
- Weber, M. (1978). Economy and Society. Berkeley CA: University of California Press.Google Scholar
- Winter, J., Happee, R., Martens, M. H., & Stanton, N. A. (2014). Effects of adaptive cruise control and highly automated driving on workload and situation awareness: A review of the empirical evidence. Transportation Research Part F: Traffic Psychology and Behaviour, 27, 196–217. https://doi.org/10.1016/j.trf.2014.06.016.CrossRefGoogle Scholar