Machine ethicists have offered six reasons (found in the literature) in favor of and/or promoting the development of moral machines. These are not stand alone reasons; rather, they are often intertwined. Part of the reason it sounds so convincing (at first glance) is because of their interdependency rather than the strength of any reason on its own. Disentangling these reasons shows their dubious foundation and allows one to challenge the endeavor of machine ethics.
Robots with moral decision making abilities will become a technological necessity (Wallach 2007).
[Artificial Moral Agents] are necessary and, in a weak sense, inevitable (Colin Allen and Wallach 2011).
Machine ethicists claim that robots in morally salient contexts will not and cannot be avoided, i.e. their development is inevitable (Anderson and Anderson 2010; Moor 2006; Scheutz 2016; Wallach 2010).
First, what exactly is meant by morally salient contexts is unclear. For some researchers this would include contexts such as healthcare, elder care, childcare, sex, and or the military—where life and death decisions are being made on a daily (or hourly) basis (Arkin 2009; Lokhorst and van den Hoven 2011; Sharkey 2016; Sharkey 2008; Sharkey and Sharkey 2011; Sharkey et al. 2017; van Wynsberghe 2012). There is no question that robots are entering these service sectors. The International Federation for Robotics Executive Summary of 2016 tells us that “the total number of professional service robots sold in 2015 rose considerably by 25% to 41,060 units up from 32,939 in 2014” and “service robots in defense applications accounted for 27% of the total number of service robots for professional use sold in 2015”. Moreover, sales of medical robots increased by 7% from 2014 to 2015.Footnote 7
For others, morally salient context is much broader than a pre-defined space or institution:
any ordinary decision-making situation from daily life can be turned into a morally charged decision-making situation, where the artificial agent finds itself presented with a moral dilemma where any choice of action (or inaction) can potentially cause harm to other agents. (Scheutz, 2016, p. 516)
From the above quote Scheutz is saying that a morally charged situation can arise at any moment in the event that someone could be harmed through (in)action of a robot. This thin description of a morally charged decision making situation adds further ambiguity to the discussion, namely (1) what level of autonomy does the robot have, and (2) what definition of harm is Scheutz talking about? There seems to be an assumption being made in the above quote concerning the robot that the robot must make a choice for action or inaction and thus that the robot must be autonomous. According to Scheutz then any autonomous robot interacting with a human user that has the potential to harm its user should be endowed with moral reasoning capabilities. What would Scheutz have us do with industrial robots that possess divergent levels of autonomy, work with humans in their presence, and for which it has already been shown that the robots can bring serious harm or sometimes death, to humans? Scheutz’s position would imply that industrial robots as well ought to be developed into AMAs.
Consider also the definition of ‘harm’ that ought to be adopted. Is it only physical harm to the corporeal body and mind that is the object of discussion here and if so what about the robot’s, or AI algorithm’s, ability to collect, store and share information about its users in a home setting? Considering the real possibility that home robots will be connected to the Internet of Things (IoT) which holds the potential for hackers and/or for companies not related to the robotics company to access personal data from users. The harm that can come from the mis-appropriation of one’s data has proven to be noteworthy of late: people can be refused mortgage loans, stalked, blackmailed, harassed online, or worse. If harm is to be extended to include the risk of one’s digital information, and interaction with a machine that might cause harm demands that it be endowed with ethical reasoning capacities, then one must concede that every device that one interacts with in a day (your tv, phone, fridge, alarm clock, kettle, etc.) ought to have such capabilities. Thus, Scheutz’s position leads to the conclusion that any technology that one interacts with and for which there is a potential for harm (physical or otherwise) must be developed as an AMA and this is simply untenable.
Second, a distinction must be made between being in a morally charged situation, on the one hand, and being delegated a moral role on the other. Consider animals used for therapeutic purposes in an elderly care facility; one would never demand that a dog placed in this context would need to reason ethically because of its role in therapy and the potential for harm in this context. Indeed the dog would be trained to ensure a degree of safety and reliability when interacting with it but would the dog be a moral dog in the end?Footnote 8
With this thought in mind let us day the discussion will be limited to an examination of a morally salient context to contexts such as the military and healthcare, which are often thought of as morally salient, and agree that it is inevitable that robots will be placed within these contexts. In this case there is a different, more nuanced problem that can be put into the form of a dilemma: when placed in a morally salient context either machines will be delegated a moral role or they will not. If one chooses the first horn—that the machine is delegated a moral role—then one must accept that it is inevitable that machines will be delegated a moral role in addition to the inevitability of the machine being in this morally salient context. However, this is simply not the case. There are plenty of machines operating in morally salient contexts which have not been delegated a moral role and are providing a valuable service. Consider for example:
Corti is an AI which listens in on emergency phone calls and makes correlations between the breathing and speech patterns of a caller and the risk of heart attack (Peters 2018). The information from the AI is presented to the phone operator to assist in their decision making. In short, the AI is a support system for the decision making of the operator. This may be akin to the cancer detection AI or other more traditional technologies in the operating suite (e.g. electrocardiogram, respiratory monitor and so on). Corti is clearly in a morally salient context (life and death), yet the machine is not delegated a moral role. The human being is still in charge and holds all of the responsibility for decision making. If one agrees with machine ethicists then one should accept that it is inevitable that a moral role reserved for the human in this case will be assigned to the machine. While this is probably unnecessary and most likely harmful, the point is that there is simply no reason to believe that this is inevitable.
If, however, one takes the other horn of the dilemma then the claim is as follows: robots will inevitably be in morally salient contexts without being delegated a morally salient role. The problem with this is that there is little new here. Microwaves and coffee machines exist in the hospital with no need for moral reasoning capabilities; this horn should be of little interest to machine ethicists. In short, there is not any evidence to suggest that it is inevitable that there will be a need for machines with moral reasoning capabilities regardless of whether or not they function in a morally salient context.
Artificial Moral Machines to Prevent Harm to Humans
For many scholars the development of moral machines is aimed at preventing a robot from hurting human beings. To ensure that humans can overcome the potential for physical harm, a technological solution is presented; namely, to develop AMAs:
the only way to minimize human harm is to build morally competent robots that can detect and resolve morally charged situations in human-like ways (Scheutz 2016).
The line of reasoning here is pretty straight forward in that: “it is clear that machines…will be capable of causing harm to human beings” (Anderson and Anderson 2010) and this can be mitigated and/or reduced by endowing the robot with ethical reasoning capabilities. This also speaks to the interconnection of the reasons in favor of AMAs; robots are inevitable, robots could harm us, therefore robots should be made into AMAs.
It is unclear that AMAs are the solution to this problem, however. There are plenty of technologies capable of harming human beings (e.g. lawn mowers, automatic doors, curling irons, blenders); the solution has always been either to design them with safety features or to limit the contexts in which a technology can be used. An elevator door has a sensor so that it does not close on people; lawn mowers have a guard to protect us from their blades; and ovens have lights to warn us when our stovetop is hot. One does not normally use barbeques indoors or chainsaws in daycare centers. Machine ethicists are the first to suggest endowing technology with moral reasoning capabilities as a solution to problems of safety.
Furthermore, machine ethicists may also agree with the pursuit of safe robots and then the real concern for ethicists is that ethics is being reduced to safety. Notions such as values, rights, freedoms, good vs bad, right vs wrong, are central to the study of ethics and form the basis for a discussion of competing conceptions of the good life. One may believe that the values of safety and security are fundamental to achieving the good life; however, ethics cannot be reduced to these issues. So if AMAs are simply a solution to possibly harmful machines, then safety—not moral agency—is the object of debate.
In this case the world ‘moral’ is a linguistic ‘trojan horse’—a word
that smuggles in a rich interconnected web of human concepts that are not part of a computer system or how it operates (Sharkey 2012, p. 793)
The concept of moral machines or artificial moral agents invites, or more strongly requests that, the user believe the robot may care for him/her, or that the robot can experience feelings. For robot developers this could increase desirability of the robot and therefore profits. However, this is problematic for the public in that it invites a kind of fictive, asymmetric, deceptive relationship between human and robot.
Thus, machine ethicists must either distinguish what makes their machines “moral” above and beyond “safe” or they must stop using the world “moral” as the word is not appropriate—only the most reductionist account of morality would equate it with preventing harm.
as systems get more sophisticated and their ability to function autonomously in different contexts and environments expands, it will become more important for them to have ‘ethical subroutines’ of their own (Allen et al. 2006, p. 14)
The idea behind using complexity as an argument in favor of AMAs is that robots are, and will increasingly become, so complex in terms of their programming that it is no longer possible know what they will do in novel situations. This uncertainty results in the impossibility of the engineer to predict every scenario and as such it will not be possible for the engineer to predict the robot’s actions. Consequently, one cannot simply foresee a morally problematic situation and pre-program what the robot should do. Instead, authors who use the complexity argument to promote development of AMAs claim the robot needs to have moral competence in order to govern its unpredictable actions in the inevitably unpredictable and unstructured human environments that the robot will be placed.
First, using complexity as a reason for developing AMAs expects both that there will be complex robots and that such robots ought to be placed in contexts for which this complexity (i.e. unpredictability) could cause problems for human beings.
Next, of importance for this issue is the context within which the robot will be placed. In other words, this problem can be mitigated simply by restricting the context within which these machines are used. For example, designers of Google’s complex machine AlphaGo may not have any idea what their machine will do next (which move it will make in the notoriously difficult game of GO); however, this is not an ethical or moral problem because the context (the game of GO) is restricted. Its complexity will not pose a problem for us.
One may argue that human beings are unpredictable and can cause harm to other human beings. The solution has not been to prevent the delegation of moral roles to human beings. One might ask: why treat machines differently? While it is outside of the scope of this paper to engage in a debate on just how predictable humans are, it can be noted that with regard to serious moral values—killing, non-consensual sex, harming innocent people for fun—society places restrictions on unpredictable human beings (i.e. imprisonment). Humans may be unpredictable in terms of what they will do next, but most of us assume that a random person will not intentionally cause us harm.
Other machine ethicists argue that making AMAs will increase public trust: “Constructing artificial moral agents serves at least two purposes: one, better understanding of moral reasoning, and, two, increasing our trust and confidence in creating autonomous agents acting on our behalf” (Wiegel 2006). There has been talk in the media expressing concerns surrounding AI and robotics—voiced by the likes of Elon Musk and Steven Hawking (Cellan-Jones 2014; Markoff 2015). Rather than preventing the development of robots that are the source of these fears “machine ethics may offer a viable, more realistic solution” (Anderson and Anderson 2007).
This line of thinking assumes that if robots are given moral competence then this will put the public at ease and lead to public acceptance. It should be noted here that acceptance differs from acceptability. As an example, the public may accept geo tagging and tracking algorithms on their smartphone devices but this does not meant that such privacy breaching technologies and/or the lack of transparency about their existence are acceptable practices for upholding societal values.
Some important clarifications are needed when discussing trust as a concept. Traditionally speaking, trust is described as an interaction between persons or between a person and an institution and so on. For scholar John Hardwig trust can be placed in people, processes and in knowledge (Hardwig 1991). In more recent years scholars are discussing a new form of trust; trust in algorithms (Simon 2010). This new form of trust is most commonly referred to as ‘algorithmic authority’ and is described as a practice of placing confidence in the decisions made by an algorithm (Shirky 2009). Wikipedia is an example of this form of trust as it requires trust not in persons but in the algorithms regulating the content on the website.Footnote 9
If trust is broken the result will be feelings of disappointment on the part of the truster. These resulting negative feelings are what relate trust to the concept of reliability: if either one is misplaced the result is oftentimes feelings of disappointment (Simon 2010). Trust is distinguished from reliability in the intensity of the emotions experienced afterwards; “trust differs from reliance because if we are let down we feel betrayed and not just disappointed” (Baier 1986; Simon 2010). Relatedly, Simon claims that one cannot speak of trust for socio-technical systems but rather of reliance: “we usually do not ascribe intentionality to unanimated objects, which is why we do not feel betrayed by them” (p 347). Hence, we do not trust unanimated objects, we rely on them.
With the formulation of Hardwig in mind—trust can be placed in people, processes, and in knowledge and when it concerns placing trust in robots one must ask: who, or what, are machine ethicists asking the public to trust: the algorithm directing the robot; the designer; or, the development process?
If the public is being asked to trust the algorithm then one must consider that:
unfortunately we often trustFootnote 10 algorithms blindly. Algorithms are hidden within a system. In most cases we are not aware of how they work and we cannot assess their impact on the information we receive. In other words: algorithms are black-boxed (Simon 2010).
Consequently, if the public is being asked to trust an algorithm and it is considered a black-box, then, as Simon rightly asserts, it must be opened—the way it works, the decisions made in its development, and alternatives—must be made transparent and subject to scrutiny.
If, however, the public is being asked to trust the designer then designers and developers ought to develop a code of conduct (perhaps in the form of soft law) to adhere to. Again, transparency of this is required for the public to have the knowledge required for trust.
Last, if the public is being asked to trust the process through which the robot is being developed, a kind of procedural trust, then standards and certifications must be developed to once again provide the user with the knowledge required to place trust in the process through which the robot was developed. Examples of such procedural trust are FairTrade, ISO, GMOs, and so on.
In any case it is important to point out the inconsistency between the promotion of AMAs for reasons of complexity and for reasons of trust: it is inconsistent to expect unpredictability in a machine and to expect trust in a machine at the same time. While this may not be the case for people—one might trust persons who are at the same time unpredictable—more clarity is needed in understanding who/what society is being asked to trust and what level of (un)predictability one can assume.
Preventing Immoral Use
In the 2012 American science fiction comedy-drama movie “Robot & Frank” there is a compelling story of how a retired cat burglar convinces his robot to help him enter the business once again. The story raises the question about human–robot interaction not in the sense of safe or reliable interactions but rather should the robot be capable of evaluating a human’s request for action. Thus, another reason put forward for the development of AMAs can be stated as: preventing humans from misusing, or inappropriately using, a robot requires that the robot be developed as a moral machine and can thus prevent misuse of itself, itself.
The main problem with this reason has to do with the potential to constrain the autonomy of humans. It’s not always clear what is the good thing to do and oftentimes context is required for this (Miller et al. 2017). Consider for example a couple who are at home having a few drinks together, domestic violence ensues after a heated argument, the women tries to get away in her car but the breathalyzer in her car picks up the alcohol in her system and the car won’t start. What would be the right thing to do in such an instance? How should the device be programmed? A deontologist would have us believe that one should never get behind the wheel after consuming alcohol, while a utilitarian might suggest that if the woman’s life is saved in doing so the overall good is maximized (unless of course the women were to get into a car accident and harm two others). These are not easy problems to solve and require particular details of context and individuals.
Consider another example where misuse is unclear. If an elderly person at home wants to have a fourth glass of wine and asks his/her robot to fetch it. If the robot fetches the wine, is the robot being misused in so far as it is contributing to poor health choices of the user? Or is the robot ‘good’ in so far as it fulfilled the request of its user. Presenting scenarios like these is meant to show the difficulty in determining the right or the good thing to do. And yet if one is claiming that robots should be involved in the decision making procedure it must be very clear how a ‘good’ robot is distinguished from a ‘bad’ one.
Morality: Better with Moral Machines
Endowing the robot with the capability to override or edit a human’s decisions draws us into the discussion of the robot as a superior moral reasoner to a human. Computer science Professor James Gips suggested back in 1994 that “not many human beings live their lives flawlessly as moral saints. But a robot could (Gips 1994, p. 250). Also along the same lines, Professor of Philosophy Eric Dietrich has suggested that:
humans are genetically hardwired to be immoral…let us – the humans – exit the stage, leaving behind a planet populated with machines who, although not perfect angels, will nevertheless be a vast improvement over us (Dietrich 2001)
The assumption here is that a robot could be better at moral decision making than a human given that it would be impartial, unemotional, consistent, and rational every time it made a decision. Thus, no decisions would be based on bias or emotions, no decision would be the result of an affinity towards one person (or group of people) over another. More importantly, the robot would never tire but would have the energy to be consistent in decision making: to make the same choice time after time.
This line of reasoning to promote AMAs is also often invoked when speaking of robots in military contexts. In particular computer scientist/roboticist Ronald Arkin discusses the power of autonomous military robots for overcoming the shortcomings of humans on the battlefield (Arkin 2009). These robots would not rape or pillage the villages taken over during wartime and would be programmed as ethical agents according to the Laws of Just war and/or the Rules of Engagement.
There are some general concerns with this reason. First, the underlying programming which will enable machines to reason morally implies that one has an understanding of moral epistemology such that one can program machines to “learn” the correct moral truths—or at least know enough to have AMAs learn something that works. This gets complicated as there is no moral epistemology which does not have serious philosophical objections and therefore presents a barrier to being reduced to a programming language.
Machines could only be better if there is some standard of moral truth with which to judge. This implies that there are objective moral truths in a moral realist sense and further that it is possible to know what they are. This is opposed to error theory (the idea that there are no moral truths at all—so nothing to know), and moral skepticism (there are moral truths, but it is not possible that we as humans can know them).
Furthermore, based on the above quotes it seems that the moral truths that machines would be better at knowing are truths which are independent of human attitudes. Russ Shafer-Laundau calls these stance-independent moral truths (Shafer-Landau 1994). If—and that is a big if—there are stance independent moral truths whereby the truths have no dependence upon human desires, beliefs, needs, etc. then there are objections to how one could come to know such truths (Finlay 2007). If a machine was built which did somehow discover moral truths that have heretofore yet to be discovered (because morality would be a lot easier if we simply knew the moral truths) then one would have to accept on faith that machines are better than we are.
The moral consistency promised by machine ethicists is only a public good if the moral truths are known in advance—the opposite of the situation human beings find themselves in. For, as shown in previous sections, AMAs are argued to be needed because one cannot predict the kind of situations or moral dilemmas they will be faced with. But this is not a chess game where the outcome is a win or a loss. An autonomous car which drives off a cliff—killing its one passenger—in order to save five passengers in another car would not be a clear cut situation that everyone could agree was the correct decision. Indeed, books are written about that very decision and human disagreement about what should be done (i.e. the trolley problem) (see e.g. Greene 2013).
Lastly this all presumes that human emotions, human desires, and our evolutionary history are all getting in the way of our moral reasoning—causing it to be worse than it could be. There are those who include moral emotions as a necessary part of moral judgment and reasoning (Kristjánsson 2006; Pizarro 2000; Roeser 2010). If this is so, then AMAs would require emotions—something not even on the horizon of AI and robotics.
Let us say that there are moral principles and that humans can know what they are. So there is a standard with which to judge AMAs. Furthermore, let us also assume they live up to their promise and are better moral reasoners than humans. It might then make sense to outsource our moral decisions to machines. This would assume that being good at moral reasoning is not a necessary part of a human being’s good life. Aristotle believed leading a moral life and gaining a moral understanding through practice was necessary to leading a good life (Aristotle et al. 1998). Many contemporary philosophers agree. Outsourcing our moral reasoning to machines could cause an undesirable moral deskilling in human beings (Vallor 2015). The point is that it is not clear at all if machines were better moral reasoners than us that this would be a good reason to use them. Added to this, to make such an assumption is to assume we have an understanding of morality and the good life that we may not.
Better Understanding of Morality
Finally, machine ethicists sometimes argue that developing robots with moral reasoning capabilities will ultimately lead to a better understanding of human morality:
the hope is that as we try to implement ethical systems on the computer we will learn much more about the knowledge and assumptions built into the ethical theories themselves. That as we build the artificial ethical reasoning systems we will learn how to behave more ethically ourselves (Gips 1994)
In short, regardless of the resulting machine the very process of attempting to create such a machine would benefit humans in so far as we would learn about ourselves and our moral attributes (Gips 1994; Moor 2006; Wiegel 2006).
The most important consideration in response to this claim is that ethical theories are not (and have little to do with) how people reason morally so the work doesn’t help understand human morality. Experiments in moral psychology show us that human morality is deeply influenced by irrelevant situational factors (Doris 1998; Merritt 2000), is driven by emotion (Haidt 2001; Haidt and Joseph 2008), and influenced by our evolutionary past (Street 2006). To be sure, there is an intense debate in the literature with regard to each of these studies. The point is that human morality, in the descriptive sense, is dependent upon many complex factors and building a machine that tries to perfectly emulate human morality must use each of these factors combined rather than rely on ethical theory alone.