We proceed now to examine and address some likely objections against our proposal. Some of these objections can also be raised against Firth’s theory, while others only apply to the AMA. We will either argue that the objection in question is not valid or not strong enough; or we will take the point of the objection and use it to refine our proposal by suggesting how the AMA could be modified to address the objection.
Objection (i): the AMA Would Return Counterintuitive or Overdemanding Responses
Presumably, the AMA would at times return responses that are counterintuitive. This is only to be expected, if the AMA is meant to make up for the limitations of our intuitive and emotive moral psychology. One might think that the AMA would be of no use in such cases, because agents would not be willing to endorse or act upon a counterintuitive moral judgment. However, in this way the AMA might prompt us to balance our intuitions against the piece of advice, and vice versa, to attain a condition of “reflective equilibrium,” which can be understood in the narrow or in the wide sense. In the narrow sense, reflective equilibrium requires a certain degree of coherence between one’s considered judgments and principles within a certain moral perspective (for example, an egalitarian, or utilitarian, or religious perspective). In its wide sense, reflective equilibrium requires balancing different moral perspectives against one another (e.g., a utilitarian against an egalitarian or a deontological or a religious) (Daniels 2013, Mikhail 2010, Rawls 2001). However, most of us are incapable of balancing different competing fundamental principles; for example, recent findings in moral psychology suggest that liberals and conservatives base their moral approaches on different sets of intuitive and emotive foundations almost inaccessible to reason (Haidt 2012, Haidt and Graham 2007).
Here is where an apparent objection to AMA—i.e., the possibility of counterintuitive responses—turns out to be one of its strengths. The AMA’s response would be an external contribution towards reaching a condition of reflective equilibrium. More precisely, the AMA could help reach a narrow reflective equilibrium because the counterintuitive response would introduce a new element in the dynamics through which we balance our considered judgments and our principles. A counterintuitive response from the AMA would prompt us to question our considered judgments or some of our principles, adjust either or both according to the counterintuitive responses, and/or adjust the counterintuitive response from the AMA, for example by taking an attenuated version of it.
Here is an example. If I want to be altruistic in a utilitarian way, i.e., in a way that is maximally effective and impartial (Singer 2011), I would be often advised by the AMA to do things that my intuitions would deem supererogatory, like giving most of my income to charities. In such cases, I might want to slightly adjust my moral principles and/or considered judgments in a more altruistic direction, e.g., convince myself that giving a substantial part of my income to charities is a moral duty, and/or take a milder version of AMA’s advice and agree to give, say, just a third, rather than a half, of my income to charities.
The contrast between the ideal observer’s advice and my intuitive judgment could also help us achieve a wide reflective equilibrium. The counterintuitive responses can lead to question our fundamental moral views and balance them not only against the counterintuitive responses, but also against alternative moral views. For example, the counterintuitive advice provided by the AMA instructed with utilitarian operational criteria might convince me that after all, utilitarianism is not a moral theory I want to subscribe to because of its overdemandingness. Operational criteria consistent with different moral theories or approaches could be memorized by the AMA and used for future decisions, so that the pieces of advice I would eventually receive would be closer to the balance of different moral views that are consistent with my considered judgments.
Thus, a counterintuitive response resulting from utilitarian operational criteria would lead me to adjust my general moral views towards a milder version consistent with less counterintuitive particular judgments, for example, with a an “easy rescue” utilitarianism that would require maximizing utility as long as this is not achieved at too high a cost for me (Savulescu 2007), or to adopt a different moral perspective or a mix of elements from different moral perspectives encompassing, say, deontological or religious elements alongside utilitarian ones.
In addition, the AMA could prompt certain kind of reflection or empathic engagement, through using personal stories, media, games, etc.
Objection (ii): Dangers of Relativism
Because we have hypothesized that the agent provides the operational criteria, the AMA would work equally well regardless of whether these criteria are ethical or unethical, egoistic or altruistic, of a saint or a psychopath. The contribution the AMA would give us towards adjusting our principles and judgments—discussed in the previous section—would not make us more moral unless we already have an at least basic commitment to be moral, which many people do not have.
This objection should be taken into account when programming the AMA. We propose that the AMA, while accepting the widest range of moral criteria possible, should be programmed with some basic “moral filters” that constrain the range of possible operational criteria to be used as input. We need not take a stand on the issue whether moral relativism is true or false to justify adding such constraints. For one, relativism is not the same as the view that any possible moral principle is as valid, good, true as any other (at the metaethical level), or that any moral view should be equally respected (at the normative level). Although relativism can be defined as the theory that the truth of moral judgments and principles is relative to a certain culture or group, it is not incompatible with the view that some systems of morality—understood as agreements among members of a community—are better than others (Harman 1975, 4; Wong 2006). Besides, and more importantly at the practical level, we need to put constraints on people’s behavior regardless of whether we think there are objective standards of ethics, because there are pragmatic or political ends—most notably regulating the cohabitation of different people—whose justification need not be based on metaethical theories. Western society is already organized around some basic principles—such as reciprocal respect, tolerance, protection of persons’ lives—which are enforced regardless of whether some people or moral systems acknowledge them. The AMA would simply follow the same approach we already adopt in shaping the institutions of liberal societies. We could make sure that some basic moral requirements are met while at the same time allowing people to use the AMA according to their different moral criteria. For example, the AMA could be programmed to avoid advising about killing people or stealing. As we suggested above, a pool of moral experts could be consulted when programming the AMA to make sure that its advice is always consistent with such basic moral principles.
To be sure, there would be cases in which killing and stealing are morally acceptable or even, one might think, morally obligatory (for instance to defend others, to save someone from starvation). These circumstances however would be very rare, and the fact that the AMA would not be reliable in such cases does not detract from its usefulness in the majority of our everyday ethical decisions that do not involve such exceptional actions as killing or stealing for a greater good. As long as human agents are informed that the AMA cannot advise about how and when to kill, and that they can autonomously decide whether or not to take the AMA’s advice, the situation in such circumstances would not be worse than it would be if the AMA were not used. Also, as technology evolves and software becomes more sophisticated, we might have an AMA capable of distinguishing those cases in which the basic moral constraints can be permissibly waived—for example, killing someone who is about to blow up a building in order to save many lives—and those in which it cannot—for example, killing in a terroristic attack.
While we believe ethical relativism is not a defensible position, beginning with a relativistic AMA would be a practical first step towards non-relativism. It would engage the individual’s own ethical code but develop that by showing the full consequences and circumstances of action, while also suggesting alternative courses of actions or moral values. In more full blown versions, it could be programmed to suggest what moral paragons or leaders had chosen in similar situations, striving towards at least a rational intersubjectivity. The AMA could even facilitate development towards more objectivist moral codes by engaging with an individual’s own morality and psychology. Ideally, it would function as a moral adviser and perhaps even persuader.
Objection (iii): The AMA Does Not Take Emotions into Due Consideration
As a matter of fact, intuitions and emotions drive most of our moral and practical decision-making. “Emotionism,” i.e., the idea that emotions are in some way essential to morality (Prinz 2007), is not only a philosophical view (in the form of an emotivist metaethical theory), but also a psychological and neuroscientific theory which has received a lot of empirical support in recent years. Humans base most of their practical judgments on “affect heuristics,” i.e., intuitive and emotive, rapid responses (Finucane et al. 2000; Gilovich and Griffin 2002). Objects, events, and possible outcomes are often tagged with different, readily available emotions (positive or negative), which serve as cues for many important judgments we make and for which mental resources or time availability are likely to be limited. According to recent theories in moral psychology, our sense of morality is based on a limited set of intuitive and emotive responses, where principles and reasons are, more often than not, mere post-hoc rationalizations (Cushman et al. 2003, Haidt and Graham 2007, Haidt 2012). In general terms, humans “are effort-minimizing information processors inclined to adopt the simplest and least effortful processing strategy” (Forgas 1995, 46).
Consider disgust, for instance. Disgust was originally a physiological mechanism that served as a defense from poisons and parasites. It has been suggested that disgust has eventually been “co-opted” by our moral psychology (Kelly 2011) and upgraded into a more complex evaluation system which signals perceived social and moral violations pertaining to the dimension of “purity” and “sacredness” (Rozin et al. 2009), such as incest, but probably also fairness violations (Sanfey et al. 2003, Skarlicki et al. 2013). Disgust is often a useful heuristic for quickly recognizing potentially morally problematic situations that would otherwise require a lot of reflection, observation, and mental efforts for which we might not have the mental or even conceptual resources (Hauskeller 2006, 599; Kass 1997), or more specifically behaviors that, like microbes and pathogens, can be contaminating and lead to “social contagion” (e.g., heavily drinking in public) (Plakias 2013).
Intuitive and emotive responses can also result from the automatization of rational reflection or of the application of consciously held moral goals (Fishbach et al. 2003; Bargh and Chartrand 1999). Our moral principles, in other words, “may be overlearned to the point of their a) subliminal activation, and b) relative independence of cognitive resources” (Fishbach et al. 2003, 298). Also in this case, at the basis of our emotive and intuitive responsiveness is the typically human necessity to minimize cognitive resources and time.
The other side of the coin is that—as we have seen—often intuitions and emotions are also sources of biases and other types of irrational or immoral judgments that, as discussed at the beginning, make us bad information processors, bad moral judges, and bad moral agents. Some people are disgusted by the idea of homosexual sex, and indeed disgust sensitivity has been shown to be a predictor of condemnation of homosexuality (Inbar et al. 2009). But the disgust reaction is not necessarily linked to any morally relevant aspect of the practice or object being judged (Kelly and Morar 2014; Giubilini 2015)—unless, of course, we want to say that disgust is itself a reason for considering the object of disgust immoral (Kass 1997).
Thus, the very same characteristics that make emotions and intuitions an essential component of our practical and moral judgments and decisions (automaticity, independence from cognitive resources, minimization of mental effort) also make emotions and intuitions unreliable because sources of biases.
The AMA would maintain the positive functions of emotions, while avoiding the downsides. More in particular, emotions fulfill two main functions in human moral psychology. First, they are necessary but imperfect proxies for complicated reasoning and calculation for which we do not have the mental resources and for which we need heuristics. The AMA would have the advantage of providing the “real thing”—information processing, calculation, weighing of expected utilities, and so on—rather than the imperfect proxy. Second, human emotions and intuitions have the function of automatically drawing our attention to morally relevant aspects of certain situations (for instance disgust might draw our attention to anti-social behaviors like heavily drinking in public or to sexual taboos) prior to and independent of any conscious reflection. Again, as seen above, the way emotions perform this activity is inevitably suboptimal. The AMA, on the other hand, would perform the same function by immediately pointing at morally relevant aspects (according to the agent’s own standards of morality) of a certain situation that has been modeled and categorized in its software. Admittedly, some of the situations and the choices humans have to face are very complex, and in many such cases, humans’ skills are still better than any software we currently have. Our claim is not that computers are better than humans in any possible circumstance, but that work needs to (and can) be done to increase the reliability of computers as moral advisors and make them of assistance to humans in an increasingly wider set of circumstances.
Emotions are often considered to have at least two other important functions in human morality. First, in a Humean perspective, the belief that something is morally right or wrong boils down to approving or disapproving of it, where approval and disapproval are emotional attitudes. In this view, the role of emotions would be required in order for us to be able to endorse the AMA’s moral or practical advice. Admittedly, this is an aspect of an emotion based moral system that the AMA cannot replace.
Secondly, emotions are necessary to the motivational aspect of morality. They provide the indispensable link between moral judgment and moral action. It is noteworthy, for example, that without the correct functioning of certain emotions (sympathy, shame, guilt, grief) following brain damage, people with adequate knowledge of appropriate social behavior turn into psychopaths that simply do not behave as they know they should (Damasio 1994).
The distorting influence of our emotional states might be present at these two levels, making us incapable not only of accepting what would otherwise seem a reasonable moral judgment, but also of acting upon it. We propose that the AMA could be integrated with emotion and neurophysiological detection technologies in a way that could assist our moral judgments and actions by making us aware of emotional and potentially distorting factors. Consider, for example, a system for “ambient-assisted emotional regulation” (Garzo et al. 2010). The system is intended for use in elder care, and it “aims at exploring the combination of physiological emotion detection and ambient intelligence in the context of emotional regulation” (Garzo et al. 2010; see also Nasoz et al. 2003, Leon et al. 2007). The system uses software incorporated in people’s clothes that can monitor neurophysiological states; when an abnormal and potentially threatening state is detected, the system sends out a signal that prompts assistance for that person. For example, when the system detects a neurophysiological state of fear, it automatically switches on the light, or contacts emergency services. We propose the same technology could be used to notify us of an emotional state that might negatively affect our moral or practical decisions. Google is currently developing contact lenses with a microchip in them that monitors levels of glucose in the blood of diabetics (Liu 2013).Footnote 3 Although the purpose of this project is to assist medical professionals and patients, the same technology could be used to improve human decision-making in everyday life. A recent study has shown that low glucose levels are correlated with higher aggressiveness in couples (Bushman et al. 2014). Knowing when our neurophysiological state is likely to lead to aggressive behavior may assist us in making better decisions, for example by suggesting to postpone a certain discussion or to put more effort into self-control.
A further technological help in this direction could come—so we suggest—from so-called “neurofeedback training” (NFT). NFT is the use of monitoring devices “to provide the individual with explicit information regarding specific aspects of his cortical activity in an easy-to-understand format and in doing so encourage him to alter particular target components, such as amplitude and frequency” (Vernon et al. 2009, 2). In other words, subjects can be trained to alter their brain activity in certain respects by being made aware of some neurophysiological parameters. For example, NFT has been used to reduce anxiety and improve mood (Vernon et al. 2009). But once again, we want to suggest that this technology could be used to improve our moral capacities. A recent study has shown that receiving neurofeedback about their brain activity enabled subjects to change brain network function of areas related to empathy (Moll et al. 2014). Research on neurofeedback is however still in its infancy, and its methodology and results are still debated (Vernon et al. 2009 provide a useful review of the results). However, should this methodology prove itself to be efficient, we suggest it could be used as integration to the AMA and the neurophysiology detection technologies for moral enhancement purposes.