Does the socio-cognitive TA apply to the case of the relation HA? To answer to this question, we have to analyse the main features of autonomous agents: such analysis can enable us to assess to what extent autonomous agent may be held compatible with socio-cognitive tenets from the point of both the internal attribution and the external attribution of trust (displayed in Section 3.1). This is particularly important as researchers, who suggest that trust is to be reserved for the case of people only (that people can trust other people and not inanimate objects), insist on the following: “One reason to reserve the concept of trust for a relation between people is the role motives and intentions play in it. [...] The philosopher Lawrence Becker, for example, has argued that far more relevant to our readiness to trust are the motives and intentions we perceive others to have than actions and outcomes” (Camp et al. 2001, p. 6). However, such a consideration is not already a reason to exclude autonomous agents from the realm of trust. I may be willing to endorse the view that: “If it is not possible to design a computer security system without assumptions about human behaviour the design of computer security systems should be informed by philosophical and social sciences theories about trust” (Camp et al. 2001, p. 8). On the other hand, I must also note that to revise cognitive models on the basis of autonomous agents’ way of behaving may help us to draw (new) assumptions about human behaviour. In other words, I do not suggest comparing autonomous agents with human agents or to subsume the former within the comprehension of the latter. This could be charged with an accusation of anthropocentrism. I suggest understanding both human and autonomous agents’ decision to trust, at a higher level of abstraction, in terms of cognitive states and relations.
Main Features of Autonomous Agents
The following are the main features inherent to the internal status of the autonomous agents (Franklin and Graesser 1996; Nwana 1996), which are related to the internal attribution of trust:
▪ Autonomy: the agent’s ability to perform a task that originates from the agent itself and is neither a simple adjustment determined by the environment nor the reaction to the intervention of a human user or of another agent; it is the ability to choose between alternatives that are not fully predictable at a determined LoA, which is not that of programmers but of users (Floridi 2004). Compare this notion of agent autonomy with Floridi’s definition, given in Section 3.2.
▪ Persistence: the agent’s ability to keep its own internal state while performing its task as well as after its performance.
▪ Vitality: the agent’s ability to face and solve anomalous situations that are otherwise capable of determining in the agent a state of instability that would menace the agent’s persistence.
▪ Organisation: the agent’s ability to organise its task, by distributing the work and communicating it to each part that compose its applications;
▪ Proactivity: the agent’s ability to create new situations in the environment, by requesting the intervention of other agents or coordinating their activities in a way to perform its task.
The following are the main features inherent to the autonomous agent’s ability to interact with the environment, with users or other agents (Franklin and Graesser 1996; Nwana 1996), which are related to the external attribution of trust:
▪ Reactivity: the agent’s ability to interact with the environment or agents, by changing its states (compare this definition of agent’s reactivity with Floridi’s more sophisticated explanation of adaptability given in Section 3.2).
▪ Ability to communicate: the agent’s ability to communicate with other agents to cooperate or share resources or learn from experience (this ability is higher when autonomous agents communicate with artificial agents rather than with human agents).
▪ Mobility: the agent’s ability to move within the web or in another intelligent ambient, in order to change its states or partners, when it is necessary to react to the varied environmental conditions.
▪ Benevolence: the agent’s attitude which motivated it to do everything that is required to perform the task exactly, without being deterred from its course of action by external environmental stimulus;
▪ Attitude to be veridical: the agent’s attitude to release true information when interacting with other agents.
These characteristics appear to be fairly compatible with the socio-cognitive analysis of trustor’s internal and external attribution of trust. The trustor is enabled to judge both the internal and the external qualities and defects of the autonomous agent, by assigning to each feature:
An evaluation (also quantitative: <0, 1>) of its probability (possibility, plausibility or ranking);
An evaluation (also quantitative: <0, 1>) of its relative importance, compared with other features, in relation to a specific goal to be obtained. This ranking function is of great importance to us, since in a cognitive model we judge not only what are the qualities and defects of the trustee; but how significantly these qualities and defect interplay in a given context;
An evaluation (also quantitative: <0, 1> of its ascertainability by the trustor. This ranking function (how much the trustor can control the trustee and ascertain her cognitive states) is related both to the trustee’s ascertainability and correlatively to the trustor’s reflexivity (capacity of self-evaluating her own cognitive mind-set).
We could certainly strengthen such results by taking into account elements that directly refer to trustworthiness such as probabilities to defect or to cooperate (Ghanea-Hercock 2007) or rating functions of positive outcomes (Taddeo 2010) but these features can hardly be associated with evaluations, motives, intentions and cognitive judgments. Even if the characteristics we have considered above as associated to the internal and external attribution of trust appear, in their whole, consistent with cognitive schemas and judgments, they may raise problems as to their full applicability to the case of autonomous agents.
Problems of Applicability of Cognitive Evaluations of TA and TD to AAs
Autonomy is a very troublesome feature for two different kinds of reasons (we will insist on this in the subsection 5.3.2). First of all, it is debatable at what level of abstraction we should investigate agent’s autonomy: either at the LoA of programmers (who design, implement and deploy an AA), a group we will call the AA’s developers (Grodzinsky et al. 2010)” or at the LoA of users? The answer, as we will see later on, may change radically the way we perceive the interaction with autonomous agents: “if computers are perceived as elements of a single undifferentiated network, then trust in computers will increase as computing experience increases” (Camp et al. 2001, p. 1). This question is inherent to the second point at play: that is, the level of predictability of the trusted agent’s behaviour. As it has been stressed (Grodzinsky et al. 2010): “predictability is a central theme that we wish to emphasize”, since it “has important consequences in issues of trust”. These issues concern not only agents’ autonomy but also persistence/change and adaptation (reactivity) and, in the end, the “rationality” itself of the autonomous agent and hence of the AoT. I will consider predictability, firstly, from the point of view of changes driven by autonomy and, then, from the point of view of changes driven by adaptation.
“Predictability is an important attribute from which to draw important distinctions between humans and AAs. AAs are distinct in the sense that we expect that they are capable of much faster changes than humans. Also, the discrete nature of binary encoded programmes increases the likelihood of abrupt and dramatic changes; we expect slower, more gradual changes in processes that at least appear to follow laws described with continuous values and mathematics. (That is, in general we expect binary processes to appear more ‘jumpy’ and analogue processes to appear ‘smoother’). Because software moves at speeds that are beyond the perception of humans, AAs can go through a dramatic self-modification process multiple times during a relatively slow interaction with a human. This sort of change can be disruptive to any existing trust relationship that relies on predictability and that grows out of past experience with that AA” (Grodzinsky et al. 2010). Clearly, it is reasonable to assess that the more predictable a given behaviour is the more certain and stable are the expectations of such behaviour. However, this assessment cannot be turned into an absolute value, since a full predictability would transform incomplete information into a form of certainty that would exclude the interaction between agents from the realm of trust (and place it into the domain of a mere reliance). So, the requirement of predictability is not to be inflated (driven to an absolute value) if one intends to remain in the conceptual area of trust In other words, mere reliance, based on full predictability, is not sufficient in a multi-agent system, where agents continually adapt to their environment—an environment that consists largely of other agents.
As regards to change and adaptation, in addition to what previously said, I wonder whether this ability to change their internal states might at times enable the agents to face unexpected adverse conditions. This may prove true when adaptation is concerned only with changing “the value of a variable”. On the contrary, when adaptation is affected with a “self-modifying code”, this becomes much more unclear, and we can hardly distinguish whether or not we still interact with the same agent (Grodzinsky et al. 2010). In other words, online users cannot entirely determine themselves on the basis of predictions of future, if such predictions derive from established past knowledge (norms, programmes, instructions, data) that depend on original conditions, which are likely to be, more or less apparently, modified by the mobility and reactivity of other agents. Rather, a rational expectation may be based on present shared information coming from party relations that can be cognitively evaluated and revised (Durante 2008).
In human to human relations, the trustees know that their self-representations and declarations will be judged, relied on and shared: this can deter the agent from a misrepresentation or a false declaration, setting aside the cases in which the trustees take advantage from modifying the original conditions of party relation. Trustees will communicate fairly when they share a concern for the task to be performed and when their performance is subject to shared information or, to put it differently, when their performance is visible and traceable: it can become a trailed part of agents’ past history that trustors can keep track of. Similarly, in human to autonomous agents relations, the issue of “transparency at best, and traceability at least, is a theme” of a great impact, since “if humans are to trust AAs, then AA developers should produce systems whose criteria and process for making decisions are accessible to humans. If these systems’ decision-making processes are obscure or hidden, humans are less likely to trust AAs over the long run, and we assert that humans should not trust such systems” (Grodzinsky et al. 2010). As regards to mobile agents in particular, the issue of traceability of migration path is crucial and can be faced efficiently by means of distributed trust: i.e. by distributing trust to several hosts on a migration path, which store and communicate information each other, since “hosts cannot cut out or replace the tail of a migration chain (including cycles), as the next receiving host will check the migration with the previous host in the migration chain, using both the host and the sequence number. The essence of this approach is that the responsibility of a migration is spread over two hosts” (Warnier et al. 2007).
We can rephrase here, as regards to H > AAs relations, what we have stated about H > H relations: to the extent to which decisions and behaviours may be traced and expectations be shared, which can give rise to forms of cooperation. In fact, distributed trust turns an individual risk in a collective one, and allows human and autonomous agents to cope more efficiently with the lack of certainties (Durante 2008). It is different to face the lack of certainties by means of a communication process (that does not exclude the presence of an inherent risk but only transforms it into a distributed one) or by achieving a full predictability through design (that aims at eliminating the presence of an inherent risk).
How to Discuss Arguments Contrary to the Application of Trust to AAs
We have tried to show, to this point, that a socio-cognitive evaluation of AAs features and attitudes is feasible and can enable us to apply both a TA and a TD to the functioning of AAs, while admitting that such application is not devoid of problems. These problems appear at any time we attempt to compare human behaviour (decisions and actions) with autonomous agents’ way of behaving: such a comparison leads to a specific problem (Taddeo 2010): “how an artificial agent could be programmed to behave in a manner similar to how humans behave when they report that they have learned to trust someone or something” (Grodzinsky et al. 2010). Of course, this approach, as has been stated, is theoretically correct but too difficult to achieve in practice. Developers take crucial indications from the comparative analysis of human behaviour but they cannot simply attempt to design AAs on the basis of such similarities. They can only attempt to (1) elaborate cognitive models to represent agents’ behaviour; (2) to figure out the representation of agents’ behaviour not as a whole but as a sum of parts; (3) to represent each part in syntactical terms that make this model applicable to the design of AAs along these guidelines:
▪ Traceability is to the decision-making process what is memory to human experience (Ferraris 2009): we should not try to fully understand and reproduce what means to make experience; we have to consider experience as a selection of meaningful data (meaningful because selected);
▪ Information feedback is to the decision-making process what past experience is to human decision: a trusted third party that registers positive outcomes and makes them part of a distributed trust system may be equivalent to a crucial aspect of human decisions: that is, it is reasonable to expect that, ceteris paribus, same information would lead agents to the same decisions;
▪ Assigning variables to values and weighing values against each other within a frame is to the decision-making process what judging priorities is to human evaluation: evaluation is one of the most complex human activities, since it always entails (from a syntactical standpoint) a combination of (1) variables of values, (2) hierarchies of values, and a (3) framework for hierarchies;
▪ Relations between states of affairs (conceived in terms of degrees of control) are to the decision-making process what relations between humans are to the construction of human knowledge: like Foucault has shown it, to know is to know what I can control (Foucault 1980, 1994).
These guidelines may help us to discuss the arguments commonly addressed to the possibility to apply trust to AAs with a milder anthropocentric bias (on this, see what has already been said in Section 4).
Is Trust for Human and Not for Artificial Agents?
Trust is a concept entangled with the moral standing of agents: it requires free will, autonomy, reward, appraisal, responsibility, blameworthiness, repentance, betrayal, disapproval, that is, the whole catalogue of moral categories. It is a commonplace that only human beings can endorse moral values, judgments and behaviours. Luciano Floridi’s ethics of information (2001, 2004) stands against such commonplace and his moral theory has already been proved to be fruitfully applicable to the domain of trust (Taddeo 2009, 2010). Floridi’s ethics of information (2004) is concerned with rebutting five main objections with regards to the possibility of qualifying AAs in moral terms (the teleological objection; the intentional objection; the freedom objection; the responsibility objection and the objection of concreteness), which we cannot expound here (see on this debate, Ethics and Information Technology 2008, pp. 10.2–3). Whether or not Floridi is persuasive—and I think he is to a large extent—there is one point which appears crucial to me in relation to the notion of trust, that is, Floridi’s idea of a practical counterfactual that points out the limits of determinism (which I will focus on in the next subsection): “The AAs are already free in the sense of being non-deterministic systems. This much is uncontroversial, scientifically sound and can be guaranteed about human beings as well [...]. All one needs to do is to realise that the agents in question satisfy the usual practical counterfactual: they could have acted differently had they chosen differently, and they could have chosen differently because they are interactive, informed, autonomous and adaptive” (Floridi 2004, p. 17).
As stated above, this idea of a practical counterfactual that makes visible the limits of determinism further enlightens the notion of trust. I have already emphasized this point (Section 4): trust is concerned with a particular form of reduction of uncertainty. Helen Nissenbaum has expressed it efficaciously, with reference to Niklas Luhmann (1979): “To express the trade-off in Luhmann’s terms, we may say that while both trust and security are mechanisms for reducing complexity and making life more manageable, trust enables people to act in a richly complex world, whereas security reduces the richness and complexity” (p. 179).
What I only would add to such formulation is that, thanks to that practical counterfactual (which shows otherness: things could have been happened otherwise), trust enables people to see (and to some extent to represent and to measure) how rich a complex world is: the whole act of trusting is not only a way of entering into the relation with the other, but also of discovering who is the other (Durante 2008).
Lack of Trustee’s Autonomy
We have already spoken about autonomy but we have to insist here on one core point. As has been noted, the requirement of autonomy is essential, since trust demands the agents to be able to choose between alternatives in a way that is not entirely predictable (like for many intentional states). What we have to highlight is that predictability is not to be judged in some absolute terms but only in relation to the LoA at which agents are studied and analysed: “If a piece of software that exhibits machine learning is studied at a LoA which registers its interactions with its environment, then the software will appear interactive, autonomous and adaptive, i.e. to be an agent. But if the programme code is revealed then software is shown to be simply following rules and hence not to be adaptive. Those two LoAs are at variance. One reflects the ‘open source’ view of software: the user has access to the code. The other reflects the commercial view that, although the user has bought the software and can use it at will, he has no access to the code. At stake is whether or not the software forms an (artificial) agent” (Floridi 2004, 13).
Obviously, such a consideration does not go so far to exempt developers from responsibility as regards to the design of AAs, both for Floridi (who casts, however, some doubts on this approach: “Our insistence on dealing directly with an agent rather than seeking its ‘creator’ [...] has led to a nonstandard but perfectly workable conclusion” (Floridi 2004, p. 24))—and for Grodzinsky, Miller &Wolf: “All this does not mean that the concept of ‘responsibility’ is redundant. On the contrary, our previous analysis makes clear the need for further analysis of the concept of responsibility itself, when the latter refers to the ontological commitment of creators of new AAs and environments. As we have argued, Information Ethics is an ethics addressed not just to ‘users’ of the world but also to demiurges who are ‘divinely’ responsible for its creation and well-being” (Floridi 2004, p. 26).
“We prefer that AAs be boringly predictable. We are far more concerned about the trustworthiness of AAs, and far less concerned that they mimic human’s adaptability. In almost all situations (with the possible exception of computer gaming), we think that AA developers have a duty to the safety of the public that should restrict their use of the self-modifying code to implement AAs, including limitations on the use of neural nets in AAs” (Grodzinsky et al. 2010).
Trustee’s Lack of Repentance or Perception of Trustor’s Disapproval
In principle, it may be held true that: “In terms of trust and forgiveness in the context of computer-mediated activities, there is no significant systematic difference in people’s reactions to betrayals that originate from human actions, on the one hand, and computer failure, on the other” (Camp et al. 2001, p. 5). Theoretically, trustor’s reactions should not vary in relation to the same betrayals of trust. In practice, from a cognitive standpoint, such reaction is likely to be intertwined with a judgment—in particular when the issue of forgiveness is at play—that is concerned with trustee’s repentance for betrayal or at least with trustee’s perception of the trustor’s disapproval for the failure (awareness). From this perspective, a trustee’s lack of repentance or perception of trustor’s disapproval for betrayal could be an argument against the application of trust to AAs. More precisely, it can be an argument that undermines the possibility to develop an atmosphere of trust with AAs on the basis of information feedbacks (on this Section 3.3). On the contrary, we could say that it is precisely an information feedback cycle that could fill the gap between a trustee’s lack of perception and repentance: in plain terms, perception of someone else’s disapproval and repentance amount to public admission of our own failures and betrayals. In order to obtain a public admission, it is sufficient that failures and betrayals are traced, communicated and stored by a third trusted party that bring them into the general notice of interacting parties.
Trustee’s Lack of a ‘Fear of Sanctions’ for Betrayal
In a previous paper (Durante 2008), I have argued that a trust relation belongs to the area of normativity because social norms grow out of the communication process between trusting parties. If this holds, the normative communication process can be reinforced by the provision of sanctions for betrayal (the same can be said for rewards in case of agents fulfilling their task (Nissenbaum 2004, p. 161). A sanction is not intended here in psychological terms as the fear for the trustor’s reaction or judgment. If a sanction is conceived in such terms, this leads us to understand fear as an emotion that only human beings can experience. Conversely, a sanction can be described in strict legal terms as a negative consequence that stems from the betrayal (the unobserved norm). How can a consequence be negative? It can be negative in that it deprives me of something. In other words, a sanction is a diminishment of me. If a sanction is interpreted in this way, fear should no longer be conceived as a human emotion: it can be intended as a scalable state of affairs that correspond to a diminishment that lowers my probabilities to act. Probabilities to act, attached to a negative success rate, can be lowered down until the point that the trustee will be automatically prevented from acting by a further diminishment in its success rate: in this perspective, we may say that a ‘fear of sanctions’ may prevent it from acting badly (on the applicability of sanctions to artificial agents see L. Floridi 2004).
Lack of Embodied Relations Between Agents
The last point is to reiterate that human touch is not, in my view, a needed requirement for trust. From a philosophical point of view, it would take time to expound this reflection: suffice to say that human relations are not only empirical relations that require touch, contact, physical proximity etc. The fear for a progressive disembodiment of human relations and experiences is not caused by the fact that embodiment is a necessary character of both relation and experience. On the contrary, it originates from the fact that embodiment has became, throughout time, a sort of guarantee of the truthfulness of relations and experiences. An empirical, embodied situation seems to endow us with the feeling and the epistemic belief of being able to have control over our relations and experiences. This brings us back to problem of control—the real cognitive and practical problem—which we have started from. Things are far more complicated than we have attempted to represent, since it is an oversimplification to speak of a relation between control and lack of control, which is only justified by the analysis of trust. A (certain) lack of control is, most of the times, insular to any cognitive and epistemic ‘control’ (as I have defined it from the start): it is not something that comes from the outside of a situation of control but it is inherent to it. Even when I look at my clock to check what time it is, I can easily ‘control’ what time it is, but I can still doubt whether or not my clock is on time. I always need, in the case, a further guarantee, which is not necessarily at hand. In this perspective, trust is concerned with the agents’ need (be human or artificial agents) to step out of their limited circle of guaranties (of course, the reverse situation is not necessarily true: not all that escapes from our control deserves trust). As a final point, trust is not only a relation between interacting agents but, primarily, a cognitive relation with what remains out of control within what we believe to hold control over.