Modeling culture in intelligent virtual agents

Abstract

This work addresses the challenge of creating virtual agents that are able to portray culturally appropriate behavior when interacting with other agents or humans. Because culture influences how people perceive their social reality it is important to have agent models that explicitly consider social elements, such as existing relational factors. We addressed this necessity by integrating culture into a novel model for simulating human social behavior. With this model, we operationalized a particular dimension of culture—individualism versus collectivism—within the context of an interactive narrative scenario that is part of an agent-based tool for intercultural training. Using this scenario we conducted a cross-cultural study in which participants from a collectivistic country (Portugal) were compared with participants from an individualistic country (the Netherlands) in the way they perceived and interacted with agents whose behavior was either individualistic or collectivistic, according to the configuration of the proposed model. In the obtained results, Portuguese subjects rated the collectivistic agents more positively than the Dutch but both countries had a similarly positive opinion about the individualistic agents. This experiment sheds new light on how people from different countries differ when assessing the social appropriateness of virtual agents, while also raising new research questions on this matter.

Introduction

Due to the incredible advances made in computer graphics and physics, it is currently possible to create visually stunning virtual environments. Moreover, these environments can now be populated with appealing characters that can look very realistic and with whom we may interact in multiple ways. However, many difficult challenges arise when we try to procedurally generate human-like social behavior for these characters. Nevertheless, over the past few decades, researchers in the field of intelligent virtual agents have made significant advances [22]. These advances have enabled the development of innovative agent-based applications for education [3, 33], health [6], and entertainment [46], among others. The potential shown by such applications continues to increase the demand for agents with even greater social skills.

As the development of virtual agents focuses increasingly more on the social aspects of human interaction, it becomes crucial to address the notion of culture and how it affects human behavior. As argued by Charles Ellwood, “It is impossible to understand human society without understanding human culture; for the social behavior of man, whether it is domestic, economic, political, legal, moral or religious, is dominated by the culture of his group” [10]. Given its importance, research in modeling culture in autonomous agents is currently growing. Still, many different types of cultural phenomena have yet to be addressed. In virtual agents, most of the research performed so far has narrowly focused on specific aspects of conversational behavior such as language [30], dialect [15], speech pauses [13], posture [11], or proxemics [29].

Apart from the aforementioned aspects, it is also important that culture be considered in the general cognitive processes of an agent. The development of cognitive models that address cultural factors is quite challenging because culture is associated to groups of people, not single individuals. It represents the shared knowledge of a societal group that includes assumptions of what is right and wrong, and what is appropriate and inappropriate. Such assumptions have a major impact on human reasoning and behavior. However, careful consideration of these shared assumptions has been, for the most part, absent from general cognitive agent architectures that try to create agents with human-like intelligence. SOAR [32], ACT-R [2], MicroPSI [4], EMA [18], and PRS [17] are some examples of such architectures.

With a few exceptions [34], most cognitive agent models of culture have focused on specific norms [47, 48] or on exploring how culture affects instrumental decisions, such as in negotiation tasks [16, 39]. Instead, the work described here focuses on representing cognitive biases in judging the appropriateness of social actions in everyday social interaction. Our approach is based on existing theoretical work from the social sciences literature. For the notion of culture, we chose a theory that is widely known and has been empirically validated across several countries. This theory, put forth by Hofstede et al. [25], proposes a set of culture dimensions that indicate broad tendencies of behavior and reasoning. Unlike specific practices or norms, these tendencies can be found in all countries. Our proposed model is also strongly based on the Status-Power theory of Kemper [31]. The main purpose of this theory is to structure the ultimate motives that underlie all socio-cultural behavior.

The contributions presented here can be summarized as follows. Firstly, we present a novel agent model, named the social importance dynamics (SID) model, which formalizes the notion of status as proposed by Kemper [31]. By describing how the model integrates with the well known BDI paradigm [43], other researchers can apply the same constructs and associated dynamics to create agents that are more aware of the social consequences of their actions. Second, through the use of the SID model, we demonstrate how general cultural tendencies that are shown in everyday social interaction can be explicitly represented in a structured manner, rather than implicitly in the agents’ goals and actions. One of the main advantages of making culture explicit in an agent model is that it becomes possible to conduct agent-based simulations and experiments where one manipulates the culture of a group of agents as a separate parameter. These experiments can provide new insights into the field of human-computer interaction. With this in mind, our third contribution is a user study with participants from two different countries: Portugal and the Netherlands. According to Hofstede’s model, the culture of the Netherlands is highly individualistic whereas Portuguese culture is strongly collectivistic. As such, the aim of the study described here was to investigate whether individuals from these two countries perceive and interact more favorably with agents that have a similar culture profile. The results obtained showed that collectivistic agents were indeed perceived significantly more favorably by Portuguese participants than Dutch participants. The general opinion about individualistic agents was similarly positive in both countries, possibly due to limitations in the scenario design.

This paper is organized as follows. In the following section, we discuss the theoretical background of the notion of culture. In Sect. 3, we establish the state of the art in addressing cultural factors in the behavior of intelligent agents and how our work differs from that of others. In Sect. 4, we discuss the different theoretical concepts that are related to modeling socio-cultural behavior. In Sect. 5, we present the proposed SID model and its integration into the BDI paradigm, which is then followed by a description of how the model was used to implement cultural differences in Sect. 6. In Sect. 7, our case study is presented and in Sect. 8 we discuss its evaluation and the results obtained. Finally, we draw conclusions and discuss future work.

Theoretical background

There are many definitions of culture, and they can range from visible elements, such as the music styles of a group of people (or ‘pop-culture’), to non-visible elements, such as certain behavioral tendencies in a group. Some researchers take a view that includes both elements such as Stella-Ting Toomey’s conceptualization of culture: “Culture is like an iceberg: the deeper layers (e.g. traditions, beliefs, values) are hidden from our view; we only see and hear the uppermost layer of cultural artifacts (e.g. fashion, trends) and of verbal and non-verbal symbols” [50]. She then explains that to truly understand the nature of a culture, we must be able to match the upper layers with the lower layers.

This approach is also mentioned by Hofstede et al. [25] when they discuss the different manifestations of cultures. For them, values are manifestations of the deeper layers of culture, and practices are manifestations of the upper layers. Moreover, values are quite abstract, and take a generic form, such as “the importance of family over work”, while practices take a more concrete form, such as “how low you bow when your boss enters the room.” In their work, which was based on empirical data, they attempted to describe high-level values [25] for different cultures and to determine how visible behavior reflects these values [24].

Dimensions of culture

Hofstede and his colleagues [25] conceptualize culture as a limited number of major societal issues, to each of which a society finds a shared solution. These issues are conceptualized as a continua—scales with a lower and an upper end. Hofstede et al. call these ‘dimensions of values’ and they describe broad cultural differences across nationalities. Hofstede’s model is based on questions about everyday work practices; the dimensions of values were a serendipitous finding. They refer not to convictions or beliefs, but to broad tendencies to perceive the social world in a certain way. The model has grown over time, as more sources of data were consulted. The latest model consists of six dimensions: (1) individualism versus collectivism; (2) power distance (3); masculinity versus femininity (4); uncertainty avoidance (5); long-term versus short-term orientation; and (6) indulgence versus restraint. Each of them is modeled as a continuum along a scale from 0 to 100.

It is important to note that the picture drawn is necessarily simplified. The authors mainly provide descriptions for the extremes of the dimensions, and, as such, they are only abstract representations of how cultures can differ. In reality, almost all real world cultures have intermediate positions on the dimensions, and thus all cultures will share characteristics of both extremes. Furthermore, the dimensions of culture can only be isolated from one another in an artificial way; the six dimensions are abstractions that capture behavioral trends across cultures.

Synthetic cultures

The dimensions of culture unpack differences between real-world cultures. An attempt has been made to make these dimensions even more tangible, through so-called synthetic cultures [24]. These synthetic cultures have already been used in numerous simulation games [27], and they represent extreme manifestations of the value orientations at the ends of the dimensions of culture. They are simplifications to the extent that they represent only a single aspect of social behavior; they are not meant to describe the interdependency between dimensions of culture.

The advantage of these synthetic cultures is that they are among the very few scripts for cross-cultural simulations available that are based on theory. Harry Triandis, writes in his foreword to Hofstede’s book: “In fact it has been shown [5] that theory-based cross-cultural training is more effective than training that consists of scattered samples of beliefs, attitudes, and experiences. Why? It is easier for the learner to absorb the material and generalise to new situations if the training is based on theory.”

The synthetic cultures are instantiated on different levels of human behavior. They describe certain elements: core values, core distinctions, seven key elements, words with a positive and a negative connotation. To provide an example, the individualism dimension deals with the extent to which members of a society feel responsible for themselves, or for the larger group they belong to. In individualistic cultures, ties between individuals are loose and rights and obligations should be the same for all people, whereas in collectivistic cultures, people are integrated into strong, cohesive groups and the boundary of the in-group is also a moral boundary beyond which the same rights and obligations do not hold.

Related work

Because culture can manifest itself in many different aspects of human behavior, researchers have developed different kinds of models for creating culturally adaptive agents [51]. Some models focus exclusively on addressing cultural differences in the external aspects of behavior that are closely tied to the agent’s embodiment. For instance, Jan et al. proposed a model that addresses cultural differences in proxemics, gaze and overlap in turn taking [29]. More specifically, the authors chose to manually adapt these aspects in background multi-party conversations for three particular cultures: Anglo-American, Mexican, and Arab. A corpus containing approximately 20 h of audiovisual multiparty conversation in the three target cultures was later gathered and analyzed [23]. The results obtained illustrated that the people’s behavior often deviated from the predictions found in the literature.

Endrass et al. [11–13] have also investigated the cultural adaptation of non-verbal features in virtual agents. More precisely, the authors focused on how topic selection, gesture expressivity, usage of pauses, speech overlap and body posture are different between Germany and Japan. The authors created prototypical Japanese and German versions of two agents talking to each other following an approach that considered both the existing social sciences literature and a video corpus analysis of dyadic interactions in the two target cultures [44]. The perception of the prototypical versions was then evaluated with participants from both countries. With regards to topic selection, both German and Japanese participants significantly preferred their cultural version of the agents. However, for the nonverbal features, German participants showed a statistically significant preference only for some features and the Japanese participants only showed a significant preference with regards to posture. The aforementioned work shows that the cultural adaptation of virtual agents with regards to the externally visible features of their behavior is a research area that is quite challenging and yet important for the field of human-computer interaction.

With regard to decision-making, the lack of agent architectures that integrate cultural aspects in the agents’ cognitive processes has led some researchers to propose new models addressing such aspects. For instance, the CAB model [48] allows the explicit encoding of specific cultural norms and stereotypes in a task-oriented model. Furthermore, Thespian [47], an agent architecture based on the PsychSim framework [42], focuses on representing social obligations during conversations. Both the CAB Model and Thespian allow the encoding of specific cultural norms that have a strong impact on the way the agents act. However, they do not allow the representation of cultural differences in the agent’s decision making that are more generalizable than specific norms. Still, these models have important aspects that we also considered in our approach. Namely, both CAB and Thespian use a Theory of Mind mechanism and Thespian has an explicit model of social relations. These aspects allow the agent to have a better notion of the social reality of a situation, thus facilitating the modeling of cultural elements in social interaction.

A particular domain where researchers have shown interest in exploring more general cultural differences in decision-making is human-computer negotiation. Gal et al. [16] developed an adaptive agent for a repeated bilateral negotiation game, named Colored Trails [19], and investigated its performance across two different countries, the US and Lebanon. The developed agent, named PURB, uses hand-designed rules to explicitly reason about the observed behavioral traits of the human negotiators it interacts with. The results obtained showed that the agent’s adaptation mechanism made it quite proficient at negotiating with people from the two countries. Interestingly, the agent was able to outperform participants from the US but was outperformed by Lebanese people. The latter result was attributed to the fact that the agent overestimated the reliability of Lebanese participants. An improved agent was later developed [21] that was able to outperform participants in the US, Lebanon, and Israel. This was achieved by combining the approach taken with probabilistic models of human behavior. These models were constructed from data gathered from people playing the game in the three target countries.

Nouri and Traum have also proposed a decision-making model for negotiation that is culturally sensitive [39]. The model, named MARV, was applied to the Ultimatum game [20], an extensively researched scenario where studies have shown that people exhibit significantly different behavior across cultures [41]. Instead of a classical monolithic view of utility, the model developed by the authors considers several metrics (e.g. self-interest, other interest, uncertainty) for evaluating the value of a particular outcome in a negotiation game. Cultural differences are represented in the model by assigning different weights to such metrics. Initially, the authors defined different weights for Hofstede’s culture dimensions [25] according to their intuitions about what these dimensions represent. An experiment was then performed in which agents played against each other under different cultural parameters. The results were then compared with the existing data that had previously been gathered from human players in different cultures. For many countries, the comparison showed a strong resemblance with the results obtained from the simulation. Still, for some countries the results were quite different, which according to the authors, was an indication that the culture-specific weights needed to be more robust. This led the authors to investigate the use of Inverse Reinforcement Learning [1] to learn weights automatically using data from different cultures. To test this possibility, the authors used data of four different cultures, namely, the US, Japan, Israel, and former Yugoslavia. The learned weights were able to outperform a baseline strategy with random and greedy weights [38, 39]. Also, when playing against opponents of one culture the learned weights of that culture outperformed the learned weights of different cultures. This result further illustrates that negotiation is a culture-specific process. More recently, the authors also designed a short pre-survey on the participants’ value orientations toward a negotiation task and investigated how the reported values can be mapped to the model to improve its performance [40].

The previous negotiation models have brought valuable insights on how cultural biases affect human decision-making, particularly in instrumental decisions. Our work differs by exploring cultural biases on how actions, particularly non-instrumental ones, are judged in terms of their moral appropriateness given a specific relational context (interacting with a stranger versus interacting with a friend). Given our focus in the behavioral appropriateness of virtual agents, our evaluation approach is closer to the aforementioned work regarding non-verbal behavior in the sense that it focuses on assessing how people respond and judge different cultural configurations of our model.

In summary, research on cultural models for agents is currently growing, addressing either cognitive or non-cognitive aspects of culture. With regards to cognitive aspects, our main interest, most work thus far has focused on specific norms or negotiation behavior. A notable exception is the model proposed by Mascarenhas et al. [34], a cognitive model that was based on two specific cultural dimensions from Hofstede’s theory [25], namely, Individualism versus Collectivism and Power Distance. The main idea was to establish a direct link between these two dimension and the utility of goals and thus decision making. In the model discussed in this paper, culture is not linked directed to the utility of individual goals or actions. Instead, culture changes how the agents perceives their social relations with others and the expectations associated with those relations.

A conceptual model for socio-cultural agents

As mentioned previously, culture can be conceptualized as a limited number of major societal issues. As such, it influences how people perceive the social world (that perception, and its associated interpretation, will then influence people’s behavior). The synthetic cultures allude to different levels of human behavior, but they do not define which elements need to be present in the mind of an agent for them to use these cultural differences as part of the agent’s decision-making process.

In previous work, we identified three different levels of instantiation that are important to consider when creating socio-cultural agents [8]. They range from the more specified, i.e., the interaction, to the more abstract, i.e., the society, and they help to define important concepts that are required for agents to act out culturally varying behavior:

• The ‘interaction’ level is composed of elements that are visible to an outsider and that may change depending on the people involved.

• The ‘group’ level is composed of elements that are part of group level behavior and that may change depending on the group affiliation of the people involved.

• The ‘society’ level is composed of elements that are part of the underlying culture and that may change depending on the cultural background of those involved.

At the interaction level, there are three important elements to consider: the ritual, its physical context, and its participants.

A ritual is the process of two or more people interacting. Besides the physical effects of actions, there is also the need to consider their symbolic meaning. As argued in Rothenbuhler’s work [45], rituals may range from the ceremonial and memorable to the mundane and transient. In fact, any group of people that has a degree of shared attention can be said to be engaged in a ritual [26]; in other words, all social actions an agent may take can be considered part of a ritual. We call those people who are part of a ritual participants and the location the physical context. Both the participants and the physical context impact the ritual. Depending on where it is and who is present, some symbolic actions may have a different meaning. For example, shaking someone’s hand may be intended as a greeting, but in a different setting it may be intended as a way to form an agreement.

At the ‘group’ level, there are also three important elements to consider: moral circles, social norms, and relational variables.

The concept of moral circles is an important determinant of behavior across social groups [52]. Moral circles are primarily used to help describe those who are worthy, versus those who are not. Those who are viewed as worthy are granted certain (moral) rights and duties. A moral circle is composed of three elements: the people to whom it applies (i.e., moral circle membership), their mutual perceptions of social attributes (i.e. relational primitives); and the rules that regulate their behavior (i.e., social norms). In our computational model, a moral circle is represented by a group of agents that shares the same configuration of the model.

Relational variables represent relational properties of other individuals, and may differ per person. For example, a boss may receive a greater prestige among the employees due to his or her professional standing. However, if the boss behaves in a reproachable manner, for instance, by making employees work late every day, a lower reputation will be attributed to him or her. Both prestige and reputation are relational properties that will influence how one treats an individual. The notion of social importance, as we will define in the following section, is a generalization of all relational variables that have a positive connotation in human social interaction, which is similar to how the notion of utility in agents is a generalization of all motivational sources the agent has.

Social norms can be used to establish appropriate actions that should be acted out, may be acted out, or are prohibited in specific contexts [28]. Each moral circle may have a set of different or overlapping social norms, and they help to determine appropriate behavior. For example, it may be acceptable to kiss your spouse in private, but it may not be as appropriate in public. As will be explained in the following section, the notion of conferral and claim represent social norms whose activation depends on the existing relational standing.

At the ‘society’ level, there is one major determinant that drives behavior across societies: cultural meta-norms [37]. In contrast to the specific guides to behavior, i.e., social norms, meta-norms are non-instantiated guides to social behavior. They influence the relational fundamentals of social life, and they are shared within any society that has the same culture. Cultural meta-norms act as a weighing and salience mechanism for the concepts at the ‘group’ level. The cultural background may influence which moral circles and social norms are more important. For example, is it more important to attend a friend’s or a boss’ house-warming party if one can not attend both? The cultural background may influence which decision is most appropriate. In our computational model, we formalize this notion of cultural meta-norms using the construct “Culture Influence.”

Social importance dynamics model

Thus far, we have argued that to address the problem of creating agents that are capable of portraying different cultures it is important to consider fundamental aspects of human social behavior as culture is essentially a social phenomenon. To this end, we described a number of theoretical constructs that pertain to the social reality of human beings. The aim of these concepts is to describe the social world from a more theoretical perspective.

Moving toward a computational perspective, our aim was to create a model that enabled the emergence of these theoretical concepts in the agent’s behavior, while also minimizing the number of different formal constructs that are required. To achieve this, we developed the SID model [35], which is grounded on the status-power theory proposed by Kemper [31].

This theory was chosen as the basis for the computational model because it argues that all human social activity, including participating in rituals, forming moral circles, and following or violating social norms is ultimately motivated by two distinct constructs: status and power. Both are represented as a relational scalar that exists between two social entities. The amount each entity attributes to the other is not necessarily reciprocal in status or in power.

In Kemper’s theory [31], the status one attributes to another signifies the extent to which one will voluntarily respect or comply with the wishes, needs, and interests of the other. Power, on the other hand, represents the negative side of relational behavior as it refers to our ability to coerce others to act in our favor. However, using power reduces our status and if the source of power is lost then those previously coerced will be less willing to comply afterward.

The notion of social importance (SI) in the SID model aims to operationalize Kemper’s notion of status. As a starting point, we assume that neither agents nor users will attempt to coerce or manipulate others and therefore, the proposed SID model does not represent Kemper’s notion of power. There are several factors that will influence how much SI we attribute to others, including but not restricted to, friendship, reputation, group membership, professional occupation, conformity to existing norms, and family ties. All these factors can have an impact on our willingness to act in the interest of another. Moreover, our cultural background plays a major role in determining which factors are more important than others. For instance, group membership will be more important in collectivistic than in individualistic cultures.

In terms of how SI affects our behavior, it works both as a motivational source and a restraining factor. The former is visible when someone is compelled to perform a conferral act to signify the amount of SI that someone has. Such desire comes from the need to reinforce or improve the relation, with different acts conferring different amounts of SI. For instance, consider the difference between explaining directions to someone who is lost and accompanying the person to the desired destination. The restraining aspect takes place when considering how much one should ask to have others act in our interest, as it will largely depend on the amount of SI they attribute to us. If our request exceeds the limit of what we could request, the other person will likely not comply in the way we would like and it is possible that our SI will be lowered in their mind.

Aside from the conferral acts that are made in response to explicit claims, it is also possible that the situation itself implicitly evokes a conferral as part of an established ritual. For instance, the situation of meeting a friend implicitly evokes a greeting action as a conferral act, with different types of greetings conferring different amounts of importance.

The aim of the SID model (see Fig. 1) is to increase the social intelligence of regular BDI agents by integrating the aforementioned notions in their reasoning and behavior. We chose the BDI model [43] as the basis for our work as it is a very well-known model for rational agents and has been extensively developed and applied in the field of autonomous agents since its creation. The central concepts of the BDI model are the following:

• Beliefs—these represent the knowledge the agent has of the environment, of it self or of other agents. Usually, beliefs are represented as logical predicates. For instance, the knowledge that John is an in-group member can be represented with the predicate inGroup(John). In our model, $B_{self}$ represents the set of the agent’s beliefs obtained through the agent’s sensors and $B_x$ contains the agent’s beliefs about the beliefs of agent ‘x’.

  Fig. 1

  

  Social importance model applied to the general BDI architecture

• Desires—these are the agent’s motivations. They represent states of the world the agent wants to achieve. In our model, the desires of the agent are represented by G, which denotes the set of abstract goals the agent has. Such goals have the following properties: (1) Name—a unique identifier; (2) Preconditions—a list of logical conditions that need to be grounded and verified in order for the goal to be instantiated; (3) SuccessConditions—a list of logical conditions that dictate when the goal is considered to have been achieved; and (5) ImportanceOfSuccess—a numeric value that specifies how important the goal is for the agent.

• Intentions—these represent states of the world the agent is focused on and committed to achieve at a given moment. In the proposed model, I denotes the set of intentions the agent currently has. Each intention in I is composed of an instantiated goal and a plan of actions to achieve the goal’s SuccessConditions. These actions assume the form of STRIP-like operators [14] where each has a list of preconditions that are required to be verified in order for the action to be performed and a list of effects that are expected to become true if the action is performed successfully.

The aforementioned concepts augment the traditional execution cycle of a BDI agent, resulting on the process detailed on Algorithm 1. In this cycle, the agents continuously sense the environment for new events. When an event corresponds to the appearance of a new agent ag, the mental model of that agent’s beliefs $(B_{ag})$ is initialized with a copy of the agent’s own beliefs $(B_{self})$ and the agent is added to the set of known agents. Then, the agent’s beliefs in $B_{self}$ and the belief model of other agents ($B_{x}$) are revised according to the event perceived, including the beliefs about SI relationships. This process is described in Algorithm 2. After the belief revision process is completed, a list of options is generated by seeing which abstract goals in G can be instantiated and activated according to the revised beliefs. A goal is considered active if a valid substitution is found for all the variables defined in the Preconditions and SucessConditions of the goal. In the third step, the generated options (active instantiated goals) are filtered with regards to their expected utility. If the agent is not committed to an existing intention, the option with the most utility is chosen to become the current intention of the agent. In our model, the utility of an option corresponds directly to the ImportanceOfSuccess of the associated goal. Finally, the agent builds a plan to achieve the success conditions of its current intention and, based on that plan, decides the next action to perform. A plan must not contain actions that are associated to a claim to which the agent has not enough SI to perform. When such claims are encountered, the action is dropped from the plan and an alternative plan is built.

In the following sections, a more detailed description is provided of how each SI element influences the cycle just described.

Impact on belief revision

The values of the SI relationships the agent has with others are stored in $B_{self}$ in the form of $SI([x])=[y]$ where the variable [x]¹ represents the name of the target agent and the variable [y] is how much SI is attributed to that agent. This value is determined through SI Attribution rules, which are meant to encode the shared cultural knowledge of how different factors influence the SI of other individuals. Formally, an SI Attribution Rule is defined as a tuple $\langle T, A, V \rangle$ where:

• T specifies the target of the rule.

• A corresponds to a list of conditions that specify when the rule is activated.

• V the amount of SI the target of the rule gains or loses.

When the agent perceives a new agent in the simulation, it will create an initial belief of how much SI is attributed to the encountered agent. The value is determined by the sum of all SI Attribution rules that are activated when applied to that agent. At each cycle, after revising all other beliefs, the SI of all other agents is also revised according to the rules that are active at that point. This process is detailed in Algorithm 2.

For illustration purposes, consider a simple social simulation involving a user and two agents, John and Paul. Furthermore, consider that both agents are initialized with the following belief set:

• $B_{self} = \{isPerson(John), isPerson(Paul)$, isPerson(User), inGroup(John), $inGroup(Paul), outGroup(User)\}$

When they perceive each other, agent John and agent Paul will use a copy of their own beliefs to initialize their mental model of one another, which for John will be $B_{Paul}$ and for Paul will be $B_{John}$. Both agents will also initialize their mental model of the user, $B_{User}$, in a similar manner. This is a crude initialization process that can be refined in the future, for instance, through the use of stereotypes as it is done in the CAB model [48].

Let us assume that the following SI Attribution Rules are associated with these two agents²:

• AR1 $: \langle T=x, A=isPerson(x),V=20 \rangle$

• AR2 $: \langle T=x, A=inGroup(x),V=5\rangle$

• AR3 $: \langle T=x, A=outGroup(x),V=-5\rangle$

Agents will determine the SI they attribute to one another by checking which of the three rules above can be verified when substituting the variable ‘x’ with the other agent’s name. For instance, Agent John will add $SI(Paul)=25$ to $B_{self}$ by checking that both AR1 and AR3 are verified when the substitution ‘x/Paul’ is made. As for the user, AR1 and AR3 are verified instead so John will add $SI(User)=15$ to $B_{self}$.

All agents will repeat the same process but assuming the perspective of one another. For instance, Paul will use the beliefs it has inferred in $B_{John}$ to determine which rules will be applied when the substitution ‘x/Paul’ is made. The result will be that Paul adds the belief $SI(Paul)=25$ to its model of John’s beliefs ($B_{John}$).

Other than the SI attribution rules, there is an automatic mechanism that decreases the SI attributed to another agent whenever that agent performs an inappropriate claim. The rationale behind this mechanism is to represent the notion that people lose SI if they violate the standards of conduct. The amount decreased is linearly proportional to the exceeded amount of the claim.

Impact on option generation

Options correspond to instantiated goals that represent the desires of the agent, or more precisely, the states the agent is motivated to achieve. In his theory, Kemper argues that there are two main motivations concerning status: (1) wanting to obtain status from others and (2) wanting to confer status to others when it is appropriate. Our model focuses on the latter, by endowing agents with a general desire to perform acts to signify the amount of SI they have ascribed to others. For the sake of simplicity, the model currently does not include strategies for agents to increase their SI in the perspective of others. While this would be relevant to further increase the social intelligence of an agent, implementing such strategies is not trivial and falls beyond the scope of this article, which focuses on culturally appropriate behaviosr. As stated by Kemper, “Culture specifies what concrete acts and to what degree they signify status-conferral” [31]. The aim of the SI Conferrals of our model is to encode such cultural knowledge. In our model, a SI Conferral is formally defined as a tuple $\langle C, A, T, V \rangle$ where:

• C is a set of preconditions that dictate the context in which the conferral is appropriate.

• A is the name of the action that symbolically represents an SI conferral.

• T corresponds to the target agent to which the conferral applies.

• V specifies the amount of social importance conferred by the action.

Concerning the process of generating options, SI Conferrals are converted to regular abstract goals. This conversion occurs in the initialization process of the agent (see Algorithm 3). First, the conditions specified in C are used as the goal’s Preconditions and the performance of A as the goal’s SuccessConditions. Second, the condition $SI([T])>=[V]$ is added to the goal’s Preconditions. This condition verifies whether the SI attributed by the agent to the conferral’s target T is equal or greater than V. Finally, V is used as the goal’s ImportanceOfSuccess after being normalized to the same scale. This means that the utility of an instantiated conferral is linearly proportional to the amount of SI it confers. The rationale is that agents want to confer as much as they think the other agent deserves but not more.

To illustrate the process just described, consider a scenario where the user is playing the role of a character that needs a place to sleep but cannot find a hotel nearby. The user then arrives at a bar where agent John and Paul are talking to one another. The user then decides to ask directions to John. Consider that both agents do not know where the hotel is and both of them have the following SI Conferrals associated:

• CF1 $: \langle C=askedForHotelDirections([x]) \wedge \lnot knowsHotelDirections([SELF])$, $A=apologizeForNotKnowingHotelDirections([x]), T=[x], V=10 \rangle$

• CF2 $: \langle C=askedForHotelDirections([x]) \wedge knowsHotelDirections([SELF])$, $A=explainDirections([x]), T=[x], V=15 \rangle$

• CF3 $: \langle C=askedForHotelDirections([x])\vee askedToSleepOver([x]) \wedge \lnot knowsHotelDirections([SELF])$, $A=offerPlaceToStay([x]), T=[x], V=60 \rangle$

As described previously, John attributes an SI of 15 to the user. As such, only CF1 and CF2 satisfy the condition $SI([T])>=[V]$. As such, if agent John knew the directions to the hotel, both CF1 and CF2 would become active goals and, when filtering the two options, the agent would choose to commit to the intention of achieving CF2, given that its utility, the amount it confers, is higher than CF1. However, because John does not know the directions to the hotel, only CF1 is activated. Consequently, agent John will confer less than it was willing to, which also happens frequently in human interaction.

Note that, in general, any action that is done in the interest of the other person should be associated with a conferral, such as giving advice, offering help, and so on. The greater the effort made to please or satisfy the other, the greater the amount of SI conferred. Ritualistic actions toward others such as greetings, expressing gratitude, and applauding, should also be coded as conferrals.

Impact on action selection

After committing to the intention with the highest utility, the agent must build or choose a plan of actions to achieve the intention and then, from this plan, select the next action to perform. When the aim is to simulate social scenarios, it is often the case that agents can greatly benefit from the help of others, similar to what happens with humans whom are constantly interacting with one another. Cultural conventions establish what seems appropriate or inappropriate to ask of another. The purpose of the SI Claims in our proposed model is to endow the agent with knowledge about such conventions, so that its plans are appropriate in a particular socio-cultural context. Formally, a SI Claim is defined as a tuple $\langle A, T, V \rangle$ where:

• A is the name of the action that is perceived as a claim for social importance.

• T is the target of the claim.

• V it the amount of social importance the action is claiming.

SI Claims affect the BDI process of action selection in the following manner. Before a plan is set in motion, the agent determines whether any action has an associated SI Claim. If such action is found, the agent checks the mental model of the claim’s target T to infer whether the SI that T attributes to the agent is greater than or equal to the claim’s associated V. If it is lower, then the agent determines that it lacks sufficient SI to perform the action and an alternative plan must be developed. If no alternative is possible, then the agent considers that the intention is impossible to achieve in a socially appropriate way and drops it. A person in this situation might seek a secondary plan for increasing SI to then enact the claim. Although interesting, modeling this type of status-seeking is left to future work given its complexity.

To exemplify how SI claims affect behavior, we return to the scenario of the previous example. Consider that agent John and Paul have the following associated SI Claims:

• CL1 $: \langle A=askDirectionsToHotel([x]), T=[x], V=10 \rangle$

• CL2 $: \langle A=askToSleepOver([x]), T=[x], V=60 \rangle$

If, similar to the user, agent Paul also needed a place to stay for the night, the agent could come up with two valid plans: (1) go to the nearby hotel, or (2) ask John to sleep over at agent John’s place. Although the second plan has a much lower execution cost than the first, the claim associated (CL2) is much higher. In order to proceed with the second plan, Paul must evaluate if it whether it has sufficient SI in the perspective of John. This is done by verifying whether the condition $SI(Paul)>=60$ is true in $B_{John}$, which is Paul’s mental model of agent John. Considering the SI Attribution rules described earlier, Paul believes that agent John attributes a SI of 25 to it, so this test is unsuccessful. As a result, the second plan is discarded and agent Paul turns its attention to the plan of sleeping at a hotel after finding out how to get there. Given that Paul has enough SI to perform the action of asking agent John for the directions, the plan is put in motion. After discovering that agent John also does not know the directions, Paul needs, once more, build a new plan on how to get to the hotel.

Although quite simple, the previous example illustrates how relational factors come into play when trying to achieve an instrumental goal (find a place to sleep). Typically, it would be quite socially inappropriate to ask a stranger to stay at his or her home even though it would be more cost efficient that renting a room at a hotel. However, if instead of a stranger, we are sufficiently important to that person from a relational perspective, then it becomes acceptable to make the request and save money. With the SID model, an agent is capable of reasoning in a similar way whereas an agent that only evaluates actions instrumentally is not.

Modeling culture

Thus far, we have established how the SID model can be used to represent the dynamics found in social interactions between people by mediating relationships through conferrals and claims. Depending on the amount of social importance people think is attributed to them, they can form expectations about how much they can claim of others or what conferrals should they expect. Deviations from these expectations can cause negative emotions such as anger and anxiety [49, 53].

We will now discuss how to model the influence of culture on the social interactions that are mediated by the SID model. One possible approach is to manually configure the values assigned to all the elements of the model (attribution rules, claims, and conferrals) in such a way that the resulting behavior of a group of agents reflects the behavior found in a particular culture.

The main disadvantage of this approach is that it cannot be easily adapted to model several distinct cultures. As such, we propose a more flexible approach that is based on the association between a SI component and a Cultural Influence (this matches the concept of a cultural meta-norm from Sect. 4). The latter is formally defined as a pair $\langle D, M \rangle$ where D corresponds to the name of a dimension of culture (e.g. Individualism vs. Collectivism) and M is a multiplier, either positive or negative, that is applied to modify the value V of the associated SI component. This is done by using the following equation, in which Score(D) corresponds to the score associated with the dimension D in the agent’s cultural profile and ranging from 0 to 100:

$$\begin{aligned} V_{modified} = V_{initial} + |V_{initial}| \times M \times \frac{Score(D)}{100} \end{aligned}$$

(1)

The advantage of using this approach is that it becomes possible to adapt the agent’s cultural behavior just by merely changing the scores associated with their cultural dimensions. This adaptation occurs in the initialization process of the agent as described in Algorithm 3.

Culturally-varying SI attribution

As previously mentioned, the SI attribution rules will influence the Social Importance agents attribute to others depending on relational primitives. One very important relational primitive that is affected by the individualism (IDV) versus collectivism (COL) dimension is the in-group/out-group boundary, or moral circle membership [25]. Collectivistic cultures are very sensitive to this distinction, in the sense that people are loyal and helpful toward in-group members while, at the same time, they tend to keep a ’relational distance’ toward out-group members. Individualistic cultures on the other hand are less sensitive to this distinction and everyone expects to be treated equally, regardless of the groups they belong to.

Our model represents the aforementioned notions with the general SI attribution rules that are defined in Table 1. The result of applying these rules is that, the higher the value of COL specified for the agent’s culture, the less the SI agents will attribute to the members of an out-group and the more they will attribute to members of the in-group.

Table 1 General SI attribution rules concerning the in-group/out-group distinction; $v_1$, and $m_1$, are positive real numbers that need to be instantiated for a scenario

Culturally-varying claims and conferrals

Culture affects not only the amount of SI that is attributed to others but also how we judge the appropriateness of certain actions, which in the SID model, is influenced by how much SI an action claims or confers.

Concerning claims, Table 2 shows those in our model that are impacted by the Individualism versus Collectivism dimension. The first corresponds to any form of casual greeting, such as saying “Hi there!” The second claim is associated with questions such as “How are you?” or “What are you doing here?” Finally, the third claim corresponds to an action of joining an existing group, for example sitting next to a group of people in a bar. In all three cases, the amount of SI claimed is lowered by how much IDV is specified for the agent’s culture. This means that, if the culture of the agents is highly individualistic, such actions are less likely to be perceived as inappropriate, even when performed by a stranger. Conversely, in a collectivistic culture, the potential for being morally offensive when performing such actions is much higher, particularly if you are an out-group member.

Table 2 General SI claims that have a lower value in individualistic cultures

With regards to conferrals, Table 3 details those influenced by the Individualism versus Collectivism dimension in our proposed model. As one may see, both also defined as claims in Table 2. The rationale is that given the smaller social risk for strangers to perform these actions in individualistic cultures, their performance is also less socially rewarding and meaningful. This is reflected in the sentence “Hi! How are you?” which corresponds to a typical greeting in the US (a highly individualistic culture) and is often misinterpreted by people from other countries who assume that it is an expression of concern.

Case study

The proposed SID model has been implemented into an existing agent architecture, named FAtiMA Modular [9]. This architecture was chosen as it already provided functionality that, while not specifically related to culture, was essential to have in order to create believable virtual agents. This includes, for instance, the ability to remember past events or the ability to plan a course of actions to achieve an intention. The resulting architecture was then used to develop an intercultural training application named Traveller [7, 36]. Traveller is a tool aimed at young adults (18–25 years old), in which users are encouraged to interact with intelligent virtual agents from different cultures. They come across several interaction scenes, so-called critical incidents, which may be misinterpreted due to differences in culture, in the hopes of increasing their intercultural sensitivity and awareness.

Table 3 General SI conferrals that have a lower value in individualistic cultures

Each critical incident features a different setting, in which the user has multiple interaction options. Based on the choice he or she makes, and the culture of the agents, the perceived SI of the user may change, and the agents will behave differently. In this article, we will focus on the first incident in Traveller, the Beach Bar. The reason is that this particular episode was designed to showcase the cultural influences of our model that are associated with the Individualism versus Collectivism dimension, which we described previously. In total, the socio-cultural behaviour of the agents in the beach bar episode involved the specification of 5 SI Attribution Rules, 8 SI Claims, and 34 SI Conferrals.

The Beach Bar incident takes place in a bar located on a beach to which the user arrives late at night after failing to find the way to his or her hotel. At the start of the scene, there are only two characters sitting in the bar and they are talking to each other. The barman is absent (although he later appears). The goal of the user is to find directions to his or her hotel. To solve this problem there are two main approaches: (1) questioning the bar customers or (2) patiently waiting for the barman to arrive and ask him the directions. There are moments where it is possible for the users to switch between the two approaches. For instance, the user might initially go and talk to the characters, but then change his or her mind and sit away from them and just wait for the barman to arrive or vice-versa (see Fig. 2).

Cultural differences in agent behaviour—individualism versus collectivism

At the start of the scene, the main cultural difference is in the SI the agents attribute to the user based on the rules defined in Table 1. Because the user plays the role of a foreigner, his or her initial SI will be lower if the agents are collectivistic. The agents will also lower the SI attributed to the user when he or she performs an action that is perceived as an inappropriate claim. To make the user aware that he or she did something inappropriate, the agents respond with a frowning facial expression as shown in Fig. 3.

Fig. 2

Finite-state diagram of the flow of the interaction in the beach bar episode. In states $S_1$ and $S_3$ the user is near the two characters and can talk to them whereas in $S_2$ and $S_4$ the user is sitting alone away from the characters. In state $S_5$ the bartender appears and interacts with the user before the scene ends

Fig. 3

Example of two possible conferrals in the beach bar scene in response to a “How are you?” question. The image on the left shows the response in a highly individualistic culture and the image on the right shows the response in a highly collectivistic culture. The frown occurs as a reaction to the user over claiming more than the attributed SI in this culture

Figure 2 shows the different interaction states of how the scenario progresses. In the initial state, $S_0$, the user is presented with only two options. The first, which leads to $S_1$, is to approach the pair of agents talking to one another. The second option is to avoid the characters and sit alone at the bar, which puts the user in the state $S_2$. If the user chooses the second option, individualistic agents will invite the user to join them whereas collectivistic agents will not. This difference occurs because the action of inviting the user to join them is associated to a conferral whose SI value exceeds the user’s initial SI in the collectivistic culture but not in the individualistic one. If the user accepts the invitation, the interaction state becomes $S_1$. Alternatively, the user can only choose to wait for the barman. After the first time the user waits, the barman is still absent and the collectivistic agents ask if the user needs help. The user can then approach the characters, or wait a second time. If the latter is chosen, the barman finally appears and the interaction state becomes $S_5$.

At the start of $S_1$, when the users approaches the characters, the individualistic agents will turn to face the user immediately. However, this conferral will not be activated by the collectivistic agents given the lower SI they attribute to the user. Instead, to make the collectivistic agents turn to the user he or she must explicitly draw their attention. Afterwards, the user can greet the characters either casually by saying “Hi guys, nice to meet you!” or with a more formal approach by saying “Good evening.” Regardless of the user’s choice, individualistic agents will reply with a casual greeting while collectivistic agents will reply with a formal greeting. Moreover, collectivistic agents will exhibit a frown if the user greets them in a casual manner, as they do not think the user has enough SI to perform such action.

After the greeting exchange, the user can remain in state $S_1$ and talk to the characters, or the user can decide to sit next to them, reaching state $S_3$. A third option is to go to state $S_4$ by leaving the characters and sitting alone and then waiting for the barman until he comes. Regarding the option of talking with the characters, the following cultural differences can occur. As shown in Fig. 3, collectivistic agents will not find it appropriate for the user to ask “How are you?” because this action was coded as an ask-personal-information claim. On the other hand, individualistic agents will find this action appropriate and they will take the initiative to then ask the user what he or she is doing there and what the user thinks about the bar. Another cultural difference occurs when the user sits next to the characters, an action that was associated to the join-group claim from Table 2. Individualistic agents will not find it inappropriate. However, the collectivistic agents will and they will hint at it if the user asks permission beforehand, either by reluctantly agreeing (“Ok$\ldots$”) or by making an excuse that they are waiting for a friend to come. If the user sits with the characters he or she can then continue talking or wait for the barman to arrive. When the latter option is chosen, the barman arrives and the interaction progresses to state $S_5$.

Concerning the hotel the user is searching for, he or she can ask the two characters for the directions but, in both versions, the agents will reveal that they do not know where it is because they genuinely do not know. Then, if the user behaved appropriately up to this point, both cultural versions will recommend that the user asks the barman when he returns. However, if the user lost SI by behaving inappropriately, the characters will choose a lesser conferral and suggest that the user to go ask someone on the street.

In state $S_5$, after the barman is back at the bar, the user must once again decide how to greet the character, following the same logic as before when greeting the other two characters. The barman will then ask the user’s character if he wants to order a drink. The user can then either choose to order a single drink, buy a round for everyone, or refuse to order anything. Both collectivistic and individualistic agents will express their gratitude to the user if they are offered a drink and they will also increase their SI toward the user. However, refusing to order anything will be perceived as inappropriate in both cultural versions. Finally, when the user asks the barman for directions, both cultural versions of the agent will explain the directions, provided that the user behaved in a sufficiently appropriate manner and has not lost SI. No directions are given if the user was inappropriate multiple times during the interaction. In both cases, the only option left after this point is for the user to leave the bar, which ends the episode.

Note that, for pedagogical reasons, the differences between the two cultural versions in this scenario were slightly exaggerated on purpose. The reason is that the goal of Traveller is to use scenarios to promote intercultural awareness. As such, using more subtle differences would be more realistic but is also likely that users would not notice them.

Cross-cultural evaluation

We conducted a cross-cultural study to determine how users perceived and responded to the two cultural configurations of the agents in the beach bar episode. The aim was to compare users from both a collectivistic country, Portugal, and an individualistic country, the Netherlands, concerning how they perceived the behaviour of the agents. For reference, Portugal is the most collectivistic country in Europe (scoring 27 on that dimension), and the Netherlands is one of the most individualistic countries in Europe (scoring 80 on that dimension).

The experiment consisted of a $2\,\times \,2$ between-subjects design in which the independent variables were the participant’s nationality (Dutch or Portuguese) and the cultural parameterization of the agents at the beach bar (Individualistic or Collectivistic). The main hypotheses we wanted to verify within this study were the following:

• H1: Portuguese participants have a more positive opinion of the collectivistic agents’ behavior than the Dutch.

• H2: Dutch participants have a more positive opinion of the individualistic agents behavior than the Portuguese.

Experimental design

In both Portugal and in the Netherlands, participants were randomly assigned to play the beach bar scene either with agents whose culture was highly collectivistic (IDV = 0) or agents whose culture was highly individualistic (IDV = 100). All other configuration options of the agents’ behavior were the same. Afterward, they were given a brief explanation of how the user interface worked and that the interaction in Traveller takes place in fictional foreign countries. In the case of the Beach Bar scene, the name of the country is Malahide. Participants were also instructed that Traveller was not a game in the sense that there were no points nor victory conditions. Finally, we explicitly asked participants to try to behave in the simulation as closely as possible to how they think they would behave in the real world if they were facing a similar situation. The reason behind this request was to discourage participants from trying to act in a strange way just to see what would happen in the virtual world, given that this sort of behavior is very common when playing computer games.

After the participants completed the beach bar episode, they were asked to fill in an online questionnaire about their experience. This questionnaire starts with two open-ended questions: (1) “Did the characters behave in a socially appropriate manner? If not, tell us on which occasions they were socially inappropriate.”, (2) “What were you focused on during the interaction?” After these questions, the participants were asked to give their opinion, using a 7-point Likert scale, on how much they agreed on several statements. The following statements are associated with the impression of the characters’ behavior: (1) “My general impression about the behaviour of these characters is positive”; (2) “The characters behaved appropriately toward me”; (3) “The characters made me feel unwelcome”; (4) “The characters treated me as one of their own”; (5) “My presence made the characters uncomfortable”; (6) “The characters enjoyed interacting with me”. Another four statements were related to the participant’s experience and behavior: (1) “It was frustrating interacting with the characters”; (2) “I enjoyed interacting with the characters”; (3) “I felt bored during the interaction in the beach bar”; (4) “I behaved appropriately toward the characters”. Afterwards, participants were asked to describe the characters in terms of several adjective pairs: (1) “Trustworthy/Untrustworthy”; (2) “Assertive/Unassertive”; (3) “Respectful/Disrespectful”; (4) “Polite/Impolite”; (5) “Unfriendly/Friendly”; (6) “Relaxed/Tense”; (7) “Likeable/Unlikeable”; (8) “Caring/Uncaring”; (9) “Tolerant/Intolerant”; (10) “Serious/Cheerful”. Finally, at the end of the questionnaire, we asked participants to fill in their gender, age, and nationality.

For this study, we had 72 participants of which 37 were Dutch and 35 were Portuguese. In both cases, roughly half of the participants interacted with the individualistic culture and the other half with the collectivistic culture. The participants were mostly university students and their average age was 23 in Portugal and 22 in the Netherlands. There were fewer female than male participants in both countries. More precisely, the percentage of female participants was 35 % in the Netherlands and 22 % in Portugal.

Qualitative analysis

As previously mentioned, the questionnaire begins with the following open-ended question: “Did the characters behave in a socially appropriate manner? If not, tell us in which occasions they were socially inappropriate.” Regarding the collectivistic agents, the answers to this question were evenly split for the Dutch participants. Roughly half of them (52 %) replied negatively and the other half replied positively. For the Portuguese participants, there were more positive answers (72 %) than negative ones.

The Dutch participants seemed to be mostly upset by the distance shown by the characters at the bar and the fact they were unhelpful in giving directions. As an example, consider the following answers given by three different Dutch individuals:

The two men sitting at the bar didn’t want me to sit with them, there were enough chairs for their “friend”. Maybe they didn’t know the location of the hotel, but I think they could have helped a little bit more.

Not totally since they behaved in a distant manner and did not do any more effort to help a stranger out.

Both the group and the barman were somewhat socially inappropriate at times. When I said I didn’t want a drink to the barman and when I asked the group how they were doing. I found the first one to be especially rude, because he made me wait an eternity without me getting angry over it, But when I say I don’t want a drink he has the right to be pissed!

Interestingly, although some Dutch participants perceived the characters as distant but, they thought they had the right to act this way. For instance, consider the following elaborate answer:

The characters behaved in a way that probably best suited their own interests. While it is always an interesting experience to learn about their culture by talking to them, if they do not wish to, it should not be asked of them. I would have personally been more open to the random stranger (me) in question, but I do not blame others for receiving a friend or the like (even if that may not be the case, and the friend turns out to be a lie).

The previous answer reflects the notion that individual rights are a primary concern for individualistic cultures [25]. For the Portuguese participants, while there were a few that also thought that the collectivistic agents should have been more helpful, most of them considered their behavior as appropriate. Interestingly, many Portuguese participants explicitly mentioned that they were a stranger to these agents, as shown in the following answers:

Yes, they were actually very friendly considering I was a complete stranger to them.

Yes, they behaved in a correct manner, they were communicating with a stranger and they were quite polite.

Yes, they behaved very well. They seemed a little distant in the end of the conversation but that was normal because I was a stranger to them.

We argue that these answers are supportive of the cultural influence defined in our model that makes the in-group/out-group distinction more relevant for collectivistic cultures.

With regards to the individualistic agents, 67 % of the Dutch participants answered affirmatively to the question related to the social appropriateness of the agents’ behavior. Moreover, most participants who answered negatively focused on aspects that were not very related to our model but were more related to limitations of the scenario itself and the specific choices made for the agents’ utterances. As an example, consider the following answer:

The barman behaved in an appropriate manner. The other two guys at the bar behaved a little bit strange, not necessarily distant, but the replies of them were sometimes odd. The conversation felt scripted instead of social. Some points of the conversation repeated like the location of the hotel.

For the Portuguese participants, the majority of them (70 %) answered that the individualistic characters were socially appropriate. The fact that these agents were more receptive to talking and asking personal questions of the user did not seem to bother the participants. Still, there were a few notable exceptions:

I think maybe not. Possibly because most Portuguese people are somewhat close-minded, and do not invite other people to their group just like that.

I think the characters acted in a socially acceptable manner, yet i think they invited my character to sit with them too early in the interaction. Personally, I would have waited a bit more.

Finally, there was another open-ended question that asked “What were you focused on during the interaction?” In this case, the answers given were very similar across all conditions. The vast majority of both the Dutch and Portuguese participants reported that they were focused on getting directions to the hotel while avoiding being rude to the characters at the bar. For illustration purposes, some of these answers were as follows:

I was focused on my aim (finding the hotel) but not being to stressed and in a hurry.

My main focus was to get the directions for the hotel the most friendly way as possible.

Still, there were a few notable exceptions to this pattern. For instance, the following was said by a Portuguese subject:

People’s receptive reaction. I need to feel that in order to be more comfortable while interacting with people.

User action analysis

The application logged all the actions chosen by the participants in the beach bar scenario. As shown in Fig. 4, the majority of users from both countries decided to approach the group of strangers as their first choice instead of sitting alone at the bar and waiting for the barman to arrive. Given that Portugal is a highly collectivistic culture, we were expecting that there would be more users who sit at the bar initially to avoid interacting with strangers they did not know. However, they behaved very similarly to the Dutch in this regard. It is possible, given the fact that they were controlling an avatar in a virtual environment, that the Portuguese users felt more at ease and were more willing to interact with strangers, even when told beforehand that they should act as close as possible to how they would if facing a similar situation in real life. Alternatively, a large percentage of users may have initially thought that the barman would never appear in the scenario. If so, the two customers were their only option to find the directions to the hotel. Further experiments are needed to have a clearer answer on this matter.

Fig. 4

User’s first choice in the Beach Bar episode

Concerning only the users who talked to the customers in the beach bar, Fig. 5 shows the percentage who asked the question “How are you?” in both countries, for each cultural version. It is important to remember that this question is frowned upon by the collectivistic agents as they perceive it as an excessive claim when coming from a stranger. Overall, the Dutch participants were more inclined to ask this particular question than the Portuguese, especially when interacting with the individualistic agents. This supports our model’s association of this action with a claim that has a lower SI value in individualistic cultures such as the Netherlands. Still, the number of Portuguese participants who also performed this action was higher than expected, particularly when interacting with the collectivistic agents. The fact that Traveller is in English might have led the Portuguese participants to judge this action as less inappropriate than they would have if it was in their native language. This hypothesis would be interesting to test, provided that Traveller is translated into Portuguese in the future.

Fig. 5

Percentage of users who asked “How are you?” in both cultural versions

Finally, after the user has had the chance to speak to the two agents sitting at the bar, the barman finally arrives and asks if the user would like to order a drink. The user can then order a drink just for himself or offer a drink to the two customers as well. He or she may also decide to not order anything at all. Figure 6 shows the percentage of users that made each of these choices in both countries, for each cultural version. The results show that very few Dutch participants offered a drink to the collectivistic agents, particularly when compared with the amount of drinks they offered to the individualistic agents. In contrast, the collectivistic agents received more drinks from the Portuguese participants than individualistic agents. However, the difference in this case was much smaller. Interestingly, it seems that although the Dutch participants were split in their opinion about the appropriateness of the collectivistic agents’ behavior, they do seem to have less affinity for them compared with the affinity they have toward the individualistic agents.

Fig. 6

Percentage obtained for each possible choice when the barman asks to order

Quantitative results and discussion

The initial step in the quantitative analysis of the cross-cultural study consisted of determining whether the scores obtained for the questionnaire’s statements and adjectives followed a normal distribution. By applying the Kolmogorov–Smirnov test, it was determined that most distributions of the variables measured were significantly non-normal. This issue was addressed by applying a rank transformation to the data in order to apply the two-way factorial ANOVA statistical test.

A significant interaction effect between the agents’ culture and the participants’ nationality was found for the statement “My general impression about the behavior of these characters is positive” (F(1,68) = 6.441, p = 0.013) and for the statement “The characters behaved appropriately toward me” (F(1,68) = 4.922, p = 0.03). For these two statements (see Fig. 7), the participants’ opinions about the two cultural versions of the agents was affected differently by their nationalities. A simple effects analysis revealed that the score of the collectivistic agents was significantly higher in Portugal than in the Netherlands for the first statement (F(1,68) = 6.685 p = 0.012) and for the second statement (F(1,68) = 4.678, p = 0.034). Both results support our hypothesis H1. In contrast, hypothesis H2 was not supported as the score given to the individualistic agents by the Dutch was not significantly higher than the score given by the Portuguese, neither for the first statement (F = 1.052, p = 0.309) nor for the second one (F(1,68) = 0.982, p = 0.325).

Fig. 7

Results of the statements that had a significant interaction effect between the participants’ nationality and the cultural profile of the agents

Still, there was a significant main effect of the agents’ cultural version for the first, (F(1,68) = 12.248, p = 0.001) and second (F(1,68) = 5.45, p = 0.023) statements. This effect indicates that, regardless of the country, there was a positive bias toward the individualistic agents for these two statements.

For the remaining statements, the interaction effect was non-significant but most of them also had a significant main effect of the agents’ culture. Again, regardless of the participants’ nationality, the individualistic agents scored higher (see Fig. 8) on the following statements “The characters treated me as one of their own” (F(1, 68) = 18.429, $p < 0.001$), “The characters enjoyed interacting with me” (F(1, 68) = 29.347, $p < 0.001$) and “I enjoyed interacting with the characters” (F(1, 68) = 8.016, p = 0.006). The same agents also obtained lower scores (see Fig. 9) on the statements “My presence made the characters uncomfortable” (F(1,68) = 5.554 p = 0.021), “The characters made me feel unwelcome” (F(1,68) = 18.528, $p < 0.001$) and “The characters were annoyed by my presence” (F(1,68) = 17.16, $p< 0.001$).

Fig. 8

Results of the positive statements that had a significant main effect of the agents’ culture with individualistic agents scoring higher in both countries

Fig. 9

Results of the negative statements that had a significant main effect of the agents’ culture with collectivistic agents scoring higher in both countries

The previous results do not support our hypothesis H2, in the sense that both Portuguese and Dutch participants had a similar positive opinion of the individualistic agents. A possible explanation is that the scenario chosen for this study, namely the beach bar, was not rich enough in the sense that it lacked opportunities for the user to perform socially inappropriate actions from the individualistic agents’ perspective and it lacked opportunities to show aspects of individualistic cultures that would be more strongly viewed as inappropriate by participants of a collectivistic culture. This was due to the strong design focus of the scenario on exploring the negative consequences of being an out-group member (a foreigner), which is associated with collectivism.

Concerning the adjectives the participants chose to describe the agents, there was only one significant interaction effect between the agents’ culture and the participants nationality. This effect concerned the Respectful/Disrespectful pair (F(1,68) = 4.895, p = 0.03). Compared with the Dutch participants, the Portuguese found the collectivistic agents more respectful and the individualistic agents more disrespectful (Fig. 10). This result follows the same logic as the interaction effect obtained for the “The characters behaved appropriately toward me”, which was also significant as mentioned earlier. The closely related pair “Polite/Impolite” showed a similar trend but the interaction effect was not significant (F(1,68) = 1.919, p = 0.171).

Fig. 10

Results for the “Respectful/Disrespectful” pair, the only adjective pair where a significant interaction effect occurred

Apart from the single interaction effect, there was a significant main effect of the culture manipulation for almost all adjective pairs. The only exception was the “Trustworthy/Untrustworthy” pair ($F=0.165, p = 0.686$), whose results are shown in Fig. 11. In both countries, both collectivistic and individualistic agents were perceived as slightly trustworthy. For the other pairs, participants from both countries were again positively biased toward the individualistic agents (see Fig. 12), regarding them, overall, as more assertive ($F(1,68) = 6.632, p = 0.012$), more respectful ($F(1,68) = 5.705, p = 0.02$), more polite ($F(1,68) = 4.905, p = 0.03$), more friendly ($F(1,68) = 19.879, p = 0.001$), more relaxed ($F(1,68) = 7.877, p = 0.007$), more likeable ($F(1,68) = 24.833, p = 0.007$), more caring ($F(1,68) = 10.978, p = 0.001$), more tolerant ($F(1,68) = 7.249, p = 0.009$) and more cheerful ($F(1,68) = 11.296, p = 0.001$).

Fig. 11

Results for the “Trustworthy/Untrustworthy” pair, the only adjective pair with no significant effects

Fig. 12

Results for the adjective pairs that had a significant main effect of the agents’ culture

Finally, there was a significant main effect of nationality for the pair “Assertive/Unassertive” ($F(1,68) = 8.326, p = 0.005$). This means, that regardless of the cultural version they interacted with, Portuguese participants were more inclined to rate the characters as more assertive than the Dutch. This result suggests that the notion of being assertive differs in these two countries. Given the more frequent use of the direct verbal style in individualistic cultures [50], it is possible that the Dutch participants were less sensitive to its use by the agents in the scenario.

Overall, the cultural influences of the proposed SID model, when applied to the beach bar scenario, were able to create collectivistic agents that were perceived in a more negative light by the Dutch than by the Portuguese, which supports our hypothesis H1. This result is important as it constitutes evidence of the proposed model’s capability in adapting the agents’ cultural behavior toward one extreme of the Individualism versus Collectivism dimension. However, the adaptation of the model to make agents more individualistic did not cause a negative impression in the Portuguese participants. Therefore, we could not find evidence to support hypothesis H2. For both Dutch and Portuguese, the individualistic agents were seen in a more positive manner than the collectivistic agents, with a few exceptions. It is important to remember that, in their qualitative answers, some Portuguese participants wrote negative remarks about the appropriateness of the individualistic agents. For instance, they found it strange that these agents invited the user to sit with them immediately. However, it is plausible that for many Portuguese participants, this strangeness was interpreted as unexpected friendliness rather than unwarranted intrusiveness, which would then explain the obtained results.

Conclusion

In this article, we have described an approach to integrate cultural cognitive biases into the behaviour of virtual agents. There were two parts to this approach: the analysis of relevant theoretical constructs that should be considered in such challenge and the concrete operationalization of one dimension of culture—Individualism versus Collectivism—as part of a specific application for intercultural training.

The main contributions of this work can be summarized as follows. The first one is the SID model, a novel extension to the BDI paradigm [43], which makes agents more aware of the social factors that drive human behavior, such as the distinction between in-group and out-group members, and allows for an explicit representation of cultural influences on such factors. The model was formerly introduced in [35] and is described here in much greater detail concerning its formalization and implementation.

The second main contribution is a cross-cultural study of how people from two different countries, Portugal and the Netherlands, perceive and respond to agents with distinct cultural biases as represented in our proposed model. In the conducted experiment, users from both nations had to socially interact with a small group of agents from a fictional country in order to be able to find directions to a hotel. This scenario was designed to showcase the cultural influences from our model that are related to Individualism versus Collectivism. Given that the culture of Portugal is rated as highly collectivistic and the culture of Netherlands is strongly individualistic, the goal was to investigate how subjects from the two countries would differ in their reactions and judgements about the agents in the two cultural versions.

Right after finishing the beach bar episode, participants had to answer an open-ended question that asked in what ways the characters behave inappropriately. The answers given were quite insightful. For instance, many Dutch participants complained that the collectivistic agents were too distant whereas many Portuguese participants mention that the distance shown was appropriate given that they were a stranger to the agents. We also did an analysis of the choices made by the participants in each country and found some interesting results as well. For example, more Dutch participants offered a drink to the individualistic agents than they did to the collectivistic ones and the opposite was true for the Portuguese subjects. Finally, participants were asked to fill in their opinions on a series of Likert scales about the agent’s behaviour and their user experience and to describe the agents in terms of several adjective pairs. Overall, the results obtained for these statements and adjectives showed that the Portuguese participants had a significantly higher opinion of the collectivistic agents than the Dutch. However, for the individualistic agents, both countries had a similarly positive opinion.

There are a few limitations to this work. The first is that we have applied the work only to a specific context in which the social interaction is extremely limited; the agents do not build lasting relationships and the social norms do not change. We only take the first steps in this research. In the future, additional social dynamics would need to be modeled (such as the influence of power, and establishing lasting relationships).

The second limitation is that our evaluation focused on a single dimension of culture in one specific scenario. Other dimensions were also implemented in the model but were explored in other critical incidents that take place in different episodes of Traveller. In future work we aim to test the impact of the biases that are associated with other dimensions of culture to determine whether the agents are able to show a wider range of appropriate culturally varying behavior. This would require additional evaluations with people from different cultures.

The third is that the values for the Social Importance attribution, claims and conferrals, together with the cultural modifiers, are currently predefined according to the interpretations made from the theory that supports the model. In future work, we would like to derive such values by exploring a more data-driven approach. This is one of the most important challenges we plan to address when further developing the proposed model. Still, by focusing on general cultural biases that are associated with social factors such as the relation between cultural dimensions and the in-group/out-group distinction, the model has the potential to scale better when compared to an approach where cultural differences are conveyed by using different specific norms/goals/actions.

The fourth limitation is that the context, and the actions that the user was allowed to select, had a significant impact on the perception and interpretation of the user. In a different context, and with different actions, the user might have had a completely different experience. For instance, by adding more negatively perceived behavior to the individualistic agents, participants from the two countries might have had a more distinct opinion about them. This is related to the authoring process, and the underlying design choices that have been made. These choices are inevitable when designing an experiment that involves an interactive social scenario with virtual agents.