1 Introduction

The research on human-computer interaction (HCI) models has made significant progress over the last twenty years, with the appearance of many interaction devices, as well as design and evaluation methods. Interactive systems are increasingly used in collective contexts, addressing teams and organizations. In parallel, the use of new systems, which introduce assistants for users, whether for the general public or in industrial systems, has spread rapidly over the last few decades [1,2,3].

At the same time, the works in Artificial Intelligence (AI) and in Distributed Artificial Intelligence (DAI), and particularly in multi-agent systems (MAS), have led to different advances in the modeling and the understanding of interactions between intelligent entities [4, 5]. The accepted idea is based on a global coherent behavior of a set of entities (called agents), which are immersed in a dynamic environment.

A cross-fertilization between HCI and AI [6, 7], and between HCI and DAI/MAS [8] gives rise to new perspectives: the interaction becomes intelligent, personalized or so-called context-aware [9, 10]. The notion of agent, able of cooperating by an intelligent way with human beings thanks to a relevant combination of AI and HCI methods, has been also introduced [3, 11]. Such researches represent a cornerstone for the study of team involving humans and agents, contributing to new challenges concerning smart environments [12, 13].

Our goal is to propose interactions between assistant agents and humans, without explicit communication, which has the advantage to reduce significantly the processing costs for the exchanged information. It is also important that agents do not systematically assist human beings, continuously, so as not to risk decreasing their level of expertise [14]. With this objective, it is possible to investigate the approaches inspired by game theory. At each iteration, the interaction is thus formalized by a decision matrix, representing a game between players (active entities corresponding to assistant agents and humans). This idea was initiated, for example by [15] for software agents without human participants. According to us, such a model remains valid in our context involving also humans, since the software agents will make a decision following the building of their own matrix.

The research question is thus the following: How to design and evaluate a model of interaction between these different actors, based on decision matrices? In order to show its feasibility, a participatory simulation applied to a road traffic was studied. This application seems to us to be suitable, because the traffic management problem is recognized as a difficult one [16,17,18]. Thus, this task for human participants requires an analysis of the global situation: to respond to a congestion, the search for the intersection to select depends on the other intersections connected to it. A wrong choice can thus lead to an increase of the congestion, or even to a degradation of the global traffic. Furthermore, by increasing the density of vehicles in the environment, the human can quickly become overwhelmed by the task, most often with the need to share efforts with other human participants and/or software agent(s).

The remainder of this paper is organized as follows. Section 2 proposes related works concerning existing interaction models. Section 3 presents our proposal for an interaction model based on decision matrices inspired by game theory. Our proposal is evaluated in Sect. 4, involving a participatory simulation in the realization of a road traffic management. The results obtained by five groups of participants will be also analyzed in order to show the feasibility of our model. Section 5 discusses our approach in a general context. The last section concludes and proposes prospects for future research.

2 Related Works

We briefly present existing models of interaction between humans and software agents 2.1, the participatory simulation 2.2, and finally the target application domain 2.3.

2.1 Models of Humans and Software Agents Team

The interaction modeling involving between humans and assistant agents has been the subject of several proposals and case studies. A stochastic model has been proposed by Levin et al. [19] for human-agent interactions based on Markov decision processes, and was tested for learning strategies in an air traffic control task. Holzinger et al. [20] demonstrated that the intelligence of human beings can improve the intelligence of agents in cooperation with them in a game based on the principle of ant colonies. This game is about finding the fastest way to collect resources.

Azaria et al. [21] proposed another interaction model using strategies that exploit agents which can become advisors for the human beings who cooperate with them. In addition, Barange and Pauchet [22] have proposed agents qualified as pedagogical agents, in order to allow the user to benefit from a kind of teaching and human learning through a new architecture of pedagogical agent.

Another approach is user profiling in personal information agents. In this case, in order to provide personalized assistance, agents rely on representations of user information interests and preferences; they are most often contained in user profiles [23]. Our approach does not consider the individual preferences and interests, but it focuses on collective objectives.

Finally, a model inspired by game theory has been proposed in the context of interactions between humans and agents [24]. However, this approach did not directly describe the human characteristics taken into account; it does not consider explicitly the context of use related to the environment in which an interactive system evolves.

With regard to the methods described, it can be seen that no approach, whether centralized or distributed, makes explicit the direct involvement of user contexts related to interactions.

2.2 Participatory Simulation

Etymologically, participatory simulation can be defined as a simulation in which several actors can participate simultaneously or in turn in a simulated system [25]. To our knowledge, this type of simulation has been developed for the first time in the context of systems dynamics and systems human learning by [26] at MIT during the sixties.

In the field of multi-agent simulation, several platforms offer this possibility of participatory simulation. In particular, Wilensky and Stroup [27] used participatory simulation to be able to set up a human learning system based on Netlogo platform. In this one, the teacher plays the role of monitor or observer, while the students are the participants who must log in to participate in the simulation, interacting with the simulator. In addition, Taillandier et al. [28] presented participatory simulation based on GAMA modeling.

Cases of participatory simulation have concerned the simulation of road traffic [29], air traffic [30], and more recently, in a context related to the Covid-19 pandemic [31]. But to our knowledge, the proposed participatory simulations essentially allow human beings to participate in a simulation via a platform. There is no direct involvement of assistant agents or multi-agent assistance systems.

2.3 Our Case Study: Road Traffic Simulation

With the increasing size of road traffic in most countries, enforcement is becoming more and more complicated, particularly in order to limit congestions in urban road traffic. Several works focused on the use of intelligent agents to solve this traffic problem. To reduce the congestion problems at road intersections, various approaches have been explored [32,33,34,35].

In spite of their diversity, none of the above-mentioned approaches explicitly took into account human operators who may be assisted by agents in the traffic supervision. In an interactive system, involving humans, each of them can be characterized by several aspects according to the context model. For example for the user profile, one can consider the knowledge level in relation to the functioning of the system [36]. The instantaneous workload level can also be considered [37, 38].

To our knowledge, an interaction model using decision matrices inspired by game theory has not been the subject of an approach involving directly several human beings and assistant agents. In the following, we propose a generalization of the interaction model previously proposed in our works [39, 40]. Let us note that the objective of this model is to ensure the effectiveness of the human-agent team.

3 Proposal of a Model Involving Humans and Agents

This section deals with the description of the proposed interaction model. As it is inspired by game theory principles, this section begins with brief reminders 3.1, before tackling the formalization itself 3.2 and 3.3. Finally, a numerical analysis of decision matrices is provided 3.4.

3.1 Game Theory Reminders

A game [41, 42] is a decision matrix based on numerical values associated to utilities \(u_i\) representing the gains or losses (payoffs) of each player i according to their strategic choices (among a set of possible actions called strategies). We will call \(s_i \in S_i\) a strategy for player i; and \(s_{-i}\) the strategies of other players. Their decision-makings are based on game rules, characterized by a search of equilibria.

In our context, the interactions between several players (humans and agents) may be described by using the notion of multiplayer, repeated (analysis of the game matrix at each iteration), non-zero-sum game (if the sum of the players’ gains is not always equal to zero). The resolution is based on Nash equilibria [43], which will be used by the agents for their decision-making.

The step of choosing a equilibrium and its resolution are not really the subject of this paper; we propose to use the algorithm described in [44]. In our opinion, the difficulty lies in the construction of the decision matrices which should take into account their heterogeneous behaviors. But the advantage is to provide a calculation of their utilities and to search for an equilibrium (and thus to obtain the decision of each agent at each iteration).

3.2 Modeling a Multi-Player Game

3.2.1 General Principle of Interaction

We assume that human beings are all players capable of making rational decisions. However, they are not going to choose an action based on their payoffs because only the assistant agents build the game matrix to make their decision. They carry out their task autonomously, and can ask implicitly for help if necessary through context descriptors (such as their experience level and their workload level); while agents characterize their game payoff matrix (based on perceived information) and can intervene (or not) on a specific task. In the case of non-intervention, they can take care of their own tasks.

Fig. 1
figure 1

Use case diagram for the proposed system

Full size image

The use case diagram of the participatory simulator using our interaction model is visible in Fig. 1. It shows the different actors of participatory simulations: (human) observer, one or several human operators and one or several assistant agents. The following subsection details the possible actions of the game matrix; while the human tasks through the user interface are presented in Sect. 4.1.3.

3.2.2 Description of Possible Actions

For reasons of simplification, we build a game payoff matrix by assuming that the different players/actors can choose one action from two. The designation of the two actions resulting from the prisoner’s dilemma game: C for an action to cooperate, and D a second action not to cooperate (corresponding to ‘Defect’).

The human being requests the intervention of an assistant agent (noted action C). In this case, the current situation leads the human operator to request the assistant’s help. The human being works without the help of the assistant (noted action D); this one does not need the intervention of one of the agents. The current situation is such that the operator thinks the evolution of the environment is under control. Similarly, software agents can intervene, inform or warn human beings (represented by the action C). The situation evolves critically or one of them responds to a request from a human being. They can also let human beings work alone (characterized by action D).

As part of their decision-making, software agents perform the following two steps in a cyclic process. A first step considers a task allocation between the agents according to the evolution of the environment (in our case, road traffic to be regulated by traffic lights). In the current state, the algorithm described for this task allocation is based on FIFO processing. A second step is based on the intervention or not of each assistant agent, with respect to the resolution of the game matrix.

3.3 Payoff Modeling for the Representation of the Game Matrix

The goal is to obtain a context-sensitive interactive system. For this purpose, we propose to use the following elements as context descriptors in the definition of utility functions 3.3.1, and of the game matrix 3.3.2 and 3.3.3.

3.3.1 Context Descriptors

Several context descriptors may be used, as suggested in [45]. We propose three context descriptors: (i) workload level, (ii) experience level of the human operator, and (iii) criticality level of the situation.

According to Lyall [46], “Workload has been described as an indicator of the level of total mental and/or physical effort required to carry out one or more tasks at a specific performance level”. This notion is particularly interesting for the case of a context-sensitive interactive system; because it can influence the performance of the human operators and also of the global system. When the workload level is higher, it will necessarily modify their behavior and ability to control. Thus, each human operator may make errors. According to Kirwan et al. [37], we can consider five values for the workload level of a human operator: No activity (Level 1), Relaxed (Level 2), Comfortable (Level 3), High (Level 4) and Excessive (Level 5).

According to Dreyfus’ model [36], experience level depends on a person’s acquisition of knowledge in relation to the education level. They have proposed a scale of five values to express the experience level: Level 1 (Novice), 2 (Advanced Beginner), 3 (Competent), 4 (Proficient) and Level 5 (Expert).

By definition, criticality level refers to the consequences of incorrect behavior of a system that is under the supervision of human operators. In the study of an interactive system, the risk is the main factor to be considered in order to minimize possible damages. By extending the study initiated by Eckhoff [47], we choose to define a five-level scale for the criticality level of the situation. These levels correspond to degrees of criticality: Normal (Level 1), Harmless (2), Unpleasant (3), Dangerous (4) and Uncontrollable/critical (Level 5).

In the following, we assume the following notations to describe these payoffs: \(H_k\) denotes the human being of index k and \(A_j\) refers to the j-index agent; \(s_{-k}\) the actions of the other players (i.e. those of assistants and human beings other than \(H_k\)) and \(s_{-j}\) the actions of the other players (i.e. those of assistants and human beings other than \(A_j\)); crt is the criticality level; ca characterizes the number of agents which choose to act as assistants and da represents the number of agents who do not answer to human demands; ch means the number of human beings who have chosen to cooperate (they request the intervention of the agents); dh is the number of human beings who have chosen autonomy (they do not seek assistance) and dh denotes the number of human operators who do not request help from the agents (i.e., the number of human operators who select strategy D); n is the total number of human operators; \(usrl_k\) denotes the experience level of the human operator of index k; and \(wl_i\) (respectively \(wl_k\)) corresponds to workload level of the human operator \(H_i\) (respectively \(H_k\)). We now propose the modeling of utilities/gains based on these descriptors for the different types of players (human beings, software agents).

3.3.2 Definition of Human Beings’ Payoffs

Each human being’s payoff/gain also depends on the responses of other human beings. If the human beings ask for help, their gain increases with the number of assistant agents which respond, as well as with their workload level. On the other hand, the gain decreases with their experience level, the number of agents which do not intervene, and the criticality level of the situation. If a human being is autonomous (i.e. does not need help), the intervention of assistant agents is of no real importance for this one, in relation to other humans who need help. This may be the case if the human being has a high experience level, without having a high workload level. In spite of this, the criticality level is also introduced in the calculation of the gain to consider its importance in relation to the decisions of each human and agent.

The k-index human being asks for help (Action C):

$$\begin{aligned} U_{H_k} (C,s_{-k}) = ca \cdot \sum _{i=1}^n wl_i - da \cdot \sum _{i=1}^n usrl_i - dh \cdot crt \end{aligned}$$
(1)

This equation results from the following three terms. The first term \(ca \cdot \sum _{i=1}^n wl_i\) is the product of the number of agents wishing to intervene (cooperate) by the sum of workload levels. The higher the demand due to the workload level and the number of agents wishing to intervene, the more justified the request for help from the human \(H_k\) will be. The second term \(- da \cdot \sum _{i=1}^n usrl_i\) is the product of the number of agents not wishing to help, and an evaluation of the collective experience level. The higher the collective experience level, the more the request for help may not be really adapted (i.e., the human has the capacity to work without the intervention of assistant agents). The third term \(dh \cdot crt\) is the product of the number of humans who do not wish to be assisted, and the criticality level. In case of high criticality levels and the non-intervention desired by a majority of participants, not being helped is likely to have a negative impact.

The k-index human being works autonomously (Action D):

$$\begin{aligned} U_{H_k} (D,s_{-k}) = - ca \cdot \sum _{i=1}^n wl_i + usrl_k - \left( crt + wl_k\right) \end{aligned}$$
(2)

Equation 2 is also a sum of three terms. The first term \(-ca \cdot \sum _{i=1}^n wl_i\) is the product of the number of agents willing to cooperate, and the collective workload. If the human does not want the intervention of an agent while the collective workload level can be high, then this leads to a negative cost. This one will increase with the number of agents willing to cooperate. The second term \(usrl_k\) is the only positive value for this payoff because by working without any request for help, the humans rely only on their experience level in the hope that they will have the ability to handle the situation. This highlights the importance of the experience level. The third term \(- \left( crt + wl_k\right)\) is also a negative gain for a human who works without asking for help. Indeed, if their workload level is high, the humans should have asked for help rather than taking the risk of working alone. Similarly, if criticality level is high, the humans should seek help rather than work alone (and risk increasing their workload level further).

3.3.3 Definition of Agents’ Payoffs Modeling

Assistants’ payoffs also depend on the choices of humans (whether to solicit them or not), as well as on the choices of other assistants, while taking into account information from the context.

In case of no intervention, the values can be chosen as appropriate. When humans do not need help, the assistant should not react to the risk of wasting time unnecessarily. The assistant leaves the human beings work alone and understand the situation themselves, and then try to solve it [48]. When one or more humans have asked for help, the intervention of an agent is important because it has reacted as one would expect. In this case, its gains also increase with the workload level of the human beings and with the criticality level of the situation.

According to the above principle, we can propose the following utility functions in order to be able to build the game matrix for the case of assistants and human beings. The utilities are aggregated functions depending on the difference between the workload and experience levels for the human participants, related to the criticality level. Two equations characterize the payoffs of the assistant agents for their two actions, namely the willingness to intervene (denoted C) or the willingness to not cooperate in order to perform other tasks (Action D).

The j-index assistant does intervene:

$$\begin{aligned} U_{A_j} (C, s_{_j}) = ch \cdot \sum _{i=1}^{n} wl_i -\sum _{i=1}^{n} usrl_i - dh \cdot crt \end{aligned}$$
(3)

Equation 3 results from the following three terms. The first term \(ch \cdot \sum _{i=1}^{n} wl_i\) is the product of the number of humans willing to cooperate and the collective workload. The assistance of an agent is much more useful (justified) if humans need help, in case of a high collective workload. The second term \(- \sum _{i=1}^{n} usrl_i\) is the sum of human experience levels. This quantity is negative for the agent if it wants to intervene, as it is better to be busy with another task rather than helping humans with a high collective experience level. The third term \(- dh \cdot crt\) is the product of the number of humans not willing to cooperate and the criticality level. It is negative for the agent because, in a critical situation, most humans should not take the risk of working alone.

The j-index assistant does not intervene:

$$\begin{aligned} U_{A_j} (D, s_{_j}) = ch \cdot \sum _{i=1}^{n} usrl_i -\sum _{i=1}^{n} wl_i - dh \cdot crt \end{aligned}$$
(4)

In Eq. 4, the first term \(ch \cdot \sum _{i=1}^{n} usrl_i\) is the product of the number of humans willing to cooperate and the sum of the human experience levels. It underlines the importance of the collective experience level for the agent which does not intervene. Indeed, it constitutes a positive value for the agent: if this quantity is high, the non-intervention is justified. In this case, the agent has the possibility of dealing with another task. The second term \(- \sum _{i=1}^{n} wl_i\) refers to the collective workload. It is a negative value for the agent’s payoff. Indeed, if it is high, this agent should intervene to try to help the humans to decrease it. The third term \(- dh \cdot crt\) is the product of the number of humans not willing to cooperate and the criticality level. As expressed for Eq. 3, it is negative for the agent because in a critical situation most humans should not take the risk of working alone.

3.4 Numerical Analysis of Decision Matrices

The decision matrices are based on a set of winning strategies for each player. To express the notation used, for instance for two humans and two assistant agents, an equilibrium noted CCDD means that the two humans need help (Action C) and that both assistant agents decide not to intervene (Action D). We propose a brief analysis of different team compositions 3.4.1, and then, we focus on illustrations for our case study 3.4.2.

3.4.1 Numerical Analysis for Different Human-Agent Team Compositions

We have performed different team compositions: from one to three humans, assisted by one to three agents. In a global way, let us study the cooperative strategies where all the actors (human beings, agents) of the team cooperate (CC, CCC, etc.), as well as those where none of the actors cooperate (so-called competitive strategies: DD, DDD, etc.). We varied the values (from one to five) of the three context descriptors (workload, experience and criticality levels), and determined the Nash equilibria for these different decision matrices.

Table 1 presents the percentage rate of these two strategies, relative to the set of possible equilibria. We notice that for a fixed number of humans, the increase in the number of agents decreases the percentage of these strategies. This first observation can be explained by the difficulty of obtaining these two strategies among those determined by the Nash equilibria. To confirm it, let us note that the various numerical simulations have highlighted an average of 1.77 Nash equilibria for all the team compositions visible in this table. For our case study (2 humans, 2 agents), this percentage is 1.51. This result seems to us reasonable for the choice of the agents’ strategy, with an aim of implementation and simulation.

Table 1 Nash equilibrium percentages for cooperative and competitive strategies (H: Human, A: Assistant agent)
Full size table

Among the different Nash equilibria obtained, we were interested in the strategies requiring at least one cooperating agent. In our case study, this percentage is 62.30%. The agents will not systematically intervene, which should allow the humans to keep their expertise by working autonomously. From the point of view of humans, assistance will not be systematic: this is a design choice. Indeed, as suggested by several authors, such as [14], humans must continue to perform their task, without thinking that they can be replaced or continuously assisted by one or several agents. To go further in the objective of maintaining expertise, a research perspective could consist in setting thresholds in the percentage of participation of at least one agent.

3.4.2 Numerical Illustrations for a Team of Two Humans and Two Assistants

Consider again the case studied in this paper, of two humans and two agents. As first illustration, for \(wl_1=wl_2=1\), \(usrl_1=usrl_2=1\) and \(crt=2\), the building of the decision matrix based on the Eqs. 1–4 (see 3.3.2), provides a single equilibrium: DDDD. In this case, where the workload is low and the criticality low, the cooperation of the agents is not necessary.

As second illustration, for \(wl_1=wl_2=4\), \(usrl_1=usrl_2=1\) and \(crt=1\), the building of the matrix provides the following two equilibria: CCCC and CDCC. The criticality is low, but the human workload here is high, with one of the two having a low experience level. This configuration requires the cooperation of the agents.

As third illustration, for \(wl_1=wl_2=1\), \(usrl_1=usrl_2=1\) and \(crt=1\), the building of the decision matrix provides the following five equilibria: CCCC, DDCC, DDCD, DDDC and DDDD. This implies that the two agents can cooperate in two cases out of five; (ii) only one of the two agents can cooperate in two cases out of five; (iii) they cannot cooperate in one case out of five, this one being plausible given the low criticality. We have chosen a variability of the agents’ behavior in their decision, hence the possibility for them to select (randomly) a strategy (as specified in 3.1). The absence of systematic interventions should allow it not to decrease the level of expertise. It should also be remembered that human beings cannot explicitly request the agents, and are therefore not directly concerned by the strategies proposed in these equilibria. Thus, the calculation of the matrix takes into account information concerning humans and the environmental context, but is not taken into account in human decision-making.

In summary, the payoffs previously introduced allow the building of decision matrices for the different software agents. Thus, at each iteration, they can determine the action to be performed according to the context. The proposed modeling has been implemented in Netlogo and studied in a road traffic supervision task. In this participatory simulation, the feasibility was evaluated with groups of humans assisted by software agents.

4 Case Study Involving 2 Humans Assisted by 2 Agents

The section illustrates a case study, in a traffic supervision context. We present some details on our tool supporting the proposed interaction model 4.1, give different obtained results 4.2, and propose an analysis of these results 4.3.

4.1 Implementation Based on Netlogo Platform

After a brief introduction of Netlogo platform 4.1.1, we describe the overall architecture 4.1.2 and the application 4.1.3.

4.1.1 Presentation of Netlogo Platform

Netlogo is a platform for agent-based simulations [49, 50]. An overview of different multi-agent platforms can be found in [34] and [51].

For this work, we used Netlogo’s initial basic traffic model to develop a road traffic simulator. In Netlogo, the hubnet server allows the connection and the access to the user interfaces for the different users: the participants of the participatory simulation and the observer. We have completed and revised this initial basic model by adding functionalities. They include the possibility of lanes of any width, the management of traffic lights at intersections, the use of Hubnets in the framework of a participatory simulation with one or more users as well as the creation of one or more autonomous agents capable of carrying out management tasks at any intersection.

4.1.2 Overall Architecture of the Proposed System

Road traffic management is recognized as one of the complex tasks that can involve human beings to deal with regulation at each intersection. The assistants are intended to help them in the performance of these tasks with two actions: to take special care of an intersection or to leave the task to human operators; and to manage the traffic lights installed at each intersection. For this purpose, rules may be evoked to justify the importance of the intervention or not of the agents: an agent may look for an intersection that has the maximum number of vehicles waiting to enter it (for example, vehicles stopped by the red traffic light in front of them). In this case, the agent may change the color of the traffic light, considering that this change could allow traffic to flow smoothly. In this way, each assistant has an occupancy zone and can select an intersection that has the minimum number of vehicles waiting to enter the intersection (stopped by the red traffic light).

Figure 2 shows the overall architecture, proposed in order to launch the participatory simulation (by the observer). Several users (or human operators) may connect with the traffic simulation application, housing internal agents such as vehicles and traffic lights installed on each side of the entrance to an intersection. Then, the application can record the data related to the traffic and the connected users for each cycle, allowing an analysis of the results after each simulation run. For the traffic supervision, a fixed number of assistant agents is initially created by the observer.

Fig. 2
figure 2

Architecture of our participatory simulation

Full size image

4.1.3 Description of the Traffic Simulator Application

The hubnet server allows a participatory simulation. Thus, the connected clients can participate in a cooperative way in the management of traffic lights in order to obtain the best possible traffic flow. To simplify, we only consider the intervention on the traffic lights, although human operators can act on other components of the client interface.

The interface of the simulator server is used by the observer (Fig. 3). It contains three parts (from left to right). The left part (Command area) contains all the elements allowing the initialization of the simulator environment, such as the choice of road dimensions (number of lanes, of intersections, etc.). The next part contains the traffic view (vehicles moving on roads) and an area showing the evolution of the equilibria (Nash’s equilibria results). The last part is a hubnet control center window containing the list of connected clients, the IP address of the server and the port number used (hidden for security reasons).

Fig. 3
figure 3

Screen page used by the observer of the participatory simulation

Full size image

In the screenshot shown in Fig. 4, the left side of the screen contains three types of components. The components enabling the participants (in Netlogo platform, it is called a hubnet client) to act on the traffic (Command area): to change the signal lights at intersection, or to change the maximum speed (in this case, only the last change made by a client is taken into account). The components (Level estimation area) allowing human operators to simulate the estimation of their own workload and experience level. The components (Information area) allow to see certain information, such as the list of information about the human operators, the number of interventions carried out by each human operator, and the criticality level of the traffic.

Fig. 4
figure 4

Screen page used by a participant

Full size image

Thanks to the hubnet client interface, the human operators connected to the simulator server have the choice to take the initiative on an intersection. They can choose and change the corresponding traffic lights, and can also wait for assistants’ help in managing traffic lights. Their estimation is based on the workload and the experience levels, according to their expertise and the evolution of the simulated environment. Each connected client can also consult information about the workload level of the other operators as well as their experience level, to help their decision through the hubnet client interface.

For this task, the agents and the humans can thus carry out changes of traffic lights with the common aim of obtaining a maximum traffic flow.

4.2 Participatory Simulation and Results

In this sub-section, the protocol and the participants involved are presented 4.2.1. Then, the simulation sequence is explained 4.2.2. Finally, the results of participatory simulation are described 4.2.3.

4.2.1 Description of the Protocol and the Involved Participants

The protocol followed by the different participants (adapted from [40]) during the experiments is based on five steps: (1) answers to a questionnaire: usual questions (e.g., age, gender, education level) and the knowledge of our context (e.g., road traffic simulator, Netlogo platform), (2) explanations about Netlogo platform and the user interface (description of the environment and hubnet of this platform, estimation for experience level and workload of users, and the main commands for the traffic control), (3) explanations concerning the possible interventions of assistant agents (which are characterized by different elements such as the evolution of the traffic, the detection of gridlocks and the human-agent cooperation), (4) participation through their interventions during the participatory simulation, and finally (5) giving a feedback by answering a questionnaire (Fig. 5) after the simulation.

Fig. 5
figure 5

Questionnaire after the experiments

Full size image

For our study, 10 participants were recruited and randomly assigned (see Table 2). They had to meet certain criteria to carry out the participatory simulation: they all have good visual and motor skills and are not prone to dizziness. Five groups of two people were formed, because our case study focuses on the interaction between two humans and two agents. Their ages range from 21 to 28 years old. They were characterized by education (from Bachelor 1 to Master 1) and experience levels (1 as beginner level, 2 as intermediate level, for participants who have already had the opportunity to use a traffic simulator).

Table 2 Characteristics of participants in the experiments
Full size table

4.2.2 Simulation Sequence

On a two-lane road network, we started out with 200 vehicles (a number of vehicles giving a good compromise between the simulation constraints and its difficulty level for the traffic management) that can circulate freely and randomly distributed for each section of road in 2D space. Initially at rest, the vehicles move along one axis through the junctions. Each vehicle must respect an inter-vehicle safety distance of 0.5 patch (in the sense of the Netlogo space grids) and must stop if it is behind a stopped vehicle or in front of a red traffic light at the entrance of an intersection. On the other hand, a vehicle can move forward if it crosses a green traffic light, while always avoiding contact that could cause an accident.

The environment contains nine intersections, each with four traffic lights at the entrance. Traffic management aims to obtain a maximum value for the average speed of traffic through the lights at each intersection. Another objective is also to minimize the average waiting time of vehicles and the number of vehicles stopped at each cycle. Each human operator is responsible for a traffic area by managing the corresponding traffic lights.

Each agent can intervene at any intersection in the road network by giving priority to its area by acting on the traffic lights. In this case study, the intersections are divided into two subsets of intersections. Consequently, for nine intersections, four of them are allocated to each agent at the initialization of the simulator. To emphasize the importance of other tasks that the agents can handle when available, an intersection is not explicitly assigned to an agent at the beginning of the simulation. Thus, an agent could take care of other tasks: management of the last intersection or other tasks which are not specified here. Let us note that one intersection can be managed by one human or one agent.

4.2.3 Participatory Simulation and Results for 5 Groups

Our evaluation is based on decisions taken by each player on the overall traffic behavior. Using Netlogo, we ran the simulator for 5 groups of 2 human operators during 60 iterations.

The performance can be measured in terms of average traffic speed, numbers of stopped vehicles and average waiting time for these vehicles (Fig. 6). The following figures show the obtained results according the measured criteria: average speed (Fig. 6a), number of stopped vehicles (Fig. 6b) and waiting time (Fig. 6c) for five groups. The notation Gi (\(i=1\) to 5) refers to the evolution in average, at each iteration, of vehicles for Group i.

Fig. 6
figure 6

Obtained results for the five groups

Full size image

Figure 6a illustrates the average speed of vehicles. At beginning of different participatory simulations, the average speed increases strongly, and then tends to decrease very quickly for the different groups. The reasons of the abrupt decreasing after increasing is the following: when the various vehicles (defined with zero speed) are created, their movement allows them to reach their desired speed (a maximum value). But as traffic density increases, their speed decreases just as sharply as the initial increase. A unique group (G4) succeeds to maintain a higher speed (approximately, twice as fast as the other groups).

Figure 6b shows the number of stopped vehicles for the five group of participants. Similarly to Fig. 6a, at the beginning of the simulation, the reasons of the abrupt increasing after decreasing is the following: the number of stopped vehicles (i.e. with zero speed) decreases sharply as vehicles accelerate; these vehicles are then confronted with intersections as they move, forcing them to slow down, and for some of them to stop at traffic lights. After several iterations, the number of stopped vehicles seems to be steady (the set of 200 vehicles should be stopped). The different participants have some trouble to maintain a normal functioning of the global system. However, we note that group G5 is able to better adapt the traffic flow for this criterion.

Figure 6c deals with the waiting time for the five groups, and differentiates the different groups for this criterion. This figure shows again that group G4 succeeded to better manage the global system. Another group (G2) owns the worst result for this criterion (close to 1200 for cumulative waiting times, i.e. the double of other groups).

In these figures, let us note that the performance measures are increasing over the iterations. When the humans play multiple rounds of traffic simulations, their performance cannot be improved. Indeed, the environment continuously introduce new vehicles in the simulation, in order to increase the human workload.

4.3 Results Analysis

Following the participatory simulations carried out, the results show the feasibility of the interaction model. Indeed, we analyze below, group by group, these results (note that only two colors, red and green, were used for the traffic lights; yellow was not used in order to simplify (1) the participatory simulation for the participants, and (2) the simulation of the intervention of the assistant agents). This case also shows the interactions between two agents and two human operators in the traffic management. Here, each agent involved in the game seeks Nash equilibria for decision making on all intersections (including those supervised by human operators). The participatory simulation was carried out with a number of iterations equal to 60 to have a margin of choice for both human beings.

Let us start with the results obtained by the participants of group G1. At the start, like all other groups, there is a peak in Fig. 6, for all groups, in the average speed corresponding to the starting of all vehicles. Before the iteration 14, we also observed a still stable traffic situation. Between iterations 14 and 21, during which the assistants have always considered equilibrium, it can be seen that the assistants do not intervene. During this period, they all have a lower workload level and a high expertise level. This explains the non-intervention of agents which consider that humans have the possibility to act and should not request their intervention. In this case, if the humans do not intervene, the slowing down of the traffic is noticed by the decrease of the average speed of the vehicles. From iteration 21 on-wards, there is a stability in the average speed of traffic. Still considering the equilibrium, an optimization of the traffic is also observed if humans have a high workload level with a slightly lower expertise level: hence the importance of the intervention of the two agents which in turn prioritize the busiest intersections.

The bad results of group G2 are interesting to study. Indeed it is important to note that despite the total interlocking of the traffic that occurred more precisely at iteration 14, they managed to restart the traffic thanks to the cooperation with the assistant agents.

As far as group G3 is concerned, it should be noted that it was not possible to complete all 60 iterations. This group stopped the participatory simulation at the 48th iteration due to a LAN network failure during the simulation. Before this interruption, this group managed to stabilize the traffic in terms of average speed.

Group G4 outperformed all other groups. Indeed, with the same number of vehicles, this group obtained the maximum average speed, the minimum waiting time and a smaller number of stopped vehicles. This performance is observed during almost all the recorded iterations.

Finally, as for the result obtained by group G5, it can be seen that the average speed of the traffic reflects well the logic of the intervention or not of the agents. Indeed, the two humans chose to intervene between cycles 25 and 32. We can observe a slight stability of the traffic situation during this period in which the two agents do not intervene. Between cycles 33 and 40, where humans do not intervene, the agents managed to maintain the situation of the traffic. This shows once again the feasibility of the approach.

Moreover, a questionnaire provided in Fig. 5 was completed by each participant after the experiment. The marks and verbatims visible in Table 3, were obtained from this questionnaire.

Table 3 Qualitative results for each of the five groups of participants after the experiments
Full size table

5 Discussion and Limitations

Our research question was about how to design and evaluate an interaction model, without explicit communication between humans and agents. This section investigates the potential of our approach 5.1, and mentions a few limitations 5.2.

5.1 Discussion

This discussion successively addresses a positioning with regards to participatory simulation 5.1.1, and road traffic management 5.1.2.

5.1.1 Positioning with Regards to Participatory Simulation

Badeig et al. [16] explore four properties for the design of advanced interactive and collaborative systems: autonomy, proactiveness, context-awareness and situatedness. We think that an analysis based on these properties is well suited for the participatory simulation.

Autonomy suggests “the necessity to articulate the different entities”, by considering two types of autonomous entities, human beings (in our case: observer, human participants) and software agents (in our case: assistant agents, agents from the road traffic application). In our model, the articulation is thus defined by the roles played by these different entities. Our proposal focuses on the main interactions between humans and agents, which are oriented by the different payoffs. To improve the interactions of agents, it could be possible to investigate a form of adjustable autonomy, as described in [52] for human-robot interactions.

Proactiveness is “the way to maintain a consistent coupling between processing structures and dynamic environments”. In our model, the proactivity is expressed through the behaviors that can vary following three criteria, which allow the agents to modify their decision to cooperate or not. Thus, we assume two criteria based on individual characteristics of human beings, and one last criteria based on the environment. The criticality level seems to us sufficient for describing the dynamic state of the environment. Other criteria could be added such as the estimated pollution level leading to limit the speed of vehicles [53].

Context-awareness is the way “to the design of systems in which there is a need to constantly adapt to evolving situations that may be hard to understand”. We think that this property could be better exploited by a finer model of software agents facilitating the anticipation of events (for instance, deadlocks at intersections), as suggested in a next subsection (Cf. 5.2.1).

Situatedness is “supported by continuous interactions with physical elements in the environment like traces as a result of their activity”. The records based on these traces allow us to have a better knowledge of their actions. In our case, the traces allowed us to present the different obtained results (Cf. 4.2.3). We could propose an in-depth analysis, in which these different activities could be evaluated. Future works could concern the estimation of different parameters for the human participants (reasoning time, temporal pressure, etc.).

5.1.2 Positioning with Regards to Traffic Management

We described a road traffic management like a proof of concept for our proposal. In terms of participatory simulation, the cases available in the existing researches only mentioned participants performing simultaneous or parallel tasks on a simulator. The case study presents alternative conditions since the participants are helped by agents, with the overall goal of collective efficiency.

Let us also note that the context of participatory simulation is here highly dynamic: for evidence, the task is not solved by the participants (the traffic is always evolving). However, the results obtained by all the groups clearly show the feasibility of interaction model using the multi-agent assistance system. The slowing down of traffic due to non-intervention is often observed. The numerical value of each assistant’s payoffs reflects their availability to take care of other tasks on the one hand; on the other hand this value expresses their usefulness as an assistant which can reduce the workload of the human being.

In future studies, by increasing the number of groups, it will be possible to observe if patterns can emerge (identical behaviors under similar conditions).

5.2 Limitations

We propose to focus on two limitations: the improvement of models for the different actors 5.2.1, and the consideration of organizational structures 5.2.2.

5.2.1 Improvement of the Model for Different Actors

We have made strong assumptions for the modeling of human beings and software agents. Regarding the human operators, we assumed two crucial parameters for the evaluation of the agents’ utilities, namely the workload and the experience level. These two parameters are significant since they play on the collaboration or not of the agents, to help (or not) the human actors. At present, the human actors estimate their own experience level, via the user interface. Nothing can be said about the real estimation of their expertise or their workload. In our study, we assumed that operators are rational and really want to perform the global task. Some existing studies attempted to characterize the human workload, such as [54] by an evaluation of the tasks performed in relation to a strong temporal requirement. However, it is difficult to formally characterize the experience level for a given problem. In a real context, we could take into account the time spent in the common task to be performed.

Concerning the agents, their task is performed according to the Nash equilibria. Their decision is therefore based on a decision process computed at each cycle. As suggested in 5.1.1, we could improve their expertise by considering anticipation capabilities. Discussions about the anticipation are available in [55, 56]. The anticipation remains possible in systems whose decision-making process is based on predicting possible payoffs. A particular solution is based on learning techniques by reinforcement [57,58,59,60].

5.2.2 Consideration of the Organizational Structure

Our proposal envisages the ability of agents to help any humans involved in participatory simulation. However, realistic (or even real) applications require taking into account the organizational specificity of the application. Concerning this subject, we can cite works on structured inter-agent relations in multi-agent organizations [61] or humans-agents organizations [8]. These motivations may be due to real constraints, or simply to computational constraints. The computational problems could be prohibitive in dynamic and particularly complex environments, due for instance to a significant increase in the number of vehicles to be managed, or to a high number of assistant agents and human participants involved. In case of large number of agents, the cost of helping a particular human could probably influence collective actions. More broadly, the development of complex environments such that no human would have a global vision remains a crucial issue.

6 Conclusion

Our research aims to propose a model without explicit communication between humans and agents composing a team. Thus, decision matrices make it possible to consider human operators, by helping them in the accomplishment of the task. In our analysis, several criteria have been introduced: the experience and the workload levels for humans, and the criticality level of the situation. Our model uses a repeated multi-player game, where several agents and human beings interact in a given environment. In this context, we described a participatory simulation. The simplicity of the approach and the promising results encourage us to further develop it in collective solving of complex tasks. An advantage of using decision matrices is justified by (1) the possibility of designing an adaptive system that is context-sensitive, (2) the consideration of agent’s availability to take care of other tasks.

In the future, we would like to consider improvements of the participatory simulation platform. It could be possible to investigate a form of adjustable autonomy, to study different parameters for the human participants (reasoning time, temporal pressure, etc.) and to add other environmental criteria such as the estimated pollution and noise levels [62]. Context sensitivity in interacting with an adaptive multi-agent system will also be another challenge in trying to introduce an anticipatory mechanism for the agents and an organizational structure for the different actors. A second perspective focuses on the traffic management. To find different emerging patterns, it would be possible to involve a set of additional participants while varying their characteristics (e.g., age, education level, disabilities). For road traffic supervision, our third perspective is to enrich the rules used by the assistant agents. They could take on other aspects of traffic management in the context of their interventions, by adding new expertise on this problem, for example: to add other mixed traffic of autonomous vehicles, pedestrians and trams [63] and to consider explicitly emergency vehicles [64]; one or several agents, as well as human beings, may be dedicated to their management. A last research perspective would be to consider different team configurations (from 0 to several assistant agents/human operators), while varying the level of difficulty in traffic management (numbers of vehicles and intersections), and compare the performance of the proposed interaction model.