Keywords

1 Introduction

Playing games is an activity that is embedded in social and situated practices [18]. Game experiences are often driven by a certain social context where the presence of other players is an integral part of enjoyment. This results in social interactions between players (e.g., coordination of actions or encouraging another player) that are deeply rooted in natural human communication. In human-human collaboration gaze and eye contact are crucial factors building the foundation for social interaction [10, p. 86] and information flow [9, p. 67]. However, in remote settings, like online coop games, typical interpersonal means of face-to-face communication (e.g., gaze, mimic, etc.) that are prevalent in co-located gaming, are not or only limitedly present. These non-verbal signals have an impact on social presence and manifest themselves in various ways like facial expression, gesture, head movement, body posture, etc., which are factors that can also influence how players perceive each other in cooperative gameplay. This is especially relevant in games that require players to act cooperatively in order to be successful. In cooperative gameplay social interactions are often targeted at creating a shared understanding of the current scene and to coordinate each other’s actions. We argue that gaze interaction can play an important part in how such cooperative actions are coordinated and thus, also influences social presence.

The approach discussed in this paper uses a game in combination with a shared gaze visualization as a shared embodied resource in order to foster social presence and connectedness between players. By giving the gaze of a cooperating player a “body” and a representation within the game we aim at exploring this gaze-based interaction modality as a means for collaboration. We argue that by providing a means of non-verbal communication (i.e., a shared gaze visualization) within a game, even if not actively part of the gameplay itself, a new layer of non-verbal communication for the players is created, that can result in new forms of human-human collaboration in online games.

In this paper, we investigate the impact of a shared gaze on social presence in a remote cooperative game setting. Further, we want to find out how a shared gaze visualization changes how people collaborate (e.g., different strategies) and how it is used in combination with other communication modalities like verbal input. We aim at using gaze as a “moderator” and facilitator of cooperation, and use shared-gaze as a way to create a coupling between players in order to foster a shared sense making process towards increased social presence. With our findings we want to inform game designers with insights on how and why the inclusion of gaze-based interaction in remote game settings fosters the social bond between players.

1.1 Social Presence in Online Games

Several authors share the opinion that presence is composed of the three dimensions: social presence, spatial presence, and self presence [e.g., 20]. As this work specifically focuses on the social aspect of presence a more elaborated definition of the term shall be given. The concept of social presence was introduced by Short et al. and defined as “the degree of salience of the other person in the interaction and the consequent salience of the interpersonal relationships” [17, p. 65]. The researchers claimed that communication medium has an impact on the intimacy and immediacy. After Bulu [1] theories based on the work of Short et al. explain that the level of social presence increases as the communication capabilities of the medium increases, ranging from text-based, to face-to-face media or to computer mediated communication. Following Hudson and Cairns [6] social presence can be compared with the concept of “Shared Involvement”, coined by Calleja [3]. Shared Involvement can be understood as a sense of being with subjects in a common environment, whether that involvement is composed of explicit communication, acting as a team, or by being aware that one’s actions are taking place in a shared environment. Regarding the role of social presence in the context of games Hudson and Cairns [6] emphasize the importance of social presence by noting that social relationships between players as mediated by games (as well as social presence as its foundation) have a significant influence on the gaming experience. This does not only apply to massively multiplayer games but also to console & PC games (multiplayer, co-located play, etc.). Cairns et al. [2, p. 1069] state that “playing socially is a prevalent and important aspect of digital gaming.” Furthermore, several studies identified a correlation between a positive game experience and a high degree of social presence and social interaction (e.g., [5, 14]).

1.2 Nonverbal Interaction and Gaze-Based Interaction in Games

Nonverbal communication manifests itself in a variety of forms such as facial expression, gestures, paralinguistic, body language and posture, proxemics, haptics, and gaze [4]. Several studies revealed that there is a relation between social presence and nonverbal communication [8]. Mansour et al. [13] argue that there is a positive relationship between the perception of social interaction (reciprocal processes in which humans negotiate and regulate the quality of their relationships) and the feeling of social presence in online multiplayer games. The authors note that the coordination of a conversation within games (e.g., clarifying mutual goals, discussing strategies, tactics, etc.) is a dynamic and complex process including exchanges of nonverbal communication. Their proposed framework shows that both verbal and nonverbal communication acts are two main components of social interaction among players. After Mansour et al. [13] the application of verbal and nonverbal behavior helps players to familiarize with their partners, fosters the management of the interaction flow and has a positive influence on the feeling of trust among players. Manninen and Kujanpää [12] suggest that an efficient combination of different communication channels enhances interaction among players. By combining various forms of interaction players would not have to rely on verbal communication alone, but have more flexible means in representing themselves and their ideas. Further, Shahid et al. [16] investigated the effects of mutual gaze on the game experience and perceived social presence during video-mediated and co-present gameplay. Their results indicate that the absence of mutual gaze dramatically affects the quality of interpersonal interaction in video-mediated gameplay settings. Including mutual gaze interaction in such a setting, on the other hand, leads to increased social interaction and mutual understanding among remote collaborators. They argue for the inclusion of mutual gaze interaction as an integral part of playful cooperative settings.

2 Contribution

Based on related research, we argue that there is much potential for gaze-based interaction in cooperative online gameplay. Research on gaze interaction is rather limited in the context of cooperative online gaming. Besides using gaze as a substitute and as enrichment for interaction we want to investigate whether the non-diegetic use of a shared gaze visualization can support social interaction in remote collaborative gameplay. In contrast to other research, we see gaze communication as an integral part of the gaming experience and explore its potential to be used as a means of explicit and implicit interaction. Further, we explore how a shared gaze visualization changes cooperative in-game communication and how teams appropriate the use and meaning of it.

We aim at using gaze as an input method to foster social presence and enhance team collaboration. We argue that by providing this additional communication channel in an online co-op can potentially increase connectedness between cooperating players. In that regard, the game acts as a mediatory artifact generating a social bonding between the cooperating team of players. In a remote setting, typical human face-to-face qualities and non-verbal communication (e.g., eye contact) is very limited. Incorporating eye-based interaction during cooperative online play (in our case visualizing another person’s gaze point on a player’s screen) can potentially enrich such remote co-op settings. With our approach and gaze-based interaction concept we aim at providing non-verbal information without distracting players but to support them in their gaming activities and explore shared gaze as a subconscious and implicit means of communication but also as an explicit tool for cooperation.

3 Experiment Description

In order to explore gaze-based collaboration in an online co-op game, we conducted an experiment that investigates the effects of shared gaze visualizations on social presence among the participants. The main idea behind the experiment was to visualize the cooperating person’s gaze on the player’s screen and vice versa (see Fig. 1). We chose the game Ibb & Obb [19] as a test bed for our experiment and gaze-based interaction concept. Ibb & Obb is a 2D platformer that lets two players cooperate online to succeed the different puzzle based levels. In this game, players have to coordinate each others actions quite carefully to be able to finish the puzzles the game presents.

Fig. 1.
figure 1

Left side: Picture of study participant playing the game while eye tracking is used to communicate with the other player. Right side: In-game visualization of gaze point indicated via a black circle rendered above the actual game

Full size image

The experiment consisted of four conditions with varying communication modalities for the cooperating players which were completely randomized to the participants (i.e., randomized block factorial design): (1) no additional communication channels (condition name: none), (2) verbal communication (condition name: voice), (3) gaze (condition name: gaze), and (4) verbal communication + gaze (condition name: voice and gaze). In conditions with shared gaze (i.e., condition 1 and 3), the gaze of the other player (i.e., where the other player is currently looking at) was visualized in real-time on the other player’s screen as a black circle (see Fig. 1, right picture). In conditions 2 and 3 participants could additionally communicate via online voice chat. After every playing condition, participants filled out a questionnaire regarding social presence (see section on Measures). The participant pairings played the levels of the game in ascending order to assure that players are always confronted with an increasing level of difficulty. The playing condition of every level was randomized for every participant pairing.

With this study experiment we wanted to investigate the following hypothesis: With the integration of gaze interaction (i.e., the visualization of the other person’s gaze) in online co-op games, subjects will experience an increased level of social presence in comparison to a gaming scenario that is solely based on verbal and in-game interactions. We argue that with shared gaze, players receive an additional communication channel that is either used consciously or unconsciously, but which in any case, provides a means of non-verbal communication that is deeply rooted in natural human face-to-face communication. This potentially effects social presence during online co-op play and gives players a new tool to be used for e.g., in-game spatial referencing.

3.1 Participants and Procedure

The study was conducted at two separate research facilities. The sample consisted of 20 participants), age 21 to 37 years (6 female, mean age = 26.20, SD = 4.51). All participants were either students of the University of Applied Sciences Upper Austria (65 %) or research staff working at the University of Salzburg (35 %). Furthermore, subjects represented a variety of disciplines of education having a background in psychology, software engineering, or in digital arts. About half of the participants (11 persons) share an affinity to play video games as they indicated to play games at several times a week or daily, whereas only one participant stated to never play games.

Each evaluation session consisted of two subjects that played the game collaboratively in two separate rooms. The evaluation was divided into four parts based on the previously described conditions (none, voice, gaze, gaze and voice). By choosing a within-subject design, all participant teams had to play each of the mentioned scenarios (play time limit for each condition: 5 min). The order in which the teams were assigned to the conditions was randomized to limit the effects of the condition and game difficulty. As a first step, the experimenter provided a short introduction to give an overview of the overall procedure. After the eye tracking devices were calibrated the evaluation part started with a sandbox level (“limbo” level with level selection) showing the basic means of interaction. Participants were instructed about the setting and the game goals. As the control scheme and the genre itself (i.e., platform game) were easy to comprehend, subjects had no difficulties to get into the game. When subjects confirmed that the control scheme of the game was clear to them the experiment began. All subjects started with the first game level of Ibb & Obb in conjunction with one of the four conditions, followed by level 2, 3 and 4. After each level the condition was changed. For example, team 1 started the experiment with level 1 and the gaze condition, while team 2 played level 1 with the voice condition. This should enable players to get into the game (no steep learning curve), as well as make sure that at the beginning of the experiment subjects were not confronted with too difficult game situations.

After every condition, participants were instructed to fill out the CCPIG questionnaire (see section on Measures for a detailed description). After finishing all conditions, participants had the possibility to give comments on the played conditions with a focus on social presence and their experience with the gaze visualization, via a team interview carried out by the experimenter. One experimenter was present during the whole study and took notes on observations and participants’ interactions in every condition (i.e., how they used the gaze visualization and potential collaboration strategies emerging from the gaze visualization). The procedure took between 50 to 60 min.

3.2 Technical Setup

The technical setup for our experiment consisted of two Tobii EyeX eye trackers (http://www.tobii.com/xperience/) and two separate applications, one for data communication and another one for gaze visualization on each player’s computer. The gaze position of a player (i.e., X and Y in screen coordinates) was captured by the eye tracker and visualized in real-time on the corresponding players screen “above” the actual game. The communication between the eye tracking application on one player’s side and the gaze visualization on the other player’s computer was realized via Spacebrew (docs.spacebrew.cc), which is an open-source websocket-based prototyping framework. In order to provide a smooth movement of the gaze visualization, we filtered the gaze point values before sending them to the visualization application. Figure 2 illustrates the overall technical setup.

Fig. 2.
figure 2

Schematic of the technical study setup; Two separate applications sending and receiving gaze data for visualization within the game on each participant’s side

Full size image

3.3 Measures

To measure the social presence dimensions the Competitive and Cooperative Presence in Gaming Questionnaire (CCPIG v1.2), which is a validated scale developed by Hudson and Cairns [7], was employed. We decided to use this questionnaire, because it specifically aims at exploring social presence in the complex social environments that are team-based online games. It focuses explicitly on games and is not designed to be a general measure for social presence across multiple media. The CCPIG is composed of two parts: part 1 measures competitive social presence, which is the level of social presence experienced by players towards their opponents. The second part measures cooperative social presence, the level of social presence experienced by a participant towards their teammates. The authors of the questionnaire note that it is possible to use the two parts independently to investigate different kinds of games (competitive, cooperative). Since the game of our evaluation features cooperative and not competitive gameplay, we applied only the second part of the CCPIG. This part is made up of two modules. The first module measures the perceived level of team cohesion (25 items – example: “It was as much about the team as about my own game”), while module 2 measures team involvement (11 items – example: “The actions of my teammates affected my thoughts and actions”). All items were rated on a five-point Likert scale (ranging from 1 = “strongly disagree” to 5 = “strongly agree”). The internal consistencies were acceptable to good with Cronbach’s alpha reliability estimates ranging from α = .67 to α = .92 (cf. Table 1).

Table 1. Means, standard deviations, zero-order correlations, and Cronbach’s alpha reliabilities (in the diagonal) of the studied variables
Full size table

Apart from the CCPIG the evaluation also included an open interview at the end of the test session in order to gain a deeper understanding of the game experience and how participants perceived the gaze interaction. Participants were asked if the gaze visualization was a helpful tool to interact with their teammate as well as how they used it for communication. Additionally, the participants were asked if they felt their teammate as being more present in conditions with a gaze visualization and if they had any suggestions for improvements in regard to game design and the usage of the gaze visualization.

3.4 Data Analysis

All dependent variables were normally distributed, meeting the requirements for parametric analyses. Hence, data were analyzed by means of a repeated measure analysis of variance (rANOVA) with the experimental condition (4 conditions) as the within-subjects factor and the 2 indicators of social presence (team cohesion and team involvement) as the dependent variables. To account for violations in sphericity, the Greenhouse-Geisser correction of degrees of freedom was employed. Post hoc analyses on mean differences between each pair of the four conditions were conducted using Sidak’s method. The conventional level of p < .05 was used when evaluating the results with regard to statistical significance.

4 Results

Table 1 reports the descriptive statistics and bivariate Pearson correlation coefficients between the variables. As expected, participants experienced higher levels in social presence in conditions with additional communication channels. Social presence was highest in conditions where voice communication was possible (i.e., conditions 2 and 4). Gender was not significantly related to social presence, but age was associated with higher levels in social presence, particularly in the voice condition. Moreover, the more frequently participants were playing games, the less team involvement they were experiencing in the condition without gaze and voice communication.

The results of the rANOVA revealed a significant main effect of experimental condition, F(1.61, 35.70) = 12.42, p < .001, and a significant interaction effect between experimental condition and the social presence indicators, F(1.88, 35.70) = 6.03, p = .006. Both effect sizes were large with part. ɳ2 = .40 for the main effect and ɳ2 = .24 for the interaction. Thus, social presence differed significantly between experimental conditions (main effect), and the effects of the experimental condition were different for team cohesion and team involvement (interaction effect). Figure 3 illustrates the results.

Fig. 3.
figure 3

Means of Social Presence in the four experimental conditions (error bars represent 95 % confidence intervals)

Full size image

Although condition 3 (gaze) had higher ratings than condition 1 (none) post hoc tests regarding the main effect of experimental conditions demonstrated that condition 3 (gaze) did not result in significantly higher social presence experiences as compared with condition 1 (none), M difference  = 0.11, SE difference  = 0.07, p = .613. Moreover, condition 4 (voice + gaze) was significantly superior to condition 3 (gaze), M difference  = 0.36, SE difference  = 0.09, p = .003, but not to condition 2 (voice), M difference  = 0.06, SE difference  = 0.05, p = .894. In sum, these results suggest that adding gaze as a communication channel tends to have positive effects on social presence experiences of players, however not significantly, which is contrary to expectations.

As far as the interaction effect is concerned, post hoc analyses revealed that the effect of the experimental condition was stronger for team cohesion as the dependent variable, ɳ2 = .44, than for team involvement, ɳ2 = .25. The differential effects of the experimental condition on team cohesion and team involvement became particularly evident when conditions 2 (voice) and 4 (voice + gaze) were compared with condition 1 (none): Team cohesion could be significantly improved by adding voice, M difference  = 0.54, SE difference  = 0.13, p = .003, or voice and gaze as communication channels, M difference  = 0.57, SE difference  = 0.15, p = .007. On the other hand, team involvement was significantly higher when voice and gaze were added simultaneously, M difference  = 0.36, SE difference  = 0.12, p = .045; adding voice only tended to be superior to condition 1 as well, but this difference failed to reach significance, M difference  = 0.27, SE difference  = 0.10, p = .090.

4.1 Qualitative Data Results

The qualitative data consisted of the answers from the team interviews as well as observations and notes taken during the study. The notes were assigned to one of four pre-defined categories (i.e., one for every condition) to gather the study observations in a structured way. During the study the observing experimenter wrote down how participants coordinated their actions (e.g., potential strategies) in every condition and what instructions they gave to each other (e.g., either through gaze or verbal communication). The observation data as well as the team interview after the playing sessions was analysed according to the basics of qualitative content analysis [15] in order to identify “common themes” and similarities among the participant teams in every condition.

The observations as well as the interviews showed that in conditions where voice communication was enabled (i.e., condition voice + gaze and condition voice), voice was the dominant information channel for the teams. Voice was used to negotiate actions, coordinate planning, turn taking, potential hazards, avatar capabilities etc. Further, teams also reported that they felt less verbal misunderstandings in the condition that combined gaze and voice input. This suggests a positive effect of the gaze visualization towards gaining a shared understanding among the team members. For instance, when the teams encountered new, yet unexplored level areas, players stopped their avatars, discussed and planned together, before executing the negotiated actions in a cooperative manner (“I think you have to jump right up there and then I can reach over here”). This step-by-step planning was supported by the gaze visualization as it was used to draw paths showing the team member how to proceed and link verbal input with spatial referencing. The more communication channels the teams had (within the different conditions), the more “planning ahead” occurred. In contrast to that, the condition with no additional communication channels (i.e., condition none) appeared to be most prone to trial and error behavior of the teams.

During the interviews, 9 out of 10 teams reported, that compared to the condition with no additional communication channels (i.e., condition none), conditions with a shared gaze visualization made them perceive the other player as being “more present”. They reported that they tried to use the gaze visualization as a tool for communication, e.g., for pointing reasons to show the team member where to move. However, some participants felt that using the gaze as a tool was too difficult, as the meaning of, e.g., potential movements and gestures drawn via the visualization, could not be negotiated in advance with the teammate, thus, tended to be unclear to them. Further, the players’ gaze was naturally more focused on their own avatar, except when they tried to use the gaze visualization as a tool for pointing and turn taking, in order to draw the other player’s attention to something. The concept of perceiving the gaze visualization as a mere tool and not as a representation of the other player may also explain why the effect of social presence was not as pronounced as anticipated. Some players noted that it would be a good idea to visually link the other player’s game character with the gaze information. A detailed discussion on the design possibilities in this regard can be found in Lankes et al. [11]. An issue participants had with the gaze visualization was its “always-on” behavior. They felt a strong loss of meaning in the partner’s gaze visualization as it was constantly visible, thus, making it hard to distinguish between deliberate communication and unwanted inputs. In that regard, players highlighted that for future iterations of the prototypes they would like to see both, their own and their partner’s gaze visualized, a visual link between the avatar and the gaze information, as well as having a way to activate/deactivate the visualization when necessary.

5 Discussion & Lessons Learned

Our study has shown, that gaze-based interaction bears much potential as an additional layer of collaboration in games with shared gaze showing positive influence on perceived social presence. We explored how the non-diegetic use of a shared gaze visualization can support social interactions in a remote setting. It proofed to be a viable means for communication, however, it needs improvements to act as a useful tool for explicit interaction and communication.

Gaze input needs affordances and activation strategies:

In our prototype the partner’s gaze was visualized in an abstract manner (i.e., via a black circle) without giving the gaze point a semantic meaning within the game world. We did this in order to explore how people react to this new interaction modality and how its use and meaning will be negotiated by the teams themselves letting the players freely appropriate the gaze input towards facilitating common ground. However, a crucial aspect towards making gaze a useful “tool” is to provide a certain affordance and meaning in order to let the players know how and when to use it appropriately. This could be done by, e.g., integrating the gaze point directly into the game world (e.g., letting avatars react to the gaze point) to assure that there is no disconnect between in-game interactions and player interactions. Additionally, one of the lessons learned from the study was that players didn’t like the “always-on” aspect of the prototypes current implementation of the gaze visualization. Thus, we argue, that in order to be adopted as a tool by players, we also need to design for meaningful and understandable ways of activation of such a gaze-based interaction (e.g., the player can trigger the visualization actively).

Gaze as an additional layer of communication supporting team processes:

Players used their gaze as a tool for communicating in situations that required e.g., exact pointing, or as a substitute to verbal interaction to establish a shared understanding between players and lessen the occurrence of misunderstandings due to potential ambiguous verbal communication. The more communication channels the teams had, the more negotiation between players was happening. This notion of gaze as an additional communication layer could in turn help players to connect and communicate with each other, e.g., in game settings where verbal input might be undesired but team coordination is still necessary (e.g., cooperative mobile online gaming in a public transport scenario).

Gaze as a subconscious non-verbal communication channel:

Besides explicit gaze interactions like pointing, we believe that players can also form a kind of subconscious communication through an omnipresent gaze visualization of the other player. Over time players could potentially learn to understand the gaze movements and resulting gestures and intentions of another player, which could lead to a more integrated visual “team language” and connectedness. If and how such implicit gaze-based communication is established between two remote cooperating players will be part of future work, though. In that regard, we plan to do a study in a similar setup with more participants over a longer period of time in order to investigate if such a shared gaze visualization can be used and “learned” over time as a subconscious means communication.

Limitations:

The questionnaire we used included many items regarding the players’ feelings of acting together as a team. We are aware that the game we used in our study was already strongly based on working together as a team, which could have led to the increases in team coherence and presence in the questionnaire data being not significant among the different conditions. Although we found only little quantitative improvement in social presence between gaze and non-gaze conditions, post-play interviews nonetheless indicate that subjects reacted positively to the inclusion of shared gaze, particularly in conjunction with voice communication.

6 Conclusion

This paper reports on a study that investigated a shared gaze approach and its effects on social presence and collaborative interactions in online cooperative play. We explored different combinations of voice and gaze input as a communication channel for players during a team-based game. Our study findings show that shared gaze can have a positive influence on team cohesion and involvement, leading to players perceiving their team member as being more present when their partner’s gaze is visualized. Participants tried to use the gaze visualization as a tool for communication, e.g., for spatial referencing, and described it as being a useful substitute to verbal communication. Based on our findings we argue, that shared gaze can foster social couplings in remote collaborative gaming settings which in turn can contribute to increased connectedness between collaborating players. The integration of such non-verbal interpersonal communication qualities such as gaze, is yet an underrepresented and underused design resource, but nevertheless, a promising direction for future game designs.