Moving together: toward understanding the mechanisms of joint action
- First Online:
- Cite this article as:
- Obhi, S.S. & Sebanz, N. Exp Brain Res (2011) 211: 329. doi:10.1007/s00221-011-2721-0
- 1.1k Downloads
The study of joint action is becoming increasingly popular (indexed by the fact that this special issue is a double volume) as the realization grows that much of human information processing and behavior occurs in social interaction. For example, when two people move a piece of furniture together, there is a requirement for them to work in concert and there are mechanisms by which they are able to coordinate their actions. A defining characteristic of such a scenario is that each person must adjust to some aspect(s) of the intentions, action plans, and movements of the other (Clark 1996). This is a fundamentally different situation from that in which a person acts in isolation to achieve a goal. Accordingly, the ‘experimental quarantine’ approach (Richardson et al. 2008) of studying isolated individuals that has guided the majority of cognitive (neuro-)science research has been giving way to studies of social interaction that try to do justice to the complexity of interpersonal cognition and action. At the same time, this new endeavor provides a bridge between traditional experimental psychology traditions and more anthropologically inspired studies of cognition in the wild (Hutchins 1995; Jordan 2008).
Everyday examples of the need to coordinate actions with others abound, which highlights the relevance of developing a greater understanding of the underlying mechanisms. For instance, activities such as dance, music, sports, and major surgery all require close coordination, be it at the level of intentions, action plans or ongoing movements. Joint action has been defined as any form of social interaction where two or more individuals coordinate their actions in space and time to bring about a change in the environment (Sebanz et al. 2006). Within this broad definition of joint action, there are many sub-areas of investigation that relate to different aspects of joint performance, and that involve different levels of analysis (Knoblich et al. 2011).
A relatively recent but quickly growing line of research deals with the question of how and when co-actors form representations of each other’s actions and tasks, and how such “co-representation” affects individual task performance (Wenke et al. 2011). Studies on co-representation have consistently shown that people have a strong tendency to keep in mind what their co-actors need to be doing and monitor others’ performance, even when this impairs their own task performance.
The more long-standing focus on temporal coordination, which can be traced back to when researchers first started comparing intra- and interpersonal coordination about 20 years ago (Schmidt et al. 1990), has been fundamental in advancing our understanding of how synchronization between individuals emerges. In addition to synchrony (Richardson et al. 2007), recent studies on temporal coordination have also begun to address the mechanisms underlying coordination of non-rhythmical activities and the use of individual strategies to facilitate temporal coordination.
A third line of research has focused on perceptual processes and, in particular, on the role of close links between perception and action for joint action. From behavioral and neuro-physiological studies, we have learnt that perceiving an action (whether this involves seeing someone dancing or hearing someone playing a piano piece) activates representations of corresponding motor programs in the perceiver (Rizzolatti and Sinigaglia 2010). Recent studies have started to explore how this principle, variously referred to as ‘motor resonance’, ‘motor simulation’ or ‘mirroring’, contributes to joint action and how social context affects action perception. Furthermore, recent investigations of perceptual processes in the context of joint action include the study of joint action perception and joint attention.
Our special issue brings together the latest research on these three sub-themes, showing the enormous progress that has recently been made in understanding co-representation, interpersonal coordination, and (action) perception in the context of joint action. In addition, the special issue highlights the role of developmental work and the role of computational modeling for the understanding of mechanisms of joint action. Last but not least, this issue also features very recent developments in the phenomenology of joint action.
We hope that the present collection of papers will be informative and inspiring not only to those already hooked on the study of joint action but also to those who are new to this area. There is much to discover and discuss in the thirty contributions to this special issue, and we hope that the many conceptual insights and methodological advances will spread across various sub-disciplines of cognitive, social and developmental psychology and neuroscience. The following section provides an overview of the topics covered by the special issue, including a short summary of each contribution.
When two people perform independent parts of a spatial compatibility task next to each other, their performance has striking similarities with the performance of single individuals performing both parts of the spatial compatibility task alone (Sebanz et al. 2003). This indicates that even when coordination is not required co-actors take into account particular aspects of each other’s tasks and include each other’s actions in their planning. Recent work on co-representation has focused on two questions. First, what exactly about the co-actor’s task is represented and what are the cognitive processes underlying co-representation (Wenke et al. 2011)? Second, to what extent does the social relation between co-actors affect co-representation (Hommel et al. 2009; Ruys and Aarts 2010)? Several papers in this special issue provide exciting new findings that help to answer these questions.
Ferraro et al. assessed whether the joint spatial compatibility effect that has been taken as an index of co-representation reflects both facilitation and interference processes, and whether or not these two processes equally affect performance. Does the presence of a co-actor make performing one’s own task easier or harder? Their results suggest that interference dominates, as evidenced by slower reaction times when a feature of the stimulus to be responded to overlaps with the other co-actor’s response. This indicates that the co-actor is represented as an agent responsible for particular action events (Philipp and Prinz, 2010).
Holländer et al. explored whether co-representation leads to motor activation in the non-acting person when it is the co-actor’s turn to respond. Their electroencephalography (EEG) study revealed a lateralized readiness potential, reflecting activation of motor areas contra-lateral to the responding hand, not only when participants prepared to act themselves, but also when they anticipated their partner’s action. This study also revealed that motor activation when it was the co-actor’s turn was consistent with participants preparing to act themselves and did not depend on the specific hand to be used by the co-actor, indicating that participants simulated performing the action to be performed by the co-actor themselves rather than adopting the other’s perspective.
Nevertheless, co-representation does seem to require advanced social cognition abilities. Humphreys and Bedford report findings that suggest common processes involved in particular aspects of Theory of Mind (ToM) and co-representation. Neurological patients with impairments in ToM did not include a co-actor in their action planning, as evidenced by the lack of a joint spatial compatibility effect. Interestingly however, when asked to pay attention to their co-actor, ToM patients with lesions of posterior parietal cortex who tend to have difficulties attending to social cues showed a joint spatial compatibility effect. In contrast, ToM patients with lesions involving frontal regions who may be impaired in recruiting resources to code others’ actions did not show a sustained effect.
The joint Flanker effect reported by Atmaca et al. demonstrates that co-representation is not restricted to tasks involving automatic stimulus–response links such as spatial compatibility tasks, but generalizes to tasks involving arbitrary stimulus–response mappings. In line with Humphreys and Bedford’s findings suggesting that ToM plays a role in co-representation, this study also found that for co-representation effects to occur, it is crucial that people perceive their co-actor as acting intentionally. Participants did not show a Flanker effect when their co-actor’s actions were controlled by a machine, indicating that only tasks of intentional agents are co-represented.
Three further studies shed light on performance monitoring during co-action. Liepelt and Prinz developed a social dual-task paradigm where two individuals share two tasks and compared dual-task performance within and across individuals. Their findings provide evidence that a social Psychological Refractory Period effect can be induced when two people share two tasks, provided that the task to be executed first is prioritized. This finding provides a new perspective on how and when co-actors monitor each other’s performance.
In an EEG study, De Bruijn et al. investigated how knowledge about a co-actor’s task affects error monitoring and behavioral adjustments following errors. The analysis of event-related potentials showed a modulation in the error-related negativity depending on the co-actor’s task. This suggests that when performing a task together, co-actors include each other’s tasks in their error monitoring.
Findings by Radke et al. demonstrate that when acting together, people consider the impact of their actions on their co-actor. In a functional magnetic resonance imaging (fMRI) study, the authors found increased activation in parts of the brain network involved in ToM (medial prefrontal cortex) following errors that had consequences for the co-actor, compared with errors that did not affect the co-actor.
A further series of studies sheds light on the impact of interpersonal relations on co-representation. Milanese et al. used a social transfer of learning paradigm (Milanese et al. 2010) to test whether a joint learning experience affects performance on a subsequent joint task when the identity of the co-actor changes. Whereas learning transfer occurred when the new task was performed together with a different co-actor, keeping the spatial relation between the co-actors constant turned out to be crucial for transfer to occur.
Other findings suggest that perceived group membership affects co-representation. As Müller et al. report, white participants showed a joint compatibility effect when interacting with an animated image of a white hand, but not when interacting with a black hand. However, co-representation of an out-group member’s actions could be induced by instructing participants to take the perspective of an out-group member before performing the joint spatial compatibility task.
In line with these findings, He et al. found that the relation between co-actors modulated co-representation in a task that measures how items to be remembered by one’s co-actor affect one’s own attention during visual search. Strangers tended to be attracted by items their co-actor had to keep in mind, demonstrating that the other’s memory task was co-represented. Surprisingly, pairs with in-group relations (friends and Chinese people living in the UK) did not show this effect. This raises new questions about the relation between perceived group membership and self-presentation, trust, and arousal levels that may be responsible for mediating co-representation.
Whereas perceived group membership based on ethnicity modulates the extent to which a co-actor’s task is taken into account, experimentally induced perceptions of group membership may not have the same effects. In a joint spatial compatibility task, Iani et al. failed to find differences in co-representation between people who were told that they and their co-actor belonged to the same or to a different group based on cognitive style. However, this study revealed a clear effect of the social context, as a joint compatibility effect was found when participants intended to cooperate but not when they competed against each other. This finding extends earlier studies suggesting that interdependency is a key factor for co-representation (Ruys and Aarts 2010).
In this special issue, several papers consider the general issue of behavioral coordination. Although the range of experimental tasks and manipulations varies, there is a common aim to understand the factors and constraints that affect interpersonal coordination, and the mechanisms by which these factors exert their effects, be they cognitive, perceptual, perceptuo-motor, dynamical, or multi-level in nature.
On the question of constraints on interpersonal coordination, Ramenzoni et al. investigate the perceptuo-motor processes that contribute to successful performance in a joint coordination task. In their experiments, a pair of participants engage in a coordination task that demands actions from one partner that are wholly dependent on the actions of the other partner (experiment 1), and additionally dependent on the degree of their own postural stability (experiment 2). A key question posed by these authors relates to the effects of task constraints at the interpersonal and intrapersonal levels on successful performance. The authors conclude that increased task demands at the interpersonal and intrapersonal levels have corresponding effects on coordinative structures at both intrapersonal and interpersonal levels (Bernstein 1967; Latash et al. 2007).
Fine and Amazeen give consideration to intrapersonal and interpersonal coordination in a rhythmic Fitt’s Law tapping task. The fact that spontaneous coupling seems to occur for both intrapersonal coordination and interpersonal coordination (albeit induced via anatomical and visual coupling, respectively) led the authors to predict that similar violations of Fitt’s law would occur in interpersonal and intrapersonal Fitt’s type rhythmic movement tasks. This is exactly what the experimental results showed, and the authors discuss their results in the context of non-centralized control, which refers to the idea that interpersonal coupling emerges despite the fact that there is no centralized controller for the constituent movements. The authors highlight the potential efficacy of the dynamical approach to explain such interpersonal coupling, and underscore the benefits of being able to derive predictions based on the known workings and physical laws that govern the behavior of coupled oscillators.
To investigate whether intrapersonal and interpersonal coupling effects are similar, and to shed light on the mechanisms that mediate them, Jung et al. performed two experiments incorporating bimanual (intrapersonal) and joint (interpersonal) variants of a two-target aiming task. The authors contextualize their work with regard to previous evidence suggesting that interpersonal coupling effects are similar, but weaker than intrapersonal coupling effects (e.g., Schmidt et al. 1998). By comparing bimanual to joint action conditions, the authors found that interpersonal coordination in a non-rhythmical choice response task is characterized by weak local alignment but strong global alignment. That is, coupling effects seem to depend on processes that operate over a long timescale that transcends individual trials. The idea that coupling effects can be observed and characterized over different timescales suggests the existence of different underlying mechanisms that govern coordination.
While intrapersonal coordination dynamics and interpersonal coordination dynamics have been studied separately in the past (e.g., Black et al. 2007), many real-world coordination scenarios involve concurrent intra and interpersonal coordination (e.g., talking and walking together), and hence, an important research goal is to understand these types of situation more thoroughly. To address this issue, Coey et al. required pairs of participants to produce intrapersonal coordination patterns of differing stability (i.e., in-phase or anti-phase), and demonstrated that intrapersonal coordination stability and the emergence of interpersonal coordination are independent of one another. They conclude that intrapersonal coordination synergies or coordinative structures are the units that become coordinated interpersonally (i.e., it is not the constituent movements within these intrapersonal coordinative structures that are the units of coupling).
Interpersonal synchrony has been suggested to impact the quality of social exchanges (Chartrand and Bargh, 1999), and as such, understanding the social factors mediating synchrony is an important goal for the field. Probing the conditions under which interpersonal synchrony occurs, Miles et al. found that when individuals interact with members of an ostensibly different social group (minimally defined by wearing of different color stickers purportedly reflecting their artistic taste), they display more behavioral synchrony than when interacting with members of their own minimal group. The authors discuss this somewhat counterintuitive result with respect to the idea that behavioral synchrony could be a strategy that the system uses to reduce intergroup distance.
In many joint activities, especially in domains such as music or dance, it is critical for successful performance that co-actors synchronize their actions either to each other, or to some external pattern of sensory information, such as a musical piece. To investigate this ability, Pecenka and Keller build upon the finding that expert musicians are able to predict the ongoing actions of their musical partners in order to be synchronous. In previous studies, it was found that two-thirds of participants were able to predict ongoing tempo changes in a sensorimotor synchronization (tapping) task, with the remaining participants tracking the changes (Pecenka and Keller 2009). Pecenka and Keller paired combinations of individuals who are strong ‘predictors’ or strong ‘trackers’ in a dyadic sensorimotor synchronization task. Interestingly, pairs of ‘predictors’ were more synchronous with one another as indexed by higher accuracy and lower variability than pairs of ‘trackers’. Mixed pairs showed levels of synchronization in between predictor and tracker pairings. The authors conclude that individual differences in the ability for temporal prediction are an important factor in the interaction between various cognitive, motor, and social processes that are involved in action coordination tasks within musical domains.
Pecenka and Keller’s study highlights the importance of the capacity to synchronize actions when engaged in certain kinds of joint action tasks. An interesting and relevant question is whether or not individual members of a pair change their own performance to facilitate the attainment of synchronous performance. To address this question, Vesper et al. report three demonstrating that increasing predictability of performance is a coordination strategy employed in joint action tasks in which continuous feedback from a partner’s actions is not available. The dependency on an intention to coordinate suggests that reducing variability is a coordination strategy that is different from known entrainment tendencies and motor simulation processes in joint coordination tasks.
Attention and perceptual processing in joint action
In a large number of joint action scenarios, sharing attention with others and perceiving interaction partners’ actions is critical for the successful attainment of task goals. Several studies in the special issue address questions relating to attention and perceptual processing in joint action tasks.
Böckler et al. investigated whether joint attention to a stimulus from different perspectives has an influence on the frame of reference (egocentric or allocentric) that people adopt. In particular, they determined whether joint attention to hand stimuli changes performance on a mental rotation task in which handedness is judged. By examining changes in reaction time as a function of degree of rotation, the authors found that joint attention changes the frame of reference from an egocentric to an allocentric one. Importantly, this effect seems to depend on joint attention and not the mere presence of another individual. The switch from an egocentric frame of reference to an allocentric frame of reference in joint attention conditions is suggested to facilitate the formation of perceptual common ground in joint action and communication.
A key ability for successful joint action, and indeed successful social interaction more generally, is the ability to predict what another person’s intentions are, not least whether they are a friend or foe, and want to act cooperatively or competitively. Manera et al. report results from an experiment designed to address whether individuals can uncover the social intentions of another person simply by observing the kinematics of their movement. Using a temporal-visual occlusion task, they presented participants with videos, or point light displays of a person reaching and grasping for a wooden block in various contexts, that differed in the social intention underlying the movement. Interestingly, participants were able to accurately ascertain the social intention of the model in the point light display conditions, suggesting that kinematic information alone is sufficient for decoding movement intentions. The authors discuss the results in the context of motor simulation (e.g., Kilner et al. 2004; Prinz 1997).
Sartori et al. report a transcranial magnetic stimulation (TMS) study aimed at determining whether or not social context changes the response of the motor system to observed action. In their experiment, participants watched a video of another person grasping different sized objects (i.e., an almond or an apple) and placing them on a tray. Later on in the observed sequence, the actor held out their hand as if to ask for one of the objects that was out of reach. By examining the size of TMS-induced motor-evoked potentials (MEPs) in the muscle specific to the precision and whole-hand grasps required for grasping the almond or apple, respectively, the authors demonstrated that observation of the initial grasp led to motor resonance for the same action (i.e., a true “mirror” response). Crucially, however, when TMS was delivered at the point where the model beckoned for an object, motor representations that would be involved in the complementary response of picking up the requested object were found to be excited. The authors discuss their results in the light of new ideas about the role of the putative human mirror system in complementary and “mirror” actions (e.g., Newman-Norlund et al. 2007).
Shibata et al. examine the neural correlates of joint action observation using functional magnetic resonance imaging (fMRI). These authors presented video clips of two people in a joint action scenario in which one person held out two objects and asked the other person to take one, or in which only the active participant was shown with an imaginary “requesting agent” off screen. In both experiments, the action performed by the “taker” was congruent action or incongruent with respect to the request. The authors found that the right inferior frontal gyrus (rIFG) and right cerebellum were more activated in the incongruent action condition than in the congruent action condition in the joint action scenario, whereas the posterior superior temporal sulcus (pSTS) and the medial prefrontal cortex (mPFC) showed activation for both the single and joint action conditions. The results extend previous research by demonstrating that the rIFG region may mediate higher-order action understanding relating to the congruence of a requested complementary action (Newman-Norlund et al. 2007). These results are discussed with respect to the potential role of expectation violation processing and action prediction mechanisms linked to brain areas implicated in the human mirror system (e.g., Kilner et al. 2007).
Much progress in understanding the mechanisms underlying joint action has been made by considering joint action from a developmental perspective (Brownell 2011; Tomasello 2009). It is beyond the scope of our special issue to cover all the exciting new findings in this field, which concern the development of shared attention (Carpenter and Liebal in press), the use of pointing gestures (Liszkowski et al. 2008), the emergence of helping behavior (Warneken and Tomasello 2007), as well as the understanding of commitment (Gräfenhain et al. 2009) and social norms (Rakoczy et al. 2008). The studies in this special issue, however, provide a sample of some of the exciting research questions currently being addressed in this field, including questions about children’s processing of interaction partners’ actions, their understanding of others’ action capabilities, and the development of social cognition in the adolescent brain.
Meyer et al. conducted a joint action study in 3-year-olds based on previous research that has shown increased motor system involvement (indexed by changes in the contingent negative variation (CNV) and reduced activity in the beta-frequency band of the electroencephalogram) in adults engaged in predicting the actions of a joint action partner (Kourtis et al. 2010). The authors investigate whether an increase in motor system involvement during action observation in a joint action scenario can already been seen in 3-year-olds, and whether this increased involvement is correlated with better performance in the joint action task. Results showed that children exhibited greater motor activity when they observed actions of their joint action partner. Furthermore, the degree of motor system involvement was negatively correlated with the number of errors made by the child in their own performance. That is, the less the motor system was engaged during observation, the more errors the child made. The authors suggest that the increased relevance of a joint action partner could enhance attentional and motivational processes, thereby increasing engagement of the motor system.
A key ability for many real-world joint action tasks is the ability to take into account a potential partner’s action capabilities as they relate to the goal of a particular task. For example, when an object is placed on a high shelf out of reach, most adults would naturally ask a taller person as opposed to a shorter person to assist them with retrieving the object. Paulus and Moore address a gap in knowledge regarding development of this ability. Using a dolls game in which one doll required help from one of two other dolls, the authors investigated the capacity to take into account others’ action capabilities when asking for help in a sample of 2.5-, 3.5- and 5-year-old children. They found clear developmental effects, with 3.5- and 5-year-old children able to identify the doll that had the action capabilities to help meaningfully with the task. In contrast 2.5-year-olds did not demonstrate this ability. Hence, the ability to take others’ action capabilities into account when requesting help seems to emerge at around 3 years of age. The authors discuss their results in terms of the potential role for motor resonance and affordance perception.
Whereas some important capacities for successful joint action performance emerge at a fairly young age, during adolescence many important brain changes and developments take place that might underpin several capacities that are important for effective joint action. Cook and Bird investigate the relationship between social attitudes and imitative behavior in adolescents and adults. Previous research in both the social psychological and cognitive neuroscience domains has demonstrated that pro-social attitudes are associated with greater imitation, and that imitation itself can foster more pro-social attitudes (e.g., Chartrand and Bargh 1999; Leighton and Heyes 2010; Obhi and Hogeveen 2010). Specifically, Cook and Bird ask whether the effects of pro-sociality on automatic imitation differ for adolescents compared with adults, given that brain areas important for key social abilities do not mature until well into adolescence. Using an automatic imitation task, the authors replicated previous findings for the adult group that confirm an effect of pro-social priming on automatic imitation. In contrast, there were no effects of social priming on automatic imitation for the adolescent group. The authors discuss their results in relation to the maturation of brain regions thought to be crucial for social cognition and behavior.
Efforts to model the perceptual, motor, and cognitive processes involved in joint action have been key in guiding experimentation and in providing constraints for theorizing (Bicho et al. 2011; Butz and Pezzulo 2008; Cuijpers et al. 2006). Three contributions in this special issue offer computational models that respectively target the role of shared representations for joint action, the processes governing continuous motor coordination, and the role of perception–action links.
In combination with experimental data from a human–computer experiment, Pezzulo and Dindo provide a probabilistic model of joint action that explores how shared representations help solving interaction problems. The model demonstrates how shared representations facilitate prediction by aligning agents’ mental states, and highlights the importance of signaling actions performed to increase the probability that representations become shared. This contribution not only serves to spell out the functions of shared representations in the context of joint action, but also demonstrates new ways to study the emergence and strategic use of signaling actions during joint action (Galantucci 2005).
Braun et al. provide a model of continuous motor coordination within a game theoretical framework. The basic idea is that just like well-known coordination problems such as the “stag hunt” game, motor coordination also requires choosing particular strategies whereby none of the other players gains anything by changing only his or her strategy unilaterally (Nash equilibrium). Employing motor versions of different coordination games containing Nash equilibria, the authors determined general features that characterize successful coordination, including stereotypical motor patterns and an early convergence toward the final position by one of the agents. Their model, in which each agent samples the noisy payoff gradient with respect to their movement and acts so as to reduce effort, suggests that coordination can arise through gradient-descent-like co-adaptation.
Finally, based on the findings from single neuron recordings in monkeys’ observing and performing actions, Chersi has developed a neural network model describing the role of close perception–action links in joint action. In this model, motor sequences are represented in a way that allows individuals to switch between observing actions and performing actions in response to others’. Additionally, the model describes how the connections between parietal mirror neurons and prefrontal cortex can be exploited for monitoring and predicting others’ actions.
Agency in joint action
An important topic that has only very recently started to be addressed concerns the phenomenology of joint action (Pacherie 2011). How does joint action modulate agency, the experience of controlling one’s actions and their effects on the world? Might there be experiences of joint agency that are qualitatively different from individual agency? The special issue includes two studies that provide some first answers to these challenging questions.
Obhi and Hall compared an explicit and an implicit measure of agency for effects following from jointly produced actions that were initiated by one of two co-actors. When individuals perform an action that is followed by an effect such as an auditory tone, they perceive the interval between the action and the effect as shorter than it actually was. As this only occurs for intentionally performed actions, this compression has been termed ‘intentional binding’ and is considered to provide an implicit measure of agency. Interestingly, even though only the person initiating the joint action reported a reliable subjective experience of agency, the initiator and the person joining in both showed intentional binding. The authors discuss the possibility that the person joining in might experience the outcome as their own at a pre-reflective level, because they experience ‘we’ agency rather than ‘I’ agency. This interpretation is in line with findings of a second study by the same authors where they found intentional binding when participants interacted with another human, but not when they interacted with a computer.
From the short summaries provided above, it is evident that this special issue has brought together some of the latest developments from many prominent researchers in the field of Joint Action. The number of contributions shows that this is an increasingly popular area of study, and as the intensity of research grows, we are optimistic that key insights into the functional and neural mechanisms that govern joint action across different tasks and contexts will continue to emerge in the years to come.
SSO and NS would like to thank Mel Goodale, Neil Solomon and Margrit Derrick for their editorial guidance in putting this special issue together.