The Human Dynamic Clamp: A Probe for Coordination Across Neural, Behavioral, and Social Scales
Humans (with their brains, bodies and behaviors) are complex dynamical systems embedded in an environment that includes a multitude of other conspecifics. Moving beyond previous brain- centered views of the human mind requires to develop a parsimonious yet integrative account that relates neural, behavioral, and social scales. Social neuroscience has recently started to acknowledge the importance of relational dynamics when it extended its purview from social stimuli to human-human interactions. Human-machine interactions also constitute promising tools to probe multiple scales in a controlled manner. Inspired by the electrophysiological method of the dynamic clamp, Virtual Partner Interaction (VPI) allows real time interaction between human subjects and their simulations as dynamical system. This provides a new test bed for operationalizing theoretical models in experimental settings. We discuss how VPI can be generalized into a Human Dynamic Clamp (HDC), a paradigm that allows the exploration of the parameter spaces of interactional dynamics in various contexts: from rhythmic and discrete coordination to adaptive and intentional behaviors, including learning. HDC brings humans and machines together to question our understanding of the natural and our theory behind the artificial.
Social neuroscience seeks to bridge the gap between the neural, the behavioral and the social. Such an agenda contrasts with cognitive science and the shortcoming of its brain-centered and individualistic approach to the mind. Recently, several approaches have proposed to go beyond a third person representational account of others by investigating social interaction from developmental, dynamical and relational viewpoints. This departure from a strictly reductionist view calls for new manners of empirical investigation of social systems along with a theoretical account of their various scales of organization. With those advances, one aims to integrate complementary aspects of the problem of social coordination into a coherent, comprehensive and parsimonious whole. In this respect, non-linear dynamical systems theory has already proved a good formalism to relate biological, psychological and more recently social levels . This paper discusses a new experimental paradigm grounded in the framework of Coordination Dynamics [2, 3, 4]. We describe the development of Virtual Partner Interaction (VPI), a system allowing to couple a human with a theoretical model of movement coordination in real time [5, 6]. We review its generalization into the “Human Dynamic Clamp” (HDC), a new paradigm for Cognitive Science to study the multiple scales of coordination that govern human brain and behavior.
This novel paradigm pursues an already ongoing grip of Cognitive Science toward multiscale coordination [1, 3, 7]. In the exemplary case of hand movements for instance, social interactions span multiple scales in time: from position, phase and frequency of movements to the turn-taking between people (e.g. ). Such social interaction also gives rise to neural coordination within and across brains [9, 10, 11]. Multiple scales are also present in space, from the processing of information at synaptic levels to the level of large neural assemblies giving rise to different brain rhythms . Moreover, neurophysiology shows how temporal and spatial dimensions are intertwined: neural oscillations at large time-scales (i.e. low frequencies) tend to cover larger scales in space, whereas shorter time-scales (i.e. high frequencies) appear to be more localized . Thus, both brain and behavior are meshed together across multiple scales of time and space.
Since the present scientific approach aims to combine experimental studies with theoretical models, the key challenge is to connect these observations across scales and levels of organization within a coherent theoretical framework . Coordination dynamics aims at such understanding through the synergetic concepts of self-organization  and the mathematical tools of dynamical systems theory [3, 16, 17]. It seeks both general principles and functionally-specific mechanisms of coordination  and aims at connecting multiple scales by emphasizing reciprocal coupling between levels, upward and downward . In this perspective, coordination between humans represents an operational playground for experimental investigation at the crossroad of the neural, the behavioral and the social.
Recently, hyperscanning techniques have offered access to the simultaneous recording of brain activity from interacting people and thus to the study of brain and behavior coordination at both intra- and inter-individual scales [11, 18, 19, 20, 21, 22]. In doing so, this technique has also reintroduced real social interaction into laboratory studies of human behavior, a key feature that was oddly lacking from earlier work within a (cognitively-inspired) social neuroscience, as it resorted to exposing one subject to social “stimuli” rather than examining interactions [23, 24, 25, 26]. Further, the use of reciprocal paradigms and a real second-person approach of social cognition do not necessarily require the presence of two or more subjects in the experimental task . Instead, one of the interacting partners can be substituted with a virtual agent whose design sustains bi-directional coupling between real and simulated partners [6, 27, 28].
2 Human-Machine Interaction as a Research Tool
Meanwhile, in other areas of science and engineering, a plethora of studies was concentrating on subjective perception of artificial agents by humans, with the goal of designing realistic avatars for potential applications to, e.g. video games, cinema, or eLearning assistants  to name just a few. In this line of research, the exercise was to mimic facets of human behavior rather than to model foundational neurobehavioral mechanisms. Interestingly, participants’ beliefs of realism were influenced by emotionally and behaviorally contingent responses made by the artificial agent ; see also ; this finding hints at the importance of reciprocal coupling with the human.
The development of realistic artificial agents extended the toolset available to social psychological research , with more to come as those agents are embedded in virtual realities that are increasingly indistinguishable from “normal” reality. The breakthrough of virtualization has reconciled ecological validity and experimental control, e.g. in the study of visual perception, spatial cognition and social interaction [5, 33].
A first level of social interaction is the mere presence of someone else . Regarding this issue, virtual reality fits particularly well since it creates a sense of presence through mediated environments carrying dynamic animations of virtual characters . Virtual characters are readily perceived as social agents and are thus capable of exerting social influence on humans . Those virtual characters with strong similarity to real human interactions  can easily and valuably be combined with neuroimaging recording .
Human-machine interaction was also used to investigate motor coordination: for instance a finger tapping study by Repp and Keller  used a simple linear phase correction model to drive a virtual agent. It showed that subjects’ behavior was systematically modulated by the computational parameters governing that agent. Reframed in a functional neuroimaging study by Fairhurst et al. , the same paradigm uncovered some neural basis for motor synchronization and more importantly, for the socio-emotional consequences of different degrees of entrainment success.
In the following, we describe another paradigm, the Human Dynamic Clamp (HDC), that embraces a continuous, multiscale and nonlinear coupling between a human and a machine. By departing from information processing approaches and design-oriented modeling, the HDC offers: (a) a new way to bridge the gap between theory, experiment and models; and (b) an integrative solution to linking neural, behavioral, and social dynamics. HDC puts well established equations of human coordination dynamics into the machine and studies real-time interactions between human and virtual partners. This opens up the possibility to explore and understand a wide variety of interactions [5, 6, 40]. Ultimately, HDC may prove useful to establishing a much friendlier union of man and machine, based on sound interactional design, and perhaps it will even lead to the creation of a different kind of machine altogether.
3 A Principle-Based Virtual Partner
The study of movement coordination is at the core of coordination dynamics and for the last 30-odd years the catchy phrase “let your fingers do the walking” has opened a rich experimental window into human behavior at both intra-individual and inter-individual levels. In a first move, it is important to clarify what we are looking at . What is the behavior? What are the relevant variables and control parameters? These fundamental issues are addressed by uncovering qualitative changes in collective variables from the system called order parameters [15, 41]. Qualitative changes appear in two main flavors within the formalism of dynamical system theory: phase transitions and bifurcations. Although they are both revealed in the phenomenon of transition in collective dynamics, the first is related to the switch between potential modes of behavior simultaneously accessible to the system, and the second concerns global changes of the system’s behavioral landscape. The landscape is usually described with a manifold in phase space (the frame of reference representing the relationship between variables associated with each degree of freedom). The challenge then is to uncover the most parsimonious model that can exhibit these qualitative changes, and fit its parameters according to the experimental data (see the discussion of Phenomenological Synergetics in . One key issue to keep in mind lies with the biological constraints that make it possible to link a model to actual physiological mechanisms. In this perspective, it is fundamental to recognize that all models are false by definition. However, dynamical system theory offers good candidates for a universal class of models, given the needed parsimony for elegant theory [43, 44].
Born from this aim was our recently developed Human Dynamic Clamp, a paradigm that took inspiration from the electrophysiological dynamic clamp [45, 46] to allow real-time interaction between a human subject and a computational model. Using empirically-grounded models not only validated reciprocal and fully dynamical design protocols for experimenters to use, but also provided the opportunity to explore parameter ranges and perturbations that were out of reach of traditional experimental designs with live interactions. The symmetry between the human and the machine and the fact that they carry the same laws of coordination dynamics were keys to our approach . The design of the virtual partner (VP) was grounded in the equations of motion for the coordination of the human neurobehavioral system. These laws were obtained from accumulated studies over the last 30-odd years to describe how parts of the human body and brain self-organize, and to address the issue of self-reference, a condition leading to complexity.
The first version of the Human Dynamic Clamp called Virtual Partner Interaction  embodied the Haken–Kelso–Bunz (HKB) model . The original form of HKB describes and predicts the coordination dynamics of two rhythmically moving fingers, with its characteristically complex phenomena such as multistability, phase transitions, hysteresis, critical slowing-down and fluctuation enhancement ([42, 17] for reviews). Since then, the model has also been successfully validated experimentally for the coordination between different limbs (e.g. ), between people (e.g. ) and even between species , within unimodal and multimodal contexts . It has been supported by empirical evidence ranging from brain dynamics within [51, 52] and between brain areas [46, 53, 54, 55], to coordination with external stimuli  and neural counterparts thereof [57, 58]. This universal characteristic supports HKB as an ideal candidate for the Human Dynamic Clamp.
However, since computers do not have direct access to the relative phase, the internal dynamics of VP is governed by the HKB model at the component level (see Fig. 1b). In this form, two non-linearly coupled non-linear oscillators represent the interaction between the two fingers. The collective form in Eq. (1) can be derived from the equations at the component level (Fig. 1b). At the component level, variables are no longer the relative phase but the individual finger positions (and velocities by derivation). x and y represent VP’s and human’s finger positions, α, β and γ are constants associated with the intrinsic dynamics of VP, ω is VP’s pulsation (frequency), A and B are constants associated with the coupling from VP to human, and finally μ is a constant fixed to either +1 or −1, indicating VP’s preference for in-phase or anti-phase coordination.
In the original study , VP and human behaviors were chosen to be quite simple. Both partners were tasked to coordinate finger movements with one another, the human with the intention of achieving in-phase coordination with the VP (trying to synchronize his/her flexion and extension movements with VP’s). On the VP side, the parameter μ was set to −1, inducing a VP preference for anti-phase coordination and thus a goal opposite to human’s. Subjects were instructed to maintain a smooth and continuous rhythmic movement with their right index finger (flexion-extension) and to avoid stopping their movement at any time. Visual coupling was experimentally manipulated: from unidirectional in two conditions (VP “perceives” human movement but human does not perceive VP’s behavior; or reciprocally), to bi-directional in another (both VP and human have access to each other’s finger movement). VPI accommodated the whole set of behavioral coordination modes described by the HKB model. For instance, when VP and human participants did not have the same preferred movement frequency, their relative phase conformed to predictions by the extended version of HKB  and exhibited phase wrapping (not shown) or metastability (see Fig. 1c). Pitting machine against human through opposing task demands is a way to enhance the formation of emergent behavior, and also allowed us to examine each partner’s individual contribution to the collective behavior. An intriguing outcome of the experiments was that subjects ascribed intentions to the machine, reporting that it was “messing” with them. A later study further suggested that VP elicits emotional experiences in human: subjects’ emotional arousal was greatest when VP interactions were (falsely) deemed to be with a human rather than with a machine .
In summary, Kelso et al.  initial VPI experiment demonstrated the feasibility of the Human Dynamic Clamp in the context of a continuous coordination of rhythmic movements. It uncovered unexpected behaviors, which were theoretically tested afterward. In the following, we show how to explore a new set of behaviors with other theoretical models of human behavior.
4 Expanding the Behavioral Repertoire
4.1 Discrete Behavior: Phase-Space Sculpture
Although it is undeniable that living organisms rely both on rhythmic and discrete behaviors, the field of motor control has traditionally studied them separately. This led to two different ways of theoretically approaching and modeling them. While rhythmic movements have been extensively studied through the prism of dynamical systems, discrete movements’ modeling has focused on equilibrium points or control signals . Unifying rhythmic and discrete movements is often posed to be a key theoretical challenge in behavioral science [62, 63]. However, there is no specific need to invoke two separate mechanisms for discrete and rhythmic behavior [64, 65, 66]. For instance, Schöner  extended the HKB model to the case of discrete bistable coordination by changing the intrinsic dynamics. Sternad et al.  proposed another model for unimanual coordination with two mutually inhibiting subsystems, each of which handled the discrete and the continuous cases respectively.
Along similar lines, Jirsa and Kelso  modeled discrete and rhythmic movement based on the phase flow topology of the so-called “Excitator” model (see also ). The Excitator defines a universal class of two-dimensional dynamical systems able to exhibit limit cycles for rhythmic movement, and fixed point dynamics for discrete movement. This model is based on topological considerations and is a parsimonious way to handle discrete and continuous behaviors simultaneously. Furthermore, in line with the approach of HKB modeling, the Excitator provides predictions regarding false-start phenomena that have been confirmed experimentally . Finally, it is a biologically realistic model since it follows the self-excitable property that the FitzHugh-Nagumo model drew from single neurons .
When this is put in unidimensional form, we retain the same coupling terms as HKB model’s. The coupling causes either convergence or divergence of the trajectories in phase space depending on initial conditions. Since trajectories are bounded, constraints lead to in-phase or anti-phase modes of coordination (for more details, see ).
Implementation of the Excitator dynamics in an HDC is quite straightforward: one only needs to substitute the relevant equations from (2) in the software controlling VP’s behavior. Note that these equations introduce a new term of importance: parameter I allows to modify the phase flow according to an external input. An external input can originate from the experimenter or the human partner him/herself. It is a key component for modeling discrete behaviors, which rely on external information and is non-autonomous in a mathematical sense. The introduction of the new variable allows VP and human to coordinate diverse movements that range from simple rhythms to discrete actions. Figure 2a presents an interaction between a human and a VP governed by the Excitator model, and shows a transition from discrete movement (flexions and extensions interrupted by quiescent behavior) to continuous movement.
4.2 Adaptive Behavior: Parameter Dynamics and Modularity
The Excitator model shows how a single dynamical system may give rise to different behavioral modes of coordination between human and virtual partner. However, each mode required a different set of parameters. Once those parameters are fixed, the differential equations set the functional structure of the system for a specific behavioral context. But structure, function and dynamics are not separated in nature; everything is constantly evolving on different time scales [7, 3]. In biology, organisms change their own behavior and learn new ones to better face the world, and interact with their peers in a more effective manner. Robert Rosen even associated adaptation as the most characteristic property of living things . The process of adaptation is ubiquitous in so-called complex adaptive systems that may also encompass physical or artificial aspects . In the case of the brain, it is not surprising to observe such ongoing anticipation continuously . Adaptation is especially important in social behavior, for instance mimicry at the morphological level  or interactional synchrony during cooperative imitation and skill learning .
Coordination may be seen as a subtle blend of reaction and adaptation to the other . Whereas reaction takes place at a given time, adaptation builds up over time. For instance, humans may have a preferred movement frequency but they can adapt to different partners by slowing down or speeding up their movements. In the case of the Human Dynamic Clamp, frequency adjustment is a good candidate to address adaptive behavior in a manner that is fully compatible with the previously described systems, and uses the same formalism. Basically, frequency adaptation requires a new equation in the system of differential equations that manages the rate of change of frequency ω. At a more conceptual level, it fits with the idea that adaptation depends on the system’s ongoing intrinsic dynamics. Furthermore, adaptation can enhance the realism of the interaction, by expanding beyond an instantaneous coordination with the position of a finger or the phase of a movement.
Different strategies for modeling frequency adaptation have been proposed. In a pure Artificial Intelligence (AI) tradition, a specific module detects the frequency of the human partner which then controls VP’s actual frequency. This shows that it is possible to successfully design an artificial device that is able to do the job. In the Bayesian approach, adaptation is error-based and relies on reinforcement learning . This approach is inspired from real physiological processes. In predictive coding, adaptation of model parameters is associated with Hebbian and synaptic plasticity in the brain . Other bottom-up strategies have been developed in the fields of signal processing  and robotics . Here we continue to follow the strategy of Coordination Dynamics and Dynamical System Theory. That approach was shown to better account for frequency adaptation in fireflies  and in tempo adaptation to musical rhythms ; see also . In contrast with the AI approach, it is worth noting that the equations stay totally continuous and do not relate to an artificial measurement of the human frequency. This illustrates how adaptation relies on parameter dynamics according to the scale of observation .
4.3 Intentional Behavior: Symmetry Breaking and Forcing
In the case of an adaptive system, we have seen that adding a third dimension renders the dynamics less predictable. The system is nevertheless not random and appears more autonomous while still being governed by deterministic rules. This balance between autonomy and coupling creates successful agency illusion and can trigger an attribution of intention to the human observer [89, 90]. Keeping in mind that the Human Dynamic Clamp aims at operationalizing models for experimental purposes, a teleonomic system is not adequate, because its intention is not directly controllable by the experimenter.
This modification of VP dynamics makes it possible to direct a collective behavior towards any desired pattern of coordination (see Fig. 2c). This offers new experimental perspectives, e.g. to study how new dynamical patterns are learned on top of a subject’s spontaneous behavioral repertoire .
In this chapter, we have seen how a hybrid system called the Human Dynamic Clamp allows for real-time interaction between humans and virtual partners, based on the equations of coordination dynamics built originally from HKB and its extensions. A key aspect is that the human and its virtual partner are reciprocally coupled: the human acquires information about the partner’s behavior through perception, and the virtual partner continuously detects the human’s behavior through the input of sensors. Our approach is analogous to that of the original dynamic clamp used to study the dynamics of interactions between neurons, but now scaled up to the level of behaving humans. This principle-based approach offers a new paradigm for the study of social interaction. While stable and intermittent coordination behaviors emerged that had previously been observed in ordinary human social interactions, we also discovered novel behaviors or strategies that had never been observed in human social behavior. Those novel behaviors pertained to unexplored regions of the theoretical model and were possible ways of coordination for people to interact with each other. Such emergence of novel behaviors demonstrates the scientific potential of HDC as a human-machine framework. Modifying the dynamics of the virtual partner with the purpose of inducing a desired human behavior, such as learning a new skill or as a tool for therapy and rehabilitation, is one of several applications of VPI.
HDC allows to study social interaction with more experimental control than other recent social neuroscience methods (e.g. hyperscanning); it is also a test bed for theoretical models. HDC moves away from simple protocols in which systems are ‘poked’ by virtue of ‘stimuli’ to address more complex, reciprocally connected systems where meaningful interactions occur. Thus, the Human Dynamic Clamp supports the development of a computational social neuroscience where theory, experiment and modeling work hand-in-hand across neural, behavioral and social scales .
The research described herein was supported by grants from the National Institute of Mental Health (MH080838), the Davimos Family Endowment for Excellence in Science and The FAU Foundation (Eminent Scholar in Science).
- 3.J.A.S. Kelso, Dynamic Patterns: The Self-Organization of Brain and Behavior (The MIT Press, Cambridge, 1995)Google Scholar
- 8.O. Oullier, G.C. de Guzman, K.J. Jantzen, J.A.S. Kelso, On context dependence of behavioral variability in inter-personal coordination. Int. J. Comput. Sci. Sport 2, 126–128 (2003)Google Scholar
- 14.H.R. Maturana, F.J. Varela, The Tree of Knowledge: The Biological Roots of Human Understanding (New Science Library/Shambhala Publications, 1987)Google Scholar
- 17.G. Schöner, A dynamic theory of coordination of discrete movement. Biol. Cybern. 63(4), 257–270 (1990)Google Scholar
- 27.J. Mattout. Brain-computer interfaces: a neuroscience paradigm of social interaction? A matter of perspective. Front. Hum. Neurosci. (2012). https://doi.org/10.3389/fnhum.2012.00114
- 29.J. Scholtz, Theory and evaluation of human robot interactions, in Proceedings of the 36th Annual Hawaii International Conference on System Sciences, 2003 (IEEE, 2003), p 10Google Scholar
- 30.M. Ochs, C. Pelachaud, D. Sadek, An empathic virtual dialog agent to improve human-machine interaction, in Proceedings of the 7th International Joint Conference on Autonomous Agents and Multiagent Systems (Estoril, Portugal, 2008)Google Scholar
- 32.L. Schilbach, A.M. Wohlschlaeger, N.C. Kraemer et al., Being with virtual others: neural correlates of social interaction. Neuropsychologia 44, 718–730 (2006). https://doi.org/10.1016/j.neuropsychologia.2005.07.017 CrossRefGoogle Scholar
- 41.J.A.S. Kelso, Phase transitions: foundations of behavior, in Synergetics of Cognition, ed. by H. Haken, M. Stadler (Springer, Berlin, 1990), pp. 249–268Google Scholar
- 43.M. Golubitsky, I. Stewart, The Symmetry Perspective: From Equilibrium to Chaos in Phase Space and Physical Space (Springer Science and Business Media, 2003)Google Scholar
- 46.S.H. Strogatz, Nonlinear Dynamics and Chaos (Westview Press, Boulder, 2008)Google Scholar
- 56.J.A.S. Kelso, J. DelColle, G. Schöner, Action perception as a pattern formation process, in Attention and Performance XIII, ed. by M. Jeannerod (Erlbaum, Hills-dale, NJ, 1990), pp. 139–169Google Scholar
- 59.A. Fuchs, J.A.S. Kelso, Movement coordination, in Encyclopedia of Complexity and Systems Science (Springer, 2009), pp. 5718–5736Google Scholar
- 64.R. Huys, V.K. Jirsa, Nonlinear Dynamics in Human Behavior (Springer, Heidelberg, 2010)Google Scholar
- 66.P.N. Kugler, J.A.S. Kelso, M.T. Turvey, On the concept of coordinative structures as dissipative structures: I. Theoretical lines of convergence. Tutor. Mot. Behav. 3, 47 (1980)Google Scholar
- 67.D. Sternad, W.J. Dean, S. Schaal, Interaction of rhythmic and discrete pattern generators in single-joint movements. Hum. Mov. Sci. 19(4), 627–664 (2000)Google Scholar
- 71.R. Rosen, Life itself: a comprehensive inquiry into the nature, origin, and fabrication of life (Columbia University Press, New York, 1991)Google Scholar
- 72.J. Holland, Adaptation in Natural and Artificial Systems (MIT Press, Cambridge, MA, 1992)Google Scholar
- 75.A. Fogel, Two principles of communication: co-regulation and framing. New Perspect. Early Commun. Dev. 9–22 (1993)Google Scholar
- 77.J. Peters, S. Schaal, Policy gradient methods for robotics, in 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems (IEEE, 2006), pp. 2219–2225Google Scholar
- 78.Friston KJ (2010) The free-energy principle: a unified brain theory? 11:127–138. https://doi.org/10.1038/nrn2787
- 83.I. Konvalinka, P. Vuust, A. Roepstorff, A coupled oscillator model of interactive tapping, in Proceedings of the 7th Triennial Conference of European Society for the Cognitive Sciences of Music (Jyvaskyla, Finland, 2009)Google Scholar
- 88.W.J. Freeman, Consciousness, intentionality and causality. J. Conscious. Stud. 6, 143–172 (1999)Google Scholar
- 93.E. Tognoli, G. Dumas, J.A.S. Kelso, A roadmap to computational social neuroscience, in Advances in Cognitive Neurodynamics (V) (Springer, Dordrecht, in press)Google Scholar