Advertisement

Analyzing Social Interactions: The Promises and Challenges of Using Cross Recurrence Quantification Analysis

  • Riccardo FusaroliEmail author
  • Ivana Konvalinka
  • Sebastian Wallot
Conference paper
Part of the Springer Proceedings in Mathematics & Statistics book series (PROMS, volume 103)

Abstract

The scientific investigation of social interactions presents substantial challenges: interacting agents engage each other at many different levels and timescales (motor and physiological coordination, joint attention, linguistic exchanges, etc.), often making their behaviors interdependent in non-linear ways. In this paper we review the current use of Cross Recurrence Quantification Analysis (CRQA) in the analysis of social interactions, and assess its potential and challenges. We argue that the method can sensitively grasp the dynamics of human interactions, and that it has started producing valuable knowledge about them. However, much work is still necessary: more systematic analyses and interpretation of the recurrence indexes and more consistent reporting of the results,more emphasis on theory-driven studies, exploring interactions involving more than 2 agents and multiple aspects of coordination,and assessing and quantifying complementary coordinative mechanisms. These challenges are discussed and operationalized in recommendations to further develop the field.

Keywords

High Recurrence Rate Coordinative Structure Recurrence Quantification Analysis Diagonal Structure Interpersonal Coordination 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Notes

Acknowledgments

This research is supported by the Danish Council for Independent Research—Humanities & Technology and Production Sciences, the Interacting Minds Center (Aarhus University), the ERC Marie Curie Training Network Towards an Embodied Science of Intersubjectivity (TESIS) and the EUROCORES project: Digging the Roots For Understanding (DRUST).

References

  1. 1.
    Kimmel, M.: Intersubjectivity at close quarters: how dancers of Tango Argentino use imagery for interaction and improvisation. J. Cogn. Semiot. 4, 76–124 (2012)Google Scholar
  2. 2.
    Kiefer, A.W., et al.: Quantifying the coherence of pedestrian groups. in Cog Sci 2013, Berlin (2013)Google Scholar
  3. 3.
    Fusaroli, R., et al.: Coming to terms: an experimental quantification of the coordinative benefits of linguistic interaction. Psychol. Sci. 23(8), 931–939 (2012)CrossRefGoogle Scholar
  4. 4.
    Hutchins, E.: How a cockpit remembers its speeds. Cogn. Sci. 19(3), 265–288 (1995)CrossRefGoogle Scholar
  5. 5.
    Miller, J.H., Page, S.E.: Complex Adaptive Systems: An Introduction to Computational Models of Social Life. Princeton Studies in Complexity. Princeton University Press, Princeton (2007)Google Scholar
  6. 6.
    Dale, R., et al.: The self-organization of human interaction. Psychol. Learn. Motiv. 59, 43–95 (2013)CrossRefGoogle Scholar
  7. 7.
    Hasson, U., et al.: Brain-to-brain coupling: a mechanism for creating and sharing a social world. Trends Cogn. Sci. 16(2), 114–121 (2012)CrossRefGoogle Scholar
  8. 8.
    Steptoe, A., et al.: Social isolation, loneliness, and all-cause mortality in older men and women. PNAS 110(15), 5797–5801 (2013)CrossRefGoogle Scholar
  9. 9.
    Van Orden, G.C., Holden, J.G., Turvey, M.T.: Self-organization of cognitive performance. J. Exp. Psychol. Gen. 132(3), 331 (2003)CrossRefGoogle Scholar
  10. 10.
    Fusaroli, R., Raczaszek-Leonardi, J., Tylén, K.: Dialog as interpersonal synergy. N. Ideas Psychol. 32, 147–157 (2014)CrossRefGoogle Scholar
  11. 11.
    Riley, M.A., et al.: Interpersonal synergies. Frontiers Psychol. 2, 38 (2011)CrossRefGoogle Scholar
  12. 12.
    Konvalinka, I., Roepstorff, A.: The two-brain approach: how can mutually interacting brains teach us something about social interaction? Frontiers Human Neurosci. 6, 1 (2012)CrossRefGoogle Scholar
  13. 13.
    Bakeman, R., Gottman, J.: Observing Interaction: An Introduction to Sequential Analysis, 2nd edn. Cambridge University Press, Cambridge (1997)CrossRefGoogle Scholar
  14. 14.
    Chartrand, T.L., Bargh, J.A.: The chameleon effect: the perception-behavior link and social interaction. J. Pers. Soc. Psychol. 76(6), 893–910 (1999)CrossRefGoogle Scholar
  15. 15.
    Pickering, M.J., Garrod, S.: Toward a mechanistic psychology of dialogue. Behav. Brain Sci. 27(02), 169–190 (2004)Google Scholar
  16. 16.
    Fusaroli, R., Tylén, K.: Carving Language for Social Coordination: a dynamic approach Interaction studies 13(1), 103–123 (2012)Google Scholar
  17. 17.
    Sacheli, L.M., et al.: Kinematics fingerprints of leader and follower role-taking during cooperative joint actions. Exp. Brain Res. 226, 473–486 (2013)CrossRefGoogle Scholar
  18. 18.
    Skewes, J., et al.: Implicit negotiation of leader-follower dynamics in an asymmetric joint aiming task. (under Revision)Google Scholar
  19. 19.
    Noy, L., Dekel, E., Alon, U.: The mirror game as a paradigm for studying the dynamics of two people improvising motion together. PNAS 108(52), 20947–20952 (2011)CrossRefGoogle Scholar
  20. 20.
    Sebanz, N., Bekkering, H., Knoblich, G.: Joint action: bodies and minds moving together. Trends Cogn. Sci. 10(2), 70–76 (2006)CrossRefGoogle Scholar
  21. 21.
    Masumoto, J., Inui, N.: Two heads are better than one: both complementary and synchronous strategies facilitate joint action. J. Neurophysiol. 109(5), 1307–1314 (2013)CrossRefGoogle Scholar
  22. 22.
    Mills, G.: Dialogue in joint activity: complementarity, convergence and conventionalization. N. Ideas Psychol. 32, 158–173 (2014)CrossRefGoogle Scholar
  23. 23.
    Vilar, L., et al.: Coordination tendencies are shaped by attacker and defender interactions with the goal and the ball in futsal. Hum. Mov. Sci. 33, 14–24 (2014)CrossRefGoogle Scholar
  24. 24.
    Dale, R., Warlaumont, A.S., Richardson, D.C.: Nominal cross recurrence as a generalized lag sequential analysis for behavioral streams. Int. J. Bifurcat. Chaos 21, 1153–1161 (2011)CrossRefzbMATHGoogle Scholar
  25. 25.
    Coco, M.I., Dale, R.: Cross-recurrence quantification analysis of categorical and continuous time series: an R package. arXiv preprint arXiv:1310.0201, (2013)
  26. 26.
    Zbilut, J.P., Giuliani, A., Webber, C.L.: Detecting deterministic signals in exceptionally noisy environments using cross-recurrence quantification. Phys. Lett. A 246(1), 122–128 (1998)CrossRefGoogle Scholar
  27. 27.
    Marwan, N., et al.: Recurrence plots for the analysis of complex systems. Phys. Rep. 438(5–6), 237–329 (2007)CrossRefMathSciNetGoogle Scholar
  28. 28.
    Takens, F., Rand, D.A., Young, L.S.: Detecting Strange Attractors in Turbulence, in Dynamical Systems and Turbulence, pp. 366–381. Springer, Berlin (1981)Google Scholar
  29. 29.
    Marwan, N.: Cross recurrence plot toolbox. Available at http://tocsy.pik-potsdam.de/crp.php (2012)
  30. 30.
    Orsucci, F., Giuliani, A., Webber, C.: Combinatorics and synchronization in natural semiotics. Physica A: Stat. Mech. Appl. 361(2), 665–676 (2006)CrossRefGoogle Scholar
  31. 31.
    Isenhower, R.W., et al.: Rhythmic bimanual coordination is impaired in young children with autism spectrum disorder. Res. Autism Spectr. Disord. 6(1), 25–31 (2012)CrossRefGoogle Scholar
  32. 32.
    Kelso, J.A.S.: Dynamic Patterns: The Self-organization of Brain and Behavior. MIT Press, Cambridge (1995). (4 p. of plates)Google Scholar
  33. 33.
    Richardson, M.J., et al.: Rocking together: dynamics of intentional and unintentional interpersonal coordination. Hum. Mov. Sci. 26(6), 867–891 (2007)CrossRefGoogle Scholar
  34. 34.
    Shockley, K., et al.: Cross recurrence quantification of coupled oscillators. Phys. Lett. A 305(1–2), 59–69 (2002)CrossRefzbMATHGoogle Scholar
  35. 35.
    Richardson, M.J., et al.: Comparing the attractor strength of intra- and interpersonal interlimb coordination using cross-recurrence analysis. Neurosci. Lett. 438(3), 340–345 (2008)CrossRefGoogle Scholar
  36. 36.
    Louwerse, M.M., et al.: Behavior matching in multimodal communication is synchronized. Cogn. Sci. 36(8), 1404–1426 (2012)CrossRefGoogle Scholar
  37. 37.
    Ramenzoni, V.C., et al.: Joint action in a cooperative precision task: nested processes of intrapersonal and interpersonal coordination. Exp. Brain Res. Experimentelle Hirnforschung. Experimentation Cerebrale 211(3–4), 447–457 (2011)CrossRefGoogle Scholar
  38. 38.
    Konvalinka, I., et al.: Synchronized arousal between performers and related spectators in a fire-walking ritual. PNAS 108(20), 8514–8519 (2011)CrossRefGoogle Scholar
  39. 39.
    Fusaroli, R., Bjørndahl, J., Tylén, K.: A Heart for Coordination: investigating speech, gesture and heart rate in a collective, creative construction task. submittedGoogle Scholar
  40. 40.
    Ramenzoni, V.C., et al.: Interpersonal and intrapersonal coordinative modes for joint and single task performance. Human Movement Science, 2012Google Scholar
  41. 41.
    Varni, G., et al.: Toward a real-time automated measure of empathy and dominance. In: IEEE International Conference on Computational Science and Engineering, CSE’09 (2009)Google Scholar
  42. 42.
    Varni, G., Volpe, G., Camurri, A.: A system for real-time multimodal analysis of nonverbal affective social interaction in user-centric media. IEEE Trans. Multimedia 12(6), 576–590 (2010)CrossRefGoogle Scholar
  43. 43.
    Reddy, V., Markova, G., Wallot, S.: Anticipatory adjustments to being picked up in infancy. PLoS ONE 8(6), e65289 (2013)CrossRefGoogle Scholar
  44. 44.
    Wallot, S., et al.: Division of labor as a cooperative strategy during complex joint action. submittedGoogle Scholar
  45. 45.
    Tylén, K., et al.: Language as a tool for interacting minds. Mind Lang. 25(1), 3–29 (2010)CrossRefGoogle Scholar
  46. 46.
    Clark, H.H.: Using Language. Cambridge University Press, Cambridge (1996)CrossRefGoogle Scholar
  47. 47.
    Shockley, K., Santana, M.V., Fowler, C.A.: Mutual interpersonal postural constraints are involved in cooperative conversation. J. Exp. Psychol. Hum. Percept. Perform 29(2), 326–332 (2003)CrossRefGoogle Scholar
  48. 48.
    Shockley, K., Richardson, D.C., Dale, R.: Conversation and coordinative structures. Topics Cogn. Sci. 1(2), 305–319 (2009)CrossRefGoogle Scholar
  49. 49.
    Shockley, K., et al.: Articulatory constraints on interpersonal postural coordination. J. Exp. Psychol. Hum. Percept. Perform 33(1), 201–208 (2007)CrossRefGoogle Scholar
  50. 50.
    Richardson, M.J., Marsh, K.L., Schmidt, R.C.: Effects of visual and verbal interaction on unintentional interpersonal coordination. J. Exp. Psychol. Hum. Percept. Perform. 31, 62–79 (2005)CrossRefGoogle Scholar
  51. 51.
    Marsh, K.L., Richardson, M.J., Schmidt, R.C.: Social connection through joint action and interpersonal coordination. Topics Cogn. Sci. 1(2), 320–339 (2009)CrossRefGoogle Scholar
  52. 52.
    Reuzel, E., et al.: Conversational synchronization in naturally occurring settings: a recurrence-based analysis of gaze directions and speech rhythms of staff and clients with intellectual disability. J. Nonverbal Behav. 37(4), 281–305 (2013)CrossRefGoogle Scholar
  53. 53.
    Ashenfelter, K.T.: Simultaneous Analysis of Verbal and Nonverbal Data During Conversation\(\vert \) Symmetry and Turn-taking. University of Notre Dame, Indiana (2008)Google Scholar
  54. 54.
    Richardson, D.C., Dale, R.: Looking to understand: the coupling between speakers’ and listeners’ eye movements and its relationship to discourse comprehension. Cogn. Sci. 29(6), 1045–1060 (2005)CrossRefGoogle Scholar
  55. 55.
    Richardson, D.C., Dale, R., Kirkham, N.Z.: The art of conversation is coordination: common ground and the coupling of eye movements during dialogue. Psychol. Sci. 18(5), 407–413 (2007)CrossRefGoogle Scholar
  56. 56.
    Richardson, D.C., Dale, R., Tomlinson, J.M.: Conversation, gaze coordination, and beliefs about visual context. Cogn. Sci. 33(8), 1468–1482 (2009)CrossRefGoogle Scholar
  57. 57.
    Dale, R., Kirkham, N.Z., Richardson, D.C.: The dynamics of reference and shared visual attention. Front. Psychol. 2, 355 (2011)CrossRefGoogle Scholar
  58. 58.
    Diderichsen, P.: Cross recurrence quantification analysis of indefinite anaphora in swedish dialog. An eye-tracking pilot experiment. In: Brandial’06: Proceedings of the 10th Workshop on the Semantics and Pragmatics of Dialogue (SemDial-10), Universitätsverlag Potsdam, Germany, 11–13 September 2006Google Scholar
  59. 59.
    Jermann, P., Nüssli, M.-A.: Effects of sharing text selections on gaze cross-recurrence and interaction quality in a pair programming task. In: Proceedings of the ACM 2012 Conference on Computer Supported Cooperative Work. ACM, (2012)Google Scholar
  60. 60.
    Fusaroli, R., Gangopadhyay, N., Tylén, K.: The dialogically extended mind: making a case for language as skilful intersubjective engagement. Cogn. Syst. Res. 29–30, 31–39 (2014)CrossRefGoogle Scholar
  61. 61.
    Fusaroli, R., Tylén, K.: Investigating conversational dynamics: interactive alignment, interpersonal synergies, and collective task performance. (under Revision)Google Scholar
  62. 62.
    Warlaumont, A.S., et al.: Vocal interaction dynamics of children with and without autism. In: Proceedings of the 32nd Annual Conference of the Cognitive Science Society. TX: Cognitive Science Society, Austin, (2010)Google Scholar
  63. 63.
    Cox, R.F., van Dijk, M.: Microdevelopment in parent-child conversations: from global changes to flexibility. Ecol. Psychol. 25(3), 304–315 (2013)CrossRefGoogle Scholar
  64. 64.
    Rączaszek-Leonardi, J., et al.: Linguistic interaction as coordinative structure: Relationship between supraindividual and subjective. submittedGoogle Scholar
  65. 65.
    Gorman, J.C., et al.: Measuring patterns in team interaction sequences using a discrete recurrence approach. Hum. Factors: J Hum. Factors Ergon. Soc. 54(4), 503–517 (2012)CrossRefGoogle Scholar
  66. 66.
    Buder, E.H., et al.: Dynamic indicators of mother-infant prosodic and illocutionary coordination. In: Proceedings of the 5th International Conference on Speech Prosody, (2010)Google Scholar
  67. 67.
    Michael, J., et al.: Compensatory Strategies Enhance Rapport in Interactions Involving People with Möbius Syndrome. submittedGoogle Scholar
  68. 68.
    Orsucci, F., Giuliani, A., Zbilut, J.: Structure & coupling of semiotic sets. Experimental Chaos 742, 83–93 (2004)Google Scholar
  69. 69.
    Orsucci, F., et al.: Prosody and synchronization in cognitive neuroscience. EPJ Nonlinear Biomed. Phys. 1(1), 1–11 (2013)CrossRefGoogle Scholar
  70. 70.
    Orsucci, F., et al.: Orthographic structuring of human speech and texts: linguistic application of recurrence quantification analysis. Arxiv preprint cmp-lg/9712010 (1997)Google Scholar
  71. 71.
    Dale, R., Spivey, M.J.: Unraveling the dyad: using recurrence analysis to explore patterns of syntactic coordination between children and caregivers in conversation. Lang. Learn. 56(3), 391–430 (2006)CrossRefGoogle Scholar
  72. 72.
    Dale, R., Louwerse. M.M.: Human interaction as a multimodal network structure. In: Conceptual Structures, Discourse, and Language (2013)Google Scholar
  73. 73.
    Bahrami, B., et al.: Optimally interacting minds. Science 329, 1081–1085 (2010)CrossRefGoogle Scholar
  74. 74.
    Angus, D., Smith, A., Wiles, J.: Human communication as coupled time series: quantifying multi-participant recurrence. IEEE Trans. Audio Speech Lang. Process. 20, 1795–1807 (2012)CrossRefGoogle Scholar
  75. 75.
    Angus, D., Smith, A., Wiles, J.: Conceptual recurrence plots: revealing patterns in human discourse. IEEE Trans. Visual Comput. Graphics 18(6), 988–997 (2012)CrossRefGoogle Scholar
  76. 76.
    Angus, D., et al.: Visualising conversation structure across time: insights into effective doctor-patient consultations. PloS ONE 7(6), e38014 (2012)CrossRefGoogle Scholar
  77. 77.
    Leonardi, G.: The study of language and conversation with recurrence analysis methods. Psychol. Lang. Commun. 16(2), 165–183 (2012)MathSciNetGoogle Scholar
  78. 78.
    Marsh, K.L., et al.: Autism and social disconnection in interpersonal rocking. Frontiers Integr. Neurosci. 7, 4 (2013)CrossRefGoogle Scholar
  79. 79.
    Lakens, D.: Calculating and reporting effect sizes to facilitate cumulative science: a practical primer for t-tests and anovas. Frontiers Psychol. 4, 863 (2013)CrossRefGoogle Scholar
  80. 80.
    Dienes, Z.: Understanding Psychology as a Science : An Introduction to Scientific and Statistical Inference, p. 170. Palgrave Macmillan, New York (2008)Google Scholar
  81. 81.
    Plonsky, L.: In: Porte, G. (ed.) Replication, Meta-analysis, and Generalizability. Replication Research in Applied Linguistics. Cambridge University Press, Cambridge (2012)Google Scholar
  82. 82.
    Cumming, G.: The new statistics why and how. Psychol. Sci. 25(1), 7–29 (2014)CrossRefGoogle Scholar
  83. 83.
    Dale, R., et al.: Beyond synchrony: complementarity and asynchrony in joint action. In: Cognitive Science (2013)Google Scholar
  84. 84.
    Thomasson, N., Webber, C., Zbilut, J.P.: Application of recurrence quantification analysis to EEG signals. Int. J. Comput. Appl. 9, 9–14 (2002)Google Scholar
  85. 85.
    Marwan, N., Kurths, J., Saparin, P.: Generalised recurrence plot analysis for spatial data. Phys. Lett. A 360(4), 545–551 (2007)CrossRefGoogle Scholar
  86. 86.
    Kokal, I., Gazzola, V., Keysers, C.: Acting together in and beyond the mirror neuron system. Neuroimage 47(4), 2046–2056 (2009)CrossRefGoogle Scholar
  87. 87.
    Newman-Norlund, S.E., et al.: Recipient design in tacit communication. Cognition 111, 46–54 (2009)CrossRefGoogle Scholar
  88. 88.
    Sebanz, N., Bekkering, H., Knoblich, G.: Joint action: bodies and minds moving together. Trends Cogn. Sci. 10(2), 70–76 (2006)CrossRefGoogle Scholar
  89. 89.
    Lang, M., et al.: Lost in the rhythm: the effects of rhythm on subsequent interpersonal co-ordination. underRevisionGoogle Scholar
  90. 90.
    Fusaroli, R., et al.: Non-linear dynamics of speech and voice in schizophrenia. in Neurobiology of Language 2013, San Diego (2013)Google Scholar
  91. 91.
    Fusaroli, R., Bang, D., Weed, E.: Non-linear analyses of speech and prosody in asperger’s syndrome. In: IMFAR 2013, San Sebastian (2013)Google Scholar
  92. 92.
    Fusaroli, R., et al.: Non-linear dynamics of voice in mental disorders. in Cog Sci 2013, Berlin (2013)Google Scholar
  93. 93.
    Weed, E., Fusaroli, R.: Prosodic production in right-hemisphere stroke patients: using temporal dynamics to characterize voice quality. in Neurobiology of Language (2013)Google Scholar
  94. 94.
    Rączaszek-Leonardi, J., Kelso, J.A.S.: Reconciling symbolic and dynamic aspects of language: toward a dynamic psycholinguistics. N. Ideas Psychol. 26(2), 193–207 (2008)CrossRefGoogle Scholar
  95. 95.
    Fusaroli, R., et al.: Conversation, coupling and complexity: matching scaling laws predict performance in a joint decision task. in Cog Sci 2013 (2013)Google Scholar
  96. 96.
    Chen, Y., Yang, H.: Multiscale recurrence analysis of long-term nonlinear and nonstationary time series. Chaos Solitons Fractals 45(7), 978–987 (2012)CrossRefGoogle Scholar
  97. 97.
    Xiang, R., et al.: Multiscale characterization of recurrence-based phase space networks constructed from time series. Chaos 22(1), 013107 (2012)CrossRefGoogle Scholar
  98. 98.
    Lancia, L., Avelino, H. Voigt, D.: Measuring laryngealization in running speech: interaction with contrastive tones in Yalálag Zapotec. In: Interspeech 2013. in pressGoogle Scholar
  99. 99.
    Lancia, L., Fuchs, S., Tiede, M.: Application of concepts from cross-recurrence analysis in speech production: an overview and a comparison to other nonlinear methods. J. Speech Lang. Hearing Res. 57, 743–757 (2013)Google Scholar
  100. 100.
    Lancia, L., Tiede, M.: A survey of methods for the analysis of the temporal evolution of speech articulator trajectories. In: Fuchs, A., et al. (eds.) Speech Planning and Dynamics, Peter Lang (2012)Google Scholar
  101. 101.
    Schinkel, S., et al.: Confidence bounds of recurrence-based complexity measures. Phys. Lett. A 373(26), 2245–2250 (2009)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  • Riccardo Fusaroli
    • 1
    • 2
    Email author
  • Ivana Konvalinka
    • 3
  • Sebastian Wallot
    • 1
  1. 1.Interacting Minds CenterAarhus UniversityAarhusDenmark
  2. 2.Center for SemioticsAarhus UniversityAarhusDenmark
  3. 3.Cognitive Systems, DTU ComputeTechnical University of DenmarkCopenhagenDenmark

Personalised recommendations