Advertisement

Journal of Nonverbal Behavior

, Volume 41, Issue 2, pp 85–102 | Cite as

Quantifying Facial Expression Synchrony in Face-To-Face Dyadic Interactions: Temporal Dynamics of Simultaneously Recorded Facial EMG Signals

  • Marcel RiehleEmail author
  • Jürgen Kempkensteffen
  • Tania M. Lincoln
Original Paper

Abstract

Human social interaction is enriched with synchronous movement which is said to be essential to establish interactional flow. One commonly investigated phenomenon in this regard is facial mimicry, the tendency of humans to mirror facial expressions. Because studies investigating facial mimicry in face-to-face interactions are lacking, the temporal dynamics of facial mimicry remain unclear. We therefore developed and tested the suitability of a novel approach to quantifying facial expression synchrony in face-to-face interactions: windowed cross-lagged correlation analysis (WCLC) for electromyography signals. We recorded muscle activations related to smiling (Zygomaticus Major) and frowning (Corrugator Supercilii) of two interaction partners simultaneously in 30 dyadic affiliative interactions. We expected WCLC to reliably detect facial expression synchrony above chance level and, based on previous research, expected the occurrence of rapid synchronization of smiles within 200 ms. WCLC significantly detected synchrony of smiling but not frowning compared to a control condition of chance level synchrony in six different interactional phases (smiling: d z s = .85–1.11; frowning: d z s = .01–.30). Synchronizations of smiles between interaction partners predominantly occurred within 1000 ms, with a significant amount occurring within 200 ms. This rapid synchronization of smiles supports the notion of the existence of an anticipated mimicry response for smiles. We conclude that WCLC is suited to quantify the temporal dynamics of facial expression synchrony in dyadic interactions and discuss implications for different psychological research areas.

Keywords

Social interaction Assessment Time series analysis Nonverbal synchrony Ecological validity Facial mimicry 

Notes

Acknowledgements

We wish to thank all participants for their participation and Rukiye Köysürenbars, Mathias Osterried, Beatrice Salewski, Olga Schulz, and Katrin Zilliken, who helped with the data collection.

Compliance with Ethical Standards

Conflict of interest

The authors declare that they have no conflict of interest.

Ethical approval

All procedures were approved by a local ethics committee (Psychotherapeutenkammer Hamburg) and performed in accordance with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.

Informed consent

Informed consent was obtained from all individual participants included in the study.

Supplementary material

10919_2016_246_MOESM1_ESM.docx (774 kb)
Supplementary material 1 (DOCX 773 kb)

References

  1. Bailey, P. E., & Henry, J. D. (2009). Subconscious facial expression mimicry is preserved in older adulthood. Psychology and Aging, 24(4), 995–1000. doi: 10.1037/a0015789.CrossRefPubMedGoogle Scholar
  2. Bernieri, F. J. (1988). Coordinated movement and rapport in teacher–student interactions. Journal of Nonverbal Behavior, 12(2), 120–138. doi: 10.1007/BF00986930.CrossRefGoogle Scholar
  3. Boker, S. M., Cohn, J. F., Theobald, B.-J., Matthews, I., Brick, T. R., & Spies, J. R. (2009). Effects of damping head movement and facial expression in dyadic conversation using real-time facial expression tracking and synthesized avatars. Philosophical Transactions of the Royal Society of London. Series B, Biological sciences, 364(1535), 3485–3495. doi: 10.1098/rstb.2009.0152.CrossRefPubMedPubMedCentralGoogle Scholar
  4. Boker, S. M., Rotondo, J. L., Xu, M., & King, K. (2002). Windowed cross-correlation and peak picking for the analysis of variability in the association between behavioral time series. Psychological Methods, 7(3), 338–355. doi: 10.1037/1082-989X.7.3.338.CrossRefPubMedGoogle Scholar
  5. Chartrand, T. L., & Bargh, J. A. (1999). The chameleon effect: The perception-behavior link and social interaction. Journal of Personality and Social Psychology, 76(6), 893–910. doi: 10.1037/0022-3514.76.6.893.CrossRefPubMedGoogle Scholar
  6. Chartrand, T. L., & Lakin, J. L. (2013). The antecedents and consequences of human behavioral mimicry. Annual Review of Psychology, 64(1), 285–308. doi: 10.1146/annurev-psych-113011-143754.CrossRefPubMedGoogle Scholar
  7. Cohen, A. S., Morrison, S. C., & Callaway, D. A. (2013). Computerized facial analysis for understanding constricted/blunted affect: Initial feasibility, reliability, and validity data. Schizophrenia Research, 148(1–3), 111–116. doi: 10.1016/j.schres.2013.05.003.CrossRefPubMedPubMedCentralGoogle Scholar
  8. Dimberg, U. (1982). Facial reactions to facial expressions. Psychophysiology, 19(6), 643–647. doi: 10.1111/j.1469-8986.1982.tb02516.x.CrossRefPubMedGoogle Scholar
  9. Dimberg, U., Thunberg, M., & Elmehed, K. (2000). Unconscious facial reactions to emotional facial expressions. Psychological Science, 11(1), 86–89. doi: 10.1111/1467-9280.00221.CrossRefPubMedGoogle Scholar
  10. Epstein, L. H. (1990). Perception of activity in the Zygomaticus Major and Corrugator Supercilii muscle regions. Psychophysiology, 27(1), 68–72. doi: 10.1111/j.1469-8986.1990.tb02181.x.CrossRefPubMedGoogle Scholar
  11. Fischer, A. H., & van Kleef, G. A. (2010). Where have all the people gone? A plea for including social interaction in emotion research. Emotion Review, 2(3), 208–211. doi: 10.1177/1754073910361980.CrossRefGoogle Scholar
  12. Fridlund, A. J., & Cacioppo, J. T. (1986). Guidelines for human electromyographic research. Psychophysiology, 23(5), 567–589. doi: 10.1111/j.1469-8986.1986.tb00676.x.CrossRefPubMedGoogle Scholar
  13. Heerey, E. A. (2015). Decoding the dyad: Challenges in the study of individual differences in social behavior. Current Directions in Psychological Science, 24(4), 285–291. doi: 10.1177/0963721415570731.CrossRefGoogle Scholar
  14. Heerey, E. A., & Crossley, H. M. (2013). Predictive and reactive mechanisms in smile reciprocity. Psychological Science, 24(8), 1446–1455. doi: 10.1177/0956797612472203.CrossRefPubMedGoogle Scholar
  15. Hess, U., & Blairy, S. (2001). Facial mimicry and emotional contagion to dynamic emotional facial expressions and their influence on decoding accuracy. International Journal of Psychophysiology, 40(2), 129–141. doi: 10.1016/S0167-8760(00)00161-6.CrossRefPubMedGoogle Scholar
  16. Hess, U., Blairy, S., & Kleck, R. E. (2000). The influence of facial emotion displays, gender, and ethnicity on judgments of dominance and affiliation. Journal of Nonverbal Behavior, 24(4), 265–283. doi: 10.1023/A:1006623213355.CrossRefGoogle Scholar
  17. Hess, U., & Bourgeois, P. (2010). You smile—I smile: emotion expression in social interaction. Biological Psychology, 84(3), 514–520. doi: 10.1016/j.biopsycho.2009.11.001.CrossRefPubMedGoogle Scholar
  18. Hess, U., & Fischer, A. H. (2013). Emotional mimicry as social regulation. Personality and Social Psychology Review, 17(2), 142–157. doi: 10.1177/1088868312472607.CrossRefPubMedGoogle Scholar
  19. Hühnel, I., Fölster, M., Werheid, K., & Hess, U. (2014). Empathic reactions of younger and older adults: No age related decline in affective responding. Journal of Experimental Social Psychology, 50(1), 136–143. doi: 10.1016/j.jesp.2013.09.011.CrossRefGoogle Scholar
  20. Kupper, Z., Ramseyer, F., Hoffmann, H., & Tschacher, W. (2015). Nonverbal synchrony in social interactions of patients with schizophrenia indicates socio-communicative deficits. PLoS ONE, 10(12), e0145882. doi: 10.1371/journal.pone.0145882.CrossRefPubMedPubMedCentralGoogle Scholar
  21. Lakin, J. L., & Chartrand, T. L. (2003). Using nonconscious behavioral mimicry to create affiliation and rapport. Psychological Science, 14(4), 334–339. doi: 10.1111/1467-9280.14481.CrossRefPubMedGoogle Scholar
  22. Lanzetta, J. T., & Englis, B. G. (1989). Expectations of cooperation and competition and their effects on observers’ vicarious emotional responses. Journal of Personality and Social Psychology, 56(4), 543–554. doi: 10.1037/0022-3514.56.4.543.CrossRefGoogle Scholar
  23. Laroche, J., Berardi, A. M., & Brangier, E. (2014). Embodiment of intersubjective time: Relational dynamics as attractors in the temporal coordination of interpersonal behaviors and experiences. Frontiers in Psychology, 5(October), 1–17. doi: 10.3389/fpsyg.2014.01180.Google Scholar
  24. Larsen, J. T., Norris, C. J., & Cacioppo, J. T. (2003). Effects of positive and negative affect on electromyographic activity over Zygomaticus Major and Corrugator Supercilii. Psychophysiology, 40, 776–785.CrossRefPubMedGoogle Scholar
  25. Lehrl, S. (1999). Mehrfachwahl-Wortschatz-Intelligenztest: MWT-B. Balingen: Spitta.Google Scholar
  26. Likowski, K. U., Mühlberger, A., Seibt, B., Pauli, P., & Weyers, P. (2008). Modulation of facial mimicry by attitudes. Journal of Experimental Social Psychology, 44(4), 1065–1072. doi: 10.1016/j.jesp.2007.10.007.CrossRefGoogle Scholar
  27. McIntosh, D. N., Reichmann-Decker, A., Winkielman, P., & Wilbarger, J. L. (2006). When the social mirror breaks: Deficits in automatic, but not voluntary, mimicry of emotional facial expressions in autism. Developmental Science, 9(3), 295–302. doi: 10.1111/j.1467-7687.2006.00492.x.CrossRefPubMedGoogle Scholar
  28. Messinger, D. S., Mahoor, M. H., Chow, S.-M., & Cohn, J. F. (2009). Automated measurement of facial expression in infant–mother interaction: A pilot study. Infancy, 14(3), 285–305. doi: 10.1080/15250000902839963.CrossRefPubMedPubMedCentralGoogle Scholar
  29. Ramseyer, F., & Tschacher, W. (2011). Nonverbal synchrony in psychotherapy: Coordinated body movement reflects relationship quality and outcome. Journal of Consulting and Clinical Psychology, 79(3), 284–295. doi: 10.1037/a0023419.CrossRefPubMedGoogle Scholar
  30. Stel, M., & Vonk, R. (2010). Mimicry in social interaction: Benefits for mimickers, mimickees, and their interaction. British Journal of Psychology, 101(Pt 2), 311–323. doi: 10.1348/000712609X465424.CrossRefPubMedGoogle Scholar
  31. Tassinary, L. G., Cacioppo, J. T., & Vanman, E. J. (2007). The skeletomotor system: Surface electromyography. In J. T. Cacioppo, L. G. Tassinary, & G. Bernston (Eds.), Handbook of psychophysiology (3rd ed., pp. 267–299). Cambridge: Cambridge University Press.CrossRefGoogle Scholar
  32. Tickle-Degnen, L., & Rosenthal, R. (1990). The nature of rapport and its nonverbal correlates. Psychological Inquiry, 1(4), 285–293. doi: 10.1207/s15327965pli0104_1.CrossRefGoogle Scholar
  33. Tschacher, W., Rees, G. M., & Ramseyer, F. (2014). Nonverbal synchrony and affect in dyadic interactions. Frontiers in Psychology, 5, 1323. doi: 10.3389/fpsyg.2014.01323.CrossRefPubMedPubMedCentralGoogle Scholar
  34. van der Schalk, J., Fischer, A. H., Doosje, B., Wigboldus, D., Hawk, S. T., Rotteveel, M., et al. (2011). Convergent and divergent responses to emotional displays of ingroup and outgroup. Emotion, 11(2), 286–298. doi: 10.1037/a0022582.CrossRefPubMedGoogle Scholar
  35. Varcin, K. J., Bailey, P. E., & Henry, J. D. (2010). Empathic deficits in schizophrenia: The potential role of rapid facial mimicry. Journal of the International Neuropsychological Society, 16(4), 621–629. doi: 10.1017/S1355617710000329.CrossRefPubMedGoogle Scholar
  36. Varlet, M., Marin, L., Raffard, S., Schmidt, R. C., Capdevielle, D., Boulenger, J.-P., et al. (2012). Impairments of social motor coordination in schizophrenia. PLoS ONE, 7(1), e29772. doi: 10.1371/journal.pone.0029772.CrossRefPubMedPubMedCentralGoogle Scholar
  37. Wexler, B. E., Levenson, L., Warrenburg, S., & Price, L. H. (1994). Decreased perceptual sensitivity to emotion-evoking stimuli in depression. Psychiatry Research, 51(2), 127–138. doi: 10.1016/0165-1781(94)90032-9.CrossRefPubMedGoogle Scholar
  38. Wittchen, H.-U., Fydrich, T., & Zaudig, M. (1997). SKID: Strukturiertes Klinisches Interview für DSM-IV; Achse I und II. Achse I: psychische Störungen. SKID-I. Göttingen: Hogrefe.Google Scholar
  39. Zhang, Z. (2012). Microsoft kinect sensor and its effect. IEEE Multimedia, 19(2), 4–10. doi: 10.1109/MMUL.2012.24.CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2017

Authors and Affiliations

  1. 1.Department of PsychologyUniversity of HamburgHamburgGermany

Personalised recommendations