Abstract
Human mimicry is one of the important behavioral cues displayed during social interaction that inform us about the interlocutors’ interpersonal states and attitudes. For example, the absence of mimicry is usually associated with negative attitudes. A system capable of analyzing and understanding mimicry behavior could enhance social interaction, both in human-human and human-machine interaction, by informing the interlocutors about each other’s interpersonal attitudes and feelings of affiliation. Hence, our research focus is the investigation of mimicry in social human-human and human-machine interactions with the aim to helpimprove the quality of these interactions. In particular, we aim to develop automatic multimodal mimicry analyzers, to enhance affect recognition and social signal understanding systems through mimicry analysis, and to implement mimicry behavior in Embodied Conversational Agents. This paper surveys and discusses the recent work we have carried out regarding these aims. It is meant to serve as an ultimate goal and a guide for determining recommendations for the development of automatic mimicry analyzers to facilitate affective computing and social signal processing.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Chartrand, T.L., Bargh, J.A.: The chameleon effect: The perception-behavior link and social interaction. Journal of Personality and Social Psychology 76, 893–910 (1999)
Bernieri, F.J.: Coordinated movement and rapport in teacher-student interactions. Journal of Nonverbal Behavior 12, 120–138 (1988)
Chartrand, T.L., Jefferis, V.E.: Consequences of automatic goal pursuit and the case of nonconscious mimicry. In: Forgas, J.P., Williams, K.D., von Hippel, W. (eds.) Responding to the Social World: Implicit and Explicit Processes in Social Judgments and Decisions, pp. 290–305. Psychology Press, Philadelphia (2003)
Dijksterhuis, A.: Automatic social influence: The perception-behavior link as an explanatory mechanism for behavior matching. In: Forgas, J.P., Williams, K.D. (eds.) Social Influence: Direct and Indirect Processes, pp. 95–108. Psychology Press, Philadelphia (2001)
Dijksterhuis, A., Bargh, J.A.: The perception-behavior expressway: Automatic effects of social perception on social behavior. In: Zanna, M. (ed.) Advances in Experimental Social Psychology, pp. 1–40. Academic Press, San Diego (2001)
Gueguen, N., Jacob, C., Martin, A.: Mimicry in social interaction: Its effect on human judgment and behavior. European Journal of Sociences 8 (2009)
Lakin, J.L., Chartrand, T.L.: Using nonconscious behavioral mimicry to create affiliation and rapport. Psychological Science 14, 334–339 (2003)
Lakin, J.L., Jefferis, V.E., Cheng, C.M., Chartrand, T.L.: The Chameleon Effect as social glue: Evidence for the evolutionary significance of nonconscious mimicry. Journal of Nonverbal Behavior 27, 145–162 (2003)
Huang, H.H., Cerekovic, A., Tarasenko, K., Levacic, V., Zoric, G., Pandzic, I.S., Nakano, Y., Nishida, Y.: Integration embodied conversational agent components with generic framework. Multiagent and Grid Systems-An Interational Journal 4, 371–386 (2008)
Kopp, S.: Social resonance and embodied coordination in face-to- face conversation with artificial interlocutors. Speech Communication 52(6), 587–597 (2010)
Pantic, M., Nijholt, A., Plentland, A., Huang, T.S.: Human-Centred Intelligent Human-Computer Interaction (HCI2)): how far are we from attaining it? Journal of Autonomous and Adaptive Communications Systems 1(2), 168–187 (2008)
Sun, X.F., Lichtenauer, J., Valstar, M., Nijholt, A., Pantic, M.: A Multimodal Database for Mimicry Analysis. In: D´Mello, S., et al. (eds.) ACII 2011, Part I, vol. 6974, pp. 367–376. Springer, Heidelberg (2011)
Sun, X.F., Truong, K., Nijholt, A., Pantic, M.: Automatic Visual Mimicry Expression Analysis in Interpersonal Interaction. In: Fourth IEEE Workshop on CVPR for Human Communicative Behavior Analysis, Held in Conjunction with CVPR 2011, Colorado Springs, USA (to appear, 2011)
Sun, X.F., Nijholt, A., Pantic, M.: Towards Mimicry Recognition during Human Interactions: Automatic Feature Selection and Representation. In: Proceedings 4th International ICST Conference on Intelligent Technologies for Interactive Entertainment (INTETAIN 2011), Genoa, Italy, May 25-27 ( to appear, 2011)
Sun, X.F., Truong, K.P., Pantic, M., Nijholt, A.: Towards Visual and Vocal Mimicry Recognition in Human-Human Interactions. In: Proceedings 2011Â IEEE International Conference on Systems, Man, and Cybernetics (IEEE SMC 2011), Special Session on Social Signal Processing, Anchorage, Alaska, October 9-12 (to appear, 2011)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2011 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Sun, X., Nijholt, A., Truong, K.P., Pantic, M. (2011). Automatic Understanding of Affective and Social Signals by Multimodal Mimicry Recognition. In: D’Mello, S., Graesser, A., Schuller, B., Martin, JC. (eds) Affective Computing and Intelligent Interaction. ACII 2011. Lecture Notes in Computer Science, vol 6975. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-24571-8_32
Download citation
DOI: https://doi.org/10.1007/978-3-642-24571-8_32
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-24570-1
Online ISBN: 978-3-642-24571-8
eBook Packages: Computer ScienceComputer Science (R0)