Towards Analysis of Expressive Gesture in Groups of Users: Computational Models of Expressive Social Interaction

  • Antonio Camurri
  • Giovanna Varni
  • Gualtiero Volpe
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5934)


In this paper we present a survey of our research on analysis of expressive gesture and how it is evolving towards the analysis of expressive social interaction in groups of users. Social interaction and its expressive implications (e.g., emotional contagion, empathy) is an extremely relevant component for analysis of expressive gesture, since it provides significant information on the context expressive gestures are performed in. However, most of the current systems analyze expressive gestures according to basic emotion categories or simple dimensional approaches. Moreover, almost all of them are intended for a single user, whereas social interaction is often neglected. After briefly recalling our pioneering studies on collaborative robot-human interaction, this paper presents two steps in the direction of novel computational models and techniques for measuring social interaction: (i) the interactive installation Mappe per Affetti Erranti for active listening to sound and music content, and (ii) the techniques we developed for explicitly measuring synchronization within a group of users. We conclude with the research challenges we will face in the near future.


expressive gesture analysis and processing analysis of social interaction in small groups multimodal interactive systems 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Bernhardt, D., Robinson, P.: Detecting Affect from Non-stylised Body Motions. In: Paiva, A.C.R., Prada, R., Picard, R.W. (eds.) ACII 2007. LNCS, vol. 4738, pp. 59–70. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  2. Boccaletti, S., Kurths, J., Osipov, G., Valladeres, D.L., Zhou, C.S.: The Synchronization of Cahotic Systems. Phys. Rep. 366(1-2), 1–101 (2002)zbMATHCrossRefMathSciNetGoogle Scholar
  3. Boone, R.T., Cunningham, J.G.: Children’s decoding of emotion in expressive body movement: The development of cue attunement. Developmental Psychology 34, 1007–1016 (1998)CrossRefGoogle Scholar
  4. Camurri, A., Coglio, A.: An Architecture for Emotional Agents. In: IEEE Multimedia, October-December 1998, pp. 24–33. IEEE CS Press, Los Alamitos (1998)Google Scholar
  5. Camurri, A., Ferrentino, P.: Interactive environments for music and multimedia. Multimedia Systems 7, 32–47 (1999)CrossRefGoogle Scholar
  6. Camurri, A., Coletta, P., Ricchetti, M., Volpe, G.: Expressiveness and Physicality in Interaction. Journal of New Music Research 29(3), 187–198 (2000)CrossRefGoogle Scholar
  7. Camurri, A., Lagerlöf, I., Volpe, G.: Recognizing emotion from dance movement: Comparison of spectator recognition and automated techniques. Intl. J. Human-Computer Studies 59, 213–225 (2003)CrossRefGoogle Scholar
  8. Camurri, A., De Poli, G., Leman, M., Volpe, G.: Toward Communicating Expressiveness and Affect in Multimodal Interactive Systems for Performing Art and Cultural Applications. IEEE Multimedia 12(1), 43–53 (2005)CrossRefGoogle Scholar
  9. Camurri, A., Canepa, C., Coletta, P., Ferrari, N., Mazzarino, B., Volpe, G.: The Interactive Piece The Bow is bent and drawn. In: Proc. 3rd ACM Intl. Conf. on Digital Interactive Media in Entertainment and Arts (DIMEA 2008), Athens, Greece, pp. 376–383 (2008)Google Scholar
  10. De Meijer, M.: The contribution of general features of body movement to the attribution of emotions. J. of Nonverbal Behavior 13, 247–268 (1989)CrossRefGoogle Scholar
  11. De Rosis, F., Cavalluzzi, A., Mazzotta, I., Novielli, N.: Can embodied conversational agents induce empathy in users? In: Proceedings of AISB 2005, Virtual Social Character Symposium (2005)Google Scholar
  12. Dong, W., Lepri, B., Cappelletti, A., Pentland, A., Pianesi, F., Zancanaro, M.: Using the influence model to recognize functional roles in meetings. In: Proc. 9th Intl. ACM Conf. on Multimodal Interfaces, pp. 271–278 (2007)Google Scholar
  13. Eckmann, J.P., Kamphorst, S.O., Ruelle, D.: Recurrence plots of dynamical system. Eurpph. Lett. 5, 973–977 (1987)CrossRefGoogle Scholar
  14. Juslin, P.N.: Cue utilization in communication of emotion in music performance: relating performance to perception. Journal of Experimental Psychology: Human Perception and Performance 26(6), 1797–1813 (2000)CrossRefGoogle Scholar
  15. Kapur, A., Kapur, A., Virji-Babul, N., Tzanetakis, G., Driessen, P.F.: Gesture-Based Affective Computing on Motion Capture Data. In: Tao, J., Tan, T., Picard, R.W. (eds.) ACII 2005. LNCS, vol. 3784, pp. 1–7. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  16. Laban, R., Lawrence, F.C.: Effort. Macdonald&Evans Ltd., London (1947)Google Scholar
  17. Marwan, N., Romano, M.C., Thiel, M., Kurths, J.: Recurrence plots for the analysis of complex systems. Phsysics Reports 438, 237–329 (2007)CrossRefMathSciNetGoogle Scholar
  18. McQuiggan, S.W., Lester, J.C.: Modeling and evaluating empathy in embodied companion agents. Intl. J. Human-Computer Studies 65(4), 348–360 (2007)CrossRefGoogle Scholar
  19. Pentland, A.: Social signal processing. IEEE Signal Processing Magazine 24(4), 108–111 (2007)CrossRefGoogle Scholar
  20. Poincaré, H.: Sur la problèmedes trois corps et les equations de la dynamique. Acta Math 13, 1–271 (1890)Google Scholar
  21. Russell, J.A.: A circumplex model of affect. J. of Personality and Social Psychology 39, 1161–1178 (1980)CrossRefGoogle Scholar
  22. Suzuki, K., Camurri, A., Hashimoto, S., Ferrentino, P.: Intelligent Agent System for Human-Robot Interaction through Artificial Emotion. In: Proc. IEEE Intl. Conf. on Systems Man and Cybernetics SMC 1998, San Diego, USA (1998)Google Scholar
  23. Tellegen, A., Watson, D., Clark, L.A.: On the dimensional and hierarchical structure of affect. Psychological Science 10(4), 297–303 (1999)CrossRefGoogle Scholar
  24. Varni, G., Camurri, A., Coletta, P., Volpe, G.: Emotional entrainment in music performance. In: Proc. of 8th Intl. Conf. on Automatic Face and Gesture Recognition FG 2008 (2008)Google Scholar
  25. Varni, G., Camurri, A., Coletta, P., Volpe, G.: Toward Real-time Automated Measure of Empathy and Dominance. In: Proc. 2009 IEEE Intl. Conf. on Social Computing SocialCom, Vancouver, Canada (2009a)Google Scholar
  26. Varni, G., Mancini, M., Volpe, G., Camurri, A.: Sync’n’Move: social interaction based on music and gesture. In: Proc. 1st Intl. ICST Conference on User Centric Media, Venice, Italy, December 2009 (2009b)Google Scholar
  27. Vines, B.W., Krumhansl, C.L., Wanderley, M.M., Ioana, M.D., Levitin, D.J.: Dimensions of Emotion in Expressive Musical Performance. Ann. N.Y. Acad. Sci. 1060, 462–466 (2005)CrossRefGoogle Scholar
  28. Wallbott, H.G.: Bodily expression of emotion. Eur. J. Soc. Psychol. 28, 879–896 (1998)CrossRefGoogle Scholar
  29. Zbilut, J., Webber Jr., C.L.: Embeddings and delays as derived from quantification of recurrence plots. Phys. Lett. A 5, 199–203 (1992)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Antonio Camurri
    • 1
  • Giovanna Varni
    • 1
  • Gualtiero Volpe
    • 1
  1. 1.Casa Paganini – InfoMus LabDIST – University of GenovaGenovaItaly

Personalised recommendations