Recognising Human Emotions from Body Movement and Gesture Dynamics

  • Ginevra Castellano
  • Santiago D. Villalba
  • Antonio Camurri
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4738)


We present an approach for the recognition of acted emotional states based on the analysis of body movement and gesture expressivity. According to research showing that distinct emotions are often associated with different qualities of body movement, we use non- propositional movement qualities (e.g. amplitude, speed and fluidity of movement) to infer emotions, rather than trying to recognise different gesture shapes expressing specific emotions. We propose a method for the analysis of emotional behaviour based on both direct classification of time series and a model that provides indicators describing the dynamics of expressive motion cues. Finally we show and interpret the recognition rates for both proposals using different classification algorithms.


Body Movement Emotion Recognition Dynamic Time Warping Human Emotion Correlation Base Feature Selection 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Picard, R.W.: Affective Computing. The MIT Press, Cambridge (1997)Google Scholar
  2. 2.
    Scherer, K.R.: On the nature and function of emotion: a component process approach. In: Scherer, K.R., Ekman, P. (eds.) Approaches to emotion, pp. 293–317. Hillsdale, NJ: Erlbaum (1984)Google Scholar
  3. 3.
    Pollick, F., Paterson, H., Bruderlin, A., Sanford, A.: Perceiving affect from arm movement. Cognition 82, B51–B61 (2001)CrossRefGoogle Scholar
  4. 4.
    Shiffrar, M., Pinto, J.: The visual analysis of bodily motion. In: Prinz, W., Hommel, B. (eds.) Common mechanisms in perception and action: Attention and Performance, pp. 381–399. Oxford University Press, Oxford (2002)Google Scholar
  5. 5.
    Giese, M.A., Poggio, T.: Neural mechanisms for the recognition of biological movements. Nature Reviews Neuroscience 4(3), 179–192 (2003)CrossRefGoogle Scholar
  6. 6.
    Rizzolatti, G., Fogassi, L., Gallese, V.: Mirrors in the mind. Scientific American 295(5), 54–61 (2006)CrossRefGoogle Scholar
  7. 7.
    Boone, R.T., Cunningham, J.G.: Children’s decoding of emotion in expressive body movement: the development of cue attunement. Developmental psychology 34(5), 1007–1016 (1998)CrossRefGoogle Scholar
  8. 8.
    De Meijer, M.: The contribution of general features of body movement to the attribution of emotions. Journal of Nonverbal Behavior 13(4), 247–268 (1989)CrossRefGoogle Scholar
  9. 9.
    Wallbott, H.G.: Bodily expression of emotion. European Journal of Social Psychology 28(6), 879–896 (1998)CrossRefGoogle Scholar
  10. 10.
    Burgoon, J.K., Jensen, M.L., Meservy, T.O., Kruse, J., Nunamaker, J.F.: Augmenting human identification of emotional states in video. In: Intelligence Analysis Conference, McClean, VA (2005)Google Scholar
  11. 11.
    Camurri, A., Lagerlof, I., Volpe, G.: Recognizing emotion from dance movement: comparison of spectator recognition and automated techniques. International Journal of Human-Computer Studies 59(1-2), 213–225 (2003)CrossRefGoogle Scholar
  12. 12.
    Kapur, A., Kapur, A., Babul, N.V., Tzanetakis, G., Driessen, P.F.: Gesture-based affective computing on motion capture data. In: ACII, pp. 1–7 (2005)Google Scholar
  13. 13.
    Bianchi-Berthouze, N., Kleinsmith, A.: A categorical approach to affective gesture recognition. Connection Science 15(4), 259–269 (2003)CrossRefGoogle Scholar
  14. 14.
    Balomenos, T., Raouzaiou, A., Ioannou, S., Drosopoulos, A.I., Karpouzis, K., Kollias, S.D.: Emotion analysis in man-machine interaction systems. In: Machine Learning for Multimodal Interaction, pp. 318–328 (2004)Google Scholar
  15. 15.
    Gunes, H., Piccardi, M.: Bi-modal emotion recognition from expressive face and body gestures. Journal of Network and Computer Applications In Press, Corrected ProofGoogle Scholar
  16. 16.
    el Kaliouby, R., Robinson, P.: Generalization of a vision-based computational model of mind-reading. In: ACII, pp. 582–589 (2005)Google Scholar
  17. 17.
    Camurri, A., Coletta, P., Massari, A., Mazzarino, B., Peri, M., Ricchetti, M., Ricci, A., Volpe, G.: Toward real-time multimodal processing: Eyesweb 4.0. In: AISB 2004 Convention: Motion, Emotion and Cognition (March 2004)Google Scholar
  18. 18.
    Camurri, A., Mazzarino, B., Volpe, G.: Analysis of expressive gesture: The Eyesweb Expressive Gesture processing library. In: Camurri, A., Volpe, G. (eds.) GW 2003. LNCS (LNAI), vol. 2915, pp. 460–467. Springer, Heidelberg (2004)Google Scholar
  19. 19.
    Kadous, M.W.: Temporal Classification: Extending the Classification Paradigm to Multivariate Time Series. PhD thesis, School of Computer Science & Engineering, University of New South Wales (2002)Google Scholar
  20. 20.
    Witten, I.H., Frank, E.: Data Mining: Practical Machine Learning Tools and Techniques, 2nd edn. Morgan Kaufmann Series in Data Management Systems. Morgan Kaufmann, San Francisco (2005)zbMATHGoogle Scholar
  21. 21.
    Keogh, E., Ratanamahatana, C.A.: Exact indexing of dynamic time warping. Knowledge and Information Systems 7(3), 358–386 (2005)CrossRefGoogle Scholar
  22. 22.
    Heloir, A., Courty, N., Gibet, S., Multon, F.: Temporal alignment of communicative gesture sequences. Computer Animation and Virtual Worlds 17(3-4), 347–357 (2006)CrossRefGoogle Scholar
  23. 23.
    Rodríguez, J.J., Alonso, C.J., Maestro, J.A.: Support vector machines of interval-based features for time series classification. Knowledge-Based Systems 18(4-5), 171–178 (2005)CrossRefGoogle Scholar
  24. 24.
    Sebe, N., Cohen, I., Cozman, F.G., Gevers, T., Huang, T.S.: Learning probabilistic classifiers for human-computer interaction applications. Multimedia Systems V10(6), 484–498 (2005)CrossRefGoogle Scholar
  25. 25.
    Zhang, H., Jiang, L., Su, J.: Hidden naive bayes. In: AAAI 2005, The Twentieth National Conference on Artificial Intelligence and the Seventeenth Innovative Applications of Artificial Intelligence, pp. 919–924 (2005)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2007

Authors and Affiliations

  • Ginevra Castellano
    • 1
  • Santiago D. Villalba
    • 2
  • Antonio Camurri
    • 1
  1. 1.Infomus Lab, DIST, University of Genoa 
  2. 2.MLG, School of Computer Science and Informatics, University College Dublin 

Personalised recommendations