Advertisement

International Journal of Social Robotics

, Volume 6, Issue 4, pp 621–632 | Cite as

Recognizing Emotions Conveyed by Human Gait

  • Gentiane VentureEmail author
  • Hideki Kadone
  • Tianxiang Zhang
  • Julie Grèzes
  • Alain Berthoz
  • Halim Hicheur
Article

Abstract

Humans convey emotions through different ways. Gait is one of them. Here we propose to use gait data to highlight features that characterize emotions. Gait analysis study usually focuses on stance phase, frequency, footstep length. Here the study is based on the joint angles obtained from inverse kinematics computation from the 3D motion-capture data using a combination of degrees of freedom (DOF) out of a 34DOF human body model obtained from inverse kinematics of markers 3D position. The candidates are four professional actors, and five emotional states are simulated: Neutral, Joy, Anger, Sadness, and Fear. The paper presents first a psychological approach which results are used to propose numerical approaches. The first study provides psychological results on motion perception and the possibility of emotion recognition from gait by 32 observers. Then, the motion data is studied using a feature vector approach to verify the numerical identifiability of the emotions. Finally each motion is tested against a database of reference motions to identify the conveyed emotion. Using the first and second study results, we utilize a 6DOF model then a 12DOF model. The experimental results show that by using the gait characteristics it is possible to characterize each emotion with good accuracy for intra-subject data-base. For inter-subject database results show that recognition is more prone to error, suggesting strong inter-personal differences in emotional features.

Keywords

Emotion Gait Recognition 

Notes

Acknowledgments

This research is partially supported by a grant of the NeuroCreative Lab (NPO), by the funds of the “Women’s Future Development Organization”, and by the European project “COBOL”. The author thanks Mr. Stephane Dalberra from the Laboratoire de Physiologie de la Perception et de l’Action, Collège de France, Paris, France.

References

  1. 1.
    de Gelder B, Vroomer J (2000) The perception of emotions by ear and eye. Cognit Emot 14(3):289–311CrossRefGoogle Scholar
  2. 2.
    Janssen D, Schollhorn W, Lubienetzki J, Folling K, Kokenge H, Davids K (2008) Recognition of emotions in gait patterns by means of artificial neural nets. J Nonverbal Behav 32:79–92CrossRefGoogle Scholar
  3. 3.
    Troje NF (2002) Decomposing biological motion: a framework for analysis and synthesis of human gait patterns. J Vis 2(5):371–387CrossRefGoogle Scholar
  4. 4.
    Atkinson A, Dittrich W, Gemmell A, Young A (2004) Emotion perception from dynamic and static body expressions in point-light and full-light displays. Perception 33:717–746CrossRefGoogle Scholar
  5. 5.
    Janssen D, Schollhorn W, Lubienetzki J, Folling K, Kokenge H, Davids K (2008) Recognition of emotions in gait patterns by means of artificial neural nets. J Nonverbal Behav 32:79–92CrossRefGoogle Scholar
  6. 6.
    Loula F, Prasad S, Harber K, Shiffrar M (2005) Recognizing people from their movement. J Exp Psychol 31(1):210–220Google Scholar
  7. 7.
    Jokisch D, Daum I, Troje N (2006) Self recognition versus recognition of others by biological motion: viewpoint-depedent effects. Perception 35:911–920CrossRefGoogle Scholar
  8. 8.
    Cutting J, Kozlowski L (1977) Recognizing friends by their walk: gait perception withouit familiarity cues. Bull Psychon Soc 9(5):353–356CrossRefGoogle Scholar
  9. 9.
    Jain A, Bolle J, Pankanti S (2002) Biometrics-personal identification in networked society. Springer, New YorkGoogle Scholar
  10. 10.
    Biometrics and Identity Management, vol. 5372 of Lecture Notes in Computer Science, Springer, Berlin/Heidelberg, (2008)Google Scholar
  11. 11.
    Nixon M (2008) Gait biometrics. Biom Technol Today 16(7–8):8–9CrossRefGoogle Scholar
  12. 12.
    Kleinsmith A, Bianchi-Berthouze N, Steed A (2011) Automatic recognition of non-acted affective postures. IEEE Trans Syst Man Cybern Part B 41(4):1027–1038CrossRefGoogle Scholar
  13. 13.
    Handri S, Nomura S, Nakamura K (2011) Determination of age and gender based on features of human motion using AdaBoost algorithms. Int J Soc Robot 3(3):233–241CrossRefGoogle Scholar
  14. 14.
    Murray P, Drought B, Kory R (1964) Walking patterns of normal men. J Bone Joint Surg 46(2):335–360Google Scholar
  15. 15.
    Boyd J, Little J (2005) Advanced studies in biometrics. Lecture Notes in Computer Science, Springer, Berlin/HeidelbergGoogle Scholar
  16. 16.
    de Gelder B (2006) Towards the neurobiology of emotional body language. Nat Rev Neurosci 7:242–249CrossRefGoogle Scholar
  17. 17.
    Roether C, Omlor L, Christensen A, Giese M (2009) Critical features for the perception of emotion from gait. J Vis 9(6):1–32CrossRefGoogle Scholar
  18. 18.
    Beck A, Canamero L, Hiolle A, Damiano L, Cosi P, Tesser F, Sommavilla G (2013) Interpretation of emotional body language displayed by a humanoid robot: a case study with children. Int J Soc Robot 5(3):325–334CrossRefGoogle Scholar
  19. 19.
    McColl D, Nejat G (2014) Recognizing emotional body language displayed by a human-like social robot. Int J Soc Robot 6(2):261–280CrossRefGoogle Scholar
  20. 20.
    Izard CE (2009) Emotion theory and research: highlights, unanswered questions, and emerging issues. Ann Rev Psychol 60:1–25CrossRefGoogle Scholar
  21. 21.
    Venture G, Ayusawa K, Nakamura Y (2009) Real-time identification and visualization of human segment parameters. In: Proceedings of the IEEE international conference on engineering in medicine and biology, pp 3983–3986Google Scholar
  22. 22.
    Wua G, van der Helmb F, Veegerc H, Makhsouse M, Van Royf P, Angling C, Nagelsh J, Kardunai A, McQuadej K, Wangk X, Wernerl F, Buchholz B (2002) Isb recommendation on definitions of joint coordinate systems of various joints for the reporting of human joint motion.part i: ankle, hip, and spine. J Biomech 35:543–548CrossRefGoogle Scholar
  23. 23.
    Wua G, van der Helmb F, Veegerc H, Makhsouse M, Van Royf P, Angling C, Nagelsh J, Kardunai A, McQuadej K, Wangk X, Wernerl F, Buchholz B (2005) Isb recommendation on definitions of joint coordinate systems of various joints for the reporting of human joint motion.part ii: shoulder, elbow, wrist and hand. J Biomech 38:981–992CrossRefGoogle Scholar
  24. 24.
    Kadone H, Nakamura Y (2006) Segmentation, memorization, recognition and abstraction of humanoid motions based on correlations and associative memory. In: IEEE/RAS proceedings of the international conference on humanoid robots, pp 1–6Google Scholar
  25. 25.
    Kadone H, Nakamura Y (2008) Symbolic memory of motion patterns by an associative memory dynamics with self-organizing nonmonotonicity. Neural Inf Process 4985(2008):203–213CrossRefGoogle Scholar
  26. 26.
    Hicheur H, Kadone H, Grezes J, Berthoz A (2013) The combined role of motion-related cues and upper body posture for the expression of emotions during human walking. In: Mombaur K, Berns K (eds) Modeling, simulation and optimization of bipedal walking, vol 18. Springer, Berlin Heidelberg, pp 71–85CrossRefGoogle Scholar
  27. 27.
    Karg M, Jenke R, Seiberl W, Kuuhnlenz K, Schwirtz A, Buss M (2009) ‘A comparison of pca, kpca and lda for feature extraction to recognize affect in gait kinematics. In: Proceedings of the international conference on affective computing and intelligent interaction and workshops, pp 1–6Google Scholar
  28. 28.
    Roether C, Omlor L, Giese M (2008) Lateral asymmetry of bodily emotion expression. Curr Biol 18(8):R329–R330CrossRefGoogle Scholar
  29. 29.
    Venture G (2010) Human characterization and emotion characterization from gait. In: Proceedings of the IEEE international conference engineering in medicine and biology, pp 1292–1295Google Scholar

Copyright information

© Springer Science+Business Media Dordrecht 2014

Authors and Affiliations

  • Gentiane Venture
    • 1
    Email author
  • Hideki Kadone
    • 2
  • Tianxiang Zhang
    • 1
  • Julie Grèzes
    • 4
  • Alain Berthoz
    • 3
  • Halim Hicheur
    • 5
  1. 1.Tokyo University of Agriculture and TechnologyTokyoJapan
  2. 2.University of TsukubaTsukubaJapan
  3. 3.Collège de FranceParisFrance
  4. 4.Ecole Normale SupérieureParisFrance
  5. 5.Department of MedicineUniversity of FribourgFribourgSwitzerland

Personalised recommendations