Advertisement

The Visual Computer

, Volume 32, Issue 1, pp 83–98 | Cite as

Laban descriptors for gesture recognition and emotional analysis

  • Arthur Truong
  • Hugo Boujut
  • Titus Zaharia
Original Article

Abstract

In this paper, we introduce a new set of 3D gesture descriptors based on the laban movement analysis model. The proposed descriptors are used in a machine learning framework (with SVM and different random forest techniques) for both gesture recognition and emotional analysis purposes. In a first experiment, we test our expressivity model for action recognition purposes on the Microsoft Research Cambridge-12 dataset and obtain very high recognition rates (more than 97 %). In a second experiment, we test our descriptors’ ability to qualify the emotional content, upon a database of pre-segmented orchestra conductors’ gestures recorded in rehearsals. The results obtained show the relevance of our model which outperforms results reported in similar works on emotion recognition.

Keywords

Gesture expressivity model Laban movement analysis  Motion features Gesture recognition Expressivity analysis Machine learning 

References

  1. 1.
    McNeill, D.: Language and gesture. Cambridge University Press, Cambridge (2000)CrossRefGoogle Scholar
  2. 2.
    Wang, Y., Huang, K., Tan, T.: Human activity recognition based on r transform. In: IEEE Conference on Computer Vision and Pattern Recognition. CVPR’07, IEEE, pp. 1–8 (2007)Google Scholar
  3. 3.
    Huang, F., Xu, G.: Viewpoint insensitive action recognition using envelop shape. In: Computer Vision-ACCV 2007. Springer, Berlin Heidelberg, pp. 477–486 (2007)Google Scholar
  4. 4.
    Song, Y., Demirdjian, D., Davis, R.: Tracking body and hands for gesture recognition: Natops aircraft handling signals database. In: 2011 IEEE International Conference on Automatic Face & Gesture Recognition and Workshops (FG 2011). IEEE, pp. 500–506 (2011)Google Scholar
  5. 5.
    Balomenos, T., et al.: Emotion analysis in man-machine interaction systems. Machine learning for multimodal interaction, pp. 318–328 (2005)Google Scholar
  6. 6.
    Camurri, A., Mazzarino, B., Ricchetti, M., Timmers, R., Volpe, G.: Multimodal analysis of expressive gesture in music and dance performances. In: Gesture-based communication in human-computer interaction, pp. 20–39 (2004)Google Scholar
  7. 7.
    Gunes, H., Piccardi, M.: Bi-modal emotion recognition from expressive face and body gestures. J. Netw. Comput. Appl. 30(4), 1334–1345 (2007)CrossRefGoogle Scholar
  8. 8.
    Nicolaou, M.A., Gunes, H., Pantic, M.: Continuous prediction of spontaneous affect from multiple cues and modalities in valence-arousal space. IEEE Trans. Affect. Comput. 2(2), 92–105 (2011)CrossRefGoogle Scholar
  9. 9.
    Swaminathan, D., et al.: A dynamic bayesian approach to computational laban shape quality analysis. Adv. Hum. Comput. Interact. pp. 1–17 (2009)Google Scholar
  10. 10.
    Zhao, L., Badler, N.I.: Acquiring and validating motion qualities from live limb gestures. Graph. Models 67(1), 1–16 (2005)CrossRefGoogle Scholar
  11. 11.
    Samadani, AA., Burton, S., Gorbet, R., Kulic, D.: Laban effort and shape analysis of affective hand and arm movements. In: 2013 Humaine Association Conference on Affective Computing and Intelligent Interaction (ACII), IEEE, pp. 343–348 (2013)Google Scholar
  12. 12.
    Braem, P.B., Bräm, T.: A pilot study of the expressive gestures used by classical orchestra conductors. J. Conduct. Guild 22(1–2), 14–29 (2001)Google Scholar
  13. 13.
    Glowinski, Donald, et al.: Toward a minimal representation of affective gestures. IEEE Trans. Affect. Comput. 2(2), 106–118 (2011)CrossRefMathSciNetGoogle Scholar
  14. 14.
    Boutet, D.: Une morphologie de la gestualité: structuration articulaire. Cahiers de linguistique analogique, vol. 5, pp. 81–115 (2008)Google Scholar
  15. 15.
    Luo, P., Neff, M.: A perceptual study of the relationship between posture and gesture for virtual characters. In: Motion in Games. Springer, Berlin, pp. 254–265 (2012)Google Scholar
  16. 16.
    Laban, R.: La Maîtrise du Mouvement. Actes Sud, Arles (1994)Google Scholar
  17. 17.
    Junejo, I.N., Junejo, K.N., Al Aghbari, Z.: Silhouette-based human action recognition using SAX-Shapes. Vis. Comput. 30(3), 259–269 (2014)CrossRefGoogle Scholar
  18. 18.
    Chen, H.S., Chen, H.T., Chen, Y.W., Lee, S.Y.: Human action recognition using star skeleton. In: Proceedings of the 4th ACM international workshop on Video surveillance and sensor networks. ACM, pp. 171–178 (2006)Google Scholar
  19. 19.
    Cimen, G., Ilhan, H., Capin, T., Gurcay, H.: Classification of human motion based on affective state descriptors. Comput. Animat. Virtual Worlds 24(3–4), 355–363 (2013)CrossRefGoogle Scholar
  20. 20.
    Etemad, S.A., Arya, A.: Correlation-optimized time warping for motion. Vis. Comput. (2014)Google Scholar
  21. 21.
    Singh, V.K., Nevatia, R.: Simultaneous tracking and action recognition for single actor human actions. Vis. Comput. 27(12), 1115–1123 (2011)CrossRefGoogle Scholar
  22. 22.
    Kellokumpu, V., Zhao, G., Pietikäinen, M.: Human activity recognition using a dynamic texture based method. BMVC, pp. 1–10 (2008)Google Scholar
  23. 23.
    Blank, M., Gorelick, L., Shechtman, E., Irani, M., Basri, R.: Actions as space-time shapes. In: Tenth IEEE International Conference on Computer Vision, 2005. ICCV 2005, IEEE, vol. 2, pp. 1395–1402 (2005)Google Scholar
  24. 24.
    Jiang, X., Zhong, F., Peng, Q., Qin, X.: Online robust action recognition based on a hierarchical model. Vis. Comput. 30, 1021–1033 (2014)CrossRefGoogle Scholar
  25. 25.
    Oikonomopoulos, A., Patras, I., Pantic, M.: Spatiotemporal salient points for visual recognition of human actions. IEEE Trans. Syst. Man Cybern. Part B Cybern. 36(3), 710–719 (2005)CrossRefGoogle Scholar
  26. 26.
    Rapantzikos, K., Avrithis, Y., Kollias, S.: Dense saliency-based spatiotemporal feature points for action recognition. In: IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2009, IEEE, pp. 1454–1461 (2009)Google Scholar
  27. 27.
    Schuldt, C., Laptev, I., Caputo, B.: Recognizing human actions: a local SVM approach. In: Proceedings of the 17th International Conference on Pattern Recognition. ICPR 2004, IEEE, vol. 3, pp. 32–36 (2004)Google Scholar
  28. 28.
    Laptev, I., Marszalek, M., Schmid, C., Rozenfeld, B.: Learning realistic human actions from movies. In: IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2008, IEEE, pp. 1–8 (2008)Google Scholar
  29. 29.
    Kaâniche, M., Brémont, F.: Recognizing gestures by learning local motion signatures of HOG descriptors. IEEE Trans. Pattern Mach. Intell. (2012)Google Scholar
  30. 30.
    Li, Y., Ye, J., Wang, T., Huang, S.: Augmenting bag-of-words: a robust contextual representation of spatiotemporal interest points for action recognition. Vis. Comput. (2014)Google Scholar
  31. 31.
    Wu, J., Hu, D., Chen, F.: Action recognition by hidden temporal models. Vis. Comput. 30, 1395–1404 (2013)CrossRefGoogle Scholar
  32. 32.
    Alon, J., Athitsos, V., Yuan, Q., Sclaroff, S.: A unified framework for gesture recognition and spatiotemporal gesture segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 31(9), 1685–1699 (2009)CrossRefGoogle Scholar
  33. 33.
    Pedersoli, F., Benini, S., Adami, N., Leonardi, R.: XKin: an open source framework for hand pose and gesture recognition using kinect. Vis. Comput. 30, 1107–1122 (2014)CrossRefGoogle Scholar
  34. 34.
    Bouchard, D., Badler, N.: Semantic segmentation of motion capture using laban movement analysis. In: Intelligent virtual agents. Springer, Berlin Heidelberg, pp. 37–44 (2007)Google Scholar
  35. 35.
    Camurri, A., Lagerlöf, I., Volpe, G.: Recognizing emotion from dance movement: comparison of spectator recognition and automated techniques. Int. J. Hum. Comput. Stud. 59(1), 213–225 (2003)CrossRefGoogle Scholar
  36. 36.
    Gunes, H., Schuller, B.: Categorical and dimensional affect analysis in continuous input: current trends and future directions. Image Vision Comput. (2012)Google Scholar
  37. 37.
    Bernhardt, D., Robinson, P.: Detecting affect from non-stylised body motions. In: Affective Computing and Intelligent Interaction. Springer, Berlin Heidelberg, pp. 59–70 (2007)Google Scholar
  38. 38.
    Nakata, T., Mori, T., Sato, T.: Analysis of impression of robot bodily expression. J. Robot. Mechatron. 14(1), 27–36 (2002)Google Scholar
  39. 39.
    Hachimura, K., Takashina, K., Yoshimura, M.: Analysis and evaluation of dancing movement based on LMA. In: IEEE International Workshop on Robot and Human Interactive Communication. ROMAN 2005, IEEE, pp. 294–299 (2005)Google Scholar
  40. 40.
    Kapadia, M., Chiang, I.K., Thomas, T., Badler, NI., Kider J.T. Jr: Efficient motion retrieval in large motion databases. In: Proceedings of the ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games. ACM, pp. 19–28 (2013)Google Scholar
  41. 41.
    Laban, R.: Espace Dynamique. Contredanse (2003)Google Scholar
  42. 42.
    Clauser, C.E., McConville, J.T., Young J.W.: Weight, volume and center of mass of segments of the human body. Wright-Patterson Air Force Base, Ohio (AMRL-TR-69-70): ANTIOCH COLL YELLOW SPRINGS OH (1969)Google Scholar
  43. 43.
    Fothergill, S., Mentis, H., Kohli, P., Nowozin, S.: Instructing people for training gestural interactive systems. In: Proceedings of the 2012 ACM annual conference on Human Factors in Computing Systems. ACM, pp. 1737–1746 (2012)Google Scholar
  44. 44.
    Song, Y., Morency, L.P., Davis, R.: Distribution-Sensitive Learning for Imbalanced Datasets. In: 2013 IEEE International Conference on Automatic Face & Gesture Recognition and Workshops (FG 2013). IEEE (2013)Google Scholar
  45. 45.
    Suykens, J.A.K., Vandewalle, J.: Least squares support vector machine classifiers. Neural Process. Lett. 9(3), 293–300 (1999)CrossRefMathSciNetGoogle Scholar
  46. 46.
    Milgram, J., Cheriet, M., Sabourin, R.: “One Against One” or “One Against All”: Which One is Better for Handwriting Recognition with SVMs? In: Tenth International Workshop on Frontiers in Handwriting Recognition (2006)Google Scholar
  47. 47.
    Breiman, L.: Random forests. 45(1), 5–32 (2001)Google Scholar
  48. 48.
    Geurts, Pierre, Ernst, Damien, Wehenkel, Louis: Extremely randomized trees. Mach. Learn. 63(1), 3–42 (2006)CrossRefzbMATHGoogle Scholar
  49. 49.
    Scikit-learn. (bibOnline). http://scikit-learn.org/stable/
  50. 50.
    Hripcsak, G., Rothschild, A.S.: Agreement, the f-measure, and reliability in information retrieval. J. Am. Med. Inf. Assoc. 12(3), 296–298 (2005)CrossRefGoogle Scholar
  51. 51.
    Adrien, J.M: Une approche polyvalente: Direction Musicale Dansée, Captation Gestuelle Causale. In: Actes des Journées d’Informatique Musicale. Rennes (2010)Google Scholar
  52. 52.
    Jorland, G., Thirioux, B.: Note sur l’origine de l’empathie. Revue de métaphysique et de morale. février, no. 58, pp. 269–280 (2008)Google Scholar
  53. 53.
    Hautbois, X.: Les Unités Sémiotiques Temporelles : de la sémiotique musicale vers une sémiotique générale du temps dans les arts. In: ICMS 8. Gestes, formes et processus signifiants en musique et sémiotique interarts, Paris, Huitième Congrès International sur la Signification Musicale (2004)Google Scholar
  54. 54.
    Shove, P., Repp, BH.: Musical motion and performance: theoretical and empirical perspectives. In: Rink, J. (ed.) The practice of performance. Cambridge University Press, pp. 55–83 (1995)Google Scholar
  55. 55.
    Grewe, O., Kopiez, R., Altenmüller, E.: L’évaluation des sentiments musicaux: une comparaison entre le modèle circomplexe et les inventaires d’émotions à choix forcé”, Musique, langage, émotion. Approche neuro-cognitive, pp. 49–73 (2010)Google Scholar
  56. 56.
    Chen, Y., Zhou, X.S., Huang, T.S.: One-class SVM for learning in image retrieval. In: 2001 International Conference on Image Processing Proceedings IEEE, vol. 1, pp. 34–37 (2001)Google Scholar
  57. 57.
    Kanluan, I., Grimm, M., Kroschel, K.: Audio-visual emotion recognition using an emotion space concept. In: 16th European Signal Processing Conference. Lausanne (2008)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2015

Authors and Affiliations

  1. 1.ARTEMIS Department, Institut Mines-TelecomTelecom SudParis, CNRS UMR 8145-MAP5Évry CedexFrance

Personalised recommendations