Analysis of Choreographed Human Movements Using Depth Cameras: A Systematic Review

  • Danilo RibeiroEmail author
  • João Bernardes
  • Norton Roman
  • Marcelo Antunes
  • Enrique Ortega
  • Antonio Sousa
  • Luciano Digiampietri
  • Luis Cura
  • Valdinei Silva
  • Clodoaldo Lima
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9732)


The use of computer vision to analyze human movement has been growing considerably, facilitated by the increased availability of depth cameras This paper describes the results of a systematic review about the techniques used for movement tracking and recognition, focusing on metrics to compare choreographed movements using Microsoft Kinect as a sensor. Several techniques for data analysis and pattern recognition are explored for this task, particularly Dynamic Time Warping and Hidden Markov Models. Most papers we discuss used a single sensor instead of more complex setups and most took advantage of the Kinect SDK instead of alternatives. Rhythm is rarely considered in these systems due to the temporal alignment strategies used. While most systems that use the sensors for some form of interaction instead claim that this interaction is natural, very few actually perform any sort of usability or user experience analysis.


Hide Markov Model Human Movement Dynamic Time Warping Depth Camera Dance Movement 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Zhang, Z.: Microsoft kinect sensor and its effect. IEEE MultiMedia 19(2), 4–10 (2012)CrossRefGoogle Scholar
  2. 2.
    Chye, C., Nakajima, T.: Game based approach to learn martial arts for beginners. In Embedded and Real-Time Computing Systems and Applications (RTCSA), 2012 IEEE 18th International Conference on, pp. 482–485, August 2012. doi: 10.1109/RTCSA.2012.37
  3. 3.
    Hachaj, T., Ogiela, M.R., Piekarczyk, M.: Dependence of kinect sensors number and position on gestures recognition with gesture description language semantic classifier. In: 2013 Federated Conference on Computer Science and Information Systems (FedCSIS), pp. 571–575, September 2013Google Scholar
  4. 4.
    Ogawa, T., Kambayashi, Y.: Physical instructional support system using virtual avatars. In: Proceedings of the 2012 International Conference on Advances in Computer-Human Interactions, pp. 262–265 (2012)Google Scholar
  5. 5.
    Ganganath, N., Leung, H.: Mobile robot localization using odometry and kinect sensor. In: 2012 IEEE International Conference on Emerging Signal Processing Applications (ESPA), pp. 91–94, January 2012. doi: 10.1109/ESPA.2012.6152453
  6. 6.
    Fang, C.: From dynamic time warping (DTW) to hidden markov model (HMM). Univ. Cincinnati 3, 19 (2009)Google Scholar
  7. 7.
    Rosana Ferreira Sampaio and Marisa Cotta Mancini: Estudos de revisão sistemática: um guia para síntese criteriosa da evidência científica. Braz. J. Phys. Ther. (Impr.) 11(1), 83–89 (2007)CrossRefGoogle Scholar
  8. 8.
    Zoubin Ghahramani. Hidden markov models. chapter An Introduction to Hidden Markov Models and Bayesian Networks, pp. 9–42. World Scientific Publishing Co., Inc, River Edge (2002). ISBN: 981-02-4564-5, URL
  9. 9.
    Rabiner, L.R.: A tutorial on hidden markov models, selected applications in speech recognition. Proc. IEEE 77(2), 257–286 (1989). doi: 10.1109/5.18626. ISSN: 0018-9219CrossRefGoogle Scholar
  10. 10.
    Li, S.Z., Jain, A. (eds.): Encyclopedia of Biometrics, chapter Dynamic Time Warping (DTW), pp. 231–231. Springer, Boston (2009). ISBN: 978-0-387-73003-5, doi: 10.1007/978-0-387-73003-5768
  11. 11.
    Sun, G., Muneesawang, P., Kyan, M., Li, H., Zhong, L., Dong, N., Elder, B., Guan, L.: An advanced computational intelligence system for training of ballet dance in a cave virtual reality environment. In: 2014 IEEE International Symposium on Multimedia (ISM), pp. 159–166, December 2014. doi: 10.1109/ISM.2014.55
  12. 12.
    Hachaj, T., Ogiela, M.R.: Semantic description and recognition of human body poses and movement sequences with gesture description language. In: Kim, T., Kang, J.-J., Grosky, W.I., Arslan, T., Pissinou, N. (eds.) MulGraB, BSBT and IUrC 2012. CCIS, vol. 353, pp. 1–8. Springer, Heidelberg (2012). doi: 10.1007/978-3-642-35521-9 1 CrossRefGoogle Scholar
  13. 13.
    Hachaj, T., Ogiela, M.R.: Qualitative evaluation of full body movements with gesture description language. In: 2014 2nd International Conference on Artificial Intelligence, Modelling and Simulation (AIMS), pp. 176–181, November 2014. doi: 10.1109/AIMS.2014.32
  14. 14.
    Zhu, H.-M., Pun, C.-M.: Human action recognition with skeletal information from depth camera. In: 2013 IEEE International Conference on Information and Automation (ICIA), pp. 1082–1085, August 2013. doi: 10.1109/ICInfA.2013.6720456
  15. 15.
    Masurelle, A., Essid, S., Richard, G.: Multimodal classification of dance movements using body joint trajectories and step sounds. In: 2013 14th International Workshop on Image Analysis for Multimedia Interactive Services (WIAMIS), pp. 1–4, July 2013. doi: 10.1109/WIAMIS.2013.6616151
  16. 16.
    Bianco, S., Tisato, F.: Karate moves recognition from skeletal motion (2013). URL
  17. 17.
    Kaewplee, K., Khamsemanan, N., Nattee, C.: A rule-based approach for improving kinect skeletal tracking system with an application on standard muay thai maneuvers. In: 2014 Joint 7th International Conference on and Advanced Intelligent Systems (ISIS). 15th International Symposium on Soft Computing and Intelligent Systems (SCIS), pp. 281–285, December 2014. doi: 10.1109/SCIS-ISIS.2014.7044763
  18. 18.
    Saha, S. Ghosh, S., Konar, A., Nagar, A.K.: Gesture recognition from indian classical dance using kinect sensor. In: 2013 Fifth International Conference on Computational Intelligence, Communication Systems and Networks (CICSyN), pp. 3–8, June 2013. doi: 10.1109/CICSYN.2013.11
  19. 19.
    Lin, X., Kitanovski, V., Zhang, Q., Izquierdo, E.: Enhanced multi-view dancing videos synchronisation. In: 2012 13th International Workshop on Image Analysis for Multimedia Interactive Services (WIAMIS), pp. 1–4, May 2012. doi: 10.1109/WIAMIS.2012.6226773
  20. 20.
    Dancs, J., Sivalingam, R., Somasundaram, G., Morellas, V., Papanikolopoulos, N.: Recognition of ballet micro-movements for use in choreography. In: 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 1162–1167, November 2013. doi: 10.1109/IROS.2013.6696497
  21. 21.
    Gupta, S., Goel, S.: Pogest: A vision based tool for facilitating kathak learning. In: 2014 Seventh International Conference on Contemporary Computing (IC3), pp. 24–29, August 2014. doi: 10.1109/IC3.2014.6897142
  22. 22.
    Pisharady, P.K., Saerbeck, M.: Robust gesture detection and recognition using dynamic time warping and multi-class probability estimates. In: 2013 IEEE Symposium on Computational Intelligence for Multimedia, Signal and Vision Processing (CIMSIVP), pp. 30–36, April 2013. doi: 10.1109/CIMSIVP.2013.6583844.
  23. 23.
    Alexiadis, D.S., Daras, P.: Quaternionic signal processing techniques for automatic evaluation of dance performances from MoCap data. IEEE Trans. Multimed. 16(5), 1391–1406 (2014). doi: 10.1109/TMM.2014.2317311. ISSN: 1520-9210CrossRefGoogle Scholar
  24. 24.
    Keerthy, N.K.: Virtual kung fu sifu with kinect (2012)Google Scholar
  25. 25.
    Anbarsanti, N., Prihatmanto, A.S.: Dance modelling, learning and recognition system of aceh traditional dance based on hidden markov model. In: 2014 International Conference on Information Technology Systems and Innovation (ICITSI), pp. 86–92, November 2014. doi: 10.1109/ICITSI.2014.7048243
  26. 26.
    Holsti, L., Takala, T., Martikainen, A., Kajastila, R., Hämäläinen, P.: Body-controlled trampoline training games based on computer vision. In: CHI 2013 Extended Abstracts on Human Factors in Computing Systems, CHI EA 2013, pp. 1143–1148. ACM, New York (2013). doi: 10.1145/2468356.2468560 ISBN 978-1-4503-1952-2
  27. 27.
    Wada, S., Fukase, M., Nakanishi, Y., Tatsuta, L.: In search of a usability of kinect in the training of traditional japanese. In: 2013 Second International Conference on e-Learning and e-Technologies in Education (ICEEE) (2014)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  • Danilo Ribeiro
    • 1
    Email author
  • João Bernardes
    • 1
  • Norton Roman
    • 1
  • Marcelo Antunes
    • 2
  • Enrique Ortega
    • 2
  • Antonio Sousa
    • 3
  • Luciano Digiampietri
    • 1
  • Luis Cura
    • 4
  • Valdinei Silva
    • 1
  • Clodoaldo Lima
    • 1
  1. 1.University of São PauloSão PauloBrazil
  2. 2.Central Kung Fu AcademyCampinasBrazil
  3. 3.São Paulo Faculty of TechnologySão PauloBrazil
  4. 4.Campo Limpo Paulista FacultyCampo Limpo PaulistaBrazil

Personalised recommendations