Synchronized Video and Motion Capture Dataset and Quantitative Evaluation of Vision Based Skeleton Tracking Methods for Robotic Action Imitation

  • Selamawet AtnafuEmail author
  • Conci Nicola
Conference paper
Part of the Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering book series (LNICST, volume 244)


Marker-less skeleton tracking methods are being widely used for applications such as computer animation, human action recognition, human robot collaboration and humanoid robot motion control. Regarding robot motion control, using the humanoid’s 3D camera and a robust and accurate tracking algorithm, vision based tracking could be a wise solution. In this paper we quantitatively evaluate two vision based marker-less skeleton tracking algorithms (the first, Igalia’s Skeltrack skeleton tracking and the second, an adaptable and customizable method which combines color and depth information from the Kinect.) and perform comparative analysis on upper body tracking results. We have generated a common dataset of human motions by synchronizing an XSENS 3D Motion Capture System, which is used as a ground truth data and a video recording from a 3D sensor device. The dataset, could also be used to evaluate other full body skeleton tracking algorithms. In addition, sets of evaluation metrics are presented.


Joint angle Accuracy Tracking ability Human motion dataset 3D camera Ground truth 


  1. 1.
    Shotton, J., Fitzgibbon, A., Cook, M., Sharp, T., Finocchio, M., Moore, R., Kipman, A., Blake, A.: Real time human pose recognition in parts from single depth images. In: CVPR. Microsoft Research Cambridge and Xbox Incubation (2011)Google Scholar
  2. 2.
    Buys, K., Cagniart, C., Baksheev, A., De Laet, T., De Schutter, J., Pantofaru, C.: An adaptable system for RGB-D based human body detection and pose estimation. J. Vis. Commun. Image Represent. 25, 39–52 (2014)CrossRefGoogle Scholar
  3. 3.
    Baak, A., Müller, M., Bharaj, G., Seidel, H.P., Theobalt, C.: A data-driven approach for real-time full body pose reconstruction from a depth camera. In: Fossati, A., Gall, J., Grabner, H., Ren, X., Konolige, K. (eds.) Consumer Depth Cameras for Computer Vision. Advances in Computer Vision and Pattern Recognition (ACVPR), pp. 70–98. Springer, London (2013). Scholar
  4. 4.
    Joaquim, R.: IgaliaSkeltrackGoogle Scholar
  5. 5.
    Luo, R.C., Shih, B.H., Lin, T.W: Real time human motion imitation of anthropomorphic dual arm robot based on cartesian impedance controlGoogle Scholar
  6. 6.
    Ott, C., Lee, D., Nakamura, Y.: Motion capture based human motion recognition and imitation by direct marker controlGoogle Scholar
  7. 7.
    Shon, P., Keith, G., Rao, P.N.: Robotic imitation from human motion capture using Gaussian processesGoogle Scholar
  8. 8.
    Azad, P., Ude, A., Asfour, T., Dillmann, R.: Stereo-based markerless human motion capture for humanoid robot systems. In: IEEE (2007)Google Scholar
  9. 9.
    Zhang, L., Sturm, J., Cremers, D., Lee, D.: Real-time human motion tracking using multiple depth camerasGoogle Scholar
  10. 10.
    The point cloud documentation.
  11. 11.
    Sigal, L., Balan, A.O., Black, M.J.: HumanEva: Synchronized Video and Motion Capture Dataset and Baseline Algorithm for Evaluation of Articulated Human MotionGoogle Scholar
  12. 12.
    MVN Sensor suit ManualGoogle Scholar
  13. 13.
    Ong, A., Harris, I.S., Hamill, J.: The efficacy of a video-based marker-less tracking system for gait analysis. Comput. Methods Biomech. Biomed. Eng. 20, 1089–1095 (2017)CrossRefGoogle Scholar
  14. 14.
    Wang, P., Rehg, J.M.: A modular approach to the analysis and evaluation of particle filters for tracking. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), vol. 1, pp. 790–797 (2006)Google Scholar
  15. 15.
    Gross, R., Shi, J.: The CMU motion of body (MoBo) database. Robotics Institute, Carnegie Mellon University, Technical report CMU-RI-TR-01-18 (2001)Google Scholar
  16. 16.
    CMU motion capture database.

Copyright information

© ICST Institute for Computer Sciences, Social Informatics and Telecommunications Engineering 2018

Authors and Affiliations

  1. 1.Bahir Dar Institute of TechnologyBahir DarEthiopia
  2. 2.University of TrentoTrentoItaly

Personalised recommendations