Markerless Vision-Based Skeleton Tracking in Therapy of Gross Motor Skill Disorders in Children

  • B. KaranEmail author
  • Š. Golubović
  • M. Gnjatović
Part of the Mechanisms and Machine Science book series (Mechan. Machine Science, volume 20)


This chapter presents a research towards implementation of a computer vision system for markerless skeleton tracking in therapy of gross motor skill disorders in children suffering from mild cognitive impairment. The proposed system is based on a low-cost 3D sensor and a skeleton tracking software. The envisioned architecture is scalable in the sense that the system may be used as a stand-alone assistive tool for tracking the effects of therapy or it may be integrated with an advanced autonomous conversational agent to maintain the spatial attention of the child and to increase her motivation to undergo a long-term therapy.


3D sensing Skeleton tracking Computer-aided therapy Human-machine interaction 



This research is funded in part by the Ministry of Education, Science, and Technological Development of the Republic of Serbia under the contracts III44008 and TR32035. The research is complementary supported by the project Inclusive physical education in Vojvodina schools: challenges and perspectives, co-financed by the Provincial Secretariat for Science and Technological Development of AP Vojvodina. The pictures given in Fig. 5 are produced by Dragan Matić and Dragan Živančević, affiliated with the Academy of Arts at the University of Novi Sad, and are not published before. The authors thank D. Matić and D. Živančević for allowing them to include these pictures in the chapter.


  1. Belokopytov, M. and Fridin, M.: Motivation of children with cerebral palsy during motor involvement by RAC-CP fun. In: Proc. of the Workshop on Motivational Aspects of Robotics in Physical Therapy, IEEE/RSJ Int. Conf. Intelligent Robots and Systems, (2012).Google Scholar
  2. Bojanin, S.: Reedukacija psihomotorike ili tretman pokretom. Psihijatrija danas, 38(1), 11–27 (2006).Google Scholar
  3. Brass, M. et al.: Compatibility between observed and executed finger movements: comparing symbolic, spatial and imitative cues. Brain and Cognition, 44, 124–143 (2000).Google Scholar
  4. Buccino, G. and Riggio, L.: The role of the mirror neuron system in motor learning. Kinesiology, 38(1), 5–15 (2006).Google Scholar
  5. Buys, K. et al.: An adaptable system for RGB-D based human body detection and pose estimation. Journal of Visual Communication and Image Representation, 25(1), 39−52 (2014).Google Scholar
  6. Calderita, L. et al.: THERAPIST: Towards an autonomous socially interactive robot for motor and neurorehabilitation therapies for children. In: Proc. 2013 7th Int. Conf. Pervasive Computing Technologies for Healthcare, PervasiveHealth 2013, pp. 374–377 (2013).Google Scholar
  7. Clark, R. A. et al.: Validity of the Microsoft Kinect for assessment of postural control. Gait & Posture, 36, 372–377 (2012).Google Scholar
  8. Colton, M.B. et al.: Toward therapist-in-the-loop assistive robotics for children with autism and specific language impairment. In: Proc. of the AISB 2009 Symposium on New Frontiers in Human-Robot Interaction, Edinburgh, Scotland (2009).Google Scholar
  9. Craighero, L., Bello, A., Fadiga, L., and Rizzolatti, G.: Hand action preparation influences the responses to hand pictures. Neuropsychologia, 40, 492–502 (2002).Google Scholar
  10. Dautenhahn, K. et al.: KASPAR – A minimally expressive humanoid robot for human-robot interaction research. Applied Bionics and Biomechanics 6(3-4), 369−397 (2009).Google Scholar
  11. Dias, O.P., Amaral, T.G.B., and Fernão Pires, V.: Computer assisted learning in manipulative therapy education. Manual Therapy, 16(3), 270–272 (2011).Google Scholar
  12. Dutta, T.: Evaluation of the Kinect sensor for 3-D kinematic measurement in the workplace. Applied Ergonomics, 43(4), 645−649 (2012).Google Scholar
  13. Gnjatović, M., and Rösner, D.: Inducing genuine emotions in simulated speech-based human-machine interaction: The NIMITEK corpus. IEEE Trans. Affect. Comput. 1, 132−144 (2010).Google Scholar
  14. Gnjatović, M., Janev, M., and Delić, V.: Focus tree: Modeling attentional information in task-oriented human-machine interaction. Applied Intelligence, 37(3), 305−320 (2012).Google Scholar
  15. Gnjatović, M. et al.: Linguistic encoding of motion events in robotic system. In: Proc. 6th PSU-UNS International Conference on Engineering and Technology (2013).Google Scholar
  16. Gnjatović, M. and Delić, V.: End-user design of emotion-adaptive dialogue strategies for therapeutic purposes. In: Proc. 23rd Italian Workshop Neural Networks, WIRN 2013 (Series: Smart Innovation, Systems and Technologies). Springer, 26, 371−378 (2014).Google Scholar
  17. Huber, M. et al.: PlayStation 3-based telerehabilitation for children with hemiplegia. In: Proc. Virtual Rehabilitation Conference, Vancouver, 2008, pp. 105–112 (2008).Google Scholar
  18. Joković-Turalija, I., Ivkić, D., and Oberman-Babić, M.: Neki aspekti rane dijagnostike i terapije djece sa cerebralnom paralizom. Hrvatska revija za rehabilitacijska istraživanja, 38(1), 121–126 (2002).Google Scholar
  19. Karan, B.: Calibration of depth measurement model for Kinect-type 3D vision sensors. In: Proc. 21st Int. Conf. Central Europe on Computer Graphics, Visualization, and Computer Vision, Plzen, pp. 61–74 (2013a).Google Scholar
  20. Karan, B.: Accuracy improvements of consumer-grade 3D sensors for robotic applications. In: Proc. IEEE 11th Int. Symp. Intelligent Systems and Informatics (2013b).Google Scholar
  21. Kelleher, C., et al.: Towards a therapist-centered programming environment for creating rehabilitation games. Proc. IEEE Computer Games, 2011, pp. 240−247 (2011).Google Scholar
  22. Khoshelham, K. and Elberink, S.O.: Accuracy and resolution of Kinect depth data for indoor mapping applications. Sensors, 12, 1437–1454 (2012).Google Scholar
  23. Krstić, N.: Reedukacija psihomotorike kao oblik neuropsihološke rehabilitacije. Psihijatrija danas, 38(1), 51–66 (2006).Google Scholar
  24. Microsoft Corporation: Kinect for Windows. Online: (2013).
  25. Moreno, J.C. et al..: Effects of robotic guidance on the coordination of locomotion. Journal of NeuroEngineering and Rehabilitation, 10, 79 (2013).Google Scholar
  26. Obdržálek, Š. et al.: Accuracy and robustness of Kinect pose estimation in the context of coaching of elderly population. In: Proc. Annual Int. Conf. IEEE Engineering in Medicine and Biology Society (EMBS), 2012, pp. 1188–1193 (2012).Google Scholar
  27. OpenNI Consortium: OpenNI, the standard framework for 3D sensing. Online: (2013).
  28. PrimeSense Ltd.: NiTE 2 API programmer tutorial guide. PrimeSense (2013).Google Scholar
  29. Ricks, D.J. and Colton, M.B.: Trends and considerations in robot-assisted autism therapy. In: Proc. of the IEEE International Conference on Robotics and Automation (ICRA), Anchorage, AK 2010, pp. 4354–4359 (2010).Google Scholar
  30. Rocha, J.: Skeltrack - A free software skeleton tracking library. Online: (2013).
  31. Schmidt, R.A.: Motor learning and performance: from principle to practice. Human Kinetics Pub. (1991).Google Scholar
  32. SoftKinetic: DepthSense cameras. Online: (2013).
  33. Thill, al.: Robot-assisted therapy for autism spectrum disorders with (partially) autonomous control: Challenges and outlook. Paladyn. Journal of Behavioral Robotics 3(4), 209–217 (2012).Google Scholar
  34. Vanderborght, B. et al.: Using the social robot probo as a social story telling agent for children with ASD. Interaction Studies 13(3), 348–372 (2012).Google Scholar
  35. Yamaguchi, T. et al.: Upper-body interactive rehabilitation system for children with cerebral palsy: the effect of control/display ratios. In: Proc. 9th Int. Conf. Disability, Virtual Reality & Associated Technologies, Laval, France, 2012, pp.181–186 (2012).Google Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  1. 1.Institute of Technical Sciences of the Serbian Academy of Sciences and ArtsBelgradeSerbia
  2. 2.Faculty of MedicineUniversity of Novi SadNovi SadSerbia
  3. 3.Faculty of Technical SciencesUniversity of Novi SadNovi SadSerbia

Personalised recommendations