A human–computer interface for wrist rehabilitation: a pilot study using commercial sensors to detect wrist movements

  • Ericka Janet Rechy-Ramirez
  • Antonio Marin-Hernandez
  • Homero Vladimir Rios-Figueroa
Original Article


Health conditions might cause muscle weakness and immobility in some body parts; hence, physiotherapy exercises play a key role in the rehabilitation. To improve the engagement during the rehabilitation process, we therefore propose a human–computer interface (serious game) in which five wrist movements (extension, flexion, pronation, supination and neutral) are detected via two commercial sensors (Leap motion controller and Myo armband). Leap motion provides data regarding positions of user’s finger phalanges through two infrared cameras, while Myo armband facilitates electromyography signal and inertial motion of user’s arm through its electrodes and inertial measurement unit. The main aim of this study is to explore the performance of these sensors on wrist movement recognition in terms of accuracy, sensitivity and specificity. Eight healthy participants played 5 times a proposed game with each sensor in one session. Both sensors reported over 85% average recognition accuracy in the five wrist movements. Based on t test and Wilcoxon signed-rank test, early results show that there were significant differences between Leap motion controller and Myo armband recognitions in terms of average sensitivities on extension (\(p = 0.0356\)), flexion (\(p = 0.0356\)) and pronation (\(p = 0.0440\)) movements, and average specificities on extension (\(p = 0.0276\)) and pronation (\(p = 0.0249\)) movements.


Human–computer interface Leap motion controller Myo armband Serious game Wrist movement 


Compliance with ethical standards

Conflict of interest

The authors declare that they have no conflict of interest.


  1. 1.
    Gunasekera, W.L., Bendall, J.: Rehabilitation of neurologically injured patients in neurosurgery. Part of the series Springer Specialist Surgery Series. In: Moore, A.J., Newell, D.W. (eds.) Springer, London, pp. 407–421 (2005)Google Scholar
  2. 2.
    Rego, P., Moreira, P.M., Reis, L.P.: Serious games for rehabilitation: a survey and a classification towards a taxonomy. In: Proceedings of the 5th Iberian Conference on Information Systems and Technologies, pp. 1–6. IEEE (2010)Google Scholar
  3. 3.
    Granic, I., Lobel, A., Engels, R.C.: The benefits of playing video games. Am. Psychol. 69(1), 66–78 (2014)CrossRefGoogle Scholar
  4. 4.
    Russoniello, C.V., O’Brien, K., Parks, J.M.: EEG, HRV and psychological correlates while playing Bejeweled II: a randomized controlled study. In: Wiederhold, B.K., Riva, G. (eds.) Annual Review of Cybertherapy and Telemedicine. Advanced Technologies in the Behavioral, Social and Neurosciences, pp. 189–192. IOS Press, Amsterdam (2009)Google Scholar
  5. 5.
    Zyda, M.: From visual simulation to virtual reality to games. Computer 38(9), 25–32 (2005)CrossRefGoogle Scholar
  6. 6.
    Ma, M., Bechkoum, K.: Serious games for movement therapy after stroke. In: Proceedings of the IEEE International Conference on Systems, Man and Cybernetics, pp. 1872–1877. IEEE (2008)Google Scholar
  7. 7.
    Boian, R., Sharma, A., Han, C., Merians, A., Burdea, G., Adamovich, S., Poizner, H.: Virtual reality-based post-stroke hand rehabilitation. In: Proceedings of the Medicine Meets Virtual Reality Conference, pp. 64–70. IOS Press (2002)Google Scholar
  8. 8.
    Asadipour, A., Debattista, K., Chalmers, A.: Visuohaptic augmented feedback for enhancing motor skills acquisition. Vis. Comput. 33(4), 401–411 (2017)CrossRefGoogle Scholar
  9. 9.
    Burke, J.W., McNeill, M.D.J., Charles, D.K., Morrow, P.J., Crosbie, J.H., McDonough, S.M.: Optimising engagement for stroke rehabilitation using serious games. Vis. Comput. 25(12), 1085–1099 (2009)CrossRefGoogle Scholar
  10. 10.
    Burke, J.W., McNeill, M., Charles, D., Morrow, P., Crosbie, J., McDonough, S.: Serious games for upper limb rehabilitation following stroke. In: Proceedings of the Games and Virtual Worlds for Serious Applications, pp. 103–110. IEEE (2009)Google Scholar
  11. 11.
    Aristidou, A.: Hand tracking with physiological constraints. Vis. Comput. (2016). doi: 10.1007/s00371-016-1327-8
  12. 12.
    Chang, Y.J., Han, W.Y., Tsai, Y.C.A.: Kinect-based upper limb rehabilitation system to assist people with cerebral palsy. Res. Dev. Disabil. 34(11), 3654–3659 (2013)CrossRefGoogle Scholar
  13. 13.
    Roy, A.K., Soni, Y., Dubey, S.: Enhancing effectiveness of motor rehabilitation using Kinect motion sensing technology. In: Proceedings of the Global Humanitarian Technology Conference: South Asia Satellite (GHTC-SAS), pp. 298–304. IEEE (2013)Google Scholar
  14. 14.
    Pedersoli, F., Benini, S., Adami, N., Leonardi, R.: XKin: an open source framework for hand pose and gesture recognition using kinect. Vis. Comput. 30(10), 1107–1122 (2014)CrossRefGoogle Scholar
  15. 15.
    Da Gama, A., Fallavollita, P., Teichrieb, V., Navab, N.: Motor rehabilitation using Kinect: a systematic review. Games Health J. 4(2), 123–135 (2015)CrossRefGoogle Scholar
  16. 16.
    Chuan, C.H., Regina, E., Guardino, C.: American Sign Language recognition using Leap motion sensor. In: Proceedings of the 13th International Conference on Machine Learning and Applications (ICMLA), pp. 541–544. IEEE (2014)Google Scholar
  17. 17.
    Marin, G., Dominio, F., Zanuttigh, P.: Hand gesture recognition with Leap motion and Kinect devices. In: Proceedings of the International Conference on Image Processing (ICIP), pp. 1565–1569. IEEE (2014)Google Scholar
  18. 18.
    Funasaka, M., Ishikawa, Y., Takata, M., Joe, K.: Sign language recognition using leap motion controller. In: Proceedings of the International Conference on Parallel and Distributed Processing Techniques and Applications (PDPTA), pp. 263–269 (2015)Google Scholar
  19. 19.
    Bizzotto, N., Costanzo, A., Bizzotto, L.: Leap motion gesture control with OsiriX in the operating room to control imaging: first experiences during live surgery. Surg. Innov. 21(6), 655–656 (2014)CrossRefGoogle Scholar
  20. 20.
    Mewes, A., Saalfeld, P., Riabikin, O., Skalej, M., Hansen, C.A.: Gesture-controlled projection display for CT-guided interventions. Int. J. Comput. Assist. Radiol. Surg. 11(1), 157–164 (2016)CrossRefGoogle Scholar
  21. 21.
    Bassily, D., Georgoulas, C., Guettler, J., Linner, T., Bock, T.: Intuitive and adaptive robotic arm manipulation using the Leap motion controller. In: Proceedings of the International Symposium on Robotics ISR/Robotik, pp. 1–7. IEEE (2014)Google Scholar
  22. 22.
    dos Reis Alves, S.F., Uribe-Quevedo, A.J., Nunes da Silva, I., Ferasoli Filho, H.: Pomodoro, a mobile robot platform for hand motion exercising. In: Proceedings of the International Conference on Biomedical Robotics and Biomechatronics, pp. 970–974. IEEE (2014)Google Scholar
  23. 23.
    Yu, N., Xu, C., Wang, K., Yang, Z., Liu, J.: Gesture-based telemanipulation of a humanoid robot for home service tasks. In: Proceedings of the International Conference on Cyber Technology in Automation, Control, and Intelligent Systems (CYBER), pp. 1923–1927. IEEE (2015)Google Scholar
  24. 24.
    Sonntag, D., Orlosky, J., Weber, M., Gu, Y., Sosnovsky, S., Toyama, T., Toosi, E.N.: Cognitive monitoring via eye tracking in virtual reality pedestrian environments. In: Proceedings of the 4th International Symposium on Pervasive Displays, pp. 269–270. ACM (2015)Google Scholar
  25. 25.
    Khademi, M., Mousavi Hondori, H., McKenzie, A., Dodakian, L., Lopes, C.V., Cramer, S.C.: Free-hand interaction with Leap motion controller for stroke rehabilitation. In: Proceedings of the CHI’14 Extended Abstracts on Human Factors in Computing Systems, pp. 1663–1668. ACM (2014)Google Scholar
  26. 26.
    Charles, D., Pedlow, K., McDonough, S., Shek, K., Charles, T.: Close range depth sensing cameras for virtual reality based hand rehabilitation. J. Assist. Technol. 8(3), 138–149 (2014)CrossRefGoogle Scholar
  27. 27.
    Grubišić, I., Skala Kavanagh, H.A.N.A., Grazio, S.: Novel approaches in hand rehabilitation. Period. Biol. 117(1), 139–145 (2015)Google Scholar
  28. 28.
    Qamar, A., Rahman, M.A., Basalamah, S.: Adding inverse kinematics for providing live feedback in a serious game-based rehabilitation system. In: Proceedings of the International Conference on Intelligent Systems, Modelling and Simulation (ISMS), pp. 215–220. IEEE (2014)Google Scholar
  29. 29.
    Liang, H., Chang, J., Kazmi, I.K., Zhang, J.J., Jiao, P.: Hand gesture-based interactive puppetry system to assist storytelling for children. Vis. Comput. 33(4), 517–531 (2017)CrossRefGoogle Scholar
  30. 30.
    Shen, J., Luo, Y., Wu, Z., Tian, Y., Deng, Q.: CUDAbased real-time hand gesture interaction and visualization for CT volume dataset using leap motion. Vis. Comput. 32(3), 359–370 (2016)CrossRefGoogle Scholar
  31. 31.
    Hettig, J., Mewes, A., Riabikin, O., Skalej, M., Preim, B., Hansen, C.: Exploration of 3D medical image data for interventional radiology using myoelectric gesture control. In: Proceedings of the Eurographics Workshop on Visual Computing for Biology and Medicine, pp. 177–185. ACM (2015)Google Scholar
  32. 32.
    Gándara, C.V., Bauza, C.G.: IntelliHome: A framework for the development of ambient assisted living applications based in low-cost technology. In: Proceedings of the Latin American Conference on Human Computer Interaction, p. 18. ACM (2015)Google Scholar
  33. 33.
    Qamar, A.M., Khan, A.R., Husain, S.O., Rahman, M.A., Baslamah, S.A.: Multi-sensory gesture-based occupational therapy environment for controlling home appliances. In: Proceedings of the 5th ACM on International Conference on Multimedia Retrieval, pp. 671–674. ACM (2015)Google Scholar
  34. 34.
    McCullough, M., Xu, H., Michelson, J., Jackoski, M., Pease, W., Cobb, W., Williams, B.: Myo arm: swinging to explore a VE. In: Proceedings of the ACM SIGGRAPH Symposium on Applied Perception, pp. 107–113. ACM (2015)Google Scholar
  35. 35.
    Rondinelli, R.D., Genovese, E., Brigham, C.R.: Guides to the Evaluation of Permanent Impairment. American Medical Association, Chicago (2008)Google Scholar
  36. 36.
    Norkin, C.C., White, D.J.: Measurement of Joint Motion: A Guide to Goniometry, 5th edn. FA Davis Company, Philadelphia (2016)Google Scholar
  37. 37.
    O’brien, A.V., Jones, P., Mullis, R., Mulherin, D., Dziedzic, K.: Conservative hand therapy treatments in rheumatoid arthritisa randomized controlled trial. Rheumatology 45(5), 577–583 (2006)CrossRefGoogle Scholar
  38. 38.
    Wakefield, A.E., McQueen, M.M.: The role of physiotherapy and clinical predictors of outcome after fracture of the distal radius. J. Bone Joint Surg. 82(7), 972–976 (2000)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag GmbH Germany 2017

Authors and Affiliations

  • Ericka Janet Rechy-Ramirez
    • 1
  • Antonio Marin-Hernandez
    • 1
  • Homero Vladimir Rios-Figueroa
    • 1
  1. 1.Research center for artificial intelligenceUniversidad VeracruzanaXalapaMexico

Personalised recommendations