Multimedia Tools and Applications

, Volume 76, Issue 13, pp 14847–14867 | Cite as

Augmented reality-based training system for hand rehabilitation

  • Jia LiuEmail author
  • Jianhui Mei
  • Xiaorui Zhang
  • Xiong Lu
  • Jing Huang


This study designs a training system for hand rehabilitation on the basis of augmented reality technology, which enables patients to simultaneously interact with real and virtual environments. The system framework is introduced, and four rehabilitation programs, namely, trajectory training, shelf training, batting training, and spile training, are presented. As a requirement of hand rehabilitation training, a color marker that is suitable for hand rehabilitation training is adopted. Following the Hamming coding principle, this marker is designed as a 7 × 7 square that is filled up by four designated colors with a binary bit of “0” or “1”. The check code in each row of the color marker is applied to restore the occluded binary bits, solve the occlusion issue of color markers, and complete the tracking registration of the color markers. The effectiveness of the developed system is evaluated via a usability study and questionnaires. The evaluation provides positive results. Therefore, the developed system has potential as an effective rehabilitation system for upper limb impairment.


Augmented reality Hand rehabilitation Stroke Marker 



This work was supported by the National Natural Science Foundation of China (No. 61203316, 61203319, 61502240), the Natural Science Foundation of Jiangsu Province (BK20141002), Jiangsu Government Scholarship for Overseas Studies, and the Jiangsu Students’ Project for Innovation and Entrepreneurship Training Program (No. 201510300090).


  1. 1.
    Alamri A, Cha J, Saddik AE (2010) AR-REHAB: an augmented reality framework for poststroke-patient rehabilitation. IEEE Trans Instrum Meas 59(10):2554–2563CrossRefGoogle Scholar
  2. 2.
    Alamri A, Heung-Nam K, Saddik AE (2010) A decision model of stroke patient rehabilitation with augmented reality-based games. Proc Int Conf Autonomous Intell Syst, 1–6Google Scholar
  3. 3.
    Aung YM, Al-Jumaily A (2011) Rehabilitation exercise with real-time muscle simulation based EMG and AR. Proc Int Conf Hybrid Intell Syst, 641–646Google Scholar
  4. 4.
    Aung YM, Al-Jumaily A (2012) AR based upper limb rehabilitation system. Proc IEEE RAS EMBS Int Conf Biomed Robot Biomechatron, 213–218Google Scholar
  5. 5.
    Aung YM, Al-Jumaily A, Anam K (2014) A novel upper limb rehabilitation system with self-driven virtual arm illusion. Proc Int Conf IEEE Eng Med Biol Soc, 3614–3617Google Scholar
  6. 6.
    Burke JW, McNeill MDJ, Charles DK, Morrow PJ (2010) Augmented reality games for upper-limb stroke rehabilitation. Proc IEEE Int Conf Games Virtual Worlds Serious Appl, 75–78Google Scholar
  7. 7.
    Carbonaro N, Mura GD, Lorussi F et al (2014) Exploiting wearable goniometer technology for motion sensing gloves. IEEE J Biomed Health Inform 18(6):1788–1795CrossRefGoogle Scholar
  8. 8.
    Choi Y (2011) Ubi-REHAB: an android-based portable augmented reality stroke rehabilitation system using the eGlove for multiple participants. Proc IEEE Int Conf Virtual Rehab (ICVR), 1–2Google Scholar
  9. 9.
    Collins J, Hoermann S, Regenbrecht H (2014) Virtualising the nine hole peg test of finger dexterity. Proc 10th Int Conf Disability, Virtual Reality Assoc Technol, 181–188Google Scholar
  10. 10.
    Cramer SC, Sur M, Dobkin BH et al (2011) Harnessing neuroplasticity for clinical applications. Brain 134(6):1591–1609CrossRefGoogle Scholar
  11. 11.
    Hoermann S, Hale L, Winser SJ, Regenbrecht H (2012) Augmented reflection technology for stroke rehabilitation–A clinical feasibility study. Proc Int Conf Disability, Virtual Reality Assoc Technol, 1–9Google Scholar
  12. 12.
    Holden MK (2005) Virtual environments for motor rehabilitation: review. Cyberpsychol Behav 8(3):187–211CrossRefGoogle Scholar
  13. 13.
    Hondor HM, Khademi M, Dodakian L, Cramer SC, Lopes CV (2013) A spatial augmented reality rehab system for post-stroke hand rehabilitation. Stud Health Technol Inform 184:279–285Google Scholar
  14. 14.
    Langhorne P, Coupar F, Pollock A (2009) Motor recovery after stroke: a systematic review. Lancet Neurol 8(8):741–754CrossRefGoogle Scholar
  15. 15.
    Luo X, Kenyon RV, Kline T, Waldinger HC, Kamper DG (2005) An augmented reality training environment for post-stroke finger extension rehabilitation. Proc IEEE 9th Int Conf Rehab Robot, 329–332Google Scholar
  16. 16.
    Luo X, Kline T, Fischer H et al (2005) Integration of augmented reality and assistive devices for post-stroke hand opening rehabilitation. Proc IEEE Int Conf Eng Med Biol Soc 7:6855–6858Google Scholar
  17. 17.
    Mathiowetz V, Weber K, Kashman N, Volland G (1985) Adult norms for the nine hole peg test of finger dexterity. OTJR 5(1):24–38Google Scholar
  18. 18.
    Ong SK, Shen Y, Zhang J, Nee AYC (2011) Augmented reality in assistive technology and rehabilitation engineering. In: Furht B (ed) Handbook of Augmented Reality. Springer, New York, pp 603–630Google Scholar
  19. 19.
    Pons TP, Garraghty PE, Ommaya AK et al (1991) Massive cortical reorgazation after sensory deafferentation in adult macaques. Science 252:1857–1860CrossRefGoogle Scholar
  20. 20.
    Regenbrecht H, McGregor G, Ott C, Mueller L, Franz E (2014) Manipulating the experience of reality for rehabilitation applications. Proc IEEE 102(102):170–184CrossRefGoogle Scholar
  21. 21.
    Regenbrecht H, McGregor G, Ott C, Hoermann S (2011) Out of reach? a novel AR interface approach for motor rehabilitation. Proc IEEE Int Symp Mixed Augmented Reality (ISMAR), 219–228Google Scholar
  22. 22.
    Shen Y, Ong SK, Nee AYC (2009) Hand rehabilitation based on augmented reality. Proc ACM 3rd Int Convention Rehab Eng Assistive Technol, 1–4Google Scholar
  23. 23.
    Sucar LE, Leder RS, Reinkensmeyer D, Hernandez J, Azcarate G (2008) Gesture therapy: a low-cost vision-based system for rehabilitation after stroke. Proc AMC 1st Int Conf Health Inform, 107–111Google Scholar
  24. 24.
    Takeuchi N, Izumi SI (2013) Rehabilitation with poststroke motor recovery: a review with a focus on neural plasticity. Stroke Res Treatment 2013Google Scholar
  25. 25.
    Zhang D, Shen Y, Ong SK, Nee AYC (2010) An affordable augmented reality based rehabilitation system for hand motions. Proc IEEE Int Conf Cyberworlds 7(8):346–353Google Scholar
  26. 26.
    Zhou JM, Huang JW, Lao J et al (2012) The clinical utility of hand function rehabilitation science. Shanghai World Book Publishing Company, Shanghai, pp 8–14Google Scholar

Copyright information

© Springer Science+Business Media New York 2016

Authors and Affiliations

  • Jia Liu
    • 1
    Email author
  • Jianhui Mei
    • 1
  • Xiaorui Zhang
    • 1
  • Xiong Lu
    • 2
  • Jing Huang
    • 1
  1. 1.B-DAT & CICAEET, School of Information and ControlNanjing University of Information Science & TechnologyNanjingChina
  2. 2.School of Automation EngineeringNanjing University of Aeronautics and AstronauticsNanjingChina

Personalised recommendations