Skip to main content
SpringerLink
Log in

Innovative haptic-based system for upper limb rehabilitation in visually impaired individuals: a multilayer approach

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

The integration of technology in healthcare has revolutionized physical rehabilitation of patients affected by neurological conditions, such as spinal cord injuries and strokes. However, a significant gap remains in addressing the needs of the visually impaired, as most current solutions are visually-centric. This paper presents a novel haptic-based system tailored for the visually impaired that aims to bridge this gap in upper limb rehabilitation. The system is underpinned by a multi-layer architecture that allows both patient guidance during rehabilitation and the definition and analysis of exercises by the therapist. The architecture design includes functionality to track the user’s body by means of natural user interfaces, to register the user’s movement, and to guide them through the vibrations of the haptic glove or through voice commands. Thus, the proposed solution empowers visually impaired individuals to perform therapist-defined hand exercises autonomously, fostering independence and optimizing therapeutic resources. The system captures detailed kinematic data, offering therapists a comprehensive insight into the patient’s exercise execution. To assess the system’s functionality, a pilot trial was conducted. This study also allowed us to compare the similarity of exercise performance under vibration-based guidance to the exercises defined by the therapist with that of verbal guidance. The results highlighted a significant increase in similarity to therapist-defined exercises when using the vibration-based guidance facilitated by the designed haptic glove.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

Data Availability

The dataset is publicly available at https://github.com/AIR-Research-Group-UCLM/Haptic-Rehab (accessed on 19 November 2023).

Notes

  1. https://irisvision.com/irisvision-inspire/

  2. https://www.orcam.com/

  3. https://www.esighteyewear.com/

  4. https://developers.google.com/mediapipe/solutions/vision/pose_landmarker

References

  1. Mekki M, Delgado AD, Fry A, Putrino D, Huang V (2018) Robotic rehabilitation and spinal cord injury: a narrative review. Neurotherapeutics. 15(3):604–617. https://doi.org/10.1007/s13311-018-0642-3

    Article  Google Scholar 

  2. Bravo VP, Muñoz JA (2022) Wearables and their applications for the rehabilitation of elderly people. Med Biol Eng Comput 60(5):1239–1252. https://doi.org/10.1007/s11517-022-02544-w

    Article  Google Scholar 

  3. Pereira MF, Prahm C, Kolbenschlag J, Oliveira E, Rodrigues NF (2020) A virtual reality serious game for hand rehabilitation therapy. In: 2020 IEEE 8th International conference on serious games and applications for health (SeGAH) pp 1–7. https://doi.org/10.1109/SeGAH49190.2020.9201789. ISSN: 2573-3060

  4. Lamash L, Klinger E, Josman N (2017) Using a virtual supermarket to promote independent functioning among adolescents with Autism Spectrum Disorder. In: 2017 International conference on virtual rehabilitation (ICVR) pp 1–7. https://doi.org/10.1109/ICVR.2017.8007467 . ISSN: 2331-9569

  5. Araújo AVL, Neiva JFdO, Monteiro CBdM, Magalhães FH (2019) Efficacy of Virtual Reality Rehabilitation after Spinal Cord Injury: A Systematic Review. Biomed Res Int 2019:7106951 https://doi.org/10.1155/2019/7106951 . Publisher: Hindawi

  6. Klinker K, Wiesche M, Krcmar H (2020) Digital transformation in health care: Augmented reality for hands-free service innovation. Inf Syst Front 22:1419–1431. https://doi.org/10.1007/s10796-019-09937-7

    Article  Google Scholar 

  7. Carpinello A, Vezzetti E, Ramieri G, Moos S, Novaresio A, Zavattero E, Borbon C (2021) Evaluation of hmds by qfd for augmented reality applications in the maxillofacial surgery domain. Appl Sci. 11(22):11053. https://doi.org/10.3390/app112211053

  8. Barcali E, Iadanza E, Manetti L, Francia P, Nardi C, Bocchi L (2022) Augmented reality in surgery: a scoping review. Appl Sci 12(14):6890. https://doi.org/10.3390/app12146890

    Article  Google Scholar 

  9. García FM, Moraleda R, Schez-Sobrino S, Monekosso DN, Vallejo D, Glez-Morcillo C (2023) Health-5g: a mixed reality-based system for remote medical assistance in emergency situations. IEEE Access 11:59016–59032. https://doi.org/10.1109/ACCESS.2023.3285420

    Article  Google Scholar 

  10. Herrera V, Vallejo D, Castro-Schez JJ, Monekosso DN, de los Reyes A, Glez-Morcillo C, Albusac J (2023) Rehab-immersive: A framework to support the development of virtual reality applications in upper limb rehabilitation. SoftwareX. 23:101412. https://doi.org/10.1016/j.softx.2023.101412

  11. Hao J, Xie H, Harp K, Chen Z, Siu K-C (2022) Effects of virtual reality intervention on neural plasticity in stroke rehabilitation: a systematic review. Arch Phys Med Rehabil 103(3):523–541. https://doi.org/10.1016/j.apmr.2021.06.024

    Article  Google Scholar 

  12. Kang Y, Ding H, Zhou H, Wei Z, Liu L, Pan D, Feng S (2017) Epidemiology of worldwide spinal cord injury: a literature review. J Neurorestoratology 6:1–9. https://doi.org/10.2147/JN.S143236

    Article  Google Scholar 

  13. Hodkin EF, Lei Y, Humby J, Glover IS, Choudhury S, Kumar H, Perez MA, Rodgers H, Jackson A (2018) Automated fes for upper limb rehabilitation following stroke and spinal cord injury. IEEE Trans Neural Syst Rehabil Eng 26(5):1067–1074. https://doi.org/10.1109/TNSRE.2018.2816238

    Article  Google Scholar 

  14. Spooren AIF, Janssen-Potten YJM, Kerckhofs E, Bongers HMH, Seelen HAM (2011) Evaluation of a task-oriented client-centered upper extremity skilled performance training module in persons with tetraplegia. Spinal Cord 49(10):1049–1054. https://doi.org/10.1038/sc.2011.54

    Article  Google Scholar 

  15. Organization WH (2020) The Top 10 Causes of Death. https://www.who.int/news-room/fact-sheets/detail/the-top-10-causes-of-death

  16. Zhao Q, Wang X, Wang T et al (2021) Cognitive rehabilitation interventions after stroke: protocol for a systematic review and meta-analysis of randomized controlled trials. Syst Rev 10:66. https://doi.org/10.1186/s13643-021-01607-7

    Article  Google Scholar 

  17. Parisi A, Bellinzona F, Di Lernia D, Repetto C, De Gaspari S, Brizzi G, Riva G, Tuena C (2022) Efficacy of multisensory technology in post-stroke cognitive rehabilitation: A systematic review. J Clin Med 11(21):6324. https://doi.org/10.3390/jcm11216324

    Article  Google Scholar 

  18. Real S, Araujo A (2019) Navigation systems for the blind and visually impaired: Past work, challenges, and open problems. Sensors 19(15):3404. https://doi.org/10.3390/s19153404

    Article  Google Scholar 

  19. Creed C, Al-Kalbani M, Theil A, Sarcar S, Williams I (2023) Inclusive AR/VR: accessibility barriers for immersive technologies. Universal Access in the Information Society. https://doi.org/10.1007/s10209-023-00969-0

  20. Hassler AP, Menasalvas E, García-García FJ, Rodríguez-Mañas L, Holzinger A (2019) Importance of medical data preprocessing in predictive modeling and risk factor discovery for the frailty syndrome. BMC Med Inform Decis Mak 19:1–17. https://doi.org/10.1186/s12911-019-0747-6

    Article  Google Scholar 

  21. Demmin DL, Silverstein SM (2020) Visual impairment and mental health: unmet needs and treatment options. Clinical Ophthalmology, pp 4229–4251. https://doi.org/10.2147/OPTH.S258783

  22. Santos ADP, Ferrari ALM, Medola FO, Sandnes FE (2022) Aesthetics and the perceived stigma of assistive technology for visual impairment. Disabil Rehabil Assist Technol 17(2):152–158. https://doi.org/10.1080/17483107.2020.1768308. PMID: 32501732

  23. Kuriakose B, Shrestha R, Sandnes FE (2022) Tools and technologies for blind and visually impaired navigation support: a review. IETE Tech Rev 39(1):3–18. https://doi.org/10.1080/02564602.2020.1819893

    Article  Google Scholar 

  24. Manjari K, Verma M, Singal G (2020) A survey on assistive technology for visually impaired. Internet of Things 11:100188. https://doi.org/10.1016/j.iot.2020.100188

    Article  Google Scholar 

  25. Fernandes H, Costa P, Filipe V, Paredes H, Barroso J (2019) A review of assistive spatial orientation and navigation technologies for the visually impaired. Univ Access Inf Soc 18:155–168. https://doi.org/10.1007/s10209-017-0570-8

    Article  Google Scholar 

  26. Howard J, Fisher Z, Kemp AH, Lindsay S, Tasker LH, Tree JJ (2022) Exploring the barriers to using assistive technology for individuals with chronic conditions: a meta-synthesis review. Disabil Rehabil Assist Technol 17(4):390–408. https://doi.org/10.1080/17483107.2020.1788181

    Article  Google Scholar 

  27. Okonji PE, Ogwezzy DC (2019) Awareness and barriers to adoption of assistive technologies among visually impaired people in nigeria. Assist Technol 31(4):209–219. https://doi.org/10.1080/10400435.2017.1421594

    Article  Google Scholar 

  28. Martiniello N, Eisenbarth W, Lehane C, Johnson A, Wittich W (2022) Exploring the use of smartphones and tablets among people with visual impairments: Are mainstream devices replacing the use of traditional visual aids? Assist Technol 34(1):34–45. https://doi.org/10.1080/10400435.2019.1682084

    Article  Google Scholar 

  29. Cardillo E, Caddemi A (2019) Insight on electronic travel aids for visually impaired people: A review on the electromagnetic technology. Electronics 8(11):1281. https://doi.org/10.3390/electronics8111281

    Article  Google Scholar 

  30. Jeong G-Y, Yu K-H (2016) Multi-section sensing and vibrotactile perception for walking guide of visually impaired person. Sensors 16(7):1070. https://doi.org/10.3390/s16071070

    Article  Google Scholar 

  31. Budrionis A, Plikynas D, Daniušis P, Indrulionis A (2022) Smartphone-based computer vision travelling aids for blind and visually impaired individuals: A systematic review. Assist Technol 34(2):178–194. https://doi.org/10.1080/10400435.2020.1743381

    Article  Google Scholar 

  32. Katzschmann RK, Araki B, Rus D (2018) Safe local navigation for visually impaired users with a time-of-flight and haptic feedback device. IEEE Trans Neural Syst.Rehabil Eng 26(3):583–593. https://doi.org/10.1109/TNSRE.2018.2800665

    Article  Google Scholar 

  33. Meshram VV, Patil K, Meshram VA, Shu FC (2019) An astute assistive device for mobility and object recognition for visually impaired people. IEEE Trans Hum-Mach Syst 49(5):449–460. https://doi.org/10.1109/THMS.2019.2931745

    Article  Google Scholar 

  34. Debnath B, O’brien M, Yamaguchi M, Behera A (2022) A review of computer vision-based approaches for physical rehabilitation and assessment. Multimed Syst 28(1):209–239. https://doi.org/10.1007/s00530-021-00815-4

  35. Schez-Sobrino S, Vallejo D, Monekosso DN, Glez-Morcillo C, Remagnino P (2020) A distributed gamified system based on automatic assessment of physical exercises to promote remote physical rehabilitation. IEEE Access 8:91424–91434. https://doi.org/10.1109/ACCESS.2020.2995119

    Article  Google Scholar 

  36. Roccetti M, Marfia G, Semeraro A (2012) Playing into the wild: A gesture-based interface for gaming in public spaces. J Vis Commun Image Represent 23(3):426–440. https://doi.org/10.1016/j.jvcir.2011.12.006

    Article  Google Scholar 

  37. Ren L, Peng Y (2019) Research of fall detection and fall prevention technologies: A systematic review. IEEE Access 7:77702–77722. https://doi.org/10.1109/ACCESS.2019.2922708

    Article  Google Scholar 

  38. Gmez-Portes C, Castro-Schez JJ, Albusac J, Monekosso DN, Vallejo D (2021) A fuzzy recommendation system for the automatic personalization of physical rehabilitation exercises in stroke patients. Mathematics 9(12):1427. https://doi.org/10.3390/math9121427

    Article  Google Scholar 

  39. Cavazos Quero L, Iranzo Bartolomé J, Cho J (2021) Accessible visual artworks for blind and visually impaired people: comparing a multimodal approach with tactile graphics. Electronics 10(3):297. https://doi.org/10.3390/electronics10030297

    Article  Google Scholar 

  40. Guerrón NE, Cobo A, Olmedo JJS, Martín C (2020) Sensitive interfaces for blind people in virtual visits inside unknown spaces. Int J Hum-Comput Stud 133:13–25. https://doi.org/10.1016/j.ijhcs.2019.08.004

    Article  Google Scholar 

  41. Herrera V, Reyes-Guzmán A, Vallejo D, Castro-Schez JJ, Monekosso DN, González-Morcillo C, Albusac J (2023)Performance analysis for upper limb rehabilitation in non-immersive and immersive scenarios. In: ICEIS (2), pp. 231–242

  42. Tölgyessy M et al (2021) Evaluation of the azure kinect and its comparison to kinect v1 and kinect v2. Sensors 21(2):413. https://doi.org/10.3390/s21020413

    Article  Google Scholar 

Download references

Acknowledgements

This work has been founded by the Spanish Ministry of Science and Innovation MCIN/AEI/10.13039/501100011033 under the Research Project: Platform for Upper Extremity Rehabilitation based on Immersive Virtual Reality (Rehab-Immersive), PID2020-117361RB-C21, and by the University of Castilla-La Mancha (Spain), thanks to the call for proposals to support research groups (Ref. 2023-GRIN-34400).

Author information

Author notes

  1. Javier Albusac, Vanesa Herrera, Santiago Schez-Sobrino, Rubén Grande, Dorothy N. Monekosso and David These authors contributed equally to this work.

Authors and Affiliations

  1. Department of Information Technologies and Systems, University of Castilla-La Mancha, P. de la Universidad 4, Ciudad Real, 13071, Spain

    Javier Albusac, Vanesa Herrera, Santiago Schez-Sobrino, Rubén Grande & David Vallejo

  2. Department of Computer Science, Durham University, Stockton Road, Durham, DH1 3LE, United Kingdom

    Dorothy N. Monekosso

Authors
  1. Javier Albusac
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Vanesa Herrera
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Santiago Schez-Sobrino
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Rubén Grande
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Dorothy N. Monekosso
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. David Vallejo
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to David Vallejo.

Ethics declarations

Conflicts of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Albusac, J., Herrera, V., Schez-Sobrino, S. et al. Innovative haptic-based system for upper limb rehabilitation in visually impaired individuals: a multilayer approach. Multimed Tools Appl 83, 60537–60563 (2024). https://doi.org/10.1007/s11042-023-17892-4

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-023-17892-4

Keywords

Search

Navigation