Skip to main content
SpringerLink
Log in

A virtual reality system for strengthening awareness and participation in rehabilitation for post-stroke patients

  • Original Paper
  • Published:
Journal on Multimodal User Interfaces Aims and scope Submit manuscript

Abstract

This paper presents a system called REAPP, a virtual environment aiming at supporting upper-limb robotic neurorehabilitation for post-stroke patients. In the last few years different systems using either or both virtual environments and robots have been developed. However, such advanced systems often lack attention on the physical and psychological conditions perceived by the patient, who should be the key focus in therapies. In this context, REAPP aims not to be just an entertaining interface but also a means to investigate patients’ feelings toward the therapy and to enhance their self-perception while performing the rehabilitation exercises. Before and after the rehabilitation session the patients are inquired about their emotional state, their physical condition and a self-evaluation of the performance is done. While performing the exercises patients look at a 2D/3D interface which has the aim of enhancing their self-awareness by showing them, through an avatar, the movements they are doing. Widgets and a virtual assistant provide additional feedback about the quality of the movements which is computed through a specifically developed arm model that can be customized on the residual capabilities of each patient.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. Mendis S (2013) Stroke disability and rehabilitation of stroke: World Health Organization perspective. Int J Stroke 8(1):3–4

    Article  Google Scholar 

  2. Janssen H, Ada L, Bernhardt J, MvElduff P, Pollack M, Nilsson M, Spratt N (2014) Physical, cognitive and social activity levels of stroke patients undergoing rehabilitation within a mixed rehabilitation unit. Clin Rehabil 28(1):91–101

    Article  Google Scholar 

  3. Pekna M, Pekny M, Nilsson M (2012) Modulation of Neural Plasticity as a Basis for Stroke Rehabilitation. Stroke 43:2819–2828

    Article  Google Scholar 

  4. Chang WH, Kim Y-H (2013) Robot-assisted therapy in stroke rehabilitation. J Stroke 3(15):174–181

    Article  Google Scholar 

  5. Lohse KR, Hilderman CGE, Cheung KL, Tatla S, Van der Loos HFM (2014) Virtual reality therapy for adults post-stroke: a systematic review and meta-analysis exploring virtual environments and commercial games in therapy. Plos One 9(3):e93318

    Article  Google Scholar 

  6. Fluet GG, Deutsch JE (2013) Virtual reality for sensorimotor rehabilitation post-stroke: the promise and current state of the field. Curr Phys Med Rehabil Rep 1(1):9–20

    Article  Google Scholar 

  7. Novak D, Nagke A, Keller U, Riener R (2014) Increasing motivation in robot-aided arm rehabilitation with competitive and cooperative gameplay. J Neuroeng Rehabil 11:64

    Article  Google Scholar 

  8. Riener R, Nef T, Colombo G (2005) Robot-aided neurorehabilitation of the upper extremities. Med Biol Eng Comput 43:2–10

    Article  Google Scholar 

  9. Coote S, Stokes E (2003) Robot mediated therapy: attitudes of patients and therapists toward the first prototype of the GENTLE/s system. Technol Disabil 15:27–34

    Google Scholar 

  10. Lum PS, Burgar CG, Shor SC, Majmundar M, Van der Loos M (2002) Robot-assisted movement training compared with conventional therapy techniques for the rehabilitation of upper limb motor function after stroke. Arch Phys Med Rehabil 83(7):952–959

    Article  Google Scholar 

  11. Maclean N, Piund P (2000) A critical review of the concept of patient motivation in the literature on physical reahabilitation. Soc Sci Med 50:495–506

    Article  Google Scholar 

  12. Borghese NA, Pirovano M, Lanzi PL, Wüest S, de Bruin ED (2013) Computational intelligence and game design for effective at home rehabilitation. Games Health J 2(2):81–88

    Article  Google Scholar 

  13. Bertilsson AS, Ranner M, Von Kich L, Eriksson G, Johansson U, Ytterberg C, Guidetti S, Tham K (2014) A client-centered ADL intervention: three-month follow-up of a randomized controlled trial. Scand J Occup Ther 1–15

  14. Pastor I, Hayes HA, Bamberg SJM (2012) A feasibility study of an upper limb rehabilitation system using Kinect and computer games. In: 34th annual international conference of the IEEE EMBS, San Diego, California USA

  15. Mirelman A, Bonato P, Deutsch DE (2009) Effects of training with a robot-virtual reality system compared with a robot alone on the gait of individuals after stroke. Stroke 40(1):169–174

    Article  Google Scholar 

  16. Brütsch K, Koenig A, Zimmerli L, Mérillat-Koeneke S, Riener R, Jäncke L, van Hedel HJ, Meyer-Heim A (2011) Virtual reality for enhancement of robot-assisted gait training in children with central gait disorders. J Rehabil Med 43(6):493–499

    Article  Google Scholar 

  17. Shah N, Basteris A, Amirabdollahian F (2014) Design parameters in multimodal games for rehabilitation. Games Health J 3(1):13–20

    Article  Google Scholar 

  18. Malosio M, Pedrocchi N, Molinari Tosatti L (2010) Robot-assisted upper-limb rehabilitation platform. In: Proceedings of the 5th ACM/IEEE international conference on Human-robot interaction, Nara, Japan

  19. Scano A, Caimmi M, Malosio M, Pedrocchi N, Molinari Tosatti L, Molteni F (2013) Upper limb robotic rehabilitation: Treatment customization. Gait Posture 37(1):S13–S14

    Article  Google Scholar 

  20. Caimmi M, Carda S, Giovanzana C, Maini E, Sabatini AM, Smania N, Molteni F (2008) Using kinematic analysis to evaluate constraint-induced movement therapy in chronic stroke patients. Neurorehabil Neural Repair 22:31–39

    Article  Google Scholar 

  21. Aprile I, Rabuffetti M, Padua L, Di Sipio E, Simbolotti C, Ferrarin M (2014) Using kinematic analysis to evaluate constraint-induced movement therapy in chronic stroke patients. BioMed Res Int

  22. Redaelli C, Caimmi M, Molteni F (2014) Percezione dell’interazione uomo/robot nella riabilitazione assistita: il punto di vista del paziente. In: \(42^{\circ }\) congresso nazionale SIMFER, Torino

  23. Lin KC, Chen YT, Huang PC (2014) Effect of mirror therapy combined with somatosensory stimulation on motor recovery and daily function in stroke patients: A pilot study. J Formos Med Assoc 113(7):422–428

    Article  Google Scholar 

  24. Zucconi C, Valt V, Agostini M, Turolla A, Tonin P, Piron L (2011) Assessment of a virtual teacher feedback for the recovery of the upper limb after stroke. Ital J Physiother

  25. Albert SJ, Kesselring J (2011) Neurorehabilitation of stroke. J Neurol 259(5):817–832

    Article  Google Scholar 

  26. Cherney LR, van Vuuren S (2012) Telerehabilitation, virtual therapists, and acquired neurologic speech and language disorders. In: seminars in speech and language

  27. Parker J, Mawson S, Mountain G, Nasr N, Zheng H (2014) Stroke patients’ utilisation of extrinsic feedback from computer-based technology in the home: a multiple case study realistic evaluation. BMC Med Inform Decis Mak 14(1):14–46

    Article  Google Scholar 

  28. Ekman P, Friesen W, Hager J (2002) Facial action coding system: a technique for the measurement of facial movement. Consulting Psychologists Press, Mountain View

    Google Scholar 

  29. Smith VA, DeVellis BM, Kalet A, Roberts JC, DeVellis RF (2005) Encouraging patient adherence: primary care physicians’ use of verbal compliance-gaining strategies in medical interviews. Patient Educ Couns 57:62–76

    Article  Google Scholar 

  30. Koenig A, Omlin X, Zimmerli L, Sapa M, Krewer C, Bollinger M, Müller F, Riener R (2011) Psychological state estimation from physiological recordings during robot-assisted gait rehabilitation. J Rehabil Res Dev 48(4):367–385

    Article  Google Scholar 

  31. Zangiacomi A, Redaelli C, Valentini F, Bernardelli G (2014) Design of interaction in a Virtual Environment for post-stroke rehabilitation: a cognitive perspective. In: 5th IEEE cogInfoCom conference, Vietri sul Mare, Italy

  32. Cherkassky T, Oksenberg A, Froom P, Ring H (2003) Sleep-related breathing disorders and rehabilitation outcome of stroke patients: a prospective study. Am J Phys Med Rehabil 82(6):452–455

    Google Scholar 

  33. Dib S, Ramos AR, Douglas WM, Rundek T (2005) Sleep and stroke. Period Biol 114(3):369–375

    Google Scholar 

  34. Spalletta G, Pasini A, Costa A, De Angelis D, Ramundo N, Paolucci S, Caltagirone C (2001) Alexithymic features in stroke: effects of laterality and gender. Psychosom Med 63(6):944–950

    Article  Google Scholar 

  35. Lane RD, Sechrest L, Reidel R, Weldon V, Kaszniak A (1996) Impaired verbal and nonverbal emotion recognition in alexithymia. Psychosom Med 58(3):203–210

    Article  Google Scholar 

  36. Yuvaraj RA, Murugappan MA, Norlinah MI, Sundaraj KA, Khairiyah MB (2013) Review of emotion recognition in stroke patients. Dement Geriatr Cogn Disord 36:179–196

    Article  Google Scholar 

  37. Wiles R, Ashburn A, Payne S, Murphy C (2002) Patients’ expectations of recovery following stroke: a qualitative study. Disabil Rehabil 24(16):841–850

    Article  Google Scholar 

  38. Lange B, Change CY, Suma E, Newman B, Rizzo AS, Bolas M (2011) Development and evaluation of low cost game- based balance rehabilitation tool using the Microsoft Kinect sensor. In: 33rd annual international conference of the IEEE EMBS, Massachusetts USA, Boston

  39. Kitzmüller G, Häggström T, Asplund K (2013) Living an unfamiliar body: the significance of the long-term influence of bodily change on the perception of self after stroke. Med Health Care Philos 29:16–19

    Google Scholar 

  40. Prochnow D, Bermùz i Badia S, Schmidt J, Duff A, Brunheim S, Kleiser R, Seitz RJ, Verschure PFMJ (2013) A functional magnetic resonance imaging study of visuomotor processing in a virtual reality - based paradigm: rehabilitation gaming system. Eur J Neurosci 37:1441–1447

    Article  Google Scholar 

  41. Rizzolatti G, Fabbri-Destro M, Cattaneo L (2009) Mirror neuron and their clinical relevance. Nat Clin Pract Neurol 5:24–34

    Article  Google Scholar 

  42. Stefan K, Classen J, Celnik P, Cohen LG (2008) Cuncurrent action observation modulates practise-induced motor memory formation. Eur J Neurosci 27(3):730–738

    Article  Google Scholar 

  43. Ertelt D, Small S, Solodkin A, Dettmers C, McNamara A, Binkofski F, Buccino G (2007) Action observation has a positive impact on rehabilitation of motor deficits after stroke. Neuroimage 36(2):164–173

    Article  Google Scholar 

  44. Kil-Soo S, Hongki K, Eung KS (2011) What if your avatar looks like you? Dualcongruity perspectives for avatar use. MIS Q 35(3):711–729

    Google Scholar 

  45. Changsoo K, Sang-Gun L, Minchoel K (2012) I became an attractive person in the virtual world: users’ identification with virtual communities and avatars. Comput Hum Behav 28(5):1663– 1669

    Article  Google Scholar 

  46. Vasalou A, Joinson AN (2009) Me, myself and I: The role of interactional context on self-presentation through avatars. Comput Hum Behav 25(2):510–520

    Article  Google Scholar 

  47. Teng C-I (2010) Customization, immersion satisfaction, and online gamer loyalty. Comput Hum Behav 26(6):1547–1554

    Article  Google Scholar 

  48. Li DD, Kien Liau A, Khoo A (2013) layer-Avatar Identification in video gaming: concept and measurement. Comput Hum Behav 29(1):257–263

    Article  Google Scholar 

  49. Nowak KL (2013) Choosing Buddy Icons that look like me or represent my personality: using Buddy Icons for social presence. Comput Hum Behav 29(4):1456–1464

    Article  Google Scholar 

  50. Bailey R, Wise K, Bolls P (2009) How avatar customizability affects children’s arousal and subjective presence during junk food-sponsored online video games. Cyber psychol Behav 12(3):277–283

    Article  Google Scholar 

  51. Biocca F (1997) The Cyborg’s dilemma: progressive embodiment in Virtual Environments. J Comput Med Commun 3:(2)

  52. Youjeong K, Shyam Sundar S (2012) Visualizing ideal self vs. actual self through avatars: impact on preventive health outcomes. Comput Hum Behav 28(4):1356–1364

    Article  Google Scholar 

  53. Jin SAA (2010) I feel more connected to the physically deal Mini Me than the mirror-image Mini Me: theoretical implications of the “Malleable Self” for speculations on the effects of avatar creation on Avatar-Self Connection in Wii. Cyberpsychol Behav Soc Netw 13(5):567–570

    Article  Google Scholar 

  54. Keppel CC, Crowe SF (2000) Changes to body image and self-esteem following stroke in young adults. Neuropsychol Rehabil 10(1):15–31

    Article  Google Scholar 

  55. Hole E, Stubbs B, Roskell C, Soundy A (2014) The Patient’s Experience of the Psychosocial Process That Influences Identity following Stroke Rehabilitation: A Metaethnography. Sci World J 2014:349151

    Article  Google Scholar 

  56. Krakauer JW (2006) Motor learning: its relevance to stroke recovery and neurorehabilitation. Curr Opin Neurol 19(1):84–90

    Article  Google Scholar 

  57. Liao WW, Wu CY, Hsieh YW, Lin KC, Chang WY (2011) Effects of robot-assisted upper limb rehabilitation on daily function and real-world arm activity in patients with chronic stroke: a randomized controlled trial. Clin Rehabil 26(2):111–120

    Article  Google Scholar 

  58. Thielman G (2010) Rehabilitation of reaching poststroke: a randomized pilot investigation of tactile versus aditory feedback for trunk control. J Neurol Phys Ther 34(3):138–144

    Article  Google Scholar 

  59. Arya KN, Pandian S, Verma R, Garg KR (2011) Movement therapy induced neural reorganization and motor recovery in stroke: a review. J Bodyw Mov Ther 15(4):528–537

    Article  Google Scholar 

  60. McPhail SM, Nalder E, Hill AM, Haines TP (2013) Physiotherapists have accurate expectations of their patients’ future health-related quality of life after first assessment in a subacute rehabilitation setting. Biomed Res Int 2013:340371

    Article  Google Scholar 

  61. Baranyi P, Csapo A (2010) Cognitive Infocommunications: CogInfoCom. In: Proceedings of 11th IEEE international symposium on computational intelligence and informatics, Budapest

  62. Baranyi P, Csapo A (2012) Definition and synergies of Cognitive Infocommunications. Acta Polytechnica Hungarica 9(1):67–83

    Google Scholar 

  63. Wright RW (1997) Affective computing. MIT Press, Cambridge

    Google Scholar 

  64. Csapo A, Wersenyi G (2013) Overview of auditory representations in human-machine interfaces. ACM Comput Surv 46(2):1–23

    Article  Google Scholar 

  65. Blattner MM, Sumikava DA, Greenberg RM (1989) Earcons and icons: their structure and common principles design. Hum Comput Interact 4:11–44

    Article  Google Scholar 

  66. Rosati G, Rodà A, Avanzini F, Masier S (2013) On the role of the auditory feedback in robot-assisted movement training after stroke: review of the literature. Comput Intell Neurosci 2013:586138

    Article  Google Scholar 

  67. Levin MF, Kleim JA, Wolf SL (2008) What do motor “recovery” and “compensation” mean in patients following stroke? Neurorehabil Neural Repair 23(4):313–319

    Article  Google Scholar 

  68. Taati B, Wang R, Huq R, Snoek J, Mihailidis A (2012) Vision-based posture assessment to detect and categorize compensation during robotic rehabilitation therapy. In: the fourth IEEE RAS/EMBS internationalconference on biomedical robotics and biomechatronics, Roma, Italy

  69. Colombo R, Sterpi I, Mazzone A, Delconte C, Pisano F (2013) Robot-aided neurorehabilitation in sub-acute and chronic stroke: does spontaneous recovery have a limited impact on outcome? NeuroRehabilitation 33(4):621–629

    Google Scholar 

  70. Cirstea MC, Levin MF (2000) Compensatory strategies for reaching in stroke. Brain 123:940–953

    Article  Google Scholar 

  71. Yeunga L, Chengb KC, Fonga C, Leea WC, Tonga K-Y (2013) Evaluation of the Microsoft Kinect as a clinical assessment tool of body sway. Gait Posture 40(4):532–538

    Article  Google Scholar 

  72. Hu MC, Chen CW, Cheng WH, Chang CH, Lai JH, Wu JL (2014) Real-time human movement retrieval and assessment with kinect sensor. In: Proceedings of IEEE transactions cybernetics

  73. Rikakis T (2011) Utilizing media arts principles for developing effective interactive neurorehabilitation systems. In: 33rd annual international conference of the IEEE EMBS, Boston, Massachusetts, USA

  74. Malosio M, Caimmi M, Legnani G, Molinari Tosatti L (2014) LINarm: a low-cost variable stiffness device for upper-limb rehabilitation. In: 2014 IEEE/RSJ international conference on intelligent robots and systems, Chicago

Download references

Acknowledgments

The presented work is carried out within RIPRENDO@Home Project, regional research project funded inside the Framework Agreement between Regione Lombardia and National Research Council, D.G.R. n. 3728 - 11/07/2012.

Author information

Authors and Affiliations

  1. Institute of Industrial Technologies and Automation, National Research Council, Milan, Italy

    Stefano Mottura, Luca Fontana, Sara Arlati, Andrea Zangiacomi, Claudia Redaelli & Marco Sacco

Authors
  1. Stefano Mottura
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Luca Fontana
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Sara Arlati
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Andrea Zangiacomi
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Claudia Redaelli
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Marco Sacco
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Stefano Mottura.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Mottura, S., Fontana, L., Arlati, S. et al. A virtual reality system for strengthening awareness and participation in rehabilitation for post-stroke patients. J Multimodal User Interfaces 9, 341–351 (2015). https://doi.org/10.1007/s12193-015-0184-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12193-015-0184-5

Keywords

Search

Navigation