Skip to main content
SpringerLink
Log in

Performance-based outcome measures of upper extremity in virtual reality and telerehabilitation: a systematic review

  • Review Article
  • Published:
Neurological Sciences Aims and scope Submit manuscript

Abstract

Objectives

This systematic review aimed to identify and synthesize evidence regarding measurement properties of upper extremity performance-based outcome measures performed in virtual reality or in telerehabilitation to inform clinical applications and research endeavors.

Methods

Five bibliographic databases, PubMed, Embase, CINAHL, APA PsycINFO, and Scopus, were searched on July 12, 2023. Studies assessing the measurement properties (reliability, validity, responsiveness) of upper extremity performance-based outcome measures in virtual reality and telerehabilitation were eligible for inclusion. The COSMIN risk of bias checklist was used for methodological quality assessment. Study selection, data extraction, and quality assessment were completed by two independent reviewers.

Results

A total of 240 records were identified from the five databases. Nine cross-sectional studies published from 2016 to 2023 were included. Participants included 210 patients with neurological conditions and 184 healthy subjects. In virtual reality studies, four implemented the Box and Block Test, one the Arm Research Action Test, and one the Peg Insertion Test. In telerehabilitation studies, three implemented the Fugl-Meyer Assessment Upper Extremity. For quality assessment, one study was rated inadequate, one was rated doubtful, and all others demonstrated adequate to good quality. Most studies demonstrated good test-retest reliability and concurrent validity to the original in-person assessments.

Conclusion

Implementing upper extremity performance-based outcome measures in virtual reality and telerehabilitation is feasible and promising. Further studies are warranted to develop and refine remote assessment paradigms and validate them on a larger scale to inform clinical application and promote digital health in rehabilitation.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

References

  1. O'Sullivan SB, Schmitz TJ, Fulk G. Physical rehabilitation. 2019.

    Google Scholar 

  2. Ahmadvand A, Kavanagh D, Clark M, Drennan J, Nissen L (2019) Trends and visibility of “digital health” as a keyword in articles by JMIR publications in the new millennium: bibliographic-bibliometric analysis. J Med Internet Res 21:e10477

    Article  PubMed  PubMed Central  Google Scholar 

  3. Keshner EA, Weiss PT, Geifman D, Raban D (2019) Tracking the evolution of virtual reality applications to rehabilitation as a field of study. J Neuroeng Rehabil 16:1–15

    Article  Google Scholar 

  4. Bohil CJ, Alicea B, Biocca FA (2011) Virtual reality in neuroscience research and therapy. Nat Rev Neurosci 12:752–762

    Article  CAS  PubMed  Google Scholar 

  5. Peretti A, Amenta F, Tayebati SK, Nittari G, Mahdi SS (2017) Telerehabilitation: review of the state-of-the-art and areas of application. JMIR Rehab Assist Technol 4:e7511

    Google Scholar 

  6. Suso-Martí L, La Touche R, Herranz-Gómez A, Angulo-Díaz-Parreño S, Paris-Alemany A, Cuenca-Martínez F (2021) Effectiveness of telerehabilitation in physical therapist practice: an umbrella and mapping review with meta–meta-analysis. Phys Ther 101:pzab075

    Article  PubMed  Google Scholar 

  7. Jin M, Pei J, Bai Z, Zhang J, He T, Xu X, Zhu F, Yu D, Zhang Z (2022) Effects of virtual reality in improving upper extremity function after stroke: a systematic review and meta-analysis of randomized controlled trials. Clin Rehabil 36:573–596

  8. Mekbib DB, Han J, Zhang L, Fang S, Jiang H, Zhu J, Roe AW, Xu D (2020) Virtual reality therapy for upper limb rehabilitation in patients with stroke: a meta-analysis of randomized clinical trials. Brain Inj 34:456–465

    Article  PubMed  Google Scholar 

  9. Prinsen CA, Mokkink LB, Bouter LM, Alonso J, Patrick DL, De Vet HC, Terwee CB (2018) COSMIN guideline for systematic reviews of patient-reported outcome measures. Qual Life Res 27:1147–1157

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  10. Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, Shamseer L, Tetzlaff JM, Moher D (2021) Updating guidance for reporting systematic reviews: development of the PRISMA 2020 statement. J Clin Epidemiol 134:103–112

    Article  PubMed  Google Scholar 

  11. Mokkink LB, De Vet HC, Prinsen CA, Patrick DL, Alonso J, Bouter LM, Terwee CB (2018) COSMIN risk of bias checklist for systematic reviews of patient-reported outcome measures. Qual Life Res 27:1171–1179

    Article  CAS  PubMed  Google Scholar 

  12. Koo TK, Li MY (2016) A guideline of selecting and reporting intraclass correlation coefficients for reliability research. J Chiropr Med 15:155–163

    Article  PubMed  PubMed Central  Google Scholar 

  13. Portney LG. Foundations of clinical research: applications to evidence-based practice. 2020.

    Google Scholar 

  14. Burton Q, Lejeune T, Dehem S, Lebrun N, Ajana K, Edwards MG, Everard G (2022) Performing a shortened version of the Action Research Arm Test in immersive virtual reality to assess post-stroke upper limb activity. J Neuroeng Rehabil 19:1–12

    Article  Google Scholar 

  15. Dong Y, Liu X, Tang M, Huo H, Chen D, Wu Z, An R, Fan Y (2023) A haptic-feedback virtual reality system to improve the box and block test (BBT) for upper extremity motor function assessment. Virtual Reality 27:1199–1219

    Article  Google Scholar 

  16. Everard G, Otmane-Tolba Y, Rosselli Z, Pellissier T, Ajana K, Dehem S, Auvinet E, Edwards MG, Lebleu J, Lejeune T (2022) Concurrent validity of an immersive virtual reality version of the Box and Block Test to assess manual dexterity among patients with stroke. J Neuroeng Rehabil 19:7

    Article  PubMed  PubMed Central  Google Scholar 

  17. Carmona C, Sullivan JE, Arceo R, Drogos J, Besser S, Gutierrez S, Jeteric Z, Wyman J, Yao J (2023) Development and preliminary validity study of a modified version of the upper extremity Fugl-Meyer assessment for use in telerehabilitation. J Neurol Phys Ther 47:208–216

  18. Liz L, da Silva TG, Michaelsen SM (2023) Validity, reliability, and measurement error of the remote Fugl-Meyer assessment by videoconferencing: tele-FMA. Phys Ther 103:pzad054

  19. Rowe V, Blanton S, Aycock D, Hayat MJ, Ali SZ (2023) Remote delivery of the Fugl-Meyer assessment for the upper extremity: a pilot study to assess feasibility, reliability, and validity. Arch Rehab Res Clin Transl 5:100261

    Google Scholar 

  20. Tobler-Ammann BC, De Bruin ED, Fluet M, Lambercy O, de Bie RA, Knols RH (2016) Concurrent validity and test-retest reliability of the Virtual Peg Insertion Test to quantify upper limb function in patients with chronic stroke. J Neuroeng Rehabil 13:1–14

    Article  Google Scholar 

  21. Oña ED, Jardón A, Cuesta-Gómez A, Sánchez-Herrera-Baeza P, Cano-de-la-Cuerda R, Balaguer C (2020) Validity of a fully-immersive VR-based version of the box and blocks test for upper limb function assessment in Parkinson’s disease. Sensors 20:2773

    Article  ADS  PubMed  PubMed Central  Google Scholar 

  22. Alvarez-Rodríguez M, López-Dolado E, Salas-Monedero M, Lozano-Berrio V, Ceruelo-Abajo S, Gil-Agudo A, de los Reyes-Guzmán A. (2020) Concurrent validity of a virtual version of Box and Block Test for patients with neurological disorders. World. J Neurosci 10:79

    Google Scholar 

  23. Oliveira CS, Almeida CS, Freias LC, Santana R, Fernandes G, Fonseca Junior PR, Moura RCF (2016) Use of the Box and Block Test for the evaluation of manual dexterity in individuals with central nervous system disorders: a systematic review. Manual Ther, Posturol Rehab J 14:0

    Google Scholar 

  24. Hernandez ED, Galeano CP, Barbosa NE, Forero SM, Nordin Å, Sunnerhagen KS, Murphy MA (2019) Intra-and inter-rater reliability of Fugl-Meyer assessment of upper extremity in stroke. J Rehabil Med 51:652–659

    Article  PubMed  Google Scholar 

  25. Lewis JR (2018) The system usability scale: past, present, and future. Int J Human–Comp Inter 34:577–590

    Google Scholar 

  26. Chen H, Chen CC, Hsueh I, Huang S, Hsieh C (2009) Test-retest reproducibility and smallest real difference of 5 hand function tests in patients with stroke. Neurorehabil Neural Repair 23:435–440

    Article  PubMed  Google Scholar 

  27. Parker SM, Andreasen SC, Ricks B, Kaipust MS, Zuniga J, Knarr BA (2022) Comparison of brain activation and functional outcomes between physical and virtual reality box and block test: a case study. Disab Rehab Assist Technol. https://doi.org/10.1080/17483107.2022.2085334

  28. Wang Y, Bohannon RW, Kapellusch J, Garg A, Gershon RC (2015) Dexterity as measured with the 9-Hole Peg Test (9-HPT) across the age span. J Hand Ther 28:53–60

    Article  PubMed  Google Scholar 

  29. Hao J, He Z, Yu X, Remis A (2023) Comparison of immersive and non-immersive virtual reality for upper extremity functional recovery in patients with stroke: a systematic review and network meta-analysis. Neurol Sci 44:2679–2697

  30. Woytowicz EJ, Rietschel JC, Goodman RN, Conroy SS, Sorkin JD, Whitall J, Waller SM (2017) Determining levels of upper extremity movement impairment by applying a cluster analysis to the Fugl-Meyer assessment of the upper extremity in chronic stroke. Arch Phys Med Rehabil 98:456–462

    Article  PubMed  Google Scholar 

  31. Wang I, Li P, Lee S, Lee Y, Wang C, Hsieh C (2023) Development of a Berg Balance Scale short-form using a machine learning approach in patients with stroke. J Neurol Phys Ther 47:44–51

    Article  CAS  PubMed  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Health & Rehabilitation Sciences, College of Allied Health Professions, University of Nebraska Medical Center, Omaha, NE, 68198, USA

    Jie Hao, Yanfei Li & Yuqi Pu

  2. Health Research Association of Keck Medicine, University of Southern California, Los Angeles, CA, 90033, USA

    Andréas Remis

  3. Department of Epidemiology, Bloomberg School of Public Health, Johns Hopkins University, Baltimore, MD, 21205, USA

    Zhengting He

  4. Department of Rehabilitation Medicine, Beijing Hospital, National Center of Gerontology, Institution of Geriatric Medicine, Chinese Academy of Medical Science, Beijing, 100051, People’s Republic of China

    Zixuan Yao

Authors
  1. Jie Hao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yanfei Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Andréas Remis
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Zhengting He
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Zixuan Yao
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Yuqi Pu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jie Hao.

Ethics declarations

Ethics approval

Ethical approval is not applicable to this review paper.

Conflict of interest

The authors declare no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendix. Search strategy

Appendix. Search strategy

PubMed

(“Upper extremit*” OR “Upper Extremity”[Mesh]) AND (Assessment OR evaluation OR measurement OR test* OR “Outcome Assessment, Health Care”[Mesh]) AND (“Virtual reality” OR “telehealth” OR “telerehabilitation” OR “telemedicine” OR “videoconferenc*” OR “Virtual Reality”[Mesh] OR “Telerehabilitation”[Mesh]) AND (“Validity” OR “reliability” OR “Psychometrics”[Mesh] OR “Reproducibility of Results”[Mesh])

Embase

('upper limb'/exp/mj OR 'upper limb' OR 'upper limb':ti,ab OR 'upper extremit*':ti,ab) AND (assessment:ti,ab OR evaluation:ti,ab OR measurement:ti,ab OR test*:ti,ab OR 'outcome measure':ti,ab) AND (validity:ti,ab OR reliability:ti,ab) AND ('virtual reality'/exp/mj OR 'virtual reality' OR 'telerehabilitation'/exp/mj OR telerehabilitation OR 'virtual reality':ti,ab OR telerehabilitation:ti,ab OR telehealth:ti,ab OR telemedicine:ti,ab OR videoconference*:ti,ab)

Scopus

( TITLE-ABS-KEY ( “Upper extremit*” ) AND TITLE-ABS-KEY ( assessment OR evaluation OR measurement OR test* OR “outcome measure” ) AND TITLE-ABS-KEY ( validity OR reliability ) AND TITLE-ABS-KEY ( “Virtual reality” OR telehealth OR telerehabilitation OR telemedicine OR videoconferenc* ) )

CINAHL

“Upper extremit*” AND ( Assessment OR evaluation OR measurement OR test* ) AND ( “Virtual reality” OR “telehealth” OR “telerehabilitation” OR “telemedicine” OR “videoconferenc*” ) AND ( “Validity” OR “reliability” OR “Psychometrics” )

PsycINFO

“Upper extremit*” AND ( Assessment OR evaluation OR measurement OR test* ) AND ( “Virtual reality” OR “telehealth” OR “telerehabilitation” OR “telemedicine” OR “videoconferenc*” ) AND ( “Validity” OR “reliability” OR “Psychometrics” )

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Hao, J., Li, Y., Remis, A. et al. Performance-based outcome measures of upper extremity in virtual reality and telerehabilitation: a systematic review. Neurol Sci 45, 977–986 (2024). https://doi.org/10.1007/s10072-023-07158-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10072-023-07158-0

Keywords

Search

Navigation