Skip to main content
Log in

Skills assessment metrics of electric powered wheelchair driving in a virtual environment: a survey

  • Review Article
  • Published:
Medical & Biological Engineering & Computing Aims and scope Submit manuscript

Abstract

The purpose of this review is to present studies on the parameters for assessing the skills of users of electric wheelchair driving simulators in a virtual environment. In addition, this study also aims to identify the most widely used and validated parameters for the quantification of electric wheelchair driving ability in a virtual environment and to suggest challenges for future research. To carry out this research, the criteria of the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) were adopted. Literature searches in English, French, and Portuguese were conducted up to December 2020 in the PubMed, SciELO, Science Direct, World Wide Science, and Scopus databases. The keywords used were electric wheelchair, simulator, performance indicators, performance skills, driving skills, training platform, virtual environment, and virtual reality. We excluded studies involving “real” wheelchairs without a simulator in a virtual environment. We have selected a total of 42 items. In these studies, we identified 32 parameters (3 qualitative and 29 quantitative) that are used as parameters for the evaluation of the ability to control a powered wheelchair in a virtual environment. Although the amount of research in this area has increased significantly in recent years, additional studies are still needed to provide a more accurate and objective assessment of skills among the target population. A challenge for future work is the increasing application of artificial intelligence techniques and the exploration of biomedical data measurements, which may be a promising alternative to improve the quantification of user competencies.

Graphical abstract

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

Data availability

All data generated or analyzed during this study are included in this published article.

Abbreviations

ECG:

Electrooculography

EEG:

Electroencephalography

EMG:

Electromyography

EPW:

Electric powered wheelchair

GSR:

Galvanic skin response

PCI:

Physiological cost index

PMRT:

Power mobility road test

PRISMA:

Preferred Reporting Items for Systematic Reviews and Meta-Analyses

QoE:

Quality of experience

References

  1. Diez P, 2018 Smart wheelchairs and brain-computer interfaces. Elsevier - Academic Press

  2. Zolotas M, Elsdon J, and  Demiris Y, 2018“Head-mounted augmented reality for explainable robotic wheelchair assistance,” in International Conference on Intelligent Robots and Systems, 1823–1829.

  3. Guillaume Vailland Y. G, Grzeskowiak F, Louise Devigne, R. L. B. Bastien Fraudet, Emilie Leblong, Florian Nouviale, Francois Pasteau, Sylvain Guegan M. B, Gouranton V, Arnaldi B, 2019 “User-centered design of a multisensory power wheelchair simulator: towards training and rehabilitation applications,” in IEEE 16th International Conference on Rehabilitation Robotics (ICORR)

  4. Škrabaa A, Stojanovi R, Zupanc A, Koložvari, and Kofjacˇ D, 2015“Speech-controlled cloud-based wheelchair platform for disabled persons,” Microprocess. Microsyst., 17, 48

  5. Rabhi Y, Mrabet M, Fnaiech F (2018) A facial expression controlled wheelchair for people with disabilities. Comput Methods Programs Biomed 165:89–105

    Article  Google Scholar 

  6. Silva A. N, Morère Y, Naves E. L. M, Sa A. A. R , Soares A. B, 2013 “Virtual electric wheelchair controlled by electromyographic signals,” in 2013 ISSNIP Biosignals and Biorobotics Conference: Biosignals and Robotics for Better and Safer Living (BRC)

  7. Borges LR, Martins FR, Naves EL (2016) Electric-powered wheelchair control using eye tracking techniques. Int J Innov Res Comput Commun Eng 4(9):16690–16695

    Article  Google Scholar 

  8. Fehr L, Langbein W, Skaar S (2000) Adequacy of power wheelchair control interface for persons with severe disabilities: a clinical survey. J Rehabil Res Dev 37(1):353–360

    CAS  PubMed  Google Scholar 

  9. Barea R. Boquete L. Lopes E, Mazo M, 2000 “Guidance of a wheelchair using electrooculography,” in IEEE 4th World CSCC (Circuits, Systems, Communications & Computers, CSCC 2000)

  10. Wästlund E, Sponseller K, Pettersson O, Bared A (2015) Evaluating gaze-driven power wheelchair with navigation support for persons with disabilities. J Rehabil Res Dev 52:815–826

    Article  Google Scholar 

  11. Martins F. R, 2017 “Simulador para treinamento de cadeirantes em ambiente virtual acionado por comandos musculares e/ou visuais,” Universidade Federal de Uberlândia

  12. Martins F. R, Salgado D. P, Naves E. L. M, 2016 “Realidade Virtual e tecnologia assistiva: ambiente seguro para treinamento de cadeirantes controlado por sinais eletromiográficos,” in IX Simpósio em Engenharia biomédica, 86–89.

  13. Simpson R, Lopresti E, Cooper R (2008) How many people would benefit from a smart wheelchair. J Rehabil Res Dev 45(1):53–72

    Article  Google Scholar 

  14. Wood JM, Worringham C, Kerr G, Mallon K, Silburn P (2005) Quantitative assessment of driving performance in Parinson´s disease. J Neurol Neurosurg Psychiatry 76:176–180

    Article  CAS  Google Scholar 

  15. Morère Y, Bourhis G, Cosnuau K, Guilmois G, Rumilly E,  Blangy E, 2018 “ViEW: a wheelchair simulator for driving analysis,” Assist. Technol. Off. J. RESNA, 1–11

  16. Lee Kirby FRR, Miller WC, Louise Demers JMP, Alex Mihailidis MM, Rushton PW, Titus L, Cher Smith BS, Theriault C, Thompson K (2015) Effectiveness of a wheelchair skills training program for powered wheelchair users: a randomized controlled trial. Arch Phy. Med Rehabil 9(6):2017–2026

    Article  Google Scholar 

  17. Moghaddam A. K et al., 2011 “Mobility profile and wheelchair driving skills of powered wheelchair users: sensor-based event recognition using a support vector machine classifier,” in 33rd Annual International Conference of the IEEE EMBS,  7336–7340.

  18. Erren-Wolters CV (2007) Virtual reality for mobility devices: training applications and clinical results: a review. Int journ rehab res 30:91–96

    Article  Google Scholar 

  19. Pithon K (2009) Weiss, Richir, “Wheelchairs simulators: a review.” Technol Disabil 21:1–10

    Article  Google Scholar 

  20. Niniss H, Nadif A, 2000 “Simulation of the behaviour of a powered wheelchair using virtual reality,” in 3rd International Conference on Disability, Virtual Reality and Associated Technologies, 9–14.

  21. Spaeth D, Cooper R, Guo S, Chaves E (2002) The application of optical sensors for quantifying electric powered wheelchair driving skill. RESNA 2002:258–260

    Google Scholar 

  22. Kamaraj DC, Dicianno BE, Schmid M, Boyanoski T, Cooper RA (2014) Quantifying power wheelchair driving ability. Conference Proceedings RESNA 2014:1–4

    Google Scholar 

  23. Archambault P, Blackburn E, Reid D, Routhier F, Miller W. C,  2015“Development and user validation of driving tasks for a power wheelchair simulator,” in International Conference on Virtual Rehabilitation (ICVR)

  24. Nunnerley J, 2016 “Training wheelchair navigation in immersive virtual environments for patients with spinal cord injury,” Disabil. Rehabil. Assist. Technol

  25. Inman DP, Loge K, Cram A, Peterson M (2011) Learning to drive a wheelchair in virtual reality. J Spec Educ Technol 26(3):21–34

    Article  Google Scholar 

  26. Ktena S. I, Abbot W, Faisal A. A, 2015 “A virtual reality platform for safe evaluation and training of natural gaze-based wheelchair driving,” in 7th International IEEE/EMBS Conference on Neural Engineering Montpellier, 236–239

  27. Abellard P, 2010 “Electric wheelchair navigation simulators: why, when, how?,” Mechatron. Syst. Appl., 161–168

  28. Arlati S, Colombo V, Ferrigno G, Sacchetti R,  Sacco M, 2019 “Virtual reality-based wheelchair simulators: a scoping review,” Assist. Technol., 1–12

  29. Moher D, Liberati A, Tetzlaff J, and D. G,  2009 “Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement,” BMJ, vol. 339, no. b2535, 1–8

  30. Moher D, Liberati A, Tetzlaff J, Altman DG (2009) Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med 6(7):e1000097

    Article  Google Scholar 

  31. Brunnhuber K, Chalmers I, Chalkidou K, Clarke M (2006) How to formulate research recommendations. BMJ 333(7572):804–806

    Article  Google Scholar 

  32. Archambault PS (2012) Driving performance in a power wheelchair simulator. Disabil Rehabil Assist Technol 7(3):226–233

    Article  Google Scholar 

  33. Grant M, Harrison C, Conway B, 2010 “The enhancement of a virtual reality wheelchair simulator to include qualitative and quantitative performance metrics,” J. Assist. Technol., 20–31

  34. Crichlow L. R, 2011 “Development of a comprehensive mathematical model and physical interface for manual wheelchair simulation,” University of Toronto

  35. Harrison CS, Grant M, Conway BA (2004) Haptic interfaces for wheelchair navigation in the built environment. Presence 5:520–534

    Article  Google Scholar 

  36. Buxbaum LJ, Palermo MA, Mastrogiovann D, Schmidt M (2008) Assessment of spatial attention and neglect with a virtual wheelchair navigation task. J Clin Exp Neuropsychol Publ details Incl Instr 0(6):650–660

    Article  Google Scholar 

  37. Archambault P, Gagnon D, Routhier F, Miller W (2016) Effectiveness of power wheelchair simulator training, delivered at home, on wheelchair driving skills. Ann Phys Rehabil Med 59:37–38

    Article  Google Scholar 

  38. Caetano D et al (2020) Proposal of an augmented reality telerehabilitation system for powered wheelchair user´s training. J Commun Inf Syst 35(1):51–60

    Google Scholar 

  39. Kamaraj D.C, Dicianno B.E, Mahajan H. P, Buhari A. M, Cooper R.A. 2016 “Inter-rater reliability of the power mobility road test in the virtual reality based simulator-2,” Arch. Phys. Med. Rehabil., 2, 5

  40. Kamaraj D.C, Mahajan H, Terhorst L, 2016 “Discriminative ability of the quantitative electric powered wheelchair driving metrics in VRSIM-2,” Arch. Phys. Med. Rehabil., 47

  41. Valentini C.A.M, 2019 “Protocolo para condução de cadeira de rodas motorizada usando realidade virtual,” Universidade Federal de Uberlandia

  42. Stredney D, Yagel R, Carlson W, Möller T, Shih P.W, Fontana M. 1997 “Assessing user proficiency through virtual simulations,” in Proceedings of RESNA’97, annual meeting of the Rehabilitation Engineering and Assistive Technology Society of North America, 366–368.

  43. Zatlaa K.C.H, Morère Y, Hadj-Abdelkader A, Bourhis G, Demet K, Guilmois G, Bigaut N, 2018 “Preview distance index for the analysis of powered wheelchair driving,” IRBM

  44. Niniss H, Nadif A, 2003 “Simulation system for powered wheelchairs: evaluation of driving skills using virtual reality,” in Assistive Technology: Shaping the Future : AAATE’03, 112–116.

  45. Salgado D, Rodrigues T, Martins F, Keighrey C. 2018 “A QoE assessment method based on EDA, heart rate and EEG of a virtual reality assistive technology system,” in Proceedings of the 9th ACM Multimedia Systems Conference, 517–520.

  46. Borges L.R, Martins F. R, Naves E, Bastos T, Lucena V. 2016 “Multimodal system for training at distance in a virtual or augmented reality environment for users of electric-powered wheelchairs,” in International Federation of Automatic Control, 156–160.

  47. Gacem A, Monacelli E, Wang T, Rabreau O, Al‑ani T, 2019 “Assessment of wheelchair skills based on analysis of driving style,” Cogn. Technol. Work, pp. 1–15

  48. Spaeth D, Mahajan H, Karmarar A, Collins D, Cooper R, Boninger M, 2008 “Development of a wheelchair virtual driving environment: trials with subjects with traumatic brain injury,” Arch Phys Med Rehabil., pp. 996–1003

  49. Morére Y, Abdelkader H, Cosnuau K, Guilmois G,  Bourhis G, 2015 “Haptic control for powered wheechair driving assistance,” IRBM, no. 293–304

  50. Archambault P.S, Chong J.N.F, Sorrento G, Routhier F, Boissy P, 2011 “Comparison of powered wheelchair driving performance in a real and in a simulated environment,” in International Conference on Virtual Rehabilitation

  51. Silva Y, Souza V, Naves E, Filho T, V. L. Jr;, 2018 “Teleoperation training environment for new users of electric powered wheelchairs,” in The 8th International Conference on Current and Future Trends of Information and Communication Technologies in Healthcares (ICTH 2018), 343–350.

  52. Cooper RA et al (2002) Driving characteristics of electric-powered wheelchair users: how far, fast and often do people drive? Arch Phys Med Rehabil 83:250–255

    Article  Google Scholar 

  53. Niniss H, Inoue T (2006) Assessment of driving skills using virtual reality: comparative survey on experts and unskilled users of electric wheelchairs. Technol Disabil 18:217–226

    Article  Google Scholar 

  54. Mahajan H.P, 2012 “Development and validation of simulators for power wheelchair driving evaluations,” University of Pittsburgh

  55. Marchuk N.D, Ding D, Gaukrodger S, 2007 “Development of a virtual platform for assessment and training of power wheelchair driving,” in 30th RESNA International Conference, 2007.

  56. Mahajan HP, Dicianno BE, Cooper RA, Ding D (2013) Assessment of wheelchair driving performance in a virtual reality-based simulator. J Spinal Cord Med 6(4):322–333

    Article  Google Scholar 

  57. Liu L, Wang J, Chen W. 2014 “A virtual simulation and driver evaluation platform for smart wheelchairs,” in Communications in Computer and Information Science, 307–318.

  58. Majdolashrafi M, Ahmadabadi M.N, Ghazavi A, 2002 “A desktop virtual environment to train motorized wheelchair driving,” in IEEE International Conference on Systems, Man and Cybernetics

  59. Niniss H, Inoue T. 2006 “Electric wheelchair simulator for rehabilitation of persons with motor disability,” in Sysmposium on Virtual Reality VIII

  60. Linden MA, Whyatt C, Craig C, Kerr C (2013) Efficacy of a powered wheelchair simulator for school aged children: a randomised controlled trial. Rehabil Psychol 58(4):405–411

    Article  Google Scholar 

  61. Carlozzia NE, Gadeb V, Rizzoc A, Tulskya DS (2015) Using virtual reality driving simulators in persons with spinal cord injury: three screen display versus head mounted display. Disabil Rehabil Assist Technol 8(2):176–180

    Article  Google Scholar 

  62. Alshaer A, Regenbrecht H, O’Hare D (2017) Immersion factors affecting perception and behaviour in a virtual reality power wheelchair simulator. Appl Ergon 58:1–12

    Article  Google Scholar 

  63. John N, Pop S, Day T, Ritsos P, Headleand C (2018) The implementation and validation of a virtual environment for training powered wheelchair manoueuvres. IEEE Trans Vis Comput Graph 24(5):1867–1877

    Article  Google Scholar 

  64. De Santis A, Di Gironimo G, Marzano A, Siciliano B, Tarallo A, 2008 “A virtual-reality-based evaluation environment for wheelchair-mounted manipulators,” in Eurographics Italian Chapter Conference, 2008, pp. 1–8.

  65. Hernandez-Ossa  K, Longo B, Montenegro-Couto E, Romero-Laiseca A, Bastos-FIlho T, 2017 “Development and pilot test of a virtual reality system for electric powered wheelchair simulation,” in IEEE International Conference on Systems, 2355–2360.

  66. Hernandez-Ossa K, Montenegro-Couto E, Longo B, Bastos-FIlho T, 2019“Virtual reality simulator for electric powered wheelchairs using a joystick,” in XXVI Brazilian Congress on Biomedical Engineering,  729–736.

  67. Zatla H, Hadj-Abdelkader A, Morere Y, Bourhis G. 2015 “OPCM model application on a 3D simulator for powered wheelchair,” in International Conference on Virtual Rehabilitation (ICVR), 131–132.

  68. Randria I, Abellard A, Ben Khelifa M, Abellard P, Ramanantsizehena P, 2008 “Evaluation of trajectory applied to collaborative rehabilitation for a wheelchair driving simulator,” in IFMBE Proceeedings,  1843–1846.

  69. Onyango S.O, Hamam Y, Djouani K, Daachi B, Steyn N, 2016 “A driving behaviour model of electrical wheelchair users,” Comput. Intell. Neurosci., 1–20

  70. Kalawsky RS (1999) VRUSE–a computerised diagnostic tool: for usability evaluation of virtual/synthetic environment systems. Appl Ergon 30:11–25

    Article  CAS  Google Scholar 

  71. MacGillivray M, Sawatzy B, Miller W, Routhier F, Kirby L,  2017 “Goal satisfaction improves with individualized powered wheelchair skills training,” Disabil Rehabil Assist Technol, 1–4

  72. Massengale S, Folden D, McConnell P, Stratton L, Whitehead V (2010) Effect of visual perception, visual function, cognition, and personality on power wheelchair use in adults. Assist Technol 17(2):108–121

    Article  Google Scholar 

  73. Martins FR, Naves ELM, Morère Y, de Sá AAR (2021) Preliminary assessment of a multimodal electric-powered wheelchair simulator for training of activities of daily living. J Multimodal User Interfaces 21:1–29

    Google Scholar 

  74. Javanmardi S, Bideaux E, Trégouët J.F, Trigui R, Tattegrain H, NicouleauBourles E. 2017 “Driving style modelling for eco-driving applications,” in The 20th World congress of the international federation of automatic control, 13866–13871.

  75. Egan E, Brennan S, Barrett J, Qiao Y, Murray N. 2016 “An evaluation of heart rate and electrodermal activity as an objective QoE evaluation method for immersive virtual reality environments,” in 8th International Conference on Quality of Multimedia Experience (QoMEX)

  76. D. M. Gabrielli F, Schiro J, Pudlo P, Bouilland S, Thévenon A 2012 “Indicateur pour différencier les conducteurs automobiles durant des manoeuvres à basse vitesse,” in Handicap 2012 conference, 93–103.

  77. D. Y. Carlson T 2008 “Collaborative control in human wheelchair interaction reduces the need for dexterity in precise manoeuvres,” in Robotic helpers: user interaction, interfaces and companions in assis-tive and therapy robotics”, a workshop at ACM/IEEE HRI, 59–66.

  78. S.-T. I. Urdiales C, Peula J, Fdez-Carmona M, Barrué C, Pérez E, 2010 “A new multi-criteria optimization strategy for shared control in wheelchair assisted navigation,” Aut. Robot., 30, 2, 179–197

  79. Li Q, Chen W, Wang J. 2011 “Dynamic shared control for human-wheelchair cooperation,” in IEEE International Conference on Robotics and Automation

  80. Hershey JR, Olsen PA (2007) Approximating the Kullback Leibler divergence between Gaussian mixture models. ICASSP 4:317–320

    Google Scholar 

  81.  Faria B.M, 2013“Intelligent wheelchair simulator for users’ training: cerebral palsy children’s case study,” in 8th Iberian Conference on Information Systems and Technologies,  1–6.

  82. Kirby R.L. 2017 Wheelchair skills assessment and training. CRC Press

  83. Borges LR, Naves ELM, Sa AAR (2021) Usability evaluation of an electric-powered wheelchair driven by eye tracking. Univers Access Inf Soc 1(1):1–20

    Google Scholar 

  84. FDA 2016 “Applying human factors and usability engineering to medical devices,” 

  85. Bertin R.J.V, Guillot A, Collet C, Vienne F, Espié S, Graf W. 2004 “Objective measurement of simulator sickness and the role of visual-vestibular conflict situations: a study with vestibular-loss (a-reflexive) subjects,” in Proceeding Neuroscience,  30.

  86. Affanni PZA, Bernardini R, Piras A, Rinaldo R (2018) Driver’s stress detection using skin potential response signals. Measurement 122:264–274

    Article  Google Scholar 

  87. Lanatà FBA, Valenza G, Greco A, Gentili C, Bartolozzi R, Frendo EPSF (2014) How the autonomic nervous system and driving style change with incremental stressing conditions during simulated driving. IEEE Trans Intell Transp Syst 16(3):1505–1507

    Article  Google Scholar 

  88. Hernandez R.J, McDuff D, Benavides X, Amores J, Maes P, Picard, 2014“Autoemotive: bringing empathy to the driving experience to manage stress,” in Proceedings of the 2014 companion publication on Designing interactive system, 253–56.

  89. Healey JA, Picard RW (2015) Detecting stress during real-world driving tasks using physiological sensors. IEEE Trans Intell Transp Syst 6(2):156–166

    Article  Google Scholar 

  90. Subasi A. 2019“Biomedical signal analysis and its usage in healthcare,” in Biomedical Engineering and its Applications in Healthcare, S. Paul, Ed. Springer,

  91. Cerutti S, Marchesi C. 2011, Advanced methods of biomedical signal processing. Wiley-IEEE,

  92. WSP, 2021 “Wheelchair skills program (WSP) manual and forms,” 

  93. Kirby RL, Swuste J, Dupuis DJ, MacLeod DA, Monroe R (2002) The wheelchair skills test: a pilot study of a new outcome measure. Arch Phys Med Rehabil 83(1):10–18

    Article  Google Scholar 

  94. Letts L.J, Dawson D. R, Gleason J. 2007 “Reliability and validity of the power-mobility community driving assessment,” Assist. Technol. Off. J. RESNA, pp. 1–8

  95. Dawson D. R,  2006 Power-mobility indoor driving assessment manual (PIDA). Toronto-Canadá: Sunnybrook and Wonmen´s College Health Sciences Centre

  96. Hernandez-Ossa K et al (2020) Simulation system of electric-powered wheelchairs for training purposes. Sensors 20:3565

    Article  Google Scholar 

  97. Pithon T, Weiss T, Richir S, Klinger E, 2016 “Wheelchair simulators: a review,” Technol. Disabil., 1–7

  98. Day T.W, Dobson W.H, Headleand C.J, John N.W, Pop S.R. 2017“Using virtual reality to experience different powered wheelchair configurations,” in 2017 International Conference on Cyberworlds (CW)

  99. Gerling K.M, Mandryk R.L, Kalyn M.R, 2013 “Wheelchair-based game design for older adults,” in ASSETS ’13: Proceedings of the 15th International ACM SIGACCESS Conference on Computers and Accessibility, 1–8.

Download references

Funding

This study was financed in part by the Coordenação de Aperfeiçoamento de Pessoal de Nível Superior – Brasil (CAPES). Finance code 001.

Author information

Authors and Affiliations

Authors

Contributions

AARS was involved in drafting the manuscript and revising it critically for important intellectual content. YM and ELMN supervised, revised, and gave the final approval of the manuscript. All authors were fully involved in the study and preparation of the manuscript. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Angela A. R. de Sá.

Ethics declarations

Ethics approval and consent to participate

Not applicable.

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

de Sá, A.A.R., Morère, Y. & Naves, E.L.M. Skills assessment metrics of electric powered wheelchair driving in a virtual environment: a survey. Med Biol Eng Comput 60, 323–335 (2022). https://doi.org/10.1007/s11517-022-02500-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11517-022-02500-8

Keywords

Navigation