1 Introduction

According to estimates of the World Health Organization, 75 million people in the world were using a wheelchair in 2018 (WHO 2018). Wheelchairs are needed for mobility in many different kinds of disabilities, among which are amputation and spinal cord injury (Archambault et al. 2012; Karmarkar et al. 2009; Singh et al. 2020; van den Akker et al. 2020), paralysis (Aarai et al. 2011; AlAbboudi et al. 2020; Fadhil et al. 2019; Kuntal et al. 2020; Ruzaij et al. 2017), cerebral palsy (Alkhateeb et al. 2019; Rodby-Bousquet and Hägglund 2010; Rozas Llontop et al. 2020), multiple sclerosis (Devitt et al. 2004; Revathi et al. 2008; Silveira et al. 2020), muscular dystrophy (Bayley et al. 2020; Richardson and Frank 2009; Xi et al. 2019), traumatic brain injury (Quilico et al. 2020; Spaeth et al. 2008).

Wheelchairs increase mobility of people with physical impairments (Borg et al. 2011; WHO 2018), enabling them to become more involved and active in their communities, with a significant improvement in their social participation (Brandt et al. 2004; Cooper et al. 2011; Davies et al. 2003; Domingues et al. 2019; Evans et al. 2007; Hosseini et al. 2012; Labbé et al. 2020; Miles-Tapping 1996; Rousseau-Harrison et al. 2009). Manual or powered mobility can have a great effect on the lives of people with mobility impairments. People using wheelchairs reported feeling empowered and more productive, enjoying more free time and self-care (Domingues et al. 2020; Miles-Tapping 1996).

On the other hand, wheelchair driving requires complex physical and cognitive abilities, such as vision, balance, spatial awareness and a general knowledge of the wheelchair characteristics and reaction times (Bigras et al. 2020; Morgan et al. 2017; Pithon et al. 2009). Thus, it is important for people with physical disabilities to manage all these skills. Wheelchair use has indeed many implications for the safety of the user as well as other people, and in terms of safeguarding objects in the environment (Field 1999; Frost et al. 2020). Some studies have highlighted that the risk of tipping over, falling and colliding with static or moving objects in the environment is not negligible (Corfman et al. 2003; Kirby et al. 1994; Rice et al. 2019; Xiang et al. 2006).

New wheelchair users report some difficulties in learning to drive the device, be it a powered wheelchair (PW) (Torkia et al. 2015) or a manual one (MW), especially when secondary pain conditions affect the upper limbs or hands (Briley et al. 2020; Dieruf et al. 2008; Walford et al. 2019). Therefore, training is usually required in order to improve independent mobility in users (Bigras et al. 2020; Keeler et al. 2019; R. Lee Kirby et al. 2015; Mountain et al. 2010a, b; Mountain et al. 2010a, b). Typically, such training aims to teach—with different levels of difficulty—abilities like obstacle avoidance, turning in open or thigh space, passing through doors, moving backward/forward while maneuvering, driving over different terrains, going up/down stairs and ramps. Such training on a real wheelchair can be potentially unsafe (Frost et al. 2020; Mountain et al. 2010a, b; Mountain et al. 2010a, b). By contrast, a virtual reality (VR)-based training can improve maneuvering abilities while ensuring user safety and strongly reducing the risk of injuries (Archambault et al. 2012; Archambault and Bigras 2019; Desbonnet 1998; A. Harrison et al. 2002; Hasdai et al. 1998; Nunnerley et al. 2017; Torkia et al. 2015; Webster et al. 2001).

With the increasing diffusion of VR technology (Arlati et al. 2019), different types of wheelchair simulators were developed in recent years. Most papers in the literature describe PW simulators (Archambault et al. 2012, 2017; Carlozzi et al. 2013; Desbonnet 1998; Devigne et al. 2017; Harrison et al. 2002; Hasdai et al. 1998; Harshal P. Mahajan et al. 2013; Harshal Prabhakar Mahajan 2013; Nunnerley et al. 2017; Panadero et al. 2014; Spaeth et al. 2008; Webster et al. 2001), while only few authors developed MW-specific solutions (Blouin et al. 2015; Chenier et al. 2014; Crichlow et al. 2011; Crichlow and Crichlow 2011; C. S. Harrison et al. 2000; Maxhall et al. 2004). Furthermore, to the best of our knowledge, no studies were published in the literature describing flexible systems that integrate both driving modalities.

Moreover, most simulators are based on non-immersive VR (Archambault et al. 2012, 2017; Blouin et al. 2015; Chenier et al. 2014; Desbonnet 1998; A. Harrison et al. 2002; Hasdai et al. 1998; Harshal Prabhakar Mahajan 2013; Spaeth et al. 2008; Webster et al. 2001), and thus, they do not take advantage of the increased engagement and motivation often related to the increased level of immersion (Kim and Biocca 2018).

Some papers demonstrated that VR training is effective in improving driving abilities (Archambault and Bigras 2019; Keeler et al. 2019), in particular virtual maneuverability tasks seemed to be a useful pre-training activity (Archambault et al. 2012; Desbonnet 1998; A. Harrison et al. 2002; Hasdai et al. 1998; Nunnerley et al. 2017; Webster et al. 2001). Training with non-immersive PW simulators revealed to make the participants able to control their PW in a real environment: Driving tasks were performed in the same way in the virtual environment as the real one, suggesting that similar driving skills were acquired (Archambault et al. 2012). Additionally, such simulators are also useful for training people to perform specific daily living tasks such as cooking, doing self-care activities or moving in small spaces such as a supermarket (Archambault et al. 2012). Since training proved effective regardless of the immersion level (Carlozzi et al. 2013; Harshal P. Mahajan et al. 2013), it is plausible to think that an increased immersion could make such simulators effective while also benefitting of higher engagement and motivation (Kim and Biocca 2018).

On the other hand, considering MW simulators developed in immersive VR, interaction with the virtual environment and control of the virtual wheelchair motion were based on specific hardware interfaces, such as rollers or encoders to traduce real wheel movement in virtual wheelchair motion (Crichlow and Crichlow 2011; . Harrison et al. 2000; Maxhall et al. 2004). Although hardware interface guarantees an appropriate force feedback useful for training, setup is rather complex and does not necessarily enable use of a personal wheelchair.

As a side effect of VR wheelchair simulators, the literature mentions cybersickness which could be related to a combination of the immersion level and the feedback provided to users (Carlozzi et al. 2013; Harshal P. Mahajan et al. 2013; Mittelstaedt et al.2018; Nunnerley et al. 2017; Weech et al. 2019). Specifically, the higher the level of immersion, the greater the sickness as well as the engagement and challenge (Weech et al. 2019).

The main purpose of this work is to describe the development of a new wheelchair simulation system based on an immersive VR CAVE and a real wheelchair. This system allows simulating both manual and powered wheelchair modalities. Furthermore, the simulation system in PW modality integrates interaction between movements of the non-driving hand and objects in the virtual environment. A secondary aim of this work is to verify the reliability of the mathematical model used to simulate the MW motion comparing it to the movement of a real MW overground.

The current work is part of the Rientr@ project whose aim is promoting return to work of wheelchair users after a job-related injury.

The newly developed simulation system could be potentially used as a tool to teach to new wheelchair users—and assess—wheelchair driving skills and interaction with objects in the environment in safe and realistic conditions. Furthermore, this system is independent of scenarios and could also be used to validate newly developed virtual environments.

2 Materials and methods

2.1 The grail system

The wheelchair simulation system was developed using the Gait Real-time Analysis Interactive Lab (GRAIL by Motek, NL)Footnote 1. The GRAIL system comprises many different hardware components, which are controlled by the D-flow software (Geijtenbeek et al. 2011). The GRAIL is an instrumented 2 degrees of freedom (i.e. forward/backward tilt and lateral sway) multisensor platform based on immersive VR for gait training and rehabilitation in engaging VR environments. A Vicon motion capture system (Oxford Metrics, Oxford, UK) captures the kinematics of passive markers in real time. The GRAIL system projects VR environments on a 180 ° semicylindrical screen and gives multisensory (audio, video and proprioceptive) feedback, thus allowing users to feel immersed in the computer-generated world. Finally, the GRAIL system is equipped with 3 video cameras and a Dolby surround system to support this immersive experience (Fig. 1a).

Fig. 1
figure 1

a The GRAIL system and its components; b available D-flow software modules

Full size image

The GRAIL system, controlled by the D-flow software, allows an operator to define feedback strategies through a flexible application development framework. The D-flow software is based on visual and Lua script programming, and it is designed for the development of interactive and immersive VR applications, for the purpose of clinical research and rehabilitation. D-flow is based on the concept of modules with specific functionalities, which can be combined to create complex interactive VR applications. Some software modules directly control specific hardware devices, such as the treadmill or the motion platform. Other modules provide access to real-time data streams from input devices or control of the virtual environment. D-flow integrates and synchronizes all hardware components and makes the information available in real time. Furthermore, D-flow manages the relationship between patient, scenery and interactive feedback and simulations.

The D-flow modules available for the GRAIL system used in this research have different features and can be grouped in four categories according to their functionalities (Fig. 1b). Scene includes all modules connected to the scene and those enabling its modification and interaction. Modules included in Resources receive signals from the hardware device (e.g. the motion capture system) or send signals to another hardware device (e.g. platform and treadmill). Preset comprises embedded modules to manipulate marker coordinates which can provide hardware feedback. Modules belonging to Editor are software modules to perform simple operations, such as sound activation, as well as develop complex programs and algorithms, e.g. the Script module. Finally, Data incorporate all modules related to input and output aimed at defining input features for a specific module or at recording an output.

The main D-flow modules used for the simulator development are detailed below:

  • the Mocap module receives all data coming from motion capture and force plate systems and makes them available as output

  • the VGait module allows to modify pitch (from  − 10 ° to + 10 °) and sway (max displacement 5 cm) of the moving platform

  • the Marker Matcher module allows to label markers according to a custom setup

  • the Dynamic Balance module uses input marker coordinates to calculate a combined virtual marker position, as a sort of barycenter; when calibrated, the module sets the zero level position of the virtual marker. This module also enables to scale the virtual marker movements in order to simulate larger or smaller displacements

  • the Navigator module dynamically controls objects in the 3D scene, e.g. it converts an input displacement into speed or acceleration

  • the Camera module is used to set and control the point of view in the 3D scene

  • the Parameter module is used to develop and configure a graphic user interface where the user can define input parameters and control the whole system

  • the Script module is the most flexible module and works as algorithm editor; it can be programmed in Lua and allows to define complex logics and system behaviors not included in standard module functionalities.

2.2 Physical setup

To develop the simulation system in both modalities (i.e. MW and PW), a real wheelchair (KSP Italia, model KSP N23R24Footnote 2) was used and positioned in the middle of the motion platform of the GRAIL system. The wheelchair was placed over 4 blocks, to leave the rear wheels free to turn in the manual modality (Fig. 2a), and a commercial joystick of a PW was placed on an armrest (Fig. 2b), to allow for motion control in power modality. The joystick can be mounted on the right or the left armrest, to accommodate for right- and left-handed people.

Fig. 2
figure 2

Wheelchair simulator setup. a configuration with 3 markers for each driving wheel, as required for MW simulator; rear wheels are free to move thanks to 4 blocks that raise the wheelchair; b configuration with the commercial joystick on which marker support was mounted, as needed for the PW simulator

Full size image

Passive markers were placed on the wheels and the joystick to track wheels or joystick movements, respectively, according to the modality used, as described in the next sections. The wheelchair was firmly secured with ropes to the GRAIL platform in both modalities.

2.3 Manual wheelchair modeling

For the MW simulation, 6 spherical markers with a 12.5 mm diameter were placed on the wheels, 3 markers for each wheel: One marker was positioned in the center of the wheel, and two markers were located on two spokes at 90 °, about halfway between the wheel tire and the center (Fig. 2a).

A custom 3D Vicon model was created to recognize and track the motion of the three identified rigid bodies: the left wheel, defined by markers labeled Axis1, Lwheel1 and Lwheel2; the right wheel, represented by markers labeled Axis2, Rwheel1 and Rwheel2; and the wheelchair axis connecting the centers of the two wheels, defined by markers Axis1 and Axis2 (Fig. 3).

Fig. 3
figure 3

Three rigid bodies are identified by the Vicon model of the manual wheelchair simulator: the left wheel, defined by markers labeled Axis1, Lwheel1 and Lwheel2; the right wheel, represented by markers named Axis2, Rwheel1 and Rwheel2; and the wheelchair axis connecting the centers of the two wheels, defined by markers Axis1 and Axis2

Full size image

The main D-flow modules used for the development of the MW simulator were Mocap, Camera, and Parameter. However, the MW simulator was mainly developed by using the Script module, as described below.

The D-flow received, through the Mocap module, the 3D coordinates (x, y, z) of each marker labeled from the Vicon system in real time. An interactive interface was developed using a Parameter for different models of wheelchair or marker position. The operator needs to enter the wheel radius (Fig. 4a parameter r, in green), the wheel-to-ground distance (Fig. 4a parameter g, in blue), the distance of the markers on the wheel spokes from the center of the wheel itself (Fig. 4b parameter m, in white) and the offset between the real wheel center and the marker on the rotation axis (Fig. 4b parameter c, in red). The application combines these inputs with the marker coordinates to calculate the geometry of the wheelchair that is necessary to reproduce the wheelchair motion.

Fig. 4
figure 4

Measures required by interactive interface: r = wheel radius (green); g = distance between wheel and ground (blue), c = offset between the real wheel center and the marker on the axis of rotation (red); m = distance between the marker on the wheel spoke and the center of the wheel (white)

Full size image

Since the D-flow sampling rate is equal to 250 Hz and the maximum linear speed of a manual wheelchair can be estimated to be about 3 m/s, the distance between two consecutive detected positions of any marker placed on the wheel spokes (in our setup positioned approximately 305 mm from the center) could be considered less than 12 mm, corresponding to a wheel rotation angle less than 2.25°. Thus, the circumference arc length crossed by each wheel marker between two consecutive frames was approximated to the chord subtended L(t) (Eq. 1), that is the distance between two consecutive detected positions of the same marker.

$$L\left( t \right) = \sqrt {\left( {(x\left( t \right) - x\left( {t - 1} \right)^{2} + (y\left( t \right) - y\left( {t - 1} \right)^{2} + (z\left( t \right) - z\left( {t - 1} \right)^{2} } \right)} ~$$
(1)

where x, y, z are the spoke marker coordinates.

A kinematic constraint was assumed to describe movement in the virtual space: The linear speed of each wheel at the ground was set equal to zero (i.e., no slip condition). By this way, the linear displacement of both wheels is combined to calculate the wheelchair movement frame by frame.

The mathematical model used to calculate the MW kinematics was based on the differential drive model (Fig. 5). Angular speeds are ωr(t) for the right wheel and ωl(t) for the left wheel; Vm(t) and ωm(t) are the linear and the angular speed applied to the midpoint axis of the wheelchair, respectively, and θ(t) is the instant angle of the wheelchair at the Center of Instant Rotation (CIR).

Fig. 5
figure 5

Unicycle differential drive model. ωl(t) = instant left wheel angular speed; ωr(t) = instant right wheel angular speed; Vm(t) = instant midpoint linear speed; ωm(t) = instant midpoint wheelchair axis angular speed; CIR = center of instant rotation of wheelchair; and θ(t) = instant attitude angle of wheelchair at CIR

Full size image

According to this model, the angular speed ωm(t) and the linear speed Vm(t) of the wheelchair midpoint are linked in a unique way to the angular speed ωr(t) of the right wheel and ωl(t) of the left wheel, by means of Eqs. 2 and 3:

$$\omega _{m} \left( t \right) = \frac{{\left( {\omega _{r} \left( t \right) - \omega _{l} \left( t \right)} \right) * R}}{D}$$
(2)
$$V_{m} \left( t \right) = \frac{{\left( {\omega _{r} \left( t \right) + \omega _{l} \left( t \right)} \right) * R}}{2}$$
(3)

where R is the wheel radius and D is the wheelchair axis length.

As previously explained, the product between ω(t)*dt of each wheel and R was assumed to be equal to the chord subtended (Eq. 1).

Operatively, a first Lua Script module was designed to receive the actual coordinates from Mocap while saving those of the previous sample. A second Script—developed to receive dynamically the coordinates by the first one—implements Eq. 1 to calculate the instant linear displacement Lr(t) and Ll(t) (of the right and the left wheel, respectively) frame by frame. At each frame, a third Script module receives the instant linear shift Lr(t) and Ll(t) from the second one and is designed to calculate the rototranslation of the wheelchair axis midpoint according to Eqs. 4 and 5 (Fig. 6).

Fig. 6
figure 6

Ll(t) = instant left wheel shift; Lr(t) = instant right wheel shift; Lm(t) = wheelchair instant midpoint shift; θ(t) = instant attitude angle at CIR; D = wheelchair axis length; and CIR: center of instantaneous rotation

Full size image

The instant angle θ(t) of the wheelchair at CIR is computed according to Eq. 4:

$$\theta \left( t \right) = \frac{{L_{r} \left( t \right) - L_{l} \left( t \right)}}{D}$$
(4)

The midpoint axis instant shift Lm(t) is calculated according to Eq. 5:

$$L_{m} \left( t \right) = \frac{{L_{r} \left( t \right) + L_{l} \left( t \right)}}{2}$$
(5)

By applying the same considerations above and considering.

the circumference arc length approximated to the subtended chord, the bisector of θ(t) was considered perpendicular to Lm(t) and the relative instantaneous rotation of the wheelchair axis midpoint was approximated to θ(t)/2 (Fig. 7).

Fig. 7
figure 7

Wheelchair movement between two frames; red lines represent the wheelchair axis, and blue lines show the arc covered by the left and right wheels and the midpoint axis. Measures of the midpoint in the relative axis are shown in green. Ll(t) = instant left wheel shift; Lr(t) = instant right wheel shift; Lm(t) = wheelchair instant midpoint shift; θ(t) = instant attitude angle at CIR(t); (Xr;Zr) = relative system; Zm(t) = relative z coordinate; Xm(t) = relative x coordinate; and \(\theta \left(t\right)/2\)= relative instant rotation of the midpoint of wheelchair axis

Full size image

According to this, the third Script module calculates the instant wheelchair axis midpoint coordinates Xm(t) and Zm(t) along the relative axis according to Eqs. 6 and 7.

$$X_{m} \left( t \right) = L_{m} \left( t \right) * \sin \left( {\frac{{\theta \left( t \right)}}{2}} \right)~$$
(6)
$$Z_{m} \left( t \right) = L_{m} \left( t \right) * \cos \left( {\frac{{\theta \left( t \right)}}{2}} \right)$$
(7)

Lastly, by applying rototranslation equations between relative and absolute reference systems, such Script is able to calculate the absolute positions X(t) and Z(t) and attitude Rot(t) of the wheelchair midpoint axis, as shown by Eqs. 8, 9 and 10.

$$X\left( t \right) = X_{m} \left( t \right) * \cos \left( {\theta \left( t \right)} \right)~ - Z_{m} \left( t \right) * \sin ~\left( {\theta \left( t \right)} \right) + X\left( {t - 1} \right)~$$
(8)
$$Z\left( t \right) = X_{m} \left( t \right) * \sin ~\left( {\theta \left( t \right)} \right)~ + Z_{m} \left( t \right) * \cos ~\left( {\theta \left( t \right)} \right) + Z\left( {t - 1} \right)~$$
(9)
$$Rot\left( t \right) = Rot\left( {t - 1} \right) + \theta \left( t \right)$$
(10)

The position of the virtual wheelchair is updated by the algorithm frame by frame, so the simulator is able to track the position of the real wheelchair in the virtual space in real time.

The instantaneous absolute wheelchair position calculated by the algorithm is sent to the Camera module available in the D-flow software: The virtual camera was positioned at the axis midpoint to simulate the user’s point of view. The position of the camera along the vertical axis can be adjusted according to the user’s height so that users have the impression to navigate into the virtual scenario using a first-person view.

2.4 Powered wheelchair simulation

For the PW simulation, two 12.5 mm spherical markers were positioned on the joystick through a custom-made handpiece, designed and produced with 3D printing techniques. The handle of the original joystick was removed, and the cylindrical handpiece (25 mm diameter, 80 mm high) was placed onto the lever and a 100 mm metal bar (3 mm in diameter) was fit on the top of the handpiece parallel to the ground. Two markers were placed at the extremities of the bar (Fig. 2b). A third 12.5 mm marker on the back of the non-driving hand tracked its movements and enabled user interaction with objects within the virtual environment.

The main D-flow modules used for the development of the PW simulator were: Mocap, Marker Matcher, Dynamic Balance, Navigator and Camera.

The D-flow Mocap module receives the detected marker coordinates from the Vicon system in real time. The Marker Matcher module is used to receive the coordinates from Mocap and to label markers directly in D-flow according to a custom template. It was decided to use this module instead of the Vicon model because there were only 3 markers in this setup, and markers on the joystick never changed their relative position.

Labeled joystick marker coordinates are sent to the Dynamic Balance module. It computes the combined virtual marker position of the two markers on the joystick. At the beginning of each trial, this virtual marker position is zero-leveled by calibration, which must be performed by the operator before starting the simulator application. Furthermore, the Dynamic Balance module allows to change the relation between the virtual marker displacement and the navigation in the virtual environment, i.e. the joystick sensitivity, according to Eqs. 11 and 12:

$$Zout\left( t \right) = ~\alpha * Z\left( t \right)~$$
(11)
$$Xout\left( t \right) = \beta * X\left( t \right)$$
(12)

where X(t) and Z(t) are the original combined marker coordinates, Xout(t) and Zout(t) are the final coordinates of the combined marker, and α and β are configurable values used by the Dynamic Balance module to change the joystick sensitivity that can be customized, similarly to real wheelchair joysticks.

The D-flow Navigator module receives coordinates Zout(t) and Xout(t) from the Dynamic Balance module in real time. The Navigator module uses these coordinates to update its position and rotation in the virtual environment (Fig. 8). Navigation within the scene takes place thanks to the two markers positioned on the joystick: The increasing/decreasing speed is proportional to the forward/backward movement of the joystick (Zout(t)) compared to the zero position (the one measured during calibration), while rotational movements are proportional to the joystick right/left shift (Xout(t)). The maximum speed that can be reached in forward movements was limited to 3 m/s and in backward movements to 2 m/s.

Fig. 8
figure 8

Block diagram describing the PW simulator, from marker recognition to wheelchair virtual movement

Full size image

Since the position and rotation of the Navigator module in the virtual environment reflect the wheelchair movement, they are applied to the Camera module to render the user’s point of view.

This navigation method was considered reliable as several applications developed by the GRAIL manufacturer are based on the same principle. In those exergames, two markers are placed on the subject’s low back or shoulders and he/she moves in the virtual environment as the user moves the correct part of his/her body.

2.4.1 Virtual hand

In the PW simulation modality, the third marker represents the user’s hand, as explained in the previous paragraph. This marker is labeled by the Marker Matcher module (Fig. 8). The hand marker coordinates are considered as the relative position of the hand in relation to the wheelchair position in the virtual environment. The absolute position of the wheelchair is constantly updated by the Navigator module during navigation in the virtual environment. A Script module was designed to compute the absolute position of the hand in real time: The absolute hand coordinates XH(t), YH(t) and ZH(t) are computed by combining the absolute position of the wheelchair and the relative marker coordinates for the hand according to Eqs. 13, 14 and 15.

$$XH\left( t \right) = XW\left( t \right) - \left( { - Xh\left( t \right) * \cos \left( {\alpha W\left( t \right)} \right)~ - Zh\left( t \right) * \sin \left( {~\alpha W\left( t \right)} \right)} \right)$$
(13)
$$ZH\left( t \right) = ZW\left( t \right) + \left( { - Xh\left( t \right) * \sin ~\left( {\alpha W\left( t \right)} \right)~ + Zh\left( t \right) * \cos \left( {~\alpha W\left( t \right)} \right)} \right)$$
(14)
$$YH\left( t \right) = YW\left( t \right) + Yh\left( t \right)$$
(15)

where Xh(t), Yh(t) and Zh(t) are the relative marker coordinates for the hand received from the Mocap module, XW(t), YW(t) and ZW(t) are the absolute coordinates for the wheelchair, and αW(t) is the absolute rotation for the wheelchair, all received from the Navigator module.

A volume representing the user’s virtual hand was created and associated to the marker coordinates for the hand XH(t), YH(t) and ZH(t) computed by the previously described Script: The simulator could thus recognize when the hand collided with scene objects. The virtual hand was visible in the environment (Fig. 9) and could translate in 3D environment without rotating along its relative axes.

Fig. 9
figure 9

PW simulator and the virtual hand during a task completion

Full size image

Some custom rules related to collision detection were implemented to make the simulator able to manage the interactions, such as grabbing an object, opening/closing a door and moving an object from one place to another, producing an appropriate feedback for the user.

2.5 User feedback, interaction with scenes and data recording

In both PW and MW simulation modalities, a bounding box—width 50 cm, depth 80 cm and height 100 cm—was created and applied to the midpoint of the wheelchair axis. This bounding box was configured to detect the collision of the virtual wheelchair with other scene objects. When the wheelchair volume touches an object volume, even partially, the simulator system is programmed to detect the triggering collision, and to produce vibrations through treadmill movement and/or audio feedback. Moreover, the bounding box was used to configure some background sounds to be played for a greater immersion experience, e.g. it was possible to hear the traffic noise along a city street or birds chirping inside a park.

The wheelchair bounding box was also used to simulate ascents and descents in the virtual environment, for example, on sidewalks or ramps, as D-flow does not provide the functionalities to implement appropriate physics-based behaviors. Hidden volumes were created and placed at the beginning/end of the ascent/descent and used as trigger colliders. A specific Script module was developed to detect collisions of the wheelchair volume with trigger colliders and to evaluate whether the collision position corresponded to an ascent (or descent). In this case, the module was programmed to send an output command to the Vgait module so as to tilt the platform of the GRAIL system up to + (or − ) 10 ° according to the real slope of the scene.

By means of a dedicated Script developed to manage hand interactions, the simulation system in PW modality is able to detect collisions between the volume representing the user’s virtual hand and scene objects. The algorithm is able to activate visible and audible feedback to reinforce the interaction with objects. For example, if the user touches a closed door, the door opens and an appropriate sound is played by the system. When an object needs to be moved to another place, the object disappears when it is correctly grasped and it reappears when it is released in the correct position, followed by a sound. An object is considered correctly held if the hand volume collides with the object volume, while it is considered correctly placed if the volume of the hand collides with the volume of the area in which it must be placed.

The application in MW modality was configured to record a.txt file containing: midpoint axis coordinates of virtual wheelchair, absolute rotation, instant linear speed, number of collisions against objects in the VR environment and absolute coordinates of collision point, in order to make these quantities available for future analysis.

The simulation system in PW modality was configured to record a.txt file containing the same quantities registered in MW modality and hand interactions with objects.

2.6 Manual wheelchair simulation assessments

To evaluate the accuracy of the model developed for the MW simulation, the overground movement of the wheelchair was compared with the movement calculated by the simulator.

As the wheelchair over the GRAIL platform did not really move because of the available space—only wheels do turn on the simulator—overground tests of the manual wheelchair were performed in a different laboratory equipped with the Elite motion capture system (BTS Bioengineering) furnished with 8 optoelectronic cameras (sampling frequency 100 Hz).

Four markers were used for these tests: 1 marker was applied on each hand-rail wheel and 1 marker on each armrest of the wheelchair. The marker setup was different from that used for GRAIL experiments due to Vicon requirements (i.e. needs of a marker model made by joints connected by segments); however, in both setups it was possible to track wheels and wheelchair movements, thus making the two measurements comparable.

One experimenter performed ten trials for each wheel; in each trial, one wheel was free to turn, while the other was blocked using the brake; each trial consisted in a 360 degree rotation of the wheelchair with the blocked wheel as rotation center. The coordinates of each marker were collected from the Elite motion capture system and registered into a.txt file.

The same experimenter performed tests on the GRAIL system, mimicking the ones performed overground and using the same wheelchair. Markers were positioned according to the Vicon model developed for MW simulator (Sect. 2.3). Ten trials for each wheel, as described above, were performed, and the wheelchair rotation angles were calculated by the simulation system and registered in a text output file together with all the marker coordinates.

The rotational movement with a locked wheel was a type of movement that could be easily reproduced in the two environments. Therefore, this was a replicable test to assess motion equations; tire pressure and wear were also controlled to reduce possible errors.

2.7 Data analysis

Experimental data were analyzed using a custom MATLAB® scripting, which computed the absolute rotational position of the wheelchair every 5 ° of wheel rotation.

The .txt files recorded by the simulator and by the Elite system were collected and loaded in MATLAB workspace.

To track the wheel rotation, the displacement in the vertical direction of the marker on the rotating wheel was converted into the rotation angle of the wheel by using the arccosine function. The expected output was a triangular wave with values between 0 and π.

To evaluate the wheelchair rotation, two different scripts were developed for the simulator and for the overground tests. The GRAIL simulator provided directly this value in the form of a sawtooth wave with values between  − π and π. By contrast, for the overground test, the wheelchair rotation was calculated using the arctangent function for the 2 horizontal components of the segment connecting the two markers on the armrests. The output was a sawtooth wave with values between -π and π, comparable to the values provided by the GRAIL system.

The total rotation angles of the wheelchair were thus created by correcting the instantaneous change from  − π to π (or vice versa, depending on the rotation direction), for both simulator or overground data, controlling when the angle variation of two subsequent instants was greater than ± π. Data were also filtered by means of moving average, and the window length was set to 1 s for both tests. Finally, the absolute angle values for the wheelchair were extracted every 5 ° of wheel rotation using linear interpolation.

Data used for statistical analysis were collected during the second wheel 360 degree rotation, to avoid considering the initial transition.

2.8 Statistics

According to tests performed as described in paragraph 2.6, 20 trials were collected (10 for each wheel) for both the simulator (virtual wheelchair) and the real wheelchair overground. For each test, data coming from each wheel were pooled to calculate the mean values of wheel and wheelchair angles.

The mean rotational positions of the wheelchair obtained in the two tests were statistically evaluated as described below in order to compare the motion calculated by the simulator with the real motion on the ground.

To evaluate the accuracy of the simulator, the absolute (E) and relative (Er) errors between mean angles of the wheelchair in the two groups were computed (Eqs. 16 and 17).

$${\text{E}} = \left| {{\text{Mean}}~{\text{Overground}} - {\text{Mean}}~{\text{Simulator}}} \right|$$
(16)
$${\text{Er}} = \frac{{\left| {{\text{Mean~Overground}} - {\text{Mean}}~{\text{Simulator}}} \right|}}{{{\text{Mean}}~{\text{Overground}}}} * 100$$
(17)

Median and interquartile range (75th percentile–25th percentile; IQR) were computed for E and Er.

Shapiro–Wilk test was performed to verify whether data were normally distributed. Since the normality of the dataset was not verified, a Mann–Whitney U test was computed to detect any systematic bias. Spearman’s rho was computed to evaluate correlation between the two measurements.

3 Results

The quantitative test of the MW simulator assessed that the developed model, based on the hypothesis of absence of wheel slipping phenomenon and inertia, was consistent with overground movements of a real wheelchair. The mean angles of the wheelchair corresponding to a 360 ° wheel rotation were calculated for the simulator and overground data. The two curves were qualitatively compared (Fig. 10), highlighting that the simulated wheelchair was slightly more sloped than the overground one at the end of the trial.

Fig. 10
figure 10

Mean wheelchair angles, as a function of wheel rotation for the MW simulator (red) and overground test (blue)

Full size image

Table 1 shows the median (IQR) of absolute and relative errors. The absolute error was less than half degree, and the relative error was equal to 0.6%.

Table 1 Absolute (E) and relative (Er%) errors between overground and simulator data. IQR: interquartile range, computed as 75th percentile–25th percentile
Full size table

The Mann–Whitney U test evidenced no systematic bias between the two measurements (Z =  − 0.072, p = 0.943).

The correlation between the two groups is shown in Fig. 11. Spearman's rank-order correlation test was not statistically significant (rho > 0.999, p < 0.01).

Fig. 11
figure 11

Correlation between MW simulator and overground test. Data points are reported in blue. The linear interpolation line is shown in black, while the red line represents the 1st quadrant bisector

Full size image

4 Discussion

The aim of this study was to describe the development of a new wheelchair simulation system made by an immersive VR CAVE and a real wheelchair. A simulation system that can imitate both manual and powered wheelchairs was developed. The reliability of the mathematical model used to simulate the kinematics of MW motion, in the absence of wheel slipping and inertia, was assessed comparing it to the movement of a real MW overground using a single wheel rotation test. Specifically, this was achieved by evaluating wheelchair angles as function of wheel rotation: Simulated and real manual wheelchairs had very similar slopes, except for the end of wheelchair rotation, where the simulation slope is higher than the overground one.

Furthermore, no systematic bias was detected, and simulated and real MW movements were highly correlated. In terms of accuracy, the absolute and relative errors between the overground test and the MW simulation were very small, less than half degree. These errors may be due to the approximations used in the development of the simulation mathematics, as discussed below.

First of all, the approximation involving the use of the chord instead of the arc of circumference introduces an error that becomes negligible with small angles. In our case, the maximum measured chord was equivalent to 1/146 of circle. This means that we were approximating the circumference with a polygon with a minimum number of sides of 146, being other chords smaller, with a resulting theoretical error smaller than 10–2%. Hence, this approximation did not contribute to the error found.

Secondly, the hypothesis of no-slip condition prevented the simulator from considering the slipping phenomenon existing in overground tests. According to previous considerations, the slightly lower mean angle obtained after a wheel rotation of 360 ° in overground tests (177.58 °) versus the MW simulator mean angle (180.73 °) was probably due to the absence of slipping phenomenon simulation.

The literature describes several wheelchair simulators, mainly for PW (Archambault et al. 2012, 2017; Carlozzi et al. 2013; Desbonnet M, 1998; Devigne et al. 2017; Harrison et al. 2002; Hasdai et al. 1998; Harshal P. Mahajan et al. 2013; Harshal Prabhakar Mahajan 2013; Nunnerley et al. 2017; Panadero et al. 2014; Spaeth et al. 2008; Webster et al. 2001) and few for MW (Blouin et al. 2015; Chenier et al. 2014; Crichlow et al. 2011; Crichlow and Crichlow 2011; Harrison et al. 2000; Maxhall et al. 2004). Most of these simulators are based on non-immersive VR (Archambault et al. 2012, 2017; Blouin et al. 2015; Chenier et al. 2014; Desbonnet 1998; A. Harrison et al. 2002; Hasdai et al. 1998; Harshal Prabhakar Mahajan 2013; Spaeth et al. 2008; Webster et al. 2001) which is known to be less engaging and motivating than immersive VR owing to the low level of immersion (Kim and Biocca 2018).

To the best of our knowledge, the simulation system developed in this work is the first one integrating both manual and powered wheelchairs in immersive VR.

Furthermore, considering manual wheelchair simulators developed in immersive VR, the interaction with the virtual environment and the control of the virtual wheelchair motion were based on specific hardware interfaces. Specifically, Crichlow and coll. (Crichlow and Crichlow 2011) developed a hardware interface with torque sensors used to detect user input torques and a servomotor for force feedback. Similarly, Harrison et al. (Harrison et al. 2000) used rollers to detect wheel turning and a variable torque hysteresis brake to provide resistance feedback at each roller according to different virtual slopes. Maxhall and coll. (Maxhall et al. 2004) developed a hardware interface with step counters to measure wheel rotation, and some motion sensors were integrated to detect motion of the user’s hands, head and relative position between body and wheelchair.

By contrast, the MW simulation system developed in this work exploited an optoelectronic motion analysis system to track wheels motion by means of passive markers. This advantageously allows for changing wheelchair and setup easily and quickly. Furthermore, it could be adapted to different motion tracking systems, including less costly ones (e.g., using inertial sensors). The MW interface developed can be considered more manageable and less complex than an ad hoc hardware interface, such as rollers or encoders, to translate real wheel movement in virtual wheelchair motion.

PW simulators described in the literature used instead different immersive VR technologies and different interfaces. Some authors used head mounted displays (HMDs) or glasses (Carlozzi et al. 2013; Nunnerley et al. 2017; Panadero et al. 2014); others used VR screens (Devigne et al. 2017; Harshal P. Mahajan et al. 2013) to show the virtual wheelchair movement, by using joysticks as interface. Panadero and coll. developed a simulator that integrated a platform to generate motion feedback to the user (Panadero et al. 2014). By contrast, Devigne and colleagues did not integrate any kind of feedback in their simulator and reported virtual sickness in users at the end of tests (Devigne et al. 2017). Furthermore, Mahajan and Carlozzi (Carlozzi et al. 2013; Harshal P. Mahajan et al. 2013) demonstrated that PW simulators based on HMD or on 3 flat screens were similar in terms of training effectiveness and user sickness. However, driving was found to be more challenging with HMD technology than with 3 VR screens (Carlozzi et al. 2013) and more difficult with VR screens than with a PC monitor (Harshal P. Mahajan et al. 2013). Also, in their study Nunnerley et al. reported the effectiveness of a VR simulator for training, together with user nausea at the end of the tests (Nunnerley et al. 2017).

The PW simulation system developed in this work uses actual joystick shifts to drive wheelchair movements in the VR environment through a motion analysis system. Given the available hardware and the possibilities of integration provided by D-flow, this interface is more straightforward than ad hoc systems such as USB joysticks requiring the joystick to be physically connected to the simulator through dedicated electronics. Our solution is also easily adaptable to any type of joystick thus allowing each subject to use the simulator with their wheelchair.

Previous works demonstrated that simulators based on immersive VR may cause cybersickness (Carlozzi et al. 2013; Devigne et al. 2017; Harshal P. Mahajan et al. 2013; Nunnerley et al. 2017) and that appropriate vestibular feedback could reduce it. Cybersickness could be indeed related to a combination between the immersion level and the feedback provided to users, and it was also demonstrated that training is effective regardless of the immersion level and that challenging is proportional to the level of immersion.

Moreover, consistently with what has been shown in the literature, the use of a VR immersive screen and appropriate feedback may provide for realistic navigation and reduce user cybersickness (Mittelstaedt et al. 2018; Weech et al. 2019).

Consistent with the indications provided by previous studies, the simulation system described in this manuscript integrates both platform movements to generate vestibular stimuli and optic/acoustic feedback.

5 Conclusion

This work aimed at describing a new simulation system, based on an immersive VR CAVE that integrates both manual and powered wheelchairs. The innovation of this work was the development of an integrated MW/PW setup that enables users to interact with virtual objects from their own wheelchair, thanks to the virtual hand integration in powered modality.

Concerning the MW mode, this work contributed to the development and implementation in the D-flow of the mathematical model and its validation. This step proved that the simulator is able to reproduce wheelchair overground movements, under the assumption that no wheel slip occurs.

Regarding the PW simulator, the main contribution was the integration of hand movements and the adaptation of the navigation control method, used in other GRAIL applications, to the specific purpose of navigating with a wheelchair. The proposed solution is very flexible: It can be adapted to any type of wheelchair, both manual and powered, and any type of joystick, it does not require long preparations for the patient and the environment, and it could be easily adapted to different tracking systems. Furthermore, the system could also be used to analyze the movements necessary to drive a wheelchair by using a proper upper limb motion analysis protocol, being the simulator integrated in a motion analysis lab.

Finally, use of immersive VR based on a 180 ° cylindrical screen and integrating platform movement to generate vestibular stimuli and optic/acoustic feedback could be considered a trade-off between driving difficulties and user experience. This simulator should thus provide a good level of immersion flanked by a reduced user sickness (Mittelstaedt et al. 2018; Weech et al. 2019).

The authors are aware of the possible limitations of the present work. The MW simulation system described in this manuscript is not provided with force feedback and inertia, differently from other works; wheels are free to turn and extremely sensitive to every slight movement. Furthermore, the absence of force feedback would make training incomplete in terms of resistance and balance maintenance especially for MW users. Nevertheless, the system reproduces the movements of a real wheelchair overground and could thus be used to teach abilities like obstacle avoidance, turning in open or thigh space, passing through doors, moving backward/forward while maneuvering.

Furthermore, in the absence of physics in D-flow, there cannot be any gravity simulation and thus acceleration/deceleration in case slopes. This could be solved by integrating specific hardware, i.e. force feedback interface.

The absence of virtual hand in MW simulator could be considered another limitation, because both MW and PW users have the same training needs in approaching and catching objects. Nevertheless, as this was a first prototype, it was decided to ensure the minimum interference between the markers used for driving and those on the hand and to choose the simplest marker setup. For these reasons, the virtual hand was integrated in the PW simulator on the non-driving hand, and not in the MW simulator where both hands are required for driving. Hand integration in the MW model could be achieved, for example, by creating a Vicon model including markers on trunk, arm, wheels and wheelchair axis: This, however, would have required a very long setup and complex patient preparation.

Furthermore, the virtual hand simulation in PW mode includes exclusively translational movements in the 3D space. Three markers would have been necessary to identify the plane of the hand for simulating hand rotations along its axis. The hand rotation would undoubtedly have made the simulation more realistic and more effective for learning, in terms of approaching and taking an object. Nevertheless, in this prototype we wanted to demonstrate the feasibility of an integrated MW/PW simulator that could also (partially) integrate hand tracking, and therefore, the complete implementation of object dragging was out of scope.

Another limitation is that the PW simulator does not allow to configure the joystick dead space which can be found on real joysticks, thus not reducing possible instabilities.

Moreover, the implementation of the proposed setup requires a complex system integrating motion capture, moving platform and virtual reality, such as the GRAIL system whose cost is not negligible.

A final limitation of this work is that it lacks the description of the PW simulator validation. Nevertheless, the PW simulator development was mainly based on available modules that required parameterization, thus allowing to hypothesize that a correct functionality was preserved.

Despite such limitations, we believe that our simulator is a reliable prototype that can be improved with further work. Future activities must be focused on the validation of the simulator with real end users assessing their experience in VR, possible cybersickness as well as effectiveness and safety of the simulator. Furthermore, quantitative tests for PW modality have to be planned to assess the validity of the simulator to reproduce power wheelchair movements.

For future prototypes, it will also be necessary to consider the integration of force feedback to make the simulator more realistic in both modalities.

Regarding the PW simulator, we will evaluate whether it is possible to take into account the dead space available in real joysticks to improve the performance and fidelity of joystick simulation.

Lastly, for future implementation, we will evaluate possible integration of the virtual hand in MW modality in order to guarantee environment interaction tasks also for manual wheelchair users.

To conclude, future perspectives of the simulator described in this manuscript may be the following. The system could be used to validate new virtual scenarios for wheelchair training, thanks to its independence of the environment used. This simulator could have the potential to be a good wheelchair driving rehabilitation tool for MW and PW users in a safe and controlled environment. During the rehabilitation process, the simulator would also allow to evaluate movements needed to drive a wheelchair, if integrated with a proper protocol for upper limb motion analysis. Finally, it could help test driving skills and the ability to reaching objects, thus representing a useful tool to evaluate the appropriateness of a medical prescription for a specific wheelchair.