1 Introduction

Dementia is an umbrella term that includes a set of symptoms linked to disorders of the brain that progress over time and refers primarily to impairments of cognitive and motor functions (Rose et al. 2018). Cognitive functions may decline as a reaction to co-existing symptoms of dementia, such as depression, apathy, lack of motivation and loss of interest in oneself and others (Brett and Murnion 2015; Kitching 2015; Muliyala and Varghese 2010). Agitation, aggression, and a significant decline in social functions, as well as difficulties in undertaking activities of daily living, are also common symptoms in dementia (Kasl-Godley and Gatz 2000; Lyketsos et al. 2000). Additionally, symptoms can include sexual disinhibition, problems with eating, and abnormal vocalization (Douglas et al. 2004; Korczyn and Halperin 2009). Motor functions may also decline as a reaction to co-existing symptoms of dementia. Examples of some motor dementia symptoms can include weight loss, and muscle weaknesses (Alzheimer’s Society 2017). Even though a set of symptoms has been linked to the development of dementia, however, the cognitive, motor, and behavioural profiles of the people who are living with dementia vary according to the type of dementia, as well as the progression of the pathology and personal characteristics/profiling of the patient (Alzheimer’s Society 2017).

1.1 The global context

A 2017 estimate suggested that 50 million people live with dementia worldwide (WHO 2017). As the global population is increasingly ageing, it is expected that the number of people living with dementia will increase further. In 2017, there were 962 million people over 60 worldwide. Within the following thirty years, this number is expected to rise to 2.1 billion (United Nations 2017). It is therefore predicted that the number of people living with dementia will also increase to about 132 million (WHO 2017). This expected growth has made imperative the need to support people with dementia (Verbeek et al. 2010) who exhibit impairments in cognitive and motor functions.

The Global Action Plan has set out to improve the quality of life of individuals living with dementia (WHO 2017). The Global Action Plan has outlined the objectives with a similar situation to present in all health care markets. One of the objectives included is providing a higher quality of care in the diagnosis, treatment, and support of People with Dementia (PwD), setting up dementia as a public health priority. It also emphasized the drive for improvements in research and innovation. Previous research has shown that pharmacological interventions are overprescribed among PwD compared to other older adults without dementia (Banerjee 2009) and were also found to be ineffective in treating side symptoms (Banerjee et al. 2011). As a result, and in line with the Global Action Plan, it was recommended that best practice reflects the use of pharmacological interventions only as a last resort to treat complex cases where non-pharmacological interventions have proven unsuccessful. The goal, therefore, remains to deliver non-pharmacological innovations, enhanced by the development of proper information systems for dementia, that can support and enhance patients’ quality of life.

1.2 Interventions in dementia

It is well established that good practices in the field of dementia are reinforced by a person-centric approach, with positive evidence of non-pharmacological interventions for PwD (Rose et al. 2019). In particular, research has suggested a reduction in agitation (Livingston et al. 2014) and behaviour that challenges (Kasl-Godley and Gatz 2000), and improvements in cognition, mood and quality of life (Aguirre et al. 2013; Logsdon et al. 2007; Mapelli et al. 2013; Spector et al. 2013; Woods et al. 2012, 2018), for PwD who have been following non-pharmacological interventions, such as cognitive stimulation therapy, reminiscence therapy, music therapy, and other. In addition, physical activity has been also associated with health benefits in cognitive and motor functions for PwD. Many studies highlight the negative relationship between cognitive decline and physical activity, suggesting that cognitive decline could be delayed through systematic physical activity (Littbrand et al. 2011; Pitkälä et al. 2013; Potter et al. 2011). Thus, regular physical activity can improve functionalities related to the brain, preventing the further development of neurological conditions and cognitive decline (Loprinzi et al. 2013).

2 Background

2.1 Benefits of exercise in people with dementia and technological advances

Exercise and physical activity have been associated with several benefits in cognitive and motor health for PwD. Many studies suggested that PwD who exercise regularly have improved motor functions, including stronger upper and lower body muscle strength and better walking balance compared to those who do not engage in any type of physical activity (Santana-Sosa et al. 2018). A positive correlation between the deceleration of the disease and resistance training was also observed (Brown et al. 2015; Vreugdenhil et al. 2012). In support of the above, a systematic literature review of 1497 PwD reported that engagement in regular aerobic exercise can improve the cognitive abilities and delay the deterioration of memory (Trautwein et al. 2017; Zheng et al. 2016). This is further corroborated by research showing that long-term engagement in aerobic exercise improves executive functions for PwD (Cancela et al. 2016; Heath et al. 2016) and reduces depression (Conradsson et al. 2010; Eggermont et al. 2009; Rolland et al. 2007).

Promising results have been reported by studies that examined the use of assisted technology in physical training for PwD. A study (Fenney and Lee 2010) showed that the memory of dementia patients can be improved by playing bowling on Nintendo Wii. Another study (Yamaguchi et al. 2011) revealed that the cognitive levels of PwD can also be improved when performing upper and lower limbs exergaming tasks, such as catching virtual coins or playing drums in front of a screen with customized controllers for upper and lower limbs movement. Improvements in gait and balance in PwD were also reported by a study that used Wii Fit to compare virtual to normal walking tasks. Results indicated improved gait and balance for people who exercised using the Wii Fit console; this improvement was attributed to the high levels of presence associated with the virtual exposure (Flynn et al. 2003). It follows from these studies that Virtual Reality (VR) can be an effective tool in improving the quality of life for PwD since presence is a major component.

It is worth mentioning that the above studies present findings of semi-immersive VR systems, however, we foresee their benefits in other types of VR systems as well. Based on a brief definition of VR, there are three types of VR systems. A non-immersive VR system is a desktop computer-based 3D graphical system that allows the user to navigate the VE through a keyboard, mouse and a small computer screen. A semi-immersive system is an improved system; where a graphical display is projected on a large screen, and there may be some forms of gesture recognition system for natural interactions. Finally, the third type of VR is a fully-immersive Head-Mounted system where users’ vision is fully enveloped, creating a sense of full immersion (Ma and Zheng 2011). In general, within the context of VR technology in health care, research mostly focuses on the specific needs of each population with promising results (Alizadehsalehi et al. 2020, 2021; Bielsa 2021; Holzwarth et al. 2021; Pedram et al. 2021; Plotzky et. al. 2021; Scavarelli et al. 2020).

In health care, research has demonstrated the benefits of employing VR. In general, most of the research within the concept of VR and dementia focused on the enhancement of specific skills which tend to decline within the course of dementia. Such skills include training for cognitive deficiencies (Cheng et al. 2014; Ijaz et al. 2019; Loprinzi et al. 2013; Littbrand et al. 2011; Pitkälä et al. 2013; Potter et al. 2011), memory (Optale et al. 2010) spatial navigation (Cushman et al. 2008; Zakzanis et al. 2009), and executive functions such as planning activities (Manera et al. 2015) and attention enhancement (Doniger et al. 2018; Manera et al. 2016). More specifically and within the context of HCI, research is mostly focused on the specific needs that PwD need to address. Past studies have examined the important elements that a virtual environment should incorporate in terms of design and development in both semi-immersive and fully-immersive modalities. The main conclusion is that special care should be placed on creating personalized experiences that take into account the preferences and skill level of each user (Eisapour et al. 2020; Eisapour et al. 2018a, b; Hodge et al. 2018; Rose et al. 2019; Tabbaa et al. 2019).

To conclude, previous research suggests that physical activity can slow the progression of dementia and improve the quality of life for dementia patients (Cheng et al. 2014; Gallaway et al. 2017). Even though the benefits of physical activity are well documented, research also demonstrated that people living with dementia may experience several difficulties in taking up or continuing physical training. Indeed, barriers may arise from cognitive and motor impairments that limit or prevent PwD engagement in exercise. Specifically, as the disease progresses, PwD encounter difficulties with orientation, which can affect their understanding of the exercise task (Savva et al. 2009). Attention, memory and language processing difficulties also interfere with their understanding. Therefore, it is especially challenging for a person with dementia to comprehend the exercise instructions given by the physician during the physical activity (Yu and Kolanowski 2009). Another impediment related to dementia symptoms is the increase in apathy, loss of motivation and interest in the self (Clarke et al. 2008; Crombie et al. 2004; Kitching 2015; Muliyala and Varghese 2010) which contributes to patient reluctance to engage in any form of physical activity. As a result, most PwD finds physical training to be boring and tedious (Nyman 2011; Suttanon et al. 2012). As a result, further research is needed to develop novel systems that can support and enhance the physical activity for PwD who are residing in confined environments.

2.2 Aims and objectives

The aim of this paper is to present the cyclic process of prototyping, testing, analysing, and refining a VR system for the physical training of PwD. For the purposes of this study, we evaluated the feasibility (i.e. the degree of easily interact with the system and perform the exercises), acceptability (i.e. the degree of tolerance and acceptance PwD have towards the system), practicality (i.e. the degree of easily setting up /deploy the system into the clinical settings) and user experience as well as the effect such technology can have on the quality of life. Having the above in mind, we propose the following research questions:

RQ1: Is VR as an intervention a feasible and acceptable solution for PwD’s physical training?

H1: VR can feasibly and acceptably enhance the physical training of PwD.

RQ2: How does VR impact the physical training of PwD?

H2: VR will have a positive impact on PwD physical training.

RQ3: How easy it is to integrate VR in the clinical setting to enhance the physical rehabilitation of PwD?

H3: VR is a practical solution that can be deployed in real-world clinical settings.

3 Methods

3.1 Participants

Seven people with moderate to severe dementia (Male = 2; Female = 5), with ages between 76 and 86 years (M = 81.14, SD = 3.84), participated in this study. Their average number of years of living with dementia was 4.29 years (SD = 4.96) and ranged between 1 and 15 years. The mean Global Deterioration Scale rating (GDS; Lawton et al. 1999) was 6.14 (SD = 1.07) “moderate to severe dementia” (range = 4–7: “mild to severe dementia”). Primary diagnoses included dementia in Alzheimer’s (n = 5) and Unspecified (n = 2) disease. Secondary diagnoses included Major depressive disorder with recurrent depressive episodes (n = 5) and clinical anxiety (n = 3). All PwD had no history of severe motion sickness or vertigo which posed a risk of nausea, disorientation, and impending anxiety. Additional exclusion criteria were the presence of aggression and/or overfamiliar behaviour; however, behaviour that challenges it was acceptable because of the nature of the dementia disease. Patients confined to bed were also excluded from the study.

3.2 Ethics

Participants were recruited from a National Health in-patient Alzheimer’s and Dementia’s disease hospital that specializes in Dementia and Alzheimer’s disease. Ethical approval was obtained from the National Bioethics Committee (Number is hidden for blind review). All participants signed before the study. All procedures employed were in accordance with the Declaration of Helsinki. For PwD who were unable to provide consent, a close family member was requested to consent on their behalf.

3.3 Procedure

The experiment required the PwD to attend in three separate VR iterations of rapid prototyping with user testing. During each iteration, the PwD were invited, with a familiar physiotherapist in a familiar room at the ward. PwD were asked to sit on a chair, in the middle of the room, and rest their hands to their sides. The maximum time of VR exposure for each iteration was 20 min. Exposure time was consistent with previous research (Eisapour et al. 2020; Matsangidou et al. 2020) and was selected to reduce the risk of side effects. Both studies present the participatory design and evaluation process of VR to promote engagement in upper body physical activity for PwD. Both studies used similar to this study procedures to evaluate the use of VR for PwD.

All training activities involved upper body movements which can be performed actively from a seated position. Seated position exercise training is mostly preferred in older adults to avoid the risk of falling. In each iteration of rapid prototyping with user testing, the PwD did two seated exercises for 20 repetitions each time similar to their usual physical training. The selected exercises were (a) Overhead Press and (b) Side Arm Raise (Fig. 1).

Fig. 1
figure 1

To the left: illustration of the overhead press exercise. To the right: illustration of the side arm raise exercise

Full size image

At the beginning of the experiment, a Human–Computer Interaction (HCI) researcher, specialized in psychology, greeted the PwD and presented the equipment (VR headset, and interaction devices). Information related to the virtual content (patients’ attention was directed to the Virtual Environment as mirrored on the laptop’s screen) and on how to wear and use the VR headset was also given to the patients. While the HCI researcher was explaining the experimental process to the PwD, a technical researcher was managing the equipment.

At each iteration of rapid prototyping with user testing, before the start of an exercise, the user performed one repetition of the exercise. This was used to calibrate the system based on the PwD’s Range of Motion (ROM) by capturing the trajectory of the arms for the exercise, including the starting and ending positions. The calibration aimed to offer a personalized experience for each exercise task. This helped to avoid any frustrations and enable PwD to successfully perform the exercise tasks based on their level of ROM and capabilities. Once the calibration was completed successfully, two targets, one for each arm, appeared in the environment based on the calibrated starting positions. When the user touched a target, a second one appeared at the associated ending position. This was repeated 20 times for each exercise task. Between each exercise, there was a 30 s break.

During the experimental process, several quantitative measurements were recorded by an HCI researcher. Once the experiment was completed, PwD answered a set of questions in a semi-structured interview format. Questions included a series of items inquiring about their experience with the VR. Overall, each iteration of rapid prototyping with user testing lasted approximately 30 min.

3.4 Apparatus

The VR prototype for the study was developed by the authors of the study, using the Unity3DFootnote 1 game engine and the 3D models were created in MayaFootnote 2 to run on a VIVE Pro Eye VR system.Footnote 3 PwD position and movement were captured from the Head-Mounted Display (HMD) for the head position, while for the arms, initially two Vive ControllersFootnote 4 were used and later replaced with four Vive TrackersFootnote 5 (see Fig. 2 for final system outfit). The VR content was streamed on a laptop screen, mirroring the PwD’s real-time virtual interactions, while relevant information about the exercise (patient name, current count of repetitions per arm, current phase in the exercise, i.e. calibration, and current in-game state) were displayed, as an overlay, only on the laptop screen. This allowed the physiotherapist and the researchers to observe silently the procedure. Finally, the user’s gaze was tracked through the HMD’s eye tracker and visualized using a ray only visible on the laptop screen. Through the eye tracker, we were able to compute where PwD were looking by casting a ray based on the direction of their gaze and where they were positioned. The cast ray was then used to identify at which object in the virtual world PWD were looking through collision detection with the various points of interest in the environment.

Fig. 2
figure 2

The system’s outfit consists of the VIVE Pro Eye VR HMD System and four vive trackers to monitor upper limbs movements

Full size image

3.5 System design

The design and development of the system used an iterative design process following the Rapid Iterative Testing and Evaluation (RITE) methodology, which is based on a cyclic process of prototyping, testing, analysing, and refining the system (Medlock et al. 2002). For our study, three RITE iterations were carried out. Each system prototype used the feedback and observational notes from the previous system prototype and made changes to address problems that were identified. The observational notes are discussed in more detail in Sect. 4.

3.5.1 Prototype 1

For the first RITE iteration, an initial prototype was developed implementing a Minimum Viable Product (MVP) that included the core functionalities of the interaction. The purpose of this prototype was to establish the feasibility of such a system and whether it can be usable and understood by PwD. The prototype placed the PwD in an outdoor environment, at the centre of a valley, surrounded by mountains and a creek. Targets were represented by a simple red sphere that turned green once the user touched them. User input was captured from the Vive Controllers, and the hands were displayed as gloves in the scene (Fig. 3).

Fig. 3
figure 3

Prototype 1–to the left: the virtual setting when no action or correct performance of the task is taking place. To the right: the virtual setting with a virtual ball turning green with each successful performance of the task

Full size image

3.5.2 Prototype 2

At the second RITE iteration, the controllers were replaced by trackers placed on the wrists of the patients using straps to simplify the interaction and avoid distractions for the patients. Moreover, the environment was changed with one resembling a landscape that could be more familiar for the PwD. The new environment was again an outdoor landscape, but it included reminiscent elements. These included a brick bridge over a river, common trees, etc. (Fig. 4). Very positive attitudes towards this environment were recorded from PwD (see Sect. 4.1.2 in results for additional information).

Fig. 4
figure 4

Prototype 2–representation of the virtual environment which was used as the backdrop at the second iteration of the prototype

Full size image

3.5.3 Prototype 3

For the third RITE iteration, a human-stick figure was added to the environment to represent the PwD moves in an in-game panel, along with representations of the targets. The panel was right in front of the user, within their line of sight, so they can easily see where the targets are always located. This also allowed the physiotherapist to follow the user’s movements and track what they needed to perform next. The stick figure mirrored the user movements based on the motion tracking from the Vive Trackers. To better capture and represent the full-arm motion of the participants, two additional trackers were added compared to the previous prototype, for a total of four. The trackers were placed at the user’s wrists and above the elbows (see Fig. 2). These, and along with the position and orientation of the head, were used to visualize the orientation of the full arm on the stick figure. To further aid the therapist, a side panel was added as an overlay, only visible to the therapist on the laptop screen, that displayed relevant information about the exercise like the current number of repetitions, next steps, etc. (Fig. 5).

Fig. 5
figure 5

Prototype 3–representation of the virtual environment enhanced with a human-stick figure to represent the PwD moves in an in-game panel, along with representations of the targets (red spheres). To the right, there’s a representation of the therapist’s information panel

Full size image

3.6 Instruments

This paper aims to present the cyclic process of prototyping, testing, analysing, and refining a VR system for the physical training of PwD. For this study, we evaluated the feasibility, acceptability, practicality, user experience as well as the effect such technology can have on the quality of life. To do so, several instruments were used to inform the design, and to assess the usability of the system.

Accuracy and latency data from task execution were logged automatically by the computer. Additional data-independence, SQE and VAS were collected by an HCI Researcher with a background in Psychology and experience in PwD to avoid barriers in communication (i.e. cognitive impairment, other medical issues). The metrics collected in each RITE iteration were the following:

  • Observation Notes Detailed observation notes were taken to classify the patient’s interactions and behavioural responses to each VR experience and to update the design of the prototype.

  • Semi-structured Interviews Interviews with PWD aimed to reflect on their experience using VR over technology acceptance, presence, and emotional affect. Some of the questions were constructed based on the presence and usability (SUS-Presence/SUS-Usability) Questionnaires (please see below for a more detailed explanation). For PWD who were able to elaborately express their answers, questions were asked in an open-ended nature to allow discussions. A simplified version was adopted for PWD who best respond to questions that are simple and closed-ended. Finally, to ensure the reliability of the answers, the same questions were asked more than once and sometimes in a different format.

  • Observed Emotion Rating Scale (OERS) (Lawton et al. 1999). OERS was used to assess the patient’s interactions with the VR. The OERS is used routinely in the care setting since the scale offers direct observation of the time spent expressing five affect types: pleasure; anger; anxiety; sadness; and general alertness. The tool was held by the researcher for the duration of the VR exposure as determined by the participation (maximum of 20 min). The researcher chooses one of six possible defined time intervals (e.g. 1 = never; 2 =  < 16 s; 3 = 16–59 s; 4 = 1–5 min; 5 =  > 5 min; and 7 = not in view) of each of the five emotions.

  • Visual Analog Scale (VAS) (Crichton 2001). VAS was used as a psychometric response scale of depicted emoji to measure emotions (0 = happy and 5 = sad). The scale was used to allow patients to express their emotions towards each method of interaction. We asked patients to point to the specific emoji which matched their emotional state before and after each task execution.

  • Task Performance Assessed the ability to perform the exercise correctly for 20 repetitions within a two-minute timeframe.

  • Task Independence Evaluated by considering the number of times assistance was provided by the physiotherapist to the PwD to perform the exercise. More assistance yielded a lower score on this measure.

  • Time Time was measured in seconds, to assess the duration of each exercise. Each exercise ended by the time the PwD completed the 20 attempts to reach the targets.

  • Movement Movement information was captured by the system and analysed to identify patterns for each exercise and the performance of PwD over the progression of each exercise task.

  • Eye tracking Eye tracking data were collected from the eye tracking camera of the headset and used to analyse the behaviour and focus of PwD during the experience.

  • Slater-Usoh-Steed Questionnaire (SUS-Presence) (Usoh et al. 2000). A simplified version of the SUS assessed the level of presence and immersion for each prototype on a 7-point Likert scale (1 = being somewhere else and 7 = being in the Virtual Environment).

  • System Usability Scale (SUS-Usability) (Brooke 1986). A simplified version of the SUS assessed usability for each prototype on a 5-point Likert scale (1 = strongly disagree and 5 = strongly agree). Note that the analysis is performed at 0 to 100 rates. To do so SUS is required to subtract 1 from the user responses to positive statements and subtract corresponding values from 5 in the negative statements. All responses are added and multiplied by 2.5.

  • The Single Ease Question (SEQ) (Tedesco and Tullis 2006). SEQ assessed the level of difficulty for each interaction on a 7-point Likert scale (1 = very difficult and 7 = very easy).

4 Results

4.1 Observation notes

When assessing the potential of VR for PwD, it is essential to understand what are the unique aspects that may make the VR technology viable and effective for these patients’ population. The primary resource of data presented in this section is based mostly on observation notes. To provide context and illustration of the findings, passages were quoted from our observative notes, and/or the transcribed interviews. The passages were referred in this mode: People with Dementia = PwD, Researcher = R, Source = Observation Notes, and/or Interview. The findings were drawn using content analysis. The analysis revealed eight themes (Table 1).

Table 1 Thematic scheme summary
Full size table

4.1.1 Prototype 1

4.1.1.1 Neural impulses resulted in finger reflex movements

The purpose of the first prototype testing was mainly to evaluate user acceptance of the technology and the feasibility of the use of such a system. Through the design of the first prototype and based on previous studies which were conducted by the team (Hide for blind review), simple interactions were preferred to be executed. Therefore, the use of interaction buttons was avoided. However, it was noted that some PwD, more so for those with mild symptoms of dementia, were trying to interact with the system by pressing the trigger button of the controllers. This attempt was found to be distracting for the patients since it was withdrawing their attention from the virtual experience. In some cases, it was observed that this execution made the patients frustrated, as they were not getting any feedback for their action. Therefore, when designing a system for PwD, it is suggested to use interaction devices that do not include any buttons. When buttons are considered necessary for the design, we propose to use the “trigger button” which is positioned to the front of the controller instead of the “grip button” which is positioned to the side. We believe that patients naturally positioned their fingers to the front side of the controller because of the reduced finger mobility that is common with old age. Our suggestion is in line with other relevant studies which proposed avoiding the use of buttons as a control input when an interaction is required by PwD (Eisapour et al. 2018a).

“He pressed the trigger buttons of both controllers with his index fingers. The researcher asked the patient to hold the controller without pressing any buttons. He seems confused, unable to understand the researcher comments. He kept his index fingers continuously flexy-pressed on the trigger button.”

[PwD7, Observations]

4.1.1.2 Visual limitations of the virtual environment

As aforementioned, PwD were asked to perform an Overhead Press Exercise, which requires for the hands of the patients to be lifted. Due to the nature of the task, the PwD were unable to look directly at the targets unless they manage to tilt their head back to look up. Extra assistance was required by the physiotherapist for the patient to perform the exercise correctly. Therefore, and in line with previous studies (Eisapour et al. 2018a, b, Matsangidou et al. 2020), it is suggested that all interactive visual targets should be visible and within the frontal field of view of the PwD so that the patients could appropriately interact with the system.

“PwD kept her head stable, looking right in front of her. The researcher asked the PwD to rotate her head and look around, instead, she slightly moved her head to the right and then to the left. The researcher asked the PwD to look up. The PwD seemed confused. The researcher asked the PwD to look at the sky. The PwD remained still.”

[PwD3, Observation Notes]

4.1.1.3 Unengaging virtual experiences

There is extensive research that indicates that VR could be utilized to create personal and pleasant space for PwD having a positive effect on the person’s quality of life (Moyle et al. 2018; Rose et al. 2018, 2019; Tabbaa et al. 2019). Through this virtual space, the PwD are capable of momentarily physically isolate themselves from the physical world, without having to physically be removed from the real physical settings. This potential has been found from the general literature to be effective for PwD since VR can simulate realistic immersive experiences which generate a positive change in the PwD environment and the confinements of their daily lives (Rose et al. 2019; Tabbaa et al. 2019). However, we noticed that PwD did not show any special interest in the virtual environment we used but rather were disconnected from the experience. It is therefore suggested for future studies to pay attention to the environment selection process since this finding illustrates the importance of using environments that can trigger memories and enhance positive emotions of PwD.

“R: What do you see? PwD: I don’t know. R: Can you see the trees around you? PwD: I don’t know. R: Can you see the river? PwD: I can see water. R: What colour is the water? PwD: Blue. R: Is that river water? PwD: No, R: Is it a lake? PwD: No, R: Is it a beach? PwD: No. R: What else can you see? PwD: Nothing.”

[PwD4, Interview]

4.1.2 Prototype 2

4.1.2.1 Natural interactions

Due to the nature of Dementia, several difficulties and restrictions may arise in response to the acceptability of new technology from the patients. Based on a previous study, it is suggested to choose a portable and simple technology that projects a comfortable interaction for PwD (Tabbaa et al. 2019). Therefore, one of the main goals of the second prototype testing was to replace the controllers and assess whether body trackers can be tolerated by PwD. It was observed that body trackers are a comfortable device for PwD which can enhance their concentration compared to the controllers.

“R: I placed sensors on your hands how do you feel? PwD: I feel nothing. R: Is it Comfortable? PwD: Yes. R: Is it tight? PwD: No. R: Do you want me to take them off? PwD: No.”

[PwD1, Interview]

4.1.2.2 Meaningful virtual experiences

As aforementioned, previous studies have suggested that PwD are capable of experiencing presence, pay attention and can have an autonomous and meaningful experience in the virtual space (Flynn et al. 2003; Hodge et al. 2018; Rose et al. 2019; Tabbaa et al. 2019). However, this was not proved through prototype 1. At prototype 2, and through the change of the virtual environment, users seemed more immersed in the environment while it also helped them with reminiscing. Reminiscing was further accomplished by the more active role of the HCI researcher that probed PwD about their memories and experiences in the past. Almost all users associated the environment with something from their past, like their childhood village, or a place they had travelled to. These findings are in line with previous research which suggested that VR can also serve as a tool for PwD to reminiscent about the past and experiences from their life (Siriaraya and Ang 2014). Finally, it was also observed that users at the end of the exposure were livelier and happier compared with before exposure and compare to prototype 1. Considering that the only major change between prototype 1 and prototype 2 was the virtual environment, is mainly attribute that the improvement in the patients’ mood, happened because of the virtual environment.

“R: Look around you. What can you see? PwD: I see so many things -PwD giggles- I see birds, and trees, and and… -PwD trying to find the words to describe the element she was looking at. Then she pointed with her finger at the element, the researcher was able to observe through the screen what she was looking at- R: Grass? PwD: -with excitement- Ah, grass! R: Do you like the grass? PwD: Oh yes! And I like my river too. R: Is it your river? PwD: Of course, it is. Is the river I was always walking by with my husband. But today he is not here. I ‘ll wait for him, he will come soon… -PwD’s husband died years ago-”

[PwD5, Interview and Observation Notes]

4.1.3 Prototype 3

4.1.3.1 Range of movements

Even though the benefits of physical activity and exercise are well documented, research also demonstrates that PwD may experience several difficulties in taking part in physical training. Barriers may arise from physical impairments that limit or prevent patient engagement in exercise. In particular, PwD usually suffer from osteoarthritis, one of the most common chronic conditions in the elderly and the largest cause of long-term disability in this group (McAlindon et al. 1993). Osteoarthritis is usually associated with high levels of pain and significant functional disabilities that may cause a significant decrease in the ROM of the person (Johannes 1998). One of the issues that was also present in prototypes 1 and 2 but became more apparent once the usability improved was the patients’ mobility. More specifically, it was noticed that even though the system was calibrated based on an initial task attempt, PwD had trouble performing the exercise in the same way, over several repetitions. Some showed signs of strength deterioration, difficulties in keeping up to the initial range, and required some assistance from the therapist for the completion of the exercise. Others performed similar movement trajectories as the calibrated one, but as time progressed, they were unable to match the target positions. Finally, it was also observed that at the beginning of the task some PwD performed the exercise correctly, but as the repetitions progressed, they changed to a different exercise. It is therefore suggested for the systems to incorporate a reasonable range of deviation in which the patient will be allowed to perform the exercise (for more info see results Sect. 4.7 Movement).

“PwD raised both arms over the head keeping the elbows slightly bent. The red targets become green. She then lowers the arms and bought them back to the first position which is at sides at shoulder height with palms facing forward. She repeated the exercise 20 times. Even though she performed the exercise correctly, however, as time passed, due to fatigue she was unable to reach the targets”

[PwD2, Observation Notes]

4.1.3.2 Engaging virtual experiences

As aforementioned, PwD were found to be capable of engaging in meaningful virtual experiences when the virtual reality environments are appropriately designed (Flynn et al. 2003; Hodge et al. 2018; Rose et al. 2019; Tabbaa et al. 2019). In our design in prototype 2, we altered the virtual environment to fit the PwD preferences. Since the new environment was perceived positively, in prototype 3, our design incorporated a stick figure, to provide visual feedback to the patent in response to the required exercise. Having set an eye tracking technology within the HMD, it was observed that almost all the PwD’s eye-gazes were looking at the figure and the targets when performing the exercises. Moreover, it was noticeable that the PwD required fewer prompts from the therapist about what to do, and they were able to accurately perform the exercises with more precision.

“-PwD’s eye-gases are directed at the board and the stick figure- R: What are you looking at? PwD: A human R: That human is you. PwD: Is me? R: Yes. Move your hands and the human will do the same. -PwD waved her hand- PwD: Oh! It is me! -PwD continue to wave while bursting into laughter (see Fig. 6)”

[PwD6, Interview and Observation Notes]

Fig. 6
figure 6

Dementia patient is waving at the stick figure

Full size image
4.1.3.3 The role of the physiotherapist

Our design was in line with person-centric approach care where the physiotherapist had a vital role in PwD training. However, it is noted that the amount of assistance the physiotherapist offers to the PwD differs significantly due to the nature of dementia, the ROM and the physical capabilities of the person. When designing a system for PwD, assisted interactions were found to be a crucial component for effective design (Hoey et al. 2013). Therefore, and in line with suggestions made by previous research on VR and PwD, it is highly recommended to develop assistive interactions within the virtual environments, which can enhance positively the patients’ experiences. It was further suggested that such systems and environments should incorporate features that will able the caregiver (in our case the physiotherapist) to monitor the patients’ eye-gaze and aid when needed (Tabbaa et al. 2019). We believe that our design incorporated features that allowed the physiotherapist to interpret the PwD responses concerning the system even though the patients’ face was covered by the HMD, and hence, the visual information were prevented. The inclusion of the information board for the physiotherapist allowed them to fully focus on guiding the user without having trouble understanding what was happening.

“The physiotherapist silently observed the PwD interacting with the system. As time pass, the PwD is getting tired and lower her arms. The Physiotherapist smoothly asked the PwD to raise her arms a bit higher. The PwD respond to the command and increase the ROM”

[PwD3, Observation Notes]

4.2 Observed emotion rating scale (OERS)

4.2.1 Prototype 1

Friedman test indicated that there was no significant difference in ratings of pleasure χ2 (2) = 1.333, p = 0.513, anger χ2 (2) = 0.286, p = 0.867, anxiety/fear χ2 (2) = 0.000, p = 1.000, sadness χ2 (2) = 2.000, p = 0.368, or general alertness χ2 (2) = 2.000, p = 0.368, before, during and after VR exposure (Fig. 7).

Fig. 7
figure 7

Prototype 1–means and standard deviations of the observed ratings of emotions before, during and after the VR exposure using the OERS

Full size image

4.2.2 Prototype 2

Friedman test indicated that ratings of pleasure significantly differed between before (M = 1.714, SD = 0.951), during (M = 2.571, SD = 0.787) and after (M = 3.286, SD = 0.756) VR exposure, χ2 (2) = 9.500, p = 0.009. Post hoc analysis with Wilcoxon signed-rank tests was conducted with a Bonferroni correction applied, resulting in a significant increase in pleasure from before to after VR exposure (Z = − 2.041, p = 0.041), but there was no significant difference between before to during VR exposure (Z = − 1.732, p = 0.083), and between during to after VR exposure (Z = − 1.890, p = 0.059).

Friedman test also indicated significant differences for the ratings of sadness between before (M = 3.857, SD = 1.952), during (M = 1.286, SD = 0.488) and after (M = 1.143, SD = 0.378) VR exposure, χ2 (2) = 8.588, p = 0.014. Post hoc analysis with Wilcoxon signed-rank tests was conducted with a Bonferroni correction applied, resulting in a significant decrease in sadness from before to during VR exposure (Z = − 2.701, p = 0.038) and before to after exposure (Z = − 2.121, p = 0.034). No significant difference was reported between during to after VR exposure (Z = − 1.000, p = 0.317).

Finally, there is no significant difference detected in ratings of anger χ2 (2) = 0.800, p = 0.670, anxiety/fear χ2 (2) = 0.154, p = 0.926, or general alertness χ2 (2) = 0.286, p = 0.867, before, during and after VR exposure (Fig. 8).

Fig. 8
figure 8

Prototype 2–means and standard deviations of the observed ratings of emotions before, during and after the VR exposure using the OERS

Full size image

4.2.3 Prototype 3

Friedman test indicated that ratings of pleasure significantly differed between before (M = 2.71, SD = 2.14), during (M = 4.86, SD = 0.38) and after (M = 5.00, SD = 0.00) VR exposure, χ2 (2) = 7.54, p = 0.023. Post hoc analysis with Wilcoxon signed-rank tests was conducted with a Bonferroni correction applied, resulting in a significant increase in pleasure from before to after VR exposure (Z = − 2.00, p = 0.046), but there was no significant difference between before to during VR exposure (Z = − 1.89, p = 0.059), and between during to after VR exposure (Z = − 1.00, p = 0.317).

Friedman test also indicated significant differences for the ratings of anxiety/fear between before (M = 3.29, SD = 2.14), during (M = 1.86, SD = 0.69) and after (M = 1.00, SD = 0.00) VR exposure, χ2 (2) = 7.05, p = 0.029. Post hoc analysis with Wilcoxon signed-rank tests was conducted with a Bonferroni correction applied, resulting in a significant decrease in anxiety/fear from before to during (Z = − 2.12, p = 0.034) and before to after VR exposure (Z = − 2.00, p = 0.036). No significant difference was reported between before to after VR exposure (Z = − 1.79, p = 0.074).

Friedman test also indicated significant differences for the ratings of sadness between before (M = 3.86, SD = 1.95), during (M = 1.71, SD = 0.49) and after (M = 1.00, SD = 0.00) VR exposure, χ2 (2) = 10.00, p = 0.007. Post hoc analysis with Wilcoxon signed-rank tests was conducted with a Bonferroni correction applied, resulting in a significant decrease in sadness from before, during and after VR exposure (Z = − 2.236, p = 0.025).

Finally, there were no significant differences detected in ratings of anger or general alertness before, during and after VR exposure, χ2 (2) = 2.000, p = 0.368 (Fig. 9).

Fig. 9
figure 9

Prototype 3–means and standard deviations of the observed ratings of emotions before, during and after the VR exposure using the OERS

Full size image

In general, the results suggested that PwD were experiencing positive emotions like pleasure during and after the VR session, while it was also shown that negative emotions like anger, anxiety and sadness were decreased during and after the exposure. We conclude that VR can improve the patients’ mode and deal positively with behaviour that challenges.

4.3 Visual analog scale (VAS)

A paired samples t test was conducted to compare the PwD emotions before and after the VR exposure. There was a significant difference in the scores before (M = 3.429, SD = 1.618) and after (M = 1.000, SD = 1.528) the prototype 2 t(6) = 2.634, p = 0.039 and before (M = 2.429, SD = 1.512) and after (M = 0.429, SD = 0.535) prototype 3 t(6) = 3.742, p = 0.010, but no significant differences were reported in response to the scores collected before (M = 1.429, SD = 1.134) and after (M = 0.857, SD = 0.690) prototype 1 t(6) = 1.188, p = 0.280 (Fig. 10). Based on the above results, we conclude that being exposed in VR to perform their physical training improves the patient’s mood.

Fig. 10
figure 10

Means and standard deviations of the visual analog scale emotions before and after the VR exposure

Full size image

4.4 Task performance

To assess the ability of the PwD to perform the exercise correctly, 20 repetitions were carried out, within a two-minute timeframe. A Friedman test with post hoc analysis with Wilcoxon signed-rank tests indicated significant differences for the task performance ratings between the three prototypes for both the Overhead Press (χ2 (2) = 8.000, p = 0.018) and Arm Raise Exercises (χ2 (2) = 6.741, p = 0.034) (Table 2). These results suggest that the patient’s performance can be increased, when visual feedback is provided (e.g. a human-stick figure imitating the PwD moves) and less complicated interactions are encountered between the user and the system (e.g. body trackers are preferred compared to controllers).

Table 2 Task performance for the selected exercises during the three prototypes
Full size table

4.5 Task independence

Friedman test with post hoc analysis with Wilcoxon signed-rank tests indicated significant differences for the task independence ratings between the three prototypes for both the Overhead Press (χ2 (2) = 6.080, p = 0.048) and Arm Raise Exercises (χ2 (2) = 11.385, p = 0.003) (Table 3). These results suggest that VR can enhance and support PwD physical activity and can increase the patient’s independence when less complicated interactions are encountered between the user and the system (e.g. body trackers are preferred compared to controllers). Also, the results suggest that visualizing the orientation of the patient’s body can improve the independence levels of the patient.

Table 3 Task independence for the selected exercises during the three prototypes
Full size table

4.6 Time

Friedman test with post hoc analysis with Wilcoxon signed-rank tests indicated significant differences for the time ratings between the three prototypes for both the Overhead Press (χ2 (2) = 6.741, p = 0.034) and Arm Raise Exercises (χ2 (2) = 13.556, p = 0.001) (Table 4). These results suggest that PwD were performing the exercises more speedily when visual feedback was provided (e.g. a human-stick figure imitating the PwD moves) and less complicated interactions were encountered between the user and the system (e.g. body trackers are preferred compared to controllers).

Table 4 Time (measured in seconds) for the selected exercises during the three prototypes
Full size table

4.7 Movement

The trajectories of the hand’s movements were examined to evaluate the fatigue effects the repetitions of the exercise have on PwD. For each repetition, the trajectory of each hand was computed and compared to the expected trajectory based on the initial calibration movement. The comparison yielded an average magnitude offset of the trajectory for each repetition (M = 0.129, SD = 0.069). Then, Pearson’s correlation coefficient (r) was calculated between the offset of the trajectory and the number of repetitions revealing a positive strong correlation between the two variables r(6) = 0.534, p = 0.001. The observed offset of the trajectory reached up to 30 cm from the expected values. Based on the analysis performed, and depending on the exercise type, we recommend the allowance of up to 15 cm offset from the trajectories PwD perform. This will avoid repetitions performed being missed by the system because of the fatigue PwD might felt. There was no significant difference between exercises or the severity of dementia r(6) = 0.118, p = 0.069.

4.8 Eye tracking

To evaluate the usage and understanding of the stick figure introduced in prototype 3 by PwD, we analysed data recorded from the eye-tracking system. The in-game panel with the stick figure was considered as an area of interest, and different metrics were measured. The metrics were then compared using Welch’s one-way ANOVA based on the severity of dementia. The metrics used were the percentage of time spent viewing the stick figure during the exposure, the number of revisits per second and the number of fixations per second. A fixation was considered after 50 ms of minimal movement. Both the stick figure viewing time (F(2,11.5) = 8.292, p = 0.006) and the fixations per second metrics (F(2,10.9) = 5.557, p = 0.022) were significantly based on the severity of the dementia. Specifically, it was reported that people with more severe symptoms of dementia were paying less attention to the stick figure. No significant differences were reported between the severity of the disease and the revisits per second (F(2,11.4) = 2.775, p = 0.104).

4.9 Slater-usoh-steed questionnaire (SUS-presence)

Friedman test with post hoc analysis with Wilcoxon signed-rank tests indicated significant differences for the component of presence between the three prototypes (χ2 (2) = 10.640, p = 0.005) (Table 5). Results showed that presence can increase when personalized virtual environments are offered to the PwD’s. In particular, environments that are relevant to their interests and are representing memories can improve the patient’s immersion and engagement. The result also reveals that the component of presence can be enhanced when visual feedback and natural interactions are given to the patient.

Table 5 Ratings of presence between the three prototypes
Full size table

4.10 System usability scale (SUS-usability)

Friedman test with post hoc analysis with Wilcoxon signed-rank tests indicated significant differences for the usability of the system between the three prototypes. Usability tests were conducted via the System Usability Scale (SUS) χ2 (2) = 13.556, p = 0.001 and the Single Ease Question (SEQ) χ2 (2) = 11.077, p = 0.004 (Table 6). The results indicated the VR system is perceived as usable by the PwD when visual feedback and natural interactions are encountered.

Table 6 System’s usability between the three prototypes
Full size table

5 Discussion

The use of VR technology to enhance the physical training of PwD is a relatively new approach. From our study, we found that VR interventions appeared to be very effective for this sample of 76–86-year-old adults of both genders who are living with Dementia.

H1 is accepted since the results demonstrated that VR is a feasible and acceptable solution to support and enhance PwD physical training. Through our observations, we found that PwD are capable to engage in meaningful virtual experiences when the virtual reality environments are appropriately designed. Such environments should incorporate personalized features relevant to each patients’ background and interests. Interactions should also be facilitated by natural gestures and portable devices to be accepted by the PwD. Using a reliable measurement of presence (Usoh et al. 2000), our findings also revealed that when a VR system is appropriately designed, then PwD are capable of experiencing a high degree of presence. This was documented by the significant differences detected between the scores of presence during the three prototypes. More specifically, the results indicated that presence can be increased based on the relevance a VR environment has for the patients (i.e. reminisced of the PwD home-country) and based on the equipment’s simplicity and portability (i.e. use body trackers instead of controllers). This notion was further supported by the significant differences which were reported for the usability ratings. These findings were in line with previous research which suggested that PwD can experience the virtual world with a sense of presence and normality (Flynn et al. 2003; Hodge et al. 2018; Rose et al. 2019; Tabbaa et al. 2019). Analysis of the eye tracking data has however also shown that PwD with more severe symptoms could have troubles in understanding tasks they need to perform, and therefore more care is needed when designing experiences for such a population.

In addition, VR was found to have an impact on PwD emotional health. Significant improvements were observed before, during and after the VR exposure in pleasure, sadness, and fear/anxiety. The current study found no adverse effects in the form of anger and general alertness. It is worth mentioning that the observed improvements in the emotional health of the PwD might be attributed to the commonality and relevance of the chosen virtual environment (i.e. most of the PwD quoted that the VR environment was an intimate place for them, similar to their home-town), the sense of presence and the high rates in usability. Significant improvements in pleasure when a PwD is exposed to a VR environment were also reported by a relevant study (Rose et al. 2019). The study found that VR exposure can elicit positive responses and pleasant emotions for PwD and attributed pleasure to the usability and the portability of the technology. The design of the study was based on previous research (Moyle et al. 2018; Siriaraya and Ang 2014) and aimed at reducing the risk of fatigue or discomfort by having the patients seated and by zeroing the interactions between the PwD and the system. Contrary to the above, our design required high mobility of interactivity with the system which resulted in high rates of usability and immersion. We, therefore, believe that new technologies have brought to life opportunities that can deploy simple interactions where natural movements can be used and can provoke much more positive emotions in the PwD. In terms of interactivity, our findings are in line with previous studies which suggested that PwD can benefit from the use of VR technology if the system is appropriately designed and support simplified interactions which avoid any distracting factors, such as the use of buttons (Eisapour et al. 2018a).

Similarly, H2 is accepted since the positive improvements in the patients emotions made the training pleasurable and fun, which resulted in an overall positive impact on PwD physical training. In particular, previous research suggested that even though the benefits of physical activity and exercise are well documented, however, most of the PwD are reluctant to engage in any form of physical activity since they find the physical training to be boring and tedious (Nyman 2011; Suttanon et al. 2012). This reaction is explained based on the dementia symptoms, which are linked to an increase in apathy, loss of motivation and interest in the self (Clarke et al. 2008; Crombie et al. 2004; Kitching 2015; Muliyala and Varghese 2010). Our study proposed the use of VR technology as a novel intervention to improve the physical training of PwD, given that patients expressed that positive emotions were elicited after being exposed to VR physical training. To the best of our knowledge, this is the first study that reports significant improvements in PwD mood after using a fully-immersive and interactive VR system. The significance of the findings of the study is documented by the OERS and the VAS psychometric response scales, scales with high statistical reliability and validity.

Finally, H3 is accepted, supporting that VR as a set-up is a practical solution that can be deployed into real-world clinical settings since the findings demonstrated that PwD are able to perform their daily training through the use of VR when a usable environment with explanatory feedback is provided to the patients. Findings are further supported by the fact that PwD were able to perform the exercises independently with less supervision and only supported by the system. Results have also shown that fatigue, but also mental challenges, can be a hurdle for PwD successfully completing the exercise tasks. Therefore, any solution should be adaptive based on the PwD performance in order to maximize potential outcomes. The system’s design and usability resulted also in reducing the overall time which was required for the training to be performed. The findings are in line with previous research which suggested that VR as technology can enhance and support the PwD physical training (Eisapour et al. 2018a, b), however, and to the best of our knowledge, our study is the first study that implemented the technology in a hospital environment where people with dementia are living confined (Schiza et al. 2019).

The results of this study provide further evidence that VR technology can play a significant role in the improvement of PwD physical training and emotional health. In particular, our findings revealed the positive consequences of VR for PwD who are now able to experience the outside while exercising in a safe and restricted hospital environment. Overall, our results suggest that VR can increase positive emotions (e.g. pleasure and happiness), decrease negative emotions (e.g. sadness and fear/anxiety), and improve the performance and independence of the PwD when performing an exercise through gamified features (e.g. boost the patients training by targets and reinforcement). This results in PwD having an increased willingness to engage in physical training, or continues to exercise for a longer period of time (e.g. as mentioned by the physiotherapist during their normal physical training, PwD quit the training once they get tired, with the VR PwD are positively motivated and so they exercise for longer).

6 Conclusions, limitations and future work

Research suggests that physical activity can slow the progression of dementia and improve functional outcomes (Cheng et al. 2014). Although physical activity is fundamental for the improvement of the quality of life for people with dementia, patients often neglect to participate fully in sessions of physical activity due to barriers that may arise from cognitive and motor impairments that limit or prevent PwD engagement in exercise.

The purpose of this study was to explore whether VR is a feasible solution for PwD who are dealing with barriers arising from cognitive and motor impairments, such as difficulties with orientation, attention, memory, language processing and lack of motivation. Using qualitative and quantitative approaches, we found that VR could be a successful solution for physical activity in dementia patients when specific factors of the design are followed. In particular, our findings suggested that VR appeared to be very effective for this clinical population if the design includes features that respond to specific challenges, such as: 1) a simple interaction modality is preferred for the PwD to be able to interact with the system and perform the task correctly and without much supervision and assistance from the clinical staff. Having that said body trackers were found to enhance PwD concentration compared to the controllers; 2) visual targets should be stable, visible, and within the frontal field of view of the PwD so that the patients could interact easier and appropriately with the system. Also, continuous feedback should be provided to the PwD so as to be able to accurately perform the exercises with more precision; 3) personalized virtual environments relevant to the PwD’s interests and memories improve the patient’s mood, immersion and engagement, and 4) a reasonable range of deviation in which the patient will be allowed to perform the exercise is highly suggested since this can increase task performance.

Even though we believe that this study contributes to the emerging body of research by describing the process of how VR can be designed, tested, and deployed into a hospital environment based on the restrictions that dementia patients possess, several limitations were in presence. In particular, the study was limited to a relatively small sample of dementia patients, constrained by RITE methodological design, and exclusion criteria. Nevertheless, the study contributes to the research in the design community by presenting the process for the effective design of a deployable VR system for people with dementia residing within hospital units. In the future, studies should examine the use of VR for physical activity with a larger sample size to increase the reliability and validity of the findings. Future work should also consider ways to reduce exclusions. For example, people with dementia confined to bed could be addressed by setting up the equipment in their private rooms.

To conclude, this study contributes to the emerging body of research on the use of medical technology for PwD dealing with several barriers to engage in physical activity in real-world clinical settings. The study presents the cyclic process of prototyping, testing, analysing, and refining the VR system in real-world clinical settings. We believe this paper lays the foundations for the deployment of VR on a large scale in clinical environments and we can see a future where VR will be a part of the dementia patients’ physical activity training.