Background

Each year more than 1 million Europeans suffer from stroke and approximately two-thirds of survivors maintain upper limb (UL) paresis [1]. This number is expected to rise by 35% in upcoming years [2] leading to additional rehabilitation needs. To this date, few people attain complete UL recovery 6 months after stroke [3]. New interventions targeting the UL aim for better outcomes in activities of daily living (ADL), functional independence and quality of life. Alongside conventional therapies, recent developments offer possibilities in self- and tele-rehabilitation [4] which could help manage, cost-efficiently [5], increasing rehabilitation demands.

Technological improvements in robot assisted therapy (RAT) and virtual reality (VR) systems (VRS) enhance patient care and facilitate therapist assistance during UL rehabilitation [6, 7]. First, RAT promotes the use of the affected limb, intensifies rehabilitation through task repetition and offers task-specific practice [7]. Effectiveness of RAT is established for UL rehabilitation after stroke [8, 9]. Secondly, VRS provide augmented feedback, preserve motivation and are becoming cost-efficient [5]. Recent meta-analyses suggest a superior effect of VR-based interventions compared to conventional treatment on UL function and activity after stroke, especially if developed for this specific purpose [10–12]. Authors attributed these findings to the fact that VRS specifically developed for rehabilitation, as opposed to commercial video-games (CVG), fulfil numerous neurorehabilitation principles.

Typically, a common denominator of VRS and RAT is playful interventions by means of serious games [13, 14]. A serious game is defined as a game that has education or rehabilitation as primary goal. These games combine entertainment, attentional engagement and problem solving to challenge function and performance [15, 16]. Moreover, they comply with several motor relearning principles that constitute the basis of effective interventions in neurorehabilitation [11, 16]. For example, some devices adapt game difficulty to stimulate recovery and maintain motivation [15]. Others incorporate functional tasks mimicking ADL in virtual environments and provide performance feedback during and/or after task completion [17]. Characteristics of serious games differ depending on targeted rehabilitation purposes and technical specificities of the system they are implemented on.

Previous work on the efficacy of VR-based interventions indicated that serious games may enhance UL recovery after stroke [11, 12, 18]. However, why such interventions, specifically developed for rehabilitation purposes and implemented on various types of devices (such as robots, smartphones, tablets, motion capture systems, etc.), may constitute effective therapies for UL rehabilitation after stroke needs to be further investigated. Recent theoretical research proposed consolidation of commonly acknowledged neurorehabilitation principles [16]. Usually, serious games comply with several of these principles which creates an opportunity to evaluate clinical applicability of the consolidated set of principles. To this day, it remains unclear whether higher adherence to this consolidated set of neurorehabilitation principles enhances efficacy of interventions. In addition, it is not well known whether adherence to specific principles influences efficacy. Finally, rehabilitation effects on participation outcomes remain relatively unexplored. In this context, efficacy of interventions should be addressed in terms of all components of the World Health Organization’s International Classification of Function, Disability, and Health (ICF-WHO) model [19].

The main objective of this systematic review and meta-analysis was to address the following question in PICOS form: in adults after stroke (P), do serious games, implemented on various technological systems (I), show better efficacy than conventional treatment (C), to rehabilitate UL function and activity, and patient’s participation (O)? A secondary objective was to assess whether higher adherence to a consolidated set of neurorehabilitation principles enhances efficacy of games specifically designed for rehabilitation, irrespective of the technological device used.

Methods

Design

This systematic review followed the Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) guidelines [20]. The protocol was registered in International Prospective Register of Systematic Reviews (PROSPERO 2020, registration number: 156589).

Identification and study selection

A search strategy looking for relevant literature was developed for PubMed and adapted for the other databases, namely Scopus, Embase and Cochrane Library (Additional file 1). Authors received help from a professional librarian to set up the search strategy. Two investigators (GE and ID) independently retrieved studies. All references were stored in reference management software EndNote X9. After removal of duplicates, remaining references were first screened based on titles and abstracts.

Study eligibility was assessed according to the following criteria: (a) design of randomized controlled trials (RCT) (b) participants were adults undergoing stroke rehabilitation (c) the intervention consisted of games developed for neurorehabilitation purposes and implemented in the following technological devices: robotic systems, VRS, tablets, smartphones and motion capture systems (d) relevant outcomes were employed to assess UL function, UL activity and participation (e) studies were published in French or English before May 5th, 2020. All studies using additional therapeutic modalities such as brain stimulation, electrical stimulation or invasive treatments were excluded.

Systematic reviews assessing effectiveness of VR-based rehabilitation and RAT in stroke recovery were also hand-searched looking for relevant references. Finally, a selection based on full-text was conducted by the same two reviewers. Disagreements were resolved through discussion.

Quality and risk of bias assessment

The PEDro checklist was used for methodological quality assessment of trials [21]. In addition, the Cochrane Collaboration’s Risk of Bias (RoB) tool was employed to conduct a critical appraisal of each trial’s internal validity [22].

Data extraction

The following data concerning patients, interventions, control groups and outcomes were extracted from each study: number of patients enrolled in each group, mean time since stroke, corresponding stroke stage classification (subacute: 7 days to 6 months after stroke, chronic: more than 6 months after stroke) [23], dosage and duration of the intervention, technological device used, type and duration of treatment for the control group, presence of a follow-up assessment and outcomes assessed in each timepoint evaluation.

Studies were also assessed in terms of the number of neurorehabilitation principles their intervention fulfilled as described in the review of Maier et al. [11]. These authors described a total of 11 principles presented in Table 1. The two reviewers, independently, investigated whether interventions of included studies fulfilled each one of the neurorehabilitation principles. For each clearly identified principle, one point was attributed to the study. In case available information was vague, missing or did not match the neurorehabilitation principle descriptive’ (as mentioned in Table 1), no point was accorded. Then, we calculated a total score out of 11 for each included study.

Table 1 List of neurorehabilitation principles with description established by Maier et al. [11, 16]
Full size table

Outcome measurements

Outcome measures were selected in accordance to the ICF-WHO model [19]. In each category, assessment scales were chosen based on recent literature recommendations [24, 25]. The Fugl-Meyer Assessment (FMA) [26] was used for the body function domain. The Action Research Arm Test (ARAT) [27], the Box and Block Test (BBT) [28] and the Wolf Motor Function Test (WMFT) [29] were used for the activity domain. The social participation subscale of the Stroke Impact Scale (SIS) [30] was used for the participation domain.

When available, mean improvements in terms of change-from-baseline and their standard deviation (SD) were extracted for each time point. If not available, authors were contacted via email. In case of non-response, the mean improvement was calculated through subtraction between post-intervention mean score and pre-intervention mean score. Then, the SD was estimated by using a formula according to the Cochrane Handbook for Systematic Reviews of Interventions [31]. The value of the correlation coefficient was imputed by using data from other studies [17, 32, 33] included in the meta-analysis. Lastly, when only median and quartiles were available, the mean and SD were approximated using the method proposed by Wan et al. [34]. For studies using follow-up evaluations at least one month after the intervention, mean improvements in terms of change-from-baseline were calculated in order to assess long-term effect retention.

Data and statistical analysis

Articles scoring below 5/10 on the PEDro scale were excluded due to overall poor methodological quality [35]. In addition, only trials that described conventional therapy used in the comparison group as including occupational, physical or self-therapy were considered for statistical analysis. Statistical analyses were performed using the RevMan 5.3 software [36]. Since different rating scales were used for studied outcomes and results were reported in various ways, standardized mean difference (SMD) and 95% confidence interval (CI) were calculated. This method allowed standardization of results across studies. A random effects meta-analysis model was used for all analyses and statistical significance level was set at P < 0.05 [37]. Heterogeneity across trials was estimated using the I2 test. Heterogeneity was not considered to be significative for a I2 < 30% [30].

Subgroup analysis was conducted for RCT whose intervention met at least 8/11 neurorehabilitation principles compared to RCT whose intervention fulfilled less. Another subgroup analysis was performed according to stroke stage, comparing effects of interventions using serious games on subacute and chronic stroke patients. Subgroup analysis were only considered when at least two trials in each subgroup reported a given domain. Furthermore, long-term effect retention for trials that measured outcomes at follow-up was evaluated.

Publication bias was evaluated visually through funnel plot graphic representation. Sensitivity analyses was conducted to verify results robustness in case of funnel plot asymmetry, heterogeneity or presence of outliers. Additional sensitivity analyses were conducted using two different values for correlation coefficient [30]. GRADEpro program was used to assess the strength of the body evidence [38].

Finally, a Fischer’s exact test was used to compare differences in proportions among studies, depending on their results, regarding adherence to each neurorehabilitation principle.

Results

Study selection

A total of 8141 trials were identified through search across all databases and 165 additional records through other sources. After removal of duplicates, 5131 articles were screened based on titles and abstracts. Among these, 5049 were excluded and 82 full-text articles were assessed for eligibility. 51 RCT were included in the qualitative synthesis. Finally, after quality assessment was performed, 42 RCT were considered for quantitative synthesis. Further details are illustrated in the study PRISMA flow chart (Fig. 1).

Fig. 1
figure 1

Flow chart (PRISMA) of the selection process

Full size image

Study characteristics

A total of 2083 participants with a mean age ranging from 49.3 to 76.0 years were included in the qualitative synthesis. For each included study, we identified the mean age of the participants, the stroke stage classification and the type of device used for intervention (Table 2). Approximately one third (31%) of studies included stroke patients at subacute stage and two-thirds (69%) at chronic stage. Across trials, serious games were implemented on different types of devices: 26 (51%) used a motion capture system among which many low-cost systems (such as Microsoft Kinect for example), 10 (19%) used an end-effector type robot, 5 (9%) used motion capture gloves, 3 (7%) a robotic exoskeleton, 3 (6%) an immersive-VR system, 2 (4%) a smartphone or tablet, 1 (2%) a surface EMG-controlled sensor and 1 (2%) an arm support system.

Table 2 Characteristics of included studies
Full size table

For each trial, total treatment duration in terms of minutes per session, number of sessions per week and total number of weeks was identified. In addition, whether intervention and control groups were time-matched regarding these characteristics was verified (Table 3). Total number of weeks of treatment varied from 2 to 12 weeks with a mean of 5 weeks among trials. Daily duration of therapy varied widely among studies ranging from 30 to 225 min. In most trials (85%), total treatment duration was matched between the intervention and control groups (Table 3).

Table 3 Duration, matched groups, outcome measurements, overall findings, number of included neurorehabilitation principles
Full size table

The number of neurorehabilitation principles fulfilled by serious games were identified through content analysis. This number varied from 4 to 11 (Table 3). For a total of 11 neurorehabilitation principles, 32 (63%) interventions met 8 or more, 17 (33%) met between 5 and 7 and 2 (4%) interventions met less than 5. Table 1 illustrates the percentage of studies included in meta-analysis that complied with each neurorehabilitation principle. In addition, Table 1 displays differences in adherence to each neurorehabilitation principle between studies with overall positive or negative results (based on each study SMD in quantitative synthesis results). Statistically significant differences were observed regarding the principle of explicit feedback. Indeed, the group of studies with overall positive results adheres more to this principle than the other group.

Regarding main outcomes, 44 trials (87%) assessed UL motor function, 30 (59%) assessed UL activity and 9 (17%) assessed participation (Table 3). Most trials (60%) reported significantly superior results in at least one ICF-WHO component in favour of interventions using serious games compared to conventional treatment.

Methodological quality and risk of bias assessment

PEDro scores of 51 included studies ranged from 5 to 8 with a mean (SD) of 6.33 (1.15) indicating an overall moderate to high methodological quality (Table 2). Detailed PEDro scale scoring for each trial is illustrated in Additional file 1: Table S1. In addition, the detailed analysis using the Cochrane Collaboration RoB tool is presented in Additional file 1: Fig. S1.

Effect of rehabilitation through serious games on UL motor function

In total, rehabilitation using serious games led to significantly better improvements, of moderate effect size, in UL motor function compared to conventional treatment (SMD = 0.47; 95% CI = 0.24 to 0.70; P < 0.0001) (Fig. 2). Subgroup analysis highlighted differences between results of trials using serious games fulfilling 8 or more neurorehabilitation principles and those that did not (P = 0.003). Indeed, only interventions that met 8 or more principles showed significant impact of moderate effect size on upper limb motor function (SMD = 0.62; 95% CI = 0.33 to 0.92; P = 0.0001). Although total results indicated considerable heterogeneity between studies (I2 = 76%), analysis using the GRADE approach led to a moderate certainty of evidence (Additional file 1: Fig. S2 illustrates detailed summary of findings).

Fig. 2
figure 2

Forest plot of upper limb motor function as measured by the FMA-UE: studies using a serious game fulfilling ≥ 8 Npr versus studies using a serious game fulfilling < 8 Npr. FMA-UE upper extremity subscale of the Fugl-Meyer Assessment, Npr neurorehabilitation principles

Full size image

Additional subgroup analysis was conducted based on the stroke stage of included participants across studies (Fig. 3). Results suggest that interventions using serious games were effective in improving UL motor function in both subacute (SMD = 0.35; 95% CI = 0.10 to 0.59; P = 0.006) and chronic stage after stroke (SMD = 0.57; 95% CI = 0.19 to 0.95; P = 0.003). Differences among subgroups did not reach statistical significance (P = 0.33).

Fig. 3
figure 3

Forest plot of upper limb motor function as measured by the FMA-UE: studies in the subacute phase after stroke versus studies in the chronic phase after stroke. FMA-UE, upper extremity subscale of the Fugl-Meyer Assessment

Full size image

Finally, in order to address heterogeneity, sensitivity analyses were performed in two ways. A first analysis was conducted by excluding outliers identified through funnel plot graphic representation (Additional file 1: Fig. S3). Then, a second analysis was carried out by using a different correlation coefficient value. In both cases, results indicate no significant differences in total estimates when compared to initial findings (Additional file 1: Figs. S4 and S5).

Effect of rehabilitation through serious games on UL activity

In total, rehabilitation using serious games led to significantly better improvements, of low effect size, in upper limb activity compared to conventional treatment (SMD = 0.25; 95% CI = 0.05 to 0.46; P = 0.02) (Fig. 4). In a similar way to results regarding UL function, subgroup analysis showed significantly better improvements, of moderate effect size, only for interventions that fulfilled 8 or more neurorehabilitation principles (SMD = 0.42; 95% CI = 0.12 to 0.72; P = 0.006). Differences among subgroups were statistically significant (P = 0.01). Total results indicated moderate heterogeneity between studies (I2 = 56%). Additional subgroup analysis based on stroke stage did not reach statistical significance for neither subacute or chronic stage after stroke (Additional file 1: Fig. S6).

Fig. 4
figure 4

Forest plot of upper limb activity as measured by the ARAT, BBT, WMFT: studies using a serious game fulfilling ≥ 8 Npr versus studies using a serious game fulfilling < 8 Npr. ARAT action research arm test, BBT box and block test, WMFT Wolf motor function test, Npr neurorehabilitation principles

Full size image

Effect of rehabilitation through serious games on participation

In total, rehabilitation using serious games led to significantly better improvements, of large effect size, in participation compared to conventional treatment (SMD = 0.66; 95% CI = 0.29 to 1.03; P = 0.0005) (Fig. 5). No significant heterogeneity was present (I2 = 0%). All trials included in this analysis used a serious game that complied with 8 or more neurorehabilitation principles.

Fig. 5
figure 5

Forest plot of participation as measured by the social participation subscale of the SIS. SIS stroke impact scale

Full size image

Analysis of follow-up data

Separate analyses were conducted regarding follow-up data for each ICF-WHO component. Only half of the studies included in the quantitative synthesis (50%) performed follow-up evaluations. Among them, length of follow-up period ranged from 1 to 6 months with a mean (SD) of 2.3 months (1.86). An overall tendency towards improvement for interventions using serious games regarding all ICF-WHO components was observed (Additional file 1: Figs. S7, S8 and S9). Total estimates concerning UL function indicate effect retention to follow-up in favour of the experimental group of moderate effect size (SMD = 0.42; 95% CI = 0.05 to 0.79; P = 0.03). Results did not reach statistical significance regarding UL activity and participation.

Discussion

Main results

This systematic review and meta-analysis showed results in favour of rehabilitation using, purpose-built, serious games on UL motor function, UL activity and participation after stroke compared to conventional treatment. Moreover, long term effect retention was significantly maintained regarding UL function. Irrespective of the technological device used, serious games that complied with more than 8 out of 11 neurorehabilitation principles showed better overall effects.

Previous studies on effectiveness of VRS/CVG for UL rehabilitation after stroke

Previous work on the use of VRS and CVG for UL rehabilitation after stroke demonstrated similar results [11, 17]. Yet, to date, usage and efficacy of game-based interventions for UL rehabilitation after stroke remain controversial [38–40]. Initially, a meta-analysis by Saposnik et al., combining observational studies and RCT, suggested improvements in UL strength and motor function after stroke [41]. However, this review focused on various VRS, including CVG designed by the entertainment industry, not specifically developed for rehabilitation. In addition, no statistically significant differences were observed concerning UL activity outcomes and no analysis was conducted regarding ICF-WHO participation component due to limited available data.

Two other groups conducted systematic reviews on a similar topic [42, 43]. However, both reviews included studies concerning not only UL rehabilitation but also gait and balance, making it difficult to draw conclusions regarding the UL. Palma et al., solely relying on qualitative synthesis, supported positive findings on function [42]. Results were inconclusive regarding activity and participation components and further interpretation was limited due to lack of quantitative synthesis. Then, a meta-analysis by Lohse et al. showed positive effects in favour of VR-based interventions regarding three ICF-WHO components [43]. However, analysis was restricted to therapies that did not include robotic assistance. Furthermore, analyses through meta-regressions did not point out significant differences in outcomes between commercially available and custom-built systems.

Two updated Cochrane reviews covered broader aspects of VR and robotics in UL rehabilitation after stroke [8, 10]. In the review by Mehrholz et al., high quality evidence supports better improvements in ADL, arm function and arm strength in favour of RAT [8]. Nonetheless, effects of robotic training performed in form of a serious game were not studied. Then, a review by Laver et al. on VR-based interventions, demonstrated equivalent improvements in UL function and activity when comparing time-matched interventions [10]. Notably, UL function and activity outcomes were pooled in one common analysis instead of distinguishing effects in terms of the two ICF-WHO components. Further analyses in subgroups suggested better results when specific systems designed for rehabilitation were employed compared to off-the-shelf CVG, although differences did not reach statistical significance.

Finally, two recent reviews showed improvements on both UL function and activity in groups receiving VR/gaming-based training after stroke [11, 18]. However, both reviews studied broader aspects of VR-based interventions and their scope was not delimited to specific use of serious games. Karamians et al. suggested that interventions with gaming components further promote recovery compared to those providing visual feedback only [18]. Then, Maier et al. distinguished VRS specifically built for rehabilitation purposes from others destined to generic use [11]. Results illustrated that, when compared to conventional therapy, interventions specifically designed based on elements enhancing neural plasticity led to significantly better results [11]. Additionally, it was suggested that custom-made interventions, in comparison to non-specific interventions, comply better with a series of neurorehabilitation principles.

Adherence to neurorehabilitation principles of interventions using serious games for UL rehabilitation after stroke

To this date, UL stroke recovery through games developed specifically for rehabilitation and implemented on diverse systems, has not been explicitly reviewed. In addition, most recent reviews delimit their scope in technological terms by considering interventions based on the devices being used [11, 12, 18]. Some authors characterise comparison between studies using different devices as difficult [44]. However, a holistic overview of serious games, regardless of the technology used, is important in order to better understand their added value in UL rehabilitation after stroke. Comparison between studies using systems with different technical specificities, mainly in hardware, is challenging. Nonetheless, interventions through serious games implemented on different devices may share similarities. Indeed, all studies included in our review perform non-invasive treatments. Then, gamification and adaptability of interventions, to the patients’ impairments and performance, aim to maintain motivation throughout therapy sessions [45]. Additionally, all these systems have the potential to give access to kinematic data allowing objective assessment, evaluating real-time performance and tracking UL recovery [46–48]. Finally, all interventions stimulate recovery through adherence to common neurorehabilitation principles. In fact, comparison across different types of technologies and treatment modalities leads to identification of common ‘active ingredients’ in terms of effective rehabilitation [11]. In accordance with recent literature, this review contributes to identifying a rationale regarding efficacy of interventions in UL rehabilitation after stroke. Our results point out that even in a group of interventions specifically developed for rehabilitation purposes, differences in outcomes may be explained depending on higher adherence to neurorehabilitation principles. Furthermore, even though most interventions seem to fulfil certain principles (task-specific practice, variable practice, massed practice), it seems that clusters of principles met among serious games may lead to differences in efficacy. For instance, our findings suggest that providing feedback during therapy appears to be an important characteristic that interventions using serious games should satisfy. Further, to what degree each individual principle contributes in efficacy is difficult to study. However, it appears that the more an intervention adheres to principles, the better the expected outcomes can be regarding motor recovery.

To the best of our knowledge, this systematic review is the first to address, in a non-fragmented way, efficacy of specifically designed gaming interventions in UL rehabilitation after stroke. Our results confirm current trends favouring custom-made rehabilitation systems and gamification of interventions. Positive findings concerning function and activity have already been reported in previous reviews [11, 18]. It is worth noting that this review shows encouraging results in participation outcomes indicating, therefore, improvements in three ICF-WHO components.

Strengths and limitations

In a rapidly emerging field, 40% of studies included in our review were published within the last 3 years. Quantitative synthesis was performed by only using RCT of moderate to high methodological quality. However, this was not feasible for two studies due to unavailable data. Additionally, even though our work was conducted according to PRISMA guidelines for systematic reviews, no methods were used to detect unpublished trials. Also, publication bias was only assessed through funnel plot graphic representation which nonetheless did not indicate asymmetry. Heterogeneity across studies was moderate to high regarding UL function and activity outcomes. This may be partially due to variation of elements such as patient characteristics, duration of interventions and evaluation timepoints. Heterogeneity was addressed by using a random effects model for meta-analyses and by conducting additional analyses. Even though heterogeneity levels remained moderate, our results were little affected by changes in methods or outliers, indicating robustness.

Perspectives

Our work offers some suggestions regarding clinical practice and future research. Interventions using serious games may be encouraged and integrated in upper limb rehabilitation programs during subacute and chronic stage after stroke. Specifications regarding dosage, duration and selection of patients that could benefit most from these treatments need further investigation. In addition, serious games should be explored in terms of ways to provide self- or tele-rehabilitation.

From a research point of view, new developments in gaming interventions can take into consideration adherence to neurorehabilitation principles. In accordance with our findings, future developments of interventions in UL stroke rehabilitation ought to comply with as many neurorehabilitation principles as possible. Future work should study how variations in clusters of these principles may influence differently specific aspects of motor or cognitive rehabilitation. Also, richness of kinematic data, accessible through technological devices on which games are implemented, open new perspectives in assessment and follow-up of stroke patients. In our review, only 11% of studies used kinematic data, complementary to clinical rating scales, for UL function evaluation. Finally, few studies (11%) included in our review reported cognitive outcomes. Since motor performance and functional recovery can be influenced by cognitive determinants [49, 50], combined assessment of all these aspects should be further considered in future work.

Conclusion

In conclusion, this systematic review and meta-analysis showed that post-stroke UL rehabilitation through serious games, implemented on various types of technological devices, showed better improvements, compared to conventional treatment, on three ICF-WHO components. Long term effect retention was maintained for UL function. Irrespective of the technological system used, serious games that complied with more than 8 out of 11 neurorehabilitation principles led to better overall effects. Our findings emphasize the importance of adherence to neurorehabilitation principles in order to improve efficacy of interventions in UL rehabilitation after stroke.