Abstract
Background
Multiplayer games have emerged as a promising approach to increase the motivation of patients involved in rehabilitation therapy. In this systematic review, we evaluated recent publications in health-related multiplayer games that involved patients with cognitive and/or motor impairments. The aim was to investigate the effect of multiplayer gaming on game experience and game performance in healthy and non-healthy populations in comparison to individual game play. We further discuss the publications within the context of the theory of flow and the challenge point framework.
Methods
A systematic search was conducted through EMBASE, Medline, PubMed, Cochrane, CINAHL and PsycINFO. The search was complemented by recent publications in robot-assisted multiplayer neurorehabilitation. The search was restricted to robot-assisted or virtual reality-based training.
Results
Thirteen articles met the inclusion criteria. Multiplayer modes used in health-related multiplayer games were: competitive, collaborative and co-active multiplayer modes. Multiplayer modes positively affected game experience in nine studies and game performance in six studies. Two articles reported increased game performance in single-player mode when compared to multiplayer mode.
Conclusions
The multiplayer modes of training reviewed improved game experience and game performance compared to single-player modes. However, the methods reviewed were quite heterogeneous and not exhaustive. One important take-away is that adaptation of the game conditions can individualize the difficulty of a game to a player’s skill level in competitive multiplayer games. Robotic assistance and virtual reality can enhance individualization by, for example, adapting the haptic conditions, e.g. by increasing haptic support or by providing haptic resistance. The flow theory and the challenge point framework support these results and are used in this review to frame the idea of adapting players’ game conditions.
Similar content being viewed by others
Explore related subjects
Find the latest articles, discoveries, and news in related topics.Introduction
Robotic assistance and virtual reality in neuromuscular therapy
Neurological deficits can result in impaired motor function that affect a person’s quality of life. Researchers have been working to restore the nervous system and reduce the neurological deficits of people suffering from stroke, spinal cord injury, or traumatic brain injury [1]. For people with neurological deficits, impaired motor function is among the most prominent factors limiting the quality of life [2]. Motor neurorehabilitation can lead to permanent improvements in motor function [3]. Robotic assistance and virtual reality have the potential to enhance rehabilitation of neuromuscular deficits beyond the levels possible with conventional training strategies [4, 5].
Game experience and task performance in multiplayer games
Robot- and virtual reality-assisted single-player games are well integrated in neurorehabilitation schedules. Recently, multiplayer games have been tested to complement neuromuscular therapy. Multiplayer games are expected to motivate the patients and increase the potential of robot- and virtual reality-assisted neuromuscular therapy.
Multiplayer games incorporate social interaction to promote the enjoyment of the involved players. The additional player adds new possibilities to the game environment, generally missed in single-player gaming against preprogrammed challenges or artificially controlled opponents. The multiplayer environment and related game mechanics can facilitate social interaction, ranging from conversation to haptic interaction. Due to the this added social interaction, the game experience is thought to be better in multiplayer compared to single-player gaming [6].
The mode of the game specifies whether the players compete or cooperate with one another [7]. In line with the flow theory, a competitive mode requires opponents of similar skill level to achieve enjoyment as the task difficulty experienced by one opponent [8]. Comparable skill levels prevent boredom or stress and result in a meaningful challenge level that leads to a flow state when training [9]. In such training conditions the players have a positive game experience.
In positive game experience players increase their game performance [9, 10]. Increased game performance facilitates the general idea of serious games, i.e., playing for a primary purpose other than pure entertainment [11]. If enhanced game performance is achieved by increased physical activity, training intensity is also increased. In neuromuscular therapy, training intensity – alongside early treatment, user-centered, and task-oriented training – is one of the key factors in neurorehabilitation [12, 13]. Therefore, multiplayer gaming has great potential to further increase the benefits of robot-assisted neuromuscular and virtual reality-assisted therapy [14, 15].
Differently skilled patients and motor learning
Accounting for the individual’s skill level is not only important for game experience, but also for motor learning. According to the challenge point framework [16], motor learning depends on the amount of interpretable information. An increase in task difficulty can increase the amount of available information. However, the learning potential is only increased when information can still be interpreted and does not overload the individual, i.e. hamper task performance. As the ability to process information varies between individuals, the task difficulty should be adjusted to the individual in order to facilitate learning.
Manipulation of game conditions in multiplayer games
Various solutions have been offered to account for the differences in skill level in multiplayer gaming in neurorehabilitation. These can be differentiated into adaptation of game mechanics (e.g. frequency of actions) and adaptation of the interface’s mechanics (e.g. robotic support). In multiplayer games, adaptations of the game mechanics affect the game condition for all players. In contrast, adaptation at the interface level allows for individual adjustment of the task difficulty. Thus, in settings with differently skilled players, individual interface mechanics can ensure that all players are challenged according to their skill level and can therefore increase game experience.
In health-related gaming, task difficulty adaptation in multiplayer games, i.e. accounting for the skill level of the opponents, bears a challenge for game developers: Adaptations of game mechanics–as commonly done for single-player games–affect the game conditions for all players. In health-related gaming environments – particularly in patients with neuromuscular deficits – this challenge is even more prominent due to large variability in information processing abilities and motor skills. This large variability leads to differences across active range of motion, muscular strength, interlimb coordination or spasticity, among others. In multiplayer games, these deficits can make it so that two players cannot play against one another. It is therefore of interest to understand how to manipulate game conditions to balance the skill levels of patients and enable multiplayer gaming.
Contribution of this review
In this review, we investigated whether multiplayer environments have improved game experience or game performance in serious games for health-related disciplines and neuromuscular therapy. We further compared different multiplayer game modes regarding game experience and game performance. To facilitate the transfer and consolidation of multiplayer gaming from health-related gaming to robot-assisted neuromuscular therapy, we systematically reviewed the literature regarding multiplayer gaming in health-related games. In this article, we describe the available literature in the context of the flow model and the challenge point framework. To further facilitate a transfer to neuromuscular rehabilitation, this review discusses measures regarding perceived game experience and physical performance, which have been applied in studies using multiplayer modes offering social interaction. Furthermore, this review complements definitions of task difficulty with the term “conditional task difficulty” to facilitate the discussion of haptic difficulty adaptation in robot-assisted rehabilitation. All discussions refer to our research question: Do multiplayer games enhance experience and performance in robot-assisted and virtual-reality assisted neurorehabilitation?
Methods
Protocol
The review protocol was based on an initial search review of the promising aspects of social interaction in virtual environments for rehabilitation interventions.
Eligibility criteria
We developed a database adjusted electronic search strategy for EMBASE, Medline, PubMed, Cochrane, CINAHL and PsycINFO in collaboration with a librarian from the Medicinal Library of the University of Zurich. The search was restricted to English and German language literature. There was no limitation in publication date or restriction by study design. The first search was performed in April 2016 and was complemented by papers published since (until January 2018).
We used medical sub-headings as search terms, including the following main terms for the population: motor impairments, cognitive impairments, stroke, elderly people; for interventions: competition, collaboration, cooperation, coopetition, competitive behavior, collaborative behavior, exergame, multi-player-exergame, serious game, rehabilitative game, rehabilitative exergame, education game, computerized training, robot-assisted rehabilitation, robot-aided rehabilitation, virtual reality, virtual reality therapy, virtual world, social community, community integration, virtual community; for outcome: persuasion, compliance, motivation, engagement, effort, adherence, therapy progress.
The search strategy (Additional file 1) was initially run in EMBASE and then adapted to the search format requirements of the other databases included in this review. The search results were supplemented by articles found through manual searches by scanning reference lists of identified studies.
Study selection
After duplicate citations were removed, two independent reviewers (KB, AS) determined which articles should be included within this systematic review by scanning the titles, abstracts and keywords while applying the inclusion and exclusion criteria (Table 1). An article was included if both reviewers independently saw the potential that the article’s results might be of interest for multiplayer applications in robot-assisted neuromuscular therapy. To determine whether the full text should be retrieved for a given citation, the two reviewers marked each citation using a “yes,” “no,” or “unknown” (unsure whether yes or no) designation. The citations marked as “unknown” were discussed within the reviewers. The screening process involved simultaneous title and abstract screening. The reviewers evaluated the same set of citations. A study was considered eligible for inclusion in the review when the studied intervention: (1) was a multiplayer intervention in a robot-assisted, video-game based or virtual reality based, health-related setting, and (2) examined the effects of the intervention on perceived game experience or physical functioning of the players involved.
This review includes a transfer to neurorehabilitation training gaming from related disciplines, such as health related serious gaming. For this purpose, we considered typical types of exercise interfaces as described by Tanaka [17]. Feasibility studies with less than ten subjects were not included in the systematic assessment but relevant content was considered for the discussion.
The full-text article was read if the title, abstract, or keywords provided insufficient information to decide on inclusion (Fig. 1).
Data extraction process and data synthesis
The following data were extracted from the studies: (1) characteristics of the studied population: number of participants, disease and age, (2) characteristics of the interventions: the design, frequency and duration of the intervention, co-interventions and control intervention; (3) characteristics of the outcomes: outcome measures and results (Tables 2, 3, 4, and 5). The included studies were divided into three groups according to their assessments: perceived game experience (Table 3), physical performance (Table 4), and personality factors (Table 5). The review of perceived game experience included studies using inventories that assessed the players’ game experience or game playtime expressing a positive game experience. The review of physical performance included studies assessing any physical fitness related quantity measured by the game score or physical quantities measured on the player. If a study monitored both perceived game experience and physical performance, the study appears in every relevant group, accordingly.
Perceived game experience is discussed as the expressed opinions gathered from the activity. The opinion was assessed either during or after the task and either by answering a questionnaire or by assessing performance attributes (e.g. player’s decision on duration of task performance). Physical fitness is considered as a set of attributes that people have or achieve to perform a physical activity as defined by Caspersen [18]. Physical performance is considered as the performed physical activity represented by the amount of useful work accomplished within the task. Assessment of the usefulness of the work is defined by the task designers.
Because we expected the interventions and reported outcome measures to be markedly varied, we focused on a description of the studies and their results, and on thematic synthesis as defined by Thomas and Barnett-Page [19, 20].
Assessment of study quality
Our critical appraisal of the studies was based on a checklist designed for assessing the methodological quality of both randomised and non-randomised studies of healthcare interventions [21]. The checklist assesses biases related to reporting, external validity, internal validity and power. Six items were not considered in this review: adverse effects (item 8), follow-up analyses (items 9 and 26), representativeness of treatment locations and facilities (item 13), allocation concealment for participants (item 14), and blinding of investigators (item 15). These items were excluded because there were not any follow-up studies available that considered the given type of intervention (items 9 and 26), or because we considered these items as being of minor significance for this review (items 8, 13, 14 and 15).
The remaining 21 items were applied by two reviewers (KB, AS) to assess the methodological quality of the selected full text studies (Additional file 2). The total possible score was 22 points, whereas every item can be scored 0 or 1 point except for the confounding description (item 5; 0, 1 or 2 points). Scoring for consistency in recruitment time period (item 22) was, compared to the original checklist, changed to scoring for stating time period of intervention. Scoring for statistical power (item 27) was simplified to a choice between 0 or 1 points depending on the level of ability to detect a clinically important effect. The scale ranged from insufficient (β<70%=0points) to sufficient (β≥70%=1point). To assess the level of agreement between the investigators a Cohen’s kappa analysis was performed on all items on the checklist. In accordance with Landis and Koch’s benchmarks for assessing the agreement between raters a kappa-score of 0.81...1.0 was considered almost perfect, 0.61...0.8 was substantial, 0.41...0.6 was moderate, 0.21...0.4 was fair, 0.0...0.2 were slight and scores <0 were poor [22]. The PRISMA-statement was followed for reporting items of this systematic review [23]. Therefore, eligible criteria, information sources, and search strategy were defined pre-search (30.10.2014) and remained unchanged (Additional file 1). Study selection, data collection, and data reporting are fully described within this paper. However, the systematic review is not registered in any data base.
Results
Study selection
The search provided a total of 1889 references. After adjusting for duplicates, 1330 remained. Of these, 1304 were discarded because they were out of scope of this review. The remaining 26 potentially relevant articles were supplemented by 7 additional references retrieved by a manual search. This resulted in a total of 33 articles eligible for full-text reading. After full-text reading, 19 articles were excluded because neither an intervention nor social interaction was presented (Fig. 1). Thirteen articles were finally reviewed (Table 2).
Study characteristics
All studies were published in English. The publication dates ranged from 2008 to 2017. In the selected studies, participants were seventh-graders [24], high-school students [25, 26], adults [27–30], arm impaired [29–33], and undergraduate students [33–36]. The age of the participants ranged from 12 years to over 90 years whereas four publications did not include full information about minimal and maximal age [29, 31, 33, 34].
Devices, games and multiplayer modes
Six studies used a commercially available exergaming input device, such as a Nintendo Wii controller or a Kinect system, in combination with a commercially available game [24–26, 35], or an in-house developed game [34, 36]. Five studies used a haptic manipulator or arm rehabilitation system with an in-house developed game [28, 30–33]. One study used a grip transducer with an in-house developed game [29].
In six studies addressing multiplayer modes, a competitive mode was compared to a cooperative mode [25, 26, 30, 31, 35, 36]. Four of these studies also compared competitive or cooperative gaming against a single-player mode [26, 30, 31, 35]. Six studies compared either competitive or cooperative gaming against a single-player mode [24, 27–29, 32, 34]. Only one study compared a competitive mode to a single-player mode in different games [24] and one study compared multiple sessions of a competitive mode to a single-player mode [32]. One study investigated the influence of external forces, including forces from partner players, on performance and motor learning [27]. Included studies also compared discrepancy levels of a participant’s game performance to a competitor’s game performance [34], and different methods to decrease discrepancy in a multiplayer rehabilitation game [33]. Furthermore, included studies compared friend- and stranger-paired group multiplayer gaming [35], gaming in home and clinical environments [32], and different levels of interaction in a tele-rehabilitation game environment [34].
Assessment
Assessment of perceived game experience
Eleven out of thirteen articles assessed the effect of the intervention on the motivation of the player (Table 3). To measure the players’ motivation, the Intrinsic Motivation Inventory (interest/enjoyment, perceived competence, effort/importance, pressure/ tension) [37] was used seven times [28–33, 36], Malone’s theory of intrinsic motivations for learning (sensory immersion, control/choice, challenge/optimal difficulty, goal setting and feedback) [38] was used once [25], the Kids Game Experience Questionnaire [39] was used once [24], and an evaluation of motivation related adjectives (boring (reverse-coded), exciting, enjoyable, entertaining, fun, interesting, pleasant) [40] was used once [35].
A ranking regarding preferred game mode based on subsets of the Intrinsic Motivation Inventory (interest/enjoyment, perceived competence, effort/importance, pressure/tension) [37] was used in four studies [28–31] and a weighted preference in one study [33].
Assessment of goal commitment [41, 42] was used once [36], and assessment of psychosocial attractiveness of game design regarding interpersonal communication in the four factors social interaction, collaboration, individual feedback and team feedback [43] once [25].
The Koehler effect [44], assessed by measuring the difference in persistence in executing the intervention task, was used once [34].
Psychosocial factors such as self-efficacy [45], efficacy self-esteem [46] and peer support [47] were assessed in one study [26].
Intervention specific questions for perception assessment of the external forces and fatigue were used in one study [27].
Assessment of physical performance
Five out of thirteen articles assessed physical fitness by calculating energy expenditure (Table 4): weight difference by measuring weight before and after the intervention [26], comparing the heart rate during an intervention with heart rate during individual control condition [34], calculating the metabolic equivalent of task [24] as defined by Ainsworth [48], and averaging the acceleration profile assessed during the intervention [25, 35] as introduced by Puyau [49].
Five out of thirteen studies, [27–30, 36], assessed physical performance by comparing game performance scores. Three out of thirteen studies, [31–33], used the root-mean-square (RMS) of the velocity profile of the hand movement (introduced by Van Der Pas [50]); one out of these three studies also reported mean velocity [31]. One study, [28], assessed physical performance by extracting the peak velocity of the hand movement. One study, [34], assessed perceived exertion with the Borg scale [51].
Assessment of personality factors
In three studies [30–32], assessments of perceived game experience and physical performance were compared with assessments of personality as suggested by Goldberg [52] (Table 5).
Influence of multiplayer games on game experience and performance
The majority of the reviewed studies state that social interaction through multiplayer game settings improve both game experience and game performance (Fig. 2). All studies that examined game experience stated that their results proved that multiplayer modes positively influenced game experience [24, 26, 28–32, 34, 35]. Most studies stated that multiplayer games led to better game performances or higher physical exertion in most of the measured dimensions [24, 26, 27, 29, 31, 34]. Two studies stated that single-player mode improved game performance when compared to multiplayer modes or increased physical exertion in certain dimensions [29, 35]. Two studies also found correlation of game experience and game performance [25, 28].
Summary of the review. Multiplayer game modes (solid lines) have been shown to differ in their impact compared to single-player game modes (dotted lines) in health-related applications for two differently skilled players (dark red, light blue). In comparison to single-player modes (dotted lines), multiplayer game modes (solid lines) have been shown to positively influence both game experience (left) and game performance (right). The benefit of multiplayer modes is present for players of all skill levels (light blue representing less skilled players, dark red representing skilled players, respectively) and at all conditional task difficulties
Quality evaluation
The agreement on study quality between the two reviewers was almost perfect. The estimated Kappa value was 0.99 with a confidence interval ranging between 0.96 and 1.00 (α = 0.05). The percentage of agreement between the two reviewers was 99.3%. The mean quality score was 12.1 points (maximum: 22 points, range: 10-14 points), the median value was 12 points and the mode was 11 points. The mean score for reporting was 6.2 points (maximum: 9 points; range: 5-8 points), for external validity 0.8 points (maximum: 2 points; range: 0-2 points), for internal validity (bias) 3.9 points (maximum: 5 points; range: 3-4 points), for internal validity (confounding) 1.2 points (maximum: 5 points; range: 0-2 points) and for power 0.0 points (maximum 1 point; 0 points), see Additional file 2.
Discussion
Study selection
Our search resulted in thirteen articles fulfilling the inclusion criteria. These articles evaluated the psychological and physiological experience of multiplayer gaming using various forms of player interaction in studies on neuromuscular patients, overweight adolescents, and students. The thirteen articles remain small compared to 1031 papers related to “robot-assisted training” and to 444 papers related to “exergaming” that were published until July 2018 [53]. This highlights that multiplayer gaming seems not yet systematically considered in neuromuscular therapy or other health-related training. The combination of both the small number of included papers and the heterogeneity in their methods, prevents us from giving a clear answer to the research question.
Devices and games inconsistently discussed
The selection process of devices and of the commercially available games was consistently not discussed within any of the thirteen articles. The design process of the in-house developed games was discussed in three articles [27, 29, 34]. Feltz et al. [34] and Ganesh et al. [27] designed the games according to the paradigm to be tested, i.e., Koehler effect [44] and reactive motor adaptation, respectively. Mace et al. [29] explained the game design process, and how their multiplayer game mode explicitly demands a collaborative behavior [7]. To better discuss and compare the study results, the targeted behavior and the actual game modes selected or designed, should generally be reported in more detail.
Limited diversity of multiplayer modes
The multiplayer modes in the reviewed studies have been commonly named competitive, collaborative and cooperative mode. According to the taxonomy proposed by Jarrassé et al. [7] and others (Fig. 3), some modes of training in the reviewed studies should be termed co-active (a task that can be solved by individual player) rather than cooperative (playing in the same team with different roles according to own individual skills, thus, a role being either “supported” or “supportive”; see also Sawers and Ting [54]). Although a cooperative mode in that sense has not been considered in the reviewed studies, this mode may have great potential in robot-assisted rehabilitation systems since the roles of “supported” and “supportive” are often given in a therapeutic setting with patient and therapist [55, 56]. The collaborative mode (playing in the same team with equal roles) was used in four studies [27, 29, 31, 34]. Differences in effects of collaberative and competitive modes in health-promoting exergames have already been discussed by Marker and Staiano [57].
Determination of multiplayer modes. According to the taxonomies of Jarrassé [7], Konert [59], and Mueller [58], the applied multiplayer mode can be determined based on the task characteristics. The task characteristics and the players’ behavior define the behavioral characteristics of the multiplayer mode. The references of the included studies (black) and feasibility studies (white) are placed at the multiplayer modes applied in the corresponding studies. The full variety of multiplayer modes was not covered by the included studies and therefore needs further investigation
The distinction of competitive modes, as done by Mueller [58], seems reasonable to predict training outcome. For instance, in combat gameplay, physical effort was higher than in object competition when both players controlled the same object [24].
Coopetition, i.e., competition and cooperation in combination, has been proposed as a new mode of multiplayer gaming [59]. Derived from a business concept, coopetition can be linked to a social health behavior where people compare their health behavior with others sharing the same health-related goal [59, 60]. Platforms such as social media enable this health behavior. This “coopetitive” behavior can be targeted by games that monitor the health progress in an understandable and comparable representation, e.g. weight loss. Robot-assisted neurorehabilitation can provide similar representations using parameters such as range-of-motion assessments and performance measures in reaching tasks [61].
The potential of playing with friends and relatives as co-players
Another example to combine competition and cooperation has been tested in a game-based learning study [62]. In this study, a second level of social interaction was introduced. In the first level of interaction, the teams solved subtasks together in a cooperative manner. In the second level of interaction, the teams interacted with other teams in a competitive manner. Such combinations of competition and cooperation are applicable in clinical environments. Subgroups of patients in clinics or different clinics could compete against each other. Part of the competition could be a comparison of therapy progress, duration of device usage, or virtual points in rehabilitation games. The groups of patients can motivate their teammates to cooperatively contribute to their team’s score by training more. Such a setup is possible in telerehabilitation since participating in remote places does not seem to reduce the motivation to compete: game experience and game score have been shown not to be affected when competing in different rooms compared to cooperating in the same room [35].
Cooperating in a team might be even more motivating when the team member is a friend or a relative. Playing with a friend in a cooperative mode resulted in greater goal commitment compared to playing with a stranger [36]. Regarding rehabilitation games, this implies that in a cooperative game, a well-known game partner – such as a family member – may be preferred over a lesser-known patient or therapist. For competitive modes it was already shown that playing with well-known game partners does improve the game experience when compared to playing with lesser known game partners [31]. Such game play requires input devices for different skill levels, e.g. rehabilitation robots for a severely impaired patient and exergaming devices, such as a Nintendo Wii, for moderately impaired and unimpaired players. Tasks solved by players with different skill levels were discussed in the concepts of flow and of the challenge point framework [9, 16]. However, both concepts are limitedly discussed with regard to multiplayer games using different input devices for different skill levels.
Balancing multiplayer interventions in the context of the flow model and the challenge point framework
The flow model considers the immediate task difficulty and current skill level, thus, defines whether the player is experiencing boredom, flow or frustration. We use this model as a basis for a description of a player’s specific skill level: the game experience profile. The inverted-U-shape dependency of game experience on task difficulty has been confirmed in multiplayer health-related games [34]. The global maximum of the game experience profile represents the point of highest rated game experience. Flow is expected to be experienced close to this point.
Different game modes result in differently shaped game experience profiles. These profiles are influenced by both the game mode and the players’ skills in the game, thus, resulting in different maxima. According to the flow model, the game experience of skilled players demands higher task difficulty to achieve maximum game experience [9]. In studies applying multiplayer modes, a cooperative mode can result in a better game experience compared to a competitive mode [36]. This increase in game experience could be represented by an overall shift upwards in the game experience profile or a game experience profile with less declination; both represented by a widened flow zone — a hypothesis that has yet to be confirmed. Novak et al. [30] found that the player’s preferred mode may be predicted based on a correlation between the player’s preferred multiplayer mode and personality. Therefore, a player-tailored selection of multiplayer mode may result in a better game experience, too.
The challenge point framework, introduced by Guadagnoli [16], provides a theoretical basis to conceptualize the effects of task difficulty in motor learning. In competitive games, the game experience is linked to the skill level of the opponent [9]. A discrepancy in skill level makes the weaker player feel frustrated and the better player bored. Changing the conditions by adapting visual, auditory or haptic game elements can reduce the differences in skill level between players. The terminology of the challenge point framework can be used to describe the difficulty of any task, not only regarding the conditions, but also the characteristics of the task and the skill level of the subject.
Conditional task difficulty: complementing the terminology
The challenge point framework defined the terms nominal task difficulty, i.e. the characteristics of the task only, and the functional task difficulty, i.e. the difficulty of the task relative to the skill level of the player and the conditions under which the task is performed. However, the term functional task difficulty does not allow for a distinction of the skill level of the player and the conditions under which the task is performed. Therefore, we propose an extended definition by introducing the term conditional task difficulty, i.e. the difficulty of the task relative to the conditions under which the task is performed. This extended definition is particularly relevant for environments where the conditions of the task can be adapted online, as when using haptic robots that can adapt the support of a patient based on his/her performance. In addition, this extended definition allows to collectively report task difficulty when a game is played with different input devices by the involved players.
Changing the individual conditions to optimize game experience
By changing the condition individually, we can change the players’ game performance in relation to their individual skill level. In two included studies and five exemplary feasibility studies, game design features such as speed, frequency of actions, or avatar size were adapted in single and multiplayer rehabilitation games to account for differences in skill level [32, 33, 63–67]. Such design features may change the conditions generally for both players, instead of for each player individually. In addition, these features may visually reveal the different skill levels of the players which can be embarrassing for the worse performing player.
Robotic devices offer unique design features to tailor the condition to the players’ motor abilities [68]. For instance, haptic force fields have been used to adjust the task difficulty of a training task. These force fields could be used in multiplayer games to individually set the level of difficulty [69]. In all difficulty adapting strategies, we have to consider that a change in condition for one player implicitly introduces a change in condition for the other one, since the players are connected over the nominal task itself. If we support the less skilled player, or hinder the more skilled player, in succeeding in the task we approach the highest game experience for both players (Fig. 4).
Difficulty adaptation based on individual condition setting in multiplayer games. Game experience (left) can be optimized by balancing the game performance (right). – Left: The initial game experience under nominal conditions relates to the skill level of the opponent and is non-optimal for differently skilled players (squares). Optimal game experience is perceived by the players when the condition adapts the difficulty towards the players’ skill level (circles). – Right: A common initial game performance state consists of a conditional task difficulty and its corresponding player specific game performance (square). Player specific difficulty adaptation can balance the game performances of the two players (circles)
Performance-balanced games improved game experience compared to non-balanced games in one of the included studies and several feasibility studies [33, 63, 64, 69]. In therapy settings where the level of difficulty cannot be individually adapted, collaborative game designs may be preferred since they may require less similarity in skill level to achieve flow [29].
The motivation in cooperative modes can even be increased with conjunctive tasks design (i.e. collaborative according to Jarrassé [7]) facilitating the Koehler effect [70]. The Koehler effect occurs when an inferior team member performs a difficult task better in a team than one would expect from knowledge of his or her individual performance. The effect has been found to be strongest in conjunctive task conditions in which the group’s potential productivity towards a cooperative goal is equal to the productivity of its least capable member [71]. Experience with the Koehler effect implies that multiplayer difficulty adaptation targeting a moderate discrepancy compared to the player’s performance can optimise the game experience of the participants [27, 34]. Such small discrepancies can be achieved by robotic devices as they can individually assist or challenge players [15].
Different assessments hinder inter-study comparison
Assessment of perceived game experience
In general, measures of game experience can be acquired using physiological measures and questionnaires; only the latter were present in the included papers. Physiological measures, e.g. sensors measuring cardiac activity, significantly improve the assessment of game experience [72]. However, the required additional set of sensors may disturb game play and effort by the study operators. Among the presented questionnaires, the Intrinsic Motivation Inventory is widely used in various sports interventions and can therefore allow inter-study comparison [37]. However, the Intrinsic Motivation Inventory does not necessarily have good discriminative power as pointed out by one study [30]. That is why the participants have been asked to also rank the games regarding game experience related questions (e.g. “Which game mode do you prefer?”).
Inventories regarding psychosocial attractiveness of game design in the dimensions of social interaction, collaboration, individual feedback and team feedback, can extend motivation inventories when different multiplayer modes are compared [25]. The perceived multiplayer mode, e.g. playing in a cooperative or competitive mode, will influence the answers regarding psychosocial attractiveness. However, the multiplayer mode might not be perceivable to all players. Thus, should be checked after each mode if the mode is not introduced accordingly [36].
The benefit of social interaction could be increased by integrating visual, auditory/verbal, and haptic elements. In the cognitive task study of Yu et al. [62] conversely, hiding of the opponent team (anonymous opponents) in a competitive mode showed improved game experience and satisfaction compared to visual presence of the opponent team. Hence, visual presence of the opponent seems to negatively influence game experience. In contrast, Johnson et al. [28] stated that the more modalities of social interaction integrated in motor tasks, the more enjoyable for the players. The perceived intensity of social interaction in different modalities seems to vary between task and player. In the study of Yu et al. [62] the intensity of social interaction within the team members might superpose any interaction with the opponent team. Therefore, assessing the perceived intensity of social interaction provided by different modalities and involved people is suggested. In the study of Gorsic et al. [32], the level of verbal interaction was assessed by a study operator. However, a standardized assessment for perception of interaction intensity has not been established yet.
Assessment of physical performance
Energy expenditure is a common measure for physical fitness regarding physical performance [73]. Various methods to measure energy expenditure or their consequences include: the determination of acceleration profiles [25, 35], feature extraction of the velocity profile [28, 31, 33], measuring the weight drop [26], and measuring the heart rate [34]. Alternatively, the Borg scale, used as a measure of perceived physical performance in one study, is a widely accepted and valid measure of exercise intensity [34, 51, 74]. Both methods presented allow inter-game comparison or even comparisons regarding physical performance with non-gaming interventions.
Physical performance corresponds to training intensity and, therefore, predicts improvement in motor function, too [75]. However, neuromuscular therapy and movement training in general are related to motor function and execution and rather than to physical exertion only. Consequently, measures of physical fitness may not be sufficient to provide all relevant information regarding progress in physical functioning. Therefore, game performance score or gamified assessments based on conventional motor functional assessments (e.g. Fugl-Meyer [76]) should be included to complement the evaluation of the influence of the game on physical functioning progress [61].
In games where the end effector of the input device controls the avatar, the end effector position affects the success or failure within the game task. Assessing the end effector position and its impact on task success does not demand functional movement patterns in arm joints. However, functional movements are important in upper limb training. In exoskeleton devices used for rehabilitation training it is possible to measure the movements of non-game controlling joints [77]. One solution is the assessment of quality of movements to measure the player’s functional physical performance. Assessments that are independent of task dimensions do not need a developer’s definition such as the spectral arc length metric [78]. However, the validity of movement quality assessments is under discussion [79].
Limitations
We used a structured protocol to guide our search strategy, study selection, extraction of data and statistical analysis. However, limitations of this review should be noted: a publication bias may be present, as well as a language bias, given that we considered only interventions described in published studies and restricted our search to English and German language publications. A bias regarding research fields was generated since we mainly focused on neuromuscular therapy and health-related training.
Conclusions
Multiplayer modes can enhance the players’ perceived game experience and positively influence the players’ performance. Based on the small number of relevant studies published so far, a conclusion cannot yet be drawn about which multiplayer mode is best during neurorehabilitation training. A meta-analysis of game experience and game performance outcomes may be suggested as soon as more multiplayer studies with homogeneous outcome measures will be published. Nevertheless, this review demonstrated that the players’ individual skill levels and personalities, as well as their role in the game, must be taken into account when selecting and designing multiplayer modes.
Based on the model of flow and the challenge point framework, we suggest an individual adaptation of game conditions, i.e. conditional task difficulty, to assimilate differently skilled players for an optimal game experience. Furthermore, player specific selection of multiplayer modes may result in more robust interventions regarding game experience and requires less assimilation of differently skilled players.
We suggest breaking the limited variety in multiplayer modes and fully exploring multiplayer modes and co-player’s characteristics such as the co-players presence, skill level, personality and relation to the player. We further suggest that future studies use a more stringent research design in which participants are allocated to either single play or multiplayer modes of exercise through randomised assignment.
References
Krucoff MO, Rahimpour S, Slutzky MW, Edgerton VR, Turner DA. Enhancing nervous system recovery through neurobiologics, neural interface training, and neurorehabilitation. Front Neurosci. 2016; 10(DEC). https://doi.org/10.3389/fnins.2016.00584.
World Health Organisation. The World Health Report 2001. Soc Psychiatry Psychiatr Epidemiol. 2001; 36:473–4. https://doi.org/10.1007/s001270170010.
Krakauer JW. Motor learning: its relevance to stroke recovery and neurorehabilitation. Curr Opin Neurol. 2006; 19(1):84–90. https://doi.org/10.1097/01.wco.0000200544.29915.cc.
Reinkensmeyer DJ, Emken JL, Cramer SC. Robotics, motor learning, and neurologic recovery. Annu Rev Biomed Eng. 2004; 6:497–525. https://doi.org/10.1146/annurev.bioeng.6.040803.140223.
Adamovich SV, Fluet GG, Tunik E, Merians AS. Sensorimotor training in virtual reality: A review. NeuroRehabilitation. 2009; 25(1):29–44. https://doi.org/10.3233/NRE-2009-0497.
Gajadhar BJ, Kort YAWD, Ijsselsteijn W. Shared Fun Is Doubled Fun : Player Enjoyment as a Function of Social Setting. Lect Notes Comput Sci. 2008; 5294:106–117.
Jarrassé N, Charalambous T, Burdet E. A framework to describe, analyze and generate interactive motor behaviors. PloS ONE. 2012; 7(11):49945. https://doi.org/10.1371/journal.pone.0049945.
Sweetser P, Wyeth P. GameFlow: A Model for Evaluating Player Enjoyment in Games. Comput Entertain. 2005; 3(3):3. https://doi.org/10.1145/1077246.1077253.
Csikszentmihalyi M. Play and intrinsic rewards. In: Flow and the Foundations of Positive Psychology: The Collected Works of Mihaly. Csikszentmihalyi. Dordrecht: Springer: 2014. p. 135–53. https://doi.org/10.1007/978-94-017-9088-8_10.
Oxendine JB. Emotional arousal and motor performance. Quest. 1970; 13(1):23–32. https://doi.org/10.1080/00336297.1970.10519673.
Djaouti D, Alvarez J, Jessel J-P. Classifying serious games: The G/P/S model. Hand Res Improv learn Motiv Educ games: Multidiscip. 2011:118–36. https://doi.org/10.4018/978-1-60960-495-0.ch006.
Kwakkel G, Kollen BJ, van der Grond J, Prevo AJH. Probability of Regaining Dexterity in the Flaccid Upper Limb: Impact of Severity of Paresis and Time Since Onset in Acute Stroke. Stroke. 2003; 34(9):2181–6. https://doi.org/10.1161/01.STR.0000087172.16305.CD.
Sveistrup H. Motor rehabilitation using virtual reality. J NeuroEngineering Rehabil. 2004; 1:1–8. https://doi.org/10.1186/1743-0003-1-10.
Lang CE, Lohse KR, Birkenmeier RL. Dose and timing in neurorehabilitation: Prescribing motor therapy after stroke. Curr Opin Neurol. 2015; 8(6):549–55. https://doi.org/10.1158/1940-6207.CAPR-14-0359.Nrf2-dependent 1533440615334406.
Brown DA, Lee TD, Reinkensmeyer DJ, Duarte J. E. In: Reinkensmeyer DJ, Dietz V, (eds).Designing Robots That Challenge to Optimize Motor Learning. Cham: Springer; 2016, pp. 39–58. https://doi.org/10.1007/978-3-319-28603-7_3.
Guadagnoli MA, Lee TD. Challenge Point: A Framework for Conceptualizing the Effects of Various Practice Conditions in Motor Learning. J Mot Behav. 2004; 36(2):212–24. https://doi.org/10.3200/JMBR.36.2.212-224.
Tanaka K, Parker J, Baradoy G, Sheehan D, Holash JR, Katz L.A Comparison of Exergaming Interfaces for Use in Rehabilitation Programs and Research. Loading. 2012; 6(9):69–81.
Caspersen CJ, Powell KE, Christenson GM. Physical activity, exercise, and physical fitness: definitions and distinctions for health-related research. Public Health Rep. 1985; 100(2):126–31. https://doi.org/10.2307/20056429.
Thomas J, Harden A. Methods for the thematic synthesis of qualitative research in systematic reviews. BMC Med Res Methodol. 2008; 8:45. https://doi.org/10.1186/1471-2288-8-45.
Barnett-Page E, Thomas J. Methods for the synthesis of qualitative research: a critical review. BMC Med Res Methodol. 2009; 9:59. https://doi.org/10.1186/1471-2288-9-59.
Downs SH, Black N. The feasibility of creating a checklist for the assessment of the methodological quality both of randomised and non-randomised studies of health care interventions. J Epidemiol Community Health. 1998; 52(6):377–84.
Landis JR, Koch GG. An application of hierarchical kappa-type statistics in the assessment of majority agreement among multiple observers,. Biometrics. 1977; 33(2):363–74. https://doi.org/10.2307/2529786.
Liberati A, Altman DG, Tetzlaff J, Mulrow C, Gotzsche PC, Ioannidis JP, Clarke M, Devereaux PJ, Kleijnen J, Moher D. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate healthcare interventions: explanation and elaboration. BMJ. 2009; 339:2700. https://doi.org/10.1136/bmj.b2700.
Verhoeven K, Abeele VV, Gers B, Seghers J. Energy Expenditure During Xbox Kinect Play in Early Adolescents: The Relationship with Player Mode and Game Enjoyment. Game Health J. 2015; 4(6):444–51. https://doi.org/10.1089/g4h.2014.0106.
Staiano AE, Abraham AA, Calvert SL. Motivating Effects of Cooperative Exergame Play for Overweight and Obese Adolescents. J Diabetes Sci Technol. 2012; 6(4):812–9. https://doi.org/10.1177/193229681200600412.
Staiano AE, Abraham AA, Calvert SL. Adolescent exergame play for weight loss and psychosocial improvement: A controlled physical activity intervention. Obesity. 2013; 21(3):598–601. https://doi.org/10.1002/oby.20282. NIHMS150003NIHMS150003.
Ganesh G, Takagi A, Osu R, Yoshioka T, Kawato M, Burdet E. Two is better than one: physical interactions improve motor performance in humans. Sci Rep. 2014; 4:3824. https://doi.org/10.1038/srep03824.
Johnson MJ, Loureiro RCV, Harwin WS. Collaborative tele-rehabilitation and robot-mediated therapy for stroke rehabilitation at home or clinic. Intel Serv Robotics. 2008; 1(2):109–21. https://doi.org/10.1007/s11370-007-0010-3.
Mace M, Kinany N, Rinne P, Rayner A, Bentley P, Burdet E. Balancing the playing field: Collaborative gaming for physical training. J NeuroEngineering Rehabil. 2017; 14(1):1–18. https://doi.org/10.1186/s12984-017-0319-x.
Novak D, Nagle A, Keller U, Riener R. Increasing motivation in robot-aided arm rehabilitation with competitive and cooperative gameplay. J Neuroengineering Rehabil. 2014; 11(1):64.
Goršič M, Cikajlo I, Novak D. Competitive and cooperative arm rehabilitation games played by a patient and unimpaired person : effects on motivation and exercise intensity. J NeuroEngineering Rehabil. 2017; 14(1):1–18. https://doi.org/10.1186/s12984-017-0231-4.
Goršič M, Cikajlo I, Goljar N, Novak D. A multisession evaluation of an adaptive competitive arm rehabilitation game. J NeuroEngineering Rehabil. 2017; 14(1):128. https://doi.org/10.1186/s12984-017-0336-9.
Goršič M, Darzi A, Novak D. Comparison of two difficulty adaptation strategies for competitive arm rehabilitation exercises. IEEE Int Conf Rehabil Robot. 2017; 2017:1–6.
Feltz DL, Irwin B, Kerr N. Two-player partnered exergame for obesity prevention: using discrepancy in players’ abilities as a strategy to motivate physical activity. J Diabetes Sci Technol. 2012; 6(4):820–7. https://doi.org/10.1177/193229681200600413.
Peng W, Crouse J. Playing in parallel: the effects of multiplayer modes in active video game on motivation and physical exertion. Cyberpsychology Behav Soc Netw. 2013; 16(6):423–7. https://doi.org/10.1089/cyber.2012.0384.
Peng W, Hsieh G. The influence of competition, cooperation, and player relationship in a motor performance centered computer game. Comput Hum Behav. 2012; 28(6):2100–6. https://doi.org/10.1016/j.chb.2012.06.014.
McAuley E, Duncan T, Tammen VV. Psychometric properties of the Intrinsic Motivation Inventory in a competitive sport setting: a confirmatory factor analysis. 1989. https://doi.org/10.1080/02701367.1989.10607413.
Malone TW. Toward a Theory of Intrinsically Motivating Instruction *. Cogn Sci Multidiscip J. 1981; 5(4):333–69. https://doi.org/10.1207/s15516709cog0504.
Poels K, IJsselsteijn WA, de Kort YAW. Development of the Kids Game Experience Questionnaire: A self report instrument to assess digital game experiences in children. Poster Presentation at the Meaningful Play Conference, 2008. East Lansing, MI: 2008.
Song H, Kim J, Tenzek KE, Lee KM. The effects of competition and competitiveness upon intrinsic motivation in exergames. Comput Human Behav. 2013; 29(4):1702–8. https://doi.org/10.1016/j.chb.2013.01.042.
Hollenbeck JR, Klein HJ, O’Leary AM, Wright PM. Investigation of the construct validity of a self-report measure of goal commitment. J Appl Psychol. 1989; 74(6):951–6. https://doi.org/10.1037/0021-9010.74.6.951.
Tziner A, Kopelman RE, Livneh N. Effects of performance appraisal format on perceived goal characteristics, appraisal process satisfaction, and changes in rated job performance: A field experiment. J Psychol Interdisc Appl. 1993; 127(3):281–91. https://doi.org/10.1080/00223980.1993.9915562.
Johnston JD, Sheldon L, Massey AP. Influencing physical activity and healthy behaviors in college students: lessons from an alternate reality game. In: Serious Game Design and Development: Technologies for Training and Learning. Hershey: IGI Global: 2010. p. 270–89. https://doi.org/10.4018/978-1-61520-739-8-ch015. http://www.igi-global.com/gateway/chapter/41078.
Köhler O. Über den Gruppenwirkungsgrad der menschlichen Körperarbeit und die Bedingung optimaler Kollektivkraftreaktion. Industrielle Psychotechnik. 1927; 4:209–26.
Sallis JF, Pinski RB, Grossman RM, Patterson TL, Nader PR. The development of self-efficacy scales for healthrelated diet and exercise behaviors. Health Educ Res. 1988; 3(3):283–92. https://doi.org/10.1093/her/3.3.283.
Rosenberg M. Conceiving the self. New York: Basic Books; 1979.
Bukowski W, Hoza B, Boivin M. Measuring friendship quality during pre and early adolescence: the development and psychometric properties of the friendship qualities scale. J Soc Pers Relatsh. 1994; 11:471–84. https://doi.org/10.1177/0265407594113011.
Ainsworth BE, Haskell WL, Whitt MC, Irwin ML, Swartz AM, Strath SJ, O’Brian WL, Bassett DR, Schmitz KH, Emplaincourt PO, Jacobs DR, Leon AS. Compendium of Physical Activities: an update of activity codes and MET intensities. Med Sci Sports Exerc. 2000; 32(Supplement):498–516. https://doi.org/10.1097/00005768-200009001-00009.
Puyau MR, Adolph AL, Vohra FA, Zakeri I, Butte NF. Prediction of activity energy expenditure using accelerometers in children. Med Sci Sports Exerc. 2004; 36(9):1625–31. https://doi.org/10.1249/01.MSS.0000139898.30804.60.
Van Der Pas SC, Verbunt JA, Breukelaar DE, Van Woerden R, Seelen HA. Assessment of arm activity using triaxial accelerometry in patients with a stroke. Arch Phys Med Rehabil. 2011; 92(9):1437–42. https://doi.org/10.1016/j.apmr.2011.02.021.
Borg G. Borg’s perceived exertion and pain scales. Champaign [etc.]: Human Kinetics; 1998.
Goldberg LR, Johnson JA, Eber HW, Hogan R, Ashton MC, Cloninger CR, Gough HG. The international personality item pool and the future of public-domain personality measures. J Res Personal. 2006; 40(1):84–96. https://doi.org/10.1016/j.jrp.2005.08.007.
Web of Science [v.5.21] - All Databases Results. http://apps.webofknowledge.com. Accessed 04 July 2018.
Sawers A, Ting LH. Perspectives on human-human sensorimotor interactions for the design of rehabilitation robots. J Neuroengineering Rehabil. 2014; 11(1):142. https://doi.org/10.1186/1743-0003-11-142.
Lanini J, Tsuji T, Wolf P, Riener R, Novak D. Teleoperation of two six-degree-of-freedom arm rehabilitation exoskeletons. In: IEEE International Conference on Rehabilitation Robotics. 2015. p. 514–9. https://doi.org/10.1109/ICORR.2015.7281251.
Song A, Wu C, Ni D, Li H, Qin H. One-Therapist to Three-Patient Telerehabilitation Robot System for the Upper Limb after Stroke. Int J Soc Robot. 2016; 8(2):319–29. https://doi.org/10.1007/s12369-016-0343-1.
Marker AM, Staiano AE. Better Together: Outcomes of Cooperation Versus Competition in Social Exergaming. Game Health J. 2014; 4(1):141017093454003. https://doi.org/10.1089/g4h.2014.0066.
Mueller FF, Gibbs MR, Vetere F. Taxonomy of exertion games. Proceedings of the 20th Australasian Conference on Computer-Human Interaction Designing for Habitus and Habitat - OZCHI ’08. 2008:263–266. https://doi.org/10.1145/1517744.1517772.
Konert J, Göbel S, Steinmetz R. Video Game Personalization via Social Media Participation. Cham: Springer; 2014.
Brandenburger A, Nalebuff BJ. Co-opetition. New York: Currency Doubleday; 1998.
Keller U, Schölch S, Albisser U, Rudhe C, Curt A, Riener R, Klamroth-Marganska V. Robot-assisted arm assessments in spinal cord injured patients: A consideration of concept study. PLoS ONE. 2015; 10(5):0126948. https://doi.org/10.1371/journal.pone.0126948.
Yu F-Y, Han C, Chan T-W. Experimental comparisons of face-to-face and anonymous real-time team competition in a networked gaming learning environment. Cyberpsychology Behav Impact Internet Multimedia Virtual Reality Behav Soc. 2008; 11(4):511–4. https://doi.org/10.1089/cpb.2007.0171.
Caurin GA, Siqueira AA, Andrade KO, Joaquim RC, Krebs HI. Adaptive strategy for multi-user robotic rehabilitation games. Conference proceedings :... Annual International Conference of the IEEE Engineering in Medicine and Biology Society. IEEE Eng Med Biol Soc Conf. 2011; 2011(1):1395–8. https://doi.org/10.1109/IEMBS.2011.6090328.
Maier M, Ballester BR, Duarte E, Duff A, Verschure PFMJ. Social Integration of Stroke Patients through the Multiplayer Rehabilitation Gaming System. In: Games for Training, Education, Health and Sports Lecture Notes in Computer Science Vol. 8395.2014. p. 100–14.
Ballester BR, Badia SBI, Verschure PFMJ. Including Social Interaction in Stroke VR-Based Motor Rehabilitation Enhances Performance: a Pilot Study. Presence. 2012; 21(4):490–501.
Mihelj M, Novak D, Milavec M, Ziherl J, Olenšek A, Munih M. Virtual Rehabilitation Environment Using Principles of Intrinsic Motivation and Game Design. Presence: Teleoperators and Virtual Environments. 2012; 21(1):1–15. https://doi.org/10.1162/PRES_a_00078.
Wittmann F, Lambercy O, Gonzenbach RR, Van Raai MA, Hover R, Held J, Starkey ML, Curt A, Luft A, Gassert R. Assessment-driven arm therapy at home using an IMU-based virtual reality system. IEEE Int Conf Rehabil Robot. 2015; 2015:707–12. https://doi.org/10.1109/ICORR.2015.7281284.
Colombo R, Pisano F, Mazzone A, Delconte C, Micera S, Carrozza MC, Dario P, Minuco G. Design strategies to improve patient motivation during robot-aided rehabilitation. J Neuroengineering Rehabil. 2007; 4:3. https://doi.org/10.1186/1743-0003-4-3.
Baur K, Wolf P, Riener R, Duarte J. Making neurorehabilitation fun: Multiplayer training via damping forces balancing differences in skill levels. Proc 2017 IEEE Int Conf Rehabil Robot. 2017; 2017:876–81.
Hertel G, Kerr NL, Messé LA. Motivation gains in performance groups: paradigmatic and theoretical developments on the Köhler effect. J Personal Soc Psychol. 2000; 79(4):580–601. https://doi.org/10.1037/0022-3514.79.4.580.
Feltz DL, Kerr NL, Irwin BC. Buddy Up: The Kohler Effect Applied to Health Games. J Sport Exerc Psychol. 2011; 33(4):506–26. https://doi.org/10.1123/jsep.33.4.506.
Bastarache-Roberge MC, Léger PM, Courtemanche F, Sénécal S, Fredette M. Measuring flow using psychophysiological data in a multiplayer gaming context. In: Lecture Notes in Information Systems and Organisation. Cham: Springer: 2015. p. 187–91. https://doi.org/10.1007/978-3-319-18702-0_25.
Levine JA. Measurement of energy expenditure. Public Health Nutr. 2005; 8(7A):1123–32. https://doi.org/10.1079/PHN2005800.
Chen MJ, Fan X, Moe ST. Criterion-related validity of the Borg ratings of perceived exertion scale in healthy individuals: a meta-analysis. J Sports Sci. 2002; 20(11):873–99. https://doi.org/10.1080/026404102320761787.
Ouellette MM, Lebrasseur NK, Bean JF, Phillips E, Frontera WR, Fielding RA. High-Intensity Resistance Training Improves Muscle Strength, Self-Reported Function, and Disability in Long-Term Stroke Survivors. Stroke. 2004; 35:1404–9. https://doi.org/10.1161/01.STR.0000127785.73065.34.
Fugl-Meyer AR, Jääskö L, Leyman I, Olsson S, Steglind S. The post-stroke hemiplegic patient. 1. a method for evaluation of physical performance. Scand J Rehabil Med. 1975; 7(1):13–31. https://doi.org/10.1038/35081184.
Klamroth-Marganska V, Blanco J, Campen K, Curt A, Dietz V, Ettlin T, Felder M, Fellinghauer B, Guidali M, Kollmar A, Luft A, Nef T, Schuster-Amft C, Stahel W, Riener R. Three-dimensional, task-specific robot therapy of the arm after stroke: A multicentre, parallel-group randomised trial. Lancet Neurol. 2014; 13(2):159–66. https://doi.org/10.1016/S1474-4422(13)70305-3.
Balasubramanian S, Melendez-Calderon A, Roby-Brami A, Burdet E. On the analysis of movement smoothness. J NeuroEngineering Rehabil. 2015; 12(1):112. https://doi.org/10.1186/s12984-015-0090-9.
Nordin N, Xie SQ, Wünsche B. Assessment of movement quality in robot- assisted upper limb rehabilitation after stroke: a review. J Neuroengineering Rehabil. 2014; 11:137. https://doi.org/10.1186/1743-0003-11-137.
Acknowledgements
None.
Funding
This work was supported by the Swiss National Science Foundation (Grant No. SNF-160313) through the National Centre of Competence in Research on Robotics (NCCR) and by the ETH Foundation through ETH Research Grant ETH-17 13-2.
Author information
Authors and Affiliations
Contributions
KB designed the search protocol, screened the literature, assessed the quality of the included papers and drafted the manuscript.
AS designed the search protocol, screened the literature and assessed the quality of the included papers.
EB conceived of the review and supported and reviewed the design of the search protocol.
RR revised conception and design of the review, and participated in the elaboration of background literature.
JD reviewed the manuscript regarding included concepts.
PW conceived of the review and supported and reviewed the design of the search protocol, and helped to draft the manuscript.
All authors were revising the manuscript. All authors read and approved the final manuscript.
Corresponding author
Ethics declarations
Ethics approval and consent to participate
Not applicable.
Consent for publication
Not applicable.
Competing interests
The authors declare that they have no competing interests.
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Additional files
Additional file 1
Search strategy. (DOC 31 kb)
Additional file 2
Assessment of methodological quality. (DOC 78 kb)
Rights and permissions
Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver(http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.
About this article
Cite this article
Baur, K., Schättin, A., de Bruin, E.D. et al. Trends in robot-assisted and virtual reality-assisted neuromuscular therapy: a systematic review of health-related multiplayer games. J NeuroEngineering Rehabil 15, 107 (2018). https://doi.org/10.1186/s12984-018-0449-9
Received:
Accepted:
Published:
DOI: https://doi.org/10.1186/s12984-018-0449-9