Abstract
Anxiety disorder is the most prevalent and frequently diagnosed disorder in youth, and associated with serious negative health outcomes. Our most effective prevention programs, however, have several limitations. These limitations can be addressed using game-based interventions. Results from two randomized controlled trials on the video game MindLight show improvements in anxiety that are maintained up to 6 months. The game was designed based on evidence-based therapeutic techniques; however, it is unclear if children’s engagement with these techniques actually predict improvements in anxiety symptoms. An important advantage of game-based interventions is that they provide excellent opportunities to isolate therapeutic action mechanisms and test their impact on intervention outcomes. In the current study, on-screen videotaped output while playing MindLight was coded and analysed for forty-three 8 to 12-year old children with elevated levels of anxiety. Results showed that changes in in-game play behaviours representing therapeutic exposure techniques predicted improvements in anxiety symptoms 3 months later (when children had not played the game for 3 months). The current study is a first step towards identifying and validating game mechanics that can be used in new applied games to target anxiety symptoms or other psychopathologies with the same underlying deficits.
Similar content being viewed by others
Avoid common mistakes on your manuscript.
Anxiety disorders are the earliest form of psychopathology to emerge in childhood, the most prevalent and frequently diagnosed disorders in youth (Beesdo et al. 2009), and associated with adverse health outcomes (Brent et al. 1986; Essau 2003; Kessler et al. 1996; Pine et al. 1998; Weissman et al. 1999; Woodward and Fergusson 2001). Beyond the numbers for clinical diagnoses, sub-clinical levels of anxiety symptoms are estimated at 40% in children (Muris et al. 2000a), which increase the risk for full-blown anxiety disorders at older ages. Currently, programs based on Cognitive Behavioural Therapy (CBT) have been shown to be most effective (Butler et al. 2006; Fisak Jr et al. 2011; In-Albon and Schneider 2007; Kendall 2011; Mychailszyn et al. 2012). However, CBT outcomes are mixed and effect sizes are small to moderate (Fisak Jr et al. 2011; Mychailszyn et al. 2012).
These disappointing outcomes of CBT-based indicated prevention programs might be related to limitations regarding the way in which interventions are delivered, rather than the therapeutic principles on which they are based (Granic et al. 2014; Kazdin 2011). These limitations include a didactic-based approach that might not be engaging and motivating for some children (Gosch et al. 2006), few opportunities to practice newly-acquired knowledge, non-adherence of practitioners to the protocol (Eichstedt et al. 2014), and low accessibility and high costs of interventions (Collins et al. 2004; Farmer et al. 1999; Kataoka et al. 2002).
The aforementioned limitations can be addressed by using game-based interventions (see Granic et al. 2014 for full discussion; Rideout et al. 2010; Olson 2010). On this basis, the video game MindLight (GainPlay Studio 2014) was developed in order to prevent the escalation of anxiety in at-risk children. The game incorporates evidence-based techniques by translating these techniques into game mechanics. Game mechanics are the actions in the game that are designed for the player to repeat over and over; they are the vehicles by which certain skills are trained (see Table 1). Results from an indicated randomized controlled trial (RCT; Schoneveld et al. 2016) showed that after playing 5 sessions of MindLight, at-risk children showed significant reductions in anxiety symptoms by 3-months follow-up. In addition, a second indicated RCT showed that playing MindLight for 6 sessions is as effective as a CBT-based indicated prevention program (i.e., Coping Cat; van Starrenburg et al. 2017) in reducing anxiety symptoms from pre- to post-measurement, including the 3- and 6-months follow-up (Schoneveld et al. 2018).Footnote 1 Finally, results from a third study showed that compared to online CBT-based psychoeducation, playing MindLight resulted in similar improvements in anxiety symptoms and state anxiety in response to a (social and cognitive) stressor (Tsui et al. 2018, manuscript submitted for publication).
These results are promising, but it remains unclear whether the means by which children improve in anxiety symptoms are through the game mechanics that were explicitly designed into MindLight. Investigating such in-game processes is extremely valuable, not only because it gives insight into who will benefit most from the game, but also into which game mechanics are most important for anxiety symptom-reduction more generally (and those that are in need of further development). By testing the effect of the game mechanics in MindLight, the current study provides a first step towards building a toolbox of validated game mechanics. In turn, those validated game mechanics that are able to change causal processes associated with the development and maintenance of anxiety could be used in new applied games targeting anxiety symptoms or other psychopathologies with the same underlying deficits.
MindLight
MindLight is a video game for children 8 to 12 years of age, designed and developed in a cross-disciplinary collaboration among developmental psychologists, clinicians, game designers, and children themselves. In the game, the player is in grandma’s dark and decrepit mansion and needs to save her from evil forces by chasing away or uncovering “fear events” and solving puzzles. The player plays through the game by using his/her “mindlight”, a beam of light at the end of an antenna attached to the magical hat the avatar is wearing. The player needs this light in order to move through the dark game environment.
MindLight incorporates three evidence-based techniques based on cognitive-behavioural principles: relaxation through neurofeedback training (Price and Budzynski 2009), exposure training (Feske and Chambless 1995), and attention bias modification (Bar-Haim 2010; Bar-Haim et al. 2011). These techniques have been repeatedly shown to address causal processes associated with the development and maintenance of anxiety such as avoidance and negative attention bias (Mathews and MacLeod 2005; Weersing et al. 2012). In MindLight, the three techniques are translated into specific game mechanics: neurofeedback, approaching fear events, and attention bias modification puzzles (see Table 1), together aiming at teaching children how to cope with anxiety in a playful manner. Below we explain these techniques and game mechanics in more detail. More information can also be found in Appendix 1.
The first technique incorporated in MindLight is relaxation training through the use of neurofeedback. In conventional neurofeedback training individuals are presented with real-time electroencephalogram (EEG) recordings from their brain and guided through relaxation exercises in order to keep their EEG waves consistent with identified proxies of relaxation (i.e., reduction in relative beta power and increase in relative alpha power; Price and Budzynski 2009). In MindLight, the player wears an EEG headset with one (dry) sensor touching the forehead and one reference point located in the clip attached to the left earlobe. The headset detects EEG signals and converts them into a continuous data stream representing relaxation (Johnstone et al. 2012). This data stream is fed into the game and controls the amount of light that shines in the game. The brightness of players’ mindlight is proportional to the strength of the real-time relaxation of the player. When the player becomes more relaxed the mindlight becomes brighter, providing more light in the game environment. This way, players get feedback on their level of relaxation. Thus, relaxation is trained through the neurofeedback game mechanic in MindLight (see Table 1).
The second technique incorporated in MindLight is exposure training. There are fear events (i.e., fearful obstacles) in the game and by shining one’s mindlight on them they can be chased away or uncovered. Some fear events will then turn into a friendly kitten that follows the player and that reminds the player of past fears conquered. Other fear events turn into a benign object or animal and reward the player with a coin that is needed to unlock puzzles (see below). To play through the game, the fear events need to be approached. However, a player can also escape them (temporally) by hiding inside one of the chests or by turning on a ceiling light (both instances of safety behaviours). Thus, exposure training is done through the fear events in the game (see Table 1).
The third technique incorporated in MindLight is attention bias modification (ABM). Attentional biases characterized by a hyper attention towards potential threats play an important role in the development and maintenance of anxiety disorders (Muris 2016; Muris and Field 2008). With ABM-training, the attentional bias towards threats is retrained such that individuals pay more attention to positive stimuli rather than to negative stimuli (Bar-Haim 2010; Bar-Haim et al. 2011). Conventional ABM-procedures have used a modified version of the dot-probe task or a visual search task and have been shown to reduce anxiety (at least in the short term; Bar-Haim 2010; Bar-Haim et al. 2011; Hakamata et al. 2010). In MindLight, gaming elements have been added to the main principles of the dot-probe training. In the ABM puzzles, the player learns to focus on and attend to portraits of happy faces rather than threatening faces. Upon completion of the puzzle, the lights will turn back on in that particular room. Thus, ABM-training is done through the ABM-puzzles in MindLight (see Table 1).
In-Game Play Behaviours
Although the game was designed to provide children with repeated training opportunities to learn emotion-regulation strategies, there is also variability in how much children actually practice these skills. This variability is largely a function of the design of the game, which allows children to play at their own pace and explore and progress through the game in a variety of ways that foster a sense of autonomy and fun. For example, some children may be too afraid to move through the game at first, and will hide in chests to avoid the fearful stimuli, while others might prefer to explore a great deal more from the outset. These play-pattern differences may lead to differences in the amount of opportunities to practice the relaxation skills or to encounter fear events further in the game.
There are several specific in-game play behaviours that are most relevant to the intervention goals of MindLight (see Table 1). These behaviours can be classified into two types: “engaged” and “avoidant/safety” in-game play behaviours. Clinical research has shown that avoidant and/or safety behaviours are important maintenance processes for anxiety symptoms (Clark 1999; McManus et al. 2008; Salkovskis et al. 1999; Salkovskis et al. 1996), and that reducing avoidant and/or safety behaviours and increasing engagement with the treatment are predictive of better treatment outcomes (e.g., Glenn et al. 2013; Morgan and Raffle 1999; Salkovskis et al. 1999).
While playing MindLight, children can show in-game play behaviours that indicate engagement versus avoidant/safety behaviour. To be successful at MindLight, children need to show engaged in-game play behaviours, which will lead to a higher “dosage”, or more practice, with the game mechanics. Avoidant/safety in-game play behaviours limit that “dosage”. Differences in the dosage may result in anxiety symptom-reduction for some children and no anxiety symptom-reduction for other children based on how they played the game.
Table 1 presents an overview of the in-game play behaviours categorized as “engaged” versus “avoidant/safety” and how these behaviours relate to the game mechanics and cognitive-behavioural principles that are incorporated in MindLight. In the “engaged” category, all behaviours represent experiences that support players’ practice of relaxation, exposure, and modifying attention biases. For example, by exploring children are more exposed to fear events, which they can chase away or decloak and from which they can learn how to regulate their anxiety in the face of perceived threat. By solving puzzles they learn to focus more on the positive faces than on the negative ones. The “avoidant/safety” category represents behaviours that interfere with the intervention goals in MindLight. For example, hiding inside a chest or being inactive reduces exposure to fear events in the game. Turning on ceiling lights minimizes relaxation training, because under the ceiling light monsters are less likely to show up and the child needs to rely less on his or her own mindlight to brighten the environment.
Design and Hypotheses
In the current study, participants were children with elevated levels of anxiety that participated in a RCT to test the effect of 6 play-sessions of MindLight compared to CBT (Schoneveld et al. 2018). Only children from the MindLight group were included in the current study. Engaged and avoidant/safety in-game play behaviours were coded for the first and last play-sessions to examine 1) how pretest anxiety scores were related to in-game play behaviours during the first play-session, and 2) whether changes in in-game play behaviours from the first to the last play-session predicted changes in anxiety symptoms at the 3-months follow-up assessment. We hypothesized that a) higher pretest anxiety scores would be related to less engaged, and more avoidant/safety, in-game play behaviours during the first play-session, and b) increases in engaged behaviours and decreases in avoidant/safety behaviours from the first to the last play-session would predict reductions in anxiety symptoms at the 3-months follow-up assessment.
Method
Participants
Forty-three children (20 boys, 23 girls) with elevated levels of anxiety participated in the current study. Participants’ age ranged from 8.17 to 12.65 years (M = 9.94, SD = 1.14) at pretest. All children attended primary school in the east of the Netherlands and 97.7% were born in the Netherlands
Procedure
The current study was part of a 2-armed indicated prevention RCT (Schoneveld et al. 2018) which has been approved by the ethics committee of the Faculty of Social Sciences. Within this RCT, the effect of MindLight on anxiety symptoms is compared to the effect of the CBT-program Coping Cat (Kendall and Hedtke 2006; van Starrenburg et al. 2017). Participants for this RCT were selected based on their score on the child-version of the Spence Children’s Anxiety Scale (SCAS-C; Spence 1998). Children were included with a total SCAS-score one standard deviation above the mean or when they scored one standard deviation above the mean on two subscales of the SCAS (not including the obsessive compulsive disorder subscale). Children that already received mental health care were excluded. Seven hundred and ninety-one children from eight primary schools filled out the screening. Two hundred and eight children met the inclusion criteria, of which 174 children agreed to participate in the study. Eighty-six children were randomly allocated to play MindLight. Seventy-two children completed the MindLight-intervention.
Participants played MindLight (Version 1.0.1; 2014) for a total of approximately 6 hours. Gameplay was broken down into six 1-hour sessions spread out over a 6-week period. MindLight sessions took place under supervision of two research assistants at the children’s own school after school hours. MindLight was played on 15.6-in. ASUS X552CL-SX033H laptops, using a MindWave headset (Version 1.1.23, Neurosky Inc. 2011; Johnstone et al. 2012) and a Xbox 360 controller. Children and their parents filled out several questionnaires including the Spence Children’s Anxiety Scale (Spence 1998) 2 weeks before and after playing MindLight, at 3-months and 6-months follow-up (when children had not played the game for 3 respectively 6 months).
Before coding the in-game data for the current study, only children that had completed both pre- and post-measurements of anxiety symptoms were included. In addition, only children with (almost) complete video data were selected: children with less than 4 recorded and/or attended play-sessions, with no recorded first and/or last play-session, with only a short video of the first session (i.e., 6 and 9 min), or with significant disturbancesFootnote 2 during gameplay were excluded. In total, video data of 43 children were coded and analysed. Children that were excluded did not significantly differ from the children that were included with respect to their sex, age and pretest anxiety scores (χ2(1, N = 77) = 0.53, p = .466; t(75) = −0.60, p = .551; and t(75) = 0.94, p = .351, respectively).
Measures
Anxiety Symptoms
The Spence Children’s Anxiety Scale (SCAS-C; Spence 1997, 1998) was used to assess anxiety symptoms before and after playing MindLight. This scale consisted of 44 items where children are asked how often (i.e., never, sometimes, often, always; scored as 0–3) they experience symptoms of six DSM-IV defined anxiety disorders, namely separation anxiety disorder (6 items), social phobia (6 items), panic disorder and agoraphobia (9 items), physical injury fears (5 items), generalized anxiety disorder (6 items), and obsessive-compulsive disorder (6 items). Six items were positive filler items to reduce negative response bias. Mean anxiety scores were calculated over the 38 anxiety items and used in the analyses. The SCAS-C is a reliable and valid measure (Birmaher et al. 1997; Muris et al. 1998; Muris et al. 2000b; Reynolds and Richmond 1978) and in the current study internal consistency was good; Cronbach’s alpha = .89 for pretest and .90 for posttest.
Coding of in-Game Play Behaviours
During the MindLight sessions, the on-screen output was videotaped using Fraps (Version 3.5.99; 2015). This program records the exact output of the screen (i.e., what the children see on their screens while playing the game). On-screen video data was coded using observational codes with the Noldus Observer XT (Version 11.5; Noldus Information Technology 2013). Real-time in-game play behaviours during the first and last play-sessionFootnote 3 were coded following the adapted version of the MindLight Coding System (i.e., MCS-II; based on Sherman 2015; see Appendix Table 6 for written guidelines of all codes) which has been developed to measure the frequency and duration of different in-game play behaviours. The MCS-II includes mutually exclusive codes for the: [1] type of engagement, [2] location in the game-environment, [3] presence of different kinds of fear events, and [4] brightness of the mindlightFootnote 4 (which could be “none”, “some”, or “total”). In the current study, codes of interest included four engaged in-game play behaviours and two avoidant/safety in-game play behaviours (see Table 1). Engaged behaviours were the frequency of defeat and the duration of exploration, attempts to decloak or attack fear events (“fear attempts”), and the brightness of the mindlight. Avoidant/safety behaviours were the frequency of ceiling light attempts, and the duration of hiding inside a chest. A detailed description of all codes can be found in Appendix Table 6.
The first author trained research assistants in the use of Noldus Observer XT and the MindLight Coding System-II. Training took approximately 8 hours and training materials included video data from excluded cases. Coders were blind to the hypotheses. Weekly to bi-weekly recalibration training and reliability checks were conducted to monitor coding and minimize coder drift. Fifteen percent of the total amount of coded video data was independently coded by 2 or more randomly-selected coders. The average reliability was .89 kappa (range .49–1.00).
Strategy of Analyses
The frequency and duration of in-game play behaviours were transformed to frequencies per minute and proportions, respectively, to control for differences in duration of play-sessions. For the brightness of the mindlight, only the durations of the two extremes (“none” and “total”) were used in the analyses. Then, descriptive statistics of the study variables and Pearson correlations between 1) pretest anxiety scores and in-game play behaviours during the first play-session, and 2) in-game play behaviours during the first play-session and the last play-session were examined. Second, two hierarchical regression analyses (one for engaged and one for avoidant/safety in-game play behaviours) were performed to examine whether changes in in-game play behaviours from the first to the last play-session (i.e., differences in frequencies/proportions calculated as last minus first play-session) were related to changes in anxiety symptoms 3 months later. The pretest anxiety score was entered in the first step. The difference in frequency or proportion of the specific in-game play behaviours were entered as a set in the second step. The anxiety score at the 3-months follow-up assessment was entered as dependent variable. Before running the regression models, statistical assumptions necessary for multiple regression (i.e., normal distribution of measurement error, homoscedasticity of the variances, linearity of the model, and multicollinearity) were tested and met. Nevertheless, bootstrapping (n = 5000) was used to ensure accurate and valid results (and taking into account the small sample size). One participant that was an outlier (z-score > |3|) on both pretest and posttest anxiety was excluded from the analyses.
Results
Descriptive Statistics
Means and standard deviations of the study variables are presented in Table 2. Contrary to what was expected, anxiety at pretest was not significantly associated with the in-game play behaviours during the first play-session (see Table 3). In addition, age and sex were not significantly associated with anxiety at pretest nor with the different in-game play behaviours during the first play-session.
Moderate to large correlations were found between several in-game play behaviours during the first play-session (see Table 3). Exploration was positively associated with total mindlight, and negatively associated with ceiling light attempts, hiding inside a chest, and fear attempts. Further, total mindlight was negatively associated with ceiling light attempts. Thus, a child that spent large amounts of time exploring the game environment showed longer periods of bright mindlight, fewer ceiling light attempts, and shorter periods of time hiding inside a chest and decloaking or attacking fear events. Longer periods of bright mindlight were associated with fewer ceiling light attempts.
Regarding the correlations between in-game play behaviours during the first and the last play-session, Table 4 shows that none of the in-game play behaviours were associated with one another over time except for the negative association between exploration during the first play-session and fear attempts during the last play-session. It might be that children that explored more during the first session defeated more fear events during the game, resulting in less fear attempts needed during the last session.
In-Game Play Behaviours and Improvements in Anxiety 3 Months Later
Table 5 presents hierarchical regression analyses predicting anxiety at the 3-months follow-up from the difference in in-game play behaviours from the first to the last play-session controlled for anxiety at pretest. Model statistics represent the difference in the unique proportion of variance explained by the engaged and avoidant/safety in-game play behaviours respectively, beyond the proportion of variance explained by anxiety at pretest. Anxiety at pretest explained 44.4% of the variance in anxiety at the 3-months follow-up.
We hypothesized that increases in engaged in-game play behaviours and decreases in safety/avoidant in-game play behaviours from the first to the last play-session would predict reduction in anxiety symptoms at the 3-months follow-up. In line with these expectations, Table 5 shows that increases in exploration were significantly associated with lower anxiety scores 3 months later. For the avoidant/safety in-game play behaviours increases in ceiling light attempts and time spent hiding inside chests were significantly associated with higher anxiety scores 3 months later.
Discussion
The video game MindLight translates evidence-based techniques into game mechanics that teach children how to cope with anxiety in a playful manner. The aim of the present study was to investigate whether children with elevated levels of anxiety improved in their anxiety levels through these game mechanics that were explicitly designed into MindLight. Based on the anxiety literature, two types of in-game play behaviours (i.e., “engaged” and “avoidant/safety”) that are most relevant to the intervention goals of MindLight were distinguished and coded during gameplay. First, contrary to what was expected, pretest anxiety scores were not associated with the different in-game play behaviours during the first play-session. Second, in line with our predictions, increases in one of the in-game engaged behaviours and decreases in the two avoidant/safety behaviours predicted lower anxiety scores at 3-months follow-up. Together, these findings suggest that mechanics related to exposure techniques predicted improvements in children’s anxiety symptoms.
Associations between Pretest Anxiety Scores and In-Game Play Behaviours
Our finding that none of the in-game play behaviours, and the avoidant/safety behaviours in particular, were associated with pretest anxiety scores was unexpected. Because safety behaviours are an important maintenance process in anxiety disorders (Clark 1999), it was expected that children higher in anxiety would try to create more light by turning on ceiling lights and to hide more inside chests, because these objects provide a way to reduce and/or avoid exposure to fear events. The current study is part of an indicated prevention trial in which selection was based on scoring one standard deviation above the mean and clinical cases that already received mental health care were excluded. Therefore, the current finding might be due to a restricted range in anxiety scores. It might also be that the first play-session was standardized in such a way (with a lot of cut-scenes explaining the game) that little room was left for children to show very different in-game play behaviours. Nevertheless, it is promising that irrespective of their pretest anxiety scores, children started playing MindLight in a similar way because this strengthens the findings for the second hypothesis; changes in in-game play behaviours and the predicted improvements in anxiety symptoms are not due to associations between initial levels of anxiety and in-game play behaviours during the first session. Furthermore, the finding that none of the in-game play behaviours during the first session was associated with behaviours during the last session (except one association) is promising, because this suggests that MindLight is able to change the way in which children continued playing the game after the first session, enhancing opportunities to change anxiety symptoms.
Additional findings showed negative associations between several engaged and avoidant/safety in-game play behaviours during the first play-session, which provide support for the theory-based distinction between these two types of behaviours and at the same time supports the contention that the MindLight Coding System-II is able to distinguish these conceptually different in-game play behaviours.
In-Game Play Behaviours Representing Exposure Predicted Improvements in Anxiety
Results showed that changes in in-game play behaviours representing therapeutic exposure techniques predicted improvements in anxiety symptoms 3 months later. Regarding the engaged in-game behaviours, exploring the fearful game-environment for longer periods of time predicted decreases in anxiety symptoms. The time spent on actual attempts to chase away these fearful stimuli did not predict changes in anxiety symptoms nor did the frequency of getting defeated by these fear events. Regarding the avoidant/safety in-game behaviours, more attempts to seek safety under a ceiling light and increases in time spent hiding inside chests (and therefore avoiding fearful stimuli) predicted higher anxiety symptoms 3 months later. These findings are in line with previous research on avoidant/safety behaviours during traditional therapy (Glenn et al. 2013; Hammond 2005; Morgan and Raffle 1999; Salkovskis et al. 1999; Price and Budzynski 2009). However, the present study also extends this research by investigating these predictors in a game-based intervention. The current findings suggest that avoidant/safety behaviors play a similar role in an applied video game as in traditional face-to-face therapy.
Moreover, relaxation during gameplay (indicated by no or total mindlight) did not predict changes in anxiety symptoms. It might be that relaxation during gameplay is not as important for improvements in anxiety as the exposure game mechanic. Recent evidence (published after MindLight was developed) suggests that the contribution of relaxation training to anxiety improvements in youth appears to be limited, although it might be that a higher dose of relaxation is needed to improve anxiety symptoms (Peris et al. 2015).
Most importantly, the present study provides a unique contribution to the field by demonstrating that changes in the interaction with the game mechanics in MindLight predicted real-world improvements in anxiety symptoms at the 3-months follow-up assessment (when children had not played the game for 3 months). These findings show that what children learned and practiced in the game led to actual changes in their real-life (self-reported) behaviours. Most CBT approaches are not able to close this gap (Granic et al. 2014; Kendall 2011) and only a few game-based interventions have examined these effects (e.g., Girard et al. 2013). Moreover, by including a long-term follow-up, the present study is one of the few longitudinal studies in the field of applied video games (Girard et al. 2013; Granic et al. 2014; Primack et al. 2012).
Limitations and Suggestions for Future Research
There are a few limitations that should be mentioned when interpreting the findings and future research should attempt to overcome. The first limitation is that no in-game play behaviours representing the third technique (i.e., attention bias modification), that was initially incorporated in MindLight, were included in the current study. Because of how the game was designed, these behaviours were too dependent on the specific puzzle and the current location of the player and these locations varied widely across players, leaving comparisons across children unreliable. For example, a puzzle success on an easier puzzle would be different from a puzzle success on a more difficult puzzle, which requires more puzzle attempts and also the ability to distinguish different varieties of positive faces and negative faces. However, evidence for the effectiveness of attention bias modification in the literature is mixed. Some research has shown that attention bias modification is capable of reducing anxiety (Bar-Haim 2010; Bar-Haim et al. 2011; Hakamata et al. 2010), whereas other recent studies – that were published after MindLight was developed - failed to show this (for an overview see Clarke et al. 2014; Cristea et al. 2015). Clarke et al. (2014) argue that the studies finding no effects of attention bias modification failed to manipulate attention and therefore found no effects on the outcome measures. Future research should first test whether the puzzles in MindLight actually are able to modify attentional bias before in-game puzzle-solving behaviours are examined in relation to changes in anxiety.
A second limitation is that despite the fact that inactivity has good face validity for representing avoidant/safety behaviour because it reduces exposure to fear events and literally limits engagement with the game mechanics in MindLight, the inactivity code in the MindLight Coding System-II is not able to distinguish between actively avoiding the game or being inactive because the child is waiting for a research assistant to answer a question. Relatedly, the technical problem code might have been confounded too, because this code was used – but not exclusively – when the game was paused. It is possible that some children paused the game to actively avoid fear events in the game. Future research should try to separate these different behaviours, such that codes become as clear as possible and might show relations to changes in anxiety symptoms.
Finally, the current study did not test mediators or moderators. It is possible that associations between (multiple) in-game play behaviours and changes in anxiety symptoms are mediated or moderated by other factors, such as game expectancies, game experience, motivation, and enjoyment, which are known to be important predictors of treatment outcome (Castonguay et al. 1996; Ferguson and Olson 2013; Przybylski et al. 2010). Future research might want to examine these effects in a larger sample to get insight into why and for whom MindLight works best.
Practical Implications
Notwithstanding the aforementioned limitations, the current study has three important implications. The first implication applies to clinical research on anxiety. Results showed that in-game behaviours representing the exposure principle predicted improvements in anxiety. These findings underscore the importance of this technique in anxiety disorders and show that applied games can be used to reduce anxiety symptoms. Using game-based interventions contribute to the movement of tailoring and personalizing treatments (Chorpita and Weisz 2009) by engaging and motivating children that might not like the rather didactic-based approach of traditional CBT-treatments (Gosch et al. 2006) and by providing rich practice opportunities for children that find it difficult to use newly-acquired knowledge in real-life situations (Granic et al. 2014).
The second implication relates to designs aimed at improving both gameplay experiences and intervention outcomes. Because results demonstrated that the MindLight Coding System-II is useful for distinguishing between engaged and avoidant/safety in-game play behaviours, the coding system could be integrated into the game itself. In-game play behaviours can then be tracked automatically and be used as a measure for determining children’s progress. Measurements of in-game behaviours can be used to dynamically adjust the game to the player’s actions, diverse needs and learning paces (Bakkes et al. 2012; Bakkes et al. 2014). For example, the game may provide more hiding spaces in the beginning of the game for more anxious children (cf. Milosevic and Radomsky 2008; cf. Rachman et al. 2008), or provide more exposures to fear events when children are able to stay relaxed and calm in the face of less/easier fear events. Additionally, such measurements could tell clinicians and researchers when and where children are encountering difficulties in the game, and provide opportunities to help at an early stage. Finally, measuring in-game behaviours provides feedback for the players, clinicians, researchers and future game-based interventions, without the biases and stigma that are associated with self-reports.
The third implication applies to future research and the development of game-based interventions. Using observational codes, the current study provided a first step in testing the effect of the game mechanics in MindLight; the exposure to fear events is an important game mechanic that predicted anxiety symptoms 3 months later. Because of the modularity of game design, games hold the immense potential to test mechanisms of change with tightly controlled experiments. The next step for future research could be to experimentally manipulate the game mechanics in MindLight to test their causal impact on improvements in anxiety. Different versions of the game could be played with and without the game mechanics. Such experimentally controlled studies contribute to the development of a toolbox of validated game mechanics that could be used in new games targeting anxiety symptoms or other psychopathologies with the same underlying deficits.
Notes
Significant disturbances included: [1] trainer was wearing the MindWave instead of the child, [2] child had to change laptops, which led to the situation that the child had to start over with the game, [3] technical problems or no connection with the MindWave for more than 50% of a specific session.
When a child was absent during the last or the fifth session, video data from the fifth or the fourth session was coded, respectively.
Although the MindWave headset provided a more continuous measure for relaxation through the EEG data, this data could not be matched to the video data. Therefore, the brightness of the mindlight as an indication for relaxation was coded manually.
References
Bakkes, S., Tan, C. T., & Pisan, Y. (2012). Personalised gaming. Journal of Creative Technologies, 3.
Bakkes, S., Whiteson, S., Li, G., Visniuc, G. V., Charitos, E., Heijne, N., & Swellengrebel, A. (2014). Challenge balancing for personalised game spaces. IEEE Games Media Entertainment, 1–8.
Bar-Haim, Y. (2010). Research review: Attention bias modification (ABM): A novel treatment for anxiety disorders. Journal of Child Psychology and Psychiatry, 51, 859–870. https://doi.org/10.1111/j.1469-7610.2010.02251.x.
Bar-Haim, Y., Morag, I., & Glickman, S. (2011). Training anxious children to disengage attention from threat: A randomized controlled trial. Journal of Child Psychology and Psychiatry, 52, 861–869. https://doi.org/10.1111/j.1469-7610.2011.02368.x.
Beesdo, K., Knappe, S., & Pine, D. S. (2009). Anxiety and anxiety disorders in children and adolescents: Developmental issues and implications for DSM-V. Psychiatric Clinics of North America, 32, 483–524. https://doi.org/10.1016/j.psc.2009.06.002.
Birmaher, B., Khetarpal, S., Brent, D., Cully, M., Balach, L., Kaufman, J., & Neer, S. M. (1997). The screen for child anxiety related emotional disorders (SCARED): Scale construction and psychometric characteristics. Journal of the American Academy of Child and Adolescent Psychiatry, 36, 545–553. https://doi.org/10.1097/00004583-199704000-00018.
Brent, D. A., Kalas, R., Edelbrock, C., Costello, A. J., Dulcan, M. K., & Conover, N. (1986). Psychopathology and its relationship to suicidal ideation in childhood and adolescence. Journal of the American Academy of Child Psychiatry, 25, 666–673. https://doi.org/10.1016/S0002-7138(09)60292-2.
Butler, A. C., Chapman, J. E., Forman, E. M., & Beck, A. T. (2006). The empirical status of cognitive-behavioral therapy: A review of meta-analyses. Clinical Psychology Review, 26, 17–31. https://doi.org/10.1016/j.cpr.2005.07.003.
Castonguay, L. G., Goldfried, M. R., Wiser, S., Raue, P. J., & Hayes, A. M. (1996). Predicting the effect of cognitive therapy for depression: A study of unique and common factors. Journal of Consulting and Clinical Psychology, 64, 497–504. https://doi.org/10.1037/0022-006X.64.3.497.
Chorpita, B. F., & Weisz, J. R. (2009). MATCH-ADTC: Modular approach to therapy for children with anxiety, depression, trauma, or conduct problems. Satellite Beach: PracticeWise.
Clark, D. M. (1999). Anxiety disorders: Why they persist and how to treat them. Behaviour Research and Therapy, 37, S5–S27. https://doi.org/10.1016/S0005-7967(99)00048-0.
Clarke, P. J., Notebaert, L., & MacLeod, C. (2014). Absence of evidence or evidence of absence: Reflecting on therapeutic implementations of attentional bias modification. BMC Psychiatry, 14, 1–10. https://doi.org/10.1186/1471-244X-14-8.
Collins, K. A., Westra, H. A., Dozois, D. J., & Burns, D. D. (2004). Gaps in accessing treatment for anxiety and depression: Challenges for the delivery of care. Clinical Psychology Review, 24, 583–616. https://doi.org/10.1016/j.cpr.2004.06.001.
Cristea, I. A., Mogoașe, C., David, D., & Cuijpers, P. (2015). Practitioner review: Cognitive bias modification for mental health problems in children and adolescents: A meta-analysis. Journal of Child Psychology and Psychiatry, 56, 723–734. https://doi.org/10.1111/jcpp.12383.
Eichstedt, J. A., Collins, K. A., Phoenix, E., Haensel, H., & Dozois, D. J. A. (2014). The management of childhood anxiety disorders in Ontario’s public mental health system. Canadian Psychology, 55, 90–100. https://doi.org/10.1037/a0035103.
Essau, C. A. (2003). Comorbidity of anxiety disorders in adolescents. Depression and Anxiety, 18, 1–6. https://doi.org/10.1002/da.10107.
Farmer, E. M., Stangl, D. K., Burns, B. J., Costello, E., & Angold, A. (1999). Use, persistence, and intensity: Patterns of care for children’s mental health across one year. Community Mental Health Journal, 35, 31–46. https://doi.org/10.1023/A:1018743908617.
Ferguson, C. J., & Olson, C. K. (2013). Friends, fun, frustration and fantasy: Child motivations for video game play. Motivation and Emotion, 37, 154–164. https://doi.org/10.1007/s11031-012-9284-7.
Feske, U., & Chambless, D. L. (1995). Cognitive-behavioral versus exposure only treatment for social phobia: A meta-analysis. Behavior Therapy, 26, 695e720–695e720. https://doi.org/10.1016/s0005-7894(05)80040-1.
Fisak Jr., B. J., Richard, D., & Mann, A. (2011). The prevention of child and adolescent anxiety: A meta-analytic review. Prevention Science, 12, 255–268. https://doi.org/10.1007/s11121-011-021000.
Fraps (Version 3.5.99) [Computer Software]. (2015). Melbourne, Australia: Beepa Pty Ltd. http://www.fraps.com/
GainPlay Studio (2014). MindLight. http://www.gainplaystudio.com/mindlight/. Accessed 31 May 2018.
Girard, C., Ecalle, J., & Magnan, A. (2013). Serious games as new educational tools: How effective are they? A meta-analysis of recent studies. Journal of Computer Assisted Learning, 29, 207e219–207e219. https://doi.org/10.1111/j.1365-2729.2012.00489.x.
Glenn, D., Golinelli, D., Rose, R. D., Roy-Byrne, P., Stein, M. B., Sullivan, G., & Craske, M. G. (2013). Who gets the most out of cognitive behavioral therapy for anxiety disorders? The role of treatment dose and patient engagement. Journal of Consulting and Clinical Psychology, 81, 639–649. https://doi.org/10.1037/a0033-403.
Gosch, E. A., Flannery-Schroeder, E., Mauro, C. F., & Compton, S. N. (2006). Principles of cognitive-behavioral therapy for anxiety disorders in children. Journal of Cognitive Psychotherapy, 20, 247–262. https://doi.org/10.1891/jcop.20.3.247.
Granic, I., Lobel, A., & Engels, R. C. M. E. (2014). The benefits of playing video games. American Psychologist, 69, 66–78. https://doi.org/10.1037/a0034857.
Hakamata, Y., Lissek, S., Bar-Haim, Y., Britton, J. C., Fox, N. A., Leibenluft, E., et al. (2010). Attention bias modification treatment: A meta-analysis toward the establishment of novel treatment for anxiety. Biological Psychiatry, 68, 982–990. https://doi.org/10.1016/j.biopsych.2010.07.021.
Hammond, D. C. (2005). Neurofeedback treatment of depression and anxiety. Journal of Adult Development, 12, 131–137. https://doi.org/10.1007/s10804-005-7029-5.
In-Albon, T., & Schneider, S. (2007). Psychotherapy of childhood anxiety disorders: A meta-analysis. Psychotherapy and Psychosomatics, 76, 15–24. https://doi.org/10.1159/000096361.
Johnstone, S. J., Blackman, R., & Bruggemann, J. M. (2012). EEG from a single-channel dry-sensor recording device. Clinical EEG and Neuroscience, 43, 112–120. https://doi.org/10.1177/1550059411435857.
Kataoka, S. H., Zhang, L., & Wells, K. B. (2002). Unmet need for mental health care among US children: Variation by ethnicity and insurance status. American Journal of Psychiatry, 159, 1548–1555. https://doi.org/10.1176/appi.ajp.159.9.1548.
Kazdin, A. E. (2011). Evidence-based treatment research: Advances, limitations, and next steps. American Psychologist, 66, 685–698. https://doi.org/10.1037/a0024975.
Kendall, P. C. (2011). Child and adolescent therapy: Cognitive-behavioral procedures. New York: Guilford Press.
Kendall, P. C., & Hedtke, K. (2006). Cognitive-behavioral therapy for anxious children: Therapist manual (3rd ed.). Ardmore: Workbook Publishing.
Kessler, R. C., Nelson, C. B., McGonagle, K. A., Edlund, M. J., Frank, R. G., & Leaf, P. J. (1996). The epidemiology of co-occurring addictive and mental disorders: Implications for prevention and service utilization. American Journal of Orthopsychiatry, 66, 17–31. https://doi.org/10.1037/h0080151.
Lenz, A. S. (2015). Meta-analysis of the coping cat program for decreasing severity of anxiety symptoms among children and adolescents. Journal of Child and Adolescent Counseling, 1, 51–65. https://doi.org/10.1080/23727810.2015.1079116.
Mathews, A., & MacLeod, C. (2005). Cognitive vulnerability to emotional disorders. Annual Reviews of Clinical Psychology, 1, 167–195. https://doi.org/10.1146/annurev.clinpsy.1.102803.143916.
McManus, F., Sacadura, C., & Clark, D. M. (2008). Why social anxiety persists: An experimental investigation of the role of safety behaviours as a maintaining factor. Journal of Behavior Therapy and Experimental Psychiatry, 39, 147–161. https://doi.org/10.1016/j.jbtep.2006.12.002.
Milosevic, I., & Radomsky, A. S. (2008). Safety behaviour does not necessarily interfere with exposure therapy. Behaviour Research and Therapy, 46, 1111–1118. https://doi.org/10.1016/j.brat.2008.05.011.
MindWave Headset (Version 1.1.28) [Apparatus and software]. (2011). Silicon Valley, California: NeuroSky Inc.
Morgan, H., & Raffle, C. (1999). Does reducing safety behaviours improve treatment response in patients with social phobia? Australian and New Zealand Journal of Psychiatry, 33, 503–510. https://doi.org/10.1080/j.1440-1614.1999.00599.x.
Muris, P. (2016). The pathogenesis of childhood anxiety disorders: considerations from a developmental psychopathology perspective. International Journal of Behavioral Development, 30(1), 5–11.
Muris, P., & Field, A. P. (2008). Distorted cognition and pathological anxiety in children and adolescents. Cognition & Emotion, 22(3), 395–421.
Muris, P., Merckelbach, H., Schmidt, H., & Mayer, B. (1998). The revised version of the screen for child anxiety related emotional disorders (SCARED-R): Factor structure in normal children. Personality and Individual Differences, 26, 99–112. https://doi.org/10.1016/S0191-8869(98)00130-5.
Muris, P., Merckelbach, H., Mayer, B., & Prins, E. (2000a). How serious are common childhood fears? Behaviour Research and Therapy, 38, 217–228. https://doi.org/10.1016/s0005-7967(98)00204-6.
Muris, P., Schmidt, H., & Merckelbach, H. (2000b). Correlations among two self-report questionnaires for measuring DSM-defined anxiety disorder symptoms in children: The screen for child anxiety related emotional disorders and the Spence Children’s anxiety scale. Personality and Individual Differences, 28, 333–346. https://doi.org/10.1016/s0191-8869(99)00102-6.
Mychailszyn, M. P., Brodman, D. M., Read, K. L., & Kendall, P. C. (2012). Cognitive-behavioral school-based interventions for anxious and depressed youth: A meta-analysis of outcomes. Clinical Psychology: Science and Practice, 19, 129–153. https://doi.org/10.1111/j.1468-2850.2012.01279.x.
Olson, C. K. (2010). Children’s motivations for video game play in the context of normal development. Review of General Psychology, 14, 180–187. https://doi.org/10.1037/a0018984.
Peris, T. S., Compton, S. N., Kendall, P. C., Birmaher, B., Sherrill, J., March, J., et al. (2015). Trajectories of change in youth anxiety during cognitive-behavior therapy. Journal of Consulting and Clinical Psychology, 83, 239–252. https://doi.org/10.1037/a0038402.
Pine, D. S., Cohen, P., Gurley, D., Brook, J., & Ma, Y. (1998). The risk for early-adulthood anxiety and depressive disorders in adolescents with anxiety and depressive disorders. Archives of General Psychiatry, 55, 56–64. https://doi.org/10.1001/archpsyc.55.1.56.
Price, J., & Budzynski, T. (2009). Anxiety, EEG patterns, and neurofeedback. In T. Budzynski, H. Kogan Budzynski, J. R. Evans, & A. Abarbanel (Eds.), Introduction to quantitative EEG and neurofeedback: Advanced theory and applications (pp. 453–470). Amsterdam: Academic Press.
Primack, B. A., Carroll, M. V., McNamara, M., Klem, M. L., King, B., Rich, M., Chan C. W., Nayak S. (2012). Role of video games in improving health-related outcomes: A systematic review. American Journal of Preventive Medicine, 42, 630e638. https://doi.org/10.1016/j.amepre.2012.02.023, 630e638
Przybylski, A. K., Rigby, C. S., & Ryan, R. M. (2010). A motivational model of video game engagement. Review of General Psychology, 14, 154–166. https://doi.org/10.1037/a0019440.
Rachman, S., Radomsky, A. S., & Shafran, R. (2008). Safety behaviour: A reconsideration. Behaviour Research and Therapy, 46, 163–173. https://doi.org/10.1016/j.brat.2007.11.008.
Reynolds, C. R., & Richmond, B. O. (1978). What I think and feel: A revised measure of children’s manifest anxiety. Journal of Abnormal Child Psychology, 6, 271–280. https://doi.org/10.1007/BF00919131.
Rideout, V. J., Foehr, U. G., & Roberts, D. F. (2010). Generation M 2 : Media in the lives of 8-to-18-year-olds. Menlo Park: Henry J. Kaiser Family Foundation Available: https://kaiserfamilyfoundation.files.wordpress.com/2013/04/8010.pdf.
Salkovskis, P. M., Clark, D. M., & Gelder, M. G. (1996). Cognition-behaviour links in the persistence of panic. Behaviour Research and Therapy, 34, 453–458. https://doi.org/10.1016/0005-7967(95)00083-6.
Salkovskis, P. M., Clark, D. M., Hackmann, A., Wells, A., & Gelder, M. G. (1999). An experimental investigation of the role of safety-seeking behaviours in the maintenance of panic disorder with agoraphobia. Behaviour Research and Therapy, 37, 559–574. https://doi.org/10.1016/S0005-7967(98)00153-3.
Schoneveld, E. A., Malmberg, M., Lichtwarck-Aschoff, A., Verheijen, G. P., Engels, R. C. M. E., Granic, Isabela (2016). A neurofeedback video game (MindLight) to prevent anxiety in children: A randomized controlled trial. Computers in Human Behavior 63, 321–333.
Schoneveld, E. A., Lichtwarck-Aschoff, A., Granic, I. (2018). Preventing childhood anxiety disorders: is an applied game as effective as a cognitive behavioral therapy-based program? Prevention Science, 19(2), 220–232.
Sherman, A. (2015). Avoidance and engagement during a video game designed to reduce children’s anxiety. (Unpublished bachelor’s thesis). Queen’s University, Kingston, Canada.
Spence, S. H. (1997). Structure of anxiety symptoms among children: A confirmatory factor-analytic study. Journal of Abnormal Psychology, 106, 280–297. https://doi.org/10.1037//0021-843x.106.2.280.
Spence, S. H. (1998). A measure of anxiety symptoms among children. Behaviour Research and Therapy, 36, 545–566. https://doi.org/10.1016/s0005-7967(98)00034-5.
The Observer XT 11 (Version 11.5) [Computer Software]. (2013). Wageningen, The Netherlands: Noldus Information Technology.
Tsui, T. Y. L., De France, K., Khalid-Khan, S., Granic, I., & Hollenstein, T. (2018). Reductions of anxiety symptoms, state anxiety, and anxious arousal in youth playing the video game MindLight. Manuscript submitted for publication
van Starrenburg, M. L., Kuijpers, R. C., Kleinjan, M., Hutschemaekers, G. J., & Engels, R. C. M. E. (2017). Effectiveness of a cognitive behavioral therapy-based indicated prevention program for children with elevated anxiety levels: A randomized controlled trial. Prevention Science, 18, 31–39. https://doi.org/10.1007/s11121-016-0725-5.
Weersing, V. R., Rozenman, M. S., Maher-Bridge, M., & Campo, J. V. (2012). Anxiety, depression, and somatic distress: Developing a transdiagnostic internalizing toolbox for pediatric practice. Cognitive and Behavioral Practice, 19, 68–82. https://doi.org/10.1016/j.cbpra.2011.06.002.
Weissman, M. M., Wolk, S., Wickramaratne, P., Goldstein, R. B., Adams, P., Greenwald, S., …, & Steinberg, D. (1999). Children with prepubertal-onset major depressive disorder and anxiety grown up. Archives of General Psychiatry, 56, 794–801. https://doi.org/10.1001/archpsyc.56.9.794.
Woodward, L. J., & Fergusson, D. M. (2001). Life course outcomes of young people with anxiety disorders in adolescence. Journal of the American Academy of Child and Adolescent Psychiatry, 40, 1086–1093. https://doi.org/10.1016/S0005-7894(05)80088.
Acknowledgements
The authors would like to gratefully acknowledge GainPlay Studio, the game design studio who co-developed MindLight. We would also like to thank all research assistants for their help in coding the video data.
We would also like to acknowledge the financial support for the conduct of the research from the Netherlands Organisation for Scientific Research (NWO, 406-12-017). They had no role in the design of the study, data collection, analysis, interpretation of data, writing the manuscript, and in the decision to submit the article for publication.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of Interest
MindLight was produced by the PlayNice Institute. Prof. dr. Isabela Granic is co-founder of this institute. However, MindLight is not yet commercialized. The chance of competing interests will be present only in the case of commercialization of the game. Dr. Anna Lichtwarck-Aschoff, Elke A. Schoneveld (MSc), and Aniek Wols (MSc) declare that they have no competing interests. This research received financial support for the conduct of the research from the Netherlands Organisation for Scientific Research (NWO, 406–12-017). They had no role in the design of the study, data collection, analysis, interpretation of data, writing the manuscript, and in the decision to submit the article for publication.
Experiment Participants
All procedures in this study were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards. Informed consent was obtained from all individual participants included in the study.
Appendices
Appendix 1
Description of the Video Game MindLight
MindLight incorporates three evidence-based techniques (i.e., relaxation, exposure, attention bias modification) to teach children how to cope with anxiety in a playful manner (see Table 1). To trigger real feelings of anxiety and to practice regulating these feelings, the game is set in grandma’s dark and decrepit mansion. Little Arty (the player) is left on the doorstep of grandma’s mansion that has been taken over by evil forces. It is his task to save grandma who succumbed to shadows. In the house Arty finds a magical headset (Teru) and together they can save grandma. However, the player needs to overcome his fears by using his own mind. Teru teaches how to use this “mindlight”, a beam of light at the end of an antenna attached to the magical hat which players need to play through the game.
Arty’s mindlight responds to the electroencephalogram (EEG) signals that the MindWave headset, a one-channel dry-sensor EEG headset the player is wearing, picks up. The headset detects and converts EEG signals (i.e., relative beta power and relative alpha power) into two continuous data streams representing relaxation and focused attention. These data streams are fed into the game (i.e., neurofeedback training) and control Arty’s mindlight and “mindbeam”. The strength of the mindlight and mindbeam are proportional to the strength of the relaxation and focused attention signals, respectively. Being more relaxed and focused makes it possible to effectively engage with the objects and evidence-based game mechanics in the game. When the player becomes more relaxed the mindlight will become stronger, providing more light inside the game environment which makes it easier to get through the game. When exposed to fear events, players need their mindlight to chase away or “decloak” these monsters. Some fear events that are decloaked will turn into a friendly kitten that will remind the player of past fears conquered. More focused attention leads to a stronger mindbeam, which is needed to unlock hiding spaces, turn on ceiling lights and to solve attention bias modification puzzles in which the player learns and is rewarded for focusing on positive faces rather than on negative faces.
The goal in MindLight is to save grandma by completing all puzzles, which will turn the lights back on in the house. To solve a puzzle, coins have to be found first to unlock the puzzle. Every puzzle requires a predetermined amount of coins (i.e., from 3 up to 7 coins) that corresponds to the number of faces in the puzzle. The amount of coins reflects the difficulty of the puzzle; a puzzle with more coins requires more effort than a puzzle with less coins because in the puzzle with more coins the positive face has to be distinguished from more negative faces.
Appendix 2
Rights and permissions
Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.
About this article
Cite this article
Wols, A., Lichtwarck-Aschoff, A., Schoneveld, E.A. et al. In-Game Play Behaviours during an Applied Video Game for Anxiety Prevention Predict Successful Intervention Outcomes. J Psychopathol Behav Assess 40, 655–668 (2018). https://doi.org/10.1007/s10862-018-9684-4
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10862-018-9684-4