Skip to main content
  • S.I. : Virtual Reality for Therapy, Psychological Interventions, and Physical and Cognitive Rehabilitation
  • Open Access
  • Published:

The effectiveness of a virtual reality attention task to predict depression and anxiety in comparison with current clinical measures

Abstract

Previous studies have revealed that attention and inhibition are impaired in individuals with elevated symptoms of depression and anxiety. Virtual reality (VR)-based neuropsychological assessment may be a valid instrument for assessing attention and inhibition given its higher ecological validity when compared to classical tests. However, it is still unclear as to whether a VR assessment can predict depression and anxiety with the same or higher level of effectiveness and adherence as classical neuropsychological measures. The current study examined the effectiveness of a new VR test, Nesplora Aquarium, by testing participants with low (N = 41) and elevated (N = 41) symptoms of depression and anxiety. Participants completed a continuous performance test where they had to respond to stimuli (species of fish) in a virtual aquarium, as well as paper-and-pencil and computerised tests. Participants’ performance in Nesplora Aquarium was positively associated with classic measures of attention and inhibition, and effectively predicted symptoms of depression and anxiety above and beyond traditional cognitive measures such as psychomotor speed and executive functioning, spatial working memory span. Hence, VR is a safe, enjoyable, effective and more ecological alternative for the assessment of attention and inhibition among individuals with elevated anxiety and depression symptoms.

Introduction

Depression and anxiety disorders are a global problem with an estimated worldwide prevalence of 8%. Roughly, 322 million people are currently living with depression and 264 million with anxiety disorders (WHO 2017) with estimated worldwide costs of US$925 billion and 12 billion days of lost productivity per year caused by these conditions (Chisholm et al. 2016). Depression and anxiety are a major contributor to global disability, and both are predicted to affect an increasing number of individuals in the years to come (WHO 2017).

A wealth of evidence associates depression and anxiety with impaired attention (Eysenck et al. 2007; Hallion et al. 2018; Liu et al. 2020; Pacheco-Unguetti et al. 2011; Rock et al. 2014; Snyder 2013). Vigilance or sustained attention is the ability to maintain an attentional ability and be alert to stimuli over prolonged periods of time (Lezak et al. 2004). Poor inhibition indicating poor capacity to deliberately inhibit irrelevant responses (Miyake et al. 2000), a domain of executive functioning, is linked with rapid response impulsivity, an inability to adjust responses and behaviours to the situational demands (Asahi et al. 2004; Evenden 1998). To date, cognitive deficits, including attention and executive functioning, have been measured using neuropsychological tests such as Continuous Performance Tests or Stroop Test. The aim of neuropsychological assessment is to draw inferences about structural and functional characteristics of the brain by assessing individual’s observable behaviours in defined stimulus–response situations such as participant responses and subsequent performance. Results of neuropsychological assessment are used to identify the impact of the disease on various cognitive functioning domains and to make predictions regarding a person’s level of functioning in everyday life (Benton 1994; Lezak et al. 2004). Because most of these tests were developed in the early twentieth century, some argue that their principles are outdated (Chaytor and Schmitter-Edgecombe 2003; Kessels 2019). Usually, these tests ask participants to respond to various symbols or stimuli such as letter “x” following predefined rules while ignoring irrelevant stimuli such as letters “a”, “b” “c”. Nowadays, stimuli are delivered in a systematic way on computer screens thus limiting human error (and increasing accuracy) when recording participant responses. However, it has been argued that attending and responding to a set of stimuli on a computer screen lack the complexity of real environments (Parsons and Rizzo 2019), thus limiting the effectiveness of the neuropsychological assessments in predicting real-life performance (Chaytor and Schmitter-Edgecombe 2003; Kessels 2019).

A useful example to understand the limitation of such traditional computer-based tests is to consider how children may respond in a typical one-on-one assessment context and in a real classroom setting with distracting stimuli such as other children whispering or noise coming from outside. A promising alternative to increase the ecological validity of these tests, which has gained support in the last 20 years, is the use of virtual reality (VR) (Parsons and Rizzo 2019). Referring back to the previous example, the VR “solution” for assessing attention deficits among children in ecological environments is to create a VR environment, for example, a VR classroom that replicates a real classroom environment, while delivering classic cognitive tasks and measuring attention performance (Iriarte et al. 2016; Rizzo et al. 2006).

In the last few decades, VR has thus been increasingly used in clinical psychology, including neuropsychology, because of its ability to create experiences that are very similar to those in real life and the possibility to maintain control over stimuli and therapeutic strategies that would otherwise not be possible in real life settings (Negut et al. 2016; Rizzo 2019; Rizzo and Koenig 2017). For example, VR platforms allow a participant to be immersed in a virtual environment (VE) and to systematically deliver visual and auditory stimuli in the presence of various distractors (such as avatars of other humans and/or noise from the environment) thus increasing the realism of the assessment context. An increasing number of VEs like virtual classrooms, virtual shopping centres or cities have been developed to measure cognitive functions such as attention, executive functioning and memory across various populations including patients with brain injury or individuals with neurodevelopmental disorders such as children with ADHD (Negut et al. 2016; Rizzo 2019). Results are encouraging as they have shown that VR tests are valid instruments (Gilboa et al. 2018; Negut et al. 2016; Parsons et al. 2019). However, more studies are needed to identify whether VR is more effective than classical tests in identifying cognitive impairment as a meta-analysis has shown that VR tests have similar sensitivity to traditional computerised or paper-and-pencil tests in detecting cognitive deficits (Negut et al. 2016).

However, to our knowledge, no study so far has used VR as a tool to investigate the neurocognitive profile of individuals with elevated symptoms of depression and anxiety. Using VR to assess cognitive processes of individuals with depression and anxiety might help overcome issues related to lack of motivation or increased fatigue during testing and consequently increase adherence. For example, some of the poor results on cognitive functioning obtained by people with depression can be explained by reduced motivation and fatigue during prolonged periods of testing time (Marazziti et al. 2010; Vilgis et al. 2015). VR can be more engaging and can increase motivation because it has features similar to computer games (Rizzo and Kim 2005). Because VR can replicate real-life scenarios, this can increase validity over computerised tests, engaging and motivating responders. That is, if the test takers perceive that the content of the test is appropriate and relevant for the purpose of the assessment, they are more willing to cooperate (Urbina 2004). VR also has good-to-excellent usability ratings given by healthy participants (Voinescu et al. 2019), which may facilitate its adoption (Mousavi et al. 2013). Product usability is the “extent to which a system, product or service can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use” (ISO 9241-11:2018) and is strongly connected to the usefulness of a product (Nielsen and Mack 1994). Usability of VR systems is linked with acceptance and adherence rates to VR-based interventions (Laver et al. 2012; Morán et al. 2015). Acceptance and adherence rates were mostly measured using self-report scales that cover dimensions such as enjoyment, confidence and self-esteem (Rose et al. 2018). Results suggest that VR has the potential of increasing adherence (Rose et al. 2018); however, a study that measured adherence as number of days spent in therapy did not find significant differences between the VR group and standard therapy (Parry et al. 2015). While the usability of VR systems has not been tested yet with individuals with depression and anxiety, other research with vulnerable populations such as older adults and patients with Parkinson disease has shown that VR has good ratings of usability (Corno et al. 2014; Pedroli et al. 2013, 2018). Simulator sickness and presence are variables in VR that are linked with usability and performance. Simulator sickness refers to symptoms that occur during exposure in VR and cause discomfort such as fatigue, headache, eye strain, nausea, dizziness (Kennedy et al. 1993). The sensory-conflict theory offers an explanation of simulator sickness symptoms, which can be caused by a mismatch in the visual, vestibular and proprioception sensory channels (Cobb et al. 1999). Simulator sickness symptoms are usually mild and transient (Lavoie et al. 2020), but it is believed that side effects can be more problematic for vulnerable populations such as older adults or people with neurological conditions (Brooks et al. 2010; Kim et al. 2017). Presence in VR refers to the subjective feeling of being present in the environment (Slater and Wilbur 1997; Witmer and Singer 1998) and can increase user experience (Sun et al. 2015) and facilitate the transfer of skills from VR to real life settings (Slater et al. 1996). However, a recent systematic review concluded that there are inconsistent results for the relationship between presence and performance on VR rehabilitation interventions (Rose et al. 2018). Identifying whether there are individual differences between people with elevated symptoms of depression and anxiety and healthy people on cyberpsychology variables such as self-report usability, simulator sickness and presence would increase our understanding of how to design better VR applications for clinical purposes.

To benefit from potential advantages of VR assessment described above, in the current study we aimed to explore the attention and inhibition profile among participants with low and elevated symptoms of depression and anxiety using a new VR tool (Parsons et al. 2017; Rizzo and Koenig 2017; Schulthesis and Doiron 2017). Nesplora Aquarium is the first VR test designed to measure attention and executive functions in an adult population (Climent et al. 2019) and is based on the Continuous performance test paradigm (CPT) which is one of the most widely used measures of attention impairment in depression (Coles et al. 2009).

The original CPT paradigm was first developed by Rosvold et al. (1956), and today CPTs are widely used as a reliable measure of sustained vigilance, attention and inhibition (Gualtieri and Johnson 2005; Losier et al. 1996). Two forms of CPT tasks are currently used (Servera and Cardo 2006): Vigilance tasks known as X- or AX-types require the participant to respond to target stimuli and to ignore non-target stimuli (Rosvold et al. 1956). Inhibition tasks or non-X tasks ask the participant to respond to the non-target stimuli and to ignore target stimuli (Conners et al. 2003; Servera and Cardo 2006). Outcomes of CPTs are reaction time (RT) for correct targets, variability of RT, number of omission errors (missed targets) and commission errors (incorrect hits). To date, CPTs have been integrated in VR using a virtual classroom where the participant is sitting at a desk and stimuli consisting of alphabet letters are delivered on a blackboard (Iriarte et al. 2016; Rizzo et al. 2000). Studies using this VR system revealed it is effective in assessing attention deficits such as ADHD among children (Gilboa et al. 2018; Negut et al. 2016, 2017; Parsons et al. 2019).

In Nesplora Aquarium, CPTs have been integrated in VR by using an aquarium experience. The aquarium contains multiple tanks, but the participant is instructed to pay attention to the main tank which is positioned in front of them. Following a CPT rule, the participant has to pay attention and respond to correct stimuli over a prolonged period of time (18 min). For example, they have to press a button whenever they see or hear the name of a certain fish, for example, the clown fish. During the test, to increase the realism and ecological validity of the task, various contextual distractors are introduced such as people walking in front of the aquarium, a baby crying. An important and novel feature of Nesplora Aquarium is that it contains both visual and auditory stimuli. Because of this, the test can also examine the attentional sensory profile of individuals with depression and anxiety by assessing attention performance obtained on both visual and auditory stimuli to identify any deficits. Currently, the nature of attention deficits in depression and anxiety, especially in relation to sensory processing, is still lacking. Most research in anxiety and depression has focused on early pre-attentive and automatic visual and auditory processing and showed impaired deficits in early auditory and visual processing in depressed and anxious samples (Chang et al. 2011; Kahkonen et al. 2007; Qiu et al. 2011; Schirmer and Escoffier 2010; Takei et al. 2009; Weinstein 1995; Yang et al. 2019; Zweerings et al. 2019). However, sensory processing as modulated by attention has not been studied in depression and anxiety, which we plan to address in the current study by exploring attention performance on auditory and visual stimuli of individuals with elevated symptoms of depression and anxiety in comparison with individuals with low symptoms.

Furthermore, to establish whether Nesplora Aquarium for attention and inhibition assessment is a valid and effective measure over and above traditional tests, we focused on patterns of convergence and divergence, as main sources of test validity. Evidence of convergent validity of a test is usually established if two tests that are supposed to measure the same psychological construct show strong positive correlations (Urbina 2004). Assessing convergent validity of VR measures is a common practice. For example, Parsons and Barnett (2017) using a sample of younger and older adults computed Pearson’s r coefficients between a VR measure of memory (Virtual Environment Grocery Store, VEGS) and a traditional neuropsychological measure of learning and memory (California Verbal Learning Test). Because the two measures were highly correlated (correlations ranging from 0.42 to 0.56), the authors concluded that the VR measure showed convergent validity. In opposition, evidence for divergent validity is obtained based on low correlations between two tests that are supposed to measure different constructs (Urbina 2004). Similarly, in the same study, Parsons and Barnett (2017) computed Pearson’s r coefficients between the VR measure of memory (Virtual Environment Grocery Store, VEGS) and a traditional neuropsychological measure of executive functioning (Delis Kaplan Executive Functioning System, DKEFS). Because VEGS did not correlate with theoretically unrelated constructs such as executive functioning (correlations ranging from 0.07 to 0.28), the authors concluded that the VR test had adequate levels of divergent validity.

Individuals with depression and anxiety show impaired cognition on multiple attention like reaction time, rapid visual information processing and executive functioning domains: inhibition, shifting, verbal working memory, visuospatial working memory, planning, verbal fluency, processing speed) (Eysenck et al. 2007; Hallion et al. 2018; Liu et al. 2020; Rock et al. 2014; Snyder 2013). Hence, in the current study we investigated if attention and inhibition measured in VR with Nesplora Aquarium can effectively detect symptoms of depression and anxiety above and beyond classical measures. To assess the proposed incremental validity of Nesplora Aquarium over classical measures in predicting current symptoms of depression and anxiety, we selected several cognitive ability domains known to be affected in depression and anxiety: attention and inhibition, psychomotor speed and executive functioning, selective attention and inhibition, cognitive impairment, spatial working memory span which were measured with classical tests such as paper-and-pencil or computerised measures. We then assessed the ability of Nesplora Aquarium to predict current symptoms of depression and anxiety beyond cognitive domains measured by classical tests.

In summary, the scope of the current study is to explore the effectiveness of a virtual reality attention task to predict depression and anxiety in comparison with current clinical measures. The main objectives of the current study were to (1) investigate the validity of Nesplora Aquarium as a measure of attention and inhibition. We expected strong positive correlations between outcomes of Nesplora Aquarium and outcomes of the computerised version of CPT for measure of attention and inhibition; (2) examine whether attention measured with Nesplora Aquarium can predict current symptoms of depression and anxiety above and beyond cognitive domains measured with classical tests; we anticipated that outcomes of Nesplora Aquarium will predict current symptoms of depression and anxiety above and beyond cognitive domains measured with classical tests (3) to identify if Nesplora Aquarium can discriminate between individuals with low and elevated symptoms of depression and anxiety and to explore patterns of attention deficits, and their relation to specific sensory processing (visual and auditory). We predicted that participants with elevated symptoms of depression and anxiety will have poorer attention performance and inhibition (also in relation to specific visual and auditory sensory processing) as measured with Nesplora Aquarium; and (4) to assess the usability of Nesplora Aquarium as well as possible adverse effects such as motion sickness and the level of presence in VR. We expected no major simulator sickness symptoms, and good-to-excellent self-report usability and adequate level of presence in VR irrespective of the level of anxiety and depression.

Methods

Design

We used a cross-sectional design as data were collected at one single point in time. The cross-sectional design was mixed: 2 × 2 × 2. The first factor was a between-subjects factor (the groups of participants: people with low and elevated symptoms of depression and anxiety). The second factor was a within-subjects factor (type of task: VR vs. non-VR classical task). The third factor was a within-subjects factor (sensory modality: visual vs. auditory).

Participants

Participants were recruited via multiple sources: University Student Participant Pool in exchange for study credit; social media channels; general medical practices; and Psychiatric facilities in Romania. A total of 91 participants enrolled in the study and 82 participants aged between 19 and 61 years (M = 32.40, SD = 9.71) were included in the analysis. Four participants were excluded from the sample due to data loss (technical errors related to poor wi-fi connection during VR Aquarium testing) and another five based on their mild to moderate scores on the clinical scales (as detailed below). Sixty-one percent were females (N = 53) with a mean education year of 16 years. Sixty-two percent (N = 54) were employed and 40% reported previous VR experience (N = 35) (see Supplementary Materials, Table A.1 for participants’ demographic characteristics, Table A.2–3. for a full description of clinical and neuropsychological measures). We excluded participants who: (a) were younger than 18 years; (b) had a history of neurological diseases (including head injury or epilepsy), psychosis or substance dependence, (c) had uncorrected hearing and visual impairments or severe motion sickness.

Based on the severity of the mood symptoms, we divided our sample into healthy participants and participants with elevated symptoms of depression and anxiety, by using the following criteria: (a) having a clinical diagnosis of anxiety or depressive disorders and/or (b) having higher scores of depression and/or anxiety as measured with well-established psychological scales, as described below. Twenty-four participants had a clinical diagnosis as they met the ICD 10 criteria for: Major Depressive Disorder, single episode (N = 8), Recurrent Depressive Disorder (N = 6) Generalised Anxiety Disorder (N = 4), Mixed Anxiety and Depressive Disorder (N = 2), Panic Disorder (N = 3) and Social Phobia (N = 1). Seventeen participants reported increased depression and anxiety but did not meet the diagnostic criteria for a depressive or an anxiety disorder. Participants were screened for symptoms of depression and anxiety using the Beck Depression Inventory-II (BDI-II, Beck et al. 1996) and State-Trait Anxiety Inventory (Form Y) (STAI, Spielberger 1983). The cut-off used for depressive symptoms was BDI total score ≥ 20, which is equivalent to moderate to severe depression according to the normative data (Beck et al. 2012). For anxiety, a moderate to severe level of state anxiety (STAI-S) ≥ 34 was used as cut-off according to data from the normative study (Spielberger et al. 2007). Twenty-two participants were taking antidepressant, anxiolytic, hypnotic and/or antipsychotic medication at the time of the study. Overall, there were no significant differences between participants who were on medication and those who were not on most cognitive outcomes, except on omission errors measured with classic CPT. The medicated participants displayed significantly increased symptoms of depression and anxiety than non-medicated participants (Supplementary Materials, Table A.4. displays means, SDs, t tests values and alpha significance level).

Forty-one healthy participants were selected and had a score on BDI ≤ 13 which is equivalent to minimal/low depression according to the normative data (Spielberger et al. 2007) and a low level of state anxiety (STAI-S) ≤ 30 for both males and females according to data from normative study (Beck et al. 2012). We excluded five participants who displayed mild to moderate symptoms of depression and anxiety (BDI total scores between 14 and 19 (Beck et al. 2012) or state anxiety (STAI-S) scores between 31 and 33 (Spielberger et al. 2007). Supplementary Table A.1. displays information about participants demographics.

Materials

VR test for attention assessment

The VR test used in this study was Nesplora Aquarium (Climent et al. 2019), the first VR-based test developed to measure attention and executive functioning in adults over 16 years. It consists of non-X tasks that are administered in a virtual aquarium (see Fig. 1). The VR system uses a Samsung Galaxy S7 smartphone, paired with Samsung Gear VR headset. The test is monitored by the experimenter, using a laptop computer (ASUS ROG, Intel i7 processor, 8 Gb RAM, GeForce GTX 960 M video card). Both the laptop computer and the VR headset are connected using a local wireless connection.

Fig. 1
figure1

Screenshot of the Nesplora Aquarium

During the test, the participant is virtually positioned in front of a VR aquarium and must pay attention to the main fish tank where they have to respond to the stimuli. The stimuli are different species of fish such as a clown fish delivered via the visual channel using images of different species of fish that are passing through two rocks in the main fish tank or via the auditory channel using the names of the same species of fish as in the visual condition presented via speakers (see Fig. 1). In line with the CPT paradigm, the participant has to follow a rule in responding to stimuli; for example, each time they see any other fish than the “clown” or they hear any other fish except the “surgeon”. The task was to attend to either the visual or to the auditory modality and to respond to visual or auditory targets. Visual and auditory stimuli were distributed randomly during the task. Participants responses were collected for visual and auditory stimuli separately. Different contextual distractors were delivered during the task, either visually such as people walking in front of the aquarium, other animals present in the aquarium such as turtles, lights flickering or auditorily such as an invitation to coffee delivered over the PA system, a baby crying, a warning to not use the flash when taking photographs. The full description of Nesplora Aquarium is available in the Online Resources.

Participants’ responses were recorded separately for each sensory modality: visual and auditory stimuli. Besides performance on each modality, the system also computes for each outcome an average score by combining visual and auditory performance. Nesplora Aquarium outcomes were used as measures of sustained attention and inhibition, more specifically: mean reaction time (RT) for correct responses in milliseconds, standard deviation (SD) of RT for correct responses, total number of commissions, RT for commission errors, omission errors and perseverance errors. All the measures for sustained attention and inhibition as calculated with Nesplora were obtained separately for auditory and visual modalities and for the two modalities combined. Commission errors result from incorrect responses to non-target stimuli, while omission errors result from not responding to targets. Increased number of commission errors and omission errors, as well as faster RT for correct answers and increased SD of RT, reflects poor attention performance. More specifically, slow RT is an indicator of slow processing speed of the target stimulus while variability of RT suggests changes in sustained attention or fatigability during the task with lower scores reflecting fluctuation of attention. Omissions are linked to inattention and low arousal and commissions to impulsivity or a deficit in inhibitory control (Climent et al. 2019; Negut et al. 2017; Tinius 2003). RT to commission errors is proposed as an additional indicator of impulsivity and inhibitory control with faster RT indicating increased impulsivity (Climent et al. 2019; Halperin et al. 1991). Perseverance errors can indicate cognitive inflexibility, in a sense that they measure participants in/ability to change their responses following changes in the task at hand, i.e. if a participant uses old rules from the task rather than switching successfully to new rules, then this is an indicator of poor interference control and cognitive inflexibility (Climent et al. 2019; Greve et al. 2005). More perseverance errors reflect poorer interference control and cognitive flexibility (Climent et al. 2019). For all Nesplora outcomes, as no standardised data and clear cut-off scores for attention impairment among clinical samples were available in the literature, we transformed raw scores to t scores using the formula: T = (Z × 10) + 50 and we used these transformed data in the analysis (Weiner et al. 2013).

Measures

Participants’ demographic characteristics collected included age, gender, highest education degree, work status, previous VR experience, medical history and medication taken, and whether they experience any severe motion sickness. We screened for symptoms of depression using the Beck Depression Inventory-II, BDI-II (Beck et al. 1996). Symptoms of anxiety were assessed using the State and Trait Anxiety Measured with State-Trait Anxiety Inventory, STAI (Form Y) (Spielberger 1983).

Besides completing the Nesplora Aquarium task, participants were asked to complete the following paper-and-pencil and computerised measures of cognitive abilities known to be affected in cases of depression and anxiety: sustained attention and inhibition (Continuous Performance Test, CPT, Mueller and Piper 2014), selective attention and cognitive flexibility (Stroop Test, Bayard et al. 2011), spatial working memory span (Corsi Block Test, Corsi 1972), psychomotor speed and executive functioning (Trail Making Test, Reitan 2016), and cognitive impairment (Montreal Cognitive Assessment, MoCA test, Nasreddine et al. 2005). Self-reported system usability was measured with the System Usability Scale (SUS, Brooke 1986). The level of simulator sickness was measured using the Simulator Sickness Questionnaire (SSQ, Kennedy et al. 1993). Self-report presence in VR was measured using the Presence Questionnaire (Witmer and Singer 1998). Further description and psychometric properties of these measures can be found in the in the Supplementary Materials; Table A.2–3 contains a full description of clinical and neuropsychological measures.

Procedure

The study was approved by the University Research Ethics committee (REF 6667/25.04.2018) at Babes-Bolyai University, Romania where the data were collected and from University of Bath, UK where the data was analysed (PREC 18-305). Data collection took place between April 2018 and November 2019. The study was pre-registered on the Open Science Framework (https://osf.io/nrhb5). All participants gave informed consent after reading the information sheet. The screening and demographic questionnaire was completed at the beginning of the study. Participants who did not meet the inclusion criteria did not take part in the study. Participants in the study completed the paper-and-pencil version of BDI-II (Beck et al. 1996, 2012), the S-STAI (Spielberger 1983; Spielberger et al. 2007) and pre-test SSQ (Kennedy et al. 1993) followed by the Nesplora Aquarium VR test. During the VR test, the experimenter showed the participant the VR headset and explained how it worked and stressed that in the case of any motion/simulator sickness symptoms they should alert the experimenter and remove the VR headset. The experimenter helped the participant to adjust the VR headset to make sure that the VR headset was comfortable and that the image was focused. Then, the practice session began which consisted of the same VR aquarium environment. During this session, the participant was instructed first to explore the VR environment and to listen to instructions and to start the task when they were ready. The practice session lasted for approximately 5 min, depending on the needs of the participant to familiarise with the environment and controllers. Then, the main testing session started. During this, the participant had to sit down on a chair (the task did not involve active navigation in the VE and was exclusively passive) and pay attention and respond to the visual or auditory stimuli. Visual and auditory stimuli were distributed randomly. The task lasted for approximately 15 min. At the end of the VR task, they had to fill in the paper-and-pencil post-test questionnaires and measures: post-test SSQ, PQ, SUS (Brooke 1986) followed by the computerised versions of the CPT (Conners et al. 2003; Piper et al. 2016), Victoria Stroop (Strauss et al. 2006), Corsi Block Test (Corsi 1972) and the paper-and-pencil TMT A&B (Reitan 2016) and MoCA (Nasreddine et al. 2005). Written instructions were provided at the start of each test, and additional information concerning test instructions was provided by the experimenter if required. At the end of the session participants were debriefed. The testing time took approximately 90 min for each participant.

Results

As some of the variables were not normally distributed like RT which usually produces skewed distributions (Miller 1991) for our statistical analysis, we used the bootstrapping procedure with 95% bias-corrected bootstrap confidence intervals (CIs) with 10,000 resamples to obtain more robust CIs for the difference and probability values for regression model estimates (Efron and Tibshirani 1993; Fox 2015; Mallinckrodt et al. 2006). All data analyses were conducted using SPSS version 24.

Convergent and divergent validity of VR CPT Nesplora Aquarium

To investigate our first objective that aimed to establish the convergent and divergent validity of Nesplora Aquarium, we first analysed correlations between VR CPT and computerised CPT. We used Pearson’s r to assess correlations between variables which were continuous. In addition, we also calculated Spearman’s rho correlation estimates for robustness against non-normal distributions. Both tests assess vigilance, sustained attention and inhibition and the measures are comparable for the following outcomes: RT, SD RT, omission and commission errors. All correlations between these variables were significant. As expected, the correlations were positive and high for RT, omission and commission errors, except for SD RT, for which the correlation was moderate/acceptable (Drummond and Jones 2016). Table 1 shows the correlations between the variables. We expected weak to moderate correlations between VR CPT Nesplora Aquarium outcomes with classic neuropsychological measures that measure other cognitive domains such as psychomotor speed and executive functioning (TMT A&B), spatial working memory span (Corsi Blocks test), cognitive impairment (MoCA), and selective attention and cognitive flexibility (Stroop). In line with our predictions, most correlations were weak and nonsignificant with the exception of VR CPT omission errors and TMT A& B, Corsi and MoCA with moderate to high correlation coefficients (see Online Resources, Table A.5. for intercorrelations for divergent validity).

Table 1 Intercorrelations between scores from VR CPT Nesplora Aquarium and computerised CPT

Assessing covariates

Because cognitive deficits in depression and anxiety extend to multiple domains, we performed two separate hierarchical regressions to identify which of the variables that are known to be affected in depression and anxiety such as spatial working memory or processing speed are significant predictors of current symptoms of depression and anxiety. Significant predictors were included in further regression analyses as covariates to identify whether Nesplora Aquarium can predict current symptoms of depression and anxiety specifically for attention and inhibition performances. Several variables were identified as predicting depressive symptoms measured with BDI-II: cognitive impairment measured with MoCA (B = − 1.37, p = 0.005, BCa CI = [− 2.28, − 0.55]); executive functioning and psychomotor processing speed measured with TMT-A (B = 0.36, p = 0.048, BCa CI = [0.04, 0.62]); spatial working memory span measured with Corsi Block Test, total score (B = − 0.34*, p = 0.018, BCa CI = [0.62, − 0.05]). For state anxiety measured with STAI-S, cognitive impairment measured with MoCA (B = − 1.35, p = 0.002, BCa CI = [− 2.17, − 0.50]); TMT-B (B = 0.11, p = 0.054, BCa CI = [0.03, 0.23]); sustained attention measured with CPT, RT for total correct responses (B = 0.47, p = 0.018, BCa CI = [0.09, 0.77]); spatial working memory span measured with Corsi Block Test, total score (B = − 0.47, p = 0.000, BCa CI = [− 0.68, − 0.27]) (see Table 2).

Table 2 Summary of linear regression for predicting symptoms of depression and anxiety for various neuropsychological measures

Effectiveness of Nesplora Aquarium in predicting symptoms of depression and anxiety

To identify whether attention measured with the Nesplora Aquarium can significantly predict symptoms of depression and anxiety, we first conducted a standard linear regression. Predictor variables were outcomes of the Nesplora Aquarium: RT for total correct responses, omission errors, commission errors and variability (SD) of RT, RT for commission errors, perseverative errors. RT for total correct responses (B = 0.36, p = 0.043, BCa CI = [0.01, 0.74]) and omission errors (B = 0.66, p = 0.001, BCa CI = [0.22, 0.87]) were significant for depressive symptoms. Similarly, anxiety symptoms were predicted by VR CPT RT for total correct responses (B = 0.49, p = 0.011, BCa CI = [0.11, 0.90]) and omission errors (B = 0.48, p = 0.008, BCa CI [0.13, 0.79]) (Table 2).

To assess whether attention measured with Nesplora Aquarium predicted symptoms of depression and anxiety above and beyond classical measures, we performed two hierarchical multiple regression analysis, first, for depressive symptoms, and second, for anxiety symptoms. At Block 1, we included only significant classic test measures that emerged as significant predictors in the previous regression analysis (see Table 4). Similarly, at Block 2 we included only Nesplora Aquarium outcomes that emerged as significant predictors of depressive and anxiety symptoms. For symptoms of depression, results indicated that at Block 1, MoCA test, TMT-A and Corsi Blocks Test contributed significantly to the regression model (F = 6.87, p = 0.000) and accounted for 21% of the variation. At Block 2, as predicted, results showed that adding VR CPT omission errors and VR CPT RT for total correct responses explained 39%, F = 9.62, p = 0.000. After controlling for results on MoCA test, TMT-A and Corsi Blocks Test, VR CPT omission errors and RT explained an additional 18%, F change = 11.09, p = 0.000. Both VR outcomes were significant predictors, omissions, B = 0.43, p = 0.005, BCa CI = [0.13, 0.70] and RT, respectively, B = 0.43, p = 0.002, BCa CI = [0.18, 0.67]. In the final model, none of the covariates were statistically significant (all ps > 0.05). For anxiety symptoms, scores of MoCA test, CPT RT, TMT-B and Corsi total score were entered at Block 1, explaining 33% of variance. After entry of VR CPT RT for total correct responses and VR CPT omissions at Block 2, the total variance explained by the model was 40%, F = 8.42, p = 0.000. After controlling for results on MoCA test, CPT RT for correct answers, TMT-B and Corsi Blocks Test, errors and VR CPT RT and omission errors explained an additional 0.8%, F change = 4.87, p = 0.01. Only VR CPT RT was a significant predictor (B = 0.31, p = 0.014, BCa CI = [0.04, 0.63]. In the final model, none of the covariates were statistically significant (all ps > 0.05) (Table 3).

Table 3 Summary of hierarchical multiple regression results for predicting symptoms of depression and anxiety

Group Effects on Attention Measured with Nesplora Aquarium

To explore patterns of attention and inhibition deficits among participants with elevated symptoms of depression and anxiety compared with participants with low symptoms using Nesplora Aquarium, we performed a binary logistic regression. Results of a binary logistic regression with Nesplora Aquarium variables (RT for total correct responses, SD RT, omission and commission errors, RT to commission errors and perseverative errors) indicated that the full model was statistically significant, χ2 = 15.70, p = 0.02 suggesting that the model distinguished between participants with low and elevated symptoms of depression and anxiety. The model explained between 17% (Cox & Snell R2) and 23% (Nagelkerke R2) of the variability in displaying symptoms of depression and anxiety and correctly predicted 69.5% of cases. As shown in Table 4, only two variables were significant: RT for total correct responses, OR = 1.09, p = 0.05, 95% BCa CI = [1.00, 1.18] and omission errors, OR = 1.08, p = 0.035, 95% BCa CI = [1.00, 1.17] (see Table 4).

Table 4 Summary of logistic regression for predicting likelihood of displaying symptoms of depression and anxiety based on VR CPT outcomes

To identify if there were any significant differences between participants with elevated symptoms of depression, those with elevated symptoms of anxiety and participants with mixed anxiety and depression symptoms we performed a Kruskal–Wallis Test for each of the Nesplora Aquarium outcomes. We used this test given the small sample of participants in some of the groups (Field 2013). Results are presented in Table 5.

Table 5 Performance on Nesplora Aquarium between participants with elevated symptoms of depression, anxiety and mixed sample of anxiety and depression symptoms

We performed post hoc comparisons for the significant Kruskal–Wallis Tests using Mann–Whitney U tests with Bonferroni adjustments to control for multiple comparisons (p = 0.05/3, p = 0.017). The following comparisons were significant. First, significant differences in RT to correct responses were found between people with anxiety symptoms (Md N = 53.60) and mixed depression and anxiety symptoms (Md N = 61.78), U = 50.00, z = − 2.81, p = 0.005. For omission errors, people with depression symptoms performed significantly fewer omission errors (Md N = 44.95) than people with mixed anxiety and depression symptoms (Md N = 64.09), U = 9.50, z = − 2.62, p = 0.009. People with anxiety symptoms (Md N = 45.64) performed significantly better on omission errors than people with mixed anxiety and depression symptoms (Md N = 64.09), U = 32.00, z = − 3.48, p = 0.001. People with anxiety symptoms (Md N = 51.79) committed more commission errors than those with mixed symptoms (Md N = 41.21), U = 49.50, z = − 2.84, p = 0.004.

Sensory effects

To identify whether participants with elevated symptoms of depression and anxiety are more likely to have visual or auditory attention impairments compared with healthy controls with low symptoms of depression and anxiety, we performed binary logistic regressions. Table 5 displays results of multivariable binary logistic regression analysis. Our predictions were partially supported as only two variables were statistically significant: omission errors for visual stimuli, OR = 1.06, p = 0.03, 95% BCa CI = [0.99, 1.14] and SD RT for visual stimuli, OR = 1.12, p = 0.001, 95% BCa CI = [1.04, 1.21] (see Online Resources, Table A.6.).

Safety concerns in VR, usability and presence

The average ratings of Nesplora Aquarium indicated a good-to-excellent usability of the product (M = 80.18, SD = 15.30). According to norms from the literature Nesplora Aquarium obtained a class A- rating, making it more usable than 85–89 of other systems (Bangor et al. 2009; Sauro and Lewis 2016). Results from an independent samples t test showed that the participants with elevated symptoms of depression and anxiety rated the usability of the CPT VR Nesplora Aquarium significantly lower than participants with low symptoms, t = 2.62, p = 0.01, BCa CI = [2.32, 15.18]. Participants with low depressive and anxiety symptoms rated Nesplora Aquarium as excellent (M = 84.45, SD = 11.49), where a score of 85 corresponds to an A + (highest grade) and a percentile rank of 96–100. Our VR system is considered more usable then 96–100% of products and less usable then 5–10%. On the other hand, participants with high symptoms rated Nesplora Aquarium as being good (M = 75.91, SD = 17.45) with a grade of B and percentile rank of 70–79, which makes it less usable, though within the range of highly acceptable products.

To identify differences between participants with low and high depression and anxiety in the level of self-reported symptoms of simulator sickness, we performed an ANCOVA, controlling for pre-test scores of simulator sickness. In line with our predictions, results showed that there were no differences on post-test simulator sickness symptoms reported by the participants, F = 0.57, p = 0.45. Only two participants (5%) with low symptoms of depression and anxiety had a score > 15 and experienced moderate to severe simulator sickness compared to five participants from the elevated symptoms group (12%). We ran an independent samples t-test to identify any differences on self-report levels of presence in VR between participants with low symptoms of depression and anxiety and those with elevated levels. No significant differences between the two groups emerged, t = 1.39, p = 0.16, BCa CI = [− 2.74, 17.52].

Discussion

This study aimed to comprehensively validate, for the first time, the Nesplora Aquarium VR experience as a measure of selective attention and inhibition in people with low and elevated symptoms of depression and anxiety. The data generally supported the validity of the Nesplora Aquarium. We first assessed validity on the basis of correlations between Nesplora Aquarium and other standardised neuropsychological measures such as computerised CPT, Stroop test, MoCA, Corsi block test). We expected high correlations between outcomes of Nesplora Aquarium and the outcomes of the computerised version of CPT, because the two tests were highly comparable in terms of outcomes as they measured the same theoretical construct: attention and inhibition. As predicted, the outcomes of the two tests were highly and positively correlated. Comparable outcomes between VR tests and traditional tests for neuropsychological assessment were previously reported in a meta-analysis with a pooled correlation coefficient of 0.51 (Neguţ et al. 2015). In addition to the expected high correlation between tests that measures similar constructs such as attention and inhibition, we also found weak or a lack of correlations between Nesplora and other cognitive tests that measure different constructs (Urbina 2004), i.e. between Nesplora measuring attention and inhibition and TMT A& B, Corsi and MoCA measuring executive functioning, spatial working memory and cognitive impairment. However, we did find moderate to high correlations between Nesplora Aquarium omission errors and TMT A& B, Corsi and MoCA. Unexpected correlations between VR tests and traditional tests measuring different cognitive constructs have been observed in other studies using a sample of active-duty soldiers (Armstrong et al. 2013). For example, Armstrong et al. (2013) computed correlations between a VR Stroop Task and classical tests- the Automated Neuropsychological Assessment Metrics and showed that VR Stroop outcomes were correlated with subscales of ANAM that measure other cognitive processes than Stroop such as visual search, sustained attention, and working memory, spatial processing and visuo–spatial working memory. In our study, a possible explanation for a strong positive association between TMT A& B, Corsi and MoCA and Nesplora Aquarium omission errors is that this type of error might measure a component of attention processing that is closely linked with the general domain of cognitive functioning. As currently these unexpected correlations between tests measuring different cognitive functions are unclear, future research could focus on explaining their nature by using a broader range of neurophysiological measures. Testing additional cognitive abilities such as memory, general IQ could inform better the relationship between outcomes of Nesplora Aquarium and these cognitive abilities. Despite the few unexpected findings, our results do support the validity of Nesplora Aquarium for measuring sustained attention, impulsivity and inhibition.

A second objective of our study was to assess whether Nesplora Aquarium could predict symptoms of depression and anxiety above and beyond other traditional measures. Nesplora Aquarium seems to be a strong predictor of current symptoms of anxiety and depression as shown by its ability to explain these symptoms above and beyond classic tests. First, the regression models with Nesplora Aquarium as predictors explained symptoms of depression and anxiety more than regression models with traditional neuropsychological measures such as MoCA, traditional CPT, TMT A & B, and Stroop. Several measures were identified as significant covariates: general cognition (MoCA), as well as multiple cognitive domains: RT for correct responses (CPT), such as psychomotor speed, mental flexibility and shifting (TMT A & B), and spatial working memory span (Corsi Blocks Test). A full discussion on covariates can be found in the Online Resources. Second, after controlling for covariates: MoCA, traditional CPT, TMT A & B, and Stroop, Nesplora Aquarium was better at predicting current symptoms of depression and anxiety than other traditional measures of cognitive functions. However, these encouraging findings were limited to two out of six Nesplora Aquarium outcomes. This lack of association between four Nesplora Aquarium outcomes could be explained by increased heterogeneity of both depression and anxiety (Keller et al. 2019; Nandi et al. 2009) which might lead to some individuals displaying attention impairment whilst others are not that affected. The fact that omission errors and RT for correct responses were the only significant predictors of symptoms of depression can be explained by the “cognitive speed hypothesis” which states that depression is characterised by cognitive slowness which may lead to cognitive dysfunction (Den Hartog et al. 2003). Depressed individuals might display reduced attention performance due to slow processing speed (Degl'Innocenti et al. 1998; Den Hartog et al. 2003; Egeland et al. 2003) as studies showed slower RT and psychomotor slowness in depression (Kemp et al. 2010; Kertzman et al. 2010; Marazziti et al. 2010). In our study this is consistent with slower RTs for correct targets. Additionally, because omission errors are closely linked to inattention and result from failing to respond to correct targets, it might be the case that the increased number of these errors were caused by the slow processing speed, as the participants did not have enough time to process the stimuli and correctly respond to them. For anxiety, only RT for correct responses was a significant predictor which might be explained by the Attentional Control Theory (ACT) (Eysenck et al. 2007) which posits that anxiety is associated with increased distractibility and reduced efficiency of central executive functions. However, according to ACT, we also expected impaired inhibitory control in anxiety. Commission errors and RT for commission errors, an index of impulsivity and poor inhibitory control, were not significant predictors of current anxiety symptoms. As we only included participants with elevated symptoms, future studies might try and investigate these associations using samples of individuals with specific anxiety and depression disorders as deficits may be differently influenced by disorder. We also compared participants with elevated symptoms of depression alone, with elevated symptoms of anxiety alone and those with elevated and mixed anxiety and depression symptoms. The comorbidity of both anxiety and depression symptoms is well documented in the literature (Möller et al. 2016) and may result in worse impairment than each condition alone (Tiller 2013; Tyrer 2001). Our results showed a similar pattern to these previous studies as people with mixed anxiety and depression performed worse on omission errors and RT for correct responses than those with either depression or anxiety symptoms. However, no differences emerged on SD RT, RT to commission errors and perseverative errors. This brings further support in favour of the idea that not all attention domains are impaired in depression and anxiety or in conditions with mixed symptoms. Results could also be explained in light of medication taken by participants. In our sample, 22 participants were medicated which could have altered participants’ responses on cognitive tests. Research is controversial when it comes to the positive or negative effects of medication on cognitive function (Godard et al. 2011; Pachet and Wisniewski 2003; Rosenblat et al. 2015; Stein and Strickland 1998; Wagner et al. 2012; Wingo et al 2009). However, in our study we cannot exclude that because participants took medication, including on the day of testing, this may have had a positive impact on their performance thus limiting their impairment on attention outcomes as only two out of six Nesplora Aquarium outcomes were correlated with symptoms of depression and anxiety.

Both depression and anxiety seem to negatively impact effortful and complex tasks (Hartlage et al. 1993). In anxiety, tasks with increased difficulty impair attention more than less complex tasks; for example, in less difficult tasks individuals can compensate for the detrimental effects of anxiety (Eysenck et al. 2007). Similarly, depressed individuals direct their cognitive resources on depression-related themes, and they display a biased attention for emotional stimuli (Disner et al. 2011; Hartlage et al. 1993). A review of task difficulty of VR neuropsychological tests compared with analogue classic tests highlighted increased task difficulty of VR measures for executive functioning, including attention (Neguţ et al. 2016). This may clarify why Nesplora Aquarium predicts symptoms of anxiety nearly twice as much as traditional CPTs. VR tasks require additional cognitive resources because they recreate real-life scenarios which are more complex than classical tasks (Neguţ et al. 2016). For example, in Nesplora Aquarium the environment contains a real-life replica of an aquarium with many tanks, various environmental distractors to ignore and fish to pay attention to. In contrast, the classical CPT consists of letters which appear on a black computer screen. Both tasks measure the same theoretical construct, but the complexity and real-life similarity are higher for the Nesplora Aquarium. Most likely the increased realism of VR increases its power of prediction compared with traditional tests.

Our results are in agreement with previous studies which investigated the incremental value of VR in detecting attention and executive functioning impairments. For example, Lalonde et al. (2013) showed that performance on a VR Stroop task predicted everyday behavioural executive functioning better than classical paper-and-pencil tasks among a sample of healthy adolescents. Rodriguez et al. (2018) used a VR classroom to assess attention deficits of children of ADHD. Their results indicated that the VR classroom predicted ADHD better than a computerised CPT. Recent meta-analyses showed that VR-based measures discriminate between various categories of populations known to have impaired attention and executive functioning such as children with ADHD, individuals with schizophrenia or brain injury and healthy controls (Gilboa et al. 2018; Negut et al. 2016; Parsons et al. 2019). Taken together, VR seems to be an effective tool to use in assessment contexts. By creating 3D simulations of real environments and situations, it offers a challenging, engaging and highly ecological way to test attention and inhibition processes. However, in the current study we assessed if VR is better at predicting current symptoms of depression and anxiety. Future studies might validate the predictive validity of VR tests by investigating whether VR can successfully predict symptoms of depression and anxiety in the long run, for example, for the next 6 months or other real-life functioning indexes like employment status or social services. For our third objective, we investigated whether Nesplora Aquarium can distinguish between participants with low and elevated symptoms of depression and anxiety. We expected that participants with elevated depression and anxiety symptoms were more likely to have poorer scores on the Nesplora Aquarium test compared with individuals with lower symptoms and to identify attention domains which are most affected. Results showed that overall, outcomes of Nesplora significantly distinguished between participants with elevated and low symptoms of depression and anxiety, the first displaying overall poorer attention performance. Taken together, our results provide evidence in favour of reduced vigilance, increased inattention and psychomotor slowness in depression and anxiety. Our findings suggest that the attention and inhibition impairments in depression and anxiety are heterogenous. We argue that anxiety and depression affect differently various attention domains. The only attention domains that were impaired in our sample of participants were increased inattention and reduced processing speed. Most likely, there is not a global deficit of attention in depression and anxiety.

In relation to the sensory effects of Nesplora Aquarium, we predicted that individuals with elevated symptoms of anxiety and depression would be more impaired on visual and auditory modalities compared to individuals with low symptoms. In our study, we only identified some impairments (omission errors and SD to RT) in the visual domain, but not in the auditory domain. Individuals with elevated symptoms of depression and anxiety committed more omission errors in the visual modality, but not in the auditory modality which could indicate that they are less vigilant when attending to visual stimuli. This might suggest that future cognitive rehabilitation or remediation programs might focus on visual tasks. Variability of RT, a measure of consistency of performance (Stuss et al. 1989), suggests changes in sustained attention or fatigability during the task (Climent et al. 2019). Increased RT variability in the visual condition is consistent with increased inattention in this condition, which might indicate increased distractibility for visual stimuli for individuals with elevated symptoms of depression and anxiety (and possibly a sign of “wandering attention” when people get distracted by internal and external stimuli Stuss and Benson 1984; Tinius 2003). This result seems to sit in contrast to previous studies which showed impaired deficits in early auditory and visual processing in depressed and anxious samples (Chang et al. 2011; Kahkonen et al. 2007; Qiu et al. 2011; Schirmer and Escoffier 2010; Takei et al. 2009; Weinstein 1995; Yang et al. 2019; Zweerings et al. 2019). A potential explanation for our results is that previous studies assessed auditory and visual processing at early pre-attentive automatically processing of visual and auditory processing, while ours assessed attention and inhibition as higher cognitive functions. Hence, differences in early visual and auditory processes may disappear in later stage of higher cognitive functions pointing to a generalised deficit of sustained attention irrespective of sensory processing. Future studies could compare pre-attentive sensory processing with later stages of sensory processing modulated by sustained attention to examine this possibility. We also note that in our study we did not account for the cognitive styles of the participants. According to the cognitive theory of multimedia learning, people process information using the visual and the auditory channel. The visualiser–verbaliser cognitive style posits that some people prefer, rely more and perform better when they learn from pictures (visualisers) while other people prefer words or text (verbalisers) (Koć-Januchta et al. 2017; Mayer and Massa 2003). As we did not measure and control for this variable, we cannot exclude that participant’s cognitive style could have impacted our results. In our study, we only identified some impairments (omission errors and SD to RT) in the visual domain, but not in the auditory domain; we cannot rule out the fact that the two groups of participants were not similar in relation to this variable. Future studies can investigate whether individual differences in cognitive styles might moderate the performance on visual and auditory stimuli.

Finally, our last objective was to assess self-reported usability, simulator sickness symptoms and level of presence in VR across individuals with high and low symptoms of depression and anxiety. As predicted, the participants’ responses showed no significant difference between the two groups for symptoms of simulator sickness which is in accordance with results from previous studies (Bouchard et al. 2009, 2014; Kim et al. 2019; Robillard et al. 2003). It might be the case that unlike other vulnerable populations such as older adults or patients with neurological conditions who are more likely to experience side effects associated with exposure to VR (Brooks et al. 2010; Hutton et al. 2018; Matheis et al. 2007), individuals with elevated symptoms of depression and anxiety tolerate VR exposure well. Most likely, because during the assessment with Nesplora Aquarium participants were seated and did not move around the environment, they did not experience significant sensory mismatch or incongruence between visual, vestibular and proprioceptive information, which have been linked with simulator sickness symptoms (Kim et al. 2017). Similarly, as we predicted, there were no significant differences between people with low and elevated symptoms of depression and anxiety on self-report presence in VR. To our knowledge, the current study is the first to address individual differences among people with anxiety and depression compared with controls. Although there are some studies that linked presence with personality (Kober and Neuper 2013), in relation to cognitive abilities previous studies showed that there were no significant differences between spatial presence and spatial ability (Coxon et al. 2016). The current study identified a similar pattern and showed that people with low and elevated levels of anxiety and depression, which clearly differ on the level of cognitive functioning, report the same level of presence within VR.

The results of test usability showed that participants with low symptoms of depression and anxiety rated the usability of the VR system higher than those with high symptoms. However, even the group of participants with high levels of anxiety and depression gave an acceptable rating to the VR product. Other studies that assessed the usability of various VR systems with older adults reported similar ratings to our high anxiety and depression group (data for individuals with elevated symptoms of depression and anxiety) (Corno et al. 2014; Pedroli et al. 2018). The current study assessed Nesplora usability using the SUS scale; however, informally many participants said that they were so absorbed in the environment that they lost track of time. Almost unanimously participants preferred the Nesplora Aquarium over the classical tests. Some highlighted that the task in VR while more fun and engaging than the computerised task was more difficult. Differences in usability ratings between individuals with low symptoms compared with those with elevated symptoms of depression and anxiety may be explained by their self-perceived performance in VR. In the absence of formal recordings of usability other than SUS, we can only speculate that participants with elevated symptoms, especially high anxiety, were preoccupied by their level of performance which in turn might have influenced their usability ratings. Future studies might further explore these differences in usability using objective measures and examine mechanisms that can account for differences.

Limitations

Despite a number of strengths of the current study, including being the first to assess attention and inhibition in a highly ecological VR setting, and the promising results, a number of limitations should be considered. First, our design was cross-sectional, which does not allow one to infer a causal relationship between attention and inhibition and current symptoms of depression and anxiety. That being said we are not certain that attention and inhibition can influence symptoms of depression and anxiety or if the other way around, current symptoms of depression and anxiety influence attention and inhibition performance. A recent meta-analysis showed positive significant effects of computerised cognitive training in improvements in symptom severity and daily functioning for patients with depression (Motter et al. 2016) which might highlight the role of cognition on depression symptomatology. Similarly, training of working memory showed significant results in reducing anxiety symptoms (Hadwin and Richards 2016). If improvements in cognition can lead to a reduction in symptoms, this can be taken as evidence of the importance of cognitive functioning in facilitating and maintaining of these symptoms.

We did not control for medication effects on cognitive performance. Psychotropic medication may negatively impact some cognitive functions; thus, there is lack of consensus in the literature regarding the positive or negative effect of medications like antidepressants on cognitive functioning (Godard et al. 2011; Pachet and Wisniewski 2003; Rosenblat et al. 2015; Stein and Strickland 1998; Wagner et al. 2012; Wingo et al. 2009). On most cognitive functions, in our study, there were no significant differences between the medicated and non-medicated group, but we cannot rule out potential effects of medication on cognitive functioning. In our study, we included a comprehensive set of well-validated measures of neuropsychological measures; however, we did not include measures of memory, language comprehension or IQ, and thus, we could not control for their effects. Future studies might investigate the impact of these individual differences and their association with attention performance on VR tasks. Although a limitation of the current study, we did not apply strict inclusion criteria in a sense that we did not focus only on patients with a clinical diagnosis of depression and anxiety disorders. Instead, we applied a more relaxed inclusion criteria and used two well-validated scales to screen for clinically significant symptoms of depression and anxiety, such as BDI-II and STAI. We did so because we were mostly interested in the ability of the Nesplora to predict clinically significant symptoms of depression and anxiety across a wide range of participants, not only those with a clinical diagnosis of depression and anxiety disorders. For example, increased symptoms of anxiety and depression are also prevalent across various chronic medical conditions such as diabetes (Li et al. 2019), vulnerable populations like older adults, LGBT community, people who commit self-harm (Remes et al. 2016) or students (AlFaris et al. 2016).

Conclusions

The current study supported the validity of Nesplora Aquarium and its added effectiveness in predicting symptoms of depression and anxiety when compared to traditional tests. VR tests have increased ecological validity, and consequently increased task difficulty similar to real-life scenarios; hence, they may provide a more precise measure of attention and inhibition. Hence, using Nesplora could be more efficient in identifying people with increased anxiety and depression when facing everyday tasks. As the visual modality was more affected than the auditory modality in participants with high levels of anxiety and depression, it suggests the presence of domain-specific deficits; however, further studies using different tasks and tests need to replicate this finding to strengthen this conclusion. Finally, a future focus on usability in clinical psychology could help designers and researchers to develop VR tasks that are effective, easy to use and enjoyable for vulnerable populations to increase their adherence throughout the evaluation and treatment process.

References

  1. AlFaris E et al (2016) Health professions’ students have an alarming prevalence of depressive symptoms: exploration of the associated factors. BMC Med Educ 16:1–8. https://doi.org/10.1186/s12909-016-0794-y

    Article  Google Scholar 

  2. Armstrong CM, Reger GM, Edwards J, Rizzo AA, Courtney CG, Parsons TD (2013) Validity of the virtual reality stroop task (VRST) in active duty military. J Clin Exp Neuropsychol 35:113–123. https://doi.org/10.1080/13803395.2012.740002

    Article  Google Scholar 

  3. Asahi S, Okamoto Y, Okada G, Yamawaki S, Yokota N (2004) Negative correlation between right prefrontal activity during response inhibition and impulsiveness: a fMRI study. Eur Arch Psychiatry Clin Neurosci 254:245–251. https://doi.org/10.1007/s00406-004-0488-z

    Article  Google Scholar 

  4. Bangor A, Kortum P, Miller J (2009) Determining what individual SUS scores mean: adding an adjective rating scale. J Usability Stud 4:114–123

    Google Scholar 

  5. Bayard S, Erkes J, Moroni C, College des Psychologues Cliniciens specialises en Neuropsychologie du Languedoc R (2011) Victoria stroop test: normative data in a sample group of older people and the study of their clinical applications in the assessment of inhibition in Alzheimer’s disease. Arch Clin Neuropsychol 26:653–661. https://doi.org/10.1093/arclin/acr053

    Article  Google Scholar 

  6. Beck AT, David D, Dobrean A (2012) Inventarul de depresie Beck-Editia a doua (BDI-II) [Beck Depression Inventory-II]. RTS Romanian Testing Services, Cluj-Napoca

    Google Scholar 

  7. Beck AT, Steer RA, Brown GK (1996) Beck depression inventory-II. The Psychological Corporation, San Antonio

    Google Scholar 

  8. Benton AL (1994) Neuropsychological assessment. Annu Rev Psychol 45:1–23. https://doi.org/10.1146/annurev.ps.45.020194.000245

    Article  Google Scholar 

  9. Bouchard S et al (2017) Virtual reality compared with in vivo exposure in the treatment of social anxiety disorder: a three-arm randomised controlled trial. Br J Psychiatry 210:276–283. https://doi.org/10.1192/bjp.bp.116.184234

    Article  Google Scholar 

  10. Bouchard S, St-Jacques J, Renaud P, Wiederhold BK (2009) Side effects of immersions in virtual reality for people suffering from anxiety disorders. J Cyber Ther Rehabil 2:127–137

    Google Scholar 

  11. Brooke J (1986) System usability scale (SUS): a quick-and-dirty method of system evaluation user information. Digital Equipment Co Ltd, Reading

    Google Scholar 

  12. Brooks JO, Goodenough RR, Crisler MC, Klein ND, Alley RL, Koon BL, Logan WC Jr, Ogle JH, Tyrrell RA, Wills RF (2010) Simulator sickness during driving simulation studies. Accid Anal Prev 42:788–796. https://doi.org/10.1016/j.aap.2009.04.013

    Article  Google Scholar 

  13. Chang Y, Xu J, Shi N, Pang X, Zhang B, Cai Z (2011) Dysfunction of preattentive visual information processing among patients with major depressive disorder. Biol Psychiatry 69:742–747. https://doi.org/10.1016/j.biopsych.2010.12.024

    Article  Google Scholar 

  14. Chaytor N, Schmitter-Edgecombe M (2003) The ecological validity of neuropsychological tests: a review of the literature on everyday cognitive skills. Neuropsychol Rev 13:181–197. https://doi.org/10.1023/b:nerv.0000009483.91468.fb

    Article  Google Scholar 

  15. Chisholm D, Sweeny K, Sheehan P, Rasmussen B, Smit F, Cuijpers P, Saxena S (2016) Scaling-up treatment of depression and anxiety: a global return on investment analysis. Lancet Psychiatry 3:415–424. https://doi.org/10.1016/S2215-0366(16)30024-4

    Article  Google Scholar 

  16. Climent G et al (2019) New virtual reality tool (Nesplora Aquarium) for assessing attention and working memory in adults: a normative study. Appl Neuropsychol Adult. https://doi.org/10.1080/23279095.2019.1646745

    Article  Google Scholar 

  17. Cobb SV, Nichols S, Ramsey A, Wilson JR (1999) Virtual reality-induced symptoms and effects (VRISE). Presence (Camb) 8:169–186. https://doi.org/10.1162/105474699566152

    Article  Google Scholar 

  18. Coles AS, Lee Y, Subramaniapillai M, McIntyre RS (2009) Cognitive deficits in major depression: from mechanisms to management. In: McIntyre RS (ed) Major depressive disorder 2020. Elsevier, St. Louis, MO, pp. 51–62. https://doi.org/10.1016/B978-0-323-58131-8.00004-5

  19. Conners CK, Epstein JN, Angold A, Klaric J (2003) Continuous performance test performance in a normative epidemiological sample. J Abnorm Child Psychol 31:555–562. https://doi.org/10.1023/a:1025457300409

    Article  Google Scholar 

  20. Corno G, Bouchard S, Forget H (2014) Usability Assessment of the Virtual Multitasking Test (V-MT) for Elderly People. In: Proceedings of annual review of cybertherapy and telemedicine. IOS Press, pp 168–172. https://doi.org/10.3233/978-1-61499-401-5-168 P (1972) Memory and the medial temporal region of the brain. McGill University, Montreal, QB

  21. Corsi P (1972) Memory and the medial temporal region of the brain. McGill University, Montreal

    Google Scholar 

  22. Coxon M, Kelly N, Page S (2016) Individual differences in virtual reality: are spatial presence and spatial ability linked? Virtual Real 20:203–212. https://doi.org/10.1007/s10055-016-0292-x

    Article  Google Scholar 

  23. Degl’Innocenti A, Agren H, Backman L (1998) Executive deficits in major depression. Acta Psychiatr Scand 97:182–188. https://doi.org/10.1111/j.1600-0447.1998.tb09985.x

    Article  Google Scholar 

  24. Den Hartog HM, Derix MM, Van Bemmel AL, Kremer B, Jolles J (2003) Cognitive functioning in young and middle-aged unmedicated out-patients with major depression: testing the effort and cognitive speed hypotheses. Psychol Med 33:1443–1451. https://doi.org/10.1017/s003329170300833x

    Article  Google Scholar 

  25. Disner SG, Beevers CG, Haigh EA, Beck AT (2011) Neural mechanisms of the cognitive model of depression. Nat Rev Neurosci 12:467–477. https://doi.org/10.1038/nrn3027

    Article  Google Scholar 

  26. Drummond RJ, Jones KD (2016) Assessment procedures for counselors and helping professionals. Pearson, Boston

    Google Scholar 

  27. Efron B, Tibshirani R (1993) An introduction to the bootstrap. CRC Press, Florida

    Book  Google Scholar 

  28. Egeland J et al (2003) Attention profile in schizophrenia compared with depression: differential effects of processing speed, selective attention and vigilance. Acta Psychiatr Scand 108:276–284. https://doi.org/10.1034/j.1600-0447.2003.00146.x

    Article  Google Scholar 

  29. Evenden JL (1998) The pharmacology of impulsive behaviour in rats IV: the effects of selective serotonergic agents on a paced fixed consecutive number schedule. Psychopharmacology (Berl) 140:319–330. https://doi.org/10.1007/s002130050773

    Article  Google Scholar 

  30. Eysenck MW, Derakshan N, Santos R, Calvo MG (2007) Anxiety and cognitive performance: attentional control theory. Emotion 7:336–353. https://doi.org/10.1037/1528-3542.7.2.336

    Article  Google Scholar 

  31. Field A (2013) Discovering statistics using IBM SPSS statistics. Sage Publications, London

    Google Scholar 

  32. Fox J (2015) Applied regression analysis and generalized linear models. Sage Publications, California

    Google Scholar 

  33. Gilboa Y, Fogel-Grinvald H, Chevignard M (2018) Virtual classroom assessment for children and adolescents with attention deficits: a systematic review and meta-analysis of measurement properties. J Atten Disord. https://doi.org/10.1177/1087054718808590

    Article  Google Scholar 

  34. Godard J, Grondin S, Baruch P, Lafleur MF (2011) Psychosocial and neurocognitive profiles in depressed patients with major depressive disorder and bipolar disorder. Psychiatry Res 190:244–252. https://doi.org/10.1016/j.psychres.2011.06.014

    Article  Google Scholar 

  35. Greve KW, Stickle TR, Love JM, Bianchini KJ, Stanford MS (2005) Latent structure of the Wisconsin card sorting test: a confirmatory factor analytic study. Arch Clin Neuropsychol 20:355–364. https://doi.org/10.1016/j.acn.2004.09.004

    Article  Google Scholar 

  36. Gualtieri CT, Johnson LG (2005) ADHD: Is objective diagnosis possible? Psychiatry 2:44–53.

  37. Hadwin JA, Richards HJ (2016) Working memory training and CBT reduces anxiety symptoms and attentional biases to threat: a preliminary study. Front Psychol 7:47. https://doi.org/10.3389/fpsyg.2016.00047

    Article  Google Scholar 

  38. Hallion LS, Steinman SA, Kusmierski SN (2018) Difficulty concentrating in generalized anxiety disorder: an evaluation of incremental utility and relationship to worry. J Anxiety Disord 53:39–45. https://doi.org/10.1016/j.janxdis.2017.10.007

    Article  Google Scholar 

  39. Halperin JM, Wolf L, Greenblatt ER, Young G (1991) Subtype analysis of commission errors on the continuous performance test in children. Dev Neuropsychol 7:207–217. https://doi.org/10.1080/87565649109540488

    Article  Google Scholar 

  40. Hartlage S, Alloy LB, Vazquez C, Dykman B (1993) Automatic and effortful processing in depression. Psychol Bull 113:247–278. https://doi.org/10.1037/0033-2909.113.2.247

    Article  Google Scholar 

  41. Iriarte Y, Diaz-Orueta U, Cueto E, Irazustabarrena P, Banterla F, Climent G (2016) AULA-advanced virtual reality tool for the assessment of attention: normative study in Spain. J Atten Disord 20:542–568. https://doi.org/10.1177/1087054712465335

    Article  Google Scholar 

  42. Kahkonen S, Yamashita H, Rytsala H, Suominen K, Ahveninen J, Isometsa E (2007) Dysfunction in early auditory processing in major depressive disorder revealed by combined MEG and EEG. J Psychiatry Neurosci 32:316–322

    Google Scholar 

  43. Keller AS, Leikauf JE, Holt-Gosselin B, Staveland BR, Williams LM (2019) Paying attention to attention in depression. Transl Psychiatry 9:279. https://doi.org/10.1038/s41398-019-0616-1

    Article  Google Scholar 

  44. Kemp AH et al (2010) Impact of depression heterogeneity on attention: an auditory oddball event related potential study. J Affect Disord 123:202–207. https://doi.org/10.1016/j.jad.2009.08.010

    Article  Google Scholar 

  45. Kennedy RS, Lane NE, Berbaum KS, Lilienthal MG (1993) Simulator sickness questionnaire: an enhanced method for quantifying simulator sickness. Int J Aviat Psychol 3:203–220. https://doi.org/10.1207/s15327108ijap0303_3

    Article  Google Scholar 

  46. Kertzman S, Reznik I, Hornik-Lurie T, Weizman A, Kotler M, Amital D (2010) Stroop performance in major depression: selective attention impairment or psychomotor slowness? J Affect Disord 122:167–173. https://doi.org/10.1016/j.jad.2009.08.009

    Article  Google Scholar 

  47. Kessels RPC (2019) Improving precision in neuropsychological assessment: Bridging the gap between classic paper-and-pencil tests and paradigms from cognitive neuroscience. Clin Neuropsychol 33:357–368. https://doi.org/10.1080/13854046.2018.1518489

    Article  Google Scholar 

  48. Kim A, Darakjian N, Finley JM (2017) Walking in fully immersive virtual environments: an evaluation of potential adverse effects in older adults and individuals with Parkinson’s disease. J Neuroeng Rehabil 14:16. https://doi.org/10.1186/s12984-017-0225-2

    Article  Google Scholar 

  49. Kim YI et al (2019) Visuo-haptic-based multimodal feedback virtual reality solution to improve anxiety symptoms: a proof-of-concept study. Psychiatry Investig 16:167–171. https://doi.org/10.30773/pi.2018.12.25.1

    Article  Google Scholar 

  50. Kober SE, Neuper C (2013) Personality and presence in virtual reality: does their relationship depend on the used presence measure? Int J Hum Comput Interact 29:13–25. https://doi.org/10.1080/10447318.2012.668131

    Article  Google Scholar 

  51. Koć-Januchta M, Höffler T, Thoma GB, Prechtl H, Leutner D (2017) Visualizers versus verbalizers: effects of cognitive style on learning with texts and pictures—an eye-tracking study. Comput Hum Behav 68:170–179. https://doi.org/10.1016/j.chb.2016.11.028

    Article  Google Scholar 

  52. Laver K et al (2012) Virtual reality grocery shopping simulator: development and usability in neurological rehabilitation. Presence 21:183–191. https://doi.org/10.1162/PRES_a_00098

    Article  Google Scholar 

  53. Lavoie R et al (2020) Virtual experience, real consequences: the potential negative emotional consequences of virtual reality gameplay. Virtual Real. https://doi.org/10.1007/s10055-020-00440-y

    Article  Google Scholar 

  54. Lalonde G, Henry M, Drouin-Germain A, Nolin P, Beauchamp MH (2013) Assessment of executive function in adolescence: a comparison of traditional and virtual reality tools. J Neurosci Methods 219:76–82. https://doi.org/10.1016/j.jneumeth.2013.07.005

    Article  Google Scholar 

  55. Lezak MD, Howieson DB, Loring DW, Fischer S (2004) Neuropsychological assessment. Oxford University Press, New York

    Google Scholar 

  56. Li H, Ge S, Greene B, Dunbar-Jacob J (2019) Depression in the context of chronic diseases in the United States and China. Int J Nurs Sci 6:117–122. https://doi.org/10.1016/j.ijnss.2018.11.007

    Article  Google Scholar 

  57. Liu J et al. (2020) Influence of comorbid anxiety symptoms on cognitive deficits in patients with major depressive disorder. J Affect Disord 260:91–96. https://doi.org/10.1016/j.jad.2019.08.091

    Article  Google Scholar 

  58. Losier BJ, McGrath PJ, Klein RM (1996) Error patterns on the continuous performance test in non‐medicated and medicated samples of children with and without ADHD: A meta‐analytic review. J Child Psychol Psychiatry 37:971–987. https://doi.org/10.1111/j.1469-7610.1996.tb01494.x

  59. Mallinckrodt B, Abraham WT, Wei M, Russell DW (2006) Advances in testing the statistical significance of mediation effects. J Couns Psychol 53:372–378. https://doi.org/10.1037/0022-0167.53.3.372

    Article  Google Scholar 

  60. Marazziti D, Consoli G, Picchetti M, Carlini M, Faravelli L (2010) Cognitive impairment in major depression. Eur J Pharmacol 626:83–86. https://doi.org/10.1016/j.ejphar.2009.08.046

    Article  Google Scholar 

  61. Mayer RE, Massa LJ (2003) Three facets of visual and verbal learners: cognitive ability, cognitive style, and learning preference. J Educ Psychol 95:833–846. https://doi.org/10.1037/0022-0663.95.4.833

    Article  Google Scholar 

  62. Miller J (1991) Reaction time analysis with outlier exclusion: bias varies with sample size. Q J Exp Psychol A 43:907–912. https://doi.org/10.1080/14640749108400962

    Article  Google Scholar 

  63. Miyake A, Friedman NP, Emerson MJ, Witzki AH, Howerter A, Wager TD (2000) The unity and diversity of executive functions and their contributions to complex “Frontal Lobe” tasks: a latent variable analysis. Cogn Psychol 41:49–100. https://doi.org/10.1006/cogp.1999.0734

    Article  Google Scholar 

  64. Morán AL, Ramírez-Fernández C, Meza-Kubo V, Orihuela-Espina F, García-Canseco E, Grimaldo AI, Sucar E (2015) On the effect of previous technological experience on the usability of a virtual rehabilitation tool for the physical activation and cognitive stimulation of elders. J Med Syst 39:104. https://doi.org/10.1007/s10916-015-0297-0

    Article  Google Scholar 

  65. Motter JN, Pimontel MA, Rindskopf D, Devanand DP, Doraiswamy PM, Sneed JR (2016) Computerized cognitive training and functional recovery in major depressive disorder: a meta-analysis. J Affect Disord 189:184–191. https://doi.org/10.1016/j.jad.2015.09.022

    Article  Google Scholar 

  66. Mousavi M, Jen YH, Musa SNB (2013) A review on cybersickness and usability in virtual environments. In: Advanced engineering forum, vol 10, pp 34–39. https://doi.org/10.4028/www.scientific.net/aef.10.34

  67. Möller HJ, Bandelow B, Volz HP, Barnikol UB, Seifritz E, Kasper S (2016) The relevance of “mixed anxiety and depression” as a diagnostic category in clinical practice. Eur Arch Psychiatry Clin Neurosci 266:725–736. https://doi.org/10.1007/s00406-016-0684-7

    Article  Google Scholar 

  68. Mueller ST, Piper BJ (2014) The psychology experiment building language (PEBL) and PEBL test battery. J Neurosci Methods 222:250–259. https://doi.org/10.1016/j.jneumeth.2013.10.024

    Article  Google Scholar 

  69. Nandi A, Beard JR, Galea S (2009) Epidemiologic heterogeneity of common mood and anxiety disorders over the lifecourse in the general population: a systematic review. BMC Psychiatry 9:31. https://doi.org/10.1186/1471-244X-9-31

    Article  Google Scholar 

  70. Nasreddine ZS et al (2005) The Montreal cognitive assessment, MoCA: a brief screening tool for mild cognitive impairment. J Am Geriatr Soc 53:695–699. https://doi.org/10.1111/j.1532-5415.2005.53221.x

    Article  Google Scholar 

  71. Negut A, Jurma AM, David D (2017) Virtual-reality-based attention assessment of ADHD: clinicaVR: classroom-CPT versus a traditional continuous performance test. Child Neuropsychol 23:692–712. https://doi.org/10.1080/09297049.2016.1186617S

    Article  Google Scholar 

  72. Neguţ A, Matu S-A, Sava FA, David D (2016) Task difficulty of virtual reality-based assessment tools compared to classical paper-and-pencil or computerized measures: a meta-analytic approach. Comput Hum Behav 54:414–424. https://doi.org/10.1016/j.chb.2015.08.029

    Article  Google Scholar 

  73. Negut A, Matu SA, Sava FA, David D (2016) Virtual reality measures in neuropsychological assessment: a meta-analytic review. Clin Neuropsychol 30:165–184. https://doi.org/10.1080/13854046.2016.1144793

    Article  Google Scholar 

  74. Neguţ A, Matu SA, Sava FA, David D (2015) Convergent validity of virtual reality neurocognitive assessment: a meta-analytic approach. Transylv J Psychol 16:31–55

    Google Scholar 

  75. Nielsen J, Mack R (1994) Usability inspection methods. Wiley, New York

    Book  Google Scholar 

  76. Pacheco-Unguetti AP, Acosta A, Marques E, Lupianez J (2011) Alterations of the attentional networks in patients with anxiety disorders. J Anxiety Disord 25:888–895. https://doi.org/10.1016/j.janxdis.2011.04.010

    Article  Google Scholar 

  77. Pachet AK, Wisniewski AM (2003) The effects of lithium on cognition: an updated review. Psychopharmacology (Berl) 170:225–234. https://doi.org/10.1007/s00213-003-1592-x

    Article  Google Scholar 

  78. Parry I, Painting L, Bagley A, Kawada J, Molitor F, Sen S, Greenhalgh DG, Palmieri TL (2015) A pilot prospective randomized control trial comparing exercises using videogame therapy to standard physical therapy: 6 months follow-up. J Burn Care Res 36:534–44. https://doi.org/10.1097/BCR.0000000000000165

    Article  Google Scholar 

  79. Parsons TD, Barnett M (2017) Validity of a newly developed measure of memory: feasibility study of the virtual environment grocery store. J Alzheimers Dis 59:1227–1235. https://doi.org/10.3233/JAD-170295

    Article  Google Scholar 

  80. Parsons TD, Carlew AR, Magtoto J, Stonecipher K (2017) The potential of function-led virtual environments for ecologically valid measures of executive function in experimental and clinical neuropsychology. Neuropsychol Rehabil 27:777–807. https://doi.org/10.1080/09602011.2015.1109524

    Article  Google Scholar 

  81. Parsons TD, Duffield T, Asbee J (2019) A comparison of virtual reality classroom continuous performance tests to traditional continuous performance tests in delineating ADHD: a meta-analysis. Neuropsychol Rev 29:338–356. https://doi.org/10.1007/s11065-019-09407-6

    Article  Google Scholar 

  82. Parsons TD, Rizzo A (2019) A review of virtual classroom environments for neuropsychological assessment. Virtual reality for psychological and neurocognitive interventions. Springer, New York, pp 247–265

    Chapter  Google Scholar 

  83. Pedroli E, Cipresso P, Serino S, Riva G, Albani G (2013) A virtual reality test for the assessment of cognitive deficits: usability and perspectives. In: Proceedings of the 7th international conference on pervasive computing technologies for healthcare and workshops. IEEE, Venice, Italy, pp 453–458. https://doi.org/10.4108/icst.pervasivehealth.2013.252359

  84. Pedroli E et al (2018) Characteristics, usability, and users experience of a system combining cognitive and physical therapy in a virtual environment: positive bike. Sensors (Basel) 18:2343. https://doi.org/10.3390/s18072343

    Article  Google Scholar 

  85. Piper B, Mueller ST, Talebzadeh S, Ki MJ (2016) Evaluation of the validity of the psychology experiment building language tests of vigilance, auditory memory, and decision making. PeerJ 4. https://doi.org/10.7717/peerj.1772

    Article  Google Scholar 

  86. Qiu X et al (2011) Impairment in processing visual information at the pre-attentive stage in patients with a major depressive disorder: a visual mismatch negativity study. Neurosci Lett 491:53–57. https://doi.org/10.1016/j.neulet.2011.01.006

    Article  Google Scholar 

  87. Reitan RM (2016) Validity of the trail making test as an indicator of organic brain damage. Percept Mot Skills 8:271–276. https://doi.org/10.2466/pms.1958.8.3.271

    Article  Google Scholar 

  88. Remes O, Brayne C, van der Linde R, Lafortune L (2016) A systematic review of reviews on the prevalence of anxiety disorders in adult populations. Brain Behav 6:e00497. https://doi.org/10.1002/brb3.497

    Article  Google Scholar 

  89. Rizzo AA, Bowerly T, Buckwalter JG, Klimchuk D, Mitura R, Parsons TD (2006) A virtual reality scenario for all seasons: the virtual classroom. CNS Spectr 11:35–44. https://doi.org/10.1017/s1092852900024196

    Article  Google Scholar 

  90. Rizzo AA, Buckwalter JG, Bowerly T, Van Der Zaag C, Humphrey L, Neumann U, Chua C, Kyriakakis C, Van Rooyen A, Sisemore D (2000) The virtual classroom: a virtual reality environment for the assessment and rehabilitation of attention deficits. Cyberpsychol Behav 3:483–499. https://doi.org/10.1089/10949310050078940

  91. Rizzo AS (2019) Clinical virtual reality in mental health and rehabilitation: a brief review of the future! In: Infrared technology and applications XLV. Anaheim, pp 1–9. https://doi.org/10.1117/12.2524302

  92. Rizzo AS, Kim GJ (2005) A SWOT analysis of the field of virtual reality rehabilitation and therapy. Presence Teleop Virt 14:119–146. https://doi.org/10.1162/1054746053967094

    Article  Google Scholar 

  93. Rizzo AS, Koenig ST (2017) Is clinical virtual reality ready for primetime? Neuropsychology 31:877–899. https://doi.org/10.1037/neu0000405

    Article  Google Scholar 

  94. Robillard G, Bouchard S, Fournier T, Renaud P (2003) Anxiety and presence during VR immersion: a comparative study of the reactions of phobic and non-phobic participants in therapeutic virtual environments derived from computer games. Cyberpsychol Behav 6:467–476. https://doi.org/10.1089/109493103769710497

    Article  Google Scholar 

  95. Rock PL, Roiser JP, Riedel WJ, Blackwell AD (2014) Cognitive impairment in depression: a systematic review and meta-analysis. Psychol Med 44:2029–2040. https://doi.org/10.1017/S0033291713002535

    Article  Google Scholar 

  96. Rodriguez C, Areces D, Garcia T, Cueli M, Gonzalez-Castro P (2018) Comparison between two continuous performance tests for identifying ADHD: traditional vs. virtual reality. Int J Clin Health Psychol 18:254–263. https://doi.org/10.1016/j.ijchp.2018.06.003

    Article  Google Scholar 

  97. Rose T, Nam CS, Chen KB (2018) Immersion of virtual reality for rehabilitation. Review Appl Ergon 69:153–161. https://doi.org/10.1016/j.apergo.2018.01.009

    Article  Google Scholar 

  98. Rosenblat JD, Kakar R, McIntyre RS (2015) The cognitive effects of antidepressants in major depressive disorder: a systematic review and meta-analysis of randomized clinical trials. Int J Neuropsychopharmacol. https://doi.org/10.1093/ijnp/pyv082

    Article  Google Scholar 

  99. Rosvold HE, Mirsky AF, Sarason I, Bransome Jr ED, Beck LH (1956) A continuous performance test of brain damage. J Consult Psychol 20:343–350. https://doi.org/10.1037/h0043220

  100. Sauro J, Lewis JR (2016) Quantifying the user experience: practical statistics for user research. Morgan Kaufmann, Cambridge

    Google Scholar 

  101. Schirmer A, Escoffier N (2010) Emotional MMN: anxiety and heart rate correlate with the ERP signature for auditory change detection. Clin Neurophysiol 121:53–59. https://doi.org/10.1016/j.clinph.2009.09.029

    Article  Google Scholar 

  102. Schulthesis MT, Doiron M (2017) Technologies for functionally relevant neuropsychological assessment. In: Kane RL, Parsons TD (eds) The role of technology in clinical neuropsychology. Oxford University Press, Oxford, pp 47–59

    Google Scholar 

  103. Servera M, Cardo E (2006) Children sustained attention task (CSAT): Normative, reliability, and validity data. Int J Clin Health Psychol 6:697–707.

  104. Slater M, Linakis V, Usoh M, Kooper R (1996) Immersion, presence and performance in virtual environments: An experiment with tri-dimensional chess. In: Proceedings of the ACM symposium on virtual reality software and technology. Association for Computing Machinery, pp 163–172. https://doi.org/10.1145/3304181.3304216

  105. Slater M, Wilbur S (1997) A framework for immersive virtual environments (FIVE): Speculations on the role of presence in virtual environments. Presence (Camb) 6:603–616. https://doi.org/10.1162/pres.1997.6.6.603

    Article  Google Scholar 

  106. Snyder HR (2013) Major depressive disorder is associated with broad impairments on neuropsychological measures of executive function: a meta-analysis and review. Psychol Bull 139:81–132. https://doi.org/10.1037/a0028727

    Article  Google Scholar 

  107. Spielberger CD (1983) Manual for the state-trait anxiety inventory (form Y). Consulting Psychologists Press, Palo Alto

    Google Scholar 

  108. Spielberger CD, Pitariu H, Pleasa C (2007) STAI—Y. State-trait anxiety inventory Y form. Sinapsis, Cluj-Napoca

    Google Scholar 

  109. Stein RA, Strickland TL (1998) A review of the neuropsychological effects of commonly used prescription medications. Arch Clin Neuropsychol 13:259–284. https://doi.org/10.1093/arclin/13.3.259

    Article  Google Scholar 

  110. Strauss E, Sherman EMS, Spreen O (2006) A compendium of neuropsychological tests. Oxford University Press, New York

    Google Scholar 

  111. Stuss DT, Benson DF (1984) Neuropsychological studies of the frontal lobes. Psychol Bull 95:3–28. https://doi.org/10.1037/0033-2909.95.1.3

    Article  Google Scholar 

  112. Stuss DT, Stethem LL, Hugenholtz H, Picton T, Pivik J, Richard MT (1989) Reaction time after head injury: fatigue, divided and focused attention, and consistency of performance. J Neurol Neurosurg Psychiatry 52:742–748. https://doi.org/10.1136/jnnp.52.6.742

    Article  Google Scholar 

  113. Sun HM, Li SP, Zhu YQ, Hsiao B (2015) The effect of user’s perceived presence and promotion focus on usability for interacting in virtual environments. Appl Ergon 50:126–132. https://doi.org/10.1016/j.apergo.2015.03.006

    Article  Google Scholar 

  114. Takei Y et al (2009) Preattentive dysfunction in major depression: a magnetoencephalography study using auditory mismatch negativity. Psychophysiology 46:52–61. https://doi.org/10.1111/j.1469-8986.2008.00748.x

    Article  Google Scholar 

  115. Tiller JW (2013) Depression and anxiety. Med J Aust 199:S28–S31. https://doi.org/10.5694/mja12.10628

    Article  Google Scholar 

  116. Tinius T (2003) The integrated visual and auditory continuous performance test as a neuropsychological measure. Arch Clin Neuropsychol 18:439–454. https://doi.org/10.1016/s0887-6177(02)00144-0

    Article  Google Scholar 

  117. Tyrer P (2001) The case for cothymia: mixed anxiety and depression as a single diagnosis. Br J Psychiatry 179:191–193. https://doi.org/10.1192/bjp.179.3.191

    Article  Google Scholar 

  118. Urbina S (2004) Essentials of psychological testing. Wiley, New Jersey

    Google Scholar 

  119. Vilgis V, Silk TJ, Vance A (2015) Executive function and attention in children and adolescents with depressive disorders: a systematic review. Eur Child Adolesc Psychiatry 24:365–384. https://doi.org/10.1007/s00787-015-0675-7

    Article  Google Scholar 

  120. Voinescu A, Fodor L-A, Stanton Fraser D, Mejías M, David D (2019) Exploring the usability of nesplora aquarium, a virtual reality system for neuropsychological assessment of attention and executive functioning. In: IEEE conference on virtual reality and 3D user interfaces (VR). IEEE, pp 1207–1208. https://doi.org/10.1109/VR.2019.8798191

  121. Wagner S, Doering B, Helmreich I, Lieb K, Tadic A (2012) A meta-analysis of executive dysfunctions in unipolar major depressive disorder without psychotic symptoms and their changes during antidepressant treatment. Acta Psychiatr Scand 125:281–292. https://doi.org/10.1111/j.1600-0447.2011.01762.x

    Article  Google Scholar 

  122. Weiner IB, Graham JR, Naglieri JA (2013) Handbook of psychology, assessment psychology. Wiley, Hoboken

    Google Scholar 

  123. Weinstein AM (1995) Visual ERPs evidence for enhanced processing of threatening information in anxious university students. Biol Psychiatry 37:847–858. https://doi.org/10.1016/0006-3223(94)00249-3

    Article  Google Scholar 

  124. Witmer BG, Singer MJ (1998) Measuring presence in virtual environments: a presence questionnaire. Presence 7:225–240. https://doi.org/10.1162/105474698565686

    Article  Google Scholar 

  125. WHO (2017) Depression and other common mental disorders: global health estimates. World Health Organization, Geneva

    Google Scholar 

  126. Wingo AP, Wingo TS, Harvey PD, Baldessarini RJ (2009) Effects of lithium on cognitive performance: a meta-analysis. J Clin Psychiatry 70:1588–159. https://doi.org/10.4088/JCP.08r04972

    Article  Google Scholar 

  127. Yang X et al (2019) Dysfunction of pre-attentive visual information processing in drug-naive women, but not men, during the initial episode of major depressive disorder. Front Psychiatry 10:899. https://doi.org/10.3389/fpsyt.2019.00899

    Article  Google Scholar 

  128. Zweerings J et al (2019) Fronto-parietal and temporal brain dysfunction in depression: a fMRI investigation of auditory mismatch processing. Hum Brain Mapp 40:3657–3668. https://doi.org/10.1002/hbm.24623

    Article  Google Scholar 

Download references

Acknowledgements

We are thankful to our collaborators at Nesplora—Neuroscience Support Systems, Spain, for designing, developing and programming the Nesplora Aquarium. We thank Dr. Irina Dulea for her valuable assistance with participants recruitment.

Funding

The equipment used for the study (Nesplora Aquarium VR system) was funded by the European Commission under Horizon 2020 Program (Grant 733901, from Project VRMIND—Virtual Reality Based Evaluation of Mental Disorders). The funding source had no involvement in the research and preparation of the article.

Author information

Affiliations

Authors

Corresponding author

Correspondence to Alexandra Voinescu.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Below is the link to the electronic supplementary material.

Supplementary file1 (DOCX 54 kb)

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Voinescu, A., Petrini, K., Stanton Fraser, D. et al. The effectiveness of a virtual reality attention task to predict depression and anxiety in comparison with current clinical measures. Virtual Reality (2021). https://doi.org/10.1007/s10055-021-00520-7

Download citation

Keywords

  • Attention
  • Response inhibition
  • Depression
  • Anxiety
  • Cognitive ability
  • Virtual reality