1 Introduction

Depression and anxiety disorders are a global problem with an estimated worldwide prevalence of 8%. Roughly, 322 million people are currently living with depression and 264 million with anxiety disorders (WHO 2017) with estimated worldwide costs of US$925 billion and 12 billion days of lost productivity per year caused by these conditions (Chisholm et al. 2016). Depression and anxiety are a major contributor to global disability, and both are predicted to affect an increasing number of individuals in the years to come (WHO 2017).

A wealth of evidence associates depression and anxiety with impaired attention (Eysenck et al. 2007; Hallion et al. 2018; Liu et al. 2020; Pacheco-Unguetti et al. 2011; Rock et al. 2014; Snyder 2013). Vigilance or sustained attention is the ability to maintain an attentional ability and be alert to stimuli over prolonged periods of time (Lezak et al. 2004). Poor inhibition indicating poor capacity to deliberately inhibit irrelevant responses (Miyake et al. 2000), a domain of executive functioning, is linked with rapid response impulsivity, an inability to adjust responses and behaviours to the situational demands (Asahi et al. 2004; Evenden 1998). To date, cognitive deficits, including attention and executive functioning, have been measured using neuropsychological tests such as Continuous Performance Tests or Stroop Test. The aim of neuropsychological assessment is to draw inferences about structural and functional characteristics of the brain by assessing individual’s observable behaviours in defined stimulus–response situations such as participant responses and subsequent performance. Results of neuropsychological assessment are used to identify the impact of the disease on various cognitive functioning domains and to make predictions regarding a person’s level of functioning in everyday life (Benton 1994; Lezak et al. 2004). Because most of these tests were developed in the early twentieth century, some argue that their principles are outdated (Chaytor and Schmitter-Edgecombe 2003; Kessels 2019). Usually, these tests ask participants to respond to various symbols or stimuli such as letter “x” following predefined rules while ignoring irrelevant stimuli such as letters “a”, “b” “c”. Nowadays, stimuli are delivered in a systematic way on computer screens thus limiting human error (and increasing accuracy) when recording participant responses. However, it has been argued that attending and responding to a set of stimuli on a computer screen lack the complexity of real environments (Parsons and Rizzo 2019), thus limiting the effectiveness of the neuropsychological assessments in predicting real-life performance (Chaytor and Schmitter-Edgecombe 2003; Kessels 2019).

A useful example to understand the limitation of such traditional computer-based tests is to consider how children may respond in a typical one-on-one assessment context and in a real classroom setting with distracting stimuli such as other children whispering or noise coming from outside. A promising alternative to increase the ecological validity of these tests, which has gained support in the last 20 years, is the use of virtual reality (VR) (Parsons and Rizzo 2019). Referring back to the previous example, the VR “solution” for assessing attention deficits among children in ecological environments is to create a VR environment, for example, a VR classroom that replicates a real classroom environment, while delivering classic cognitive tasks and measuring attention performance (Iriarte et al. 2016; Rizzo et al. 2006).

In the last few decades, VR has thus been increasingly used in clinical psychology, including neuropsychology, because of its ability to create experiences that are very similar to those in real life and the possibility to maintain control over stimuli and therapeutic strategies that would otherwise not be possible in real life settings (Negut et al. 2016; Rizzo 2019; Rizzo and Koenig 2017). For example, VR platforms allow a participant to be immersed in a virtual environment (VE) and to systematically deliver visual and auditory stimuli in the presence of various distractors (such as avatars of other humans and/or noise from the environment) thus increasing the realism of the assessment context. An increasing number of VEs like virtual classrooms, virtual shopping centres or cities have been developed to measure cognitive functions such as attention, executive functioning and memory across various populations including patients with brain injury or individuals with neurodevelopmental disorders such as children with ADHD (Negut et al. 2016; Rizzo 2019). Results are encouraging as they have shown that VR tests are valid instruments (Gilboa et al. 2018; Negut et al. 2016; Parsons et al. 2019). However, more studies are needed to identify whether VR is more effective than classical tests in identifying cognitive impairment as a meta-analysis has shown that VR tests have similar sensitivity to traditional computerised or paper-and-pencil tests in detecting cognitive deficits (Negut et al. 2016).

However, to our knowledge, no study so far has used VR as a tool to investigate the neurocognitive profile of individuals with elevated symptoms of depression and anxiety. Using VR to assess cognitive processes of individuals with depression and anxiety might help overcome issues related to lack of motivation or increased fatigue during testing and consequently increase adherence. For example, some of the poor results on cognitive functioning obtained by people with depression can be explained by reduced motivation and fatigue during prolonged periods of testing time (Marazziti et al. 2010; Vilgis et al. 2015). VR can be more engaging and can increase motivation because it has features similar to computer games (Rizzo and Kim 2005). Because VR can replicate real-life scenarios, this can increase validity over computerised tests, engaging and motivating responders. That is, if the test takers perceive that the content of the test is appropriate and relevant for the purpose of the assessment, they are more willing to cooperate (Urbina 2004). VR also has good-to-excellent usability ratings given by healthy participants (Voinescu et al. 2019), which may facilitate its adoption (Mousavi et al. 2013). Product usability is the “extent to which a system, product or service can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use” (ISO 9241-11:2018) and is strongly connected to the usefulness of a product (Nielsen and Mack 1994). Usability of VR systems is linked with acceptance and adherence rates to VR-based interventions (Laver et al. 2012; Morán et al. 2015). Acceptance and adherence rates were mostly measured using self-report scales that cover dimensions such as enjoyment, confidence and self-esteem (Rose et al. 2018). Results suggest that VR has the potential of increasing adherence (Rose et al. 2018); however, a study that measured adherence as number of days spent in therapy did not find significant differences between the VR group and standard therapy (Parry et al. 2015). While the usability of VR systems has not been tested yet with individuals with depression and anxiety, other research with vulnerable populations such as older adults and patients with Parkinson disease has shown that VR has good ratings of usability (Corno et al. 2014; Pedroli et al. 2013, 2018). Simulator sickness and presence are variables in VR that are linked with usability and performance. Simulator sickness refers to symptoms that occur during exposure in VR and cause discomfort such as fatigue, headache, eye strain, nausea, dizziness (Kennedy et al. 1993). The sensory-conflict theory offers an explanation of simulator sickness symptoms, which can be caused by a mismatch in the visual, vestibular and proprioception sensory channels (Cobb et al. 1999). Simulator sickness symptoms are usually mild and transient (Lavoie et al. 2020), but it is believed that side effects can be more problematic for vulnerable populations such as older adults or people with neurological conditions (Brooks et al. 2010; Kim et al. 2017). Presence in VR refers to the subjective feeling of being present in the environment (Slater and Wilbur 1997; Witmer and Singer 1998) and can increase user experience (Sun et al. 2015) and facilitate the transfer of skills from VR to real life settings (Slater et al. 1996). However, a recent systematic review concluded that there are inconsistent results for the relationship between presence and performance on VR rehabilitation interventions (Rose et al. 2018). Identifying whether there are individual differences between people with elevated symptoms of depression and anxiety and healthy people on cyberpsychology variables such as self-report usability, simulator sickness and presence would increase our understanding of how to design better VR applications for clinical purposes.

To benefit from potential advantages of VR assessment described above, in the current study we aimed to explore the attention and inhibition profile among participants with low and elevated symptoms of depression and anxiety using a new VR tool (Parsons et al. 2017; Rizzo and Koenig 2017; Schulthesis and Doiron 2017). Nesplora Aquarium is the first VR test designed to measure attention and executive functions in an adult population (Climent et al. 2019) and is based on the Continuous performance test paradigm (CPT) which is one of the most widely used measures of attention impairment in depression (Coles et al. 2009).

The original CPT paradigm was first developed by Rosvold et al. (1956), and today CPTs are widely used as a reliable measure of sustained vigilance, attention and inhibition (Gualtieri and Johnson 2005; Losier et al. 1996). Two forms of CPT tasks are currently used (Servera and Cardo 2006): Vigilance tasks known as X- or AX-types require the participant to respond to target stimuli and to ignore non-target stimuli (Rosvold et al. 1956). Inhibition tasks or non-X tasks ask the participant to respond to the non-target stimuli and to ignore target stimuli (Conners et al. 2003; Servera and Cardo 2006). Outcomes of CPTs are reaction time (RT) for correct targets, variability of RT, number of omission errors (missed targets) and commission errors (incorrect hits). To date, CPTs have been integrated in VR using a virtual classroom where the participant is sitting at a desk and stimuli consisting of alphabet letters are delivered on a blackboard (Iriarte et al. 2016; Rizzo et al. 2000). Studies using this VR system revealed it is effective in assessing attention deficits such as ADHD among children (Gilboa et al. 2018; Negut et al. 2016, 2017; Parsons et al. 2019).

In Nesplora Aquarium, CPTs have been integrated in VR by using an aquarium experience. The aquarium contains multiple tanks, but the participant is instructed to pay attention to the main tank which is positioned in front of them. Following a CPT rule, the participant has to pay attention and respond to correct stimuli over a prolonged period of time (18 min). For example, they have to press a button whenever they see or hear the name of a certain fish, for example, the clown fish. During the test, to increase the realism and ecological validity of the task, various contextual distractors are introduced such as people walking in front of the aquarium, a baby crying. An important and novel feature of Nesplora Aquarium is that it contains both visual and auditory stimuli. Because of this, the test can also examine the attentional sensory profile of individuals with depression and anxiety by assessing attention performance obtained on both visual and auditory stimuli to identify any deficits. Currently, the nature of attention deficits in depression and anxiety, especially in relation to sensory processing, is still lacking. Most research in anxiety and depression has focused on early pre-attentive and automatic visual and auditory processing and showed impaired deficits in early auditory and visual processing in depressed and anxious samples (Chang et al. 2011; Kahkonen et al. 2007; Qiu et al. 2011; Schirmer and Escoffier 2010; Takei et al. 2009; Weinstein 1995; Yang et al. 2019; Zweerings et al. 2019). However, sensory processing as modulated by attention has not been studied in depression and anxiety, which we plan to address in the current study by exploring attention performance on auditory and visual stimuli of individuals with elevated symptoms of depression and anxiety in comparison with individuals with low symptoms.

Furthermore, to establish whether Nesplora Aquarium for attention and inhibition assessment is a valid and effective measure over and above traditional tests, we focused on patterns of convergence and divergence, as main sources of test validity. Evidence of convergent validity of a test is usually established if two tests that are supposed to measure the same psychological construct show strong positive correlations (Urbina 2004). Assessing convergent validity of VR measures is a common practice. For example, Parsons and Barnett (2017) using a sample of younger and older adults computed Pearson’s r coefficients between a VR measure of memory (Virtual Environment Grocery Store, VEGS) and a traditional neuropsychological measure of learning and memory (California Verbal Learning Test). Because the two measures were highly correlated (correlations ranging from 0.42 to 0.56), the authors concluded that the VR measure showed convergent validity. In opposition, evidence for divergent validity is obtained based on low correlations between two tests that are supposed to measure different constructs (Urbina 2004). Similarly, in the same study, Parsons and Barnett (2017) computed Pearson’s r coefficients between the VR measure of memory (Virtual Environment Grocery Store, VEGS) and a traditional neuropsychological measure of executive functioning (Delis Kaplan Executive Functioning System, DKEFS). Because VEGS did not correlate with theoretically unrelated constructs such as executive functioning (correlations ranging from 0.07 to 0.28), the authors concluded that the VR test had adequate levels of divergent validity.

Individuals with depression and anxiety show impaired cognition on multiple attention like reaction time, rapid visual information processing and executive functioning domains: inhibition, shifting, verbal working memory, visuospatial working memory, planning, verbal fluency, processing speed) (Eysenck et al. 2007; Hallion et al. 2018; Liu et al. 2020; Rock et al. 2014; Snyder 2013). Hence, in the current study we investigated if attention and inhibition measured in VR with Nesplora Aquarium can effectively detect symptoms of depression and anxiety above and beyond classical measures. To assess the proposed incremental validity of Nesplora Aquarium over classical measures in predicting current symptoms of depression and anxiety, we selected several cognitive ability domains known to be affected in depression and anxiety: attention and inhibition, psychomotor speed and executive functioning, selective attention and inhibition, cognitive impairment, spatial working memory span which were measured with classical tests such as paper-and-pencil or computerised measures. We then assessed the ability of Nesplora Aquarium to predict current symptoms of depression and anxiety beyond cognitive domains measured by classical tests.

In summary, the scope of the current study is to explore the effectiveness of a virtual reality attention task to predict depression and anxiety in comparison with current clinical measures. The main objectives of the current study were to (1) investigate the validity of Nesplora Aquarium as a measure of attention and inhibition. We expected strong positive correlations between outcomes of Nesplora Aquarium and outcomes of the computerised version of CPT for measure of attention and inhibition; (2) examine whether attention measured with Nesplora Aquarium can predict current symptoms of depression and anxiety above and beyond cognitive domains measured with classical tests; we anticipated that outcomes of Nesplora Aquarium will predict current symptoms of depression and anxiety above and beyond cognitive domains measured with classical tests (3) to identify if Nesplora Aquarium can discriminate between individuals with low and elevated symptoms of depression and anxiety and to explore patterns of attention deficits, and their relation to specific sensory processing (visual and auditory). We predicted that participants with elevated symptoms of depression and anxiety will have poorer attention performance and inhibition (also in relation to specific visual and auditory sensory processing) as measured with Nesplora Aquarium; and (4) to assess the usability of Nesplora Aquarium as well as possible adverse effects such as motion sickness and the level of presence in VR. We expected no major simulator sickness symptoms, and good-to-excellent self-report usability and adequate level of presence in VR irrespective of the level of anxiety and depression.

2 Methods

2.1 Design

We used a cross-sectional design as data were collected at one single point in time. The cross-sectional design was mixed: 2 × 2 × 2. The first factor was a between-subjects factor (the groups of participants: people with low and elevated symptoms of depression and anxiety). The second factor was a within-subjects factor (type of task: VR vs. non-VR classical task). The third factor was a within-subjects factor (sensory modality: visual vs. auditory).

2.2 Participants

Participants were recruited via multiple sources: University Student Participant Pool in exchange for study credit; social media channels; general medical practices; and Psychiatric facilities in Romania. A total of 91 participants enrolled in the study and 82 participants aged between 19 and 61 years (M = 32.40, SD = 9.71) were included in the analysis. Four participants were excluded from the sample due to data loss (technical errors related to poor wi-fi connection during VR Aquarium testing) and another five based on their mild to moderate scores on the clinical scales (as detailed below). Sixty-one percent were females (N = 53) with a mean education year of 16 years. Sixty-two percent (N = 54) were employed and 40% reported previous VR experience (N = 35) (see Supplementary Materials, Table A.1 for participants’ demographic characteristics, Table A.2–3. for a full description of clinical and neuropsychological measures). We excluded participants who: (a) were younger than 18 years; (b) had a history of neurological diseases (including head injury or epilepsy), psychosis or substance dependence, (c) had uncorrected hearing and visual impairments or severe motion sickness.

Based on the severity of the mood symptoms, we divided our sample into healthy participants and participants with elevated symptoms of depression and anxiety, by using the following criteria: (a) having a clinical diagnosis of anxiety or depressive disorders and/or (b) having higher scores of depression and/or anxiety as measured with well-established psychological scales, as described below. Twenty-four participants had a clinical diagnosis as they met the ICD 10 criteria for: Major Depressive Disorder, single episode (N = 8), Recurrent Depressive Disorder (N = 6) Generalised Anxiety Disorder (N = 4), Mixed Anxiety and Depressive Disorder (N = 2), Panic Disorder (N = 3) and Social Phobia (N = 1). Seventeen participants reported increased depression and anxiety but did not meet the diagnostic criteria for a depressive or an anxiety disorder. Participants were screened for symptoms of depression and anxiety using the Beck Depression Inventory-II (BDI-II, Beck et al. 1996) and State-Trait Anxiety Inventory (Form Y) (STAI, Spielberger 1983). The cut-off used for depressive symptoms was BDI total score ≥ 20, which is equivalent to moderate to severe depression according to the normative data (Beck et al. 2012). For anxiety, a moderate to severe level of state anxiety (STAI-S) ≥ 34 was used as cut-off according to data from the normative study (Spielberger et al. 2007). Twenty-two participants were taking antidepressant, anxiolytic, hypnotic and/or antipsychotic medication at the time of the study. Overall, there were no significant differences between participants who were on medication and those who were not on most cognitive outcomes, except on omission errors measured with classic CPT. The medicated participants displayed significantly increased symptoms of depression and anxiety than non-medicated participants (Supplementary Materials, Table A.4. displays means, SDs, t tests values and alpha significance level).

Forty-one healthy participants were selected and had a score on BDI ≤ 13 which is equivalent to minimal/low depression according to the normative data (Spielberger et al. 2007) and a low level of state anxiety (STAI-S) ≤ 30 for both males and females according to data from normative study (Beck et al. 2012). We excluded five participants who displayed mild to moderate symptoms of depression and anxiety (BDI total scores between 14 and 19 (Beck et al. 2012) or state anxiety (STAI-S) scores between 31 and 33 (Spielberger et al. 2007). Supplementary Table A.1. displays information about participants demographics.

2.3 Materials

2.3.1 VR test for attention assessment

The VR test used in this study was Nesplora Aquarium (Climent et al. 2019), the first VR-based test developed to measure attention and executive functioning in adults over 16 years. It consists of non-X tasks that are administered in a virtual aquarium (see Fig. 1). The VR system uses a Samsung Galaxy S7 smartphone, paired with Samsung Gear VR headset. The test is monitored by the experimenter, using a laptop computer (ASUS ROG, Intel i7 processor, 8 Gb RAM, GeForce GTX 960 M video card). Both the laptop computer and the VR headset are connected using a local wireless connection.

Fig. 1
figure 1

Screenshot of the Nesplora Aquarium

Full size image

During the test, the participant is virtually positioned in front of a VR aquarium and must pay attention to the main fish tank where they have to respond to the stimuli. The stimuli are different species of fish such as a clown fish delivered via the visual channel using images of different species of fish that are passing through two rocks in the main fish tank or via the auditory channel using the names of the same species of fish as in the visual condition presented via speakers (see Fig. 1). In line with the CPT paradigm, the participant has to follow a rule in responding to stimuli; for example, each time they see any other fish than the “clown” or they hear any other fish except the “surgeon”. The task was to attend to either the visual or to the auditory modality and to respond to visual or auditory targets. Visual and auditory stimuli were distributed randomly during the task. Participants responses were collected for visual and auditory stimuli separately. Different contextual distractors were delivered during the task, either visually such as people walking in front of the aquarium, other animals present in the aquarium such as turtles, lights flickering or auditorily such as an invitation to coffee delivered over the PA system, a baby crying, a warning to not use the flash when taking photographs. The full description of Nesplora Aquarium is available in the Online Resources.

Participants’ responses were recorded separately for each sensory modality: visual and auditory stimuli. Besides performance on each modality, the system also computes for each outcome an average score by combining visual and auditory performance. Nesplora Aquarium outcomes were used as measures of sustained attention and inhibition, more specifically: mean reaction time (RT) for correct responses in milliseconds, standard deviation (SD) of RT for correct responses, total number of commissions, RT for commission errors, omission errors and perseverance errors. All the measures for sustained attention and inhibition as calculated with Nesplora were obtained separately for auditory and visual modalities and for the two modalities combined. Commission errors result from incorrect responses to non-target stimuli, while omission errors result from not responding to targets. Increased number of commission errors and omission errors, as well as faster RT for correct answers and increased SD of RT, reflects poor attention performance. More specifically, slow RT is an indicator of slow processing speed of the target stimulus while variability of RT suggests changes in sustained attention or fatigability during the task with lower scores reflecting fluctuation of attention. Omissions are linked to inattention and low arousal and commissions to impulsivity or a deficit in inhibitory control (Climent et al. 2019; Negut et al. 2017; Tinius 2003). RT to commission errors is proposed as an additional indicator of impulsivity and inhibitory control with faster RT indicating increased impulsivity (Climent et al. 2019; Halperin et al. 1991). Perseverance errors can indicate cognitive inflexibility, in a sense that they measure participants in/ability to change their responses following changes in the task at hand, i.e. if a participant uses old rules from the task rather than switching successfully to new rules, then this is an indicator of poor interference control and cognitive inflexibility (Climent et al. 2019; Greve et al. 2005). More perseverance errors reflect poorer interference control and cognitive flexibility (Climent et al. 2019). For all Nesplora outcomes, as no standardised data and clear cut-off scores for attention impairment among clinical samples were available in the literature, we transformed raw scores to t scores using the formula: T = (Z × 10) + 50 and we used these transformed data in the analysis (Weiner et al. 2013).

2.3.2 Measures

Participants’ demographic characteristics collected included age, gender, highest education degree, work status, previous VR experience, medical history and medication taken, and whether they experience any severe motion sickness. We screened for symptoms of depression using the Beck Depression Inventory-II, BDI-II (Beck et al. 1996). Symptoms of anxiety were assessed using the State and Trait Anxiety Measured with State-Trait Anxiety Inventory, STAI (Form Y) (Spielberger 1983).

Besides completing the Nesplora Aquarium task, participants were asked to complete the following paper-and-pencil and computerised measures of cognitive abilities known to be affected in cases of depression and anxiety: sustained attention and inhibition (Continuous Performance Test, CPT, Mueller and Piper 2014), selective attention and cognitive flexibility (Stroop Test, Bayard et al. 2011), spatial working memory span (Corsi Block Test, Corsi 1972), psychomotor speed and executive functioning (Trail Making Test, Reitan 2016), and cognitive impairment (Montreal Cognitive Assessment, MoCA test, Nasreddine et al. 2005). Self-reported system usability was measured with the System Usability Scale (SUS, Brooke 1986). The level of simulator sickness was measured using the Simulator Sickness Questionnaire (SSQ, Kennedy et al. 1993). Self-report presence in VR was measured using the Presence Questionnaire (Witmer and Singer 1998). Further description and psychometric properties of these measures can be found in the in the Supplementary Materials; Table A.2–3 contains a full description of clinical and neuropsychological measures.

2.4 Procedure

The study was approved by the University Research Ethics committee (REF 6667/25.04.2018) at Babes-Bolyai University, Romania where the data were collected and from University of Bath, UK where the data was analysed (PREC 18-305). Data collection took place between April 2018 and November 2019. The study was pre-registered on the Open Science Framework (https://osf.io/nrhb5). All participants gave informed consent after reading the information sheet. The screening and demographic questionnaire was completed at the beginning of the study. Participants who did not meet the inclusion criteria did not take part in the study. Participants in the study completed the paper-and-pencil version of BDI-II (Beck et al. 1996, 2012), the S-STAI (Spielberger 1983; Spielberger et al. 2007) and pre-test SSQ (Kennedy et al. 1993) followed by the Nesplora Aquarium VR test. During the VR test, the experimenter showed the participant the VR headset and explained how it worked and stressed that in the case of any motion/simulator sickness symptoms they should alert the experimenter and remove the VR headset. The experimenter helped the participant to adjust the VR headset to make sure that the VR headset was comfortable and that the image was focused. Then, the practice session began which consisted of the same VR aquarium environment. During this session, the participant was instructed first to explore the VR environment and to listen to instructions and to start the task when they were ready. The practice session lasted for approximately 5 min, depending on the needs of the participant to familiarise with the environment and controllers. Then, the main testing session started. During this, the participant had to sit down on a chair (the task did not involve active navigation in the VE and was exclusively passive) and pay attention and respond to the visual or auditory stimuli. Visual and auditory stimuli were distributed randomly. The task lasted for approximately 15 min. At the end of the VR task, they had to fill in the paper-and-pencil post-test questionnaires and measures: post-test SSQ, PQ, SUS (Brooke 1986) followed by the computerised versions of the CPT (Conners et al. 2003; Piper et al. 2016), Victoria Stroop (Strauss et al. 2006), Corsi Block Test (Corsi 1972) and the paper-and-pencil TMT A&B (Reitan 2016) and MoCA (Nasreddine et al. 2005). Written instructions were provided at the start of each test, and additional information concerning test instructions was provided by the experimenter if required. At the end of the session participants were debriefed. The testing time took approximately 90 min for each participant.

3 Results

As some of the variables were not normally distributed like RT which usually produces skewed distributions (Miller 1991) for our statistical analysis, we used the bootstrapping procedure with 95% bias-corrected bootstrap confidence intervals (CIs) with 10,000 resamples to obtain more robust CIs for the difference and probability values for regression model estimates (Efron and Tibshirani 1993; Fox 2015; Mallinckrodt et al. 2006). All data analyses were conducted using SPSS version 24.

3.1 Convergent and divergent validity of VR CPT Nesplora Aquarium

To investigate our first objective that aimed to establish the convergent and divergent validity of Nesplora Aquarium, we first analysed correlations between VR CPT and computerised CPT. We used Pearson’s r to assess correlations between variables which were continuous. In addition, we also calculated Spearman’s rho correlation estimates for robustness against non-normal distributions. Both tests assess vigilance, sustained attention and inhibition and the measures are comparable for the following outcomes: RT, SD RT, omission and commission errors. All correlations between these variables were significant. As expected, the correlations were positive and high for RT, omission and commission errors, except for SD RT, for which the correlation was moderate/acceptable (Drummond and Jones 2016). Table 1 shows the correlations between the variables. We expected weak to moderate correlations between VR CPT Nesplora Aquarium outcomes with classic neuropsychological measures that measure other cognitive domains such as psychomotor speed and executive functioning (TMT A&B), spatial working memory span (Corsi Blocks test), cognitive impairment (MoCA), and selective attention and cognitive flexibility (Stroop). In line with our predictions, most correlations were weak and nonsignificant with the exception of VR CPT omission errors and TMT A& B, Corsi and MoCA with moderate to high correlation coefficients (see Online Resources, Table A.5. for intercorrelations for divergent validity).

Table 1 Intercorrelations between scores from VR CPT Nesplora Aquarium and computerised CPT
Full size table

3.2 Assessing covariates

Because cognitive deficits in depression and anxiety extend to multiple domains, we performed two separate hierarchical regressions to identify which of the variables that are known to be affected in depression and anxiety such as spatial working memory or processing speed are significant predictors of current symptoms of depression and anxiety. Significant predictors were included in further regression analyses as covariates to identify whether Nesplora Aquarium can predict current symptoms of depression and anxiety specifically for attention and inhibition performances. Several variables were identified as predicting depressive symptoms measured with BDI-II: cognitive impairment measured with MoCA (B = − 1.37, p = 0.005, BCa CI = [− 2.28, − 0.55]); executive functioning and psychomotor processing speed measured with TMT-A (B = 0.36, p = 0.048, BCa CI = [0.04, 0.62]); spatial working memory span measured with Corsi Block Test, total score (B = − 0.34*, p = 0.018, BCa CI = [0.62, − 0.05]). For state anxiety measured with STAI-S, cognitive impairment measured with MoCA (B = − 1.35, p = 0.002, BCa CI = [− 2.17, − 0.50]); TMT-B (B = 0.11, p = 0.054, BCa CI = [0.03, 0.23]); sustained attention measured with CPT, RT for total correct responses (B = 0.47, p = 0.018, BCa CI = [0.09, 0.77]); spatial working memory span measured with Corsi Block Test, total score (B = − 0.47, p = 0.000, BCa CI = [− 0.68, − 0.27]) (see Table 2).

Table 2 Summary of linear regression for predicting symptoms of depression and anxiety for various neuropsychological measures
Full size table

3.3 Effectiveness of Nesplora Aquarium in predicting symptoms of depression and anxiety

To identify whether attention measured with the Nesplora Aquarium can significantly predict symptoms of depression and anxiety, we first conducted a standard linear regression. Predictor variables were outcomes of the Nesplora Aquarium: RT for total correct responses, omission errors, commission errors and variability (SD) of RT, RT for commission errors, perseverative errors. RT for total correct responses (B = 0.36, p = 0.043, BCa CI = [0.01, 0.74]) and omission errors (B = 0.66, p = 0.001, BCa CI = [0.22, 0.87]) were significant for depressive symptoms. Similarly, anxiety symptoms were predicted by VR CPT RT for total correct responses (B = 0.49, p = 0.011, BCa CI = [0.11, 0.90]) and omission errors (B = 0.48, p = 0.008, BCa CI [0.13, 0.79]) (Table 2).

To assess whether attention measured with Nesplora Aquarium predicted symptoms of depression and anxiety above and beyond classical measures, we performed two hierarchical multiple regression analysis, first, for depressive symptoms, and second, for anxiety symptoms. At Block 1, we included only significant classic test measures that emerged as significant predictors in the previous regression analysis (see Table 4). Similarly, at Block 2 we included only Nesplora Aquarium outcomes that emerged as significant predictors of depressive and anxiety symptoms. For symptoms of depression, results indicated that at Block 1, MoCA test, TMT-A and Corsi Blocks Test contributed significantly to the regression model (F = 6.87, p = 0.000) and accounted for 21% of the variation. At Block 2, as predicted, results showed that adding VR CPT omission errors and VR CPT RT for total correct responses explained 39%, F = 9.62, p = 0.000. After controlling for results on MoCA test, TMT-A and Corsi Blocks Test, VR CPT omission errors and RT explained an additional 18%, F change = 11.09, p = 0.000. Both VR outcomes were significant predictors, omissions, B = 0.43, p = 0.005, BCa CI = [0.13, 0.70] and RT, respectively, B = 0.43, p = 0.002, BCa CI = [0.18, 0.67]. In the final model, none of the covariates were statistically significant (all ps > 0.05). For anxiety symptoms, scores of MoCA test, CPT RT, TMT-B and Corsi total score were entered at Block 1, explaining 33% of variance. After entry of VR CPT RT for total correct responses and VR CPT omissions at Block 2, the total variance explained by the model was 40%, F = 8.42, p = 0.000. After controlling for results on MoCA test, CPT RT for correct answers, TMT-B and Corsi Blocks Test, errors and VR CPT RT and omission errors explained an additional 0.8%, F change = 4.87, p = 0.01. Only VR CPT RT was a significant predictor (B = 0.31, p = 0.014, BCa CI = [0.04, 0.63]. In the final model, none of the covariates were statistically significant (all ps > 0.05) (Table 3).

Table 3 Summary of hierarchical multiple regression results for predicting symptoms of depression and anxiety
Full size table

3.4 Group Effects on Attention Measured with Nesplora Aquarium

To explore patterns of attention and inhibition deficits among participants with elevated symptoms of depression and anxiety compared with participants with low symptoms using Nesplora Aquarium, we performed a binary logistic regression. Results of a binary logistic regression with Nesplora Aquarium variables (RT for total correct responses, SD RT, omission and commission errors, RT to commission errors and perseverative errors) indicated that the full model was statistically significant, χ2 = 15.70, p = 0.02 suggesting that the model distinguished between participants with low and elevated symptoms of depression and anxiety. The model explained between 17% (Cox & Snell R2) and 23% (Nagelkerke R2) of the variability in displaying symptoms of depression and anxiety and correctly predicted 69.5% of cases. As shown in Table 4, only two variables were significant: RT for total correct responses, OR = 1.09, p = 0.05, 95% BCa CI = [1.00, 1.18] and omission errors, OR = 1.08, p = 0.035, 95% BCa CI = [1.00, 1.17] (see Table 4).

Table 4 Summary of logistic regression for predicting likelihood of displaying symptoms of depression and anxiety based on VR CPT outcomes
Full size table

To identify if there were any significant differences between participants with elevated symptoms of depression, those with elevated symptoms of anxiety and participants with mixed anxiety and depression symptoms we performed a Kruskal–Wallis Test for each of the Nesplora Aquarium outcomes. We used this test given the small sample of participants in some of the groups (Field 2013). Results are presented in Table 5.

Table 5 Performance on Nesplora Aquarium between participants with elevated symptoms of depression, anxiety and mixed sample of anxiety and depression symptoms
Full size table

We performed post hoc comparisons for the significant Kruskal–Wallis Tests using Mann–Whitney U tests with Bonferroni adjustments to control for multiple comparisons (p = 0.05/3, p = 0.017). The following comparisons were significant. First, significant differences in RT to correct responses were found between people with anxiety symptoms (Md N = 53.60) and mixed depression and anxiety symptoms (Md N = 61.78), U = 50.00, z = − 2.81, p = 0.005. For omission errors, people with depression symptoms performed significantly fewer omission errors (Md N = 44.95) than people with mixed anxiety and depression symptoms (Md N = 64.09), U = 9.50, z = − 2.62, p = 0.009. People with anxiety symptoms (Md N = 45.64) performed significantly better on omission errors than people with mixed anxiety and depression symptoms (Md N = 64.09), U = 32.00, z = − 3.48, p = 0.001. People with anxiety symptoms (Md N = 51.79) committed more commission errors than those with mixed symptoms (Md N = 41.21), U = 49.50, z = − 2.84, p = 0.004.

3.4.1 Sensory effects

To identify whether participants with elevated symptoms of depression and anxiety are more likely to have visual or auditory attention impairments compared with healthy controls with low symptoms of depression and anxiety, we performed binary logistic regressions. Table 5 displays results of multivariable binary logistic regression analysis. Our predictions were partially supported as only two variables were statistically significant: omission errors for visual stimuli, OR = 1.06, p = 0.03, 95% BCa CI = [0.99, 1.14] and SD RT for visual stimuli, OR = 1.12, p = 0.001, 95% BCa CI = [1.04, 1.21] (see Online Resources, Table A.6.).

3.5 Safety concerns in VR, usability and presence

The average ratings of Nesplora Aquarium indicated a good-to-excellent usability of the product (M = 80.18, SD = 15.30). According to norms from the literature Nesplora Aquarium obtained a class A- rating, making it more usable than 85–89 of other systems (Bangor et al. 2009; Sauro and Lewis 2016). Results from an independent samples t test showed that the participants with elevated symptoms of depression and anxiety rated the usability of the CPT VR Nesplora Aquarium significantly lower than participants with low symptoms, t = 2.62, p = 0.01, BCa CI = [2.32, 15.18]. Participants with low depressive and anxiety symptoms rated Nesplora Aquarium as excellent (M = 84.45, SD = 11.49), where a score of 85 corresponds to an A + (highest grade) and a percentile rank of 96–100. Our VR system is considered more usable then 96–100% of products and less usable then 5–10%. On the other hand, participants with high symptoms rated Nesplora Aquarium as being good (M = 75.91, SD = 17.45) with a grade of B and percentile rank of 70–79, which makes it less usable, though within the range of highly acceptable products.

To identify differences between participants with low and high depression and anxiety in the level of self-reported symptoms of simulator sickness, we performed an ANCOVA, controlling for pre-test scores of simulator sickness. In line with our predictions, results showed that there were no differences on post-test simulator sickness symptoms reported by the participants, F = 0.57, p = 0.45. Only two participants (5%) with low symptoms of depression and anxiety had a score > 15 and experienced moderate to severe simulator sickness compared to five participants from the elevated symptoms group (12%). We ran an independent samples t-test to identify any differences on self-report levels of presence in VR between participants with low symptoms of depression and anxiety and those with elevated levels. No significant differences between the two groups emerged, t = 1.39, p = 0.16, BCa CI = [− 2.74, 17.52].

4 Discussion

This study aimed to comprehensively validate, for the first time, the Nesplora Aquarium VR experience as a measure of selective attention and inhibition in people with low and elevated symptoms of depression and anxiety. The data generally supported the validity of the Nesplora Aquarium. We first assessed validity on the basis of correlations between Nesplora Aquarium and other standardised neuropsychological measures such as computerised CPT, Stroop test, MoCA, Corsi block test). We expected high correlations between outcomes of Nesplora Aquarium and the outcomes of the computerised version of CPT, because the two tests were highly comparable in terms of outcomes as they measured the same theoretical construct: attention and inhibition. As predicted, the outcomes of the two tests were highly and positively correlated. Comparable outcomes between VR tests and traditional tests for neuropsychological assessment were previously reported in a meta-analysis with a pooled correlation coefficient of 0.51 (Neguţ et al. 2015). In addition to the expected high correlation between tests that measures similar constructs such as attention and inhibition, we also found weak or a lack of correlations between Nesplora and other cognitive tests that measure different constructs (Urbina 2004), i.e. between Nesplora measuring attention and inhibition and TMT A& B, Corsi and MoCA measuring executive functioning, spatial working memory and cognitive impairment. However, we did find moderate to high correlations between Nesplora Aquarium omission errors and TMT A& B, Corsi and MoCA. Unexpected correlations between VR tests and traditional tests measuring different cognitive constructs have been observed in other studies using a sample of active-duty soldiers (Armstrong et al. 2013). For example, Armstrong et al. (2013) computed correlations between a VR Stroop Task and classical tests- the Automated Neuropsychological Assessment Metrics and showed that VR Stroop outcomes were correlated with subscales of ANAM that measure other cognitive processes than Stroop such as visual search, sustained attention, and working memory, spatial processing and visuo–spatial working memory. In our study, a possible explanation for a strong positive association between TMT A& B, Corsi and MoCA and Nesplora Aquarium omission errors is that this type of error might measure a component of attention processing that is closely linked with the general domain of cognitive functioning. As currently these unexpected correlations between tests measuring different cognitive functions are unclear, future research could focus on explaining their nature by using a broader range of neurophysiological measures. Testing additional cognitive abilities such as memory, general IQ could inform better the relationship between outcomes of Nesplora Aquarium and these cognitive abilities. Despite the few unexpected findings, our results do support the validity of Nesplora Aquarium for measuring sustained attention, impulsivity and inhibition.

A second objective of our study was to assess whether Nesplora Aquarium could predict symptoms of depression and anxiety above and beyond other traditional measures. Nesplora Aquarium seems to be a strong predictor of current symptoms of anxiety and depression as shown by its ability to explain these symptoms above and beyond classic tests. First, the regression models with Nesplora Aquarium as predictors explained symptoms of depression and anxiety more than regression models with traditional neuropsychological measures such as MoCA, traditional CPT, TMT A & B, and Stroop. Several measures were identified as significant covariates: general cognition (MoCA), as well as multiple cognitive domains: RT for correct responses (CPT), such as psychomotor speed, mental flexibility and shifting (TMT A & B), and spatial working memory span (Corsi Blocks Test). A full discussion on covariates can be found in the Online Resources. Second, after controlling for covariates: MoCA, traditional CPT, TMT A & B, and Stroop, Nesplora Aquarium was better at predicting current symptoms of depression and anxiety than other traditional measures of cognitive functions. However, these encouraging findings were limited to two out of six Nesplora Aquarium outcomes. This lack of association between four Nesplora Aquarium outcomes could be explained by increased heterogeneity of both depression and anxiety (Keller et al. 2019; Nandi et al. 2009) which might lead to some individuals displaying attention impairment whilst others are not that affected. The fact that omission errors and RT for correct responses were the only significant predictors of symptoms of depression can be explained by the “cognitive speed hypothesis” which states that depression is characterised by cognitive slowness which may lead to cognitive dysfunction (Den Hartog et al. 2003). Depressed individuals might display reduced attention performance due to slow processing speed (Degl'Innocenti et al. 1998; Den Hartog et al. 2003; Egeland et al. 2003) as studies showed slower RT and psychomotor slowness in depression (Kemp et al. 2010; Kertzman et al. 2010; Marazziti et al. 2010). In our study this is consistent with slower RTs for correct targets. Additionally, because omission errors are closely linked to inattention and result from failing to respond to correct targets, it might be the case that the increased number of these errors were caused by the slow processing speed, as the participants did not have enough time to process the stimuli and correctly respond to them. For anxiety, only RT for correct responses was a significant predictor which might be explained by the Attentional Control Theory (ACT) (Eysenck et al. 2007) which posits that anxiety is associated with increased distractibility and reduced efficiency of central executive functions. However, according to ACT, we also expected impaired inhibitory control in anxiety. Commission errors and RT for commission errors, an index of impulsivity and poor inhibitory control, were not significant predictors of current anxiety symptoms. As we only included participants with elevated symptoms, future studies might try and investigate these associations using samples of individuals with specific anxiety and depression disorders as deficits may be differently influenced by disorder. We also compared participants with elevated symptoms of depression alone, with elevated symptoms of anxiety alone and those with elevated and mixed anxiety and depression symptoms. The comorbidity of both anxiety and depression symptoms is well documented in the literature (Möller et al. 2016) and may result in worse impairment than each condition alone (Tiller 2013; Tyrer 2001). Our results showed a similar pattern to these previous studies as people with mixed anxiety and depression performed worse on omission errors and RT for correct responses than those with either depression or anxiety symptoms. However, no differences emerged on SD RT, RT to commission errors and perseverative errors. This brings further support in favour of the idea that not all attention domains are impaired in depression and anxiety or in conditions with mixed symptoms. Results could also be explained in light of medication taken by participants. In our sample, 22 participants were medicated which could have altered participants’ responses on cognitive tests. Research is controversial when it comes to the positive or negative effects of medication on cognitive function (Godard et al. 2011; Pachet and Wisniewski 2003; Rosenblat et al. 2015; Stein and Strickland 1998; Wagner et al. 2012; Wingo et al 2009). However, in our study we cannot exclude that because participants took medication, including on the day of testing, this may have had a positive impact on their performance thus limiting their impairment on attention outcomes as only two out of six Nesplora Aquarium outcomes were correlated with symptoms of depression and anxiety.

Both depression and anxiety seem to negatively impact effortful and complex tasks (Hartlage et al. 1993). In anxiety, tasks with increased difficulty impair attention more than less complex tasks; for example, in less difficult tasks individuals can compensate for the detrimental effects of anxiety (Eysenck et al. 2007). Similarly, depressed individuals direct their cognitive resources on depression-related themes, and they display a biased attention for emotional stimuli (Disner et al. 2011; Hartlage et al. 1993). A review of task difficulty of VR neuropsychological tests compared with analogue classic tests highlighted increased task difficulty of VR measures for executive functioning, including attention (Neguţ et al. 2016). This may clarify why Nesplora Aquarium predicts symptoms of anxiety nearly twice as much as traditional CPTs. VR tasks require additional cognitive resources because they recreate real-life scenarios which are more complex than classical tasks (Neguţ et al. 2016). For example, in Nesplora Aquarium the environment contains a real-life replica of an aquarium with many tanks, various environmental distractors to ignore and fish to pay attention to. In contrast, the classical CPT consists of letters which appear on a black computer screen. Both tasks measure the same theoretical construct, but the complexity and real-life similarity are higher for the Nesplora Aquarium. Most likely the increased realism of VR increases its power of prediction compared with traditional tests.

Our results are in agreement with previous studies which investigated the incremental value of VR in detecting attention and executive functioning impairments. For example, Lalonde et al. (2013) showed that performance on a VR Stroop task predicted everyday behavioural executive functioning better than classical paper-and-pencil tasks among a sample of healthy adolescents. Rodriguez et al. (2018) used a VR classroom to assess attention deficits of children of ADHD. Their results indicated that the VR classroom predicted ADHD better than a computerised CPT. Recent meta-analyses showed that VR-based measures discriminate between various categories of populations known to have impaired attention and executive functioning such as children with ADHD, individuals with schizophrenia or brain injury and healthy controls (Gilboa et al. 2018; Negut et al. 2016; Parsons et al. 2019). Taken together, VR seems to be an effective tool to use in assessment contexts. By creating 3D simulations of real environments and situations, it offers a challenging, engaging and highly ecological way to test attention and inhibition processes. However, in the current study we assessed if VR is better at predicting current symptoms of depression and anxiety. Future studies might validate the predictive validity of VR tests by investigating whether VR can successfully predict symptoms of depression and anxiety in the long run, for example, for the next 6 months or other real-life functioning indexes like employment status or social services. For our third objective, we investigated whether Nesplora Aquarium can distinguish between participants with low and elevated symptoms of depression and anxiety. We expected that participants with elevated depression and anxiety symptoms were more likely to have poorer scores on the Nesplora Aquarium test compared with individuals with lower symptoms and to identify attention domains which are most affected. Results showed that overall, outcomes of Nesplora significantly distinguished between participants with elevated and low symptoms of depression and anxiety, the first displaying overall poorer attention performance. Taken together, our results provide evidence in favour of reduced vigilance, increased inattention and psychomotor slowness in depression and anxiety. Our findings suggest that the attention and inhibition impairments in depression and anxiety are heterogenous. We argue that anxiety and depression affect differently various attention domains. The only attention domains that were impaired in our sample of participants were increased inattention and reduced processing speed. Most likely, there is not a global deficit of attention in depression and anxiety.

In relation to the sensory effects of Nesplora Aquarium, we predicted that individuals with elevated symptoms of anxiety and depression would be more impaired on visual and auditory modalities compared to individuals with low symptoms. In our study, we only identified some impairments (omission errors and SD to RT) in the visual domain, but not in the auditory domain. Individuals with elevated symptoms of depression and anxiety committed more omission errors in the visual modality, but not in the auditory modality which could indicate that they are less vigilant when attending to visual stimuli. This might suggest that future cognitive rehabilitation or remediation programs might focus on visual tasks. Variability of RT, a measure of consistency of performance (Stuss et al. 1989), suggests changes in sustained attention or fatigability during the task (Climent et al. 2019). Increased RT variability in the visual condition is consistent with increased inattention in this condition, which might indicate increased distractibility for visual stimuli for individuals with elevated symptoms of depression and anxiety (and possibly a sign of “wandering attention” when people get distracted by internal and external stimuli Stuss and Benson 1984; Tinius 2003). This result seems to sit in contrast to previous studies which showed impaired deficits in early auditory and visual processing in depressed and anxious samples (Chang et al. 2011; Kahkonen et al. 2007; Qiu et al. 2011; Schirmer and Escoffier 2010; Takei et al. 2009; Weinstein 1995; Yang et al. 2019; Zweerings et al. 2019). A potential explanation for our results is that previous studies assessed auditory and visual processing at early pre-attentive automatically processing of visual and auditory processing, while ours assessed attention and inhibition as higher cognitive functions. Hence, differences in early visual and auditory processes may disappear in later stage of higher cognitive functions pointing to a generalised deficit of sustained attention irrespective of sensory processing. Future studies could compare pre-attentive sensory processing with later stages of sensory processing modulated by sustained attention to examine this possibility. We also note that in our study we did not account for the cognitive styles of the participants. According to the cognitive theory of multimedia learning, people process information using the visual and the auditory channel. The visualiser–verbaliser cognitive style posits that some people prefer, rely more and perform better when they learn from pictures (visualisers) while other people prefer words or text (verbalisers) (Koć-Januchta et al. 2017; Mayer and Massa 2003). As we did not measure and control for this variable, we cannot exclude that participant’s cognitive style could have impacted our results. In our study, we only identified some impairments (omission errors and SD to RT) in the visual domain, but not in the auditory domain; we cannot rule out the fact that the two groups of participants were not similar in relation to this variable. Future studies can investigate whether individual differences in cognitive styles might moderate the performance on visual and auditory stimuli.

Finally, our last objective was to assess self-reported usability, simulator sickness symptoms and level of presence in VR across individuals with high and low symptoms of depression and anxiety. As predicted, the participants’ responses showed no significant difference between the two groups for symptoms of simulator sickness which is in accordance with results from previous studies (Bouchard et al. 2009, 2014; Kim et al. 2019; Robillard et al. 2003). It might be the case that unlike other vulnerable populations such as older adults or patients with neurological conditions who are more likely to experience side effects associated with exposure to VR (Brooks et al. 2010; Hutton et al. 2018; Matheis et al. 2007), individuals with elevated symptoms of depression and anxiety tolerate VR exposure well. Most likely, because during the assessment with Nesplora Aquarium participants were seated and did not move around the environment, they did not experience significant sensory mismatch or incongruence between visual, vestibular and proprioceptive information, which have been linked with simulator sickness symptoms (Kim et al. 2017). Similarly, as we predicted, there were no significant differences between people with low and elevated symptoms of depression and anxiety on self-report presence in VR. To our knowledge, the current study is the first to address individual differences among people with anxiety and depression compared with controls. Although there are some studies that linked presence with personality (Kober and Neuper 2013), in relation to cognitive abilities previous studies showed that there were no significant differences between spatial presence and spatial ability (Coxon et al. 2016). The current study identified a similar pattern and showed that people with low and elevated levels of anxiety and depression, which clearly differ on the level of cognitive functioning, report the same level of presence within VR.

The results of test usability showed that participants with low symptoms of depression and anxiety rated the usability of the VR system higher than those with high symptoms. However, even the group of participants with high levels of anxiety and depression gave an acceptable rating to the VR product. Other studies that assessed the usability of various VR systems with older adults reported similar ratings to our high anxiety and depression group (data for individuals with elevated symptoms of depression and anxiety) (Corno et al. 2014; Pedroli et al. 2018). The current study assessed Nesplora usability using the SUS scale; however, informally many participants said that they were so absorbed in the environment that they lost track of time. Almost unanimously participants preferred the Nesplora Aquarium over the classical tests. Some highlighted that the task in VR while more fun and engaging than the computerised task was more difficult. Differences in usability ratings between individuals with low symptoms compared with those with elevated symptoms of depression and anxiety may be explained by their self-perceived performance in VR. In the absence of formal recordings of usability other than SUS, we can only speculate that participants with elevated symptoms, especially high anxiety, were preoccupied by their level of performance which in turn might have influenced their usability ratings. Future studies might further explore these differences in usability using objective measures and examine mechanisms that can account for differences.

5 Limitations

Despite a number of strengths of the current study, including being the first to assess attention and inhibition in a highly ecological VR setting, and the promising results, a number of limitations should be considered. First, our design was cross-sectional, which does not allow one to infer a causal relationship between attention and inhibition and current symptoms of depression and anxiety. That being said we are not certain that attention and inhibition can influence symptoms of depression and anxiety or if the other way around, current symptoms of depression and anxiety influence attention and inhibition performance. A recent meta-analysis showed positive significant effects of computerised cognitive training in improvements in symptom severity and daily functioning for patients with depression (Motter et al. 2016) which might highlight the role of cognition on depression symptomatology. Similarly, training of working memory showed significant results in reducing anxiety symptoms (Hadwin and Richards 2016). If improvements in cognition can lead to a reduction in symptoms, this can be taken as evidence of the importance of cognitive functioning in facilitating and maintaining of these symptoms.

We did not control for medication effects on cognitive performance. Psychotropic medication may negatively impact some cognitive functions; thus, there is lack of consensus in the literature regarding the positive or negative effect of medications like antidepressants on cognitive functioning (Godard et al. 2011; Pachet and Wisniewski 2003; Rosenblat et al. 2015; Stein and Strickland 1998; Wagner et al. 2012; Wingo et al. 2009). On most cognitive functions, in our study, there were no significant differences between the medicated and non-medicated group, but we cannot rule out potential effects of medication on cognitive functioning. In our study, we included a comprehensive set of well-validated measures of neuropsychological measures; however, we did not include measures of memory, language comprehension or IQ, and thus, we could not control for their effects. Future studies might investigate the impact of these individual differences and their association with attention performance on VR tasks. Although a limitation of the current study, we did not apply strict inclusion criteria in a sense that we did not focus only on patients with a clinical diagnosis of depression and anxiety disorders. Instead, we applied a more relaxed inclusion criteria and used two well-validated scales to screen for clinically significant symptoms of depression and anxiety, such as BDI-II and STAI. We did so because we were mostly interested in the ability of the Nesplora to predict clinically significant symptoms of depression and anxiety across a wide range of participants, not only those with a clinical diagnosis of depression and anxiety disorders. For example, increased symptoms of anxiety and depression are also prevalent across various chronic medical conditions such as diabetes (Li et al. 2019), vulnerable populations like older adults, LGBT community, people who commit self-harm (Remes et al. 2016) or students (AlFaris et al. 2016).

6 Conclusions

The current study supported the validity of Nesplora Aquarium and its added effectiveness in predicting symptoms of depression and anxiety when compared to traditional tests. VR tests have increased ecological validity, and consequently increased task difficulty similar to real-life scenarios; hence, they may provide a more precise measure of attention and inhibition. Hence, using Nesplora could be more efficient in identifying people with increased anxiety and depression when facing everyday tasks. As the visual modality was more affected than the auditory modality in participants with high levels of anxiety and depression, it suggests the presence of domain-specific deficits; however, further studies using different tasks and tests need to replicate this finding to strengthen this conclusion. Finally, a future focus on usability in clinical psychology could help designers and researchers to develop VR tasks that are effective, easy to use and enjoyable for vulnerable populations to increase their adherence throughout the evaluation and treatment process.