Introduction

Attentional bias to motivationally relevant stimuli such as threat and reward is fundamental for survival (Pool et al., 2016; Yiend, 2010). Prior research has shown that individuals with emotional disorders such as anxiety and depression are more likely to exhibit stronger attentional bias to threat stimuli, even when task irrelevant (for review see, Abado et al., 2020). Importantly, in individuals with emotional disorders, and/or those with anxious traits, more reliable attentional bias to threat is observed, when the threat stimuli are particularly relevant to the individual (e.g. spider stimuli are more relevant to individuals with spider phobia) (for review see, MacLeod et al., 2019).

Over the last decade, there has been a surge of research on how individual differences in intolerance of uncertainty (IU), the tendency to react to uncertainty negatively, underlies different facets of anxiety-related pathology (Carleton, 2016). IU is particularly promising because: (1) it is transdiagnostic, with high levels of self-reported IU observed across many emotional disorders such as anxiety and depression (for meta-analyses see, Gentes & Ruscio, 2011; McEvoy et al., 2019), and (2) it can be reduced through short-term experimental (Oglesby et al., 2017) and long-term therapeutic intervention (Dugas et al., 2022; Mofrad et al., 2020). However, despite this progress, there still remain questions as to how IU modulates attentional bias. Addressing such questions is fundamental, in order to understand how IU operates and how IU can be optimally targeted in treatments for emotional disorders (Shihata et al., 2016).

Notably, only a handful of studies have examined how IU affects attentional processing of uncertainty-based information (Del Cristaldi et al., 2021; Fergus et al., 2013; Fergus & Carleton, 2016; Morriss & McSorley, 2019; Morriss et al., 2017; Wiese et al., 2023). A seminal study by Fergus et al. (2013) highlighted that higher IU was associated with faster identification of word stimuli related to uncertainty (e.g., ‘maybe’), compared to control word stimuli related to household items (e.g. ‘kettle’). Furthermore, Fergus et al. (2013) highlighted that the effect of IU was specific over trait worry. As far as we are aware, here for the first time, we sought to conceptually replicate Fergus et al.’s (2013) seminal study. Ascertaining the reliability of IU-related effects on attentional processes is crucial for understanding the relevance and feasibility of targeting IU-related attentional biases as part of treatment protocols for emotional disorders (e.g., cognitive restructuring, which aims to challenge attentional biases). As such, it is important to establish whether Fergus et al.’s results are robust (for discussion on issues of replication for attentional bias see McNally, 2019).

The current study aimed to conceptually replicate Fergus et al.’s (2013) study. Following Fergus et al., we first predicted that higher IU would be associated with faster reaction times (i.e., facilitation) to word stimuli relating to uncertainty (e.g., ‘maybe’), compared to control word stimuli relating to household items (e.g., ‘kettle’). We secondly anticipated that this association would be specific to IU, even when controlling for other broader negative affective traits such as trait anxiety.

Method

Conceptual replication

The design of the Fergus et al. (2013) study and the current study differed in a few ways. Firstly, the Fergus et al. (2013) study was conducted in a lab with each participant taking part one at a time, whilst the current study was conducted in a Psychology workshop with participants taking part in groups. Secondly, with regards to the visual search task, we used two different buttons for the keyboard responses, a keyboard response was needed to initiate the start of a block of trials, in between blocks of trials participants were reminded what buttons to press on the keyboard, and we adapted two of the word strings (e.g., using a British English word (colour) and non-word matched in length).

Participants

Participants were first year undergraduate Psychology students from the University of X (163 participants) in the UK, who participated as part of a group tutorial in a course module. Due to the classroom nature of the study, some participants did not complete the study and hence were excluded (for more information please see Data Exclusions section below). Demographics were recorded from 150 participants (115 female, 30 male, 2 non-binary, 2 non-disclosed; 1 gender fluid; mean age = 19.08, SD age = 2.57; 107 White, 24 Asian, 10 Multi-ethnic, 8 Black, 1 Arab; First Language: 130 English, 9 Other Language, 4 Mandarin, 3 Cantonese, 2 Italian, 2 Tamil). The procedure of this study was approved by the University of X ethics committee (ID 78950).

The participants in Fergus et al’s (2013) study were also from an undergraduate Psychology student population of similar age. However, Fergus et al’s (2013) study included a more equal balance of males and females, and a different composition of ethnic backgrounds that were representative of North America.

Power analysis

Due to the directional hypothesis, an a-priori sample size calculation for a one-tailed correlation using G*Power software (Faul et al., 2007) was conducted. The following parameters were entered: effect size = 0.22 (r value from the correlation between Intolerance of Uncertainty Scale (IUS) scores and reaction times for the facilitation condition in Fergus et al., 2013), α error probability = 0.05, power = 0.80. The total sample size suggested was n = 126. The required sample size was similar to the original Fergus et al. (2013) study (n = 104).

Questionnaires

Questionnaires were presented on a computer screen using Qualtrics survey software (Qualtrics, Provo, UT), and participant responses were collected using a keyboard.

The 12-item short version of the IUS was used to measure IU and the inhibitory and prospective IU subscales (Carleton et al., 2007). Example items included “Unforeseen events upset me greatly” and “Uncertainty makes me uneasy, anxious, or stressed”. A 5-point Likert scale was used.

The 5-item short version of the State-Trait Anxiety Inventory (STAI-T) was used to measure trait anxiety (Zsido et al., 2020). Example items included “Some unimportant thoughts run through my mind and bothers me” and “I feel that difficulties are piling up so that I cannot overcome them”. A 4-point Likert scale was used.

Visual search task

The design of the visual search task was based on Fergus et al.’s (2013) study and was near identical, except for a few modifications (please see Conceptual Replication section above).

The visual search task was programmed using JavaScript plugins in jsPsych (de Leeuw, 2015) and was hosted on a JATOS server (Lange et al., 2015). The visual search task ran on desktop computers and laptops. Participant responses were collected using a keyboard. There were six different visual search trial conditions (20 trials each; see Fig. 1). The visual search trial conditions either captured facilitation (easier identification of a target) or disengagement (harder identification of target).

Fig. 1
figure 1

Example visual search task trials. a uncertainty word target in array of non-word stimuli, b categorised (i.e., household items) neutral word target in array of non-word stimuli, c non-word target in array of non-word stimuli, d non-word target in array of uncertainty words, e non-word target in array of categorised neutral words, f uncategorised neutral word target in array of uncategorised neutral words

Full size image

Participants were instructed that the goal of the visual search task was to identify whether the unique string of letters on each trial was an English word or non-word. Participants used buttons on the keyboard to respond (‘m’ for an English word and ‘z’ for a non-word). Trials consisted of a central fixation cross (700 ms), followed by an array of four strings of letters presented together in a rectangular pattern, and then a blank screen (1000 ms). The button response to each array of four strings of letters was not time limited. The visual search array remained on the screen until the participant responded.

Participants received one block of practice trials (10 trials) with feedback (correct, incorrect), and four blocks of experimental trials without feedback (30 trials per block; 120 trials total). Each experimental block contained all three visual search trial conditions (10 trials per condition in each block) for either visual search facilitation or disengagement (i.e., a-c or d-f). Block order was counterbalanced (order 1: facilitation, disengagement, facilitation, disengagement; order 2: disengagement, facilitation, disengagement, facilitation) between participants, and the trial order within each block was randomised. Blocks were separated by a 30-second delay, and, between blocks, participants needed to press a button on the keyboard to move the experiment forward.

English word stimuli consisted of uncertainty words (e.g., maybe), categorised neutral words (household items such as kettle), and uncategorised neutral words (e.g., tower, errand). Non-word stimuli consisted of strings of letters. English word and non-word stimuli were matched based on the number of letters in each string.

Procedure

The study was conducted in a computer lab at the University of X. Participants were tested by the lead researcher in four consecutive sessions (~ 40 participants in each session). Participants were instructed about what the study involved, were asked to switch off any of their devices (e.g., phones, smart watches), and were asked to sit quietly throughout the experiment. Using a computer, participants read the information about the study, completed informed consent, completed two questionnaires, and completed a visual search task. The experiment took approximately 30 min.

Data reduction

The data reduction approach used here was very similar to Fergus et al. (2013), except we log-transformed reaction times because winsorising the reaction times was not possible due to the longer length of the reaction times (for distributions of raw reaction time scores, see supplement). Data were reduced using R software (R Core Team, 2022).

Questionnaires

Data were downloaded from Qualtrics (Qualtrics, Provo, UT). Total scores for the IUS, IUS subscales (inhibitory, prospective), and STAI-T were calculated.

Visual search task reaction times

Data were downloaded from JATOS (Lange et al., 2015). For correct trials, reaction times were checked for anticipatory responses (less than 250 ms), but none were observed (0% of trials). Reaction times from incorrect trials were replaced by the mean reaction time of correct trials for that visual search condition. After these steps, reaction times were log-transformed to reduce skew and to retain any outlying values.

Two reaction time difference scores were calculated: Facilitation (mean reaction time to categorised neutral targets in an array of non-word stimuli - mean reaction time to uncertainty targets in an array of non-word stimuli); Disengagement (mean reaction time to non-word targets in an array of categorised neutral words - mean reaction time to non-word targets in an array of uncertainty words).

Data analyses

The data analysis approach (e.g. correlations, partial correlations) used here is identical to Fergus et al. (2013). Data were analysed using R software (R version 4.2.2, R Core Team, 2022). First, correlations were conducted between facilitation/disengagement reaction time difference scores and IUS scores. Second, exploratory correlations were conducted between facilitation/disengagement reaction time difference scores and inhibitory IUS/prospective IUS, and STAI-T. Third, to assess specificity, any significant correlations were followed up with partial correlations between facilitation/disengagement reaction time difference scores and the relevant anxiety measure (e.g. IUS, STAI-T, etc.), controlling for the other anxiety measures.

Data exclusions

Participants were excluded if: (1) their visual search task and questionnaire data could not be linked due to ID number typos (n = 10), (2) they had less than 60% correct responses overall in the visual search task (n = 7), (3) they had less than 60% correct responses for any visual search task condition (n = 24), and (4) their reaction times were +/- 3 standard deviations from the mean after log transformation (n = 2). After these exclusions, the final sample consisted of 120 participants (93 female, 24 male, 2 non-binary, 1 non-disclosed; mean age = 19.08, SD age = 2.82; 89 White, 17 Asian, 8 Multi-ethnic, 5 Black, 1 Arab; First Language: 106 English, 7 Other Language, 4 Mandarin, 2 Italian, 2 Tamil).

Results

Questionnaires

IUS (M = 33.73; SD = 9.20; Range = 12.00–53.00; α = 0.91), Inhibitory IUS (M = 13.17; SD = 4.35; Range = 5.00–23.00; α = 0.83), Prospective IUS (M = 20.57; SD = 5.44; Range = 7.00–33.00; α = 0.85), and STAI-T (M = 12.79; SD = 3.55; Range = 5.00–20.00; α = 0.79) scores had high internal consistency and were similar to that previously reported for student and community samples (Carleton et al., 2007). The IUS and STAI-T questionnaires were significantly positively correlated (r(119) = 0.681, p <.001). For visualisation of distributions of questionnaires, see supplement.

Visual search task

Descriptive statistics by experimental condition for accuracy and reaction time measures are given in Table 1. Descriptive statistics were calculated for log-transformed reaction time difference scores for facilitation (M = 0.02, SD = 0.10, Range = -0.24–0.37), and for disengagement (M = 0.00, SD = 0.09, Range = -0.20–0.25).

Table 1 Descriptive Statistics by Condition for Accuracy and Log Transformed Reaction Times (ms)
Full size table

IUS, inhibitory IUS and STAI-T were all significantly positively correlated with facilitation (Table 2; Fig. 2). Such that higher IUS, inhibitory IUS and STAI-T were associated with faster identification of uncertain word targets vs. categorised neutral word targets. Prospective IUS was not significantly correlated with facilitation, but trends were in a similar direction. All correlations between anxiety measures and disengagement were non-significant.

Table 2 Correlations with 95% Confidence Intervals
Full size table
Fig. 2
figure 2

Scatterplots with line of best fit for the different individual difference questionnaires and Facilitation (Log transformed)

Full size image

The correlation between facilitation and IUS was no longer significant when controlling for STAI-T (r (119) = 0.06, p =.552). Similarly, the correlation between facilitation and STAI-T was no longer significant when controlling for IUS (r (119) = 0.12, p =.199).

Achieved power

A posthoc analysis of achieved power for the correlation between intolerance of uncertainty and facilitation was conducted (bivariate normal model: two tailed, effect size 0.18, α = 0.05, n = 120). The achieved power was 0.50. While the experiment was slightly underpowered, the extra participants required are unlikely to change the overall pattern of results.

Discussion

The current study aimed to conceptually replicate Fergus et al. (2013). In their original study, Fergus et al. (2013) observed: (1) individuals with high IU to identify uncertainty-based information (words related to uncertainty such as ‘maybe’), compared to neutral information (words related to household items) more quickly than individuals with low IU, (2) and that this effect for uncertainty-based information was specific to IU, compared to other measures related to anxiety such as worry. Here, we partially replicated the results of Fergus et al. (2013). However, we also observed that higher trait anxiety was significantly associated with faster identification of uncertainty-based information, relative to neutral information. Thus, there was no specificity of IU or trait anxiety in predicting faster identification of uncertainty-based information, relative to neutral information.

While the effect size was generally small and there was a lack of specificity of IU and trait anxiety in predicting an attentional bias to uncertain information, relative to neutral information, these results are still useful for the field. In particular, the findings from the present study provide further empirical support for the notion that individual differences in anxious traits are related to attentional biases for uncertainty-based information (Del Cristaldi et al., 2021; Fergus & Carleton, 2016; Jhang & Laing, 2023; Morriss et al., 2017; Wiese et al., 2023), even in the absence of direct threat (Fergus et al., 2013; Morriss et al., 2019). To establish the generalisability of attentional biases to uncertain-based information in those with anxious traits (for review see, Abado et al., 2020; Morriss et al., 2023) and to understand its usefulness for the diagnosis and treatment of emotional disorders (Shihata et al., 2016), further replication is warranted in diverse community and clinical samples.

A methodological limitation of the current design is that the dissociation of attentional facilitation or disengagement may not have been captured optimally because the participant must compare the properties of two stimuli on each trial to make a decision (see Clarke et al., 2013). Therefore, the participant might attentionally engage and disengage to the different stimuli several times on any given trial. Although, this is potentially problematic, as this study was a conceptual replication, we opted to use the same terminology as the original Fergus et al. (2013) paper.

To conclude, we partially replicated the results of Fergus et al. (2013), in that higher IU was associated with faster identification of uncertainty-based information, relative to neutral information. A similar pattern was also observed for trait anxiety. Although, there was no specificity of IU or trait anxiety in predicting faster identification of uncertainty-based information, relative to neutral information. Taken together, these findings highlight the potential utility of targeting attentional biases to uncertainty-based information in emotional disorders.