Skip to main content
SpringerLink
Log in

Trait Anxiety, State Anxiety, and Attentional Bias to Threat: Assessing the Psychometric Properties of Response Time Measures

  • Original Article
  • Published:
Cognitive Therapy and Research Aims and scope Submit manuscript

Abstract

Attentional biases to threat are thought to play a central role in the development and maintenance of anxiety disorders. Various measures have been developed to index these biases; unfortunately, the psychometric properties of response time measures of attentional bias have not been thoroughly evaluated, and the reliability of those that have been evaluated is poor. The current study assessed the reliability and convergent validity of dot probe and emotional cueing bias scores in high-anxious and low-anxious participants in the context of a state anxiety manipulation to increase the likelihood of producing large, potentially reliable attentional biases. One hundred sixty participants completed an anxious or calm mood induction followed by dot probe and emotional cueing tasks. Reliability estimates for bias scores in the full sample were low, ranging from 0 to .44; convergent validity estimates were also poor. The anxiety induction did not substantially improve the reliability or validity of the measures. These results underscore the importance of developing new, more reliable attentional bias measures for future research.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

Notes

  1. Actors were 15 female and 17 male. Actor numbers were 1, 2, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 18, 20, 21, 22, 23, 26, 28, 29, 31, 32, 33, 34, 36, 37, 38, 39, 40, and 41.

  2. Actors were 8 male and 8 female; they matched those in Mogg et al. (2008). Actor numbers were 1, 3, 6, 7, 9, 12, 16, 18, 23, 24, 29, 33, 34, 37, 40, and 42.

  3. The script for the mood induction can be obtained by contacting the first author.

  4. When our data were analysed using a stem and leaf plot outlier procedure (Mogg et al., 1999), the ABT results were similar and the reliability estimates were similar or lower.

  5. The results of these analyses are available upon request from the first author.

References

  • Amir, N., Beard, C., Taylor, C. T., Klumpp, H., Elias, J., Burns, M., & Chen, X. (2009). Attention training in individuals with Generalized Social Phobia: A randomized controlled trial. Journal of Consulting and Clinical Psychology, 77(5), 961.

    Article  PubMed Central  PubMed  Google Scholar 

  • Antony, M. M., Beiling, P. J., Cox, B. J., Enns, M. W., & Swinson, R. P. (1998). Psychometric properties of the 42-item and 21-item versions of the Depression Anxiety Stress Scales in clinical groups and a community sample. Psychological Assessment, 10, 176–181.

    Article  Google Scholar 

  • Bar-Haim, Y., Holoshitz, Y., Eldar, S., Frenkel, T. I., Muller, D., Charney, D. S., et al. (2010). Life-threatening danger and suppression of attention bias to threat. The American Journal of Psychiatry, 167(6), 694–698.

    Article  PubMed  Google Scholar 

  • Bar-Haim, Y., Lamy, D., Pergamin, L., Bakermans-Kranenburg, M. J., & Van Ijzendoorn, M. H. (2007). Threat-related attentional bias in anxious and nonanxious individuals: A meta-analytic study. Psychological Bulletin, 133(1), 1–24.

    Article  PubMed  Google Scholar 

  • Beard, C. (2011). Cognitive bias modification for anxiety: current evidence and future directions. Expert Review of Neurotherapeutics, 11(2), 299–311.

    Article  PubMed Central  PubMed  Google Scholar 

  • Beck, A. T., & Clark, D. A. (1997). An information processing model of anxiety: Automatic and strategic processes. Behaviour Research and Therapy, 35(1), 49–58.

    Article  CAS  PubMed  Google Scholar 

  • Bögels, S. M., & Mansell, W. (2004). Attention processes in the maintenance and treatment of social phobia: Hypervigilance, avoidance and self-focused attention. Clinical Psychology Review, 24, 827–856.

    Article  PubMed  Google Scholar 

  • Borgmann, K. W., Risko, E. F., Stolz, J. A., & Besner, D. (2007). Simon says: Reliability and the role of working memory and attentional control in the Simon task. Psychonomic Bulletin & Review, 14(2), 313–319.

    Article  Google Scholar 

  • Bradley, B. P., Mogg, K., & Millar, N. H. (2000). Covert and overt orienting of attention to emotional faces in anxiety. Cognition and Emotion, 14(6), 789–808.

    Article  Google Scholar 

  • Bradley, B. P., Mogg, K., Millar, N., Bonham-Carter, C., Fergusson, E., Jenkins, J., & Parr, M. (1997). Attentional biases for emotional faces. Cognition and Emotion, 11(1), 25–42.

    Article  Google Scholar 

  • Bradley, B. P., Mogg, K., White, J., Groom, C., & Bono, J. (1999). Attentional bias for emotional faces in generalized anxiety disorder. British Journal of Clinical Psychology, 38(3), 267–278.

    Article  PubMed  Google Scholar 

  • Broomfield, N. M., & Turpin, G. (2005). Covert and overt attention in trait anxiety: A cognitive psychophysiological analysis. Biological Psychology, 68(3), 179–200.

    Article  PubMed  Google Scholar 

  • Brosschot, J., Deruiter, C., & Kindt, M. (1999). Processing bias in anxious subjects and repressors, measured by emotional Stroop interference and attentional allocation. Personality and Individual Differences, 26(5), 777–793.

    Article  Google Scholar 

  • Brown, H. M., Eley, T. C., Broeren, S., MacLeod, C., Rinck, M., Hadwin, J. A., & Lester, K. J. (2014). Psychometric properties of reaction time based experimental paradigms measuring anxiety-related information-processing biases in children. Journal of Anxiety Disorders, 28(1), 97–107.

    Article  CAS  PubMed  Google Scholar 

  • Cisler, J. M., Bacon, A. K., & Williams, N. L. (2009). Phenomenological characteristics of attentional biases towards threat: A critical review. Cognitive Therapy and Research, 33(2), 221–234.

    Article  PubMed Central  PubMed  Google Scholar 

  • Cooper, R. M., Bailey, J. E., Diaper, A., Stirland, R., Renton, L. E., Benton, C. P., et al. (2011). Effects of 7.5% CO2 inhalation on allocation of spatial attention to facial cues of emotional expression. Cognition & Emotion, 25(4), 626–638.

    Article  Google Scholar 

  • Cooper, R. M., Rowe, A. C., Penton-Voak, I. S., & Ludwig, C. (2009). No reliable effects of emotional facial expression, adult attachment orientation, or anxiety on the allocation of visual attention in the spatial cueing paradigm. Journal of Research in Personality, 43(4), 643–652.

    Article  Google Scholar 

  • Dalgleish, T., Taghavi, R., Neshat-Doost, H., Moradi, A., Canterbury, R., & Yule, W. (2003). Patterns of processing bias for emotional information across clinical disorders: A comparison of attention, memory, and prospective cognition in children and adolescents with depression, generalized anxiety, and posttraumatic stress disorder. Journal of Clinical Child and Adolescent Psychology, 32, 10–21.

    Article  PubMed  Google Scholar 

  • Egloff, B., & Hock, M. (2003). Assessing attention allocation toward threat-related stimuli: A comparison of the emotional Stroop task and the attentional probe task. Personality and Individual Differences, 35(2), 475–483.

    Article  Google Scholar 

  • Eide, P., Kemp, A., Silberstein, R. B., Nathan, P. J., & Stough, C. (2002). Test-retest reliability of the emotional Stroop task: Examining the paradox of measurement change. The Journal of Psychology, 136(5), 514–520.

    Article  CAS  PubMed  Google Scholar 

  • Enock, P. M., Hofmann, S. G., & McNally, R. J. (2014). Attention bias modification training via smartphone to reduce social anxiety: A randomized, controlled multi-session experiment. Cognitive Therapy and Research, 38(2), 200–216.

    Article  Google Scholar 

  • Faul, F., Erdfelder, E., Lang, A.-G., & Buchner, A. (2007). G*Power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behavior Research Methods, 39, 175–191.

    Article  PubMed  Google Scholar 

  • Fischer, D. G., & Fick, C. (1993). Measuring social desirability: Short forms of the Marlowe-Crowne Social Desirability Scale. Educational and Psychological Measurement, 53, 417–424.

    Article  Google Scholar 

  • Fox, E. (1993). Allocation of visual attention and anxiety. Cognition and Emotion, 7, 207–215.

    Article  Google Scholar 

  • Fox, L. S., & Knight, B. G. (2005). The effects of anxiety on attentional processes in older adults. Aging & Mental Health, 9(6), 585–593.

    Article  CAS  Google Scholar 

  • Fox, E., Russo, R., Bowles, R., & Dutton, K. (2001). Do threatening stimuli draw or hold visual attention in subclinical anxiety? Journal of Experimental Psychology: General, 130(4), 681–700.

    Article  CAS  Google Scholar 

  • Fox, E., Russo, R., & Dutton, K. (2002). Attentional bias for threat: Evidence for delayed disengagement from emotional faces. Cognition and Emotion, 16(3), 1–22.

    Google Scholar 

  • Gotlib, I. H., Kasch, K. L., Traill, S., Joormann, J., Arnow, B. A., & Johnson, S. L. (2004). Coherence and specificity of information-processing biases in depression and social phobia. Journal of Abnormal Psychology, 113(3), 386–398.

    Article  PubMed  Google Scholar 

  • Greenwald, A. G., Nosek, B. A., & Banaji, M. R. (2003). Understanding and using the implicit association test: I. An improved scoring algorithm. Journal of Personality and Social Psychology, 85(2), 197.

    Article  PubMed  Google Scholar 

  • Grös, D. F., Antony, M. M., Simms, L. J., & McCabe, R. E. (2007). Psychometric properties of the State-Trait Inventory for Cognitive and Somatic Anxiety (STICSA): Comparison to the State-Trait Anxiety Inventory (STAI). Psychological assessment, 19(4), 369–381.

  • Jefferies, L. N., Smilek, D., Eich, E., & Enns, J. T. (2008). Emotional valence and arousal interact in attentional control. Psychological Science, 19(3), 290–295.

    Article  PubMed  Google Scholar 

  • Kappenman, E. S., Farrens, J. L., Luck, S. J., & Proudfit, G. H. (2014). Behavioral and ERP measures of attentional bias to threat in the dot-probe task: Poor reliability and lack of correlation with anxiety. Frontiers in Psychology, 5, 1368. doi:10.3389/fpsyg.2014.01368.

  • Kopriva, R. J., & Shaw, D. G. (1991). Power estimates: The effect of dependent variable reliability on the power of one-factor ANOVAs. Educational and Psychological Measurement, 51, 585–595.

    Article  Google Scholar 

  • Koster, E. H. W., Crombez, G., Verschuere, B., Van Damme, S., & Wiersema, J. R. (2006a). Components of attentional bias to threat in high trait anxiety: Facilitated engagement, impaired disengagement, and attentional avoidance. Behaviour Research and Therapy, 44(12), 1757–1771.

    Article  PubMed  Google Scholar 

  • Koster, E. H. W., Leyman, L., De Raedt, R., & Crombez, G. (2006b). Cueing of visual attention by emotional facial expressions: The influence of individual differences in anxiety and depression. Personality and Individual Differences, 41(2), 329–339.

    Article  Google Scholar 

  • Legerstee, J. S., Tulen, J. H. M., Kallen, V. L., Dieleman, G. C., Treffers, P. D. A., Verhulst, F. C., & Utens, E. M. W. J. (2009). Threat-related selective attention predicts treatment success in childhood anxiety disorders. Journal of the American Academy of Child and Adolescent Psychiatry, 48(2), 196–205.

    Article  PubMed  Google Scholar 

  • Lovibond, P. F., & Lovibond, S. H. (1995). The structure of negative emotional states: Comparison of the Depression Anxiety Stress Scales (DASS) with the Beck Depression and Anxiety Inventories. Behaviour Research and Therapy, 33, 335–342.

    Article  CAS  PubMed  Google Scholar 

  • MacLeod, J. W., Lawrence, M. A., McConnell, M. M., Eskes, G. A., Klein, R. M., & Shore, D. I. (2010). Appraising the ANT: Psychometric and theoretical considerations of the Attention Network Test. Neuropsychology, 24, 637–651.

  • MacLeod, C., & Mathews, A. (1988). Anxiety and the allocation of attention to threat. The Quarterly Journal of Experimental Psychology Section A, 40(4), 653–670.

    Article  CAS  Google Scholar 

  • MacLeod, C., Rutherford, E., Campbell, L., Ebsworthy, G., & Holker, L. (2002). Selective attention and emotional vulnerability: Assessing the causal basis of their association through the experimental manipulation of attentional bias. Journal of Abnormal Psychology, 111(1), 107.

    Article  PubMed  Google Scholar 

  • Maloney, E. A., Risko, E. F., Preston, F., Ansari, D., & Fugelsang, J. (2010). Challenging the reliability and validity of cognitive measures: The case of the numerical distance effect. Acta Psychologica, 134(2), 154–161.

    Article  PubMed  Google Scholar 

  • Mathews, A. (1990). Why worry? The cognitive function of anxiety. Behaviour Research and Therapy, 28, 455–468.

    Article  CAS  PubMed  Google Scholar 

  • Mathews, A., & MacLeod, C. (2002). Induced processing biases have causal effects on anxiety. Cognition and Emotion, 16, 331–354.

    Article  Google Scholar 

  • Miller, J. (1991). Reaction time analysis with outlier exclusion: Bias varies with sample size. The Quarterly Journal of Experimental Psychology, 43(4), 907–912.

  • Miller, J., & Ulrich, R. (2013). Mental chronometry and individual differences: Modeling reliabilities and correlations of reaction time means and effect sizes. Psychonomic Bulletin & Review, 20(5), 819–858.

    Article  Google Scholar 

  • Mogg, K., & Bradley, B. P. (1998). A cognitive-motivational analysis of anxiety. Behaviour Research and Therapy, 36(9), 809–848.

    Article  CAS  PubMed  Google Scholar 

  • Mogg, K., & Bradley, B. P. (1999). Some methodological issues in assessing attentional biases for threatening faces in anxiety: A replication study using a modified version of the probe detection task. Behaviour Research and Therapy, 37, 595–604.

    Article  CAS  PubMed  Google Scholar 

  • Mogg, K., Bradley, B. P., Dixon, C., Fisher, S., Twelftree, H., & McWilliams, A. (2000). Trait anxiety, defensiveness and selective processing of threat: an investigation using two measures of attentional bias. Personality and Individual Differences, 28(6), 1063–1077.

    Article  Google Scholar 

  • Mogg, K., Bradley, B. P., & Hallowell, N. (1994). Attentional bias to threat: Roles of trait anxiety, stressful events, and awareness. Quarterly Journal of Experimental Psychology, 47(4), 841–864.

    Article  CAS  PubMed  Google Scholar 

  • Mogg, K., Holmes, A., Garner, M., & Bradley, B. P. (2008). Effects of threat cues on attentional shifting, disengagement and response slowing in anxious individuals. Behaviour Research and Therapy, 46(5), 656–667.

    Article  PubMed Central  PubMed  Google Scholar 

  • Mogg, K., Mathews, A., Bird, C., & Macgregor-Morris, R. (1990). Effects of stress and anxiety on the processing of threat stimuli. Journal of Personality and Social Psychology, 59(6), 1230.

    Article  CAS  PubMed  Google Scholar 

  • Mogg, K., Mathews, A., & Weinman, J. (1987). Memory bias in clinical anxiety. Journal of Abnormal Psychology, 96(2), 94.

    Article  CAS  PubMed  Google Scholar 

  • Mogoaşe, C., David, D., & Koster, E. H. (2014). Clinical efficacy of attentional bias modification procedures: An updated meta‐analysis. Journal of Clinical Psychology, 70(12), 1133–1157.

  • Murphy, K., & Davidshofer, C. (2005). Psychological testing: Principles and applications (6th ed.). Upper Saddle River, NJ: Prentice Hall.

    Google Scholar 

  • Preacher, K. J., Rucker, D. D., MacCallum, R. C., & Nicewander, W. A. (2005). Use of the extreme groups approach: A critical reexamination and new recommendations. Psychological Methods, 10(2), 178.

    Article  PubMed  Google Scholar 

  • Quigley, L., Nelson, A. L., Carriere, J., Smilek, D., & Purdon, C. (2012). The effects of trait and state anxiety on attention to emotional images: An eye-tracking study. Cognition and Emotion, 26, 1390–1411.

    Article  PubMed  Google Scholar 

  • Rapee, R. M., & Heimberg, R. G. (1997). A cognitive-behavioral model of anxiety in social phobia. Behaviour Research and Therapy, 35, 741–756.

    Article  CAS  PubMed  Google Scholar 

  • Ree, M. J., French, D., MacLeod, C., & Locke, V. (2008). Distinguishing cognitive and somatic dimensions of state and trait anxiety: Development and validation of the State-Trait Inventory for Cognitive and Somatic Anxiety (STICSA). Behavioural and Cognitive Psychotherapy, 36(03), 313–332.

    Article  Google Scholar 

  • Reinecke, A., Waldenmaier, L., Cooper, M. J., & Harmer, C. J. (2013). Changes in automatic threat processing precede and predict clinical changes with exposure-based cognitive-behavior therapy for panic disorder. Biological Psychiatry, 73(11), 1064–1070.

    Article  PubMed  Google Scholar 

  • Russell, J. A., Weiss, A., & Mendelsohn, G. A. (1989). Affect Grid: A single-item scale of pleasure and arousal. Journal of Personality and Social Psychology, 57(3), 493–502.

    Article  Google Scholar 

  • Sattler, J. M. (2008). The assessment of children: Cognitive foundations (5th ed.). San Diego: Jerome M. Sattler, Publisher Inc.

    Google Scholar 

  • Schmukle, S. C. (2005). Unreliability of the dot probe task. European Journal of Personality, 19(7), 595–605.

    Article  Google Scholar 

  • Staugaard, S. (2009). Reliability of two versions of the dot probe task using photographic faces. Psychology Science Quarterly, 51(3), 339–350.

    Google Scholar 

  • Stolz, J. A., Besner, D., & Carr, T. H. (2005). Implications of measures of reliability for theories of priming: Activity in semantic memory is inherently noisy and uncoordinated. Visual Cognition, 12, 284–336.

    Google Scholar 

  • Strahan, R., & Gerbasi, K. C. (1972). Short, homogeneous versions of the Marlowe-Crowne Social Desirability Scale. Journal of Clinical Psychology, 28, 191–193.

    Article  Google Scholar 

  • R Core Development Team (2005). R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing.

  • Tottenham, N., Tanaka, J., Leon, A., et al. (2009). The NimStim set of facial expressions: Judgments from untrained research participants. Psychiatry Research, 168, 242–249.

    Article  PubMed Central  PubMed  Google Scholar 

  • Van Bockstaele, B., Verschuere, B., Koster, E. H., Tibboel, H., De Houwer, J., & Crombez, G. (2011). Differential predictive power of self-report and implicit measures on behavioural and physiological fear responses to spiders. International Journal of Psychophysiology, 79(2), 166–174.

    Article  PubMed  Google Scholar 

  • Van Bockstaele, B., Verschuere, B., Tibboel, H., De Houwer, J., Crombez, G., & Koster, E. H. (2014). A review of current evidence for the causal impact of attentional bias on fear and anxiety. Psychological Bulletin, 140(3), 682–721.

    Article  PubMed  Google Scholar 

  • Van Damme, S., Crombez, G., Hermans, D., Koster, E. H., & Eccleston, C. (2006). The role of extinction and reinstatement in attentional bias to threat: A conditioning approach. Behaviour Research and Therapy, 44(11), 1555–1563.

    Article  PubMed  Google Scholar 

  • Van Selst, M., & Jolicoeur, P. (1994). A solution to the effect of sample size on outlier elimination. The Quarterly Journal of Experimental Psychology, 47(3), 631–650.

    Article  Google Scholar 

  • Waechter, S., Nelson, A. L., Wright, C., Hyatt, A., & Oakman, J. (2014). Attentional bias to threat in social anxiety: Psychometric properties of dot probe and eye-tracking measures. Cognitive Therapy and Research, 38, 313–333.

    Article  Google Scholar 

  • Waters, A. M., Mogg, K., Bradley, B. P., & Pine, D. S. (2012). Attention bias for angry faces in children with social phobia. Journal of Experimental Psychopathology, 2, 475–489.

    Article  Google Scholar 

  • Weierich, M. R., Treat, T. A., & Hollingworth, A. (2008). Theories and measurement of visual attentional processing in anxiety. Cognition and Emotion, 22(6), 985–1018.

    Article  Google Scholar 

  • Yiend, J. (2010). The effects of emotion on attention: A review of attentional processing of emotional information. Cognition and Emotion, 24(1), 3–47.

    Article  Google Scholar 

  • Yiend, J., & Mathews, A. (2001). Anxiety and attention to threatening pictures. The Quarterly Journal of Experimental Psychology: Section A, 54(3), 665–681.

  • Zvielli, A., Bernstein, A., & Koster, E. H. (in press). Temporal dynamics of attentional bias. Clinical Psychological Science.

Download references

Acknowledgments

This work was supported by a Natural Sciences and Engineering Research Council of Canada (NSERC) grant to J. A. S. and a NSERC CGS-D scholarship to S. W. We thank Klint Fung, Arief Kartolo, Harjot Dosanjh, Jordan Kemp, Marina Ren, and Yuna Lee for their assistance with data collection.

Conflicts of Interest

Stephanie Waechter and Jennifer A. Stolz declare that they have no conflicts of interest.

Informed Consent

All procedures followed were in accordance with the ethical standards of the responsible committee on human experimentation (institutional and national) and with the Helsinki Declaration of 1975, as revised in 2,000 (5). Informed consent was obtained from all patients for being included in the study.

Animal Rights

No animal studies were carried out by the authors for this article.

Author information

Authors and Affiliations

  1. Department of Psychology, University of Waterloo, 200 University Ave. W., Waterloo, ON, N2L 3G1, Canada

    Stephanie Waechter & Jennifer A. Stolz

Authors
  1. Stephanie Waechter
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jennifer A. Stolz
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Stephanie Waechter.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Waechter, S., Stolz, J.A. Trait Anxiety, State Anxiety, and Attentional Bias to Threat: Assessing the Psychometric Properties of Response Time Measures. Cogn Ther Res 39, 441–458 (2015). https://doi.org/10.1007/s10608-015-9670-z

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10608-015-9670-z

Keywords

Search

Navigation