A Bifactor Model of the Straightforward Attentional Control Scale

  • Matt R. JudahEmail author
  • Kevin G. Saulnier
  • Nathan M. Hager
  • Nicholas P. Allan


Prior studies suggest that the Attentional Control Scale (ACS) consists of two correlated factors. These models do not include a general factor, though this is assumed often in theory and practice. Using an adult North American sample collected through Amazon Mechanical Turk (N = 419), we examined a revised version of the ACS with positive keying of all items (Straightforward Attentional Control Scale; ACS-S) to avoid the potential of a factor produced by mixing positively and negatively keyed items. Exploratory factor analysis using a bifactor approach was used to examine the structure of the ACS-S, which consisted of a general factor and a nuisance variance factor. There was mixed evidence of good fit for the full 20-item version of the ACS, but fit was good for a reduced 12-item version of the ACS-S. The full and reduced version were highly correlated (r = .98). Exploratory structural equation modeling suggested that the general factor of the ACS-S was negatively related to depression and both anxious arousal, as reflected by panic symptoms, and anxious apprehension, as reflected by worry. The findings suggest that perceived attentional control is uniquely related to depression and both dimensions of anxiety. ACS-S Short scores were tested as a moderator of the association between anxious apprehension and anxious arousal. The association between these anxiety dimensions was weaker at higher levels of perceived attentional control. Our findings suggest that the ACS-S is best represented as a single dimension of beliefs about attentional control which can be scored by totaling the items and that the ACS-S uniquely related to the internalizing symptoms we assessed. The findings also contribute to the understanding of how anxiety dimensions are influenced by perceived attentional control.


Attentional control Bifactor Depression Anxious arousal Anxious apprehension 


Compliance with Ethical Standards

Conflict of Interest

Matt R. Judah, Kevin G. Saulnier, Nathan M. Hager, and Nicholas P. Allan declare no actual or potential conflicts of interest or sources of funding for this study.

Experiment Participants

All research procedures were approved by the Institutional Review Board at Ohio University.

Informed Consent

All subjects provided informed consent prior to participation in the study.


  1. Allan, N. P., Albanese, B. J., Short, N. A., Raines, A. M., & Schmidt, N. B. (2015). Support for the general and specific bifactor model factors of anxiety sensitivity. Personality and Individual Differences, 74, 78–83. Scholar
  2. Arditte, K. A., Cek, D., Shaw, A. M., & Timpano, K. R. (2016). The importance of assessing clinical phenomena in mechanical Turk research. Psychological Assessment, 28, 684–691. Scholar
  3. Armstrong, T., Zald, D. H., & Olatunji, B. O. (2011). Attentional control in OCD and GAD: Specificity and associations with core cognitive symptoms. Behaviour Research and Therapy, 49, 756–762. Scholar
  4. Bardeen, J. R., & Fergus, T. A. (2016). Emotional distress intolerance, experiential avoidance, and anxiety sensiivity: The buffering effect of attentional control on associations with posttraumatic stress symptoms. Journal of Psychopathology and Behavioral Assessment, 38, 320–329. Scholar
  5. Bardeen, J. R., Fergus, T. A., & Orcutt, H. K. (2015). Attentional control as a prospective predictor of posttraumatic stress symptomatology. Personality and Individual Differences, 81, 124–128. Scholar
  6. Behrend, T. S., Sharek, D. J., Meade, A. W., & Wiebe, E. N. (2011). The viability of crowdsourcing for survey research. Behavior Research Methods, 43, 800–813. Scholar
  7. Bentler, P. (1990). Comparative fit indices in structural models. Psychological Bulletin, 107(2), 238–246. Scholar
  8. Browne, M., W., & Cudeck, R. (1992). Alternative ways of assessing model fit. Sociological Methods & Research, 21, 230–258.
  9. Casler, K., Bickel, L., & Hackett, E. (2013). Separate but equal? A comparison of participants data gathered via Amazon's MTurk, social media, and face-to-face behavioral testing. Computers in Human Behavior, 29, 2156–2160. Scholar
  10. Cox, R. C., Cole, D. A., Kramer, E. L., & Olatunji, B. O. (2018). Prospective associations between sleep disturbance and repetitive negative thinking: The mediating roles of focusing and shifting attentional control. Behavior Therapy, 49, 21–31. Scholar
  11. Derryberry, D., & Reed, M. A. (2002). Anxiety-related attentional biases and their regulation by attentional control. Journal of Abnormal Psychology, 111, 225–236. Scholar
  12. DiStefano, C., & Motl, R. W. (2006). Further investigating method effects associated with negatively worded items on self-report surveys. Structural Equation Modeling, 13, 440–464. Scholar
  13. Elhai, J. D., Contractor, A. A., Tamburrino, M., Fine, T. H., Prescott, M. R., Shirley, E., . . . Calabrese, J. R. (2012). The factor structure of major depression symptoms: A test of four competing models using the patient health Questionnaire-9. Psychiatry Research, 199, 169–173. CrossRefGoogle Scholar
  14. Eysenck, M. W., Derakshan, N., Santos, R., & Calvo, M. G. (2007). Anxiety and cognitive performance: Attentional control theory. Emotion, 7, 336–353.
  15. Friedman, N. P., & Miyake, A. (2017). Unity and diversity of executive functions: Individual differences as a window on cognitive structure. Cortex, 86, 186–204. Scholar
  16. Goodman, J. K., Cryder, C. E., & Cheema, A. (2013). Data collection in a flat world: The strengths and weaknesses of mechanical Turk samples. Journal of Behavioral Decision Making, 26, 213–224. Scholar
  17. Hayes, A. F. (2017a). Introduction to mediation, moderation, and conditional Process analysis: A regression-based approach (2nd ed.). New York, NY: The Guilford Press.Google Scholar
  18. Hayes, A. F. (2017b). The PROCESS macro for SPSS and SAS (version 3.2).Google Scholar
  19. Heller, W., & Nitschke, J. B. (1998). The puzzle of regional brain activity in and anxiety: The importance of subtypes and comorbidity. Cognition and Emotion, 12, 421–447. Scholar
  20. Houck, P. R., Spiegel, D. A., Shear, M. K., & Rucci, P. (2002). Reliability of the self-report version of the panic disorder severity scale. Depression and Anxiety, 15, 183–185. Scholar
  21. Hsu, K. J., Beard, C., Rifkin, L., Dillon, D. G., Pizzagalli, D. A., & Bjorgvinsson, T. (2015). Transdiagnostic mechanisms in depression and anxiety: The role of rumination and attenttional control. Journal of Affective Disorders, 188, 22–27. Scholar
  22. Hu, L., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal, 6, 1–55. Scholar
  23. Johnson, P. O., & Neyman, J. (1936). Tests of certain linear hypotheses and their application to some educational problems. Statistical Research Memoirs, 1, 57–93.Google Scholar
  24. Judah, M. R., Grant, D. M., Mills, A. C., & Lechner, W. V. (2014). Factor structure and validation of the attentional control scale. Cognition and Emotion, 28, 433–451. Scholar
  25. Kaiser, H. F. (1960). The application of electronic computers to factor analysis. Educational and Psychological Measurement, 20, 141–151.CrossRefGoogle Scholar
  26. Kocalevent, R.-D., Hinz, A., & Brähler, E. (2013). Standardization of the depression screener patient health questionnaire (PHQ-9) in the general population. General Hospital Psychiatry, 35, 551–555. Scholar
  27. Kroenke, K., Spitzer, R., & Williams, W. (2001). The PHQ-9: Validity of a brief depression severity measure. Journal of General Internal Medicine, 16, 606–616. Scholar
  28. Macatee, R. J., Allan, N. P., Gajewska, A., Norr, A. M., Raines, A. M., Albanese, B. J., . . . Cougle, J. R. (2016). Shared and distinct cognitive/affective mechanisms in intrusive cognition: An examination of worry and obsessions. Cognitive Therapy and Research, 40, 80–91. CrossRefGoogle Scholar
  29. MacCallum, R. C., Browne, M., & Sugawara, H. M. (1996). Power analysis and determination of sample size for covariance structure modeling. Psychological Methods, I, 1(1), 130–149. Scholar
  30. Manea, L., Gilbody, S., & McMillan, D. (2012). Optimal cut-off score for diagnosing depression with the patient health questionnaire (PHQ-9): A meta-analysis. Canadian Medical Assocaition Journal, 184, E191–E196. Scholar
  31. Manea, L., Gilbody, S., & McMillan, D. (2015). A diagnostic meta-analysis of the patient health Questionnaire-9 (PHQ-9) algorithm scoring method as a screen for depression. General Hospital Psychiatry, 37, 67–75. Scholar
  32. Martin, A., Rief, W., Klaiberg, A., & Braehler, E. (2006). Validity of the brief patient health questionnaire mood scale (PHQ-9) in the general population. General Hospital Psychiatry, 28, 71–77. Scholar
  33. Meyer, T. J., Miller, M. L., Metzger, R. L., & Borkovec, T. D. (1990). Development and validation of the Penn State worry questionnaire. Behaviour Research and Therapy, 28, 487–495. Scholar
  34. Mills, A. C., Grant, D. M., Judah, M. R., White, E. J., Taylor, D. L., & Frosio, K. E. (2016). Trait attentional control influences the relationship between repetitive negative thinking and psychopathology symptoms. Psychiatry Research, 238, 277–283. Scholar
  35. Muthén, L. K., & Muthén, B. O. (1998–2017). Mplus User’s Guide (Seventh ed.). CA: Los Angeles.Google Scholar
  36. Newman, M. G., Holmes, M., Zuellig, A. R., Kachin, K. E., & Behar, E. (2006). The reliability and validity of the Panic Disorder Self-Report: A new diagnostic screening measure of panic disorder. Psychological Assessment, 18, 49-61.Google Scholar
  37. Nitschke, J., Heller, W., Imig, J., McDonald, R., & Miller, G. (2001). Distinguishing dimensions of anxiety and depression. Cognitive Therapy and Research, 25, 1–22. Scholar
  38. Norman, D. A., & Shallice, T. (1986). Attention to action. In R. J. Davidson, G. E. Schwartz, & D. Shapiro (Eds.), Consciousness and Self-Regulation. Springer; Boston (pp. 1-18).Google Scholar
  39. Olafsson, R. P., Smari, J., Guomundsdottir, F., Olafsdottir, G., Haroardottir, H. L., & Einarsson, S. M. (2011). Self reported attentional control with the attentional control scale: Factor structure and relationship with symptoms of anxiety and depression. Journal of Anxiety Disorders, 25, 777–782. Scholar
  40. Paolacci, G., Chandler, J., & Ipeirotis, P. G. (2010). Running experiments on Amazon mechanical Turk. Judgment and Decision making, 5.Google Scholar
  41. Preacher, K. J., Curran, P. J., & Bauer, D. J. (2006). Computational tools for probing interactions in multiple linear regression, multilevel modeling, and latent curve analysis. Journal of Educational and Behavioral Statistics, 31, 437–448.CrossRefGoogle Scholar
  42. Posner, M. I., & Peterson, S. E. (1990). The attention system of the human brain. Annual Review of Neuroscience, 13, 25-42.
  43. Putman, P., Verkuil, B., Arias-Garcia, E., Pantazi, I., & van Schie, C. (2014). EEG theta/beta ratio as a potential biomarker for attentional control and resilience against deleterious effects of stress on attention. Cognitive, Affective, & Behavioral Neuroscience, 14, 782–791. Scholar
  44. Quigley, L., Wright, C. A., Dobson, K. S., & Sears, C. R. (2017). Measuring attentional control ability or beliefs? Evaluation of the factor structure and convergent validity of the attentional control scale. Journal of Psychopathology and Behavioral Assessment, 39, 742–754. Scholar
  45. Raines, A. M., Oglesby, M. E., Allan, N. P., Short, N. A., & Schmidt, N. B. (2016). Understanding DSM-5 hoarding disorder: A triple vulnerability model. Psychiatry, 79, 120–129. Scholar
  46. Raykov, T. (2012). Scale construction and development using structural equation modeling. In R.H. Holyle (Ed.),Handbook of structural equation modeling. New York: Guilford Press. (pp. 472-492).Google Scholar
  47. Reinholdt-Dunne, M. L., Mogg, K., & Bradley, B. P. (2009). Effects of anxiety and attention control on processing pictorial and linguistic emotional information. Behaviour Research and Therapy, 47, 410–417. Scholar
  48. Reinholdt-Dunne, M. L., Mogg, K., & Bradley, B. (2013). Attention control: Relationships between self-report and behavioural measures, and symptoms of anxiety and depression. Cognition and Emotion, 27, 430–440. Scholar
  49. Reise, S. P. (2012). The rediscovery of Bifactor measurement models. Multivariate Behavioral Research, 47(5), 667–696. Scholar
  50. Reise, S. P., Moore, T. M., & Haviland, M. G. (2010). Bifactor models and rotations: Exploring the extent to which multidimensional data yield univocal scale scores. Journal of Personality Assessment, 92, 544–559. Scholar
  51. Shapiro, D. N., Chandler, J., & Mueller, P. A. (2013). Using mechanical Turk to study clinical populations. Clinical Psychological Science, 1, 213–220. Scholar
  52. Shear, M. K., Brown, T. A., Barlow, D. H., Money, R., Sholomskas, D. E., Woods, S. W., . . . Papp, L. A. (1997). Multicenter collaborative panic disorder severity scale. American Journal of Psychiatry, 154, 1571–1575. CrossRefGoogle Scholar
  53. Shear, M. K., Rucci, P., Williams, J., Frank, E., Grochocinski, V., Bilt, J. V., . . . Wang, T. (2001). Reliability and validity of the panic disorder severity scale: Replication and extension. Journal of Psychiatric Research, 35, 293–296. CrossRefGoogle Scholar
  54. Spitzer, R. L., Williams, J. B. W., Kroenke, K., Linzer, M., Verloin, F., Hahn, S. R., . . . Johnson, J. G. (1994). Utility of a new procedure for diagnosing mental disorders in primary care. Journal of the American Medical Association, 272, 1749–1756. CrossRefGoogle Scholar
  55. Tabachnick, B. G., & Fidell, L. S. (2007). Using multivariate statistics. Boston, MA: Pearson/Allyn & Bacon.Google Scholar
  56. Topper, M., Emmelkamp, P. M. G., Watkins, E., & Ehring, T. (2014). Development and assessment of brief versions of the Penn State worry questionnaire and the ruminative response scale. British Journal of Clinical Psychology, 53, 402–421. Scholar
  57. Vasey, M. W., Chriki, L., & Toh, G. Y. (2017). Cognitive control and anxious arousal in worry and generalized anxiety: An initial test of an integrative model. Cogntiive Therapy and Research, 41, 155–169. Scholar
  58. Velicer, W. F., & Fava, J. L. (1998). Affects of variable and subject sampling on factor pattern recovery. Psychological Methods, 3, 231–251. Scholar
  59. Wiersema, J. R., & Roeyers, H. (2009). ERP correlates of effortful control in children with varying levels of ADHD symptoms. Journal of Abnormal Child Psychology, 37, 327–336. Scholar
  60. Williams, P. G., Rau, H. K., Suchy, Y., Thorgusen, S. R., & Smith, T. W. (2017). On the validity of self-report assessment of cognitive abilities: Attentional control scale associations with cognitive performance, emotional adjustment, and personality. Psychological Assessment, 29, 519–530.
  61. Wuyek, L. A., Antony, M. M., & McCabe, R. E. (2011). Psychometric properties of the panic disorder severity scale: Clinician-administered and self-report versions. Clinical Psychology & Psychotherapy, 18, 234–243. Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2019

Authors and Affiliations

  1. 1.Department of Psychology, Virginia Consortium Program in Clinical PsychologyOld Dominion UniversityNorfolkUSA
  2. 2.Virginia Consortium Program in Clinical PsychologyNorfolkUSA
  3. 3.Ohio UniversityAthensUSA

Personalised recommendations