Skip to main content

Refinement and Psychometric Evaluation of the Executive Skills Questionnaire-Revised

Abstract

Executive functioning (EF) skills are vital for academic success. Along with the recent explosion of interventions targeting these skills comes the need for affordable, efficient, and ecologically valid measures for planning and tailoring interventions and monitoring outcomes. The current study describes the refinement and initial psychometric evaluation of the Executive Skills Questionnaire-Revised (ESQ-R), a self-report EF rating scale that integrates current scientific understanding of core EF processes with an ecologically valid understanding of EF skills (ESs) that is directly applicable to academic contexts and tasks and tied to available interventions. We describe reduction of an initial 61-item pool to a final 25-item version using a series of exploratory and confirmatory factor analyses with 347 participants. Psychometric evidence for the 25-item version is promising, with excellent internal consistency (alpha = .91), adequate test-retest reliability for a small subsample (.70 with no effects of time delay on score variability), moderate correlations with other EF rating scales (.56–.74) and psychological symptom scales (.38–.55), and a significant correlation with academic engagement (− .40).

This is a preview of subscription content, access via your institution.

References

  1. Ajilchi, B., & Nejati, V. (2017). Executive functions in students with depression, anxiety, and stress symptoms. Basic and Clinical Neuroscience, 8(3), 223–232. https://doi.org/10.18869/nirp.bcn.8.3.223.

    Article  PubMed  PubMed Central  Google Scholar 

  2. Baars, M. A. E., Nije Bijvank, M., Tonnaer, G. H., & Jolles, J. (2015). Self-report measures of executive functioning are a determinant of academic performance in first-year students at a university of applied sciences. Frontiers in Psychology, 6, 1131. https://doi.org/10.3389/fpsyg.2015.01131.

    Article  PubMed  PubMed Central  Google Scholar 

  3. Barkley, R. A. (1991). The ecological validity of laboratory and analogue assessment methods of ADHD symptoms. Journal of Abnormal Child Psychology, 19(2), 149–178. https://doi.org/10.1007/BF00909976.

    Article  PubMed  Google Scholar 

  4. Barkley, R. A. (2011). Barkley deficits in executive functioning scale (BDEFS). New York, NY: Guilford Press.

    Google Scholar 

  5. Barkley, R. A. (2012). Executive functions: what they are, how they work, and why they evolved. In Choice (Vol. 50, p. 762). https://doi.org/10.5860/CHOICE.50-2366.

  6. Best, J. R., Miller, P. H., & Naglieri, J. A. (2011). Relations between executive function and academic achievement from ages 5 to 17 in a large, representative national sample. Learning and Individual Differences, 21(4), 327–336. https://doi.org/10.1016/j.lindif.2011.01.007.

    Article  PubMed  PubMed Central  Google Scholar 

  7. Biederman, J., Petty, C. R., Fried, R., Doyle, A. E., Mick, E., Aleardi, M., Monuteaux, M. C., Seidman, L. J., Spencer, T., Faneuil, A. R., Holmes, L., & Faraone, S. V. (2008). Utility of an abbreviated questionnaire to identify individuals with ADHD at risk for functional impairments. Journal of Psychiatric Research, 42(4), 304–310. https://doi.org/10.1016/j.jpsychires.2006.12.004.

    Article  PubMed  Google Scholar 

  8. Chan, R. C. K., Shum, D., Toulopoulou, T., & Chen, E. Y. H. (2008). Assessment of executive functions: Review of instruments and identification of critical issues. Archives of Clinical Neuropsychology, 23(2), 201–216. https://doi.org/10.1016/j.acn.2007.08.010.

    Article  PubMed  Google Scholar 

  9. Cohen, S., Kamarck, T., & Mermelstein, R. (1983). A global measure of perceived stress. Journal of Health and Social Behavior, 24(4), 385–396. https://doi.org/10.2307/2136404.

    Article  PubMed  Google Scholar 

  10. Collins, A., Koechlin, E., Simon, H., Kahneman, D., Tversky, A., Cohen, J. D., et al. (2012). Reasoning, learning, and creativity: Frontal lobe function and human decision-making. PLoS Biology, 10(3), e1001293. https://doi.org/10.1371/journal.pbio.1001293.

    Article  PubMed  PubMed Central  Google Scholar 

  11. Crede, M., & Kuncel, N. R. (2008). Study habits, skills, and attitudes: The third pillar supporting collegiate academic performance. Perspectives on Psychological Science, 3(6), 425–454.

    Article  Google Scholar 

  12. Credé, M., & Kuncel, N. R. (2008). Study habits, skills, and attitudes: The third pillar supporting collegiate academic performance. Perspectives on Psychological Science, 3(6), 425–453. https://doi.org/10.1111/j.1745-6924.2008.00089.x.

    Article  PubMed  Google Scholar 

  13. Dawson, P., & Guare, R. (2010). Executive skills in children and adolescents: A practical guide to assessment and intervention. Guilford Press.

  14. Dawson, P., & Guare, R. (2012). Coaching students with executive skills deficits. New York, NY: Guilford Press.

    Google Scholar 

  15. Dawson, P., & Guare, R. (2018). Executive skills in children and adolescents: A practical guide to assessment and intervention (3rd ed.). New York: Guilford Press.

  16. Dehn, M. (2008). Working memory and academic learning. Hoboken, NJ: John Wiley & Sons, Ltd..

    Google Scholar 

  17. Dempster, F. N. (1992). The rise and fall of the inhibitory mechanism: Toward a unified theory of cognitive development and aging. Developmental Review, 12(1), 45–75. https://doi.org/10.1016/0273-2297(92)90003-K.

    Article  Google Scholar 

  18. DeVellis, R. F. (2017). Scale development. SAGE publications. https://doi.org/10.1017/CBO9781107415324.004

  19. Diamond, A. (2014). Executive functions. Annual Review of Clinical Psychology, 64, 135–168. https://doi.org/10.1146/annurev-psych-113011-143750.Executive.

    Article  Google Scholar 

  20. Engle, R. W. (2002). Working memory capacity as executive attention. Current Directions in Psychological Science, 11(1), 19–23.

    Article  Google Scholar 

  21. Fok, C. C. T., & Henry, D. (2015). Increasing the sensitivity of measures to change. Prevention Science, 16(7), 978–986. https://doi.org/10.1007/s11121-015-0545-z.

    Article  PubMed  PubMed Central  Google Scholar 

  22. Handelsman, M. M., Briggs, W. L., Sullivan, N., Towler, A., Handelsman, M. M., Briggs, W. L., et al. (2005). A measure of college student course engagement. Journal of Educational Research, 98(3), 184–192. https://doi.org/10.3200/JOER.98.3.184-192.

    Article  Google Scholar 

  23. Holmbeck, G. N., & Devine, K. A. (2009). Editorial: An author’s checklist for measure development and validation manuscripts. Journal of Pediatric Psychology, 34(7), 691–696. https://doi.org/10.1093/jpepsy/jsp046.

    Article  PubMed  PubMed Central  Google Scholar 

  24. Holst, Y., & Thorell, L. B. (2018). Adult executive functioning inventory (ADEXI): Validity, reliability, and relations to ADHD. International Journal of Methods in Psychiatric Research, 27(1). https://doi.org/10.1002/mpr.1567.

  25. Isquith, P. K., Roth, R. M., & Gioia, G. (2013). Contribution of rating scales to the assessment of executive functions. Applied Neuropsychology: Child, 2(2), 125–132. https://doi.org/10.1080/21622965.2013.748389.

    Article  Google Scholar 

  26. Kuncel, N. R., Crede, M., & Thomas, L. L. (2005). The validity of self-reported grade point averages, class ranks, and test scores: A meta-analysis and review of the literature. Review of Educational Research, 75(1), 63–82. https://doi.org/10.3102/00346543075001063.

    Article  Google Scholar 

  27. Lezak, M. D. (1995). Neuropsychological assessment (3rd ed.). New York: Oxford University Press.

    Google Scholar 

  28. Lovibond, S. H., & Lovibond, P. F. (1995). Manual for the depression anxiety stress scales (2nd ed.). Sydney: Psychology Foundation.

    Google Scholar 

  29. Lunt, L., Bramham, J., Morris, R. G., Bullock, P. R., Selway, R. P., Xenitidis, K., & David, A. S. (2012). Prefrontal cortex dysfunction and ‘jumping to conclusions’: Bias or deficit? Journal of Neuropsychology, 6(1), 65–78. https://doi.org/10.1111/j.1748-6653.2011.02005.x.

    Article  PubMed  Google Scholar 

  30. Miyake, A., & Friedman, N. P. (2012). The nature and Organization of Individual Differences in executive functions: Four general conclusions. Current Directions in Psychological Science, 21(1), 8–14. https://doi.org/10.1177/0963721411429458.

    Article  PubMed  PubMed Central  Google Scholar 

  31. Miyake, A., Friedman, N. P., Emerson, M. J., Witzki, A. H., Howerter, A., & Wager, T. D. (2000). The Unity and Diversity of executive functions and their contributions to complex “frontal lobe” tasks: A latent variable analysis. Cognitive Psychology, 41(1), 49–100. https://doi.org/10.1006/cogp.1999.0734.

    Article  PubMed  Google Scholar 

  32. Muthén, L. K., & Muthén, B. O. (2013). Mplus 7.11. Los Angeles, CA: Muthén & Muthén.

  33. Petersen, R., Lavelle, E., & Guarino, A. J. (2006). The relationship between college students’ executive functioning and study strategies. Journal of College Reading and Learning, 36(2), 59–67. https://doi.org/10.1080/10790195.2006.10850188.

    Article  Google Scholar 

  34. Rabin, L. A., Fogel, J., & Nutter-Upham, K. E. (2011). Academic procrastination in college students: The role of self-reported executive function. Journal of Clinical and Experimental Neuropsychology, 33(3), 344–357. https://doi.org/10.1080/13803395.2010.518597.

    Article  PubMed  Google Scholar 

  35. Reynolds, C. R., & Kamphaus, R. W. (2015). Behavior Assessment Systems for Children: Third Edition (BASC-3) (3rd ed.). Bloomington, MN: Pearson.

    Google Scholar 

  36. Roth, R. M. (2005). Behavior rating inventory of executive function—adult version. Lutz, Florida: Psychological Assessment Resources, Inc.

  37. Samuels, W. E., Tournaki, N., Blackman, S., & Zilinski, C. (2016). Executive functioning predicts academic achievement in middle school: A four-year longitudinal study. The Journal of Educational Research, 109(5), 478–490. https://doi.org/10.1080/00220671.2014.979913.

    Article  Google Scholar 

  38. Sinclair, S. J., Siefert, C. J., Slavin-Mulford, J. M., Stein, M. B., Renna, M., & Blais, M. A. (2012). Psychometric evaluation and normative data for the depression, anxiety, and stress Scales-21 (DASS-21) in a nonclinical sample of U.S. adults. Evaluation & the Health Professions, 35(3), 259–279. https://doi.org/10.1177/0163278711424282.

    Article  Google Scholar 

  39. Spitzer, R. L., Kroenke, K., Williams, J. B. W., & Löwe, B. (2006). A brief measure for assessing generalized anxiety disorder: The GAD-7. Archives of Internal Medicine, 166(10), 1092–1097. https://doi.org/10.1001/archinte.166.10.1092.

    Article  PubMed  Google Scholar 

  40. Toplak, M. E., West, R. F., & Stanovich, K. E. (2013). Practitioner review: Do performance-based measures and ratings of executive function assess the same construct? Journal of Child Psychology and Psychiatry and Allied Disciplines, 54(2), 131–143. https://doi.org/10.1111/jcpp.12001.

    Article  Google Scholar 

  41. Visser, P. S., Krosnick, J. O. N. A., Lavraws, P. J., Visser, P. S., Krosnick, J. O. N. A., & Lavraws, P. J. (2000). Survey research. In H. T. Reis & C. M. Judd (Eds.), Handbook of research methods in social psychology (pp. 223–252). New York: Cambridge University Press.

    Google Scholar 

  42. Weinstein, C. E., Palmer, D. R., & Schulte, A. C. (1987). Learning and study strategies inventory (LASSI). Clearwater, FL: H & H.

    Google Scholar 

  43. Wingo, J., Kalkut, E., Tuminello, E., Asconape, J., & Han, S. D. (2013). Executive functions, depressive symptoms, and college adjustment in women. Applied Neuropsychology, 20(2), 136–144. https://doi.org/10.1080/09084282.2012.670154.

    Article  PubMed  Google Scholar 

  44. Zhang, Y., Gan, Y., Cham, H., Wang, M.-T., Willett, J. B., Eccles, J. S., et al. (2012). School engagement trajectories and their differential predictive relations to dropout. Journal of Adolescence, 74(4), 274–283. https://doi.org/10.1002/pits.

    Article  Google Scholar 

Download references

Funding

The authors received no external funding for this study.

Author information

Affiliations

Authors

Corresponding author

Correspondence to Julia Englund Strait.

Ethics declarations

Conflict of Interest

Julia Englund Strait, Ph.D., declares that she has no conflict of interest.

Peg Dawson, Ed.D., is a co-author of the original ESQ rating scales and the books where they appear.

Christine AP Walther, Ph.D., declares that she has no conflict of interest.

Gerald Gill Strait, Ph.D., declares that he has no conflict of interest.

Amy K. Barton declares that she has no conflict of interest.

Maryellen Brunson McClain, Ph.D., declares that she has no conflict of interest.

Ethical Approval

All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.

Informed Consent

Informed consent and, when appropriate for age, parental consent and child assent were obtained for all individual participants included in the study in procedures approved by the university IRB (Committee for the Protection of Human Subjects or CPHS).

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Strait, J.E., Dawson, P., Walther, C.A.P. et al. Refinement and Psychometric Evaluation of the Executive Skills Questionnaire-Revised. Contemp School Psychol 24, 378–388 (2020). https://doi.org/10.1007/s40688-018-00224-x

Download citation

Keywords

  • Executive functioning
  • Assessment
  • Psychometrics
  • Intervention