Abstract
Executive functioning (EF) skills are vital for academic success. Along with the recent explosion of interventions targeting these skills comes the need for affordable, efficient, and ecologically valid measures for planning and tailoring interventions and monitoring outcomes. The current study describes the refinement and initial psychometric evaluation of the Executive Skills Questionnaire-Revised (ESQ-R), a self-report EF rating scale that integrates current scientific understanding of core EF processes with an ecologically valid understanding of EF skills (ESs) that is directly applicable to academic contexts and tasks and tied to available interventions. We describe reduction of an initial 61-item pool to a final 25-item version using a series of exploratory and confirmatory factor analyses with 347 participants. Psychometric evidence for the 25-item version is promising, with excellent internal consistency (alpha = .91), adequate test-retest reliability for a small subsample (.70 with no effects of time delay on score variability), moderate correlations with other EF rating scales (.56–.74) and psychological symptom scales (.38–.55), and a significant correlation with academic engagement (− .40).
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Ajilchi, B., & Nejati, V. (2017). Executive functions in students with depression, anxiety, and stress symptoms. Basic and Clinical Neuroscience, 8(3), 223–232. https://doi.org/10.18869/nirp.bcn.8.3.223.
Baars, M. A. E., Nije Bijvank, M., Tonnaer, G. H., & Jolles, J. (2015). Self-report measures of executive functioning are a determinant of academic performance in first-year students at a university of applied sciences. Frontiers in Psychology, 6, 1131. https://doi.org/10.3389/fpsyg.2015.01131.
Barkley, R. A. (1991). The ecological validity of laboratory and analogue assessment methods of ADHD symptoms. Journal of Abnormal Child Psychology, 19(2), 149–178. https://doi.org/10.1007/BF00909976.
Barkley, R. A. (2011). Barkley deficits in executive functioning scale (BDEFS). New York, NY: Guilford Press.
Barkley, R. A. (2012). Executive functions: what they are, how they work, and why they evolved. In Choice (Vol. 50, p. 762). https://doi.org/10.5860/CHOICE.50-2366.
Best, J. R., Miller, P. H., & Naglieri, J. A. (2011). Relations between executive function and academic achievement from ages 5 to 17 in a large, representative national sample. Learning and Individual Differences, 21(4), 327–336. https://doi.org/10.1016/j.lindif.2011.01.007.
Biederman, J., Petty, C. R., Fried, R., Doyle, A. E., Mick, E., Aleardi, M., Monuteaux, M. C., Seidman, L. J., Spencer, T., Faneuil, A. R., Holmes, L., & Faraone, S. V. (2008). Utility of an abbreviated questionnaire to identify individuals with ADHD at risk for functional impairments. Journal of Psychiatric Research, 42(4), 304–310. https://doi.org/10.1016/j.jpsychires.2006.12.004.
Chan, R. C. K., Shum, D., Toulopoulou, T., & Chen, E. Y. H. (2008). Assessment of executive functions: Review of instruments and identification of critical issues. Archives of Clinical Neuropsychology, 23(2), 201–216. https://doi.org/10.1016/j.acn.2007.08.010.
Cohen, S., Kamarck, T., & Mermelstein, R. (1983). A global measure of perceived stress. Journal of Health and Social Behavior, 24(4), 385–396. https://doi.org/10.2307/2136404.
Collins, A., Koechlin, E., Simon, H., Kahneman, D., Tversky, A., Cohen, J. D., et al. (2012). Reasoning, learning, and creativity: Frontal lobe function and human decision-making. PLoS Biology, 10(3), e1001293. https://doi.org/10.1371/journal.pbio.1001293.
Crede, M., & Kuncel, N. R. (2008). Study habits, skills, and attitudes: The third pillar supporting collegiate academic performance. Perspectives on Psychological Science, 3(6), 425–454.
Credé, M., & Kuncel, N. R. (2008). Study habits, skills, and attitudes: The third pillar supporting collegiate academic performance. Perspectives on Psychological Science, 3(6), 425–453. https://doi.org/10.1111/j.1745-6924.2008.00089.x.
Dawson, P., & Guare, R. (2010). Executive skills in children and adolescents: A practical guide to assessment and intervention. Guilford Press.
Dawson, P., & Guare, R. (2012). Coaching students with executive skills deficits. New York, NY: Guilford Press.
Dawson, P., & Guare, R. (2018). Executive skills in children and adolescents: A practical guide to assessment and intervention (3rd ed.). New York: Guilford Press.
Dehn, M. (2008). Working memory and academic learning. Hoboken, NJ: John Wiley & Sons, Ltd..
Dempster, F. N. (1992). The rise and fall of the inhibitory mechanism: Toward a unified theory of cognitive development and aging. Developmental Review, 12(1), 45–75. https://doi.org/10.1016/0273-2297(92)90003-K.
DeVellis, R. F. (2017). Scale development. SAGE publications. https://doi.org/10.1017/CBO9781107415324.004
Diamond, A. (2014). Executive functions. Annual Review of Clinical Psychology, 64, 135–168. https://doi.org/10.1146/annurev-psych-113011-143750.Executive.
Engle, R. W. (2002). Working memory capacity as executive attention. Current Directions in Psychological Science, 11(1), 19–23.
Fok, C. C. T., & Henry, D. (2015). Increasing the sensitivity of measures to change. Prevention Science, 16(7), 978–986. https://doi.org/10.1007/s11121-015-0545-z.
Handelsman, M. M., Briggs, W. L., Sullivan, N., Towler, A., Handelsman, M. M., Briggs, W. L., et al. (2005). A measure of college student course engagement. Journal of Educational Research, 98(3), 184–192. https://doi.org/10.3200/JOER.98.3.184-192.
Holmbeck, G. N., & Devine, K. A. (2009). Editorial: An author’s checklist for measure development and validation manuscripts. Journal of Pediatric Psychology, 34(7), 691–696. https://doi.org/10.1093/jpepsy/jsp046.
Holst, Y., & Thorell, L. B. (2018). Adult executive functioning inventory (ADEXI): Validity, reliability, and relations to ADHD. International Journal of Methods in Psychiatric Research, 27(1). https://doi.org/10.1002/mpr.1567.
Isquith, P. K., Roth, R. M., & Gioia, G. (2013). Contribution of rating scales to the assessment of executive functions. Applied Neuropsychology: Child, 2(2), 125–132. https://doi.org/10.1080/21622965.2013.748389.
Kuncel, N. R., Crede, M., & Thomas, L. L. (2005). The validity of self-reported grade point averages, class ranks, and test scores: A meta-analysis and review of the literature. Review of Educational Research, 75(1), 63–82. https://doi.org/10.3102/00346543075001063.
Lezak, M. D. (1995). Neuropsychological assessment (3rd ed.). New York: Oxford University Press.
Lovibond, S. H., & Lovibond, P. F. (1995). Manual for the depression anxiety stress scales (2nd ed.). Sydney: Psychology Foundation.
Lunt, L., Bramham, J., Morris, R. G., Bullock, P. R., Selway, R. P., Xenitidis, K., & David, A. S. (2012). Prefrontal cortex dysfunction and ‘jumping to conclusions’: Bias or deficit? Journal of Neuropsychology, 6(1), 65–78. https://doi.org/10.1111/j.1748-6653.2011.02005.x.
Miyake, A., & Friedman, N. P. (2012). The nature and Organization of Individual Differences in executive functions: Four general conclusions. Current Directions in Psychological Science, 21(1), 8–14. https://doi.org/10.1177/0963721411429458.
Miyake, A., Friedman, N. P., Emerson, M. J., Witzki, A. H., Howerter, A., & Wager, T. D. (2000). The Unity and Diversity of executive functions and their contributions to complex “frontal lobe” tasks: A latent variable analysis. Cognitive Psychology, 41(1), 49–100. https://doi.org/10.1006/cogp.1999.0734.
Muthén, L. K., & Muthén, B. O. (2013). Mplus 7.11. Los Angeles, CA: Muthén & Muthén.
Petersen, R., Lavelle, E., & Guarino, A. J. (2006). The relationship between college students’ executive functioning and study strategies. Journal of College Reading and Learning, 36(2), 59–67. https://doi.org/10.1080/10790195.2006.10850188.
Rabin, L. A., Fogel, J., & Nutter-Upham, K. E. (2011). Academic procrastination in college students: The role of self-reported executive function. Journal of Clinical and Experimental Neuropsychology, 33(3), 344–357. https://doi.org/10.1080/13803395.2010.518597.
Reynolds, C. R., & Kamphaus, R. W. (2015). Behavior Assessment Systems for Children: Third Edition (BASC-3) (3rd ed.). Bloomington, MN: Pearson.
Roth, R. M. (2005). Behavior rating inventory of executive function—adult version. Lutz, Florida: Psychological Assessment Resources, Inc.
Samuels, W. E., Tournaki, N., Blackman, S., & Zilinski, C. (2016). Executive functioning predicts academic achievement in middle school: A four-year longitudinal study. The Journal of Educational Research, 109(5), 478–490. https://doi.org/10.1080/00220671.2014.979913.
Sinclair, S. J., Siefert, C. J., Slavin-Mulford, J. M., Stein, M. B., Renna, M., & Blais, M. A. (2012). Psychometric evaluation and normative data for the depression, anxiety, and stress Scales-21 (DASS-21) in a nonclinical sample of U.S. adults. Evaluation & the Health Professions, 35(3), 259–279. https://doi.org/10.1177/0163278711424282.
Spitzer, R. L., Kroenke, K., Williams, J. B. W., & Löwe, B. (2006). A brief measure for assessing generalized anxiety disorder: The GAD-7. Archives of Internal Medicine, 166(10), 1092–1097. https://doi.org/10.1001/archinte.166.10.1092.
Toplak, M. E., West, R. F., & Stanovich, K. E. (2013). Practitioner review: Do performance-based measures and ratings of executive function assess the same construct? Journal of Child Psychology and Psychiatry and Allied Disciplines, 54(2), 131–143. https://doi.org/10.1111/jcpp.12001.
Visser, P. S., Krosnick, J. O. N. A., Lavraws, P. J., Visser, P. S., Krosnick, J. O. N. A., & Lavraws, P. J. (2000). Survey research. In H. T. Reis & C. M. Judd (Eds.), Handbook of research methods in social psychology (pp. 223–252). New York: Cambridge University Press.
Weinstein, C. E., Palmer, D. R., & Schulte, A. C. (1987). Learning and study strategies inventory (LASSI). Clearwater, FL: H & H.
Wingo, J., Kalkut, E., Tuminello, E., Asconape, J., & Han, S. D. (2013). Executive functions, depressive symptoms, and college adjustment in women. Applied Neuropsychology, 20(2), 136–144. https://doi.org/10.1080/09084282.2012.670154.
Zhang, Y., Gan, Y., Cham, H., Wang, M.-T., Willett, J. B., Eccles, J. S., et al. (2012). School engagement trajectories and their differential predictive relations to dropout. Journal of Adolescence, 74(4), 274–283. https://doi.org/10.1002/pits.
Funding
The authors received no external funding for this study.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of Interest
Julia Englund Strait, Ph.D., declares that she has no conflict of interest.
Peg Dawson, Ed.D., is a co-author of the original ESQ rating scales and the books where they appear.
Christine AP Walther, Ph.D., declares that she has no conflict of interest.
Gerald Gill Strait, Ph.D., declares that he has no conflict of interest.
Amy K. Barton declares that she has no conflict of interest.
Maryellen Brunson McClain, Ph.D., declares that she has no conflict of interest.
Ethical Approval
All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.
Informed Consent
Informed consent and, when appropriate for age, parental consent and child assent were obtained for all individual participants included in the study in procedures approved by the university IRB (Committee for the Protection of Human Subjects or CPHS).
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Strait, J.E., Dawson, P., Walther, C.A.P. et al. Refinement and Psychometric Evaluation of the Executive Skills Questionnaire-Revised. Contemp School Psychol 24, 378–388 (2020). https://doi.org/10.1007/s40688-018-00224-x
Published:
Issue Date:
DOI: https://doi.org/10.1007/s40688-018-00224-x