Advertisement

Refinement and Psychometric Evaluation of the Executive Skills Questionnaire-Revised

  • Julia Englund StraitEmail author
  • Peg Dawson
  • Christine A. P. Walther
  • Gerald Gill Strait
  • Amy K. Barton
  • Maryellen Brunson McClain
Article
  • 6 Downloads

Abstract

Executive functioning (EF) skills are vital for academic success. Along with the recent explosion of interventions targeting these skills comes the need for affordable, efficient, and ecologically valid measures for planning and tailoring interventions and monitoring outcomes. The current study describes the refinement and initial psychometric evaluation of the Executive Skills Questionnaire-Revised (ESQ-R), a self-report EF rating scale that integrates current scientific understanding of core EF processes with an ecologically valid understanding of EF skills (ESs) that is directly applicable to academic contexts and tasks and tied to available interventions. We describe reduction of an initial 61-item pool to a final 25-item version using a series of exploratory and confirmatory factor analyses with 347 participants. Psychometric evidence for the 25-item version is promising, with excellent internal consistency (alpha = .91), adequate test-retest reliability for a small subsample (.70 with no effects of time delay on score variability), moderate correlations with other EF rating scales (.56–.74) and psychological symptom scales (.38–.55), and a significant correlation with academic engagement (− .40).

Keywords

Executive functioning Assessment Psychometrics Intervention 

Notes

Funding

The authors received no external funding for this study.

Compliance with Ethical Standards

Conflict of Interest

Julia Englund Strait, Ph.D., declares that she has no conflict of interest.

Peg Dawson, Ed.D., is a co-author of the original ESQ rating scales and the books where they appear.

Christine AP Walther, Ph.D., declares that she has no conflict of interest.

Gerald Gill Strait, Ph.D., declares that he has no conflict of interest.

Amy K. Barton declares that she has no conflict of interest.

Maryellen Brunson McClain, Ph.D., declares that she has no conflict of interest.

Ethical Approval

All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.

Informed Consent

Informed consent and, when appropriate for age, parental consent and child assent were obtained for all individual participants included in the study in procedures approved by the university IRB (Committee for the Protection of Human Subjects or CPHS).

References

  1. Ajilchi, B., & Nejati, V. (2017). Executive functions in students with depression, anxiety, and stress symptoms. Basic and Clinical Neuroscience, 8(3), 223–232.  https://doi.org/10.18869/nirp.bcn.8.3.223.CrossRefGoogle Scholar
  2. Baars, M. A. E., Nije Bijvank, M., Tonnaer, G. H., & Jolles, J. (2015). Self-report measures of executive functioning are a determinant of academic performance in first-year students at a university of applied sciences. Frontiers in Psychology, 6, 1131.  https://doi.org/10.3389/fpsyg.2015.01131.CrossRefGoogle Scholar
  3. Barkley, R. A. (1991). The ecological validity of laboratory and analogue assessment methods of ADHD symptoms. Journal of Abnormal Child Psychology, 19(2), 149–178.  https://doi.org/10.1007/BF00909976.CrossRefGoogle Scholar
  4. Barkley, R. A. (2011). Barkley deficits in executive functioning scale (BDEFS). New York, NY: Guilford Press.Google Scholar
  5. Barkley, R. A. (2012). Executive functions: what they are, how they work, and why they evolved. In Choice (Vol. 50, p. 762).  https://doi.org/10.5860/CHOICE.50-2366.
  6. Best, J. R., Miller, P. H., & Naglieri, J. A. (2011). Relations between executive function and academic achievement from ages 5 to 17 in a large, representative national sample. Learning and Individual Differences, 21(4), 327–336.  https://doi.org/10.1016/j.lindif.2011.01.007.CrossRefGoogle Scholar
  7. Biederman, J., Petty, C. R., Fried, R., Doyle, A. E., Mick, E., Aleardi, M., Monuteaux, M. C., Seidman, L. J., Spencer, T., Faneuil, A. R., Holmes, L., & Faraone, S. V. (2008). Utility of an abbreviated questionnaire to identify individuals with ADHD at risk for functional impairments. Journal of Psychiatric Research, 42(4), 304–310.  https://doi.org/10.1016/j.jpsychires.2006.12.004.CrossRefGoogle Scholar
  8. Chan, R. C. K., Shum, D., Toulopoulou, T., & Chen, E. Y. H. (2008). Assessment of executive functions: Review of instruments and identification of critical issues. Archives of Clinical Neuropsychology, 23(2), 201–216.  https://doi.org/10.1016/j.acn.2007.08.010.CrossRefGoogle Scholar
  9. Cohen, S., Kamarck, T., & Mermelstein, R. (1983). A global measure of perceived stress. Journal of Health and Social Behavior, 24(4), 385–396.  https://doi.org/10.2307/2136404.CrossRefGoogle Scholar
  10. Collins, A., Koechlin, E., Simon, H., Kahneman, D., Tversky, A., Cohen, J. D., et al. (2012). Reasoning, learning, and creativity: Frontal lobe function and human decision-making. PLoS Biology, 10(3), e1001293.  https://doi.org/10.1371/journal.pbio.1001293.CrossRefGoogle Scholar
  11. Crede, M., & Kuncel, N. R. (2008). Study habits, skills, and attitudes: The third pillar supporting collegiate academic performance. Perspectives on Psychological Science, 3(6), 425–454.CrossRefGoogle Scholar
  12. Credé, M., & Kuncel, N. R. (2008). Study habits, skills, and attitudes: The third pillar supporting collegiate academic performance. Perspectives on Psychological Science, 3(6), 425–453.  https://doi.org/10.1111/j.1745-6924.2008.00089.x.CrossRefGoogle Scholar
  13. Dawson, P., & Guare, R. (2010). Executive skills in children and adolescents: A practical guide to assessment and intervention. Guilford Press.Google Scholar
  14. Dawson, P., & Guare, R. (2012). Coaching students with executive skills deficits. New York, NY: Guilford Press.Google Scholar
  15. Dawson, P., & Guare, R. (2018). Executive skills in children and adolescents: A practical guide to assessment and intervention (3rd ed.). New York: Guilford Press.Google Scholar
  16. Dehn, M. (2008). Working memory and academic learning. Hoboken, NJ: John Wiley & Sons, Ltd..Google Scholar
  17. Dempster, F. N. (1992). The rise and fall of the inhibitory mechanism: Toward a unified theory of cognitive development and aging. Developmental Review, 12(1), 45–75.  https://doi.org/10.1016/0273-2297(92)90003-K.CrossRefGoogle Scholar
  18. DeVellis, R. F. (2017). Scale development. SAGE publications.  https://doi.org/10.1017/CBO9781107415324.004
  19. Diamond, A. (2014). Executive functions. Annual Review of Clinical Psychology, 64, 135–168.  https://doi.org/10.1146/annurev-psych-113011-143750.Executive.CrossRefGoogle Scholar
  20. Engle, R. W. (2002). Working memory capacity as executive attention. Current Directions in Psychological Science, 11(1), 19–23.CrossRefGoogle Scholar
  21. Fok, C. C. T., & Henry, D. (2015). Increasing the sensitivity of measures to change. Prevention Science, 16(7), 978–986.  https://doi.org/10.1007/s11121-015-0545-z.CrossRefGoogle Scholar
  22. Handelsman, M. M., Briggs, W. L., Sullivan, N., Towler, A., Handelsman, M. M., Briggs, W. L., et al. (2005). A measure of college student course engagement. Journal of Educational Research, 98(3), 184–192.  https://doi.org/10.3200/JOER.98.3.184-192.CrossRefGoogle Scholar
  23. Holmbeck, G. N., & Devine, K. A. (2009). Editorial: An author’s checklist for measure development and validation manuscripts. Journal of Pediatric Psychology, 34(7), 691–696.  https://doi.org/10.1093/jpepsy/jsp046.CrossRefGoogle Scholar
  24. Holst, Y., & Thorell, L. B. (2018). Adult executive functioning inventory (ADEXI): Validity, reliability, and relations to ADHD. International Journal of Methods in Psychiatric Research, 27(1).  https://doi.org/10.1002/mpr.1567.
  25. Isquith, P. K., Roth, R. M., & Gioia, G. (2013). Contribution of rating scales to the assessment of executive functions. Applied Neuropsychology: Child, 2(2), 125–132.  https://doi.org/10.1080/21622965.2013.748389.CrossRefGoogle Scholar
  26. Kuncel, N. R., Crede, M., & Thomas, L. L. (2005). The validity of self-reported grade point averages, class ranks, and test scores: A meta-analysis and review of the literature. Review of Educational Research, 75(1), 63–82.  https://doi.org/10.3102/00346543075001063.CrossRefGoogle Scholar
  27. Lezak, M. D. (1995). Neuropsychological assessment (3rd ed.). New York: Oxford University Press.Google Scholar
  28. Lovibond, S. H., & Lovibond, P. F. (1995). Manual for the depression anxiety stress scales (2nd ed.). Sydney: Psychology Foundation.Google Scholar
  29. Lunt, L., Bramham, J., Morris, R. G., Bullock, P. R., Selway, R. P., Xenitidis, K., & David, A. S. (2012). Prefrontal cortex dysfunction and ‘jumping to conclusions’: Bias or deficit? Journal of Neuropsychology, 6(1), 65–78.  https://doi.org/10.1111/j.1748-6653.2011.02005.x.CrossRefGoogle Scholar
  30. Miyake, A., & Friedman, N. P. (2012). The nature and Organization of Individual Differences in executive functions: Four general conclusions. Current Directions in Psychological Science, 21(1), 8–14.  https://doi.org/10.1177/0963721411429458.CrossRefGoogle Scholar
  31. Miyake, A., Friedman, N. P., Emerson, M. J., Witzki, A. H., Howerter, A., & Wager, T. D. (2000). The Unity and Diversity of executive functions and their contributions to complex “frontal lobe” tasks: A latent variable analysis. Cognitive Psychology, 41(1), 49–100.  https://doi.org/10.1006/cogp.1999.0734.CrossRefGoogle Scholar
  32. Muthén, L. K., & Muthén, B. O. (2013). Mplus 7.11. Los Angeles, CA: Muthén & Muthén.Google Scholar
  33. Petersen, R., Lavelle, E., & Guarino, A. J. (2006). The relationship between college students’ executive functioning and study strategies. Journal of College Reading and Learning, 36(2), 59–67.  https://doi.org/10.1080/10790195.2006.10850188.CrossRefGoogle Scholar
  34. Rabin, L. A., Fogel, J., & Nutter-Upham, K. E. (2011). Academic procrastination in college students: The role of self-reported executive function. Journal of Clinical and Experimental Neuropsychology, 33(3), 344–357.  https://doi.org/10.1080/13803395.2010.518597.CrossRefGoogle Scholar
  35. Reynolds, C. R., & Kamphaus, R. W. (2015). Behavior Assessment Systems for Children: Third Edition (BASC-3) (3rd ed.). Bloomington, MN: Pearson.Google Scholar
  36. Roth, R. M. (2005). Behavior rating inventory of executive function—adult version. Lutz, Florida: Psychological Assessment Resources, Inc.Google Scholar
  37. Samuels, W. E., Tournaki, N., Blackman, S., & Zilinski, C. (2016). Executive functioning predicts academic achievement in middle school: A four-year longitudinal study. The Journal of Educational Research, 109(5), 478–490.  https://doi.org/10.1080/00220671.2014.979913.CrossRefGoogle Scholar
  38. Sinclair, S. J., Siefert, C. J., Slavin-Mulford, J. M., Stein, M. B., Renna, M., & Blais, M. A. (2012). Psychometric evaluation and normative data for the depression, anxiety, and stress Scales-21 (DASS-21) in a nonclinical sample of U.S. adults. Evaluation & the Health Professions, 35(3), 259–279.  https://doi.org/10.1177/0163278711424282.CrossRefGoogle Scholar
  39. Spitzer, R. L., Kroenke, K., Williams, J. B. W., & Löwe, B. (2006). A brief measure for assessing generalized anxiety disorder: The GAD-7. Archives of Internal Medicine, 166(10), 1092–1097.  https://doi.org/10.1001/archinte.166.10.1092.CrossRefGoogle Scholar
  40. Toplak, M. E., West, R. F., & Stanovich, K. E. (2013). Practitioner review: Do performance-based measures and ratings of executive function assess the same construct? Journal of Child Psychology and Psychiatry and Allied Disciplines, 54(2), 131–143.  https://doi.org/10.1111/jcpp.12001.CrossRefGoogle Scholar
  41. Visser, P. S., Krosnick, J. O. N. A., Lavraws, P. J., Visser, P. S., Krosnick, J. O. N. A., & Lavraws, P. J. (2000). Survey research. In H. T. Reis & C. M. Judd (Eds.), Handbook of research methods in social psychology (pp. 223–252). New York: Cambridge University Press.Google Scholar
  42. Weinstein, C. E., Palmer, D. R., & Schulte, A. C. (1987). Learning and study strategies inventory (LASSI). Clearwater, FL: H & H.Google Scholar
  43. Wingo, J., Kalkut, E., Tuminello, E., Asconape, J., & Han, S. D. (2013). Executive functions, depressive symptoms, and college adjustment in women. Applied Neuropsychology, 20(2), 136–144.  https://doi.org/10.1080/09084282.2012.670154.CrossRefGoogle Scholar
  44. Zhang, Y., Gan, Y., Cham, H., Wang, M.-T., Willett, J. B., Eccles, J. S., et al. (2012). School engagement trajectories and their differential predictive relations to dropout. Journal of Adolescence, 74(4), 274–283.  https://doi.org/10.1002/pits.Google Scholar

Copyright information

© California Association of School Psychologists 2019

Authors and Affiliations

  1. 1.Department of Clinical, Health, and Applied SciencesUniversity of Houston-Clear LakeHoustonUSA
  2. 2.Center for Learning and Attention DisordersPortsmouthUSA
  3. 3.Department of PsychologyUniversity of Houston-Clear LakeHoustonUSA
  4. 4.Department of Psychological, Health, and Learning SciencesUniversity of HoustonHoustonUSA
  5. 5.Department of PsychologyUtah State UniversityLoganUSA

Personalised recommendations