Reliability and Validity of the Thinking Skills Inventory, a Screening Tool for Cross-Diagnostic Skill Deficits Underlying Youth Behavioral Challenges
Deficits in a range of skill domains (including executive functioning, emotion regulation, social cognition and language/communication) are associated with disrupted youth behavior and functioning across mental health diagnoses. The identification of skill deficits are important for effective treatment planning, particularly for personalized interventions. While there are multiple ways to assess these skills, parent/caregiver reports represent an important information source. To date, no single, brief measure has been developed that gathers parent/caregiver ratings across this range of constructs. We have developed a short caregiver-report questionnaire (the Thinking Skills Inventory; TSI), to screen for skill deficits. Here, we examine the reliability and validity of this rating scale in 384 youth who were consecutively referred for neuropsychiatric evaluation. A primary caregiver completed the TSI as well as other established measures. Exploratory and confirmatory factor analyses support five subscales on the TSI: Attention and Working Memory, Language and Communication, Emotion Regulation, Cognitive Flexibility, and Social Thinking Skills. The subscales showed moderate to high internal consistency (Cronbach’s alphas range from 0.84 to 0.91). Correlations with established caregiver-report measures confirm their convergent and discriminant validity, and associations with multiple clinical diagnoses and cross-diagnostic aggressive behavior further support the utility of the scale for our intended purpose. In sum, this free, brief measure is a valid and reliable way to identify variation in skill domains relevant to a range of psychopathology. The TSI may be useful in youth mental health settings to assist with treatment planning and to inform referral for further evaluation.
KeywordsCross-diagnostic traits Child psychopathology Assessment Collaborative problem solving
This research was supported in part by the David Judah Fund, the Stanley Center for Psychiatric Research, and NIH R03MH106862 awarded to Dr. Alysa E. Doyle.
Compliance with Ethical Standards
Conflict of Interest
Drs. Pollastri, Wang, Hill, and Ablon work for Think:Kids, a non-profit program in the Department of Psychiatry at Massachusetts General Hospital, from which staff train and consult on Collaborative Problem Solving and use the Thinking Skills Inventory (TSI). The TSI is available at no cost to paid clients of Think:Kids as well as to the larger research and clinical communities.
This research was approved by the appropriate Institutional Review Board and all participants provided informed consent.
- Achenbach, T. M. (2009). ASEBA: Development, findings, theory and applications. Burlington, VT: University of Vermont Research Center for Children, Youth and Families.Google Scholar
- Allen, M. J., & Yen, W. M. (2002). Introduction to measurement theory. Long Grove: Waveland Press.Google Scholar
- American Psychiatric Association. (2000). Diagnostic and statistical manual of mental disorders (4th ed., Text Revision). Washington, DC: Author.Google Scholar
- American Psychiatric Association. (2013). Diagnostic and statistical manual of mental disorders (5th ed.). Washington, DC: Author.Google Scholar
- Browne, M. W., & Cudeck, R. (1993). Alternative ways of assessing model fit. In K. A. Bollen & S. L. Long (Eds.), Testing structural equation models (pp. 136–162). London: Sage Publications.Google Scholar
- Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Routledge.Google Scholar
- Dodge, K. A., Lansford, J. E., Burks, V. S., Bates, J. E., Pettit, G. S., Fontaine, R., & Price, J. M. (2003). Peer rejection and social information-processing factors in the development of aggressive behavior problems in children. Child Development, 74(2), 374–393.PubMedPubMedCentralCrossRefGoogle Scholar
- Doyle, A. E., Vuijk, P. J., Doty, N. D., McGrath, L. M., Willoughby, B. L., O’Donnell, E. H., et al. (2017). Cross-disorder cognitive impairments in youth referred for neuropsychiatric evaluation. Journal of the International Neuropsychological Society, 1–13.Google Scholar
- Gioia, G. A. (2000). BRIEF: Behavior rating inventory of executive function: Professional manual. Odessa: Psychological Assessment Resources.Google Scholar
- Gioia, G. A., Isquith, P. K., Guy, S. C., & Kenworthy, L. (2015). Behavior rating inventory of executive functions-2. Lutz: PAR.Google Scholar
- Greene, R. W. (1998). The explosive child: A new approach for understanding and parenting easily frustrated, chronically inflexible children. New York: Harper Collins.Google Scholar
- Greene, R. W., & Ablon, J. S. (2005). Treating explosive kids: The collaborative problem-solving approach. New York: Guilford Press.Google Scholar
- Harrison, P.L. & Oakland, T. (2003). Adaptive Behavior Assessment System – Second Edition, San Antonio: The Psychological Corporation.Google Scholar
- Hooper, D., Coughlan, J., & Mullen, M. (2008). Structural equation modeling: Guidelines for determining model fit. Electronic Journal of Business Research Methods, 6(1), 53–60.Google Scholar
- Kline, P. (2013). Handbook of psychological testing. London, England. Routledge.Google Scholar
- Nunnally, J. C. (1978). Psychometric theory. New York: McGraw-Hill.Google Scholar
- Podsakoff, P. M., MacKenzie, S. B., Lee, J.-Y., & Podsakoff, N. P. (2003). Common method biases in behavioral research: A critical review of the literature and recommended remedies. Journal of Applied Psychology, 88(5), 879–903. https://doi.org/10.1037/0021-9010.88.5.879.PubMedCrossRefGoogle Scholar
- Revelle, W. (2017). psych: Procedures for Personality and Psychological Research, Northwestern University, Evanston, Illinois, USA, https://CRAN.R-project.org/package=psychVersion = 1.7.8.
- RStudio Team. (2016). RStudio: integrated development for R. Boston, MA: RStudio, Inc. Available from http://www.rstudio.com/.
- Russell, D. (2002). In search of underlying dimensions: The use (and abuse) of factor analysis in personality and social psychology bulletin. Personality and Social Psychology Bulletin, 28(12), 1629–1646.Google Scholar
- Tabachnick, B. G., & Fidell, L. S. (2007). Using multivariate statistics (5th ed.). Boston: Allyn & Bacon/Pearson Education.Google Scholar
- Van Goozen, S. H., Cohen-Kettenis, P. T., Snoek, H., Matthys, W., Swaab-Barneveld, H., & Van Engeland, H. (2004). Executive functioning in children: A comparison of hospitalized ODD and ODD/ADHD children and normal controls. Journal of Child Psychology and Psychiatry, 45(2), 284–292.PubMedCrossRefGoogle Scholar