Skip to main content

Advertisement

SpringerLink
Log in

Evaluating the Factor Validity of the Children’s Organizational Skills Scale in Youth with ADHD

  • Original Paper
  • Published:
School Mental Health Aims and scope Submit manuscript

Abstract

Children and adolescents with ADHD often have difficulties with organization, time management, and planning skills, and these skills are a common target of intervention. A limited array of tools for measuring these abilities in youth is available, and one of the most prominent measures is the Children’s Organizational Skills Scale (COSS). Although the COSS fills an important need, a replication of the COSS factor structure outside of initial measure development has not been conducted in any population. Given that the COSS is frequently used in ADHD research, the current study evaluated the factor structure of the parent-rated COSS in a sample (N = 619) of adolescents with ADHD. Results indicated that the original factor structure could be replicated, although the use of item parcels appeared to affect model fit statistics. An alternative bi-factor model was also tested that did not require the use of parcels, with results suggesting similar model fit in comparison with the original factor structure. Exploratory validity tests indicated that the domain-general factor of the bi-factor model appears related to broad executive functioning abilities.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

References

  • Abikoff, H., & Gallagher, R. (2009). COSS: Children’s organizational skills scales. North Tonawanda, NY: Multi-Health Systems.

    Google Scholar 

  • Abikoff, H., Nissley-Tsiopinis, J., Gallagher, R., Zambenedetti, M., Seyffert, M., Boorady, R., et al. (2009). Effects of MPH-OROS on the organizational, time management, and planning behaviors of children with ADHD. Journal of the American Academy of Child and Adolescent Psychiatry, 48(2), 166–175.

    Article  PubMed  Google Scholar 

  • American Psychiatric Association. (2013). Diagnostic and statistical manual of mental disorders (DSM-5). Arlington Country: American Psychiatric Publications.

    Book  Google Scholar 

  • Anesko, K. M., Schoiock, G., Ramirez, R., & Levine, F. M. (1987). The homework problem checklist: Assessing children’s homework difficulties. Behavioral Assessment, 9, 179–185.

    Google Scholar 

  • Bandalos, D. L. (2002). The effects of item parceling on goodness-of-fit and parameter estimate bias in structural equation modeling. Structural Equation Modeling, 9(1), 78–102.

    Article  Google Scholar 

  • Bandalos, D. L., & Finney, S. J. (2001). Item parceling issues in structural equation modeling. In G. A. Marcoulides & R. E. Schumacker (Eds.), New developments and techniques in structural equation modeling (pp. 269–296). Hillsdale, NJ: Erlbaum.

    Google Scholar 

  • Barbaresi, W. J., Katusic, S. K., Colligan, R. C., Weaver, A. L., & Jacobsen, S. J. (2007). Long-term school outcomes for children with attention-deficit/hyperactivity disorder: A population-based perspective. Journal of Developmental and Behavioral Pediatrics, 28(4), 265–273.

    Article  PubMed  Google Scholar 

  • Barkley, R. A. (1997). Attention-deficit/hyperactivity disorder, self-regulation, and time: Toward a more comprehensive theory. Journal of Developmental and Behavioral Pediatrics, 18(4), 271–279.

    Article  PubMed  Google Scholar 

  • Booster, G. D., DuPaul, G. J., Eiraldi, R., & Power, T. J. (2010). Functional impairments in children with ADHD: Unique effects of age and comorbid status. Journal of Attention Disorders. doi:10.1177/1087054710383239.

    PubMed  Google Scholar 

  • Conners, C. K. (2008). Conners (3rd ed.). Toronto, Ontario, Canada: Multi-Health Systems.

    Google Scholar 

  • DuPaul, G. J., & Langberg, J. L. (2014). Educational impairments in children with ADHD. In R. A. Barkley (Ed.), Attention-deficit hyperactivity disorder (4th Ed) A handbook for diagnosis and treatment. New York: Guilford Press.

    Google Scholar 

  • Evans, S. W., Allen, J., Moore, S., & Strauss, V. (2005). Measuring symptoms and functioning of youth with ADHD in middle schools. Journal of Abnormal Child Psychology, 33(6), 695–706.

    Article  PubMed  Google Scholar 

  • Evans, S. W., Langberg, J. M., Schultz, B. K., Vaughn, A., Altaye, M., Marshall, S. A., et al. (2016). Evaluation of a school-based treatment program for young adolescents with ADHD. Journal of Consulting and Clinical Psychology, 84(1), 15.

    Article  PubMed  Google Scholar 

  • Evans, S. W., Schultz, B. K., White, L. C., Brady, C., Sibley, M. H., & Van Eck, K. (2009). A school-based organization intervention for young adolescents with attention-deficit/hyperactivity disorder. School Mental Health, 1, 78–88.

    Article  Google Scholar 

  • Fabiano, G. A., Chafouleas, S. M., Weist, M. D., Sumi, W. C., & Humphrey, N. (2014). Methodology considerations in school mental health research. School Mental Health, 6(2), 68–83.

    Article  Google Scholar 

  • Frazier, T. W., Youngstrom, E. A., Glutting, J. J., & Watkins, M. W. (2007). ADHD and achievement: Meta-analysis of the child, adolescent, and adult literatures and a concomitant study with college students. Journal of Learning Disabilities, 40(1), 49–65.

    Article  PubMed  Google Scholar 

  • Fristad, M. A., Teare, M., Weller, E. B., Weller, R. A., & Salmon, P. (1998). Study III: Development and concurrent validity of the children’s interview for psychiatric syndromes- parent version (P-ChIPS). Journal of Child and Adolescent Psychopharmacology, 8(4), 221–226.

    Article  PubMed  Google Scholar 

  • Gioia, G. A., Isquith, P. K., Guy, S. C., & Kenworthy, L. (2000a). Behavior rating inventory of executive function: BRIEF. Odessa, FL: Psychological Assessment Resources.

    Google Scholar 

  • Gioia, G. A., Isquith, P. K., Guy, S. C., & Kenworthy, L. (2000b). Test review: Behavior rating inventory of executive function. Child Neuropsychology, 6, 235–238.

    Article  Google Scholar 

  • Gureasko-Moore, S., DuPaul, G. J., & White, G. P. (2007). Self-management of classroom preparedness and homework: Effects on school functioning of adolescents with attention deficit hyperactivity disorder. School Psychology Review, 36(4), 647.

    Google Scholar 

  • Harrier, L. K., & DeOrnellas, K. (2005). Performance of children diagnosed with ADHD on selected planning and reconstitution tests. Applied Neuropsychology, 12(2), 106–119.

    Article  PubMed  Google Scholar 

  • Henson, R. K., & Roberts, J. K. (2006). Use of exploratory factor analysis in published research common errors and some comment on improved practice. Educational and Psychological Measurement, 66(3), 393–416.

    Article  Google Scholar 

  • Hu, L. T., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal, 6(1), 1–55.

    Article  Google Scholar 

  • Hughes, C., & Graham, A. (2002). Measuring executive functions in childhood: Problems and solutions? Child and Adolescent Mental Health, 7(3), 131–142.

    Article  Google Scholar 

  • Jensen, P. S., Arnold, L. E., Swanson, J. M., Vitiello, B., Abikoff, H. B., Greenhill, L. L., et al. (2007). 3-Year follow-up of the NIMH MTA study. Journal of the American Academy of Child and Adolescent Psychiatry, 46(8), 989–1002.

    Article  PubMed  Google Scholar 

  • Langberg, J. M., Dvorsky, M. R., & Evans, S. W. (2013). What specific facets of executive function are associated with academic functioning in youth with attention-deficit/hyperactivity disorder? Journal of Abnormal Child Psychology, 41(7), 1145–1159.

    Article  PubMed  PubMed Central  Google Scholar 

  • Langberg, J. M., Dvorsky, M. R., Molitor, S. J., Bourchtein, E., Eddy, L. D., Smith, Z., et al. (2016). Longitudinal evaluation of the importance of homework assignment completion for the academic performance of middle school students with ADHD. Journal of School Psychology, 55, 27–38.

    Article  PubMed  PubMed Central  Google Scholar 

  • Langberg, J. M., Epstein, J. N., Becker, S. P., Girio-Herrera, E., & Vaughn, A. J. (2012). Evaluation of the Homework, Organization, and Planning Skills (HOPS) intervention for middle school students with ADHD as implemented by school mental health providers. School Psychology Review, 41(3), 342.

    PubMed  PubMed Central  Google Scholar 

  • Langberg, J. M., Epstein, J. N., Girio-Herrera, E., Becker, S. P., Vaughn, A., & Altaye, M. (2011). Materials organization, planning, and homework completion in middle-school students with ADHD: Impact on academic performance. School Mental Health, 3(2), 93–101.

    Article  PubMed  PubMed Central  Google Scholar 

  • Leffler, J. M., Riebel, J., & Hughes, H. M. (2014). A review of child and adolescent diagnostic interviews for clinical practitioners. Assessment. doi:10.1177/1073191114561253.

    PubMed  Google Scholar 

  • Little, T. D., Rhemtulla, M., Gibson, K., & Schoemann, A. M. (2013). Why the items versus parcels controversy needn’t be one. Psychological Methods, 18(3), 285.

    Article  PubMed  PubMed Central  Google Scholar 

  • Loe, I. M., & Feldman, H. M. (2007). Academic and educational outcomes of children with ADHD. Journal of Pediatric Psychology, 32(6), 643–654.

    Article  PubMed  Google Scholar 

  • Mahone, E. M., Cirino, P. T., Cutting, L. E., Cerrone, P. M., Hagelthorn, K. M., Hiemenz, J. R., et al. (2002). Validity of the behavior rating inventory of executive function in children with ADHD and/or Tourette syndrome. Archives of Clinical Neuropsychology, 17(7), 643–662.

    Article  PubMed  Google Scholar 

  • Marsh, H. W., Lüdtke, O., Nagengast, B., Morin, A. J., & Von Davier, M. (2013). Why item parcels are (almost) never appropriate: Two wrongs do not make a right—Camouflaging misspecification with item parcels in CFA models. Psychological Methods, 18(3), 257.

    Article  PubMed  Google Scholar 

  • Matsunaga, M. (2008). Item parceling in structural equation modeling: A primer. Communication Methods and Measures, 2(4), 260–293.

    Article  Google Scholar 

  • McCandless, S., & O’Laughlin, L. (2007). The clinical utility of the behavior rating inventory of executive function (BRIEF) in the diagnosis of ADHD. Journal of Attention Disorders, 10, 381–389.

    Article  PubMed  Google Scholar 

  • Muthén, L. K., & Muthén, B. O. (1998–2012). Mplus user’s guide (7th Ed.). Los Angeles, CA: Muthén & Muthén.

  • Narad, M. E., Garner, A. A., Peugh, J. L., Tamm, L., Antonini, T. N., Kingery, K. M., et al. (2015). Parent–teacher agreement on ADHD symptoms across development. Psychological Assessment, 27(1), 239.

    Article  PubMed  Google Scholar 

  • Pennington, B. F., & Ozonoff, S. (1996). Executive functions and developmental psychopathology. Journal of Child Psychology and Psychiatry, 37(1), 51–87.

    Article  PubMed  Google Scholar 

  • Pfiffner, L. J., Mikami, A. Y., Huang-Pollock, C., Easterlin, B., Zalecki, C., & McBurnett, K. (2007). A randomized, controlled trial of integrated home-school behavioral treatment for ADHD, predominantly inattentive type. Journal of the American Academy of Child and Adolescent Psychiatry, 46(8), 1041–1050.

    Article  PubMed  Google Scholar 

  • Pfiffner, L. J., Villodas, M., Kaiser, N., Rooney, M., & McBurnett, K. (2013). Educational outcomes of a collaborative school–home behavioral intervention for ADHD. School Psychology Quarterly, 28(1), 25.

    Article  PubMed  PubMed Central  Google Scholar 

  • Power, T. J., Dombrowski, S. C., Watkins, M. W., Mautone, J. A., & Eagle, J. W. (2007). Assessing children’s homework performance: Development of multi-dimensional, multi-informant rating scales. Journal of School Psychology, 45(3), 333–348.

    Article  PubMed  PubMed Central  Google Scholar 

  • Power, T. J., Watkins, M., Mautone, J. A., Walcott, C. M., Coutts, M. J., & Sheridan, S. M. (2015). Examining the validity of the Homework Performance Questionnaire: Multi-informant assessment in elementary and middle school. School Psychology Quarterly, 30(2), 260–275.

    Article  PubMed  Google Scholar 

  • Prencipe, A., Kesek, A., Cohen, J., Lamm, C., Lewis, M. D., & Zelazo, P. D. (2011). Development of hot and cool executive function during the transition to adolescence. Journal of Experimental Child Psychology, 108(3), 621–637.

    Article  PubMed  Google Scholar 

  • Reise, S. P. (2012). The rediscovery of bifactor measurement models. Multivariate Behavioral Research, 47(5), 667–696.

    Article  PubMed  PubMed Central  Google Scholar 

  • Rios, J., & Wells, C. (2014). Validity evidence based on internal structure. Psicothema, 26, 108–116.

    PubMed  Google Scholar 

  • Shallice, T. (1982). Specific impairments of planning. Philosophical Transactions of the Royal Society of London B: Biological Sciences, 298(1089), 199–209.

    Article  PubMed  Google Scholar 

  • Shimoni, M. A., Engel-Yeger, B., & Tirosh, E. (2012). Executive dysfunctions among boys with Attention Deficit Hyperactivity Disorder (ADHD): Performance-based test and parents report. Research in Developmental Disabilities, 33(3), 858–865.

    Article  PubMed  Google Scholar 

  • Toplak, M. E., West, R. F., & Stanovich, K. E. (2013). Practitioner review: Do performance-based measures and ratings of executive function assess the same construct? Journal of Child Psychology and Psychiatry, 54(2), 131–143.

    Article  PubMed  Google Scholar 

  • Visser, S. N., Danielson, M. L., Bitsko, R. H., Holbrook, M., Kogan, R., Ghandour, R., et al. (2013). Trends in the prevalence of parent-reported ADHD diagnosis and medication treatment by a health care professional: United States, 2003–2011. Journal of the American Academy of Child and Adolescent Psychiatry, 53(1), 34–46.

    Article  PubMed  PubMed Central  Google Scholar 

  • Wechsler, D.(2003). Wechsler intelligence scale for children-WISC-IV. Psychological Corporation.

  • Weller, E. B., Weller, R. A., Fristad, M. A., Rooney, M. T., & Schecter, J. (2000). Children’s interview for psychiatric syndromes (ChIPS). Journal of the American Academy of Child and Adolescent Psychiatry, 39(1), 76–84.

    Article  PubMed  Google Scholar 

  • Weyandt, L., DuPaul, G. J., Verdi, G., Rossi, J. S., Swentosky, A. J., Vilardo, B. S., et al. (2013). The performance of college students with and without ADHD: Neuropsychological, academic, and psychological functioning. Journal of Psychological Behavioral Assessment. doi:10.1007/s10862-013-9351-8.

    Google Scholar 

  • Wolraich, M. L., Wibbelsman, C. J., Brown, T. E., Evans, S. W., Gotlieb, E. M., Knight, J. R., et al. (2005). Attention-deficit/hyperactivity disorder among adolescents: A review of the diagnosis, treatment, and clinical implications. Pediatrics, 115(6), 1734–1746.

    Article  PubMed  Google Scholar 

  • Zoromski, A. K., Owens, J. S., Evans, S. W., & Brady, C. E. (2015). Identifying ADHD symptoms most associated with impairment in early childhood, middle childhood, and adolescence using teacher report. Journal of Abnormal Child Psychology, 43, 1243–1255.

    Article  PubMed  Google Scholar 

Download references

Acknowledgements

This research was supported by Grant Number R01 MH082865 awarded to Steven W. Evans by the National Institute of Mental Health, and Grant Number R305A130011 awarded to Joshua M. Langberg by the Institute of Education Sciences.

Author information

Authors and Affiliations

  1. Virginia Commonwealth University, 806 W. Franklin Street, P.O. Box 842018, Richmond, VA, 23284-2018, USA

    Stephen J. Molitor, Joshua M. Langberg, Melissa R. Dvorsky, Elizaveta Bourchtein, Laura D. Eddy, Zoe R. Smith & Lauren E. Oddo

  2. Ohio University, 200 Porter Hall, Athens, OH, 45701, USA

    Steven W. Evans

Authors
  1. Stephen J. Molitor
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Joshua M. Langberg
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Steven W. Evans
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Melissa R. Dvorsky
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Elizaveta Bourchtein
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Laura D. Eddy
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Zoe R. Smith
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Lauren E. Oddo
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Stephen J. Molitor.

Ethics declarations

Conflict of interest

Stephen J. Molitor, Joshua M. Langberg, Steven W. Evans, Melissa R. Dvorsky, Elizaveta Bourchtein, Laura D. Eddy, Zoe R. Smith, and Lauren E. Oddo declare that they have no conflict of interest.

Ethical approval

All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Declaration of Helsinki and its later amendments or comparable ethical standards. Informed consent was obtained from all individual participants included in the study.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Molitor, S.J., Langberg, J.M., Evans, S.W. et al. Evaluating the Factor Validity of the Children’s Organizational Skills Scale in Youth with ADHD. School Mental Health 9, 143–156 (2017). https://doi.org/10.1007/s12310-016-9205-0

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12310-016-9205-0

Keywords

Search

Navigation