Contribution of the Behavioral Observation of Students in Schools to ADHD Assessment
Classroom observations have long been considered a necessary component of Attention-Deficit/Hyperactivity Disorder (ADHD) evaluations. Yet, research evaluating the utility of observational ratings in ADHD assessment is limited. This study examined the contributions of the Behavioral Observation of Students in Schools (BOSS) to ADHD assessment by investigating associations between BOSS scores with ADHD symptom clusters and symptoms of frequently co-occurring externalizing and internalizing disorders. The utility of BOSS scores in predicting future ADHD-related impairment beyond standard parent and teacher ratings was also examined. One hundred and thirty-five children in grades 2–5 across 23 public schools participated in a randomized controlled trial examining a psychosocial treatment for ADHD. BOSS ratings were collected at baseline. Parent and teacher ratings of child symptoms and impairment were collected at baseline, post-treatment (3–4 months later), and follow-up (8–12 months later). Multiple regressions investigated the associations between the BOSS subscale of Task Engagement (TE) and parent and teacher ratings of ADHD symptoms and related disorders. Multi-level modeling accounted for school cluster effects. Results showed that lower BOSS TE was related to higher teacher-rated inattention but not hyperactivity/impulsivity symptoms. Lower BOSS TE was also associated with higher teacher-rated Oppositional Defiant Disorder (ODD) and depression symptoms, but not anxiety symptoms. Further, BOSS TE predicted higher future impairment beyond baseline teacher and parent ratings of ADHD symptoms and impairment, controlling for treatment. The BOSS appears sensitive to symptoms of child inattention, ODD, and depression, and may have utility in informing future impairment beyond standard informant ratings of ADHD.
KeywordsAttention-Deficit/Hyperactivity Disorder (ADHD) Behavioral Observation of Students in Schools (BOSS) Task Engagement Systematic direct observations
The authors thank John DuBois and Melissa Plageman for their contributions to data entry and management, as well as many research assistants and study staff who served as coders.
This work was supported by the Institute of Education Sciences, U.S. Department of Education Award Grant R324A120358 (Principal Investigator: Linda J. Pfiffner).
Compliance with Ethical Standards
Conflict of interest
Dr. McBurnett has received research support from Akili Interactive and Shire Pharmaceuticals. Dr. Capriotti receives grant funding from the American Academy of Neurology, Tourette Association of America (Clinical Research Training Fellowship in Tourette Syndrome), and Center for California Studies at Sacramento State University. Dr. Capriotti has received honoraria from Sanofi Genzyme for presenting on mindfulness at events for individuals with Multiple Sclerosis and from the Tourette Association of America for leading workshops on behavioral treatment of Tourette Syndrome. Drs. Jiang, Beaulieu, Rooney, and Pfiffner report no potential conflicts of interest.
All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki Declaration and its later amendments or comparable ethical standards.
Informed consent was obtained from all individual participants included in the study.
- Abikoff, H., & Gittelman, R. (1985). Hyperactive children treated with stimulants: Is cognitive training a useful adjunct? Archives of General Psychiatry, 42, 953–961. https://doi.org/10.1001/archpsyc.1985.01790330033004.CrossRefGoogle Scholar
- Abikoff, H. B., Jensen, P. S., Arnold, L. E., Hoza, B., Hechtman, L., Pollack, S., et al. (2002). Observed classroom behavior of children with ADHD: Relationship to gender and comorbidity. Journal of Abnormal Child Psychology, 30, 349–359. https://doi.org/10.1023/A:1015713807297.CrossRefGoogle Scholar
- Achenbach, T. M. (2009). The Achenbach system of empirically based assessment (ASEBA): Development, findings, theory, and applications. Burlington, VT: University of Vermont Research Center for Children, Youth, & Families.Google Scholar
- Benjamini, Y., & Hochberg, Y. (1995). Controlling the false discovery rate: A practical and powerful approach to multiple testing. Journal of the Royal Statistical Society, 57, 289–300.Google Scholar
- Biederman, J., Monuteaux, M. C., Doyle, A. E., Seidman, L. J., Wilens, T. E., Ferrero, F., et al. (2004). Impact of executive function deficits and attention-deficit/hyperactivity disorder (ADHD) on academic outcomes in children. Journal of Consulting and Clinical Psychology, 72, 757. https://doi.org/10.1037/0022-006X.72.5.757.CrossRefGoogle Scholar
- Conners, C. K. (2008). Conners 3rd edition manual. New York, NY: Multi-health Systems Inc.Google Scholar
- DuPaul, G. J., & Stoner, G. (2014). ADHD in the schools: Assessment and intervention strategies. New York: Guilford Publications.Google Scholar
- Fabiano, G. A., Pelham, W. E., Gnagy, E. M., Waschbusch, D. A., Lahey, B. B., Chronis, A. M., et al. (2006). A practical measure of impairment: Psychometric properties of the impairment rating scale in samples of children with attention deficit hyperactivity disorder and two school-based samples. Journal of Clinical Child and Adolescent Psychology, 35, 369–385. https://doi.org/10.1207/s15374424jccp3503_3.CrossRefGoogle Scholar
- Gadow, K. D., Nolan, E. E., & Sverd, J. (1992). Methylphenidate in hyperactive boys with comorbid tic disorder: II. Short-term behavioral effects in school settings. Journal of the American Academy of Child and Adolescent Psychiatry, 31, 462–471. https://doi.org/10.1097/00004583-199205000-00012.CrossRefGoogle Scholar
- Gadow, K. D., & Sprafkin, J. N. (2002). Child symptom inventory 4: Screening and norms manual. Stony Brook, NY: Checkmate Plus.Google Scholar
- Gadow, K. D., Sprafkin, J., & Nolan, E. E. (1996). ADHD school observation code. Stony Brook, NY: Checkmate Plus.Google Scholar
- Greenwood, C. R., Horton, B. T., & Utley, C. A. (2002). Academic engagement: Current perspectives on research and practice. School Psychology Review, 31, 328.Google Scholar
- Hintze, J. M. (2005). Psychometrics of direct observation. School Psychology Review, 34, 507.Google Scholar
- Lahey, B. B., Pelham, W. E., Loney, J., Kipp, H., Ehrhardt, A., Lee, S. S., et al. (2004). Three-year predictive validity of DSM-IV attention-deficit hyperactivity disorder in children diagnosed at 4–6 years of age. American Journal of Psychiatry, 161, 2014–2020. https://doi.org/10.1176/appi.ajp.161.11.2014.CrossRefGoogle Scholar
- McConaughy, S. H., Harder, V. S., Antshel, K. M., Gordon, M., Eiraldi, R., & Dumenci, L. (2010). Incremental validity of test session and classroom observations in a multimethod assessment of attention deficit/hyperactivity disorder. Journal of Clinical Child and Adolescent Psychology, 39, 650–666. https://doi.org/10.1080/15374416.2010.501287.CrossRefGoogle Scholar
- Orban, S. A., Rapport, M. D., Friedman, L. M., Eckrich, S. J., & Kofler, M. J. (2018). Inattentive behavior in boys with ADHD during classroom instruction: The mediating role of working memory processes. Journal of Abnormal Child Psychology, 46, 713–727. https://doi.org/10.1007/s10802-017-0338-x.CrossRefGoogle Scholar
- Pfiffner, L. J., Rooney, M., Haack, L., Villodas, M., Delucchi, K., & McBurnett, K. (2016). A randomized controlled trial of a school-implemented school–home intervention for attention-deficit/hyperactivity disorder symptoms and impairment. Journal of the American Academy of Child and Adolescent Psychiatry, 55, 762–770. https://doi.org/10.1016/j.jaac.2016.05.023.CrossRefGoogle Scholar
- Reynolds, C. R., & Kamphaus, R. W. (2004). BASC-2: Behavior assessment system for children (2nd ed.): Manual. Circle Pines, MN: AGS Printing.Google Scholar
- Rogers, M., Hwang, H., Toplak, M., Weiss, M., & Tannock, R. (2011). Inattention, working memory, and academic achievement in adolescents referred for attention deficit/hyperactivity disorder (ADHD). Child Neuropsychology, 17, 444–458. https://doi.org/10.1080/09297049.2010.544648.CrossRefGoogle Scholar
- Saudargas, R. A. (1997). State-event classroom observation system (SECOS): Observation manual. Knoxville, TN: University of Tennessee.Google Scholar
- Scahill, L., Schwab-Stone, M., Merikangas, K. R., Leckman, J. F., Zhang, H., & Kasl, S. (1999). Psychosocial and clinical correlates of ADHD in a community sample of school-age children. Journal of the American Academy of Child and Adolescent Psychiatry, 38, 976–984. https://doi.org/10.1097/00004583-199908000-00013.CrossRefGoogle Scholar
- Shapiro, E. S. (2004). Direct observation: Manual for the behavioral observation of students in schools (BOSS). New York, NY: Guilford Press.Google Scholar
- Skansgaard, E. P., & Burns, G. L. (1998). Comparison of DSM-IV ADHD combined and predominantly inattention types: Correspondence between teacher ratings and direct observations of inattentive, hyperactivity/impulsivity, slow cognitive tempo, oppositional defiant, and overt conduct disorder symptoms. Child & Family Behavior Therapy, 20, 1–14. https://doi.org/10.1300/J019v20n01_01.CrossRefGoogle Scholar
- Steiner, H., & Remsing, L. (2007). Practice parameter for the assessment and treatment of children and adolescents with oppositional defiant disorder. Journal of the American Academy of Child and Adolescent Psychiatry, 46, 126–141. https://doi.org/10.1097/01.chi.0000246060.62706.af.CrossRefGoogle Scholar
- Thorell, L. B. (2007). Do delay aversion and executive function deficits make distinct contributions to the functional impact of ADHD symptoms? A study of early academic skill deficits. Journal of Child Psychology and Psychiatry, 48, 1061–1070. https://doi.org/10.1111/j.1469-7610.2007.01777.x.CrossRefGoogle Scholar
- Volpe, R. J., DiPerna, J. C., Hintze, J. M., & Shapiro, E. S. (2005). Observing students in classroom settings: A review of seven coding schemes. School Psychology Review, 34, 454.Google Scholar
- Walker, H. M., Severson, H. H., & Feil, E. G. (2014). Systematic screening for behavior disorders (SSBD): Technical manual. Eugene, OR: Pacific Northwest Publishing.Google Scholar
- Wechsler, D. (2011). Wechsler abbreviated scale of intelligence. San Antonio, TX: Psychological Corporation.Google Scholar
- Willcutt, E. G., Hartung, C. M., Lahey, B. B., Loney, J., & Pelham, W. E. (1999). Utility of behavior ratings by examiners during assessments of preschool children with attention-deficit/hyperactivity disorder. Journal of Abnormal Child Psychology, 27, 463–472. https://doi.org/10.1023/A:1021984126774.CrossRefGoogle Scholar
- Wilson, M. S., & Reschly, D. J. (1996). Assessment in school psychology training and practice. School Psychology Review, 25, 923.Google Scholar