Abstract
Classroom observations have long been considered a necessary component of Attention-Deficit/Hyperactivity Disorder (ADHD) evaluations. Yet, research evaluating the utility of observational ratings in ADHD assessment is limited. This study examined the contributions of the Behavioral Observation of Students in Schools (BOSS) to ADHD assessment by investigating associations between BOSS scores with ADHD symptom clusters and symptoms of frequently co-occurring externalizing and internalizing disorders. The utility of BOSS scores in predicting future ADHD-related impairment beyond standard parent and teacher ratings was also examined. One hundred and thirty-five children in grades 2–5 across 23 public schools participated in a randomized controlled trial examining a psychosocial treatment for ADHD. BOSS ratings were collected at baseline. Parent and teacher ratings of child symptoms and impairment were collected at baseline, post-treatment (3–4 months later), and follow-up (8–12 months later). Multiple regressions investigated the associations between the BOSS subscale of Task Engagement (TE) and parent and teacher ratings of ADHD symptoms and related disorders. Multi-level modeling accounted for school cluster effects. Results showed that lower BOSS TE was related to higher teacher-rated inattention but not hyperactivity/impulsivity symptoms. Lower BOSS TE was also associated with higher teacher-rated Oppositional Defiant Disorder (ODD) and depression symptoms, but not anxiety symptoms. Further, BOSS TE predicted higher future impairment beyond baseline teacher and parent ratings of ADHD symptoms and impairment, controlling for treatment. The BOSS appears sensitive to symptoms of child inattention, ODD, and depression, and may have utility in informing future impairment beyond standard informant ratings of ADHD.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
The pattern of findings was similar when BOSS Off-Task Behavior was used instead of TE in analyses.
References
Abikoff, H., & Gittelman, R. (1985). Hyperactive children treated with stimulants: Is cognitive training a useful adjunct? Archives of General Psychiatry, 42, 953–961. https://doi.org/10.1001/archpsyc.1985.01790330033004.
Abikoff, H., Gittelman, R., & Klein, D. F. (1980). Classroom observation code for hyperactive children: A replication of validity. Journal of Consulting and Clinical Psychology, 48, 555. https://doi.org/10.1037//0022-006X.48.5.555.
Abikoff, H., Gittelman-Klein, R., & Klein, D. F. (1977). Validation of a classroom observation code for hyperactive children. Journal of Consulting and Clinical Psychology, 45, 772. https://doi.org/10.1037/0022-006X.45.5.772.
Abikoff, H. B., Jensen, P. S., Arnold, L. E., Hoza, B., Hechtman, L., Pollack, S., et al. (2002). Observed classroom behavior of children with ADHD: Relationship to gender and comorbidity. Journal of Abnormal Child Psychology, 30, 349–359. https://doi.org/10.1023/A:1015713807297.
Achenbach, T. M. (2009). The Achenbach system of empirically based assessment (ASEBA): Development, findings, theory, and applications. Burlington, VT: University of Vermont Research Center for Children, Youth, & Families.
American Psychiatric Association. (2013). Diagnostic and statistical manual of mental disorders (5th ed.). Washington, DC: America Psychiatric Association.
Atkins, M. S., & Pelham, W. E. (1991). School-based assessment of attention deficit-hyperactivity disorder. Journal of Learning Disabilities, 24, 197–204. https://doi.org/10.1177/002221949102400403.
Austerman, J. (2015). ADHD and behavioral disorders: Assessment, management, and an update from DSM-5. Cleveland Clinic Journal of Medicine, 82, 2–7. https://doi.org/10.3949/ccjm.82.s1.01.
Barkley, R. A. (1997). Behavioral inhibition, sustained attention, and executive functions: Constructing a unifying theory of ADHD. Psychological Bulletin, 121, 65.
Benjamini, Y., & Hochberg, Y. (1995). Controlling the false discovery rate: A practical and powerful approach to multiple testing. Journal of the Royal Statistical Society, 57, 289–300.
Biederman, J., Monuteaux, M. C., Doyle, A. E., Seidman, L. J., Wilens, T. E., Ferrero, F., et al. (2004). Impact of executive function deficits and attention-deficit/hyperactivity disorder (ADHD) on academic outcomes in children. Journal of Consulting and Clinical Psychology, 72, 757. https://doi.org/10.1037/0022-006X.72.5.757.
Conners, C. K. (2008). Conners 3rd edition manual. New York, NY: Multi-health Systems Inc.
Daley, D., & Birchwood, J. (2010). ADHD and academic performance: Why does ADHD impact on academic performance and what can be done to support ADHD children in the classroom? Child: Care, Health and Development, 36, 455–464. https://doi.org/10.1111/j.1365-2214.2009.01046.x.
Dodge, K. A., & Crick, N. R. (1990). Social information-processing bases of aggressive behavior in children. Personality and Social Psychology Bulletin, 16, 8–22. https://doi.org/10.1177/0146167290161002.
DuPaul, G. J., & Stoner, G. (2014). ADHD in the schools: Assessment and intervention strategies. New York: Guilford Publications.
DuPaul, G. J., Volpe, R. J., Jitendra, A. K., Lutz, J. G., Lorah, K. S., & Gruber, R. (2004). Elementary school students with AD/HD: Predictors of academic achievement. Journal of School Psychology, 42, 285–301. https://doi.org/10.1016/j.jsp.2004.05.001.
Fabiano, G. A., Pelham, W. E., Gnagy, E. M., Waschbusch, D. A., Lahey, B. B., Chronis, A. M., et al. (2006). A practical measure of impairment: Psychometric properties of the impairment rating scale in samples of children with attention deficit hyperactivity disorder and two school-based samples. Journal of Clinical Child and Adolescent Psychology, 35, 369–385. https://doi.org/10.1207/s15374424jccp3503_3.
Gadow, K. D., Nolan, E. E., & Sverd, J. (1992). Methylphenidate in hyperactive boys with comorbid tic disorder: II. Short-term behavioral effects in school settings. Journal of the American Academy of Child and Adolescent Psychiatry, 31, 462–471. https://doi.org/10.1097/00004583-199205000-00012.
Gadow, K. D., & Sprafkin, J. N. (2002). Child symptom inventory 4: Screening and norms manual. Stony Brook, NY: Checkmate Plus.
Gadow, K. D., Sprafkin, J., & Nolan, E. E. (1996). ADHD school observation code. Stony Brook, NY: Checkmate Plus.
Greenwood, C. R., Horton, B. T., & Utley, C. A. (2002). Academic engagement: Current perspectives on research and practice. School Psychology Review, 31, 328.
Handler, M. W., & DuPaul, G. J. (2005). Assessment of ADHD: Differences across psychology specialty areas. Journal of Attention Disorders, 9, 402–412. https://doi.org/10.1177/1087054705278762.
Hintze, J. M. (2005). Psychometrics of direct observation. School Psychology Review, 34, 507.
Junod, R. E. V., DuPaul, G. J., Jitendra, A. K., Volpe, R. J., & Cleary, K. S. (2006). Classroom observations of students with and without ADHD: Differences across types of engagement. Journal of School Psychology, 44, 87–104. https://doi.org/10.1016/j.jsp.2005.12.004.
Kofler, M. J., Rapport, M. D., & Alderson, R. M. (2008). Quantifying ADHD classroom inattentiveness, its moderators, and variability: A meta-analytic review. Journal of Child Psychology and Psychiatry, 49, 59–69. https://doi.org/10.1111/j.1469-7610.2007.01809.x.
Lahey, B. B., Pelham, W. E., Loney, J., Kipp, H., Ehrhardt, A., Lee, S. S., et al. (2004). Three-year predictive validity of DSM-IV attention-deficit hyperactivity disorder in children diagnosed at 4–6 years of age. American Journal of Psychiatry, 161, 2014–2020. https://doi.org/10.1176/appi.ajp.161.11.2014.
Lee, S. S., Lahey, B. B., Owens, E. B., & Hinshaw, S. P. (2008). Few preschool boys and girls with ADHD are well-adjusted during adolescence. Journal of Abnormal Child Psychology, 36, 373–383. https://doi.org/10.1007/s10802-007-9184-6.
Loe, I. M., & Feldman, H. M. (2007). Academic and educational outcomes of children with ADHD. Journal of Pediatric Psychology, 32, 643–654. https://doi.org/10.1093/jpepsy/jsl054.
Luk, S. L. (1985). Direct observation studies of hyperactive behaviors. Journal of the American Academy of Child Psychiatry, 24, 338–344. https://doi.org/10.1016/S0002-7138(09)61096-7.
McConaughy, S. H., Harder, V. S., Antshel, K. M., Gordon, M., Eiraldi, R., & Dumenci, L. (2010). Incremental validity of test session and classroom observations in a multimethod assessment of attention deficit/hyperactivity disorder. Journal of Clinical Child and Adolescent Psychology, 39, 650–666. https://doi.org/10.1080/15374416.2010.501287.
MTA Cooperative Group. (1999). 14-month randomized clinical trial of treatment strategies for attention deficit hyperactivity disorder. Archives of General Psychiatry, 56, 1073–1086. https://doi.org/10.1001/archpsyc.56.12.1073.
Nock, M. K., & Kurtz, S. M. (2005). Direct behavioral observation in school settings: Bringing science to practice. Cognitive and Behavioral Practice, 12, 359–370. https://doi.org/10.1016/S1077-7229(05)80058-6.
Orban, S. A., Rapport, M. D., Friedman, L. M., Eckrich, S. J., & Kofler, M. J. (2018). Inattentive behavior in boys with ADHD during classroom instruction: The mediating role of working memory processes. Journal of Abnormal Child Psychology, 46, 713–727. https://doi.org/10.1007/s10802-017-0338-x.
Ota, K. R., & DuPaul, G. J. (2002). Task engagement and mathematics performance in children with attention-deficit hyperactivity disorder: Effects of supplemental computer instruction. School Psychology Quarterly, 17, 242. https://doi.org/10.1521/scpq.17.3.242.20881.
Owens, M., Stevenson, J., Hadwin, J. A., & Norgate, R. (2012). Anxiety and depression in academic performance: An exploration of the mediating factors of worry and working memory. School Psychology International, 33, 433–449. https://doi.org/10.1177/0143034311427433.
Parker, A., & Corkum, P. (2016). ADHD diagnosis: As simple as administering a questionnaire or a complex diagnostic process? Journal of Attention Disorders, 20, 478–486. https://doi.org/10.1177/1087054713495736.
Pekrun, R., Elliot, A. J., & Maier, M. A. (2009). Achievement goals and achievement emotions: Testing a model of their joint relations with academic performance. Journal of Educational Psychology, 101, 115. https://doi.org/10.1037/a0013383.
Pekrun, R., & Linnenbrink-Garcia, L. (2012). Academic emotions and student engagement. In S. L. Christenson, A. L. Reschly, & C. Wylie (Eds.), Handbook of research on student engagement (pp. 259–282). New York, NY: Springer.
Pelham, W. E., Jr., Fabiano, G., & Massetti, G. (2005). Evidence-based assessment of attention deficit hyperactivity disorder in children and adolescents. Journal of Clinical Child & Adolescent Psychology, 34, 449–476. https://doi.org/10.1207/s15374424jccp3403_5.
Pfiffner, L. J., Rooney, M., Haack, L., Villodas, M., Delucchi, K., & McBurnett, K. (2016). A randomized controlled trial of a school-implemented school–home intervention for attention-deficit/hyperactivity disorder symptoms and impairment. Journal of the American Academy of Child and Adolescent Psychiatry, 55, 762–770. https://doi.org/10.1016/j.jaac.2016.05.023.
Platzman, K. A., Stoy, M. R., Brown, R. T., Coles, C. D., Smith, I. E., & Falek, A. (1992). Review of observational methods in attention deficit hyperactivity disorder (ADHD): Implications for diagnosis. School Psychology Quarterly, 7, 155.
Pliszka, S. R. (1992). Comorbidity of attention-deficit hyperactivity disorder and overanxious disorder. Journal of the American Academy of Child and Adolescent Psychiatry, 28, 882–887. https://doi.org/10.1097/00004583-199203000-00003.
Quay, H. C. (1997). Inhibition and attention deficit hyperactivity disorder. Journal of Abnormal Child Psychology, 25, 7–13. https://doi.org/10.1023/A:1025799122529.
Reynolds, C. R., & Kamphaus, R. W. (2004). BASC-2: Behavior assessment system for children (2nd ed.): Manual. Circle Pines, MN: AGS Printing.
Rogers, M., Hwang, H., Toplak, M., Weiss, M., & Tannock, R. (2011). Inattention, working memory, and academic achievement in adolescents referred for attention deficit/hyperactivity disorder (ADHD). Child Neuropsychology, 17, 444–458. https://doi.org/10.1080/09297049.2010.544648.
Saudargas, R. A. (1997). State-event classroom observation system (SECOS): Observation manual. Knoxville, TN: University of Tennessee.
Scahill, L., Schwab-Stone, M., Merikangas, K. R., Leckman, J. F., Zhang, H., & Kasl, S. (1999). Psychosocial and clinical correlates of ADHD in a community sample of school-age children. Journal of the American Academy of Child and Adolescent Psychiatry, 38, 976–984. https://doi.org/10.1097/00004583-199908000-00013.
Schatz, D. B., & Rostain, A. L. (2006). ADHD with comorbid anxiety: A review of the current literature. Journal of Attention Disorders, 10, 141–149. https://doi.org/10.1177/1087054706286698.
Shapiro, E. S. (2004). Direct observation: Manual for the behavioral observation of students in schools (BOSS). New York, NY: Guilford Press.
Shapiro, E. S. (2014). Behavioral assessment in school psychology. New York, NY: Psychology Press.
Shapiro, E. S., & Heick, P. F. (2004). School psychologist assessment practices in the evaluation of students referred for social/behavioral/emotional problems. Psychology in the Schools, 41, 551–561. https://doi.org/10.1002/pits.10176.
Skansgaard, E. P., & Burns, G. L. (1998). Comparison of DSM-IV ADHD combined and predominantly inattention types: Correspondence between teacher ratings and direct observations of inattentive, hyperactivity/impulsivity, slow cognitive tempo, oppositional defiant, and overt conduct disorder symptoms. Child & Family Behavior Therapy, 20, 1–14. https://doi.org/10.1300/J019v20n01_01.
Steiner, H., & Remsing, L. (2007). Practice parameter for the assessment and treatment of children and adolescents with oppositional defiant disorder. Journal of the American Academy of Child and Adolescent Psychiatry, 46, 126–141. https://doi.org/10.1097/01.chi.0000246060.62706.af.
Thorell, L. B. (2007). Do delay aversion and executive function deficits make distinct contributions to the functional impact of ADHD symptoms? A study of early academic skill deficits. Journal of Child Psychology and Psychiatry, 48, 1061–1070. https://doi.org/10.1111/j.1469-7610.2007.01777.x.
Volpe, R. J., DiPerna, J. C., Hintze, J. M., & Shapiro, E. S. (2005). Observing students in classroom settings: A review of seven coding schemes. School Psychology Review, 34, 454.
Walker, H. M., Severson, H. H., & Feil, E. G. (2014). Systematic screening for behavior disorders (SSBD): Technical manual. Eugene, OR: Pacific Northwest Publishing.
Wechsler, D. (2011). Wechsler abbreviated scale of intelligence. San Antonio, TX: Psychological Corporation.
Willcutt, E. G., Hartung, C. M., Lahey, B. B., Loney, J., & Pelham, W. E. (1999). Utility of behavior ratings by examiners during assessments of preschool children with attention-deficit/hyperactivity disorder. Journal of Abnormal Child Psychology, 27, 463–472. https://doi.org/10.1023/A:1021984126774.
Wilson, M. S., & Reschly, D. J. (1996). Assessment in school psychology training and practice. School Psychology Review, 25, 923.
Acknowledgements
The authors thank John DuBois and Melissa Plageman for their contributions to data entry and management, as well as many research assistants and study staff who served as coders.
Funding
This work was supported by the Institute of Education Sciences, U.S. Department of Education Award Grant R324A120358 (Principal Investigator: Linda J. Pfiffner).
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
Dr. McBurnett has received research support from Akili Interactive and Shire Pharmaceuticals. Dr. Capriotti receives grant funding from the American Academy of Neurology, Tourette Association of America (Clinical Research Training Fellowship in Tourette Syndrome), and Center for California Studies at Sacramento State University. Dr. Capriotti has received honoraria from Sanofi Genzyme for presenting on mindfulness at events for individuals with Multiple Sclerosis and from the Tourette Association of America for leading workshops on behavioral treatment of Tourette Syndrome. Drs. Jiang, Beaulieu, Rooney, and Pfiffner report no potential conflicts of interest.
Ethical Approval
All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki Declaration and its later amendments or comparable ethical standards.
Informed Consent
Informed consent was obtained from all individual participants included in the study.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Jiang, Y., Capriotti, M., Beaulieu, A. et al. Contribution of the Behavioral Observation of Students in Schools to ADHD Assessment. School Mental Health 11, 464–475 (2019). https://doi.org/10.1007/s12310-019-09313-5
Published:
Issue Date:
DOI: https://doi.org/10.1007/s12310-019-09313-5