Abstract
Discrepancies between parent and teacher ratings of problem behaviors related to Attention-Deficit/Hyperactivity Disorder (ADHD) are reported frequently. Previous studies have hypothesized that these discrepancies are the results of various informant biases and have evaluated whether the rating scales are measuring behaviors the same way across informants. The purpose of this study was to evaluate if two rating scales of ADHD behavior, the Strengths and Weaknesses of ADHD Symptoms and Normal Behavior Rating Scale (SWAN) and the Conners’ Teacher Rating Scale-15 (CTRS-15), reflected the same underlying constructs across parent and teacher report. Measurement invariance analyses were conducted using parent and teacher report data from a sample of 1645 preschool to fifth-grade children (age range 46 to 169 months) that was comprised of roughly equal number of boys and girls and had racial/ethnic diversity similar to the community (i.e., 61% White, 22% Black/African American; 4% Hispanic/Latino). Although it was hypothesized that both rating scales would demonstrate measurement invariance across parent and teacher report, at least partial weak measurement invariance was only supported for the CTRS-15 across all grade groups. These results indicate that the meaning of any rating discrepancies on the SWAN are unknown because it is not reflective of the same underlying constructs across parents and teachers across all of the examined grade groups. In general, these results have potentially important implications regarding research on ADHD symptoms and related behaviors, and raise questions regarding the utility and measurement of ADHD symptoms.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Achenbach, T. M., McConaughy, S. H., & Howell, C. T. (1987). Child/adolescent behavioral and emotional problems: Implications of cross-informant correlations for situational specificity. Psychological Bulletin, 101(2), 213–232. https://doi.org/10.1037/0033-2909.101.2.213.
Allan, D. M., & Lonigan, C. J. (2019). Examination of the structure and measurement of inattentive, hyperactive, and impulsive behaviors from preschool to grade 4. Journal of Abnormal Child Psychology, 47(6), 975–987. https://doi.org/10.1007/s10802-018-0491-x.
Amador-Campos, J. A., Forns-Santacana, M., Guàrdia-Olmos, J., & Peró-Cebollero, M. (2006). DSM-IV attention deficit hyperactivity disorder symptoms: Agreement between informants in prevalence and factor structure at different ages. Journal of Psychopathology and Behavioral Assessment, 28(1), 23–32. https://doi.org/10.1007/s10862-006-4538-x.
American Psychiatric Association. (2013). Diagnostic and statistical manual of mental disorders (5th ed.). Arlington: Author.
Antrop, I., Roeyers, H., Oosterlaan, J., & Oost, P. V. (2002). Agreement between parent and teacher ratings of disruptive behavior disorders in children with clinically diagnosed ADHD. Journal of Psychopathology and Behavioral Assessment, 24(1), 67–73. https://doi.org/10.1023/A:1014057325752.
Benjamini, Y., & Hochberg, Y. (1995). Controlling the false discovery rate: A practical and powerful approach to multiple testing. Journal of the Royal Statistical Society: Series B (Methodological), 57(1), 289–300. https://doi.org/10.1111/j.2517-6161.1995.tb02031.x.
Biederman, J., Mick, E., & Faraone, S. V. (2000). Age-dependent decline of symptoms of attention deficit hyperactivity disorder: Impact of remission definition and symptom type. American Journal of Psychiatry, 157(5), 816–818. https://doi.org/10.1176/appi.ajp.157.5.816.
Burns, G. L., Servera, M., Bernard, M., Carrillo, J. M., & Geiser, C. (2014). Ratings of ADHD symptoms and academic impairment by mothers, fathers, teachers and aides: Construct validity within and across settings as well as occasions. Psychological Assessment, 26(4), 1247–1258. https://doi.org/10.1037/pas0000008.
Burns, G. L., Walsh, J. A., Gomez, R., & Hafetz, N. (2006). Measurement and structural invariance of parent ratings of ADHD and ODD symptoms across gender for American and Malaysian children. Psychological Assessment, 18(4), 452–457. https://doi.org/10.1037/1040-3590.18.4.452.
Burns, G. L., Walsh, J. A., Servera, M., Lorenzo-Seva, U., Cardo, E., & Rodriguez-Fornells, A. (2013). Construct validity of ADHD/ODD rating scales: Recommendations for the evaluation of forthcoming DSM–V ADHD/ODD scales. Journal of Abnormal Child Psychology, 41(1), 15–26. https://doi.org/10.1007/s10802-012-9660-5.
Chen, F. F. (2007). Sensitivity of goodness of fit indexes to lack of measurement invariance. Structural Equation Modeling: A Multidisciplinary Journal, 14(3), 464–504. https://doi.org/10.1080/10705510701301834.
Chi, T. C., & Hinshaw, S. P. (2002). Mother–child relationships of children with ADHD: The role of maternal depressive symptoms and depression-related distortions. Journal of Abnormal Child Psychology, 30, 387–400.
Conners, C. K. (1969). A teacher rating scale for use in drug studies with children. American Journal of Psychiatry, 126(6), 884–888. https://doi.org/10.1176/ajp.126.6.884.
De Los Reyes, A., Augenstein, T. M., Wang, M., Thomas, S. A., Drabick, D. A., Burgers, D. E., & Rabinowitz, J. (2015). The validity of the multi-informant approach to assessing child and adolescent mental health. Psychological Bulletin, 141(4), 858–900. https://doi.org/10.1037/a0038498.
De Los Reyes, A., & Kazdin, A. E. (2005). Informant discrepancies in the assessment of childhood psychopathology: A critical review, theoretical framework, and recommendations for further study. Psychological Bulletin, 131(4), 483–509. https://doi.org/10.1037/0033-2909.131.4.483.
De Los Reyes, A., Thomas, S. A., Goodman, K. L., & Kundey, S. M. (2013). Principles underlying the use of multiple informants' reports. Annual Review of Clinical Psychology, 9, 123–149. https://doi.org/10.1146/annurev-clinpsy-050212-185617.
Duhig, A. M., Renk, K., Epstein, M. K., & Phares, V. (2000). Interparental agreement on internalizing, externalizing, and total behavior problems: A meta-analysis. Clinical Psychology: Science and Practice, 7, 435–453.
DuPaul, G. J., Fu, Q., Anastopoulos, A. D., Reid, R., & Power, T. J. (2020). ADHD parent and teacher symptom ratings: Differential item functioning across gender, age, race, and ethnicity. Journal of Abnormal Child Psychology, 48, 679–691. https://doi.org/10.1007/s10802-020-00618-7.
Gooch, D., Maydew, H., Sears, C., & Norbury, C. F. (2017). Does a child’s language ability affect the correspondence between parent and teacher ratings of ADHD symptoms? BMC Psychiatry, 17(1), 129. https://doi.org/10.1186/s12888-017-1300-8.
Harvey, E. A., Fischer, C., Weieneth, J. L., Hurwitz, S. D., & Sayer, A. G. (2013). Predictors of discrepancies between informants’ ratings of preschool-aged children's behavior: An examination of ethnicity, child characteristics, and family functioning. Early Childhood Research Quarterly, 28(4), 668–682. https://doi.org/10.1016/j.ecresq.2013.05.002.
Hu, L. T., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal, 6(1), 1–55. https://doi.org/10.1080/10705519909540118.
Javo, C., Heyerdahl, S., & Rønning, J. A. (2000). Parent reports of child behavior problems in young Sami children: A cross-cultural comparison. European Child & Adolescent Psychiatry, 9(3), 202–211.
Lakes, K. D., Swanson, J. M., & Riggs, M. (2012). The reliability and validity of the English and Spanish strengths and weaknesses of ADHD and normal behavior rating scales in a preschool sample: Continuum measures of hyperactivity and inattention. Journal of Attention Disorders, 16(6), 510–516. https://doi.org/10.1177/1087054711413550.
Larsson, H., Dilshad, R., Lichtenstein, P., & Barker, E. D. (2011). Developmental trajectories of DSM-IV symptoms of attention-deficit/hyperactivity disorder: Genetic effects, family risk and associated psychopathology. Journal of Child Psychology and Psychiatry, 52(9), 954–963. https://doi.org/10.1111/j.1469-7610.2011.02379.x.
Lee, S., Burns, G. L., Snell, J., & McBurnett, K. (2014). Validity of the sluggish cognitive tempo symptom dimension in children: Sluggish cognitive tempo and ADHD inattention as distinct symptom symptoms. Journal of Abnormal Child Psychology, 42, 7–19. https://doi.org/10.1007/s10802-013-9714-3.
Lonigan, C. J., Bloomfield, B. G., Anthony, J. L., Bacon, K. D., Phillips, B. M., & Samwel, C. S. (1999). Relations among emergent literacy skills, behavior problems, and social competence in preschool children from low-and middle-income backgrounds. Topics in Early Childhood Special Education, 19(1), 40–53. https://doi.org/10.1177/027112149901900104.
Lubke, G. H., & Muthén, B. O. (2004). Applying multigroup confirmatory factor models for continuous outcomes to Likert scale data complicates meaningful group comparisons. Structural Equation Modeling, 11(4), 514–534. https://doi.org/10.1207/s15328007sem1104_2.
Makransky, G., & Bilenberg, N. (2014). Psychometric properties of the parent and teacher ADHD rating scale (ADHD-RS) measurement invariance across gender, age, and informant. Assessment, 21(6), 694–705. https://doi.org/10.1177/1073191114535242.
Massetti, G. M., Lahey, B. B., Pelham, W. E., Loney, J., Ehrhardt, A., Lee, S. S., & Kipp, H. (2008). Academic achievement over 8 years among children who met modified criteria for attention-deficit/hyperactivity disorder at 4–6 years of age. Journal of Abnormal Child Psychology, 36(3), 399–410. https://doi.org/10.1007/s10802-007-9186-4.
Meade, A. W., Johnson, E. C., & Braddy, P. W. (2008). Power and sensitivity of alternative fit indices in tests of measurement invariance. Journal of Applied Psychology, 93(3), 568–592. https://doi.org/10.1037/0021-9010.93.3.568.
Murray, D. W., Kollins, S. H., Hardy, K. K., Abikoff, H. B., Swanson, J. M., Cunningham, C., & Chuang, S. Z. (2007). Parent versus teacher ratings of attention-deficit/hyperactivity disorder symptoms in the Preschoolers with Attention-Deficit/Hyperactivity Disorder Treatment Study (PATS). Journal of Child and Adolescent Psychopharmacology, 17(5), 605–619. https://doi.org/10.1089/cap.2007.0060.
Muthén, L. K., & Muthén, B. O. (2014). Mplus 7.4 [software]. Los Angeles: Muthén and Muthén.
Narad, M. E., Garner, A. A., Peugh, J. L., Tamm, L., Antonini, T. N., Kingery, K. M., Simon, J. O., & Epstein, J. N. (2015). Parent–teacher agreement on ADHD symptoms across development. Psychological Assessment, 27(1), 239–248. https://doi.org/10.1037/a0037864.
Purpura, D. J., & Lonigan, C. J. (2009). Conners' teacher rating scale for preschool children: A revised, brief, age-specific measure. Journal of Clinical Child & Adolescent Psychology, 38(2), 263–272. https://doi.org/10.1080/15374410802698446.
Putnick, D. L., & Bornstein, M. H. (2016). Measurement invariance conventions and reporting: The state of the art and future directions for psychological research. Developmental Review, 41, 71–90. https://doi.org/10.1016/j.dr.2016.06.004.
Rabiner, D., Coie, J. D., & Conduct Problems Prevention Research Group. (2000). Early attention problems and children's reading achievement: A longitudinal investigation. Journal of the American Academy of Child & Adolescent Psychiatry, 39(7), 859–867. https://doi.org/10.1097/00004583-200007000-00014.
Richters, J. E. (1992). Depressed mothers as informants about their children: a critical review of the evidence for distortion. Psychological bulletin, 112(3), 485.
Richters, J., & Pellegrini, D. (1989). Depressed mothers’ judgments about their children: an examination of the depression-distortion hypothesis. Child Development, 60, 1068–1075. https://doi.org/10.2307/1130780.
Roberts, M. A., Milich, R., Loney, J., & Caputo, J. (1981). A multitrait-multimethod analysis of variance of teachers' ratings of aggression, hyperactivity, and inattention. Journal of Abnormal Child Psychology, 9(3), 371–380. https://doi.org/10.1007/BF00916841.
Sass, D. A., Schmitt, T. A., & Marsh, H. W. (2014). Evaluating model fit with ordered categorical data within a measurement invariance framework: A comparison of estimators. Structural Equation Modeling: A Multidisciplinary Journal, 21(2), 167–180. https://doi.org/10.1080/10705511.2014.882658.
Shi, D., Song, H., & Lewis, M. D. (2019). The impact of partial factorial invariance on cross-group comparisons. Assessment, 26(7), 1217–1233. https://doi.org/10.1177/1073191117711020.
Sims, D. M., & Lonigan, C. J. (2013). Inattention, hyperactivity, and emergent literacy: Different facets of inattention relate uniquely to preschoolers' reading-related skills. Journal of Clinical Child & Adolescent Psychology, 42(2), 208–219. https://doi.org/10.1080/15374416.2012.738453.
Sonuga-Barke, E. J., Koerting, J., Smith, E., McCann, D. C., & Thompson, M. (2011). Early detection and intervention for attention-deficit/hyperactivity disorder. Expert Review of Neurotherapeutics, 11(4), 557–563. https://doi.org/10.1586/ern.11.39.
Stanger, C., & Lewis, M. (1993). Agreement among parents, teachers, and children on internalizing and externalizing behavior problems. Journal of Clinical Child Psychology, 22(1), 107–116. https://doi.org/10.1207/s15374424jccp2201_11.
Swanson, J. M., Schuck, S., Porter, M. M., Carlson, C., Hartman, C. A., Sergeant, J. A., Clevenger, W., Wasdell, M., McCleary, R., Lakes, K., & Wigal, T. (2012). Categorical and dimensional definitions and evaluations of symptoms of ADHD: History of the SNAP and the SWAN rating scales. The International Journal of Educational and Psychological Assessment, 10(1), 51–70.
Timmons, K., Pelletier, J., & Corter, C. (2016). Understanding children’s self-regulation within different classroom contexts. Early Child Development and Care, 186(2), 249–267. https://doi.org/10.1080/03004430.2015.1027699.
Touliatos, J., & Lindholm, B. W. (1981). Congruence of parents' and teachers' ratings of children's behavior problems. Journal of Abnormal Child Psychology, 9(3), 347–354. https://doi.org/10.1007/BF00916839.
Treutler, C. M., & Epkins, C. C. (2003). Are discrepancies among child, mother, and father reports on children’s behavior related to parents' psychological symptoms and aspects of parent–child relationships? Journal of Abnormal Child Psychology, 31(1), 13–27. https://doi.org/10.1023/A:1021765114434.
Tripp, G., Schaughency, E. A., & Clarke, B. (2006). Parent and teacher rating scales in the evaluation of attention-deficit hyperactivity disorder: Contribution to diagnosis and differential diagnosis in clinically referred children. Journal of Developmental & Behavioral Pediatrics, 27(3), 209–218.
Vitoratou, S., Garcia-Rosales, A., Banaschewski, T., Sonuga-Barke, E., Buitelaar, J., Oades, R. D., Rothenberger, A., Steinhausen, H. C., Taylor, E., Faraone, S. V., & Chen, W. (2019). Is the endorsement of the attention deficit hyperactivity disorder symptom criteria ratings influenced by informant assessment, gender, age, and co-occurring disorders? A measurement invariance study. International Journal of Methods in Psychiatric Research, 28(4), e1794. https://doi.org/10.1002/mpr.1794.
Walcott, C. M., Scheemaker, A., & Bielski, K. (2010). A longitudinal investigation of inattention and preliteracy development. Journal of Attention Disorders, 14(1), 79–85. https://doi.org/10.1177/1087054709333330.
Willcutt, E. G., Nigg, J. T., Pennington, B. F., Solanto, M. V., Rohde, L. A., Tannock, R., Loo, S. K., Carlson, C. L., McBurnett, K., & Lahey, B. B. (2012). Validity of DSM-IV attention deficit/hyperactivity disorder symptom dimensions and subtypes. Journal of Abnormal Psychology, 121(4), 991–1010. https://doi.org/10.1037/a0027347.
Willcutt, E. G., & Pennington, B. F. (2000). Comorbidity of reading disability and attention-deficit/hyperactivity disorder: Differences by gender and subtype. Journal of Learning Disabilities, 33(2), 179–191. https://doi.org/10.1177/002221940003300206.
Willcutt, E. G., Pennington, B. F., & DeFries, J. C. (2000). Etiology of inattention and hyperactivity/impulsivity in a community sample of twins with learning difficulties. Journal of Abnormal Child Psychology, 28(2), 149–159. https://doi.org/10.1023/A:1005170730653.
Xia, Y., & Yang, Y. (2019). RMSEA, CFI, and TLI in structural equation modeling with ordered categorical data: The story they tell depends on the estimation methods. Behavior Research Methods, 51(1), 409–428. https://doi.org/10.3758/s13428-018-1055-2.
Youngstrom, E., Loeber, R., & Stouthamer-Loeber, M. (2000). Patterns and correlates of agreement between parent, teacher, and male adolescent ratings of externalizing and internalizing problems. Journal of Consulting and Clinical Psychology, 68(6), 1038–1050. https://doi.org/10.1037/0022-006X.68.6.1038.
Funding
Portions of this research were supported by grants from the Institute of Educational Science (R305F100027) and the Eunice Kennedy Shriver National Institute of Child Health and Human Development (HD052120). The views expressed herein are those of the authors and have not been reviewed or approved by the granting agencies.
Author information
Authors and Affiliations
Corresponding authors
Ethics declarations
Conflict of Interest
Colleen M. Jungersen and Christopher J. Lonigan declare that they have no conflict of interest.
Experiment Participants
The previous study, during which the data used in this study was collected, was reviewed and approved by the Florida State University Institutional Review Board with HSC Number: 2015.15149. The approval allowed for de-identified data to be used for future studies, such as the current one.
Informed Consent
Prior to any data collection, district, school, and classroom agreement to participate in the study was obtained, and informed consent/permission was obtained from the children’s parents. Assent was obtained from the children who participated in this study.
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Supplementary Information
ESM 1
(DOCX 25 kb)
Rights and permissions
About this article
Cite this article
Jungersen, C.M., Lonigan, C.J. Do Parent and Teacher Ratings of ADHD Reflect the Same Constructs? A Measurement Invariance Analysis. J Psychopathol Behav Assess 43, 778–792 (2021). https://doi.org/10.1007/s10862-021-09874-3
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10862-021-09874-3