Skip to main content
SpringerLink
Log in

Description of Criterion Validity of the Autism Spectrum Rating Scales 6–18 Parent Report: Initial Exploration in a Large Community Sample

  • Original Article
  • Published:
Child Psychiatry & Human Development Aims and scope Submit manuscript

Abstract

Research regarding psychometric properties of autism spectrum disorder questionnaires is lacking. This study explored the criterion validity of the ASRS 6–18 parent report (ASRS-6–18-P) in a large, well-characterized, real-world clinical sample of 422 children (\(\bar{X}_{\text{age}} = 10.04;\) autism [AUT] n = 139; non-autism [NOT] n = 283) evaluated with the Autism Diagnostic Observation Schedule-2, a gold-standard measure. Significant mean differences were observed for DSM-5, social, and unusual behaviors. Total, DSM-5, social, and unusual behaviors demonstrated significant correlations with ADOS-2 comparison scores (modules 1–3), but not with module 4 (raw) scores. DSM-5 and Unusual Behaviors demonstrated significant but poor AUCs (0.60). Findings with/without covariates (IQ/age) were overall similar. Sensitivity and specificity could not be optimized. The suggested cutpoint (T-score = 60) demonstrated unacceptably high false positive rates (> 76.33%). While findings suggest limited diagnostic utility of the ASRS-6–18-P, the sample’s complex psychiatric presentation and measurement error inherent in cutoff score application should be considered when generalizing results. Further research is recommended.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. American Psychiatric Association (2013) Diagnostic and statistical manual of mental disorders, 5th edn. American Psychiatric Publishing, Rosslyn

    Book  Google Scholar 

  2. Norris M, Lecavalier L (2010) Screening accuracy of level 2 autism spectrum disorder rating scales. Autism 14:263–284

    PubMed  Google Scholar 

  3. Mckenney ELW, Dorencz J, Bristol RM, Hall LP (2015) Publishing about ASD in school psychology journals. Psychol Schools 52:265–283

    Google Scholar 

  4. Falkmer T, Anderson K, Falkmer M, Horlin C (2013) Diagnostic procedures in autism spectrum disorders: a systematic literature review. Eur Child Adolesc Psychiatry 22:329–340

    PubMed  Google Scholar 

  5. Hampton J, Strand PS (2015) A review of level 2 parent-report instruments used to screen children aged 1.5–5 for autism: a meta-analytic update. J Autism Dev Disord 45:2519–2530

    PubMed  Google Scholar 

  6. Hus V, Bishop S, Gotham K, Huerta M, Lord C (2013) Factors influencing scores on the social responsiveness scale. J Child Psychol Psychiatry 54:216–224

    PubMed  Google Scholar 

  7. Rutter M, LeCouteur A, Lord C (2003) Autism diagnostic interview-revised. Western Psychological Services, Torrance

    Google Scholar 

  8. Lord C, Rutter M, DiLavore PC, Risi S, Gotham K, Bishop SL (2012) ADOS-2: autism diagnostic observation schedule, 2nd edn, (part I): modules 1-4. Western Psychological Services, Torrance

    Google Scholar 

  9. Allen RA, Robins DL, Decker SL (2008) Autism spectrum disorders: neurobiology and current assessment practices. Psychol Schools 45:905–917

    Google Scholar 

  10. Cook JR, Hausman EM, Jensen-Doss A, Hawley KM (2017) Assessment practices of child clinicians: results from a national survey. Assessment 24:210–221

    PubMed  Google Scholar 

  11. Pugliese CE, Kenworthy L, Bal VH, Wallace GL, Yerys BE, Maddox BB et al (2015) Replication and comparison of the newly proposed ADOS-2, module 4 algorithm in ASD without ID: a multi-site study. J Autism Dev Disord 45:3919–3931

    PubMed  PubMed Central  Google Scholar 

  12. Goldstein S, Naglieri JA (2013) Autism spectrum rating scales (ASRS) technical manual. Multi-Health Systems, New York

    Google Scholar 

  13. Goldstein S, Naglieri JA (2014) Autism spectrum rating scales (ASRS) technical report #2. Multi-Health Systems, New York

    Google Scholar 

  14. Zhou H, Zhang L, Luo X, Wu L, Zou X, Xia K et al (2017) Modifying the autism spectrum rating scale (6–18 years) to a chinese context: an exploratory factor analysis. Neurosci Bull 33:175–182

    PubMed  PubMed Central  Google Scholar 

  15. Borenstein M, Hedges LV, Higgins JPT, Rothstein HR (2009) Introduction to meta-analysis. Wiley, New York

    Google Scholar 

  16. Kreiser NL, White SW (2015) ASD traits and co-occurring psychopathology: the moderating role of gender. J Autism Dev Disord 45:3932–3938

    PubMed  Google Scholar 

  17. Tureck K, Matson JL, May A, Davis TE, Whiting SE (2013) Investigation of the rates of comorbid symptoms in children with ADHD compared to children with ASD. J Dev Phys Disabil 25:405–417

    Google Scholar 

  18. Youngstrom EA (2014) A primer on receiver operating characteristic analysis and diagnostic efficiency statistics for pediatric psychology: we are ready to ROC. J Pediatr Psychol 39:204–221

    PubMed  Google Scholar 

  19. Swets JA, Dawes RM, Monahan J, Bjork RA (2000) Psychological science can improve diagnostic decisions. Psychol Sci Public Interest 1:1–26

    PubMed  Google Scholar 

  20. Cheng D, Branscum AJ, Johnson WO (2012) Sample size calculations for ROC studies: parametric robustness and bayesian nonparametrics. Stat Med 31:131–142

    PubMed  Google Scholar 

  21. Tabachnick BG, Fidell LS (2012) Using multivariate statistics, 6th edn. Pearson, London

    Google Scholar 

  22. Wechsler D (2014) Technical and interpretative manual of the wechsler intelligence scale for children, 5th edn. PsychCorp, San Antonio

    Google Scholar 

  23. Wechsler D (2011) Technical and interpretative manual of the wechsler abbreviated scale of intelligence, 2nd edn. PsychCorp, San Antonio

    Google Scholar 

  24. Kaufman AS, Kaufman NL (2004) Kaufman assessment battery for children: technical manual, 2nd edn. American Guidance Service, Circle Pines

    Google Scholar 

  25. Roid GH (2003) Stanford–binet intelligence scales, 5th edn. Riverside Publishing, Itasca

    Google Scholar 

  26. Wechsler D (2003) Technical and interpretative manual of the wechsler intelligence scale for children, 4th edn. PsychCorp, San Antonio

    Google Scholar 

  27. Wechsler D (2012) Technical and interpretative manual of the wechsler preschool and primary scale of intelligence, 2nd edn. PsychCorp, San Antonio

    Google Scholar 

  28. Wechsler D (2008) Technical and interpretive manual of the wechsler adult intelligence scale, 4th edn. PsychCorp, San Antonio

    Google Scholar 

  29. Bracken B, McCallum S (2016) Universal nonverbal intelligence test, 2nd edn. Western Psychological Publishers, Torrance

    Google Scholar 

  30. Reynolds CR, Kamphaus RW (2003) Handbook of psychological and educational assessment of children: intelligence, aptitude, and achievement, 2nd edn. Guilford Press, New York

    Google Scholar 

  31. Riccio CA, Sullivan JR, Cohen MJ (2010) Neuropsychological assessment and intervention for childhood and adolescent disorders. Wiley, New York

    Google Scholar 

  32. Langmann A, Becker J, Poustka L, Becker K, Kamp-Becker I (2017) Diagnostic utility of the autism diagnostic observation schedule in a clinical sample of adolescents and adults. Res Autism Spectr Disord 34:34–43. https://doi.org/10.1016/j.rasd.2016.11.012

    Article  Google Scholar 

  33. Zander E, Willfors C, Berggren S, Choque-Olsson N, Coco C, Elmund A et al (2016) The objectivity of the autism diagnostic observation schedule (ADOS) in naturalistic clinical settings. Eur Child Adolesc Psychiatry 25:769–780

    PubMed  Google Scholar 

  34. Chojnicka I, Pisula E (2017) Adaptation and validation of the ADOS-2, polish version. Front Psychol 8:1916–1930

    PubMed  Google Scholar 

  35. Medda JE, Cholemkery H, Freitag CM (2019) Sensitivity and specificity of the ADOS-2 algorithm in a large german sample. J Autism Dev Disord 49:750–761

    PubMed  Google Scholar 

  36. Oosterling IJ, Roos S, Bildt AD, Lambregts-Rommelse NNJ, Jonge MD, Visser J et al (2010) Improved diagnostic validity of the ADOS revised algorithms: a replication study in an independent sample. J Autism Dev Disord 40:689–703

    PubMed  Google Scholar 

  37. Fusar-Poli L, Brondino N, Rocchetti M, Panisi C, Provenzani U, Damiani S, Politi P (2017) Diagnosing ASD in adults without ID: accuracy of the ADOS-2 and the ADI-R. J Autism Dev Disord 47:3370–3379

    PubMed  Google Scholar 

  38. Kamp-Becker I, Albertowski K, Becker J, Ghareman M, Langmann A, Mingebach T et al (2018) Diagnostic accuracy of the ADOS and ADOS-2 in clinical practice. Eur Child Adolesc Psychiatry 27:1193–1207

    PubMed  Google Scholar 

  39. Louwerse I, Eussen M, Ende J, Nijs P, Gool A, Dekker LP et al (2015) ASD symptom severity in adolescence of individuals diagnosed with PDD-NOS in childhood: stability and the relation with psychiatric comorbidity and societal participation. J Autism Dev Disord 45:3908–3918

    PubMed  Google Scholar 

  40. Camodeca A (2018) Utility of three N-item scales of the child behavior checklist 6–18 in autism diagnosis. Res Autism Spectrum Disord 51:75–85

    Google Scholar 

  41. Duchan JF (1998) Describing the unusual behavior of children with autism. J Commun Disord 31:93–112

    PubMed  Google Scholar 

  42. Hedley D, Brewer N, Nevill R, Uljarević M, Butter E, Mulick JA (2016) The relationship between clinicians’ confidence and accuracy, and the influence of child characteristics, in the screening of autism spectrum disorder. J Autism Dev Disord 46:2340–2348

    PubMed  Google Scholar 

  43. Dolan BK, Van Hecke AV, Carson AM, Karst JS, Stevens S, Schohl KA et al (2016) Brief report: assessment of intervention effects on in vivo peer interactions in adolescents with autism spectrum disorder (ASD). J Autism Dev Disord 46:2251–2259

    PubMed  PubMed Central  Google Scholar 

  44. Green JL, Sciberras E, Anderson V, Efron D, Rinehart N (2016) Association between autism symptoms and functioning in children with ADHD. Arch Dis Child 101:922–928

    PubMed  Google Scholar 

  45. Hinshaw SP, Carte ET, Sami N, Treuting JJ, Zupan BA (2002) Preadolescent girls with attention-deficit/hyperactivity disorder—II: neuropsychological performance in relation to subtypes and individual classification. J Consult Clin Psychol 70:1099–1111

    PubMed  Google Scholar 

  46. Perugini EM, Harvey EA, Lovejoy DW, Sandstrom K, Webb AH (2000) The predictive power of combined neuropsychological measures for attention-deficit/hyperactivity disorder in children. Child Neuropsychol 6:101–114

    PubMed  Google Scholar 

  47. Grzadzinski R, Dick C, Lord C, Bishop S (2016) Parent-reported and clinician-observed autism spectrum disorder (ASD) symptoms in children with attention deficit/hyperactivity disorder (ADHD): implications for practice under DSM-5. Mol Autism 7:7

    PubMed  PubMed Central  Google Scholar 

  48. Berry K, Russell K, Frost K (2018) Restricted and repetitive behaviors in autism spectrum disorder: a review of associated features and presentation across clinical populations. Curr Dev Disord Rep 5:108–115

    Google Scholar 

  49. Green D, Chandler S, Charman T, Simonoff E, Baird G (2016) Brief report: DSM-5 sensory behaviours in children with and without an autism spectrum disorder. J Autism Dev Disord 46:3597–3606

    PubMed  Google Scholar 

  50. Lam KSL, Bodfish JW, Piven J (2008) Evidence for three subtypes of repetitive behavior in autism that differ in familiality and association with other symptoms. J Child Psychol Psychiatry 49:1193–1200

    PubMed  PubMed Central  Google Scholar 

  51. Esbensen AJ, Seltzer MM, Lam KSL, Bodfish JW (2009) Age-related differences in restricted repetitive behaviors in autism spectrum disorders. J Autism Dev Disord 39:57–66

    PubMed  Google Scholar 

  52. Bishop SL, Richler J, Lord C (2006) Association between restricted and repetitive behaviors and nonverbal IQ in children with autism spectrum disorders. Child Neuropsychol 12:247–267

    PubMed  Google Scholar 

  53. Harrop C, McConachie H, Emsley R, Leadbitter K, Green J, PACT Consortium, The PACT Consortium (2014) Restricted and repetitive behaviors in autism spectrum disorders and typical development: cross-sectional and longitudinal comparisons. J Autism Dev Disord 44:1207–1219

    Google Scholar 

  54. Magyar CI, Pandolfi V (2017) Utility of the CBCL DSM-oriented scales in assessing emotional disorders in youth with autism. Res Autism Spectrum Disord 37:11–20

    Google Scholar 

  55. Joshi G, Petty C, Wozniak J, Henin A, Fried R, Galdo M et al (2010) The heavy burden of psychiatric comorbidity in youth with autism spectrum disorders: a large comparative study of a psychiatrically referred population. J Autism Dev Disord 40:1361–1370

    PubMed  Google Scholar 

  56. Rieske RD, Matson JL, Beighley JS, Cervantes PE, Goldin RL, Jang J (2015) Comorbid psychopathology rates in children diagnosed with autism spectrum disorders according to the DSM-IV-TR and the proposed DSM-5. Dev Neurorehabil 18:218

    PubMed  Google Scholar 

  57. Bent CA, Barbaro J, Dissanayake C (2017) Change in autism diagnoses prior to and following the introduction of DSM-5. J Autism Dev Disord 47:163–171

    PubMed  Google Scholar 

  58. Aldridge FJ, Gibbs VM, Schmidhofer K, Williams M (2012) Investigating the clinical usefulness of the social responsiveness scale (SRS) in a tertiary level, autism spectrum disorder specific assessment clinic. J Autism Dev Disord 42:294–300

    PubMed  Google Scholar 

  59. Barnard-Brak L, Brewer A, Chesnut S, Richman D, Schaeffer AM (2016) The sensitivity and specificity of the social communication questionnaire for autism spectrum with respect to age. Autism Res 9:838–845

    PubMed  Google Scholar 

  60. Mazefsky CA, Oswald DP (2006) The discriminative ability and diagnostic utility of the ADOS-G, ADI-R, and GARS for children in a clinical setting. Autism 10:533–549

    PubMed  Google Scholar 

  61. Sizoo BB, Horwitz EH, Teunisse JPWM, Kan CC, Vissers CTWM, Forceville EJM et al (2015) Predictive validity of self-report questionnaires in the assessment of autism spectrum disorders in adults. Autism 19:842–849

    PubMed  Google Scholar 

  62. Blader JC, Pliszka SR, Kafantaris V, Sauder C, Posner J, Foley CA et al (2016) Prevalence and treatment outcomes of persistent negative mood among children with attention-deficit/hyperactivity disorder and aggressive behavior. J Child Adolesc Psychopharmacol 26:164–173

    PubMed  PubMed Central  Google Scholar 

  63. Esbjørn BH, Hoeyer M, Dyrborg J, Leth I, Kendall PC (2010) Prevalence and co-morbidity among anxiety disorders in a national cohort of psychiatrically referred children and adolescents. J Anxiety Disord 24:866–872

    PubMed  Google Scholar 

  64. Vasiliadis H, Diallo FB, Rochette L, Smith M, Langille D, Lin E et al (2017) Temporal trends in the prevalence and incidence of diagnosed ADHD in children and young adults between 1999 and 2012 in Canada: a data linkage study. Can J Psychiatry 62:818–826

    PubMed  PubMed Central  Google Scholar 

  65. Warner LA, Song NK, Pottick KJ (2014) Outpatient psychotropic medication use in the US: a comparison based on foster care status. J Child Fam Stud 23:652–665

    Google Scholar 

  66. Pandolfi V, Magyar CI, Dill CA (2012) An initial psychometric evaluation of the CBCL 6–18 in a sample of youth with autism spectrum disorders. Res Autism Spectrum Disord 6:96–108

    Google Scholar 

  67. Ooi YP, Rescorla L, Ang RP, Woo B, Fung DSS (2011) Identification of autism spectrum disorders using the child behavior checklist in Singapore. J Autism Dev Disord 41:1147–1156

    PubMed  Google Scholar 

  68. So P, Greaves-Lord K, van der Ende J, Verhulst FC, Rescorla L, de Nijs PF (2012) Using the child behavior checklist and the teacher’s report form for identification of children with autism spectrum disorders. Autism 17:595–607

    PubMed  Google Scholar 

  69. Achenbach TM, Rescorla LA (2001) Manual for the ASEBA school-age forms & profiles. University of Vermont, Research Center for Children, Youth, and Families, Burlington

    Google Scholar 

  70. Reynolds CR, Kamphaus RW (2015) Behavior assessment system for children, 3rd edn. Pearson, Bloomington

    Google Scholar 

  71. Gilliam JE (2014) Gilliam Autism Rating Scale, 3rd edn. Pro-Ed, Austin

    Google Scholar 

  72. Myles B, Bock S, Simpson R (2001) Asperger Syndrome Diagnostic Scale manual. MHS, North Tonawanda

    Google Scholar 

  73. Constantino JN, Gruber CP (2012) Social Responsiveness Scale, 2nd edn. Western Psychological Services, Torrance

    Google Scholar 

  74. Kotov R, Krueger RF, Watson D, Achenbach TM, Althoff RR, Bagby RM et al (2017) The hierarchical taxonomy of psychopathology (HiTOP): a dimensional alternative to traditional nosologies. J Abnorm Psychol 126:454–477

    PubMed  Google Scholar 

  75. Ingersoll B, Hopwood CJ, Wainer A, Donnellan MB (2011) A comparison of three self-report measures of the Broad Autism Phenotype in a non-clinical sample. J Autism Dev Dis 12:1646–1657

    Google Scholar 

  76. Ruzich E, Allison C, Smith P, Watson P, Auyeung B, Ring H, Baron-Cohen S (2015) Measuring autistic traits in the general population: a systematic review of the autism-spectrum quotient (AQ) in a nonclinical population sample of 6,900 typical adult males and females. Mol Autism 6:2

    PubMed  Google Scholar 

  77. Wainer AL, Ingersoll BR, Hopwood CJ (2011) The structure and nature of the broader autism phenotype in a non-clinical sample. J Psychopathol Behav 33:459–469

    Google Scholar 

  78. Altman DG, Royston P (2006) The cost of dichotomizing continuous variables. BMJ Br Med J 332:1080

    Google Scholar 

  79. Royston P, Altman DG, Sauerbrei W (2006) Dichotimizing continuous predictors in multiple regression: a bad idea. Stat Med 25:127–141

    PubMed  Google Scholar 

  80. Barnwell-Ménard J, Cohen AA (2015) Effects of categorization method, regression type, and variable distribution on the inflation of Type-1 error rate when categorizing a confounding variable. Stat Med 34:936–949

    PubMed  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. The Pennsylvania State University, Beaver Campus, 100 University Drive, Monaca, PA, 15061, USA

    Amy Camodeca

Authors
  1. Amy Camodeca
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Amy Camodeca.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Ethical Approval

This article does not contain any studies with animals performed by any of the authors. All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards. For this type of study formal consent is not required.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (DOCX 36 kb)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Camodeca, A. Description of Criterion Validity of the Autism Spectrum Rating Scales 6–18 Parent Report: Initial Exploration in a Large Community Sample. Child Psychiatry Hum Dev 50, 987–1001 (2019). https://doi.org/10.1007/s10578-019-00899-0

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10578-019-00899-0

Keywords

Search

Navigation