Advertisement

Journal of Autism and Developmental Disorders

, Volume 46, Issue 7, pp 2340–2348 | Cite as

The Relationship Between Clinicians’ Confidence and Accuracy, and the Influence of Child Characteristics, in the Screening of Autism Spectrum Disorder

  • Darren HedleyEmail author
  • Neil Brewer
  • Rose Nevill
  • Mirko Uljarević
  • Eric Butter
  • James A. Mulick
Original Paper

Abstract

The study examined the confidence accuracy relationship, and the influence of child characteristics on clinician confidence, when predicting a diagnosis of Autism Spectrum Disorder during screening of 125 referred children aged under 3.5 years. The diagnostic process included observation, interview, language and developmental testing. Clinical judgement accuracy was compared against final diagnosis for high and low confidence levels (with confidence assessed on a 0–100 % scale). We identified a significant CA relationship with predictive accuracy highest at confidence levels of 90–100 %. Parent report of unusual behaviors was the only significant independent predictor of confidence. Clinicians’ confidence may be important when evaluating decisions to refer, or not to refer, children for further diagnostic assessment.

Keywords

Autism Spectrum Disorder Screening Diagnosis Clinical judgement Confidence Accuracy Assessment 

Notes

Acknowledgments

This research was supported by Nationwide Children’s Hospital Clinical and Translational Intramural Grant #203213 awarded to Darren Hedley. We would like to thank the staff and families who participated in the study. We also thank Sarah Beinkampen, Natalie Fields, Mark Jones, Emily Mariotti, Yessica Monroy Moreno and Brianna Murphy for their assistance with data collection, and Dr. Elizabeth Kryszak and Dr. Nathan Weber for their contributions to the study design.

Author Contributions

DH, NB and JAM conceived and designed the study. DH and RN collected the data. DH, NB, RN and MU performed the analysis. All authors contributed to the writing and review of the manuscript. All authors approved the final version.

Compliance with Ethical Standards

Conflict of interest

The authors confirm that there are no conflict of interest to declare.

Ethical Approval

All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.

Informed Consent

Informed consent was obtained from all individual participants included in the study where required to do so by the Institutional Review Board.

References

  1. Achenbach, T. M., & Rescorla, L. A. (2000). Manual for the ASEBA Preschool forms and profiles. Burlington, VT: University of Vermont Department of Psychiatry.Google Scholar
  2. American Psychiatric Association. (2013). Diagnostic and statistical manual of mental disorders (5th ed.). Washington, DC: American Psychiatric Publishing.Google Scholar
  3. Barbaro, J., Ridgway, L., & Dissanayake, C. (2011). Developmental surveillance of infants and toddlers by maternal and child health nurses in an Australian community-based setting: Promoting the early identification of Autism Spectrum Disorders. Journal of Pediatric Nursing, 26, 334–347.CrossRefPubMedGoogle Scholar
  4. Bauminger, N., Solomon, M., & Rogers, S. J. (2010). Internalizing and externalizing behaviors in ASD. Autism Research, 3, 101–112.CrossRefPubMedGoogle Scholar
  5. Brewer, N. (2006). Uses and abuses of eyewitness confidence. Legal and Criminological Psychology, 11, 3–23.CrossRefGoogle Scholar
  6. Brewer, N., & Wells, G. (2006). The confidence-accuracy relationship in eyewitness identification: Effects of lineup instructions, foil similarity, and target-absent base rates. Journal of Experimental Psychology: Applied, 12, 11–30.PubMedGoogle Scholar
  7. Carlin, A. S., & Hewitt, P. L. (1990). The discrimination of patient generated and randomly generated MMPIs. Journal of Personality Assessment, 54, 24–29.CrossRefPubMedGoogle Scholar
  8. Centers for Disease Control and Prevention. (2012). Prevalence of autism spectrum disorders—Autism and Developmental Disabilities Monitoring Network, 14 sites, United States, 2008. MMWR 2012; 61 (No. SS-3).Google Scholar
  9. Charman, T., & Baird, G. (2002). Practitioner review: Diagnosis of autism spectrum disorder in 2-and 3-year-old children. Journal of Child Psychology and Psychiatry, 43, 289–305.CrossRefPubMedGoogle Scholar
  10. Corsello, C., Akshoomoff, N., & Stahmer, A. (2013). Diagnosis of autism spectrum disorders in 2-year-olds: A study of community practice. Journal of Child Psychology and Psychiatry, 54, 178–185.CrossRefPubMedGoogle Scholar
  11. Dawson, G., Jones, E. J. H., Merkle, K., Venema, K., Lowy, R., Faja, S., & Webb, S. J. (2012). Early behavioral intervention is associated with normalized brain activity in young children with autism. Journal of the American Academy of Child and Adolescent Psychiatry, 51, 1150–1159.CrossRefPubMedPubMedCentralGoogle Scholar
  12. Dawson, G., Rogers, S., Munson, J., Smith, M., Winter, J., Greenson, J., & Varley, J. (2010). Randomized, controlled trial of an intervention for toddlers with autism: The Early Start Denver Model. Pediatrics, 125, 17–23.CrossRefGoogle Scholar
  13. Goldstein, S., & Naglieri, J. A. (2010). Autism Spectrum Rating Scales. North Tonawanda, NY: Multi Health Systems.Google Scholar
  14. Guthrie, W., Swineford, L., Nottke, C., & Wetherby, A. (2013). Early diagnosis of autism spectrum disorder: Stability and change in clinical diagnosis and symptom presentation. Journal of Child Psychology and Psychiatry, 54, 582–590.CrossRefPubMedPubMedCentralGoogle Scholar
  15. Hedley, D., Nevill, R. E., Monroy Moreno, Y., Fields, N., Wilkins, J., Butter, E., & Mulick, J. A. (2015). Efficacy of the ADEC in identifying Autism Spectrum Disorder in clinically referred toddlers in the US. Journal of Autism and Developmental Disorders, 45, 2337–2348.CrossRefPubMedGoogle Scholar
  16. Insabato, A., Pannunzi, M., Rolls, E. T., & Deco, G. (2010). Confidence-related decision making. Journal of Neurophysiology, 104, 539–547.CrossRefPubMedGoogle Scholar
  17. Levy, S. E., Giarelli, E., Lee, L. C., Schieve, L. A., Kirby, R. S., Cunniff, C., & Rice, C. E. (2010). Autism spectrum disorder and co-occurring developmental, psychiatric, and medical conditions among children in multiple populations of the United States. Journal of Developmental and Behavioral Pediatrics, 31, 267–275.CrossRefPubMedGoogle Scholar
  18. Lichtenstein, S., & Fischhoff, B. (1977). Do those who know more also know more about how much they know? Organizational Behavior and Human Performance, 20, 159–183.CrossRefGoogle Scholar
  19. Lichtenstein, S., Fischhoff, B., & Phillips, L. (1982). Calibration of probabilities: The state of the art to 1980. In D. Kahneman, P. Slavic, & A. Tversky (Eds.), Judgment under uncertainty: Heuristics and biases. New York: Cambridge University Press.Google Scholar
  20. Lord, C., Luyster, R. J., Gotham, K., & Guthrie, W. (2012a). Autism diagnostic observation schedule, Second Edition (ADOS-2) Manual (Part II): Toddler Module. Torrance, CA: Western Psychological Services.Google Scholar
  21. Lord, C., Risi, S., DiLavore, P. S., Shulman, C., Thurm, A., & Pickles, A. (2006). Autism from 2 to 9 years of age. Archives of General Psychiatry, 63, 694–701.CrossRefPubMedGoogle Scholar
  22. Lord, C., Rutter, M., DiLavore, P. C., Risi, S., Gotham, K., & Bishop, S. L. (2012b). Autism diagnostic observation schedule, Second Edition (ADOS-2) Manual (Part I): Modules 1–4. Torrance, CA: Western Psychological Services.Google Scholar
  23. Messinger, D., Young, G. S., Ozonoff, S., Dobkins, K., Carter, A., Zwaigenbaum, L., & Sigman, M. (2013). Beyond autism: A baby siblings research consortium study of high-risk children at three years of age. Journal of the American Academy of Child and Adolescent Psychiatry, 52, 300–308.CrossRefPubMedPubMedCentralGoogle Scholar
  24. Miller, D. J., Spengler, E. S., & Spengler, P. M. (2015). A meta-analysis of confidence and judgment accuracy in clinical decision making. Journal of Counselling Psychology, 62, 553–567.CrossRefGoogle Scholar
  25. Mullen, E. M. (1995). Mullen scales of early learning (AGS ed.). Circle Pines, MN: American Guidance Service Inc.Google Scholar
  26. Murphy, A. H., & Winkler, R. L. (1974). Credible interval temperature forecasting: Some experimental results. Monthly Weather Review, 102, 784–794.CrossRefGoogle Scholar
  27. National Autism Center. (2015). Findings and conclusions: National Standard Project, Phase 2: Addressing the need for evidence-based practice guidelines for Autism Spectrum Disorder. Randolph, MA: National Autism Center.Google Scholar
  28. Oskamp, S. (1965). Overconfidence in case-study judgements. Journal of Consulting Psychology, 29, 261–265.CrossRefPubMedGoogle Scholar
  29. Ozonoff, S., Heung, K., Byrd, R., Hansen, R., & Hertz-Picciotto, I. (2008). The onset of Autism: Patterns of symptom emergence in the first years of life. Autism Research, 1, 320–328.CrossRefPubMedPubMedCentralGoogle Scholar
  30. Ozonoff, S., Young, G. S., Steinfeld, M. B., Hill, M. M., Cook, I., Hutman, T., & Sigman, M. (2009). How early do parent concerns predict later Autism diagnosis? Journal of Developmental and Behavioral Pediatrics, 30, 367–375.CrossRefPubMedPubMedCentralGoogle Scholar
  31. Ronis, D. L., & Yates, J. F. (1987). Components of probability judgement accuracy: Individual consistency and effects of subject matter and assessment method. Organizational Behavior and Human Decision Processes, 40, 193–218.CrossRefGoogle Scholar
  32. Shimabukuro, T. T., Grosse, S. D., & Rice, C. (2008). Medical expenditures for children with an autism spectrum disorder in a privately insured population. Journal of Autism and Developmental Disorders, 38, 546–552.CrossRefPubMedGoogle Scholar
  33. Simonoff, E., Pickles, A., Charman, T., Chandler, S., Loucas, T., & Baird, G. (2008). Psychiatric disorders in children with autism spectrum disorders: Prevalence, comorbidity, and associated factors in a population-derived sample. Journal of the American Academy of Child and Adolescent Psychiatry, 47, 921–929.CrossRefPubMedGoogle Scholar
  34. Sparrow, S., Cicchetti, D., & Balla, D. (2005). Vineland adaptive behavior scales (2nd ed.). Minneapolis, MN: Pearson Assessment.Google Scholar
  35. Wiggins, L. D., Baio, J., & Rice, C. (2006). Examination of the time between first evaluation and first Autism Spectrum Diagnosis in a population-based sample. Developmental and Behavioral Pediatrics, 27, 79–87.CrossRefGoogle Scholar
  36. Wolff, J., Botteron, K. N., Dager, S. R., Elison, J. T., Estes, A. M., Gu, H., & Piven, J. (2014). Longitudinal patterns of repetitive behavior in toddlers with autism. Journal of Child Psychology and Psychiatry, 55, 945–953.CrossRefPubMedPubMedCentralGoogle Scholar
  37. Young, R. (2007). Autism detection in early childhood: ADEC. Camberwell, VIC: ACER Press.Google Scholar

Copyright information

© Springer Science+Business Media New York 2016

Authors and Affiliations

  • Darren Hedley
    • 1
    • 2
    • 3
    Email author
  • Neil Brewer
    • 4
  • Rose Nevill
    • 2
  • Mirko Uljarević
    • 1
    • 5
  • Eric Butter
    • 2
    • 3
  • James A. Mulick
    • 2
  1. 1.Olga Tennison Autism Research Centre, School of Psychology and Public HealthLa Trobe UniversityMelbourneAustralia
  2. 2.Department of PediatricsOhio State UniversityColumbusUSA
  3. 3.Child Development CenterNationwide Children’s HospitalColumbusUSA
  4. 4.School of PsychologyFlinders UniversityAdelaideAustralia
  5. 5.Cooperative Research Centre for Living with Autism (Autism CRC)BrisbaneAustralia

Personalised recommendations