What Drives Detection and Diagnosis of Autism Spectrum Disorder? Looking Under the Hood of a Multi-stage Screening Process in Early Intervention

  • R. Christopher SheldrickEmail author
  • Elizabeth Frenette
  • Juan Diego Vera
  • Thomas I. Mackie
  • Frances Martinez-Pedraza
  • Noah Hoch
  • Abbey Eisenhower
  • Angel Fettig
  • Alice S. Carter
Original Paper


U.S. guidelines for detecting autism emphasize screening and also incorporate clinical judgment. However, most research focuses on the former. Among 1,654 children participating in a multi-stage screening protocol for autism, we used mixed methods to evaluate: (1) the effectiveness of a clinical decision rule that encouraged further assessment based not only on positive screening results, but also on parent or provider concern, and (2) the influence of shared decision-making on screening administration. Referrals based on concern alone were cost-effective in the current study, and reported concerns were stronger predictors than positive screens of time-to-complete referrals. Qualitative analyses suggest a dynamic relationship between parents’ concerns, providers’ concerns, and screening results that is central to facilitating shared decision-making and influencing diagnostic assessment.


Autism spectrum disorder Screening Costs Decision-making Process assessment 



Autism spectrum disorders


Early intervention


Institute of Medicine/National Academy of Medicine


Brief infant and toddler social emotional assessment


Parent observation of social interaction


Screening tool for autism in toddlers



The ABCD Project Team gratefully acknowledges the numerous people who helped shape our learning over the past several years and who provided specific statements on this article, as well as support from HRSA and from NIMH grant R01MH104400. We also thank our Early Intervention collaborators for their enduring partnership and the caregivers who participated in this study for so generously sharing both their time and their experiences with us.

Author Contributions

RCS participated in the design of the study, conducted primary quantitative analyses, created figures, drafted sections of the manuscript, and directed the editing process. EF and JDV assisted with initial data analyses, writing of the initial draft, and editing of the final manuscript. TM conceptualized and directed all qualitative data analyses, wrote all qualitative sections, and edited the final manuscript. FMP conceptualized the overarching study design and edited the final manuscript. NH conducted descriptive data analyses, assisted with interpretation of results, and substantively edited the final manuscript. AE and AF participated in the design of the study, the interpretation of data, and the editing of the final manuscript. AS participated in the design of the study, conceptualization of the paper, interpretation of qualitative and quantitative evidence, and substantive revisions of the final manuscript.

Compliance with Ethical Standards

This research was supported in part by a NIMH grant to Drs. Carter and Sheldrick (R01MH104400). Dr. Sheldrick is the co-creator of the POSI, which is one of the two screeners used in this study. He conducts research related to this instrument but receives no royalties. Dr. Carter is the co-creator of the BITSEA, which is one of the two screeners used in this study. She receives royalties related to the licensing of this instrument.

Ethical Approval

Ethical approval was granted by the Institutional Review Board of the University of Massachusetts Boston. All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.

Informed Consent

Informed consent was obtained from all individual participants included in the study.


  1. Ægisdóttir, S., White, M. J., Spengler, P. M., Maugherman, A. S., Anderson, L. A., Cook, R. S., et al. (2006). The meta-analysis of clinical judgment project: Fifty-six years of accumulated research on clinical versus statistical prediction. The Counseling Psychologist, 34(3), 341–382. Scholar
  2. Balogh, E., Miller, B. T., Ball, J., & Institute of Medicine (U.S.). Committee on Diagnostic Error in Health Care. (2015). Improving diagnosis in health care. Washington, DC: The National Academies Press. Scholar
  3. Briggs-Gowan, M., & Carter, A. S. (2006). Brief infant toddler social emotional assessment (BITSEA). (pp. 17–19) London: Pearson.Google Scholar
  4. Brożek, J. L., Akl, E. A., Alonso-Coello, P., Lang, D., Jaeschke, R., Williams, J. W., … Schünemann, H. J. (2009, May). Grading quality of evidence and strength of recommendations in clinical practice guidelines: Part 1 of 3. An overview of the GRADE approach and grading quality of evidence about interventions. Allergy: European Journal of Allergy and Clinical Immunology. Scholar
  5. Calzada, L. R., Pistrang, N., & Mandy, W. P. (2012). High-functioning autism and Asperger’s disorder: Utility and meaning for families. Journal of Autism and Developmental Disorders, 42(2), 230–243.Google Scholar
  6. Creswell, J. W., Klassen, A. C., Clark, P., V. L., & Smith, C., K (2011). Best practices for mixed methods research in the health sciences. Bethesda: Commissioned by the Office of Behavioral and Social Sciences Research.Google Scholar
  7. Curran, G. M., Bauer, M., Mittman, B., Pyne, J. M., & Stetler, C. (2012). Effectiveness-implementation hybrid designs: Combining elements of clinical effectiveness and implementation research to enhance public health impact. Medical Care, 50(3), 217.Google Scholar
  8. Dawes, R., Faust, D., & Meehl, P. (1989). Clinical versus actuarial judgment. Science, 243(4899), 1668–1674. Scholar
  9. Fryback, D. G., & Thornbury, J. R. (1991). The efficacy of diagnostic imaging. Medical Decision Making, 11(2), 88–94.Google Scholar
  10. Fusch, P. I., & Ness, L. R. (2015). Are we there yet? Data saturation in qualitative research. The Qualitative Report, 20(9), 1408–1416.Google Scholar
  11. Gayes, L. A., & Steele, R. G. (2014). A meta-analysis of motivational interviewing interventions for pediatric health behavior change. Journal of Consulting and Clinical Psychology, 82(3), 521.Google Scholar
  12. Giserman Kiss, I., Feldman, M. S., Sheldrick, R. C., & Carter, A. S. (2017). Developing autism screening criteria for the brief infant toddler social emotional assessment (BITSEA). Journal of Autism and Developmental Disorders, 47(5), 1269–1277. Scholar
  13. Godoy, L., & Carter, A. S. (2013). Identifying and addressing mental health risks and problems in primary care pediatric settings: A model to promote developmental and cultural competence. American Journal of Orthopsychiatry, 83(1), 73–88. Scholar
  14. Godoy, L., Mian, N. D., Eisenhower, A. S., & Carter, A. S. (2014). Pathways to service receipt: Modeling parent help-seeking for childhood mental health problems. Administration and Policy in Mental Health and Mental Health Services Research, 41(4), 469–479. Scholar
  15. Grove, W. M., & Meehl, P. E. (1996). Comparative efficiency of informal (subjective, impressionistic) and formal (mechanical, algorithmic) prediction procedures: The clinical-statistical controversy. Psychology, Public Policy, and Law, 2(2), 293–323. Scholar
  16. Grove, W. M., Zald, D. H., Lebow, B. S., Snitz, B. E., & Nelson, C. (2000). Clinical versus mechanical prediction: A meta-analysis. Psychological Assessment, 12(1), 19–30. Scholar
  17. Guinchat, V., Chamak, B., Bonniau, B., Bodeau, N., Perisse, D., Cohen, D., & Danion, A. (2012). Very early signs of autism reported by parents include many concerns not specific to autism criteria. Research in Autism Spectrum Disorders, 6(2), 589–601.Google Scholar
  18. Johnson, C. P., & Myers, S. M. (2007). Identification and evaluation of children with autism spectrum disorders. Pediatrics, 120(5), 1183–1215.Google Scholar
  19. King, T. M., Tandon, S. D., Macias, M. M., et al. (2010). Implementing developmental screening and referrals: Lessons learned from a national project. Pediatrics, 125(2), 350–360.Google Scholar
  20. Lord, C., Risi, S., Lambrecht, L., Cook, E. H., Leventhal, B. L., DiLavore, P. C., et al. (2000). Autism diagnostic observation schedule (ADOS). Journal of Autism and Developmental Disorders. Scholar
  21. McPheeters, M. L., Weitlauf, A., Vehorn, A., Taylor, C., Sathe, N. A., Krishnaswami, S., et al. (2015). Screening for autism spectrum disorder in young children: A systematic evidence review for the U.S. preventive services task force. AHRQ Publication No., 13-05185, EF-1, (121), 202.Google Scholar
  22. Council on Children with Disabilities, Section on Developmental Behavioral Pediatrics, Bright Futures Steering Committee & Medical Home Initiatives for Children With Special Needs Project Advisory Committee. (2006). Identifying infants and young children with developmental disorders in the medical home: An algorithm for developmental surveillance and screening. Pediatrics, 118(1), 405–420. Scholar
  23. Meehl, P. E. (1959). A comparison of clinicians with five statistical methods of identifying psychotic MMPI profiles. Journal of Counseling Psychology, 6(2), 102–109. Scholar
  24. Meehl, P. E., & Rosen, A. (1955). Antecedent probability and the efficiency of psychometric signs, patterns, or cutting scores. Psychological Bulletin, 52(3), 194.Google Scholar
  25. Mullen, E. M. (1995). Mullen scales of early learning, AGS Edition: Manual and item administrative books. (pp. 1–92). Noida: American Guidance Services Inc.Google Scholar
  26. Ozonoff, S., Young, G. S., Landa, R. J., Brian, J., Bryson, S., Charman, T., … Zwaigenbaum, L. (2015). Diagnostic stability in young children at risk for autism spectrum disorder: A baby siblings research consortium study. Journal of Child Psychology and Psychiatry, 56(9), 988–998.Google Scholar
  27. Pauker, S. G., & Kassirer, J. P. (1980). The threshold approach to clinical decision making. New England Journal of Medicine, 302(20), 1109–1117.Google Scholar
  28. Pizur-Barnekow, K., Muusz, M., McKenna, C., O’Connor, E., & Cutler, A. (2013). Service coordinators’ perceptions of autism-specific screening and referral practices in early intervention. Topics in Early Childhood Special Education, 33(3), 153–161.Google Scholar
  29. Robins, D. L., Casagrande, K., Barton, M., Chen, C.-M. A., Dumont-Mathieu, T., & Fein, D. (2013). Validation of the modified checklist for autism in toddlers, revised with follow-up (M-CHAT-R/F). Pediatrics, 133(1), 37–45.Google Scholar
  30. Robins, D. L., Fein, D., Barton, M. L., & Green, J. A. (2001). The modified checklist for autism in toddlers: An initial study investigating the early detection of autism and pervasive developmental disorders. Journal of Autism and Developmental Disorders, 31(2), 131–144.Google Scholar
  31. Salisbury, L., Nyce, J. D., Hannum, C., Sheldrick, R. C., & Perrin, E. C. (2018). Sensitivity and Specificity of 2 Autism Screeners Among Referred Children Between 16 and 48 Months of Age. Journal of Developmental and Behavioral Pediatrics, 39, 254–258.Google Scholar
  32. Sanders, S., Doust, J., & Glasziou, P. (2015). A systematic review of studies comparing diagnostic clinical prediction rules with clinical judgment. PLOS ONE, 10(6), e0128233. Scholar
  33. Sheldrick, R. C., Benneyan, J. C., Kiss, I. G., Briggs-Gowan, M. J., Copeland, W., & Carter, A. S. (2015). Thresholds and accuracy in screening tools for early detection of psychopathology. Journal of Child Psychology and Psychiatry and Allied Disciplines, 56(9), 936–948. Scholar
  34. Sheldrick, R. C., Breuer, D. J., Hassan, R., Chan, K., Polk, D., & Benneyan, J. (2016). A system dynamics model of clinical decision thresholds for the detection of developmental-behavioral disorders. Implementation Science, 11, 1–14. Scholar
  35. Sheldrick, R. C., & Garfinkel, D. (2017). Is a Positive developmental-behavioral screening score sufficient to justify referral? A review of evidence and theory. Academic Pediatrics, 17(5), 464–470. Scholar
  36. Smith, N. J., Sheldrick, R. C., & Perrin, E. C. (2013). An abbreviated screening instrument for autism spectrum disorders. Infant Mental Health Journal, 34(2), 149–155. Scholar
  37. Sparrow, S. S., Cicchetti, D. V., & Saulnier, C. A. (2016). Vineland adaptive behavior scales, (3rd ed.). London: Pearson.Google Scholar
  38. Stahmer, A. C. (2007). The basic structure of community early intervention programs for children with autism: Provider descriptions. Journal of Autism and Developmental Disorders, 37(7), 1344–1354.Google Scholar
  39. Stahmer, A. C., Collings, N. M., & Palinkas, L. A. (2005). Early intervention practices for children with autism: Descriptions from community providers. Focus on Autism and Other Developmental Disabilities, 20(2), 66–79.Google Scholar
  40. Stone, W. L., Coonrod, E. E., & Ousley, O. Y. (2000). Brief report: Screening tool for autism in two-year-olds (STAT): Development and preliminary data. Journal of Autism and Developmental Disorders, 30(6), 607–612. Scholar
  41. Stone, W. L., McMahon, C. R., & Henderson, L. M. (2008). Use of the screening tool for autism in two-year-olds (STAT) for children under 24 months. Autism, 12(5), 557–573. Scholar
  42. Swets, J. A., Dawes, R. M., & Monahan, J. (2000). Psychological science can improve diagnostic decisions. Psychological Science in the Public Interest, 1(1), 1–26. Scholar
  43. Tobin, G. A., & Begley, C. M. (2004). Methodological rigour within a qualitative framework. Journal of Advanced Nursing, 48(4), 388–396.Google Scholar
  44. Trikalinos, T. A., Siebert, U., & Lau, J. (2009). Decision-analytic modeling to evaluate benefits and harms of medical tests: Uses and limitations. Medical Decision Making, 29(5), E22–E29. Scholar
  45. Wiggins, L. D., Levy, S. E., Daniels, J., Schieve, L., Croen, L. A., DiGuiseppi, C., et al. (2015). Autism spectrum disorder symptoms among children enrolled in the Study to Explore Early Development (SEED). Journal of Autism and Developmental Disorders, 45(10), 3183–3194.Google Scholar
  46. Willms, D. G., Best, J. A., Taylor, D. W., Gilbert, J. R., Wilson, D. M. C., Lindsay, E. A., & Singer, J. (1990). A Systematic approach for using qualitative methods in primary prevention research. Medical Anthropology Quarterly, 4(4), 391–409. Scholar
  47. Zuckerman, K. E., Sinche, B., Cobian, M., Cervantes, M., Mejia, A., Becker, T., & Nicolaidis, C. (2014a). Conceptualization of autism in the Latino community and its relationship with early diagnosis. Journal of Developmental and Behavioral Pediatrics: JDBP, 35(8), 522.Google Scholar
  48. Zuckerman, K. E., Sinche, B., Mejia, A., Cobian, M., Becker, T., & Nicolaidis, C. (2014b). Latino parents’ perspectives on barriers to autism diagnosis. Academic Pediatrics, 14(3), 301–308.Google Scholar
  49. Zwaigenbaum, L., Bauman, M. L., Fein, D., Pierce, K., Buie, T., Davis, P. A., et al. (2015). Early screening of autism spectrum disorder: Recommendations for practice and research. Pediatrics, 136(Supplement 1), S41–S59.Google Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2019

Authors and Affiliations

  • R. Christopher Sheldrick
    • 2
    • 6
    Email author
  • Elizabeth Frenette
    • 1
  • Juan Diego Vera
    • 1
  • Thomas I. Mackie
    • 3
    • 4
  • Frances Martinez-Pedraza
    • 5
  • Noah Hoch
    • 1
  • Abbey Eisenhower
    • 1
  • Angel Fettig
    • 1
  • Alice S. Carter
    • 1
  1. 1.Department of PsychologyUniversity of Massachusetts BostonBostonUSA
  2. 2.Department of Health Law, Policy and Management, School of Public HealthBoston UniversityBostonUSA
  3. 3.Rutgers School of Public HealthPiscatawayUSA
  4. 4.Institute for Health, Health Care Policy and Aging ResearchRutgers UniversityNew BrunswickUSA
  5. 5.Department of PsychologyFlorida International UniversityMiamiUSA
  6. 6.Boston UniversityBostonUSA

Personalised recommendations