Abstract
Although research in autism has grown more sophisticated, the gap between research knowledge and applicability of research in real world settings has grown. There have been a number of different reviews of evidence-based practices of treatments for young children with autism. Reviews which have critically evaluated the empirical evidence have not found any treatments that can be considered evidence-based. Reasons for this shortcoming are explored, and a new method for the evaluation of empirical evidence is provided. Future uses of this evaluative method are provided as well as a discussion of how this tool might aid in narrowing the research to practice gap.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
With regard to the treatment of single subject research, our concerns are twofold; some definitions did not allow single subject studies to be used in the determination of EBP (Francis 2005; Shavelson and Towne 2002), and no definition of EBP that included single subject research provided a way to synthesize research across both group and single subject research (Kratochwill and Stoiber 2002; Lonigan et al. 1998; Lord et al. 2001; Odom et al. 2005). Because much of the research on treatments for young children with autism uses single subject research methods (see Odom et al. 2003), we feel it is imperative that single subject research be included in a definition of EBP used to review treatments for young children with autism.
References
Baker, B. L., & Feinfield, K. A. (2003). Early intervention. Current Opinion in Psychiatry, 16, 503–509.
Bristol, M. M., Cohen, D. J., Costello, E. J., Denckla, M., Eckberg, T. J., Kallen, R., et al. (1996). State of the science in autism: Report to the National Institutes of Health. Journal of Autism and Developmental Disorders, 26, 121–154.
Chorpita, B. F. (2003). The frontier of evidence-based practice. In A. E. Kazdin & J. R. Weisz (Eds.), Evidence-based psychotherapies for children and adolescents (pp. 42–59). New York: Guilford.
Cicchetti, D. V. (2001). The precision of reliability and validity estimates re-visited: Distinguishing between clinical and statistical significance of sample size requirements. Journal of Clinical and Experimental Neuropsychology, 23, 695–700.
Cutspec, P. A. (2004). Origins of evidence-based approaches to best practice: Evidence-based medicine. Centerscope, 2(1), 1–12.
Doehring, P., Reichow, B., & Volkmar, F. R. (2007, March). Is it evidenced-based? How to evaluate claims of effectiveness for autism. Paper presented at the International Association for Positive Behavior Support Conference, Boston, MA.
Filipek, P. A., Accardo, P. J., Ashwal, S., Baranek, G. T., Cook, E. H., Jr., Dawson, G., et al. (2000). Practice parameter: Screening and diagnosis of autism. Neurology, 55, 468–479.
Francis, K. (2005). Autism interventions: A critical update. Developmental Medicine & Child Neurology, 47, 493–499.
Goodheart, C. D., Kazdin, A. E., & Sternberg, R. J. (2006). Evidence-based psychotherapy: Where practice and research meet. Washington, DC: American Psychological Association.
Greenhalgh, T. (2001). How to read a paper: The basics of evidence based medicine (2nd ed.). London: BMJ Books.
Gresham, F. M., Beebe-Frankenberger, M. E., MacMillan, D. L. (1999). A selective review of treatments for children with autism: Description and methodological considerations. The School Psychology Review, 28, 559–575.
Grice, J. W., Jackson, B. J., & McDaniel, B. L. (2006). Bridging the idiographic-nomothetic divide: A follow-up study. Journal of Personality, 74, 1191–1218.
Hamilton, J. (2005). Clinicians’ guide to evidence-based practice. Journal of American Academy of Child and Adolescent Psychiatry, 44, 494–498.
Horner, R. H., Carr, E. G., Halle, J., McGee, G., Odom, S., & Wolery, M. (2005). The use of single-subject research to identify evidence-based practice in special education. Exceptional Children, 71, 165–179.
Kasari, C. (2002). Assessing change in early intervention programs for young children with autism. Journal of Autism and Developmental Disorders, 32, 447–461.
Kazdin, A. E. (2001). Bridging the enormous gaps of theory with therapy research and practice. Journal of Clinical Child Psychology, 30, 59–66.
Kazdin, A. E., & Weisz, J. R. (Eds.). (2003). Evidence-based psychotherapies for children and adolescents. New York: Guilford.
Klin, A., Lang, J., Cicchetti, D. V., & Volkmar, F. (2000). Inter-rater reliability of clinical diagnosis and DSM-IV criteria for autistic disorder: Results of the DSM-IV autism field trial. Journal of Autism and Developmental Disorders, 30, 163–167.
Kratochwill, T. R., & Stoiber, K. C. (2002). Evidence-based interventions in school psychology: Conceptual foundations of the Procedural and Coding Manual of Division 16 and the Society for the Study of School Psychology Task Force. School Psychology Quarterly, 17, 341–389.
Landis, J. R., & Koch, G. G. (1977). The measure of observer agreement for categorical data. Biometrics, 3, 159–174.
Levin, J. R. (2002). How to evaluate the evidence of evidence-based interventions. School Psychology Quarterly, 17, 483–492.
Lonigan, C. J., Elbert, J. C., & Johnson, S. B. (1998). Empirically supported psychosocial interventions for children: An overview. Journal of Clinical Child Psychology, 27, 138–145.
Lord, C., Bristol-Power, M., Cafiero, J. M., Filipek, P. A., Gallagher, J. J., Harris, S. L., et al. (2001). Educating children with autism. Washington, DC: National Academy Press.
Lucas, S. M., & Cutspec, P. A. (2005). The role and process of literature searching in the preparation of research synthesis. Centerscope, 4(3), 1–26.
McHugo, G. J., & Drake, R. E. (2003). Finding and evaluating the evidence: A critical step in evidence-based medicine. Psychiatric Clinics of North America, 26, 821–831.
Norcross, J. C., Beutler, L. E., & Levant, R. F. (2006). Evidence-based practices in mental health: Debate and dialogue on the fundamental questions. Washington, DC: American Psychological Association.
Odom, S. L., Brantlinger, E., Gersten, R., Horner, R. H., Thompson, B., & Harris, K. R. (2005). Research in special education: Scientific methods and evidence-based practices. Exceptional Children, 71, 137–148.
Odom, S. L., Brown, W. H., Frey, T., Karasu, N., Smith-Canter, L. L., & Strain, P. S. (2003). Evidence-based practices for young children with autism: Contributions for single-subject design research. Focus on Autism and Other Developmental Disorders, 18, 166–175.
Petticrew, M., & Roberts, H. (2006). Systematic reviews in the social sciences: A practical guide. Malden, MA: Blackwell.
Reichow, B., Barton, E. E., Volkmar, F. R., & Cicchetti, D. V. (2007, May). The status of research on interventions for young children with autism spectrum disorders. Poster presented at the International Meeting for Autism Research, Seattle, WA.
Rogers, S. J. (1998). Empirically supported comprehensive treatments for young children with autism. Journal of Clinical Child Psychology, 27, 168–179.
Shavelson, R. J., & Towne, L. (Eds.). (2002). Scientific research in education. Washington, DC: National Academy Press.
Urbaniak, G. C., & Plous, S. (2006). Research randomizer (retrieved July 15, 2005 from http://www.randomizer.org).
Wampold, B. E., Lichtenberg, J. W., & Waehler, C. A. (2002). Principles of empirically supported interventions in counseling psychology. The Counseling Psychologist, 30, 197–217.
Weisz, J. R., & Hawley, K. M. (1998). Finding, evaluating, refining, and applying empirically supported treatments for children and adolescents. Journal of Clinical Child Psychology, 27, 202–216.
Weisz, J. R., Hawley, K. M., Pilkonis, P. A., Woody, S. R., & Follette, W. C. (2000). Stressing the (other) three rs in the search for empirically supported treatments: Review procedures, research quality, relevance to practice, and the public interest. Clinical Psychology: Science and Practice, 7, 243–258.
Acknowledgments
The authors would like to thank Erin Barton and Peter Doehring for their assistance in the development of the reliability of the rubrics. This manuscript was prepared while Brian Reichow was completing an internship at the Yale Child Study Center.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Reichow, B., Volkmar, F.R. & Cicchetti, D.V. Development of the Evaluative Method for Evaluating and Determining Evidence-Based Practices in Autism. J Autism Dev Disord 38, 1311–1319 (2008). https://doi.org/10.1007/s10803-007-0517-7
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10803-007-0517-7