Skip to main content
SpringerLink
Log in

A Systematic Review of Treatment Integrity Assessment from 2004 to 2014: Examining Behavioral Interventions for Students with Autism Spectrum Disorder

  • Systematic Review
  • Published:
Contemporary School Psychology Aims and scope Submit manuscript

Abstract

Many students with autism spectrum disorder (ASD) receive behavioral interventions to improve academic and prosocial functioning and remediate current skill deficits. Sufficient treatment integrity is necessary for these interventions to be successful. However, a literature review of studies that evaluated behavioral intervention for students with ASD from 1993 to 2003 found that only 18% of studies included an operational definition of the intervention and measured treatment integrity. The purpose of the present study was to update this review and incorporate recent advances in implementation science (e.g., dimensions, implementation support). Of the 130 studies reviewed, treatment integrity was assessed in 43% of studies. When treatment integrity data were included, it was most often adherence data that were assessed through observation or self-report. Implications for future research and practice are discussed.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Baer, D. M., Wolf, M. M., & Risley, T. R. (1968). Some current dimensions of applied behavior analysis. Journal of Applied Behavior Analysis, 1, 91–97.

    Article  PubMed  PubMed Central  Google Scholar 

  • Centers for Disease Control and Prevention. (2015). Autism spectrum disorder. Retrieved from. http://www.cdc.gov/ncbddd/autism/data.html

  • Codding, R. S., Livanis, A., Pace, G. M., & Vaca, L. (2008). Using performance feedback to improve treatment integrity of class wide behavior plans: an investigation of observer reactivity. Journal of Applied Behavior Analysis, 41, 417–422.

    Article  PubMed  PubMed Central  Google Scholar 

  • Collier-Meek, M. A., & Sanetti, L. M. H. (2018). Treatment integrity of a homework intervention: evaluating parent and student adherence, quality, and program differentiation. International Journal of School and Educational Psychology., 1–13. https://doi.org/10.1080/21683603.2017.1376726.

  • Collier-Meek, M. A., Fallon, L. M., Sanetti, L. M. H., & Maggin, D. M. (2013). Focus on implementation: strategies for problem-solving teams to assess and promote treatment fidelity. Teaching Exceptional Children, 45, 52–59.

    Article  Google Scholar 

  • Collier-Meek, M. A., Sanetti, L. M. H., Fallon, L. M., & Chafouleas, S. M. (2018). An exploratory investigation of the influences of observation timing on treatment fidelity estimates. Manuscript submitted for publication.

  • Cooper, J. O., Heron, T. E., & Heward, W. L. (2007). Applied behavior analysis (2nd ed.). Upper Saddle River, NJ: Pearson.

    Google Scholar 

  • Council for Exceptional Children. (2014). Standards for evidence-based practices in special education. Retrieved from https://www.cec.sped.org/~/media/Files/Standards/Evidence%20based%20Practices%20and%20Practice/EBP%20FINAL.pdf.

  • Dawson, G., & Watling, R. (2000). Interventions to facilitate auditory, visual, and motor integration in autism: a review of the evidence. Journal of Autism and Developmental Disorders, 30, 415–421.

    Article  PubMed  Google Scholar 

  • DiGennaro Reed, F. D., & Codding, R. S. (2014). Advancements in procedural fidelity assessment and intervention: introduction to the special issue. Journal of Behavioral Education, 23, 1–18. https://doi.org/10.1007/s10864-013-9191-3.

    Article  Google Scholar 

  • DiGennaro Reed, F. D., Reed, D. D., Baez, C. N., & Maguire, H. (2011). A parametric analysis of errors of commission during discrete-trial training. Journal of Applied Behavior Analysis, 44, 611–615. https://doi.org/10.1901/jaba.2011.44-611.

    Article  PubMed  PubMed Central  Google Scholar 

  • DiGennaro, F. D., Martens, B. K., & Kleinmann, A. E. (2007). A comparison of performance feedback on teachers’ treatment implementation integrity and students’ inappropriate behavior in special education classrooms. Journal of Applied Behavior Analysis, 40, 447–461. https://doi.org/10.1901/jaba.2007.40447.

    Article  PubMed  PubMed Central  Google Scholar 

  • Elliott, S. N., & Treuting, M. V. (1991). The Behavior Intervention Rating Scale: development and validation of a pretreatment acceptability and effectiveness measure. Journal of School Psychology, 29, 43–51.

    Article  Google Scholar 

  • Fallon, L. M., Collier-Meek, M. A., Sanetti, L. M. H., Feinberg, A. B., & Kratochwill, T. R. (2015). Implementation planning to promote parents’ treatment integrity of behavioral interventions for children with autism. Journal of Educational and Psychological Consultation, 25, 1–23. https://doi.org/10.1080/10474412.2015.1039124.

    Article  Google Scholar 

  • Gresham, F. M., Gansle, K. A., Noell, G. (1993). Treatment integrity in applied behavior analysis with children. Journal of Applied Behavior Analysis, 26, 257–263.

  • Howlin, P., Magiati, I., & Charman, T. (2009). Systematic review of early intensive behavioral interventions for children with autism. American Association on Intellectual and Developmental Disabilities, 114, 23–41.

    Article  Google Scholar 

  • Lane, K. L., Kalberg, J. R., Bruhn, A. L., Driscoll, S. A., Wehby, J. H., & Elliott, S. N. (2009). Assessing social validity of school-wide positive behavior support plans: evidence for the reliability and structure of the primary intervention rating scale. School Psychology Review, 38, 135–144.

    Google Scholar 

  • McIntyre, L. L., Gresham, F. M., DiGennaro, F. D., & Reed, D. D. (2007). Treatment integrity of school-based intervention with children in the Journal of Applied Behavior Analysis 1991–2005. Journal of Applied Behavior Analysis, 40, 659–672. https://doi.org/10.1901/jaba.2007.659-672.

    Article  PubMed  Google Scholar 

  • Mouzakitis, A., Codding, R. S., & Tryon, G. (2015). The effects of self-monitoring and performance feedback on the treatment integrity of behavior intervention plan implementation and generalization. Journal of Positive Behavior Interventions, 17, 223–234.

    Article  Google Scholar 

  • National Autism Center. (2015). Findings and conclusions: National Standards Project, phase 2. Retrieved from http://www.nationalautismcenter.org/090605-2/

  • Noell, G. H., & Gansle, K. A. (2014). The use of performance feedback to improve intervention implementation in schools. In L. M. Sanetti & T. R. Kratochwill (Eds.), Treatment integrity: a foundation for evidence-based practice in applied psychology (pp. 161–183). Washington, DC: American Psychological Association.

    Chapter  Google Scholar 

  • Noell, G. H., Witt, J. C., Gilbertson, D. N., Ranier, D. D., & Freeland, J. T. (1997). Increasing teacher intervention implementation in general education settings through consultation and performance feedback. School Psychology Quarterly, 12, 77–88.

    Article  Google Scholar 

  • Noell, G. H., Gresham, F. M., & Gansle, K. A. (2002). Does treatment integrity matter? A preliminary investigation of instructional implementation and mathematics performance. Journal of Behavioral Education, 11, 51–67.

    Article  Google Scholar 

  • Noell, G. H., Witt, J. C., Slider, N. J., Connell, J. E., Gatti, S. L., Williams, K. L., et al. (2005). Treatment implementation following behavioral consultation in schools: a comparison of three follow-up strategies. School Psychology Review, 34, 87–106 ISSN 0279–6015.

    Google Scholar 

  • Peterson, L., Horner, A., & Wonderlich, S. (1982). The integrity of independent variables in behavior analysis. Journal of Applied Behavior Analysis, 15, 477–492. https://doi.org/10.1901/jaba.1982.15-477.

    Article  PubMed  PubMed Central  Google Scholar 

  • Power, T. J., Blom-Hoffman, J., Clarke, A. T., Riley-Tillman, T. C., Kelleher, C., & Manz, P. (2005). Reconceptualizing intervention integrity: a partnership-based framework for linking research with practice. Psychology in the Schools, 42, 495–507. https://doi.org/10.1002/pits.20087.

    Article  Google Scholar 

  • Sanetti, L. M. H., & Collier-Meek, M. A. (2014). Increasing the rigor of procedural fidelity assessment: an empirical comparison of direct observation and permanent product review methods. Journal of Behavioral Education, 23, 60–88. https://doi.org/10.1007/s10864-013-9179-z.

    Article  Google Scholar 

  • Sanetti, L. M. H., & Collier-Meek, M. A. (2019). Supporting successful interventions in schools: tools to plan, evaluate, and sustain effective implementation. New York: Guilford Press.

    Google Scholar 

  • Sanetti, L. M. H., & Fallon, L. M. (2012). Treatment integrity assessment: how estimates of adherence, quality, and exposure influence interpretation of implementation. Journal of Educational and Psychological Consultation, 21, 209–232. https://doi.org/10.1080/10474412.2011.595163.

    Article  Google Scholar 

  • Sanetti, L. M. H., & Kratochwill, T. R. (2009). Toward developing a science of treatment integrity: introduction to the special series. School Psychology Review, 38, 445–459. https://doi.org/10.1080/10474412.2011.620927.

    Article  Google Scholar 

  • Sanetti, L. M. H., Gritter, K. L., & Dobey, L. (2011). Treatment integrity of interventions with children in the school psychology literature from 1995 to 2008. School Psychology Review, 40, 72–84.

    Google Scholar 

  • Sanetti, L. M. H., Collier-Meek, M. A., Long, A. C. J., Byron, J. R., & Kratochwill, T. R. (2015). Increasing teacher treatment integrity of behavior support plans through consultation and implementation planning. Journal of School Psychology, 53, 209–229.

    Article  Google Scholar 

  • Sarokoff, R. A., & Sturmey, P. (2004). The effects of behavioral skills training on staff implementation of discrete-trial teaching. Journal of Applied Behavior Analysis, 37, 535–538.

    Article  PubMed  PubMed Central  Google Scholar 

  • Schulte, A. C., Easton, J. E., & Parker, J. (2009). Advances in treatment integrity research: multidisciplinary perspectives on the conceptualization, measurement, and enhancement of treatment integrity. School Psychology Review, 38, 460–475.

    Google Scholar 

  • Shadish, W. R., Cook, T. D., & Campbell, D. T. (2002). Experimental and quasi-experimental designs for generalized causal inference. Boston: Houghton Mifflin.

    Google Scholar 

  • Sterling-Turner, H. E., Watson, T. S., & Moore, J. W. (2002). The effects of direct training and treatment integrity on treatment outcomes in school consultation. School Psychology Quarterly, 17, 47–77. https://doi.org/10.1037/h0089179.

    Article  Google Scholar 

  • Watkins, M. W., Crosby, E. G., & Pearson, J. L. (2001). Role of the school psychologist: perceptions of school staff. School Psychology International, 22, 64–73.

    Article  Google Scholar 

  • Wheeler, J. J., Baggett, B. A., Fox, J., & Blevins, L. (2006). Treatment integrity: a review of intervention studies conducted with children with autism. Focus on Autism and Other Developmental Disabilities, 21, 45–54.

    Article  Google Scholar 

  • Wickstrom, K. F., Jones, K. M., LaFleur, L. H., & Witt, J. C. (1998). An analysis of treatment integrity in school-based behavioral consultation. School Psychology Quarterly, 13, 141–154.

    Article  Google Scholar 

  • Wilder, D. A., Atwell, J., & Wine, B. (2006). The effects of varying levels of treatment integrity on child compliance during treatment with a three-step prompting procedure. Journal of Applied Behavior Analysis, 39, 369–373.

    Article  PubMed  PubMed Central  Google Scholar 

  • Witt, J. C., & Elliott, S. N. (1985). Acceptability of classroom intervention strategies. In T. R. Kratochwill (Ed.), Advances in school psychology (pp. 251–288). Hillsdale: Erlbaum.

    Google Scholar 

  • Witt, J. C., & Martens, B. K. (1983). Assessing the acceptability of behavioral interventions used in classrooms. Psychology in the Schools, 20, 510–517.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Counseling and School Psychology, University of Massachusetts Boston, 100 Morrissey Blvd, Boston, MA, 02125, USA

    Kaitlin M. Gould, Melissa Collier-Meek, Emily R. DeFouw, Meghan Silva & Whitney Kleinert

Authors
  1. Kaitlin M. Gould
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Melissa Collier-Meek
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Emily R. DeFouw
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Meghan Silva
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Whitney Kleinert
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Kaitlin M. Gould.

Ethics declarations

On behalf of all authors, the corresponding author states that there is no conflict of interest. This article does not contain any studies with human participants performed by any of the authors.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Gould, K.M., Collier-Meek, M., DeFouw, E.R. et al. A Systematic Review of Treatment Integrity Assessment from 2004 to 2014: Examining Behavioral Interventions for Students with Autism Spectrum Disorder. Contemp School Psychol 23, 220–230 (2019). https://doi.org/10.1007/s40688-019-00233-4

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s40688-019-00233-4

Keywords

Search

Navigation