Skip to main content
SpringerLink
Log in

Improving Treatment Plan Implementation in Schools: A Meta-analysis of Single Subject Design Studies

Journal of Behavioral Education volume 23, pages 168–191 (2014)Cite this article

Abstract

Twenty-nine peer-reviewed journal articles that analyzed intervention implementation in schools using single-case experimental designs were meta-analyzed. These studies reported 171 separate data paths and provided 3,991 data points. The meta-analysis was accomplished by fitting data extracted from graphs in mixed linear growth models. This analytic approach was adopted to provide a model whose assumptions were met and could model both level and trend. The extent to which interventions were implemented as planned in baseline was low with a downward trend. Performance feedback, performance feedback augmented with additional procedures, and self-monitoring with environmental supports were found to be highly effective for improving intervention implementation. Follow-up meetings without review of implementation data were ineffective. Maintenance data were positive across studies. Implications of the results for future research and practice are discussed.

This is a preview of subscription content, access via your institution.

Access options

Buy single article

Instant access to the full article PDF.

USD 39.95

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Fig. 1
Fig. 2
Fig. 3

References

* Indicates that the article was included in the meta-analysis.

  • *Belfiore, P. J., Fritts, K. M., & Herman, B. C. (2008). The role of procedural integrity: Using self-monitoring to enhance discrete trial instruction (DTI). Focus on Autism and Other Developmental Disabilities, 23, 95–102. doi:10.1177/1088357607311445.

    Article  Google Scholar 

  • Bergan, J. R., & Kratochwill, T. R. (1990). Behavioral consultation and therapy. New York: Plenum Press.

    Google Scholar 

  • *Casey, A. M., & McWilliam, R. A. (2011). The impact of checklist-based training on teachers’ use of the zone defense schedule. Journal of Applied Behavior Analysis, 44, 397–401. doi:10.1901/jaba.2011.44-397.

    Article  PubMed  Google Scholar 

  • *Codding, R. S., Feinberg, A. B., Dunn, E. K., & Pace, G. M. (2005). Effects of immediate performance feedback on implementation of behavior support plans. Journal of Applied Behavior Analysis, 38, 205–219. doi:10.1901/jaba.2005.98-04.

    Article  PubMed  Google Scholar 

  • *Codding, R. S., Livanis, A., Pace, G. M., & Vaca, L. (2008). Using performance feedback to improve treatment integrity of classwide behavior plans: An investigation of observer reactivity. Journal of Applied Behavior Analysis, 41, 417–422. doi:10.1901/jaba.2008.41-417.

    Article  PubMed  Google Scholar 

  • Cohen, J. (1992). A power primer. Psychological Bulletin, 112, 155–159. doi:10.1037/0033-2909.112.1.155.

    Article  PubMed  Google Scholar 

  • Collaborative for Academic, Social, and Emotional Learning (CASEL). (2003). Safe and sound: An educational leader’s guide to evidence-based social and emotional learning programs. Chicago, IL: Collaborative for Academic, Social, and Emotional Learning.

    Google Scholar 

  • Cooper, H., & Hedges, L. V. (1994). The handbook of research synthesis. New York: Russell Sage Foundation.

    Google Scholar 

  • *DiGennaro, F. D., Martens, B. K., & Kleinmann, A. E. (2007). A comparison of performance feedback on teachers’ treatment implementation integrity and students’ inappropriate behavior in special education classrooms. Journal of Applied Behavior Analysis, 40, 447–461. doi:10.1901/jaba.2007.40-447.

    Article  PubMed  Google Scholar 

  • *DiGennaro, F. D., Martens, B. K., & McIntyre, L. L. (2005). Increasing treatment integrity through negative reinforcement: Effects on teacher and student behavior. School Psychology Review, 34, 220–231.

    Google Scholar 

  • *Digennaro-Reed, F. D., Codding, R., Catania, C. N., & Maguire, H. (2010). Effects of video modeling on treatment integrity of behavioral interventions. Journal of Applied Behavior Analysis, 43, 291–295. doi:10.1901/jaba.2010.43-291.

    Article  PubMed  Google Scholar 

  • Dufrene, B. A., & Noell, G. H. (2005). Monitoring implementation of reciprocal peer tutoring: Identifying and intervening with students who do not maintain accurate implementation. School Psychology Review, 34, 74–86.

    Google Scholar 

  • *Fox, L., Hemmeter, M., Snyder, P., Binder, D. P., & Clarke, S. (2011). Coaching early childhood special educators to implement a comprehensive model for promoting young children’s social competence. Topics in Early Childhood Special Education, 31, 178–192. doi:10.1177/0271121411404440.

    Article  Google Scholar 

  • *Franzen, K., & Kamps, D. (2008). The utilization and effects of positive behavior support strategies on an urban school playground. Journal of Positive Behavior Interventions, 10, 150–161. doi:10.1177/1098300708316260.

    Article  Google Scholar 

  • Fuchs, L. S., Fuchs, D., & Compton, D. L. (2010). Rethinking response to intervention at middle and high school. School Psychology Review, 39, 22–28.

    Google Scholar 

  • Garbacz, S. A., Woods, K. E., Swanger-Gagné, M. S., Taylor, A. M., Black, K. A., & Sheridan, S. M. (2008). The effectiveness of a partnership-centered approach in conjoint behavioral consultation. School Psychology Quarterly, 23, 313–326. doi:10.1037/1045-3830.23.3.313.

    Article  Google Scholar 

  • *Gilbertson, D., Witt, J. C., Singletary, L. L., & Van Der Heyden, A. (2007). Supporting teacher use of interventions: Effects of response dependent performance feedback on teacher implementation of a math intervention. Journal of Behavioral Education, 16, 311–326. doi:10.1007/s10864-007-9043-0.

    Article  Google Scholar 

  • *Jones, K. M., Wickstrom, K. F., & Friman, P. C. (1997). The effects of observational feedback on treatment integrity in school-based behavioral consultation. School Psychology Quarterly, 12, 316–326.

    Article  Google Scholar 

  • Kallestad, J. H., & Olweus, D. (2003). Predicting teachers’ and schools’ implementation of the Olweus bullying prevention program: A multilevel study. Prevention & Treatment, 6(1), ArtID 21. doi:10.1037/1522-3736.6.1.621a.

  • *Kelleher, C., Riley-Tillman, T., & Power, T. J. (2008). An initial comparison of collaborative and expert-driven consultation on treatment integrity. Journal of Educational and Psychological Consultation, 18, 294–324. doi:10.1080/10474410802491040.

    Article  Google Scholar 

  • Kratochwill, T. R., Hitchcock, J., Horner, R. H., Levin, J. R., Odom, S. L., Rindskopf, D. M. & Shadish, W. R. (2010). Single-case design technical documentation. Retrieved from What Works Clearinghouse Website:http://ies.ed.gov/ncee/wwc/pdf/wwc_scd.pdf.

  • *LaFleur, L., Witt, J. C., Naquin, G., Harwell, V., & Gilbertson, D. M. (1998). Use of coaching to enhance proactive classroom management by improvement of student transitioning between classroom activities. Effective School Practices, 17, 72–84.

    Google Scholar 

  • Lipsey, M. W., & Wilson, D. B. (2001). Practical meta-analysis. Thousand Oaks, CA: Sage Publications.

    Google Scholar 

  • Mastropieri, M. A., & Scruggs, T. E. (1985). Early intervention for socially withdrawn children. Journal of Special Education, 19, 429–441. doi:10.1177/002246698501900407.

    Article  Google Scholar 

  • Moncher, F. J., & Prinz, R. J. (1991). Treatment fidelity in outcome studies. Clinical Psychology Review, 11, 247–266. doi:10.1016/0272-7358(91)90103-2.

    Article  Google Scholar 

  • *Mortenson, B. P., & Witt, J. C. (1998). The use of weekly performance feedback to increase teacher implementation of a prereferral intervention. School Psychology Review, 27, 613–627.

    Google Scholar 

  • Nock, M. K., & Kazdin, A. E. (2005). Randomized controlled trial of a brief intervention for increasing participation in parent management training. Journal of Consulting and Clinical Psychology, 73, 872–879. doi:10.1037/0022-006X.73.5.872.

    Article  PubMed  Google Scholar 

  • Noell, G. H. (2008). Research examining the relationships among consultation process, treatment integrity, and outcomes. In W. P. Erchul & S. M. Sheridan (Eds.), Handbook of research in school consultation: Empirical foundations for the field (pp. 323–342). Mahwah, NJ: Lawrence Erlbaum Associates, Inc.

    Google Scholar 

  • Noell, G. H. (2010). Empirical and pragmatic issues in assessing and supporting intervention implementation in schools. In G. G. Peacock, R. A. Ervin, E. J. Daly III, & K. W. Merrell (Eds.), Practical handbook of school psychology: Effective practices for the 21st century (pp. 513–530). New York: Guilford.

    Google Scholar 

  • *Noell, G. H., Duhon, G. J., Gatti, S. L., & Connell, J. E. (2002). Consultation, follow-up, and implementation of behavior management interventions in general education. School Psychology Review, 31, 217–234.

    Google Scholar 

  • Noell, G. H., & Gansle, K. A. (2008). Moving from good ideas in educational systems change to sustainable program implementation: Coming to terms with some of the realities. Psychology in the Schools, 46, 79–89. doi:10.1002/pits.20355.

    Article  Google Scholar 

  • *Noell, G. H., Witt, J. C., Gilbertson, D. N., Ranier, D. D., & Freeland, J. L. (1997). Increasing teacher intervention implementation in general education settings through consultation and performance feedback. School Psychology Quarterly, 12, 77–88. doi:10.1037/h0088949.

    Article  Google Scholar 

  • *Noell, G. H., Witt, J. C., LaFleur, L. H., Mortenson, B. P., Ranier, D. D., & LeVelle, J. (2000). Increasing intervention implementation in general education following consultation: A comparison of two follow-up strategies. Journal of Applied Behavior Analysis, 33, 271–284. doi:10.1901/jaba.2000.33-271.

    Article  PubMed  Google Scholar 

  • Noell, G. H., Witt, J. C., Slider, N. J., Connell, J. E., Gatti, S. L., Williams, K. L., et al. (2005). Treatment implementation following behavioral consultation in schools: A comparison of three follow-up strategies. School Psychology Review, 34, 87–106.

    Google Scholar 

  • Peterson, L., Homer, A. L., & Wonderlich, S. A. (1982). The integrity of independent variables in behavior analysis. Journal of Applied Behavior Analysis, 15, 477–492. doi:10.1901/jaba.1982.15-477.

    Article  PubMed  Google Scholar 

  • *Petscher, E. S., & Bailey, J. S. (2006). Effects of training, prompting, and self-monitoring on staff behavior in a classroom for students with disabilities. Journal of Applied Behavior Analysis, 39, 215–226. doi:10.1901/jaba.2006.02-05.

    Article  PubMed  Google Scholar 

  • *Plavnick, J. B., Ferreri, S. J., & Maupin, A. N. (2010). The effects of self-monitoring on the procedural integrity of a behavioral intervention for young children with developmental disabilities. Journal of Applied Behavior Analysis, 43, 315–320. doi:10.1901/jaba.2010.43-315.

    Article  PubMed  Google Scholar 

  • Raudenbush, S. W., & Bryk, A. S. (2002). Hierarchical linear models: Applications and data analysis methods (2nd ed.). London: Sage.

    Google Scholar 

  • *Reinke, W. M., Lewis-Palmer, T., & Martin, E. (2007). The effect of visual performance feedback on teacher use of behavior-specific praise. Behavior Modification, 31, 247–263. doi:10.1177/0145445506288967.

    Article  PubMed  Google Scholar 

  • *Rodriguez, B., Loman, S. L., & Horner, R. H. (2009). A preliminary analysis of the effects of coaching feedback on teacher implementation fidelity of First Step to Success. Behavior Analysis in Practice, 2(2), 11–21.

    PubMed Central  PubMed  Google Scholar 

  • *Sanetti, L. M. H., Luiselli, J. K., & Handler, M. W. (2007). Effects of verbal and graphic performance feedback on behavior support plan implementation in a public elementary school. Behavior Modification, 31, 454–465. doi:10.1177/0145445506297583.

    Article  Google Scholar 

  • *Sarokoff, R. A., & Sturmey, P. (2008). The effects of instructions, rehearsal, modeling, and feedback on acquisition and generalization of staff use of discrete trial teaching and student correct responses. Research in Autism Spectrum Disorders, 2, 125–136. doi:10.1016/j.rasd.2007.04.002.

    Article  Google Scholar 

  • Schulte, A. C., Easton, J. E., & Parker, J. (2009). Advances in treatment integrity research: Multidisciplinary perspectives on the conceptualization, measurement, and enhancement of treatment integrity. School Psychology Review, 38, 460–475.

    Google Scholar 

  • Sheridan, S. M., & Gutkin, T. B. (2000). The ecology of school psychology: Examining and changing our paradigm for the 21st century. School Psychology Review, 29, 485–502.

    Google Scholar 

  • Shernoff, E. S., & Kratochwill, T. R. (2007). Transporting an evidence-based classroom management program for preschoolers with disruptive behavior problems to a school: An analysis of implementation, outcomes, and contextual variables. School Psychology Quarterly, 22, 449–472. doi:10.1037/1045-3830.22.3.449.

    Article  Google Scholar 

  • *Smith, S. C., Lewis, T. J., & Stormont, M. (2011). The effectiveness of two universal behavioral supports for children with externalizing behavior in Head Start classrooms. Journal of Positive Behavior Interventions, 13, 133–143. doi:10.1177/1098300710379053.

    Article  Google Scholar 

  • Solomon, B. G., Klein, S. A., & Politylo, B. C. (2012). The effect of performance feedback on teachers’ treatment integrity: A meta-analysis of the single-case literature. School Psychology Review, 4, 160–175.

    Google Scholar 

  • Stage, S., Jackson, H., Jensen, M., Moscovitz, K., Bush, J., Violette, H., et al. (2008). A validity study of functionally-based behavioral consultation with students with emotional/behavioral disabilities. School Psychology Quarterly, 23, 327–353. doi:10.1037/1045-3830.23.3.327.

    Article  Google Scholar 

  • *Sterling-Turner, H. E., Watson, T. S., & Moore, J. W. (2002). The effects of direct training and treatment integrity on treatment outcomes in school consultation. School Psychology Quarterly, 17, 47–77. doi:10.1521/scpq.17.1.47.19906.

    Article  Google Scholar 

  • Stevens, J. (1996). Applied multivariate statistics for the social sciences. Mahwah, New Jersey: Lawrence Erlbaum.

    Google Scholar 

  • *Stormont, M. A., Smith, S. C., & Lewis, T. J. (2007). Teacher implementation of precorrection and praise statements in Head Start classrooms as a component of a program-wide system of positive behavior support. Journal of Behavioral Education, 16, 280–290. doi:10.1007/s10864-007-9040-3.

    Article  Google Scholar 

  • Van den Noortgate, W., & Onghena, P. (2003). Combining single-case experimental data using hierarchical linear models. School Psychology Quarterly, 18, 325–346. doi:10.1521/scpq.18.3.325.22577.

    Article  Google Scholar 

  • Verbeke, G., & Molenberghs, G. (2001). Linear mixed models for longitudinal data. New York: Springer.

    Google Scholar 

  • *Ward, P., Johnson, M., & Konukman, F. (1998). Directed rehearsal and preservice teachers’ performance of instructional behaviors. Journal of Behavioral Education, 8, 369–380. doi:10.1023/A:1022827415544.

    Article  Google Scholar 

  • *Ward, P., Smith, S., & Makasci, K. (1997). Teacher training: Effects of directed rehearsal on the teaching skills of physical education majors. Journal of Behavioral Education, 7, 505–517. doi:10.1023/A:1022863520165.

    Article  Google Scholar 

  • Wickstrom, K. F., Jones, K. M., LaFleur, L. H., & Witt, J. C. (1998). An analysis of treatment integrity in school-based behavioral consultation. School Psychology Quarterly, 13, 141–154. doi:10.1037/h0088978.

    Article  Google Scholar 

  • *Witt, J. C., Noell, G. H., LaFleur, L. H., & Mortenson, B. P. (1997). Teacher use of interventions in general education settings: Measurement and analysis of the independent variable. Journal of Applied Behavior Analysis, 30, 693–696. doi:10.1901/jaba.1997.30-693.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Psychology, Louisiana State University, 236 Audubon Hall, Baton Rouge, LA, 70803, USA

    George H. Noell

  2. School of Education, Louisiana State University, 223 Peabody Hall, Baton Rouge, LA, 70803, USA

    Kristin A. Gansle

  3. Marcus Autism Center, Atlanta, GA, USA

    Joanna Lomas Mevers

  4. Louisiana Department of Education, Baton Rouge, LA, USA

    R. Maria Knox

  5. Johns Hopkins School of Medicine, Kennedy Krieger Institute, Neurobehavioral Unit Outpatient Clinic, 9810 Patuxent Woods Drive, Suite F, Columbia, MD, 21046, USA

    Joslyn Cynkus Mintz

  6. Autism Spectrum Therapies, Seattle, WA, USA

    Amanda Dahir

Authors
  1. George H. Noell
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kristin A. Gansle
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Joanna Lomas Mevers
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. R. Maria Knox
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Joslyn Cynkus Mintz
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Amanda Dahir
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Kristin A. Gansle.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Noell, G.H., Gansle, K.A., Mevers, J.L. et al. Improving Treatment Plan Implementation in Schools: A Meta-analysis of Single Subject Design Studies. J Behav Educ 23, 168–191 (2014). https://doi.org/10.1007/s10864-013-9177-1

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10864-013-9177-1

Keywords

search

Navigation