Skip to main content
SpringerLink
Log in

Implementation Integrity of Practice-Based Coaching: Preliminary Results from the BEST in CLASS Efficacy Trial

  • Original Paper
  • Published:
School Mental Health Aims and scope Submit manuscript

Abstract

The purpose of this article is to describe the practice-based coaching model used in BEST in CLASS, a Tier-2 classroom-based intervention comprised of evidence-based instructional practices designed to prevent and ameliorate the chronic problem behaviors of young children at risk for the development of emotional/behavioral disorders. Following a description of the model, data from year two of an ongoing 4-year randomized control trial are presented that describe (a) the amount (i.e., dosage) of coaching teachers received during BEST in CLASS implementation, (b) the integrity with which coaches implemented the BEST in CLASS coaching model, and (c) subsequent teacher implementation of the BEST in CLASS strategies. Forty-eight (23 BEST in CLASS; 25 comparison) teachers and ten coaches participated in this descriptive study. Data indicate that following coaches’ observations, teachers received approximately 30 min of practice-based coaching during coaching meetings each week of implementation, and integrity data indicate that coaches implemented the critical coaching skills during the coaching meetings with integrity. Adherence data indicate that teachers in the treatment group increased their extensiveness of the use of BEST in CLASS practices at both post-treatment and 1-month follow-up compared to comparison teachers; competence data indicated that teachers in the treatment group increased the quality of delivery of practices at post-treatment compared to comparison teachers. Implications of these findings for both future research and practice-based coaching implementation are discussed.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  • Abry, T., Rimm-Kaufman, S. E., Larsen, R. A., & Brewer, A. J. (2013). The influence of fidelity of implementation on teacher–student interaction quality in the context of a randomized controlled trial of the Responsive Classroom approach. Journal of School Psychology, 51(4), 437–453. doi:10.1016/j.jsp.2013.03.001

    Article  PubMed  Google Scholar 

  • Barber, J. P., Sharpless, B. A., Klostermann, S., & McCarthy, K. S. (2007). Assessing intervention competence and its relation to therapy outcome: A selected review derived from the outcome literature. Professional Psychology: Research and Practice, 38(5), 493–500. doi:10.1037/0735-7028.38.5.493

    Article  Google Scholar 

  • Becker, K. D., & Domitrovich, C. E. (2011). The conceptualization, integration, and support of evidence-based interventions in the schools. School Psychology Review, 40(4), 582–589.

    Google Scholar 

  • Bradley, R. H., & Corwyn, R. F. (2002). Socioeconomic status and child development. Annual Review of Psychology, 53, 371–399.

    Article  PubMed  Google Scholar 

  • Brauner, C. B., & Stephens, C. B. (2006). Estimating the prevalence of early childhood serious emotional/behavioral disorders: Challenges and recommendations. Public Health Reports, 121(3), 303–310.

    PubMed Central  PubMed  Google Scholar 

  • Burns, M., Jacob, S., & Wagner, A. (2008). Ethical and legal issues associated with using response-to-intervention to assess learning disabilities. Journal of School Psychology, 46, 263–279.

    Article  PubMed  Google Scholar 

  • Carter, A. S., Wagmiller, R. J., Gray, S. A. O., McCarthy, K. J., Horwitz, S. M., & Briggs-Gowan, M. J. (2010). Prevalence of DSM-IV disorder in a representative, healthy birth cohort at school entry: Sociodemographic risks and social adaptation. Journal of the American Academy of Child and Adolescent Psychiatry, 49(7), 686–698. doi:10.1016/j.jaac.2010.03.018

    PubMed Central  PubMed  Google Scholar 

  • Cicchetti, D. (1994). Guidelines, criteria, and rules of thumb for evaluating normed and standardized assessment instruments in psychology. Psychological Assessment, 6, 284–290. doi:10.1037/1040-3590.6.4.284

    Article  Google Scholar 

  • Conroy, M. A., Alter, P., Boyd, B., & Bettini, E. (2014). Teacher preparation for students who demonstrate challenging behaviors. In P. T. Sindelar, E. D. McCray, M. T. Brownell, & B. Lingnugaris/Kraft (Eds.), Handbook of research on special education teacher preparation (pp. 320–333). New York: Routledge, Taylor, & Francis.

    Google Scholar 

  • Conroy, M. A., Sutherland, K. S., Algina, J., Wilson, R., Martinez, J., & Whalon, K. (2014). Measuring teacher implementation of the BEST in CLASS intervention program and corollary child outcomes. Journal of Emotional and Behavioral Disorders. doi:10.1177/1063426614532949

  • Conroy, M. A., Sutherland, K. S., Vo, A., Carr, S. E., & Ogston, P. (2014). Early childhood teachers’ use of effective instructional practices and the collateral effects on young children’s behavior. Journal of Positive Behavioral Interventions, 16(2), 81–92. doi:10.1177/1098300713478666

    Article  Google Scholar 

  • Curby, T. W., Rimm-Kaufman, S. E., & Abry, T. (2013). Do emotional support and classroom organization earlier in the year set the stage for higher quality instruction. Journal of School Psychology, 51, 557–569. doi:10.1016/j.jsp.2013.06.001

    Article  PubMed  Google Scholar 

  • Domitrovich, C. E., Gest, S. D., Jones, D., Gill, S., & DeRousie, R. M. S. (2010). Implementation quality: Lessons learned in the context of the Head Start REDI trial. Early Childhood Research Quarterly, 25(3), 284–298. doi:10.1016/j.ecresq.2010.04.001

    Article  PubMed Central  PubMed  Google Scholar 

  • Durlak, J. A., & DuPre, E. P. (2008). Implementation matters: A review of research on the influence of implementation on program outcomes and the factors affecting implementation. American Journal of Community Psychology, 41, 327–350.

  • Durlak, J. A. (2010). The importance of doing well in whatever you do: A commentary on the special section, “Implementation research in early childhood education”. Early Childhood Research Quarterly, 25(3), 348–357. doi:10.1016/j.ecresq.2010.03.003

    Article  Google Scholar 

  • Fox, L., Hemmeter, M. L., Snyder, P., Binder, D. P., & Clarke, S. (2011). Coaching early childhood special educators to implement a comprehensive model for promoting young children’s social competence. Topics in Early Childhood Special Education, 3(3), 178–192. doi:10.1177/0271121411404440

    Article  Google Scholar 

  • Goncy, L. A., Sutherland, K. S., Farrell, A. D., Sullivan, T. N., & Doyle, S. T. (in press). Measuring teacher implementation in delivery of a bullying prevention program: The impact of instructional and protocol adherence and competence on student responsiveness. Prevention Science.

  • Hagermoser Sanetti, L. M., Dobey, L. M., & Gritter, K. L. (2012). Treatment integrity of interventions with children in the Journal of Positive Behavior Interventions from 1999 to 2009. Journal of Positive Behavior Interventions, 14(1), 29–46. doi:10.1177/1098300711405853

    Article  Google Scholar 

  • Harn, B., Parisi, D., & Stoolmiller, M. (2013). Balancing fidelity with flexibility and fit: What do we really know about fidelity of implementation in schools? Exceptional Children, 79(2), 181–193.

    Google Scholar 

  • Hemmeter, M. L., Corso, R., & Cheatham, G. (2006). A national survey of early childhood educators: Training needs and strategies. Paper presented at the Conference on Research Innovations in Early Intervention. Dan Diego, CA.

  • Hemmeter, M. L., Snyder, P., Kinder, K., & Artman, K. (2011). Impact of performance feedback delivered via electronic mail on preschool teachers’ use of descriptive praise. Early Childhood Research Quarterly, 26, 96–106. doi:10.1016/j.ecresq.2010.05.004

    Article  Google Scholar 

  • Hetzner, N. P., Johnson, A. D., & Brooks-Gunn, J. (2011). Poverty, effects on social and emotional development. In S. Jarvela (Ed.), Social and emotional aspects of learning (pp. 77–86). Oxford: Elsevier.

    Google Scholar 

  • Hogue, A., Henderson, C. E., Dauber, S., Barajas, P. C., Fried, A., & Liddle, H. A. (2008). Treatment adherence, competence, and outcome in individual and family therapy for adolescent behavior problems. Journal of Consulting and Clinical Psychology, 76(4), 544–555. doi:10.1037/0022-006X.60.1.73

    Article  PubMed Central  PubMed  Google Scholar 

  • Joyce, B., & Showers, B. (2002). Student achievement through staff development (3rd ed.). Alexandria, VA: Association for Supervision and Curriculum Development.

    Google Scholar 

  • McCabe, L. A., & Frede, E. C. (2007). Challenging behavior and the role of preschool education. New Brunswick: National Institute for Early Education.

    Google Scholar 

  • Miller, K., Fitzgerald, G., Koury, K., Mitchem, K., & Hollingsead, C. (2007). KidTools: Self-management, problem solving, organizational and planning tools for children and teachers. Intervention in School and Clinic, 43(1), 12–19.

    Article  Google Scholar 

  • Mitchem, K., Koury, K., Fitzgerald, G., Hollingsead, C., Miller, K., Tsai, H. H., Zha, S. (2009). The effects of instructional implementation on learning with interactive multimedia case-based instruction. Teacher Education and Special Education, 32, 297–318.

    Article  Google Scholar 

  • National Professional Development Center on Inclusion. (2008). What do we mean by professional development in the early childhood field?. Chapel Hill: The University of North Carolina, FPG Child Development Institute.

    Google Scholar 

  • Patterson, G. R., Reid, J. B., & Eddy, J. M. (2002). A brief history of the Oregon model. In J. B. Reid, G. R. Patterson, & J. Snyder (Eds.), Antisocial behavior in children and adolescents: A developmental analysis and model for intervention (pp. 3–20). Washington, DC: American Psychological Association.

    Chapter  Google Scholar 

  • Sanetti, L. M. H., Gritter, K. L., & Dobey, L. M. (2011). Treatment integrity of intervention with children in the school psychology literature from 1995 to 2008. School Psychology Review, 40(1), 72–84.

    Google Scholar 

  • Sanetti, L. M., & Kratochwill, T. R. (2009). Toward developing a science of treatment integrity: Introduction to the special series. School Psychology Review, 38, 445–459.

    Google Scholar 

  • Snyder, P. (2007). Impact of professional development on preschool teachers’ use of embedded-instruction practices. Funded from the US Department of Education, Institute of Education Sciences, Washington DC (Grant No. R324A070008).

  • Snyder, P., Denney, M., Pasia, C., Rakap, S., & Crowe, C. (2011). Professional development in early childhood intervention. In C. J. Groark (Series Ed.) & L. Kaczmarek (Vol. Ed.), Early childhood intervention: Shaping the future for children with disabilities and their families: Vol. 3. Emerging issues (pp. 169–204). Santa Barbara, CA: ABC-Clio/Praeger.

  • Snyder, P., Hemmeter, M. L., & McLaughlin, T. (2011). Professional development in early childhood intervention: Where we stand on the silver anniversary of PL 99-457. Journal of Early Intervention, 33, 357–370. doi:10.1177/1053815111428336

    Article  Google Scholar 

  • Snyder, P., Hemmeter, M. L., Meeker, K. A., Kinder, K., Pasia, C., & McLaughlin, T. (2012). Characterizing key features of the early childhood professional development literature. Infants & Young Children, 25(3), 188–212.

    Article  Google Scholar 

  • Snyder, P., & Wolfe, B. (2008). The big three process components of effective professional development: Needs assessment, evaluation, and follow-up. In P. J. Winton, J. A. McCollum, & C. Catlett (Eds.), Practical approaches to early childhood professional development: Evidence, strategies, and resources (pp. 13–52). Washington, DC: Zero to Three.

    Google Scholar 

  • Sutherland, K. S., Conroy, M. A., Vo, A., Abrams, L., & Ogston, P. (2013). An initial evaluation of the Teacher–Child Direct Observation System: Measuring teacher–child interaction behaviors in classroom settings. Assessment for Effective Intervention, 39(1), 12–23. doi:10.1177/1534508412463814

    Article  Google Scholar 

  • Sutherland, K. S., McLeod, B. D., Conroy, M. A., Abrams, L. M., & Smith, M. M. (2014). Preliminary psychometric properties of the BEST in CLASS Adherence and Competence Scale. Journal of Emotional and Behavioral Disorders. doi:10.1177/1063426613497258

  • Sutherland, K. S., McLeod, B. D., Conroy, M. A., & Cox, J. R. (2013). Measuring implementation of evidence-based programs targeting young children at risk for emotional/behavioral disorders: Conceptual issues and recommendations. Journal of Early Intervention, 35(2), 129–149. doi:10.1177/1053815113515025

    Article  Google Scholar 

  • Trivette, C. M., Dunst, C. J., Hamby, D. W., & O’Herin, C. E. (2009). Characteristics and consequences of adult learning methods and strategies. Research Brief, 3(1), 1–33.

    Google Scholar 

  • U.S. Department of Health and Human Services, Administration for Children and Families. (2012). Advisory committee on head start research and evaluation: Final report. Washington, DC: Author.

    Google Scholar 

  • Williford, A. P., Vick Whittaker, J. E., Vitiello, V. E., & Downer, J. T. (2013). Children’s engagement within the preschool classroom and their development of self-regulation. Early Education and Development, 24, 162–187. doi:10.1080/10409289.2011.628270

    Article  PubMed Central  PubMed  Google Scholar 

  • Yoshikawa, H., Aber, J. L., & Beardslee, W. R. (2012). The effects of poverty on the mental, emotional, and behavioral health of children and youth: Implications for prevention. American Psychologist, 67, 272–284.

    Article  PubMed  Google Scholar 

Download references

Acknowledgments

This research was supported by a Grant (R324A110173) from the US Department of Education, Institute for Education Sciences, with additional support from an NIH/NCATS Clinical and Translational Science Award to the University of Florida UL1 TR000064. The opinions expressed by the authors are not necessarily reflective of the position of or endorsed by the US Department of Education.

Author information

Authors and Affiliations

  1. Virginia Commonwealth University, Richmond, VA, USA

    Kevin S. Sutherland & Abigail Vo

  2. University of Florida, Gainesville, FL, USA

    Maureen A. Conroy & Crystal Ladwig

Authors
  1. Kevin S. Sutherland
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Maureen A. Conroy
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Abigail Vo
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Crystal Ladwig
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Kevin S. Sutherland.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Sutherland, K.S., Conroy, M.A., Vo, A. et al. Implementation Integrity of Practice-Based Coaching: Preliminary Results from the BEST in CLASS Efficacy Trial. School Mental Health 7, 21–33 (2015). https://doi.org/10.1007/s12310-014-9134-8

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12310-014-9134-8

Keywords

Search

Navigation