Abstract
Teachers sometimes struggle to deliver evidence-based programs designed to prevent and ameliorate chronic problem behaviors of young children with integrity. Identifying factors associated with variations in the quantity and quality of delivery is thus an important goal for the field. This study investigated factors associated with teacher treatment integrity of BEST in CLASS, a tier-2 prevention program designed for young children at risk for developing emotional/behavioral disorders. Ninety-two early childhood teachers and 231 young children at-risk for emotional/behavioral disorders participated in the study. Latent growth curve analyses indicated that both adherence and competence of delivery increased across six observed time points. Results suggest that teacher education and initial levels of classroom quality may be important factors to consider when teachers deliver tier-2 (i.e., targeted to children who are not responsive to universal or tier-1 programming) prevention programs in early childhood settings. Teachers with higher levels of education delivered the program with more adherence and competence initially. Teachers with higher initial scores on the Emotional Support subscale of the Classroom Assessment Scoring System (CLASS) delivered the program with more competence initially and exhibited higher growth in both adherence and competence of delivery across time. Teachers with higher initial scores on the Classroom Organization subscale of the CLASS exhibited lower growth in adherence across time. Contrary to hypotheses, teacher self-efficacy did not predict adherence, and teachers who reported higher initial levels of Student Engagement self-efficacy exhibited lower growth in competence of delivery. Results are discussed in relation to teacher delivery of evidence-based programs in early childhood classrooms.
Similar content being viewed by others
Notes
Two items assess rules: three to five rules are visible in classroom and teacher reviews rules, addresses rule violations. Only the second item was included in the current study as we focused on teacher-delivered practices; three to five rules are visible in classroom is a static item that did not change from observation to observation.
References
Achenbach, T. M., & Rescorla, L. A. (2000). CBCL/1, 5-5 & C-TRF/1, 5–5 profiles. Burlington: Research Center for Children, Youth, and Families.
Baker, C. N., Kupersmidt, J. B., Voegler-Lee, M. E., Arnold, D., & Willoughby, M. T. (2010). Predicting teacher participation in a classroom-based, integrated preventive intervention for preschoolers. Early Childhood Research Quarterly, 25, 270–283. doi:10.1016/j.ecresq.2009.09.005.
Barber, J. P., Crits-Christoph, P., & Luborsky, L. (1996). Effects of therapist adherence and competence on patient outcome in brief dynamic therapy. Journal of Consulting and Clinical Psychology, 64, 619–622. doi:10.1037/0022-006X.64.3.619.
Berlin, L. J., Brooks-Gunn, J., McCarton, C., & McCormick, M. C. (1998). The effectiveness of early intervention: Examining risk factors and pathways to enhanced development. Preventive Medicine, 27, 238–245. doi:10.1006/pmed.1998.0282.
Bronfenbrenner, U. (1979). The ecology of human development: Experiments by nature and design. Cambridge: Harvard University Press.
Bronfenbrenner, U. (2005). Making human beings human: Bioecological perspectives on human development. Thousand Oaks: Sage Publications.
Bruhn, A.L., Lane, K.L., Hirsch, S.E. (2013). A Review of Tier 2 Interventions Conducted Within Multitiered Models of Behavioral Prevention. Journal of Emotional and Behavioral Disorders, 22, 171–189
Carroll, K. M., & Nuro, K. F. (2002). One size cannot fit all: A stage model for psychotherapy manual development. Clinical Psychology: Science and Practice, 9, 396–406. doi:10.1093/clipsy.9.4.396.
Carroll, K. M., Nich, C., Sifty, R. L., Nuro, K. F., Frankforter, T. L., Ball, S. A., et al. (2000). A general system for evaluating therapist adherence and competence in psychotherapy research. Drug and Alcohol Dependence, 57, 225–238. doi:10.1016/S0376-8716(99)00049-6.
Carter, A. S., Wagmiller, R. J., Gray, S. A., McCarthy, K. J., Horwitz, S. M., & Briggs-Gowan, M. J. (2010). Prevalence of DSM-IV disorder in a representative, healthy birth cohort at school entry: Sociodemographic risks and social adaptation. Journal of the American Academy of Child & Adolescent Psychiatry, 49, 686–698. doi:10.1016/j.jaac.2010.03.018.
Cicchetti, D. V. (1994). Guidelines, criteria, and rules of thumb for evaluating normed and standardized assessment instruments in psychology. Psychological Assessment, 6, 284–290. doi:10.1037/1040-3590.6.4.284.
Conroy, M. A., Sutherland, K. S., Algina, J., Ladwig, C., Werch, B., Martinez, J., Jessee, G., & Gyure, M. (2017). BEST in CLASS: A professional development intervention fostering high quality classroom experiences for young children with problem behavior. Manuscript submitted for publication.
Conroy, M. A., Sutherland, K. S., Algina, J. J., Wilson, R. E., Martinez, J. R., & Whalon, K. J. (2015). Measuring teacher implementation of the BEST in CLASS intervention program and corollary child outcomes. Journal of Emotional and Behavioral Disorders, 23(3), 144–155.
Conroy, M. A., Sutherland, K. S., Vo, A. K., Carr, S. E., & Ogston, P. (2014). Early childhood teachers’ use of effective instructional practices and the collateral effects on young children’s behavior. Journal of Positive Behavior Interventions, 16, 81–92. doi:10.1177/1098300713478666.
Dane, A. V., & Schneider, B. H. (1998). Program integrity in primary and early secondary prevention: Are implementation effects out of control. Clinical Psychology Review, 18, 23–45. doi:10.1016/S0272-7358(97)00043-3.
Domitrovich, C. E., Bradshaw, C. P., Poduska, J. M., Hoagwood, K., Buckley, J. A., Olin, S., Hunter Romanelli, L., Leaf, P. J., Greenberg, M. T., & Ialongo, N. S. (2008). Maximizing the implementation quality of evidence-based preventive interventions in schools: A conceptual framework. Advances in School Mental Health Promotion, 1, 6–28.
Domitrovich, C. E., Cortes, R. C., & Greenberg, M. T. (2007). Improving young children’s social and emotional competence: A randomized trial of the preschool “PATHS” curriculum. The Journal of Primary Prevention, 28, 67–91. doi:10.1007/s10935-007-0081-0.
Domitrovich, C. E., Gest, S. D., Jones, D., Gill, S., & DeRousie, R. M. S. (2010). Implementation quality: Lessons learned in the context of the head start REDI trial. Early Childhood Research Quarterly, 25, 284–298. doi:10.1016/j.ecresq.2010.04.001.
Domitrovich, C. E., Pas, E. T., Bradshaw, C. P., Becker, K. D., Keperling, J. P., Embry, D. D., & Ialongo, N. (2015). Individual and school organizational factors that influence implementation of the PAX Good Behavior Game Intervention. Prevention Science, 16(8), 1064–1074. doi:10.1007/s11121-015-0557-8.
Durlak, J. A. (2015). Studying program implementation is not easy but it is essential. Prevention Science, 16(8), 1123–1127.
Durlak, J. A. (2010). The importance of doing well in whatever you do: A commentary on the special section, “Implementation research in early childhood education.”. Early Childhood Research Quarterly, 25, 348–357. doi:10.1016/j.ecresq.2010.03.003.
Durlak, J. A., Weissberg, R. P., Dymnicki, A. B., Taylor, R. D., & Schellinger, K. B. (2011). The impact of enhancing students’ social and emotional learning: A meta-analysis of school-based universal interventions. Child Development, 82, 405–432. doi:10.1111/j.1467-8624.2010.01564.x.
Feil, E. G., Severson, H. H., & Walker, H. M. (1998). Screening for emotional and behavioral delays: The Early Screening Project. Journal of Early Intervention, 21, 252–266. doi:10.1177/105381519802100306.
Fixsen, D. L., Naoom, S. F., Blasé, K. A., Friedman, R. M., & Wallace, F. (2005). Implementation research: A synthesis of the literature. Tampa: University of South Florida, Louis de la Parte Florida Mental Health Institute, The National Implementation Research Network (FMHI Publication #231).
Guo, Y., Connor, C. M., Yang, Y., Roehrig, A. D., & Morrison, F. J. (2012). The effects of teacher qualification, teacher self-efficacy, and classroom practices on fifth graders' literacy outcomes. The Elementary School Journal, 113, 3–24. doi:10.1086/665816.
Han, S. S., & Weiss, B. (2005). Sustainability of teacher implementation of school-based mental health programs. Journal of Abnormal Child Psychology, 33, 665–679. doi:10.1007/s10802-005-7646-2.
Hemmeter, M. L., Corso, R., & Cheatham, G. (2006). A national survey of early childhood educators: Training needs and strategies. Paper presented at the Conference on Research Innovations in Early Intervention, San Diego, CA.
Hogue, A., Liddle, H. A., & Rowe, C. (1996). Treatment adherence process research in family therapy: A rationale and some practical guidelines. Psychotherapy: Theory, Research, Practice, Training, 33, 332–345. doi:10.1037/0033-3204.33.2.332.
Hogue, A., Henderson, C. E., Dauber, S., Barajas, P. C., Fried A., Liddle, H. A. (2008) Treatment adherence, competence, and outcome in individual and family therapy for adolescent behavior problems. Journal of Consulting and Clinical Psychology, 76, 544–555
Little, M. A., Sussman, S., Sun, P., & Rohrbach, L. A. (2013). The effects of implementation fidelity in the Towards No Drug Abuse dissemination trial. Health Education, 113, 281–296. doi:10.1108/09654281311329231.
McCabe, P. C., & Altamura, M. (2011). Empirically valid strategies to improve social and emotional competence of preschool children. Psychology in the Schools, 48, 513–540. doi:10.1002/pits.20570.
McLeod, B. D., Southam-Gerow, M. A., Rodriguez, A., Quinoy, A., Arnold, C., Kendall, P. C., & Weisz, J., R. (2016). Development and initial psychometrics for a therapist competence instrument for CBT for youth with anxiety. Journal of Clinical and Adolescent Psychology. Advance online publication. doi:10.1080/15374416.2016.1253018.
McLeod, B. D., Southam-Gerow, M. A., Tully, C. B., Rodriguez, A., & Smith, M. M. (2013). Making a case for treatment integrity as a psychological treatment quality indicator. Clinical Psychology: Science and Practice, 20, 14–32. doi:10.1111/cpsp.12020.
Mendel, P., Meredith, L. S., Schoenbaum, M., Sherbourne, C. D., & Wells, K. B. (2008). Interventions in organizational and community context: A framework for building evidence on dissemination and implementation in health services research. Administration and Policy in Mental Health and Mental Health Services Research, 35(1–2), 21–37. doi:10.1007/s10488-007-0144-9.
Mitchell, B. S., Stormont, M. A., & Gage, N. A. (2011). Tier 2 interventions within the context of school-wide positive behavior support. Behavioral Disorders, 36, 241–261.
Muthén, L. K., & Muthén, B. O. (1998-2017). Mplus user’s guide (Eighth ed.). Los Angeles: Muthén & Muthén.
Nelson, J. R., Stage, S., Duppong-Hurley, K., Synhorst, L., & Epstein, M. H. (2007). Risk factors predictive of the problem behavior of children at risk for emotional and behavioral disorders. Exceptional Children, 73, 367–379. doi:10.1177/001440290707300306.
Newborg, J. (2005). Battelle developmental inventory, 2nd edition, examiner’s manual. Rolling Meadows: Riverside Publishing.
Olejnik, S., & Algina, J. (2000). Measures of effect size for comparative studies: Applications, interpretations, and limitations. Contemporary Educational Psychology, 25, 241–286.
Pas, E. T., Bradshaw, C. P., & Hershfeldt, P. A. (2012). Teacher-and school-level predictors of teacher efficacy and burnout: Identifying potential areas for support. Journal of School Psychology, 50, 129–145. doi:10.1016/j.jsp.2011.07.003.
Pas, E. T., Waasdorp, T. E., & Bradshaw, C. P. (2015). Examining contextual influences on classroom-based implementation of Positive Behavior Support strategies: Findings from a randomized controlled effectiveness trial. Prevention Science, 16(8), 1096–1106. doi:10.1007/s11121-014-0492-0.
Pianta, R., Howes, C., Burchinal, M., Bryant, D., Clifford, R., Early, D., & Barbarin, O. (2005). Features of pre-kindergarten programs, classrooms, and teachers: Do they predict observed classroom quality and child-teacher interactions? Applied Developmental Science, 9(3), 144–159. doi:10.1207/s1532480xads0903_2.
Pianta, R. C., Karen, M., Paro, L., & Hamre, B. K. (2008). Classroom assessment scoring system (CLASS) manual, pre-K. Baltimore: Paul H. Brookes Publishing Company.
Proctor, E., Silmere, H., Raghavan, R., Hovmand, P., Aarons, G., Bunger, A., Griffey, R., & Hensley, M. (2011). Outcomes for implementation research: Conceptual distinctions, measurement challenges, and research agenda. Administration and Policy in Mental Health, 38, 65–76.
Sameroff, A. J. (1995). General systems theories and developmental psychopathology. In E. D. Cicchetti & D. J. Cohen (Eds.), Developmental Psychopathology, Vol. 1: Theory and Methods. Oxford: John Wiley & Sons.
Sanetti, L. M. H., & Kratochwill, T. R. (2009). Toward developing a science of treatment integrity: Introduction to the special series. School Psychology Review, 38, 445–459.
Schoenwald, S. K., Garland, A. F., Chapman, J. E., Frazier, S. L., Sheidow, A. J., & Southam-Gerow, M. A. (2011). Toward the effective and efficient measurement of implementation fidelity. Administration and Policy in Mental Health and Mental Health Services Research, 38, 32–43. doi:10.1007/s10488-010-0321-0.
Sutherland, K. S., Conroy, M. A., Algina, J., Ladwig, C., Jessee, G., & Gyure, M. (2017). Reducing child problem behaviors and improving teacher-child interactions and relationships: A randomized controlled trial of BEST in CLASS. Early Childhood Research Quarterly. in press.
Sutherland, K. S., Conroy, M. A., Vo, A., & Ladwig, C. (2015). Implementation integrity of practice-based coaching: Preliminary results from the BEST in CLASS efficacy trial. School Mental Health, 7, 21–33. doi:10.1007/s12310-014-9134-8.
Sutherland, K. S., McLeod, B. D., Conroy, M. A., Abrams, L., & Smith, M. M. (2014). Preliminary psychometric properties of the BEST in CLASS Adherence and Competence Scale. Journal of Emotional and Behavioral Disorders, 22, 249–259. doi:10.1177/1063426613493669.
Sutherland, K. S., McLeod, B. D., Conroy, M. A., & Cox, J. R. (2013). Measuring implementation of evidence-based programs targeting young children at risk for emotional/behavioral disorders conceptual issues and recommendations. Journal of Early Intervention, 35, 129–149. doi:10.1177/1053815113515025.
Tschannen-Moran, M., & Hoy, A. W. (2001). Teacher efficacy: Capturing an elusive construct. Teaching and Teacher Education, 17, 783–805. doi:10.1016/S0742-051X(01)00036-1.
Wanless, S. B., Rimm-Kaufman, S. E., Abry, T., Larsen, R. A., & Patton, C. L. (2015). Engagement in training as a mechanism to understanding fidelity of implementation of the Responsive Classroom Approach. Prevention Science, 16(8), 1107–1116.
Webster-Stratton, C., Reid, M. J., & Hammond, M. (2004). Treating children with early-conduct problems: Intervention outcomes for parent, child, and teacher training. Journal of Clinical Child and Adolescent Psychology, 33, 105–124. doi:10.1207/S15374424JCCP3301_11.
Williford, A. P., Wolcott, C. S., Whittaker, J. V., & Locasale-Crouch, J. (2015). Program and teacher characteristics predicting the implementation of banking time with preschoolers who display disruptive behaviors. Prevention Science, 16, 1054–1063. doi:10.1007/s11121-015-0544-0.
Wolery, M. (2011). Intervention research: The importance of fidelity measurement. Topics in Early Childhood Special Education, 31, 155–157. doi:10.1177/0271121411408621.
Acknowledgements
This research was supported by a grant (R324A110173) from the U. S. Department of Education, Institute for Education Sciences, with additional support from an NIH/NCATS Clinical and Translational Science Award to the University of Florida UL1 TR000064 and Grant H325H140001 from the U.S. Department of Education, Office of Special Education Programs. The opinions expressed by the authors are not necessarily reflective of the position of or endorsed by the U. S. Department of Education.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Funding Information
This research was supported by a grant (R324A110173) from the US Department of Education, Institute for Education Sciences, with additional support from an NIH/NCATS Clinical and Translational Science Award to the University of Florida UL1 TR000064. The opinions expressed by the authors are not necessarily reflective of the position of or endorsed by the US Department of Education.
Conflict of Interest
The authors declare that they have no conflict of interest.
Ethical Approval
All study procedures involving human participants were in accordance with the ethical standards of the researchers’ Institutional Review Boards and with the 1964 Helsinki Declaration and its later amendments of comparable ethical standards.
Informed Consent
Informed consent was obtained from all individual participants included in the study.
Rights and permissions
About this article
Cite this article
Sutherland, K.S., Conroy, M.A., McLeod, B.D. et al. Factors Associated with Teacher Delivery of a Classroom-Based Tier 2 Prevention Program. Prev Sci 19, 186–196 (2018). https://doi.org/10.1007/s11121-017-0832-y
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11121-017-0832-y