Abstract
As the demand for mindfulness-based interventions grows, so does the need to monitor implementation to ensure effective delivery. Research has shown that the fidelity and quality of program implementation can impact intervention outcomes. This study reports on the development and use of implementation monitoring measures for the Cultivating Awareness and Resilience in Education (CARE) professional development program. Three facilitator-focused observer-rated measures of fidelity and quality of intervention delivery and three participant-focused measures that examined responsiveness and attendance to the program were used. Data were evaluated from six intervention implementations including 166 teachers from 36 schools in New York; 80% of participants provided follow-up data on engagement in mindful practices. We report on regression analyses that examined relationships between implementation monitoring measures and follow-up teacher outcomes related to uptake and sustainment of mindful awareness practices. Results indicated significant positive relationships with an alternative measure of adherence focused on completion of participant objectives and facilitation quality with follow-up teacher reports of continued practice (mindfulness practices used weekly, length of practices, extent that practices helped with stress). Further, participants’ perceived usefulness of the material demonstrated a strong positive relationship with length of practice and whether practices helped to reduce stress. Finally, interactions were found between attendance and adherence, and between facilitation quality and participant knowledge on practice outcomes. We explore the way these measures were used to support program delivery and facilitator training, and discuss the utility of an alternative measure of adherence focused on completion of participant learning objectives.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Baron, R. M., & Kenny, D. A. (1986). The moderator–mediator variable distinction in social psychological research: Conceptual, strategic, and statistical considerations. Journal of Personality and Social Psychology, 51(6), 1173–1182. https://doi.org/10.1037/0022-3514.51.6.1173
Berkel, C., Mauricio, A. M., Schoenfelder, E., & Sandler, I. N. (2011). Putting the pieces together: an integrated model of program implementation. Prevention Science, 12(1), 23–33. https://doi.org/10.1007/s11121-010-0186-1.
Cicchetti, D. V. (1994). Guidelines, criteria, and rules of thumb for evaluating normed and standardized assessment instruments in psychology. Psychological Assessment, 6(4), 284–290. https://doi.org/10.1037/1040-3590.6.4.284.
Coatsworth, J. D., Duncan, L. G., Nix, R. L., Greenberg, M. T., Gayles, J. G., Bamberger, K. T., et al. (2015). Integrating mindfulness with parent training: effects of the mindfulness-enhanced strengthening families program. Developmental Psychology, 51(1), 26–35. https://doi.org/10.1037/a0038212.
Cohen, J. (1960). A coefficient of agreement for nominal scales. Educational and Psychological Measurement, 20, 37–46. https://doi.org/10.1177/001316446002000104
Cohen, J. (1992). A power primer. Psychological Bulletin, 112(1), 155–159 Retrieved from http://www.bwgriffin.com/workshop/SamplingACohentables.pdf.
Cutbush, S., Gibbs, D., Krieger, K., Clinton-Sherrod, M., & Miller, S. (2017). Implementers’ perspectives on fidelity of implementation: “Teach every single part” or “Be right with the curriculum”? Health Promotion Practice, 18(2), 275–282. https://doi.org/10.1177/1524839916672815.
Dane, A. V., & Schneider, B. H. (1998). Program integrity in primary and early secondary prevention: are implementation effects out of control? Clinical Psychology Review, 18(1), 23–45. https://doi.org/10.1016/S0272-7358(97)00043-3.
Dobkin, P. L., Hickman, S., & Monshat, K. (2014). Holding the heart of mindfulness-based stress reduction: balancing fidelity and imagination when adapting MBSR. Mindfulness, 5(6), 710–718. https://doi.org/10.1007/s12671-013-0225-7.
Domitrovich, C., & Greenberg, M. T. (2000). The study of implementation: current findings from effective programs that prevent mental disorders in school-aged children. Journal of Educational and Psychological Consultation, 11, 193–221. https://doi.org/10.1207/S1532768XJEPC1102_04.
Doyle, S. L., Jennings, P. A., DeWeese, A., & Frank, J. (2014). Evaluating the fidelity of the Cultivating Awareness and Resilience in Education (CARE) professional development program. In 22nd Annual Society for Prevention Research Conference. Washington, D.C.
Durlak, J. A., & DuPre, E. P. (2008). Implementation matters: a review of research on the influence of implementation on program outcomes and the factors affecting implementation. American Journal of Community Psychology, 41(3–4), 327–350. https://doi.org/10.1007/s10464-008-9165-0.
Durlak, J. A., & Wells, A. M. (1997). Primary prevention mental health programs for children and adolescents: a meta-analytic review. Annals of Clinical Psychiatry. Wiley-Blackwell. https://doi.org/10.1023/A:1024654026646.
Dusenbury, L., Brannigan, R., Falco, M., & Hansen, W. B. (2003). A review of research on fidelity of implementation: implications for drug abuse prevention in school settings. Health Education Research, 18(2), 237–256. https://doi.org/10.1093/her/18.2.237.
Erdfelder, E., Faul, F., Buchner, A., & Lang, A. G. (2009). Statistical power analyses using G*Power 3.1: tests for correlation and regression analyses. Behavior Research Methods, 41(4), 1149–1160. https://doi.org/10.3758/BRM.41.4.1149.
Feagans Gould, L., Mendelson, T., Dariotis, J. K., Ancona, M., Smith, A. S. R., Gonzalez, A. A., et al. (2014). Assessing fidelity of core components in a mindfulness and yoga intervention for urban youth: applying the CORE Process. New Directions for Youth Development, 2014(142), 59–81. https://doi.org/10.1002/yd.20097.
Feagans Gould, L., Dariotis, J. K., Greenberg, M. T., & Mendelson, T. (2016). Assessing fidelity of implementation (FOI) for school-based mindfulness and yoga interventions: a systematic review. Mindfulness, 7(1), 5–33. https://doi.org/10.1007/s12671-015-0395-6.
Greenberg, M. T., Brown, J. L., & Abenavoli, R. M. (2016). Teacher stress and health: effects on teachers, students, and schools. Edna Bennett Pierce Prevention Research Center, The Pennsylvania State University, 1–12. Retrieved from http://www.rwjf.org/en/library/research/2016/07/teacher-stress-and-health.html.
Hallgren, K. A. (2012). Computing inter-rater reliability for observational data: an overview and tutorial. Tutorial in Quantitative Methods for Psychology, 8(1), 23–34. https://doi.org/10.20982/tqmp.08.1.p023.
Hayes, A. F. (2012). PROCESS: a versatile computational tool for observed variable mediation, moderation, and conditional process modeling. White paper.
Iowa State University Extension and Outreach. (2010). Strengthening Families Program for parents & youth 10-14: fidelity observation form. Retrieved from http://www.episcenter.psu.edu/sites/default/files/SFPParent7ObservationUPDATED08.29.16.pdf.
Jennings, P. A. (2015). Early childhood teachers’ well-being, mindfulness, and self-compassion in relation to classroom quality and attitudes towards challenging students. Mindfulness, 6(4), 732–743. https://doi.org/10.1007/s12671-014-0312-4.
Jennings, P. A., Snowberg, K. E., Coccia, M. A., & Greenberg, M. T. (2011). Improving classroom learning environments by Cultivating Awareness and Resilience in Education (CARE): results of two pilot studies. The Journal of Classroom Interaction, 46(1), 37–48 Retrieved from http://www.jstor.org/stable/23870550.
Jennings, P. A., Frank, J. L., Snowberg, K. E., Coccia, M. A., & Greenberg, M. T. (2013). Improving classroom learning environments by Cultivating Awareness and Resilience in Education (CARE): results of a randomized controlled trial. School Psychology Quarterly, 28(4), 374–390. https://doi.org/10.1037/spq0000035.
Jennings, P. A., Turksma, C., & Brown, R. (2016). Cultivating Awareness and Resilience in Education (CARE) professional development program manual. Garrison: Garrison Institute.
Jennings, P. A., Brown, J. L., Frank, J. L., Doyle, S., Oh, Y., Davis, R., et al. (2017). Impacts of the CARE for teachers program on teachers’ social and emotional competence and classroom interactions. Journal of Educational Psychology, 1–19. https://doi.org/10.1037/edu0000187.
Milat, A. J., Bauman, A., & Redman, S. (2015). Narrative review of models and success factors for scaling up public health interventions. Implementation Science, 10(1), 113. https://doi.org/10.1186/s13012-015-0301-6.
Moncher, F. J., & Prinz, R. J. (1991). Treatment fidelity in outcome studies. Clinical Psychology Review, 11(3), 247–266. https://doi.org/10.1016/0272-7358(91)90103-2.
Mowbrey, C. T., Holter, M. C., Teague, G. B., & Bybee, D. (2003). Fidelity criteria: development, measurement, and validation. American Journal of Evaluation, 24(3), 315–340. https://doi.org/10.1177/109821400302400303.
Prowse, P. T., Meadows, G., & Enticott, J. (2015). An exploratory study into the effectiveness of fidelity scales in the delivery of Mindfulness-Based Cognitive Therapy. Mindfulness, 6(6), 1401–1410. https://doi.org/10.1007/s12671-015-0412-9.
Schoenwald, S. K. (2011). It’s a bird, it’s a plane, it’s...fidelity measurement in the real world. Clinical Psychology, 18(2), 142–147. https://doi.org/10.1111/j.1468-2850.2011.01245.x.It.
Schonert-Reichl, K. A., & Lawlor, M. S. (2010). The effects of a mindfulness-based education program on pre- and early adolescents’ well-being and social and emotional competence. Mindfulness, 1(3), 137–151. https://doi.org/10.1007/s12671-010-0011-8.
Segal, Z. V., Teasdale, J. D., Williams, J. M., & Gemar, M. C. (2002). The mindfulness-based cognitive therapy adherence scale: inter-rater reliability, adherence to protocol and treatment distinctiveness. Clinical Psychology & Psychotherapy, 9(2), 131–138. https://doi.org/10.1002/cpp.320.
Acknowledgements
We would like to thank the teachers and schools who participated in this research. Without you, we could not do this work.
Funding
The project described was supported by Grant Award R305A120180 from the Institute of Educational Sciences (IES). The content is solely the responsibility of the authors and does not necessarily represent the official views of the IES or the U.S. Department of Education.
Author information
Authors and Affiliations
Contributions
SLD: designed and executed the study, performed data analyses, and wrote the paper. PAJ and JLB: assisted with design and execution of current study, assisted with editing of the paper, and designed and executed parent study. DR: assisted with revisions of paper and data analysis. AW, JLF, and CT: assisted with creation of the measures used in the study. MTG: provided feedback on drafts and helped with design and execution of parent study.
Corresponding author
Ethics declarations
Ethical Approval
All procedures performed involving human participants were in accordance with the ethical standards of the University of Virginia, Pennsylvania State University, and the New York City Department of Education Institutional Review Boards and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.
Informed Consent
Informed consent was obtained from all participants in the study.
Conflict of Interest
In accordance with ethical obligations, PAJ and CT report that they are the developers of the CARE program and, as such, may benefit from publication of this research. SLD, JLB, DR, AW, JLF, and MG have no conflicts of interest to declare.
Rights and permissions
About this article
Cite this article
Doyle, S., Jennings, P.A., Brown, J.L. et al. Exploring Relationships Between CARE Program Fidelity, Quality, Participant Responsiveness, and Uptake of Mindful Practices. Mindfulness 10, 841–853 (2019). https://doi.org/10.1007/s12671-018-1034-9
Published:
Issue Date:
DOI: https://doi.org/10.1007/s12671-018-1034-9