Abstract
Introduction Implementation fidelity is a challenge for the adoption of evidence-based programs within social service broadly and child welfare specifically. However, implementation fidelity is critical for maintaining the integrity of clinical trials and for ensuring successful delivery of services in public health settings. Methods Promoting First Relationships ® (PFR), a 10-week home visiting parenting intervention, was evaluated in two randomized clinical trials with populations of families in child welfare. Seven providers from community agencies participated in the trials and administered PFR. Fidelity data collected included observational measures of provider behavior, provider records, and input from clients to assess training uptake, adherence to content, quality of delivery, program dosage, and participant satisfaction. Results In mock cases to assess training uptake, providers demonstrated an increase in PFR verbalization strategies and a decrease non-PFR verbalizations from pre to post PFR training, and overall this was maintained a year later (Mann–Whitney U’s = 0, p’s < .01). Adherence to content in actual cases was high, with M = 97% of the program elements completed. Quality of delivery varied across providers, indicated by PFR consultation strategies (Wilks’ Lambda F = 18.24, df = 15, p < .001) and global ratings (F = 13.35, df = 5, p < .001). Program dosage was high in both trials (71 and 86% receiving 10 sessions), and participant satisfaction was high (M = 3.9, SD = 0.2; 4 = greater satisfaction). Discussion This system of training and monitoring provides an example of procedures that can be used effectively to achieve implementation fidelity with evidence-based programs in social service practice.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Berkel, C., Mauricio, A. M., Schoenfelder, E., & Sandler, I. N. (2011). Putting the pieces together: An integrated model of program implementation. Prevention Science, 12(1), 23–33.
Breitenstein, S. M., Gross, D., Garvey, C. A., Hill, C., Fogg, L., & Resnick, B. (2010). Implementation fidelity in community-based interventions. Research in Nursing & Health, 33(2), 164–173.
Dane, A. V., & Schneider, B. H. (1998). Program integrity in primary and early secondary prevention: Are implementation effects out of control? Clinical Psychology Review, 18(1), 23–45.
Durlak, J. A., & DuPre, E. P. (2008). Implementation matters: A review of research on the influence of implementation on program outcomes and the factors affecting implementation. American Journal of Community Psychology, 41(3–4), 327–350.
Kelly, J., Sandoval, D., Zuckerman, T. G., & Buehlman, K. (2008). Promoting First Relationships: A program for service providers to help parents and other caregivers nurture young children’s social and emotional development (2nd edn.). Seattle: NCAST Programs.
Lee, S., Aos, S., & Miller, M. (2008). Evidence-based programs to prevent children from entering and remaining in the child welfare system: Benefits and costs for Washington (Document No. 08-07-3901). Olympia: Washington State Institute for Public Policy.
Hastings, P., Kahle, S., Fleming, C., Lohr, M., Katz, L., & Oxford, M. (2017). An intervention that increases parental sensitivity in families referred to Child Protective Services also changes toddlers’ parasympathetic regulation (Submitted).
Nelson, E. M., & Spieker, S. J. (2013). Intervention effects on morning and stimulated cortisol responses among toddlers in foster care. Infant mental health journal, 34(3), 211–221.
Oxford, M. L., Fleming, C. B., Nelson, E. M., Kelly, J. F., & Spieker, S. J. (2013). Randomized trial of Promoting First Relationships: Effects on maltreated toddlers’ separation distress and sleep regulation after reunification. Children and youth services review, 35(12), 1988–1992.
Oxford, M. L., Marcenko, M., Fleming, C. B., Lohr, M. J., & Spieker, S. J. (2016a). Promoting birth parents’ relationships with their toddlers upon reunification: Results from Promoting First Relationships® home visiting program. Children and youth services review, 61, 109–116.
Oxford, M. L., Spieker, S. J., Lohr, M. J., & Fleming, C. B. (2016b). Promoting First Relationships®: Randomized trial of a 10-week home visiting program with families referred to child protective services. Child maltreatment, 21(4), 267–277.
Spieker, S. J., Oxford, M. L., Kelly, J. F., Nelson, E. M., & Fleming, C. B. (2012). Promoting first relationships: Randomized trial of a relationship-based intervention for toddlers in child welfare. Child maltreatment, 17(4), 271–286.
Spieker, S. J., Oxford, M. L., & Fleming, C. B. (2014). Permanency outcomes for toddlers in child welfare two years after a randomized trial of a parenting intervention. Children and youth services review, 44, 201–206.
Acknowledgements
Research reported in this manuscript was supported by the Eunice Kennedy Shriver National Institute of Child Health & Human Development of the National Institutes of Health under Award Numbers R01HD061362 and U54HD083091, and from the National Institute of Mental Health under Award Number R01MH077329. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no conflict of interest.
Rights and permissions
About this article
Cite this article
Oxford, M.L., Spieker, S.J., Lohr, M.J. et al. Ensuring Implementation Fidelity of a 10-Week Home Visiting Program in Two Randomized Clinical Trials. Matern Child Health J 22, 376–383 (2018). https://doi.org/10.1007/s10995-017-2387-8
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10995-017-2387-8