Advertisement

Prevention Science

, Volume 12, Issue 1, pp 23–33 | Cite as

Putting the Pieces Together: An Integrated Model of Program Implementation

  • Cady BerkelEmail author
  • Anne M. Mauricio
  • Erin Schoenfelder
  • Irwin N. Sandler
Article

Abstract

Considerable evidence indicates that variability in implementation of prevention programs is related to the outcomes achieved by these programs. However, while implementation has been conceptualized as a multidimensional construct, few studies examine more than a single dimension, and no theoretical framework exists to guide research on the effects of implementation. We seek to address this need by proposing a theoretical model of the relations between the dimensions of implementation and outcomes of prevention programs that can serve to guide future implementation research. In this article, we focus on four dimensions of implementation, which we conceptualize as behaviors of program facilitators (fidelity, quality of delivery, and adaptation) and behaviors of participants (responsiveness) and present the evidence supporting these as predictors of program outcomes. We then propose a theoretical model by which facilitator and participant dimensions of implementation influence participant outcomes. Finally, we provide recommendations and directions for future implementation research.

Keywords

Program implementation Fidelity Quality Adaptation Participant responsiveness Program outcomes 

Notes

Acknowledgements

Cady Berkel and Erin Schoenfelder’s involvement in the preparation of this manuscript were supported by Training Grant T32MH18387.

References

  1. Backer, T. E. (2002). Finding the balance: Program fidelity and adaptation in substance abuse prevention. Rockville, MD: Center for Substance Abuse Prevention.Google Scholar
  2. Baydar, N., Reid, M. J., & Webster-Stratton, C. (2003). The role of mental health factors and program engagement in the effectiveness of a preventive parenting program for Head Start mothers. Child Development, 74, 1433–1453.CrossRefPubMedGoogle Scholar
  3. Berkel, C., Hagan, M., Jones, S., Ayers, T. S., Wolchik, S. A., & Sandler, I. N. (2009, November). Longitudinal examination of facilitator implementation: A case study across multiple cohorts of delivery. Paper presented at the annual meeting of the American Evaluation Association, Orlando, FL.Google Scholar
  4. Blake, S. M., Simkin, L., Ledsky, R., Perkins, C., & Calabrese, J. M. (2001). Effects of a parent-child communications intervention on young adolescents’ risk for early onset of sexual intercourse. Family Planning Perspectives, 33, 52–61.CrossRefPubMedGoogle Scholar
  5. Blakely, C. H., Mayer, J. P., Gottschalk, R. G., Schmitt, N., Davidson, W. S., Roitman, D. B., et al. (1987). The fidelity-adaptation debate: Implications for the implementation of public sector social programs. American Journal of Community Psychology, 15, 253–268.CrossRefGoogle Scholar
  6. Boruch, R. F., & Gomez, H. (1977). Sensitivity, bias and theory in impact evaluations. Professional Psychology, 8, 411–434.CrossRefGoogle Scholar
  7. Botvin, G. J. (2004). Advancing prevention science and practice: Challenges, critical issues, and future directions. Prevention Science, 5, 69–72.CrossRefPubMedGoogle Scholar
  8. Botvin, G. J., Dusenbury, L., Baker, E., James-Ortiz, S., & Kerner, J. F. (1989). A skills training approach to smoking prevention among Hispanic youth. Journal of Behavioral Medicine, 12, 279–296.CrossRefPubMedGoogle Scholar
  9. Brach, C., & Fraser, I. (2000). Can cultural competency reduce racial and ethnic health disparities? A review and conceptual model. Medical Care Research & Review, 57, 181–217.Google Scholar
  10. Bush, P. J., Zuckerman, A. E., Taggart, V. S., Theiss, P. K., Peleg, E. O., & Smith, S. A. (1989). Cardiovascular risk factor prevention in Black school children: The “Know Your Body” evaluation project. Health Education Quarterly, 16, 215–227.PubMedGoogle Scholar
  11. Castro, F. G., Barrera, M., & Martinez, C. R. (2004). The cultural adaptation of prevention interventions: Resolving tensions between fidelity and fit. Prevention Science, 5, 41–45.CrossRefPubMedGoogle Scholar
  12. Charlebois, P., Brendgen, M., Vitaro, F., Normandeau, S., & Boudreau, J.-F. (2004). Examining dosage effects on prevention outcomes: Results from a multi-modal longitudinal preventive intervention for young disruptive boys. Journal of School Psychology, 42, 201–220.CrossRefGoogle Scholar
  13. Charlebois, P., Vitaro, F., Normandeau, S., Brendgen, M., & Rondeau, N. (2004). Trainers’ behavior and participants’ persistence in a longitudinal preventive intervention for disruptive boys. Journal of Primary Prevention, 25, 375–388.CrossRefGoogle Scholar
  14. Coatsworth, J. D., Duncan, L. G., Pantin, H., & Szapocznik, J. (2006a). Patterns of retention in a preventive intervention with ethnic minority families. Journal of Primary Prevention, 27, 171–193.CrossRefPubMedGoogle Scholar
  15. Coatsworth, J. D., Duncan, L. G., Pantin, H., & Szapocznik, J. (2006b). Retaining ethnic minority parents in a preventive intervention: The quality of group process. Journal of Primary Prevention, 27, 367–389.CrossRefPubMedGoogle Scholar
  16. Conduct Problems Prevention Research Group. (1999). Initial impact of the Fast Track prevention trial for conduct problems: II. Classroom effects. Journal of Consulting & Clinical Psychology, 67, 648–657.CrossRefGoogle Scholar
  17. Cross, T. L., Bazron, B. J., Dennis, K. W., & Isaacs, M. R. (1989). Towards a culturally competent system of care: A monograph on effective services for minority children who are severely emotionally disturbed: Volume I. Washington, DC: National Technical Assistance Center for Children’s Mental Health, Georgetown University Child Development Center.Google Scholar
  18. Cross, W., West, J., Wyman, P. A., & Schmeelk-Cone, K. H. (2009). Disaggregating and measuring implementor adherence and competence. Paper presented at the 2nd Annual NIH Conference on the Science of Dissemination and Implementation, Bethesda, MD.Google Scholar
  19. Dane, A. V., & Schneider, B. H. (1998). Program integrity in primary and early secondary prevention: Are implementation effects out of control? Clinical Psychology Review, 18, 23–45.CrossRefPubMedGoogle Scholar
  20. Diamond, G. M., Liddle, H. A., Hogue, A., & Dakof, G. A. (1999). Alliance-building interventions with adolescents in family therapy: A process study. Psychotherapy: Theory, Research, Practice, Training, 36, 355–368.CrossRefGoogle Scholar
  21. Dillman Carpentier, F., Mauricio, A. M., Gonzales, N. A., Millsap, R. E., Meza, C. M., Dumka, L. E., et al. (2007). Engaging Mexican origin families in a school-based preventive intervention. Journal of Primary Prevention, 28, 521–546.CrossRefPubMedGoogle Scholar
  22. Durlak, J. (1998). Why program implementation is important. Journal of Prevention & Intervention in the Community, 17, 5–18.CrossRefGoogle Scholar
  23. Durlak, J., & DuPre, E. (2008). Implementation matters: A review of research on the influence of implementation on program outcomes and the factors affecting implementation. American Journal of Community Psychology, 41, 327–350.CrossRefPubMedGoogle Scholar
  24. Dusenbury, L. A., Brannigan, R., Falco, M., & Hansen, W. B. (2003). A review of research on fidelity of implementation: Implications for drug abuse prevention in school settings. Health Education Research, 18, 237–256.CrossRefPubMedGoogle Scholar
  25. Dusenbury, L. A., Brannigan, R., Hansen, W. B., Walsh, J., & Falco, M. (2005). Quality of implementation: Developing measures crucial to understanding the diffusion of preventive interventions. Health Education Research, 20, 308–313.CrossRefPubMedGoogle Scholar
  26. Eames, C., Daley, D., Hutchings, J., Whitaker, C. J., Jones, K., Hughes, J. C., et al. (2009). Treatment fidelity as a predictor of behaviour change in parents attending group-based parent training. Child Care, Health & Development, 35, 603–612.CrossRefGoogle Scholar
  27. Eddy, J. M., Dishion, T. J., & Stoolmiller, M. (1998). The analysis of intervention change in children and families: Methodological and conceptual issues embedded in intervention studies. Journal of Abnormal Child Psychology, 26, 53–69.CrossRefPubMedGoogle Scholar
  28. Elliott, D. S., & Mihalic, S. (2004). Issues in disseminating and replicating effective prevention programs. Prevention Science, 5, 47–52.CrossRefPubMedGoogle Scholar
  29. Emshoff, J., Blakely, C., Gray, D., Jakes, S., Brounstein, P., Coulter, J., et al. (2003). An ESID case study at the federal level. American Journal of Community Psychology, 32, 345–357.CrossRefPubMedGoogle Scholar
  30. Fagan, A. A., Hanson, K., Hawkins, J. D., & Arthur, M. W. (2008). Bridging science to practice: Achieving prevention program implementation fidelity in the Community Youth Development study. American Journal of Community Psychology, 41, 235–249.CrossRefPubMedGoogle Scholar
  31. Forgatch, M. S., Patterson, G. R., & DeGarmo, D. S. (2005). Evaluating fidelity: Predictive validity for a measure of competent adherence to the Oregon model of parent management training. Behavior Therapy, 36, 3–13.CrossRefPubMedGoogle Scholar
  32. Garvey, C., Julion, W., Fogg, L., Kratovil, A., & Gross, D. (2006). Measuring participation in a prevention trial with parents of young children. Research in Nursing & Health, 29, 212–222.CrossRefGoogle Scholar
  33. Giles, S., Jackson-Newsom, J., Pankratz, M., Hansen, W., Ringwalt, C., & Dusenbury, L. (2008). Measuring quality of delivery in a substance use prevention program. Journal of Primary Prevention, 29, 489–501.CrossRefPubMedGoogle Scholar
  34. Gingiss, P., Roberts-Gray, C., & Boerm, M. (2006). Bridge-It: A system for predicting implementation fidelity for school-based tobacco prevention programs. Prevention Science, 7, 197–207.CrossRefPubMedGoogle Scholar
  35. Gonzales, N. A., Wolchik, S. A., Sandler, I. N., Winslow, E. B., Martinez, C. R., & Cooley, M. (2006, June). Quality management methods to adapt interventions for cultural diversity. Paper presented at the annual meeting of the Society for Prevention Research, San Antonio, TX.Google Scholar
  36. Haggerty, K. P., Fleming, C. B., Lonczak, H. S., Oxford, M. L., Harachi, T. W., & Catalano, R. F. (2002). Predictors of participation in parenting workshops. Journal of Primary Prevention, 22, 375–387.CrossRefGoogle Scholar
  37. Hansen, W. B., Graham, J. W., Wolkenstein, B. H., & Rohrbach, L. A. (1991). Program integrity as a moderator of prevention program effectiveness: Results for fifth-grade students in the Adolescent Alcohol Prevention Trial. Journal of Studies on Alcohol, 52, 568–579.PubMedGoogle Scholar
  38. Hubble, M. A., Duncan, B. L., & Miller, S. D. (1999). The heart and soul of change: What works in therapy. Washington, DC: American Psychological Association.CrossRefGoogle Scholar
  39. Ialongo, N. S., Werthamer, L., Kellam, S. G., Brown, C. H., Wang, S., & Lin, Y. (1999). Proximal impact of two first-grade preventive interventions on the early risk behaviors for later substance abuse, depression, and antisocial behavior. American Journal of Community Psychology, 27, 599–641.CrossRefPubMedGoogle Scholar
  40. Kam, C.-M., Greenberg, M. T., & Walls, C. T. (2003). Examining the role of implementation quality in school-based prevention using the PATHS curriculum. Prevention Science, 4, 55–63.CrossRefPubMedGoogle Scholar
  41. Kerr, D. M., Kent, L., & Lam, T. C. (1985). Measuring program implementation with a classroom observation instrument: The Interactive Teaching Map. Evaluation Review, 9, 461–482.CrossRefGoogle Scholar
  42. Kosterman, R., Hawkins, J. D., Haggerty, K. P., Spoth, R., & Redmond, C. (2001). Preparing for the Drug Free Years: Session-specific effects of a universal parent-training intervention with rural families. Journal of Drug Education, 31, 47–68.CrossRefPubMedGoogle Scholar
  43. Kumpfer, K. L., Alvarado, R., Smith, P., & Bellamy, N. (2002). Cultural sensitivity and adaptation in family-based prevention interventions. Prevention Science, 3, 241–246.CrossRefPubMedGoogle Scholar
  44. Lillehoj, C. J., Griffin, K. W., & Spoth, R. (2004). Program provider and observer ratings of school-based preventive intervention implementation: Agreement and relation to youth outcomes. Health Education & Behavior, 31, 242–257.CrossRefGoogle Scholar
  45. Lochman, J. E., Boxmeyer, C., Powell, N., Qu, L., Wells, K., & Windle, M. (2009). Dissemination of the Coping Power program: Importance of intensity of counselor training. Journal of Consulting & Clinical Psychology, 77, 397–409.CrossRefGoogle Scholar
  46. McGraw, S. A., Sellers, D. E., Stone, E. J., Bebchuk, J., Edmundson, E. W., Johnson, C. C., et al. (1996). Using process data to explain outcomes: An illustration from the Child and Adolescent Trial for Cardiovascular Health (CATCH). Evaluation Review, 20, 291–312.CrossRefPubMedGoogle Scholar
  47. NRC/IOM. (2009). Preventing mental, emotional, and behavioral disorders among young people: Progress and possibilities. Washington, DC: National Research Council and Institute of Medicine.Google Scholar
  48. Nye, C. L., Zucker, R. A., & Fitzgerald, H. E. (1995). Early intervention in the path to alcohol problems through conduct problems: Treatment involvement and child behavior change. Journal of Consulting & Clinical Psychology, 63, 831–840.CrossRefGoogle Scholar
  49. Palinkas, L. A., Aarons, G. A., Chorpita, B. F., Hoagwood, K., Landsverk, J., & Weisz, J. R. (2009). Cultural exchange and the implementation of evidence-based practices: Two case studies. Research on Social Work Practice, 19, 602–612.CrossRefGoogle Scholar
  50. Parcel, G. S., Ross, J. G., Lavin, A. T., Portnoy, B., Nelson, G. D., & Winters, F. (1991). Enhancing implementation of the Teenage Health Teaching Modules. Journal of School Health, 61, 35–38.CrossRefPubMedGoogle Scholar
  51. Patterson, G. R., & Forgatch, M. S. (1985). Therapist behavior as a determinant for client noncompliance: A paradox for the behavior modifier. Journal of Consulting & Clinical Psychology, 53, 846–851.CrossRefGoogle Scholar
  52. Pentz, M. A., Trebow, E. A., Hansen, W. B., MacKinnon, D. P., Dwyer, J. H., Johnson, C. A., et al. (1990). Effects of program implementation on adolescent drug use behavior: The Midwestern Prevention Project (MPP). Evaluation Review, 14, 264–289.CrossRefGoogle Scholar
  53. Prado, G., Pantin, H., Schwartz, S. J., Lupei, N. S., & Szapocznik, J. (2006). Predictors of engagement and retention into a parent-centered, ecodevelopmental HIV preventive intervention for Hispanic adolescents and their families. Journal of Pediatric Psychology, 31, 874–890.CrossRefPubMedGoogle Scholar
  54. Resnicow, K., Davis, M., Smith, M., Lazarus-Yaroch, A., Baranowski, T., Baranowski, J., et al. (1998). How best to measure implementation of school health curricula: A comparison of three measures. Health Education Research, 13, 239–250.CrossRefPubMedGoogle Scholar
  55. Ringwalt, C. L., Vincus, A., Ennett, S., Johnson, R., & Rohrbach, L. A. (2004). Reasons for teachers’ adaptation of substance use prevention curricula in schools with non-White student populations. Prevention Science, 5, 61–67.CrossRefPubMedGoogle Scholar
  56. Ringwalt, C. L., Pankratz, M., Jackson-Newsom, J., Gottfredson, N., Hansen, W. B., Giles, S. et al. (2010). Three-year trajectory of teachers’ fidelity to a drug prevention curriculum. Prevention Science, 11, 67–76.Google Scholar
  57. Rogers, E. M. (1995). Diffusion of drug abuse prevention programs: Spontaneous diffusion, agenda setting, and reinvention. Bethesda, MD: NIDA Research Monograph.Google Scholar
  58. Rogers, E. M. (2003). Diffusion of innovations (5th ed.). New York: The Free Press.Google Scholar
  59. Rohrbach, L. A., Gunning, M., Sun, P., & Sussman, S. (2010). The Project Towards No Drug Abuse (TND) dissemination trial: Implementation fidelity and immediate outcomes. Prevention Science, 11, 77–88.CrossRefPubMedGoogle Scholar
  60. Shelef, K., Diamond, G. M., Diamond, G. S., & Liddle, H. A. (2005). Adolescent and parent alliance and treatment outcome in Multidimensional Family Therapy. Journal of Consulting & Clinical Psychology, 73, 689–698.CrossRefGoogle Scholar
  61. Slavin, R. E. (1990). Comprehensive cooperative learning models: Embedding cooperative learning in the curriculum and school. In S. Sharan (Ed.), Cooperative learning: Theory and research (pp. 261–283). New York: Praeger.Google Scholar
  62. Spoth, R. L., & Redmond, C. (2002). Project Family prevention trials based in community-university partnerships: Toward scaled-up preventive interventions. Prevention Science, 3, 203–221.CrossRefPubMedGoogle Scholar
  63. Spoth, R., Guyll, M., Trudeau, L., & Goldberg-Lillehoj, C. (2002). Two studies of proximal outcomes and implementation quality of universal preventive interventions in a community-university collaboration context. Journal of Community Psychology, 30, 499–518.CrossRefGoogle Scholar
  64. Stanton, B., Harris, C., Cottrell, L., Li, X., Gibson, C., Guo, J., et al. (2006). Trial of an urban adolescent sexual risk-reduction intervention for rural youth: A promising but imperfect fit. Journal of Adolescent Health, 38, 25–36.CrossRefGoogle Scholar
  65. Tobler, N. S., & Stratton, H. H. (1997). Effectiveness of school-based drug prevention programs: A meta-analysis of the research. Journal of Primary Prevention, 18, 71–128.CrossRefGoogle Scholar
  66. Tolan, P. H., Hanish, L. D., McKay, M. M., & Dickey, M. H. (2002). Evaluating process in child and family interventions: Aggression prevention as an example. Journal of Family Psychology, 16, 220–236.CrossRefPubMedGoogle Scholar
  67. Widaman, K. F., & Kagan, S. (1987). Cooperativeness and achievement: Interaction of student cooperativeness with cooperative versus competitive classroom organization. Journal of School Psychology, 25, 355–365.CrossRefGoogle Scholar
  68. Wilson, B. D. M., & Miller, R. L. (2003). Examining strategies for culturally grounded HIV prevention: A review. AIDS Education & Prevention, 15, 184–202.CrossRefGoogle Scholar
  69. Wilson, S. J., & Lipsey, M. W. (2007). School-based interventions for aggressive and disruptive behavior: Update of a meta-analysis. American Journal of Preventive Medicine, 33, S130–S143.CrossRefPubMedGoogle Scholar
  70. Wolchik, S. A., Sandler, I. N., Jones, S., Gonzales, N., Doyle, K., Winslow, E., et al. (2009). The New Beginnings Program for divorcing and separating families: Moving from efficacy to effectiveness. Family Court Review, 47, 416–435.CrossRefPubMedGoogle Scholar

Copyright information

© Society for Prevention Research 2010

Authors and Affiliations

  • Cady Berkel
    • 1
    Email author
  • Anne M. Mauricio
    • 1
  • Erin Schoenfelder
    • 1
  • Irwin N. Sandler
    • 1
  1. 1.Prevention Research CenterArizona State UniversityPhoenixUSA

Personalised recommendations