Skip to main content
SpringerLink
Log in

Implementation Matters: A Review of Research on the Influence of Implementation on Program Outcomes and the Factors Affecting Implementation

  • Original Paper
  • Published:
American Journal of Community Psychology

Abstract

The first purpose of this review was to assess the impact of implementation on program outcomes, and the second purpose was to identify factors affecting the implementation process. Results from over quantitative 500 studies offered strong empirical support to the conclusion that the level of implementation affects the outcomes obtained in promotion and prevention programs. Findings from 81 additional reports indicate there are at least 23 contextual factors that influence implementation. The implementation process is affected by variables related to communities, providers and innovations, and aspects of the prevention delivery system (i.e., organizational functioning) and the prevention support system (i.e., training and technical assistance). The collection of implementation data is an essential feature of program evaluations, and more information is needed on which and how various factors influence implementation in different community settings.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

Notes

  1. We use the term dissemination to refer to the first of Roger’s stages of diffusion although some apply the term to multiple stages of the diffusion process.

  2. These journals included American Journal of Community Psychology, Health Education and Behavior, Health Education Research, Journal of Community Psychology, Journal of Primary Prevention, and Prevention Science.

  3. We refer here to traditional distinctions regarding what type of data were collected and how the data were analyzed. Several studies combined qualitative and quantitative methods.

  4. The risk status of students was the most important factor; i.e., students selected for intervention because of their early aggressive behavior improved the most.

  5. A fourth factor, gender of the participants, was also related to program outcomes but was not included in their subsequent analyses.

  6. Based on their pioneering survey of school-based implementation, Berman and McLaughlin (1976) were the first to stress the importance of mutual adaptation, that is, the organization should adapt to the innovation at the same time as the innovation is adapted to fit the organization. To our knowledge, the extent to which this has occurred during the diffusion of prevention or promotion programs has not been assessed.

References

  • *Abbott, R. D., O’Donnell, J., Hawkins, J. D., Hill, K. G., Kosterman, R., & Catalano, R. F. (1998). Changing teaching practices to promote achievement and bonding to school. American Journal of Orthopsychiatry, 68, 542–552.

    PubMed  Google Scholar 

  • *Aber, J. L., Jones, S. M., Brown, J. L., Chaudry, N., & Samples, F. (1998). Resolving conflict creatively: Evaluating the developmental effects of a school based violence prevention program in neighborhood and classroom context. Development and Psychopathology, 10, 187–213.

    PubMed  Google Scholar 

  • *Allison, K. R., Silverman, G., & Dignam, C. (1990). Effects on students of teacher training in use of a drug education curriculum. Journal of Drug Education, 20, 31–46.

    PubMed  Google Scholar 

  • Altschuld, J. W., Kumar, D. D., Smith, D. W., & Goodway, J. D. (1999). School-based educational innovations: Case illustrations of context sensitive evaluations. Family and Community Health, 22, 66–79.

    Google Scholar 

  • American Psychological Association. (2003). Guidelines on multicultural education, training, research, practice and organizational change for psychologists. American Psychologist, 58, 377–402.

    Google Scholar 

  • Ary, D. V., Biglan, A., Glasgow, R., Zoref, L., Black, C., Ochs, L., et al. (1990). The efficacy of social-influence prevention programs versus “standard care”: Are new initiatives needed? Journal of Behavioral Medicine, 13, 281–296.

    PubMed  Google Scholar 

  • *August, G. J., Bloomquist, M. L., Lee, S. S., Realmuto, G. M., & Hektner, J. M. (2006). Can evidence-based prevention programs be sustained in community practice settings? The Early Risers; advanced-stage effectiveness trial. Prevention Science, 7, 151–165.

    PubMed  Google Scholar 

  • *August, G. J., Egan, E. A., Realmuto, G. M., & Hektner, J. M. (2003a). Parceling component effects of a multifaceted prevention program for disruptive elementary school children. Journal of Abnormal Child Psychology, 31, 515–527.

    PubMed  Google Scholar 

  • *August, G. J., Lee, S. S., Bloomquist, M. L., Realmuto, G. M., & Hektner, J. M. (2003b). Dissemination of an evidence-based preventive intervention for aggressive children living in diverse, urban neighborhoods. Prevention Science, 4, 271–286.

    PubMed  Google Scholar 

  • *Backer, T. E. (2002). Finding the balance: Program fidelity and adaptation in substance abuse prevention: A state-of-the-art review. Rockville, MD: Center for Substance Abuse Prevention, Substance Abuse and Mental Health Services Administration.

    Google Scholar 

  • *Barr, J. E., Tubman, J. G., Montgomery, M. J., & Soza-Vento, R. M. (2002). Amenability and implementation in secondary school antitobacco programs. American Journal of Health Behavior, 26, 3–15.

    PubMed  Google Scholar 

  • *Basch, C. E., Sliepcevich, E. M., Gold, R. S., Duncan, D. F., & Kolbe, L. J. (1985). Avoiding type III errors in health education program evaluations: A case study. Health Education Quarterly, 12, 315–331.

    PubMed  Google Scholar 

  • *Basen-Engquist, K., O’Hara-Tompkins, N., Lovato, C. Y., Lewis, M. J., Parcel, G. S., & Gingiss, P. (1994). The effect of two types of teacher training on implementation of smart choices: A tobacco prevention curriculum. Journal of School Health, 64, 334–339.

    PubMed  Google Scholar 

  • *Battistich, V., Schaps, E., Watson, M., Solomon, D., & Lewis, C. (2000). Effects of the child development project on students’ drug use and other problem behaviors. The Journal of Primary Prevention, 21, 75–99.

    Google Scholar 

  • *Battistich, V., Schaps, E., & Wilson, N. (2004). Effects of an elementary school intervention on students’ “connectedness” to school and social adjustment during middle school. Journal of Primary Prevention, 24, 243–262.

    Google Scholar 

  • *Bauman, L., Stein, R., & Ireys, H. (1991). Reinventing fidelity: The transfer of social technology among settings. American Journal of Community Psychology, 19, 619–639.

    PubMed  Google Scholar 

  • *Bell, M. L., Kelley-Baker, T., Rider, R., & Ringwalt, C. (2005). Protecting You/Protecting me: Effects of an alcohol prevention and vehicle safety program on elementary students. Journal of School Health, 75, 171–177.

    PubMed  Google Scholar 

  • *Bellg, A., Borrelli, B., Resnick, B., Hecht, J., Minicucci, D. S., Ory, M., et al. (2004). Enhancing treatment fidelity in health behavior change studies: Best practices and recommendations from the NIH behavior change consortium. Health Psychology, 23, 443–451.

    PubMed  Google Scholar 

  • *Berman, P., & McLaughlin, M. W. (1976). Implementation of educational innovation. The Educational Forum, 40, 345–370.

    Google Scholar 

  • *Binford, V. M., & Newwell, J. M. (1991). Richmond, Virginia’s two decades of experience with Ira Gordon’s approach to parent education. The Elementary School Journal, 91, 233–237.

    Google Scholar 

  • *Blakely, C. H., Mayer, J. P., Gottschalk, R. G., Schmitt, N., Davidson, W. S., Roitman, D. B., et al. (1987). The fidelity-adaptation debate: Implications for the implementation of public sector social programs. American Journal of Community Psychology, 15, 253–268.

    Google Scholar 

  • *Bloom, H. S., Hill, C. J., & Riccio, J. A. (2003). Linking program implementation and effectiveness: Lessons from a pooled sample of welfare-to-work experiments. Journal of Policy Analysis and Management, 22, 551–575.

    Google Scholar 

  • Botvin, G. J. (2000). Preventing drug abuse in schools: Social and competence enhancement approaches targeting individual-level etiologic factors. Addictive Behaviors, 25, 887–897.

    PubMed  Google Scholar 

  • *Botvin, G. J., Baker, E., Dusenbury, L., Botvin, E. M., & Diaz, T. (1995). Long-term follow-up results of a randomized drug abuse prevention trial in a white middle-class population. Journal of the American Medical Association, 273, 1106–1112.

    PubMed  Google Scholar 

  • *Botvin, G. J., Baker, E., Dusenbury, L., Tortu, S., & Botvin, E. (1990). Preventing adolescent drug abuse through a multimodal cognitive-behavioral approach: Results of a 3 year study. Journal of Consulting and Clinical Psychology, 58, 437–446.

    PubMed  Google Scholar 

  • *Botvin, G. J., Baker, E., Filazzola, A. D., & Botvin, E. M. (1990). A cognitive-behavioral approach to substance abuse prevention: 1 Year follow-up. Addictive Behaviors, 15, 47–63.

    PubMed  Google Scholar 

  • *Botvin, G. J., Baker, E., Renick, N. L., Filazzola, A. D., & Botvin, E. M. (1994). A cognitive-behavioral approach to substance abuse prevention. Addictive Behaviors, 9, 137–147.

    Google Scholar 

  • *Botvin, G. J., Dusenbury, L., Baker, E., James-Ortiz, S., Botvin, E. M., & Kerner, J. (1992). Smoking prevention among urban minority youth: Assessing effects on outcome and mediating variables. Health Psychology, 11, 290–299.

    PubMed  Google Scholar 

  • *Botvin, G. J., Dusenbury, L., Baker, E., James-Ortiz, S., & Kerner, J. (1989). A skills training approach to smoking prevention among Hispanic youth. Journal of Behavioral Medicine, 12, 279–296.

    PubMed  Google Scholar 

  • *Bush, P. J., Zuckerman, A. E., Taggart, V. S., Theiss, P. K., Peleg, E. O., & Smith, S. A. (1989). Cardiovascular risk factor prevention in black school children: The “Know Your Body” evaluation project. Health Education Quarterly, 16, 215–227.

    PubMed  Google Scholar 

  • Chinman, M., Hannah, G., Wandersman, A., Ebener, P. P., Hunter, S. B., Imm, P., et al. (2005). Developing a community science research agenda for building community capacity for effective preventive interventions. American Journal of Community Psychology, 35, 143–157.

    PubMed  Google Scholar 

  • *Chinman, M., Imm, P., & Wandersman, A. (2004). Getting to outcomes 2004: Promoting accountability through methods and tools for planning, implementation, and evaluation (Technical Report). Santa Monica, CA: RAND Corporation.

    Google Scholar 

  • *Cho, H., Hallfors, D. D., & Sanchez, V. (2005). Evaluation of a high school peer group intervention for at-risk youth. Journal of Abnormal Child Psychology, 33, 363–374.

    PubMed  Google Scholar 

  • *Conduct Problems Prevention Research Group. (1999). Initial impact of the Fast Track prevention trial for conduct problems: II. Classroom effects. Journal of Consulting and Clinical Psychology, 67, 648–657.

    Google Scholar 

  • Connell, D. B., Turner, R. R., & Mason, E. F. (1985). Summary of findings on the school health evaluation: Health promotion, effectiveness, implementation, and costs. Journal of School Health, 55, 316–321.

    PubMed  Google Scholar 

  • *Cook, T. D., Habib, F. N., Phillips, M., Settersten, R. A., Shagle, S. C., & Degirmencioglu, S. M. (1999). Comer’s school development program in prince George’s county, Maryland: A theory-based evaluation. American Educational Research Journal, 36, 543–597.

    Google Scholar 

  • *Cook, T. D., Murphy, R. F., & Hunt, H.D. (2000). Comer’s school development program in Chicago: A theory-based evaluation. American Educational Research Journal, 37, 535–597.

    Google Scholar 

  • *Cooke, M. (2000). The dissemination of a smoking cessation program: Predictors of program awareness, adoption and maintenance. Health Promotion International, 15, 113–124.

    Google Scholar 

  • *Cullen, K. W., Baranowski, T., Baranowski, J., Herbert, D., deMoor, C., Hearn, M. D., et al. (1999). Influence of school organizational characteristics on the outcomes of a school health promotion program. Journal of School Health, 69, 376–380.

    PubMed  Google Scholar 

  • *Cunningham, P. B., & Henggeler, S. W. (2001). Implementation of an empirically based drug and violence prevention and intervention program in public school settings. Journal of Consulting and Clinical Psychology, 30, 221–232.

    Google Scholar 

  • Dane, A. V., & Schneider, B. H. (1998). Program integrity in primary and early secondary prevention: Are implementation effects out of control. Clinical Psychology Review, 18, 23–45.

    PubMed  Google Scholar 

  • *Derzon, J. H., Sale, E., Springer, J. F., & Brounstein, P. (2005). Estimating intervention effectiveness: Synthetic projection of field evaluation results. The Journal of Primary Prevention, 26, 321–343.

    PubMed  Google Scholar 

  • Domitrovich, C. E., & Greenberg, M. T. (2000). The study of implementation: Current findings from effective programs that prevent mental disorders in school-aged children. Journal of Educational and Psychological Consultation, 11, 193–221.

    Google Scholar 

  • *Dubas, J. S., Lynch, K. B., Galano, J., Geller, S., & Hunt, D. (1998). Preliminary evaluation of a resiliency-based preschool substance abuse and violence prevention project. Journal of Drug Education, 28, 235–255.

    PubMed  Google Scholar 

  • *DuBois, D. L., Holloway, B. E., Valentine, J. C., & Cooper, H. (2002). Effectiveness of mentoring programs for youth: A meta-analytic review. American Journal of Community Psychology, 30, 157–198.

    PubMed  Google Scholar 

  • *Dufrene, B. A., Noell, G. H., Gilbertson, D. N., & Duhon, G. J. (2005). Monitoring implementation of reciprocal peer tutoring: Identifying and intervening with students who do not maintain accurate implementation. School Psychology Review, 34, 74–86.

    Google Scholar 

  • Durlak, J. A. (1997). Successful prevention programs for children and adolescents. New York: Plenum.

    Google Scholar 

  • Durlak, J. A. (1985). School-based prevention programs for children and adolescents. Thousand Oaks, CA: Sage.

  • Durlak, J. A. (1998). Why program implementation is important. Journal of Prevention & Intervention in the Community, 17, 5–18.

    Google Scholar 

  • Dusenbury, L., Brannigan, R., Falco, F., & Hansen, W. B. (2003a). A review of research on fidelity of implementation: implications for drug abuse prevention in school settings. Health Education Research, 18, 237–256.

    PubMed  Google Scholar 

  • *Dusenbury, L., Brannigan, R., Falco, M., & Lake, A. (2003b). An exploration of fidelity of implementation in drug abuse prevention among five professional groups. Journal of Alcohol and Drug Education, 47, 4–19.

    Google Scholar 

  • *Elder, J. P., Perry, C. L., Stone, E. J., Johnson, C. C., Yang, M., Edmundson, E. W., et al. (1996). Tobacco use measurement, prediction, and intervention in elementary schools in four states: The CATCH study. Preventive Medicine, 25, 489–494.

    Google Scholar 

  • *Elias, M. J., Gara, M., Ubriaco, M., Rothbaum, P. A., Clabby, J. F., & Schuyler, T. (1986). Impact of a preventive social problem solving intervention on children’s coping with middle-school stressors. American Journal of Community Psychology, 14, 259–275.

    PubMed  Google Scholar 

  • *Elias, M. J., Zins, J. E., Graczyk, P. A., & Weissberg, R. P. (2003). Implementation, sustainability, and scaling up of social-emotional and academic innovations in public schools. School Psychology Review, 32, 303–319.

    Google Scholar 

  • *Engstrom, M., Jason, L. A., Townsend, S. M., Pokorny, S. B., & Curie, C. J. (2002). Community readiness for prevention: Applying stage theory to multi-community interventions. Journal of Prevention & Intervention in the Community, 24, 29–46.

    Google Scholar 

  • *Epstein, A. S. (1993). Training for quality: Improving early childhood programs systematic training. Monographs of the High/Scope Educational Research Foundation, No. 9. Ypsilanti, MI: The High/Scope Press.

    Google Scholar 

  • *Fagan, A. A., & Mihalic, S. (2003). Strategies for enhancing the adoption of school-based prevention programs: Lessons learned from the blueprints for violence prevention replications of the life skills training program. Journal of Community Psychology, 31, 235–253.

    Google Scholar 

  • Felner, R. D., Favazza, A., Shim, M., Brand, S., Gu, K., & Noonan, N. (2001). Whole school improvement and restructuring as prevention and promotion. Journal of School Psychology, 39, 177–202.

    Google Scholar 

  • *Felner, R. D., Jackson, A., Kasak, D., Mulhall, P., Brand, S., & Flowers, N. (1997). The impact of middle school reform for the middle years: Longitudinal study of a network engaged in Turning Points-based comprehensive school transformation. Phi Delta Kappan, 78, 528–532.

    Google Scholar 

  • Fixsen, D. L., Naoom, S. F., Blasé, K. A., Friedman, R. M., & Wallace, F. (2005). Implementation research: A synthesis of the literature. Tampa, FL: University of South Florida, Louis de la Parte Florida Mental Health Institute, The National Implementation Research Network (FMHI Publication #231). Retrieved November 1, 2006, from http://nirn.fmhi.usf.edu/resources/publications/Monograph/pdf/monograph_full.pdf.

  • *Forgatch, M. S., Patterson, G. R., & DeGarmo, D. S. (2005). Evaluating fidelity: Predictive validity for a measure of component adherence to the Oregon model of parent management training. Behavior Therapy, 36, 3–13.

    PubMed  Google Scholar 

  • *Fors, S. W., & Doster, M. E. (1985). Implications of results: Factors for success. Journal of School Health, 55, 332–334.

    PubMed  Google Scholar 

  • *Fullan, M., & Pomfret, A. (1977). Research on curriculum and instruction implementation. Review of Educational Research, 47, 335–397.

    Google Scholar 

  • *Gerstenblith, S. A., Soule, D. A., Gottfredson, D. C., Lu, S. Kellstrom, M. A., Womer, S. C., et al. (2005). After-school programs, antisocial behavior, and positive youth development: An exploration of the relationship between program implementation and changes in youth behavior. In J. L. Mahoney, J. S. Eccles & R. W. Larson (Eds.), Organized activities as contexts of development: Extracurricular activities, after-school and community programs. Mahwah, NJ: Erlbaum.

  • *Gingiss, P. L. (1992). Enhancing program implementation and maintenance through a multi-phase approach to peer-based staff development. Journal of School Health, 62, 161–166.

    PubMed  Google Scholar 

  • *Gingiss, P. L., Gottlieb, N. H., & Brink, S. G. (1994). Increasing teacher receptivity toward use of tobacco prevention education programs. Journal of Drug Education, 24, 163–176.

    PubMed  Google Scholar 

  • *Gingiss, P. M., Roberts-Gray, C., & Boerm, M. (2006). Bridge-It: A system for predicting implementation fidelity for school-based tobacco prevention programs. Prevention Science, 7, 197–207.

    PubMed  Google Scholar 

  • *Goldman, K. D. (1994). Perceptions of innovations as predictors of implementation levels: The diffusion of a nationwide health education campaign. Health Education Quarterly, 21, 433–443.

    PubMed  Google Scholar 

  • *Gottfredson, D. C., & Gottfredson, G. D. (2002). Quality of school-based prevention programs: Results from a national survey. Journal of Research in Crime and Delinquency, 39, 3–35.

    Google Scholar 

  • *Gottfredson, D. C., Gottfredson, G. D., & Hybl, L. G. (1993). Managing adolescent behavior: A multiyear, multischool study. American Educational Research Journal, 30, 179–215.

    Google Scholar 

  • Granner, M. L., & Sharpe, P. A. (2004). Evaluating community coalition characteristics and functioning: A summary of measurement tools. Health Education Research, 19, 514–532.

    PubMed  Google Scholar 

  • Greenhalgh, T., Robert, G., Macfarlane, F., Bate, P., Kyriakidou, O., & Peacock, R. (2005). Diffusion of innovations in health service organizations: A systematic literature review. Oxford: Blackwell.

    Google Scholar 

  • *Greenwood, C. R., Dinwiddie, G., Bailey, V., Carta, J. J., Dorsey, D., Kohler, F. W., et al. (1987). Field replication of classwide peer tutoring. Journal of Applied Behavior Analysis, 20, 151–160.

    PubMed  Google Scholar 

  • *Greenwood, C. R., Tapia, Y., Abbott, M., & Walton, C. (2003). A building-based case study of evidence-based literacy practices: Implementation, reading behavior, and growth in reading fluency, K-4. The Journal of Special Education, 37, 95–110.

    Google Scholar 

  • *Greenwood, C. R., Terry, B., Arreaga-Mayer, C., & Finney, R. (1992). The classwide peer tutoring program: Implementation factors moderating students’ achievement. Journal of Applied Behavior Analysis, 25, 101–116.

    PubMed  Google Scholar 

  • *Hahn, E. J., Noland, M. P., Rayens, M. K., & Christie, D. M. (2002). Efficacy of training and fidelity of implementation of the life skills training program. Journal of School Health, 72, 282–287.

    PubMed  Google Scholar 

  • *Hansen, W. B., Graham, J. W., Wolkenstein, B. H., & Rohrbach, L. A. (1991). Program integrity as a moderator of prevention program effectiveness: Results for fifth grade students in the adolescent alcohol prevention trial. Journal of Studies on Alcohol, 52, 568–579.

    PubMed  Google Scholar 

  • *Hansen, W. B., & McNeal, R. B. (1999). Drug education practice: Results of an observational study. Health Education Research, 14, 85–97.

    PubMed  Google Scholar 

  • *Harachi, T. W., Abbott, R. D., Catalano, R. F., Haggerty, K. P., & Fleming, C. B. (1999). Opening the black box: Using process evaluation measures to assess implementation and theory building. American Journal of Community Psychology, 27, 711–731.

    PubMed  Google Scholar 

  • *Henderson, J. L., MacKay, S., & Peterson-Badali, M. (2006). Closing the research-practice gap: Factors affecting adoption and implementation of a children’s mental health program. Journal of Clinical Child and Adolescent Psychology, 35, 2–12.

    PubMed  Google Scholar 

  • *Hogan, J. A., Baca, I., Daley, C., Garcia, T., Jaker, J., Lowther, M., et al. (2003). Disseminating science-based prevention: Lessons learned from CSAP’s CAPTs. Journal of Drug Education, 33, 233–244.

    PubMed  Google Scholar 

  • Hogue, A., Liddle, H. A., Singer, A., & Leckrone, J. (2005). Intervention fidelity in family-based prevention counseling for adolescent problem behaviors. Journal of Community Psychology, 33, 191–211.

    Google Scholar 

  • *Hopkins, R. H., Mauss, A. L., Kearney, K. A., & Weisheit, R. A. (1988). Comprehensive evaluation of a model alcohol education curriculum. Journal of Studies on Alcohol, 49, 38–50.

    PubMed  Google Scholar 

  • *Hopper, C. A., Munoz, K. D., Gruber, M. B., MacConnie, S., Schonfeldt, B., & Shunk, T. (1996). A school-based cardiovascular exercise and nutrition program with parent participation: An evaluation study. Children’s Health Care, 25, 221–235.

    Google Scholar 

  • *Ialongo, N., Werthamer, L., Kellam, S. G., Brown, C. H., Wang, S., & Lin, Y. (1999). Proximal impact of two first-grade preventive interventions on the early risk behaviors for later substance abuse, depression, and antisocial behavior. American Journal of Community Psychology, 27, 599–641.

    PubMed  Google Scholar 

  • *James, S., Reddy, P., Ruiter, A. M., & van den Borne, B. (2006). The impact of an HIV and AIDS life skills program on secondary school students in Kwazulu-Natal, South Africa. AIDS Education and Prevention, 18, 281–294.

    Google Scholar 

  • *Jason, L. A., Pokorny, S. B., Kunz, C., & Adams, M. (2004). Maintenance of community change: Enforcing youth access to tobacco laws. Journal of Drug Education, 34, 105–119.

    PubMed  Google Scholar 

  • *Kalafat, J., & Ryerson, D. M. (1999). The implementation and institutionalization of a school-based youth suicide prevention program. Journal of Primary Prevention, 19, 157–175.

    Google Scholar 

  • *Kallestad, J. H., & Olweus, D. (2003). Predicting teachers’ and schools’ implementation of the Olweus bullying prevention program: A multilevel study. Prevention & Treatment, 6. Retrieved June 20, 2005.

  • *Kam, C.-M., Greenberg, M. T., & Walls, C. T. (2003). Examining the role of implementation quality in school-based prevention using the PATHS curriculum. Prevention Science, 4, 55–63.

    PubMed  Google Scholar 

  • Kazdin, A. E., Bass, D., Ayers, W. A., & Rodgers, A. (1990). Empirical and clinical focus of child and adolescent psychotherapy research. Journal of Consulting and Clinical Psychology, 58, 729–740.

    PubMed  Google Scholar 

  • *Kealey, K. A., Peterson, A. V., Gaul, M. A., & Dinh, K. T. (2000). Teacher training as a behavior change process: Principles and results from a longitudinal study. Health Education & Behavior, 27, 64–81.

    Google Scholar 

  • *Kegler, M. C., Steckler, A., Malek, S. H., & McLeroy, K. (1998a). A multiple case study of implementation in 10 local project ASSIST coalitions in North Carolina. Health Education Research, 13, 225–238.

    PubMed  Google Scholar 

  • *Kegler, M. C., Steckler, A., McLeroy, K., & Malek, S. H. (1998b). Factors that contribute to effective community health promotion coalitions: A study of 10 project ASSIST coalitions in North Carolina. Health Education & Behavior, 25, 338–353.

    Google Scholar 

  • *Kegler, M. C., & Wyatt, V. H. (2003). A multiple case study of neighborhood partnerships for positive youth development. American Journal of Health Behavior, 27, 156–169.

    PubMed  Google Scholar 

  • *Kelly, J. A., Somlai, A. M., DiFranceisco, W. J., Otto-Salaj, L. L., McAuliffe, T. L., Hackl, K. L., et al. (2000). Bridging the gap between the science and service of HIV prevention: Transferring effective research-based HIV prevention interventions to community AIDS service providers. American Journal of Public Health, 90, 1082–1088.

    PubMed  Google Scholar 

  • Kendrick, J. S., Zahniser, C., Miller, N., Salas, N., Stine, J. Gargiuillo, P. M., et al. (1995). Integrating smoking cessation into routine public prenatal care: The smoking cessation pregnancy project. American Journal of Public Health, 85, 217–222.

    PubMed  Google Scholar 

  • *Kerr, D. M., Kent, L., & Lam, T. C. M. (1985). Measuring program implementation with a classroom observation instrument: The interactive teaching map. Evaluation Review, 9, 461–482.

    Google Scholar 

  • *Komro, K. A., Perry, C. L., Veblen-Mortenson, S., Bosma, L. M., Dudovitz, B. S., Williams, C. L., et al. (2004). Brief report: The adaptation of Project Northland for urban youth. Journal of Pediatric Psychology, 29, 457–466.

    PubMed  Google Scholar 

  • *Komro, K. A., Perry, C. L., Veblen-Mortenson, S., Farbakhsh, K., Kugler, K. C., Alfano, K. A., et al. (2006). Cross-cultural adaptation and evaluation of a home-based program for alcohol use prevention among urban youth: The “Slick Tracy Home Team Program”. The Journal of Primary Prevention, 27, 135–154.

    PubMed  Google Scholar 

  • *Kramer, L., Laumann, G., & Brunson, L. (2000). Implementation and diffusion of the rainbows program in rural communities: Implications for school-based prevention programming. Journal of Educational and Psychological Consultation, 11, 37–64.

    Google Scholar 

  • *Lapan, R. T., Gysbers, N. C., & Petroski, G. F. (2001). Helping seventh graders be safe and successful: A statewide study of the impact of comprehensive guidance and counseling programs. Journal of Consulting & Development, 79, 320–330.

    Google Scholar 

  • *Levenson-Gingiss, P., Gottlieb, N. H., & Brink, S. G. (1994). Increasing teacher receptivity toward use of tobacco prevention education programs. Journal of Drug Education, 24, 163–176.

    Google Scholar 

  • *Lillehoj, C. J. G., Griffin, K. W., & Spoth, R. (2004). Program provider and observer ratings of school-based preventive intervention implementation: Agreement and relation to youth outcomes. Health Education & Behavior, 31, 242–257.

    Google Scholar 

  • *Lynch, K. B., Geller, S. R., Hunt, D. R., Galano, J., & Dubas, J. S. (1998). Successful program develoment using implementation evaluation. Journal of Prevention & Intervention in the Community, 17, 51–64.

    Google Scholar 

  • *MacDonald, M. A., & Green, L. W. (2001). Reconciling concept and context: The dilemma of implementation in school-based health promotion. Health Education & Behavior, 28, 749–768.

    Google Scholar 

  • *McCormick, L. K., Steckler, A., & McLeroy, K. R. (1994). Diffusion of innovation in schools: A study of adoption and implementation of school-based tobacco prevention curricula. American Journal of Health Promotion, 9, 210–219.

    Google Scholar 

  • *McGraw, S., Sellers, D., Stone, E., Bebchuk, J., Edmundson, E., Johnson, C., et al. (1996). Using process data to explain outcomes: An illustration from the child and adolescent trial for cardiovascular health (CATCH). Evaluation Review, 20, 291–312.

    PubMed  Google Scholar 

  • *McKenzie, T. L., Nader, P. R., Strikmiller, P. K., Yang, M., Stone, E. J., erry, C. L, et al. (1996). School physical education: Effect of the child and adolescent trial for cardiovascular health. Preventive Medicine, 25, 423–431.

    PubMed  Google Scholar 

  • *Mihalic, S., Irwin, K., Fagan, A., Ballard, D., & Elliott, D. (2004). Successful program implementation: Lessons from blueprints. Electronic report. U.S. Department of Justice, Office of Justice Programs). Retrieved August 10, 2006, from http://www.ojp.usdoj.gov/ojjdp.

  • Miranda, J., Bernal, G., Lau, A., Kohn, L., Hwangh, W., & LaFromboise, T. (2005). State of the science on psychosocial interventions for ethnic minorities. Annual Review of Clinical Psychology, 1, 113–142.

    PubMed  Google Scholar 

  • Mitchell, C. M. (1983). The dissemination of a social intervention: Process and effectiveness of two types of paraprofessional change agents. American Journal of Community Psychology, 11, 723–738.

    PubMed  Google Scholar 

  • *Moskowitz, J. M., Schaps, E., & Malvin, J. H. (1982). Process and outcome evaluations in primary prevention: The magic circle program. Evaluation Review, 6, 775–788.

    Google Scholar 

  • Mowbray, C. T., Holter, M. C., Teague, G. B., & Bybee, D. (2003). Fidelity criteria: Developmental, measurement, and validation. American Journal of Evaluation, 24, 315–340.

    Google Scholar 

  • Nastasi, B. K., & Schensul, S. L. (2005). Contributions of qualitative research to the validity of intervention research. Journal of School Psychology, 43, 177–195.

    Google Scholar 

  • National Research Council, Institute of Medicine. (2002). Community programs to promote youth development. Washington, DC: National Academy Press.

    Google Scholar 

  • *Noell, G. H., Duhon, G. J., Gatti, S. L., & Connell, J. E. (2002). Consultation, follow-up, and implementation of behavior management interventions in general education. School Psychology Review, 31, 217–234.

    Google Scholar 

  • *Noell, G. H., & Witt, J. C. (1999). When does consultation lead to intervention implementation? The Journal of Special Education, 33, 29–35.

    Google Scholar 

  • *Noell, G. H., Witt, J. C., Gilbertson, D. N., Ranier, D. D., & Freeland, J. T. (1997). Increasing teacher intervention implementation in general education settings through consultation and performance feedback. School Psychology Quarterly, 12, 77–88.

    Google Scholar 

  • *Noell, G. H., Witt, J. C., Slider, N. J., Connell, J. E., Gatti, S. L., Williams, K. L. et al. (2005). Treatment implementation following behavioral consultation in schools: A comparison of three follow-up strategies. School Psychology Review, 34, 87–106.

    Google Scholar 

  • *Parcel, G. S., Ross, J. G., Lavin, A. T., Portnoy, B., Nelson, G. D., & Winters, F. (1991). Enhancing implementation of the teenage health teaching modules. Journal of School Health, 61, 35–38.

    PubMed  Google Scholar 

  • *Payne, A. A., Gottfredson, D. C., & Gottfredson, G. D. (2006). School predictors of the intensity of implementation of school-based prevention programs: Results from a national study. Prevention Science, 7, 225–237.

    PubMed  Google Scholar 

  • *Pentz, M. A., Trebow, E. A., Hansen, W. B., MacKinnon, D. B., Dwyer, J. H., Johnson, C. A. et al. (1990). Effects of program implementation on adolescent drug use behavior: The Midwestern Prevention Project (MPP). Evaluation Review, 14, 264–289.

    Google Scholar 

  • *Perry, C. L., Murray, D. M., & Griffin, G. (1990). Evaluating the statewide dissemination of smoking prevention curricula: Factors in teacher compliance. Journal of School Health, 60, 501–504.

    PubMed  Google Scholar 

  • *Perry, C. L., Sellers, D. E., Johnson, C., Pedersen, S., Bachman, K. J., Parcel, G. S., et al. (1997). The Child and Adolescent Trial for Cardiovascular Health (CATCH): Intervention, implementation, and feasibility for elementary schools in the United States. Health Education and Behavior, 24, 717–735.

    Google Scholar 

  • *Piper, D. L., King, M. J., & Moberg, D. P. (1993). Implementing a middle school health promotion research project: Lessons our textbook didn’t teach us. Evolution & Program Planning, 16, 171–180.

    Google Scholar 

  • *Pluye, P., Potvin, L., Denis, J. L., Pelletier, J., & Mannoni, C. (2005). Program sustainability begins with the first events. Evaluation and Program Planning, 28, 123–137.

    Google Scholar 

  • *Rankin, W., Tarnai, J., Fagan, N. J., Mauss, A. L., & Hopkins, R. H. (1978). An evaluation of workshops designed to prepare teachers in alcohol education. Journal of Alcohol & Drug Education, 23, 1–13.

    Google Scholar 

  • Resnicow, K., Baranowski, T., Ahluwalia, J. S., & Braithwaite, R. L. (1999). Cultural sensitivity in public health: Defined and demystified. Ethnicity and Disease, 9, 10–21.

    PubMed  Google Scholar 

  • *Resnicow, K., Cohn, L., Reinhardt, J., Cross, D., Futterman, R., Kirschner, E., et al. (1992). A 3-year evaluation of the Know Your Body program in inner-city schoolchildren. Health Education Quarterly, 19, 463–480.

    PubMed  Google Scholar 

  • *Resnicow, K., Davis, M., Smith, M., Baranowski, T., Lin, L. S., Baranowski, J., et al. (1998a). Results of the TeachWell worksite wellness program. American Journal of Public Health, 88, 250–257.

    PubMed  Google Scholar 

  • *Resnicow, K., Davis, M., Smith, M., Lazarus-Yaroch, A., Baranowski, T., Baranowski, J., et al. (1998b). How best to measure implementation of school health curricula: A comparison of three measures. Health Education Research, 13, 239–250.

    PubMed  Google Scholar 

  • *Reynolds, K. D., Franklin, F. A., Leviton, L. C., Maloy, J., Harrington, K. F., Yaroch, A. L., et al. (2000). Methods, results, and lessons learned from process evaluation of the High 5 school-based nutrition intervention. Health Education & Behavior, 27, 177–186.

    Google Scholar 

  • *Richard, L., Lehoux, P., Breton, E., Denis, J., Labrie, L., & Leonard, C. (2004). Implementing the ecological approach in tobacco control programs: Results of a case study. Evaluation and Program Planning, 27, 409–421.

    Google Scholar 

  • *Riley, B. L. (2003). Dissemination of heart health promotion in the Ontario public health system: 1989–1999. Health Education Research, 18, 15–31.

    PubMed  Google Scholar 

  • *Riley, B. L., Taylor, S. M., & Elliott, S. J. (2001). Determinants of implementing heart healthy promotion activities in Ontario public health units: A social ecological perspective. Health Education Research, 16, 425–441.

    PubMed  Google Scholar 

  • *Riley, B. L., Taylor, S. M., & Elliott, S. J. (2003). Organizational capacity and implementation change: A comparative case study of heart health promotion in Ontario public health agencies. Health Education Research, 18, 754–769.

    PubMed  Google Scholar 

  • *Ringwalt, C. L., Ennett, S., Johnson, R., Rohrbach, L. A., Simons-Rudolph, A., Vincus, A., et al. (2003). Factors associated with fidelity to substance use prevention curriculum guides in the nation’s middle schools. Health Education & Behavior, 30, 375–391.

    Google Scholar 

  • *Rogers, E. M. (2003). Diffusion of innovations (5th ed.). New York: Free Press.

  • *Rohrbach, L. A., Graham, J. W., & Hansen, W. B. (1993). Diffusion of a school-based substance abuse prevention program: Predictors of program implementation. Preventive Medicine, 22, 237–260.

    PubMed  Google Scholar 

  • *Ross, J. G., Kuepker, R. V., Nelson, G. D., Saavedra, P., & Hubbard, B. M. (1991). Teenage health teaching modules: Impact of teacher training on implementation and student outcomes. Journal of School Health, 61, 31–34.

    PubMed  Google Scholar 

  • Saunders, R. P., Ward, D., Felton, G. M., Dowda, M., & Pate, R. R. (2006). Examining the link between program implementation and behavior outcomes in the lifestyle education for activity program (LEAP). Evaluation and Program Planning, 29, 352–364.

    Google Scholar 

  • *Scheirer, M. A. (1990). The life cycle of an innovation: Adoption versus discontinuation of the fluoride mouth rinse program in schools. Journal of Health and Social Behavior, 31, 203–215.

    PubMed  Google Scholar 

  • *Scheirer, M. A. (2005). Is sustainability possible: A review and commentary on empirical studies of program sustainability. American Journal of Evaluation, 26, 320–347.

    Google Scholar 

  • *Scheirer, M. A., Shediac, M. C., & Cassady, C. E. (1995). Measuring the implementation of health promotion programs: The case of the Breast and Cervical Cancer Program in Maryland. Health Education Research, 10, 11–25.

    PubMed  Google Scholar 

  • Shediac-Rizkallah, M. C., & Bone, L. R. (1998). Planning for the sustainability of community-based health programs: Conceptual frameworks and future directions for research, practice and policy. Health Education Research, 13, 87–108.

    PubMed  Google Scholar 

  • *Smith, D. W., McCormick, L. K., Steckler, A. B., & McLeroy, K. R. (1993). Teachers’ use of health curricula: Implementation of growing healthy, project SMART, and the teenage health teaching modules. Journal of School Health, 63, 349–354.

    PubMed  Google Scholar 

  • *Smith, D. W., Redican, K. J., & Olsen, L. K. (1992). The longevity of growing healthy: An analysis of the eight original sites implementing the school health curriculum project. Journal of School Health, 62, 83–87.

    PubMed  Google Scholar 

  • *Smith, J. D., Schneider, B. H., Smith, P. K., & Ananiadou, K. (2004). The effectiveness of whole-school antibullying programs: A synthesis of evaluation research. School Psychology Review, 33, 547–560.

    Google Scholar 

  • *Solomon, D., Battistich, V., Watson, M., Schaps, E., & Lewis, C. (2000). A six-district study of educational change: Direct and mediated effects of the Child Development Project. Social Psychology Education, 4, 3–51.

    Google Scholar 

  • *Spoth, R., Guyll, M., Trudeau, L., & Goldberg-Lillehoj, C. (2002). Two studies of proximal outcomes and implementation quality of universal preventive interventions in a community-university collaboration context. Journal of Community Psychology, 30, 499–518.

    Google Scholar 

  • *Steckler, A., McLeroy, K. R., Goodman, R. M., Smith, D., Dawson, L., & Howell, K. (1989). The importance of school district policies in the dissemination of tobacco use curricula in North Carolina schools. Family & Community Health, 12, 14–25.

    Google Scholar 

  • *Sterling-Turner, H. E., Watson, T. S., & Moore, J. W. (2002). The effects of direct training and treatment integrity on treatment outcomes in school consultation. School Psychology Quarterly, 17, 47–77.

    Google Scholar 

  • *Stevens, V., Van Oost, P., & De Bourdeaudhuij, I. (2001). Implementation process of the Flemish antibullying intervention and relation with program effectiveness. Journal of School Psychology, 39, 303–317.

    Google Scholar 

  • Stith, S., Pruitt, I., Dees, J., Fronce, M., Green, N., Som, A. et al. (2006). Implementing community-based prevention programming: A review of the literature. Journal of Primary Prevention, 27, 599–617.

    PubMed  Google Scholar 

  • *Story, M., Mays, R. W., Bishop, D. B., Perry, C. L., Taylor, G., Smyth, M., et al. (2000). 5-a-day power plus: Process evaluation of a multicomponent elementary school program to increase fruit and vegetable consumption. Health Education & Behavior, 27, 187–200.

    Google Scholar 

  • *Stotts, A. L., diClemente, C. C., & Dolan-Mullen, P. (2002). One-to-one: A motivational intervention for resistant pregnant smokers. Addictive Behaviors, 27, 275–292.

    PubMed  Google Scholar 

  • *St. Pierre, T. L., & Kaltreider, D. L. (2001). Reflections on implementing a community agency-school prevention program. Journal of Community Psychology, 29, 107–116.

    Google Scholar 

  • *St. Pierre, Y. L., & Kaltreider, L. (2004). Tales of refusal, adoption, and maintenance: Evidence-based substance abuse prevention via school extension collaborations. American Journal of Evaluation, 25, 479–491.

    Google Scholar 

  • *Taggart, V. S., Bush, P. J., Zuckerman, A. E., & Theiss, P. K. (1990). A process evaluation of the District of Columbia “Know Your Body” project. Journal of School Health, 60, 60–66.

    PubMed  Google Scholar 

  • *Tappe, M. K., Galer-Unti, R. A., & Bailey, K. C. (1995). Long-term implementation of the teenage health teaching modules by trained teachers: A case study. Journal of School Health, 65, 411–415.

    Article  PubMed  Google Scholar 

  • *Telzrow, C. F., McNamara, K., & Hollinger, C. L. (2000). Fidelity of problem-solving implementation and relationship to student performance. School Psychology Review, 29, 443–461.

    Google Scholar 

  • Tobler, N. S. (1986). Meta-analysis of 143 adolescent drug prevention programs: Quantitative outcome results of program participants compared to a control or comparison group. Journal of Drug Issues, 16, 537–567.

    Google Scholar 

  • *Tortu, S., & Botvin, G. (1989). School-based smoking prevention: The teacher training process. Preventive Medicine, 18, 280–289.

    PubMed  Google Scholar 

  • *Vadasy, P. F., Jenkins, J. R., Antil, L. R., Phillips, N. B., & Pool, K. (1997). The research-to-practice ball game. Remedial and Special Education, 18, 143–156.

    Article  Google Scholar 

  • *Vincent, M. L., Paine-Andrews, A., Fisher, J., Devereaux, R. S., Gonyear Dolan, H., Harris, K. J., et al. (2000). Replication of a community-based multicomponent teen pregnancy prevention model: Realities and challenges. Family and Community Health, 23, 28–45.

    Google Scholar 

  • *Wall, M. A., Severson, H. H., Andrews, J. A., Lichtenstein, E., & Zoref, L. (1995). Pediatric office-based smoking intervention: Impact on maternal smoking and relapse. Pediatrics, 96, 622–628.

    PubMed  Google Scholar 

  • Wandersman, A. (2003). Community science: Bridging the gap between science and practice with community-centered models. American Journal of Community Psychology, 31, 227–242.

    PubMed  Google Scholar 

  • Wandersman, A., Duffy, J., Flaspohler, P., Noonan, R., Lubell, K., Stillman, L., et al. (2008). Bridging the gap between prevention research and practice: The interactive systems framwork for dissemination and implementation. American Journal of Community Psychology, 37, (this issue). doi:10.1007/s10464-008-9174-z

  • *Wandersman, A., Morissey, E., Davino, K., Seybolt, D., Crusto, C., Nation, M., et al. (1998). Comprehensive quality programming and accountability: Eight essential strategies for implementing successful prevention programs. The Journal of Primary Prevention, 19, 3–30.

    Google Scholar 

  • Weisman, S. A., Womer, S. C., Kellstrom, M., Bryner, S., Kahler, A., Slocum, L. A., et al. (2003). Maryland after school grant program part 1: Report on the 2001–2002 school year evaluation of the phase 3 after school programs. Unpublished manuscript, University of Maryland, College Park.

  • *Wiecha, J. L., El Ayadi, A. M., Fuemmeler, B. F., Carter, J. E., Handler, S., Johnson, S., et al. (2004). Diffusion of an integrated health education program in an urban school system: Planet Health. Journal of Pediatric Psychology, 29, 467–474.

    PubMed  Google Scholar 

  • Wilson, S. J., Lipsey, M. W., & Derzon, J. H. (2003). The effects of school-based intervention programs on aggressive behavior: A meta-analysis. Journal of Consulting and Clinical Psychology, 71, 136–149.

    PubMed  Google Scholar 

  • *Winfree, L. T., Lynsky, D. P., & Maupin, J. R. (1999). Developing local police and federal law enforcement partnerships: G.R.E.A.T. as a case study of policy implementation. Criminal Justice Review, 24, 145–168.

    Google Scholar 

  • Wolff, T. (2001). A practitioner’s guide to successful coalitions. American Journal of Community Psychology, 29, 173–181.

    PubMed  Google Scholar 

Download references

Acknowledgement

Preparation of this paper was supported in part from a grant awarded to the first author by the William T. Grant Foundaton (Grant #2212).

Author information

Authors and Affiliations

  1. Department of Psychology, Loyola University Chicago, 6525 North Sheridan Road, Chicago, IL, 60626, USA

    Joseph A. Durlak & Emily P. DuPre

Authors
  1. Joseph A. Durlak
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Emily P. DuPre
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Joseph A. Durlak.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Durlak, J.A., DuPre, E.P. Implementation Matters: A Review of Research on the Influence of Implementation on Program Outcomes and the Factors Affecting Implementation. Am J Community Psychol 41, 327–350 (2008). https://doi.org/10.1007/s10464-008-9165-0

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10464-008-9165-0

Keywords

Search

Navigation