Implementation Matters: A Review of Research on the Influence of Implementation on Program Outcomes and the Factors Affecting Implementation

Original Paper

Abstract

The first purpose of this review was to assess the impact of implementation on program outcomes, and the second purpose was to identify factors affecting the implementation process. Results from over quantitative 500 studies offered strong empirical support to the conclusion that the level of implementation affects the outcomes obtained in promotion and prevention programs. Findings from 81 additional reports indicate there are at least 23 contextual factors that influence implementation. The implementation process is affected by variables related to communities, providers and innovations, and aspects of the prevention delivery system (i.e., organizational functioning) and the prevention support system (i.e., training and technical assistance). The collection of implementation data is an essential feature of program evaluations, and more information is needed on which and how various factors influence implementation in different community settings.

Keywords

Implementation Youth programs Prevention Health promotion 

References

  1. *Abbott, R. D., O’Donnell, J., Hawkins, J. D., Hill, K. G., Kosterman, R., & Catalano, R. F. (1998). Changing teaching practices to promote achievement and bonding to school. American Journal of Orthopsychiatry, 68, 542–552.PubMedGoogle Scholar
  2. *Aber, J. L., Jones, S. M., Brown, J. L., Chaudry, N., & Samples, F. (1998). Resolving conflict creatively: Evaluating the developmental effects of a school based violence prevention program in neighborhood and classroom context. Development and Psychopathology, 10, 187–213.PubMedGoogle Scholar
  3. *Allison, K. R., Silverman, G., & Dignam, C. (1990). Effects on students of teacher training in use of a drug education curriculum. Journal of Drug Education, 20, 31–46.PubMedGoogle Scholar
  4. Altschuld, J. W., Kumar, D. D., Smith, D. W., & Goodway, J. D. (1999). School-based educational innovations: Case illustrations of context sensitive evaluations. Family and Community Health, 22, 66–79.Google Scholar
  5. American Psychological Association. (2003). Guidelines on multicultural education, training, research, practice and organizational change for psychologists. American Psychologist, 58, 377–402.Google Scholar
  6. Ary, D. V., Biglan, A., Glasgow, R., Zoref, L., Black, C., Ochs, L., et al. (1990). The efficacy of social-influence prevention programs versus “standard care”: Are new initiatives needed? Journal of Behavioral Medicine, 13, 281–296.PubMedGoogle Scholar
  7. *August, G. J., Bloomquist, M. L., Lee, S. S., Realmuto, G. M., & Hektner, J. M. (2006). Can evidence-based prevention programs be sustained in community practice settings? The Early Risers; advanced-stage effectiveness trial. Prevention Science, 7, 151–165.PubMedGoogle Scholar
  8. *August, G. J., Egan, E. A., Realmuto, G. M., & Hektner, J. M. (2003a). Parceling component effects of a multifaceted prevention program for disruptive elementary school children. Journal of Abnormal Child Psychology, 31, 515–527.PubMedGoogle Scholar
  9. *August, G. J., Lee, S. S., Bloomquist, M. L., Realmuto, G. M., & Hektner, J. M. (2003b). Dissemination of an evidence-based preventive intervention for aggressive children living in diverse, urban neighborhoods. Prevention Science, 4, 271–286.PubMedGoogle Scholar
  10. *Backer, T. E. (2002). Finding the balance: Program fidelity and adaptation in substance abuse prevention: A state-of-the-art review. Rockville, MD: Center for Substance Abuse Prevention, Substance Abuse and Mental Health Services Administration.Google Scholar
  11. *Barr, J. E., Tubman, J. G., Montgomery, M. J., & Soza-Vento, R. M. (2002). Amenability and implementation in secondary school antitobacco programs. American Journal of Health Behavior, 26, 3–15.PubMedGoogle Scholar
  12. *Basch, C. E., Sliepcevich, E. M., Gold, R. S., Duncan, D. F., & Kolbe, L. J. (1985). Avoiding type III errors in health education program evaluations: A case study. Health Education Quarterly, 12, 315–331.PubMedGoogle Scholar
  13. *Basen-Engquist, K., O’Hara-Tompkins, N., Lovato, C. Y., Lewis, M. J., Parcel, G. S., & Gingiss, P. (1994). The effect of two types of teacher training on implementation of smart choices: A tobacco prevention curriculum. Journal of School Health, 64, 334–339.PubMedGoogle Scholar
  14. *Battistich, V., Schaps, E., Watson, M., Solomon, D., & Lewis, C. (2000). Effects of the child development project on students’ drug use and other problem behaviors. The Journal of Primary Prevention, 21, 75–99.Google Scholar
  15. *Battistich, V., Schaps, E., & Wilson, N. (2004). Effects of an elementary school intervention on students’ “connectedness” to school and social adjustment during middle school. Journal of Primary Prevention, 24, 243–262.Google Scholar
  16. *Bauman, L., Stein, R., & Ireys, H. (1991). Reinventing fidelity: The transfer of social technology among settings. American Journal of Community Psychology, 19, 619–639.PubMedGoogle Scholar
  17. *Bell, M. L., Kelley-Baker, T., Rider, R., & Ringwalt, C. (2005). Protecting You/Protecting me: Effects of an alcohol prevention and vehicle safety program on elementary students. Journal of School Health, 75, 171–177.PubMedGoogle Scholar
  18. *Bellg, A., Borrelli, B., Resnick, B., Hecht, J., Minicucci, D. S., Ory, M., et al. (2004). Enhancing treatment fidelity in health behavior change studies: Best practices and recommendations from the NIH behavior change consortium. Health Psychology, 23, 443–451.PubMedGoogle Scholar
  19. *Berman, P., & McLaughlin, M. W. (1976). Implementation of educational innovation. The Educational Forum, 40, 345–370.Google Scholar
  20. *Binford, V. M., & Newwell, J. M. (1991). Richmond, Virginia’s two decades of experience with Ira Gordon’s approach to parent education. The Elementary School Journal, 91, 233–237.Google Scholar
  21. *Blakely, C. H., Mayer, J. P., Gottschalk, R. G., Schmitt, N., Davidson, W. S., Roitman, D. B., et al. (1987). The fidelity-adaptation debate: Implications for the implementation of public sector social programs. American Journal of Community Psychology, 15, 253–268.Google Scholar
  22. *Bloom, H. S., Hill, C. J., & Riccio, J. A. (2003). Linking program implementation and effectiveness: Lessons from a pooled sample of welfare-to-work experiments. Journal of Policy Analysis and Management, 22, 551–575.Google Scholar
  23. Botvin, G. J. (2000). Preventing drug abuse in schools: Social and competence enhancement approaches targeting individual-level etiologic factors. Addictive Behaviors, 25, 887–897.PubMedGoogle Scholar
  24. *Botvin, G. J., Baker, E., Dusenbury, L., Botvin, E. M., & Diaz, T. (1995). Long-term follow-up results of a randomized drug abuse prevention trial in a white middle-class population. Journal of the American Medical Association, 273, 1106–1112.PubMedGoogle Scholar
  25. *Botvin, G. J., Baker, E., Dusenbury, L., Tortu, S., & Botvin, E. (1990). Preventing adolescent drug abuse through a multimodal cognitive-behavioral approach: Results of a 3 year study. Journal of Consulting and Clinical Psychology, 58, 437–446.PubMedGoogle Scholar
  26. *Botvin, G. J., Baker, E., Filazzola, A. D., & Botvin, E. M. (1990). A cognitive-behavioral approach to substance abuse prevention: 1 Year follow-up. Addictive Behaviors, 15, 47–63.PubMedGoogle Scholar
  27. *Botvin, G. J., Baker, E., Renick, N. L., Filazzola, A. D., & Botvin, E. M. (1994). A cognitive-behavioral approach to substance abuse prevention. Addictive Behaviors, 9, 137–147.Google Scholar
  28. *Botvin, G. J., Dusenbury, L., Baker, E., James-Ortiz, S., Botvin, E. M., & Kerner, J. (1992). Smoking prevention among urban minority youth: Assessing effects on outcome and mediating variables. Health Psychology, 11, 290–299.PubMedGoogle Scholar
  29. *Botvin, G. J., Dusenbury, L., Baker, E., James-Ortiz, S., & Kerner, J. (1989). A skills training approach to smoking prevention among Hispanic youth. Journal of Behavioral Medicine, 12, 279–296.PubMedGoogle Scholar
  30. *Bush, P. J., Zuckerman, A. E., Taggart, V. S., Theiss, P. K., Peleg, E. O., & Smith, S. A. (1989). Cardiovascular risk factor prevention in black school children: The “Know Your Body” evaluation project. Health Education Quarterly, 16, 215–227.PubMedGoogle Scholar
  31. Chinman, M., Hannah, G., Wandersman, A., Ebener, P. P., Hunter, S. B., Imm, P., et al. (2005). Developing a community science research agenda for building community capacity for effective preventive interventions. American Journal of Community Psychology, 35, 143–157.PubMedGoogle Scholar
  32. *Chinman, M., Imm, P., & Wandersman, A. (2004). Getting to outcomes 2004: Promoting accountability through methods and tools for planning, implementation, and evaluation (Technical Report). Santa Monica, CA: RAND Corporation.Google Scholar
  33. *Cho, H., Hallfors, D. D., & Sanchez, V. (2005). Evaluation of a high school peer group intervention for at-risk youth. Journal of Abnormal Child Psychology, 33, 363–374.PubMedGoogle Scholar
  34. *Conduct Problems Prevention Research Group. (1999). Initial impact of the Fast Track prevention trial for conduct problems: II. Classroom effects. Journal of Consulting and Clinical Psychology, 67, 648–657.Google Scholar
  35. Connell, D. B., Turner, R. R., & Mason, E. F. (1985). Summary of findings on the school health evaluation: Health promotion, effectiveness, implementation, and costs. Journal of School Health, 55, 316–321.PubMedGoogle Scholar
  36. *Cook, T. D., Habib, F. N., Phillips, M., Settersten, R. A., Shagle, S. C., & Degirmencioglu, S. M. (1999). Comer’s school development program in prince George’s county, Maryland: A theory-based evaluation. American Educational Research Journal, 36, 543–597.Google Scholar
  37. *Cook, T. D., Murphy, R. F., & Hunt, H.D. (2000). Comer’s school development program in Chicago: A theory-based evaluation. American Educational Research Journal, 37, 535–597.Google Scholar
  38. *Cooke, M. (2000). The dissemination of a smoking cessation program: Predictors of program awareness, adoption and maintenance. Health Promotion International, 15, 113–124.Google Scholar
  39. *Cullen, K. W., Baranowski, T., Baranowski, J., Herbert, D., deMoor, C., Hearn, M. D., et al. (1999). Influence of school organizational characteristics on the outcomes of a school health promotion program. Journal of School Health, 69, 376–380.PubMedGoogle Scholar
  40. *Cunningham, P. B., & Henggeler, S. W. (2001). Implementation of an empirically based drug and violence prevention and intervention program in public school settings. Journal of Consulting and Clinical Psychology, 30, 221–232.Google Scholar
  41. Dane, A. V., & Schneider, B. H. (1998). Program integrity in primary and early secondary prevention: Are implementation effects out of control. Clinical Psychology Review, 18, 23–45.PubMedGoogle Scholar
  42. *Derzon, J. H., Sale, E., Springer, J. F., & Brounstein, P. (2005). Estimating intervention effectiveness: Synthetic projection of field evaluation results. The Journal of Primary Prevention, 26, 321–343.PubMedGoogle Scholar
  43. Domitrovich, C. E., & Greenberg, M. T. (2000). The study of implementation: Current findings from effective programs that prevent mental disorders in school-aged children. Journal of Educational and Psychological Consultation, 11, 193–221.Google Scholar
  44. *Dubas, J. S., Lynch, K. B., Galano, J., Geller, S., & Hunt, D. (1998). Preliminary evaluation of a resiliency-based preschool substance abuse and violence prevention project. Journal of Drug Education, 28, 235–255.PubMedGoogle Scholar
  45. *DuBois, D. L., Holloway, B. E., Valentine, J. C., & Cooper, H. (2002). Effectiveness of mentoring programs for youth: A meta-analytic review. American Journal of Community Psychology, 30, 157–198.PubMedGoogle Scholar
  46. *Dufrene, B. A., Noell, G. H., Gilbertson, D. N., & Duhon, G. J. (2005). Monitoring implementation of reciprocal peer tutoring: Identifying and intervening with students who do not maintain accurate implementation. School Psychology Review, 34, 74–86.Google Scholar
  47. Durlak, J. A. (1997). Successful prevention programs for children and adolescents. New York: Plenum.Google Scholar
  48. Durlak, J. A. (1985). School-based prevention programs for children and adolescents. Thousand Oaks, CA: Sage.Google Scholar
  49. Durlak, J. A. (1998). Why program implementation is important. Journal of Prevention & Intervention in the Community, 17, 5–18.Google Scholar
  50. Dusenbury, L., Brannigan, R., Falco, F., & Hansen, W. B. (2003a). A review of research on fidelity of implementation: implications for drug abuse prevention in school settings. Health Education Research, 18, 237–256.PubMedGoogle Scholar
  51. *Dusenbury, L., Brannigan, R., Falco, M., & Lake, A. (2003b). An exploration of fidelity of implementation in drug abuse prevention among five professional groups. Journal of Alcohol and Drug Education, 47, 4–19.Google Scholar
  52. *Elder, J. P., Perry, C. L., Stone, E. J., Johnson, C. C., Yang, M., Edmundson, E. W., et al. (1996). Tobacco use measurement, prediction, and intervention in elementary schools in four states: The CATCH study. Preventive Medicine, 25, 489–494.Google Scholar
  53. *Elias, M. J., Gara, M., Ubriaco, M., Rothbaum, P. A., Clabby, J. F., & Schuyler, T. (1986). Impact of a preventive social problem solving intervention on children’s coping with middle-school stressors. American Journal of Community Psychology, 14, 259–275.PubMedGoogle Scholar
  54. *Elias, M. J., Zins, J. E., Graczyk, P. A., & Weissberg, R. P. (2003). Implementation, sustainability, and scaling up of social-emotional and academic innovations in public schools. School Psychology Review, 32, 303–319.Google Scholar
  55. *Engstrom, M., Jason, L. A., Townsend, S. M., Pokorny, S. B., & Curie, C. J. (2002). Community readiness for prevention: Applying stage theory to multi-community interventions. Journal of Prevention & Intervention in the Community, 24, 29–46.Google Scholar
  56. *Epstein, A. S. (1993). Training for quality: Improving early childhood programs systematic training. Monographs of the High/Scope Educational Research Foundation, No. 9. Ypsilanti, MI: The High/Scope Press.Google Scholar
  57. *Fagan, A. A., & Mihalic, S. (2003). Strategies for enhancing the adoption of school-based prevention programs: Lessons learned from the blueprints for violence prevention replications of the life skills training program. Journal of Community Psychology, 31, 235–253.Google Scholar
  58. Felner, R. D., Favazza, A., Shim, M., Brand, S., Gu, K., & Noonan, N. (2001). Whole school improvement and restructuring as prevention and promotion. Journal of School Psychology, 39, 177–202.Google Scholar
  59. *Felner, R. D., Jackson, A., Kasak, D., Mulhall, P., Brand, S., & Flowers, N. (1997). The impact of middle school reform for the middle years: Longitudinal study of a network engaged in Turning Points-based comprehensive school transformation. Phi Delta Kappan, 78, 528–532.Google Scholar
  60. Fixsen, D. L., Naoom, S. F., Blasé, K. A., Friedman, R. M., & Wallace, F. (2005). Implementation research: A synthesis of the literature. Tampa, FL: University of South Florida, Louis de la Parte Florida Mental Health Institute, The National Implementation Research Network (FMHI Publication #231). Retrieved November 1, 2006, from http://nirn.fmhi.usf.edu/resources/publications/Monograph/pdf/monograph_full.pdf.
  61. *Forgatch, M. S., Patterson, G. R., & DeGarmo, D. S. (2005). Evaluating fidelity: Predictive validity for a measure of component adherence to the Oregon model of parent management training. Behavior Therapy, 36, 3–13.PubMedGoogle Scholar
  62. *Fors, S. W., & Doster, M. E. (1985). Implications of results: Factors for success. Journal of School Health, 55, 332–334.PubMedGoogle Scholar
  63. *Fullan, M., & Pomfret, A. (1977). Research on curriculum and instruction implementation. Review of Educational Research, 47, 335–397.Google Scholar
  64. *Gerstenblith, S. A., Soule, D. A., Gottfredson, D. C., Lu, S. Kellstrom, M. A., Womer, S. C., et al. (2005). After-school programs, antisocial behavior, and positive youth development: An exploration of the relationship between program implementation and changes in youth behavior. In J. L. Mahoney, J. S. Eccles & R. W. Larson (Eds.), Organized activities as contexts of development: Extracurricular activities, after-school and community programs. Mahwah, NJ: Erlbaum.Google Scholar
  65. *Gingiss, P. L. (1992). Enhancing program implementation and maintenance through a multi-phase approach to peer-based staff development. Journal of School Health, 62, 161–166.PubMedGoogle Scholar
  66. *Gingiss, P. L., Gottlieb, N. H., & Brink, S. G. (1994). Increasing teacher receptivity toward use of tobacco prevention education programs. Journal of Drug Education, 24, 163–176.PubMedGoogle Scholar
  67. *Gingiss, P. M., Roberts-Gray, C., & Boerm, M. (2006). Bridge-It: A system for predicting implementation fidelity for school-based tobacco prevention programs. Prevention Science, 7, 197–207.PubMedGoogle Scholar
  68. *Goldman, K. D. (1994). Perceptions of innovations as predictors of implementation levels: The diffusion of a nationwide health education campaign. Health Education Quarterly, 21, 433–443.PubMedGoogle Scholar
  69. *Gottfredson, D. C., & Gottfredson, G. D. (2002). Quality of school-based prevention programs: Results from a national survey. Journal of Research in Crime and Delinquency, 39, 3–35.Google Scholar
  70. *Gottfredson, D. C., Gottfredson, G. D., & Hybl, L. G. (1993). Managing adolescent behavior: A multiyear, multischool study. American Educational Research Journal, 30, 179–215.Google Scholar
  71. Granner, M. L., & Sharpe, P. A. (2004). Evaluating community coalition characteristics and functioning: A summary of measurement tools. Health Education Research, 19, 514–532.PubMedGoogle Scholar
  72. Greenhalgh, T., Robert, G., Macfarlane, F., Bate, P., Kyriakidou, O., & Peacock, R. (2005). Diffusion of innovations in health service organizations: A systematic literature review. Oxford: Blackwell.Google Scholar
  73. *Greenwood, C. R., Dinwiddie, G., Bailey, V., Carta, J. J., Dorsey, D., Kohler, F. W., et al. (1987). Field replication of classwide peer tutoring. Journal of Applied Behavior Analysis, 20, 151–160.PubMedGoogle Scholar
  74. *Greenwood, C. R., Tapia, Y., Abbott, M., & Walton, C. (2003). A building-based case study of evidence-based literacy practices: Implementation, reading behavior, and growth in reading fluency, K-4. The Journal of Special Education, 37, 95–110.Google Scholar
  75. *Greenwood, C. R., Terry, B., Arreaga-Mayer, C., & Finney, R. (1992). The classwide peer tutoring program: Implementation factors moderating students’ achievement. Journal of Applied Behavior Analysis, 25, 101–116.PubMedGoogle Scholar
  76. *Hahn, E. J., Noland, M. P., Rayens, M. K., & Christie, D. M. (2002). Efficacy of training and fidelity of implementation of the life skills training program. Journal of School Health, 72, 282–287.PubMedGoogle Scholar
  77. *Hansen, W. B., Graham, J. W., Wolkenstein, B. H., & Rohrbach, L. A. (1991). Program integrity as a moderator of prevention program effectiveness: Results for fifth grade students in the adolescent alcohol prevention trial. Journal of Studies on Alcohol, 52, 568–579.PubMedGoogle Scholar
  78. *Hansen, W. B., & McNeal, R. B. (1999). Drug education practice: Results of an observational study. Health Education Research, 14, 85–97.PubMedGoogle Scholar
  79. *Harachi, T. W., Abbott, R. D., Catalano, R. F., Haggerty, K. P., & Fleming, C. B. (1999). Opening the black box: Using process evaluation measures to assess implementation and theory building. American Journal of Community Psychology, 27, 711–731.PubMedGoogle Scholar
  80. *Henderson, J. L., MacKay, S., & Peterson-Badali, M. (2006). Closing the research-practice gap: Factors affecting adoption and implementation of a children’s mental health program. Journal of Clinical Child and Adolescent Psychology, 35, 2–12.PubMedGoogle Scholar
  81. *Hogan, J. A., Baca, I., Daley, C., Garcia, T., Jaker, J., Lowther, M., et al. (2003). Disseminating science-based prevention: Lessons learned from CSAP’s CAPTs. Journal of Drug Education, 33, 233–244.PubMedGoogle Scholar
  82. Hogue, A., Liddle, H. A., Singer, A., & Leckrone, J. (2005). Intervention fidelity in family-based prevention counseling for adolescent problem behaviors. Journal of Community Psychology, 33, 191–211.Google Scholar
  83. *Hopkins, R. H., Mauss, A. L., Kearney, K. A., & Weisheit, R. A. (1988). Comprehensive evaluation of a model alcohol education curriculum. Journal of Studies on Alcohol, 49, 38–50.PubMedGoogle Scholar
  84. *Hopper, C. A., Munoz, K. D., Gruber, M. B., MacConnie, S., Schonfeldt, B., & Shunk, T. (1996). A school-based cardiovascular exercise and nutrition program with parent participation: An evaluation study. Children’s Health Care, 25, 221–235.Google Scholar
  85. *Ialongo, N., Werthamer, L., Kellam, S. G., Brown, C. H., Wang, S., & Lin, Y. (1999). Proximal impact of two first-grade preventive interventions on the early risk behaviors for later substance abuse, depression, and antisocial behavior. American Journal of Community Psychology, 27, 599–641.PubMedGoogle Scholar
  86. *James, S., Reddy, P., Ruiter, A. M., & van den Borne, B. (2006). The impact of an HIV and AIDS life skills program on secondary school students in Kwazulu-Natal, South Africa. AIDS Education and Prevention, 18, 281–294. Google Scholar
  87. *Jason, L. A., Pokorny, S. B., Kunz, C., & Adams, M. (2004). Maintenance of community change: Enforcing youth access to tobacco laws. Journal of Drug Education, 34, 105–119.PubMedGoogle Scholar
  88. *Kalafat, J., & Ryerson, D. M. (1999). The implementation and institutionalization of a school-based youth suicide prevention program. Journal of Primary Prevention, 19, 157–175.Google Scholar
  89. *Kallestad, J. H., & Olweus, D. (2003). Predicting teachers’ and schools’ implementation of the Olweus bullying prevention program: A multilevel study. Prevention & Treatment, 6. Retrieved June 20, 2005.Google Scholar
  90. *Kam, C.-M., Greenberg, M. T., & Walls, C. T. (2003). Examining the role of implementation quality in school-based prevention using the PATHS curriculum. Prevention Science, 4, 55–63.PubMedGoogle Scholar
  91. Kazdin, A. E., Bass, D., Ayers, W. A., & Rodgers, A. (1990). Empirical and clinical focus of child and adolescent psychotherapy research. Journal of Consulting and Clinical Psychology, 58, 729–740.PubMedGoogle Scholar
  92. *Kealey, K. A., Peterson, A. V., Gaul, M. A., & Dinh, K. T. (2000). Teacher training as a behavior change process: Principles and results from a longitudinal study. Health Education & Behavior, 27, 64–81.Google Scholar
  93. *Kegler, M. C., Steckler, A., Malek, S. H., & McLeroy, K. (1998a). A multiple case study of implementation in 10 local project ASSIST coalitions in North Carolina. Health Education Research, 13, 225–238.PubMedGoogle Scholar
  94. *Kegler, M. C., Steckler, A., McLeroy, K., & Malek, S. H. (1998b). Factors that contribute to effective community health promotion coalitions: A study of 10 project ASSIST coalitions in North Carolina. Health Education & Behavior, 25, 338–353.Google Scholar
  95. *Kegler, M. C., & Wyatt, V. H. (2003). A multiple case study of neighborhood partnerships for positive youth development. American Journal of Health Behavior, 27, 156–169.PubMedGoogle Scholar
  96. *Kelly, J. A., Somlai, A. M., DiFranceisco, W. J., Otto-Salaj, L. L., McAuliffe, T. L., Hackl, K. L., et al. (2000). Bridging the gap between the science and service of HIV prevention: Transferring effective research-based HIV prevention interventions to community AIDS service providers. American Journal of Public Health, 90, 1082–1088.PubMedGoogle Scholar
  97. Kendrick, J. S., Zahniser, C., Miller, N., Salas, N., Stine, J. Gargiuillo, P. M., et al. (1995). Integrating smoking cessation into routine public prenatal care: The smoking cessation pregnancy project. American Journal of Public Health, 85, 217–222.PubMedGoogle Scholar
  98. *Kerr, D. M., Kent, L., & Lam, T. C. M. (1985). Measuring program implementation with a classroom observation instrument: The interactive teaching map. Evaluation Review, 9, 461–482.Google Scholar
  99. *Komro, K. A., Perry, C. L., Veblen-Mortenson, S., Bosma, L. M., Dudovitz, B. S., Williams, C. L., et al. (2004). Brief report: The adaptation of Project Northland for urban youth. Journal of Pediatric Psychology, 29, 457–466.PubMedGoogle Scholar
  100. *Komro, K. A., Perry, C. L., Veblen-Mortenson, S., Farbakhsh, K., Kugler, K. C., Alfano, K. A., et al. (2006). Cross-cultural adaptation and evaluation of a home-based program for alcohol use prevention among urban youth: The “Slick Tracy Home Team Program”. The Journal of Primary Prevention, 27, 135–154.PubMedGoogle Scholar
  101. *Kramer, L., Laumann, G., & Brunson, L. (2000). Implementation and diffusion of the rainbows program in rural communities: Implications for school-based prevention programming. Journal of Educational and Psychological Consultation, 11, 37–64.Google Scholar
  102. *Lapan, R. T., Gysbers, N. C., & Petroski, G. F. (2001). Helping seventh graders be safe and successful: A statewide study of the impact of comprehensive guidance and counseling programs. Journal of Consulting & Development, 79, 320–330.Google Scholar
  103. *Levenson-Gingiss, P., Gottlieb, N. H., & Brink, S. G. (1994). Increasing teacher receptivity toward use of tobacco prevention education programs. Journal of Drug Education, 24, 163–176.Google Scholar
  104. *Lillehoj, C. J. G., Griffin, K. W., & Spoth, R. (2004). Program provider and observer ratings of school-based preventive intervention implementation: Agreement and relation to youth outcomes. Health Education & Behavior, 31, 242–257.Google Scholar
  105. *Lynch, K. B., Geller, S. R., Hunt, D. R., Galano, J., & Dubas, J. S. (1998). Successful program develoment using implementation evaluation. Journal of Prevention & Intervention in the Community, 17, 51–64.Google Scholar
  106. *MacDonald, M. A., & Green, L. W. (2001). Reconciling concept and context: The dilemma of implementation in school-based health promotion. Health Education & Behavior, 28, 749–768.Google Scholar
  107. *McCormick, L. K., Steckler, A., & McLeroy, K. R. (1994). Diffusion of innovation in schools: A study of adoption and implementation of school-based tobacco prevention curricula. American Journal of Health Promotion, 9, 210–219.Google Scholar
  108. *McGraw, S., Sellers, D., Stone, E., Bebchuk, J., Edmundson, E., Johnson, C., et al. (1996). Using process data to explain outcomes: An illustration from the child and adolescent trial for cardiovascular health (CATCH). Evaluation Review, 20, 291–312.PubMedGoogle Scholar
  109. *McKenzie, T. L., Nader, P. R., Strikmiller, P. K., Yang, M., Stone, E. J., erry, C. L, et al. (1996). School physical education: Effect of the child and adolescent trial for cardiovascular health. Preventive Medicine, 25, 423–431.PubMedGoogle Scholar
  110. *Mihalic, S., Irwin, K., Fagan, A., Ballard, D., & Elliott, D. (2004). Successful program implementation: Lessons from blueprints. Electronic report. U.S. Department of Justice, Office of Justice Programs). Retrieved August 10, 2006, from http://www.ojp.usdoj.gov/ojjdp.
  111. Miranda, J., Bernal, G., Lau, A., Kohn, L., Hwangh, W., & LaFromboise, T. (2005). State of the science on psychosocial interventions for ethnic minorities. Annual Review of Clinical Psychology, 1, 113–142.PubMedGoogle Scholar
  112. Mitchell, C. M. (1983). The dissemination of a social intervention: Process and effectiveness of two types of paraprofessional change agents. American Journal of Community Psychology, 11, 723–738.PubMedGoogle Scholar
  113. *Moskowitz, J. M., Schaps, E., & Malvin, J. H. (1982). Process and outcome evaluations in primary prevention: The magic circle program. Evaluation Review, 6, 775–788.Google Scholar
  114. Mowbray, C. T., Holter, M. C., Teague, G. B., & Bybee, D. (2003). Fidelity criteria: Developmental, measurement, and validation. American Journal of Evaluation, 24, 315–340.Google Scholar
  115. Nastasi, B. K., & Schensul, S. L. (2005). Contributions of qualitative research to the validity of intervention research. Journal of School Psychology, 43, 177–195.Google Scholar
  116. National Research Council, Institute of Medicine. (2002). Community programs to promote youth development. Washington, DC: National Academy Press.Google Scholar
  117. *Noell, G. H., Duhon, G. J., Gatti, S. L., & Connell, J. E. (2002). Consultation, follow-up, and implementation of behavior management interventions in general education. School Psychology Review, 31, 217–234.Google Scholar
  118. *Noell, G. H., & Witt, J. C. (1999). When does consultation lead to intervention implementation? The Journal of Special Education, 33, 29–35.Google Scholar
  119. *Noell, G. H., Witt, J. C., Gilbertson, D. N., Ranier, D. D., & Freeland, J. T. (1997). Increasing teacher intervention implementation in general education settings through consultation and performance feedback. School Psychology Quarterly, 12, 77–88.Google Scholar
  120. *Noell, G. H., Witt, J. C., Slider, N. J., Connell, J. E., Gatti, S. L., Williams, K. L. et al. (2005). Treatment implementation following behavioral consultation in schools: A comparison of three follow-up strategies. School Psychology Review, 34, 87–106.Google Scholar
  121. *Parcel, G. S., Ross, J. G., Lavin, A. T., Portnoy, B., Nelson, G. D., & Winters, F. (1991). Enhancing implementation of the teenage health teaching modules. Journal of School Health, 61, 35–38.PubMedGoogle Scholar
  122. *Payne, A. A., Gottfredson, D. C., & Gottfredson, G. D. (2006). School predictors of the intensity of implementation of school-based prevention programs: Results from a national study. Prevention Science, 7, 225–237.PubMedGoogle Scholar
  123. *Pentz, M. A., Trebow, E. A., Hansen, W. B., MacKinnon, D. B., Dwyer, J. H., Johnson, C. A. et al. (1990). Effects of program implementation on adolescent drug use behavior: The Midwestern Prevention Project (MPP). Evaluation Review, 14, 264–289.Google Scholar
  124. *Perry, C. L., Murray, D. M., & Griffin, G. (1990). Evaluating the statewide dissemination of smoking prevention curricula: Factors in teacher compliance. Journal of School Health, 60, 501–504.PubMedGoogle Scholar
  125. *Perry, C. L., Sellers, D. E., Johnson, C., Pedersen, S., Bachman, K. J., Parcel, G. S., et al. (1997). The Child and Adolescent Trial for Cardiovascular Health (CATCH): Intervention, implementation, and feasibility for elementary schools in the United States. Health Education and Behavior, 24, 717–735.Google Scholar
  126. *Piper, D. L., King, M. J., & Moberg, D. P. (1993). Implementing a middle school health promotion research project: Lessons our textbook didn’t teach us. Evolution & Program Planning, 16, 171–180.Google Scholar
  127. *Pluye, P., Potvin, L., Denis, J. L., Pelletier, J., & Mannoni, C. (2005). Program sustainability begins with the first events. Evaluation and Program Planning, 28, 123–137.Google Scholar
  128. *Rankin, W., Tarnai, J., Fagan, N. J., Mauss, A. L., & Hopkins, R. H. (1978). An evaluation of workshops designed to prepare teachers in alcohol education. Journal of Alcohol & Drug Education, 23, 1–13.Google Scholar
  129. Resnicow, K., Baranowski, T., Ahluwalia, J. S., & Braithwaite, R. L. (1999). Cultural sensitivity in public health: Defined and demystified. Ethnicity and Disease, 9, 10–21.PubMedGoogle Scholar
  130. *Resnicow, K., Cohn, L., Reinhardt, J., Cross, D., Futterman, R., Kirschner, E., et al. (1992). A 3-year evaluation of the Know Your Body program in inner-city schoolchildren. Health Education Quarterly, 19, 463–480.PubMedGoogle Scholar
  131. *Resnicow, K., Davis, M., Smith, M., Baranowski, T., Lin, L. S., Baranowski, J., et al. (1998a). Results of the TeachWell worksite wellness program. American Journal of Public Health, 88, 250–257.PubMedGoogle Scholar
  132. *Resnicow, K., Davis, M., Smith, M., Lazarus-Yaroch, A., Baranowski, T., Baranowski, J., et al. (1998b). How best to measure implementation of school health curricula: A comparison of three measures. Health Education Research, 13, 239–250.PubMedGoogle Scholar
  133. *Reynolds, K. D., Franklin, F. A., Leviton, L. C., Maloy, J., Harrington, K. F., Yaroch, A. L., et al. (2000). Methods, results, and lessons learned from process evaluation of the High 5 school-based nutrition intervention. Health Education & Behavior, 27, 177–186.Google Scholar
  134. *Richard, L., Lehoux, P., Breton, E., Denis, J., Labrie, L., & Leonard, C. (2004). Implementing the ecological approach in tobacco control programs: Results of a case study. Evaluation and Program Planning, 27, 409–421.Google Scholar
  135. *Riley, B. L. (2003). Dissemination of heart health promotion in the Ontario public health system: 1989–1999. Health Education Research, 18, 15–31.PubMedGoogle Scholar
  136. *Riley, B. L., Taylor, S. M., & Elliott, S. J. (2001). Determinants of implementing heart healthy promotion activities in Ontario public health units: A social ecological perspective. Health Education Research, 16, 425–441.PubMedGoogle Scholar
  137. *Riley, B. L., Taylor, S. M., & Elliott, S. J. (2003). Organizational capacity and implementation change: A comparative case study of heart health promotion in Ontario public health agencies. Health Education Research, 18, 754–769.PubMedGoogle Scholar
  138. *Ringwalt, C. L., Ennett, S., Johnson, R., Rohrbach, L. A., Simons-Rudolph, A., Vincus, A., et al. (2003). Factors associated with fidelity to substance use prevention curriculum guides in the nation’s middle schools. Health Education & Behavior, 30, 375–391.Google Scholar
  139. *Rogers, E. M. (2003). Diffusion of innovations (5th ed.). New York: Free Press.Google Scholar
  140. *Rohrbach, L. A., Graham, J. W., & Hansen, W. B. (1993). Diffusion of a school-based substance abuse prevention program: Predictors of program implementation. Preventive Medicine, 22, 237–260.PubMedGoogle Scholar
  141. *Ross, J. G., Kuepker, R. V., Nelson, G. D., Saavedra, P., & Hubbard, B. M. (1991). Teenage health teaching modules: Impact of teacher training on implementation and student outcomes. Journal of School Health, 61, 31–34.PubMedGoogle Scholar
  142. Saunders, R. P., Ward, D., Felton, G. M., Dowda, M., & Pate, R. R. (2006). Examining the link between program implementation and behavior outcomes in the lifestyle education for activity program (LEAP). Evaluation and Program Planning, 29, 352–364.Google Scholar
  143. *Scheirer, M. A. (1990). The life cycle of an innovation: Adoption versus discontinuation of the fluoride mouth rinse program in schools. Journal of Health and Social Behavior, 31, 203–215.PubMedGoogle Scholar
  144. *Scheirer, M. A. (2005). Is sustainability possible: A review and commentary on empirical studies of program sustainability. American Journal of Evaluation, 26, 320–347.Google Scholar
  145. *Scheirer, M. A., Shediac, M. C., & Cassady, C. E. (1995). Measuring the implementation of health promotion programs: The case of the Breast and Cervical Cancer Program in Maryland. Health Education Research, 10, 11–25.PubMedGoogle Scholar
  146. Shediac-Rizkallah, M. C., & Bone, L. R. (1998). Planning for the sustainability of community-based health programs: Conceptual frameworks and future directions for research, practice and policy. Health Education Research, 13, 87–108.PubMedGoogle Scholar
  147. *Smith, D. W., McCormick, L. K., Steckler, A. B., & McLeroy, K. R. (1993). Teachers’ use of health curricula: Implementation of growing healthy, project SMART, and the teenage health teaching modules. Journal of School Health, 63, 349–354.PubMedGoogle Scholar
  148. *Smith, D. W., Redican, K. J., & Olsen, L. K. (1992). The longevity of growing healthy: An analysis of the eight original sites implementing the school health curriculum project. Journal of School Health, 62, 83–87.PubMedGoogle Scholar
  149. *Smith, J. D., Schneider, B. H., Smith, P. K., & Ananiadou, K. (2004). The effectiveness of whole-school antibullying programs: A synthesis of evaluation research. School Psychology Review, 33, 547–560.Google Scholar
  150. *Solomon, D., Battistich, V., Watson, M., Schaps, E., & Lewis, C. (2000). A six-district study of educational change: Direct and mediated effects of the Child Development Project. Social Psychology Education, 4, 3–51.Google Scholar
  151. *Spoth, R., Guyll, M., Trudeau, L., & Goldberg-Lillehoj, C. (2002). Two studies of proximal outcomes and implementation quality of universal preventive interventions in a community-university collaboration context. Journal of Community Psychology, 30, 499–518.Google Scholar
  152. *Steckler, A., McLeroy, K. R., Goodman, R. M., Smith, D., Dawson, L., & Howell, K. (1989). The importance of school district policies in the dissemination of tobacco use curricula in North Carolina schools. Family & Community Health, 12, 14–25.Google Scholar
  153. *Sterling-Turner, H. E., Watson, T. S., & Moore, J. W. (2002). The effects of direct training and treatment integrity on treatment outcomes in school consultation. School Psychology Quarterly, 17, 47–77.Google Scholar
  154. *Stevens, V., Van Oost, P., & De Bourdeaudhuij, I. (2001). Implementation process of the Flemish antibullying intervention and relation with program effectiveness. Journal of School Psychology, 39, 303–317.Google Scholar
  155. Stith, S., Pruitt, I., Dees, J., Fronce, M., Green, N., Som, A. et al. (2006). Implementing community-based prevention programming: A review of the literature. Journal of Primary Prevention, 27, 599–617.PubMedGoogle Scholar
  156. *Story, M., Mays, R. W., Bishop, D. B., Perry, C. L., Taylor, G., Smyth, M., et al. (2000). 5-a-day power plus: Process evaluation of a multicomponent elementary school program to increase fruit and vegetable consumption. Health Education & Behavior, 27, 187–200.Google Scholar
  157. *Stotts, A. L., diClemente, C. C., & Dolan-Mullen, P. (2002). One-to-one: A motivational intervention for resistant pregnant smokers. Addictive Behaviors, 27, 275–292.PubMedGoogle Scholar
  158. *St. Pierre, T. L., & Kaltreider, D. L. (2001). Reflections on implementing a community agency-school prevention program. Journal of Community Psychology, 29, 107–116.Google Scholar
  159. *St. Pierre, Y. L., & Kaltreider, L. (2004). Tales of refusal, adoption, and maintenance: Evidence-based substance abuse prevention via school extension collaborations. American Journal of Evaluation, 25, 479–491.Google Scholar
  160. *Taggart, V. S., Bush, P. J., Zuckerman, A. E., & Theiss, P. K. (1990). A process evaluation of the District of Columbia “Know Your Body” project. Journal of School Health, 60, 60–66.PubMedGoogle Scholar
  161. *Tappe, M. K., Galer-Unti, R. A., & Bailey, K. C. (1995). Long-term implementation of the teenage health teaching modules by trained teachers: A case study. Journal of School Health, 65, 411–415.PubMedCrossRefGoogle Scholar
  162. *Telzrow, C. F., McNamara, K., & Hollinger, C. L. (2000). Fidelity of problem-solving implementation and relationship to student performance. School Psychology Review, 29, 443–461.Google Scholar
  163. Tobler, N. S. (1986). Meta-analysis of 143 adolescent drug prevention programs: Quantitative outcome results of program participants compared to a control or comparison group. Journal of Drug Issues, 16, 537–567.Google Scholar
  164. *Tortu, S., & Botvin, G. (1989). School-based smoking prevention: The teacher training process. Preventive Medicine, 18, 280–289.PubMedGoogle Scholar
  165. *Vadasy, P. F., Jenkins, J. R., Antil, L. R., Phillips, N. B., & Pool, K. (1997). The research-to-practice ball game. Remedial and Special Education, 18, 143–156.CrossRefGoogle Scholar
  166. *Vincent, M. L., Paine-Andrews, A., Fisher, J., Devereaux, R. S., Gonyear Dolan, H., Harris, K. J., et al. (2000). Replication of a community-based multicomponent teen pregnancy prevention model: Realities and challenges. Family and Community Health, 23, 28–45.Google Scholar
  167. *Wall, M. A., Severson, H. H., Andrews, J. A., Lichtenstein, E., & Zoref, L. (1995). Pediatric office-based smoking intervention: Impact on maternal smoking and relapse. Pediatrics, 96, 622–628.PubMedGoogle Scholar
  168. Wandersman, A. (2003). Community science: Bridging the gap between science and practice with community-centered models. American Journal of Community Psychology, 31, 227–242.PubMedGoogle Scholar
  169. Wandersman, A., Duffy, J., Flaspohler, P., Noonan, R., Lubell, K., Stillman, L., et al. (2008). Bridging the gap between prevention research and practice: The interactive systems framwork for dissemination and implementation. American Journal of Community Psychology, 37, (this issue). doi:10.1007/s10464-008-9174-z
  170. *Wandersman, A., Morissey, E., Davino, K., Seybolt, D., Crusto, C., Nation, M., et al. (1998). Comprehensive quality programming and accountability: Eight essential strategies for implementing successful prevention programs. The Journal of Primary Prevention, 19, 3–30.Google Scholar
  171. Weisman, S. A., Womer, S. C., Kellstrom, M., Bryner, S., Kahler, A., Slocum, L. A., et al. (2003). Maryland after school grant program part 1: Report on the 2001–2002 school year evaluation of the phase 3 after school programs. Unpublished manuscript, University of Maryland, College Park.Google Scholar
  172. *Wiecha, J. L., El Ayadi, A. M., Fuemmeler, B. F., Carter, J. E., Handler, S., Johnson, S., et al. (2004). Diffusion of an integrated health education program in an urban school system: Planet Health. Journal of Pediatric Psychology, 29, 467–474.PubMedGoogle Scholar
  173. Wilson, S. J., Lipsey, M. W., & Derzon, J. H. (2003). The effects of school-based intervention programs on aggressive behavior: A meta-analysis. Journal of Consulting and Clinical Psychology, 71, 136–149.PubMedGoogle Scholar
  174. *Winfree, L. T., Lynsky, D. P., & Maupin, J. R. (1999). Developing local police and federal law enforcement partnerships: G.R.E.A.T. as a case study of policy implementation. Criminal Justice Review, 24, 145–168.Google Scholar
  175. Wolff, T. (2001). A practitioner’s guide to successful coalitions. American Journal of Community Psychology, 29, 173–181.PubMedGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2008

Authors and Affiliations

  1. 1.Department of Psychology Loyola University ChicagoChicagoUSA

Personalised recommendations