Abstract
This paper describes the development, application, and results of an implementation monitoring component of the Communities That Care (CTC) prevention framework used in the Community Youth Development Study (CYDS) to ensure high-fidelity prevention program implementation. This system was created based on research that community-based implementation of evidence-based prevention programs often includes adaptations in program design, content, or manner of delivery (Gottfredson and Gottfredson, Journal of research in crime and delinquency, 39, 3–35, 2002; Hallfors and Godette, Health Education Research, 17, 461–470, 2002; Wandersman and Florin, American Psychologist, 58, 441–448, 2003). A lack of fidelity to the implementation standards delineated by program designers is one indicator of a gap between prevention science and practice which can lessen the likelihood that communities will realize the positive participant effects demonstrated in research trials. By using the CTC model to select and monitor the quality of prevention activities, the 12 CYDS communities replicated 13 prevention programs with high rates of adherence to the programs’ core components and in accordance with dosage requirements regarding the number, length, and frequency of sessions. This success indicates the potential of the CTC program implementation monitoring system to enhance community Prevention Delivery Systems (Wandersman et al. American Journal of Community Psychology, this issue) and improve the likelihood of desired participant changes.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
Dichotomous (yes/no) ratings were used for all programs except Program Development Evaluation (PDE), which assessed adherence using a 3-point rating scale (not met, partially met, or fully met).
Participant attendance was also recorded at every session, providing verification of dosage information.
The five programs not observed included a self-administered program (Family Matters), a schoolwide intervention (PDE), and three programs [tutoring, Valued Youth, and Big Brothers/Big Sisters (BBBS)] for which observations would have been overly intrusive given the one-on-one or small-group administration.
Three programs [Parents Who Care® (PWC), Participate and Learn Skills (PALS), and Tutoring] did not provide developer-led training workshops. In these cases, program coordinators met with implementers to review program principles and procedures.
This definition refers to the amount of dosage offered during implementation; that is, the amount of the curriculum delivered to individuals, not individual dosage (i.e., the amount of material received by participants).
We did not penalize program cycles in which the number of sessions or session length exceeded the requirements. These deviations usually occurred in school-based curricula, as teachers often found that two class periods were needed to cover all the required information, and in parent-training programs, when sessions were lengthened to provide meals or allow more social interaction.
A dosage score for the PDE program was not calculated, as the program does not specify any of the three dosage elements.
Student attendance records were available for six of the nine cycles of school-based programs.
However, overall effects of the CTC intervention are assessed using student-reported levels of risk and protective factors and involvement in substance use and delinquency, from students in intervention communities compared to control communities.
References
Abbott, R. D., O’Donnell, J., Hawkins, J. D., Hill, K. G., Kosterman, R., & Catalano, R. F. (1998). Changing teaching practices to promote achievement and bonding to school. American Journal of Orthopsychiatry, 68, 542–552.
Bauman, K. E., Ennett, S. T., Foshee, V. A., Pemberton, M., & Hicks, K. (2001). Correlates of participation in a family-directed tobacco and alcohol prevention program for adolescents. Health Education and Behavior, 28, 440–461.
Bodisch Lynch, K., Geller, S. R., Hunt, D. R., Galano, J., & Semon Dubas, J. (1998). Successful program development using implementation evaluation. Journal of Prevention and Intervention in the Community, 17, 51–64.
Botvin, G. J., Mihalic, S. F., & Grotpeter, J. K. (1998). Blueprints for Violence Prevention, Book 5: Life Skills Training. Boulder, CO: Center for the Study and Prevention of Violence, Institute of Behavioral Science, University of Colorado.
Center for Substance Abuse Prevention (2000). CSAP’s model programs. Available on the World Wide Web: http://nrepp.samhsa.gov/find.asp. Accessed 13 Feb 2007.
Chinman, M., Hannah, G., Wandersman, A., Ebener, P., Hunter, S. B., Imm, P., et al. (2005). Developing a community science research agenda for building community capacity for effective preventive interventions. American Journal of Community Psychology, 35, 143–157.
Dumas, J. E., Lynch, A. M., Laughlin, J. E., Smith, E. P., & Printz, R. J. (2001). Promoting intervention fidelity: Conceptual issues, methods, and preliminary results from the Early Alliance Prevention Trial. American Journal of Preventive Medicine, 20, 38–47.
Dumka, L. E., Garza, C. A., Roosa, M. W., & Stoerzinger, H. D. (1997). Recruitment and retention of high-risk families into a preventive parent training intervention. The Journal of Primary Prevention, 18, 25–39.
Durlak, J. A. (1998). Common risk and protective factors in successful prevention programs. American Journal of Orthopsychiatry, 68, 512–520.
Dusenbury, L., Brannigan, R., Falco, M., & Hansen, W. B. (2003). A review of research on fidelity of implementation: Implications for drug abuse prevention in school settings. Health Education Research, 18, 237–256.
Elliott, D. S. (1997). Blueprints for Violence Prevention. Boulder, CO: University of Colorado, Institute of Behavioral Science, Center for the Study and Prevention of Violence.
Elliott, D. S., & Mihalic, S. (2004). Issues in disseminating and replicating effective prevention programs. Prevention Science, 5, 47–53.
Ennett, S. T., Ringwalt, C., Thorne, J., Rohrbach, L. A., Vincus, A. A., Simons-Rudolph, A., et al. (2003). A comparison of current practice in school-based substance use prevention programs with meta-analysis findings. Prevention Science, 4, 1–14.
Fagan, A. A., & Mihalic, S. (2003). Strategies for enhancing the adoption of school-based prevention programs: Lessons learned from the Blueprints for Violence Prevention replications of the Life Skills Training Program. Journal of Community Psychology, 31, 235–254.
Fixsen, D. L., Naoom, S. F., Blase, K. A., Friedman, R. M., & Wallace, F. (2005). Implementation research: A synthesis of the literature. Tampa, FL: University of South Florida, Louis de la Parte Florida Mental Health Institute, The National Implementation Research Network (FMHI Publication #231).
Gottfredson, D. C., & Gottfredson, G. D. (2002). Quality of school-based prevention programs: Results from a national survey. Journal of Research in Crime and Delinquency, 39, 3–35.
Greenberg, M. T., Feinberg, M. E., Gomez, B. J., & Osgood, D. W. (2005). Testing a community prevention focused model of coalition functioning and sustainability: A comprehensive study of Communities That Care in Pennsylvania. In T. Stockwell, P. Gruenewald, J. Toumbourou, & W. Loxley (Eds.), Preventing harmful substance use: The evidence base for policy and practice (pp. 129–142). West Sussex, England: Wiley.
Hallfors, D., & Godette, D. (2002). Will the “Principles of Effectiveness” improve prevention practice? Early findings from a diffusion study. Health Education Research, 17, 461–470.
Hawkins, J. D., & Catalano, R. F. (1992). Communities That Care: Action for drug abuse prevention. San Francisco, CA: Jossey-Bass Publishers.
Hawkins, J. D., & Catalano, R. F. (2004). Communities That Care Prevention strategies guide. South Deerfield, MA: Channing Bete Company, Inc. http://preventionplatform.samhsa.gov
Hawkins, J. D., Catalano, R. F., & Arthur, M. W. (2002). Promoting science-based prevention in communities. Addictive Behaviors, 27, 951–976.
Heinrichs, N., Bertram, H., Kuschel, A., & Hahlweg, K. (2005). Parent recruitment and retention in a universal prevention program for child behavior and emotional problems: Barriers to research and program participation. Prevention Science, 6, 276–286.
Henggeler, S. W., Melton, G. B., Brondino, M. J., Scherer, D. G., & Hanley, J. H. (1997). Multisystemic therapy with violence and chronic juvenile offenders and their families: The role of treatment fidelity in successful dissemination. Journal of Consulting and Clinical Psychology, 65, 821–833.
Kam, C.-M., Greenberg, M. T., & Walls, C. T. (2003). Examining the role of implementation quality in school-based prevention using the PATHS curriculum. Prevention Science, 4, 55–63.
Kealey, K. A., Peterson, A. V. J., Gaul, M. A., & Dinh, K. T. (2000). Teacher training as a behavior change process: Principles and results from a longitudinal study. Health Education and Behavior, 27, 64–81.
Mihalic, S., Fagan, A. A., Irwin, K., Ballard, D., & Elliott, D. (2004). Blueprints for violence prevention. Washington, D.C.: Office of Juvenile Justice and Delinquency Prevention.
Mihalic, S., & Irwin, K. (2003). Blueprints for Violence Prevention: From research to real world settings—factors influencing the successful replication of model programs. Youth Violence and Juvenile Justice, 1, 307–329.
Mitchell, R., Florin, P., & Stevenson, J. F. (2002). Supporting community-based prevention and health promotion initiatives: Developing effective technical assistance systems. Health Education and Behavior, 29, 620–639.
Mowbray, C. T., Holter, M. C., Teague, G. B., & Bybee, D. (2003). Fidelity criteria: Development, measurement and validation. American Journal of Evaluation, 24, 315–340.
Olweus, D., Limber, S., & Mihalic, S. F. (1999). Blueprints for Violence Prevention, Book 9: Bullying Prevention program. Boulder, CO: Center for the Study and Prevention of Violence, Institute of Behavioral Science, University of Colorado.
Paulsell, D., Kisker, E. E., Love, J. M., & Raikes, H. H. (2002). Understanding implementation in early Head Start programs: Implications for policy and practice. Infant Mental Health Journal, 23, 14–35.
Quinby, R., Fagan, A. A., Hanson, K., Brooke-Weiss, B., Arthur, M. W., & Hawkins, J. D. (2008). Installing the Communities That Care prevention system: Implementation progress and fidelity in a randomized controlled trial. Journal of Community Psychology, 36(3), 1–20.
Rohrbach, L. A., Graham, J. W., & Hansen, W. B. (1993). Diffusion of a school-based substance abuse prevention program: predictors of program implementation. Preventive Medicine, 22, 237–260.
Roth, D., Panzano, P. C., Crane-Ross, D., Massatti, R., Carstens, C., Seffrin, B., et al. (2005). The innovation diffusion and adoption research project (IDARP): Moving from the diffusion of research results to promoting the adoption of evidence-based innovations in the Ohio mental health system. In D. Roth & W. J. Lutz (Eds.), New research in mental health (Vol. 16, pp. 78–89). Columbus, OH: Ohio Department of Mental Health.
Schoenwald, S. K., Sheidow, A. J., & Letourneau, E. J. (2004). Toward effective quality assurance in evidence-based practice: Links between expert consultation, therapist fidelity, and child outcomes. Journal of Clinical Child and Adolescent Psychology, 33, 94–104.
Sherman, L. W., Gottfredson, D. C., MacKenzie, D., Eck, J., Reuter, P., & Bushway, S. (Eds.) (1997). Preventing crime: What works, what doesn’t, what’s promising: A report to the United States Congress. Washington, D.C.: U.S. Department of Justice, Office of Justice Programs.
Spoth, R. L., Guyll, M., Trudeau, L., & Goldberg-Lillehoj, C. (2002). Two studies of proximal outcomes and implementation quality of universal preventive interventions in a community-university collaboration context. Journal of Community Psychology, 30, 499–518.
Spoth, R. L., & Redmond, C. (2002). Project Family prevention trials based in community–university partnerships: Toward scaled-up preventive interventions. Prevention Science, 3, 203–222.
Taggart, V. S., Bush, P. J., Zuckerman, A. E., & Theiss, P. K. (1990). A process evaluation of the District of Columbia “Know Your Body” project. Journal of School Health, 60, 60–66.
U.S. Department of Health and Human Services (2001). Youth violence: A report of the Surgeon General. Rockville, MD: U.S. Department of Health and Human Services, Centers for Disease Control and Prevention, National Center for Injury Prevention and Control; Substance Abuse and Mental Health Services Administration, Center for Mental Health Services; National Institutes of Health, National Institute of Mental Health.
Wandersman, A., Duffy, J., Flaspohler, P., Noonan, R., Lubell, K., Stillman, L., et al. (this issue). Bridging the gap between prevention research and practice: An interactive systems framework for building capacity to disseminate and implement innovations. American Journal of Community Psychology.
Wandersman, A., & Florin, P. (2003). Community intervention and effective prevention. American Psychologist, 58, 441–448.
Welsh, B., & Farrington, D. P. (Eds.) (2006). Preventing crime: What works for children, offenders, victims and places. Dordrecht, The Netherlands: Springer.
Acknowledgements
This work was supported by a research grant from the National Institute on Drug Abuse (R01 DA015183-01A1) with co-funding from the National Cancer Institute, the National Institute of Child Health and Development, the National Institute of Mental Health, and the Center for Substance Abuse Prevention. The authors wish to acknowledge the contributions of the communities participating in the Community Youth Development Study. An earlier version of this paper was presented in November 2005 at the American Society of Criminology conference held in Toronto, Ontario, Canada.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Fagan, A.A., Hanson, K., Hawkins, J.D. et al. Bridging Science to Practice: Achieving Prevention Program Implementation Fidelity in the Community Youth Development Study. Am J Community Psychol 41, 235–249 (2008). https://doi.org/10.1007/s10464-008-9176-x
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10464-008-9176-x