Skip to main content

Long-Term Sustainability of Evidence-Based Prevention Interventions and Community Coalitions Survival: a Five and One-Half Year Follow-up Study

Prevention Science volume 18, pages 610–621 (2017)Cite this article

Abstract

This study examines (1) coalition survival, (2) prevalence of evidence-based prevention interventions (EBPIs) to reduce substance abuse implemented as part of the Tennessee Strategic Prevention Framework (SPF) State Incentive Grant (SIG), (3) EBPI sustainability, and (4) factors that predict EBPI sustainability. Secondary data were collected on 27 SPF SIG-funded coalitions and 88 EBPI and non-EBPI implementations. Primary data were collected by a telephone interview/web survey five and one-half years after the SPF SIG ended. Results from secondary data show that 25 of the 27 coalitions survived beyond the SPF SIG for one to five and one-half years; 19 coalitions (70%) were still active five and one-half years later. Further, 88 EBPIs and non-EBPIs were implemented by 27 county SPF SIG coalitions. Twenty-one (21) of 27 coalitions (78%) implemented one to three EBPIs, totaling 37 EBPI implementations. Based on primary survey data on 29 of the 37 EBPI implementations, 28 EBPIs (97%) were sustained between two and five and one-half years while 22 EBPI implementations (76%) were sustained for five and one-half years. When controlling for variability among coalitions (nesting of EBPIs in coalitions), increases in data resources (availability of five types of prevention data) was a strong predictor of length of EBPI sustainability. Positive change in extramural funding resources and level of expertise during SPF SIG implementation, as well as level of coalition formalization at the end of the SPF SIG predicted EBPI sustainability length. One intervention attribute (trialability) also predicted length of sustainability. Implications are discussed.

This is a preview of subscription content, access via your institution.

Access options

Buy single article

Instant access to the full article PDF.

USD 39.95

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Fig. 1
Fig. 2

References

  • Blanchet, K., & James, P. (2014). Can international health programmes be sustained after the end of international funding?: The case of eye care interventions in Ghana. BMC Health Services Research, 14 Retrieved from http://bmchealthservres.biomedcentral.com/articles/10.1186/1472-6963-14-77.

  • Bond, G. R., Drake, R. E., McHugo, G. J., Peterson, A. E., Jones, A. M., & Williams, J. (2014). Long-term sustainability of evidence-based practices in community mental health agencies. Administration and Policy in Mental Health and Mental Health Services Research, 41, 228–236.

    Article  PubMed  Google Scholar 

  • Bradford, N. K., Caffery, L. J., & Smith, A. C. (2016). Telehealth services in rural and remote Australia: A systematic review of models of care and factors influencing success and sustainability. Rural and Remote Health, 16. Retrieved from www.rrh.org.au/publishedarticles/article_print_3808.pdf.

  • Brown, L. D., Feinberg, M. E., & Greenberg, M. T. (2010). Determinants of community coalition ability to support evidence-based programs. Prevention Science, 11, 287–297.

    Article  PubMed  PubMed Central  Google Scholar 

  • Cernat, A., Couper, M. P., & Ofstedal, M. B. (2016). Estimation of mode effects in the health and retirement study using measurement models. Journal of Survey Statistics and Methodology, 4, 501–524.

    Article  PubMed Central  Google Scholar 

  • Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale: Erlbaum.

    Google Scholar 

  • Cohen, J. (1990). Things I have learned (so far). American Psychologist, 45, 1304–1312.

    Article  Google Scholar 

  • Collins, D., Shamblen, S., Harris, M., Johnson, K., & Dwivedi, P. (2009). Evaluation of Tennessee SPF SIG local capacity building: Final report. Louisville: PIRE Retrieved from http://www.pire.org/documents/TN_Local_%20Capacity_%20Final_Report.pdf.

    Google Scholar 

  • Cooper, B. R., Bumbarger, B. K., & Moore, J. E. (2015). Sustaining evidence-based prevention programs: Correlates in a large-scale dissemination initiative. Prevention Science, 16, 145–157.

    Article  PubMed  Google Scholar 

  • De Leeuw, E. D., & Hox, J. J. (2015). Survey mode and mode effects. In U. Engel, B. Jann, P. Lynn, A. Scherpenzeel, & P. Sturgis (Eds.), Improving survey methods: Lessons from recent research (pp. 22–34). New York: Routledge/Taylor & Francis.

    Google Scholar 

  • Edwards, J. M., Stein-Seroussi, A., Flewelling, R. L., Orwin, R. G., & Zhang, L. (2015). Sustainability of state-level substance abuse prevention infrastructure after the completion of the SPF SIG. The Journal of Primary Prevention, 36, 177–186.

    Article  PubMed  Google Scholar 

  • Fagan, A. A., Hanson, K., Hawkins, J. D., & Arthur, M. W. (2008). Implementing effective community-based prevention programs in the Community Youth Development Study. Youth Violence and Juvenile Justice, 6, 256–278.

    Article  Google Scholar 

  • Federal Register (2004). Notice of request for applications for Strategic Prevention Framework State Incentive Grants (SPF SIG) (SP 04–002). Retrieved from https://www.federalregister.gov/articles/2004/04/29/04-9656/notice-of-request-for-applications-for-strategic-prevention-framework-state-incentive-grants-spf-sig#h-12.

  • Feinberg, M. E., Bontempo, D. E., & Greenberg, M. T. (2008). Predictors and level of sustainability of community prevention coalitions. American Journal of Preventive Medicine, 34, 495–501.

    Article  PubMed  Google Scholar 

  • Flay, B. R., Biglan, A., Boruch, R. F., Gonzalez Castro, F., Gottfredson, D., Kellam, S., …, Ji, P. (2005). Standards of evidence: Criteria for efficacy, effectiveness and dissemination. Prevention Science, 6, 151–175.

  • Flewelling, R. L., & Hanley, S. M. (2016). Assessing community coalition capacity and its association with underage drinking prevention effectiveness in the context of the SPF SIG. Prevention Science, 17, 830–840.

    Article  PubMed  Google Scholar 

  • Gloppen, K. M., Arthur, M. W., Hawkins, J. D., & Shapiro, V. B. (2012). Sustainability of the Communities That are prevention system by coalitions participating in the Community Youth Development Study. Journal of Adolescent Health, 51, 259–264.

    Article  PubMed  PubMed Central  Google Scholar 

  • Gloppen, K. M., Brown, E. C., Wagenaar, B. H., Hawkins, J. D., Rhew, I. C., & Oesterle, S. (2016). Sustaining adoption of science-based prevention through Communities That Care. Journal of Community Psychology, 44, 78–89.

    Article  PubMed  Google Scholar 

  • Greenberg, M. T., Feinberg, M. E., Johnson, L. E., Perkins, D. F., Welsh, J. A., & Spoth, R. L. (2015). Factors that predict financial sustainability of community coalitions: Five years of findings from the PROSPER partnership project. Prevention Science, 16, 158–167.

    Article  PubMed  PubMed Central  Google Scholar 

  • IBM Corp. (2011). IBM SPSS Statistics for Windows, Version 20.0. Armonk: IBM Corp.

    Google Scholar 

  • ICPSR (2016). Strategic Prevention Framework State Incentive Grant (SPF SIG) national cross-site evaluation. Retrieved from http://www.icpsr.umich.edu/icpsrweb/NAHDAP/studies/28921.

  • Ihaka, R., & Gentleman, R. (1996). R: A language for data analysis and graphics. Journal of Computational and Graphical Statistics, 5, 299–314.

    Google Scholar 

  • Johnson, K., Hays, C., Center, H., & Daley, C. (2004). Building capacity and sustainable prevention innovations: A sustainability planning model. Evaluation and Program Planning, 27, 135–149.

    CAS  Article  Google Scholar 

  • Johnson, K., Collins, D., & Wandersman, A. (2013). Sustaining innovations in community prevention systems: A data-informed sustainability strategy. Journal of Community Psychology, 41, 322–340.

    Article  Google Scholar 

  • Lean, M., Leavey, G., Killaspy, H., Green, N., Harrison, I., Cook, S., Craig, T., Holloway, F., Arbuthnott, M., & King, M. (2015). Barriers to the sustainability of an intervention designed to improve patient engagement within NHS mental health rehabilitation units: A qualitative study nested within a randomised controlled trial. BMC Psychiatry, 15. doi:10.1186/s12888-015-0592-9.

  • Livet, M., Courser, M., & Wandersman, A. (2008). The prevention delivery system: Organizational context and use of comprehensive programming frameworks. American Journal of Community Psychology, 41, 361–378.

    Article  PubMed  Google Scholar 

  • Luke, D. A., Calhoun, A., Robichaux, C. B., Elliott, M. B., & Moreland-Russell, S. (2014). The program sustainability assessment tool: A new instrument for public health programs. Preventing Chronic Disease, 11, E12. doi:10.5888/pcd11.130184.

    Article  PubMed Central  Google Scholar 

  • McIntosh, K., Kim, J., Mercer, S. H., Strickland-Cohen, M. K., & Horner, R. H. (2015). Variables associated with enhanced sustainability of school-wide positive behavioral interventions and supports. Assessment for Effective Intervention, 40, 184–191.

    Article  Google Scholar 

  • Nargiso, J. E., Friend, K. B., Egan, C., Florin, P., Stevenson, J., Amodei, B., & Barovier, L. (2013). Coalitional capacities and environmental strategies to prevent underage drinking. American Journal of Community Psychology, 51, 222–231.

    Article  PubMed  Google Scholar 

  • Pacific Institute for Research and Evaluation (PIRE). (2009). Strategies to reduce underage alcohol use: Typology and brief overview. Calverton: PIRE.

    Google Scholar 

  • Peterson, A. E., Bond, G. R., Drake, R. E., McHugo, G. J., Jones, A. M., & Williams, J. R. (2014). Predicting the long-term sustainability of evidence-based practices in mental health care: An 8-year longitudinal analysis. Journal of Behavioral Health Services & Research, 41, 337–346.

    Article  Google Scholar 

  • Pinheiro, J., Bates, D., DebRoy, S., Sarkar, D., EISPACK authors, Heisterkamp, S., …, R-core (2016). nlme: Linear and nonlinear mixed effects models (Computer Software: v. 3.1–127). Retrieved from https://cran.r-project.org/web/packages/nlme.

  • Prevention Alliance of Tennessee (2015). PAT coalition portfolio. Retrieved from http://www.tncoalitions.org/coalitions.

  • Proctor, E., Silmere, H., Raghavan, R., Hovmand, P., Aarons, G., Bunger, A., …, Hensley, M. (2011). Outcomes for implementation research: Conceptual distinctions, measurement challenges, and research agenda. Administration and Policy in Mental Health and Mental Health Services, 38, 65–76.

  • Rogers, E. M. (1995). Diffusion of innovations (4th ed.). New York: Free Press.

    Google Scholar 

  • Rogers, E. M. (2003). Diffusion of innovations (5th ed.). New York: Free Press.

    Google Scholar 

  • Scaccia, J. P., Cook, B. S., Lamont, A., Wandersman, A., Castellow, J., Katz, J., & Beidas, R. S. (2015). A practical implementation science heuristic for organizational readiness: R=MC2. Journal of Community Psychology, 43, 484–501.

    Article  PubMed  PubMed Central  Google Scholar 

  • Scheirer, M. A. (2005). Is sustainability possible? A review and commentary on empirical studies of program sustainability. American Journal of Evaluation, 26, 320–347.

    Article  Google Scholar 

  • Scheirer, M. A., & Dearing, J. W. (2011). An agenda for research on the sustainability of public health programs. American Journal of Public Health, 101, 2059–2067.

    Article  PubMed  PubMed Central  Google Scholar 

  • Shults, R. A., Elder, R. W., Sleet, D. A., Nichols, J. L., Alao, M. O., Carande-Kulis, V., & Thompson, R. S. (2001). Reviews of evidence regarding interventions to reduce alcohol-impaired driving. American Journal of Preventive Medicine, 21, 66–88.

    CAS  Article  PubMed  Google Scholar 

  • Stirman, S. W., Kimberly, J., Cook, N., Calloway, A., Castro, F., & Charns, M. (2012). The sustainability of new programs and innovations: A review of the empirical literature and recommendations for future research. Implementation Science, 7. doi:10.1186/1748-5908-7-17.

  • Substance Abuse and Mental Health Services Administration (2016). National registry of evidence-based programs and practices. Retrieved from http://nrepp.samhsa.gov/01_landing.aspx.

  • Tennessee Department of Mental Health and Substance Abuse Services (2012). Prevention services provider directory, version 3. Retrieved from https://www.tn.gov/assets/entities/behavioral-health/sa/attachments/TPN-Prevention_Services_Directory_FY_13.pdf.

  • Tibbits, M. K., Bumbarger, B. K., Kyler, S., & Perkins, D. F. (2010). Sustaining evidence-based interventions under real world conditions: Results of a large-scale diffusion project. Prevention Science, 11, 252–262.

    Article  PubMed  PubMed Central  Google Scholar 

  • U.S. Department of Health and Human Services (2014). Performance improvement 2013–2014. Retrieved from https://aspe.hhs.gov/report/performance-improvement-2013-2014/what-are-program-outcomes-program-effectiveness-and-community-change-among-strategic-prevention-framework-state-incentive-grants.

Download references

Acknowledgments

This study was funded in part by the Pacific Institute for Research and Evaluation–Louisville Center and the University of South Carolina, Department of Psychology. We thank Leah Festa, the Director of the Prevention Alliance of Tennessee, the many former and current community coalition staff members, and the Tennessee Department of Mental Health and Substance Abuse Services staff for their assistance in providing data to the study. Chris Bayer, Louisville Center Production Manager, also provided important manuscript editing and formatting. A special thank you goes to the Prevention Science reviewers who provided valuable feedback on an earlier version of the manuscript.

Author information

Authors and Affiliations

  1. Pacific Institute for Research and Evaluation, Louisville Center, Louisville, KY, USA

    Knowlton Johnson, David Collins & Steve Shamblen

  2. Department of Psychology, University of South Carolina, Columbia, SC, USA

    Tara Kenworthy & Abraham Wandersman

Authors
  1. Knowlton Johnson
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. David Collins
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Steve Shamblen
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Tara Kenworthy
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Abraham Wandersman
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Knowlton Johnson.

Ethics declarations

Funding

This study was conducted using internal funding including in-kind funding from the Pacific Institute for Research and Evaluation and the University of South Carolina, Department of Psychology.

Conflict of Interest

The authors declare that they have no conflict of interest.

Ethical Approval

PIRE’s IRB reviewed the protocols for this study and deemed the study exempt under 45 CFR 46.101(b)(2). This study was performed in accordance with the ethical standards as laid down in the 1964 Declaration of Helsinki and its later amendments or comparable ethical standards.

Informed Consent

Written informed consent was not obtained from key respondents of our telephone interview/web-based survey conducted in 2015 since the PIRE IRB board deemed this study exempt under 45 CFR 46.101(b)(2). However, we did obtain verbal informed consent from the 2015–2016 sustainability survey participants.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Johnson, K., Collins, D., Shamblen, S. et al. Long-Term Sustainability of Evidence-Based Prevention Interventions and Community Coalitions Survival: a Five and One-Half Year Follow-up Study. Prev Sci 18, 610–621 (2017). https://doi.org/10.1007/s11121-017-0784-2

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11121-017-0784-2

Keywords

  • Sustainability
  • Substance abuse prevention
  • Evidence-based interventions
  • Community coalitions