Advertisement

Prevention Science

, Volume 18, Issue 5, pp 610–621 | Cite as

Long-Term Sustainability of Evidence-Based Prevention Interventions and Community Coalitions Survival: a Five and One-Half Year Follow-up Study

  • Knowlton JohnsonEmail author
  • David Collins
  • Steve Shamblen
  • Tara Kenworthy
  • Abraham Wandersman
Article

Abstract

This study examines (1) coalition survival, (2) prevalence of evidence-based prevention interventions (EBPIs) to reduce substance abuse implemented as part of the Tennessee Strategic Prevention Framework (SPF) State Incentive Grant (SIG), (3) EBPI sustainability, and (4) factors that predict EBPI sustainability. Secondary data were collected on 27 SPF SIG-funded coalitions and 88 EBPI and non-EBPI implementations. Primary data were collected by a telephone interview/web survey five and one-half years after the SPF SIG ended. Results from secondary data show that 25 of the 27 coalitions survived beyond the SPF SIG for one to five and one-half years; 19 coalitions (70%) were still active five and one-half years later. Further, 88 EBPIs and non-EBPIs were implemented by 27 county SPF SIG coalitions. Twenty-one (21) of 27 coalitions (78%) implemented one to three EBPIs, totaling 37 EBPI implementations. Based on primary survey data on 29 of the 37 EBPI implementations, 28 EBPIs (97%) were sustained between two and five and one-half years while 22 EBPI implementations (76%) were sustained for five and one-half years. When controlling for variability among coalitions (nesting of EBPIs in coalitions), increases in data resources (availability of five types of prevention data) was a strong predictor of length of EBPI sustainability. Positive change in extramural funding resources and level of expertise during SPF SIG implementation, as well as level of coalition formalization at the end of the SPF SIG predicted EBPI sustainability length. One intervention attribute (trialability) also predicted length of sustainability. Implications are discussed.

Keywords

Sustainability Substance abuse prevention Evidence-based interventions Community coalitions 

Notes

Acknowledgments

This study was funded in part by the Pacific Institute for Research and Evaluation–Louisville Center and the University of South Carolina, Department of Psychology. We thank Leah Festa, the Director of the Prevention Alliance of Tennessee, the many former and current community coalition staff members, and the Tennessee Department of Mental Health and Substance Abuse Services staff for their assistance in providing data to the study. Chris Bayer, Louisville Center Production Manager, also provided important manuscript editing and formatting. A special thank you goes to the Prevention Science reviewers who provided valuable feedback on an earlier version of the manuscript.

Compliance with Ethical Standards

Funding

This study was conducted using internal funding including in-kind funding from the Pacific Institute for Research and Evaluation and the University of South Carolina, Department of Psychology.

Conflict of Interest

The authors declare that they have no conflict of interest.

Ethical Approval

PIRE’s IRB reviewed the protocols for this study and deemed the study exempt under 45 CFR 46.101(b)(2). This study was performed in accordance with the ethical standards as laid down in the 1964 Declaration of Helsinki and its later amendments or comparable ethical standards.

Informed Consent

Written informed consent was not obtained from key respondents of our telephone interview/web-based survey conducted in 2015 since the PIRE IRB board deemed this study exempt under 45 CFR 46.101(b)(2). However, we did obtain verbal informed consent from the 2015–2016 sustainability survey participants.

References

  1. Blanchet, K., & James, P. (2014). Can international health programmes be sustained after the end of international funding?: The case of eye care interventions in Ghana. BMC Health Services Research, 14 Retrieved from http://bmchealthservres.biomedcentral.com/articles/10.1186/1472-6963-14-77.
  2. Bond, G. R., Drake, R. E., McHugo, G. J., Peterson, A. E., Jones, A. M., & Williams, J. (2014). Long-term sustainability of evidence-based practices in community mental health agencies. Administration and Policy in Mental Health and Mental Health Services Research, 41, 228–236.CrossRefPubMedGoogle Scholar
  3. Bradford, N. K., Caffery, L. J., & Smith, A. C. (2016). Telehealth services in rural and remote Australia: A systematic review of models of care and factors influencing success and sustainability. Rural and Remote Health, 16. Retrieved from www.rrh.org.au/publishedarticles/article_print_3808.pdf.
  4. Brown, L. D., Feinberg, M. E., & Greenberg, M. T. (2010). Determinants of community coalition ability to support evidence-based programs. Prevention Science, 11, 287–297.CrossRefPubMedPubMedCentralGoogle Scholar
  5. Cernat, A., Couper, M. P., & Ofstedal, M. B. (2016). Estimation of mode effects in the health and retirement study using measurement models. Journal of Survey Statistics and Methodology, 4, 501–524.CrossRefPubMedCentralGoogle Scholar
  6. Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale: Erlbaum.Google Scholar
  7. Cohen, J. (1990). Things I have learned (so far). American Psychologist, 45, 1304–1312.CrossRefGoogle Scholar
  8. Collins, D., Shamblen, S., Harris, M., Johnson, K., & Dwivedi, P. (2009). Evaluation of Tennessee SPF SIG local capacity building: Final report. Louisville: PIRE Retrieved from http://www.pire.org/documents/TN_Local_%20Capacity_%20Final_Report.pdf.Google Scholar
  9. Cooper, B. R., Bumbarger, B. K., & Moore, J. E. (2015). Sustaining evidence-based prevention programs: Correlates in a large-scale dissemination initiative. Prevention Science, 16, 145–157.CrossRefPubMedGoogle Scholar
  10. De Leeuw, E. D., & Hox, J. J. (2015). Survey mode and mode effects. In U. Engel, B. Jann, P. Lynn, A. Scherpenzeel, & P. Sturgis (Eds.), Improving survey methods: Lessons from recent research (pp. 22–34). New York: Routledge/Taylor & Francis.Google Scholar
  11. Edwards, J. M., Stein-Seroussi, A., Flewelling, R. L., Orwin, R. G., & Zhang, L. (2015). Sustainability of state-level substance abuse prevention infrastructure after the completion of the SPF SIG. The Journal of Primary Prevention, 36, 177–186.CrossRefPubMedGoogle Scholar
  12. Fagan, A. A., Hanson, K., Hawkins, J. D., & Arthur, M. W. (2008). Implementing effective community-based prevention programs in the Community Youth Development Study. Youth Violence and Juvenile Justice, 6, 256–278.CrossRefGoogle Scholar
  13. Federal Register (2004). Notice of request for applications for Strategic Prevention Framework State Incentive Grants (SPF SIG) (SP 04–002). Retrieved from https://www.federalregister.gov/articles/2004/04/29/04-9656/notice-of-request-for-applications-for-strategic-prevention-framework-state-incentive-grants-spf-sig#h-12.
  14. Feinberg, M. E., Bontempo, D. E., & Greenberg, M. T. (2008). Predictors and level of sustainability of community prevention coalitions. American Journal of Preventive Medicine, 34, 495–501.CrossRefPubMedGoogle Scholar
  15. Flay, B. R., Biglan, A., Boruch, R. F., Gonzalez Castro, F., Gottfredson, D., Kellam, S., …, Ji, P. (2005). Standards of evidence: Criteria for efficacy, effectiveness and dissemination. Prevention Science, 6, 151–175.Google Scholar
  16. Flewelling, R. L., & Hanley, S. M. (2016). Assessing community coalition capacity and its association with underage drinking prevention effectiveness in the context of the SPF SIG. Prevention Science, 17, 830–840.CrossRefPubMedGoogle Scholar
  17. Gloppen, K. M., Arthur, M. W., Hawkins, J. D., & Shapiro, V. B. (2012). Sustainability of the Communities That are prevention system by coalitions participating in the Community Youth Development Study. Journal of Adolescent Health, 51, 259–264.CrossRefPubMedPubMedCentralGoogle Scholar
  18. Gloppen, K. M., Brown, E. C., Wagenaar, B. H., Hawkins, J. D., Rhew, I. C., & Oesterle, S. (2016). Sustaining adoption of science-based prevention through Communities That Care. Journal of Community Psychology, 44, 78–89.CrossRefPubMedGoogle Scholar
  19. Greenberg, M. T., Feinberg, M. E., Johnson, L. E., Perkins, D. F., Welsh, J. A., & Spoth, R. L. (2015). Factors that predict financial sustainability of community coalitions: Five years of findings from the PROSPER partnership project. Prevention Science, 16, 158–167.CrossRefPubMedPubMedCentralGoogle Scholar
  20. IBM Corp. (2011). IBM SPSS Statistics for Windows, Version 20.0. Armonk: IBM Corp.Google Scholar
  21. ICPSR (2016). Strategic Prevention Framework State Incentive Grant (SPF SIG) national cross-site evaluation. Retrieved from http://www.icpsr.umich.edu/icpsrweb/NAHDAP/studies/28921.
  22. Ihaka, R., & Gentleman, R. (1996). R: A language for data analysis and graphics. Journal of Computational and Graphical Statistics, 5, 299–314.Google Scholar
  23. Johnson, K., Hays, C., Center, H., & Daley, C. (2004). Building capacity and sustainable prevention innovations: A sustainability planning model. Evaluation and Program Planning, 27, 135–149.CrossRefGoogle Scholar
  24. Johnson, K., Collins, D., & Wandersman, A. (2013). Sustaining innovations in community prevention systems: A data-informed sustainability strategy. Journal of Community Psychology, 41, 322–340.CrossRefGoogle Scholar
  25. Lean, M., Leavey, G., Killaspy, H., Green, N., Harrison, I., Cook, S., Craig, T., Holloway, F., Arbuthnott, M., & King, M. (2015). Barriers to the sustainability of an intervention designed to improve patient engagement within NHS mental health rehabilitation units: A qualitative study nested within a randomised controlled trial. BMC Psychiatry, 15. doi: 10.1186/s12888-015-0592-9.
  26. Livet, M., Courser, M., & Wandersman, A. (2008). The prevention delivery system: Organizational context and use of comprehensive programming frameworks. American Journal of Community Psychology, 41, 361–378.CrossRefPubMedGoogle Scholar
  27. Luke, D. A., Calhoun, A., Robichaux, C. B., Elliott, M. B., & Moreland-Russell, S. (2014). The program sustainability assessment tool: A new instrument for public health programs. Preventing Chronic Disease, 11, E12. doi: 10.5888/pcd11.130184.CrossRefPubMedCentralGoogle Scholar
  28. McIntosh, K., Kim, J., Mercer, S. H., Strickland-Cohen, M. K., & Horner, R. H. (2015). Variables associated with enhanced sustainability of school-wide positive behavioral interventions and supports. Assessment for Effective Intervention, 40, 184–191.CrossRefGoogle Scholar
  29. Nargiso, J. E., Friend, K. B., Egan, C., Florin, P., Stevenson, J., Amodei, B., & Barovier, L. (2013). Coalitional capacities and environmental strategies to prevent underage drinking. American Journal of Community Psychology, 51, 222–231.CrossRefPubMedGoogle Scholar
  30. Pacific Institute for Research and Evaluation (PIRE). (2009). Strategies to reduce underage alcohol use: Typology and brief overview. Calverton: PIRE.Google Scholar
  31. Peterson, A. E., Bond, G. R., Drake, R. E., McHugo, G. J., Jones, A. M., & Williams, J. R. (2014). Predicting the long-term sustainability of evidence-based practices in mental health care: An 8-year longitudinal analysis. Journal of Behavioral Health Services & Research, 41, 337–346.CrossRefGoogle Scholar
  32. Pinheiro, J., Bates, D., DebRoy, S., Sarkar, D., EISPACK authors, Heisterkamp, S., …, R-core (2016). nlme: Linear and nonlinear mixed effects models (Computer Software: v. 3.1–127). Retrieved from https://cran.r-project.org/web/packages/nlme.
  33. Prevention Alliance of Tennessee (2015). PAT coalition portfolio. Retrieved from http://www.tncoalitions.org/coalitions.
  34. Proctor, E., Silmere, H., Raghavan, R., Hovmand, P., Aarons, G., Bunger, A., …, Hensley, M. (2011). Outcomes for implementation research: Conceptual distinctions, measurement challenges, and research agenda. Administration and Policy in Mental Health and Mental Health Services, 38, 65–76.Google Scholar
  35. Rogers, E. M. (1995). Diffusion of innovations (4th ed.). New York: Free Press.Google Scholar
  36. Rogers, E. M. (2003). Diffusion of innovations (5th ed.). New York: Free Press.Google Scholar
  37. Scaccia, J. P., Cook, B. S., Lamont, A., Wandersman, A., Castellow, J., Katz, J., & Beidas, R. S. (2015). A practical implementation science heuristic for organizational readiness: R=MC2. Journal of Community Psychology, 43, 484–501.CrossRefPubMedPubMedCentralGoogle Scholar
  38. Scheirer, M. A. (2005). Is sustainability possible? A review and commentary on empirical studies of program sustainability. American Journal of Evaluation, 26, 320–347.CrossRefGoogle Scholar
  39. Scheirer, M. A., & Dearing, J. W. (2011). An agenda for research on the sustainability of public health programs. American Journal of Public Health, 101, 2059–2067.CrossRefPubMedPubMedCentralGoogle Scholar
  40. Shults, R. A., Elder, R. W., Sleet, D. A., Nichols, J. L., Alao, M. O., Carande-Kulis, V., & Thompson, R. S. (2001). Reviews of evidence regarding interventions to reduce alcohol-impaired driving. American Journal of Preventive Medicine, 21, 66–88.CrossRefPubMedGoogle Scholar
  41. Stirman, S. W., Kimberly, J., Cook, N., Calloway, A., Castro, F., & Charns, M. (2012). The sustainability of new programs and innovations: A review of the empirical literature and recommendations for future research. Implementation Science, 7. doi: 10.1186/1748-5908-7-17.
  42. Substance Abuse and Mental Health Services Administration (2016). National registry of evidence-based programs and practices. Retrieved from http://nrepp.samhsa.gov/01_landing.aspx.
  43. Tennessee Department of Mental Health and Substance Abuse Services (2012). Prevention services provider directory, version 3. Retrieved from https://www.tn.gov/assets/entities/behavioral-health/sa/attachments/TPN-Prevention_Services_Directory_FY_13.pdf.
  44. Tibbits, M. K., Bumbarger, B. K., Kyler, S., & Perkins, D. F. (2010). Sustaining evidence-based interventions under real world conditions: Results of a large-scale diffusion project. Prevention Science, 11, 252–262.CrossRefPubMedPubMedCentralGoogle Scholar

Copyright information

© Society for Prevention Research 2017

Authors and Affiliations

  • Knowlton Johnson
    • 1
    Email author
  • David Collins
    • 1
  • Steve Shamblen
    • 1
  • Tara Kenworthy
    • 2
  • Abraham Wandersman
    • 2
  1. 1.Pacific Institute for Research and Evaluation, Louisville CenterLouisvilleUSA
  2. 2.Department of PsychologyUniversity of South CarolinaColumbiaUSA

Personalised recommendations