Predicting the Long-Term Sustainability of Evidence-Based Practices in Mental Health Care: An 8-Year Longitudinal Analysis

  • Alison E. Peterson
  • Gary R. Bond
  • Robert E. Drake
  • Gregory J. McHugo
  • Amanda M. Jones
  • Jessica R. Williams


Few studies have examined predictors of long-term sustainability of evidence-based practices in mental health. This study used assessments of five evidence-based practices implemented in 49 sites in eight states at baseline and years 2, 4, and 8. Program characteristics, implementation characteristics, reinforcement activities, and sustainability factors were used to predict program survival status. The majority of predictors were not significant. Supervisor turnover in year 4 predicted survival status in year 8, but site characteristics, fidelity at implementation, quality improvement activities, and post-implementation activities had little impact on long-term program survival. This study extends previous sustainability research by examining the long-term impact of internal program factors over a substantial period of time using longitudinal prediction. Future research should also consider the influence of external factors such as financial policies.


Assertive Community Treatment Program Survival Program Sustainability Quality Improvement Activity Family Psychoeducation 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.



We thank the many site leaders who agreed to be interviewed and who provided thoughtful responses. This study was supported in part by Substance Abuse and Mental Health Services Administration (SAMHSA) contract No. HHSS283200700040I/HHSS28342003T.

Conflict of interest statement

The authors report no conflicts of interest.


  1. 1.
    Drake RE, Torrey WC, McHugo GJ. Strategies for implementing evidence-based practices in routine mental health settings. Evidence-Based Mental Health. 2003;6: 6–7.Google Scholar
  2. 2.
    Frey WD, Azrin ST, Goldman HH, et al.. The Mental Health Treatment Study. Psychiatric Rehabilitation Journal. 2008;31 (4):306–12.PubMedCrossRefGoogle Scholar
  3. 3.
    Mancini AD, Moser LL, Whitley R, et al. Assertive community treatment: facilitators and barriers to implementation in routine mental health settings. Psychiatric Services. 2009;60:189–95.PubMedCrossRefGoogle Scholar
  4. 4.
    Moser LL, DeLuca NL, Bond GR, et al. Implementing evidence based psychosocial practices: lessons learned from statewide implementation of two practices. CNS Spectrums. 2004;9(926–936).Google Scholar
  5. 5.
    Bjorklund RW, Monroe-DeVita M, Reed D, et al. Washington State's initiative to disseminate and implement high-fidelity ACT teams. Psychiatric Services. 2009;60:24–7.PubMedCrossRefGoogle Scholar
  6. 6.
    McFarlane WR, McNary S, Dixon L, et al. Predictors of dissemination of family psychoeducation in community mental health centers in Maine and Illinois. Psychiatric Services. 2001;52:935–42.PubMedCrossRefGoogle Scholar
  7. 7.
    Wilson DC, Crisanti AS. Psychometric properties of the Dual-Disorder Treatment Fidelity Scale: Inter-rater reliability and concurrent validity. Community Mental Health Journal. 2009;45(3):171–8.PubMedCrossRefGoogle Scholar
  8. 8.
    Becker DR, Drake RE, Bond GR, et al. A mental health learning collaborative on supported employment. Psychiatric Services. 2011;62(7):704–6.PubMedCrossRefGoogle Scholar
  9. 9.
    Fixsen DL, Naoom SF, Blase KA, et al. Implementation research: a synthesis of the literature. Tampa, FL: University of South Florida; 2005.Google Scholar
  10. 10.
    Bhattacharyya O, Reeves S, Zwarenstein M. What is implementation research? Rationale, concepts, and practices. Research on Social Work Practice. 2009;19(5):491–502.CrossRefGoogle Scholar
  11. 11.
    Drake RE, Bond GR, Essock SM. Implementing evidence-based practices for people with schizophrenia. Schizophrenia Bulletin. 2009;35(4):704–13.PubMedCentralPubMedCrossRefGoogle Scholar
  12. 12.
    Proctor EK, Landsverk J, Aarons GA, et al. Implementation research in mental health services: an emerging science with conceptual, methodological, and training challenges. Administration and Policy in Mental Health and Mental Health Services Research. 2009;36:24–34.PubMedCrossRefGoogle Scholar
  13. 13.
    Proctor E, Silmere H, Raghavan R, et al. Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda. Administration and Policy in Mental Health. 2011;38:65–76.PubMedCentralPubMedCrossRefGoogle Scholar
  14. 14.
    Shediac-Rizkallah MC, Bone LR. Planning for the sustainability of community-based health programs: conceptual frameworks and future directions for research, practice and policy. Health Education Research. 1998;13(1):87–108.PubMedCrossRefGoogle Scholar
  15. 15.
    Stirman S, Kimberly J, Cook N, et al. The sustainability of new programs and innovations: a review of the empirical literature and recommendations for future research. Implementation Science. 2012;7(17).Google Scholar
  16. 16.
    Damschroder LJ, Hagedorn HJ. A guiding framework and approach for implementation research in substance use disorders treatment. Psychology of Addictive Behaviors. 2011;25(2):194–205.PubMedCrossRefGoogle Scholar
  17. 17.
    Gruen RL, Elliott JH, Nolan ML, et al. Sustainability science: an integrated approach for health programme planning. Lancet. 2008;372(9649):1579–89.PubMedCrossRefGoogle Scholar
  18. 18.
    Aarons GA, Hurlburt M, Horwitz SM. Advancing a conceptual model of evidence-based practice implementation in public service sectors. Administration and Policy in Mental Health and Mental Health Services Research. 2011;38:4–23.PubMedCentralPubMedCrossRefGoogle Scholar
  19. 19.
    Glanz K, Bishop DB. The role of behavioral science theory in development and implementation of public health interventions. Annual Review of Public Health. 2010;31:399–418.PubMedCrossRefGoogle Scholar
  20. 20.
    Johnson K, Hays C, Center H, et al. Building capacity and sustainable prevention innovations: a sustainability planning model. Evaluation and Program Planning. 2004;27:135–49.CrossRefGoogle Scholar
  21. 21.
    McHugo GJ, Drake RE, Whitley R, et al. Fidelity outcomes in the National Implementing Evidence-Based Practices Project. Psychiatric Services. 2007;58:1279.PubMedCrossRefGoogle Scholar
  22. 22.
    Torrey WC, Lynde DW, Gorman P. Promoting the implementation of practices that are supported by research: the National Implementing Evidence-Based Practices Project. Child and Adolescent Psychiatric Clinics of North America. 2005;14:297–306.PubMedCrossRefGoogle Scholar
  23. 23.
    Swain K, Whitley R, McHugo G, et al. The sustainability of evidence-based practices in routine mental health agencies. Community Mental Health Journal. 2009.Google Scholar
  24. 24.
    Bond GR, Drake RE, McHugo GJ, et al. Long-term sustainability of evidence-based practices in community mental health agencies. Administration and Policy in Mental Health. 2012 [online ahead of print].Google Scholar
  25. 25.
    Massatti R, Sweeney H, Panzano P, et al. The de-adoption of innovative mental health practices (IMHP): why organizations choose not to sustain an IMHP. Administration and Policy in Mental Health [serial on the Internet]. 2008; 35.Google Scholar
  26. 26.
    Torrey WC, Bond GR, McHugo GJ, et al. Evidence-based practice implementation in community mental health settings: the relative importance of key domains of implementation activity. Administration and Policy in Mental Health and Mental Health Services Research. 2012;39:353–364.PubMedCrossRefGoogle Scholar
  27. 27.
    Bond G, Drake R, Rapp C, et al. Individualization and quality improvement: two new scales to complement measurement of program fidelity. Administration and Policy in Mental Health. 2009;36:349–57.PubMedCrossRefGoogle Scholar
  28. 28.
    Wiltsey Stirman S, Kimberly J, Cook N, et al. The sustainability of new programs and innovations: a review of the empirical literature and recommendations for future research. Implementation Science. 2012;7:17 doi: 10.1186/1748-5908-7-17.
  29. 29.
    Pluye P, Potvin L, Denis J. Making public health programs last: conceptualizing sustainability. Evaluation and Program Planning. 2004;27:121–33.CrossRefGoogle Scholar
  30. 30.
    Pluye P, Potvin L, Denis J, et al. Program sustainability begins with the first events. Evaluation and Program Planning. 2005;28:123–37.CrossRefGoogle Scholar
  31. 31.
    Yin RK. Life histories of innovations: how new practices become routinized. Public Administration Review. 1981;41:21–8.CrossRefGoogle Scholar
  32. 32.
    Becker D, Baker S, Carlson L, et al. Critical strategies for implementing supported employment. Journal of Vocational Rehabilitation. 2007;27:13–20.Google Scholar
  33. 33.
    Carlson L, Rapp CA, Eichler MS. The experts rate: supervisory behaviors that impact the implementation of evidence-based practices. Community Mental Health Journal. 2012;48:179–86.PubMedCrossRefGoogle Scholar
  34. 34.
    Rogers EM. Diffusion of innovations. 5th ed. New York: Free Press; 2003.Google Scholar

Copyright information

© Springer Science+Business Media, LLC (outside the USA) 2013

Authors and Affiliations

  • Alison E. Peterson
    • 1
  • Gary R. Bond
    • 1
  • Robert E. Drake
    • 1
  • Gregory J. McHugo
    • 1
  • Amanda M. Jones
    • 2
  • Jessica R. Williams
    • 2
    • 3
  1. 1.Dartmouth Psychiatric Research CenterGeisel School of Medicine at Dartmouth, Rivermill Commercial CenterLebanonUSA
  2. 2.MANILA Consulting Group, IncMcLeanUSA
  3. 3.University of Miami School of Nursing and Health StudiesCoral GablesUSA

Personalised recommendations