Skip to main content

The Sustainability of Evidence-Based Practices in Routine Mental Health Agencies

Abstract

The research presented here reports on sustainability of the practices within the National Implementing Evidence Based Practices Project for people with serious mental illness. Forty-nine sites completed the initial 2-year implementation phase and were the focus of our study. Our aims were to discern the number of sites that sustained practices 2 years after implementation, the reasons for sustaining or not sustaining, differences in characteristics between the two groups, and the extent and nature of practice adaptations. We used a mixed-methods approach, based on a telephone survey that gathered qualitative and quantitative data from site representatives and others familiar with the sites and practices during the follow-up period. We found that 80% of sites sustained their practices for 2 years post-implementation, that sustainers differed from non-sustainers in several domains: financing, training, fidelity and agency leadership, and that most sites adapted practices moderately to meet state and local needs.

This is a preview of subscription content, access via your institution.

References

  1. Battistich, V., Schaps, E., Watson, M., & Solomon, D. (1996). Prevention effects of the child development project: Early findings from an ongoing multisite demonstration trial. Journal of Adolescent Research, 11(1), 12–35. doi:10.1177/0743554896111003.

    Article  Google Scholar 

  2. Blasinski, M., Goldman, H. H., & Unutzer, J. (2006). Project IMPACT: A report on barriers and facilitators to sustainability. Administration and Policy in Mental Health and Mental Health Services Research, 33, 718–729. doi:10.1007/s10488-006-0086-7.

    Article  Google Scholar 

  3. Bond, G. R., McHugo, G. J., Becker, D. R., Rapp, C. A., & Whitley, R. (2008). Fidelity of supported employment: Lessons learned from the national evidence-based practice project. Psychiatric Rehabilitation Journal, 31(4), 300–305. doi:10.2975/31.4.2008.300.305.

    Article  PubMed  Google Scholar 

  4. Botvin, G. J., Baker, E., Dusenbury, L., Botvin, E. M., & Diaz, T. (1995). Long-term follow-up results of a randomized drug abuse prevention trial in a white middle-class population. Journal of the American Medical Association, 273(14), 1106–1112. doi:10.1001/jama.273.14.1106.

    Article  CAS  PubMed  Google Scholar 

  5. Botvin, G. J., Griffin, K. W., Diaz, T., & Ifill-Williams, M. (2001). Drug abuse prevention among minority adolescents: Posttest and one-year follow-up of a school-based preventive intervention. Prevention Science, 2(1), 1–13. doi:10.1023/A:1010025311161.

    Article  CAS  PubMed  Google Scholar 

  6. Brunette, M., Asher, D., Whitley, R., Lutz, W. J., Wieder, B. L., Jones, A. M., et al. (2008). Implementation of integrated dual disorder treatment: A qualitative analysis of facilitators and barriers. Psychiatric Services, 59(9), 989–995. doi:10.1176/appi.ps.59.9.989.

    Article  PubMed  Google Scholar 

  7. Carpinello, S. E., Rosenberg, L., Stone, J., Schwager, M., & Felton, C. J. (2002). New York State’s campaign to implement evidence-based practices for people with serious mental disorders. Psychiatric Services, 53(2), 153–155. doi:10.1176/appi.ps.53.2.153.

    Article  PubMed  Google Scholar 

  8. Dixon, L., McFarlane, W. R., Lefley, H., Lucksted, A., Cohen, M., Falloon, I., et al. (2001). Evidence-based practices for services to families of people with psychiatric disabilities. Psychiatric Services, 52(7), 903–910. doi:10.1176/appi.ps.52.7.903.

    Article  CAS  PubMed  Google Scholar 

  9. Drake, R. E., Becker, D. R., Goldman, H. H., & Martinez, R. A. (2006). The Johnson & Johnson-Dartmouth community mental health program: Disseminating evidence-based practice. Psychiatric Services, 57(3), 302–304. doi:10.1176/appi.ps.57.3.302.

    Article  PubMed  Google Scholar 

  10. Drake, R. E., Goldman, H. H., Leff, H. S., Lehman, F., Dixon, L., Mueser, K., et al. (2001). Implementing evidence-based practices in routine mental health service settings. Psychiatric Services, 52(1), 179–182. doi:10.1176/appi.ps.52.2.179.

    Article  CAS  PubMed  Google Scholar 

  11. Drake, R. E., Merrens, M. R., & Lynde, D. W. (Eds.). (2005). Evidenced-based mental health practice: A textbook (1st ed.). New York: W.W. Norton & Company.

    Google Scholar 

  12. Elliot, D. S., & Mihalic, S. (2004). Issues in disseminating and replicating effective prevention programs. Prevention Science, 5(1), 47–53. doi:10.1023/B:PREV.0000013981.28071.52.

    Article  Google Scholar 

  13. Goldman, H. H., Ganju, V., Drake, R. E., Gorman, P., Hogan, M., Hyde, P. S., et al. (2001). Policy implications for implementing evidence-based practices. Psychiatric Services, 52(12), 1591–1597. doi:10.1176/appi.ps.52.12.1591.

    Article  CAS  PubMed  Google Scholar 

  14. Gottfredson, D. C., & Koper, C. S. (1996). Race and sex differences in the prediction of drug use. Journal of Consulting and Clinical Psychology, 66(2), 305–313. doi:10.1037/0022-006X.64.2.305.

    Article  Google Scholar 

  15. Greenhalgh, T., Glenn, R., McFarlane, F., Bate, P., & Kyriakidou, O. (2004). Diffusion of innovations in service organizations, systematic review and recommendations. The Milbank Quarterly, 82(4), 581–629. doi:10.1111/j.0887-378X.2004.00325.x.

    Article  PubMed  Google Scholar 

  16. Institute of Medicine of the National Academies (2005). Improving the quality of health care for mental and substance use conditions: Quality chasm series. Retrieved from http://www.iom.edu/CMS/3809/19405.30836.aspx.

  17. Isett, K. R., Burnman, M. A., Coleman-Beattie, B., Hyde, P. S., Morrissey, J. P., Magnabosco, J., et al. (2007). Implementation issues for evidence-based practices: The state policy context. Psychiatric Services, 58(7), 914–921. doi:10.1176/appi.ps.58.7.914.

    Article  PubMed  Google Scholar 

  18. Jerrel, J. M., & Ridgely, M. S. (1999). Impact of robustness of program implementation on outcomes of clients in dual diagnosis programs. Psychiatric Services, 50(1), 109–112.

    Google Scholar 

  19. Levine, S., Unutzer, J., Yip, J., Hoffing, M., Leung, M., Fan, M., et al. (2005). Physician satisfaction with a collaborative disease management program for treating late life depression in primary care. General Hospital Psychiatry, 27, 383–391. doi:10.1016/j.genhosppsych.2005.06.001.

    Article  PubMed  Google Scholar 

  20. Magnabosco, J. L. (2006). Innovations in mental health services implementation, a report on state-level data from the US evidence-based practices project. Implementation Science, 1(13), 13. doi:10.1186/1748-5908-1-13.

    Article  PubMed  Google Scholar 

  21. Mancini, A. D., Moser, L. L., Whitley, R., McHugo, G. J., Bond, G. R., Finnerty, M. T., & Burns, B. J. (2009). Assertive community treatment: Facilitators and barriers to implementation in routine mental health settings. Psychiatric Services, 60(2), 189–195. doi:10.1176/appi.ps.60.2.189.

    Google Scholar 

  22. Marshall, T., Rapp, C. A., Becker, D. R., & Bond, G. R. (2008). Key factors for implementing supported employment. Psychiatric Services, 59(8), 886–892. doi:10.1176/appi.ps.59.8.886.

    Article  PubMed  Google Scholar 

  23. Marty, D., Rapp, C., & McHugo, G. J. (2007). Factors influencing consumer outcome monitoring in implementation of evidence-based practices, results from the national EBP implementation project. Administration and Policy in Mental Health. doi:10.1007/s10488-007-0157-4.

  24. Maryland Department of Health and Mental Hygiene (2006). Evidence-based practice in supported employment policy implementation memorandum. Retrieved from http:/www.dhmh.state.md.us.

  25. Massati, R. R., Sweeney, H. A., Panzano, P. C., & Roth, D. (2007). The de-adoption of innovative mental health practices (IMHP): Why organizations choose not to sustain an IMHP. Administration and Policy in Mental Health, 35, 50–65. doi:10.1007/s10488-007-0141-z.

    Google Scholar 

  26. McDonnell, J., Nofs, D., Hardman, J., & Chambless, C. (1989). An analysis of the procedural components of supported employment programs associated with employment outcomes. Journal of Applied Behavior Analysis, 22(4), 417–428. doi:10.1901/jaba.1989.22-417.

    Article  CAS  PubMed  Google Scholar 

  27. McHugo, G. J., Drake, R. E., Teague, G. B., & Xie, H. (1999). Fidelity to assertive community treatment and client outcomes in the New Hampshire dual disorders study. Psychiatric Services, 50(6), 818–824.

    CAS  PubMed  Google Scholar 

  28. McHugo, G. M., Drake, R. E., Whitley, R., Bond, G. R., Campbell, K., Rapp, C. A., et al. (2007). Fidelity outcomes in the national implementing evidence-based practices project. Psychiatric Services, 58(10), 1279–1284.

    Article  PubMed  Google Scholar 

  29. Meredith, L. S., Mendel, P., Pearson, M., Wu, S. Y., Joyce, G., Straus, J. B., et al. (2006). Implementation and maintenance of quality improvement for treating depression in primary care. Psychiatric Services, 57(1), 48–55. doi:10.1176/appi.ps.57.1.48.

    Article  PubMed  Google Scholar 

  30. Miles, M. B., & Huberman, A. M. (1994). Qualitative data analysis: An expanded sourcebook (2nd ed.). Thousand Oaks, CA: Sage.

    Google Scholar 

  31. Mueser, K. T., Torrey, W., Lynde, D., Singer, P., & Drake, R. E. (2003). Implementing evidence-based practices for people with severe mental illness. Behavior Modification, 27(3), 387–411. doi:10.1177/0145445503027003007.

    Article  PubMed  Google Scholar 

  32. Rapp, C. A., Etzel-Wise, D., Marty, D., Coffman, M., Carlson, L., Asher, D., Callaghan, J., & Whitley, R. (2007). Evidence-based practice implementation strategies, results of a qualitative study. Community Mental Health Journal. doi:10.1007/s10597-007-9109-4.

  33. Salyers, M. P., McKasson, M., Bond, G. R., McGrew, J. H., Rollins, A. L., & Boyle, C. (2007). The role of technical assistance centers in implementing evidence-based practices: Lessons learned. American Journal of Psychiatric Rehabilitation, 10(2), 85–101. doi:10.1080/15487760701345968.

    Article  Google Scholar 

  34. Torrey, W. C., Drake, R. E., Dixon, L., Burns, B. J., Flynn, L., Rush, A. J., et al. (2001). Implementing evidence-based treatment for persons with severe mental illnesses. Psychiatric Services, 52(1), 45–50. doi:10.1176/appi.ps.52.1.45.

    Article  CAS  PubMed  Google Scholar 

  35. Unutzer, J., Katon, W. J., Callahan, C. M., Williams, J. W., Hunkeler, E., Harpole, L., et al. (2002). Collaborative care management of late-life depression in primary care: A randomized controlled trial. Journal of the American Medical Association, 288(22), 2836–2845. doi:10.1001/jama.288.22.2836.

    Article  PubMed  Google Scholar 

  36. US Department of Health and Human Services Center for Substance Abuse Prevention (CSAP) (2002). Finding the balance: Program fidelity and adaptation in substance abuse. Rockville, MD. Retrieved from http:/www.samhsa.gov.

  37. US Department of Health and Human Services. The President’s New Freedom Commission on Mental Health (2003). Achieving the promise: Transforming mental health care in America. Rockville, MD: Pub. No. SMA-03-3832.

  38. van Erp, N. H. J., Giesen, F. B. M., van Weeghel, J., Kroon, H., Michon, H. W. C., Becker, D., et al. (2007). A multisite study of implementing supported employment in the Netherlands. Psychiatric Services, 58(11), 1421–1426. doi:10.1176/appi.ps.58.11.1421.

    Article  PubMed  Google Scholar 

  39. Whitley, R., Gingerich, S., Lutz, W., et al. (2009). Implementing the Illness Management and Recovery Program in Community Mental Health Settings: Facilitators and Barriers. Psychiatric Services, 60(2), 209–209. doi:10.1176/appi.ps.60.2.202.

  40. Wieder, B. L., & Kruszynski, R. (2007). The salience of staffing in IDDT implementation: One agency’s experience. American Journal of Psychiatric Rehabilitation, 10(2), 103–112. doi:10.1080/15487760701346016.

    Article  Google Scholar 

  41. Woltmann, E. M., Whitley, R., McHugo, G. J., Brunette, M., Torrey, W. C., Coots, L., et al. (2008). The role of team turnover in implementation of evidence-based practices in mental health care. Psychiatric Services, 59(7), 732–737. doi:10.1176/appi.ps.59.7.732.

    Article  PubMed  Google Scholar 

Download references

Author information

Affiliations

Authors

Corresponding author

Correspondence to Karin Swain.

Additional information

This research was funded by the West Family Foundation.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Swain, K., Whitley, R., McHugo, G.J. et al. The Sustainability of Evidence-Based Practices in Routine Mental Health Agencies. Community Ment Health J 46, 119–129 (2010). https://doi.org/10.1007/s10597-009-9202-y

Download citation

Keywords

  • Evidence-based practice
  • Sustainability
  • Serious mental illness
  • Mixed-methods
  • Services implementation