Evidence-Based Practice Implementation in Community Mental Health Settings: The Relative Importance of Key Domains of Implementation Activity

  • William C. Torrey
  • Gary R. Bond
  • Gregory J. McHugo
  • Karin Swain
Original Paper


Implementation research has examined practice prioritization, implementation leadership, workforce development, workflow re-engineering, and practice reinforcement, but not addressed their relative importance as implementation drivers. This study investigated domains of implementation activities and correlated them to implementation success during a large national evidence-based practice implementation project. Implementation success was correlated with active leadership strategically devoted to redesigning the flow of work and reinforcing implementation through measurement and feedback. Relative attention to workforce development was negatively correlated with implementation. Active leaders should focus on redesigning the flow of work to support the implementation and on reinforcing program improvements.


Practice implementation Severe mental illness Evidence-based practice 



This research was supported by grant 036805 from the Robert Wood Johnson Foundation; contract 280-02-8070 from the Center for Mental Health Services, Substance Abuse and Mental Health Services Administration; a gift from the West Family Foundation; and the Johnson & Johnson–Dartmouth Community Mental Health Program. The authors thank the members of the National Implementing Evidence-Based Practices Project team and the participants at the sites in the eight states.


  1. Aarons, G. A. (2004). Mental health provider attitudes toward adoption of evidence-based practice: The Evidence-Based Practice Attitude Scale (EBPAS). Mental Health Services Research, 6, 61–74.PubMedCrossRefGoogle Scholar
  2. Aarons, G. A. (2006). Transformational and transactional leadership: Association with attitudes toward evidence-based practice. Psychiatric Services, 57, 1162–1169.PubMedCrossRefGoogle Scholar
  3. Atlas.ti 2.0 (2002) Atlas.ti 2.0 [Computer software]. Berlin, Germany: GmbH.Google Scholar
  4. Batalden, P. B., & Stoltz, P. K. (1993). A framework for the continual improvement of health care: Building and applying professional and improvement knowledge to test changes in daily work. Joint Commission Journal on Quality Improvement, 19, 424–445.PubMedGoogle Scholar
  5. Berwick, D. M. (2003). Disseminating innovations in health care. Journal of the American Medical Association, 289, 1969–1975.PubMedCrossRefGoogle Scholar
  6. Bodenheimer, T. (1999). The American health care system—the movement for improved quality in health care. New England Journal of Medicine, 340, 488–492.PubMedCrossRefGoogle Scholar
  7. Bond, G. R., Becker, D. R., & Drake, R. E. (2011). Measurement of fidelity of implementation of evidence-based practices: Case example of the IPS Fidelity Scale. Clinical Psychology: Science and Practice, 18, 125–140.CrossRefGoogle Scholar
  8. Bond, G. R., Drake, R. E., McHugo, G. J., Rapp, C. A., & Whitley, R. (2009). Strategies for improving fidelity in the National Evidence-Based Practices Project. Research on Social Work Practice, 19, 569–581.CrossRefGoogle Scholar
  9. Bond, G. R., Evans, L., Salyers, M. P., Williams, J., & Kim, H. K. (2000). Measurement of fidelity in psychiatric rehabilitation. Mental Health Services Research, 2, 75–87.PubMedCrossRefGoogle Scholar
  10. Brunette, M. F., Asher, D., Whitley, R., Lutz, W. J., Wieder, B. L., Jones, A. M., et al. (2008). Implementation of integrated dual disorders treatment: A qualitative analysis of facilitators and barriers. Psychiatric Services, 59, 989–995.PubMedCrossRefGoogle Scholar
  11. Chambers, D. A. (2008). Advancing the science of implementation: A workshop summary. Administration and Policy in Mental Health and Mental Health Services Research, 35, 3–10.PubMedCrossRefGoogle Scholar
  12. Corrigan, P. W., Lickey, S. E., Campion, J., & Rashid, F. (2000). Mental health team leadership and consumers satisfaction and quality of life. Psychiatric Services, 51, 781–785.PubMedCrossRefGoogle Scholar
  13. Davis, D. A., Thomson, M. A., Oxman, A. D., & Haynes, R. B. (1995). Changing physician performance. A systematic review of the effect of continuing medical education strategies. Journal of the American Medical Association, 274, 700–705.PubMedCrossRefGoogle Scholar
  14. Drake, R. E., & Essock, S. M. (2009). The science-to-service gap in real-world schizophrenia treatment: The 95% problem. Schizophrenia Bulletin, 35, 677–678.PubMedCrossRefGoogle Scholar
  15. Drake, R. E., Goldman, H. H., Leff, H. S., Lehman, A. F., Dixon, L., Mueser, K. T., et al. (2001). Implementing evidence-based practices in routine mental health service settings. Psychiatric Services, 52, 179–182.PubMedCrossRefGoogle Scholar
  16. Durlak, J. A., & DuPre, E. P. (2011). Implementation matters: A review of research on the influence of implementation on program outcomes and the factors affecting implementation. American Journal of Community Psychology, 41, 327–350.CrossRefGoogle Scholar
  17. Finnerty, M. T., Rapp, C. A., Bond, G. R., Lynde, D. W., Ganju, V. J., & Goldman, H. H. (2009). The State Health Authority Yardstick (SHAY). Community Mental Health Journal, 45, 228–236.PubMedCrossRefGoogle Scholar
  18. Fixsen, D. L., Naoom, S. F., Blase, K. A., Friedman, R. M., & Wallace, F. (2005). Implementation research: A synthesis of the literature. Tampa, FL: University of South Florida.Google Scholar
  19. Forsetlund, L., Bjorndal, A., Rashidian, A., Jamtvedt, G., O’Brien, M. A., Wolf, F., et al. (2009). Continuing education meetings and workshops: Effects on professional practice and health care outcomes. Cochrane Database of Systematic Reviews (Online), 2(2), CD003030. doi: 10.1002/14651858.CD14003030.pub14651852.
  20. Godfrey, J. L. (2010). Re-implementing assertive community treatment: One agency’s challenge of meeting state standards. Unpublished dissertation. Indianapolis: Indiana University-Purdue University Indianapolis.Google Scholar
  21. Grimshaw, J. M., & Eccles, M. P. (2004). Is evidence-based implementation of evidence-based care possible? Medical Journal of Australia, 180(6 Suppl), S50–S51.PubMedGoogle Scholar
  22. Grol, R. (1997). Personal paper: Beliefs and evidence in changing clinical practice. BMJ, 315, 418–421.PubMedCrossRefGoogle Scholar
  23. Grol, R., & Grimshaw, J. M. (2003). From best evidence to best practice: Effective implementation of change in patients’ care. Lancet, 362, 1225–1230.PubMedCrossRefGoogle Scholar
  24. Hermann, R. C., Chan, J. A., Zazzali, J. L., & Lerner, D. (2006). Aligning measurement-based quality improvement with implementation of evidence-based practices. Administration and Policy in Mental Health and Mental Health Services Research, 34, 636–645.CrossRefGoogle Scholar
  25. Hoffman, L. R. (1979). The group problem solving process: Studies of a valence model. New York: Praeger.Google Scholar
  26. Hoge, M. A., Tondora, J., & Stuart, G. W. (2003). Training in evidence-based practice. Psychiatric Clinics of North America, 26, 851–865.PubMedCrossRefGoogle Scholar
  27. Insel, T. R. (2007). Devising prevention and treatment strategies for the nation’s diverse populations with mental illness. Psychiatric Services, 58, 395.PubMedCrossRefGoogle Scholar
  28. Institute of Medicine. (2001). Crossing the quality chasm: A new health system for the 21st century. Washington, DC: National Academy Press.Google Scholar
  29. Jamtvedt, G., Young, J. M., Kristoffersen, D. T., O’Brien, M. A., & Oxman, A. D. (2006). Audit and feedback: Effects on professional practice and health care outcomes. Cochrane Database of Systematic Reviews (Online), 2(2). doi: 10.1002/14651858.CD14000259.pub14651852.
  30. Joyce, B., & Showers, B. (2002). Student achievement through staff development (3rd ed.). Alexandria, VA: Association for Supervision and Curriculum Development.Google Scholar
  31. Kimberly, J., & Cook, J. M. (2008). Organizational measurement and the implementation of innovations in mental health services. Administration and Policy in Mental Health and Mental Health Services Research, 35, 11–20.PubMedCrossRefGoogle Scholar
  32. McFarlane, W. R., McNary, S., Dixon, L., Hornby, H., & Cimett, E. (2001). Predictors of dissemination of family psychoeducation in community mental health centers in Maine and Illinois. Psychiatric Services, 50, 818–824.Google Scholar
  33. McGrew, J. H., Bond, G. R., Dietzen, L. L., & Salyers, M. P. (1994). Measuring fidelity of implementation of a mental health program model. Journal of Consulting and Clinical Psychology, 62, 670–678.PubMedCrossRefGoogle Scholar
  34. McHugo, G. J., Drake, R. E., Teague, G. B., & Xie, H. (1999). Fidelity to assertive community treatment and client outcomes in the New Hampshire Dual Disorders Study. Psychiatric Services, 50, 818–824.PubMedGoogle Scholar
  35. McHugo, G. J., Drake, R. E., Whitley, R., Bond, G. R., Campbell, K., Rapp, C. A., et al. (2007). Fidelity outcomes in the National Implementing Evidence-Based Practices Project. Psychiatric Services, 58, 1279–1284.PubMedCrossRefGoogle Scholar
  36. Mendel, P., Meredith, L. S., Schoenbaum, M., Sherbourne, C. D., & Wells, K. B. (2008). Interventions in organizational and community context: A framework for building evidence on dissemination and implementation in health services research. Administration and Policy in Mental Health and Mental Health Services Research, 35, 21–37.PubMedCrossRefGoogle Scholar
  37. New Freedom Commission on Mental Health. (2003). Achieving the promise: Transforming mental health care in America. Final Report. DHHS Pub. No. SMA-03-3832. Rockville, MD: Substance Abuse and Mental Health Services Administration.Google Scholar
  38. O’Brien, M. A., Rogers, S., Jamtvedt, G., Oxman, A. D., Odgaard-Jensen, J., Kristoffersen, D. T., et al. (2007). Educational outreach visits: Effects on professional practice and health care outcomes. Cochrane Database of Systematic Reviews (Online), 4(4):CD000409. doi: 10.1002/14651858.CD14000409.pub14651852.
  39. Proctor, E. K., Landsverk, J., Aarons, G., Chambers, D., Glisson, C., & Mittman, B. (2009). Implementation research in mental health services: An emerging science with conceptual, methodological, and training challenges. Administration and Policy in Mental Health and Mental Health Services Research, 36, 24–34.PubMedCrossRefGoogle Scholar
  40. Rapp, C. A., & Goscha, R. J. (2006). The strengths model: Case management with people with psychiatric diabilities (2nd ed.). New York: Oxford University Press.Google Scholar
  41. Rapp, C. A., Goscha, R. J., & Carlson, L. S. (2010). Evidence-based practice implementation in Kansas. Community Mental Health Journal, 46, 461–465.PubMedCrossRefGoogle Scholar
  42. Rollins, A. L., Salyers, M. P., Tsai, J., & Lydick, J. M. (2010). Staff turnover in statewide implementation of ACT: Relationship with ACT fidelity and other team characteristics. Administration and Policy in Mental Health and Mental Health Services Research, 37, 417–426.PubMedCrossRefGoogle Scholar
  43. Rosenheck, R. (2001). Organizational process: A missing link between research and practice. Psychiatric Services, 52, 1607–1612.PubMedCrossRefGoogle Scholar
  44. Schoenwald, S. K., Garland, A. F., Chapman, J. E., Frazier, S. L., Sheidow, A. J., & Southam-Gerow, M. A. (2010a). Toward the effective and efficient measurement of implementation fidelity. Administration and Policy in Mental Health and Mental Health Services Research, 37, doi: 10.1007/s10488-10010-10321-10480.
  45. Schoenwald, S. K., Hoagwood, K. E., Atkins, M. S., Evans, M. E., & Ringeisen, H. (2010b). Workforce development and the organization of work: The science we need. Administration and Policy in Mental Health and Mental Health Services Research, 37, 71–80.PubMedCrossRefGoogle Scholar
  46. Seffrin, B., Panzano, P. C., & Roth, D. (2008). What gets noticed: How barrier and facilitator perceptions relate to the adoption and implementation of innovative mental health practices. Community Mental Health Journal, 44, 475–484.PubMedCrossRefGoogle Scholar
  47. Shojania, K. G., & Grimshaw, J. M. (2005). Evidence-based quality improvement: The state of the science. Health Affairs, 24, 138–150.PubMedCrossRefGoogle Scholar
  48. Shortell, S. M., Bennett, C. L., & Byck, G. R. (1998). Assessing the impact of continuous quality improvement on clinical practice: What it will take to accelerate progress. Milbank Quarterly, 76, 593–624.PubMedCrossRefGoogle Scholar
  49. Siskind, D., & Wiley-Exley, E. (2009). Comparison of assertive community treatment programs in urban Massachusetts and rural North Carolina. Administration and Policy in Mental Health and Mental Health Services Research, 36, 236–246.PubMedCrossRefGoogle Scholar
  50. Sluyter, G. V. (1995). Mental health leadership training: A survey of state directors. Journal of Mental Health Administration, 22, 201–204.PubMedCrossRefGoogle Scholar
  51. Surgeon General. (2000). Surgeon General’s report on mental health. Washington, DC: US Government Printing Office.Google Scholar
  52. Teague, G. B., Bond, G. R., & Drake, R. E. (1998). Program fidelity in assertive community treatment: Development and use of a measure. American Journal of Orthopsychiatry, 68, 216–232.PubMedCrossRefGoogle Scholar
  53. Thomson O’Brien, M. A., Oxman, A. D., Haynes, R. B., Davis, D. A., Freemantle, N., & Harvey, E. L. (2000). Local opinion leaders: Effects on professional practice and health care outcomes. Cochrane Database of Systematic Reviews (Online), 2(2), CD000125. doi: 10.1002/14651858.CD000125.
  54. Torrey, W. C., Drake, R. E., Dixon, L., Burns, B. J., Rush, A. J., Clark, R. E., et al. (2001). Implementing evidence-based practices for persons with severe mental illness. Psychiatric Services, 52, 45–50.PubMedCrossRefGoogle Scholar
  55. Torrey, W. C., Lynde, D. W., & Gorman, P. (2005). Promoting the implementation of practices that are supported by research: The National Implementing Evidence-Based Practice Project. Child and Adolescent Psychiatric Clinics of North America, 14, 297–306.PubMedCrossRefGoogle Scholar
  56. Wang, M. C., Hyun, J. K., Harrison, M. I., Shortell, S. M., & Fraser, I. (2006). Redesigning health systems for quality: Lessons from emerging practices. Joint Commission Journal on Quality and Patient Safety, 11, 599–611.Google Scholar
  57. West, J. C., Wilk, J. E., Olfson, M., Rae, D. S., Marcus, S., Narrow, W. E., et al. (2005). Patterns and quality of treatment for patients with schizophrenia in routine psychiatric practice. Psychiatric Services, 56, 283–291.PubMedCrossRefGoogle Scholar
  58. Whitley, R., Gingerich, S., & Lutz, W. (2009). Facilitators and barriers to the implementation of the illness management and recovery program in community mental health settings. Psychiatric Services, 60, 202–209.PubMedCrossRefGoogle Scholar
  59. Woltmann, E. M., Whitley, R., McHugo, G. J., Brunette, M., Torrey, W. C., Coots, L., et al. (2008). The role of staff turnover in the implementation of evidence-based practices in mental health care. Psychiatric Services, 59, 732–737.PubMedCrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2011

Authors and Affiliations

  • William C. Torrey
    • 1
    • 3
  • Gary R. Bond
    • 1
  • Gregory J. McHugo
    • 1
    • 2
  • Karin Swain
    • 4
  1. 1.Department of PsychiatryDartmouth Medical School, Dartmouth Psychiatric Research CenterLebanonUSA
  2. 2.Department of Community & Family MedicineDartmouth Medical School, Dartmouth Psychiatric Research CenterLebanonUSA
  3. 3.Department of PsychiatryDartmouth-Hitchcock Medical CenterLebanonUSA
  4. 4.Dartmouth Psychiatric Research CenterLebanonUSA

Personalised recommendations