Therapist and Organizational Factors Associated with Participation in Evidence-Based Practice Initiatives in a Large Urban Publicly-Funded Mental Health System

  • Laura C. Skriner
  • Courtney Benjamin Wolk
  • Rebecca E. Stewart
  • Danielle R. Adams
  • Ronnie M. Rubin
  • Arthur C. Evans
  • Rinad S. Beidas

In recent years, there has been an emphasis on the implementation of evidence-based practices (EBPs) in practice settings. This focus is due to a growing understanding that EBPs are not reaching the most vulnerable, particularly those served by the public health and mental health systems.[1,2] For example, despite there being over 1200 evidence-based treatments for childhood psychiatric disorders,[3] these treatments are rarely found in public mental health settings.[3,4] Implementation science, the study of methods to promote the systematic uptake of research,[5] was developed to understand how to more effectively implement EBP in community settings. Conceptual frameworks,[6,7] corroborated by empirical evidence,[810] suggest that EBP implementation necessitates an understanding of the individuals (e.g., provider knowledge and attitudes),[1113] organizations (e.g., organizational culture and climate),[14,15] and systems (e.g., financing)[16,17]involved, as well as the particular...



We are especially grateful for the support that the Department of Behavioral Health and Intellectual disAbility Services has provided for this project and for the Evidence Based Practice and Innovation (EPIC) group. We would also like to thank the following experts who provided their consultation on this project: Dr. Steven Marcus and Dr. David Mandell.

Compliance with Ethical Standards

Conflict of Interest

Dr. Beidas receives royalties from Oxford University Press and has served as a consultant for Merck and Kinark Child and Family Services. None of the reported disclosures are related to implementation of evidence-based practices for youth in the City of Philadelphia. The remaining authors (Dr. Skriner, Dr. Benjamin Wolk, Dr. Stewart, Ms. Adams, Dr. Rubin, and Dr. Evans) have no disclosures or conflicts of interest to report.


  1. 1.
    New Freedom Comission on Mental Health. Achieving the promise: Transforming mental health Care in America. Final report. Rockville, MD: U.S. Department of Health and Human Services; 2003.Google Scholar
  2. 2.
    U.S. Department of Health and Human Services. Mental health: A report of the surgeon general. Rockville, MD: National Institute of Mental Health; 1999.Google Scholar
  3. 3.
    Weisz JR, Ng MY, Bearman SK. Odd couple? Reenvisioning the relation between science and practice in the dissemination-implementation era. Clinical Psychology Science 2014;2(1):58–74.CrossRefGoogle Scholar
  4. 4.
    Weisz JR, Jensen-Doss A, Hawley KM. Evidence-based youth psychotherapies versus usual clinical care: A meta-analysis of direct comparisons. Am Psychol 2006;61(7):671–689.CrossRefPubMedGoogle Scholar
  5. 5.
    Eccles MP, Mittman BS. Welcome to implementation science. Implement Sci 2006;1(1):1–1.CrossRefPubMedCentralGoogle Scholar
  6. 6.
    Aarons GA, Hurlburt M, Horwitz SM. Advancing a conceptual model of evidence-based practice implementation in public service sectors. Adm Policy Ment Health Ment Health Serv Res 2011;38(1):4–23.CrossRefGoogle Scholar
  7. 7.
    Damschroder LJ, Aron DC, Keith RE, et al. Fostering implementation of health services research findings into practice: A consolidated framework for advancing implementation science. Implement Sci 2009;4(1):50–64.CrossRefPubMedPubMedCentralGoogle Scholar
  8. 8.
    Beidas RS, Marcus S, Aarons GA, et al. Predictors of community therapists’ use of therapy techniques in a large public mental health system. JAMA Pediatr 2015;169(4):374–382.CrossRefPubMedPubMedCentralGoogle Scholar
  9. 9.
    Glisson C, Hemmelgarn A. The effects of organizational climate and interorganizational coordination on the quality and outcomes of children’s service systems. Child Abuse Negl 1998;22(5):401–421.CrossRefPubMedGoogle Scholar
  10. 10.
    Glisson C, Green P. Organizational climate, services, and outcomes in child welfare systems. Child Abuse Negl 2011;35(8):582–591.CrossRefPubMedGoogle Scholar
  11. 11.
    Henggeler SW, Chapman JE, Rowland MD, et al. Statewide adoption and initial implementation of contingency management for substance-abusing adolescents. J Consult Clin Psychol 2008;76(4):556–567.CrossRefPubMedPubMedCentralGoogle Scholar
  12. 12.
    Beidas RS, Edmunds J, Ditty M, et al. Are inner context factors related to implementation outcomes in cognitive-behavioral therapy for youth anxiety? Adm Policy Ment Health Ment Health Serv Res 2014;41(6):788–799.CrossRefGoogle Scholar
  13. 13.
    Aarons GA. Mental health provider attitudes toward adoption of evidence-based practice: The evidence-based practice Attitude scale (EBPAS). Ment Health Serv Res 2004;6(2):61–74.CrossRefPubMedPubMedCentralGoogle Scholar
  14. 14.
    Glisson C, Dukes D, Green P. The effects of the ARC organizational intervention on caseworker turnover, climate, and culture in children’s service systems. Child Abuse Negl 2006;30(8):855–880.CrossRefPubMedGoogle Scholar
  15. 15.
    Peterson AE, Bond GR, Drake RE, et al. Predicting the long-term sustainability of evidence-based practices in mental health care: An 8-year longitudinal analysis. Journal of Behavioral Health Services & Research. 2014;41(3):337–346.CrossRefGoogle Scholar
  16. 16.
    Raghavan R, Bright CL, Shadoin AL. Toward a policy ecology of implementation of evidence-based practices in public mental health settings. Implement Sci 2008;3(1):26–34.CrossRefPubMedPubMedCentralGoogle Scholar
  17. 17.
    Willging CE, Green AE, Gunderson L, et al. From a "perfect storm" to "smooth sailing": Policymaker perspectives on implementation and sustainment of an evidence-based practice in two states. Child Maltreatment 2015;20(1):24–36.CrossRefPubMedGoogle Scholar
  18. 18.
    Williams JR, Blais MP, Banks D, et al. Predictors of the decision to adopt motivational interviewing in community health settings. The Journal of Behavioral Health Services & Research 2014;41(3):294–307.CrossRefGoogle Scholar
  19. 19.
    Aarons GA, Sawitzky AC. Organizational culture and climate and mental health provider attitudes toward evidence-based practice. Psychol Serv 2006; 3(1): 61–72.CrossRefPubMedPubMedCentralGoogle Scholar
  20. 20.
    Gallo KP, Barlow DH. Factors involved in clinician adoption and nonadoption of evidence-based interventions in mental health. Clinical Psychology: Science & Practice. 2012;19(1):93–106.Google Scholar
  21. 21.
    Stewart RE, Chambless DL, Baron J. Theoretical and practical barriers to practitioners’ willingness to seek training in empirically supported treatments. J Clin Psychol 2012;68(1):8–23.CrossRefPubMedGoogle Scholar
  22. 22.
    Hemmelgarn AL, Glisson C, James LR. Organizational culture and climate: Implications for services and interventions research. Clinical Psychology: Science & Practice. 2006;13(1):73–89.Google Scholar
  23. 23.
    Wang W, Saldana L, Brown CH, et al. Factors that influenced county system leaders to implement an evidence-based program: A baseline survey within a randomized controlled trial. Implement Sci 2010;5(1):72–79.CrossRefPubMedPubMedCentralGoogle Scholar
  24. 24.
    Patterson DA, Dulmus CN, Maguin E. Empirically supported treatment’s impact on organizational culture and climate. Res Soc Work Pract 2012;22(6):665–671.CrossRefGoogle Scholar
  25. 25.
    Beidas RS, Aarons GA, Barg FK, et al. Policy to implementation: Evidence-based practice in community mental health - study protocol. Implement Sci 2013;8(1):38–46.CrossRefPubMedPubMedCentralGoogle Scholar
  26. 26.
    Beidas RS, Stewart RE, Adams DR, et al. A multi-level examination of stakeholder perspectives of implementation of evidence-based practices in a large urban publicly-funded mental health system. Adm Policy Ment Health Ment Health Serv Res 2016;43(6):893–908.CrossRefGoogle Scholar
  27. 27.
    Chorpita BF, Yim LM, Donkervoet JC, et al. Toward large-scale implementation of empirically supported treatments for children: A review and observations by the Hawaii empirical basis to services task force. Clinical Psychology: Science & Practice. 2002;9(2):165–190.Google Scholar
  28. 28.
    Dorsey S, Berliner L, Lyon AR, et al. A statewide common elements initiative for children’s mental health. Journal of Behavioral Health Services & Research 2016;43(2):246–261.CrossRefGoogle Scholar
  29. 29.
    Southam-Gerow MA, Daleiden EL, Chorpita BF, et al. MAPping Los Angeles County: Taking an evidence-informed model of mental health care to scale. Journal of Clinical Child and Adolescent Psychology. 2014;43(2):190–200.CrossRefPubMedGoogle Scholar
  30. 30.
    Glisson C. The organizational context of children’s mental health services. Clin Child Fam Psychol Rev 2002;5(4):233–253.CrossRefPubMedGoogle Scholar
  31. 31.
    Gotham HJ. Diffusion of mental health and substance abuse treatments: Development, dissemination, and implementation. Clinical Psychology: Science & Practice 2004;11(2):160–176.Google Scholar
  32. 32.
    Ehrhart MG, Aarons GA, Farahnak LR. Assessing the organizational context for EBP implementation: The development and validity testing of the implementation climate scale (ICS). Implement Sci 2014;9(1):157–167.CrossRefPubMedPubMedCentralGoogle Scholar
  33. 33.
    Stirman SW, Kimberly J, Cook N, et al. The sustainability of new programs and innovations: A review of the empirical literature and recommendations for future research. Implement Sci 2012;7(1):17–35.CrossRefGoogle Scholar
  34. 34.
    Stirman SW, Spokas M, Creed TA, et al. Training and consultation in evidence-based psychosocial treatments in public mental health settings: The ACCESS model. Professional Psychology: Research & Practice 2010;41(1):48–56.CrossRefGoogle Scholar
  35. 35.
    Beidas RS, Adams DR, Kratz HE, et al. Lessons learned while building a trauma-informed public mental health system in the city of Philadelphia. Evaluation Program and Planning 2016;59:21–32.CrossRefGoogle Scholar
  36. 36.
    Powell BJ, Beidas RS, Rubin RM, et al. Applying the policy ecology framework to Philadelphia’s behavioral health transformation efforts. Adm Policy Ment Health Ment Health Serv Res 2016;43(6):909–926.CrossRefGoogle Scholar
  37. 37.
    Beidas SR, Williams NJ, Green PD, et al. Concordance between administrator and clinician ratings of organizational culture and climate. Administration and Policy in Mental Health and Mental Health Services Research. Available online first, November 5, 2016: 1–10.Google Scholar
  38. 38.
    Aarons GA, Ehrhart MG, Torres EM, et al. The humble leader: Association of discrepancies in leader and follower ratings of implementation leadership with organizational climate in mental health. Psychiatr Serv 2017;68(2):115–122.CrossRefPubMedGoogle Scholar
  39. 39.
    Torres EM, Ehrhart MG, Beidas RS, et al. Validation of the Implementation Leadership Scale (ILS) with supervisors’ self-ratings. Community Mental Health Journal. Available online first, February 8, 2017:1–5.Google Scholar
  40. 40.
    Weisz JR. Therapist background questionnaire. Los Angeles: University of California; 1997.Google Scholar
  41. 41.
    Maslach C, Jackson SE, Leiter MP. Maslach Burnout Inventory Manual. Consulting Psychologists Press; 1996.Google Scholar
  42. 42.
    Maslach C, Schaufeli WB, Leiter MP. Job burnout. Annu Rev Psychol 2001;52:397–422.CrossRefPubMedGoogle Scholar
  43. 43.
    Schaufeli WB, Bakker AB, Hoogduin K, et al. On the clinical validity of the maslach burnout inventory and the burnout measure. Psychol Health 2001;16(5):565–582.CrossRefPubMedGoogle Scholar
  44. 44.
    Glass DC, Mcknight JD. Perceived control, depressive symptomatology, and professional burnout: A review of the evidence. Psychol Health 1996;11(1):23–48.CrossRefGoogle Scholar
  45. 45.
    Aarons GA, Glisson C, Hoagwood KE, et al. Psychometric properties and U.S. national norms of the evidence-based practice Attitude scale (EBPAS). Psychol Assess 2010;22(2):356–365.CrossRefPubMedGoogle Scholar
  46. 46.
    Stumpf RE, Higa-McMillan CK, Chorpita BF. Implementation of evidence-based services for youth assessing provider knowledge. Behav Modif 2009;33(1):48–65.CrossRefPubMedGoogle Scholar
  47. 47.
    Weersing VR, Weisz JR, Donenberg GR. Development of the therapy procedures checklist: A therapist-report measure of technique use in child and adolescent treatment. Journal of Clinical Child and Adolescent Psychology 2002;31(2):168–180.CrossRefPubMedGoogle Scholar
  48. 48.
    Kolko DJ, Cohen JA, Mannarino AP, et al. Community treatment of child sexual abuse: A survey of practitioners in the national child traumatic stress network. Adm Policy Ment Health Ment Health Serv Res 2009;36(1):37–49.CrossRefGoogle Scholar
  49. 49.
    Glisson C, Landsverk JA, Schoenwald S, et al. Assessing the organizational social context (OSC) of mental health services: Implications for research and practice. Adm Policy Ment Health Ment Health Serv Res 2008;35(1–2):98–113.CrossRefGoogle Scholar
  50. 50.
    Glisson C, Green PD, Williams NJ. Assessing the organizational social context (OSC) of child welfare systems: Implications for research and practice. Child Abuse Negl 2012;36(9):621–632.CrossRefPubMedGoogle Scholar
  51. 51.
    Brown RD, Hauenstein NMA. Interrater agreement reconsidered: An alternative to the r(wg) indices. Organ Res Methods 2005;8(2):165–184.CrossRefGoogle Scholar
  52. 52.
    James LR, Demaree RG, Wolf G. Estimating within-group interrater reliability with and without response bias. J Appl Psychol 1984;69(1):85–98.CrossRefGoogle Scholar
  53. 53.
    Bliese PD. Within-group agreement, non-independence, and reliability: Implications for data aggregation and analysis. In: Klein K, Kozlowski S, eds. Multilevel theory, research, and methods in organizations. San Francisco: Jossey-Bass; 2000, pp. 349–380.Google Scholar
  54. 54.
    Aarons GA, Sommerfeld DH, Hecht DB, et al. The impact of evidence-based practice implementation and fidelity monitoring on staff turnover: Evidence for a protective effect. J Consult Clin Psychol 2009;77(2):270–280.CrossRefPubMedPubMedCentralGoogle Scholar
  55. 55.
    Rothman KJ. No adjustments are needed for multiple comparisons. Epidemiology 1990;1(1):43–46.CrossRefPubMedGoogle Scholar
  56. 56.
    Beidas RS, Stewart RE, Wolk CB, et al. Independent contractors in public mental health clinics: Implications for use of evidence-based practices. Psychiatr Serv 2016;67(7):710–717.CrossRefPubMedPubMedCentralGoogle Scholar
  57. 57.
    Birdi K, Allan C, Warr P. Correlates and perceived outcomes of four types of employee development activity. J Appl Psychol 1997;82(6):845–857.CrossRefPubMedGoogle Scholar
  58. 58.
    Beidas RS, Marcus S, Wolk CB, et al. A prospective examination of clinician and supervisor turnover within the context of implementation of evidence-based practices in a publicly-funded mental health system. Adm Policy Ment Health Ment Health Serv Res 2015;43(5):640–649.CrossRefGoogle Scholar
  59. 59.
    Shore LM, Barksdale K, Shore TH. Managerial perceptions of employee commitment to the organization. Acad Manag J 1995;38(6):1593–1615.CrossRefGoogle Scholar
  60. 60.
    Aarons GA. Measuring provider attitudes toward evidence-based practice: Consideration of organizational context and individual differences. Child Adolescent Psychiatric Clinics of North America 2005;14(2):255–271.CrossRefPubMedPubMedCentralGoogle Scholar

Copyright information

© National Council for Behavioral Health 2017

Authors and Affiliations

  • Laura C. Skriner
    • 1
  • Courtney Benjamin Wolk
    • 1
  • Rebecca E. Stewart
    • 1
  • Danielle R. Adams
    • 1
  • Ronnie M. Rubin
    • 2
  • Arthur C. Evans
    • 1
    • 2
  • Rinad S. Beidas
    • 1
  1. 1.Center for Mental Health Policy and Services ResearchUniversity of Pennsylvania Perelman School of MedicinePhiladelphiaUSA
  2. 2.Department of Behavioral Health and Intellectual disAbility ServicesPhiladelphiaUSA

Personalised recommendations