Community-Sourced Intervention Programs: Review of Submissions in Response to a Statewide Call for “Promising Practices”
This study was initiated to add to the nascent literature on locally-grown intervention programs in the youth mental health, child welfare, and juvenile justice service sectors, many of which demonstrate practice-based or community-defined evidence, but may not have been subjected to empirical evaluation. Characteristics of applications submitted in response to three public calls for additions to an inventory of research-supported intervention programs were reviewed on evidence for effectiveness, the use of key quality assurance (QA) elements (e.g., clearly specified training or integrity monitoring procedures), and cultural specificity. Findings indicate that four QA processes were identified in approximately half of all submissions: a specific initial training process, the existence of intervention integrity measures, routine outcome monitoring, and ongoing support post-training. An initial training process and integrity measurement were more commonly described among programs determined to have greater research evidence for their effectiveness. Overall, cultural elements were described relatively infrequently and most often reflected surface-level program delivery characteristics (e.g., offering services in languages other than English). Discussion is focused on the alignment of submitted programs with the larger literatures focused on implementation science and cultural competence.
KeywordsEvidence-based practice Policy Implementation science Quality assurance Culture
This publication was made possible in part by funding from Grant number K08 MH095939, awarded to the first author from the National Institute of Mental Health (NIMH). Dr. Lyon is an investigator with the Implementation Research Institute (IRI), at the George Warren Brown School of Social Work, Washington University in St. Louis; through an award from the National Institute of Mental Health (R25 MH080916) and the Department of Veterans Affairs, Health Services Research & Development Service, Quality Enhancement Research Initiative (QUERI).
- Aarons, G. A., Glisson, C., Green, P. D., Hoagwood, K., Kelleher, K. J., Landsverk, J. A., & The Research Network on Youth Mental Health. (2012). The organizational social context of mental health services and clinician attitudes toward evidence-based practice: A United States national study. Implementation Science. doi: 10.1186/1748-5908-7-56.Google Scholar
- Beidas, R.S., Edmunds, J.M., Ditty, M., Watkins, J., Walsh, L., & Kendall, P. (2014). Are inner context factors related to implementation outcomes in cognitive-behavioral therapy for youth anxiety? Administration and Policy in Mental Health. Advance online publication. doi: 10.1007/s10488-013-0529-x.
- Bickman, L., Kelley, S.D., Breda, C., de Andrade, A.R., & Riemer, M. (2011). Effects of routine feedback to clinicians on mental health outcomes of youths: results of a randomized trial. Psychiatry Services (Washington D.C.), 62(12), 1423-1429. doi: 10.1176/appi.ps.002052011.
- Bruns, E.J & Hoagwood, E.K. (2008). State implementation of evidence-based practice for youths, Pt. I: Responses to the state of the evidence. Journal of Child and Adolescent Psychiatry, 47(4), 369–373. doi: 10.1097/CHI.0b013e31816485f4.
- Cardemil, E. V. (2010). Cultural adaptions to empirically supported treatments: a research agenda. Scientific Review of Mental Health Practice, 7(1), 8–21.Google Scholar
- Fixsen, D.L., Naoom, S.F., Blase, K.A., Friedman, R.M., & Wallace, F. (2005). Implementation research: A synthesis of the literature. FMHI Publication #231. Tampa, FL: University of South Florida, Louis de la Parte Florida Mental Health Institute, The National Implementation Research Network.Google Scholar
- Garland, A. F., Haine-Schlagel, R., Brookman-Frazee, L., Baker-Ericzen, M., Trask, E., & Fawley-King, K. (2013). Improving community-based mental health care for children: translating knowledge into action. Administration and Policy In Mental Health, 40(1), 6–22. doi: 10.1007/s10488-012-0450-8.CrossRefPubMedPubMedCentralGoogle Scholar
- Herschell, A. D., Kolko, D. J., Baumann, B. L., & Davis, A. C. (2010). The role of therapist training in the implementation of psychosocial treatments: a review and critique with recommendations. Clinical Psychology Review, 30(4), 448–466. doi: 10.1016/j.cpr.2010.02.005.CrossRefPubMedPubMedCentralGoogle Scholar
- Hoagwood, K. E., Olin, S. S., Horwitz, S., McKay, M., Cleek, A., Gleacher, A., et al. (2014). Scaling up evidence-based practices for children and families in New York State: Toward evidence-based policies on implementation for state mental health systems. Journal of Clinical Child and Adolescent Psychology, 43(2), 145–157.CrossRefPubMedPubMedCentralGoogle Scholar
- Isaacs, L.N., Huang, M., Hernandez, H., & Echo-Hawk (2005). The road to evidence: The intersection of evidence-based practices and cultural competence in children’s mental health. Washington, DC: National Alliance of Multi-ethnic Behavioral Health Associations.Google Scholar
- Isett, K.R., Burnam, M.A., Coleman-Beattie, B., Hyde, P.S., Morrissey, J.P., Magnabosco, J., & Goldman, H.H. (2007). The state policy context of implementation issues for evidence-based practices in mental health. Psychiatry Services (Washington D.C.), 58(7), 914–921. doi: 10.1176/appi.ps.58.7.914
- Lipsey, M. W., Howell, J. C., Kelly, M. R., Chapman, G., & Carver, D. (2010). Improving the effectiveness of juvenile justice programs: A new perspective on evidence-based practice. Washington, DC: Center for Juvenile Justice Reform.Google Scholar
- Lyon, A. R., Lau, A. S., McCauley, E., Vander Stoep, A., & Chorpita, B. F. (2014). A case for modular design: implications for implementing evidence-based interventions with culturally diverse youth. Professional Psychology: Research and Practice, 45(1), 57–66. doi: 10.1037/a0035301.CrossRefGoogle Scholar
- Lyon, A. R., Stirman, S. W., Kerns, S. E., & Bruns, E. J. (2011). Developing the mental health workforce: review and application of training approaches from multiple disciplines. Administration and Policy In Mental Health, 38(4), 238–253. doi: 10.1007/s10488-010-0331-y.CrossRefPubMedPubMedCentralGoogle Scholar
- Marsiglia, F. F., & Kulis, S. (2009). Diversity, oppression and change: Culturally grounded social work. Chicago, IL: Lyccum.Google Scholar
- Martinez, K. (2008). Culturally defined evidence: what is it? And what can it do for Latinos/as?. El Boletín: Newsletter of the National Latino/a Psychological Association, Fall/Winter.Google Scholar
- Martinez, K., Callejas, L., & Hernandez, M. (2010). Community-defined evidence: A bottom-up behavioral health approach to measure what works in communities of color. Report on Emotional and Behavioral Disorders in Youth, 10(1), 11–16.Google Scholar
- Reniscow, K., Baranowski, T., Ahluwalia, J. S., & Braithwaite, R. L. (1999). Cultural sensitivity in public health: defined and demystified. Ethnicity and Disease, 9(1), 10–21.Google Scholar
- Rhoades, B. L., Bumbarger, B. K., & Moore, J. E. (2012). The role of state-level prevention support system in promoting high-quality implementation and sustainability of evidence-based programs. American Journal of Community Psychology, 50(3–4), 386–401. doi: 10.1007/s10464-012-9502-1.CrossRefPubMedGoogle Scholar
- Sanders, M. R & Kirby, J. N. (2014). Surviving or thriving: quality assurance mechanisms to promote innovation in the development of evidence-based parenting interventions. Implementation Science, 1–11. doi: 10.1007/s11121-014-0475-1.
- Schoenwald, S. K., Garland, A. F., Chapman, J. E., Frazier, S. L., Sheidow, A. J., & Southam-Gerow, M. A. (2011). Toward the effective and efficient measurement of implementation fidelity. Administration and Policy in Mental Health and Mental Health Services Research, 38(1), 32–43. doi: 10.1007/s10488-010-0321-0.CrossRefPubMedPubMedCentralGoogle Scholar
- Walker, S.C., Lyon, A.R., Aos, S., & Trupin, E. (under review). The consistencies and vagaries of the Washington State Inventory of Evidence-Based, Research-Based and Promising Practices: The definition of “evidence-based” in a policy context.Google Scholar
- Warren, J. S., Nelson, P. L., Mondragon, S. A., Baldwin, S. A., & Burlingame, G. M. (2010). Youth psychotherapy change trajectories and outcomes in usual care: Community mental health versus managed care settings. Journal of Consulting and Clinical Psychology, 78(2), 144–155. doi: 10.1037/a0018544.CrossRefPubMedGoogle Scholar
- Weisz, J. R., Kuppens, S., Eckshtain, D., Ugueto, A. M., Hawley, K. M., & Jensen-Doss, A. (2013). Performance of evidence-based youth psychotherapies compared with usual clinical care: a multilevel meta-analysis. JAMA Psychiatry, 70(7), 750–761. doi: 10.1001/jamapsychiatry.2013.1176.CrossRefPubMedGoogle Scholar
- Zane, N., Sue, S., Chang, J., Huang, L., Huang, J., Lowe, S., & Lee, E. (2005). Beyond ethnic match: Effects of client–therapist cognitive match in problem perception, coping orientation, and therapy goals on treatment outcomes. Journal of Community Psychology, 33(5), 569–585. doi: 10.1002/jcop.20067.CrossRefGoogle Scholar