Advertisement

Community-Sourced Intervention Programs: Review of Submissions in Response to a Statewide Call for “Promising Practices”

  • Aaron R. Lyon
  • Michael D. Pullmann
  • Sarah Cusworth Walker
  • Gabrielle D’Angelo
Original Paper

Abstract

This study was initiated to add to the nascent literature on locally-grown intervention programs in the youth mental health, child welfare, and juvenile justice service sectors, many of which demonstrate practice-based or community-defined evidence, but may not have been subjected to empirical evaluation. Characteristics of applications submitted in response to three public calls for additions to an inventory of research-supported intervention programs were reviewed on evidence for effectiveness, the use of key quality assurance (QA) elements (e.g., clearly specified training or integrity monitoring procedures), and cultural specificity. Findings indicate that four QA processes were identified in approximately half of all submissions: a specific initial training process, the existence of intervention integrity measures, routine outcome monitoring, and ongoing support post-training. An initial training process and integrity measurement were more commonly described among programs determined to have greater research evidence for their effectiveness. Overall, cultural elements were described relatively infrequently and most often reflected surface-level program delivery characteristics (e.g., offering services in languages other than English). Discussion is focused on the alignment of submitted programs with the larger literatures focused on implementation science and cultural competence.

Keywords

Evidence-based practice Policy Implementation science Quality assurance Culture 

Notes

Acknowledgments

This publication was made possible in part by funding from Grant number K08 MH095939, awarded to the first author from the National Institute of Mental Health (NIMH). Dr. Lyon is an investigator with the Implementation Research Institute (IRI), at the George Warren Brown School of Social Work, Washington University in St. Louis; through an award from the National Institute of Mental Health (R25 MH080916) and the Department of Veterans Affairs, Health Services Research & Development Service, Quality Enhancement Research Initiative (QUERI).

References

  1. Aarons, G. A., Glisson, C., Green, P. D., Hoagwood, K., Kelleher, K. J., Landsverk, J. A., & The Research Network on Youth Mental Health. (2012). The organizational social context of mental health services and clinician attitudes toward evidence-based practice: A United States national study. Implementation Science. doi: 10.1186/1748-5908-7-56.Google Scholar
  2. Aarons, G., Hurlburt, M., & Horwitz, S. (2011). Advancing a conceptual model of evidence-based practice implementation in child welfare. Administration and Policy In Mental Health, 38(1), 4–23. doi: 10.1007/s10488-010-0327-7.CrossRefPubMedGoogle Scholar
  3. Aarons, G. A., & Palinkas, L. (2007). Implementation of evidence-based practice in child welfare: Service provider perspectives. Administration and Policy In Mental Health, 34(4), 411–419. doi: 10.1007/s10488-007-0121-3.CrossRefPubMedGoogle Scholar
  4. Beidas, R. S., Aarons, G., Barg, F., Evans, A., Hadley, T., Hoagwood, K., & Mandell, D. S. (2013). Policy to implementation: evidence-based practice in community mental health—study protocol. Implementation Science,. doi: 10.1186/1748-5908-8-38.CrossRefPubMedPubMedCentralGoogle Scholar
  5. Beidas, R.S., Edmunds, J.M., Ditty, M., Watkins, J., Walsh, L., & Kendall, P. (2014). Are inner context factors related to implementation outcomes in cognitive-behavioral therapy for youth anxiety? Administration and Policy in Mental Health. Advance online publication. doi:  10.1007/s10488-013-0529-x.
  6. Beidas, R. S., Edmunds, J. M., Marcus, S. C., & Kendall, P. C. (2012). Training and consultation to promote implementation of an empirically supported treatment: A randomized trial. Psychiatric Services, 63(7), 660–665. doi: 10.1176/appi.ps.201100401.CrossRefPubMedPubMedCentralGoogle Scholar
  7. Beidas, R. S., & Kendall, P. C. (2010). Training therapists in evidence-based practice: A critical review of studies from a systems-contextual perspective. Clinical Psychology, 17(1), 1–30. doi: 10.1111/j.1468-2850.2009.01187.x.PubMedPubMedCentralGoogle Scholar
  8. Benish, S. G., Quintana, S., & Wampold, B. E. (2011). Culturally adapted psychotherapy and the legitimacy of myth: A direct comparison meta-analysis. Journal of Counseling Psychology, 58(3), 279–289. doi: 10.1037/a0023626.CrossRefPubMedGoogle Scholar
  9. Bernal, G., & Scharrón-del-Río, M. R. (2001). Are empirically supported treatments valid for ethnic minorities? Toward an alternative approach for treatment research. Cultural Diversity and Ethnic Minority Psychology, 7(4), 328–342. doi: 10.1037/1099-9809.7.4.328.CrossRefPubMedGoogle Scholar
  10. Bickman, L., Kelley, S.D., Breda, C., de Andrade, A.R., & Riemer, M. (2011). Effects of routine feedback to clinicians on mental health outcomes of youths: results of a randomized trial. Psychiatry Services (Washington D.C.), 62(12), 1423-1429. doi:  10.1176/appi.ps.002052011.
  11. Bruns, E.J & Hoagwood, E.K. (2008). State implementation of evidence-based practice for youths, Pt. I: Responses to the state of the evidence. Journal of Child and Adolescent Psychiatry, 47(4), 369–373. doi:  10.1097/CHI.0b013e31816485f4.
  12. Bumbarger, B. K., & Campbell, E. M. (2011). A state agency-university partnership for translational research and the dissemination of evidence-based prevention and intervention. Administration and Policy In Mental Health, 39(4), 268–277. doi: 10.1007/s10488-011-0372-x.CrossRefGoogle Scholar
  13. Cardemil, E. V. (2010). Cultural adaptions to empirically supported treatments: a research agenda. Scientific Review of Mental Health Practice, 7(1), 8–21.Google Scholar
  14. Cicchetti, D. V. (1994). Guidelines, criteria, and rules of thumb for evaluating normed and standardized assessment instruments in psychology. Psychological Assessment, 6(4), 284–290. doi: 10.1037/1040-3590.6.4.284.CrossRefGoogle Scholar
  15. Eccles, M. P., & Mittman, B. S. (2006). Welcome to implementation science. Implementation Science,. doi: 10.1186/1748-5908-1-1.Google Scholar
  16. Evans, M. E., Boothroyd, R. A., & Armstrong, M. I. (1997). Development and implementation of an experimental study of the effectiveness of intensive in-home crisis services for children and their families. Journal of Emotional and Behavioral Disorders, 5, 93–105.CrossRefGoogle Scholar
  17. Fixsen, D.L., Naoom, S.F., Blase, K.A., Friedman, R.M., & Wallace, F. (2005). Implementation research: A synthesis of the literature. FMHI Publication #231. Tampa, FL: University of South Florida, Louis de la Parte Florida Mental Health Institute, The National Implementation Research Network.Google Scholar
  18. Garland, A. F., Bickman, L., & Chorpita, B. F. (2010). Change what? Indetidying quality improvement targets by investigating usual mental health care. Administration and Policy In Mental Health, 37(1–2), 15–26. doi: 10.1007/s10488-010-0279-y.CrossRefPubMedPubMedCentralGoogle Scholar
  19. Garland, A. F., Haine-Schlagel, R., Brookman-Frazee, L., Baker-Ericzen, M., Trask, E., & Fawley-King, K. (2013). Improving community-based mental health care for children: translating knowledge into action. Administration and Policy In Mental Health, 40(1), 6–22. doi: 10.1007/s10488-012-0450-8.CrossRefPubMedPubMedCentralGoogle Scholar
  20. Herschell, A. D., Kolko, D. J., Baumann, B. L., & Davis, A. C. (2010). The role of therapist training in the implementation of psychosocial treatments: a review and critique with recommendations. Clinical Psychology Review, 30(4), 448–466. doi: 10.1016/j.cpr.2010.02.005.CrossRefPubMedPubMedCentralGoogle Scholar
  21. Hoagwood, K. E., Olin, S. S., Horwitz, S., McKay, M., Cleek, A., Gleacher, A., et al. (2014). Scaling up evidence-based practices for children and families in New York State: Toward evidence-based policies on implementation for state mental health systems. Journal of Clinical Child and Adolescent Psychology, 43(2), 145–157.CrossRefPubMedPubMedCentralGoogle Scholar
  22. Holleran Steiker, L. K., Castro, F. G., Kumpfer, K., Marsiglia, F. F., Coard, S., & Hopson, L. M. (2008). A dialogue regarding cultural adaption of interventions. Journal of Social Work Practice in the Addictions, 8(1), 154–162. doi: 10.1080/15332560802112094.CrossRefGoogle Scholar
  23. Hsieh, H. F., & Shannon, S. E. (2005). Three approaches to qualitative content analysis. Qualitative Health Research, 15(9), 1277–1288. doi: 10.1177/1049732305276687.CrossRefPubMedGoogle Scholar
  24. Huey, S. J., & Polo, A. J. (2008). Evidence-based psychosocial treatments for ethnic minority youth. Journal of Clinical Child and Adolescent Psychology, 37(1), 262–301. doi: 10.1080/15374410701820174.CrossRefPubMedPubMedCentralGoogle Scholar
  25. Hwang, W. C. (2009). The Formative Method for Adapting Psychotherapy (FMAP): A community-based developmental approach to culturally adapted therapy. Professional Psychology: Research and Practice, 40(4), 369–377. doi: 10.1037/a0016240.CrossRefGoogle Scholar
  26. Isaacs, L.N., Huang, M., Hernandez, H., & Echo-Hawk (2005). The road to evidence: The intersection of evidence-based practices and cultural competence in children’s mental health. Washington, DC: National Alliance of Multi-ethnic Behavioral Health Associations.Google Scholar
  27. Isett, K.R., Burnam, M.A., Coleman-Beattie, B., Hyde, P.S., Morrissey, J.P., Magnabosco, J., & Goldman, H.H. (2007). The state policy context of implementation issues for evidence-based practices in mental health. Psychiatry Services (Washington D.C.), 58(7), 914–921. doi:  10.1176/appi.ps.58.7.914
  28. Jensen, D. R., Abbott, M. K., Beecher, M. E., Griner, D., Golightly, T. R., & Cannon, J. A. N. (2012). Professional Psychology: Research and Practice, 43(4), 388–394. doi: 10.1037/a0029033.CrossRefGoogle Scholar
  29. Jensen-Doss, A., Hawley, K. H., Lopez, M., & Osterberg, L. D. (2009). Using evidence-based treatments: The experiences of youth providers working under a mandate. Professional Psychology: Research and Practice, 40(4), 417–424. doi: 10.1037/a0014690.CrossRefGoogle Scholar
  30. Kendall, P., & Beidas, R. (2007). Smoothing the trail for dissemination of evidence-based practices for youth: Flexibility within fidelity. Professional Psychology and Research Practice, 38(1), 13–20. doi: 10.1037/0735-7028.38.1.13.CrossRefGoogle Scholar
  31. Lambert, M. J., Whipple, J. L., Hawkins, E. J., Vermeersch, D. A., Nielson, S. L., & Smart, D. W. (2003). Is it time for clinicians to routinely track patient outcome? A meta-analysis. Clinical Psychology: Science and Practice, 10(3), 288–301. doi: 10.1093/clipsy.bpg025.Google Scholar
  32. Lau, A. S. (2006). Making the case for selective and directed cultural adaptations of evidence-based treatments: Examples from parent training. Clinical Psychology, 13(4), 295–310. doi: 10.1111/j.1468-2850.2006.00042.x.Google Scholar
  33. Lipsey, M. W., Howell, J. C., Kelly, M. R., Chapman, G., & Carver, D. (2010). Improving the effectiveness of juvenile justice programs: A new perspective on evidence-based practice. Washington, DC: Center for Juvenile Justice Reform.Google Scholar
  34. Lyon, A. R., Lau, A. S., McCauley, E., Vander Stoep, A., & Chorpita, B. F. (2014). A case for modular design: implications for implementing evidence-based interventions with culturally diverse youth. Professional Psychology: Research and Practice, 45(1), 57–66. doi: 10.1037/a0035301.CrossRefGoogle Scholar
  35. Lyon, A. R., Stirman, S. W., Kerns, S. E., & Bruns, E. J. (2011). Developing the mental health workforce: review and application of training approaches from multiple disciplines. Administration and Policy In Mental Health, 38(4), 238–253. doi: 10.1007/s10488-010-0331-y.CrossRefPubMedPubMedCentralGoogle Scholar
  36. Maramba, G., & Nagayama Hall, G. C. (2002). Meta-analyses of ethnic match as a predictor of dropout, utilization, and level of functioning. Cultural Diversity And Ethnic Minority Psychology, 8(3), 290–297. doi: 10.1037/1099-9809.8.3.290.CrossRefPubMedGoogle Scholar
  37. Marsiglia, F. F., & Kulis, S. (2009). Diversity, oppression and change: Culturally grounded social work. Chicago, IL: Lyccum.Google Scholar
  38. Martinez, K. (2008). Culturally defined evidence: what is it? And what can it do for Latinos/as?. El Boletín: Newsletter of the National Latino/a Psychological Association, Fall/Winter.Google Scholar
  39. Martinez, K., Callejas, L., & Hernandez, M. (2010). Community-defined evidence: A bottom-up behavioral health approach to measure what works in communities of color. Report on Emotional and Behavioral Disorders in Youth, 10(1), 11–16.Google Scholar
  40. McHugh, R. K., & Barlow, D. H. (2010). The dissemination and implementation of evidence-based psychological treatments. A review of current efforts. The American Psychologist, 65(2), 73–84. doi: 10.1037/a0018121.CrossRefPubMedGoogle Scholar
  41. Miranda, J., Nakamura, R., & Bernal, G. (2003). Including ethnic minorities in mental health intervention research: A practical approach to a long-standing problem. Culture, Medicine and Psychiatry, 27(4), 467–486. doi: 10.1023/B:MEDI.0000005484.26741.79.CrossRefPubMedGoogle Scholar
  42. Nicolas, G., Arntz, D. L., Hirsch, B., & Schmiedigen, A. (2009). Cultural adaptation of a group treatment for Haitian American adolescents. Professional Psychology: Research and Practice, 40(4), 378–384. doi: 10.1037/a0016307.CrossRefGoogle Scholar
  43. Potter, J. W., & Levine-Donnerstein, D. (1999). Rethinking validity and reliability in content analysis. Journal of Applied Communication Research, 27(3), 258–284. doi: 10.1080/00909889909365539.CrossRefGoogle Scholar
  44. Proctor, E. K., Powell, B. J., & McMilen, J. C. (2013). Implementation strategies: Recommendations for specifying and reporting. Implementation Science, 2013(8), 139. doi: 10.1186/1748-5908-8-139.CrossRefGoogle Scholar
  45. Reniscow, K., Baranowski, T., Ahluwalia, J. S., & Braithwaite, R. L. (1999). Cultural sensitivity in public health: defined and demystified. Ethnicity and Disease, 9(1), 10–21.Google Scholar
  46. Rhoades, B. L., Bumbarger, B. K., & Moore, J. E. (2012). The role of state-level prevention support system in promoting high-quality implementation and sustainability of evidence-based programs. American Journal of Community Psychology, 50(3–4), 386–401. doi: 10.1007/s10464-012-9502-1.CrossRefPubMedGoogle Scholar
  47. Sanders, M. R & Kirby, J. N. (2014). Surviving or thriving: quality assurance mechanisms to promote innovation in the development of evidence-based parenting interventions. Implementation Science, 1–11. doi:  10.1007/s11121-014-0475-1.
  48. Schoenwald, S. K., Garland, A. F., Chapman, J. E., Frazier, S. L., Sheidow, A. J., & Southam-Gerow, M. A. (2011). Toward the effective and efficient measurement of implementation fidelity. Administration and Policy in Mental Health and Mental Health Services Research, 38(1), 32–43. doi: 10.1007/s10488-010-0321-0.CrossRefPubMedPubMedCentralGoogle Scholar
  49. Schoenwald, S. K., Hoagwood, K. E., Atkins, M. S., Evans, M. E., & Ringeisen, H. (2010). Workforce development and the organization of work: the science we need. Administration and Policy In Mental Health, 37(1–2), 71–80. doi: 10.1007/s10488-010-0278-z.CrossRefPubMedPubMedCentralGoogle Scholar
  50. Southam-Gerow, M. A., & McLeod, B. D. (2013). Advances in applying treatment integrity research for dissemination and implementation science: introduction to special issue. Clinical Psychology, 20(1), 1–13. doi: 10.1111/cpsp.12019.PubMedPubMedCentralGoogle Scholar
  51. Swenson, C. C., Schaeffer, C. M., Henggeler, S. W., Faldowski, R., & Mayhew, A. M. (2010). Multisystemic therapy for child abuse and neglect: A randomized effectiveness trial. Journal of Family Psychology, 24(4), 497–507. doi: 10.1037/a0020324.CrossRefPubMedPubMedCentralGoogle Scholar
  52. Walker, S.C., Lyon, A.R., Aos, S., & Trupin, E. (under review). The consistencies and vagaries of the Washington State Inventory of Evidence-Based, Research-Based and Promising Practices: The definition of “evidence-based” in a policy context.Google Scholar
  53. Warren, J. S., Nelson, P. L., Mondragon, S. A., Baldwin, S. A., & Burlingame, G. M. (2010). Youth psychotherapy change trajectories and outcomes in usual care: Community mental health versus managed care settings. Journal of Consulting and Clinical Psychology, 78(2), 144–155. doi: 10.1037/a0018544.CrossRefPubMedGoogle Scholar
  54. Weisz, J. R., Jensen-Doss, A., & Hawley, K. M. (2006). Evidence-based youth psychotherapies versus usual clinical care: a meta-analysis of direct comparisons. The American Psychologist, 61(7), 671–689. doi: 10.1037/0003-066X.61.7.671.CrossRefPubMedGoogle Scholar
  55. Weisz, J. R., Kuppens, S., Eckshtain, D., Ugueto, A. M., Hawley, K. M., & Jensen-Doss, A. (2013). Performance of evidence-based youth psychotherapies compared with usual clinical care: a multilevel meta-analysis. JAMA Psychiatry, 70(7), 750–761. doi: 10.1001/jamapsychiatry.2013.1176.CrossRefPubMedGoogle Scholar
  56. Zane, N., Sue, S., Chang, J., Huang, L., Huang, J., Lowe, S., & Lee, E. (2005). Beyond ethnic match: Effects of client–therapist cognitive match in problem perception, coping orientation, and therapy goals on treatment outcomes. Journal of Community Psychology, 33(5), 569–585. doi: 10.1002/jcop.20067.CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2015

Authors and Affiliations

  • Aaron R. Lyon
    • 1
  • Michael D. Pullmann
    • 1
  • Sarah Cusworth Walker
    • 1
  • Gabrielle D’Angelo
    • 1
  1. 1.Department of Psychiatry and Behavioral SciencesUniversity of WashingtonSeattleUSA

Personalised recommendations