Unintended Consequences of Evidence-Based Treatment Policy Reform: Is Implementation the Goal or the Strategy for Higher Quality Care?
This study examined patterns of evidence-based treatment (EBT) delivery following a county-wide EBT reform initiative. Data were gathered from 60 youth and their 21 providers, who were instructed to deliver therapy as they normally would under the EBT initiative. Results showed limited applicability of county-supported EBTs to this service sample, and that most youth did not receive traditional delivery of EBTs. Findings suggest that it may be unrealistic to expect providers to deliver EBTs with fidelity with all clients, and that EBT implementation may be best thought of as a strategy for improving mental health services rather than a goal.
KeywordsDissemination and implementation Evidence-based treatment Mental health care reform
This work was supported by the John D. and Catherine T. MacArthur Foundation, who did not shape the design or conduct of the study; collection, management, analysis, or interpretation of the data, or preparation, review, or approval of the manuscript.
Compliance with Ethical Standards
Conflict of interest
The authors declare that they have no conflict of interest.
Approval by the institutional review board of the University of California, Los Angeles, as well as those institutional review boards of participating service agencies that requested independent reviews, was obtained for this research. All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards. This article does not contain any studies with animals performed by any of the authors.
Informed consent was obtained from all individual participants included in the study.
- Asarnow, J. R., Jaycox, L. H., Duan, N., LaBorde, A. P., Rea, M. M., Murray, P., … Wells, K. B. (2005). Effectiveness of a quality improvement intervention for adolescent depression in primary care clinics: A randomized controlled trial. JAMA, 293(3), 311–319. https://doi.org/10.1001/jama.293.3.311.CrossRefPubMedGoogle Scholar
- Beidas, R. S., Koerner, K., Weingardt, K. R., & Kendall, P. C. (2011). Training research: Practical recommendations for maximum impact. Administration and Policy in Mental Health and Mental Health Services Research, 38(4), 223–237. https://doi.org/10.1007/s10488-011-0338-z.CrossRefPubMedPubMedCentralGoogle Scholar
- Brookman-Frazee, L., Stadnick, N., Roesch, S., Regan, J., Barnett, M., Bando, L., … Lau, A. (2016). Measuring sustainment of multiple practices fiscally mandated in children’s mental health services. Administration and Policy in Mental Health and Mental Health Services Research, 43(6), 1009–1022. https://doi.org/10.1007/s10488-016-0731-8.CrossRefPubMedPubMedCentralGoogle Scholar
- California Department of Mental Health (2016). Mental Health Services Act expenditure report: Fiscal year 2016–2017. Retrieved from http://www.dhcs.ca.gov/formsandpubs/Documents/Legislative%20Reports/Mental%20Health/MHSAExpend_Report_Jan2016.pdf.
- Chen, H., Reeves, J. H., Fincham, J. E., Kennedy, W. K., & Martin, B. C. (2006). Off-label use of antidepressant, anticonvulsant, and antipsychotic medications among Georgia Medicaid enrollees in 2001. The Journal of Clinical Psychiatry, 67(6), 972–982. Retrieved from http://www.psychiatrist.com/JCP/article/Pages/2006/v67n06/v67n0615.aspx.
- Chorpita, B. F., Bernstein, A., & Daleiden, E. L. (2008). Driving with roadmaps and dashboards: Using information resources to structure the decision models in service organizations. Administration and Policy in Mental Health and Mental Health Services Research, 35(1–2), 114–123. https://doi.org/10.1007/s10488-007-0151-x.CrossRefPubMedGoogle Scholar
- Chorpita, B. F., Bernstein, A., & Daleiden, E. L. (2011). Empirically guided coordination of multiple evidence-based treatments: An illustration of relevance mapping in children’s mental health services. Journal of Consulting and Clinical Psychology, 79(4), 470–480. https://doi.org/10.1037/a0023982.CrossRefPubMedGoogle Scholar
- Chorpita, B. F., Bernstein, A. D., Becker, K. D., Ebesutani, C., Park, A. L., Brown, T., Tsai, K., Moskowitz, A. L., & Daleiden, E. L. (2016). Measurement feedback and beyond: The place for evidence in clinical decision support. In J. Persons (Chair), Monitoring progress in psychotherapy: Why and how. Symposium presented at the 50th Annual Convention of the Association for Behavioral and Cognitive Therapies. New York, NY.Google Scholar
- Chorpita, B. F., Daleiden, E. L., Park, A. L., Ward, A. M., Levy, M. C., Cromley, T., … Krull, J. L. (2017). Child STEPs in California: A cluster randomized effectiveness trial comparing modular treatment with community implemented treatment for youth with anxiety, depression, conduct problems, or traumatic stress. Journal of Consulting and Clinical Psychology, 85(1), 13–25. https://doi.org/10.1037/cco0000133.CrossRefPubMedGoogle Scholar
- Daleiden, E. L., Chorpita, B. F., Donkervoet, C., Arensdorf, A. M., Brogan, M., & Hamilton, J. D. (2006). Getting better at getting them better: Health outcomes and evidence-based practice within a system of care. Journal of the American Academy of Child and Adolescent Psychiatry, 45(6), 749–756. https://doi.org/10.1097/01.chi.0000215154.07142.63.CrossRefPubMedGoogle Scholar
- Damschroder, L. J., Aron, D. C., Keith, R. E., Kirsh, S. R., Alexander, J. A., & Lowery, J. C. (2009). Fostering implementation of health services research findings into practice: A consolidated framework for advancing implementation science. Implementation Science, 4, 50–65. https://doi.org/10.1186/1748-5908-4-50.CrossRefPubMedPubMedCentralGoogle Scholar
- Ebesutani, C., Bernstein, A., Nakamura, B. J., Chorpita, B. F., & Weisz, J. R. (2010). A psychometric analysis of the revised child anxiety and depression scale—parent version in a clinical sample. Journal of Abnormal Child Psychology, 38(2), 249–260. https://doi.org/10.1007/s10802-009-9363-8.CrossRefPubMedGoogle Scholar
- Ebesutani, C., Chorpita, B. F., Higa-McMillan, C. K., Nakamura, B. J., Regan, J., & Lynch, R. E. (2011). A psychometric analysis of the revised child anxiety and depression scales—parent version in a school sample. Journal of Abnormal Child Psychology, 39(2), 173–185. https://doi.org/10.1007/s10802-010-9460-8.CrossRefPubMedGoogle Scholar
- Ebesutani, C., Tottenham, N., & Chorpita, B. (2015). The revised child anxiety and depression scale-parent version: Extended applicability and validity for use with younger youth and children with histories of early-life caregiver neglect. Journal of Psychopathology and Behavioral Assessment, 37(4), 705–718. https://doi.org/10.1007/s10862-015-9494-x.CrossRefGoogle Scholar
- Ehrenreich-May, J., Southam-Gerow, M. A., Hourigan, S. E., Wright, L. R., Pincus, D. B., & Weisz, J. R. (2011). Characteristics of anxious and depressed youth seen in two different clinical contexts. Administration and Policy in Mental Health and Mental Health Services Research, 38(5), 398–411. https://doi.org/10.1007/s10488-010-0328-6.CrossRefPubMedGoogle Scholar
- Garland, A. F., Brookman-Frazee, L., Hurlburt, M. S., Accurso, E. C., Zoffness, R. J., Haine-Schlagel, R., & Ganger, W. (2010). Mental health care for children with disruptive behavior problems: A view inside therapists’ offices. Psychiatric Services, 61(8), 788–795. https://doi.org/10.1176/ps.2010.61.8.788.CrossRefPubMedPubMedCentralGoogle Scholar
- Glisson, C., Schoenwald, S. K., Kelleher, K., Landsverk, J., Hoagwood, K. E., Mayberg, S., & Green, P. (2008). Therapist turnover and new program sustainability in mental health clinics as a function of organizational culture, climate, and service structure. Administration and Policy in Mental Health and Mental Health Services Research, 35(1–2), 124–133. https://doi.org/10.1007/s10488-007-0152-9.CrossRefPubMedGoogle Scholar
- Greenberg, M. T., Mihalic, S. F., & Kusché, C. A. (1998). Promoting alternative thinking strategies (PATHS). Center for the Study and Prevention of Violence, Institute of Behavioral Science, University of Colorado at Boulder.Google Scholar
- Guan, K., Levy, M. C., Kim, R. E., Brown, T. E., Reding, M. E., Rith-Najarian, L., … Chorpita, B. F. (2017). Managing in-session “surprises:” Provider responses to emergent life events during evidence-based treatment implementation. Administration and Policy in Mental Health and Mental Health Services Research, 44(2), 164–176. https://doi.org/10.1007/s10488-015-0692-3.CrossRefPubMedGoogle Scholar
- Lieberman, A. F., Van Horn, P., & Ippen, C. G. (2005). Toward evidence-based treatment: Child-parent psychotherapy with preschoolers exposed to marital violence. Journal of the American Academy of Child and Adolescent Psychiatry, 44(12), 1241–1248. https://doi.org/10.1097/01.chi.0000181047.59702.58.CrossRefPubMedGoogle Scholar
- Los Angeles County Department of Mental Health (2011). Prevention and Early Intervention (PEI) evidence-based practices, promising practices, and community-defined evidence practices: Resource Guide 2.0. Retrieved from http://file.lacounty.gov/SDSInter/dmh/159575_EBPPPCDEResourceGuide2_0.pdf.
- Los Angeles County Department of Mental Health (2013). Prevention and Early Intervention (PEI) evidence-based practices (EBP) outcome measures. Retrieved from http://file.lacounty.gov/SDSInter/dmh/201946_PEI_Outcome_Measures_REV._07-22-13.pdf.
- Los Angeles County Department of Mental Health (2014). Mental Health Services Act: Three year program and expenditure plan: Fiscal years 2014–2015 through 2016–2017. Retrieved from http://file.lacounty.gov/SDSInter/dmh/211892_ThreeYearPlan_072414.pdf.
- Los Angeles County Department of Mental Health (2016a). Countywide data report for PEI programs. Retrieved from http://file.lacounty.gov/SDSInter/dmh/1003274_CW-Report-March-31-2016.pdf.
- Los Angeles County Department of Mental Health (2016b). Training protocols for Prevention and Early Intervention practices. Retrieved from http://file.lacounty.gov/SDSInter/dmh/201947_PEITrainingProtocolsrevised4-1-16.pdf.
- Najavits, L. M., Gallop, R. J., & Weiss, R. D. (2006). Seeking Safety therapy for adolescent girls with PTSD and substance use disorder: A randomized controlled trial. The Journal of Behavioral Health Services and Research, 33(4), 453–463. https://doi.org/10.1007/s11414-006-9034-2.CrossRefPubMedGoogle Scholar
- National Advisory Mental Health Council Workgroup on Child and Adolescent Mental Health Intervention and Deployment. (2001). Blueprint for change: Research on child and adolescent mental health. Rockville, MD: U.S. Department of Health and Human Services, Public Health Service, National Institutes of Health.Google Scholar
- Palinkas, L. A., Weisz, J. R., Chorpita, B. F., Levine, B., Garland, A. F., Hoagwood, K. E., & Landsverk, J. (2013). Continued use of evidence-based treatments after a randomized controlled effectiveness trial: A qualitative study. Psychiatric Services, 64(11), 1110–1118. https://doi.org/10.1176/appi.ps.004682012.CrossRefPubMedGoogle Scholar
- Park, A. L., Moskowitz, A. L., & Chorpita, B. F. (2016). Community-based providers’ selection of practices for children and adolescents with comorbid mental health problems. Journal of Clinical Child and Adolescent Psychology. https://doi.org/10.1080/15374416.2016.1188706.PubMedCentralCrossRefPubMedGoogle Scholar
- Powell, B. J., Beidas, R. S., Rubin, R. M., Stewart, R. E., Wolk, C. B., Matlin, S. L., … Mandell, D. S. (2016). Applying the policy ecology framework to Philadelphia’s behavioral health transformation efforts. Administration and Policy in Mental Health and Mental Health Services Research, 43(6), 909–926. https://doi.org/10.1007/s10488-016-0733-6.CrossRefPubMedPubMedCentralGoogle Scholar
- Reding, M. E., Chorpita, B. F., Lau, A. S., & Innes-Gomberg, D. (2014). Providers’ attitudes toward evidence-based practices: Is it just about providers, or do practices matter, too? Administration and Policy in Mental Health and Mental Health Services Research, 41(6), 767–776. https://doi.org/10.1007/s10488-013-0525-1.CrossRefPubMedPubMedCentralGoogle Scholar
- Reding, M. E. J., Guan, K., Regan, J., Palinkas, L. A., Lau, A. S., & Chorpita, B. F. (2017). Implementation in a changing landscape: Provider experiences during rapid scaling of use of evidence-based treatments. Cognitive and Behavioral Practice. https://doi.org/10.1016/j.cbpra.2017.05.005.CrossRefGoogle Scholar
- Sanders, M. R., Markie-Dadds, C., Turner, K. M., & Ralph, A. (2004). Using the triple P system of intervention to prevent behavioural problems in children and adolescents. In P. Barrett & T. H. Ollendick (Eds.), Handbook of interventions that work with children and adolescents: Prevention and treatment (pp. 489–516). Malden: Wiley Interscience.CrossRefGoogle Scholar
- Southam-Gerow, M. A., Chorpita, B. F., Miller, L. M., & Gleacher, A. A. (2008). Are children with anxiety disorders privately referred to a university clinic like those referred from the public mental health system? Administration and Policy in Mental Health and Mental Health Services Research, 35(3), 168–180. https://doi.org/10.1007/s10488-007-0154-7.CrossRefPubMedGoogle Scholar
- Southam-Gerow, M. A., Daleiden, E. L., Chorpita, B. F., Bae, C., Mitchell, C., Faye, M., & Alba, M. (2014). MAPping Los Angeles County: Taking an evidence-informed model of mental health care to scale. Journal of Clinical Child & Adolescent Psychology, 43(2), 190–200. https://doi.org/10.1080/15374416.2013.833098.CrossRefGoogle Scholar
- Ward, A. M., Regan, J., Chorpita, B. F., Starace, N., Rodriguez, A., & Okamura, K. … Research Network on Youth Mental Health. (2013). Tracking evidence based practice with youth: Validity of the MATCH and standard manual consultation records. Journal of Clinical Child and Adolescent Psychology, 42(1), 44–55. https://doi.org/10.1080/15374416.2012.700505.CrossRefPubMedGoogle Scholar
- Webster-Stratton, C. (2006). The incredible years: A trouble-shooting guide for parents of children aged 2–8 years. Seattle, WA: Incredible Years.Google Scholar
- Weisz, J. R., & Chorpita, B. F. (2012). “Mod squad” for youth psychotherapy: Restructuring evidence-based treatment for clinical practice. In P. C. Kendall (Ed.), Child and adolescent therapy: Cognitive-behavioral procedures (4th edn., pp. 379–397, Chapter xiv, 538 Pages). New York, NY: Guilford Press.Google Scholar
- Weisz, J. R., Chorpita, B. F., Frye, A., Ng, M. Y., Lau, N., Bearman, S. K., … Hoagwood, K. E. (2011). Youth Top Problems: Using idiographic, consumer-guided assessment to identify treatment needs and to track change during psychotherapy. Journal of Consulting and Clinical Psychology, 79(3), 369. https://doi.org/10.1037/a0023307.CrossRefPubMedGoogle Scholar
- Weisz, J. R., Chorpita, B. F., Palinkas, L. A., Schoenwald, S. K., Miranda, J., Bearman, S. K., … Gray, J. (2012). Testing standard and modular designs for psychotherapy treating depression, anxiety, and conduct problems in youth: A randomized effectiveness trial. Archives of General Psychiatry, 69(3), 274–282. https://doi.org/10.1001/archgenpsychiatry.2011.147.CrossRefPubMedGoogle Scholar
- Zima, B. T., Hurlburt, M. S., Knapp, P., Ladd, H., Tang, L., Duan, N., … Wells, K. B. (2005). Quality of publicly-funded outpatient specialty mental health care for common childhood psychiatric disorders in California. Journal of the American Academy of Child and Adolescent Psychiatry, 44(2), 130–144. https://doi.org/10.1097/00004583-200502000-00005.CrossRefPubMedGoogle Scholar