Unintended Consequences of Evidence-Based Treatment Policy Reform: Is Implementation the Goal or the Strategy for Higher Quality Care?

  • Alayna L. Park
  • Katherine H. Tsai
  • Karen Guan
  • Bruce F. Chorpita
Original Article


This study examined patterns of evidence-based treatment (EBT) delivery following a county-wide EBT reform initiative. Data were gathered from 60 youth and their 21 providers, who were instructed to deliver therapy as they normally would under the EBT initiative. Results showed limited applicability of county-supported EBTs to this service sample, and that most youth did not receive traditional delivery of EBTs. Findings suggest that it may be unrealistic to expect providers to deliver EBTs with fidelity with all clients, and that EBT implementation may be best thought of as a strategy for improving mental health services rather than a goal.


Dissemination and implementation Evidence-based treatment Mental health care reform 



This work was supported by the John D. and Catherine T. MacArthur Foundation, who did not shape the design or conduct of the study; collection, management, analysis, or interpretation of the data, or preparation, review, or approval of the manuscript.

Compliance with Ethical Standards

Conflict of interest

The authors declare that they have no conflict of interest.

Ethical Approval

Approval by the institutional review board of the University of California, Los Angeles, as well as those institutional review boards of participating service agencies that requested independent reviews, was obtained for this research. All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards. This article does not contain any studies with animals performed by any of the authors.

Informed Consent

Informed consent was obtained from all individual participants included in the study.


  1. Alexander, J. F., & Robbins, M. S. (2011). Functional family therapy. In A. D. Kidman, T. H. Ollendick & R. C. Murrihy (Eds.), Clinical handbook of assessing and treating conduct problems in youth (pp. 245–271). New York: Springer.CrossRefGoogle Scholar
  2. Asarnow, J. R., Jaycox, L. H., Duan, N., LaBorde, A. P., Rea, M. M., Murray, P., … Wells, K. B. (2005). Effectiveness of a quality improvement intervention for adolescent depression in primary care clinics: A randomized controlled trial. JAMA, 293(3), 311–319.  https://doi.org/10.1001/jama.293.3.311.CrossRefPubMedGoogle Scholar
  3. Beidas, R. S., Koerner, K., Weingardt, K. R., & Kendall, P. C. (2011). Training research: Practical recommendations for maximum impact. Administration and Policy in Mental Health and Mental Health Services Research, 38(4), 223–237.  https://doi.org/10.1007/s10488-011-0338-z.CrossRefPubMedPubMedCentralGoogle Scholar
  4. Brookman-Frazee, L., Stadnick, N., Roesch, S., Regan, J., Barnett, M., Bando, L., … Lau, A. (2016). Measuring sustainment of multiple practices fiscally mandated in children’s mental health services. Administration and Policy in Mental Health and Mental Health Services Research, 43(6), 1009–1022.  https://doi.org/10.1007/s10488-016-0731-8.CrossRefPubMedPubMedCentralGoogle Scholar
  5. California Department of Mental Health (2016). Mental Health Services Act expenditure report: Fiscal year 20162017. Retrieved from http://www.dhcs.ca.gov/formsandpubs/Documents/Legislative%20Reports/Mental%20Health/MHSAExpend_Report_Jan2016.pdf.
  6. Chen, H., Reeves, J. H., Fincham, J. E., Kennedy, W. K., & Martin, B. C. (2006). Off-label use of antidepressant, anticonvulsant, and antipsychotic medications among Georgia Medicaid enrollees in 2001. The Journal of Clinical Psychiatry, 67(6), 972–982. Retrieved from http://www.psychiatrist.com/JCP/article/Pages/2006/v67n06/v67n0615.aspx.
  7. Chorpita, B. F., Bernstein, A., & Daleiden, E. L. (2008). Driving with roadmaps and dashboards: Using information resources to structure the decision models in service organizations. Administration and Policy in Mental Health and Mental Health Services Research, 35(1–2), 114–123.  https://doi.org/10.1007/s10488-007-0151-x.CrossRefPubMedGoogle Scholar
  8. Chorpita, B. F., Bernstein, A., & Daleiden, E. L. (2011). Empirically guided coordination of multiple evidence-based treatments: An illustration of relevance mapping in children’s mental health services. Journal of Consulting and Clinical Psychology, 79(4), 470–480.  https://doi.org/10.1037/a0023982.CrossRefPubMedGoogle Scholar
  9. Chorpita, B. F., Bernstein, A. D., Becker, K. D., Ebesutani, C., Park, A. L., Brown, T., Tsai, K., Moskowitz, A. L., & Daleiden, E. L. (2016). Measurement feedback and beyond: The place for evidence in clinical decision support. In J. Persons (Chair), Monitoring progress in psychotherapy: Why and how. Symposium presented at the 50th Annual Convention of the Association for Behavioral and Cognitive Therapies. New York, NY.Google Scholar
  10. Chorpita, B. F., & Daleiden, E. L. (2014). Structuring the collaboration of science and service in pursuit of a shared vision. Journal of Clinical Child and Adolescent Psychology, 43(2), 323–338.  https://doi.org/10.1080/15374416.2013.828297.CrossRefPubMedGoogle Scholar
  11. Chorpita, B. F., Daleiden, E. L., Park, A. L., Ward, A. M., Levy, M. C., Cromley, T., … Krull, J. L. (2017). Child STEPs in California: A cluster randomized effectiveness trial comparing modular treatment with community implemented treatment for youth with anxiety, depression, conduct problems, or traumatic stress. Journal of Consulting and Clinical Psychology, 85(1), 13–25.  https://doi.org/10.1037/cco0000133.CrossRefPubMedGoogle Scholar
  12. Chorpita, B. F., Moffitt, C. E., & Gray, J. (2005). Psychometric properties of the revised child anxiety and depression scale in a clinical sample. Behaviour Research and Therapy, 43(3), 309–322.  https://doi.org/10.1016/j.brat.2004.02.004.CrossRefPubMedGoogle Scholar
  13. Cohen, J. A., & Mannarino, A. P. (1996). A treatment outcome study for sexually abused preschool children: Initial findings. Journal of the American Academy of Child and Adolescent Psychiatry, 35(1), 42–50.  https://doi.org/10.1097/00004583-199601000-00011.CrossRefPubMedGoogle Scholar
  14. Cooper, J. L., & Aratani, Y. (2009). The status of states’ policies to support evidence-based practices in children’s mental health. Psychiatric Services, 60(12), 1672–1675.  https://doi.org/10.1176/ps.2009.60.12.1672.CrossRefPubMedGoogle Scholar
  15. Daleiden, E. L., Chorpita, B. F., Donkervoet, C., Arensdorf, A. M., Brogan, M., & Hamilton, J. D. (2006). Getting better at getting them better: Health outcomes and evidence-based practice within a system of care. Journal of the American Academy of Child and Adolescent Psychiatry, 45(6), 749–756.  https://doi.org/10.1097/01.chi.0000215154.07142.63.CrossRefPubMedGoogle Scholar
  16. Damschroder, L. J., Aron, D. C., Keith, R. E., Kirsh, S. R., Alexander, J. A., & Lowery, J. C. (2009). Fostering implementation of health services research findings into practice: A consolidated framework for advancing implementation science. Implementation Science, 4, 50–65.  https://doi.org/10.1186/1748-5908-4-50.CrossRefPubMedPubMedCentralGoogle Scholar
  17. Ebesutani, C., Bernstein, A., Nakamura, B. J., Chorpita, B. F., & Weisz, J. R. (2010). A psychometric analysis of the revised child anxiety and depression scale—parent version in a clinical sample. Journal of Abnormal Child Psychology, 38(2), 249–260.  https://doi.org/10.1007/s10802-009-9363-8.CrossRefPubMedGoogle Scholar
  18. Ebesutani, C., Chorpita, B. F., Higa-McMillan, C. K., Nakamura, B. J., Regan, J., & Lynch, R. E. (2011). A psychometric analysis of the revised child anxiety and depression scales—parent version in a school sample. Journal of Abnormal Child Psychology, 39(2), 173–185.  https://doi.org/10.1007/s10802-010-9460-8.CrossRefPubMedGoogle Scholar
  19. Ebesutani, C., Tottenham, N., & Chorpita, B. (2015). The revised child anxiety and depression scale-parent version: Extended applicability and validity for use with younger youth and children with histories of early-life caregiver neglect. Journal of Psychopathology and Behavioral Assessment, 37(4), 705–718.  https://doi.org/10.1007/s10862-015-9494-x.CrossRefGoogle Scholar
  20. Ehrenreich-May, J., Southam-Gerow, M. A., Hourigan, S. E., Wright, L. R., Pincus, D. B., & Weisz, J. R. (2011). Characteristics of anxious and depressed youth seen in two different clinical contexts. Administration and Policy in Mental Health and Mental Health Services Research, 38(5), 398–411.  https://doi.org/10.1007/s10488-010-0328-6.CrossRefPubMedGoogle Scholar
  21. Eyberg, S. M., Boggs, S. R., & Algina, J. (1995). Parent-child interaction therapy: A psychosocial model for the treatment of young children with conduct problem behavior and their families. Psychopharmacology Bulletin, 31(1), 83–91.PubMedGoogle Scholar
  22. Garland, A. F., Brookman-Frazee, L., Hurlburt, M. S., Accurso, E. C., Zoffness, R. J., Haine-Schlagel, R., & Ganger, W. (2010). Mental health care for children with disruptive behavior problems: A view inside therapists’ offices. Psychiatric Services, 61(8), 788–795.  https://doi.org/10.1176/ps.2010.61.8.788.CrossRefPubMedPubMedCentralGoogle Scholar
  23. Glisson, C., Schoenwald, S. K., Kelleher, K., Landsverk, J., Hoagwood, K. E., Mayberg, S., & Green, P. (2008). Therapist turnover and new program sustainability in mental health clinics as a function of organizational culture, climate, and service structure. Administration and Policy in Mental Health and Mental Health Services Research, 35(1–2), 124–133.  https://doi.org/10.1007/s10488-007-0152-9.CrossRefPubMedGoogle Scholar
  24. Goodman, R. (2001). Psychometric properties of the strengths and difficulties questionnaire. Journal of the American Academy of Child and Adolescent Psychiatry, 40(11), 1337–1345.  https://doi.org/10.1097/00004583-200111000-00015.CrossRefPubMedGoogle Scholar
  25. Greenberg, M. T., Mihalic, S. F., & Kusché, C. A. (1998). Promoting alternative thinking strategies (PATHS). Center for the Study and Prevention of Violence, Institute of Behavioral Science, University of Colorado at Boulder.Google Scholar
  26. Guan, K., Levy, M. C., Kim, R. E., Brown, T. E., Reding, M. E., Rith-Najarian, L., … Chorpita, B. F. (2017). Managing in-session “surprises:” Provider responses to emergent life events during evidence-based treatment implementation. Administration and Policy in Mental Health and Mental Health Services Research, 44(2), 164–176.  https://doi.org/10.1007/s10488-015-0692-3.CrossRefPubMedGoogle Scholar
  27. Gyani, A., Shafran, R., Myles, P., & Rose, S. (2014). The gap between science and practice: How therapists make their clinical decisions. Behavior Therapy, 45(2), 199–211.  https://doi.org/10.1016/j.beth.2013.10.004.CrossRefPubMedGoogle Scholar
  28. Hawley, K. M., & Weisz, J. R. (2002). Increasing the relevance of evidence-based treatment review to practitioners and consumers. Clinical Psychology: Science and Practice, 9(2), 225–230.  https://doi.org/10.1093/clipsy.9.2.225.Google Scholar
  29. Huey, S. J. Jr., & Polo, A. J. (2008). Evidence-based psychosocial treatments for ethnic minority youth. Journal of Clinical Child and Adolescent Psychology, 37(1), 262–301.  https://doi.org/10.1080/15374410701820174.CrossRefPubMedPubMedCentralGoogle Scholar
  30. Lewinsohn, P. M., Clarke, G. N., Hops, H., & Andrews, J. (1990). Cognitive-behavioral treatment for depressed adolescents. Behavior Therapy, 21(4), 385–401.  https://doi.org/10.1016/S0005-7894(05)80353-3.CrossRefGoogle Scholar
  31. Lieberman, A. F., Van Horn, P., & Ippen, C. G. (2005). Toward evidence-based treatment: Child-parent psychotherapy with preschoolers exposed to marital violence. Journal of the American Academy of Child and Adolescent Psychiatry, 44(12), 1241–1248.  https://doi.org/10.1097/01.chi.0000181047.59702.58.CrossRefPubMedGoogle Scholar
  32. Los Angeles County Department of Mental Health (2011). Prevention and Early Intervention (PEI) evidence-based practices, promising practices, and community-defined evidence practices: Resource Guide 2.0. Retrieved from http://file.lacounty.gov/SDSInter/dmh/159575_EBPPPCDEResourceGuide2_0.pdf.
  33. Los Angeles County Department of Mental Health (2013). Prevention and Early Intervention (PEI) evidence-based practices (EBP) outcome measures. Retrieved from http://file.lacounty.gov/SDSInter/dmh/201946_PEI_Outcome_Measures_REV._07-22-13.pdf.
  34. Los Angeles County Department of Mental Health (2014). Mental Health Services Act: Three year program and expenditure plan: Fiscal years 2014–2015 through 2016–2017. Retrieved from http://file.lacounty.gov/SDSInter/dmh/211892_ThreeYearPlan_072414.pdf.
  35. Los Angeles County Department of Mental Health (2016a). Countywide data report for PEI programs. Retrieved from http://file.lacounty.gov/SDSInter/dmh/1003274_CW-Report-March-31-2016.pdf.
  36. Los Angeles County Department of Mental Health (2016b). Training protocols for Prevention and Early Intervention practices. Retrieved from http://file.lacounty.gov/SDSInter/dmh/201947_PEITrainingProtocolsrevised4-1-16.pdf.
  37. McHugh, R. K., & Barlow, D. H. (2010). The dissemination and implementation of evidence-based psychological treatments: A review of current efforts. American Psychologist, 65(2), 73–84.  https://doi.org/10.1037/a0018121.CrossRefPubMedGoogle Scholar
  38. Najavits, L. M., Gallop, R. J., & Weiss, R. D. (2006). Seeking Safety therapy for adolescent girls with PTSD and substance use disorder: A randomized controlled trial. The Journal of Behavioral Health Services and Research, 33(4), 453–463.  https://doi.org/10.1007/s11414-006-9034-2.CrossRefPubMedGoogle Scholar
  39. National Advisory Mental Health Council Workgroup on Child and Adolescent Mental Health Intervention and Deployment. (2001). Blueprint for change: Research on child and adolescent mental health. Rockville, MD: U.S. Department of Health and Human Services, Public Health Service, National Institutes of Health.Google Scholar
  40. Palinkas, L. A., Weisz, J. R., Chorpita, B. F., Levine, B., Garland, A. F., Hoagwood, K. E., & Landsverk, J. (2013). Continued use of evidence-based treatments after a randomized controlled effectiveness trial: A qualitative study. Psychiatric Services, 64(11), 1110–1118.  https://doi.org/10.1176/appi.ps.004682012.CrossRefPubMedGoogle Scholar
  41. Park, A. L., Moskowitz, A. L., & Chorpita, B. F. (2016). Community-based providers’ selection of practices for children and adolescents with comorbid mental health problems. Journal of Clinical Child and Adolescent Psychology.  https://doi.org/10.1080/15374416.2016.1188706.PubMedCentralGoogle Scholar
  42. Powell, B. J., Beidas, R. S., Rubin, R. M., Stewart, R. E., Wolk, C. B., Matlin, S. L., … Mandell, D. S. (2016). Applying the policy ecology framework to Philadelphia’s behavioral health transformation efforts. Administration and Policy in Mental Health and Mental Health Services Research, 43(6), 909–926.  https://doi.org/10.1007/s10488-016-0733-6.CrossRefPubMedPubMedCentralGoogle Scholar
  43. Radley, D. C., Finkelstein, S. N., & Stafford, R. S. (2006). Off-label prescribing among office-based physicians. Archives of Internal Medicine, 166(9), 1021–1026.  https://doi.org/10.1001/archinte.166.9.1021.CrossRefPubMedGoogle Scholar
  44. Raghavan, R., Bright, C. L., & Shadoin, A. L. (2008). Toward a policy ecology of implementation of evidence-based practices in public mental health settings. Implementation Science, 3(1), 26–35.  https://doi.org/10.1186/1748-5908-3-26.CrossRefPubMedPubMedCentralGoogle Scholar
  45. Reding, M. E., Chorpita, B. F., Lau, A. S., & Innes-Gomberg, D. (2014). Providers’ attitudes toward evidence-based practices: Is it just about providers, or do practices matter, too? Administration and Policy in Mental Health and Mental Health Services Research, 41(6), 767–776.  https://doi.org/10.1007/s10488-013-0525-1.CrossRefPubMedPubMedCentralGoogle Scholar
  46. Reding, M. E. J., Guan, K., Regan, J., Palinkas, L. A., Lau, A. S., & Chorpita, B. F. (2017). Implementation in a changing landscape: Provider experiences during rapid scaling of use of evidence-based treatments. Cognitive and Behavioral Practice.  https://doi.org/10.1016/j.cbpra.2017.05.005.Google Scholar
  47. Rotheram-Borus, M. J., Swendeman, D., & Chorpita, B. F. (2012). Disruptive innovations for designing and diffusing evidence-based interventions. American Psychologist, 67(6), 463–476.  https://doi.org/10.1037/a0028180.CrossRefPubMedPubMedCentralGoogle Scholar
  48. Sanders, M. R., Markie-Dadds, C., Turner, K. M., & Ralph, A. (2004). Using the triple P system of intervention to prevent behavioural problems in children and adolescents. In P. Barrett & T. H. Ollendick (Eds.), Handbook of interventions that work with children and adolescents: Prevention and treatment (pp. 489–516). Malden: Wiley Interscience.CrossRefGoogle Scholar
  49. Shah, S. S., Hall, M., Goodman, D. M., Feuer, P., Sharma, V., Fargason, C., … Levin, J. E. (2007). Off-label drug use in hospitalized children. Archives of Pediatrics and Adolescent Medicine, 161(3), 282–290.  https://doi.org/10.1001/archpedi.161.3.282.CrossRefPubMedGoogle Scholar
  50. Southam-Gerow, M. A., Chorpita, B. F., Miller, L. M., & Gleacher, A. A. (2008). Are children with anxiety disorders privately referred to a university clinic like those referred from the public mental health system? Administration and Policy in Mental Health and Mental Health Services Research, 35(3), 168–180.  https://doi.org/10.1007/s10488-007-0154-7.CrossRefPubMedGoogle Scholar
  51. Southam-Gerow, M. A., Daleiden, E. L., Chorpita, B. F., Bae, C., Mitchell, C., Faye, M., & Alba, M. (2014). MAPping Los Angeles County: Taking an evidence-informed model of mental health care to scale. Journal of Clinical Child & Adolescent Psychology, 43(2), 190–200.  https://doi.org/10.1080/15374416.2013.833098.CrossRefGoogle Scholar
  52. Stein, B. D., Jaycox, L. H., Kataoka, S. H., Wong, M., Tu, W., Elliott, M. N., & Fink, A. (2003). A mental health intervention for schoolchildren exposed to violence: A randomized controlled trial. JAMA, 290(5), 603–611.  https://doi.org/10.1001/jama.290.5.603.CrossRefPubMedGoogle Scholar
  53. Steinberg, A. M., Brymer, M. J., Decker, K. B., & Pynoos, R. S. (2004). The University of California at Los Angeles post-traumatic stress disorder reaction index. Current Psychiatry Reports, 6(2), 96–100.  https://doi.org/10.1007/s11920-004-0048-2.CrossRefPubMedGoogle Scholar
  54. Ward, A. M., Regan, J., Chorpita, B. F., Starace, N., Rodriguez, A., & Okamura, K. … Research Network on Youth Mental Health. (2013). Tracking evidence based practice with youth: Validity of the MATCH and standard manual consultation records. Journal of Clinical Child and Adolescent Psychology, 42(1), 44–55.  https://doi.org/10.1080/15374416.2012.700505.CrossRefPubMedGoogle Scholar
  55. Webster-Stratton, C. (2006). The incredible years: A trouble-shooting guide for parents of children aged 2–8 years. Seattle, WA: Incredible Years.Google Scholar
  56. Weisz, J. R., & Chorpita, B. F. (2012). “Mod squad” for youth psychotherapy: Restructuring evidence-based treatment for clinical practice. In P. C. Kendall (Ed.), Child and adolescent therapy: Cognitive-behavioral procedures (4th edn., pp. 379–397, Chapter xiv, 538 Pages). New York, NY: Guilford Press.Google Scholar
  57. Weisz, J. R., Chorpita, B. F., Frye, A., Ng, M. Y., Lau, N., Bearman, S. K., … Hoagwood, K. E. (2011). Youth Top Problems: Using idiographic, consumer-guided assessment to identify treatment needs and to track change during psychotherapy. Journal of Consulting and Clinical Psychology, 79(3), 369.  https://doi.org/10.1037/a0023307.CrossRefPubMedGoogle Scholar
  58. Weisz, J. R., Chorpita, B. F., Palinkas, L. A., Schoenwald, S. K., Miranda, J., Bearman, S. K., … Gray, J. (2012). Testing standard and modular designs for psychotherapy treating depression, anxiety, and conduct problems in youth: A randomized effectiveness trial. Archives of General Psychiatry, 69(3), 274–282.  https://doi.org/10.1001/archgenpsychiatry.2011.147.CrossRefPubMedGoogle Scholar
  59. Wittich, C. M., Burkle, C. M., & Lanier, W. L. (2012). Ten common questions (and their answers) about off-label drug use. Mayo Clinic Proceedings, 87(10), 982–990.  https://doi.org/10.1016/j.mayocp.2012.04.017.CrossRefPubMedPubMedCentralGoogle Scholar
  60. Zima, B. T., Hurlburt, M. S., Knapp, P., Ladd, H., Tang, L., Duan, N., … Wells, K. B. (2005). Quality of publicly-funded outpatient specialty mental health care for common childhood psychiatric disorders in California. Journal of the American Academy of Child and Adolescent Psychiatry, 44(2), 130–144.  https://doi.org/10.1097/00004583-200502000-00005.CrossRefPubMedGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  • Alayna L. Park
    • 1
  • Katherine H. Tsai
    • 2
  • Karen Guan
    • 1
  • Bruce F. Chorpita
    • 1
  1. 1.Department of PsychologyUniversity of CaliforniaLos AngelesUSA
  2. 2.Five Acres – The Boys’ and Girls’ Aid Society of Los Angeles CountyLos AngelesUSA

Personalised recommendations