Skip to main content
SpringerLink
Log in

How Do Treatment Protocols Affect the Use of Engagement Practices in Youth Mental Health Services?

  • Original Article
  • Published:
Administration and Policy in Mental Health and Mental Health Services Research Aims and scope Submit manuscript

Abstract

Purpose

Treatment engagement poses challenges for youth mental health providers. With the expansion of evidence-based treatments (EBTs), providers face complex decisions regarding how to engage youth and families using available information sources. This study investigated how EBT protocols are associated with the selection and delivery of engagement practices.

Method

Twenty engagement practices were coded in a sample of digital recordings of early treatment sessions (N = 193) from the Child STEPs in California study, a randomized trial testing modular treatment and community-implemented treatment for youth mental health problems. Data were collected on which protocols mental health providers reportedly used to guide their sessions and the protocols in which they had received training. We examined which information sources (i.e., the guiding protocol, other protocols in training history, unspecified source) were associated with observed engagement practices.

Results

In sessions guided by a protocol, most observed engagement practices were accounted for by the guiding protocol (p < .001), rather than protocols in training history or unspecified sources (p < .001). In sessions not guided by a protocol, most observed practices were accounted for by training history (p < .001). Practice frequency and extensiveness was generally greater when a protocol guided the session.

Conclusions

Inclusion in protocols is associated with the selection and delivery of engagement practices, but this strategy might be insufficient for supporting the use of the full range of engagement practices supported by evidence. Supports are needed that leverage the engagement evidence base to ensure that selected practices empirically fit the engagement needs of youth and families.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Asarnow, J. R., Jaycox, L. H., Duan, N., LaBorde, A. P., Rea, M. M., Murray, P. … Wells, K. B. (2005). Effectiveness of a quality improvement intervention for adolescent depression in primary care clinics: A randomized controlled trial. Journal Of The American Medical Association, 293(3), 311–319. https://doi.org/10.1001/jama.293.3.311

    Article  CAS  Google Scholar 

  • Barnett, M. L., Brookman-Frazee, L., Gonzalez, J. C., Zhan, C., Rodriguez, A., Stadnick, N. A., & Lau, A. S. (2019). Qualitative reports of how and when therapists adapt children’s evidence-based practices during community implementation. Journal of Clinical Child & Adolescent Psychology, 48(6), 894–905. https://doi.org/10.1080/15374416.2018.1485107

    Article  Google Scholar 

  • Bearman, S. K., Weisz, J. R., Chorpita, B. F., Hoagwood, K., Ward, A., Ugueto, A. M., & Research Network on Youth Mental Health. (2013). More practice, less preach? The role of supervision processes and therapist characteristics in EBP implementation. Administration and Policy in Mental Health and Mental Health Services Research, 40(6), 518–529. https://doi.org/10.1007/s10488-013-0485-5

    Article  Google Scholar 

  • Becker, K. D., Boustani, M., Gellatly, R., & Chorpita, B. F. (2018). Forty years of engagement research in children’s mental health services: Multidimensional measurement and practice elements. Journal of Clinical Child & Adolescent Psychology, 47(1), 1–23. https://doi.org/10.1080/15374416.2017.1326121

    Article  Google Scholar 

  • Becker, K. D., Dickerson, K., Boustani, M. M., & Chorpita, B. F. (2021a). Knowing what to do and when to do it: Mental health professionals and the evidence base for treatment engagement. Administration and Policy in Mental Health and Mental Health Services Research, 48(2), 201–218. https://doi.org/10.1007/s10488-020-01067-6

    Article  Google Scholar 

  • Becker, K. D., Lee, B. R., Daleiden, E. L., Lindsey, M., Brandt, N. E., & Chorpita, B. F. (2015). The common elements of engagement in children’s mental health services: Which elements for which outcomes? Journal of Clinical Child & Adolescent Psychology, 44(1), 30–43. https://doi.org/10.1080/15374416.2013.814543

    Article  Google Scholar 

  • Becker, K. D., Wu, E. G., Hukill, A., Brandt, N., & Chorpita, B. F. (2021b). How do mental health providers assess treatment engagement of youth and caregivers? Journal of Child and Family Studies, 30(10), 2527–2538. https://doi.org/10.1007/s10826-021-02042-x

    Article  Google Scholar 

  • Chacko, A., Jensen, S. A., Lowry, L. S., Cornwell, M., Chimklis, A., Chan, E. … Pulgarin, B. (2016). Engagement in behavioral parent training: Review of the literature and implications for practice. Clinical Child and Family Psychology Review, 19(3), 204–215. https://doi.org/10.1007/s10567-016-0205-2

    Article  Google Scholar 

  • Chen, H., Cohen, P., & Chen, S. (2010). How big is a big odds ratio? Interpreting the magnitudes of odds ratios in epidemiological studies. Communications in Statistics—Simulation and Computation, 39(4), 860–864. https://doi.org/10.1080/03610911003650383

    Article  Google Scholar 

  • Chorpita, B. F., & Becker, K. D. (2022). Dimensions of treatment engagement among youth and caregivers: Structural validity of the REACH framework. Journal of Consulting and Clinical Psychology. https://doi.org/10.1037/ccp0000711

    Article  Google Scholar 

  • Chorpita, B. F., & Daleiden, E. L. (2018). Coordinated strategic action: Aspiring to wisdom in mental health service systems. Clinical Psychology: Science and Practice, 25(4), e12264. https://doi.org/10.1111/cpsp.12264

    Article  Google Scholar 

  • Chorpita, B. F., Daleiden, E. L., Park, A. L., Ward, A. M., Levy, M. C., Cromley, T. … Krull, J. L. (2017). Child STEPs in California: A cluster randomized effectiveness trial comparing modular treatment with community implemented treatment for youth with anxiety, depression, conduct problems, or traumatic stress. Journal of Consulting and Clinical Psychology, 85(1), 13. https://doi.org/10.1037/ccp0000133

    Article  Google Scholar 

  • Chorpita, B. F., Park, A., Tsai, K., Korathu-Larson, P., Higa-McMillan, C. K., Nakamura, B. J. … Krull, J. (2015). Balancing effectiveness with responsiveness: Therapist satisfaction across different treatment designs in the Child STEPs randomized effectiveness trial. Journal of Consulting and Clinical Psychology, 83(4), 709. https://doi.org/10.1037/a0039301

    Article  Google Scholar 

  • Chorpita, B. F., & Weisz, J. R. (2009). Modular approach to therapy for children with anxiety, depression, trauma, or conduct problems. MATCH-ADTC)

  • Cohen, J. A., & Mannarino, A. P. (1996). A treatment outcome study for sexually abused preschool children: Initial findings. Journal of the American Academy of Child & Adolescent Psychiatry, 35(1), 42–50. https://doi.org/10.1097/00004583-199601000-00011

    Article  CAS  Google Scholar 

  • Cooper, J. L., & Aratani, Y. (2009). The status of states’ policies to support evidence-based practices in children’s mental health. Psychiatric Services, 60(12), 1672–1675. https://doi.org/10.1176/ps.2009.60.12.1672

    Article  Google Scholar 

  • De Haan, A. M., Boon, A. E., De Jong, J. T., Hoeve, M., & Vermeiren, R. R. (2013). A meta-analytic review on treatment dropout in child and adolescent outpatient mental health care. Clinical Psychology Review, 33(5), 698–711. https://doi.org/10.1016/j.cpr.2013.04.005

    Article  Google Scholar 

  • Eyberg, S. M., Boggs, S. R., & Algina, J. (1995). Parent-child interaction therapy: A psychosocial model for the treatment of young children with conduct problem behavior and their families. Psychopharmacology Bulletin, 31(1), 83–91

    CAS  Google Scholar 

  • Fisher, R. A. (1992). Statistical Methods for Research Workers. In S. Kotz, & N. L. Johnson (Eds.), Breakthroughs in Statistics (pp. 66–70). New York, NY: Springer. https://doi.org/10.1007/978-1-4612-4380-9_6

    Chapter  Google Scholar 

  • Garland, A. F., Brookman-Frazee, L., Hurlburt, M. S., Accurso, E. C., Zoffness, R. J., Haine-Schlagel, R., & Ganger, W. (2010). Mental health care for children with disruptive behavior problems: A view inside therapists’ offices. Psychiatric Services, 61(8), 788–795. https://doi.org/10.1176/ps.2010.61.8.788

    Article  Google Scholar 

  • Harkin, B., Webb, T. L., Chang, B. P., Prestwich, A., Conner, M., Kellar, I. … Sheeran, P. (2016). Does monitoring goal progress promote goal attainment? A meta-analysis of the experimental evidence. Psychological Bulletin, 142(2), 198. https://doi.org/10.1037/bul0000025

    Article  Google Scholar 

  • Herschell, A. D., Taber-Thomas, S. M., Kolko, D. J., McLeod, B. D., & Jackson, C. B. (2020). Treatment-as-usual for child physical abuse in community mental health centers: therapist characteristics, client profiles, and therapy processes. Journal of Emotional and Behavioral Disorders, 28(4), 223–234. https://doi.org/10.1177/1063426619866188

    Article  Google Scholar 

  • Hogue, A., Liddle, H. A., & Rowe, C. (1996). Treatment adherence process research in family therapy: A rationale and some practical guidelines. Psychotherapy: Theory Research Practice Training, 33(2), 332. https://doi.org/10.1037/0033-3204.33.2.332

    Article  Google Scholar 

  • Kim, J. J., Brookman-Frazee, L., Barnett, M. L., Tran, M., Kuckertz, M., Yu, S., & Lau, A. S. (2020). How community therapists describe adapting evidence‐based practices in sessions for youth: Augmenting to improve fit and reach. Journal of Community Psychology, 48(4), 1238–1257. https://doi.org/10.1002/jcop.22333

    Article  Google Scholar 

  • Koo, T. K., & Li, M. Y. (2016). A guideline for selecting and reporting intraclass correlation coefficients for reliability research. Journal of Chiropractic Medicine, 15, 155–163

    Article  Google Scholar 

  • Kourany, R. F., Garber, J., & Tornusciolo, G. (1990). Improving first appointment attendance rates in child psychiatry outpatient clinics. Journal of the American Academy of Child and Adolescent Psychiatry, 29(4), 657–660. https://doi.org/10.1097/00004583-199007000-00022

    Article  CAS  Google Scholar 

  • Lakind, D., Bradley, W. J., Patel, A., Chorpita, B. F., & Becker, K. D. (2021). A Multidimensional Examination of the Measurement of Treatment Engagement: Implications for Children’s Mental Health Services and Research. Journal of Clinical Child & Adolescent Psychology, 1–16. https://doi.org/10.1080/15374416.2021.1941057

  • Landis, J. R., & Koch, G. G. (1977). An application of hierarchical kappa-type statistics in the assessment of majority agreement among multiple observers. Biometrics, 3, 363–374. https://doi.org/2529786

  • Lieberman, A. F., Van Horn, P., & Ippen, C. G. (2005). Toward evidence-based treatment: Child-parent psychotherapy with preschoolers exposed to marital violence. Journal of the American Academy of Child & Adolescent Psychiatry, 44(12), 1241–1248. https://doi.org/10.1097/01.chi.0000181047.59702.58

    Article  Google Scholar 

  • Martinez, J. I., Lau, A. S., Chorpita, B. F., Weisz, J. R., & Research Network on Youth Mental Health. (2017). Psychoeducation as a mediator of treatment approach on parent engagement in child psychotherapy for disruptive behavior. Journal of Clinical Child & Adolescent Psychology, 46(4), 573–587. https://doi.org/10.1080/15374416.2015.1038826

    Article  Google Scholar 

  • McGraw, K. O., & Wong, S. P. (1996). Forming inferences about some intraclass correlation coefficients. Psychological Methods, 1, 30–46. https://doi.org/10.1037/1082-989X.1.1.30

    Article  Google Scholar 

  • McKay, M., McCadam, K., & Gonzales, J. (1996). Addressing the barriers to mental health services for inner city children and their caretakers. Community Mental Health Journal, 32(4), 353–361. https://doi.org/10.1007/BF02249453

    Article  CAS  Google Scholar 

  • McLeod, B. D. (2011). Relation of the alliance with outcomes in youth psychotherapy: A meta-analysis. Clinical Psychology Review, 31(4), 603–616. https://doi.org/10.1016/j.cpr.2011.02.001

    Article  Google Scholar 

  • McLeod, B. D., Martinez, R. G., Southam-Gerow, M. A., Weisz, J. R., & Chorpita, B. F. (2022). Can a Single Measure Estimate Protocol Adherence for Two Psychosocial Treatments for Youth Anxiety Delivered in Community Mental Health Settings? Behavior Therapy, 53(1), 119–136. https://doi.org/10.1016/j.beth.2021.06.008

    Article  Google Scholar 

  • McLeod, B. D., & Weisz, J. R. (2010). The therapy process observational coding system for child psychotherapy strategies scale. Journal of Clinical Child & Adolescent Psychology, 39(3), 436–443. https://doi.org/10.1080/15374411003691750

    Article  Google Scholar 

  • Meza, R. D., Jungbluth, N., Sedlar, G., Martin, P., Berliner, L., Wiltsey-Stirman, S., & Dorsey, S. (2020). Clinician-reported modification to a CBT approach in children’s mental health. Journal of Emotional and Behavioral Disorders, 28(2), 104–113. https://doi.org/10.1177/1063426619828369

    Article  Google Scholar 

  • Najavits, L. M., Gallop, R. J., & Weiss, R. D. (2006). Seeking safety therapy for adolescent girls with PTSD and substance use disorder: A randomized controlled trial. The Journal of Behavioral Health Services & Research, 33(4), 453–463. https://doi.org/10.1007/s11414-006-9034-2

    Article  Google Scholar 

  • Nock, M. K., & Ferriter, C. (2005). Parent management of attendance and adherence in child and adolescent therapy: A conceptual and empirical review. Clinical Child and Family Psychology Review, 8(2), 149–166. https://doi.org/10.1007/s10567-005-4753-0

    Article  Google Scholar 

  • Nock, M. K., & Kazdin, A. E. (2001). Parent expectancies for child therapy: Assessment and relation to participation in treatment. Journal of Child and Family Studies, 10(2), 155–180. https://doi.org/10.1023/A:1016699424731

    Article  Google Scholar 

  • Nock, M. K., & Kazdin, A. E. (2005). Randomized controlled trial of a brief intervention for increasing participation in parent management training. Journal of Consulting and Clinical Psychology, 73(5), 872. https://doi.org/10.1037/0022-006X.73.5.872

    Article  Google Scholar 

  • Park, A. L., Chorpita, B. F., Regan, J., Weisz, J. R., & Research Network on Youth Mental Health. (2015). Integrity of evidence-based practice: Are providers modifying practice content or practice sequencing? Administration and Policy in Mental Health and Mental Health Services Research, 42(2), 186–196. https://doi.org/10.1007/s10488-014-0559-z

    Article  Google Scholar 

  • Park, A. L., Moskowitz, A. L., & Chorpita, B. F. (2018a). Community-based providers’ selection of practices for children and adolescents with comorbid mental health problems. Journal of Clinical Child & Adolescent Psychology, 47(5), 796–807. https://doi.org/10.1080/15374416.2016.1188706

    Article  Google Scholar 

  • Park, A. L., Tsai, K. H., Guan, K., & Chorpita, B. F. (2018b). Unintended consequences of evidence-based treatment policy reform: Is implementation the goal or the strategy for higher quality care? Administration and Policy in Mental Health and Mental Health Services Research, 45(4), 649–660. https://doi.org/10.1007/s10488-018-0853-2

    Article  Google Scholar 

  • Pellerin, K., Costa, N., Weems, C., & Dalton, R. (2010). An examination of treatment completers and non-completers at a child and adolescent community mental health clinic. Community Mental Health Journal, 46(3), 273–281. https://doi.org/10.1007/s10597-009-9285-5

    Article  Google Scholar 

  • Reding, M. E., Chorpita, B. F., Lau, A. S., & Innes-Gomberg, D. (2014). Providers’ attitudes toward evidence-based practices: Is it just about providers, or do practices matter. too? Administration and Policy in Mental Health and Mental Health Services Research, 41(6), 767–776. https://doi.org/10.1007/s10488-013-0525-1

    Article  Google Scholar 

  • Regan, J., Park, A. L., & Chorpita, B. F. (2019). Choices in treatment integrity: Considering the protocol and consultant recommendations in child and adolescent therapy. Journal of Clinical Child & Adolescent Psychology, 48(sup1), S79–S89. https://doi.org/10.1080/15374416.2016.1212360

    Article  Google Scholar 

  • Rodriguez, A., Kim, J. J., Zhan, C., Lau, A. S., Hamilton, A. B., Palinkas, L. A. … Brookman-Frazee, L. (2021). A mixed-method analysis on the impacts of a system-driven implementation of multiple child evidence-based practices on community mental health providers. Professional Psychology: Research and Practice, 52(1), 67. https://doi.org/10.1037/pro0000353

    Article  Google Scholar 

  • Saloner, B., Carson, N., & Cook, B. L. (2014). Episodes of mental health treatment among a nationally representative sample of children and adolescents. Medical Care Research and Review, 71(3), 261–279. https://doi.org/10.1177/1077558713518347

    Article  Google Scholar 

  • Sanders, M. R., Markie-Dadds, C., Turner, K. M., & Ralph, A. (2004). Using the Triple P system of intervention to prevent behavioural problems in children and adolescents. Handbook of interventions that work with children and adolescents: Prevention and treatment, 489–516. https://doi.org/10.1002/9780470753385.ch20

  • Shirk, S. R., & Karver, M. (2003). Prediction of treatment outcome from relationship variables in child and adolescent therapy: a meta-analytic review. Journal of Consulting and Clinical Psychology, 71(3), 452. https://doi.org/10.1037/0022-006X.71.3.452

    Article  Google Scholar 

  • Shrout, P. E., & Fleiss, J. L. (1979). Intraclass correlations: Uses in assessing rater reliability. Psychological Bulletin, 86, 420–428. https://doi.org/10.1037/0033-2909.86.2.420

    Article  CAS  Google Scholar 

  • Shuman, A. L., & Shapiro, J. P. (2002). The effects of preparing parents for child psychotherapy on accuracy of expectations and treatment attendance. Community Mental Health Journal, 38(1), 3–16. https://doi.org/10.1023/A:1013908629870

    Article  Google Scholar 

  • Southam-Gerow, M. A., Weisz, J. R., Chu, B. C., McLeod, B. D., Gordis, E. B., & Connor-Smith, J. K. (2010). Does cognitive behavioral therapy for youth anxiety outperform usual care in community clinics? An initial effectiveness test. Journal of the American Academy of Child & Adolescent Psychiatry, 49(10), 1043–1052. https://doi.org/10.1016/j.jaac.2010.06.009

    Article  Google Scholar 

  • Stein, B. D., Jaycox, L. H., Kataoka, S. H., Wong, M., Tu, W., Elliott, M. N., & Fink, A. (2003). A mental health intervention for schoolchildren exposed to violence: A randomized controlled trial. Journal Of The American Medical Association, 290(5), 603–611. https://doi.org/10.1001/jama.290.5.603

    Article  Google Scholar 

  • Stewart, R. E., & Chambless, D. L. (2007). Does psychotherapy research inform treatment decisions in private practice? Journal of Clinical Psychology, 63(3), 267–281. https://doi.org/10.1002/jclp.20347

    Article  Google Scholar 

  • Thomas, S. L., & Heck, R. H. (2001). Analysis of large-scale secondary data in higher education research: Potential perils associated with complex sampling designs. Research in Higher Education, 42(5), 517–540. https://doi.org/10.1023/A:1011098109834

    Article  Google Scholar 

  • Verbist, A. N., Winters, A. M., Antle, B. F., & Collins-Camargo, C. (2020). A review of treatment decision-making models and factors in mental health practice. Families in Society, 101(4), 444–455. https://doi.org/10.1177/1044389420921069

    Article  Google Scholar 

  • Ward, A. M., Regan, J., Chorpita, B. F., Starace, N., Rodriguez, A., Okamura, K., & Research Network on Youth Mental Health. (2013). Tracking evidence based practice with youth: Validity of the MATCH and Standard Manual Consultation Records. Journal of Clinical Child & Adolescent Psychology, 42(1), 44–55. https://doi.org/10.1080/15374416.2012.700505

    Article  Google Scholar 

  • Webster-Stratton, C. H., & Reid, M. J. (2011). The Incredible Years Program for children from infancy to pre-adolescence: Prevention and treatment of behavior problems. Clinical handbook of assessing and treating conduct problems in youth (pp. 117–138). New York, NY: Springer

    Chapter  Google Scholar 

  • Weersing, V. R., & Weisz, J. R. (2002). Community clinic treatment of depressed youth: benchmarking usual care against CBT clinical trials. Journal of Consulting and Clinical Psychology, 70(2), 299. https://doi.org/10.1037/0022-006X.70.2.299

    Article  Google Scholar 

  • Weisz, J. R., Chorpita, B. F., Palinkas, L. A., Schoenwald, S. K., Miranda, J., Bearman, S. K., & Research Network on Youth Mental Health. (2012). Testing standard and modular designs for psychotherapy treating depression, anxiety, and conduct problems in youth: A randomized effectiveness trial. Archives of General Psychiatry, 69(3), 274–282. https://doi.org/10.1001/archgenpsychiatry.2011.147

    Article  Google Scholar 

  • Wiltsey-Stirman, S., Gamarra, J. M., Bartlett, B. A., Calloway, A., & Gutner, C. A. (2017). Empirical examinations of modifications and adaptations to evidence-based psychotherapies: Methodologies, impact, and future directions. Clinical Psychology: Science and Practice, 24(4), 396. https://doi.org/10.1037/h0101747

    Article  Google Scholar 

  • Wolk, C. B., Marcus, S. C., Weersing, V. R., Hawley, K. M., Evans, A. C., Hurford, M. O., & Beidas, R. S. (2016). Therapist-and client-level predictors of use of therapy techniques during implementation in a large public mental health system. Psychiatric Services, 67(5), 551–557. https://doi.org/10.1176/appi.ps.201500022

    Article  Google Scholar 

  • Yu, S. H., Brookman-Frazee, L., Kim, J. J., Barnett, M. L., Wright, B., & Lau, A. S. (2021). Therapist adaptations to evidence-based practices and associations with implementation outcomes in child therapy sessions. Journal of Consulting and Clinical Psychology. https://doi.org/10.1037/ccp0000667

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Psychology, University of South Carolina, 1512 Pendleton Street, Barnwell College, Suite #220, 29208, Columbia, SC, USA

    Eleanor G. Wu & Kimberly D. Becker

  2. Judge Baker Children’s Center, 53 Parker Hill Avenue, 02120, Boston, MA, USA

    Rachel E. Kim

  3. Department of Psychology, California State University, 18111 Nordhoff Street, 91330, Northridge, CA, USA

    Jonathan I. Martinez

  4. Department of Psychology, University of California, Los Angeles, Box 951563, 90095, Los Angeles, CA, USA

    Jennifer M. Gamarra & Bruce F. Chorpita

Authors
  1. Eleanor G. Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kimberly D. Becker
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Rachel E. Kim
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jonathan I. Martinez
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jennifer M. Gamarra
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Bruce F. Chorpita
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Eleanor G. Wu.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary Material 1

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wu, E.G., Becker, K.D., Kim, R.E. et al. How Do Treatment Protocols Affect the Use of Engagement Practices in Youth Mental Health Services?. Adm Policy Ment Health 49, 943–961 (2022). https://doi.org/10.1007/s10488-022-01210-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10488-022-01210-5

Keywords

Search

Navigation