Abstract
Purpose
Treatment engagement poses challenges for youth mental health providers. With the expansion of evidence-based treatments (EBTs), providers face complex decisions regarding how to engage youth and families using available information sources. This study investigated how EBT protocols are associated with the selection and delivery of engagement practices.
Method
Twenty engagement practices were coded in a sample of digital recordings of early treatment sessions (N = 193) from the Child STEPs in California study, a randomized trial testing modular treatment and community-implemented treatment for youth mental health problems. Data were collected on which protocols mental health providers reportedly used to guide their sessions and the protocols in which they had received training. We examined which information sources (i.e., the guiding protocol, other protocols in training history, unspecified source) were associated with observed engagement practices.
Results
In sessions guided by a protocol, most observed engagement practices were accounted for by the guiding protocol (p < .001), rather than protocols in training history or unspecified sources (p < .001). In sessions not guided by a protocol, most observed practices were accounted for by training history (p < .001). Practice frequency and extensiveness was generally greater when a protocol guided the session.
Conclusions
Inclusion in protocols is associated with the selection and delivery of engagement practices, but this strategy might be insufficient for supporting the use of the full range of engagement practices supported by evidence. Supports are needed that leverage the engagement evidence base to ensure that selected practices empirically fit the engagement needs of youth and families.
Similar content being viewed by others
References
Asarnow, J. R., Jaycox, L. H., Duan, N., LaBorde, A. P., Rea, M. M., Murray, P. … Wells, K. B. (2005). Effectiveness of a quality improvement intervention for adolescent depression in primary care clinics: A randomized controlled trial. Journal Of The American Medical Association, 293(3), 311–319. https://doi.org/10.1001/jama.293.3.311
Barnett, M. L., Brookman-Frazee, L., Gonzalez, J. C., Zhan, C., Rodriguez, A., Stadnick, N. A., & Lau, A. S. (2019). Qualitative reports of how and when therapists adapt children’s evidence-based practices during community implementation. Journal of Clinical Child & Adolescent Psychology, 48(6), 894–905. https://doi.org/10.1080/15374416.2018.1485107
Bearman, S. K., Weisz, J. R., Chorpita, B. F., Hoagwood, K., Ward, A., Ugueto, A. M., & Research Network on Youth Mental Health. (2013). More practice, less preach? The role of supervision processes and therapist characteristics in EBP implementation. Administration and Policy in Mental Health and Mental Health Services Research, 40(6), 518–529. https://doi.org/10.1007/s10488-013-0485-5
Becker, K. D., Boustani, M., Gellatly, R., & Chorpita, B. F. (2018). Forty years of engagement research in children’s mental health services: Multidimensional measurement and practice elements. Journal of Clinical Child & Adolescent Psychology, 47(1), 1–23. https://doi.org/10.1080/15374416.2017.1326121
Becker, K. D., Dickerson, K., Boustani, M. M., & Chorpita, B. F. (2021a). Knowing what to do and when to do it: Mental health professionals and the evidence base for treatment engagement. Administration and Policy in Mental Health and Mental Health Services Research, 48(2), 201–218. https://doi.org/10.1007/s10488-020-01067-6
Becker, K. D., Lee, B. R., Daleiden, E. L., Lindsey, M., Brandt, N. E., & Chorpita, B. F. (2015). The common elements of engagement in children’s mental health services: Which elements for which outcomes? Journal of Clinical Child & Adolescent Psychology, 44(1), 30–43. https://doi.org/10.1080/15374416.2013.814543
Becker, K. D., Wu, E. G., Hukill, A., Brandt, N., & Chorpita, B. F. (2021b). How do mental health providers assess treatment engagement of youth and caregivers? Journal of Child and Family Studies, 30(10), 2527–2538. https://doi.org/10.1007/s10826-021-02042-x
Chacko, A., Jensen, S. A., Lowry, L. S., Cornwell, M., Chimklis, A., Chan, E. … Pulgarin, B. (2016). Engagement in behavioral parent training: Review of the literature and implications for practice. Clinical Child and Family Psychology Review, 19(3), 204–215. https://doi.org/10.1007/s10567-016-0205-2
Chen, H., Cohen, P., & Chen, S. (2010). How big is a big odds ratio? Interpreting the magnitudes of odds ratios in epidemiological studies. Communications in Statistics—Simulation and Computation, 39(4), 860–864. https://doi.org/10.1080/03610911003650383
Chorpita, B. F., & Becker, K. D. (2022). Dimensions of treatment engagement among youth and caregivers: Structural validity of the REACH framework. Journal of Consulting and Clinical Psychology. https://doi.org/10.1037/ccp0000711
Chorpita, B. F., & Daleiden, E. L. (2018). Coordinated strategic action: Aspiring to wisdom in mental health service systems. Clinical Psychology: Science and Practice, 25(4), e12264. https://doi.org/10.1111/cpsp.12264
Chorpita, B. F., Daleiden, E. L., Park, A. L., Ward, A. M., Levy, M. C., Cromley, T. … Krull, J. L. (2017). Child STEPs in California: A cluster randomized effectiveness trial comparing modular treatment with community implemented treatment for youth with anxiety, depression, conduct problems, or traumatic stress. Journal of Consulting and Clinical Psychology, 85(1), 13. https://doi.org/10.1037/ccp0000133
Chorpita, B. F., Park, A., Tsai, K., Korathu-Larson, P., Higa-McMillan, C. K., Nakamura, B. J. … Krull, J. (2015). Balancing effectiveness with responsiveness: Therapist satisfaction across different treatment designs in the Child STEPs randomized effectiveness trial. Journal of Consulting and Clinical Psychology, 83(4), 709. https://doi.org/10.1037/a0039301
Chorpita, B. F., & Weisz, J. R. (2009). Modular approach to therapy for children with anxiety, depression, trauma, or conduct problems. MATCH-ADTC)
Cohen, J. A., & Mannarino, A. P. (1996). A treatment outcome study for sexually abused preschool children: Initial findings. Journal of the American Academy of Child & Adolescent Psychiatry, 35(1), 42–50. https://doi.org/10.1097/00004583-199601000-00011
Cooper, J. L., & Aratani, Y. (2009). The status of states’ policies to support evidence-based practices in children’s mental health. Psychiatric Services, 60(12), 1672–1675. https://doi.org/10.1176/ps.2009.60.12.1672
De Haan, A. M., Boon, A. E., De Jong, J. T., Hoeve, M., & Vermeiren, R. R. (2013). A meta-analytic review on treatment dropout in child and adolescent outpatient mental health care. Clinical Psychology Review, 33(5), 698–711. https://doi.org/10.1016/j.cpr.2013.04.005
Eyberg, S. M., Boggs, S. R., & Algina, J. (1995). Parent-child interaction therapy: A psychosocial model for the treatment of young children with conduct problem behavior and their families. Psychopharmacology Bulletin, 31(1), 83–91
Fisher, R. A. (1992). Statistical Methods for Research Workers. In S. Kotz, & N. L. Johnson (Eds.), Breakthroughs in Statistics (pp. 66–70). New York, NY: Springer. https://doi.org/10.1007/978-1-4612-4380-9_6
Garland, A. F., Brookman-Frazee, L., Hurlburt, M. S., Accurso, E. C., Zoffness, R. J., Haine-Schlagel, R., & Ganger, W. (2010). Mental health care for children with disruptive behavior problems: A view inside therapists’ offices. Psychiatric Services, 61(8), 788–795. https://doi.org/10.1176/ps.2010.61.8.788
Harkin, B., Webb, T. L., Chang, B. P., Prestwich, A., Conner, M., Kellar, I. … Sheeran, P. (2016). Does monitoring goal progress promote goal attainment? A meta-analysis of the experimental evidence. Psychological Bulletin, 142(2), 198. https://doi.org/10.1037/bul0000025
Herschell, A. D., Taber-Thomas, S. M., Kolko, D. J., McLeod, B. D., & Jackson, C. B. (2020). Treatment-as-usual for child physical abuse in community mental health centers: therapist characteristics, client profiles, and therapy processes. Journal of Emotional and Behavioral Disorders, 28(4), 223–234. https://doi.org/10.1177/1063426619866188
Hogue, A., Liddle, H. A., & Rowe, C. (1996). Treatment adherence process research in family therapy: A rationale and some practical guidelines. Psychotherapy: Theory Research Practice Training, 33(2), 332. https://doi.org/10.1037/0033-3204.33.2.332
Kim, J. J., Brookman-Frazee, L., Barnett, M. L., Tran, M., Kuckertz, M., Yu, S., & Lau, A. S. (2020). How community therapists describe adapting evidence‐based practices in sessions for youth: Augmenting to improve fit and reach. Journal of Community Psychology, 48(4), 1238–1257. https://doi.org/10.1002/jcop.22333
Koo, T. K., & Li, M. Y. (2016). A guideline for selecting and reporting intraclass correlation coefficients for reliability research. Journal of Chiropractic Medicine, 15, 155–163
Kourany, R. F., Garber, J., & Tornusciolo, G. (1990). Improving first appointment attendance rates in child psychiatry outpatient clinics. Journal of the American Academy of Child and Adolescent Psychiatry, 29(4), 657–660. https://doi.org/10.1097/00004583-199007000-00022
Lakind, D., Bradley, W. J., Patel, A., Chorpita, B. F., & Becker, K. D. (2021). A Multidimensional Examination of the Measurement of Treatment Engagement: Implications for Children’s Mental Health Services and Research. Journal of Clinical Child & Adolescent Psychology, 1–16. https://doi.org/10.1080/15374416.2021.1941057
Landis, J. R., & Koch, G. G. (1977). An application of hierarchical kappa-type statistics in the assessment of majority agreement among multiple observers. Biometrics, 3, 363–374. https://doi.org/2529786
Lieberman, A. F., Van Horn, P., & Ippen, C. G. (2005). Toward evidence-based treatment: Child-parent psychotherapy with preschoolers exposed to marital violence. Journal of the American Academy of Child & Adolescent Psychiatry, 44(12), 1241–1248. https://doi.org/10.1097/01.chi.0000181047.59702.58
Martinez, J. I., Lau, A. S., Chorpita, B. F., Weisz, J. R., & Research Network on Youth Mental Health. (2017). Psychoeducation as a mediator of treatment approach on parent engagement in child psychotherapy for disruptive behavior. Journal of Clinical Child & Adolescent Psychology, 46(4), 573–587. https://doi.org/10.1080/15374416.2015.1038826
McGraw, K. O., & Wong, S. P. (1996). Forming inferences about some intraclass correlation coefficients. Psychological Methods, 1, 30–46. https://doi.org/10.1037/1082-989X.1.1.30
McKay, M., McCadam, K., & Gonzales, J. (1996). Addressing the barriers to mental health services for inner city children and their caretakers. Community Mental Health Journal, 32(4), 353–361. https://doi.org/10.1007/BF02249453
McLeod, B. D. (2011). Relation of the alliance with outcomes in youth psychotherapy: A meta-analysis. Clinical Psychology Review, 31(4), 603–616. https://doi.org/10.1016/j.cpr.2011.02.001
McLeod, B. D., Martinez, R. G., Southam-Gerow, M. A., Weisz, J. R., & Chorpita, B. F. (2022). Can a Single Measure Estimate Protocol Adherence for Two Psychosocial Treatments for Youth Anxiety Delivered in Community Mental Health Settings? Behavior Therapy, 53(1), 119–136. https://doi.org/10.1016/j.beth.2021.06.008
McLeod, B. D., & Weisz, J. R. (2010). The therapy process observational coding system for child psychotherapy strategies scale. Journal of Clinical Child & Adolescent Psychology, 39(3), 436–443. https://doi.org/10.1080/15374411003691750
Meza, R. D., Jungbluth, N., Sedlar, G., Martin, P., Berliner, L., Wiltsey-Stirman, S., & Dorsey, S. (2020). Clinician-reported modification to a CBT approach in children’s mental health. Journal of Emotional and Behavioral Disorders, 28(2), 104–113. https://doi.org/10.1177/1063426619828369
Najavits, L. M., Gallop, R. J., & Weiss, R. D. (2006). Seeking safety therapy for adolescent girls with PTSD and substance use disorder: A randomized controlled trial. The Journal of Behavioral Health Services & Research, 33(4), 453–463. https://doi.org/10.1007/s11414-006-9034-2
Nock, M. K., & Ferriter, C. (2005). Parent management of attendance and adherence in child and adolescent therapy: A conceptual and empirical review. Clinical Child and Family Psychology Review, 8(2), 149–166. https://doi.org/10.1007/s10567-005-4753-0
Nock, M. K., & Kazdin, A. E. (2001). Parent expectancies for child therapy: Assessment and relation to participation in treatment. Journal of Child and Family Studies, 10(2), 155–180. https://doi.org/10.1023/A:1016699424731
Nock, M. K., & Kazdin, A. E. (2005). Randomized controlled trial of a brief intervention for increasing participation in parent management training. Journal of Consulting and Clinical Psychology, 73(5), 872. https://doi.org/10.1037/0022-006X.73.5.872
Park, A. L., Chorpita, B. F., Regan, J., Weisz, J. R., & Research Network on Youth Mental Health. (2015). Integrity of evidence-based practice: Are providers modifying practice content or practice sequencing? Administration and Policy in Mental Health and Mental Health Services Research, 42(2), 186–196. https://doi.org/10.1007/s10488-014-0559-z
Park, A. L., Moskowitz, A. L., & Chorpita, B. F. (2018a). Community-based providers’ selection of practices for children and adolescents with comorbid mental health problems. Journal of Clinical Child & Adolescent Psychology, 47(5), 796–807. https://doi.org/10.1080/15374416.2016.1188706
Park, A. L., Tsai, K. H., Guan, K., & Chorpita, B. F. (2018b). Unintended consequences of evidence-based treatment policy reform: Is implementation the goal or the strategy for higher quality care? Administration and Policy in Mental Health and Mental Health Services Research, 45(4), 649–660. https://doi.org/10.1007/s10488-018-0853-2
Pellerin, K., Costa, N., Weems, C., & Dalton, R. (2010). An examination of treatment completers and non-completers at a child and adolescent community mental health clinic. Community Mental Health Journal, 46(3), 273–281. https://doi.org/10.1007/s10597-009-9285-5
Reding, M. E., Chorpita, B. F., Lau, A. S., & Innes-Gomberg, D. (2014). Providers’ attitudes toward evidence-based practices: Is it just about providers, or do practices matter. too? Administration and Policy in Mental Health and Mental Health Services Research, 41(6), 767–776. https://doi.org/10.1007/s10488-013-0525-1
Regan, J., Park, A. L., & Chorpita, B. F. (2019). Choices in treatment integrity: Considering the protocol and consultant recommendations in child and adolescent therapy. Journal of Clinical Child & Adolescent Psychology, 48(sup1), S79–S89. https://doi.org/10.1080/15374416.2016.1212360
Rodriguez, A., Kim, J. J., Zhan, C., Lau, A. S., Hamilton, A. B., Palinkas, L. A. … Brookman-Frazee, L. (2021). A mixed-method analysis on the impacts of a system-driven implementation of multiple child evidence-based practices on community mental health providers. Professional Psychology: Research and Practice, 52(1), 67. https://doi.org/10.1037/pro0000353
Saloner, B., Carson, N., & Cook, B. L. (2014). Episodes of mental health treatment among a nationally representative sample of children and adolescents. Medical Care Research and Review, 71(3), 261–279. https://doi.org/10.1177/1077558713518347
Sanders, M. R., Markie-Dadds, C., Turner, K. M., & Ralph, A. (2004). Using the Triple P system of intervention to prevent behavioural problems in children and adolescents. Handbook of interventions that work with children and adolescents: Prevention and treatment, 489–516. https://doi.org/10.1002/9780470753385.ch20
Shirk, S. R., & Karver, M. (2003). Prediction of treatment outcome from relationship variables in child and adolescent therapy: a meta-analytic review. Journal of Consulting and Clinical Psychology, 71(3), 452. https://doi.org/10.1037/0022-006X.71.3.452
Shrout, P. E., & Fleiss, J. L. (1979). Intraclass correlations: Uses in assessing rater reliability. Psychological Bulletin, 86, 420–428. https://doi.org/10.1037/0033-2909.86.2.420
Shuman, A. L., & Shapiro, J. P. (2002). The effects of preparing parents for child psychotherapy on accuracy of expectations and treatment attendance. Community Mental Health Journal, 38(1), 3–16. https://doi.org/10.1023/A:1013908629870
Southam-Gerow, M. A., Weisz, J. R., Chu, B. C., McLeod, B. D., Gordis, E. B., & Connor-Smith, J. K. (2010). Does cognitive behavioral therapy for youth anxiety outperform usual care in community clinics? An initial effectiveness test. Journal of the American Academy of Child & Adolescent Psychiatry, 49(10), 1043–1052. https://doi.org/10.1016/j.jaac.2010.06.009
Stein, B. D., Jaycox, L. H., Kataoka, S. H., Wong, M., Tu, W., Elliott, M. N., & Fink, A. (2003). A mental health intervention for schoolchildren exposed to violence: A randomized controlled trial. Journal Of The American Medical Association, 290(5), 603–611. https://doi.org/10.1001/jama.290.5.603
Stewart, R. E., & Chambless, D. L. (2007). Does psychotherapy research inform treatment decisions in private practice? Journal of Clinical Psychology, 63(3), 267–281. https://doi.org/10.1002/jclp.20347
Thomas, S. L., & Heck, R. H. (2001). Analysis of large-scale secondary data in higher education research: Potential perils associated with complex sampling designs. Research in Higher Education, 42(5), 517–540. https://doi.org/10.1023/A:1011098109834
Verbist, A. N., Winters, A. M., Antle, B. F., & Collins-Camargo, C. (2020). A review of treatment decision-making models and factors in mental health practice. Families in Society, 101(4), 444–455. https://doi.org/10.1177/1044389420921069
Ward, A. M., Regan, J., Chorpita, B. F., Starace, N., Rodriguez, A., Okamura, K., & Research Network on Youth Mental Health. (2013). Tracking evidence based practice with youth: Validity of the MATCH and Standard Manual Consultation Records. Journal of Clinical Child & Adolescent Psychology, 42(1), 44–55. https://doi.org/10.1080/15374416.2012.700505
Webster-Stratton, C. H., & Reid, M. J. (2011). The Incredible Years Program for children from infancy to pre-adolescence: Prevention and treatment of behavior problems. Clinical handbook of assessing and treating conduct problems in youth (pp. 117–138). New York, NY: Springer
Weersing, V. R., & Weisz, J. R. (2002). Community clinic treatment of depressed youth: benchmarking usual care against CBT clinical trials. Journal of Consulting and Clinical Psychology, 70(2), 299. https://doi.org/10.1037/0022-006X.70.2.299
Weisz, J. R., Chorpita, B. F., Palinkas, L. A., Schoenwald, S. K., Miranda, J., Bearman, S. K., & Research Network on Youth Mental Health. (2012). Testing standard and modular designs for psychotherapy treating depression, anxiety, and conduct problems in youth: A randomized effectiveness trial. Archives of General Psychiatry, 69(3), 274–282. https://doi.org/10.1001/archgenpsychiatry.2011.147
Wiltsey-Stirman, S., Gamarra, J. M., Bartlett, B. A., Calloway, A., & Gutner, C. A. (2017). Empirical examinations of modifications and adaptations to evidence-based psychotherapies: Methodologies, impact, and future directions. Clinical Psychology: Science and Practice, 24(4), 396. https://doi.org/10.1037/h0101747
Wolk, C. B., Marcus, S. C., Weersing, V. R., Hawley, K. M., Evans, A. C., Hurford, M. O., & Beidas, R. S. (2016). Therapist-and client-level predictors of use of therapy techniques during implementation in a large public mental health system. Psychiatric Services, 67(5), 551–557. https://doi.org/10.1176/appi.ps.201500022
Yu, S. H., Brookman-Frazee, L., Kim, J. J., Barnett, M. L., Wright, B., & Lau, A. S. (2021). Therapist adaptations to evidence-based practices and associations with implementation outcomes in child therapy sessions. Journal of Consulting and Clinical Psychology. https://doi.org/10.1037/ccp0000667
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
About this article
Cite this article
Wu, E.G., Becker, K.D., Kim, R.E. et al. How Do Treatment Protocols Affect the Use of Engagement Practices in Youth Mental Health Services?. Adm Policy Ment Health 49, 943–961 (2022). https://doi.org/10.1007/s10488-022-01210-5
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10488-022-01210-5