Skip to main content

Advertisement

SpringerLink
Log in

Measuring Evidence-Based Treatment Strategies in Youth Community Mental Health Care: the Evidence-Based Strategies Scale

  • Published:
The Journal of Behavioral Health Services & Research Aims and scope Submit manuscript

Abstract

While there are established measures for fidelity to specific evidence-based treatments, there is no widely accepted, feasible measure of the use of evidence-based treatment strategies in youth mental health (MH) care. This study examined the factor structure of a provider self-report measure of evidence-based treatment strategy use, the Evidence-Based Strategies Scale (EBSS). MH providers completed the EBSS as part of a larger mailed survey. The factor structure of the EBSS was examined using exploratory factor analysis in a national, multidisciplinary sample of MH providers (N = 1092), and confirmatory factor analysis was subsequently conducted to replicate this factor structure in a state-wide, multidisciplinary sample of Medicaid MH providers (N = 780). Findings indicated a three-factor structure, representing working alliance, youth-focused, and family-focused evidence-based treatment strategies factors. The EBSS offers a potential method for measuring the evidence-based treatment strategies being delivered in youth community MH care, though more research is needed.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Balas EA, Boren SA. Managing Clinical Knowledge for Health Care Improvement. In: Bemmel J, McCray AT (Eds). Yearbook of Medical Informatics 2000: Patient-Centered Systems. Stuttgart, Germany: Schattauer Verlagsgesellschaft mbH,2000,pp. 65-70.

    Google Scholar 

  2. Grant J, Green L, Mason B. Basic research and health: A reassessment of the scientific basis for the support of biomedical science. Research Evaluation. 2003;12(3):217-224. https://doi.org/10.3152/147154403781776618

    Article  Google Scholar 

  3. Institute of Medicine Committee on Quality of Health Care in America. Crossing the Quality Chasm: A New Health System for the 21st Century. Washington, DC: National Academy Press, 2001.

  4. Dorsey S, Berliner L, Lyon AR, et al. A statewide common elements initiative for children’s mental health. The Journal of Behavioral Health Services & Research. 2016;43(2):246-261. https://doi.org/10.1007/s11414-014-9430-y

    Article  Google Scholar 

  5. Jensen-Doss A, Hawley KM, Lopez M, et al. Using evidence-based treatments: The experiences of youth providers working under a mandate. Professional Psychology: Research and Practice. 2009;40(4):417-424. https://doi.org/10.1037/a0014690

    Article  Google Scholar 

  6. Martinez RG, Lewis CC, Weiner BJ. Instrumentation issues in implementation science. Implementation Science. 2014;9(1):118. https://doi.org/10.1186/s13012-014-0118-8

    Article  PubMed  PubMed Central  Google Scholar 

  7. Borntrager CF, Chorpita BF, Orimoto T, et al. Validity of clinician’s self-reported practice elements on the monthly treatment and progress summary. Journal of Behavioral Health Services & Research. 2015;42(3):367-382. https://doi.org/10.1007/s11414-013-9363-x

    Article  Google Scholar 

  8. Falendar CA, Shafranske EP. Psychotherapy-based supervision models in an emerging competency-based era: A commentary. Psychotherapy: Theory, Research, Practice, Training. 2010;47(1):45–50. https://doi.org/10.1037/a0018873

  9. Muse K, McManus F. A systematic review of methods for assessing competence in cognitive-behavioral therapy. Clinical Psychology Review. 2013;33(3):484-499. https://doi.org/10.1016/J.CPR.2013.01.010

    Article  PubMed  Google Scholar 

  10. McLeod BD, Weisz JR. The therapy process observational coding system for child psychotherapy strategies scale. Journal of Clinical Child & Adolescent Psychology. 2010;39(3): 436-443. https://doi.org/10.1080/15374411003691750

    Article  Google Scholar 

  11. Bearman S, Herren J, Weisz JR. Therapy Integrity in Evidence Based Interventions: Observational Coding System Coding Manual. Austin, Texas: University of Texas, 2012.

    Google Scholar 

  12. Bearman SK, Schnedierman RL, Zoloth E. Building an evidence base for effective supervision practices: An analogue experiment of supervision to increase EBT fidelity. Administration and Policy in Mental Health and Mental Health Services Research. 2017;44(2):293-307. https://doi.org/10.1007/s10488-016-0723-8

    Article  PubMed  Google Scholar 

  13. Weisz JR, Bearman SK, Santucci LC, et al. Initial test of a principle-guided approach to transdiagnostic psychotherapy with children and adolescents. Journal of Clinical Child & Adolescent Psychology. 2017;46(1):44-58. https://doi.org/10.1080/15374416.2016.1163708

    Article  Google Scholar 

  14. Perepletchikova F, Hilt LM, Chereji E, et al. Barriers to implementing treatment integrity procedures: Survey of treatment outcome researchers. Journal of Consulting and Clinical Psychology. 2009;77(2):212-218. https://doi.org/10.1037/a0015232

    Article  PubMed  Google Scholar 

  15. Rozek DC, Serrano JL, Marriott BR, et al. Cognitive behavioral therapy competency: Pilot data from a comparison of multiple perspectives. Behaivoural and Cognitive Psychotherapy. 2018;46(2):244-250. https://doi.org/10.1017/S1352465817000662

    Article  Google Scholar 

  16. Hanson RF, Saunders BE, Ralston E, et al. Statewide implementation of child trauma-focused practices using the community-based learning collaborative model. Psychological Services. 2019;16(1):170-181. https://doi.org/10.1037/ser0000319

    Article  PubMed  Google Scholar 

  17. Kolko DJ, Cohen JA, Mannarino AP, et al. Community treatment of child sexual abuse: A survey of practitioners in the national child traumatic stress network. Administration and Policy in Mental Health and Mental Health Services Research. 2009;36(1):37-49. https://doi.org/10.1007/s10488-008-0180-0

    Article  PubMed  Google Scholar 

  18. Weersing VR, Weisz JR, Donenberg GR. Development of the therapy procedures checklist: A therapist-report measure of technique use in child and adolescent treatment. Journal of Clinical Child and Adolescent Psychology. 2002;31(2):168-180. https://doi.org/10.1207/S15374424JCCP3102_03

    Article  PubMed  Google Scholar 

  19. Child and Adolescent Mental Health Division. Service Provider Monthly Treatment and Progress Summary. Honolulu: HI: Hawaii Department of Health Child and Adolescent Mental Health Division, 2003.

  20. Child and Adolescent Mental Health Division. Instructions and Codebook for Provider Monthly Treatment and Progress Summary. Honolulu: HI: Hawaii Department of Health Child and Adolescent Mental Health Division, 2008. https://health.hawaii.gov/camhd/files/2017/11/MTPS-codebook.pdf

  21. Orimoto TE, Higa-McMillan CK, Mueller CW, et al. Assessment of therapy practices in community treatment for children and adolescents. Psychiatric Services. 2012;63(4):343-350. https://doi.org/https://doi.org/10.1176/appi.ps.201100129

    Article  PubMed  Google Scholar 

  22. Southam-Gerow MA, McLeod BD, Arnold CC, et al. Initial development of a treatment adherence measure for cognitive-behavioral therapy for child anxiety. Psychological Assessment. 2016;28(1):70-80. https://doi.org/10.1037/pas0000141

    Article  PubMed  Google Scholar 

  23. Cho E, Wood PK, Taylor EK, Hausman EM, et al. Evidence-based treatment strateiges in youth mental health services: results from a national survey of providers. Administration and Policy in Mental Health and Mental Health Services Research. 2019;46(1):71-81. https://doi.org/10.1007/s10488-018-0896-4

    Article  PubMed  Google Scholar 

  24. Hawley KM, Cook JR, Jensen-Doss, A. Do noncontingent incentives increase survey response rates among mental health providers? A randomized trial comparison. Administration and Policy in Mental Health and Mental Health Services Research. 2009;36(5):343-348. https://doi.org/10.1007/s10488-009-0225-z

    Article  PubMed  Google Scholar 

  25. Jensen-Doss A, Haimes EMB, Smith AM, et al. Monitoring treatment progress and providing feedback is viewed favorably but rarely used in practice. Administration and Policy in Mental Health and Mental Health Services Research. 2018;45(1):48-61. https://doi.org/10.1007/s10488-016-0763-0

    Article  PubMed  Google Scholar 

  26. Cho E, Tugendrajch SK, Marriott BR, et al. Evidence-based assessment in routine mental health services for youths. Psychiatric Services. 2021;72(3):325-328. https://doi.org/10.1176/appi.ps.201900537

    Article  PubMed  Google Scholar 

  27. Garland AF, Hawely KM, Brookman-Frazee, L, et al. Identifying common elements of evidence-based psychosocial treatments for children’s disruptive behavior problems. Journal of the American Academy of Child & Adolescent Psychiatry. 2008;47(5):505-514. https://doi.org/10.1097/CHI.0b013e31816765c2

    Article  Google Scholar 

  28. Horn JL. A rationale and test for the number of factors in factor analysis. Psychometrika. 1965;30(2):179-185. https://doi.org/10.1007/BF02289447

    Article  CAS  PubMed  Google Scholar 

  29. O’connor BP. SPSS and BAS programs for determining the number of components usign parallel analysis and Velicer’s MAP test. Behavior Research Methods, Instruments, & Computers. 2000;32:396-402. https://doi.org/10.3758/BF03200807

    Article  Google Scholar 

  30. Bentler PM. Comparative fit indexes in structural models. Psychological Bulletin. 1990;107(2):238-246. https://doi.org/10.1037/0033-2909.107.2.238

    Article  CAS  PubMed  Google Scholar 

  31. Tucker LR, Lewis C. A reliability coefficient for maximum likelihood factor analysis. Psychometrika. 1973;38(1):1-10. https://doi.org/10.1007/BF02291170

    Article  Google Scholar 

  32. Steiger JH. Structural model evaluation and modification: An interval estimation approach. Multivariate Behavioral Research. 1990;25(2):173-180. https://doi.org/10.1207/s15327906mbr2502_4

    Article  CAS  PubMed  Google Scholar 

  33. Hu L, Bentler PM. Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal. 1999;6(1):1-55. https://doi.org/10.1080/10705519909540118

    Article  Google Scholar 

  34. Marsh HW, Hau KT, Wen Z. In search of golden rules: Comment on hypothesis-testing approaches to setting cutoff values for fit indexes and dangers in overgeneralizing Hu and Bentler’s (1999) findings. Structural Equation Modeling: A Multidisciplinary Journal. 2004;11(3):320-341. https://doi.org/10.1207/s15328007sem1103_2

    Article  Google Scholar 

  35. Muthén LK, Muthén BO. Mplus User’s Guide. 8th ed. Muthén & Muthén,1998–2015.

  36. Dillman D. Mail and Internet Surveys: The Tailored Design Method. John Wiley & Sons, Inc., 2000.

  37. Hurlburt MS, Garland AF, Nguyen K, et al. Child and family therapy process: Concordance of therapist and observational perspectives. Administration and Policy in Mental Health and Mental Health Services Research. 2010;37(3):230-244. https://doi.org/10.1007/s10488-009-0251-x

    Article  PubMed  Google Scholar 

  38. Castro-Ramirez F, Al-Suwaidi M, Garcia P, et al. Racism and poverty are barriers to the treatment of youth mental health concerns. Journal of Clinical Child & Adolescent Psychology. 2021;50(4):534-546. https://doi.org/10.1080/15374416.2021.1941058

Download references

Acknowledgements

The authors would like to thank Jonathon Cook, Marcia Kearns, Amanda Jensen-Doss, Stacey McCrary, Curtis McMillan, and Enola Proctor for their assistance with the surveys.

Funding

This research was supported in part by R03 MH077752 from the National Institute of Mental Health to Dr. Kristin M. Hawley and by P30 MH068579 from the National Institute of Mental Health to Dr. Enola K. Proctor.

Author information

Authors and Affiliations

  1. Department of Psychological Sciences, University of Missouri, Columbia, MO, USA

    Brigid R. Marriott PhD, Evelyn Cho PhD, Siena K. Tugendrajch MA & Kristin M. Hawley PhD

Authors
  1. Brigid R. Marriott PhD
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Evelyn Cho PhD
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Siena K. Tugendrajch MA
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Kristin M. Hawley PhD
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Kristin M. Hawley PhD.

Ethics declarations

Conflict of Interest

The authors declare no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Marriott, B.R., Cho, E., Tugendrajch, S.K. et al. Measuring Evidence-Based Treatment Strategies in Youth Community Mental Health Care: the Evidence-Based Strategies Scale. J Behav Health Serv Res 49, 335–345 (2022). https://doi.org/10.1007/s11414-021-09779-9

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11414-021-09779-9

Search

Navigation