Skip to main content

Advertisement

SpringerLink
Log in

Behavioral Intervention Fidelity in Routine Practice: Pragmatism Moves to Head of the Class

  • Review Paper
  • Published:
School Mental Health Aims and scope Submit manuscript

Abstract

This commentary for the special issue on integrity measurement in school mental health research advocates that pragmatism is centralized in implementation and fidelity efforts for evidence-based behavioral interventions (EBIs). Specific recommendations include focusing on augmenting existing practices and core element interventions; developing practitioner self-report fidelity measures; and emphasizing practitioner training in how to use EBI fidelity methods. An exemplar model of practitioner training in fidelity methods, featuring observational coder training and fidelity measurement feedback, is presented.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

References

  • Becker-Haimes, E. M., Klein, M. R., McLeod, B. D., Schoenwald, S. K., Dorsey, S., Hogue, A., Fugo, P. B., Phan, M. L., Hoffacker, C., & Beidas, R. S. (2021). The TPOCS-self-reported therapist intervention fidelity for youth (TPOCS-SeRTIFY): A case study of pragmatic measure development. Implementation Research and Practice, 2, 2633489521992553. https://doi.org/10.1177/2633489521992553

    Article  Google Scholar 

  • Beidas, R. S., Cross, W., & Dorsey, S. (2014). Show me, don’t tell me: Behavioral rehearsal as a training and analogue fidelity tool. Cognitive and Behavioral Practice, 21, 1–11.

    Article  Google Scholar 

  • Brookman-Frazee, L., Stadnick, N. A., Lind, T., Roesch, S., Terrones, L., Barnett, M. L., Regan, J., Kennedy, C. A., Garland, F., & A., & Lau, A. S. (2021). Therapist-observer concordance in ratings of EBP strategy delivery: Challenges and targeted directions in pursuing pragmatic measurement in children’s mental health services. Administration and Policy in Mental Health and Mental Health Services Research. 48(1), 155–170. https://doi.org/10.1007/s10488-020-01054-x

    Article  PubMed  Google Scholar 

  • Caron, E. B., & Dozier, M. (2019). Effects of fidelity-focused consultation on clinicians’ implementation: An exploratory multiple baseline design. Administration and Policy in Mental Health and Mental Health Services Research, 46, 445–457.

    Article  Google Scholar 

  • Chorpita, B. F., Daleiden, E. L., Park, A. L., Ward, A. M., Levy, M. C., Cromley, T., Chiu, A. W., Letamendi, A. M., Tsai, K. H., & Krull, J. L. (2017). Child STEPs in California: A cluster randomized effectiveness trial comparing modular treatment with community implemented treatment for youth with anxiety, depression, conduct problems, or traumatic stress. Journal of Consulting and Clinical Psychology, 85, 13–25.

    Article  Google Scholar 

  • Chorpita, B. F., Park, A., Tsai, K., Korathu-Larson, P., Higa-McMillan, C. K., Nakamura, B. J., Weisz, J. R., & Krull, J. (2015). Balancing effectiveness with responsiveness: Therapist satisfaction across different treatment designs in the Child STEPs randomized effectiveness trial. Journal of Consulting and Clinical Psychology, 83, 709–718.

    Article  Google Scholar 

  • Chorpita, B. F., Weisz, J. R., Daleiden, E. L., Schoenwald, S. K., Palinkas, L. A., Miranda, J., Higa-McMillan, C. K., Nakamura, B. J., Austin, A. A., Borntrager, C. F., & Research Network on Youth Mental Health. (2013). Long-term outcomes for the Child STEPs randomized effectiveness trial: A comparison of modular and standard treatment designs with usual care. Journal of Consulting and Clinical Psychology, 81, 999–1009.

    Article  Google Scholar 

  • Frank, H. E., Becker-Haimes, E. M., & Kendall, P. C. (2020). Therapist training in evidence-based interventions for mental health: A systematic review of training approaches and outcomes. Clinical Psychology: Science and Practice, 27(3), e12330.

    Google Scholar 

  • Gallo, K. P., & Barlow, D. H. (2012). Factors involved in clinician adoption and nonadoption of evidence-based interventions in mental health. Clinical Psychology: Science and Practice, 19, 93–106.

    Google Scholar 

  • Glasgow, R. E., Fisher, L., Strycker, L. A., Hessler, D., Toobert, D. J., King, D. K., & Jacobs, T. (2014). Minimal intervention needed for change: Definition, use, and value for improving health and health research. Translational Behavioral Medicine, 4, 26–33.

    Article  Google Scholar 

  • Glasgow, R. E., & Riley, W. T. (2013). Pragmatic measures: What they are and why we need them. American Journal of Preventive Medicine, 45(2), 237–243. https://doi.org/10.1016/j.amepre.2013.03.010

    Article  PubMed  Google Scholar 

  • Gonder, J., Metlay, W., & Shapiro, T. (2018). Testing assumptions: Can performance rating feedback result in objective performance improvements? Journal of Management and Innovation, 4(2), 1–31.

    Google Scholar 

  • Herschell, A. D., Quetsch, L. B., & Kolko, D. J. (2020). Measuring adherence to key teaching techniques in an evidence-based treatment: A comparison of caregiver, therapist, and behavior observation ratings. Journal of Emotional and Behavioral Disorders, 28(2), 92–103. https://doi.org/10.1177/1063426618821901

    Article  Google Scholar 

  • Hogue, A., Bobek, M., Porter, N., MacLean, A., Bruynesteyn, L., Jensen-Doss, A., Dauber, S., & Henderson, C. E. (2021). Therapist self-report of fidelity core elements of family therapy for adolescent behavior problems: Psychometrics of a pragmatic quality indicator tool. Administration and Policy in Mental Health and Mental Health Services Research. https://doi.org/10.1007/s10488-021-01164-0

    Article  PubMed  Google Scholar 

  • Hogue, A., Dauber, S., & Henderson, C. E. (2017). Benchmarking family therapy for adolescent behavior problems in usual care: Fidelity, outcomes, and therapist performance differences. Administration and Policy in Mental Health and Mental Health Services Research, 44, 626–641. https://doi.org/10.1007/s10488-016-0769-7

    Article  PubMed  Google Scholar 

  • Hogue, A., Ozechowski, T. J., Robbins, M. S., & Waldron, H. B. (2013). Making fidelity an intramural game: Localizing quality assurance procedures to promote sustainability of evidence-based practices in usual care. Clinical Psychology: Science and Practice, 20, 60–77.

    Google Scholar 

  • Hogue, A., Porter, N., Bobek, M., MacLean, A., Bruynesteyn, L., Jensen-Doss, A., Dauber, S., & Henderson, C. E. (2022). Online training of community therapists in observational coding of family therapy techniques: Reliability and accuracy. Administration and Policy in Mental Health and Mental Health Services Research. 49, 139–151. https://doi.org/10.1007/s10488-021-01152-4

    Article  PubMed  Google Scholar 

  • Hurlburt, M. S., Garland, A. F., Nguyen, K., & Brookman-Frazee, L. (2010). Child and family therapy process: Concordance of therapist and observational perspectives. Administration and Policy in Mental Health and Mental Health Services Research, 37, 230–244. https://doi.org/10.1007/s10488-009-0251-x

    Article  PubMed  Google Scholar 

  • Institute of Medicine. (2015). Psychosocial interventons for mental and substance use disorders: A framework for establishing evidence-based standards. National Academies Press.

    Google Scholar 

  • Kerns, S. E., Perrine, C. M., Sedlar, G., Peterson, R., & Monroe-DeVita, M. (2021). Keeping the faith while keeping it real: practical, empirical approaches to evaluating treatment fidelity. Global Implementation Research and Applications, 1(2), 1–12.

    Article  Google Scholar 

  • LeBreton, J. M., & Senter, J. L. (2008). Answers to 20 questions about interrater reliability and interrater agreement. Organizational Research Methods, 11(4), 815–852.

    Article  Google Scholar 

  • Lyon, A. R., Stanick, C., & Pullmann, M. D. (2018). Toward high-fidelity treatment as usual: Evidence-based intervention structures to improve usual care psychotherapy. Clinical Psychology: Science and Practice, 25, e12265.

    Google Scholar 

  • Mastroleo, N. R., Humm, L., Williams, C. M., Kiluk, B. D., Hoadley, A., & Magill, M. (2020). Initial testing of a computer-based simulation training module to support clinicians’ acquisition of CBT skills for substance use disorder treatment. Journal of Substance Abuse Treatment, 114, 108014.

    Article  Google Scholar 

  • McLeod, B. D., Cook, C. R., Sutherland, K. S., Lyon, A. R., Dopp, A., Broda, M., & Beidas, R. S. (2021). A theory-informed approach to locally managed learning school systems: Integrating treatment integrity and youth mental health outcome data to promote youth mental health. School Mental Health.

  • McLeod, B. D., Cox, J. R., Jensen-Doss, A., Herschell, A., Ehrenreich-May, J., & Wood, J. J. (2018). Proposing a mechanistic model of clinician training and consultation. Clinical Psychology: Science and Practice, 25(3), e12260.

    Google Scholar 

  • Park, A. L., Tsai, K. H., Guan, K., & Chorpita, B. F. (2018). Unintended consequences of evidence-based treatment policy reform: Is implementation the goal or the strategy for higher quality care? Administration and Policy in Mental Health and Mental Health Services Research, 45(4), 649–660.

    Article  Google Scholar 

  • Regan, J., Daleiden, E. L., & Chorpita, B. F. (2013). Integrity in mental health systems: An expanded framework for managing uncertainty in clinical care. Clinical Psychology: Science and Practice, 20(1), 78.

    Google Scholar 

  • Schleider, J. L., & Weisz, J. R. (2017). Little treatments, promising effects? Meta-Analysis of single-session interventions for youth psychiatric problems. Journal of the American Academy of Child & Adolescent Psychiatry, 56, 107–115.

    Article  Google Scholar 

  • Shimokawa, K., Lambert, M. J., & Smart, D. W. (2010). Enhancing treatment outcome of patients at risk of treatment failure: Meta-analytic and mega-analytic review of a psychotherapy quality assurance system. Journal of Consulting and Clinical Psychology, 78(3), 298.

    Article  Google Scholar 

  • Stirman, S. W. (2020). Commentary: Challenges and opportunites in the assessment of fidelity and related constructs. Administration and Policy in Mental Health and Mental Health Services Research, 47(6), 932–934.

    Article  Google Scholar 

  • Sutherland, K. S., & McLeod, B. D. (2021). Advancing the science of integrity measurement in school mental health research. School Mental Health.

  • Walton, H., Spector, A., Tombor, I., & Michie, S. (2017). Measures of fidelity of delivery of, and engagement with, complex, face-to-face health behaviour change interventions: A systematic review of measure quality. British Journal of Health Psychology, 22(4), 872–903.

    Article  Google Scholar 

  • Weisz, J., Bearman, S. K., Santucci, L. C., & Jensen-Doss, A. (2017). Initial test of a principle-guided approach to transdiagnostic psychotherapy with children and adolescents. Journal of Clinical Child & Adolescent Psychology, 46, 44–58.

    Article  Google Scholar 

  • Weisz, J. R., Krumholz, L. S., Santucci, L., Thomassin, K., & Ng, M. Y. (2015). Shrinking the gap between research and practice: Tailoring and testing youth psychotherapies in clinical care contexts. Annual Review of Clinical Psychology, 11, 139–163.

    Article  Google Scholar 

Download references

Acknowledgments

The author gratefully acknowledges the contributions of esteemed colleagues Molly Bobek, Sarah Dauber, Amanda Jensen-Doss, Alexandra MacLean, Bryce McLeod, Nicole Porter, and Michael Southam-Gerow to the ideas presented herein.

Author information

Authors and Affiliations

  1. Family and Adolescent Clinical Technology and Science, Partnership to End Addiction, 711 Third Avenue, Suite 500, New York, NY, 10017, USA

    Aaron Hogue

Authors
  1. Aaron Hogue
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Aaron Hogue.

Ethics declarations

Conflicts of Interest

The authors declare that they have no conflict of interest.

Ethical Approval

This work did not involve human subjects or animal participants.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Hogue, A. Behavioral Intervention Fidelity in Routine Practice: Pragmatism Moves to Head of the Class. School Mental Health 14, 103–109 (2022). https://doi.org/10.1007/s12310-021-09488-w

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12310-021-09488-w

Keywords

Search

Navigation