Measuring Treatment Integrity: Use of and Experience with Measurements in Child and Youth Care Organizations

  • Pauline Brigitta Goense
  • Leonieke Boendermaker
  • Tom van Yperen


Performance feedback and supervision are essential to the adoption of evidence-based interventions with high treatment integrity in child and youth care organizations. Little is known about the use of treatment integrity measurements in these organizations. For this study, 12 interventions for children and young people in the Netherlands with externalizing behavioral problems were selected. For each intervention, an expert, two supervisors, and two therapists were approached for an interview. In total, 54 semi-structured interviews were conducted. The results show that almost all interventions used treatment integrity instruments (N = 11, 91.7%). Only two used measurements for both QA procedures (certification and recertification) and supervision purposes. Therapists regard treatment integrity measurements as valuable when they are used for multiple purposes and feedback is provided. The results of this study suggest the feasibility of the use of measurements for multiple purposes. Collaborative action is required to develop instruments that effectively contribute to continuous improvement.


Treatment integrity Instruments Quality assurance (QA) Interventions Supervision 



We thank Sanne Pronk, Rosan Bakker, Lianne Bertling, Inge Busschers, and Joyce Berentsen for their contribution to the data collection and analysis.


The research included in this article was financially supported by Stichting Innovatie Alliantie (a Dutch funding body for research conducted by applied universities in the Netherlands) in the form of a grant (2012-14-18P).

Compliance with Ethical Standards

Conflict of Interest

The authors declare no potential conflicts of interest with respect to the research, authorship, or publication of this article.


  1. 1.
    Fixsen DL, Ogden T. Facing the Challenges of Implementation. Zeitschrift für Psychologie. 2014;222(1):1–3.CrossRefGoogle Scholar
  2. 2.
    Goense PB, Assink M, Stams GJJM, et al. Making ‘What Works’ Work: A Meta-Analytic Study of the Effect of Treatment Integrity on Outcomes of Evidence-Based Interventions for Juveniles with Antisocial Behavior. Aggression and Violent Behavior. 2016. doi:
  3. 3.
    Schoenwald SK, Chapman JE, Sheidow AJ, et al. Long-Term Youth Criminal Outcomes in MST Transport: The Impact of Therapist Adherence and Organizational Climate and Structure. Journal of Clinical Child and Adolescent Psychology. 2009;38(1):91–105.CrossRefPubMedPubMedCentralGoogle Scholar
  4. 4.
    Sexton T, Turner CW. The Effectiveness of Functional Family Therapy for Youth with Behavioral Problems in a Community Practice Setting. Journal of Family Psychology. 2010;24:339–348.CrossRefPubMedPubMedCentralGoogle Scholar
  5. 5.
    Tennyson HR. Reducing Juvenile Recidivism: A Meta-Analysis of Treatment Outcomes. Pacific University CommonKnowledge. Available online at Accessed on November 25, 2016.
  6. 6.
    Schoenwald SK, Garland AF. A Review of Treatment Adherence Measurement Methods. Psychological Assessment. 2013;25(1):146–156.CrossRefPubMedGoogle Scholar
  7. 7.
    Fixsen DL, Naoom SF, Blase KA, et al. Implementation Research: A Synthesis of the Literature. Tampa, FL: Louis de la Parte Florida Mental Health Institute, National Implementation Research Network, 2005.Google Scholar
  8. 8.
    Garland AF, Schoenwald SK. Use of Effective and Efficient Quality Control Methods to Implement Psychosocial Interventions. Clinical Psychology: Science and Practice. 2013;20:33–43.Google Scholar
  9. 9.
    Goense PB, Boendermaker L, van Yperen T. Support Systems for Treatment Integrity. Research on Social Work Practice. 2015;1–5.Google Scholar
  10. 10.
    Hogue A, Ozechowski TJ, Robbins MS, et al. Making Fidelity an Intramural Game: Localizing Quality Assurance Procedures to Promote Sustainability of Evidence-Based Practices in Usual Care. Clinical Psychology: Science and Practice. 2013;20:60–77.Google Scholar
  11. 11.
    Kerby NT (Ed). Helping Others Help Children; Clinical Supervision of Child Psychotherapy. Washington, DC: American Psychological Association, 2006.Google Scholar
  12. 12.
    Miller WR, Sorensen JL, Selzer JA, et al. Disseminating Evidence Based Practices in Substance Abuse Treatment: A Review with Suggestions. Journal of Substance Abuse Treatment. 2006;31:25–39.CrossRefPubMedGoogle Scholar
  13. 13.
    Schoenwald SK, Sheidow AJ, Chapman JE. Clinical Supervision in Treatment Transport: Effects on Adherence and Outcomes. Journal of Consulting and Clinical Psychology. 2009;77(3):410–421.CrossRefPubMedPubMedCentralGoogle Scholar
  14. 14.
    Perepletchikova F, Treat TA, Kazdin AE. Treatment Integrity in Psychotherapy Research: Analysis of the Studies and Examination of the Associated Factors. Journal of Consulting and Clinical Psychology. 2007;75(6):829–841.CrossRefPubMedGoogle Scholar
  15. 15.
    Barber JP, Gallop R, Crits-Christoph P, et al. The role of Therapist Adherence, Therapist Competence, and Alliance in Predicting Outcome of Individual Drug Counseling: Results from the National Institute Drug Abuse Collaborative Cocaine Treatment Study. Psychotherapy Research. 2006;16(2):229–240.CrossRefGoogle Scholar
  16. 16.
    Barber JP, Sharpless BA, Klostermann S, et al. Assessing Intervention Competence and Its Relation to Therapy Outcome. Professional Psychology: Research and Practice. 2007;38(5):493–500.CrossRefGoogle Scholar
  17. 17.
    Barber J, Triffleman E, Marmar C. Considerations in Treatment Integrity: Implications and Recommendations for PTSD Research. Journal of Traumatic Stress. 2007;20(5):793–805.CrossRefPubMedGoogle Scholar
  18. 18.
    Goense P, Boendermaker L, Van Yperen T, et al. Implementation of Treatment Integrity Procedures: An Analysis of Outcome Studies of Youth Interventions Targeting Externalizing Behavioral Problems. Zeitschrift für Psychologie. 2014;222(1):12–21.CrossRefGoogle Scholar
  19. 19.
    McLeod BD, Southam-Gerow MA, Tully CB, et al. Making a Case for Treatment Integrity as a Psychosocial Treatment Quality Indicator for Youth Mental Health Care. Clinical Psychology. 2013;20(1):1–23.Google Scholar
  20. 20.
    Schoenwald SK, Garland AF, Chapman JE, et al. Toward the Effective and Efficient Measurement of Implementation Fidelity. Administration and Policy in Mental Health. 2011;38:32–43.CrossRefPubMedPubMedCentralGoogle Scholar
  21. 21.
    Nápoles AM, Santoyo-Olsson J, Stewart AL. Methods for Translating Evidence-Based Behavioral Interventions for Health-Disparity Communities. Preventing Chronic Disease. 2013;10.Google Scholar
  22. 22.
    Knox LM, Aspy CB. Quality Improvement as a Tool for Translating Evidence-Based Interventions into Practice: What the Youth Violence Prevention Community Can Learn from Healthcare. American Journal of Community Psychology. 2011;48(1–2):56–64.CrossRefPubMedGoogle Scholar
  23. 23.
    Sutherland KS, McLeod BD, Conroy MA, et al. Measuring Implementation of Evidence-Based Programs Targeting Young Children at Risk for Emotional/Behavioral Disorders: Conceptual Issues and Recommendations. Journal of Early Intervention. 2013;35(2):129–149.CrossRefGoogle Scholar
  24. 24.
    Kazdin A. Methodology, design, and evaluation in psychotherapy research. In: AE Bergin, SL Garfield (Eds). Handbook of Psychotherapy and Behavior Change, Fourth Edition. Oxford: John Wiley & Sons, 1994, pp. 19–71.Google Scholar
  25. 25.
    McLeod BD, Islam NY, Wheat E. Designing, conducting, and evaluating therapy process research. In: J Comer, P Kendall (Eds). The Oxford Handbook of Research Strategies for Clinical Psychology. New York: Oxford University Press, 2013, pp. 142–164.Google Scholar
  26. 26.
    Schoenwald SK. It’s a Bird, It’s a Plane, It’s…Fidelity Measurement in the Real World. Clinical Psychology: Science and Practice. 2011;18(2):142–147.Google Scholar
  27. 27.
    Goense PB, Pronk S, Boendermaker L, et al. Leren op de Werkvloer. Organisatie en Inhoud van Leren op de Werkvloer. Delft, NL: Uitgeverij Eburon, 2015.Google Scholar
  28. 28.
    Weisz JR, Jensen-Doss A, Hawley KM. Evidence-Based Youth Psychotherapies Versus Usual Clinical Care: A Meta-Analysis of Direct Comparisons. American Psychologist. 2006;61:671–689.CrossRefPubMedGoogle Scholar
  29. 29.
    Goldstein AP, Glick B, Gibbs JC. Aggression Replacement Training; A Comprehensive Intervention for Aggressive Youth, Revised Edition. Champaign, IL: Research Press, 1998.Google Scholar
  30. 30.
    MAXQDA. MAXQDA 11: The Art of Data Analysis, Reference Manual for Windows. Berlin, DE: VERBI Software, Consult, Sozialforschung, 2012.Google Scholar
  31. 31.
    Hsieh H, Shannon SE. Three Approaches to Qualitative Content Analysis. Qualitative Health Research. 2005;15:1277–1288.CrossRefPubMedGoogle Scholar
  32. 32.
    Morrow SL. Quality and Trustworthiness in Qualitative Research in Counseling Psychology. Journal of Counseling Psychology. 2005;52:250–260.CrossRefGoogle Scholar

Copyright information

© National Council for Behavioral Health 2018

Authors and Affiliations

  • Pauline Brigitta Goense
    • 1
  • Leonieke Boendermaker
    • 2
  • Tom van Yperen
    • 3
  1. 1.The Netherlands Organisation for Health Research and DevelopmentThe HagueThe Netherlands
  2. 2.Centre for Applied Research in Social Work and LawAmsterdam University of Applied SciencesAmsterdamThe Netherlands
  3. 3.Faculty of Social and Behavioural SciencesUniversity of GroningenGroningenThe Netherlands

Personalised recommendations