Measuring Treatment Integrity: Use of and Experience with Measurements in Child and Youth Care Organizations
Performance feedback and supervision are essential to the adoption of evidence-based interventions with high treatment integrity in child and youth care organizations. Little is known about the use of treatment integrity measurements in these organizations. For this study, 12 interventions for children and young people in the Netherlands with externalizing behavioral problems were selected. For each intervention, an expert, two supervisors, and two therapists were approached for an interview. In total, 54 semi-structured interviews were conducted. The results show that almost all interventions used treatment integrity instruments (N = 11, 91.7%). Only two used measurements for both QA procedures (certification and recertification) and supervision purposes. Therapists regard treatment integrity measurements as valuable when they are used for multiple purposes and feedback is provided. The results of this study suggest the feasibility of the use of measurements for multiple purposes. Collaborative action is required to develop instruments that effectively contribute to continuous improvement.
KeywordsTreatment integrity Instruments Quality assurance (QA) Interventions Supervision
We thank Sanne Pronk, Rosan Bakker, Lianne Bertling, Inge Busschers, and Joyce Berentsen for their contribution to the data collection and analysis.
The research included in this article was financially supported by Stichting Innovatie Alliantie (a Dutch funding body for research conducted by applied universities in the Netherlands) in the form of a grant (2012-14-18P).
Compliance with Ethical Standards
Conflict of Interest
The authors declare no potential conflicts of interest with respect to the research, authorship, or publication of this article.
- 2.Goense PB, Assink M, Stams GJJM, et al. Making ‘What Works’ Work: A Meta-Analytic Study of the Effect of Treatment Integrity on Outcomes of Evidence-Based Interventions for Juveniles with Antisocial Behavior. Aggression and Violent Behavior. 2016. doi: https://doi.org/10.1016/j.avb.2016.08.003
- 5.Tennyson HR. Reducing Juvenile Recidivism: A Meta-Analysis of Treatment Outcomes. Pacific University CommonKnowledge. Available online at https://commons.pacificu.edu/cgi/viewcontent.cgi?referer=https://www.google.com/&httpsredir=1&article=1212&context=spp. Accessed on November 25, 2016.
- 7.Fixsen DL, Naoom SF, Blase KA, et al. Implementation Research: A Synthesis of the Literature. Tampa, FL: Louis de la Parte Florida Mental Health Institute, National Implementation Research Network, 2005.Google Scholar
- 8.Garland AF, Schoenwald SK. Use of Effective and Efficient Quality Control Methods to Implement Psychosocial Interventions. Clinical Psychology: Science and Practice. 2013;20:33–43.Google Scholar
- 9.Goense PB, Boendermaker L, van Yperen T. Support Systems for Treatment Integrity. Research on Social Work Practice. 2015;1–5.Google Scholar
- 10.Hogue A, Ozechowski TJ, Robbins MS, et al. Making Fidelity an Intramural Game: Localizing Quality Assurance Procedures to Promote Sustainability of Evidence-Based Practices in Usual Care. Clinical Psychology: Science and Practice. 2013;20:60–77.Google Scholar
- 11.Kerby NT (Ed). Helping Others Help Children; Clinical Supervision of Child Psychotherapy. Washington, DC: American Psychological Association, 2006.Google Scholar
- 15.Barber JP, Gallop R, Crits-Christoph P, et al. The role of Therapist Adherence, Therapist Competence, and Alliance in Predicting Outcome of Individual Drug Counseling: Results from the National Institute Drug Abuse Collaborative Cocaine Treatment Study. Psychotherapy Research. 2006;16(2):229–240.CrossRefGoogle Scholar
- 19.McLeod BD, Southam-Gerow MA, Tully CB, et al. Making a Case for Treatment Integrity as a Psychosocial Treatment Quality Indicator for Youth Mental Health Care. Clinical Psychology. 2013;20(1):1–23.Google Scholar
- 21.Nápoles AM, Santoyo-Olsson J, Stewart AL. Methods for Translating Evidence-Based Behavioral Interventions for Health-Disparity Communities. Preventing Chronic Disease. 2013;10.Google Scholar
- 24.Kazdin A. Methodology, design, and evaluation in psychotherapy research. In: AE Bergin, SL Garfield (Eds). Handbook of Psychotherapy and Behavior Change, Fourth Edition. Oxford: John Wiley & Sons, 1994, pp. 19–71.Google Scholar
- 25.McLeod BD, Islam NY, Wheat E. Designing, conducting, and evaluating therapy process research. In: J Comer, P Kendall (Eds). The Oxford Handbook of Research Strategies for Clinical Psychology. New York: Oxford University Press, 2013, pp. 142–164.Google Scholar
- 26.Schoenwald SK. It’s a Bird, It’s a Plane, It’s…Fidelity Measurement in the Real World. Clinical Psychology: Science and Practice. 2011;18(2):142–147.Google Scholar
- 27.Goense PB, Pronk S, Boendermaker L, et al. Leren op de Werkvloer. Organisatie en Inhoud van Leren op de Werkvloer. Delft, NL: Uitgeverij Eburon, 2015.Google Scholar
- 29.Goldstein AP, Glick B, Gibbs JC. Aggression Replacement Training; A Comprehensive Intervention for Aggressive Youth, Revised Edition. Champaign, IL: Research Press, 1998.Google Scholar
- 30.MAXQDA. MAXQDA 11: The Art of Data Analysis, Reference Manual for Windows. Berlin, DE: VERBI Software, Consult, Sozialforschung, 2012.Google Scholar