Observational Coding Training Methods for CBT Treatment Fidelity: A Systematic Review
Observational coding is the gold standard for coding treatment fidelity in Cognitive Behavioral Therapy (CBT). This method allows researchers to objectively assess therapist behaviors to determine if CBT was delivered as intended. However, there is no standard method or protocol that ensures reliability within and between research groups, compromising comparability between studies. The aim of this study was to evaluate procedures used to train raters to quantify CBT fidelity. A systematic review was conducted. PsycINFO and Web of Science databases were reviewed for articles up until July 2015; the parameters of the search included CBT, depression, treatment fidelity, and coding/rating. A total of 521 articles were screened for inclusion, which yielded 81 articles. Data was extracted across four categories (study characteristics, training process, inter-rater reliability, and rating), and 62 codes. The results revealed variability in training approaches (e.g., 25.9% of articles reported 10–50 h of training). Very little information was provided about the training process (i.e. only one study specified training duration). Training materials were: manuals (21.0%), videos (18.5%), audiotapes (2.4%), readings (1.2%), and lectures (1.2%). Almost all (90.1%) studies did not describe the trainer, and 28.4% did not describe raters. The relation between CBT fidelity and clinical outcomes is equivocal, perhaps because intra-team and inter-team reliability is difficult to achieve. This systematic review revealed variability in training others to rate CBT fidelity, of which some approaches might inform the field for future use.
KeywordsSystematic review Observational coding Training Fidelity CBT
We would like to acknowledge the following research assistants for their hard work in coding articles for this study: Paige Schultz, Madison Walker, Kayne Mettert, and Jasmine Shaffer.
Compliance with Ethical Standards
Conflict of Interest
Natalie Rodriguez-Quintana and Cara C. Lewis declare that they have no conflict of interest.
Human and Animal Rights
This article does not contain any studies with human participants or animals performed by any of the authors.
All procedures followed were in accordance with the ethical standards of the responsible committee on human experimentation (national and institutional). Informed consent was obtained from all individual subjects participating in the study.
- Bakeman, R., & Gottman, J. M. (1987). Applying observational methods: A systematic view. In J. D. Osofsky (Ed.), Handbook of infant development (2nd edn., pp. 818–854). New York: Wiley.Google Scholar
- Baldwin, S. A., & Imel, Z. E. (2013). Therapist Effects: Findings and Methods. In B. A. Garfield (Ed.), Handbook of psychotherapy and behavior change (6th edn., pp. 258–297). New York: Wiley.Google Scholar
- Beck, J. S. (2011). Cognitive behavior therapy: Basics and beyond. New York: Guilford Press.Google Scholar
- Brauhardt, A., de Zwaan, M., Herpertz, S., Zipfel, S., Svaldi, J., Friederich, H. -C., & Hilbert, A. (2014). Therapist adherence in individual cognitive-behavioral therapy for binge-eating disorder: Assessment, course, and predictors. Behaviour Research and Therapy, 61, 55–60. https://doi.org/10.1016/j.brat.2014.07.014.CrossRefPubMedGoogle Scholar
- Castorr, A. H., Thompson, K. O., Ryan, J. W., Phillips, C. Y., Prescott, P. A., & Soeken, K. L. (1990). The process of rater training for observational instruments: Implications for interrater reliability. Research in Nursing & Health, 13, 311–318. https://doi.org/10.1002/nur.4770130507.CrossRefGoogle Scholar
- Corace, K. M., Srigley, J. A., Hargadon, D. P., Yu, D., MacDonald, T. K., Fabrigar, L. R., & Garber, G. E. (2016). Using behavior change frameworks to improve healthcare worker influenza vaccination rates: A systematic review. Vaccine, 34, 3235–3242. https://doi.org/10.1016/j.vaccine.2016.04.071.CrossRefPubMedGoogle Scholar
- Crits-Christoph, P., Siqueland, L., Blaine, J., Frank, A., Luborsky, L., Onken, L. S., … Gastfriend, D. R. (1999). Psychosocial treatments for cocaine dependence: National Institute on Drug Abuse collaborative cocaine treatment study. Archives of General Psychiatry, 56, 493–502. https://doi.org/10.1001/pubs.ArchGenPsychiatry-ISSN-0003-990x-56-6-yoa8244.CrossRefPubMedGoogle Scholar
- Guttman, H. A., Spector, R. M., Sigal, J. J., Rakoff, V., & Epstein, N. B. (1971). Reliability of coding affective communication in family therapy sessions: Problems of measurement and interpretation. Journal of Consulting and Clinical Psychology, 37, 397–402. https://doi.org/10.1037/h0032003.CrossRefPubMedGoogle Scholar
- Jacobson, N. S. (1998, July 21–26). How important are cognitive interventions in cognitive therapy? Paper presented at the World Congress of Behavioral and Cognitive Therapies, Acapulco, Mexico.Google Scholar
- Jacobson, N. S., Dobson, K. S., Truax, P. A., Addis, M. E., Koerner, K., Gollan, J. K., … Prince, S. E. (1996). A component analysis of cognitive-behavioral treatment for depression. Journal of Consulting and Clinical Psychology, 64, 295–304. https://doi.org/10.1037/0022-006X.70.2.288.CrossRefPubMedGoogle Scholar
- Maissi, E., Ridge, K., Treasure, J., Chalder, T., Roche, S., Bartlett, J., … Ismail, K. (2011). Nurse-led psychological interventions to improve diabetes control: Assessing competencies. Patient Education and Counseling, 84, e37-43. https://doi.org/10.1016/j.pec.2010.07.036.CrossRefPubMedGoogle Scholar
- Markowitz, J. C., Kocsis, J. H., Fishman, B., Spielman, L. A., Jacobsberg, L. B., Frances, A. J., … Perry, S. W. (1998). Treatment of depressive symptoms in human immunodeficiency virus-positive patients. Archives of General Psychiatry, 55, 452–457. https://doi.org/10.1001/archpsyc.55.5.452.CrossRefPubMedGoogle Scholar
- McLeod, B. D., & Weisz, J. R. (2005). The therapy process observational coding system-alliance scale: Measure characteristics and prediction of outcome in usual clinical practice. Journal of Consulting and Clinical Psychology, 73, 323–333. https://doi.org/10.1037/0022-006X.73.2.323.CrossRefPubMedGoogle Scholar
- McManus, F., Rakovshik, S., Kennerley, H., Fennell, M., & Westbrook, D. (2012). An investigation of the accuracy of therapists’ self-assessment of cognitive-behaviour therapy skills. British Journal of Clinical Psychology, 51, 292–306. https://doi.org/10.1111/j.2044-8260.2011.02028.x.CrossRefPubMedGoogle Scholar
- Perepletchikova, F., Treat, T. A., & Kazdin, A. E. (2007). Treatment integrity in psychotherapy research: Analysis of the studies and examination of the associated factors. Journal of Consulting and Clinical Psychology, 75, 829–841. https://doi.org/10.1037/0022-006X.75.6.829.CrossRefPubMedGoogle Scholar
- Rozek, D. C., Serrano, J. L., Marriott, B., Hickman, L. B., Lewis, C. C., Brothers, B. M., & Simons, A. D. (2017). Cognitive Behavioral Therapy competency: Pilot data from a comparison of multiple perspectives. Behavioural and Cognitive Psychotherapy. https://doi.org/10.1017/S1352465817000662 PubMedGoogle Scholar
- Sheidow, A. J., Donohue, B. C., Hill, H. H., Henggeler, S. W., & Ford, J. D. (2008). Development of an audiotape review system for supporting adherence to an evidence-based treatment. Professional Psychology: Research and Practice, 39, 553–560. https://doi.org/10.1037/0735-7028.39.5.553.CrossRefGoogle Scholar
- Simons, A. D., Padesky, C. A., Montemarano, J., Lewis, C. C., Murakami, J., Lamb, K., … Beck, A. T. (2010). Training and dissemination of cognitive behavior therapy for depression in adults: A preliminary examination of therapist competence and client outcomes. Journal of Consulting and Clinical Psychology, 78, 751–756. https://doi.org/10.1037/a0020569.CrossRefPubMedGoogle Scholar
- Weck, F., Hilling, C., Schermelleh-Engel, K., Rudari, V., & Stangier, U. (2011b). Reliability of adherence and competence assessment in cognitive behavioral therapy: Influence of clinical experience. The Journal of Nervous and Mental Disease, 199, 276–279. https://doi.org/10.1037/adb0000216.CrossRefPubMedGoogle Scholar