Abstract
This paper presents the psychometric evaluation of brief measures of therapeutic alliance (TA) for youths, clinicians and caregivers and a longitudinal analysis of relationships between changes in TA and changes in youth symptom and functioning severity. Psychometric analyses using methods from Classical Test Theory, Item Response Theory, and Factor Analysis indicate that the measures of TA used in this study offer something new for both practice and research. The measures have variability, sensitivity to change over time, brevity and can be used with multiple parties through parallel forms. The longitudinal analyses, employing hierarchical linear modeling with time-varying covariates, found that TA ratings of the clinician correlated with symptom improvement as rated by the clinician, caregiver and youth. Additional analyses showed that decreases in clinician-rated youth TA was most important in predicting a lower rate of youth improvement. Implications for future research and clinical practice are discussed.
Similar content being viewed by others
References
Anker, M. G., Owen, J., Duncan, B. L., & Sparks, J. A. (2010). The alliance in couple therapy: Partner influence, early change, and alliance patterns in a naturalistic sample. Journal of Consulting and Clinical Psychology, 78(5), 635–645. doi:10.1037/a0020051.
Armbruster, P., & Kazdin, A. E. (1994). Attrition in child-psychotherapy. Advances in Clinical Child Psychology, 16(16), 81–108.
Athay, M. M., Riemer, M., & Bickman, L. (2012). The Symptoms and Functioning Severity Scale (SFSS): Psychometric evaluation and discrepancies among youth, caregiver, and clinician ratings over time. Administration and Policy in Mental Health and Mental Health Services Research. doi:10.1007/s10488-012-0403-2.
Barber, J. P., Connoly, M. B., Crits-Christoph, P., Gladis, L., & Siqueland, L. (2009). Alliance predicts patients’ outcome beyond in-treatment change in symptoms. Personality Disorders: Theory, Research, and Treatment, S(1), 80–89. doi:10.1037/1949-2715.S.1.80.
Bickman, L. (2008). A measurement feedback system (MFS) is necessary to improve mental health outcomes. Journal of the American Academy of Child and Adolescent Psychiatry, 47(10), 1114–1119. doi:10.1097/CHI.0b013e3181825af8.
Bickman, L., Rosof-Williams, J., Salzer, M. S., Summerfelt, W. T., Noser, K., Wilson, S. J., et al. (2000). What information do clinicians value for monitoring adolescent client progress and outcomes? Professional Psychology: Research and Practice, 31(1), 70–74.
Bickman, L., Andrade, A. R., Lambert, E. W., Doucette, A., Sapyta, J., Boyd, A. S., et al. (2004). Youth therapeutic alliance in intensive treatment settings. Journal of Behavioral Health Services and Research, 31(2), 134–148.
Bickman, L., Riemer, M., Lambert, E. W., Kelley, S. D., Breda, C., Dew, S., et al. (2007). Manual of the Peabody Treatment Progress Battery (Electronic version). Nashville, TN: Vanderbilt University. http://peabody.vanderbilt.edu/ptpb/.
Bickman, L., Athay, M. M., Riemer, M., Lambert, E. W., Kelley, S. D., Breda, C., Vides de Andrade, A. R. (2010). Manual of the Peabody Treatment Progress Battery (2nd ed.). Nashville, TN: Vanderbilt University. Retrieved from http://peabody.vanderbilt.edu/ptpb/.
Bickman, L., Kelley, S., Breda, C., Vides de Andrade, A. R., & Riemer, M. (2011). Effects of routine feedback to clinicians on youth mental health outcomes: A randomized cluster design. Psychiatric Services, 62(12), 1423–1429.
Bordin, E. S. (1979). The generalizability of the psychoanalytic concept of the working alliance. Psychotherapy: Theory, Research and Practice, 16(3), 252–260. doi:10.1037/h0085885.
Chiu, A. W., McLeod, B. D., Har, K., & Wood, J. J. (2009). Child-therapist alliance and clinical outcomes in cognitive behavioral therapy for child anxiety disorders. Journal of Child Psychology and Psychiatry and Allied Disciplines, 50(6), 751–758. doi:10.1111/j.1469-7610.2008.01996.x.
Chu, B. C., & Kendall, P. C. (2004). Positive association of child involvement and treatment outcome within a manual-based cognitive-behavioral treatment for children with anxiety. Journal of Consulting and Clinical Psychology, 72(5), 821–829.
Crits-Christoph, P., Gibbons, M. B. C., Hamilton, J., Ring-Kurtz, S., & Gallop, R. (2011). The dependability of alliance assessments: The alliance-outcome correlation is larger than you might think. Journal of Consulting and Clinical Psychology, 79(3), 267–278. doi:10.1037/a0023668.
Crocker, L., & Algina, J. (2006). Introduction to classical and modern test theory. Mason, OH: Wadsworth.
De Bolle, M., Johnson, J. G., & De Fruyt, F. (2010). Patient and clinician predictors of therapeutic alliance as predictors of improvement in depression. Psychotherapy Psychosomatics, 79(6), 378–385. doi:10.1159/000320895.
DeRubeis, R. J., & Feeley, M. (1990). Determinants of change in cognitive therapy for depression. Cognitive Therapy and Research, 14(5), 469–482. doi:10.1007/bf01172968.
DeRubeis, R. J., Brotman, M. A., & Gibbons, C. J. (2005). A conceptual and methodological analysis of the nonspecifics argument. Clinical Psychology: Science and Practice, 12(2), 174–183. doi:10.1093/clipsy.bpi022.
Eltz, M. J., Shirk, S. R., & Sarlin, N. (1995). Alliance formation and treatment outcome among maltreated adolescents. Child Abuse and Neglect, 19(4), 419–431. doi:10.1016/0145-2134(95)00008-v.
Faw, L., Hogue, A., Johnson, S., Diamond, G. M., & Liddle, H. A. (2005). The Adolescent Therapeutic Alliance Scale (ATAS): Initial psychometrics and prediction of outcome in family-based substance abuse prevention counseling. Psychotherapy Research, 15(1–2), 141–154.
Feeley, M., DeRubeis, R. J., & Gelfand, L. A. (1999). The temporal relation of adherence and alliance to symptom change in cognitive therapy for depression. Journal of Consulting and Clinical Psychology, 67(4), 578–582.
Fernandez, M., & Eyberg, S. (2009). Predicting treatment and follow-up attrition in parent–child interaction therapy. Journal of Abnormal Child Psychology, 37(3), 431–441. doi:10.1007/s10802-008-9281-1.
Friedlander, M. L., Lambert, J. E., & de la Peņa, C. M. (2008). A step toward disentangling the alliance/improvement cycle in family therapy. Journal of Counseling Psychology, 55(1), 118–124.
Gonzalez, A., Weersing, V., Warnick, E., Scahill, L., & Woolston, J. (2011). Predictors of treatment attrition among an outpatient clinic sample of youths with clinically significant anxiety. Administration and Policy in Mental Health and Mental Health Services Research, 38(5), 356–367. doi:10.1007/s10488-010-0323-y.
Green, J. (2009). The therapeutic alliance. Child: Care, Health and Development, 35(3), 298–301.
Hawley, K. M., & Garland, A. F. (2008). Working alliance in adolescent outpatient therapy: Youth, parent and therapist reports and associations with therapy outcomes. Child and Youth Care Forum, 37, 59–74.
Hawley, K. M., & Weisz, J. R. (2005). Youth versus parent working alliance in usual clinical care: Distinctive associations with retention, satisfaction, and treatment outcome. Journal of Clinical Child and Adolescent Psychology, 34(1), 117–128. doi:10.1207/s15374424jccp3401_11.
Hedeker, D., & Gibbons, R. D. (1997). Application of random-effects pattern-mixture models for missing data in longitudinal studies. Psychological Methods, 2(1), 64–78.
Hogue, A., Dauber, S., Stambaugh, L. F., Cecero, J. J., & Liddle, H. A. (2006). Early therapeutic alliance and treatment outcome in individual and family therapy for adolescent behavior problems. Journal of Consulting and Clinical Psychology, 74(1), 121–129.
Horvath, A. O. (2001). The alliance. Psychotherapy: Theory Research and Practice, 38(4), 365–372.
Horvath, A. O., Del Re, A. C., Flückiger, C., & Symonds, D. (2011). Alliance in individual psychotherapy. Psychotherapy: Theory/Research/Practice/Training, 48(1), 9–16. doi:10.1037/a0022186.
Hughes, A. A., & Kendall, P. C. (2007). Prediction of cognitive behavior treatment outcome for children with anxiety disorders: Therapeutic relationship and homework compliance. Behavioural and Cognitive Psychotherapy, 35(4), 487–494. doi:10.1017/s1352465807003761.
Karver, M. S., Handelsman, J. B., Fields, S., & Bickman, L. (2006). Meta-analysis of therapeutic relationship variables in youth and family therapy: The evidence for different relationship variables in the child and adolescent treatment outcome literature. Clinical Psychology Review, 26(1), 50–65. doi:10.1016/j.cpr.2005.09.001.
Karver, M. S., Shirk, S., Handelsman, J. B., Fields, S., Crisp, H., Gudmundsen, G., et al. (2008). Relationship processes in youth psychotherapy. Journal of Emotional and Behavioral Disorders, 16(1), 15–28. doi:10.1177/1063426607312536.
Kazdin, A. E. (1996). Dropping out of child psychotherapy: Issues for research and implications for practice. Clinical Child Psychology and Psychiatry, 1(1), 133–156. doi:10.1177/1359104596011012.
Kazdin, A. E., & Whitley, M. K. (2006). Pretreatment social relations, therapeutic alliance, and improvements in parenting practices in parent management training. Journal of Consulting and Clinical Psychology, 74, 346–355.
Kazdin, A. E., Marciano, P. L., & Whitley, M. (2005). The therapeutic alliance in cognitive-behavioral treatment of children referred for oppositional, aggressive, and antisocial behavior. Journal of Consulting and Clinical Psychology, 73, 726–730.
Kazdin, A. E., Whitley, M., & Marciano, P. L. (2006). Child-therapist and parent-therapist alliance and therapeutic change in the treatment of children referred for oppositional, aggressive, and antisocial behavior. Journal of Child Psychology and Psychiatry, 47, 436–445.
Kendall, P. C., Flannery-Schroeder, E., Panichelli-Mindel, S. M., Southam-Gerow, M., Henin, A., & Warman, M. (1997). Therapy for youths with anxiety disorders: A second randomized clinical trial. Journal of Consulting and Clinical Psychology, 65(3), 366–380.
Kendall, P. C., Comer, J. S., Marker, C. D., Creed, T. A., Puliafico, A. C., Hughes, A. A., et al. (2009). In-session exposure tasks and therapeutic alliance across the treatment of childhood anxiety disorders. Journal of Consulting and Clinical Psychology, 77(3), 517–525. doi:10.1037/a0013686.
Kivlighan, D. M., & Shaughnessy, P. (1995). Analysis of the development of the working alliance using hierarchical linear modeling. Journal of Counseling Psychology, 42(3), 338–349.
Kivlighan, D. M., & Shaughnessy, P. (2000). Patterns of working alliance development: A typology of client’s working alliance ratings. Journal of Counseling Psychology, 47(3), 362–371. doi:10.1037//0022-0167.47.3.362.
Kramer, U., de Roten, Y., Beretta, V., Michel, L., & Despland, J. N. (2009). Alliance patterns over the course of short-term dynamic psychotherapy: The shape of productive relationships. Psychotherapy Research, 19(6), 699–706. doi:10.1080/10503300902956742.
Lambert, M. J., Hansen, N. B., & Finch, A. E. (2001a). Patient-focused research: Using patient outcome data to enhance treatment effects. Journal of Consulting and Clinical Psychology, 69(2), 159–172.
Lambert, M. J., Whipple, J. L., Smart, D. W., Vermeersch, D. A., Nielsen, S. L., & Hawkins, E. J. (2001b). The effects of providing therapists with feedback on patient progress during psychotherapy: Are outcomes enhanced? Psychotherapy Research, 11(1), 49–68. doi:10.1093/ptr/11.1.49.
McLeod, B. D. (2011). Relation of the alliance with outcomes in youth psychotherapy: A meta-analysis. Clinical Psychology Review, 31(4), 603–616. doi:10.1016/j.cpr.2011.02.001.
Miller, L., Southam-Gerow, M., & Allin, R. (2008). Who stays in treatment? Child and family predictors of youth client retention in a public mental health agency. Child and Youth Care Forum, 37(4), 153–170. doi:10.1007/s10566-008-9058-2.
National Institute of Mental Health. (2001). Blueprint for change: Research on child and adolescent mental health’s council’s workgroup on child and adolescent mental health intervention development and deployment. (DHHS Publication No. CG 031–591). Washington, DC: US Government Printing Office.
Nich, C., & Carroll, K. (1997). Now you see it, now you don’t: A comparison of traditional versus random-effects regression models in the analysis of longitudinal follow-up data from a clinical trial. Journal of Consulting and Clinical Psychology, 65(2), 252–261.
Rector, N. A., Zuroff, D. C., & Segal, Z. V. (1999). Cognitive change and the therapeutic alliance: The role of technical and nontechnical factors in cognitive therapy. Psychotherapy, 36(4), 320–328.
Riemer, M., Athay, M. M., Bickman, L., Breda, C., Kelley, S. D., & Vides de Andrade, A. R. (2012). The Peabody Treatment Progress Battery: History and methods for developing a comprehensive measurement battery for youth mental health. Administration and Policy in Mental Health and Mental Health Services Research. doi:10.1007/s10488-012-0404-1.
Shelef, K., Diamond, G. M., Diamond, G. S., & Liddle, H. A. (2005). Adolescent and parent alliance and treatment outcome in multidimensional family therapy. Journal of Consulting and Clinical Psychology, 73(4), 689–698. doi:10.1037/0022-006x.73.4.689.
Shirk, S. R., & Karver, M. (2003). Prediction of treatment outcome from relationship variables in child and adolescent therapy: A meta-analytic review. Journal of Consulting and Clinical Psychology, 71(3), 452–464.
Shirk, S. R., Caporino, N., & Karver, M. (2010). The alliance in adolescent therapy: Conceptual, operational, and predictive issues. In D. Castro-Blanco & M. Karver (Eds.), Elusive alliance: Treatment engagement strategies with high-risk adolescents. Washington, DC: American Psychological Association Press.
Shirk, S. R., Karver, M. S., & Brown, R. (2011). The alliance in child and adolescent psychotherapy. Psychotherapy: Theory/Research/Practice/Training, 48(1), 17–24. doi:10.1037/a0022181.
Singer, J. D., & Willett, J. B. (2003). Applied longitudinal data analysis. New York: Oxford University Press.
Stiles, W. B., Glick, M. J., Osatuke, K., Hardy, G. E., Shapiro, D. A., Agnew-Davies, R., et al. (2004). Patterns of alliance development and the rupture-repair hypothesis: Are productive relationships U-shaped or V-shaped? Journal of Counseling Psychology, 51(1), 81–92. doi:10.1037/0022-0167.51.1.81.
Topham, G. L., Wampler, K. S., Titus, G., & Rolling, E. (2011). Predicting parent and child outcomes of a filial therapy program. International Journal of Play Therapy, 20(2), 79–93. doi:10.1037/a0023261.
Wright, B. D., & Linacre, J. M. (1994). Reasonable mean-square fit values. Rasch Measurement Transactions, 8, 370. Retrieved 10 March 2011 from http://www.rasch.org/rmt/rmt83b.htm.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Bickman, L., de Andrade, A.R.V., Athay, M.M. et al. The Relationship Between Change in Therapeutic Alliance Ratings and Improvement in Youth Symptom Severity: Whose Ratings Matter the Most?. Adm Policy Ment Health 39, 78–89 (2012). https://doi.org/10.1007/s10488-011-0398-0
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10488-011-0398-0