The Relationship Between Change in Therapeutic Alliance Ratings and Improvement in Youth Symptom Severity: Whose Ratings Matter the Most?

  • Leonard BickmanEmail author
  • Ana Regina Vides de Andrade
  • M. Michele Athay
  • Jason I. Chen
  • Alessandro S. De Nadai
  • Brittany L. Jordan-Arthur
  • Marc S. Karver
Original Paper


This paper presents the psychometric evaluation of brief measures of therapeutic alliance (TA) for youths, clinicians and caregivers and a longitudinal analysis of relationships between changes in TA and changes in youth symptom and functioning severity. Psychometric analyses using methods from Classical Test Theory, Item Response Theory, and Factor Analysis indicate that the measures of TA used in this study offer something new for both practice and research. The measures have variability, sensitivity to change over time, brevity and can be used with multiple parties through parallel forms. The longitudinal analyses, employing hierarchical linear modeling with time-varying covariates, found that TA ratings of the clinician correlated with symptom improvement as rated by the clinician, caregiver and youth. Additional analyses showed that decreases in clinician-rated youth TA was most important in predicting a lower rate of youth improvement. Implications for future research and clinical practice are discussed.


Therapeutic alliance Youth mental health Improvement Ratings of therapeutic alliance 


  1. Anker, M. G., Owen, J., Duncan, B. L., & Sparks, J. A. (2010). The alliance in couple therapy: Partner influence, early change, and alliance patterns in a naturalistic sample. Journal of Consulting and Clinical Psychology, 78(5), 635–645. doi: 10.1037/a0020051.PubMedCrossRefGoogle Scholar
  2. Armbruster, P., & Kazdin, A. E. (1994). Attrition in child-psychotherapy. Advances in Clinical Child Psychology, 16(16), 81–108.Google Scholar
  3. Athay, M. M., Riemer, M., & Bickman, L. (2012). The Symptoms and Functioning Severity Scale (SFSS): Psychometric evaluation and discrepancies among youth, caregiver, and clinician ratings over time. Administration and Policy in Mental Health and Mental Health Services Research. doi: 10.1007/s10488-012-0403-2.
  4. Barber, J. P., Connoly, M. B., Crits-Christoph, P., Gladis, L., & Siqueland, L. (2009). Alliance predicts patients’ outcome beyond in-treatment change in symptoms. Personality Disorders: Theory, Research, and Treatment, S(1), 80–89. doi: 10.1037/1949-2715.S.1.80.CrossRefGoogle Scholar
  5. Bickman, L. (2008). A measurement feedback system (MFS) is necessary to improve mental health outcomes. Journal of the American Academy of Child and Adolescent Psychiatry, 47(10), 1114–1119. doi: 10.1097/CHI.0b013e3181825af8.PubMedCrossRefGoogle Scholar
  6. Bickman, L., Rosof-Williams, J., Salzer, M. S., Summerfelt, W. T., Noser, K., Wilson, S. J., et al. (2000). What information do clinicians value for monitoring adolescent client progress and outcomes? Professional Psychology: Research and Practice, 31(1), 70–74.CrossRefGoogle Scholar
  7. Bickman, L., Andrade, A. R., Lambert, E. W., Doucette, A., Sapyta, J., Boyd, A. S., et al. (2004). Youth therapeutic alliance in intensive treatment settings. Journal of Behavioral Health Services and Research, 31(2), 134–148.PubMedGoogle Scholar
  8. Bickman, L., Riemer, M., Lambert, E. W., Kelley, S. D., Breda, C., Dew, S., et al. (2007). Manual of the Peabody Treatment Progress Battery (Electronic version). Nashville, TN: Vanderbilt University.
  9. Bickman, L., Athay, M. M., Riemer, M., Lambert, E. W., Kelley, S. D., Breda, C., Vides de Andrade, A. R. (2010). Manual of the Peabody Treatment Progress Battery (2nd ed.). Nashville, TN: Vanderbilt University. Retrieved from
  10. Bickman, L., Kelley, S., Breda, C., Vides de Andrade, A. R., & Riemer, M. (2011). Effects of routine feedback to clinicians on youth mental health outcomes: A randomized cluster design. Psychiatric Services, 62(12), 1423–1429.PubMedCrossRefGoogle Scholar
  11. Bordin, E. S. (1979). The generalizability of the psychoanalytic concept of the working alliance. Psychotherapy: Theory, Research and Practice, 16(3), 252–260. doi: 10.1037/h0085885.CrossRefGoogle Scholar
  12. Chiu, A. W., McLeod, B. D., Har, K., & Wood, J. J. (2009). Child-therapist alliance and clinical outcomes in cognitive behavioral therapy for child anxiety disorders. Journal of Child Psychology and Psychiatry and Allied Disciplines, 50(6), 751–758. doi: 10.1111/j.1469-7610.2008.01996.x.CrossRefGoogle Scholar
  13. Chu, B. C., & Kendall, P. C. (2004). Positive association of child involvement and treatment outcome within a manual-based cognitive-behavioral treatment for children with anxiety. Journal of Consulting and Clinical Psychology, 72(5), 821–829.PubMedCrossRefGoogle Scholar
  14. Crits-Christoph, P., Gibbons, M. B. C., Hamilton, J., Ring-Kurtz, S., & Gallop, R. (2011). The dependability of alliance assessments: The alliance-outcome correlation is larger than you might think. Journal of Consulting and Clinical Psychology, 79(3), 267–278. doi: 10.1037/a0023668.PubMedCrossRefGoogle Scholar
  15. Crocker, L., & Algina, J. (2006). Introduction to classical and modern test theory. Mason, OH: Wadsworth.Google Scholar
  16. De Bolle, M., Johnson, J. G., & De Fruyt, F. (2010). Patient and clinician predictors of therapeutic alliance as predictors of improvement in depression. Psychotherapy Psychosomatics, 79(6), 378–385. doi: 10.1159/000320895.CrossRefGoogle Scholar
  17. DeRubeis, R. J., & Feeley, M. (1990). Determinants of change in cognitive therapy for depression. Cognitive Therapy and Research, 14(5), 469–482. doi: 10.1007/bf01172968.CrossRefGoogle Scholar
  18. DeRubeis, R. J., Brotman, M. A., & Gibbons, C. J. (2005). A conceptual and methodological analysis of the nonspecifics argument. Clinical Psychology: Science and Practice, 12(2), 174–183. doi: 10.1093/clipsy.bpi022.CrossRefGoogle Scholar
  19. Eltz, M. J., Shirk, S. R., & Sarlin, N. (1995). Alliance formation and treatment outcome among maltreated adolescents. Child Abuse and Neglect, 19(4), 419–431. doi: 10.1016/0145-2134(95)00008-v.PubMedCrossRefGoogle Scholar
  20. Faw, L., Hogue, A., Johnson, S., Diamond, G. M., & Liddle, H. A. (2005). The Adolescent Therapeutic Alliance Scale (ATAS): Initial psychometrics and prediction of outcome in family-based substance abuse prevention counseling. Psychotherapy Research, 15(1–2), 141–154.CrossRefGoogle Scholar
  21. Feeley, M., DeRubeis, R. J., & Gelfand, L. A. (1999). The temporal relation of adherence and alliance to symptom change in cognitive therapy for depression. Journal of Consulting and Clinical Psychology, 67(4), 578–582.PubMedCrossRefGoogle Scholar
  22. Fernandez, M., & Eyberg, S. (2009). Predicting treatment and follow-up attrition in parent–child interaction therapy. Journal of Abnormal Child Psychology, 37(3), 431–441. doi: 10.1007/s10802-008-9281-1.PubMedCrossRefGoogle Scholar
  23. Friedlander, M. L., Lambert, J. E., & de la Peņa, C. M. (2008). A step toward disentangling the alliance/improvement cycle in family therapy. Journal of Counseling Psychology, 55(1), 118–124.CrossRefGoogle Scholar
  24. Gonzalez, A., Weersing, V., Warnick, E., Scahill, L., & Woolston, J. (2011). Predictors of treatment attrition among an outpatient clinic sample of youths with clinically significant anxiety. Administration and Policy in Mental Health and Mental Health Services Research, 38(5), 356–367. doi: 10.1007/s10488-010-0323-y.PubMedCrossRefGoogle Scholar
  25. Green, J. (2009). The therapeutic alliance. Child: Care, Health and Development, 35(3), 298–301.CrossRefGoogle Scholar
  26. Hawley, K. M., & Garland, A. F. (2008). Working alliance in adolescent outpatient therapy: Youth, parent and therapist reports and associations with therapy outcomes. Child and Youth Care Forum, 37, 59–74.CrossRefGoogle Scholar
  27. Hawley, K. M., & Weisz, J. R. (2005). Youth versus parent working alliance in usual clinical care: Distinctive associations with retention, satisfaction, and treatment outcome. Journal of Clinical Child and Adolescent Psychology, 34(1), 117–128. doi: 10.1207/s15374424jccp3401_11.PubMedCrossRefGoogle Scholar
  28. Hedeker, D., & Gibbons, R. D. (1997). Application of random-effects pattern-mixture models for missing data in longitudinal studies. Psychological Methods, 2(1), 64–78.CrossRefGoogle Scholar
  29. Hogue, A., Dauber, S., Stambaugh, L. F., Cecero, J. J., & Liddle, H. A. (2006). Early therapeutic alliance and treatment outcome in individual and family therapy for adolescent behavior problems. Journal of Consulting and Clinical Psychology, 74(1), 121–129.PubMedCrossRefGoogle Scholar
  30. Horvath, A. O. (2001). The alliance. Psychotherapy: Theory Research and Practice, 38(4), 365–372.CrossRefGoogle Scholar
  31. Horvath, A. O., Del Re, A. C., Flückiger, C., & Symonds, D. (2011). Alliance in individual psychotherapy. Psychotherapy: Theory/Research/Practice/Training, 48(1), 9–16. doi: 10.1037/a0022186.CrossRefGoogle Scholar
  32. Hughes, A. A., & Kendall, P. C. (2007). Prediction of cognitive behavior treatment outcome for children with anxiety disorders: Therapeutic relationship and homework compliance. Behavioural and Cognitive Psychotherapy, 35(4), 487–494. doi: 10.1017/s1352465807003761.CrossRefGoogle Scholar
  33. Karver, M. S., Handelsman, J. B., Fields, S., & Bickman, L. (2006). Meta-analysis of therapeutic relationship variables in youth and family therapy: The evidence for different relationship variables in the child and adolescent treatment outcome literature. Clinical Psychology Review, 26(1), 50–65. doi: 10.1016/j.cpr.2005.09.001.PubMedCrossRefGoogle Scholar
  34. Karver, M. S., Shirk, S., Handelsman, J. B., Fields, S., Crisp, H., Gudmundsen, G., et al. (2008). Relationship processes in youth psychotherapy. Journal of Emotional and Behavioral Disorders, 16(1), 15–28. doi: 10.1177/1063426607312536.CrossRefGoogle Scholar
  35. Kazdin, A. E. (1996). Dropping out of child psychotherapy: Issues for research and implications for practice. Clinical Child Psychology and Psychiatry, 1(1), 133–156. doi: 10.1177/1359104596011012.CrossRefGoogle Scholar
  36. Kazdin, A. E., & Whitley, M. K. (2006). Pretreatment social relations, therapeutic alliance, and improvements in parenting practices in parent management training. Journal of Consulting and Clinical Psychology, 74, 346–355.PubMedCrossRefGoogle Scholar
  37. Kazdin, A. E., Marciano, P. L., & Whitley, M. (2005). The therapeutic alliance in cognitive-behavioral treatment of children referred for oppositional, aggressive, and antisocial behavior. Journal of Consulting and Clinical Psychology, 73, 726–730.PubMedCrossRefGoogle Scholar
  38. Kazdin, A. E., Whitley, M., & Marciano, P. L. (2006). Child-therapist and parent-therapist alliance and therapeutic change in the treatment of children referred for oppositional, aggressive, and antisocial behavior. Journal of Child Psychology and Psychiatry, 47, 436–445.PubMedCrossRefGoogle Scholar
  39. Kendall, P. C., Flannery-Schroeder, E., Panichelli-Mindel, S. M., Southam-Gerow, M., Henin, A., & Warman, M. (1997). Therapy for youths with anxiety disorders: A second randomized clinical trial. Journal of Consulting and Clinical Psychology, 65(3), 366–380.PubMedCrossRefGoogle Scholar
  40. Kendall, P. C., Comer, J. S., Marker, C. D., Creed, T. A., Puliafico, A. C., Hughes, A. A., et al. (2009). In-session exposure tasks and therapeutic alliance across the treatment of childhood anxiety disorders. Journal of Consulting and Clinical Psychology, 77(3), 517–525. doi: 10.1037/a0013686.PubMedCrossRefGoogle Scholar
  41. Kivlighan, D. M., & Shaughnessy, P. (1995). Analysis of the development of the working alliance using hierarchical linear modeling. Journal of Counseling Psychology, 42(3), 338–349.CrossRefGoogle Scholar
  42. Kivlighan, D. M., & Shaughnessy, P. (2000). Patterns of working alliance development: A typology of client’s working alliance ratings. Journal of Counseling Psychology, 47(3), 362–371. doi: 10.1037//0022-0167.47.3.362.CrossRefGoogle Scholar
  43. Kramer, U., de Roten, Y., Beretta, V., Michel, L., & Despland, J. N. (2009). Alliance patterns over the course of short-term dynamic psychotherapy: The shape of productive relationships. Psychotherapy Research, 19(6), 699–706. doi: 10.1080/10503300902956742.PubMedCrossRefGoogle Scholar
  44. Lambert, M. J., Hansen, N. B., & Finch, A. E. (2001a). Patient-focused research: Using patient outcome data to enhance treatment effects. Journal of Consulting and Clinical Psychology, 69(2), 159–172.PubMedCrossRefGoogle Scholar
  45. Lambert, M. J., Whipple, J. L., Smart, D. W., Vermeersch, D. A., Nielsen, S. L., & Hawkins, E. J. (2001b). The effects of providing therapists with feedback on patient progress during psychotherapy: Are outcomes enhanced? Psychotherapy Research, 11(1), 49–68. doi: 10.1093/ptr/11.1.49.CrossRefGoogle Scholar
  46. McLeod, B. D. (2011). Relation of the alliance with outcomes in youth psychotherapy: A meta-analysis. Clinical Psychology Review, 31(4), 603–616. doi: 10.1016/j.cpr.2011.02.001.PubMedCrossRefGoogle Scholar
  47. Miller, L., Southam-Gerow, M., & Allin, R. (2008). Who stays in treatment? Child and family predictors of youth client retention in a public mental health agency. Child and Youth Care Forum, 37(4), 153–170. doi: 10.1007/s10566-008-9058-2.PubMedCrossRefGoogle Scholar
  48. National Institute of Mental Health. (2001). Blueprint for change: Research on child and adolescent mental health’s council’s workgroup on child and adolescent mental health intervention development and deployment. (DHHS Publication No. CG 031–591). Washington, DC: US Government Printing Office.Google Scholar
  49. Nich, C., & Carroll, K. (1997). Now you see it, now you don’t: A comparison of traditional versus random-effects regression models in the analysis of longitudinal follow-up data from a clinical trial. Journal of Consulting and Clinical Psychology, 65(2), 252–261.PubMedCrossRefGoogle Scholar
  50. Rector, N. A., Zuroff, D. C., & Segal, Z. V. (1999). Cognitive change and the therapeutic alliance: The role of technical and nontechnical factors in cognitive therapy. Psychotherapy, 36(4), 320–328.CrossRefGoogle Scholar
  51. Riemer, M., Athay, M. M., Bickman, L., Breda, C., Kelley, S. D., & Vides de Andrade, A. R. (2012). The Peabody Treatment Progress Battery: History and methods for developing a comprehensive measurement battery for youth mental health. Administration and Policy in Mental Health and Mental Health Services Research. doi: 10.1007/s10488-012-0404-1.
  52. Shelef, K., Diamond, G. M., Diamond, G. S., & Liddle, H. A. (2005). Adolescent and parent alliance and treatment outcome in multidimensional family therapy. Journal of Consulting and Clinical Psychology, 73(4), 689–698. doi: 10.1037/0022-006x.73.4.689.PubMedCrossRefGoogle Scholar
  53. Shirk, S. R., & Karver, M. (2003). Prediction of treatment outcome from relationship variables in child and adolescent therapy: A meta-analytic review. Journal of Consulting and Clinical Psychology, 71(3), 452–464.PubMedCrossRefGoogle Scholar
  54. Shirk, S. R., Caporino, N., & Karver, M. (2010). The alliance in adolescent therapy: Conceptual, operational, and predictive issues. In D. Castro-Blanco & M. Karver (Eds.), Elusive alliance: Treatment engagement strategies with high-risk adolescents. Washington, DC: American Psychological Association Press.Google Scholar
  55. Shirk, S. R., Karver, M. S., & Brown, R. (2011). The alliance in child and adolescent psychotherapy. Psychotherapy: Theory/Research/Practice/Training, 48(1), 17–24. doi: 10.1037/a0022181.CrossRefGoogle Scholar
  56. Singer, J. D., & Willett, J. B. (2003). Applied longitudinal data analysis. New York: Oxford University Press.CrossRefGoogle Scholar
  57. Stiles, W. B., Glick, M. J., Osatuke, K., Hardy, G. E., Shapiro, D. A., Agnew-Davies, R., et al. (2004). Patterns of alliance development and the rupture-repair hypothesis: Are productive relationships U-shaped or V-shaped? Journal of Counseling Psychology, 51(1), 81–92. doi: 10.1037/0022-0167.51.1.81.CrossRefGoogle Scholar
  58. Topham, G. L., Wampler, K. S., Titus, G., & Rolling, E. (2011). Predicting parent and child outcomes of a filial therapy program. International Journal of Play Therapy, 20(2), 79–93. doi: 10.1037/a0023261.CrossRefGoogle Scholar
  59. Wright, B. D., & Linacre, J. M. (1994). Reasonable mean-square fit values. Rasch Measurement Transactions, 8, 370. Retrieved 10 March 2011 from

Copyright information

© Springer Science+Business Media, LLC 2012

Authors and Affiliations

  • Leonard Bickman
    • 1
    Email author
  • Ana Regina Vides de Andrade
    • 1
  • M. Michele Athay
    • 1
  • Jason I. Chen
    • 2
  • Alessandro S. De Nadai
    • 2
  • Brittany L. Jordan-Arthur
    • 2
  • Marc S. Karver
    • 2
  1. 1.Vanderbilt UniversityNashvilleUSA
  2. 2.Department of PsychologyUniversity of South FloridaTampaUSA

Personalised recommendations