Skip to main content
SpringerLink
Log in

Expanding the Conceptualization of Outcome and Clinical Effectiveness

  • Original Paper
  • Published:
Journal of Contemporary Psychotherapy Aims and scope Submit manuscript

Abstract

There is a need for studies that advance our knowledge of therapist effectiveness, expanding the definition of what constitutes therapeutic success. In the present study, four aspects of clinical outcome were analyzed using a sample of highly experienced psychotherapists (mean years of experience as therapy practitioners = 30) who also serve as clinical teachers. The four aspects were: (1) overall change in different outcome domains, including long-term outcome; (2) outcome with clinically distressed clients (i.e. clients above the level of clinical disturbance at pretreatment); (3) level of drop-out; and (4) the degree to which clients re-entered treatment during the follow-up period. Participants were 48 clients treated by 18 therapists. Treatments were open-ended, ‘therapy as usual’, mostly within the setting of independent private practice. Outcome measurements included the OQ-45.2 and IIP-64, with repeated measurements during therapy and extended long-term follow up (up to 3.5 years). Cohen’s d effect sizes and multilevel growth trajectories were estimated for patient change in symptomatic and interpersonal distress. The findings indicate that these therapists obtain very good outcomes overall, particularly with clients above cut-off for clinical disturbance. Moreover, there was a high level of maintenance and increase of treatment gains over time. No clients dropped out. Overall, the findings suggest that a sample of highly experienced therapists obtains particularly good outcomes as defined in more comprehensive terms than what is typical in psychotherapy research. Implications for the understanding of therapeutic success and clinical expertise are discussed.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Amble, I., Gude, T., Stubdal, S., Øktedalen, T., Skjorten, A. M., Andersen, B. J., et al. (2014). Psychometric properties of the Outcome Questionnaire-45.2: The Norwegian version in an international context. Psychotherapy Research, 24(4), 504–513. https://doi.org/10.1080/10503307.2013.849016.

    Article  PubMed  Google Scholar 

  • Baldwin, S. A., & Imel, Z. (2013). Therapist effects. In M. J. Lambert (Ed.), Bergin and Garfield’s handbook of psychotherapy and behavior change (6th ed., pp. 258–297). New York: Wiley.

    Google Scholar 

  • Beutler, L. E., Malik, M., Alimohamed, S., Harwood, T. M., Talebi, H., Noble, S., & Wong, E. (2004). Therapist variables. In M. J. Lambert (Ed.), Bergin and Garfield‘s handbook of psychotherapy and behavior change (pp. 227–306). New York: Wiley.

    Google Scholar 

  • Bludworth, J. L., Tracey, T. J. G., & Glidden-Tracey, C. (2010). The bilevel structure of the Outcome Questionnaire-45. Psychological Assessment, 22(2), 350–355. https://doi.org/10.1037/a0019187.

    Article  PubMed  Google Scholar 

  • Burlingame, G., Paul, S., Fuhriman, A., & Ogles, B. M. (1989). Implementing a time-limited therapy program: Differential effects of training and experience. Psychotherapy: Theory/Research/Practice/Training, 26, 303–313.

    Article  Google Scholar 

  • Chi, M. T. H. (2006). Two approaches to the study of experts’ characteristics. In K. A. Ericsson, N. Charness, R. R. Hoffman & P. J. Feltovich (Eds.), Cambridge handbook of expertise and expert performance (pp. 21–30). New York: Cambridge University Press.

    Chapter  Google Scholar 

  • Cohen, J. (1977). Statistical power analysis for behavioural sciences. New York: Academic Press.

    Google Scholar 

  • Dinger, U., Zimmermann, J., Masuhr, O., & Spitzer, C. (2016). Therapist effects on outcome and alliance in inpatient psychotherapy: The contribution of patients’ symptom severity. Psychotherapy. https://doi.org/10.1037/pst0000059.

    Article  PubMed  Google Scholar 

  • Ekroll, V. B., & Rønnestad, M. H. (2016). Processes and changes experienced by clients during and after naturalistic good outcome therapies conducted by experienced psychotherapists. Psychotherapy Research, 27, 450–468.

    Article  Google Scholar 

  • Ekroll, V. B., & Rønnestad, M. H. (2018). Exploring associations between therapy factors and post- therapy development after naturalistic psychotherapies. Journal of Contemporary Psychotherapy, 48(1), 115–125. https://doi.org/10.1007/s10879-017-9366-7.

    Article  Google Scholar 

  • Goldberg, S. B., Babins-Wagner, R., Rousmaniere, T., Berzins, S., Hoyt, W. T., Whipples, J. L., et al. (2016c). Creating a climate for therapist inprovement: A case study of an agency focused on outcome and deliberate practice. Psychotherapy, 53, 367–375. https://doi.org/10.1037/pst0000060.

    Article  PubMed  Google Scholar 

  • Goldberg, S. B., Hoyt, W. T., Nissen-Lie, H. A., Nielsen, S. L., & Wampold, B. (2016b). Unpacking the therapist effect: Impact of treatment length differs for high- and low-performing therapists. Psychotherapy Research. https://doi.org/10.1080/10503307.2016.1216625.

    Article  PubMed  Google Scholar 

  • Goldberg, S. B., Rousmaniere, T., Miller, S. D., Whipple, J., Nielsen, S. L., Hoyt, W. T., & Wampold, B. E. (2016a). Do psychotherapists improve with time and experience? A longitudinal analysis of outcomes in a clinical setting. Journal of Counseling Psychology, 63(1), 1–11. https://doi.org/10.1037/cou0000131.

    Article  PubMed  Google Scholar 

  • Gurtman, M. B., & Balakrishnan, J. D. (1998). Circular measurement redux: The analysis and interpretation of Interpersonal Circle Profiles. Clinical psychology: Science and Practice, 5(3), 344–360. https://doi.org/10.1111/j.1468-2850.1998.tb00154.x.

    Article  Google Scholar 

  • Hatchett, G. T., & Park, H. L. (2003). Comparisons of four operational definitions of premature termination. Psychotherapy: Theory, Research, Practice, Training, 40, 226–231. https://doi.org/10.1037/0033-3204.40.3.226.

    Article  Google Scholar 

  • Hill, C. E., Spiegel, S. B., Hoffman, M. A., Kivlighan, D. M., Jr., & Gelso, C. J. (2017). Therapist expertise in psychotherapy revisited. The Counseling Psychologist, 45, 7–53. https://doi.org/10.1177/0011000016641192.

    Article  Google Scholar 

  • Horowitz, L. M., Alden, L. E., Wiggins, J. S., & Pincus, A. L. (2000). IIP-64/IIP-32 professional manual. San Antonio: The Psychological Corporation.

    Google Scholar 

  • Horvath, A. O., & Bedi, R. P. (2002). The alliance. In J. C. Norcross (Ed.), Psychotherapy relationships that work (pp. 37–69). New York: Oxford University Press.

    Google Scholar 

  • Hox, J. (2010). Multilevel analysis. Techniques and applications (2nd ed.). Mahwah: Lawrence Erlbaum Associates.

    Book  Google Scholar 

  • Hubble, M. A., Duncan, B. L., Miller, S. D., & Wampold, B. E. (2009). Introduction. In B. L. Duncan, S. D. Miller, B. E. Wampold & M. A. Hubble (Eds.), The heart and soul of change: Delivering what works in therapy (pp. 23–46). Washington, DC: American Psychological Association.

    Google Scholar 

  • Jacobson, N. S., & Truax, P. (1991). Clinical significance: A statistical approach to defining meaningful change in psychotherapy research. Journal of Consulting and Clinical Psychology, 59(1), 12–19. https://doi.org/10.1037/0022-006X.59.1.12.

    Article  CAS  PubMed  Google Scholar 

  • Jennings, L., & Skovholt, T. M. (2016). Expertise in counseling and psychotherapy: Master therapist studies from around the world. New York: Oxford University Press.

    Book  Google Scholar 

  • Jennings, L., Skovholt, T. M., Goh, M., & Lian, F. (2013). Master therapists: Explorations of expertise. In M. H. Rønnestad & T. M. Skovholt (Eds.), The developing practitioner: Growth and stagnation of therapists and counselors (pp. 213–264). New York: Routledge.

    Google Scholar 

  • Keefe, J. R., McCarthy, K. S., Dinger, U., Zilcha-Mano, S., & Barber, J. P. (2014). A meta-analytic review of psychodynamic therapies for anxiety disorders. Clinical Psychology Review, 34, 309–323. https://doi.org/10.1016/j.cpr.2014.03.004.

    Article  PubMed  Google Scholar 

  • Kim, S., Beretvas, S., & Sherry, A. R. (2010). A validation of the factor structure of OQ-45 scores using factor mixture modeling. Measurement and Evaluation in Counseling and Development, 42, 275–295. https://doi.org/10.1177/0748175609354616.

    Article  Google Scholar 

  • Kivlighan, D. M. III, Goldberg, S. B., Abbas, M., Pace, B. T., Yulish, N. E., Thomas, J. G., et al. (2015). The enduring effects of psychodynamic treatments vis-à-vis alternative treatments: A multilevel longitudinal meta-analysis.. Clinical Psychology Review, 40, 1–14. https://doi.org/10.1016/j.cpr.2015.05.003.

    Article  PubMed  Google Scholar 

  • Lambert, M. J., Burlingame, G. M., Umphress, V., Hansen, N. B., Vermeersch, D. A., Clouse, G. C., & Yanchar, S. C. (1996). The reliability and validity of the Outcome Questionnaire. Clinical Psychology & Psychotherapy, 3, 249–258. https://doi.org/10.1002/(SICI)1099-0879(199612)3.

    Article  Google Scholar 

  • Lambert, M. J., Morton, J. J., Hatfield, D., Harmon, C., Hamilton, S., Reid, R. C., et al. (2004). Administration and scoring manual for the OQ®-45.2. OQ-Measures: Brigham Young University.

    Google Scholar 

  • Lambert, M. J., & Ogles, B. M. (2004). The efficacy and effectiveness of psychotherapy. In M. J. Lambert (Ed.), Bergin and Garfield‘s handbook of psychotherapy and behavior change (pp. 139–193). New York: Wiley. https://doi.org/10.1002/(SICI)1099-0879(199612)3:4%3C249::AID-CPP106%3E3.0.CO;2-S

    Chapter  Google Scholar 

  • Laska, K. M., Smith, T. L., Wislocki, A. P., & Wampold, B. E. (2013). Uniformity of evidence-based treatments in practice? Therapist effects in the delivery of cognitive processing therapy for PTSD. Journal of Counseling Psychology, 60, 31–41. https://doi.org/10.1037/a0031294.

    Article  PubMed  Google Scholar 

  • Leichsenring, F., & Rabung, S. (2008). Effectiveness of long-term psychodynamic psychotherapy: A meta-analysis. JAMA, 300(13), 1551–1565. https://doi.org/10.1001/jama.300.13.1551.

    Article  CAS  PubMed  Google Scholar 

  • Liberman, B. L. (1978). The role of mastery in psychotherapy: Maintenance of improvement and prescriptive change. In J. D. Frank, R. S. D. Hoehn-Saric, B. L. Imber, A. Liberman & R. Stone (Eds.), Effective ingredients of successful psychotherapy (pp. 35–72). New York: Brunner/Mazel.

    Google Scholar 

  • Luborsky, L., McLellan, A. T., Diguer, L., Woody, G., & Seligman, D. A. (1997). The psychotherapist matters: Comparison of outcomes across twenty-two therapists and seven patient samples. Clinical Psychology, 4(1), 53–65. https://doi.org/10.1111/j.1468-2850.1997.tb00099.x.

    Article  Google Scholar 

  • Monsen, J., Odland, T., Faugli, A., Daae, E., & Eilertsen, D. E. (1995). Personality disorders: Changes and stability after intensive psychotherapy focusing on affect consciousness. Psychotherapy Research, 5(1), 33–48.

    Article  Google Scholar 

  • Monsen, J. T., Hagtvet, K. A., Havik, O. E., & Eilertsen, D. E. (2006). Circumplex Structure and personality disorder correlates of the Interpersonal Problems Model (IIP-C): Construct validity and clinical implications. Psychological Assessment, 18(2), 165–173.

    Article  Google Scholar 

  • Okiishi, J., Lambert, M. J., Nielsen, S. L., & Ogles, B. M. (2003). Waiting for supershrink: An empirical analysis of therapist effects. Clinical Psychology and Psychotherapy, 10, 361–373. https://doi.org/10.1002/cpp.383.

    Article  Google Scholar 

  • Orlinsky, D. E., Botermans, J.-F., & Rønnestad, M. H. (2001). Towards an empirically, grounded model of psychotherapy training: Four thousand therapists rate influences on their development. Australian Psychologist, 36, 139–148.

    Article  Google Scholar 

  • Orlinsky, D. E., & Rønnestad, M. H. (2005). How psychotherapists develop: A study of therapeutic work and professional growth. Washington DC: American Psychological Association.

    Book  Google Scholar 

  • Orlinsky, D. E., Rønnestad, M. H., & Willutzki, U. (2004). Fifty years of psychotherapy process-outcome research: Continuity and change. In M. Lambert (Ed.), Bergin and Garfield’s hand- book of psychotherapy and behavior change (5th ed., pp. 307–389). New York: Wiley.

    Google Scholar 

  • Raudenbush, S. W., & Bryk, A. S. (2002). Hierarchical linear models (2nd ed.). Thousand Oaks: Sage Publications.

    Google Scholar 

  • Rønnestad, M. H. (2009). An intensive process-outcome study of the interpersonal aspects of psychotherapy. Research protocol, Department of Psychology, University of Oslo, Norway.

  • Rønnestad, M. H. (2016). Is expertise in psychotherapy a useful construct? Psychotherapy Bulletin, 51(1), 11–13.

    Google Scholar 

  • Rønnestad, M. H., & Skovholt, T. M. (2013). The developing practitioner: Growth and stagnation of therapists and counselors. New York: Routledge.

    Google Scholar 

  • Roos, J., & Werbart, A. (2013). Therapist and relationship factors influencing dropout from individual psychotherapy: A literature review. Psychotherapy Research, 23(4), 394–418. https://doi.org/10.1080/10503307.2013.775528.

    Article  PubMed  Google Scholar 

  • Rosenthal, J. A. (1996). Qualitative descriptors of strength of association and effect size. Journal of Social Service Research, 21, 37–59.

    Article  Google Scholar 

  • Rosenthal, R., & Rosnow, R. L. (1984). Essentials of behavioral research: Methods and data analysis. New York: McGraw-Hill.

    Google Scholar 

  • Rousmaniere, T. (2017). Deliberate practice for psychotherapists: A Guide to Improving Clinical Effectiveness. New York: Routledge.

    Google Scholar 

  • Sandell, R., Blomberg, J., Lazar, A., Carlsson, J., Broberg, J., & Schubert, J. (2000). Varieties of long-term outcome among patients in psychoanalysis and long-term psycho- therapy: A review of findings in the Stockholm Outcome of Psychoanalysis and Psychotherapy Project (STOPPP). International Journal of Psychoanalysis, 81, 921–942.

    Article  Google Scholar 

  • Saxon, D., & Barkham, M. (2012). Patterns of therapist variability: Therapist effects and the contribution of patient severity and risk. Journal of Consulting and Clinical Psychology, 80(4), 535–546. https://doi.org/10.1037/a0028898.

    Article  PubMed  Google Scholar 

  • Shanteau, J. (1992). Competence in experts: The role of task characteristics. Organizational Behavior and Human Decision Processes, 53, 252–266. https://doi.org/10.1016/0749-5978(92)90064-E.

    Article  Google Scholar 

  • Skovholt, T. M., Rønnestad, M. H., & Jennings, L. (1997). Searching for expertise in counseling, psychotherapy, and professional psychology. Educational Psychology Review, 9(4), 361–369. https://doi.org/10.1023/A:1024798723295.

    Article  Google Scholar 

  • Smith, M. L., & Glass, G. V. (1977). Meta-analysis of psychotherapy outcome studies. American Psychologist, 32, 752–760. https://doi.org/10.1037/0003-066X.32.9.752.

    Article  CAS  PubMed  Google Scholar 

  • Snijders, T. A., & Bosker, R. J. (2012). Multilevel analysis: An introduction to basic and advanced multilevel modeling (2nd ed.). London: Sage.

    Google Scholar 

  • Stein, D. M., & Lambert, M. J. (1984). On the relationship between therapist experience and psychotherapy outcome. Clinical Psychology Review, 4, 127–142. https://doi.org/10.1016/0272-7358(84)90025-4.

    Article  Google Scholar 

  • Tasca, G. A., & Gallop, R. (2009). Multilevel modeling of longitudinal data for psychotherapy researchers: I. The basics. Psychotherapy Research, 19(4–5), 429–437.

    Article  Google Scholar 

  • Tracey, J. G., Wampold, B. E., Lichtenberg, J. W., & Goodyear, R. K. (2014). Expertise in psychotherapy: An elusive goal?. American Psychologist, 69, 218–229. https://doi.org/10.1037/a0035099.

    Article  PubMed  Google Scholar 

  • Tschuschke, V., Crameri, A., Koehler, M., Berglar, J., Muth, K., Staczan, P., et al. (2015). The role of therapists’ treatment adherence, professional experience, therapeutic alliance, and clients’ severity of psychological problems: Prediction of treatment outcome in eight different psychotherapy approaches. Preliminary results of a naturalistic study. Psychotherapy Research, 25, 420–434. https://doi.org/10.1080/10503307.2014.896055.

    Article  PubMed  Google Scholar 

  • von der Lippe, A., Oddli, H. W., & Halvorsen, M. S. (2017). Therapist strategies early in therapy associated with good or poor outcomes among clients with low proactive agency. Psychotherapy Research. https://doi.org/10.1080/10503307.2017.1373205.

    Article  PubMed  Google Scholar 

  • Wampold, B. E., & Imel, Z. E. (2015). The great psychotherapy debate: The evidence for what makes psychotherapy work (2nd ed.). New York: Routledge.

    Book  Google Scholar 

  • Wierzbicki, M., & Pekarik, G. (1993). A meta-analysis of psychotherapy dropout. Professional Psychology: Research and Practice, 24(2), 190–195.

    Article  Google Scholar 

Download references

Funding

This study was financed by the Norwegian Research Council and the University of Oslo. We want to thank David E. Orlinsky and John McLeod for their valuable contributions to the project.

Author information

Authors and Affiliations

  1. Department of Psychology, University of Oslo, Pb. 1094 Blindern, 0317, Oslo, Norway

    Michael Helge Rønnestad, Helene A. Nissen-Lie, Hanne Weie Oddli, Kirsten Benum, Vidar Blokhus Ekroll, Siri E. Gullestad, Hanne Haavind, Sissel Reichelt, Marit Råbu, Erik Stänicke, Anna Louise von der Lippe & Margrethe Seeger Halvorsen

Authors
  1. Michael Helge Rønnestad
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Helene A. Nissen-Lie
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hanne Weie Oddli
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Kirsten Benum
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Vidar Blokhus Ekroll
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Siri E. Gullestad
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Hanne Haavind
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Sissel Reichelt
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Marit Råbu
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Erik Stänicke
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. Anna Louise von der Lippe
    View author publications

    You can also search for this author in PubMed Google Scholar

  12. Margrethe Seeger Halvorsen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Michael Helge Rønnestad.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Rønnestad, M.H., Nissen-Lie, H.A., Oddli, H.W. et al. Expanding the Conceptualization of Outcome and Clinical Effectiveness. J Contemp Psychother 49, 87–97 (2019). https://doi.org/10.1007/s10879-018-9405-z

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10879-018-9405-z

Keywords

Search

Navigation