Deriving Implementation Strategies for Outcome Monitoring Feedback from Theory, Research and Practice

  • Kim de Jong
Original Article


Outcome monitoring feedback is a promising intervention to enhance outcomes of clinical practice. However, effective implementation can be tough and research suggests that feedback is not equally effective under all circumstances. In this article, feedback theory, research and experience from clinical practice is used to provide implementation strategies. Factors that moderate the effectiveness of outcome monitoring feedback, including feedback, recipient and organization characteristics are discussed. It is important to pay attention to implementation processes, such as providing sufficient training for clinicians, in order for feedback to be capable of enhancing outcomes.


Outcome monitoring Feedback Implementation 


  1. Aoun, S., Pennebaker, D., & Janca, A. (2002). Outcome measurement in rural mental health care: A field trial of rooming-in models. Australian Journal of Rural Health, 10, 302–307. doi: 10.1046/j.1440-1584.2002.00484.x.CrossRefPubMedGoogle Scholar
  2. Bickman, L. (2000). Summing up program theory. New Directions for Evaluation, 2000, 103–112. doi: 10.1002/ev.1186.CrossRefGoogle Scholar
  3. Carlier, I. V. E., Meuldijk, D., Van Vliet, I. M., Van Fenema, E., Van der Wee, N. J. A., & Zitman, F. G. (2012). Routine outcome monitoring and feedback on physical or mental health status: evidence and theory. Journal of Evaluation in Clinical Practice, 18, 104–110. doi: 10.1111/j.1365-2753.2010.01543.x.CrossRefPubMedGoogle Scholar
  4. Claiborn, C. D., & Goodyear, R. K. (2005). Feedback in psychotherapy. Journal of Consulting and Clinical Psychology, 61, 209–217. doi: 10.1002/jclp.20112.CrossRefGoogle Scholar
  5. De Jong, K., Segaar, J., Van Ingenhoven, T., Busschbach, J. V., & Timman, R. (in preparation). Adverse effects of feedback in inpatients and day patients with personality disorders.Google Scholar
  6. De Jong, K., Timman, R., Hakkaart-Van Roijen, L., Vermeulen, P., Kooiman, K., & Passchier, J. (2014). The effect of outcome monitoring feedback to clinicians and patients in short and long-term psychotherapy: A randomized controlled trial. Psychotherapy Research,. doi: 10.1080/10503307.2013.871079.PubMedGoogle Scholar
  7. De Jong, K., van Sluis, P., Nugter, M. A., Heiser, W. J., & Spinhoven, P. (2012). Understanding the differential impact of outcome monitoring: Therapist variables that moderate feedback effects in a randomized clinical trial. Psychotherapy Research, 22, 464–474. doi: 10.1080/10503307.2012.673023.CrossRefPubMedGoogle Scholar
  8. Dikkentman, E. (2009). Het effect van person-organization fit, regulatory focus en regulatory fit op de attitude van behandelaars: De implementatie van een feedback instrument. Master Thesis, University of Amsterdam, Amsterdam.Google Scholar
  9. Drapeau, M. (2012). The value of progress tacking in psychotherapy. Integrating Science and Practice, 2, 5–6.Google Scholar
  10. Durlak, J., & DuPre, E. (2008). Implementation matters: A review of research on the influence of implementation on program outcomes and the factors affecting implementation. American Journal of Community Psychology, 41, 327–350. doi: 10.1007/s10464-008-9165-0.CrossRefPubMedGoogle Scholar
  11. Hannan, C., Lambert, M. J., Harmon, C., Nielsen, S. L., Smart, D. W., & Shimokawa, K. (2005). A lab test and algorithms for identifying clients at risk for treatment failure. Journal of Clinical Psychology, 61, 155–163. doi: 10.1002/jclp.20108.CrossRefPubMedGoogle Scholar
  12. Harmon, C. S., Hawkins, E. J., Lambert, M. J., Slade, K., & Whipple, J. L. (2005). Improving outcomes for poorly responding clients: The use of clinical support tools and feedback to clients. Journal of Clinical Psychology, 61, 175–185. doi: 10.1002/jclp.20109.CrossRefPubMedGoogle Scholar
  13. Harmon, C. S., Lambert, M. J., Smart, D. M., Hawkins, E., Nielsen, S. L., & Slade, K. (2007). Enhancing outcome for potential treatment failures: Therapist-client feedback and clinical support tools. Psychotherapy research, 17, 379–392. doi: 10.1080/10503300600702331.CrossRefGoogle Scholar
  14. Hawkins, E. J., Lambert, M. J., Vermeersch, D. A., Slade, K. L., & Tuttle, K. C. (2004). The therapeutic effects of providing patient progress information to therapists and patients. Psychotherapy Research, 14(3), 308–327. doi: 10.1093/ptr/kph027.CrossRefGoogle Scholar
  15. Herold, D. M., & Fedor, D. B. (2003). Individual differences in feedback propensities and training performance. Human Resource Management Review, 13, 675–689. doi: 10.1016/j.hrmr.2003.11.008.CrossRefGoogle Scholar
  16. Herold, D. M., Parsons, C. K., & Rensvold, R. B. (1996). Individual differences in the generation and processing of performance feedback. Educational and Psychological Measurement, 56, 5–25. doi: 10.1177/0013164496056001001.CrossRefGoogle Scholar
  17. Kluger, A. N., & De Nisi, A. (1996). The effects of feedback interventions on performance: A historical review, a meta-analysis, and a preliminary feedback intervention theory. Psychological Bulletin, 119, 254–284. doi: 10.1037/0033-2909.119.2.254.CrossRefGoogle Scholar
  18. Knaup, C., Koesters, M., Schoefer, D., Becker, T., & Puschner, B. (2009). Effect of feedback of treatment outcome in specialist mental healthcare: meta-analysis. The British Journal of Psychiatry, 195, 15–22. doi: 10.1192/bjp.bp.108.053967.CrossRefPubMedGoogle Scholar
  19. Lambert, M. J. (2007). Presidential address: What we have learned from a decade of research aimed at improving psychotherapy outcome in routine care. Psychotherapy Research, 17, 1–14. doi: 10.1080/10503300601032506.CrossRefGoogle Scholar
  20. Lambert, M. J., Whipple, J. L., Hawkins, E. J., Vermeersch, D. A., Nielsen, S. L., & Smart, D. W. (2003). Is it time for clinicians to routinely track patient outcome? A meta-analysis. Clinical Psychology: Science and Practice, 10(3), 288–301. doi: 10.1093/clipsy/bpg025.Google Scholar
  21. Lutz, W., Boehnke, J. R., Köck, K., & Bitterman, A. (2011). Diagnostik und psychometrische Verlaufsrückmeldungen im Rahmen eines Modellprojektes zur Qualitätssicherung in der ambulanten Psychotherapie. Zeitschrift fur Klinische Psychologie und Psychotherapie, 40, 283–297. doi: 10.1026/1616-3443/a000125.CrossRefGoogle Scholar
  22. Lutz, W., Lambert, M. J., Harmon, S., Tschitsaz, A., Schurch, E., & Stulz, N. (2006). The probability of treatment success, failure and duration- what can be learned from empirical data to support decision making in clinical practice? Clinical Psychology & Psychotherapy, 13, 223–232. doi: 10.1002/cpp.496.CrossRefGoogle Scholar
  23. Maslach, C., Schaufeli, W. B., & Leiter, M. P. (2001). Job burnout. Annual Review of Psychology, 52, 397–422. doi: 10.1146/annurev.psych.52.1.397.CrossRefPubMedGoogle Scholar
  24. Mezulis, A. H., Abramson, L. Y., Hyde, J. S., & Hankin, B. L. (2004). Is there a universal positivity bias in attributions? A meta-analytic review of individual, developmental, and cultural differences in the self-serving attributional bias. Psychological Bulletin, 130, 711–747. doi: 10.1037/0033-2909.130.5.711.CrossRefPubMedGoogle Scholar
  25. Riemer, M., & Bickman, L. (2011). Using program theory to link social psychology and program evaluation. In M. M. Mark, S. I. Donaldson, & B. Campbell (Eds.), Social psychology and evaluation. New York: Guilford.Google Scholar
  26. Riemer, M., Rosof-Williams, J., & Bickman, L. (2005). Theories related to changing clinician practice. Child and Adolescent Psychiatric Clinics of North America, 14, 241–254. doi: 10.1016/j.chc.2004.05.002.CrossRefPubMedGoogle Scholar
  27. Sapyta, J., Riemer, M., & Bickman, L. (2005). Feedback to clinicians: Theory, research and practice. Journal of Clinical Psychology. In Session, 61, 145–153. doi: 10.1002/jclp.20107.CrossRefPubMedGoogle Scholar
  28. Shimokawa, K., Lambert, M. J., & Smart, D. W. (2010). Enhancing treatment outcome of patients at risk of treatment failure: Meta-analytic and mega-analytic review of a psychotherapy quality assurance system. Journal of Consulting and Clinical Psychology, 78, 298–311. doi: 10.1037/a0019247.CrossRefPubMedGoogle Scholar
  29. Simon, W., Lambert, M. J., Harris, M. W., Busath, G., & Vazquez, A. (2012). Providing patient progress information and clinical support tools to therapists: Effects on patients at risk of treatment failure. Psychotherapy Research, 22, 638–647. doi: 10.1080/10503307.2012.698918.CrossRefPubMedGoogle Scholar
  30. Slade, K., Lambert, M. J., Harmon, S., Smart, D. W., & Bailey, R. (2008). Improving psychotherapy outcome: The use of immediate electronic feedback and revised clinical support tools. Clinical Psychology & Psychotherapy, 15, 287–303. doi: 10.1002/cpp.594.CrossRefGoogle Scholar
  31. Sng, A. A. H., Newnham, E. A., & Page, A. C. (in preparation). Does presentation style matter: Exploring therapists’ perceptions of the utility of feedback graphs depicting progress in psychotherapy.Google Scholar
  32. Walter, G., Cleary, M., & Rey, J. M. (1998). Attitudes of mental health personnel towards rating outcome. Journal of Quality in Clinical Practice, 18, 109–115.PubMedGoogle Scholar
  33. Weiner, B. (1985). An attributional theory of achievement motivation and emotion. Psychological Review, 92, 548–573. doi: 10.1037/0033-295X.92.4.548.CrossRefPubMedGoogle Scholar
  34. Willis, A., Deane, F. P., & Coombs, T. (2009). Improving clinicians’ attitudes toward providing feedback on routine outcome assessments. International Journal of Mental Health Nursing, 18, 211–215. doi: 10.1111/j.1447-0349.2009.00596.x.CrossRefPubMedGoogle Scholar
  35. Youn, S. J., Kraus, D. R., & Castonguay, L. G. (2012). The treatment outcome package: Facilitating practice and clinically relevant research. Psychotherapy, 49, 115–122. doi: 10.1037/a0027932.CrossRefPubMedGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2014

Authors and Affiliations

  1. 1.Department of Clinical Psychology, Institute of PsychologyLeiden UniversityLeidenThe Netherlands
  2. 2.Department of PsychologyUniversity of PennsylvaniaPhiladelphiaUSA

Personalised recommendations