Leading Horses to Water: Lessons from a Decade of Helping Psychological Therapy Services Use Routine Outcome Measurement to Improve Practice

  • John Mellor-ClarkEmail author
  • Simone Cross
  • James Macdonald
  • Tommy Skjulsvik
Point of View


We summarise the recent reflections of five thought leaders in the field of routine outcome measurement (ROM) for psychological therapy, and then add our own experience of introducing a national ROM system in the UK. We highlight, in particular, the post-implementation challenge of securing data of sufficient reliability to help inform service quality improvements. We ground our conclusions and recommendations in the rapidly evolving discipline of implementation science, and offer a best practice model for applying research recommendations in practice settings. In this context we portray ROM implementation as significant organizational change that benefits from rigorous process and clearly defined, well-communicated targets.


Psychological therapies Routine outcome measurement (ROM) Feedback systems Implementation 


  1. Aarons, G. A., Hurlburt, M., & McCue Horwitz, S. (2011). Advancing a conceptual model of evidence-based practice implementation in public service sectors. Administration and Policy In Mental Health, 38, 4–23.CrossRefPubMedPubMedCentralGoogle Scholar
  2. Anker, M., Duncan, B., & Sparks, J. (2009). Using client feedback to improve couples therapy outcomes: A randomised clinical trial in a naturalistic setting. Journal of Consulting and Clinical Psychology, 77, 693–704.CrossRefPubMedGoogle Scholar
  3. Barkham, M., & Mellor-Clark, J. (2003). Bridging evidence-based practice and practice-based evidence: Developing a rigorous and relevant knowledge for the psychological therapies. Journal of Clinical Psychology and Psychotherapy, 10, 319–327.CrossRefGoogle Scholar
  4. Boswell, J. F., Kraus, D. R., Miller, S., & Lambert, M. J. (in press). Implementing routine outcome assessment in clinical practice: Benefits, challenges, and solutions. Psychotherapy Research.
  5. Curtis-Tyler, K., Wolpert, M., & Edbrooke-Childs, J. (submitted). A qualitative exploration of clinicians and service user views on Patient Reported Outcome Measures in child mental health and diabetes services in the United Kingdom. Administration & Policy in Mental Health and Mental Health Services Research.Google Scholar
  6. Damschroder, L., Aron, D., Keith, R., Kirsh, S., Alexander, J., & Lowery, J. (2009). Fostering implementation of health service research findings into practice: A consolidated framework for advancing implementation science. Implementation Science, 4, 50.CrossRefGoogle Scholar
  7. de Jong, K. (submitted). Deriving effective implementation strategies for outcome monitoring from feedback theory. Administration and Policy in Mental Health. Google Scholar
  8. de Jong, K., van Sluis, P., Nuger, M. A., Heiser, W. J., & Spinhoven, P. (2012). Understanding the differential impact of outcome monitoring: Therapist variables that moderate feedback effects in a randomized controlled trial. Psychotherapy Research, 22, 464–474.CrossRefPubMedGoogle Scholar
  9. Douglas, S., Button, S., & Casey, S. E. (submitted). Implementing for sustainability: Promoting use of a Measurement Feedback System for innovation and quality improvement. Administration and Policy in Mental Health. Google Scholar
  10. Duncan, B. (2010). On Becoming a Better Therapist. Washington, DC: APA.CrossRefGoogle Scholar
  11. Edbrooke-Childs, J., Wolpert, M., Deighton, J. (submitted). Using Patient Reported Outcome Measures to Improve Service Effectiveness (UPROMISE): A service evaluation of training for clinicians to use outcome measures in child mental health. Administration & Policy in Mental Health and Mental Health Services Research.Google Scholar
  12. Evans, C. (2012). Cautionary notes on power steering for psychotherapy. Canadian Psychology, 53(2), 131–139.CrossRefGoogle Scholar
  13. Feldstein, A., & Glasgow, R. (2008). A practical robust implementation and sustainability model (PRISM). Joint Commission Journal on Quality and Patient Safety, 34(4), 228–243.PubMedGoogle Scholar
  14. Fleming, I., Jones, M., Bradley, J., & Wolpert, M. (submitted). Learning from a learning collaboration: the CORC approach to combining research, evaluation and practice in child mental health. Administration & Policy in Mental Health and Mental Health Services Research.Google Scholar
  15. Green, D., & Latchford, G. (2012). Maximising the Benefits of Psychotherapy: A Practice-Based Evidence Approach. Chichester: John Wiley & Sons.CrossRefGoogle Scholar
  16. Griffiths, S., Foster, J., Steen, S., & Pietroni, P. (2013). Mental health’s market experiment: Commissioning Psychological Therapies Through Any Qualified Provider. Chester: Centre for Psychological Therapies in Primary Care.Google Scholar
  17. Kraus, D. R., Seligman, D., & Jordan, J. R. (2005). Validation of a behavioral health treatment outcome and assessment tool designed for naturalistic settings. The Treatment Outcome Package. Journal of Clinical Psychology, 61, 285–314. doi: 10.1002/jclp.20084.CrossRefPubMedGoogle Scholar
  18. Kraus, D. R., Castonguay, L. G., Boswell, J. F., Norberg, S. S., & Hayes, J. A. (2011). Therapist effectiveness: Implications for accountability and patient care. Psychotherapy Research, 21, 267–276. doi: 10.1080/10503307.2011.563249.CrossRefPubMedGoogle Scholar
  19. Lambert, M. (2010). Prevention of Treatment Failure: The Use of Measuring, Monitoring, and Feedback in Clinical Practice. Washington, DC: APA.CrossRefGoogle Scholar
  20. Lambert, M. (2012). Helping clinicians to use and learn from research-based systems: The OQ Analyst. Psychotherapy, 49, 109–114. doi: 10.1037/a0027110.CrossRefPubMedGoogle Scholar
  21. Lambert, M. (2013). The efficacy and effectiveness of psychotherapy. In M. J. Lambert (Ed.), Bergin and Garfield’s Handbook of Psychotherapy and Behaviour Change (6th ed., pp. 169–218). Hoboken, NJ: John Wiley & Sons.Google Scholar
  22. Lambert, M., & Shimokawa, K. (2011). Collecting Client Feedback. In J. Norcross (Ed.), Psychotherapy Relationships that Work: Evidence-based responsiveness (2nd ed., pp. 203–223). New York: Oxford University Press.CrossRefGoogle Scholar
  23. Mellor-Clark, J., & Barkham, M. (2006). The CORE System: Developing and delivering practice-based evidence through quality evaluation. In C. Feltham & I. Horton (Eds.), Handbook of Counselling and Psychotherapy (2nd ed.). London: Sage Publications.Google Scholar
  24. Mellor-Clark, J., Curtis Jenkins, A., Evans, R., Mothersole, G., & McIness, B. (2006). Resourcing a CORE Network to develop a National Research Database to help enhance psychological therapy and counselling service provision. Counselling & Psychotherapy Research, 6, 16–22.CrossRefGoogle Scholar
  25. Meyers, D. C., Durlack, J. A., & Wandersman, A. (2012). The quality implementation framework: A synthesis of critical steps in the implementation process. American Journal of Community Psychology, 50(3–4), 462–480.CrossRefPubMedGoogle Scholar
  26. Miller, S. D., Duncan, B. L., Sorrell, R., & Brown, G. S. (2005). The Partners for Change Outcome Management System. Journal of Clinical Psychology: In-session, 61, 199–208. doi: 10.1002/jclp.20111.CrossRefPubMedGoogle Scholar
  27. Shimokawa, K., Lambert, M., & Smart, D. (2010). Enhancing treatment outcome of patients at risk of treatment failure: Meta-analytic and mega-analytic review of a psychotherapy quality assurance system. Journal of Consulting and Clinical Psychology, 78, 298–311.CrossRefPubMedGoogle Scholar
  28. Stiles, W. B., Barkham, M., Twigg, E., Mellor-Clark, J., & Cooper, M. (2006). Effectiveness of cognitive-behavioural, person-centred, and psychodynamic therapies as practiced in National Health Service settings. Psychological Medicine, 36, 555–566.CrossRefPubMedGoogle Scholar
  29. Stiles, W. B., Barkham, M., Mellor-Clark, J., & Connell, J. (2008). Effectiveness of cognitive-behavioural, person-centred, and psychodynamic therapies in UK primary care routine practice: Replication in a larger sample. Psychological Medicine, 38, 677–688.PubMedGoogle Scholar
  30. van Oenen, F. J., Schipper, S., Van, R., Schoevers, R., Visch, I., Peen, J., et al. (2013). Efficacy of immediate patient feedback in emergency psychiatry: A randomized controlled trail in a crisis intervention and brief therapy team. BMC Psychiatry, 13, 331–342.CrossRefPubMedPubMedCentralGoogle Scholar
  31. Wolpert, M. (2014). Uses and abuses of Patient Reported Outcome Measures (PROMs): Potential iatrogenic impact of PROMs implementation and how it can be mitigated. Administration and Policy in Mental Health and Mental Health Services Research, 41, 141–145.Google Scholar
  32. Wolpert, M., Fugard, A. J. B., Deighton, J., & Gorzig, A. (2012). Routine outcomes monitoring as part of children and young person’s Improving Access to Psychological Therapies (CYP IAPT)—improving care or unhelpful burden? Childs and Adolescent Mental Health, 17(3), 129–130.CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2014

Authors and Affiliations

  • John Mellor-Clark
    • 1
    Email author
  • Simone Cross
    • 1
  • James Macdonald
    • 1
  • Tommy Skjulsvik
    • 2
  1. 1.CORE Information Systems Ltd. (IMS)RugbyUK
  2. 2.Linköping UniversityLinköpingSweden

Personalised recommendations