Skip to main content

Advertisement

SpringerLink
Log in

The Expanding Relevance of Routinely Collected Outcome Data for Mental Health Care Decision Making

  • POINT OF VIEW
  • Published:
Administration and Policy in Mental Health and Mental Health Services Research Aims and scope Submit manuscript

Abstract

Evidence shows that routine outcome monitoring (ROM) and feedback using standardized measurement tools enhances the outcomes of individual patients. When outcome data from a large number of patients and clinicians are collected, patterns can be tracked and comparisons can be made at multiple levels. Variability in skills and outcomes among clinicians and service settings has been documented, and the relevance of ROM for decision making is rapidly expanding alongside the transforming health care landscape. In this article, we highlight several developing core implications of ROM for mental health care, and frame points of future work and discussion.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Akbari, A., Mayhew, A., Al-Alawi, M. A., Grimshaw, J., Winkens, R., Glidewell, E., et al. (2008). Interventions to improve outpatient referrals from primary care to secondary care. Cochrane Database of Systematic Reviews, 4, CD005471. doi:10.1002/14651858.CD005471.pub2.

    PubMed  Google Scholar 

  • Baldwin, S. A., & Imel, Z. E. (2013). Therapist effects: Findings and methods. In M. J. Lambert (Ed.), Bergin and Garfield’s handbook of psychotherapy and behavior change (pp. 258–297). Hoboken, NJ: Wiley.

    Google Scholar 

  • Berwick, D. M., Nolan, T. W., & Whittington, J. (2008). The triple aim: Care, health, and cost. Health Affairs, 27, 759–769.

    Article  PubMed  Google Scholar 

  • Bickman, L., Riemer, M., Breda, C., & Kelley, S. D. (2006). CFIT: A system to provide a continuous quality improvement infrastructure through organizational responsiveness, measurement, training, and feedback. Report on Emotional and Behavioral Disorders in Youth, 6(86–87), 93–94.

    Google Scholar 

  • Bickman, L., Rosof-Williams, J., Salzer, M. S., Summerfelt, W. T., Noser, K., Wilson, S. J., et al. (2000). What information do clinicians value for monitoring adolescent client progress and outcomes? Professional Psychology: Research and Practice, 31, 70–74.

    Article  Google Scholar 

  • Boswell, J.F., Constantino, M.J., & Kraus. D.R. (2014). Promoting context-responsiveness with outcomes monitoring. Paper presented at the society for the exploration of psychotherapy integration conference. Montreal.

  • Boswell, J. F., Gallagher, M. W., Sauer-Zavala, S. E., Bullis, J., Gorman, J. M., Shear, M. K., et al. (2013). Patient characteristics and variability in adherence and competence in cognitive-behavioral therapy for panic disorder. Journal of Consulting and Clinical Psychology, 81, 443–454.

    Article  PubMed  Google Scholar 

  • Boswell, J. F., Kraus, D. R., Miller, S., & Lambert, M. J. (2015). Implementing routine outcome assessment in clinical practice: Benefits, challenges, and solutions. Psychotherapy Research, 25, 6–19. doi:10.1080/10503307.2013.817696.

    Article  PubMed  Google Scholar 

  • Bremer, R. W., Scholle, S. H., Keyser, D., Knox Houtsinger, J. V., & Pincus, H. A. (2008). Pay for performance in behavioral health. Psychiatric Services, 59, 1419–1429.

    Article  PubMed  Google Scholar 

  • Brownson, R. C., Colditz, G. A., & Proctor, E. K. (2012). Dissemination and implementation research in health. New York: Oxford University Press.

    Google Scholar 

  • Castonguay, L. G., Barkham, M., Lutz, W., & McAleavey, A. (2013). Practice-oriented research. In M. J. Lambert’s (Ed.), Bergin and Garfield’s handbook of psychotherapy and behavior change (Vol. 6, pp. 85–133). Hoboken, NJ: Wiley.

    Google Scholar 

  • Castonguay, L. G., Boswell, J. F., Constantino, M. J., Hill, C. E., & Goldfried, M. R. (2010). Training implications of harmful effects of psychological treatments. American Psychologist, 65, 34–49.

    Article  PubMed  Google Scholar 

  • Cerasoli, C. P., Nicklin, J. M., & Ford, M. T. (2014). Intrinsic motivation and extrinsic incentives jointly predict performance: A 40-year meta-analysis. Psychological Bulletin, 140, 980–1008.

    Article  PubMed  Google Scholar 

  • Chafe, R., Neville, D., Rathwell, T., & Deber, R. (2008). A framework for involving the public in health care coverage and resource allocation decisions. Health Manage Forum, 21, 6–13.

    Article  Google Scholar 

  • Chamberlin, J. (2014). One-star therapy? Monitor on Psychology, 45, 52–55.

    Google Scholar 

  • Constantino, M. J., Boswell, J. F., Bernecker, S. L., & Castonguay, L. G. (2013). Context-responsive integration as a framework for unified psychotherapy and clinical science: Conceptual and empirical considerations. Journal of Unified Psychotherapy and Clinical Science, 2, 1–20.

    Google Scholar 

  • Crits-Christoph, P., & Mintz, J. (1991). Implications of therapist effects for the design and analysis of comparative studies of psychotherapies. Journal of Consulting and Clinical Psychology, 59, 20–26.

    Article  CAS  PubMed  Google Scholar 

  • De Jong, K., van Sluis, P., Nugter, M. A., Heiser, W. J., & Spinhoven, P. (2012). Understanding the differential impact of outcome monitoring: Therapist variables that moderate feedback effects in a randomized clinical trial. Psychotherapy Research, 22, 464–474. doi:10.1080/10503307.2012.673023.

    Article  PubMed  Google Scholar 

  • Dudley, R. A. (2005). Pay-for-performance research: How to learn what clinicians and policy makers need to know. JAMA, 294, 1821–1823.

    Article  CAS  PubMed  Google Scholar 

  • Edwards, A., & Elwyn, G. (2009). Shared decision-making in health care: Achieving evidence-based patient choice (2nd ed.). New York, NY: Oxford University Press.

    Google Scholar 

  • Ellsworth, J. R., Lambert, M. J., & Johnson, J. (2006). A comparison of the outcome questionnaire-45 and outcome questionnaire-30 in classification and prediction of treatment outcome. Clinical Psychology and Psychotherapy, 13, 380–391. doi:10.1002/cpp.503.

    Article  Google Scholar 

  • Farley, D. O., Elliott, M. N., Short, P. F., Damiano, P., Kanouse, D. E., & Hays, R. D. (2002). Effect of CAPHS performance information on health plan choices by Iowa Medicaid. Medical Care Research and Review, 59, 319–336.

    Article  PubMed  Google Scholar 

  • Fotaki, M., Roland, M., Boyd, A., McDonald, R., Scheaff, R., & Smith, L. (2008). What benefits will choice bring to patients? Literature review and assessment of implications. Journal of Health Service Research Policy, 13, 178–184.

    Article  Google Scholar 

  • Fung, C., Yee-Wei, L., Soeren, M., Damberg, C., & Shekelle, P. (2008). Systematic review: The evidence that publishing patient care performance data improves quality of care. Annals of Internal Medicine, 148, 111–123.

    Article  PubMed  Google Scholar 

  • Garb, H. N. (2005). Clinical judgment and decision making. Annual Review of Clinical Psychology, 1, 67–89. doi:10.1146/annurev.clinpsy.1.102803.143810.

    Article  PubMed  Google Scholar 

  • Gawande, A. (2004). The bell curve. The New Yorker. Retrieved from http://www.newyorker.com/magazine/2004/12/06/the-bell-curve. Accessed 1 Aug 2014.

  • Hannan, C., Lambert, M. J., Harmon, C., Nielsen, S. L., Smart, D. W., Shimokawa, K., et al. (2005). A lab test and algorithms for identifying clients at risk for treatment failure. Journal of Clinical Psychology, 61, 155–163. doi:10.1002/jclp.20108.

    Article  PubMed  Google Scholar 

  • Hansen, N. B., Lambert, M. J., & Forman, E. V. (2002). The psychotherapy dose-response effect and its implications for treatment delivery services. Clinical Psychology: Science and Practice, 9, 329–343.

    Google Scholar 

  • Harmon, S. C., Lambert, M. J., Smart, D. W., Hawkins, E. J., Nielsen, S. L., Slade, K., et al. (2007). Enhancing outcome for potential treatment failures: Therapist/client feedback and clinical support tools. Psychotherapy Research, 17, 379–392. doi:10.1080/10503300600702331.

    Article  Google Scholar 

  • Hatfield, D. R., & Ogles, B. M. (2004). The current climate of outcome measures use in clinical practice. Professional Psychology: Research and Practice, 35, 485–491. doi:10.1037/0735-7028.35.5.485.

    Article  Google Scholar 

  • Hatfield, D. R., & Ogles, B. M. (2007). Why some clinicians use outcome measures and others do not. Administrative Policy Mental Health and Mental Health Services Research, 34, 283–291. doi:10.1007/s10488-006-0110-y.

    Article  Google Scholar 

  • Hayes, J.A., Owen, J., & Bieschke, K.J. (2014). Therapist differences in symptom change with racial and ethnic minority clients. Psychotherapy. doi:10.1037/a0037957.

  • Henderson, A., & Henderson, S. (2010). Provision of a surgeon’s performance data for people considering elective surgery. Cochrane Database of Systematic Reviews, 11, CD006327. doi:10.1002/14651858.CD006327.pub2.

    PubMed  Google Scholar 

  • Herman, B., & Evans, M. (2015). Transformers’ push to modify pay must overcome legacy system. Modern Healthcare, http://www.modernhealthcare.com/article/20150131/MAGAZINE/301319987. Accessed 27 Feb 2015.

  • Hibbard, J. H., Peters, E., Slovic, P., Finucane, M. L., & Tusler, M. (2001). Making health care quality reports easier to use. Joint Commission Journal on Quality Improvement, 27, 591–604.

    CAS  PubMed  Google Scholar 

  • Hirdes, J. P., Botz, C. A., Kozak, J., & Lepp, V. (1996). Identifying an appropriate case-mix measure for chronic care: Evidence from an Ontario Pilot Study. Healthcare Management Forum, 9, 40–46.

    Article  CAS  PubMed  Google Scholar 

  • Howard, K. I., Moras, K., Brill, P. L., Martinovich, Z., & Lutz, W. (1996). Evaluation of psychotherapy. American Psychologist, 51, 1059–1064. doi:10.1037/0003-066X.51.10.1059.

    Article  CAS  PubMed  Google Scholar 

  • Hoagwood, K. E. (2013). Don’t mourn: Organize. Reviving mental health services research for healthcare quality improvement. Clinical Psychology: Science and Practice, 20, 120–126.

    Google Scholar 

  • Imel, Z. E., Baer, J. S., Martino, S., Ball, S. A., & Carroll, K. M. (2011). Mutual influence in therapist competence and adherence to motivational enhancement therapy. Drug and Alcohol Dependence, 115, 229–236. doi:10.1016/j.drugalcdep.2010.11.010.

    Article  PubMed  PubMed Central  Google Scholar 

  • Institute of Medicine. (2001). Crossing the quality chasm: A new health system for the 21st century. Washington, D.C.: National Academy Press.

    Google Scholar 

  • Institute of Medicine Committee on Redesigning Health Insurance Performance Measures, Payment, and Performance Improvement Programs. (2007). Rewarding provider performance: Aligning incentives in Medicare: Pathways to quality health care series. Washington, D.C: National Academy Press.

    Google Scholar 

  • Ketelaar, N. A. B. M., Faber, M. J., Flottorp, S., Rygh, L. H., Deane, K. H. O., & Eccles, M. P. (2011). Public release of performance data in changing the behaviour of healthcare consumers, professionals or organisations. Cochrane Database of Systematic Reviews, 11, CD004538.

    PubMed  Google Scholar 

  • Kielser, D. J. (1966). Some myths of psychotherapy research and the search for a paradigm. Psychological Bulletin, 65, 110–136.

    Article  Google Scholar 

  • Kraus, D. R., Castonguay, L. G., Boswell, J. F., Nordberg, S. S., & Hayes, J. A. (2011). Therapist effectiveness: Implications for accountability and patient care. Psychotherapy Research, 21, 267–276.

    Article  PubMed  Google Scholar 

  • Krause, M. S., Lutz, W., & Saunders, S. M. (2007). Empirically certified treatments or therapists: The issue of separability. Psychotherapy, 44, 347–353.

    Article  PubMed  Google Scholar 

  • Kravitz, R. L., Duan, N., & Braslow, J. (2004). Evidence based medicine, heterogeneity of treatment effects, and the trouble with averages. The Milbank Quarterly, 82, 661–687. doi:10.1111/j.0887-378X.2004.00327.x.

    Article  PubMed  PubMed Central  Google Scholar 

  • Lambert, M. J. (2010). Prevention of treatment failure: The use of measuring, monitoring, and feedback in clinical practice. Washington, D.C.: American Psychological Association Press.

    Book  Google Scholar 

  • Lambert, M. J. (2013). The efficacy and effectiveness of psychotherapy. In M. J. Lambert (Ed.), Bergin & Garfield’s handbook of psychotherapy and behavior change (6th ed., pp. 169–218). New York, NY: Wiley.

    Google Scholar 

  • Lambert, M. J., & Shimokawa, K. (2011). Collecting client feedback. Psychotherapy, 48, 72–79. doi:10.1037/a0022238.

    Article  PubMed  Google Scholar 

  • Lutz, W., Leon, S. C., Martinovich, Z., Lyons, J. S., & Stiles, W. B. (2007). Therapist effects in outpatient psychotherapy: A three-level growth curve approach. Journal of Counseling Psychology, 54, 32–39. doi:10.1037/0022-0167.54.1.32.

    Article  Google Scholar 

  • McKay, K. M., Imel, Z. E., & Wampold, B. E. (2006). Psychiatrist effects in the psychopharmacological treatment of depression. Journal of Affective Disorders, 92, 287–290. doi:10.1016/j.jad.2006.01.020.

    Article  PubMed  Google Scholar 

  • Okiishi, J., Lambert, M. J., Nielsen, S. L., & Ogles, B. M. (2003). Waiting for Supershrink: An empirical analysis of therapist effects. Clinical Psychology and Psychotherapy, 10, 361–373.

    Article  Google Scholar 

  • Riemer, M., Rosof-Williams, J., & Bickman, L. (2005). Theories related to changing clinician practice. Child and Adolescent Psychiatric Clinics of North America, 14, 241–254.

    Article  PubMed  Google Scholar 

  • Rogers, E. M. (2003). Diffusion of innovations (5th ed.). New York: Free Press.

    Google Scholar 

  • Safran, J. D., Abreu, I., Ogilvie, J., & DeMaria, A. (2011). Does psychotherapy research influence the clinical practice of researcher-clinicians? Clinical Psychology: Science and Practice, 18, 357–371.

    Google Scholar 

  • Saxon, D., & Barkham, M. (2012). Patterns of therapist variability: Therapist effects and the contribution of patient severity and risk. Journal of Consulting and Clinical Psychology, 80, 535–546. doi:10.1037/a0028898.

    Article  PubMed  Google Scholar 

  • Scanlon, D. P., Lindrooth, R. C., & Christianson, J. B. (2008). Steering patients to safer hospitals? The effect of a tiered hospital network on hospital admissions. Health Services Research, 43, 1849–1868.

    Article  PubMed  PubMed Central  Google Scholar 

  • Shimokawa, K., Lambert, M. J., & Smart, D. (2010). Enhancing treatment outcome of patients at risk of treatment failure: Meta-analytic and mega-analytic review of a psychotherapy quality assurance system. Journal of Consulting and Clinical Psychology, 78, 298–311. doi:10.1037/a0019247.

    Article  PubMed  Google Scholar 

  • Stricker, G., & Trierweiler, S. J. (1995). The local clinical scientist: A bridge between science and practice. American Psychologist, 50, 995–1002.

    Article  CAS  PubMed  Google Scholar 

  • Tracey, T. J. G., Wampold, B. E., Lichtenburg, J. W., & Goodyear, R. K. (2014). Expertise in psychotherapy: An elusive goal? American Psychologist, 69, 218–229. doi:10.1037/a0035099.

    Article  PubMed  Google Scholar 

  • Tremblay, M. C., Hevner, A. R., & Berndt, D. J. (2012). Design of an information volatility measure for health care decision making. Decision Support Systems, 52, 331–341. doi:10.1016/j.dss.2011.08.009.

    Article  Google Scholar 

  • Walfish, S., McAlister, B., O’Donnell, P., & Lambert, M. J. (2012). An investigation of self-assessment bias in mental health providers. Psychological Reports, 110, 639–644. doi:10.2466/02.07.17.

    Article  PubMed  Google Scholar 

  • Wampold, B. E. (2001). The great psychotherapy debate: Models, methods, and findings. Mahwah, NJ: Erlbaum.

    Google Scholar 

  • Wampold, B. E., & Bolt, D. M. (2006). Therapist effects: Clever ways to make them (and everything else) disappear. Psychotherapy Research, 16, 184–187. doi:10.1080/10503300500265181.

    Article  Google Scholar 

  • Wampold, B. E., & Brown, G. S. (2005). Estimating variability in outcomes attributable to therapists: A naturalistic study of outcomes in managed care. Journal of Consulting and Clinical Psychology, 73, 914–923. doi:10.1037/0022-006X.73.5.914.

    Article  PubMed  Google Scholar 

  • Warren, J. S., Nelson, P. L., Mondragon, S. A., Baldwin, S. A., & Burlingame, G. M. (2010). Youth psychotherapy change trajectories & outcome in usual care: Community mental health versus managed care. Journal of Clinical and Consulting Psychology, 78, 144–155.

    Article  Google Scholar 

  • Weissman, N. W., Allison, J. J., Kiefe, C. I., Farmer, R., Weaver, M. T., Williams, O. D., et al. (1999). Achievable benchmarks of care: The ABCs of benchmarking. Journal of Evaluation in Clinical Practice, 5, 269–281. doi:10.1046/j.1365-2753.1999.00203.x.

    Article  CAS  PubMed  Google Scholar 

  • Youn, S. J., Kraus, D. R., & Castonguay, L. G. (2012). The Treatment Outcome Package: Facilitating practice and clinically relevant research. Psychotherapy, 49, 115–122. doi:10.1037/a002.

    Article  PubMed  Google Scholar 

  • Zima, B. T., Murphy, J. M., Scholle, S. H., Hoagwood, K. E., Sachdeva, R. C., Mangione-Smith, R., et al. (2013). National quality measures for child mental health care: Background, progress, and next steps. Pediatrics, 131(1), S38–S49.

    Article  PubMed  PubMed Central  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Psychology, Social Science 399, University at Albany SUNY, 1400 Washington Avenue, Albany, NY, 12222, USA

    James F. Boswell

  2. Department of Psychological and Brain Sciences, University of Massachusetts Amherst, 612 Tobin Hall - 135 Hicks Way, Amherst, MA, USA

    Michael J. Constantino

  3. Outcome Referrals, Inc., 1 Speen St., Framingham, MA, 01701, USA

    David R. Kraus

  4. Department of Psychology, University at Albany SUNY, Albany, NY, 12222, USA

    Matteo Bugatti & Jennifer M. Oswald

Authors
  1. James F. Boswell
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Michael J. Constantino
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. David R. Kraus
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Matteo Bugatti
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jennifer M. Oswald
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to James F. Boswell.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Boswell, J.F., Constantino, M.J., Kraus, D.R. et al. The Expanding Relevance of Routinely Collected Outcome Data for Mental Health Care Decision Making. Adm Policy Ment Health 43, 482–491 (2016). https://doi.org/10.1007/s10488-015-0649-6

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10488-015-0649-6

Keywords

Search

Navigation