Skip to main content

Advertisement

Log in

Moving from significance to real-world meaning: methods for interpreting change in clinical outcome assessment scores

  • Special Section: Measuring What Matters (by invitation only)
  • Published:
Quality of Life Research Aims and scope Submit manuscript

Abstract

Purpose

Clinical outcome assessments (COAs) require evidence not only of reliability, validity, and ability to detect change, but also a definition of what constitutes a meaningful change on the instrument. The responder definition specifies the amount of change on the COA that may be interpreted as a treatment benefit and is critical for interpreting what constitutes a meaningful change on the COA scores. However, the literature that describes methods for developing and applying responder definitions can be difficult to navigate. Clear and concise guidelines regarding which methods to apply under what circumstances and how to interpret the results are lacking. This article provides a guide to the variety of available methods and issues that should be considered when establishing responder definitions for interpreting meaningful changes in COA scores.

Methods

An overview is provided for selecting anchors, developing study designs, planning psychometric analyses, using psychometric results to set responder thresholds, and applying responder thresholds in demonstrating treatment efficacy.

Results

There are a variety of anchor-based methods for consideration, but they all rely on a preference for strongly related and easily interpretable anchors. The benefits of applying multiple anchors and multiple analytic methods are discussed. The process of triangulation can synthesize results across multiple sources to gain confidence in a proposed responder definition. Though a link to meaningfulness from the patient’s perspective is absent, distribution-based methods provide lower bound estimates of score precision and have a role in triangulation. Responder definitions are typically required within regulatory review, but their application may differ across clinical trial programs.

Conclusions

By careful planning of anchor selection, study design, and psychometric methods, COA researchers can establish defensible responder thresholds that ultimately aid patients and clinicians in making informed treatment decisions.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  1. US Food and Drug Administration. (2009). Guidance for industry on patient-reported outcome measures: Use in medical product development to support labeling claims. Federal Register, 74(235), 65132–65133.

    Google Scholar 

  2. Revicki, D., Hays, R. D., Cella, D., & Sloan, J. (2008). Recommended methods for determining responsiveness and minimally important differences for patient-reported outcomes. Journal of Clinical Epidemiology, 61(2), 102–109.

    Article  PubMed  Google Scholar 

  3. Hays, R. D., Farivar, S. S., & Liu, H. (2005). Approaches and recommendations for estimating minimally important differences for health-related quality of life measures. COPD, 2(1), 63–67.

    Article  PubMed  Google Scholar 

  4. Coon, C. D., & Cappelleri, J. C. (2016). Interpreting change in scores on patient-reported outcome instruments. Therapeutic Innovation & Regulatory Science, 50(1), 22–29.

    Article  Google Scholar 

  5. Coon, C. D. (2016). Telling the interpretation story: the case for strong anchors and multiple methods. Plenary presentation at the 23rd annual conference of the International Society of Quality for Life Research; October 2016. Copenhagen, Denmark.

  6. Norman, G. R., Stratford, P., & Regehr, G. (1997). Methodological problems in the retrospective computation of responsiveness to change: The lesson of Cronbach. Journal of Clinical Epidemiology, 50(8), 869–879.

    Article  CAS  PubMed  Google Scholar 

  7. Wyrwich, K. W., Norquist, J. M., Lenderking, W. R., Acaster, S., & The Industry Advisory Committee of International Society for Quality of Life Research (ISOQOL). (2013). Methods for interpreting change over time in patient-reported outcome measures. Quality of Life Research, 22(3), 475–483.

    Article  CAS  PubMed  Google Scholar 

  8. Nixon, A., Doll, H., Kerr, C., Burge, R., & Naegeli, A. N. (2016). Interpreting change from patient reported outcome (PRO) endpoints: patient global ratings of concept versus patient global ratings of change, a case study among osteoporosis patients. Health and Quality Life Outcomes, 14, 25.

    Article  Google Scholar 

  9. Fayers, P. M., & Hays, R. D. (2014). Don’t middle your MIDs: Regression to the mean shrinks estimates of minimally important differences. Quality of Life Research, 23(1), 1–4.

    Article  PubMed  Google Scholar 

  10. Gerlinger, C., Schumacher, U., Faustmann, T., Colligs, A., Schmitz, H., & Seitz, C. (2010). Defining a minimal clinically important difference for endometriosis-associated pelvic pain measured on a visual analog scale: analyses of two placebo-controlled, randomized trials. Health and Quality Life Outcomes, 8(1), 138.

    Article  Google Scholar 

  11. Gerlinger, C., & Schmelter, T. (2011). Determining the non-inferiority margin for patient reported outcomes. Pharmaceutical Statistics, 10(5), 410–413.

    Article  PubMed  Google Scholar 

  12. Wyrwich, K. W., Bullinger, M., Aaronson, N., Hays, R. D., Patrick, D. L., & Symonds, T. (2005). Estimating clinically significant differences in quality of life outcomes. Quality of Life Research, 14(2), 285–295.

    Article  PubMed  Google Scholar 

  13. Uryniak, T., Chan, I. S. F., Fedorov, V. V., et al. (2011). Responder analyses—a PhRMA position paper. Statistics in Biopharmaceutical Research, 3(3), 476–487.

    Article  Google Scholar 

  14. Xermelo [package insert]. (2017). The Woodlands. Texas: Lexicon Pharmaceuticals, Inc.

    Google Scholar 

  15. US Food and Drug Administration, Center for Drug Evaluation and Research. Xermelo NDA 208794 summary review, February 28, 2017. Retrieved May 8, 2017, from https://www.accessdata.fda.gov/drugsatfda_docs/nda/2017/208794Orig1s000SumR.pdf.

  16. US Food and Drug Administration, Center for Drug Evaluation and Research. Xermelo NDA 208794 statistical review and evaluation, clinical outcome assessment, November 29, 2016. Retrieved May 8, 2017, from https://www.accessdata.fda.gov/drugsatfda_docs/nda/2017/208794Orig1s000StatR.pdf.

Download references

Acknowledgements

This manuscript is based on work presented at the DIA 2016 Annual Meeting, Philadelphia, PA, June 26–30, 2016 moderated by Ms. Marian Strazzeri with Drs. Scott Komo and Cheryl Coon as panelists. The authors thank Ms. Strazzeri and Dr. Komo, as well as Drs. Laura Lee Johnson and Wen-Hung Chen, for their contribution and insight during the development of the forum session and for their feedback on this manuscript.

Funding

The authors received no funding for this work.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Cheryl D. Coon.

Ethics declarations

Conflict of interest

Dr. Coon declares that she has no conflict of interest. Dr. Cook declares that she has no conflict of interest.

Ethical approval

This article does not contain any studies with human participants performed by any of the authors.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Coon, C.D., Cook, K.F. Moving from significance to real-world meaning: methods for interpreting change in clinical outcome assessment scores. Qual Life Res 27, 33–40 (2018). https://doi.org/10.1007/s11136-017-1616-3

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11136-017-1616-3

Keywords

Navigation