Patient-reported outcome (PRO) data from clinical trials can promote valuable patient–clinician communication and aid the decision-making process regarding treatment options. Despite these benefits, both patients and doctors face challenges in interpreting PRO scores. The purpose of this study was to identify best practices for presenting PRO results expressed as proportions of patients with changes from baseline (improved/stable/worsened) for use in patient educational materials and decision aids.
We electronically surveyed adult cancer patients/survivors, oncology clinicians, and PRO researchers, and conducted one-on-one cognitive interviews with patients/survivors and clinicians. Participants saw clinical trial data comparing two treatments as proportions changed using three different formats: pie charts, bar graphs, icon arrays. Interpretation accuracy, clarity, and format preference were analyzed quantitatively and online survey comments and interviews, qualitatively.
The internet sample included 629 patients, 139 clinicians, and 249 researchers; 10 patients and 5 clinicians completed interviews. Bar graphs were less accurately interpreted than pie charts (OR 0.39; p < .0001) and icon arrays (OR 0.47; p < .0001). Bar graphs and icon arrays were less likely to be rated clear than pie charts (OR 0.37 and OR 0.18; both p < .0001). Qualitative data informed interpretation of these findings.
For communicating PROs as proportions changed in patient educational materials and decision aids, these results support the use of pie charts.
This is a preview of subscription content, log in to check access.
Buy single article
Instant access to the full article PDF.
Price includes VAT for USA
Subscribe to journal
Immediate online access to all issues from 2019. Subscription will auto renew annually.
This is the net price. Taxes to be calculated in checkout.
U.S. Food and Drug Administration: Guidance for Industry. (2009). Patient reported outcome measures: Use in medical product development to support labeling claims. Federal Register, 74, 65132–65133.
Acquadro, C., Berzon, R., Dubois, D., Leidy, N. K., Marquis, P., Revicki, D., & Rothman, M. (2003). Incorporating the patient’s perspective into drug development and communication: An ad hoc task force report of the Patient-Reported Outcomes (PRO) Harmonization Group meeting at the Food and Drug Administration, February 16, 2001. Value Health, 6(5), 522–531.
Lipscomb, J., Gotay, C. C., & Snyder, C. (Eds.). (2005). Outcomes assessment in cancer: Measures, methods and applications. Cambridge, MA: Cambridge University Press.
Greenhalgh, J. (2009). The applications of PROs in clinical practice: What are they, do they work, and why? Quality of Life Research, 18(1), 115–123.
Bruner, D. W., Bryan, C. J., Aaronson, N., Blackmore, C. C., Brundage, M., Cella, D., et al. (2007). Issues and challenges with integrating patient-reported outcomes in clinical trials supported by the National Cancer Institute-sponsored clinical trials networks. Journal of Clinical Oncology, 25(32), 5051–5057.
Till, J. E., Osoba, D., Pater, J. L., & Young, J. R. (1994). Research on health-related quality of life: Dissemination into practical applications. Quality of Life Research, 3(4), 279–283.
Au, H. J., Ringash, J., Brundage, M., Palmer, M., Richardson, H., & Meyer, R. M. (2010). Added value of health-related quality of life measurement in cancer clinical trials: The experience of the NCIC CTG. Expert Review of Pharmacoeconomics & Outcomes Research, 10(2), 119–128. https://doi.org/10.1586/erp.10.15.
Brundage, M. D., Feldman-Stewart, D., Bezjak, A., Leis, A., Degner, L., Fleming, S. et al. (2005). The value of quality of life information in a cancer treatment decision. In ISOQOL 11th annual conference, San Francisco
Brundage, M., Bass, B., Jolie, R., & Foley, K. (2011). A knowledge translation challenge: Clinical use of quality of life data from cancer clinical trials. Quality of Life Research, 20(7), 979–985.
Snyder, C. F., & Aaronson, N. K. (2009). Use of patient-reported outcomes in clinical practice. Lancet, 374(9687), 369–370.
Bezjak, A., Ng, P., Skeel, R., Depetrillo, A. D., Comis, R., & Taylor, K. M. (2001). Oncologists’ use of quality of life information: Results of a survey of eastern cooperative oncology group physicians. Quality of Life Research, 10(1), 1–13.
Brundage, M. D., Smith, K. C., Little, E. A., Bantug, E. T., Snyder, C. F., & PRO Data Presentation Stakeholder Advisory Board. (2015). Communicating patient-reported outcome scores using graphic formats: Results from a mixed methods evaluation. Quality of Life Research, 24(10), 2457–2472.
Smith, K. C., Brundage, M. D., Tolbert, E., Little, E. A., Bantug, E. T., Snyder, C. F., & PRO Data Presentation Stakeholder Advisory Board. (2016). Engaging stakeholders to improve presentation of patient-reported outcomes data in clinical practice. Support Care in Cancer, 24(10), 1–9. https://doi.org/10.1007/s00520-016-3240-0.
Brundage, M., Blackford, A., Tolbert, E., Smith, K., Bantug, E., Snyder, C., & PRO Data Presentation Stakeholder Advisory Board. (2018). Presenting comparative study PRO results to clinicians and researchers: Beyond the eye of the beholder. Quality of Life Research, 27(1), 75–90.
Tolbert, E., Brundage, M., Bantug, E., Blackford, A. L., Smith, K., Snyder, C., & PRO Data Presentation Stakeholder Advisory Board (2018). Picture this: Presenting longitudinal patient-reported outcome research study results to patients. Medical Decision Making. https://doi.org/10.1177/0272989X18791177.
Zipkin, D. A., Umscheid, C. A., Keating, N. L., et al. (2014). Evidence-based risk communication: A systematic review. Annals of Internal Medicine, 161, 270–280.
Creswell, J. W., Plano Clark, V. L., Gutmann, M. L., & Hanson, W. E. (2003). Advanced mixed methods research designs. Handbook of mixed methods in social and behavioral research (pp. 209–240). Thousand Oaks: Sage
AtlasTi, in, ATLAS.ti Scientific Software Development GmbH. 2014.
Bantug, E. T., Coles, T., Smith, K. C., Snyder, C. F., Rouette, J., & Brundage, M. D. (2016). Graphical displays of patient-reported outcomes (PRO) for use in clinical practice: What makes a pro picture worth a thousand words? Patient Education and Counseling, 99(4), 483–490. https://doi.org/10.1016/j.pec.2015.10.027.
Feldman-Stewart, D., Brundage, M. D., & Zotov, V. (2007). Further insight into the perception of quantitative information: Judgments of gist in treatment decisions. Medical Decision Making, 27(1), 34–43. https://doi.org/10.1177/0272989X06297101.
Le, T., Aragon, C., Thompson, H. J., & Demiris, G. (2014). Elementary graphical perception for older adults: A comparison with the general population. Perception, 43(11), 1249–1260.
Corbin, J. C., Reyna, V. F., Weldon, R. B., & Brainerd, C. J. (2015). How reasoning, judgment, and decision making are colored by gist-based intuition: A fuzzy-trace theory approach. Journal of Applied Research in Memory and Cognition, 4(4), 344–355.
Joseph-Williams, N., Newcombe, R., Politi, M., Durand, M. A., Sivell, S., Stacey, D., et al. (2014). Toward minimum standards for certifying patient decision aids a modified Delphi consensus process. Medical Decision Making, 34(6), 699–710.
McDonald, H., Charles, C., & Gafni, A. (2014). Assessing the conceptual clarity and evidence base of quality criteria/standards developed for evaluating decision aids. Health Expectations, 17(2), 232–243.
This analysis was supported by a Patient-Centered Outcomes Research Institute (PCORI) Award (R-1410-24904). All statements in this report, including its findings and conclusions, are solely those of the authors and do not necessarily represent the views of the Patient-Centered Outcomes Research Institute (PCORI), its Board of Governors or Methodology Committee. Drs. Smith and Snyder are members of the Sidney Kimmel Comprehensive Cancer Center at Johns Hopkins (P30 CA 006973). The PRO Data Presentation Stakeholder Advisory Board includes Neil K. Aaronson, PhD (Netherlands Cancer Institute); Patricia (A) Ganz, MD (University of California-Los Angeles and Jonsson Comprehensive Cancer Center); Ravin Garg, MD (Anne Arundel Medical Center); Michael Fisch, MD (M.D. Anderson Cancer Center); Vanessa Hoffman, MPH (Bladder Cancer Advocacy Network); Bryce (B) Reeve, PhD (University of North Carolina at Chapel Hill and Lineberger Comprehensive Cancer Center); Eden Stotsky-Himelfarb (Sidney Kimmel Comprehensive Cancer Center at Johns Hopkins); Ellen Stovall (National Coalition for Cancer Survivorship); Matthew Zachary (Stupid Cancer). The Johns Hopkins Clinical Research Network (JHCRN) site investigators and staff include Ravin Garg, MD, and Steven P. DeMartino, CCRC, CRT, RPFT (Anne Arundel Medical Center), Melissa Gerstenhaber, MAS, MSN, RN, CCRN (JHCRN/Anne Arundel Medical Center); Gary Cohen, MD, and Cynthia MacInnis, BS, CCRP (Greater Baltimore Medical Center); James Zabora, ScD, MSW (Inova Health System), and Sandra Schaefer, BSN, RN, OCN (JHCRN/Inova Health System); Paul Zorsky, MD, Lynne Armiger, MSN, CRNP, ANP-C, Sandra L. Heineken, BS, RN, OCN, and Nancy J. Mayonado, MS (Peninsula Regional Medical Center); Michael Carducci, MD (Johns Hopkins Sibley Memorial Hospital); Carolyn Hendricks, MD, Melissa Hyman, RN, BSN, OCN, and Barbara Squiller, MSN, MPH, CRNP (Suburban Hospital). Lastly, we are most grateful to the patients and clinicians who contributed and participated in this study.
Drs. Snyder and Smith are members of the Sidney Kimmel Comprehensive Cancer Center at Johns Hopkins (P30CA006973). Financial support for this study was provided by the Patient-Centered Outcomes Research Institute (R-1410-24904). The funding agreement ensured the authors’ independence in designing the study, interpreting the data, writing, and publishing the report.
Conflict of interest
The authors declare that they have no conflict of interest.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
The Members of PRO Data Presentation Stakeholder Advisory Board are listed in Acknowledgements.
About this article
Cite this article
Tolbert, E., Brundage, M., Bantug, E. et al. In proportion: approaches for displaying patient-reported outcome research study results as percentages responding to treatment. Qual Life Res 28, 609–620 (2019). https://doi.org/10.1007/s11136-018-2065-3
- Patient-reported outcomes (PROs)
- Educational materials
- Decision aids