Quality of Life Research

, Volume 28, Issue 3, pp 609–620 | Cite as

In proportion: approaches for displaying patient-reported outcome research study results as percentages responding to treatment

  • Elliott TolbertEmail author
  • Michael Brundage
  • Elissa Bantug
  • Amanda L. Blackford
  • Katherine Smith
  • Claire Snyder
  • PRO Data Presentation Stakeholder Advisory Board



Patient-reported outcome (PRO) data from clinical trials can promote valuable patient–clinician communication and aid the decision-making process regarding treatment options. Despite these benefits, both patients and doctors face challenges in interpreting PRO scores. The purpose of this study was to identify best practices for presenting PRO results expressed as proportions of patients with changes from baseline (improved/stable/worsened) for use in patient educational materials and decision aids.


We electronically surveyed adult cancer patients/survivors, oncology clinicians, and PRO researchers, and conducted one-on-one cognitive interviews with patients/survivors and clinicians. Participants saw clinical trial data comparing two treatments as proportions changed using three different formats: pie charts, bar graphs, icon arrays. Interpretation accuracy, clarity, and format preference were analyzed quantitatively and online survey comments and interviews, qualitatively.


The internet sample included 629 patients, 139 clinicians, and 249 researchers; 10 patients and 5 clinicians completed interviews. Bar graphs were less accurately interpreted than pie charts (OR 0.39; p < .0001) and icon arrays (OR 0.47; p < .0001). Bar graphs and icon arrays were less likely to be rated clear than pie charts (OR 0.37 and OR 0.18; both p < .0001). Qualitative data informed interpretation of these findings.


For communicating PROs as proportions changed in patient educational materials and decision aids, these results support the use of pie charts.


Cancer Patient-reported outcomes (PROs) Decision-making Communication Educational materials Decision aids 



This analysis was supported by a Patient-Centered Outcomes Research Institute (PCORI) Award (R-1410-24904). All statements in this report, including its findings and conclusions, are solely those of the authors and do not necessarily represent the views of the Patient-Centered Outcomes Research Institute (PCORI), its Board of Governors or Methodology Committee. Drs. Smith and Snyder are members of the Sidney Kimmel Comprehensive Cancer Center at Johns Hopkins (P30 CA 006973). The PRO Data Presentation Stakeholder Advisory Board includes Neil K. Aaronson, PhD (Netherlands Cancer Institute); Patricia (A) Ganz, MD (University of California-Los Angeles and Jonsson Comprehensive Cancer Center); Ravin Garg, MD (Anne Arundel Medical Center); Michael Fisch, MD (M.D. Anderson Cancer Center); Vanessa Hoffman, MPH (Bladder Cancer Advocacy Network); Bryce (B) Reeve, PhD (University of North Carolina at Chapel Hill and Lineberger Comprehensive Cancer Center); Eden Stotsky-Himelfarb (Sidney Kimmel Comprehensive Cancer Center at Johns Hopkins); Ellen Stovall (National Coalition for Cancer Survivorship); Matthew Zachary (Stupid Cancer). The Johns Hopkins Clinical Research Network (JHCRN) site investigators and staff include Ravin Garg, MD, and Steven P. DeMartino, CCRC, CRT, RPFT (Anne Arundel Medical Center), Melissa Gerstenhaber, MAS, MSN, RN, CCRN (JHCRN/Anne Arundel Medical Center); Gary Cohen, MD, and Cynthia MacInnis, BS, CCRP (Greater Baltimore Medical Center); James Zabora, ScD, MSW (Inova Health System), and Sandra Schaefer, BSN, RN, OCN (JHCRN/Inova Health System); Paul Zorsky, MD, Lynne Armiger, MSN, CRNP, ANP-C, Sandra L. Heineken, BS, RN, OCN, and Nancy J. Mayonado, MS (Peninsula Regional Medical Center); Michael Carducci, MD (Johns Hopkins Sibley Memorial Hospital); Carolyn Hendricks, MD, Melissa Hyman, RN, BSN, OCN, and Barbara Squiller, MSN, MPH, CRNP (Suburban Hospital). Lastly, we are most grateful to the patients and clinicians who contributed and participated in this study.


Drs. Snyder and Smith are members of the Sidney Kimmel Comprehensive Cancer Center at Johns Hopkins (P30CA006973). Financial support for this study was provided by the Patient-Centered Outcomes Research Institute (R-1410-24904). The funding agreement ensured the authors’ independence in designing the study, interpreting the data, writing, and publishing the report.

Compliance with ethical standards

Conflict of interest

The authors declare that they have no conflict of interest.


  1. 1.
    U.S. Food and Drug Administration: Guidance for Industry. (2009). Patient reported outcome measures: Use in medical product development to support labeling claims. Federal Register, 74, 65132–65133.Google Scholar
  2. 2.
    Acquadro, C., Berzon, R., Dubois, D., Leidy, N. K., Marquis, P., Revicki, D., & Rothman, M. (2003). Incorporating the patient’s perspective into drug development and communication: An ad hoc task force report of the Patient-Reported Outcomes (PRO) Harmonization Group meeting at the Food and Drug Administration, February 16, 2001. Value Health, 6(5), 522–531.CrossRefPubMedGoogle Scholar
  3. 3.
    Lipscomb, J., Gotay, C. C., & Snyder, C. (Eds.). (2005). Outcomes assessment in cancer: Measures, methods and applications. Cambridge, MA: Cambridge University Press.Google Scholar
  4. 4.
    Greenhalgh, J. (2009). The applications of PROs in clinical practice: What are they, do they work, and why? Quality of Life Research, 18(1), 115–123.CrossRefPubMedGoogle Scholar
  5. 5.
    Bruner, D. W., Bryan, C. J., Aaronson, N., Blackmore, C. C., Brundage, M., Cella, D., et al. (2007). Issues and challenges with integrating patient-reported outcomes in clinical trials supported by the National Cancer Institute-sponsored clinical trials networks. Journal of Clinical Oncology, 25(32), 5051–5057.CrossRefPubMedPubMedCentralGoogle Scholar
  6. 6.
    Till, J. E., Osoba, D., Pater, J. L., & Young, J. R. (1994). Research on health-related quality of life: Dissemination into practical applications. Quality of Life Research, 3(4), 279–283.CrossRefPubMedGoogle Scholar
  7. 7.
    Au, H. J., Ringash, J., Brundage, M., Palmer, M., Richardson, H., & Meyer, R. M. (2010). Added value of health-related quality of life measurement in cancer clinical trials: The experience of the NCIC CTG. Expert Review of Pharmacoeconomics & Outcomes Research, 10(2), 119–128. Scholar
  8. 8.
    Brundage, M. D., Feldman-Stewart, D., Bezjak, A., Leis, A., Degner, L., Fleming, S. et al. (2005). The value of quality of life information in a cancer treatment decision. In ISOQOL 11th annual conference, San FranciscoGoogle Scholar
  9. 9.
    Brundage, M., Bass, B., Jolie, R., & Foley, K. (2011). A knowledge translation challenge: Clinical use of quality of life data from cancer clinical trials. Quality of Life Research, 20(7), 979–985.CrossRefPubMedGoogle Scholar
  10. 10.
    Snyder, C. F., & Aaronson, N. K. (2009). Use of patient-reported outcomes in clinical practice. Lancet, 374(9687), 369–370.CrossRefPubMedGoogle Scholar
  11. 11.
    Bezjak, A., Ng, P., Skeel, R., Depetrillo, A. D., Comis, R., & Taylor, K. M. (2001). Oncologists’ use of quality of life information: Results of a survey of eastern cooperative oncology group physicians. Quality of Life Research, 10(1), 1–13.CrossRefPubMedGoogle Scholar
  12. 12.
    Brundage, M. D., Smith, K. C., Little, E. A., Bantug, E. T., Snyder, C. F., & PRO Data Presentation Stakeholder Advisory Board. (2015). Communicating patient-reported outcome scores using graphic formats: Results from a mixed methods evaluation. Quality of Life Research, 24(10), 2457–2472.CrossRefPubMedPubMedCentralGoogle Scholar
  13. 13.
    Smith, K. C., Brundage, M. D., Tolbert, E., Little, E. A., Bantug, E. T., Snyder, C. F., & PRO Data Presentation Stakeholder Advisory Board. (2016). Engaging stakeholders to improve presentation of patient-reported outcomes data in clinical practice. Support Care in Cancer, 24(10), 1–9. Scholar
  14. 14.
    Brundage, M., Blackford, A., Tolbert, E., Smith, K., Bantug, E., Snyder, C., & PRO Data Presentation Stakeholder Advisory Board. (2018). Presenting comparative study PRO results to clinicians and researchers: Beyond the eye of the beholder. Quality of Life Research, 27(1), 75–90.CrossRefPubMedGoogle Scholar
  15. 15.
    Tolbert, E., Brundage, M., Bantug, E., Blackford, A. L., Smith, K., Snyder, C., & PRO Data Presentation Stakeholder Advisory Board (2018). Picture this: Presenting longitudinal patient-reported outcome research study results to patients. Medical Decision Making. Scholar
  16. 16.
    Zipkin, D. A., Umscheid, C. A., Keating, N. L., et al. (2014). Evidence-based risk communication: A systematic review. Annals of Internal Medicine, 161, 270–280.CrossRefPubMedGoogle Scholar
  17. 17.
    Creswell, J. W., Plano Clark, V. L., Gutmann, M. L., & Hanson, W. E. (2003). Advanced mixed methods research designs. Handbook of mixed methods in social and behavioral research (pp. 209–240). Thousand Oaks: SageGoogle Scholar
  18. 18.
    AtlasTi, in, ATLAS.ti Scientific Software Development GmbH. 2014.Google Scholar
  19. 19.
    Bantug, E. T., Coles, T., Smith, K. C., Snyder, C. F., Rouette, J., & Brundage, M. D. (2016). Graphical displays of patient-reported outcomes (PRO) for use in clinical practice: What makes a pro picture worth a thousand words? Patient Education and Counseling, 99(4), 483–490. Scholar
  20. 20.
    Feldman-Stewart, D., Brundage, M. D., & Zotov, V. (2007). Further insight into the perception of quantitative information: Judgments of gist in treatment decisions. Medical Decision Making, 27(1), 34–43. Scholar
  21. 21.
    Le, T., Aragon, C., Thompson, H. J., & Demiris, G. (2014). Elementary graphical perception for older adults: A comparison with the general population. Perception, 43(11), 1249–1260.CrossRefPubMedGoogle Scholar
  22. 22.
    Corbin, J. C., Reyna, V. F., Weldon, R. B., & Brainerd, C. J. (2015). How reasoning, judgment, and decision making are colored by gist-based intuition: A fuzzy-trace theory approach. Journal of Applied Research in Memory and Cognition, 4(4), 344–355.CrossRefPubMedPubMedCentralGoogle Scholar
  23. 23.
    Joseph-Williams, N., Newcombe, R., Politi, M., Durand, M. A., Sivell, S., Stacey, D., et al. (2014). Toward minimum standards for certifying patient decision aids a modified Delphi consensus process. Medical Decision Making, 34(6), 699–710.CrossRefPubMedGoogle Scholar
  24. 24.
    McDonald, H., Charles, C., & Gafni, A. (2014). Assessing the conceptual clarity and evidence base of quality criteria/standards developed for evaluating decision aids. Health Expectations, 17(2), 232–243.CrossRefPubMedGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  • Elliott Tolbert
    • 1
    • 2
    Email author
  • Michael Brundage
    • 3
  • Elissa Bantug
    • 4
  • Amanda L. Blackford
    • 4
    • 5
  • Katherine Smith
    • 2
    • 4
    • 6
  • Claire Snyder
    • 1
    • 2
    • 4
    • 7
  • PRO Data Presentation Stakeholder Advisory Board
  1. 1.Johns Hopkins School of MedicineBaltimoreUSA
  2. 2.Johns Hopkins Bloomberg School of Public HealthBaltimoreUSA
  3. 3.Cancer Clinic of Southeastern OntarioQueens Cancer Research InstituteKingstonCanada
  4. 4.Sidney Kimmel Comprehensive Cancer Center at Johns HopkinsBaltimoreUSA
  5. 5.Johns Hopkins School of MedicineBaltimoreUSA
  6. 6.Johns Hopkins Bloomberg School of Public HealthBaltimoreUSA
  7. 7.Johns Hopkins Bloomberg School of Public HealthBaltimoreUSA

Personalised recommendations