Skip to main content

Advertisement

SpringerLink
Log in

Making a picture worth a thousand numbers: recommendations for graphically displaying patient-reported outcomes data

  • Published:
Quality of Life Research Aims and scope Submit manuscript

Abstract

Purpose

Patient-reported outcomes (PROs) can promote patient-centered care in multiple ways: (1) using an individual patient’s PRO data to inform his/her management, (2) providing PRO results from comparative research studies in patient educational materials/decision aids, and (3) reporting PRO results from comparative research studies in peer-reviewed publications. Patients and clinicians endorse the value of PRO data; however, variations in how PRO measures are scored and scaled, and in how the data are reported, make interpretation challenging and limit their use in clinical practice. We conducted a modified Delphi process to develop stakeholder-engaged, evidence-based recommendations for PRO data display for the three above applications to promote understanding and use.

Methods

The Consensus Panel included cancer survivors/caregivers, oncologists, PRO researchers, and application-specific end-users (e.g., electronic health record vendors, decision aid developers, journal editors). We reviewed the data display issues and their evidence base during pre-meeting webinars. We then surveyed participants’ initial perspectives, which informed discussions during an in-person meeting to develop consensus statements. These statements were ratified via a post-meeting survey.

Results

Issues addressed by consensus statements relevant to both individual and research data applications were directionality (whether higher scores are better/worse) and conveying score meaning (e.g., none/mild/moderate/severe). Issues specific to individual patient data presentation included representation (bar charts vs. line graphs) and highlighting possibly concerning scores (absolute and change). Issues specific to research study results presentation included handling normed data, conveying statistically significant differences, illustrating clinically important differences, and displaying proportions improved/stable/worsened.

Conclusions

The recommendations aim to optimize accurate and meaningful interpretation of PRO data.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

References

  1. U. S. Food and Drug Administration. (2009). Guidance for industry. Patient reported outcome measures: Use in medical product development to support labeling claims. Federal Register, 74(35), 65132–65133.

    Google Scholar 

  2. Acquadro, C., Berzon, R., Dubois, D., Leidy, N. K., Marquis, P., Revicki, D., et al. (2003). Incorporating the patient’s perspective into drug development and communication: An ad hoc task force report of the patient-reported outcomes (PRO) harmonization group meeting at the food and drug administration, February 16, 2001. Value Health, 6, 522–531.

    Article  PubMed  Google Scholar 

  3. Greenhalgh, J. (2009). The applications of PROs in clinical practice: What are they, do they work, and why? Quality Life Research, 18, 115–123.

    Article  Google Scholar 

  4. Snyder, C. F., & Aaronson, N. K. (2009). Use of patient-reported outcomes in clinical practice. Lancet, 374, 369–370.

    Article  PubMed  Google Scholar 

  5. Jensen, R. E., Snyder, C. F., Abernethy, A. P., Basch, E., Reeve, B. B., Roberts, A., et al. (2014). A review of electronic patient reported outcomes systems used in cancer clinical care. Journal of Oncology Practice, 10, e215–e222.

    Article  PubMed  Google Scholar 

  6. Till, J. E., Osoba, D., Pater, J. L., & Young, J. R. (1994). Research on health-related quality of life: Dissemination into practical applications. Quality Life Research, 3(4), 279–283.

    Article  CAS  Google Scholar 

  7. Au, H.-J., Ringash, J., Brundage, M., Palmer, M., Richardson, H., Meyer, R. M., et al. (2010). Added value of health-related quality of life measurement in cancer clinical trials: The experience of the NCIC CTG. Expert Review of Pharmacoeconomics & Outcomes Research, 10(2), 119–128.

    Article  Google Scholar 

  8. Lipscomb, J., Gotay, C. C., & Snyder, C. (Eds.). (2005). Outcomes assessment in cancer: Measures, methods, and applications. Cambridge: Cambridge Univ Press.

    Google Scholar 

  9. Brundage, M., Bass, B., Ringash, J., & Foley, K. (2011). A knowledge translation challenge: Clinical use of quality of life data from cancer clinical trials. Quality Life Research, 20, 979–985.

    Article  Google Scholar 

  10. Bezjak, A., Ng, P., Skeel, R., Depetrillo, A. D., Comis, R., & Taylor, K. M. (2001). Oncologists’ use of quality of life information: Results of a survey of Eastern Cooperative Oncology Group physicians. Quality Life Research, 10, 1–13.

    Article  CAS  Google Scholar 

  11. Snyder, C. F., Blackford, A. L., Wolff, A. C., Carducci, M. A., Herman, J. M., Wu, A. W., et al. (2013). Feasibility and value of PatientViewpoint: A web system for patient-reported outcomes assessment in clinical practice. Psychooncology, 22, 895–901.

    Article  PubMed  Google Scholar 

  12. Brundage, M. D., Smith, K. C., Little, E. A., Bantug, E. T., & Snyder, C. F., PRO Data Presentation Stakeholder Advisory Board (2015). Communicating patient-reported outcome scores using graphic formats: Results from a mixed methods evaluation. Quality Life Research, 24, 2457–2472.

    Article  Google Scholar 

  13. Bantug, E. T., Coles, T., Smith, K. C., Snyder, C. F., Rouette, J., Brundage, M. D., et al. (2016). Graphical displays of patient-reported outcomes (PRO) for use in clinical practice: What makes a PRO picture worth a thousand words? Patient Education and Counselling, 99, 483–490.

    Article  Google Scholar 

  14. Smith, K. C., Brundage, M. D., Tolbert, E., Little, E. A., Bantug, E. T., Snyder, C., et al. (2016). Engaging stakeholders to improve presentation of patient-reported outcomes data in clinical practice. Support Care Cancer, 24, 4149–4157.

    Article  PubMed  Google Scholar 

  15. Snyder, C. F., Smith, K. C., Bantug, E. T., Tolbert, E. E., Blackford, A. L., Brundage, M. D., et al. (2017). What do these scores mean? Presenting patient-reported outcomes data to patients and clinicians to improve interpretability. Cancer, 123, 1848–1859.

    Article  PubMed  PubMed Central  Google Scholar 

  16. Tolbert, E., Snyder, C., Bantug, E., Blackford, A., Smith, K., Brundage, M., et al. (2016). Graphing group-level data from research studies for presentation to patients in educational materials and decision aids. Quality Life Research, 15, 17.

    Google Scholar 

  17. Brundage, M. D., Blackford, A., Tolbert, E., Smith, K., Bantug, E., Snyder, C., et al. (2018). Presenting comparative study PRO results to clinicians and researchers: Beyond the eye of the beholder. Quality Life Research, 27, 75–90.

    Article  Google Scholar 

  18. Qualtrics Survey Software. Retrieved August 7, 2018, from https://ictr.johnshopkins.edu/programs_resources/programs-resources/informatics/qualtrics-survey/.

Download references

Acknowledgements

The study was funded by the Patient-Centered Outcomes Research Institute (PCORI). Drs. Snyder and Smith are members of the Sidney Kimmel Comprehensive Cancer Center at Johns Hopkins (P30CA006973).

PRO Data Presentation Delphi Panel members

The Consensus Panel Participants included the project Stakeholder Advisory Board (SAB) and other invitees, in addition to the project team named authors. The SAB members were Daniel Weber (National Coalition for Cancer Survivorship); Ethan Basch, MD (Lineberger Cancer Center at the University of North Carolina); Neil Aaronson, PhD (Netherlands Cancer Institute); Bryce Reeve, PhD (Duke University); Galina Velikova, BMBS(MD), PhD (University of Leeds, Leeds Institute of Cancer and Pathology); Andrea Heckert, PhD, MPH (Patient-Centered Outcomes Research Institute); Eden Stotsky-Himelfarb (patient representative); Cynthia Chauhan† (patient representative); Vanessa Hoffman, MPH (caregiver representative); Patricia Ganz, MD* (UCLA Jonsson Comprehensive Cancer Center); Lisa Barbera, MD, MPA† (University of Toronto and Cancer Care Ontario). Additional Consensus Panel participants included Elizabeth Frank* (patient advocate); Mary Lou Smith, JD† (patient advocate); Arturo Durazo (patient advocate); Judy Needham (patient advocate); Shelley Fuld Nasso (National Coalition for Cancer Survivorship); Robert Miller, MD (American Society of Clinical Oncology); Tenbroeck Smith, MA (American Cancer Society); Deborah Struth, MSN, RN, PhD(c) (Oncology Nursing Society); Alison Rein, MS (AcademyHealth); Andre Dias, PhD* (International Consortium for Health Outcomes Measurement); Charlotte Roberts, MBBS, BSc† (International Consortium for Health Outcomes Measurement); Nancy Smider, PhD* (Epic); Gena Cook* (Navigating Cancer); Jakob Bjorner, MD, PhD* (Optum); Holly Witteman, PhD* (Laval University); James G. Dolan, MD* (University of Rochester); Jane Blazeby, MD, MSc† (University of Bristol); Robert M. Golub, MD† (JAMA); Christine Laine, MD, MPH† (Annals of Internal Medicine); Scott Ramsey, MD, PhD† (Fred Hutchinson Cancer Research Center). Additional details on all meeting participants are listed in Appendix Table 3. (*Meeting 1 only; †Meeting 2 only.)

Funding

This study was funded by the Patient-Centered Outcomes Research Institute (PCORI). Drs. Snyder and Smith are members of the Sidney Kimmel Comprehensive Cancer Center at Johns Hopkins (P30CA006973).

Author information

Authors and Affiliations

  1. Johns Hopkins School of Medicine, 624 N. Broadway, Room 649, Baltimore, MD, 21205, USA

    Claire Snyder

  2. Johns Hopkins Bloomberg School of Public Health, Baltimore, MD, USA

    Claire Snyder, Katherine Smith & Yonaira M. Rivera

  3. Sidney Kimmel Comprehensive Cancer Center at Johns Hopkins, Baltimore, MD, USA

    Claire Snyder, Katherine Smith & Elissa Bantug

  4. Medical University of Innsbruck, Innsbruck, Austria

    Bernhard Holzner

  5. Queens Cancer Research Institute, Kingston, ON, Canada

    Michael Brundage

Authors
  1. Claire Snyder
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Katherine Smith
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Bernhard Holzner
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yonaira M. Rivera
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Elissa Bantug
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Michael Brundage
    View author publications

    You can also search for this author in PubMed Google Scholar

Consortia

PRO Data Presentation Delphi Panel

  • Daniel Weber
  • , Ethan Basch
  • , Neil Aaronson
  • , Bryce Reeve
  • , Galina Velikova
  • , Andrea Heckert
  • , Eden Stotsky-Himelfarb
  • , Cynthia Chauhan
  • , Vanessa Hoffman
  • , Patricia Ganz
  • , Lisa Barbera
  • , Elizabeth Frank
  • , Mary Lou Smith
  • , Arturo Durazo
  • , Judy Needham
  • , Shelley Fuld Nasso
  • , Robert Miller
  • , Tenbroeck Smith
  • , Deborah Struth
  • , Alison Rein
  • , Andre Dias
  • , Charlotte Roberts
  • , Nancy Smider
  • , Gena Cook
  • , Jakob Bjorner
  • , Holly Witteman
  • , James G. Dolan
  • , Jane Blazeby
  • , Robert M. Golub
  • , Christine Laine
  •  & Scott Ramsey

Corresponding author

Correspondence to Claire Snyder.

Ethics declarations

Conflict of interest

No authors declare a conflict of interest.

Ethical approval

This article does not contain any studies with human participants or animals performed by any of the authors.

Additional information

PRO Data Presentation Delphi Panel members are listed in “Acknowledgements”.

Appendix

Appendix

See Table 3.

Table 3 Meeting participants
Full size table

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Snyder, C., Smith, K., Holzner, B. et al. Making a picture worth a thousand numbers: recommendations for graphically displaying patient-reported outcomes data. Qual Life Res 28, 345–356 (2019). https://doi.org/10.1007/s11136-018-2020-3

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11136-018-2020-3

Keywords

Search

Navigation