Making a picture worth a thousand numbers: recommendations for graphically displaying patient-reported outcomes data
Patient-reported outcomes (PROs) can promote patient-centered care in multiple ways: (1) using an individual patient’s PRO data to inform his/her management, (2) providing PRO results from comparative research studies in patient educational materials/decision aids, and (3) reporting PRO results from comparative research studies in peer-reviewed publications. Patients and clinicians endorse the value of PRO data; however, variations in how PRO measures are scored and scaled, and in how the data are reported, make interpretation challenging and limit their use in clinical practice. We conducted a modified Delphi process to develop stakeholder-engaged, evidence-based recommendations for PRO data display for the three above applications to promote understanding and use.
The Consensus Panel included cancer survivors/caregivers, oncologists, PRO researchers, and application-specific end-users (e.g., electronic health record vendors, decision aid developers, journal editors). We reviewed the data display issues and their evidence base during pre-meeting webinars. We then surveyed participants’ initial perspectives, which informed discussions during an in-person meeting to develop consensus statements. These statements were ratified via a post-meeting survey.
Issues addressed by consensus statements relevant to both individual and research data applications were directionality (whether higher scores are better/worse) and conveying score meaning (e.g., none/mild/moderate/severe). Issues specific to individual patient data presentation included representation (bar charts vs. line graphs) and highlighting possibly concerning scores (absolute and change). Issues specific to research study results presentation included handling normed data, conveying statistically significant differences, illustrating clinically important differences, and displaying proportions improved/stable/worsened.
The recommendations aim to optimize accurate and meaningful interpretation of PRO data.
KeywordsPatient-reported outcomes Consensus statements Cancer Data display Clinical practice
The study was funded by the Patient-Centered Outcomes Research Institute (PCORI). Drs. Snyder and Smith are members of the Sidney Kimmel Comprehensive Cancer Center at Johns Hopkins (P30CA006973).
PRO Data Presentation Delphi Panel members
The Consensus Panel Participants included the project Stakeholder Advisory Board (SAB) and other invitees, in addition to the project team named authors. The SAB members were Daniel Weber (National Coalition for Cancer Survivorship); Ethan Basch, MD (Lineberger Cancer Center at the University of North Carolina); Neil Aaronson, PhD (Netherlands Cancer Institute); Bryce Reeve, PhD (Duke University); Galina Velikova, BMBS(MD), PhD (University of Leeds, Leeds Institute of Cancer and Pathology); Andrea Heckert, PhD, MPH (Patient-Centered Outcomes Research Institute); Eden Stotsky-Himelfarb (patient representative); Cynthia Chauhan† (patient representative); Vanessa Hoffman, MPH (caregiver representative); Patricia Ganz, MD* (UCLA Jonsson Comprehensive Cancer Center); Lisa Barbera, MD, MPA† (University of Toronto and Cancer Care Ontario). Additional Consensus Panel participants included Elizabeth Frank* (patient advocate); Mary Lou Smith, JD† (patient advocate); Arturo Durazo (patient advocate); Judy Needham (patient advocate); Shelley Fuld Nasso (National Coalition for Cancer Survivorship); Robert Miller, MD (American Society of Clinical Oncology); Tenbroeck Smith, MA (American Cancer Society); Deborah Struth, MSN, RN, PhD(c) (Oncology Nursing Society); Alison Rein, MS (AcademyHealth); Andre Dias, PhD* (International Consortium for Health Outcomes Measurement); Charlotte Roberts, MBBS, BSc† (International Consortium for Health Outcomes Measurement); Nancy Smider, PhD* (Epic); Gena Cook* (Navigating Cancer); Jakob Bjorner, MD, PhD* (Optum); Holly Witteman, PhD* (Laval University); James G. Dolan, MD* (University of Rochester); Jane Blazeby, MD, MSc† (University of Bristol); Robert M. Golub, MD† (JAMA); Christine Laine, MD, MPH† (Annals of Internal Medicine); Scott Ramsey, MD, PhD† (Fred Hutchinson Cancer Research Center). Additional details on all meeting participants are listed in Appendix Table 3. (*Meeting 1 only; †Meeting 2 only.)
This study was funded by the Patient-Centered Outcomes Research Institute (PCORI). Drs. Snyder and Smith are members of the Sidney Kimmel Comprehensive Cancer Center at Johns Hopkins (P30CA006973).
Compliance with ethical standards
Conflict of interest
No authors declare a conflict of interest.
This article does not contain any studies with human participants or animals performed by any of the authors.
- 1.U. S. Food and Drug Administration. (2009). Guidance for industry. Patient reported outcome measures: Use in medical product development to support labeling claims. Federal Register, 74(35), 65132–65133.Google Scholar
- 2.Acquadro, C., Berzon, R., Dubois, D., Leidy, N. K., Marquis, P., Revicki, D., et al. (2003). Incorporating the patient’s perspective into drug development and communication: An ad hoc task force report of the patient-reported outcomes (PRO) harmonization group meeting at the food and drug administration, February 16, 2001. Value Health, 6, 522–531.CrossRefGoogle Scholar
- 7.Au, H.-J., Ringash, J., Brundage, M., Palmer, M., Richardson, H., Meyer, R. M., et al. (2010). Added value of health-related quality of life measurement in cancer clinical trials: The experience of the NCIC CTG. Expert Review of Pharmacoeconomics & Outcomes Research, 10(2), 119–128.CrossRefGoogle Scholar
- 8.Lipscomb, J., Gotay, C. C., & Snyder, C. (Eds.). (2005). Outcomes assessment in cancer: Measures, methods, and applications. Cambridge: Cambridge Univ Press.Google Scholar
- 12.Brundage, M. D., Smith, K. C., Little, E. A., Bantug, E. T., & Snyder, C. F., PRO Data Presentation Stakeholder Advisory Board (2015). Communicating patient-reported outcome scores using graphic formats: Results from a mixed methods evaluation. Quality Life Research, 24, 2457–2472.CrossRefGoogle Scholar
- 13.Bantug, E. T., Coles, T., Smith, K. C., Snyder, C. F., Rouette, J., Brundage, M. D., et al. (2016). Graphical displays of patient-reported outcomes (PRO) for use in clinical practice: What makes a PRO picture worth a thousand words? Patient Education and Counselling, 99, 483–490.CrossRefGoogle Scholar
- 16.Tolbert, E., Snyder, C., Bantug, E., Blackford, A., Smith, K., Brundage, M., et al. (2016). Graphing group-level data from research studies for presentation to patients in educational materials and decision aids. Quality Life Research, 15, 17.Google Scholar
- 18.Qualtrics Survey Software. Retrieved August 7, 2018, from https://ictr.johnshopkins.edu/programs_resources/programs-resources/informatics/qualtrics-survey/.