Healthcare stakeholders have pronounced both enthusiasm and apprehension over the expanding use of real-world evidence (RWE). The patient community—those who benefit from new treatments but are vulnerable to potential safety risks and whose routine medical encounters are used to generate RWE—has been less vocal. Understanding patient perspectives on the use of RWE to guide clinical decision making and inform regulatory decisions and value assessments is imperative.
We convened a day-long, multi-stakeholder roundtable in Washington D.C., USA, on 31 July 2017 to gather patient-community views on RWE and related concerns and the communications, information and tools needed by patients to understand, trust, and use RWE. Participants included a convenience sample of National Health Council (NHC) members primarily representing patient groups as well as non-patient members with an interest in RWE. Participants were organized into small, pre-assigned groups, ensuring representativeness across stakeholders and patient leadership. Discussions, including storyboards, notes, and illustrative examples were captured and later analyzed thematically by NHC staff.
Ten RWE themes emerged: (1) most patients were unaware of RWE and its actual or potential uses, (2) common definitions for real-world data and RWE are needed, (3) patient organizations need RWE skills and tools, (4) patient–scientist partnerships can help differentiate high-quality RWE, (5) RWE should inform decision making, (6) clinician support is needed for RWE uptake in patient decision making, (7) communications to patients should be balanced and empowering, (8) context of use impacts RWE acceptability/trust, (9) privacy/data ownership require clarity, and (10) patient-generated data are also real-world data (RWD).
Patients see great possibility in using RWE to understand how a treatment works—to find someone that “looks like me” as assurance of how a treatment might benefit them personally. Patient groups will play a critical role in helping to educate constituents on understanding, contributing to, and using RWE. To maximize patient uptake and the co-development and application of RWE, patient groups require education and tools.
|Real-world evidence (RWE) should be communicated to patients in a way that is balanced, easily understandable, and supports patient decision making.|
|RWE initiatives should be aligned with patient-centricity initiatives. For example, the US FDA’s RWE initiative should be aligned with their “patient-focused drug development” initiative.|
|As researchers, policymakers, and regulators establish standards and processes for using RWE in regulatory, value, and clinical decision making, patients must be the primary focus for us to advance a patient-centered health ecosystem.|
Healthcare stakeholders have pronounced both enthusiasm and apprehension over the expanded use of real-world evidence (RWE) [1,2,3,4]. However, the patient community—those who benefit from new treatments but are vulnerable to potential safety risks and whose routine medical interactions are used to generate RWE—has been less vocal [5,6,7,8]. RWE is information derived from studies analyzing real-world data (RWD), which refers to “those data derived from sources other than randomized controlled trials (for example, patient registries, patient cohorts, administrative claims, or electronic health records).” . Emerging sources of data capturing the “real world” in terms of treatments, settings, and patient populations (e.g., subgroups) offer enormous opportunity for deeper understanding of why treatments work (or do not) and for whom . Patients, as well as those making decisions on their behalf, such as regulators, value assessors, and clinicians, can benefit from research that better reflects patient experiences to support their own information needs [10,11,12]. RWE may help clarify the best uses for regulatory agency-approved treatments and add precision to value assessment and clinical decision making .
Despite these potential advantages, standards and quality varies across RWD sources and range of study designs [7, 8, 14]. Given patient ownership of their data and rights to privacy, it is also important to understand patient perspectives on RWE for ethical reasons .
Based on the premise that patients and their advocates may not be vocal or activated on RWE because of a lack of familiarity, in 2017, the National Health Council (NHC) and National Pharmaceutical Council convened a day-long multi-stakeholder roundtable discussion. The objective was to gather patient-community views on RWE, areas of greatest concern, policy issues, and information and tools most needed to understand, trust, and use RWE . The purpose of this paper is to report insights from the discussion for consideration by patient groups, policy makers, clinicians, and researchers.
Invitations were extended to a convenience sample of NHC members primarily representing patient groups but also non-patient members with an interest in RWE (see the Electronic Supplementary Material [ESM] 1). Patient groups were oversampled to provide a majority of patient and patient-group representation, as these were the primary views sought. Representatives of non-patient stakeholder communities were invited because of their RWE content knowledge and to answer questions, provide examples, and support patient-participant engagement during each session to maximize the patient voice.
Before the meeting, a briefing document with an overview of RWE definitions, key issues, and information from patient qualitative interviews was provided (see ESM 2). To open the meeting, participants heard context-setting remarks on RWE from a government regulator, health policy research center, large US payer with research capabilities, and a large patient organization. The opening panel was followed by three breakout sessions to gather patient-community perspectives on:
RWE, including but not limited to, issues of definition, concerns, transparency, privacy/security, sources, and meaningful use;
opportunities for improving the communication and dissemination of RWE to the patient community; and
skill sets and tools needed to understand and make the best use of RWE in decision making.
Discussion questions are listed in ESM 3. Participants were organized into five small, pre-assigned groups, ensuring representativeness across stakeholders and patient representative leadership. In addition to a note-taker, an NHC staff member was assigned to each group as a facilitator and to assure that the patient voice was predominant. Participants, including note takers, were asked to prioritize patient perspectives, questions, and viewpoints in the discussion over other stakeholder views. Groups were asked to prepare storyboards summarizing key points, which were presented to the larger group at the end of each breakout session. Following the roundtable, NHC staff used storyboards, notes, and illustrative examples to develop a representative draft summary of results. The draft was circulated to participants for comment.
In total, 50 individuals participated, representing patient organizations (n = 21), industry (n = 12), research entities (n = 3), regulatory agencies (n = 1), and other healthcare organizations (n = 3). The resulting draft themes were circulated to 38 organizations, with six providing comment (two patient organizations, three biopharmaceutical organizations, one US government agency). Note that, in this context, patient refers to patient participants in our roundtable rather than patients at large, though typically patients and family caregivers are common among patient-group staff representatives.
Patient Perspectives on Real-World Evidence (RWE)
Our roundtable discussion identified ten themes with implications for policy makers, researchers, clinicians, and patient groups.
Raising Patient-Community Awareness
Most Patients are Unaware of RWE and its Actual or Potential Uses
Most patients have little understanding of RWE or that controversies exist regarding selection of types of evidence used in decision making about availability of treatment, payments based on value, or choices among various treatment options. Participants noted that, while educating individual patients (i.e., the public) may not be feasible, patient organizations can play an important role in disseminating important insights from RWE.
Enhancing Patient-Community Capacity
Common Definitions for RWD and RWE are Vital
Efforts to clearly define RWD, its sources, and how such data can be interpreted to yield useful RWE are needed. Patient advocates believe that better defining RWE is important to increasing their confidence in the data collection methods and applications.
Skills and Tools are Needed by Patient Organizations to Facilitate Uptake of RWE
Table 1 lists the skill sets and tools participants suggested for patient groups to enhance communication, understanding, and use of insights from RWE among constituents. This list can be viewed as a broad first step to public understanding of RWE. Similar to how patient groups translate findings from clinical trials to their patient members, patient groups can serve as arbiters of findings from RWE. To make the best use of and communicate about RWE to their constituents, patient organizations require education and tools related to (1) introduction to RWE and RWD sources; (2) evaluating and disseminating RWE to patients; (3) RWE for shared decision making; and (4) co-developing RWE. Suggested formats should be easy to understand, brief, and created with patient input to ensure accessibility and usability. The goal is not to make patients or patient advocacy organizations RWE “method experts”, but rather to support patient advocacy organizations in discerning between good-quality RWE and “junk science”.
Partnerships with the Scientific Community to Support High-Quality RWE
Participants noted the importance of partnerships between patient groups and the scientific community being developed to guide interpretation and understanding of RWE. For example, patient groups may need scientific advisory boards or to rely on author-provided summaries of RWE to help them interpret study findings.
Patient-Specific Uses of RWE
RWE Should Support Informed Decision Making
RWE is intended to discern how things work in real-world settings and in diverse patient populations to help guide decision making. Patient groups saw the possibility of using RWD to understand how a treatment works in diverse patient populations—to find someone that “looks like me”—as an assurance of how a treatment might benefit them personally. Studies should be designed in partnership with patients and with the intent to communicate results to patients in a balanced manner to inform decision making. Table 3 lists guiding questions that can improve trustworthiness and uptake of research findings.
Clinicians Must be Champions for Dissemination and Use
Education efforts on RWE must target the clinician community, as clinicians are a primary conduit of information to patients. For RWE to be impactful, participants emphasized the need for clinicians to understand the value of RWE to identify potential treatment options and support patient decision making.
Communications to Patients Should be Balanced and Empowering
Roundtable participants identified important principles to guide communication of RWE to patients. These recommendations are summarized in Table 2.
Acceptable Uses of RWE Must be Linked to the Context of its Use
RWE may serve different purposes for different diseases (e.g., inform surrogate outcomes for future trials for rare conditions). Patient representatives generally agreed that RWE should not be used for clinical evaluation of new, unapproved therapies. However, patients saw opportunity for RWE to inform new uses of approved therapies in, for example, new patient subgroups (e.g., children), for treatment of comorbid conditions, and to achieve patient-defined endpoints not part of completed clinical studies (Table 3).
Privacy Must be Protected and Data Ownership Clear to Promote Trust
Patients often lack clarity about who owns data about their health and often struggle to access their own medical records. Patient groups were concerned about efforts that seek to commoditize data about them, without a clear focus on improving their health outcomes and care. While the Health Insurance Portability and Accountability Act (HIPAA) may allow for data sharing, privacy issues remain a concern for patients, and standards of protection to ensure anonymity must be clarified.
RWE Should Include Authentic Sources of Patient Data
Electronic health records and claims data, two primary sources of RWD, are only “real” to an extent and may not reflect the diversity of disease experiences, preferences, or outcomes. Participants recommended RWE should aggregate these data with additional real-world sources, including patient-generated data (e.g., patient-organization registries).
Participants emphasized the importance of and opportunity inherent in the emergence of RWE. However, significant challenges are posed by study-design complexity and absence of clear standards for conducting and communicating RWE, especially to patients. These findings are largely consistent with research examining patient perspectives toward comparative effectiveness and patient-centered outcomes research. Law et al.  found that patients believe research should be (1) relevant to patient concerns, (2) presented in a way that is understandable to patients, and (3) useful for patient and clinician shared decision making.
Actions to advance awareness and uptake of RWE are described in the following subsections.
Investments in Capacity Building are Needed
Individual patients look to patient groups for information to inform understanding of their disease and guide treatment decision making. Policy makers, advocacy groups, and other stakeholders need to invest time and resources into educational efforts. Similar to prior education initiatives to inform clinicians about emerging methods and data sources, patient-group-specific RWE training could help develop patient-community content knowledge, leading to more meaningful contributions to and uptake of RWE .
Patient-Friendly Summaries of All Research – Including RWE – Should be Standard Practice
Researchers and policy makers can adopt policies to directly communicate with end users. For example, the Patient-Centered Outcomes Research Institute requires researchers to publish plain language summaries of their findings . Wider adoption of patient-friendly summaries would help patient-group staff communicate RWE findings to their constituents. Summaries could be copied alongside links to the full articles in patient-group e-newsletters.
Researchers Must Consider Patient Information Needs When Designing Studies
To ensure that RWE is initially developed to inform patient and clinician questions, and that study designs reflect patient experiences to the extent possible, early patient–researcher partnerships are necessary. Researchers are encouraged to publish open access to avoid pay firewalls.
RWE Should Complement Clinical Trial Evidence in Guidelines
Patient understanding alone is necessary but not sufficient. The primary source of evidence for clinicians and patients is often via clinical practice guidelines. Standards for developing clinical practice guidelines recognize the limitations of focusing on only clinical trials, which may not reflect the diversity of patients or their experiences, and instead focus on high-quality evidence from both randomized trials and RWE . Therefore, clinicians must also be educated on the interpretation of and the proper role for RWE in decision making .
Patient Registries can Supplement Traditional RWD Sources
Unlike traditional RWD sources, patient-generated data sources include unfiltered information that has not been reinterpreted through clinicians . These data sources may help researchers overcome limitations associated with electronic health record or administrative claims data . For example, data on over-the-counter medications, frailty, or activities of daily living, which are frequently cited by researchers as important unmeasured variables, may be captured [19, 21].
Participants were a convenience sample of NHC members, colleagues, and others who self-selected to participate. Patient representatives who participated were employees or lead volunteers of patient organizations and, while knowledgeable, were not necessarily patients or caregivers of patients with the disease of interest. This approach may not have captured the views of all stakeholders or the breadth of representation within the patient community.
Patient-system medical interactions have traditionally formed the basis for RWD. However, in the twenty-first century, passive patient engagement is insufficient. The themes we identified and the needs for moving forward will aid policy makers, researchers, clinicians, and patient groups in efforts to enhance understanding of, trust in, and engagement with RWE to inform decision making by all stakeholders. As researchers, policy makers, and regulators establish standards and processes for using RWE in regulatory, value, and clinical decision making, patients must be the primary focus for us to advance a patient-centered health ecosystem.
Food and Drug Administration. Real World Evidence. Accessed 2 Feb 2018.
Laws MB, Rose GS, Bezreh T, Beach MC, Taubin T, Kogelman L, Gethers M, Wilson IB. Treatment acceptance and adherence in HIV disease: patient identity and the perceived impact of physician-patient communication. Patient Preference Adherence. 2012;6:893–903. https://doi.org/10.2147/PPA.S36912.
Sherman RE, Anderson SA, Dal Pan GJ, Gray GW, Gross T, Hunter NL, LaVange L, Marinac-Dabic D, Marks PW, Robb MA, Shuren J, Temple R, Woodcock J, Yue LQ, Califf RM. Real-world evidence—what is it and what can it tell us? N Engl J Med. 2016;375(23):2293–7. https://doi.org/10.1056/NEJMsb1609216.
Oehrlein EM, Graff JS, Perfetto EM, Mullins CD, Dubois RW, Anyanwu C, Onukwugha E. Peer-Reviewed Journal Editors’ views on real-world evidence. Int J Technol Assess Health Care. 2018;34(1):111–9. https://doi.org/10.1017/S0266462317004408.
Institute for Clinical and Economic Review (ICER). Real World Evidence for Coverage Decisions: Opportunities and Challenges A Report from the 2017 ICER Membership Policy Summit. Accessed 13 Aug 2018.
Mursaleen LR, Stamford JA, Jones DA, Windle R, Isaacs T. Attitudes towards data collection, ownership and sharing among patients with Parkinson’s disease. J Parkinsons Dis. 2017;7(3):523–31. https://doi.org/10.3233/JPD-161045.
Asche CV, Seal B, Kahler KH, Oehrlein EM, Baumgartner MG. Evaluation of healthcare interventions and big data: review of associated data issues. Pharmacoeconomics. 2017. https://doi.org/10.1007/s40273-017-0513-5.
Basu A, Axelsen K, Grabowski DC, Meltzer DO, Polsky D, Ridley DB, Wiederkehr D, Philipson TJ. Real-world data: policy issues regarding their access and use. Med Care. 2016;54(12):1038–44. https://doi.org/10.1097/MLR.0000000000000603.
Garrison LP Jr, Neumann PJ, Erickson P, Marshall D, Mullins CD. Using real-world data for coverage and payment decisions: the ISPOR Real-World Data Task Force report. Value Health. 2007;10(5):326–35. https://doi.org/10.1111/j.1524-4733.2007.00186.x.
Forsythe LP, Frank LB, Workman TA, Hilliard T, Harwell D, Fayish L. Patient, caregiver and clinician views on engagement in comparative effectiveness research. J Comp Eff Res. 2017;6(3):231–44. https://doi.org/10.2217/cer-2016-0062.
McDonald L, Lambrelli D, Wasiak R, Ramagopalan SV. Real-world data in the United Kingdom: opportunities and challenges. BMC Med. 2016;14(1):97.
Perfetto EM, Burke L, Oehrlein EM, Epstein RS. Patient-focused drug development: a new direction for collaboration. Med Care. 2015;53(1):9–17. https://doi.org/10.1097/MLR.0000000000000273.
Fendrick M, Shope M. Precision benefit design: using clinical benefit to guide how health care dollars are spent. J Clin Pathw. 2018;4(7):39–40. https://doi.org/10.25270/JCP.2018.09.00032.
Hampson G, Towse A, Dreitlein B, Henshall C, Pearson SD. Real world evidence for coverage decisions: opportunities and challenges. A Report from the 2017 ICER Membership Policy Summit. Accessed 14 April 2018.
Law E, Harrington R, Alexander GC, Saha S, Oehrlein E, Perfetto EM. Increasing uptake of comparative effectiveness and patient-centered outcomes research among stakeholders: insights from conference discussion. J Comp Eff Res. 2018;7(2):181–91. https://doi.org/10.2217/cer-2017-0060.
Perfetto EM, Anyanwu C, Pickering MK, Zaghab RW, Graff JS, Eichelberger B. Got CER? Educating pharmacists for practice in the future: new tools for new challenges. J Manag Care Spec Pharm. 2016;22(6):609–16. https://doi.org/10.18553/jmcp.2016.22.6.609.
Patient-Centered Outcomes Research Institute. Dissemination and implementation. Accessed 14 April 2018.
Institute of Medicine (US) Committee on Standards for Developing Trustworthy Clinical Practice Guidelines, Graham R, Mancher M, Miller Wolman D, et al., editors. Clinical practice guidelines we can trust. Washington D.C.: National Academies Press; 2011.
Norgaard M, Ehrenstein V, Vandenbroucke JP. Confounding in observational studies based on large health care databases: problems and potential solutions: a primer for the clinician. Clin Epidemiol. 2017;9:185–93. https://doi.org/10.2147/CLEP.S129879.
Woods SS, Evans NC, Frisbee KL. Integrating patient voices into health information for self-care and patient-clinician partnerships: Veterans Affairs design recommendations for patient-generated data applications. J Am Med Inform Assoc. 2016;23(3):491–5. https://doi.org/10.1093/jamia/ocv199.
Kim DH, Schneeweiss S. Measuring frailty using claims data for pharmacoepidemiologic studies of mortality in older adults: evidence and recommendations. Pharmacoepidemiol Drug Saf. 2014;23(9):891–901. https://doi.org/10.1002/pds.3674.
Open Data Statement
Not applicable to this article as no datasets were generated or analyzed during the current study. Please contact the corresponding author with any questions.
This project was supported by the National Pharmaceutical Council.
Conflict of interest
EMO, JH, and EMP are employees of the NHC, a not-for-profit multi-stakeholder membership organization. As such, it receives membership and sponsorship funds from a variety of organizations and businesses. The lists of members and sponsors can be found at http://www.nationalhealthcouncil.org. As part-time faculty of the University of Maryland School of Pharmacy, EMP reports grants and contracts from the Patient-Centered Outcomes Research Institute, the Pharmaceutical Research Manufacturers Association Foundation, the National Pharmaceutical Council, Pfizer, and Exerpta Medica. JSG is employed by the National Pharmaceutical Council and owns stock in Pfizer Inc.
About this article
Cite this article
Oehrlein, E.M., Graff, J.S., Harris, J. et al. Patient-Community Perspectives on Real-World Evidence: Enhancing Engagement, Understanding, and Trust. Patient 12, 375–381 (2019). https://doi.org/10.1007/s40271-019-00356-z