Patient-Community Perspectives on Real-World Evidence: Enhancing Engagement, Understanding, and Trust
Healthcare stakeholders have pronounced both enthusiasm and apprehension over the expanding use of real-world evidence (RWE). The patient community—those who benefit from new treatments but are vulnerable to potential safety risks and whose routine medical encounters are used to generate RWE—has been less vocal. Understanding patient perspectives on the use of RWE to guide clinical decision making and inform regulatory decisions and value assessments is imperative.
We convened a day-long, multi-stakeholder roundtable in Washington D.C., USA, on 31 July 2017 to gather patient-community views on RWE and related concerns and the communications, information and tools needed by patients to understand, trust, and use RWE. Participants included a convenience sample of National Health Council (NHC) members primarily representing patient groups as well as non-patient members with an interest in RWE. Participants were organized into small, pre-assigned groups, ensuring representativeness across stakeholders and patient leadership. Discussions, including storyboards, notes, and illustrative examples were captured and later analyzed thematically by NHC staff.
Ten RWE themes emerged: (1) most patients were unaware of RWE and its actual or potential uses, (2) common definitions for real-world data and RWE are needed, (3) patient organizations need RWE skills and tools, (4) patient–scientist partnerships can help differentiate high-quality RWE, (5) RWE should inform decision making, (6) clinician support is needed for RWE uptake in patient decision making, (7) communications to patients should be balanced and empowering, (8) context of use impacts RWE acceptability/trust, (9) privacy/data ownership require clarity, and (10) patient-generated data are also real-world data (RWD).
Patients see great possibility in using RWE to understand how a treatment works—to find someone that “looks like me” as assurance of how a treatment might benefit them personally. Patient groups will play a critical role in helping to educate constituents on understanding, contributing to, and using RWE. To maximize patient uptake and the co-development and application of RWE, patient groups require education and tools.
Key Points for Decision Makers
Real-world evidence (RWE) should be communicated to patients in a way that is balanced, easily understandable, and supports patient decision making.
RWE initiatives should be aligned with patient-centricity initiatives. For example, the US FDA’s RWE initiative should be aligned with their “patient-focused drug development” initiative.
As researchers, policymakers, and regulators establish standards and processes for using RWE in regulatory, value, and clinical decision making, patients must be the primary focus for us to advance a patient-centered health ecosystem.
Healthcare stakeholders have pronounced both enthusiasm and apprehension over the expanded use of real-world evidence (RWE) [1, 2, 3, 4]. However, the patient community—those who benefit from new treatments but are vulnerable to potential safety risks and whose routine medical interactions are used to generate RWE—has been less vocal [5, 6, 7, 8]. RWE is information derived from studies analyzing real-world data (RWD), which refers to “those data derived from sources other than randomized controlled trials (for example, patient registries, patient cohorts, administrative claims, or electronic health records).” . Emerging sources of data capturing the “real world” in terms of treatments, settings, and patient populations (e.g., subgroups) offer enormous opportunity for deeper understanding of why treatments work (or do not) and for whom . Patients, as well as those making decisions on their behalf, such as regulators, value assessors, and clinicians, can benefit from research that better reflects patient experiences to support their own information needs [10, 11, 12]. RWE may help clarify the best uses for regulatory agency-approved treatments and add precision to value assessment and clinical decision making .
Despite these potential advantages, standards and quality varies across RWD sources and range of study designs [7, 8, 14]. Given patient ownership of their data and rights to privacy, it is also important to understand patient perspectives on RWE for ethical reasons .
Based on the premise that patients and their advocates may not be vocal or activated on RWE because of a lack of familiarity, in 2017, the National Health Council (NHC) and National Pharmaceutical Council convened a day-long multi-stakeholder roundtable discussion. The objective was to gather patient-community views on RWE, areas of greatest concern, policy issues, and information and tools most needed to understand, trust, and use RWE . The purpose of this paper is to report insights from the discussion for consideration by patient groups, policy makers, clinicians, and researchers.
Invitations were extended to a convenience sample of NHC members primarily representing patient groups but also non-patient members with an interest in RWE (see the Electronic Supplementary Material [ESM] 1). Patient groups were oversampled to provide a majority of patient and patient-group representation, as these were the primary views sought. Representatives of non-patient stakeholder communities were invited because of their RWE content knowledge and to answer questions, provide examples, and support patient-participant engagement during each session to maximize the patient voice.
RWE, including but not limited to, issues of definition, concerns, transparency, privacy/security, sources, and meaningful use;
opportunities for improving the communication and dissemination of RWE to the patient community; and
skill sets and tools needed to understand and make the best use of RWE in decision making.
Discussion questions are listed in ESM 3. Participants were organized into five small, pre-assigned groups, ensuring representativeness across stakeholders and patient representative leadership. In addition to a note-taker, an NHC staff member was assigned to each group as a facilitator and to assure that the patient voice was predominant. Participants, including note takers, were asked to prioritize patient perspectives, questions, and viewpoints in the discussion over other stakeholder views. Groups were asked to prepare storyboards summarizing key points, which were presented to the larger group at the end of each breakout session. Following the roundtable, NHC staff used storyboards, notes, and illustrative examples to develop a representative draft summary of results. The draft was circulated to participants for comment.
In total, 50 individuals participated, representing patient organizations (n = 21), industry (n = 12), research entities (n = 3), regulatory agencies (n = 1), and other healthcare organizations (n = 3). The resulting draft themes were circulated to 38 organizations, with six providing comment (two patient organizations, three biopharmaceutical organizations, one US government agency). Note that, in this context, patient refers to patient participants in our roundtable rather than patients at large, though typically patients and family caregivers are common among patient-group staff representatives.
3.2 Patient Perspectives on Real-World Evidence (RWE)
Our roundtable discussion identified ten themes with implications for policy makers, researchers, clinicians, and patient groups.
3.2.1 Raising Patient-Community Awareness
126.96.36.199 Most Patients are Unaware of RWE and its Actual or Potential Uses
Most patients have little understanding of RWE or that controversies exist regarding selection of types of evidence used in decision making about availability of treatment, payments based on value, or choices among various treatment options. Participants noted that, while educating individual patients (i.e., the public) may not be feasible, patient organizations can play an important role in disseminating important insights from RWE.
3.2.2 Enhancing Patient-Community Capacity
188.8.131.52 Common Definitions for RWD and RWE are Vital
Efforts to clearly define RWD, its sources, and how such data can be interpreted to yield useful RWE are needed. Patient advocates believe that better defining RWE is important to increasing their confidence in the data collection methods and applications.
184.108.40.206 Skills and Tools are Needed by Patient Organizations to Facilitate Uptake of RWE
Skill sets and tools needed by the patient advocacy community to help them make the best use of and communicate about real-world evidence to constituents
1. Standardized, concise RWD and RWE definitions, universal to all stakeholders
2. Guidance to assist organizations with creating a scientific advisory council or identifying a medical director resource to help with community understanding of RWE studies and findings
3. Patient-group education materials/program on RWE uses, sources, and key issues (e.g., 15 minutes with Q&A; offered at patient advocacy organization conferences)
4. Case study database providing examples of RWE, explaining potential impact on patients
5. Brief summaries of real-world studies for patient advocacy organizations to use in communications
6. Resource that clarifies current rules on patient data ownership/rights
7. Resource of sources of patient-generated RWD (e.g., registries)
8. Fact sheet or tool that assists patient advocacy organizations with evaluating the credibility of RWE; tools to support patient-group data literacy (e.g., what good studies look like, synthesizing the volume of evidence)
9. Education and tools to build capacity for patients to co-develop real-world studies or initiatives
10. Education and tools on shared decision making using RWE
11. Tools to help patient groups understand alignment of study rigor with the context of decision making
12. Tools for clinicians to use in interpreting and discussing RWE with patients
220.127.116.11 Partnerships with the Scientific Community to Support High-Quality RWE
Participants noted the importance of partnerships between patient groups and the scientific community being developed to guide interpretation and understanding of RWE. For example, patient groups may need scientific advisory boards or to rely on author-provided summaries of RWE to help them interpret study findings.
3.2.3 Patient-Specific Uses of RWE
18.104.22.168 RWE Should Support Informed Decision Making
RWE is intended to discern how things work in real-world settings and in diverse patient populations to help guide decision making. Patient groups saw the possibility of using RWD to understand how a treatment works in diverse patient populations—to find someone that “looks like me”—as an assurance of how a treatment might benefit them personally. Studies should be designed in partnership with patients and with the intent to communicate results to patients in a balanced manner to inform decision making. Table 3 lists guiding questions that can improve trustworthiness and uptake of research findings.
22.214.171.124 Clinicians Must be Champions for Dissemination and Use
Education efforts on RWE must target the clinician community, as clinicians are a primary conduit of information to patients. For RWE to be impactful, participants emphasized the need for clinicians to understand the value of RWE to identify potential treatment options and support patient decision making.
126.96.36.199 Communications to Patients Should be Balanced and Empowering
Recommendations for communicating real-world evidence to enhance patient understanding, trust, and empowerment
Keep it simple
Use simple language, avoid acronyms or scientific jargon, indicate the intended user (is it relevant to me?) and clearly describe how patients can use this information)
Empower the patient
Communication should enable actions for decision making and help patients see a role for their contribution to studies that generate RWE
Clarify potential uses of RWE
Communication should clearly identify potential applications of RWE for patients
Emphasize high standards for methods and qualities of RWE
It is important to establish expectations for “high quality” in RWE – high-quality RWE should be communicated to patients
Openly address limitations of RWE
Communications should identify where studies are missing data, include positive and negative studies, or are outright “junk science” masquerading as RWE
The messenger is important
Patient organizations and clinicians are key arbiters and communicators of what good RWE looks like. Patient organizations have an obligation to serve in this role. They are educators of patient populations on the concepts and sources of RWE. They can develop key questions and other “litmus tests” to assist with discerning the value of RWE for their constituencies. Patient-organization communication can be an effective method to build trust and convey information in language that is understandable and relevant to the audience
Use varied communication methods tailored to the needs of the audience
Such methods may include patient focus groups, peer-facilitated listserves or discussion groups, etc. Patient organizations can build on the narrative already at play in the community and find peer champions to convey messages and combat misinformation or junk science within the patient community. Collaborations within disease-state communities can ensure consistency and optimize resources
Clinician/providers can be a resource regarding RWE
While patient organizations are one important source about emerging evidence, clinicians provide another opportunity to convey such information and can help with translating RWE for their patients to understand and use in decision making. In turn, patients should seek their doctor’s opinion about theories and information they are finding
3.2.4 Contextual Considerations
188.8.131.52 Acceptable Uses of RWE Must be Linked to the Context of its Use
Questions patients want answered to aid in their understanding and trust of real-world evidence
1. What is the purpose/objective of the study? Does it have prespecified study aims vs. post-hoc (i.e., data mining to “see what we find”)?
2. How many people were included? What were their characteristics (e.g., subgroups, comorbidities, treatment-resistant patients, rare disease)?
3. Over what time period did the study take place?
4. Do the methods align with the question/objective?
5. Who or what group conducted the study? Was the study codeveloped with patients?
6. Who owns (or holds) the data (e.g., government data, insurer owned)?
7. Who interpreted the study? What are their qualifications (e.g., clinicians, publication peer-review group, peer-patient, scientific body, government agency)?
8. What are the key findings and how are they meaningful to patients (“Why is it relevant to me?”)?
9. Who is the evidence most likely to interest or benefit?
10. How are the findings actionable for patients and clinicians?
11. Is it data (singular study, “raw” pieces of information) or evidence (repeated findings, signs of a pattern/trend, validated by replication, readily applicable to treatment dialog)?
12. How is this a novel finding, or how does it replicate or refute past work? Where does the RWE fall on the spectrum of understanding ranging from “confirms current thinking” to “changes current thinking?”
13. How should patients deal with the reality that, for some treatments, there is no clear consensus on a given treatment?
14. What are the identified limitations, including barriers/challenges, especially for patients?
184.108.40.206 Privacy Must be Protected and Data Ownership Clear to Promote Trust
Patients often lack clarity about who owns data about their health and often struggle to access their own medical records. Patient groups were concerned about efforts that seek to commoditize data about them, without a clear focus on improving their health outcomes and care. While the Health Insurance Portability and Accountability Act (HIPAA) may allow for data sharing, privacy issues remain a concern for patients, and standards of protection to ensure anonymity must be clarified.
220.127.116.11 RWE Should Include Authentic Sources of Patient Data
Electronic health records and claims data, two primary sources of RWD, are only “real” to an extent and may not reflect the diversity of disease experiences, preferences, or outcomes. Participants recommended RWE should aggregate these data with additional real-world sources, including patient-generated data (e.g., patient-organization registries).
Participants emphasized the importance of and opportunity inherent in the emergence of RWE. However, significant challenges are posed by study-design complexity and absence of clear standards for conducting and communicating RWE, especially to patients. These findings are largely consistent with research examining patient perspectives toward comparative effectiveness and patient-centered outcomes research. Law et al.  found that patients believe research should be (1) relevant to patient concerns, (2) presented in a way that is understandable to patients, and (3) useful for patient and clinician shared decision making.
Actions to advance awareness and uptake of RWE are described in the following subsections.
4.1 Investments in Capacity Building are Needed
Individual patients look to patient groups for information to inform understanding of their disease and guide treatment decision making. Policy makers, advocacy groups, and other stakeholders need to invest time and resources into educational efforts. Similar to prior education initiatives to inform clinicians about emerging methods and data sources, patient-group-specific RWE training could help develop patient-community content knowledge, leading to more meaningful contributions to and uptake of RWE .
4.2 Patient-Friendly Summaries of All Research – Including RWE – Should be Standard Practice
Researchers and policy makers can adopt policies to directly communicate with end users. For example, the Patient-Centered Outcomes Research Institute requires researchers to publish plain language summaries of their findings . Wider adoption of patient-friendly summaries would help patient-group staff communicate RWE findings to their constituents. Summaries could be copied alongside links to the full articles in patient-group e-newsletters.
4.3 Researchers Must Consider Patient Information Needs When Designing Studies
To ensure that RWE is initially developed to inform patient and clinician questions, and that study designs reflect patient experiences to the extent possible, early patient–researcher partnerships are necessary. Researchers are encouraged to publish open access to avoid pay firewalls.
4.4 RWE Should Complement Clinical Trial Evidence in Guidelines
Patient understanding alone is necessary but not sufficient. The primary source of evidence for clinicians and patients is often via clinical practice guidelines. Standards for developing clinical practice guidelines recognize the limitations of focusing on only clinical trials, which may not reflect the diversity of patients or their experiences, and instead focus on high-quality evidence from both randomized trials and RWE . Therefore, clinicians must also be educated on the interpretation of and the proper role for RWE in decision making .
4.5 Patient Registries can Supplement Traditional RWD Sources
Unlike traditional RWD sources, patient-generated data sources include unfiltered information that has not been reinterpreted through clinicians . These data sources may help researchers overcome limitations associated with electronic health record or administrative claims data . For example, data on over-the-counter medications, frailty, or activities of daily living, which are frequently cited by researchers as important unmeasured variables, may be captured [19, 21].
Participants were a convenience sample of NHC members, colleagues, and others who self-selected to participate. Patient representatives who participated were employees or lead volunteers of patient organizations and, while knowledgeable, were not necessarily patients or caregivers of patients with the disease of interest. This approach may not have captured the views of all stakeholders or the breadth of representation within the patient community.
Patient-system medical interactions have traditionally formed the basis for RWD. However, in the twenty-first century, passive patient engagement is insufficient. The themes we identified and the needs for moving forward will aid policy makers, researchers, clinicians, and patient groups in efforts to enhance understanding of, trust in, and engagement with RWE to inform decision making by all stakeholders. As researchers, policy makers, and regulators establish standards and processes for using RWE in regulatory, value, and clinical decision making, patients must be the primary focus for us to advance a patient-centered health ecosystem.
JSG, JH, and EMP contributed to the conception and design of this work. EMO, JSG, JH, and EMP all contributed to the analysis and interpretation of data; data collection, drafting and revising the work; and final approval of the version to be published. All authors agree to be accountable for all aspects of the work.
Compliance with Ethical Standards
Open Data Statement
Not applicable to this article as no datasets were generated or analyzed during the current study. Please contact the corresponding author with any questions.
This project was supported by the National Pharmaceutical Council.
Conflict of interest
EMO, JH, and EMP are employees of the NHC, a not-for-profit multi-stakeholder membership organization. As such, it receives membership and sponsorship funds from a variety of organizations and businesses. The lists of members and sponsors can be found at http://www.nationalhealthcouncil.org. As part-time faculty of the University of Maryland School of Pharmacy, EMP reports grants and contracts from the Patient-Centered Outcomes Research Institute, the Pharmaceutical Research Manufacturers Association Foundation, the National Pharmaceutical Council, Pfizer, and Exerpta Medica. JSG is employed by the National Pharmaceutical Council and owns stock in Pfizer Inc.
- 1.Food and Drug Administration. Real World Evidence. Accessed 2 Feb 2018.Google Scholar
- 2.Laws MB, Rose GS, Bezreh T, Beach MC, Taubin T, Kogelman L, Gethers M, Wilson IB. Treatment acceptance and adherence in HIV disease: patient identity and the perceived impact of physician-patient communication. Patient Preference Adherence. 2012;6:893–903. https://doi.org/10.2147/PPA.S36912.CrossRefGoogle Scholar
- 3.Sherman RE, Anderson SA, Dal Pan GJ, Gray GW, Gross T, Hunter NL, LaVange L, Marinac-Dabic D, Marks PW, Robb MA, Shuren J, Temple R, Woodcock J, Yue LQ, Califf RM. Real-world evidence—what is it and what can it tell us? N Engl J Med. 2016;375(23):2293–7. https://doi.org/10.1056/NEJMsb1609216.CrossRefGoogle Scholar
- 5.Institute for Clinical and Economic Review (ICER). Real World Evidence for Coverage Decisions: Opportunities and Challenges A Report from the 2017 ICER Membership Policy Summit. Accessed 13 Aug 2018.Google Scholar
- 14.Hampson G, Towse A, Dreitlein B, Henshall C, Pearson SD. Real world evidence for coverage decisions: opportunities and challenges. A Report from the 2017 ICER Membership Policy Summit. Accessed 14 April 2018.Google Scholar
- 15.Law E, Harrington R, Alexander GC, Saha S, Oehrlein E, Perfetto EM. Increasing uptake of comparative effectiveness and patient-centered outcomes research among stakeholders: insights from conference discussion. J Comp Eff Res. 2018;7(2):181–91. https://doi.org/10.2217/cer-2017-0060.CrossRefGoogle Scholar
- 17.Patient-Centered Outcomes Research Institute. Dissemination and implementation. Accessed 14 April 2018.Google Scholar
- 18.Institute of Medicine (US) Committee on Standards for Developing Trustworthy Clinical Practice Guidelines, Graham R, Mancher M, Miller Wolman D, et al., editors. Clinical practice guidelines we can trust. Washington D.C.: National Academies Press; 2011.Google Scholar
- 20.Woods SS, Evans NC, Frisbee KL. Integrating patient voices into health information for self-care and patient-clinician partnerships: Veterans Affairs design recommendations for patient-generated data applications. J Am Med Inform Assoc. 2016;23(3):491–5. https://doi.org/10.1093/jamia/ocv199.CrossRefGoogle Scholar
Open AccessThis article is distributed under the terms of the Creative Commons Attribution-NonCommercial 4.0 International License (http://creativecommons.org/licenses/by-nc/4.0/), which permits any noncommercial use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.