The 21st Century Cures and Prescription Drug User Fee (Sixth Reauthorization) Acts include provisions directing US FDA actions regarding (1) using real-world evidence (RWE) to support regulatory decision making (e.g., new indications for approved drugs) and (2) development of guidance on patient engagement in drug development [1, 2]. To ensure alignment between these FDA initiatives, real-world data (RWD) (e.g., patient registries, administrative claims, or electronic health records) study designs need to reflect patient experiences, and early patient–researcher partnerships are necessary. Incorporating actual patient experiences is key to contextualizing individual treatment journeys. Despite substantial interest in both RWE and patient engagement, examples of effectively engaging patients in RWD study development are limited [3,4,5].
To pursue this nascent area of patient-centered research, in 2019 the Pfizer–Bristol Myers Squibb (BMS) Alliance (Pfizer–BMS) partnered with the National Health Council (NHC) and the Arrhythmia Alliance (A-A). The goal was to engage with people diagnosed with atrial fibrillation (AF) through an advisory board (AdBoard) to understand patient perspectives and enhance patient centricity for future AF RWD studies in the short term and AF RWE in the long term. Specifically, the objectives were to
better understand patient perspectives on living with AF and anticoagulant use for reducing risk of AF-related stroke,
obtain patient input on outcomes considered important to patients with AF and gather feedback on endpoints commonly evaluated in previous anticoagulant RWD studies (and those being explored for future RWD studies), and
gain insights on opportunities to educate patients with AF and patient organizations about RWE understanding, generation, and use.
This research letter reviews the approach and key learnings from the AF RWE AdBoard and is a case study for researchers interested in engaging patients on RWE for other clinical conditions.
RWD researchers from Pfizer–BMS approached the NHC and A-A to partner with them on the AdBoard. The NHC is a non-profit membership organization that works on behalf of people living with chronic diseases and disabilities and has a great deal of experience with patient engagement in RWE [4, 6, 7]. The A-A is a non-profit organization that provides information, support, education, and awareness for all those affected by cardiac arrhythmias, particularly AF. A-A led AF patient-recruitment efforts for the AdBoard and ensured materials were appropriate for patients with AF. The project team included representatives from Pfizer–BMS’s health economics and outcomes research group, NHC research staff, and the A-A founder and CEO. The team met regularly by teleconference from July to December 2019.
Advisory Board (AdBoard) Participant Identification
A-A recruited AdBoard participants through advertisements in its monthly newsletter and US- and UK-based online forums. A-A staff screened and selected applicants according to inclusion criteria: a confirmed AF diagnosis, over 50 years of age, US or UK resident, and ability to travel to New York for a 1-day, in-person meeting. Nine patients with AF were recruited (two from the UK and seven from the US), equally distributed by gender and with diverse comorbidities.
Patient-Friendly Introduction to Real-World Data (RWD)/Real-World Evidence (RWE)
Since RWD/RWE are new to most members of the patient community, introductory webinars were hosted by the NHC prior to the in-person AdBoard meeting . The objectives of the “prep” webinar sessions were to (1) provide an opportunity for the group to meet before the in-person session; (2) introduce RWD/RWE, including non-AF and AF-specific examples ; and (3) review the AdBoard purpose and goals. Participants also received a plain language RWD/RWE glossary in advance of the in-person AdBoard meeting.
In-Person AdBoard Meeting
The in-person, “Patient Perspectives on RWD” AdBoard meeting was held on November 14, 2019, in New York City, NY, USA. Prior to the AdBoard meeting, the project team developed a detailed discussion guide to ensure optimal use of participants’ time. The AdBoard started with an abbreviated review of the RWD/RWE introduction covered during the “prep” sessions, followed by three AF-specific sessions enumerated in the following sections. In addition, the agenda featured a “wrap up discussion” in which participants were invited to provide considerations and advice to researchers from the Pfizer–BMS Alliance (see the agenda in the electronic supplementary material [ESM]).
Session 1: Experiences and Perspectives on Living with Atrial Fibrillation (AF) and Anticoagulation Medication for Reducing Risk of AF-Related Stroke
Using a patient experience-mapping visual aid, we asked participants to describe experiences before their AF diagnosis, while being diagnosed with AF, and with treatment to reduce the risk of stroke. The visual aid used in this session was co-developed by a separate NHC multi-stakeholder advisory board . It depicts the process of pre-diagnosis (e.g., noticing something is wrong, annual wellness visit), deciding to see a health professional to receive a diagnosis (or misdiagnosis), and experiences after diagnosis (e.g., treatments, outcomes experienced). It also depicts modifiers, including family or support systems, costs, health insurance, comorbidities, etc. During these discussions, participants were asked to reflect on personal experiences and more general experiences as a patient with AF but to specifically avoid provider and treatment names.
Session 2: Obtain Input on Outcomes Important to Patients with AF
To solicit inputs on which outcomes are most important for researchers to study, participants were separated into two working groups to (1) brainstorm and list the AF outcomes most important to them, (2) discuss/critique outcomes evaluated in AF-related RWD studies from the past 5 years (obtained from a landscape analysis) and others proposed by participants, and (3) rank all outcomes from both discussions based on what is most important to them. Each group tracked their discussion and came up with a list of upwards of 20 ways in which AF impacts them. Following the small-group work, participants reconvened for a full-group discussion to produce a final ranked list. To further narrow down the most important outcomes, all patients rated their top five. Results were tabulated and discussed among the entire group.
Session 3: Opportunities to Engage Patients with AF in RWE and Effective Communication of RWE to Patients
To understand perspectives on preferred methods for communicating RWE, key findings from a published AF RWD study were presented using different visuals, including bar graphs, infographic style, and the article’s original tables . The positives and negatives of the communication vehicles were discussed.
Session 1: Experiences and Perspectives on Living with AF and Anticoagulation Medication for Reducing Risk of AF-Related Stroke
The exercise highlighted heterogeneity in how AF affects individuals in their everyday life. For example, while the risk of AF-related stroke and the benefits (and risks) of therapies were clear to some patients after diagnosis, they were not communicated effectively to all. An example of a key learning with application to RWD-based analyses was experience obtaining a diagnosis. Patients described numerous challenges seeking a diagnosis, including varying symptoms unrecognized as AF for years by healthcare providers (HCPs), resulting in a delayed or misdiagnosis. One patient described three scenarios where she believed symptoms were consistent with AF but were attributed to other reasons by her HCPs. In application to RWD studies, the first recorded AF diagnosis code in a patient’s data history may be off by several years from when first symptoms occurred. Consequently, researchers should recognize limitations in using diagnostic codes to capture AF onset and may consider sensitivity analyses with AF onset starting with AF symptoms (e.g., gastrointestinal disorders, fatigue, syncope, anxiety) preceding a formal AF diagnosis. This can be applied to other diseases. Additional insights and RWD recommendations are captured in Table 1.
Session 2: Obtain Input on Outcomes Important to Patients with AF
The outcomes that patients agreed were most important were as follows:
Ischemic stroke—The effects of a debilitating stroke and the burden it would place on caregiver(s) (family/friends) was a significant concern
Major bleeding—Patients are most worried about bleeding not easily seen or resolved
Health-related quality of life—Patients do not want AF to hinder mobility and lifestyle
Session 3: Opportunities to Engage Patients with AF in RWE and Effective Communication of RWE to Patients
Following individual reflection, participants had a group discussion to identify which RWE data-communication visuals were most easily understood and best to convey RWD findings. The originally published table was universally rejected as “too complicated” and “too much! No!”
Key takeaways included the following:
Participants preferred infographic-type presentations, e.g., simple tables, using arrows to describe trends.
Since patients often connect via patient forums, easily shareable graphics are desired, provided they link to original referred sources for interested patients.
Participants noted that limitations should be transparently communicated.
Of note, numerous studies are already examining the outcomes prioritized by patients in session 2. Translating the findings into accessible language and/or infographics may offer a relatively low-resource opportunity to improve patients’ access to information about those outcomes.
Participant Feedback and Evaluations
The AdBoard closed with participant reflections and advice on improving future AdBoards. Patients reported they felt more comfortable being recruited by a patient organization than being recruited directly by a HCP or life sciences company. One patient noted the patient experience-mapping exercise was useful and should have had more time devoted to it. Another stated it was nice to have the “opportunity to address pharma directly.” A patient raised that participants were too homogenous and future AdBoards should strive for more participant diversity.
We also sought to understand patient perspectives on how researchers could further engage patients with AF and caregivers when designing RWD studies. Participants expressed an interest in additional engagement with industry on the topic of RWD/RWE. Several stated they would enjoy working directly with research teams to develop RWD studies for AF.
Following the in-person AdBoard, A-A circulated an evaluation survey. The survey asked about satisfaction with preparation for the AdBoard, communications and logistics, the quality of each session, and the adequacy of time dedicated to discussion. The content and format were rated at least a 3 of 5 (5 = excellent; 1 = poor) for all sessions, with the majority rated as 4 or 5.
The AdBoard was a simple first step in patient engagement in AF RWE. Importantly, it also presented an opportunity for RWD researchers, who do not typically interact directly with patients, to be inspired and informed by individual patient experiences. Actions to advance patient engagement in RWD research and encourage patient awareness of RWE are outlined in the following sections.
Insights Gleaned Through Patient Engagement Can Assist RWD Researchers
Engaging patients yielded actionable steps for researchers as they seek to ensure RWD study designs more accurately reflect real-world patient experiences. For example, patients identified a number of factors that impacted their treatment decision making. Novel techniques for accounting for unmeasured confounding, such as adjusting for residual confounding by linking data sources with patient surveys, have been piloted .
Communicating in a Health Literate Way is Critical for Patient Uptake of RWE
Ample RWE related to many of the outcomes prioritized by patients already exists. To ensure patients have access to this RWE, researchers can familiarize themselves with methods for communicating complex research findings to patients and collaborate with colleagues from disciplines specializing in creating patient-friendly materials, for example health literacy experts [11,12,13]. This is consistent with the British Medical Journal initiative to translate key findings from research articles into easily digestible infographics . Additionally, while participants noted they would like easily digestible formats, they also indicated a link to the original published article should be provided. To ensure accessibility, open-access publishing is encouraged .
Patient AdBoards are One Method to Engage Patients in RWE and Should Strive for Diversity
Learnings from this AdBoard can inform planning for future, more formal engagement efforts with patients with AF and those with other conditions. The learnings can also help inform activities by other researchers and patient groups interested in partnering in RWE. An AdBoard can serve as an initial foray but cannot obtain all patient inputs needed. In addition to AdBoards, valuable insights can be identified in the existing qualitative research, such as studies published in the peer-reviewed literature or the FDA’s Voice-of-the-Patient series, as well as through other qualitative and quantitative methods . Future AdBoards should seek to include more diverse and representative populations.
Patient–AdBoard participants provided insights related to patient experiences with AF, outcomes most important to them, and preferred communication modes for study findings. Their perspectives have important implications for the design of RWD studies and communication of findings to patients. The approach described in this manuscript can assist other researchers interested in engaging patients in developing patient-centered RWE.
Food and Drug Administration. Real World Evidence. https://www.fda.gov/scienceresearch/specialtopics/realworldevidence/default.htm. Accessed 22 Apr 2019
Food and Drug Administration. FDA Patient-Focused Drug Development Guidance Series for Enhancing the Incorporation of the Patient’s Voice in Medical Product Development and Regulatory Decision Making. FDA. Published online February 9, 2019. https://www.fda.gov/drugs/development-approval-process-drugs/fda-patient-focused-drug-development-guidance-series-enhancingincorporation-patients-voice-medical. Accessed 29 Apr 2019
Okun S. The missing reality of real life in real-world evidence. Clin Pharmacol Ther. 2019;106(1):136–8. https://doi.org/10.1002/cpt.1465.
Oehrlein EM, Graff JS, Harris J, Perfetto EM. Patient-community perspectives on real-world evidence: Enhancing engagement, understanding, and trust. Patient. 2019. https://doi.org/10.1007/s40271-019-00356-z (Published online January 22, 2019).
Brice C, Isaac C, Jonsson P et al (2016) Patient-powered research networks for relative effectiveness research: results of a pragmatic literature review and desktop research. https://www.imi-getreal.eu/Portals/1/Documents/01%20deliverables/GetReal%20Report%20-%20Pragmatic%20Literature%20Review%20and%20Desktop%20Research%20of%20PPRN%20FINAL.pdf. Accessed 3 Apr 2020
ISPOR—Improving real-world evidence in healthcare with patient-provided information. ISPOR | International Society For Pharmacoeconomics and Outcomes Research. https://www.ispor.org/heor-resources/news/2019/05/22/improving-real-world-evidence-in-healthcare-with-patient-provided-information. Accessed 3 Apr 2020.
Methods principles for using patient-provided information to improve real-world evidence for patient-centered value assessment. National Health Council. Published May 17, 2019. https://www.nationalhealthcouncil.org/methods-principles-using-patient-provided-information-improve-real-world-evidence-patient-centered. Accessed 7 Oct 2019.
Xian Y, Wu J, O’Brien EC, et al. Real world effectiveness of warfarin among ischemic stroke patients with atrial fibrillation: Observational analysis from Patient-Centered Research into Outcomes Stroke Patients Prefer and Effectiveness Research (PROSPER) study. BMJ. 2015;351:h3786. https://doi.org/10.1136/bmj.h3786.
Development of the National Health Council Patient-Journey Mapping Toolbox. National Health Council. Published July 18, 2019. https://www.nationalhealthcouncil.org/blog/development-national-health-council-patient-journey-mapping-toolbox. Accessed 6 Aug 2019
Schneeweiss S, Glynn RJ, Tsai EH, Avorn J, Solomon DH. Adjusting for unmeasured confounders in pharmacoepidemiologic claims data using external information: The example of COX2 inhibitors and myocardial infarction. Epidemiology. 2005;16(1):17–24. https://doi.org/10.1097/01.ede.0000147164.11879.b5.
Testing Ways to Display Patient-Reported Outcomes Data for Patients and Clinicians. Published September 8, 2014. https://www.pcori.org/research-results/2012/testing-ways-display-patient-reported-outcomes-data-patients-and-clinicians. Accessed 3 Apr 2020
Snyder CF, Smith KC, Bantug ET, et al. What do these scores mean? Presenting patient-reported outcomes data to patients and clinicians to improve interpretability. Cancer. 2017;123(10):1848–59. https://doi.org/10.1002/cncr.30530.
Bantug ET, Coles T, Smith KC, et al. Graphical displays of patient-reported outcomes (PRO) for use in clinical practice: What makes a pro picture worth a thousand words? Patient Educ Couns. 2016;99(4):483–90. https://doi.org/10.1016/j.pec.2015.10.027.
The BMJ. Infographics. https://www.bmj.com/infographics. Accessed 1 Sep 2020
Day S, Rennie S, Luo D, Tucker JD. Open to the public: Paywalls and the public rationale for open access medical research publishing. Res Involv Engag. 2020;6(1):8. https://doi.org/10.1186/s40900-020-0182-y.
FDA. The Voice of the Patient: A Series of Reports from FDA’s Patient-Focused Drug Development Initiative. FDA. Published online November 4, 2019. https://www.fda.gov/industry/prescription-drug-user-fee-amendments/voice-patient-series-reports-fdas-patient-focused-drug-development-initiative. Accessed 26 May 2020
The authors acknowledge T. Rosie Love, MPH, PhD candidate at the University of Maryland, Baltimore, USA for her organization and drafting support.
Conflict of interest
EMO, SS, and EP are employees of the National Health Council (NHC) who were paid consultants to Pfizer Inc. and Bristol Myers Squibb Company (BMS) in support of focus group preparations and development of the manuscript. NHC is a not-for-profit multi-stakeholder membership organization. As such, it receives membership and sponsorship funds from a variety of organizations and businesses. The lists of members and sponsors can be found at https://www.nationalhealthcouncil.org. XL and MS are paid employees of and own stocks in Pfizer Inc. TL is founder and CEO of the Arrhythmia Alliance and is a caregiver for two patients with AF. The Arrhythmia Alliance receives funding from life science companies, including the Pfizer-BMS Alliance in support of focus group preparations for this work. AK and BL are paid employees of and have equity/stocks in BMS. RG is a board member at the Arrhythmia Alliance and an AF patient.
This project was supported by the Pfizer-Bristol Myers Squibb (BMS) Alliance.
Consent for publication
Consent to participate
Availability of data and material
Design and implementation of Advisory Board: All; Manuscript concept and design: All; Drafting of manuscript: EMO; Critical reviews: All; Final version approval: All.
Electronic supplementary material
Below is the link to the electronic supplementary material.
About this article
Cite this article
Oehrlein, E.M., Luo, X., Savone, M. et al. Engaging Patients in Real-World Evidence: An Atrial Fibrillation Patient Advisory Board Case Example. Patient 14, 295–300 (2021). https://doi.org/10.1007/s40271-020-00479-8