Skip to main content

Engaging Patients in Real-World Evidence: An Atrial Fibrillation Patient Advisory Board Case Example


The 21st Century Cures and Prescription Drug User Fee (Sixth Reauthorization) Acts include provisions directing US FDA actions regarding (1) using real-world evidence (RWE) to support regulatory decision making (e.g., new indications for approved drugs) and (2) development of guidance on patient engagement in drug development [1, 2]. To ensure alignment between these FDA initiatives, real-world data (RWD) (e.g., patient registries, administrative claims, or electronic health records) study designs need to reflect patient experiences, and early patient–researcher partnerships are necessary. Incorporating actual patient experiences is key to contextualizing individual treatment journeys. Despite substantial interest in both RWE and patient engagement, examples of effectively engaging patients in RWD study development are limited [3,4,5].

To pursue this nascent area of patient-centered research, in 2019 the Pfizer–Bristol Myers Squibb (BMS) Alliance (Pfizer–BMS) partnered with the National Health Council (NHC) and the Arrhythmia Alliance (A-A). The goal was to engage with people diagnosed with atrial fibrillation (AF) through an advisory board (AdBoard) to understand patient perspectives and enhance patient centricity for future AF RWD studies in the short term and AF RWE in the long term. Specifically, the objectives were to

  1. 1.

    better understand patient perspectives on living with AF and anticoagulant use for reducing risk of AF-related stroke,

  2. 2.

    obtain patient input on outcomes considered important to patients with AF and gather feedback on endpoints commonly evaluated in previous anticoagulant RWD studies (and those being explored for future RWD studies), and

  3. 3.

    gain insights on opportunities to educate patients with AF and patient organizations about RWE understanding, generation, and use.

This research letter reviews the approach and key learnings from the AF RWE AdBoard and is a case study for researchers interested in engaging patients on RWE for other clinical conditions.


Project Team

RWD researchers from Pfizer–BMS approached the NHC and A-A to partner with them on the AdBoard. The NHC is a non-profit membership organization that works on behalf of people living with chronic diseases and disabilities and has a great deal of experience with patient engagement in RWE [4, 6, 7]. The A-A is a non-profit organization that provides information, support, education, and awareness for all those affected by cardiac arrhythmias, particularly AF. A-A led AF patient-recruitment efforts for the AdBoard and ensured materials were appropriate for patients with AF. The project team included representatives from Pfizer–BMS’s health economics and outcomes research group, NHC research staff, and the A-A founder and CEO. The team met regularly by teleconference from July to December 2019.

Advisory Board (AdBoard) Participant Identification

A-A recruited AdBoard participants through advertisements in its monthly newsletter and US- and UK-based online forums. A-A staff screened and selected applicants according to inclusion criteria: a confirmed AF diagnosis, over 50 years of age, US or UK resident, and ability to travel to New York for a 1-day, in-person meeting. Nine patients with AF were recruited (two from the UK and seven from the US), equally distributed by gender and with diverse comorbidities.

Patient-Friendly Introduction to Real-World Data (RWD)/Real-World Evidence (RWE)

Since RWD/RWE are new to most members of the patient community, introductory webinars were hosted by the NHC prior to the in-person AdBoard meeting [4]. The objectives of the “prep” webinar sessions were to (1) provide an opportunity for the group to meet before the in-person session; (2) introduce RWD/RWE, including non-AF and AF-specific examples [8]; and (3) review the AdBoard purpose and goals. Participants also received a plain language RWD/RWE glossary in advance of the in-person AdBoard meeting.

In-Person AdBoard Meeting

The in-person, “Patient Perspectives on RWD” AdBoard meeting was held on November 14, 2019, in New York City, NY, USA. Prior to the AdBoard meeting, the project team developed a detailed discussion guide to ensure optimal use of participants’ time. The AdBoard started with an abbreviated review of the RWD/RWE introduction covered during the “prep” sessions, followed by three AF-specific sessions enumerated in the following sections. In addition, the agenda featured a “wrap up discussion” in which participants were invited to provide considerations and advice to researchers from the Pfizer–BMS Alliance (see the agenda in the electronic supplementary material [ESM]).

Session 1: Experiences and Perspectives on Living with Atrial Fibrillation (AF) and Anticoagulation Medication for Reducing Risk of AF-Related Stroke

Using a patient experience-mapping visual aid, we asked participants to describe experiences before their AF diagnosis, while being diagnosed with AF, and with treatment to reduce the risk of stroke. The visual aid used in this session was co-developed by a separate NHC multi-stakeholder advisory board [9]. It depicts the process of pre-diagnosis (e.g., noticing something is wrong, annual wellness visit), deciding to see a health professional to receive a diagnosis (or misdiagnosis), and experiences after diagnosis (e.g., treatments, outcomes experienced). It also depicts modifiers, including family or support systems, costs, health insurance, comorbidities, etc. During these discussions, participants were asked to reflect on personal experiences and more general experiences as a patient with AF but to specifically avoid provider and treatment names.

Session 2: Obtain Input on Outcomes Important to Patients with AF

To solicit inputs on which outcomes are most important for researchers to study, participants were separated into two working groups to (1) brainstorm and list the AF outcomes most important to them, (2) discuss/critique outcomes evaluated in AF-related RWD studies from the past 5 years (obtained from a landscape analysis) and others proposed by participants, and (3) rank all outcomes from both discussions based on what is most important to them. Each group tracked their discussion and came up with a list of upwards of 20 ways in which AF impacts them. Following the small-group work, participants reconvened for a full-group discussion to produce a final ranked list. To further narrow down the most important outcomes, all patients rated their top five. Results were tabulated and discussed among the entire group.

Session 3: Opportunities to Engage Patients with AF in RWE and Effective Communication of RWE to Patients

To understand perspectives on preferred methods for communicating RWE, key findings from a published AF RWD study were presented using different visuals, including bar graphs, infographic style, and the article’s original tables [8]. The positives and negatives of the communication vehicles were discussed.


Session 1: Experiences and Perspectives on Living with AF and Anticoagulation Medication for Reducing Risk of AF-Related Stroke

The exercise highlighted heterogeneity in how AF affects individuals in their everyday life. For example, while the risk of AF-related stroke and the benefits (and risks) of therapies were clear to some patients after diagnosis, they were not communicated effectively to all. An example of a key learning with application to RWD-based analyses was experience obtaining a diagnosis. Patients described numerous challenges seeking a diagnosis, including varying symptoms unrecognized as AF for years by healthcare providers (HCPs), resulting in a delayed or misdiagnosis. One patient described three scenarios where she believed symptoms were consistent with AF but were attributed to other reasons by her HCPs. In application to RWD studies, the first recorded AF diagnosis code in a patient’s data history may be off by several years from when first symptoms occurred. Consequently, researchers should recognize limitations in using diagnostic codes to capture AF onset and may consider sensitivity analyses with AF onset starting with AF symptoms (e.g., gastrointestinal disorders, fatigue, syncope, anxiety) preceding a formal AF diagnosis. This can be applied to other diseases. Additional insights and RWD recommendations are captured in Table 1.

Table 1 Session 1 discussion and recommendations for real-world data researchers

Session 2: Obtain Input on Outcomes Important to Patients with AF

The outcomes that patients agreed were most important were as follows:

  • Ischemic stroke—The effects of a debilitating stroke and the burden it would place on caregiver(s) (family/friends) was a significant concern

  • Major bleeding—Patients are most worried about bleeding not easily seen or resolved

  • Health-related quality of life—Patients do not want AF to hinder mobility and lifestyle

Session 3: Opportunities to Engage Patients with AF in RWE and Effective Communication of RWE to Patients

Following individual reflection, participants had a group discussion to identify which RWE data-communication visuals were most easily understood and best to convey RWD findings. The originally published table was universally rejected as “too complicated” and “too much! No!”

Key takeaways included the following:

  • Participants preferred infographic-type presentations, e.g., simple tables, using arrows to describe trends.

  • Since patients often connect via patient forums, easily shareable graphics are desired, provided they link to original referred sources for interested patients.

  • Participants noted that limitations should be transparently communicated.

Of note, numerous studies are already examining the outcomes prioritized by patients in session 2. Translating the findings into accessible language and/or infographics may offer a relatively low-resource opportunity to improve patients’ access to information about those outcomes.

Participant Feedback and Evaluations

The AdBoard closed with participant reflections and advice on improving future AdBoards. Patients reported they felt more comfortable being recruited by a patient organization than being recruited directly by a HCP or life sciences company. One patient noted the patient experience-mapping exercise was useful and should have had more time devoted to it. Another stated it was nice to have the “opportunity to address pharma directly.” A patient raised that participants were too homogenous and future AdBoards should strive for more participant diversity.

We also sought to understand patient perspectives on how researchers could further engage patients with AF and caregivers when designing RWD studies. Participants expressed an interest in additional engagement with industry on the topic of RWD/RWE. Several stated they would enjoy working directly with research teams to develop RWD studies for AF.

Following the in-person AdBoard, A-A circulated an evaluation survey. The survey asked about satisfaction with preparation for the AdBoard, communications and logistics, the quality of each session, and the adequacy of time dedicated to discussion. The content and format were rated at least a 3 of 5 (5 = excellent; 1 = poor) for all sessions, with the majority rated as 4 or 5.


The AdBoard was a simple first step in patient engagement in AF RWE. Importantly, it also presented an opportunity for RWD researchers, who do not typically interact directly with patients, to be inspired and informed by individual patient experiences. Actions to advance patient engagement in RWD research and encourage patient awareness of RWE are outlined in the following sections.

Insights Gleaned Through Patient Engagement Can Assist RWD Researchers

Engaging patients yielded actionable steps for researchers as they seek to ensure RWD study designs more accurately reflect real-world patient experiences. For example, patients identified a number of factors that impacted their treatment decision making. Novel techniques for accounting for unmeasured confounding, such as adjusting for residual confounding by linking data sources with patient surveys, have been piloted [10].

Communicating in a Health Literate Way is Critical for Patient Uptake of RWE

Ample RWE related to many of the outcomes prioritized by patients already exists. To ensure patients have access to this RWE, researchers can familiarize themselves with methods for communicating complex research findings to patients and collaborate with colleagues from disciplines specializing in creating patient-friendly materials, for example health literacy experts [11,12,13]. This is consistent with the British Medical Journal initiative to translate key findings from research articles into easily digestible infographics [14]. Additionally, while participants noted they would like easily digestible formats, they also indicated a link to the original published article should be provided. To ensure accessibility, open-access publishing is encouraged [15].

Patient AdBoards are One Method to Engage Patients in RWE and Should Strive for Diversity

Learnings from this AdBoard can inform planning for future, more formal engagement efforts with patients with AF and those with other conditions. The learnings can also help inform activities by other researchers and patient groups interested in partnering in RWE. An AdBoard can serve as an initial foray but cannot obtain all patient inputs needed. In addition to AdBoards, valuable insights can be identified in the existing qualitative research, such as studies published in the peer-reviewed literature or the FDA’s Voice-of-the-Patient series, as well as through other qualitative and quantitative methods [16]. Future AdBoards should seek to include more diverse and representative populations.


Patient–AdBoard participants provided insights related to patient experiences with AF, outcomes most important to them, and preferred communication modes for study findings. Their perspectives have important implications for the design of RWD studies and communication of findings to patients. The approach described in this manuscript can assist other researchers interested in engaging patients in developing patient-centered RWE.


  1. 1.

    Food and Drug Administration. Real World Evidence. Accessed 22 Apr 2019

  2. 2.

    Food and Drug Administration. FDA Patient-Focused Drug Development Guidance Series for Enhancing the Incorporation of the Patient’s Voice in Medical Product Development and Regulatory Decision Making. FDA. Published online February 9, 2019. Accessed 29 Apr 2019

  3. 3.

    Okun S. The missing reality of real life in real-world evidence. Clin Pharmacol Ther. 2019;106(1):136–8.

    Article  PubMed  PubMed Central  Google Scholar 

  4. 4.

    Oehrlein EM, Graff JS, Harris J, Perfetto EM. Patient-community perspectives on real-world evidence: Enhancing engagement, understanding, and trust. Patient. 2019. (Published online January 22, 2019).

    Article  PubMed  PubMed Central  Google Scholar 

  5. 5.

    Brice C, Isaac C, Jonsson P et al (2016) Patient-powered research networks for relative effectiveness research: results of a pragmatic literature review and desktop research. Accessed 3 Apr 2020

  6. 6.

    ISPOR—Improving real-world evidence in healthcare with patient-provided information. ISPOR | International Society For Pharmacoeconomics and Outcomes Research. Accessed 3 Apr 2020.

  7. 7.

    Methods principles for using patient-provided information to improve real-world evidence for patient-centered value assessment. National Health Council. Published May 17, 2019. Accessed 7 Oct 2019.

  8. 8.

    Xian Y, Wu J, O’Brien EC, et al. Real world effectiveness of warfarin among ischemic stroke patients with atrial fibrillation: Observational analysis from Patient-Centered Research into Outcomes Stroke Patients Prefer and Effectiveness Research (PROSPER) study. BMJ. 2015;351:h3786.

    Article  PubMed  PubMed Central  Google Scholar 

  9. 9.

    Development of the National Health Council Patient-Journey Mapping Toolbox. National Health Council. Published July 18, 2019. Accessed 6 Aug 2019

  10. 10.

    Schneeweiss S, Glynn RJ, Tsai EH, Avorn J, Solomon DH. Adjusting for unmeasured confounders in pharmacoepidemiologic claims data using external information: The example of COX2 inhibitors and myocardial infarction. Epidemiology. 2005;16(1):17–24.

    Article  PubMed  Google Scholar 

  11. 11.

    Testing Ways to Display Patient-Reported Outcomes Data for Patients and Clinicians. Published September 8, 2014. Accessed 3 Apr 2020

  12. 12.

    Snyder CF, Smith KC, Bantug ET, et al. What do these scores mean? Presenting patient-reported outcomes data to patients and clinicians to improve interpretability. Cancer. 2017;123(10):1848–59.

    Article  PubMed  PubMed Central  Google Scholar 

  13. 13.

    Bantug ET, Coles T, Smith KC, et al. Graphical displays of patient-reported outcomes (PRO) for use in clinical practice: What makes a pro picture worth a thousand words? Patient Educ Couns. 2016;99(4):483–90.

    Article  PubMed  Google Scholar 

  14. 14.

    The BMJ. Infographics. Accessed 1 Sep 2020

  15. 15.

    Day S, Rennie S, Luo D, Tucker JD. Open to the public: Paywalls and the public rationale for open access medical research publishing. Res Involv Engag. 2020;6(1):8.

    Article  Google Scholar 

  16. 16.

    FDA. The Voice of the Patient: A Series of Reports from FDA’s Patient-Focused Drug Development Initiative. FDA. Published online November 4, 2019. Accessed 26 May 2020

Download references


The authors acknowledge T. Rosie Love, MPH, PhD candidate at the University of Maryland, Baltimore, USA for her organization and drafting support.

Author information



Corresponding author

Correspondence to Elisabeth M. Oehrlein.

Ethics declarations

Conflict of interest

EMO, SS, and EP are employees of the National Health Council (NHC) who were paid consultants to Pfizer Inc. and Bristol Myers Squibb Company (BMS) in support of focus group preparations and development of the manuscript. NHC is a not-for-profit multi-stakeholder membership organization. As such, it receives membership and sponsorship funds from a variety of organizations and businesses. The lists of members and sponsors can be found at XL and MS are paid employees of and own stocks in Pfizer Inc. TL is founder and CEO of the Arrhythmia Alliance and is a caregiver for two patients with AF. The Arrhythmia Alliance receives funding from life science companies, including the Pfizer-BMS Alliance in support of focus group preparations for this work. AK and BL are paid employees of and have equity/stocks in BMS. RG is a board member at the Arrhythmia Alliance and an AF patient.


This project was supported by the Pfizer-Bristol Myers Squibb (BMS) Alliance.

Ethics approval

Not applicable.

Consent for publication

Not applicable.

Consent to participate

Not applicable.

Availability of data and material

Not applicable.

Code availability

Not applicable.

Author contributions

Design and implementation of Advisory Board: All; Manuscript concept and design: All; Drafting of manuscript: EMO; Critical reviews: All; Final version approval: All.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (PDF 109 kb)

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License, which permits any non-commercial use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Oehrlein, E.M., Luo, X., Savone, M. et al. Engaging Patients in Real-World Evidence: An Atrial Fibrillation Patient Advisory Board Case Example. Patient 14, 295–300 (2021).

Download citation