Journal of General Internal Medicine

, Volume 29, Supplement 3, pp 788–800 | Cite as

A Systematic Review of Approaches for Engaging Patients for Research on Rare Diseases

  • Laura P. Forsythe
  • Victoria Szydlowski
  • Mohammad Hassan Murad
  • Stanley Ip
  • Zhen Wang
  • Tarig A. Elraiyah
  • Rachael Fleurence
  • David H. Hickam
Review Article

ABSTRACT

BACKGROUND

Patients with rare diseases have limited access to useful information to guide treatment decisions. Engagement of patients and other stakeholders in clinical research may help to ensure that research efforts in rare diseases address relevant clinical questions and patient-centered health outcomes. Rare disease organizations may provide an effective means to facilitate patient engagement in research. However, the effectiveness of patient-engagement approaches, particularly for the study of rare diseases, has not been well studied.

OBJECTIVES

To synthesize evidence about engagement of patients and other stakeholders in research on rare diseases, including the role of rare disease organizations in facilitating patient-centered research.

METHODS/RESEARCH DESIGN

A systematic review and gray literature search were guided by a technical expert panel composed of patient representatives, clinicians, and researchers. English-language studies that engaged patients or other stakeholders in research on rare diseases or evaluated engagement were included. Studies were assessed on how well key research questions were answered, based on the level of detail describing engagement activities and whether outcomes from engagement were assessed.

RESULTS

Thirty-five studies were included, although many reported minimal information on engagement. Patients and other stakeholders were most commonly engaged to identify patient-centered research agendas, to select which study outcomes were important to patients, to provide input on study design, and to identify strategies for increasing enrollment in trials. Rare disease organizations mainly helped provide access to patients and communicated research opportunities and findings. They also helped promote collaborative networks and provided financial support for research infrastructures. Although authors reported benefits of engagement and identified changes to their research processes, no empirical assessments of engagement practices and their effectiveness were found.

CONCLUSIONS

Researchers studying rare diseases can obtain patient input regarding which research questions and health outcomes to study; however, the most effective approaches to engagement have not been well defined.

KEY WORDS

rare diseases engagement systematic review 

INTRODUCTION

Patients with rare diseases and their caregivers often have limited evidence-based information to guide decisions about management and symptom relief.1 Patients are increasingly interested in and calling for a more proactive role as partners in clinical research.2, 3, 4 Engaging patients and other relevant healthcare stakeholders (e.g., patients, caregivers such as parents of children with rare diseases, advocacy organizations, and clinicians)5 in the planning and conduct of research (as distinct from increased patient involvement in clinical care6) is a promising approach for addressing evidence gaps for the management of rare diseases. Engagement may promote research that evaluates health outcomes that are both relevant to patients with rare diseases7 and useful for decision making.8, 9, 10, 11, 12 Engagement is feasible for many populations and research phases,13 and can involve patients as consultants, as collaborators, or as leaders.13, 14 Further, given their interest in advancing research, rare disease organizations15 could help facilitate patient engagement in clinical research beyond efforts by the research community.

We undertook a systematic literature review to assess the extent and nature of involvement of patients with rare diseases, their caregivers, and relevant organizations in research initiatives. Five key questions guided the review:
  1. 1)

    For what purposes (goals and stages of the research process) have patients and other relevant health care stakeholders been engaged in research on rare diseases?

     
  2. 2)

    In what ways have they been engaged? How were patients and other stakeholders identified and recruited for engagement? What approaches were used to obtain their input?

     
  3. 3)

    How does engagement affect the design, conduct, dissemination, and relevance of the research to patients with rare diseases?

     
  4. 4)

    What is the role of patient organizations in bringing researchers and patients together?

     
  5. 5)

    What are the challenges for engagement in research on rare diseases?

     

METHODS

Stakeholder Involvement

The review was guided by a multi-stakeholder technical expert panel (TEP) composed of patient representatives, clinicians, and researchers recruited through the informal professional networks of the authors. The TEP provided input on the key research questions, the definition of rare disease, the search strategy, key gray literature references, interpretation of study findings, and recommendations for future research.

Electronic Database Search

Eligibility Criteria

We included original studies of any design published in English, in which patients, caregivers, or other stakeholders participated in planning or conducting biomedical or health services research related to rare diseases. Studies available only in abstract form were excluded due to insufficient information. Studies were considered to be related to rare diseases if the focus was a condition considered to be rare according to lists maintained by the National Institutes of Health16 or the National Organization of Rare Disorders,15 or if the authors stated that their research focused on a rare disease. Research engagement was defined by active involvement in the planning or conduct of research (e.g., providing input on a research agenda, contributing to study design, identifying study comparators or outcomes, monitoring study progress, interpreting results, or disseminating findings).17 Enrolling patients only as passive participants (subjects) or actively involving them in clinical care were not considered research engagement. Non-original or summarized literature was used to identify additional studies.

Search strategy

The overall search strategy was developed by a reference librarian and methodologists with expertise in conducting systematic reviews to capture both clinical studies on rare disease(s) that engaged patients or other stakeholders and studies that evaluated engagement. This review covers the origin of the biomedical data bases through September 2013, searching: PubMed/Ovid MEDLINE, Ovid EMBASE; Ovid PsycInfo, Ovid Cochrane (Sys Rev, Methods, HTA), EBSCO, CINAHL, SCOPUS (social sciences content), Web of Science (multidisciplinary scientific content), and Business Search Premier, Academic Search Premier and Google Scholar. To identify studies relevant to research engagement, medical subject heading (MeSH) terms and text words were selected based on common indexing practices. We included, among others, terms related to patients and other stakeholders (e.g., “patient*,” consumer*, stakeholder*, etc.), engagement (e.g., “participa*,” “collaborat*,”, “partner*,” etc.) (See online Appendix for full search strategy). These engagement research search terms were compiled and tested repeatedly to enhance search sensitivity and identify all potentially relevant publications. Due to the lack of indexed search terms for rare diseases, as well as the large number and heterogeneity of rare diseases, the relevant literature on research engagement was hand-screened by the authors for relevance. For studies published prior to 2011, we capitalized on a previous systematic review by three of the investigators (HM, ZW, TE) concerning research engagement broadly, although not specific to rare diseases,9 that utilized the same database search strategy with respect to engagement in research. The reference list for this original review was screened to identify studies related to rare diseases. We also screened reference lists from eligible studies, used the PubMed ‘related articles’ feature to identify eligible studies, reviewed conference proceedings, and used SciSearch for publications that cited potentially eligible studies.

Gray Literature Search

To identify relevant resources not included in bibliographic databases, we used search engines Scirus and Sciverse (which contain scientific journal content, scientists’ homepages, courseware, preprint server material, patents, and institutional repository and Website information), Google, and Bing, in addition to other websites recommended by the TEP.

Study Selection

We organized initial references in citation files using Endnote software and removed duplicates. Titles and abstracts were screened independently by two reviewers against eligibility criteria using the DistillerSR software (Evidence Partners Incorporated, Ontario, Canada). All studies judged relevant by either reviewer were included for full text review. We conducted full text screening of potentially eligible studies in duplicate using a similar procedure. If both reviewers judged a study to be relevant, it was included. Disagreements in full text screening were reconciled through arbitration by a third reviewer (MH).

Data Extraction

Data were extracted from studies with a standardized form developed using DistillerSR. Data were extracted by one investigator and reviewed for accuracy by a second. The data extraction form was tested on a sample of included studies (n = 5) to assure consistent data extraction. We extracted the following data: study description, stage(s) of research in which there was engagement, purpose of research engagement, methods for identifying and recruiting patients/stakeholders for research engagement and for obtaining their input, roles of rare disease organizations, perceived challenges to engagement, and any reported outcomes of patient engagement.

Analysis

The included studies were assessed by two reviewers (with disagreements arbitrated by a third reviewer) regarding the extent to which information answering the key research questions was provided. Studies were classified based on whether or not engagement activities were described with sufficient detail and clarity for others to replicate the approach (‘sufficiently descriptive’ vs. ‘minimally descriptive,’ respectively). Studies were also classified based on whether or not there was a direct method for measuring and examining the effect of the engagement processes on the design and conduct of the study (evaluative vs. non-evaluative studies). Findings were synthesized by key research question.

RESULTS

The systematic review and gray literature search identified 35 relevant studies (Figure 1). All studies were observational; none employed experimental or quasi-experimental designs. Eleven studies used qualitative or survey methods to obtain input from patients,4, 7, 18, 19, 20, 21, 22, 23, 24, 25, 26 and 17 studies were narrative reports describing one group’s experiences with engagement.27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43 The remaining seven studies were descriptions of specific initiatives reported on websites44, 45, 46, 47, 48, 49, 50 (Table 1 and Table 2). The studies were mostly conducted in North America and Europe, and most focused on a specific rare disease rather than rare diseases in general.7, 1821, 2331, 3538, 4042, 47, 48, 49, 50 Thirty studies reported on engagement conducted by the author(s),7, 1821, 2333, 3542, 45, 46, 47, 48, 49, 50 three described engagement conducted by others based on surveys or key informant interviews,4, 22, 43 and two broadly described contributions of patient organizations.34, 44 Nineteen reported on engagement of patients,7, 18, 20, 21, 23, 2527, 32, 33, 35, 38, 39, 43, 45, 47, 48, 49, 50 18 reported on engaging patient organizations,4, 19, 22, 2733, 35, 3739, 4143, 46 13 reported engaging parents or other caregivers,19, 23, 24, 26, 32, 35, 36, 3840, 43, 47, 50 and five reported engaging clinicians.20, 38, 39, 47, 49 Seven studies were classified as sufficiently descriptive7, 18, 19, 23, 24, 25, 26 (see Box 1 for example of a sufficiently descriptive study). No studies were classified as evaluative, because none formally assessed outcomes related to engagement.
Figure 1

Study screening and selection.

Text Box 1.

Illustrative Example of a Study that Was Sufficiently Descriptive with Respect to Research Engagement

Table 1

Characteristics of Included Studies

 

Included Studies (N = 35)

N

(%)

Study type

  

Narrative report

17

(49 %)

Website describing a specific initiative

7

(20 %)

Qualitative study

6

(17 %)

Survey study

5

(14 %)

Study location

  

United States (US)

15

(43 %)

Europe

14

(40 %)

Other

4

(11 %)

Australia

2

(6 %)

Rare disease focus

  

Specific condition*

25

(71 %)

General

10

(29 %)

Stakeholders engaged†

  

Patients

19

(54 %)

Rare disease organizations

17

(49 %)

Parents or family members

13

(37 %)

Clinicians

5

(14 %)

Description of engagement activities

  

Minimally descriptive

28

(80 %)

Sufficiently descriptive for others to replicate

7

(20 %)

Evaluation of the effects of engagement ††

  

Non-evaluative studies

35

(100 %)

Evaluative studies

0

(0 %)

*studies focused on: achalasia, neuromuscular disorders (n = 3), girls with abnormally tall stature, pulmonary arterial hypertension, cystic fibrosis, juvenile neuronal ceroid lipofuscinosis, phenylketonuria (PKU), cerebral palsy, vitiligo, teenagers and young adults with cancer, Paget disease, lupus, Li-Fraumeni Syndrome, pseudoxanthoma elasticum, fibromyalgia, tuberculosis, Parkinson’s disease, spinal muscular atrophy, pachyonychia, urea cycle disorders, degenerative ataxias, von Hippel-Lindau, and hemophilia

† Categories are not mutually exclusive; two studies2, 44 described the role of rare-disease organizations more broadly

†† Whether or not there was a direct method for measuring and examining the effect of the processes of engagement on the study design or impact of the study findings

Table 2

Detailed Characteristics of Individual Studies

Study

Study type

Rare disease

Engaged stakeholder(s)

Stage of research for which engagement is described

Purpose of engagement

Bedgood, 200727

Narrative report

Achalasia

Patients, disease-specific patient organization (ASG)

Execution Stage: Study recruitment

To access a larger patient population for developing and conducting a survey

Boon, 201028

Narrative report

Rare neuromuscular diseases

Disease-specific patient organization (Vereniging Spierziekten Nederland [VSN])

Preparatory Stage: Agenda setting; Execution Stage: Study design & procedures

To stimulate research into neuromuscular diseases

Bruinsma, 201129

Narrative report

Girls with abnormally tall stature

Disease-specific patient organization (Tall Girls Inc)

Preparatory Stage: Agenda setting; Execution Stage: Study design & procedures; Execution Stage: Study recruitment

To help determine project goals and provide input on study questions

Carroll, 201218

Qualitative study

Pulmonary arterial hypertension (PAH)

Patients

Execution Stage: Study design & procedures

To examine the motivations for and concerns regarding participation in randomized clinical trials of therapies in order to facilitate enrollment in future trials

Cystic Fibrosis Foundation, Therapeutics Development Network47

Website describing a specific initiative

Cystic fibrosis

Patients, families, clinicians

Execution Stage: Study design & procedures, Study recruitment

NR

de Blieck, 201330

Narrative report

Juvenile neuronal ceroid lipofuscinosis (JNCL).

Disease-specific patient organization (Batten Disease Support and Research Association [BDSRA])

Preparatory Stage: Research funding; Execution Stage: Study recruitment

To provide financial support and assist with recruitment (for a disease registry)

DeWard, 201331

Narrative report

Phenylketonuria (PKU)

Patient organizations

Execution Stage: Study recruitment

To improve recruitment and retention in clinical trials.

Edwards, 201119

Qualitative study

Cerebral palsy

Disease-specific organization, Parents

Preparatory Stage: Agenda setting; Execution Stage: Study design & procedures

To identify patient priorities and produce an RCT which was feasible, acceptable, based on parents’ experiences, and responsive to their needs and expectations

Eleftheriadou, 201220

Survey study

Vitiligo

Patients, clinicians

Preparatory Stage: Agenda setting

To identify disease core set of outcomes to be studied in trials

EMA (European Medicines Committees), Fifth report on the interaction with patients' and consumers' organizations (2011)33

Narrative report

General

Patients, patient organizations

Preparatory Stage: Research funding; Execution Stage: Study design & procedures; Translational Stage: Dissemination

To provide input on procedures concerning rare disease designations

EMA (European Medicines Committee), The role of patients as members of the EMA Human Scientific Committees32

Narrative report

General

Patients, parents/ Family/Friends, patient organizations

Preparatory Stage: Agenda setting; Translational Stage: Dissemination; Translational Stage: Evaluation

To advise and support regulators on the feasibility of planned investigations (e.g., for pediatric investigation, rare disease designation, risk management plan, etc.), and to guarantee that scientific opinions address patient needs and that there is a rational and adequate use of incentives (e.g., in rare disease designation) for the benefit of patients

EUPATI, EUPATI (European Patients’ Academy on Therapeutic Innovation)44

Website describing a specific initiative

General

NA

Execution Stage: Study recruitment; Translational Stage: Dissemination

NA (To address training issues and significantly improve the availability of both patient-centric information for the public as well as educated patient experts who have the capacity and capability to contribute to medical research & development)

The European Organisation for Rare Diseases (EURORDIS), Activity Report 2012 & Work plan 201334

Narrative report

General

NA

ERUORDIS encourages engagement at these stages: Preparatory Stage: Agenda setting; Preparatory Stage: Research funding, Execution Stage: Data collection; Translational Stage: Dissemination

NA

Fern, 201321

Qualitative study

Teenagers and young adults with cancer (including rare cancers)

Patients

Execution Stage: Study design & procedures; Execution Stage: Study recruitment; Execution Stage: Data collection; Translational Stage: Evaluation

To ensure that a future survey (developed through the current qualitative study) will be a true reflection of young people’s experience and that the methods of administration will be acceptable to young people in order to enhance study enrollment

Landy, 20124

Survey study

Rare genetic disorders

Non-specific patient organizations

Preparatory Stage: Research funding; agenda setting; Execution stage: study design & procedures; recruitment, data collection; Translational stage: dissemination

NR

Langston, 200535

Narrative report

Paget Disease

Disease-specific organization, Patients, caregivers

Preparatory stage: agenda setting; Execution stage: study design & procedures; Execution stage: study recruitment; Translational stage: dissemination*

To aid the design, conduct, and delivery of a clinical trial

The Lupus Research Institute, A patient’s voice in lupus research48

Website describing a specific initiative

Lupus

Patients

Preparatory Stage: Research funding

To bring the unique patient perspective to the forefront of proposal review

Mai, 201236

Narrative report

Li-Fraumeni Syndrome

Parents/Family

Preparatory Stage: Agenda setting

To bring together researchers, clinicians, and individuals from patient families to review the state of science, identify research priorities, and establish a multi-center research collaboration

Marshall, 200437

Narrative report

Pseudoxanthoma elasticum (PXE)

Disease-specific patient organization (PXE International)

Execution stage: data collection; Translation stage: dissemination

NR

Mavris, 201222

Cross-sectional survey

General

Non-disease-specific patient organization (EURORDIS), other patient organizations

Preparatory Stage: Research funding; Execution stage: study design & procedures; recruitment; data collection

NA (describes findings of a survey intended to document the role of patient organizations in rare disease research)

McCormack, 201338

Narrative report

Neuromuscular disorders

Patients, parents, patient organizations, clinicians, legal academics, sociologists, and ethicists.

Execution stage: study design & procedures

To provide a “strategic steer” on ethical issues related to the study of neuromuscular disorders

Mease, 20077

Survey study

Fibromyalgia

Patients

Preparatory Stage: Agenda Setting

To develop consensus regarding the outcomes to be assessed in clinical trials and longitudinal studies

Molster, 201239

Narrative report

General

Patients, parents, patient support groups, clinicians, funders, industry, policy makers, regulators

Preparatory Stage: Agenda setting

To get patients' input relevant to the development of a national plan for rare diseases, including the national research agenda

Nierse, 201023

Qualitative study

Non-curable neuromuscular diseases

Patients, parents

Preparatory Stage: Agenda setting

To identify patient priorities for research on neuromuscular disorders and achieve a shared research agenda among patients, clinicians and researchers

Ntshanga, 201040

Narrative report

Tuberculosis

Community members, community leaders, religious leaders, traditional healers, school principals, community health workers

Preparatory Stage: Agenda setting; Translational Stage: Dissemination

To conduct research that matches local needs and circumstances, and to enhance community participation through open communication with researchers

Orphanet, Researchers' wish-list for rare disease research in Europe45

Website describing a specific initiative

General

Patients

Preparatory Stage: Agenda setting

NR

The Parkinson Pipeline Project49

Website describing a specific initiative

Parkinson’s Disease

Patients, clinicians treating Parkinson’s Disease patients

Preparatory Stage: Agenda setting; Translational Stage: Dissemination; Translational Stage: Evaluation

To provide the patient perspective in the treatment development process

Rare disease matters, Patient-initiated research foundations46

Website describing specific initiatives

General

Patient organizations

Preparatory Stage: Research funding

NR

Rothwell, 201324

Qualitative study

Spinal muscular atrophy

Parents

Execution Stage: Study design & procedures

To assess parents' attitudes and opinions toward a pilot newborn screen study for spinal muscular atrophy

Schwartz, 201341

Narrative report

Pachyonychia

Disease-specific patient organization (PC Project)

Preparatory Stage: Agenda setting; Preparatory Stage: Research funding; Execution Stage: Study recruitment; Translational Stage: Dissemination

To help connect researchers with patients to increase enrollment in studies

Seminara, 201042

Narrative report

Urea Cycle Disorders

Non-disease-specific patient organization (Coalition of Patient Advocacy Groups), Disease-specific group (National Urea Cycle Disorders Foundation)

Preparatory Stage: Agenda setting; Execution Stage: Study design & procedures; Execution Stage: Study recruitment; Execution Stage: data collection; Translational Stage: Dissemination

NR

Serrano-Aguilar, 200925

Survey study

Degenerative ataxias; majority (67.9 %) suffered Friedreich’s ataxia

Patients

Preparatory Stage: Agenda setting

To incorporate needs and expectations of patients in the design of this systematic review

von Hippel-Lindau Alliance (VHLA)50

Website describing specific initiatives

von Hippel-Lindau

Patients, families

Preparatory Stage: Agenda Setting; Execution Stage: Study design & procedures, recruitment, data collection

To create a patient-driven registry that is unlike databases in which limited information is entered by clinicians; it contains a complete “picture” of each patient with insights and observations only a patient can give

Workman, Engaging patients in information sharing and data collection: the role of patient-powered registries and research networks prepared for AHRQ43

Narrative Report

General

Patients, families, and patient organizations

Preparatory Stage: Agenda setting; Execution Stage: Study design & procedures; Execution Stage: Study recruitment; Execution Stage: Data collection; Translational Stage: Dissemination

NA

Wu, 201226

Qualitative study

Hemophilia

Patients, Parents

Execution Stage: Study design & procedures

To explore the need for potential new items of the Canadian Hemophilia Outcomes-Kids Life Assessment Tool (CHO-KLAT)

NR = Not reported, such that information was relevant to the included study, but was not described in the publication/website; NA = Not applicable, such that the information was not relevant for the specific type of study included. *Engagement in dissemination is planned, but the project has not reached this stage as of publication

  1. Key Question 1:

    For what purposes (goals, stages of the research process) have patients and other stakeholders been engaged in research on rare diseases?

     
The purposes of engagement included identifying patient-centered research topics or agendas,19, 23, 29, 36, 39 identifying outcomes important to patients or developing measures relevant to patients’ needs,7, 19, 20, 23, 25, 26 increasing recruitment or enrollment through development of patient-centered study designs,4, 18, 21, 24, 27, 30, 31, 41 and incorporating the patient perspective into study design19, 25, 35, 39, 40 (Table 2, Table 3)
Table 3

Engagement Summary: Stage, Purpose, Method, and Role of Rare Disease Organizations

 

Studies Reporting (Total = 35)*

Stage of research in which engagement occurred

Preparatory

22

Execution

19

Translational

10

Purpose of engagement

Identifying patient-centered research topics or agendas

5

Selecting outcomes and measures

6

Increasing recruitment, enrollment, or retention

7

Incorporating patient perspective into study design

5

How patients and other stakeholders were engaged

Providing input through workshops, focus groups, or Delphi

11

methods

Developing study interview guides, study materials, websites

4

Including patients on governing or advisory committees

5

Reviewing research findings

2

Sharing study findings

6

Role of rare disease organizations

  

Providing access to or recruiting patients for engagement and as study subjects

9

Communicating research opportunities or findings

5

Providing input on study design or procedures

6

Initiating clinical trials or other studies of rare diseases

4

Funding of infrastructure (e.g., registries) or training grants

3

Initiating collaborative research networks

3

Providing training opportunities

2

* Categories are not mutually exclusive; no studies were classified as “evaluative”

. Patients and other stakeholders were most commonly engaged in the preparatory stage (n = 19 for agenda setting)4, 7, 19, 20, 23, 25, 28, 29, 32, 35, 36, 3943, 45, 49, 50 and study execution (n = 15 for study design and procedures,4, 18, 19, 21, 24, 26, 28, 29, 33, 35, 38, 42, 47, 50, 51 n = 12 for recruitment,4, 21, 27, 2931, 35, 4143, 47, 50 and n = 6 for data collection4, 21, 37, 42, 43, 50). Eleven studies reported engagement for research translation, with a greater focus on dissemination (n = 10)4, 32, 33, 35, 37, 4043, 49 than on evaluation of findings (n = 3)21, 32, 49 or implementation (n = 0). Nineteen studies4, 19, 21, 22, 2830, 3235, 37, 4044, 49, 50 described engagement at multiple stages, particularly those reporting on the role of patient organizations.33, 35, 41, 42, 43
  1. Key Question 2:

    In what ways have patients and other stakeholders been engaged in research on rare diseases?

     

Identifying Patients and Other Stakeholders for Engagement in Research

Eight studies reported information on how patients and other stakeholders were identified for engagement18, 19, 2325, 27, 35, 48 (Table 4 [online]). All of these reported convenience sampling without an explicit selection process. Recruitment occurred through patient organizations,19, 23, 27, 35 clinics,18 self-referral on the internet,48 or a recruiting agency.24 No studies engaging patient organizations described how collaborations between research teams and these organizations were initiated.

Approaches for Patient and Other Stakeholder Input

Eleven studies referred to engagement activities based on consultation with patients and stakeholders. Obtaining input through workshops, focus groups, interviews, or Delphi methods was common.7, 1820, 2326, 36, 39, 45 One study obtained patient feedback on informational materials that would be presented to study subjects.19 Thirteen studies described more collaborative involvement in at least one stage of the research process; for example, including patients on governing or advisory committees,32, 33, 38, 42, 48 in developing study interview guides, study materials and websites;21, 35, 52 in collecting data (conducting interviews),4, 21 in reviewing research findings,21, 28 and in disseminating findings.4, 33, 37, 40, 41, 42 Stakeholder-driven approaches were reported; included here are examples of patients or advocacy organizations initiating new studies4, 29, 37, 41, 42, 46 and controlling the research process by setting the research agenda, having responsibility for data collection, and disseminating findings (e.g., Patient-Powered Registries43). Conversely, in four studies, patient organizations were engaged only for providing access to their members.27, 30, 31, 46 Four studies reported that patients or other stakeholders were prepared for engagement in research, ranging from an informational video24 to in-person workshops;21, 40, 48 none reported training researchers in engagement methods.
  1. Key Question 3:

    How does engagement affect the design, conduct, dissemination, and relevance of the research?

     

Four studies clearly articulated how they used input from patient engagement to inform their work, including identifying research topics for future study,23 the study design to use,19 the outcomes to assess,7, 19 and the domains to include on a patient-reported outcome measure26 (Table 3, Table 5 [online]). Five studies used either qualitative or survey methods to obtain information directly from patients about the outcomes that were important to them.7, 19, 23, 25, 26 These reports permitted rankings of symptoms by importance7, 25 and also clarified how patients describe symptoms.7, 19, 23 For example, due to patient input, these studies included measures of pain, stiffness, sleep, motor function, quality of life, and the economic impact of disease.

No studies empirically evaluated the effect of engagement on the design, conduct, dissemination, or relevance of the research. However, many studies reported perceived impact of engagement that was not measured and could not be confirmed. Some perceived benefits relative to other studies that did not use engagement were practical, such as improved access to patients or improved recruitment methods,4, 19, 29, 30, 35, 42, 43 improved study retention,19 more in-depth responses in interviews (when using patients to conduct the interviews),21 less time to complete the project,4 and incorporation of a broader range of outcome measures to inform future risk-benefit assessments for investigational treatments.49 Other perceived benefits were less tangible, such as increased relevance of the project,4, 29, 35 broader dissemination of findings,35 improved acceptance of research findings among end-users,29 greater mutual respect between scientists and patients,32 and increased transparency of and public confidence in the research.32, 33 Only one study articulated the benefits of engagement for the engaged partners, including increased awareness of the disease studied and greater access to health information.35
  1. Key Question 4:

    What is the role of rare disease organizations in bringing researchers and patients together?

     

Nineteen studies reported on engagement of patient organizations.19, 22, 2733, 35, 3739, 4143, 46 However, no studies formally evaluated the role of patient organizations in connecting researchers and patients. Studies frequently acknowledged the role of rare disease organizations in providing access to or recruiting patients both for engagement and as study subjects,19, 22, 2931, 35, 41 and in enhancing communication about research participation opportunities and research findings (e.g., through social media, websites, conferences, and newsletters).31, 35, 36, 41 In some cases, representatives from these organizations were directly involved in the research process, serving on advisory and regulatory committees28, 42 or providing input on study protocols,19, 22, 35 for example. Some studies reported that patient organizations initiated clinical trials or other studies of rare diseases.28, 29, 41

Several studies also highlighted broader support provided by rare disease organizations. For example, these groups initiated the organization of collaborative research networks,22, 28, 41 provided financial support for research infrastructure such as registries,22, 30, 43, 46 supported training activities,22, 34, 44 and developed policy statements regarding engaging patients in research.34
  1. Key Question 5:

    What are the challenges for engagement in research on rare diseases?

     

The studies included in this review provide insights into the process of research engagement. First, four of the studies stated that engagement requires substantial time and resources from both researchers and those who are engaged as partners.26, 29, 32, 35 One author notes that scientific groups face challenges in finding time to obtain and use patient input,32 and another author notes that patient organizations have limited resources to devote to research engagement versus other priority activities.35 Second, two studies stated that engaged partners must overcome barriers to engagement such as logistical issues (e.g., access to clinics) and health problems that may prevent participation.18, 31 Third, several authors expressed concerns about possible bias among patients and patient representatives chosen for engagement due to higher motivation, high educational levels, methods of recruitment (e.g., via internet or annual meetings) or possible conflicts of interest (e.g., relationships between patient organizations and industry groups).23, 25, 27, 30, 33 , 43 Other challenges identified include lack of researcher and patient experience in or training for engagement,25 , 32 an absence of policies facilitating engagement,25 the potential for both researchers and patient partners to undervalue engagement,25 ethical concerns (e.g., patient rights43), and the possibility of uncoordinated efforts by competing patient-led groups.43

DISCUSSION

Given the diversity of conditions and limited research resources associated with rare diseases, engagement of patients and other stakeholders in research has the potential to facilitate the study of relevant clinical questions and patient-centered health outcomes for rare disease populations. In fact, these issues were among the most commonly reported reasons for engaging patients or their caregivers. In contrast, the studies generally did not address the role of engagement for dissemination of research findings, even though engagement of patients and patient organizations in sharing research results may increase the reach of findings, which is particularly important in the context of treating rare diseases.

Most studies reported a consultative approach to engagement (i.e., unidirectional communication from patients to researchers with unknown influence on the research). We expect that collaborative approaches (i.e., bidirectional communication, shared responsibility for decision-making) may be more effective for advancing the study of rare disease, given the diversity of patient experiences. Although other researchers have reported similar expectations based on their experiences with research engagement in other settings (not specific to rare diseases), definitive evidence to support this is not yet available.9 , 53 Very few studies reported training researchers, patients, or other stakeholders for engagement. Other literature supports training as increasing the impact of engagement; while some training opportunities are available for patients43 and researchers,54 these are not specific to research on rare diseases.

This systematic review suggests that rare disease organizations play multiple roles in facilitating engagement of patients and other stakeholders in research. Nearly half of the identified studies involved patient organizations; these organizations met critical needs, including facilitating access to patients who may be too geographically dispersed or difficult to identify for participation in clinical trials. Patient organizations also provided broad support for patient-centered research in rare diseases, including wide communication about research opportunities and findings, financial support, formation of collaborative research networks, and creation of policies regarding engagement.

Many studies reported the effects of engagement, but none measured these effects in ways that permit estimation of the strength of the impact of the engagement efforts. The purported benefits included both practical effects (e.g., incorporation of a broader range of outcome measures, increased study enrollment) and intangible effects (e.g., increased relevance of the research, greater public trust). However, consistent with gaps in the broader literature on engagement overall,51 no studies evaluated the effect of engagement on the inclusion of patient-centered outcomes, the impact of the research findings, or their relevance for clinical decision-making. This lack of empirical evidence precludes drawing conclusions about the impact of engagement in research on rare diseases. This gap in the literature can be addressed in future studies that directly measure the impact of engagement on health outcomes.

This review identified additional evidence gaps, including the lack of standard reporting guidelines for information about engagement of patients and other stakeholders.13 In addition, very little is known about the characteristics of those engaged, the approaches for collaboration with these individuals or groups, or the impact of these characteristics on the research process or outcomes. Connecting researchers with engaged partners is a challenge. Although researchers can learn from successful engagement approaches for the study of more common conditions (e.g., HIV),55, 56 more information about how to identify and recruit patient and stakeholder partners would allow for both shared learning and greater transparency regarding possible biases.

Several initiatives may help to fill the evidence gaps. First, the Patient-Centered Outcomes Research Institute (PCORI)57 funds studies that address methods for improving engagement, for evaluating the impact of engagement on research outcomes, and for assuring that study questions and outcomes are meaningful to patients.58 Evidence regarding best practices for research engagement will also be generated by the Agency for Healthcare Research and Quality’s Community Forum project.59 Two initiatives will support infrastructure for patient-centered approaches to rare disease research, including PCORI’s National Patient-Centered Clinical Research Network60 and the National Institutes of Health’s Global Rare Diseases Patient Registry Data Repository.9 PCORI’s Advisory Panel on Rare Diseases may also provide guidance regarding promising approaches for fostering collaborative approaches to research and involving relevant patient organizations.61.

This review was limited by the small number of peer-reviewed studies that reported on research engagement in the study of rare diseases and the lack of detail about engagement. The findings from this review could be supplemented by interviews and/or surveys with a range of rare disease organizations to ensure comprehensive capture of research engagement activities. Our assessment of the effects of engagement relied heavily on the perceptions of the researchers who conducted the included studies, which have not been empirically validated. Moreover, this review could not fully capture the wide variation in research resources and infrastructure or the engagement capacity of patient groups across the spectrum of rare diseases. The studies included in this review provide limited insight into whether the successful approaches are generalizable to other rare disease organizations. Future research should examine differences in approaches to, and the impacts of, engagement for diverse types of stakeholders (e.g., children with rare diseases compared to their parents) and assess the authenticity (i.e., the extent to which patient and stakeholder input was actually incorporated into decision making) of approaches to engagement, as the impacts of engagement may depend on the quality and genuineness (i.e., the sincerity) of engagement interactions.

This systematic review found that patient engagement for guiding research on rare diseases tended to use a limited number of multi-purpose approaches. The published studies provide examples of approaches that could be adapted and further evaluated by other organizations, and also highlight the particularly important role of patient organizations in fostering engagement in research on rare diseases. While no insurmountable barriers were identified in this review, there were meaningful challenges. Building an evidence base that documents how different approaches to patient engagement facilitate or hinder research will inform guidance on how best to make clinical research on rare diseases more useful.

Notes

Acknowledgements

We gratefully acknowledge the contributions of our Technical Expert Panel: Ned E. Calonge, MD, MPH (The Colorado Trust, President and Chief Executive Officer), Lisa Moss (Tuberous Sclerosis Alliance, Director of Donor Relations), David E. Sandberg, PhD (University of Michigan, Professor and Director, Division of Child Behavioral Health, Department of Pediatrics), Russell Teagarden, DMH (National Organization for Rare Diseases, Senior Vice President for Medical and Scientific Affairs), and Suzanne Schrandt, JD (Deputy Director of Patient Engagement, PCORI).

This review was supported by the Patient-Centered Outcomes Research Institute (PCORI).

This paper has not been previously presented.

Conflicts of Interest

The authors have no conflicts of interest to report.

Disclaimer

The views expressed in this paper are those of the authors and to do not necessarily reflect the official views of the Patient-Centered Outcomes Research Institute (PCORI).

Supplementary material

11606_2014_2895_MOESM1_ESM.docx (103 kb)
ESM 1 (DOCX 103 kb)

REFERENCES

  1. 1.
    Andersen T. The political empowerment of rare disease patient advocates both at EU and national level. Orphanet Journal Of Rare Diseases. 2012;7(2):A33.PubMedCentralCrossRefGoogle Scholar
  2. 2.
    European Organization for Rare Diseases (EURORDIS). Patients’ priorities and needs for rare disease research 2014-2020. Available at: http://www.eurordis.org/sites/default/files/publications/what_how%20_are_disease_research_0.pdf Accessed April 30, 2014.
  3. 3.
    Forsythe LP, Frank L, Walker KO, Hayes D, Levine S, Anise A, et al., editors. Patient and Clinician Views on Comparative Effectiveness Research and Engagement in Research: A Panel Discussion on PCORI Survey Results. AcademyHealth Annual Research Meeting; 2013 June 2013; Baltimore, MD.Google Scholar
  4. 4.
    Landy DC, Brinich MA, Colten ME, Horn EJ, Terry SF, Sharp RR. How disease advocacy organizations participate in clinical research: a survey of genetic organizations. Genet Med. 2012;14(2):223–228. doi: 10.1038/gim.0b013e3182310ba0.PubMedCrossRefGoogle Scholar
  5. 5.
    Concannon TW, Meissner P, Grunbaum JA, McElwee N, Guise JM, Santa J, et al. A new taxonomy for stakeholder engagement in patient-centered outcomes research. J Gen Intern Med. 2012;27(8):985–991.PubMedCentralPubMedCrossRefGoogle Scholar
  6. 6.
    Carman KL, Dardess P, Maurer M, Sofaer S, Adams K, Bechtel C, et al. Patient and family engagement: a framework for understanding the elements and developing interventions and policies. Health Aff (Millwood). 2013;32(2):223–231. doi: 10.1377/hlthaff.2012.1133.CrossRefGoogle Scholar
  7. 7.
    Mease P, Arnold LM, Bennett R, Boonen A, Buskila D, Carville S, et al. Fibromyalgia syndrome. J Rheumatol. 2007;34(6):1415–1425.PubMedGoogle Scholar
  8. 8.
    Caron-Flinterman JF, Broerse JEW, Bunders JFG. The experiential knowledge of patients: a new resource for biomedical research? Soc Sci Med. 2005;60(11):2575–2584.PubMedCrossRefGoogle Scholar
  9. 9.
    Domecq JP, Prutsky G, Elraiyah T, Wang Z, Nabhan M, Shippee N, et al. Patient engagement in research: a systematic review. BMC Health Serv Res. 2014;14(1):89. doi: 10.1186/1472-6963-14-89.PubMedCentralPubMedCrossRefGoogle Scholar
  10. 10.
    Gandhi GY, Murad MH, Fujiyoshi A, Mullan RJ, Flynn DN, Elamin MB, et al. Patient-important outcomes in registered diabetes trials. JAMA. 2008;299(21):2543–2549.PubMedCrossRefGoogle Scholar
  11. 11.
    Nass P, Levine S, Yancy C. Methods for involving patients in topic generation for patient-centered comparative effectiveness research: An international perspective; 2012. Available at: http://www.pcori.org/assets/Methods-for-Involving-Patients-in-Topic-Generation-for-Patient-Centered-Comparative-Effectiveness-Research-%E2%80%93-An-International-Perspective.pdf.; Accessed April 30, 2014.Google Scholar
  12. 12.
    Staley K. Exploring Impact: Public involvement in NHS, public health and social care research; 2009.Google Scholar
  13. 13.
    Shippee ND, Domecq GJP, Prutsky LGJ, Wang Z, Elraiyah TA, Nabhan M, et al. Patient and service user engagement in research: a systematic review and synthesized framework. Health Expectations. 2013. doi: 10.1111/hex.12090.PubMedGoogle Scholar
  14. 14.
    Hanley B BJ, Gorin S, Barnes M, Goodare H, Kelson M, Kent A, Oliver S, Thomas S, Wallcraft J. Involving the public in NHS, public health and social care research: briefing notes for researchers. 2nd ed. . 2004. Available at: http://www.twocanassociates.co.uk/perch/resources/files/Briefing%20Note%20Final_dat(2).pdf. Accessed April 30, 2014.
  15. 15.
    National Organization for Rare Disorders (NORD). Rare Disease Information. 2013. Available at: http://www.rarediseases.org/rare-disease-information. Accessed April 30, 2014.
  16. 16.
    National Institutes of Health(NIH). Genetic and Rare Disease Information Center. Available at: http://rarediseases.info.nih.gov/gard/browse-by-first-letter/A. Accessed April 30, 2014.
  17. 17.
    Patient-Centered Outcomes Research Institute (PCORI). The PCORI Methodlogy Report. 2013. Avialable at: http://www.pcori.org/assets/2013/11/PCORI-Methodology-Report.pdf. Accessed April 30, 2014.
  18. 18.
    Carroll R, Antigua J, Taichman D, Palevsky H, Forfia P, Kawut S, et al. Motivations of patients with pulmonary arterial hypertension to participate in randomized clinical trials. Clinical trials (London, England) 2012;9(3):348-57. doi: 10.1177/1740774512438981
  19. 19.
    Edwards V, Wyatt K, Logan S, Britten N. Consulting parents about the design of a randomized controlled trial of osteopathy for children with cerebral palsy. Health Expectations. 2011;14(4):429–438. doi: 10.1111/j.1369-7625.2010.00652.x.PubMedCrossRefGoogle Scholar
  20. 20.
    Eleftheriadou V, Thomas KS, Whitton ME, Batchelor JM, Ravenscroft JC. Which outcomes should we measure in vitiligo? Results of a systematic review and a survey among patients and clinicians on outcomes in vitiligo trials. Br J Dermatol. 2012;167(4):804–814. doi: 10.1111/j.13652133.2012.11056.x
  21. 21.
    Fern LA, Taylor RM, Whelan J, Pearce S, Grew T, Brooman K, et al. The art of age-appropriate aare: reflecting on a conceptual model of the cancer experience for teenagers and young adults. Cancer Nurs. 2013;36(5):E27-e38. doi: 10.1097/NCC.0b013e318288d3ce
  22. 22.
    Mavris M, Le Cam Y. Involvement of patient organisations in research and development of orphan drugs for rare diseases in Europe. Mol Syndromol. 2012;3(5):237–243.PubMedCentralPubMedGoogle Scholar
  23. 23.
    Nierse CJ, Abma TA, Horemans AM, van Engelen BG. Research priorities of patients with neuromuscular disease. Disabil Rehabil. 2013;35(5):405-12. doi: 10.3109/09638288.2012.694964
  24. 24.
    Rothwell E, Anderson RA, Swoboda KJ, Stark L, Botkin JR. Public attitudes regarding a pilot study of newborn screening for spinal muscular atrophy. Am J Med Genet A. 2013;161A(4):679–686.PubMedCrossRefGoogle Scholar
  25. 25.
    Serrano-Aguilar P, Trujillo-Martin MM, Ramos-Goni JM, Mahtani-Chugani V, Perestelo-Perez L, Posada-de la Paz M. Patient involvement in health research: a contribution to a systematic review on the effectiveness of treatments for degenerative ataxias. Soc Sci Med. 2009;69(6):920-5. doi: 10.1016/j.socscimed.2009.07.005
  26. 26.
    Wu R, Zhang J, Luke KH, Wu X, Burke T, Tang L, et al. Cross-cultural adaptation of the CHO-KLAT for boys with hemophilia in rural and urban China. Health Qual Life Outcomes. 2012;10:112. doi: 10.1186/1477-7525-10-112
  27. 27.
    Bedgood R, Sadurski R, Schade RR. The use of the internet in data assimilation in rare diseases. Dig Dis Sci. 2007;52(2):307–312.PubMedCrossRefGoogle Scholar
  28. 28.
    Boon W, Broekgaarden R. The role of patient advocacy organisations in neuromuscular disease R&D--The case of the Dutch neuromuscular disease association VSN. Neuromuscul Disord. 2010;20(2):148-51. doi: 10.1016/j.nmd.2009.10.012
  29. 29.
    Bruinsma FJ, Rayner JA, Venn AJ, Pyett P, Werther G. Looking back in time: conducting a cohort study of the long-term effects of treatment of adolescent tall girls with synthetic hormones. BMC Public Health. 2011;11(5):S7.PubMedCentralPubMedCrossRefGoogle Scholar
  30. 30.
    de Blieck EA, Augustine EF, Marshall FJ, Adams H, Cialone J, Dure L, et al. Methodology of clinical research in rare diseases: development of a research program in juvenile neuronal ceroid lipofuscinosis (JNCL) via creation of a patient registry and collaboration with patient advocates. Contemp Clin Trials. 2013;35(2):48-54. doi: 10.1016/j.cct.2013.04.004
  31. 31.
    DeWard SJ, Wilson A, Bausell H, Volz AS, Mooney K. Practical aspects of recruitment and retention in clinical trials of rare genetic diseases: the phenylketonuria (PKU) experience. J Genet Couns. 2014;23(1):20-8. doi: 10.1007/s10897-013-9642-y
  32. 32.
    European Medicine Agency (EMA). The role of patients as members of the EMA Human Scientific Committees. Available at: http://www.ema.europa.eu/docs/en_GB/document_library/Other/2011/12/WC500119614.pdf. Accessed April 30, 2014.
  33. 33.
    European Medicines Agency (EMA). Fifth report on the interaction with patients' and consumers' organizations 2011. Available at: http://www.ema.europa.eu/docs/en_GB/document_library/Report/2012/10/WC500133475.pdf. Accessed April 30, 2014.
  34. 34.
    European organisation for rare diseases (EURORDIS). Activity Report 2012 & Workplan 2013. Available at: http://www.eurordis.org/sites/default/files/activity-report-2012.pdf. Accessed April 30, 2014.
  35. 35.
    Langston AL, McCallum M, Campbell MK, Robertson C, Ralston SH. An integrated approach to consumer representation and involvement in a multicentre randomized controlled trial. Clinical Trials. 2005;2(1):80–87.PubMedCrossRefGoogle Scholar
  36. 36.
    Mai PL, Malkin D, Garber JE, Schiffman JD, Weitzel JN, Strong LC, et al. Li-Fraumeni syndrome: Report of a clinical research workshop and creation of a research consortium. Cancer Genet. 2012;205(10):479–487.PubMedCentralPubMedCrossRefGoogle Scholar
  37. 37.
    Marshall E. Genetics. Patient advocate named co-inventor on patent for the PXE disease gene. Science. 2004;305(5688):1226. doi: 10.1126/science.305.5688.1226a.PubMedCrossRefGoogle Scholar
  38. 38.
    McCormack P, Woods S, Aartsma-Rus A, Hagger L, Herczegfalvi A, Heslop E, et al. Guidance in social and ethical issues related to clinical, diagnostic care and novel therapies for hereditary neuromuscular rare diseases: "TRANSLATING" the translational. 2013(JAN):1-17.Google Scholar
  39. 39.
    Molster C, Youngs L, Hammond E, Dawkins H. Key outcomes from stakeholder workshops at a symposium to inform the development of an Australian national plan for rare diseases. 2012;7:50. doi: 10.1186/1750-1172-7-50.
  40. 40.
    Ntshanga SP, Ngcobo PS, Mabaso ML. Establishment of a Community Advisory Board (CAB) for tuberculosis control and research in the Inanda, Ntuzuma and KwaMashu (INK) area of KwaZulu-Natal, South Africa. Health Policy. 2010;95(2–3):211–215. doi: 10.1016/j.healthpol.2009.12.004.PubMedCrossRefGoogle Scholar
  41. 41.
    Schwartz ME, Zimmerman GM, Smith F, Sprecher E. Pachyonychia congenita project: A partnership of patient and medical professional. 2013;5(1):42-7. doi: 10.1097/JDN.0b013e31827d9ed5.
  42. 42.
    Seminara J, Tuchman M, Krivitzky L, Krischer K, Lee HS, Lemons C, et al. Establishing a consortium for the study of rare diseases: The Urea Cycle Disorders Consortium. Mol Genet Metab. 2010;100(1):S97–S105.PubMedCentralPubMedCrossRefGoogle Scholar
  43. 43.
    Workman TA. Engaging patients in information sharing and data collection: The role of patient-powered registries and research networks prepared for AHRQ. Available at: http://www.effectivehealthcare.ahrq.gov/ehc/assets/File/Patient-Powered-Registries-white-paper-130911.pdf Accessed April 30, 2014.
  44. 44.
    European Patients’ Academy on Therapeutic Innovation (EUPATI). EUPATI (European Patients’ Academy on Therapeutic Innovation). Available at: http://www.patientsacademy.eu/. Accessed April 30, 2014.
  45. 45.
    Orphanet. Researchers' wish-list for rare disease research in Europe. Available at: http://www.orpha.net/actor/EuropaNews/2005/050715.html. Accessed April 30, 2014.
  46. 46.
  47. 47.
    The Cystic Fibrosis Foundation. Theraputic Developments Network. Available at: http://www.cff.org/research/TDN/. Accessed April 30, 2014.
  48. 48.
    The Lupus Research Institute. A patient’s voice in lupus research. Available at: http://www.lupusresearchinstitute.org/news/discoveries/09_advocacy. Accessed April 30 2014.
  49. 49.
    The Parkinson Pipeline Project. The Parkinson Pipeline Project. Available at: http://www.pdpipeline.org/. Accessed April 30, 2014.
  50. 50.
    von Hippel-Lindau Alliance. von Hippel-Lindau Alliance,. Available at: http://www.vhl.org/wordpress/ Accessed April 30, 2014.
  51. 51.
    Workman T, Maurer M, Carman K. Unresolved tensions in consumer engagement in CER: a US research perspective. Journal of Comparative Effectiveness Research. 2013;2:127.PubMedCrossRefGoogle Scholar
  52. 52.
    Schaefer E, Astigarraga I, Haupt R, Minkov M, Price R, Donadieu J. Histio Net-A reference network for the creation of an international web portal for Langerhans cell histiocytosis and associated syndromes. Pediatric Blood and Cancer. 2011;56 (4):693.Google Scholar
  53. 53.
    Mullins CD, Abdulhalim AM, Lavallee DC. Continuous patient engagement in comparative effectiveness research. JAMA. 2012;307(15):1587–1588. doi: 10.1001/jama.2012.442.PubMedCrossRefGoogle Scholar
  54. 54.
    Colorado Clinical and Translational Sciences Institute (CCTSI). Colorado Clinical and Translational Sciences Institute (CCTSI). Available at: http://cctsi.ucdenver.edu/Pages/index.aspx. Accessed April 30, 2014.
  55. 55.
    Recommendations for Community Involvement in National Institute of Allergy and Infectious Diseases HIV/AIDS Clinical Trials Research 2009. Available at: http://www.hvtn.org/community/CAB_Recommendations_Certified.pdf. Accessed April 30, 2014
  56. 56.
    Faster Cures. Back to Basics: HIV/AIDS Advocacy as a Model for Catalyzing Change. Available at: http://www.fastercures.org/assets/Uploads/PDF/Back2BasicsFinal.pdf. Accessed April 30, 2014.
  57. 57.
    Patient-Centered Outcomes Research Institute (PCORI). PCORI Mission and Vision. Available at: http://www.pcori.org/about-us/mission-and-vision/. Accessed April 30, 2014.
  58. 58.
    Patient-Centered Outcomes Research Institute (PCORI). PCORI Funding Center. Available at: http://www.pcori.org/funding-opportunities/funding-center/. Accessed April 30, 2014.
  59. 59.
    Agency for Healthcare Research and Quality (AHRQ). Effective Healthcare Program Helping You Make Better Choices. Available at: http://effectivehealthcare.ahrq.gov/index.cfm/who-is-involved-in-the-effective-health-care-program1/ahrq-community-forum/ Accessed April 30, 2014.
  60. 60.
    Patient-Centered Outcomes Research Institute (PCORI). PCORnet: The National Patient-Centered Clinical Research Network. 2013. Available at: http://www.pcori.org/funding-opportunities/pcornet-national-patient-centered-clinical-research-network/ Accessed April 30, 2014.
  61. 61.
    Patient-Centered Outcomes Research Institute (PCORI). PCORI Proposed Charter for the Advisory Panel on Rare Diseases. Available at: http://www.pcori.org/assets/2013/11/PCORI-Board-Meeting-Proposed-Charter-Advisory-Panel-on-Rare-Disease-111813.pdf. Accessed April 30, 2014.

Copyright information

© Society of General Internal Medicine 2014

Authors and Affiliations

  • Laura P. Forsythe
    • 1
  • Victoria Szydlowski
    • 1
  • Mohammad Hassan Murad
    • 2
    • 3
  • Stanley Ip
    • 1
  • Zhen Wang
    • 2
  • Tarig A. Elraiyah
    • 2
  • Rachael Fleurence
    • 1
  • David H. Hickam
    • 1
  1. 1.Patient-Centered Outcomes Research Institute (PCORI)WashingtonUSA
  2. 2.Knowledge and Evaluation Research UnitMayo ClinicRochesterUSA
  3. 3.Division of Preventive, Occupational and Aerospace MedicineMayo ClinicRochesterUSA

Personalised recommendations