Background

Orphan diseases are often so rare that physicians have little knowledge of these conditions, while the contribution of patients to orphan drug development is under-documented [1]. Recent changes in the policy and regulatory landscape have enshrined the contribution of patients and patient organisations into the drug development lifecycle. On the 31st January 2022, the Clinical Trials Directive (EC) No. 2001/20/EC was repealed [2], and a transition period (until 2023) entered under Clinical Trials Regulation (Regulation (EU) No 536/2014) [3]. The objective of the new Clinical Trials Regulation is to harmonise the processes for assessment and supervision of clinical trials throughout the European Union (EU). The directive stipulates a greater role for patients and patient organisations (defined by the European Medicines Agency as not-for profit organisations which are patient focused, and whereby patients and/or carers -the latter when patients are unable to represent themselves- represent a majority of members in governing bodies) in the oversight of, and access to, clinical trials. In the UK, the Rare Disease Framework similarly outlined ambitions to improve patient access to new therapeutics, premised on a commitment to consultation with patient representatives, and explicitly those from Black, Asian and minority ethnic (BAME) or disadvantaged backgrounds [4].

Existing research suggests that the diversity of patient experiences of orphan drug trials has been overlooked [5, 6]. The current evidence base is dominated by surveys of patients who have had a positive experience of trial participation or treatment, practitioners who provide these treatments, or surveys of patient and public representatives interested in drug trial development [7, 8]. Where qualitative evidence of patient experiences have been collected, they have been subject to content analysis and automatic textual analysis [9, 10], rather than in-depth qualitative analyses that could inform improvement. There is a gap in knowledge around the experience of patients who have limited access to clinical trials of genomic treatments, and those who do not receive the active treatment, who withdraw from the trial, or for whom there is a perception that the drug is ineffective [11].

Recently, Brown and Bahri have proposed a conceptual and methodological framework for evaluating patient and public engagement in relation to pharmacovigilance, which delineated engagement in terms of three dimensions [12]:

  • Breadth: the diversity of patient engagement;

  • Depth: The extent of knowledge exchange between stakeholders; and

  • Texture: The interactive dynamics of what engagement feels like, means to people, and shapes their motivations to engage and change behaviour-based on values, emotions, (mis)trusts, and rationales.

Furthermore, they note that qualitative research is particularly suited to evaluating both the perspectives and mechanisms of engagement activities. Noting a rise in the volume of quantitative research that purports to concern patient and public engagement in the orphan drug development lifecycle [13, 14], we employed Brown and Bahri’s framework to establish the extent to which corresponding qualitative research could deepen our understanding of current engagement practices [12].

The aim of this paper is to explore how patients and other members of the public engage with the process of orphan drug development.

Methods

Patient advisory group

A Patient Advisory Group (PAG) were convened prior to the funding application, and met regularly to discuss the scope and content of the research. The group consisted of 6 local members of a rare disease group, and 2 members of a national group. The scope and content of the review were also discussed with the Steering Group, which also includes a patient from an international patient organisation. The PAG did not want to be cited as authors [13] but we acknowledge their contribution to this review.

Literature search

Orphan drug terminology is highly specialised, and we started our search by gathering a selection of papers using search methods that do not rely on keyword terminology. This was informed by the work of Zhao [14], who outlines the role of ‘meta-’ in the synthetic process, and the need to identify the ‘state of play’ of a given area of study. We followed Zhao’s advice to use qualitative synthesis as diagnostic. For Zhao: “[synthesis] starts with an examination of problems encountered in primary study and ends with prescriptions for resolving these” (14: 381).

To this end, we conducted forward citation searches of two topically relevant systematic reviews using Google Scholar (https://scholar.google.com/), which although had excluded qualitative papers during searching and screening exposed us to topically relevant literature [15, 16]. The lead author (JF) knew the systematic reviews from background reading. We also inspected the studies included in these reviews. Qualitative primary studies which met our inclusion criteria, and which could inform the development of the bibliographic database search strategy, were examined for keyword terminology. We also examined quantitative primary studies for topic related terminology, even though these would not be included within the analysis.

The bibliographic database search strategy was developed by an information specialist in conjunction with the review team. The search strategy was developed in MEDLINE (via Ovid)). Search terms for orphan drugs and rare diseases were derived from the titles, abstracts and indexing terms (e.g. MeSH in MEDLINE) of pre-identified relevant studies and supplemented with appropriate synonyms. As a corrective to previous reviews, which had excluded qualitative papers, we combined these terms with two published search filters: a patient and public involvement search filter [17] and a qualitative search filter [18]. However this yielded a prohibitive number to screen in full (n = 6935), as many of the studies were irrelevant (e.g. beyond our area and scope of interest, Fig. 1: Initial search). We therefore focused the search by limiting the results to articles which were indexed with any of four highly discriminating methodological MeSH terms: qualitative research, interviews, focus groups, and patient participation. This retrieved a more focused sample (n = 262; Fig. 2: Amended search).. Our intention was not to conduct an exhaustive survey of the field, but to establish the extent to which qualitative research could deepen our understanding of current engagement practices. To do this does not require a review of all papers for all rare diseases, but instead draws on established qualitative sampling approaches, seeking ‘information power’; which depends on (a) the aim of the study, (b) sample specificity, (c) use of established theory, (d) quality of dialogue, and (e) analysis strategy [19]. As we wanted to sample a selection of relevant studies rather than search exhaustively, we limited the search to the MEDLINE database and the results of forward citation searching. The results of both the forward citation searches and the MEDLINE search were exported to Endnote X8 [20] and de-duplicated using both the automated de-duplication function and manual checking. The bibliographic database search was conducted on 11th June 2021.

Fig. 1
figure 1figure 1

Orphan drugs search report

Full size image
Fig. 2
figure 2

Amended search

Full size image

Quality appraisal

Two researchers (AH, ET), independently applied the Critical Appraisal Skills Programme (CASP) checklist to assess the quality of the studies selected for inclusion [21] (Table 1). Quality appraisal is contentious in qualitative syntheses, because there is limited consensus about what makes a study good [22,23,24]. To further understand the context in which the research was conducted, we also used a validated 32-item checklist to provide a means to assess the rigour and validity of the data collection and analysis techniques used by the research authors [25].

Table 1 Quality appraisal of included papers
Full size table

Synthesis method

The purpose of qualitative synthesis is to achieve greater understanding and attain a level of conceptual or theoretical development beyond that achieved in any individual empirical study [26]. We had planned to undertake a meta-ethnography [27], to identify where similar concepts and themes from different studies or papers refer to the same entity or to opposing findings, with the objective of moving current debates about patient engagement forward [14]. However, as the included papers identified were not deemed to be ‘conceptually rich’ in that they did not extend our understanding [28, 29]. A paper is considered to be conceptually rich in qualitative synthesis if it makes a substantial contribution to the synthesis. In this context, critical appraisal is not undertaken to exclude papers prior to the synthesis, but to ‘test’ the contributions of the papers at a later stage [30]. We therefore undertook a narrative synthesis, appropriate when a wide range of research designs are included, and to tell the story of existing data [31]. The lead author (JF) tabulated data from the included papers using a standardised data extraction table, which enabled the derivation of themes that mapped onto the lifecycle of orphan drug development [32], and these were discussed and refined by the wider research team.

Results

Of the 262 abstracts identified, we excluded 192 that were explicitly quantitative (e.g. surveys, or concerning the development of patient reported outcome measures), or not about drug development (e.g. genetic sequencing and diagnostic pathways). Of the 70 full-text papers that we reviewed (Fig. 3: Identification of studies, and Additional file 1: Full texts retrieved), we excluded 57 papers that did not contain primary qualitative data (e.g. literature reviews, research protocols, opinion pieces, letters, editorials and organisations statements); were not deemed to be methodologically robust (e.g. they did not have sufficient information concerning recruitment, data collection or analysis to be replicable); or which were substantively not significant (e.g. they reported on focus groups or workshops, but the perspectives of rare disease patients, caregivers, representatives of patient organisations, or members of the public were missing, or could not be disaggregated from a wider ‘stakeholder voice’ that included health professionals or policy makers).

Fig. 3
figure 3

Identification of studies. Adapted from: Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ 2021;372:n71. https://doi.org/10.1136/bmj.n71. For more information, visit: http://www.prisma-statement.org/

Full size image

Study characteristics

We included 13 published papers from a 10 year period (2012–2022)—with studies originating from the USA (5) [33,34,35,36,37], Canada (4) [38,39,40,41], Europe (2) [42, 43], Brazil (1) [44] and China (1) [45], and which detailed 11 separate studies (e.g. both papers by Gaasterland et al. [42, 43] and both papers by Tingley et al. [39, 40] pertained to the same data sources) (Table 2). Three papers recruited the parents of children with rare diseases who considered trials as a means to access treatment [33, 34, 36]; while two papers included representatives of patient organisations concerned by the lack of access to research in specific countries [44, 45]. The included papers formed a dataset spanning key junctures of the orphan drug development lifecycle.

Table 2 Study characteristics
Full size table

Papers included patient and public involvement and engagement (PPIE) activities with patient representatives from umbrella patient organisations, as well as qualitative research with individual patients or caregivers from single disease organisations or attending clinics; but sometimes the boundaries between PPIE and research were blurred (e.g. stakeholder activities without research ethics approval presented as ‘data’). Most papers included identifiable approaches to qualitative data collection [36, 37]; however, counter to reporting guidance for qualitative research [25], only one paper [35] included an overarching methodological framework, with some citing reporting guidance, rather than qualitative methodological literature, as informing their research design [43]. Few papers provided details of how the authors conducted their qualitative analyses, although some reported findings statistically [33]. Some papers were more akin to reports of patient and public involvement/engagement (PPIE) workshops, or stakeholder events with mixed patient and clinician populations [44]. Few papers detailed the relationship between the author and participants, with limited reflexivity about their role in the construction of the research findings [21, 25]; thus making it difficult to discern how patients and other members of the public contributed to the generation of substantive knowledge about patient engagement in orphan drug development.

The 4 substantive headings, below, were those most used by the authors of the papers under review (as sub-headings) in their interpretations of the stakeholder perspectives in the primary papers, and typically follow the chronological processes of clinical trial and drug development [46, 47].

Trial design

Earlier research identified that patients with rare diseases want to see the adoption of a faster approval processes of new therapeutic agents that would produce effective treatments and improve their quality of life [36]. More recently, attention has turned to how patients, as experts in their own disease, can be active agents in the development of trial protocols, rather than merely as trial recipients [38, 41, 43]:

‘‘Because we’re talking about all the problems that happen after clinical trials are designed by people who know the science and the industry, but don’t know the disease and that’s the problem. We’re dealing with the problems because we’re not included before the trial begins.’’ (Participant, Menon et al 2015: 108).

Patient representatives reported negative perceptions of conventional randomised control trial designs and placebo controlled studies [39]. Instead, they suggested that their involvement in the trial design and protocol development could mitigate burdensome treatment regimens [41], and widen the parameters of enrolment to ensure that findings had increased external validity and thus applicability to a wider patient population [39].

Several papers suggested that patients and caregivers are not prepared to accept the outcome measures and clinical endpoints which trial designers’ currently offer [35, 39], and which fail to adequately account for quality of life [36]. The ‘true value’ of a therapy can therefore be lost because current outcome measures focus on what is easy to measure [38], or use standardised measures, rather than focusing on outcomes of interest to specific populations [39]. Authors also described the difficulty in eliciting patient outcomes and the need for new models of outcome development [42]:

“We [authors] decided that the best results would be achieved when researchers translate the patient’s preferences in outcomes, which are formulated in the first meeting, into measurement instruments and a trial protocol which can then be evaluated again with patient representatives during the second meeting.” (Gaasterland et al 2018: 1290)

The selection of clinical endpoints that were not considered as meaningful to reimbursement decision-makers was also a cause of frustration [41], and patient representatives identified that they should have a greater role in both research and reimbursement funding priorities [38].

Trial access

Patient representatives expressed disappointment when inclusion criteria, such as the phase of a disease, co-morbidities and existing medication regimens inhibit enrolment [33, 36]. Clinicians were viewed as gatekeepers, who can limit the enrolment of patients from minority backgrounds, due to beliefs that they will be unable to fulfil the research objectives or be non-compliant [33]. Lack of sponsorship can ensure that patients require sufficient ‘cultural health capital’, in order to push for access to a clinical trial [35]:

“Todd recalled the diagnosing physician telling them, “Go look for clinical trials—‘you go do your homework and I’ll do mine.’” But at their next visit, Savannah reported, “He hadn’t done anything. Like . . . he had printed out another sheet from his database . . . and told us, ‘Oh, looks like he has 2–4 years to live.’” Unwilling to accept this outcome, the Marins “did their homework” and scoured FDA databases for clinical trials.” (Gengler 2014: 346)

Several papers described the burden on patients of travelling to and from study appointments [34], which was magnified when parents of disabled children were required to stay in unfamiliar places without support networks [33]. Proposed solutions included financial remuneration, but also flexibility [34]:

“I might consider doing it depending on the leniency of when I could come in or what hours can I come into the office. I would be more drawn to a study that brought me in fewer times a week” (Participant, Carroll et al 2012:7)

A key motivator for trial participation was disease progression and associated high expectations for the benefit of trial medicines [37], even when patients themselves were unlikely to derive individual benefit [40]. In countries, yet to establish registries and normalise trial delivery, wider financial incentives were called for [44, 45].

Trial participation

While patient registries were seen as an important tool for monitoring rare diseases, and a means to recruit sufficient participants to a trial [38, 45], there were also concerns that they are only ‘suitable for highly motivated or informed volunteers who are specifically interested in research’ [33].

Despite the lack of a strong evidence base, rare disease patients are often willing to participate in trials of new drugs, even when they perceive that improvement may be minimal [36, 41]:

“There isn’t enough available for you to be able to prioritize, and so therefore you will grasp at anything that is acceptable to you. It doesn’t matter if it’s going to perhaps improve by 1% or by 50% or will get to the cure level of 100%. You will take it.” (Advocate, Kessselheim et al 2014: 78)

Throughout the drug lifecycle, decisions around what constitutes acceptable harm in order to achieve a certain magnitude of benefit are traditionally made with minimal input from patients, suggesting that there is significant scope for patients and families to be engaged as equal partners in such decisions [38]. Patients have suggested that they are concerned about the side effects of experimental medicines, and the consequences of stopping current drug regimens [34]; while others have expressed fears about making the ‘wrong’ decision, in agreeing to participate in a trial or not [41]. This can be compounded when patients are assigned to a control arm in a trial, and when the perception of any ‘potential gain’ is diminished [36]. Other patients have suggested that they have only been able to weigh up risks and harms retrospectively, that is after a trial, and specifically when any perceived improvement is subsequently lost [33].

Significant attention has also been given to the potential role of patient representatives in the design of patient-facing information. Patients and their caregivers often perceive that there is an over-abundance of information, which is fragmented, difficult to obtain, or difficult to understand [33] and which, paradoxically, can obfuscate that trials are performed in hospital with the objective of generating evidence about the effectiveness of treatments [43]. In the one example, where a trial was terminated before completion, parents reported feeling powerless, as their sense of hope receded [38]:

When he called up and said stop taking the medicine, I felt that conversation was worse than the diagnosis phone call when they told me he had muscular dystrophy... hope goes a long way, and to take that from a family is just pretty devastating ... The shattering part was because it was his cure. (Father 107, Peay et al 2014: 82)

In this example, parents were not prepared for a common trial outcome (no effect on the primary trial endpoint), which was compounded by the desperation wrought by the lack of available treatment (often typical for the rare disease community), and exacerbated by perceptions that the trial drug was of benefit for their children (in the absence of alternatives). It is suggested that patient facing information needs to better prepare participants for the possibility of a trial ending abruptly [38]; and patients have suggested that peer support and the use of social media could be used provide patient-to-patient information and support [33, 40, 42]

Dissemination

Two recent papers acknowledge that patient engagement activities typically end when their participation in a clinical trial finishes [40, 43]. Participants suggest that the results of a study need to be communicated to patients, both more often and more clearly [40, 41]:

“…I think that’s a really important piece to keep people motivated to participate in these things is to at least have some sort of follow through that allows us to see if what we shared made any kind of a difference. So that would be one thing that I’d like to see…”(Patient/caregiver 4, Tingley et al 2021: 6).

This would enable patients to evaluate their contribution to the research, as well as potentially encourage future participation in research [40].

Discussion

By formally appraising qualitative studies about how members of the public engage with orphan drug development, and employing the framework developed by Brown and Bahri [12], we identified a lack of depth in existing studies, due to the lack of understanding and rigorous application of methodologically informed qualitative research [48, 49]. Limitations include: the conflation of patient and public involvement and engagement (PPIE) and qualitative research, and a lack of detail pertaining to recognised qualitative analysis techniques (e.g. thematic or narrative analysis), and limited reflexivity about the authors relationship to those being researched [21, 26]. We also identified the adoption of a range of more quantitative techniques within papers purporting to have used qualitative methods. These include: interviews no longer than 10 min in length [34], the use of closed questions [44], and statistical analyses [33], which inhibited the development of richer understandings of patients’ unmet needs.

Recent research undertaken by EURODIS has identified that a barrier to sustaining meaningful patient engagement in rare disease research is a lack of knowledge on how to apply methodologies to capture and use patient insights [50]. More specifically, du Plessis et al. [51] have identified a lack of understanding of, and respect for, qualitative research methods that often form the foundation of patient-centric research in drug development. This lack of methodological knowledge matters, when engagement is becoming the new standard for patient-facing clinical trials and the development of new medicines to address patient needs [52].

Our analysis identified that patients and caregivers need to be involved in study design (including protocol development and the selection of measures that capture a range of outcomes, e.g. quality of life, beyond clinical endpoints) to optimise recruitment and demonstrate effectiveness. Patients and caregiver input is also necessary to ensure that trial providers enable access to the trial, develop patient facing materials that facilitate meaningful decision making, and disseminate results to patients to acknowledge their contribution and foster future collaboration. That these findings align with critical junctures for patient engagement identified in existent roadmaps for the drug development pipeline, developed by rare disease patient organisations is unsurprising [50, 52].

Earlier research about patient organisations generally, and those for rare diseases specifically, identified their imperative to engage with, and sometimes dependence upon, both the agenda and financial sponsorship of pharmaceutical organisations [53,54,55]. However, the roadmap developers are clear that these plans remain aspirational, rather than realised; with examples in the wider literature of sub-optimal patient engagement, in terms of lack of opportunities for genuine involvement in decision making [56], and acknowledgment that the patients who get to have their say in engagement activities are not representative of wider patients and publics [57, 58].

Accessing the perspectives of those who do not wish to participate in research is a so called a ‘wicked problem’ [59]. Two of the authors (JF and CP) are now undertaking a qualitative study of patients with one rare disease who have participated in, declined to participate in, or (been) withdrawn from, trials of new medicines and observational studies. We acknowledge, that in our focused study, we are only able to capture the perspective of those have agreed to be interviewed. However, to date we have interviewed patients with experience of participating in clinical trials and patients organisations, as well as those who have refused to participate in a trial, who have been declined the opportunity to participate in a trial because they have not met the inclusion criteria, as well as those who have withdrawn from a trial, and those who are not sure if they have been invited to, or participated in, research.

We suggest that more creative social science and participatory approaches might better enable those who are risk exclusion by traditional health services research methods from participating. See for example the recent initiatives from UK NIHR INCLUDE frameworks for ethnicity, and for people who lack capacity to consent [60].

Through using Brown and Bahri’s framework [12], we detected a lack of breadth in existing qualitative studies, due to the lack of diversity in the patients that have been engaged both socio-economically and geographically, and a narrow focus on the established ways of augmenting, rather than redefining, established clinical trial research [61]. For example, little or no attention in trial design is currently given to the perspectives of patients who do not want to be involved in research. More recently, Galasso and Geiger have suggested that innovation in precision and genomic medicine development risks exacerbating health care inequalities by benefitting people who are already advantaged [62]. They propose a more participatory medicine that would allow a wider public to voice their dissatisfaction, rather than being not- yet-reached or still-not-heard as at present; which might enable a more inclusive and democratic form of innovation [62].

In terms of texture, the third dimension of Brown and Bahri’s framework [12], we identified a lack of attention to both the processes of research and processes of engagement. This is important because, as Brown and Bahri suggest, understanding the processes of reactions to a new indication, the mediating factors of experience (e.g. trust, time, and resources), and any unintended consequences or barriers, are crucial to the design of effective measures and analyses of outcomes [12]. For example, although COVID has brought new opportunities for the use of remote and digital technologies for trial designs in rare diseases [63], attention must also be paid to which patients risk being further disenfranchised by the introduction of these methods.

Qualitative research is rooted in the philosophy of interpretivism, and may employ a range of methodological approaches that enable the interrogation of people’s views and perspectives, as well as any taken-for granted assumptions that may inform or provoke them [64]. We identified an absence of methodological frameworks to inform the research design and instead authors employed generic or off-the-shelf methods (such as ‘interviews’) [65], with some authors taking the patient and public perspectives at face value rather than providing an analysis [42]. Only one paper provided a methodological framework to inform data collection and analysis [35], and none engaged with the principle of reflexivity. Thus, absence in the detail and nature of the relationship between the researcher and researched ensured that any power dynamics involved typically remained obscured [66].

Considering the motives for the research were often to explore patient and caregiver perspectives, it is noteworthy that none of the studies were co-designed with patients or caregivers, nor patient-led. None of the papers reported critical reflection on the mechanisms of engagement activities [67,68,69], and instead presented case-studies as a sufficient reporting mechanism [56]. We contend that the texture of patient and public engagement in orphan drug development would be considerably improved if those researching their experiences and perspectives employed qualitative methodology, rather than ad-hoc methods, and ensured that the processes and outcomes of engagement are fit for purpose (e.g. inclusive and impactful).

In sum, we were able to identify an uneven landscape in the topography of public engagement that is indicative of both the complexity of researching rare diseases in different health care systems and using trial designs, but also the importance of structural inequalities. Data are ‘missing’ from our analysis, because of the methodological weaknesses in the papers or because some kinds of patient are absent from activities conducted with established patient organisations. However, our qualitative synthesis identified several key concerns of patients and has enabled us to suggest how future researchers may address the gaps identified (Fig. 4).

Fig. 4
figure 4

Key concerns and suggestions for how to address them

Full size image

Strength and limitations

A limitation of this research is that a wider literature search may have enabled us to understand more fully the landscape of how patients and other members of the public are currently engaged in orphan drug development. Furthermore, that the findings of our analysis were not robust enough for us to undertake a meta-ethnography is not indicative of failure. Having employed relevant search filters, and then focused upon the four highly discriminating methodological MeSH terms, we were able to evaluate any methodological weaknesses, using validated tools, and discern the substantive contributions in arguably the most relevant examples in the current qualitative literature.

A strength of this research is that we engaged with stakeholders throughout the research process. This included regular input into the design and content of the research from the Patient Advisory Group and steering meetings, as well as in presentations to industry partners and with colleagues working in clinical practice. This has enabled us to make some pragmatic recommendations about both the conduct of qualitative research and public engagement.

Conclusion

Conducting a qualitative synthesis of how patients and other members of the public engage with the orphan drug development, informed by Brown and Bahri’s framework, enabled us to identify the explicit need for methodological rigour in research with patients with rare diseases. This includes the need for appropriate and innovative use of qualitative methods and distinct PPI activities (rather than their conflation) and more strenuous efforts to capture the perspectives of under-served -researched or seldom-listened to communities with experience of rare diseases. This latter focus will require more creative recruitment and wider adoption of post-colonial practises, and a re-alignment of the research agenda (to encourage the use of co-design to enable patients to set the agenda, rather than respond to what they are being offered).