FormalPara Key Points for Decision Makers

There is overwhelming agreement by industry on what level of engagement and transparency is desired regarding the use and value of PED in decision-making by the FDA.

Challenges and concerns appear unique to different types of PED.

Consideration should be given to alignment between sponsors and FDA on ways to address PED development using existing meeting pathways, adding a standing item to agenda for FDA meetings with industry on use of PED in filings, co-development between FDA and industry of an expanded PED table template and completion for each submission with industry, as well as expanded FDA guidance on standards for different types of PED.

1 Introduction

Over the past decade, the US Congress has enacted several laws that promote incorporation of the patient voice in medical product development and encourage the collection and submission of high-quality patient experience data (PED) to the US Food and Drug Administration (FDA), which the Agency may then consider in regulatory decision-making [1,2,3]. Patient experience data (PED) captures patients’ experiences, perspectives, needs, and priorities related to (but not limited to): the symptoms of their condition and its natural history, the impact of the condition on their functioning and quality of life, their experiences with treatments, input on which outcomes are important to them, patient preferences for outcomes and treatments, and the relative importance of any issues as defined by patients [1].

The 21st Century Cures Act, signed into law in 2016, required transparency and accountability on the use of PED in the review of drug and biologic marketing applications and mandated that FDA produce clear guidance on how PED can be used for regulatory purposes [1]. To comply with this requirement, the FDA has included a PED table as part of publicly available review documents, which provides a mechanism for reviewers to summarize the types of PED that the applicant submitted as part of their application, whether FDA considered PED from other sources, and the section of the review where PED is discussed (as applicable) [4]. The PED table was first implemented by the FDA in 2017 as an initial step to increase transparency. The 21st Century Cures Act also required FDA to conduct assessments on the use of PED in regulatory decision-making with the first assessment conducted by Eastern Research Group (ERG) with a report posted by the FDA in 2021 [4]. For this initial assessment, ERG examined FDA review documents and product labeling for approved applications received between June 2017 and 2020 and approved by February 2021 to determine what PED was submitted and where and how FDA used the data. The assessment also included the results of focus group stakeholder interviews with opinions collected about the collection, submission, and use of PED data in FDA reviews and reported in FDA clinical review documents and approved product labeling and potential efforts to advance the use of PED in regulatory decision-making. The ERG interviews included a small sample of pharmaceutical industry members (N = 29 participants from 11 companies), FDA staff from the Center for Drug Evaluation and Research (CDER) and the Center for Biologics Evaluation and Research (CBER) (N = 91 participants across CDER and CBER), and other stakeholders (N = 75 participants including patients, caregivers, clinicians, and patient advocacy organizations).

The aim of the current study was to expand the initial ERG assessment to include a larger sample of industry participants with specific types of PED expertise and experience with regulatory submissions to the FDA. More specifically, the current survey targeted participants with experience with patient reported outcomes (PROs), for which there is more established precedence with regards to use in regulatory submissions [5], as well other PED for which more recent FDA guidance has been issued, including natural history study (NHS) data [6], and patient preference (PP) information [7].

2 Materials and Methods

2.1 Population

Survey participants included industry (pharmaceutical, biotechnology), and contract research organization (CRO) members with ≥ 2 years of recent experience with PED (with recent experience defined as within the last 5 years) and experience with development of FDA marketing authorization applications describing at least one of three types of PED (PRO, PP, or NHS data), and/or responding to FDA inquiries regarding the submission of these data. Participants with only qualitative research experience (e.g., conducting interviews, focus group discussions), or experience only with other types of PED (e.g., clinician reported outcomes) were excluded to narrow the focus of survey. We selected a range of PED types with potentially different considerations for collection, submission and use including PRO data, which are more commonly used, NHS data which are less frequently used, and PP data with increasing use.

2.2 Study Design and Survey Instrument

The research team developed a study protocol, and the initial draft of the two-part online survey. The survey was pretested with ten industry and CRO members for readability and comprehension. Most of the suggested revisions were editorial in nature. The final survey (Supplementary Materials File 1) was disseminated by eight large professional special interest groups (two within ISPE, two within ISOQOL, PhRMA, BIO, C-Path PRO consortium, IMI PREFER)Footnote 1 Given the carefully selected eligibility criteria to exclude those with insufficient PED experience, broad distribution of the survey was deemed necessary. Participation in the survey was voluntary and data were anonymized.

Part 1 of the survey included questions to describe survey participants and to describe experiences related to PED collection, submission, and use by FDA in the review and labeling of new drugs and biologics. For PED collection, the survey included questions related to challenges with data collection (including ranking of least and most important challenges), the use of standard operating procedures, and level of functional area support and outsourcing. For PED submission, the survey included questions related to experience meeting with the FDA, the optimal timing for meetings and need for standing agenda items, and where to include PED in the submission. For PED use by the FDA, the survey included questions on completion of the PED table and use of PED in decision-making, recognition of PED use in the review, and the use of PED in the product labeling.

Part 2 of the survey included a best-worst scaling (BWS) type 1 experiment to quantify the relative importance of potential PED initiatives to advance the use of PED in regulatory decision-making by the FDA [8]. In each of a series of questions, the participant was asked to prioritize the following five potential FDA efforts originally identified during the 2021 ERG focus group discussions [4]:

  1. i.

    a dedicated meeting pathway for discussing PED with the FDA;

  2. ii.

    a roadmap articulating what PED FDA considers useful or valuable under what circumstances;

  3. iii.

    expanded FDA guidance on development of new fit-for-purpose clinical outcome assessments (COAs) and other PED;

  4. iv.

    collaborative initiatives (encompassing all stakeholders) to develop new fit-for-purpose COAs and other PED; and

  5. v.

    a public inventory of tools or methods that FDA considers acceptable or unacceptable.

A balanced incomplete block design was used for the BWS with three items per question, resulting in a total of ten BWS questions per participant. In each of the ten questions, the participant was asked to select the effort that was most and least important to pursue to advance the use of PED in regulatory decision-making by the FDA. Practices questions were included.

2.3 Analytic Methods

Descriptive statistics were used to summarize the survey results. Frequency counts and percentages are reported.

3 Results

3.1 Participants

With 657 views of the survey, 65 participants initiated the survey and completed the initial screening questions. Among those, 15 participants were excluded as they did not meet all eligibility criteria. The remaining 50 participants (84% industry, 16% CRO) completed part 1 of the survey (Table 1), among which 46 participants (92%) completed part 2. Among the 50 participants who completed part 1, 43 (86%) had PRO experience, 25 (50%) had PP experience, and 9 (18%) had NHS experience. Of note, 37 (74%) of participants also had qualitative research experience. Most respondents had ≥ 5 years of PED experience, with a majority with PRO or NHS experience having greater than 10 years of experience (Table 1). Most participants were in North America (68%) with the remainder in Europe (32%). Most were from large organizations (250+ employees) (96%) and worked in functional areas that specialized in patient-centered research or COAs (64%) or with real-world evidence research (22%). The four most frequently reported therapeutic areas in which participants had PED experience included oncology (58%), immunology (50%), dermatology (44%), and neurology (40%). The majority had common disease (88%) and half had rare disease (52%) experience.

Table 1 Baseline characteristics of the survey participants (N = 50)
Full size table

3.2 Survey Results: Part 1

A complete list of survey questions and summary of results is presented in Supplementary Material File 2.

3.2.1 Questions Related to PED Collection for Submission to the FDA

There were some differences in challenges selected as most and least important with PED collection by PED type. For those with PRO experience, the greatest challenges included time and expense involved in design and validation (23%), and difficulty justifying inclusion without knowing if and how the data will be used (21%) (Fig. 1). For those with PP and NHS experience, the most important challenge included difficulty justifying inclusion (36% and 33%, respectively) and lack of evidentiary standards (28% and 33%, respectively). For PP and NHS data, the least important challenges were timely FDA review and input (40%) and time and expense involved in design and validation (44%), respectively (Fig. 1). There was more variability in responses among participants with PRO experience.

Fig. 1
figure 1

Challenges ranked as most important and least important with PED collection, by type of PED (N = 50)

Full size image

Industry participants (N = 42) were also asked to answer process questions concerning whether established PED standard operating procedures (SOPs) and dedicated functional areas/personnel existed in their organizations, and the degree of outsourcing (Table 2). Overall, 14 industry participants (33%) were aware of existing internal SOPs and 7 participants (17%) were unsure. Additionally, 76% of all industry participants reported they had dedicated staff to support PED activities. There was a high degree of outsourcing in each PED area (Table 2).

Table 2 Internal industry support for the collection, submission, and extent of PED outsourcing among industry participants (N = 42)
Full size table

3.2.2 Questions Related to PED Submission to the FDA

Most survey participants (72%) had experience meeting with the FDA to discuss PED submissions (Table 3). Among those without experience, the most frequent reasons for not meeting with the FDA were lack of a recognized meeting pathway (29%), meeting opportunities too late to effectively incorporate FDA feedback in the design and execution of studies (21%), or value of PED in decision-making was unclear (21%). All respondents indicated that FDA meetings should have a standing agenda item to discuss PED use in decision-making, and most (80%) reported that FDA meetings to discuss the use of PED should occur before pivotal trials, with 16% reporting that the engagement should occur as early as the preclinical setting. The top two meetings felt to be most relevant for PED discussions were the end of phase II meeting (88%), and a type C meeting scheduled for this purpose (78%). A majority (62%) of participants reported that it was not clear where to include PED in the regulatory submission (marketing application).

Table 3 Process for PED submission, transparency, and use by FDA in reviews and labeling among all survey participants (N = 50)
Full size table

3.2.3 Questions Related to PED Use by FDA in Its Review and Labeling

Most survey participants indicated that both FDA and industry applicants should complete the PED table (74%) included by the FDA in the multidisciplinary review documents (Table 3). Almost all survey participants agreed that it would be useful to add columns to the table to describe if data were used in the FDA’s decision-making and for what purpose (90%), or if data were not used, why not (96%). When asked which categories should be added to describe use, the most frequent responses included demonstration of unmet treatment need (89%), use as a primary endpoint or in endpoint selection (87%), and to support structured or quantitative benefit-risk analysis (96%). When asked which categories should be added to describe why PED was not used, the most frequent responses were data not fit for purpose (88%), data incomplete or do not address the regulatory decision (90%), lack of data quality (83%), and inconsistency with trial design (75%) (Table 3).

In addition to use of the review document to indicate the extent to which PED was used, 29 survey participants (58%) suggested that the use of PED in regulatory decision-making should be reported in other ways. These included the FDA public announcement of approval (76%), the applicant approval letter (86%), the product labeling (90%), and advisory committee meeting minutes (90%). With regards to product labeling, respondents reported that submission of PED in product labeling was highest for PRO data (74%), followed by NHS data (22%) and PP data (20%). For each PED type, data were removed by FDA from the final product labeling for approximately half of the submissions (Supplementary Materials File 2).

3.3 Survey Results: Part 2

In the introduction to part 2 of the survey, participants were asked if there were other efforts that could advance the use of PED in regulatory decision-making beyond the five initiatives previously identified by ERG during its focus group discussions: (1) a dedicated meeting pathway, (2) a roadmap of what FDA considers valuable and under what circumstances, (3) expanded FDA guidance on development of new fit for purpose clinical outcome assessments (COAs) and other PED, (4) collaborative initiatives (encompassing all stakeholders) to develop new fit-for-purposes COAs and other PED, and (5) a public inventory of tools or methods that FDA considered acceptable or unacceptable. Among the 50 survey respondents, the majority (72%) felt the list was complete (Supplementary Material File 2). Additional suggestions provided by the remaining respondents were determined to be variations of these five broader initiatives. For example, “consensus across FDA areas regarding use of data in decision-making” would likely be integrated into discussions related to development of a roadmap of what FDA considers valuable and under what circumstances and other potential initiatives.

Among the 46 survey participants (92%) who proceeded to complete the BWS exercise, the most important efforts to advance the use of PED in decision-making (Fig. 2) were a dedicated meeting pathway and expanded FDA guidance (both ranked first in 51% of choices containing the item), followed by a roadmap (26%), collaborative initiatives (19%), and public inventory of acceptable methods (17%).

Fig. 2
figure 2

Relative importance of efforts to advance the use of PED in regulatory decision-making (N = 46).

Full size image

4 Discussion

Although the FDA has taken important steps to advance PFDD [9,10,11,12], the current survey study supports the earlier finding that challenges still exist with regards to the adoption and transparency of the use of PED in regulatory decisions by the FDA [4]. The current survey adds new knowledge regarding PED use by the FDA in regulatory decision-making, noting challenges and concerns unique to distinct types of PED and with prioritization of potential efforts by the FDA for advancing PED use. Results of the current survey also confirm findings from the initial ERG focus group discussions with overwhelming agreement on the desired level of industry engagement and transparency regarding PED use and value in decision-making [4].

To advance the collection and submission of high-quality PED to the FDA for use in regulatory decision-making, further dialogue between FDA, industry, and other key constituencies is needed to identify barriers. Over the past several years, FDA has demonstrated its commitment to PFDD with development and issuance of several guidance documents and has made organizational changes to centralize its drug policy function within the Office of New Drug Policy allowing for greater consistency, for example, in application of new tools for drug development across therapeutic areas [9,10,11,12,13]. Although these FDA initiatives are important steps forward, there remains a lack of regulatory certitude and a disconnect between what industry is hearing on the policy level and what is occurring in practice during FDA and industry engagements [13, 14]. Toward this end, industry also play an important role in driving towards the future state of patient-centered drug development, with further development and alignment of internal processes and procedures, to increase reproducibility regarding the collection and submission of high-quality data that are fit for purpose. As not all PED plays a significant role in regulatory decision-making, it is also incumbent upon industry to highlight the role of PED in important decisions (e.g., related to trial design or benefit-risk assessments, which incorporate patient preferences and value judgements). Although Congress has enacted several laws that promote incorporation of the patient voice, there is little clarity on how PED may be used by the Agency to guide decisions [1,2,3]. To continue this important dialog with key constituents, the FDA has recently issued a request for information on PED in product reviews to solicit input on methodological challenges and other barriers for PED collection with plans for at least two workshops to advance the field [15, 16].

Several important considerations for FDA, industry, and other key stakeholders during these continued discussions have been borne out by the current survey. The most frequently reported challenges with PED collection were the ability to justify the effort required for PED collection without knowing if and how the agency will use PED in regulatory decision-making, and challenges related to the lack of PED evidentiary standards. This highlights again the lack of consistency embedded in the process. Not surprisingly, challenges were reported more frequently by survey participants experienced with PP and NHS data than with PRO data, for which there are established precedents and FDA guidance on the use of these data in regulatory submissions [5].

Results of the current study also show that use of a dedicated meeting pathway and expanded FDA guidance on the development of new COAs and other PED were the two highest priorities among a list of potential efforts to advance the use of PED. The ability to meet more frequently and more often with the FDA, and the availability of expanded FDA guidance on the development of new fit-for-purpose COAs and other PED would allow greater opportunity to align on what PED could be useful in decision-making and under what circumstances. Among the five efforts to advance PED use that were identified during earlier ERG focus group discussions [4], collaborative initiatives and public inventory were reported in this survey as the least important initiatives. Although the survey design limited our abilities to follow-up with participants, it is possible these initiatives were valued as least important given the longer-time horizon in terms of implementation compared with expansion of an existing guidance or implementation of a dedicated meeting pathway, which were identified as being most important. It is also possible that given several large initiatives that have been undertaken within the past decade, there is less importance placed on the need for new ones [17,18,19,20,21].

In the current survey, the most frequent reasons for not meeting with the FDA were lack of a recognized meeting pathway and opportunities to meet were too late in the drug development programs. Additionally, most survey participants reported that FDA meetings to discuss PED use should occur before pivotal trials, with some survey participants reporting that the engagement should occur as early as the preclinical setting. The top two meetings felt to be most relevant for PED discussions were end of phase II meeting (88%), and type C meeting (78%). The desire to meet earlier and in a timelier manner were consistent with the input received during the ERG focus group discussions, but the current study provided more detail regarding the relative importance of different meeting pathways and timing for engagement [4]. Another meeting that could support PED discussions, but not available at the time of the survey, is a type D meeting, which is a meeting focused on a narrow set of issues that were recently introduced by FDA in 2023 [22]. The type D meeting offers reduced response and scheduling time compared with type B and C meetings, with a response to meeting requests within 14 days and holding the meeting or providing a written response within 50 calendar days after receipt of the request. With availability of several meeting options available (type A, B, C, and now D), and noting that 29% of survey participants not meeting with FDA reported lack of a recognized meeting pathway, it is important for industry and FDA to align around which meetings are best for these conversations, the format for those meetings as it relates to PED discussions, and whether including discussion of PED as a standing agenda item should be implemented. This alignment should be formalized by the FDA and incorporated by industry into internal guidance and SOPs to the extent possible.

With regards to PED use in the FDA review, the results of this survey agreed with the initial ERG assessment, demonstrating the desire for greater transparency regarding the use and extent of PED use in regulatory decision-making. There remains an opportunity to increase the utility of the existing PED table by enhancing the content and consistency of the information included to effectively communicate FDA’s conclusions about the review and value of PED in decision-making. Improvements in content and consistency could result from development of the PED table by both the FDA and the sponsor, as desired by most survey participants. As for utility of the table, almost all survey participants agreed with the value of adding columns to the table to describe if and how PED were used (90%), or if it was not used (96%), providing the reason why. Regarding the extent of use, respondents stated that categories for use should be added to the PED table, including the demonstration of unmet treatment need, use as a primary endpoint or in endpoint selection, and use in supporting qualitative or quantitative benefit-risk analysis. With regards to benefit-risk analysis, inclusion in the PED table would be of value, but caution should be exercised not to isolate PED from the holistic discussions of benefit-risk for new medical products. PED should be viewed as an integral component alongside efficacy and safety data. Further discussion is also warranted to ensure consistency in data recorded in the PED table and the separate benefit-risk summary table, both of which are included in the clinical review document.

Most survey participants indicated that the agency should address reasons for not using data in the regulatory review. The most frequently reported categories to add to the template included data not fit for purpose, data incomplete or not addressing the regulatory decision, lack of data quality, and inconsistency with trial design. To ensure that data are captured in a more consistent manner in the PED table and other sections of the multidisciplinary review document, additional stakeholders have suggested that prompting questions could be added to FDA standard operating procedures with access for reviewers to an internal repository of examples demonstrating the link between the table and explanatory information in the review document [23].

In terms of current practices, some additional findings were related to internal practices within industry, including awareness of internal SOPs, dedicated functional area expertise, degree of outsourcing, and inclusion of PED in labeling. With regards to internal SOPs, only one-third of the survey participants were aware of existing SOPs, yet approximately three-quarters of the participants worked in areas within the organization with PED functional area expertise. This disconnect may reflect lack of internal SOPs or lack of awareness of existing SOPs among those completing and submitting marketing applications, both of which suggest further internal education and/or standardization of practices and procedures are warranted. There was also a significant amount of outsourcing of PED activities. For labeling, the reported frequency of PED use in product labeling was highest for PRO data (37%) compared with NHS data (11%) and PP data (8%). The reported frequency of PRO use is consistent with a recent review of PRO use in product labeling by the FDA, which showed that 26% of new drugs have labeling based on PROs (10% and 50% of NDAs for non-PRO and PRO-dependent new molecular entities, respectively) [24]. The lower frequency of NHS and PP data use is not surprising. For PP data, we are only aware of three studies with results included in the product labeling with patients’ views on intravenous compared with subcutaneous formulation [25,26,27]. Further alignment on the expected level of design and validation for NHS and PP data and transparency regarding their role in decision-making are likely needed before these data will be more frequently included in product labeling.

The strengths of the current study included broad distribution of the survey by eight large special interest groups with several hundred views of the survey. Given the carefully selected eligibility criteria to exclude those with insufficient PED experience, broad distribution of the survey was deemed necessary to allow completion of the survey by a meaningful number of participants with informed perspectives on current practices and challenges regarding PED collection, submission, and potential use in regulatory review. It was not deemed sufficient for participants to have had cursory PED experience at some point in their work history. Instead, participants were required to have had ≥ 2 years of PED experience within the last 5 years, including specific type(s) of PED experience, and experience describing results in FDA marketing applications and/or addressing FDA queries. Another strength of the current survey relates to the collection of challenges and experiences unique to distinct types of PED, and prioritization of efforts to advance use of PED in regulatory decision-making, which were not captured with the initial ERG assessment [4].

Although the sample size was almost twice that of the initial ERG assessment, it is still a relatively small sample. Only nine participants had experience with collection, submission and use of NHS data, so findings unique to this type of PED should be interpreted with caution. In addition, the snowball sampling used in recruitment may have resulted in selection bias. Moreover, with anonymized participation, we were unable to seek further input from survey participants and there was a potential for clustering of responses within an organization, potentially further limiting generalizability. Lastly, although not a limitation as it relates to the intent and generalizability of the current survey, it is important to recognize the importance of shifting focus from PED measures more often included and submitted for regulatory decision-making (e.g., PROs) to other types of PED to allow a more universal adoption of patient-relevant outcomes for use in regulatory decision-making. Although patients with only qualitative research were excluded from the current survey (noting that 37/50 participants also had qualitative research experience), as were specific questions on collection and submission of qualitative PED, it is important to recognize the importance of qualitative research which is foundational for development of other PED measures and has unique insights that are important to consider in the context of regulatory decision-making.

5 Conclusion

While the FDA has policy targets extending PED use, challenges still exist regarding pathways for submission and transparency in the use of PED in regulatory decision-making. Consideration should be given to alignment between FDA and industry on how best to use existing meeting pathways to facilitate discussion about the development and use of PED, adding a standing item to agenda for FDA meetings with industry on use of PED in filings, co-development between FDA and industry of an expanded PED table template and completion for each submission with industry as well as expanded FDA guidance on standard for PED. It is incumbent upon industry to drive best practices and standards alongside FDA and other thought leaders to help incorporate the voice of the patient in medical product development and allow for the adoption of PED in US regulatory decision-making.