Introduction

The US Food and Drug Administration (FDA) has an Advisory Committee (AC) program to “support the Agency’s mission of protecting and promoting public health, and to meet the requirements set forth in the Federal Advisory Committee Act of 1972” (1, 2). The FDA convenes AC meetings to seek external expert advice on scientific and technical matters related to the development and evaluation of products regulated by the agency, regulatory policy and guidance, and research. ACs are comprised of scientific experts, consumers, and industry representatives (3). As of 2021, five out of the 47 (4) FDA-established ACs provided advice specifically to the Center for Biologics Evaluation and Research (CBER), which is responsible for the regulation of biologically derived products indicated in the diagnosis, prevention, or treatment of disease, including blood, blood components and derivatives, vaccines and allergenic extracts, and cellular and gene therapy products. The five CBER-specific ACs are as follows:

  1. (i)

    The Allergenic Products AC (APAC) focuses on allergenic biological products or materials.

  2. (ii)

    The Blood Products AC (BPAC) focuses on blood, blood products, and relevant biotechnology and devices.

  3. (iii)

    The Cellular, Tissue, and Gene Therapies AC (CTGTAC) advises on human cells, human tissues, gene transfer therapies, and xenotransplantation products.

  4. (iv)

    The Vaccines and Related Biological Products AC (VRBPAC) focuses on vaccines and related biological products such as live biotherapeutics and donor-derived microbiome products.

  5. (v)

    The Transmissible Spongiform Encephalopathies AC (TSEAC).

The AC meetings discuss available data (or lack thereof) relevant to the safety, effectiveness, and adequacy of product labeling. The discussion could pertain to Investigational New Drug studies (INDs), Biological License Applications (BLAs), or post-market studies or monitoring. CBER may also consult its ACs when developing general policy recommendations and guidelines for industry and FDA regulation of a class of products. ACs also evaluate and approve site visit reports regarding the quality and mission relevance of FDA’s research programs. For AC meetings related to specific product submissions, FDA and the sponsor each prepare briefing documents related to the discussion topics. FDA may include voting questions for the AC in its briefing document (5). The meeting briefing documents are given to the AC members and posted online for the public no later than 2 days before the AC meeting. Although ACs provide recommendations to the FDA, the agency makes the final decisions related to products, policies, and research as described in Title 21 of the Code of Federal Regulations Part 14 (6): “the Commissioner has sole discretion concerning action to be taken and policy to be expressed on any matter considered by an advisory committee.” In this paper, we focus only on the AC meeting discussions recorded in the publicly available meeting materials to identify common benefit-risk considerations among ACs when they provide advice for FDA regulatory review. FDA internal discussions following an AC meeting leading to the final regulatory decision are beyond the scope of this paper.

In September 2021, FDA published a draft guidance for industry titled “Benefit-Risk Assessment for New Drug and Biological Products, Guidance for Industry” (7). This draft guidance document describes important considerations for FDA’s benefit-risk (B-R) assessments, which typically comprise case-specific, multi-disciplinary evaluations of current science and medicine, factoring in therapeutic context, evidence, uncertainty, and regulatory options. FDA’s structured B-R framework (BRF) is a flexible mechanism for identifying, assessing, and communicating the key factors that affect regulatory decision-making (7–9). The four dimensions of the structured BRF are (i) analysis of condition, (ii) current treatment options, (iii) benefit, and (iv) risk and risk management. Each dimension is analyzed with respect to its “Evidence and Uncertainties”. Finally, the conclusion is made by integrating the structured B-R assessment and the B-R trade-off. The FDA B-R guidance indicates that “some uncertainty in the body of evidence available at the time of regulatory decision-making is inevitable” and “[w]ith appropriate consideration of this uncertainty, the agency uses scientific assessment and regulatory judgement to determine whether the drug’s benefits outweigh the risks, and whether additional measures are needed and able to address or mitigate this uncertainty” (7). Since 2013, CBER has integrated the structured BRF into its BLA review template.

FDA also published guidance earlier in 2018 on “Benefit-Risk Factors to Consider When Determining Substantial Equivalence in Premarket Notifications (510(k)) with Different Technological Characteristics” (10). In a 510(k) submission, sponsors are required to demonstrate that the “new device” is “substantially equivalent” to a legally marketed predicate device. A B-R assessment should be conducted under two situations: (1) an increase in risk and increase or equivalent benefit or (2) a decrease in benefit and a decrease or equivalent risk. According to the guidance (10), the agency evaluates the aggregate benefits, including, but not limited to the type, magnitude, probability, and duration of benefits. Similarly, FDA evaluates the aggregate risks, including, but not limited to the severity, types, number, rates, and probability of each risk. FDA typically determines the approval of a drug/vaccine based on the benefit-risk profile of the product for the indicated population. After authorization or approval, other US public health entities provide guidance about the distribution, insurance reimbursement, and clinical use of the product.

A few reviews regarding the FDA AC meetings have been published from different perspectives, such as the characteristics of NDAs and BLAs (11); voting behavior of AC members (12); and the B-R analysis methods and tools (13). In this study, we evaluated AC meeting discussions within the context of FDA’s BRF and guidance to identify key regulatory challenges and B-R considerations driving the conclusions and recommendations by ACs.

Methods

We retrospectively reviewed publicly available meeting materials, such as briefing documents, minutes, summaries, and transcripts of the past CBER AC meetings (2009–2021), and conducted content analysis to identify common conceptual patterns (8, 16, 17). Furthermore, we followed FDA’s structured BRF and the guidance on B-R assessment for new drug and biological products published in 2021 (B-R guidance) (7) and for devices published in 2018 (10) and conducted an in-depth review focusing on AC meetings (2016–2021) where the B-R of a product or a category of products was discussed. We present the process of AC meeting selection and review in Fig. 1. A more detailed description of the process can be found in the S1 Supplementary Text.

Fig. 1
figure 1

A flowchart of the AC Meetings selection process. *(a) Scientific or regulatory research presentations; (b) advice on workshop planning, policy, and/or guidance development; (c) review of biological products, including IND, BLA, or EUA; (d) general advice on a biological product category; (e) influenza vaccine strains selection; (f) disease epidemiology and surveillance update; (g) general advice on a device category; (h) Humanitarian Device Exemption (HDE) or Premarket Approval (PMA) review of devices

Full size image

AC Meeting Review Codebook

We structured a review codebook for our in-depth review. The codebook is aligned with the agency’s B-R guidance, similar to that described by Lackey et al. (8) (see Table I and S2 Supplementary Text for details). There are four dimensions: therapeutic context (T), benefit (B), risk (R), and B-R trade-off (BR). These dimensions branch to capture important considerations of the evidence and uncertainties relevant to the benefits or risks of the drug/device, the disease condition, or risk management options. The dimension of “Therapeutic Context” in this codebook is an amalgamation of the “Analysis of Condition” and “Current Treatment Options” in FDA’s structured BRF for new drugs and biologics (7) and is discussed specifically as characterization of disease/condition in the guidance on 510(k) for devices (10). We added “other considerations” to the codebook to capture any important factors not fitting in the above four dimensions. In supplemental materials, we provide methodological details, a tabulated Microsoft Excel workbook (S1 Workbook), and a collection of AC voting or discussion questions posed by FDA (S3 Supplementary Text).

Table I Benefit-Risk Dimensions in the Review Codebook Used to Identify Common Regulatory Considerations in the CBER-Led AC Meetings
Full size table

Case Studies

We selected one AC case study from each of the three product offices at CBER, namely, the Office of Vaccines Research and Review (OVRR), the Office of Therapeutic Products (OTP), and the Office of Blood Research and Review (OBRR). The selected case studies encompass the diversity of key B-R considerations to illustrate the details of discussions at the AC meetings. Case studies #1 and #2 are related to regulatory reviews of two biological products, i.e., peanut allergen powder (OVRR, APAC meeting) and a gene therapy product (OTP, CTGTAC meeting). Case study #3 covers one BPAC meeting discussing the reclassification of in vitro diagnostic devices for Human Immunodeficiency Virus (HIV) (OBRR, BPAC meeting).

Results

In-Depth Review of AC Meetings (2016–2021)

Meeting Topics

Between 2009 and 2021, there were 119 CBER-led AC meetings. We grouped them into eight categories based on topic area (Fig. 2A of S1 Text). The frequency of topic areas discussed in the AC meetings is reported in the supplemental materials (Fig. 2B of S1 Text). Our in-depth review focused on 23 AC meetings (from January 1, 2016, to October 1, 2021), in which 24 topics were discussed. Of the 24 meeting topics, 12 discussed BLA, IND, or EUA applications (general topic area c), nine sought advice on a biological product category (general topic area d), and three sought advice on a device category (general topic area g); see Fig. 2C of S1 Text. During AC meetings that focused on the review of biological products (general topic area c), committees consistently discussed whether efficacy and safety data supported the licensure of the product in question for the specified patient population. Under general advice on a biological product category (general topic area d), committees discussed similar issues, but the recommendations were generally pertinent to a category of the products. Lastly, under general advice on a device category (general topic area g), the committee discussions focused mainly on device initial classifications or reclassifications.

Fig. 2
figure 2

Frequency of discussed B-R dimensions (therapeutic context, T; benefit, B; risk, R; benefit-risk trade-off, BR) and important considerations (T1–T4, B1–B4, R1–R7, BR1) among the 24 topics discussed in AC meetings 2016–2021. a Frequency of the B-R dimensions. b Frequency of important considerations within each B-R dimension. c Frequency of the discussed B-R dimensions among three selected general topic areas (c, d, and g) and frequency of committee voting requested. d Frequency of important considerations discussed among three selected general topic areas

Full size image

Important B-R Considerations

Among the B-R dimensions (Table I), the most frequently discussed in the 23 AC meetings was Risk (22 times) followed by Benefit (20 times), and the B-R trade-off (6 times) was the least discussed (Fig. 2A). In Fig. 2B, we evaluated the important considerations within each B-R dimension. The important considerations “Uncertainty about assessment of the safety profile” (R2) under Risk and “Uncertainty about assessment of the benefit based on clinical trial data” (B2) under Benefit were the most discussed. The “Uncertainty about the patient preference” (T2) under Therapeutic Context was the least discussed.

Discussions under important considerations B2 and R2 were typically on the sufficiency and generalizability of clinical trial results, and this included trial cohort sizes, age, sex, and racial/ethnic diversity. For example, in a meeting on peanut allergy immunotherapy, the APAC members discussed the accessibility of clinical trials to patients from underrepresented groups, which could translate to a lack of access to new treatments among racial and ethnic minorities. In the same APAC meeting, further discussions highlighted that, although evidence suggests that children of Black or African American descent have a higher prevalence of peanut allergy, the clinical trial participants were not appropriately representative of this demographic. Similarly, in the VRBPAC meetings that discussed the licensure of COVID-19 vaccines, the importance of including diverse research participants in randomized controlled trials (RCTs) was discussed at every meeting. Considerations regarding diversity at the COVID-19 AC meetings included race/ethnicity, age, comorbidities, and rural/urban residence, among others. AC members suggested that RCTs including participants with broader demographics are desirable so that data reflecting the safety of the overall target populations could be collected and made available in a timely manner to inform the authorization/licensure decision.

Figure 2C shows the frequency of the B-R dimensions among the three selected general topic areas (c), (d), and (g). In AC meetings focusing on (c) review of biological products and (d) general advice on a biological product category, all of the B-R dimensions (i.e., therapeutic context, benefit, risk, B-R trade-off) were discussed. Voting was more frequently requested for topic area (c) review of biological products (Fig. 2C). This is expected since the AC’s advice and recommendations are sought based on the adequacy of the data to support safety and effectiveness during these types of AC meeting discussions. Out of 12 products discussed under general topic area (c) review of biological products, two had breakthrough therapy designation (see case studies #1 and #2 below), a pathway intended to expedite the development and review of drugs for serious or life-threatening conditions (19). Three products ultimately were granted Emergency Use Authorization (EUA), and eight of the products discussed at the AC meetings were eventually approved (see S1 Workbook).

As shown in Fig. 2D, we further evaluated the frequency of the important considerations under each B-R dimension for selected general topic areas. The most-discussed important considerations include “Uncertainty about assessment of the benefit based on clinical trial data” (B2) and “Uncertainty about assessment of the safety profile” (R2), which were extensively discussed in meetings covering general topic areas (c) and (d). Noticeably, for the general topic area (g) general advice on a device category, AC discussions were similarly distributed among uncertainty on real-world benefit, uncertainty related to product use safety in the post-market, uncertainty in the effectiveness of risk management, and the trade-off between effectiveness and burden of risk management options.

Figure 3 shows the B-R dimensions discussed in different types of AC meetings. “Uncertainty in clinical relevance of the endpoint” (B1) was most often raised at the APAC’s discussions on allergen immunotherapies. At all three APAC meetings that we reviewed, there were in-depth discussions about appropriate clinical trial endpoints for evaluation of the benefit. On the other hand, VRBPACs discussed all B-R dimensions with most discussions pertaining to important considerations R2: “Uncertainty about assessment of the safety profile” (10 out of 16) and B2: “Uncertainty about assessment of the benefit based on clinical trial data” (10 out of 16).

Fig. 3
figure 3

Frequency of important considerations among the B-R dimensions discussed among CBER Advisory Committee meetings from 2016 to 2021 (APAC, 3 meeting topics; VRBPAC, 13 meeting topics; CTGTAC, 3 meeting topics; BPAC, 5 meeting topics) See Table I and S2 Supplementary Text for the detailed definitions of the important considerations within B-R dimensions

Full size image

Within CTGTAC meetings, the discussions were similarly distributed among five important consideration categories (T1, B1, B2, B3, R2, and BR1 as defined in Table I). Most of the discussions focused on the benefits of biological products that are indicated to treat life-threatening conditions with unmet medical needs. As an example, uncertainties on benefit assessment based on clinical trial data arose in the CTGTAC meeting for Lantidra (Donislecel) (14). Lantidra is an allogeneic pancreatic islet cell therapy for the treatment of adults with type 1 diabetes who are unable to achieve target glycemia due to recurrent severe hypoglycemic events (SHE) despite intensive diabetes management. Efficacy and safety data were derived from the pooled results of two clinical trials (N = 30), Study UIH-001 and UIH-002. The primary efficacy endpoint was insulin independence in Study UIH-001, while the proportion of subjects with an HbA1c ≤ 6.5% and free of SHE was the primary composite efficacy endpoint in Study UIH-002. The efficacy endpoints were measured one year after the first transplant and one year after the last transplant in both trials. In their BLA, the sponsor submitted an analysis of the pooled data from both studies using the primary composite endpoint from Study UIH-002 for both studies. “However, 83% of subjects in Studies UIH 001 and UIH-002 did not have SHE in the year prior to their first transplant, and only 37% of subjects had HbA1c at the target level at baseline” (14). The committee discussed the relevance of the proposed composite endpoint in assessing the benefit to the patients, as most of the enrolled patients achieved a level of HbA1c ≤ 6.5% and had no SHE in the year prior to first transplant. In addition, the AC discussed the product indication and the challenges in interpreting the data given that the baseline characteristics of the patients were not clearly defined in the trial. The AC members also suggested that insulin independence over a long-term follow-up period of at least 4–5 years may help demonstrate a clinically meaningful benefit to patients. The voting question for the AC was “Does donislecel delivered by intraportal administration have an overall favorable benefit-risk profile for some patients with Type 1 diabetes?” The majority of committee members voted Yes (12/17) to the voting question, indicating that this product may be an additional improved option for very small subpopulations of patients, such as those who cannot tolerate a pancreas transplantation operation and have difficult-to-manage diabetes. This example shows how uncertainties resulting from the study population and study endpoints can affect B-R assessments of the product, as well as how the FDA’s B-R includes consideration of the strength of the available evidence and takes the remaining uncertainties into account in the BRF dimensions.

Three of the BPAC meetings sought general advice on a device category, topic area (g). In these meetings, “Uncertainty in effectiveness of risk management options” (R5) was the most discussed. The meetings discussed the potential risks of either the initial classification of in vitro diagnostic devices for human leukocyte antigen (HLA), human platelet antigen (HPA), and human neutrophil antigen (HNA) markers or the reclassification of devices for human immunodeficiency virus (HIV), hepatitis C virus (HCV), from Class III to Class II. The risks from devices are primarily the risks to an individual tested incorrectly by a device. The risks discussed included a potential increase in false positives that may result in unnecessary treatments or therapeutic-related adverse events, or an increase in false negatives that could lead to the delay or denial of life-sustaining treatment for a patient or the transmission of a life-threatening disease in the population. The meetings discussed whether experience with HIV (see case study #3) and HCV devices is sufficient to establish special controls (in addition to general controls) to provide a reasonable assurance of the safety and effectiveness of the devices. In another BPAC meeting involving factor IX replacement therapy in topic area (c), the FDA and the committee members discussed the need for post-market surveillance on the potential long-term impact of factor IX replacement therapy on pediatric neurocognitive development, a potential risk suggested by the pre-clinical studies but unable to be assessed during clinical trials.

Finally, our review found that most of the CBER-led AC meetings take place when Phase 3 clinical trial data become available. However, there were meetings that were held to guide and advise product development and regulatory review in the preclinical and early phases. For example, a VRBPAC meeting was held to discuss and make recommendations on the clinical development plan of a Phase 3 RCT (Study B3451002) regarding an investigational Staphylococcus aureus vaccine intended for pre-surgical prophylaxis in elective orthopedic surgical populations of adults between 18 and 85 years of age (15). The discussion focused on whether efficacy and safety data accrued in patients undergoing elective, posterior-approach, instrumented, or multilevel spinal fusion surgery could or could not be generalized to other elective orthopedic surgical populations. Most of the committee members commented that the benefits of the vaccine may be generalized to patients undergoing other elective surgeries. However, some members argued that although the risk from surgery and infection are similar in patients undergoing various elective orthopedic surgeries, a patient’s health history plays an important role in vaccine safety. At the end, the AC members recommended that the sponsor should broaden the inclusion criteria for the Phase 3 study to include more participants undergoing various surgical procedures.

The Case Studies

In this section, we present three case studies. We investigated the details of different dimensions of B-R considerations discussed at these selected AC meetings (Fig. 4). Those considerations were summarized in Table II by the B-R dimension.

Fig. 4
figure 4

Frequency of important considerations of benefit-risk dimensions, discussed in each case study

Full size image
Table II A Deeper Look at Topics Discussed by ACs: the Case Studies
Full size table

Case Study #1: APAC — Palforzia (Peanut Allergen Powder)

On September 13, 2019, APAC met to discuss and make recommendations on the safety and efficacy of peanut allergen powder with the trade name Palforzia, manufactured by Aimmune Therapeutics Inc. Palforzia is indicated as an oral immunotherapy for the mitigation of allergic reactions, including anaphylaxis, which may occur with accidental exposure to peanuts, and it is to be used in conjunction with a peanut-avoidant diet. Initiation of Palforzia is approved for patients aged 4 through 17 with a confirmed diagnosis of peanut allergy, and its use may be continued in patients 18 years of age and older. As therapeutic context, the prevalence of peanut allergy in children less than 5 years of age is estimated to be 0.75–1.3% and about 0.7% in adults (16, 17). The committee discussed the unmet need for a regulated desensitization immunotherapy to decrease the incidence and severity of allergic reactions arising from accidental peanut exposure.

For the benefits, the committee discussed whether the available efficacy data were appropriate to support the proposed indication for Palforzia. The sponsor presented efficacy data from a Phase 3 RCT with the primary endpoint: proportion of Palforzia-treated patients, ages 4 to 17, who tolerate a single dose of at least 600 mg of peanut protein with no more than mild symptoms compared to placebo at the exit oral food challenge (after approximately a year of treatment). There was discussion regarding the relevancy of the pre-specified primary endpoint with the proposed indication since doses may be higher in accidental exposure compared to the challenge dose given in the clinical trials. Other discussions were about the diversity of the study population, as there is evidence suggesting that Black or African American children have a higher prevalence of peanut allergy (17, 18). Of the nine eligible voting committee members, seven agreed that the data were adequate to support the benefit of using Palforzia for the proposed indication.

For the risk, the committee discussed whether the available safety data in conjunction with additional safeguards were adequate to support the use of Palforzia in patients aged 4 to 17 years with confirmed diagnosis of peanut allergy. To assess these questions, pooled safety data from two RCTs were presented. Common adverse events (abdominal pain, throat irritation, pruritus, vomiting, cough, nausea, urticaria, upper abdominal pain, abdominal discomfort, oral pruritus, and sneezing) were at least 5% higher in Palforzia recipients compared to placebo recipients. Overall, treatment with Palforzia resulted in an increased risk of systemic allergic reactions, some of which resulted in increased use of epinephrine compared to the placebo group. However, the frequency of these systemic allergic reactions decreased in the maintenance phase compared to the up-dosing phase. The committee recommended that future studies should include long-term monitoring of study participants not only while on treatment (dose escalation, up-dosing, dose maintenance periods) but also beyond the treatment period, especially for those who stop treatment or fail to tolerate the treatment as they may have been increasingly sensitized, creating uncertainty of increased risk for future peanut exposure. However, eight out of nine members agreed that the available safety data in conjunction with additional safeguards were adequate to support the use of Palforzia as indicated for age 4–17 years.

The committee also discussed risk mitigation, suggesting the need for patients to take precautions when using Palforzia, in agreement with the Risk Evaluation and Mitigation Strategy (REMS) proposed by the agency, as it can require REMS for certain medications with serious safety concerns to help ensure the benefits of the medication outweigh its risks. For the use of Palforzia as part of the risk(s) mitigation strategy for systemic allergic reactions, the following was requested by the agency:

  1. A.

    Documentation that any patient prescribed Palforzia has a valid prescription for injectable epinephrine.

  2. B.

    Caregivers/patients must attest to carrying injectable epinephrine while on Palforzia.

  3. C.

    Initial dose escalation and the first dose of each up-dosing level must be administered in a certified facility capable of treating systemic allergic reactions.

Lastly, the B-R balance of using Palforzia was discussed, and the discussion was whether the risk of Palforzia outweighed the benefits for individuals who are following a peanut-avoidance diet. Most committee members concluded that the efficacy and safety data, in conjunction with additional safeguards, supported using Palforzia as a treatment to reduce the incidence and severity of allergic reactions arising from accidental peanut exposure in patients aged 4 to 17 years with a confirmed diagnosis of peanut allergy and that the REMS would provide further safeguards. Palforzia was approved on January 31, 2020, by FDA.

Case Study #2: CTGTAC — Voretigene Neparvovec

On October 12, 2017, the CTGTAC discussed the BLA for Voretigene neparvovec (AAV2-hRPE65v2, also known as Luxturna), submitted by Spark Therapeutics (19). Voretigene neparvovec is a subretinally injected gene therapy product, a recombinant adeno-associated virus serotype 2 (AAV2) vector carrying the gene for human retinal pigment epithelium 65 kDa protein (hRPE65). The product is indicated for the treatment of patients with confirmed biallelic RPE65-mutation associated retinal dystrophy, which is a rare, inherited, progressive, and devastating disease leading to blindness, for which there was no available pharmacological treatment.

Under therapeutic context, natural history data about biallelic RPE65-mutation-associated retinal dystrophy are limited. Six years prior to this AC meeting, the FDA held a CTGTAC meeting to discuss potential cellular and gene therapy clinical trials for the treatment of retinal disorders. The committee then encouraged the development of novel endpoints to analyze treatment efficacy because clinical trials may not be sufficiently powered to detect improvements in traditional endpoints (e.g., visual acuity or visual field). These endpoints may also be difficult to measure in pediatric patients and subjects with low vision. Thereafter, the sponsor of voretigene neparvovec aimed to base the primary evidence of effectiveness on a novel endpoint called the multi-luminance mobility test (MLMT). The advisory committee mainly discussed the clinical meaningfulness of MLMT score improvements and the durability of therapeutic efficacy. Results of the Phase 3 trials demonstrated a clinically meaningful benefit of the treatment, as 93% of the patients had at least 1 MLMT score improvement and the opportunity to gain activities of daily living. Additionally, these improvements were shown to be durable based on patient follow-up data obtained at least 3 years after the treatment.

The risks of voretigene neparvovec treatment include endophthalmitis, a permanent decline in visual acuity, retinal abnormalities (e.g., macular holes, foveal thinning, loss of foveal function, foveal dehiscence, and retinal hemorrhage), cataract, elevated intraocular pressure, retinal tears and holes, eye inflammation, eye pain, and maculopathy. A potential risk was related to undesired immune responses against the AAV2 vector and the hRPE65 protein synthesized by the delivered gene. However, immune reactions and extra-ocular exposure have been mild, even with sequential administration to each eye. There were no clinically significant cytotoxic T-cell responses to either the AAV2 vector capsid or the transgene product RPE65 in any of the patients. There was no inflammatory response other than occasional transient mild redness and inflammation of the eye, which was not specific. Nevertheless, to suppress the potential undesired immune responses, oral prednisone was given during the trials, which might be contraindicated for some patients. The committee discussed the treatment risks specifically for young pediatric patients since they are part of the indicated target population.

The committee discussed the potential B-R balance of repeat administrations of voretigene neparvovec that may be indicated to maintain vision or delay vision loss. The repeat administration of voretigene neparvovec was not evaluated in the clinical studies, and the human cellular immune response to AAV vectors cannot be well-predicted by preclinical animal studies. Therefore, committee members suggested further studies may be needed to support repeat administration of previously treated eyes if the therapeutic effect of earlier treatment declines over time. The committee also discussed uncertainties regarding B-R for subgroups. Developmentally delayed populations and the youngest children (age less than 3 years) were excluded from trial enrollment due to difficulty performing MLMT and other visual tests or safety concerns related to subretinal injection procedures. In addition, it is uncertain at what stage of the clinical presentation the benefits of therapy would outweigh the risks. Because the disease-causing mutation is expected to cause progressive vision loss, treatment at the late stage may not provide a substantial benefit to the patients. On the other hand, the retinal cellular proliferation is not complete until 8 to 12 months of age, and the product may be diluted or lost during the cellular proliferation process if treated before 12 months of age.

Considering the efficacy and safety information provided in the briefing document, as well as the presentations and discussions during the AC meeting, the committee unanimously concluded (16/16 members) that voretigene neparvovec has an overall favorable B-R profile for the proposed indication. Voretigene neparvovec was approved for marketing in the USA by the FDA 67 days after the CTGTAC meeting.

Case Study #3: BPAC — Reclassification of In Vitro Diagnostic Devices for HIV

A BPAC meeting was held to discuss the reclassification of in vitro diagnostic devices for HIV in July 2018 (20). The devices were regulated as Class III, which requires premarket approval (PMA), and were under consideration for reclassification into Class II, which generally requires a less burdensome premarket notification (510(k)) (21). Class III devices typically require greater FDA oversight due to the high level of risk associated with these devices (21). For Class II devices, general and additional special controls are sufficient to mitigate the identified risks and provide reasonable assurance of safety and effectiveness. The reclassification of devices from Class III to Class II would result in a reduced regulatory burden, a shorter premarket review time, and timely public access to the devices. Consideration of reclassification of a device often comes with increased experience on how to mitigate the risks with special controls, which allow FDA to regulate the devices safely as a class II.

The potential increase in the risks after reclassification and the development of appropriate risk management measures were the major considerations discussed during the meeting. The risks included a potential increase in the number of false positive test results, which could result in unnecessary treatment, treatment-related adverse events, anxiety, and stress. An increase in false negative test results could result in a lack of or delayed life-sustaining treatment and public health risk of the transmission of a life-threatening virus. False positive test results may be due to lower testing specificity due to microbial interference, cross-reactivity, and endogenous interference, whereas false negative test results may be associated with limitations of device design and insufficient analytical sensitivity. The committee discussed FDA’s proposed special controls designed to provide a reasonable assurance of the safety and effectiveness of the devices if they were reclassified as Class II. Examples included device-specific performance criteria, validation studies, product labeling to include warning statements, user instructions, detailed documentation (on device components, software, calibration standards, and design control activities, which are regulatory requirements for the development of a medical device based upon quality assurance and engineering principles), and enhanced post-market surveillance.

AC members in general supported the special controls proposed by FDA. The committee also commented that reclassifying devices to Class II will incentivize manufacturers to develop newer devices and allow them to further improve the current devices, as per the special controls’ requirements.

Discussion

In this review, we identified common regulatory topics discussed at FDA CBER AC meetings. Within the B-R dimensions, the discussion frequently gravitated towards risk, specifically, the “Uncertainty about assessment of the safety profile.” “Uncertainty about patient preference” was less often discussed. It must be reiterated that essentially all decisions that the FDA makes are accompanied by some degree of uncertainty. FDA’s B-R framework includes consideration of uncertainties as part of B-R discussion, leading to transparent and informed regulatory decisions.

The FDA requires sponsors seeking approval to provide evidence via a safety database that adequately characterizes the product’s safety profile. The generation of risk information during a RCT depends on the nature and extent or size of the clinical trial. Although most RCTs endeavor to recruit a sufficient and diverse study sample, a recurring discussion in the AC meetings was the small or statistically insufficient safety database. The appropriate size of the premarket safety database is product-specific and depends on a number of factors, including (1) novelty of treatment, (2) availability of alternative therapies, (3) intended population and condition being treated, and (4) the intended duration of use (22). The size of the study population appears to be a key issue for novel products that address an unmet medical need but have few eligible patients, such as immunotherapy products. For such products, recruiting diverse participants in sufficient numbers remains challenging.

Discussions on demographic diversity among research participants arose at many of the AC meetings. Many of the discussions were regarding the subpopulations that were excluded from some RCTs of novel products, e.g., children and racial/ethnic minorities. The ACs recognized the importance of factors such as race/ethnicity, sex, and age in clinical trial design so that benefit-risk assessments are more representative of the diversity of the target population. An example is the case of Heplisav-B, a recombinant hepatitis B vaccine sponsored by Dynavax. In 2012, Heplisav-B was first presented to the VRBPAC, and most of the members (13/14) agreed that the immunogenicity data were adequate to support its effectiveness for the prevention of hepatitis B virus infection in adults 18 through 70 years of age. However, the committee pointed out that the clinical trial data were insufficient to support the safety of Heplisav-B due to the inadequate size of the safety database, limited racial minority population representation, and a potential imbalance in immune-mediated adverse events between the control and treatment arms. Consequently, the committee voted no (8/14) to the question of whether the safety data are adequate for adults 18 through 70 years of age. Heplisav-B was not immediately approved for use. To remedy the safety concern, Dynavax launched a new clinical trial in 2014 that successfully addressed the issues. The sponsor doubled the size of the safety database and recruited a much more racially diverse population in the USA, which improved the ability to detect imbalance in infrequent, serious autoimmune events. As a result, 5 years after an initial recommendatory vote against approval, the VRBPAC agreed in 2017 with a majority “yes” vote for the adequateness of safety data for Heplisav-B. The product was approved by the FDA soon after that meeting (23).

Although most RCTs strive to have an adequate duration of follow-up, “Uncertainty about assessment of benefit based on clinical trial data” was a frequent discussion topic in the AC meetings. For example, FDA’s guidance for the EUA of investigational COVID-19 vaccines recommends that the median follow-up time in phase 3 studies be at least 2 months from the last vaccination. In addition, the agency recommended that patient follow-up should continue following EUA issuance, and the sponsor should include strategies for long-term follow-up of participants in ongoing clinical trials (24).

A detailed review of the three case studies revealed that the AC meeting discussions generally aligned with the FDA B-R guidance. For example, safety and efficacy data were very important during deliberations, as illustrated in the Palforzia (peanut allergen powder) case study. In this study, the AC members recommended the use of real-world evidence (RWE), additional clinical studies, and the collection of long-term safety data to supplement the data from clinical trials for B-R assessment.

The scientific and technical merits of risk mitigating strategies are also often discussed at AC meetings. In the meeting for Palforzia, AC members discussed FDA’s REMS proposal in addition to adverse event monitoring. The REMS included a requirement for caregivers/patients on the treatment to continually carry epinephrine. Administration of the initial dose escalation and first dosing of each up-dosing level was recommended to be done in a facility certified to treat systemic allergic reactions. Another example is the 70th CTGTAC in 2021, where AC discussed the toxicity risks of adeno-associated virus (AAV) vector-based gene therapy products. In the meeting, the committee discussed multiple risk mitigation options including reducing the amount of CpG levels in the AAV vector sequence, lowering the AAV dose, and performing baseline and intermittent toxicity assessments (25).

An example that illustrates unmet medical needs in benefit-risk assessment is the COVID-19 vaccines. These vaccines were developed as medical countermeasures (MCM) in response to the public health crisis caused by COVID-19. Although the development and clinical evaluation of COVID-19 vaccines were accelerated, the vaccines were authorized through EUAs when the accumulated scientific evidence showed that the known and potential benefits of the vaccines outweighed the known and potential risks. These vaccines were made available quickly to mitigate the public health crisis, but with continuing enhanced pharmacovigilance after EUAs to develop data appropriate for licensure (24).

Among all the important considerations described in this work, “Uncertainty about the patient preference” was the least discussed. This does not mean patient preference is less important; rather, more discussion and progress are ongoing in this area for therapeutic products, especially after the introduction of patient-focused drug development program under the Prescription Drug User Fee Act (PDUFA) V. In fact, FDA considers patient preference information (PPI) carefully and tries to incorporate the perspectives of the patients, as the patients are the ultimate stakeholders of the drug or the treatment (26). Patient preference information can inform many aspects of the B-R assessment, including the patients’ perspectives on the available treatment options, the impact of the disease, and unmet needs. Patient preference information can shed light on the meaningfulness of the clinical endpoints, the extent of the risks that the patients are willing to take, and the burden of managing the risks (7). Ideally, PPI should be collected early in the drug development process. These data could be used in consultation with FDA reviewers in discussion of clinical study designs accordingly, and the resources shared by the FDA could help stakeholders systematically gather and utilize PPI (27–29).

We observed that AC discussions for topic area (g) general advice on a device category are aligned with FDA guidance “Benefit-Risk Factors to Consider When Determining Substantial Equivalence in Premarket Notifications (510(k)) with Different Technological Characteristics” (10). These AC meetings discussed how benefit, risk, risk mitigation, and pertinent uncertainties were factored into the regulatory decisions related to devices. Some B-R considerations are more device-specific, such as the risk of false test results from diagnostic and blood screening devices; uncertainty about the characteristics of innovative technologies; the benefits of increased device accessibility to patients and clinicians; and the reduced regulatory burden associated with device reclassification from Class III to Class II.

The meeting briefing materials, including discussion items and voting/non-voting questions, are prepared by the FDA ahead of the AC meeting. During the meetings that we reviewed, FDA’s questions focused on the safety and effectiveness of the products, specifically, the therapeutic toxicities and appropriateness of animal models when discussing preclinical studies and the clinical relevance of study endpoints when discussing clinical studies (S3 Supplementary text). Discussions on the clinical endpoint for vaccine studies tended to focus on identifying and evaluating correlates/biomarkers of protection (30, 31). Two other common questions were on duration of benefit and follow-up and on target populations such as pediatric, elderly, and the immunocompromised (23, 32–34). Also posed, albeit relatively less frequently, were questions about drug administration (route, dose, frequency, setting), period of use (intermittent vs. life-long), and product manufacturing (19). Correspondingly, committee voting occurred more frequently in AC meetings discussing BLA, IND, or EUA reviews (35–38). Voting also occurred once during a meeting seeking advice on a device category, specifically discussing the initial classification of HLA, HPA, and HNA antigen or antibody devices (20).

Our work has some limitations, one of which is that we focused on the contents of meeting materials that were published on the FDA website. We did not consider why the FDA decided to hold the AC meeting for the specific products because this was not within the scope of our research. Additionally, AC meetings conducted during 2009–2021 were not evenly distributed across the four ACs (APAC, BPAC, VRBPAC, and CTGTAC). VRBPAC convened most frequently, (13/23), while APAC was the least frequent (2/23). Our period for primary reviews began in 2009, and the in-depth reviews only focused on the 2016–2021 period (both the PDUFA V and VI commitment periods were partially included). Additional considerations might have been identified had we focused on a larger time window. Also, our work considered only CBER-led AC meetings, although there are several other FDA-wide AC meetings (e.g., Pediatric Advisory Committee (PAC)) jointly organized by CBER and other centers (e.g., the Center for Drug Evaluation and Research (CDER)), which were not included in this review.

Conclusions

In conclusion, our review demonstrated the utility of FDA’s BRF and guidance. The B-R guidance could be a useful tool for both the FDA and sponsors when planning and preparing for AC meetings to facilitate the discussions and enhance the regulatory decision-making process. Identification of key B-R factors could be helpful to the FDA in the continuous process of building capacity for regulatory reviews. Such capacity building includes, but is not limited to, recruiting experts in areas of emerging technologies and enhancing regulatory science related to B-R assessment methodology, patient inputs, and real-world evidence. This in turn may further facilitate the sound risk–benefit judgments that are at the heart of all FDA regulatory decisions. Identification of key B-R factors could be helpful in sponsor’s activities and decisions throughout drug development. Our findings illustrate the discussion of AC meetings informing FDA’s product review, which may also help enhance public confidence in FDA’s decision-making for regulated medical products.