Identifying Anticipated Events of Future Clinical Trials by Leveraging Data from the Placebo Arms of Completed Trials

Background An important component of a systematic strategy for safety surveillance is prospective identification of anticipated serious adverse events (SAEs). Developing a structured approach to identify anticipated events and estimating their incidence can help align the safety strategy and the safety surveillance efforts. Methods We developed a novel approach to identify anticipated events for a hypothetical randomized, double-blind, controlled trial in subjects with bipolar disorder using the adverse events reported in the placebo arm of trials from the ClinicalTrials.gov database. We searched the ClinicalTrials.gov database for all trials on bipolar depression with similar inclusion/exclusion criteria and study duration as our hypothetical study. The frequencies of anticipated events in placebo arms were abstracted from each trial and 95% confidence intervals (CI) were calculated using the Clopper–Pearson method. Meta-analysis with a random effects model was performed to obtain a summary estimate and 95% CI for the events identified in more than one trial. Results A total of 129 clinical trials were initially identified, and 18 were ultimately selected as they met all the selection criteria. There were 69 unique anticipated SAEs identified, and 13 out of 69 were reported in at least 2 clinical trials. The top 5 anticipated SAEs for our study were: (1) hospitalization, psychiatric symptom (3.57%); (2) suicidal behavior, overdose (3.57%), (3) cholecystitis (2.86%); (4) fall (2.86%); (5) road traffic accident, injury (2.86%). Conclusion We successfully identified the anticipated events from registered trials that included a population similar to our trial. This method for identifying anticipated events could be applied to other disease areas. Electronic supplementary material The online version of this article (10.1007/s43441-020-00237-w) contains supplementary material, which is available to authorized users.


Background
Based on the 2012 Food and Drug Administration (FDA) Guidance Safety Reporting Requirements for INDs (Investigational New Drug Applications) and BA/BE (Bioavailability/Bioequivalence) studies [1], Sponsors are required to identify the anticipated events that it does not plan to report individually in an IND safety report. At the time of protocol development, these anticipated events should be included in the safety surveillance plan, together with a plan for monitoring the events.
Anticipated events are the adverse events (AEs) that the sponsor can foresee occurring with some frequency, independent of investigational drug exposure, in the general patient population under study, in patients with the disease under study, or both [2]. An important component of a systematic approach to safety surveillance is prospective identification of anticipated serious AEs (SAEs). The FDA has stated that individual case safety reports of such anticipated events are generally uninformative when reported as single events (i.e., without a comparison of the incidence of the event in treated and untreated subjects), and they do not contribute meaningfully to the developing safety profile of an investigational drug nor to subject protection [1,2]. To reduce the number of uninformative individual reports, it is necessary to take a more structured approach to identify anticipated events and estimate frequency of occurrence.
Since the anticipated events are intended to be those events that are either due to underlying disease, concomitant therapy, or demographic characteristics [3], previous trials with a similar population can provide valuable assistance in 1 3 the identification of anticipated events. The placebo arms of these analogous clinical trials are an ideal source to identify the anticipated events [4] as they by definition would exclude any events that may be truly attributable to the active medication. This formed the basis of our new approach to generate a list of anticipated events and their frequency using the AEs reported in the placebo arm of trials from the Clinical-Trials.gov results database.
In this proof of concept study, we sought to identify anticipated events from the registered clinical trials that are similar in population to a hypothetical clinical trial in bipolar depression. We used the Sherlock tool to identify the clinical trials that are similar to the hypothetical trial and extracted the information needed from the ClinicalTrials.gov database. This project provides an example of how to generate a list of anticipated events including serious and non-serious AEs using the ClinicalTrials.gov results database.

Hypothetical Study
We designed a hypothetical randomized double-blind controlled trial in subjects with bipolar disorder. The inclusion and exclusion criteria used reflect common research practice. Targeted participants are men or women between 18 and 65 years of age, with a Diagnostic and Statistical Manual of Mental Disorders 5th edition (DSM-5) diagnosis of bipolar disorder (types I and II), with or without mood stabilizing treatment. Participants must have experienced a major depressive episode (MDE), meeting the following criteria: (1) have a Hamilton Depression Rating Scale (HDRS 17 ) total score ≥ 17; (2) have a minimum of 2 manic symptoms in the current MDE (3) have a Clinical Global Impression scale for bipolar disorder (CGI-BP) ≥ 3. Patients who have a primary DSM-5 diagnosis of borderline or anti-social personality disorder, schizoaffective disorder or rapid cycling bipolar disorder or meet the DSM-5 criteria for manic episode will be excluded. The total study duration for each participant will be approximately 14 weeks.

Identification and Selection of Clinical Trials
The ClinicalTrials.gov is a registry and results database of both published and unpublished trials and contains information for privately and publicly funded clinical studies conducted worldwide. The ClinicalTrials.gov results database was launched in September 2008 to implement section 801 of the Food and Drug Administration Amendments Act of 2007 (FDAAA) (ClinicalTrials.gov) [5][6][7]. The law requires study sponsors to submit "basic results" for certain clinical trials, generally no later than 1 year after their completion date. The basic results include participant flow, baseline characteristics of the subjects included, outcome measures and statistical analyses, as well as information on observed AEs [8]. In the Clinical-Trials.gov results database, trial results are reported in a standard, tabular format which facilitates extraction of data with specialized software [6,8]. To help facilitate the identification of appropriate trials, we developed Sherlock, a system that downloads the ClinicalTrials.gov database daily, and can be used to query the trials and create analytic files for further analyses [4,9,10].
To identify potential clinical trials, we searched for clinical trials registered as of April 1, 2020, with the following search criteria: (1) trials with study results available; (2) condition terms with "bipolar disorders", "bipolar affective disorder", "bipolar I disorder", "bipolar II disorder", "depressed bipolar I disorder", "mixed bipolar I disorder", "moderate bipolar disorder", or "severe bipolar disorder"; and (3) trials with placebo arm. We only included clinical trials with enrollment ≥ 50 subjects and with study-time frame ≥ 3 weeks and ≤ 30 weeks. Furthermore, we excluded clinical trials with patients who had: (1) Age < 18 years; (2) a HDRS 17 total score < 17; (3) a manic episode, rapid cycling, cocaine/alcohol dependence or schizoaffective disorder; (4) a CGI-BP < 3 (Fig. 1). The authors reviewed the clinical trials that passed the first screening phase. In cases where there was disagreement as to the inclusion of a study, consensus was reached through discussion and by re-reviewing the clinicals trials' study description and eligibility criteria.

Identification of Anticipated Events
For the trials that met the inclusion/exclusion criteria, any AEs in the placebo arm reported in the Adverse Events section of the ClinicalTrials.gov results database were considered anticipated events. Anticipated events were further stratified according to the subcategories of the Adverse Event section: All-Cause Mortality, Serious Adverse Events and Other (not including serious) Adverse Events.

Data Extraction
The following key trial information was extracted: clinical trial ID, study type, study phase, number of arms, total number of enrollment, study status, study time frame, study inclusion and exclusion criteria, group/arm title and description, type of the AEs (MedDRA preferred term), number of participants at risk, frequency threshold, number of reported AEs, and number of reported SAEs. A data collection spreadsheet was developed as an analytic file to facilitate data analyses.

Statistical Analysis
For each selected clinical trial, the number of subjects who developed the AEs and SAEs, as well as the total number of subjects at risk in the placebo arm or the standard of care arm were extracted. For each anticipated event, incidence rates were calculated using the number of patients with the anticipated events and the total number of the subjects at risk. The frequency and the 95% confidence intervals (CI) were calculated using Clopper-Pearson method for a single proportion. For the AE and SAE identified in more than one trial, we performed a meta-analysis with a random effects model using R statistical software (https ://www.Rproje ct.org/) and the "DescTools", "meta" and "metafor" packages to obtain a summary estimate and 95% confidence interval.

Identification and Selection of Clinical Trials
There were 129 clinical trials initially identified (Fig. 1). Of these, 18 clinical trials were excluded because of the lack of a placebo arm. Then, we excluded 32 clinical trials with enrollment of less than 50 subjects or with a short study time frame (i.e. < 3 weeks) or a long study time frame (i.e. > 30 weeks). Four clinical trials which did not report AEs were also excluded. Finally, a total of 61 clinical trials were selected for the full assessment of the population eligibility.
After review in ClinicalTrials.gov, three duplicate clinical trials were excluded. We also excluded the clinical trials with pediatric patients (i.e. age at diagnosis < 18 years), and the clinical trials with the patients who had: (1) a HDRS 17 total score < 17 (N = 8); (2) a manic episode, rapid cycling, cocaine/alcohol dependence or schizoaffective disorder (N = 22); 3) a CGI-BP < 3 (N = 2). Finally, 18 clinical trials were selected since they included a similar population to our clinical trial and met all the selection criteria (Table 1).

Types and Frequency of Anticipated Serious Adverse events
We also identified a total of 69 unique anticipated SAEs (MedDRA preferred term), and 36 out of 69 had at least one case from the placebo arm in at least one of the 18 clinical trials (Supplementary Table S2). Two deaths were reported in one clinical trial. Thirteen of 69 anticipated SAEs had been reported in at least 2 clinical trials. A summary estimate and 95% confidence interval were calculated and reported for the SAEs identified in more than one trial. The top 5 anticipated SAEs for our studies were: (1) hospitalization, psychiatric symptom (3.57%); (2) suicidal behavior, overdose (3.57%), (3) cholecystitis (2.86%); (4) fall (2.86%); and (5) road traffic accident, injury (2.86%) ( Table 3, Supplementary Table S2).

Discussion
FDA safety reporting guidance indicates that sponsors should have a systematic approach to safety surveillance, including a process to pre-specify a list of anticipated SAEs to comply with the IND safety reporting requirements. In this study, we developed a novel approach to generate the list of anticipated SAEs using the placebo arm of studies with similar study designs and inclusion and exclusion criteria available in the clinicaltrials.gov results database. This list is presented here as a resource for sponsors of clinical trials to refer to while developing their pre-specified list of SAEs that are anticipated to occur in patients with bipolar disorder. This approach may serve as a model for identifying anticipated events in clinical trials investigating novel therapies for other diseases. We were able to identify a large number of clinical trials that had similar inclusion and exclusion criteria, study design and duration of follow-up to our hypothetical study. This is crucial since characteristics of the study population and previous experience with a similar population are major factors in the identification of the anticipated events. The use of meta-analytic techniques allowed the researchers to incorporate potential heterogeneity of the studies.
Although the ClinicalTrials.gov results database has information of on publicly and privately funded clinical studies on a wide range of diseases and conditions, it does not contain information about all the clinical studies conducted in the USA because not all studies are required by law to post study results [11]. In addition, it is worth noting that the submission of AE information in ClinicalTrials.gov was required beginning in September 2009, and the results  [5,6]. A major benefit of the ClinicalTrials.gov reporting system, is that it contains results of "negative" studies, studies that did not differentiate from placebo, which are often not published [12]. In addition to including information on studies that are not published, ClinicalTrials. gov also provides more complete data than those that are published, and a reliance on published data may result in missing nearly two-thirds of AEs [13]. Of note in the present study is that many AEs and SAEs were reported in only one clinical trial. If these AEs are to be considered anticipated events one might expect them to occur in the majority of trials as they all include similar patient populations; however, AE reporting may be inconsistent across studies due to the high diversity in the number and type of possible adverse effects, variations in their definitions, methods of ascertainment and the size of the placebo arm. Additionally, in many studies there is a frequency threshold for reporting AEs, typically > 5% (Table 1), and thus we can't assume there were no events in studies where they aren't reported. Because of this variability across studies, the prevalence estimates in the current study include only studies for which the AE was reported.
Other approaches that have been used to identify anticipated events is to utilize real world data [14]. We opted to use the AEs from placebo arms of similar studies because the inclusion and exclusion criteria can be mirrored, while it is more difficult to mirror the population in healthcare databases. Additionally, observational healthcare databases often do not have the depression scales needed to identify the subjects and the severity of the condition, which could change the rate of anticipated events such as suicidal ideation. Furthermore, the ascertainment of AEs in trials is completely different to how clinical events are recorded in healthcare databases, therefore, the AEs may be significantly underreported, which would make the rate of anticipated events obtained from those sources incomparable to the ones observed in a clinical trial. In addition, there is no real equivalent of a placebo patient in real-world databases, therefore, it is not possible to differentiate anticipated events from events caused by an active treatment.

Conclusion
In this study, we have developed a novel approach for identifying anticipated events in clinical trials using the Clini-calTrials.gov results database. We showed that clinical trials with similar study designs and inclusion and exclusion criteria can be found, a list anticipated events can be obtained, and their rates calculated. This study may serve as a model for identifying anticipated events in clinical trials investigating novel therapies for other diseases.

Open Access
This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a