There is an urgent need for the development of a safe and effective vaccine for COVID-19. As such, COVID-19 vaccine candidates have been developed at unprecedented speed, and there are currently a handful of vaccines in the final stages of clinical testing. It is of critical importance that the normal evaluation of safety is not reduced or compromised in any way because of high-speed clinical development.
UK trials of the Oxford and AstraZeneca vaccine have resumed after a brief pause, following safety concerns. A statement made by AstraZeneca on 12 September stated that “on 6th September, the standard review process triggered a voluntary pause to vaccination across all global trials to allow review of safety data by independent committees, and international regulators. The UK committee has concluded its investigations and recommended to the MHRA that trials in the UK are safe to resume.” A press release published on the AstraZeneca website on 2 October, 2020 confirmed that the trial had not yet been resumed in the USA . AstraZeneca and the University of Oxford have not disclosed details of this case. More recently, on the 13 October, 2020, Johnson & Johnson reported that further dosing in the phase III ENSEMBLE COVID-19 vaccine trial had been paused, owing to an unexplained illness in a study participant . In this editorial, we describe how serious adverse events reported in a vaccine clinical trial should be investigated by reviewers [regulators or members of the Data and Safety Monitoring Board (DSMB)] to assess whether a causal relationship exists.
Criteria for Assessment of Causality
The World Health Organization (WHO) issued updated guidelines for the assessment of adverse events following immunization (AEFI) in 2018 . Whilst causality can be assessed at both the population and the individual level, this editorial focuses on the latter. The WHO guidance states that at the individual level it is usually not possible to establish a definite causal relationship between a particular AEFI and a particular vaccine on the basis of a single AEFI case report. It also emphasizes the initial requirement to have a valid diagnosis for the reported AEFI, which could be an unfavorable or unintended sign, an abnormal laboratory finding, a symptom, or a disease, and that, where possible, this should meet standard case definitions .
The Brighton Collaboration is an international voluntary organization to enhance immunization safety. In 2004, The Brighton Collaboration initiated a project to facilitate comparability of immunization safety data by developing standardized case definitions for AEFIs and guidelines for case determination, recording, and data presentation . The Coalition for Epidemic Preparedness Innovation has partnered with the Brighton Collaboration, through the Task Force for Global Health, to harmonize the safety assessment of Coalition for Epidemic Preparedness Innovation-funded vaccines via its Safety Platform for Emergency vACcines (SPEAC) Project, launched in 2019 . To date, 64 case definitions have been published for adverse events of special interest (AESI) following immunization, including guidelines for the collection, analysis, and presentation of data related to each outcome .
World Health Organization Algorithm
There are several models, algorithms, and tools (including software) available for causality assessment, each with its own merits and with varying sensitivity and specificity. After a thorough review of the existing methodologies for assessing causality in adverse drug reactions and AEFI, and after pilot testing of several approaches (including scoring scales, algorithms, and questionnaires) a Global Advisory Committee on Vaccine Safety working group in consultation with experts developed the approach outlined in the recent guidance . Once the diagnosis has been validated, it is recommended that the reviewer works through a logical checklist, summarized below:
The boxes on the pathway shown in Fig. 1 below correspond to the four major sections in the checklist above. During the initial stages of the assessment when considering the eligibility, the reviewer may consider the available information to be sufficient for initiating the causality assessment process. However, after completing the checklist it may be discovered that the information is insufficient to arrive at a definite conclusion. At this stage of the review, the reviewer may decide to categorize the case as “unclassifiable” and specify the missing information that prevents the classification of the case .
The final classification has been adapted from the “Definition and Application of Terms for Vaccine Pharmacovigilance” report of the CIOMS/WHO Working Group on Vaccine Pharmacovigilance . The cause-specific definitions provide clarity on “A. Consistent causal association to immunization” and “C. Inconsistent causal association to immunization” (coincidental). The association is considered “B. indeterminate” when adequate information on the AEFI is available but it is not possible to assign it to either of the above categories. The WHO has developed an electronic software that can assist with the AEFI causality assessment process.
Further detail relevant to each of the major sections outlined in the checklist is provided below. The preliminary step involves an attempt to validate the diagnosis prior to assessing causality. As part of this process, it is suggested that all relevant information about the event is obtained: full clinical details, laboratory tests, and other investigations in chronological sequence, including any results obtained prior to the vaccination. The ultimate outcome of the event and any sequelae also need be confirmed; was it fatal, is it ongoing, did the patient recover (partially or fully), and when were these outcomes observed? All of this information will contribute to validation of the underlying diagnosis, which may change over time as additional information becomes available.
Is There Evidence for Any Other Causes?
To decide whether there is evidence for any other cause, full information about the patients’ previous medical history should be obtained, including any illnesses diagnosed prior to and at the time of the event. Family history should also be sought, and enquiries made to ascertain whether the patient experienced any known risk factors for the specific adverse event of interest.
Is There a Known Causal Association with the Vaccine or Vaccination?
Enquiries to ascertain whether there is evidence to support a known causal association with the vaccine or vaccination should involve a literature search for all reports of the specific event of interest following vaccination. Where available, information on the outcomes in these patients should be ascertained, and how the particular serious adverse event affected the development and use of any associated vaccines established. It is helpful to further widen the search by conducting a literature review for similar events that have been reported following the administration of vaccines, with particular note of whether implicated vaccines shared the same vector, excipients, or adjuvants as the vaccine under study. In addition, further evidence can be collected through a detailed search of similar events in the study and the whole program, even if these events were considered to be unrelated previously, involving collection of all relevant case details to validate the diagnosis, including the clinical path of the condition, and a review of the patients’ past medical history. A database search for all other similar or related events using a broad search strategy should be conducted, ensuring the inclusion of all relevant event terms related to the relevant system organ class is adopted.
It is recommended that assessment of biological plausibility should be considered following discussion with relevant experts and using knowledge of the underlying on- and off-target pharmacological effects. Additional enquiries to ascertain whether the vaccine may have a quality defect should be considered, i.e., was the occurrence of the event related to the batch, or any potential manufacturing issue degradation of the product during distribution and storage? The WHO guidance states that an immunization error describes an AEFI that is caused by inappropriate vaccine handling, prescribing, or administration and that therefore, by its nature, is preventable . The types of reactions caused by immunization stress responses include, but are not limited to, acute stress responses, vasovagal reactions, and conversion disorders. Furthermore, it is important to confirm whether the event took place within a “plausible” time window of increased risk. This is applicable to all questions under (ii) of the checklist.
Is There Strong Evidence Against a Causal Association?
Is there a body of published evidence (systematic reviews, Global Advisory Committee on Vaccine Safety reviews, Cochrane reviews) against a causal association between the vaccine and the event?
Other Qualifying Factors
The guidance states that Sections (i)–(iii) outline the strong evidence for or against causality for most cases of AEFI. The checklist above includes some additional factors that support the above observations. If the AEFI is still unclassified, these qualifying factors provide reviewers with indications on causality.
It would be the responsibility of the DSMB to evaluate the outputs of these enquiries and make recommendations based on the findings. A DSMB consists of a group of independent experts external to a study assessing the progress, safety data and, if needed, critical efficacy endpoints of a clinical study. Recommendations made by the DSMB can determine the fate of the study. The core objectives of a DSMB include the following:
Protection of the health of both current vaccinees enrolled in the study/program and future participants.
To ensure that the DSMB actions are both scientifically correct and proportionate to protect patients without compromising the study or the program.
To maintain the blinding of the study to the sponsors (not to compromise the study) and to ensure continuation of the integrity of the data.
To ensure that high scientific and ethical standards are maintained throughout the conduct of the study by the sponsor, investigators, and all other staff associated with the study.
To maintain the integrity of the study, the sponsors, the study staff, and investigators remain blinded to the data; however, DSMB members are unblinded. Whilst this editorial provides an overview of best practice with respect to a causality assessment of serious adverse events in a vaccine study, it is acknowledged that regulators or members of the DSMB may be forced to act even in the absence of a definite causal relationship. At the end of every meeting, the DSMB would recommend one of the following courses of action:
To continue the study with no modification of the study protocol
To continue the study with a protocol modification(s), which may include:
Possible exclusion of certain groups of people from the vaccination program that may be at increased risk of developing the serious adverse event of interest
Additional investigations or screening criteria prior to including participants in the study
Request further information, either urgently or later with an agreed timeline
Co-opt an external expert(s) to advise on specific events
To suspend the study
To stop the study
The DSMB may also elect to conduct more frequent reviews of the data. Following the review of data in the Oxford Study during the pause in the trial, the DSMB and UK regulators recommended the trial could restart. The DSMB will continue to be crucial in this study, to continually assess through review of the safety data whether the potential benefits from the vaccine continue to outweigh any associated risks.
AstraZeneca. COVID-19 vaccine AZD1222 clinical trial resumed in Japan, follows restart of trials in the UK, Brazil, South Africa and India. 2020. https://www.astrazeneca.com/media-centre/press-releases/2020/covid-19-vaccine-azd1222-clinical-trial-resumed-in-japan-follows-restart-of-trials-in-the-uk-brazil-south-africa-and-india.html. Accessed 12 Oct 2020.
Johnson & Johnson. Johnson & Johnson temporarily pauses all dosing in our Janssen COVID-19 vaccine candidate clinical trials. 2020. https://www.jnj.com/our-company/johnson-johnson-temporarily-pauses-all-dosing-in-our-janssen-covid-19-vaccine-candidate-clinical-trials. Accessed 14 Oct 2020.
WHO. Causality assessment of an adverse event following immunization (AEFI). User manual for the revised WHO classification. 2nd ed. Geneva: WHO; 2018.
Bonhoeffer J, Kohl K, Chen R, Duclos P, Heijbel H, Heininger U, et al. The Brighton Collaboration - enhancing vaccine safety. Vaccine. 2004;22(15–16):2046. https://doi.org/10.1016/j.vaccine.2004.01.016.
The Task Force for Global Health. CEPI partners with Brighton Collaboration to support safety assessment of vaccine candidates. 2019. https://cepi.net/news_cepi/cepi-partners-with-brighton-collaboration-to-support-safety-assessment-of-vaccine-candidates-against-emerging-infectious-diseases/. Accessed 12 Oct 2020.
Brighton Collaboration. Brighton Collaboration Publications and Related Tools. https://docs.google.com/spreadsheets/d/1eQf2TXXPi4Y3U1zFSo2j0pyp73gagdJx4p-VMy_qXCk/edit#gid=0. Accessed 1 Oct 2020.
Report of the CIOMS/WHO Working Group on Vaccine Pharmacovigilance. Definition and application of terms for vaccine pharmacovigilance. Geneva: CIOMS; 2012.
No sources of funding were used in the preparation of this editorial.
Conflict of interest
The Drug Safety Research Unit (DSRU) is an independent charity (No. 327206), which works in association with the University of Portsmouth. It receives unconditional donations from pharmaceutical companies. The companies have no control on the conduct or the publication of the studies conducted by the DSRU. The DSRU are leading a consortium whose aim is to conduct a post-authorization safety and effectiveness study for a vaccine(s) for COVID-19 in the UK. The DSRU is not involved with monitoring the clinical trials for COVID-19 vaccines. Saad Shakir has worked in pharmacovigilance for many years and during that time has evaluated thousands of case reports and acted as a chairman and member of Data Safety Monitoring Boards for medicines. Miranda Davies, Samantha Lane, and Saad Shakir have no conflicts of interest that are directly relevant to the content of this editorial.
Consent to participate
Consent for publication
SS was responsible for the content and contributed knowledge and expertise in this area. MD and SL assisted with the development and reviews of the manuscript. All authors read and approved the final version.
About this article
Cite this article
Shakir, S., Lane, S. & Davies, M. How to Investigate a Serious Adverse Event Reported During a Clinical Trial for a COVID-19 Vaccine. Drug Saf 44, 1–5 (2021). https://doi.org/10.1007/s40264-020-01018-y