Abstract
Every medical product requires additional study even after regulatory approval. We highlight several lines of enquiry to advance our understanding of COVID19 vaccines post authorization: identifying key population segments warranting more study, assessment of efficacy, and of safety data, harmonization of data relating to immune response and developing mechanisms for data and knowledge sharing across countries. We show how innovative trial designs and sources from real world data play a critical role in generating evidence.
Similar content being viewed by others
Avoid common mistakes on your manuscript.
Introduction
Effective vaccination strategies play a key part in halting the impact of the COVID19 pandemic. Thanks to the commitment and resources of the biotechnology, pharmaceutical, academic, government and philanthropic sectors, several vaccines are being studied in large randomized controlled trials in record time. The primary objectives of these trials are to study the effect of the vaccine in reducing the number of overt COVID19 cases relative to placebo [1]. Results from interim analyses have been presented in publications [2,3,4] and briefing documents for FDA Advisory Committees [5, 6]. As of December 2020, vaccinations have begun in several countries following authorization by Health Authorities.
Lipsicht and Dean [7] highlight many open questions that remain after these initial trials. We identify three key scientific questions: population segments, durability of protection, and safety monitoring. We briefly describe the context for each question, noting what can be reliably inferred from the trials and what remains to be addressed in the post-marketing (PM) setting. We then assess experimental approaches, existing data sources, analysis and interpretation modalities.
Population Segments
We define population segments as demographic subgroups (eg age, gender, race), comorbidities (eg prior cancer, diabetes, obesity), community factors (eg community living, schools), and COVID19 risk factors (eg healthcare workers, first responders). While the vaccine trials are large, the primary comparisons for efficacy rely on the number of COVID19 cases as defined in the protocol, typically about 200 events. Interrogation within subsets, with even fewer events, creates more uncertainty in the statistical assessments. In addition to evaluating short-term efficacy and safety across subgroups, durability of effect and other research questions need to be addressed in population segments. While manufacturers made a concerted effort to enroll a diverse population, especially minority groups, some groups were underrepresented as compared to the COVID19 incidence rate in these segments. The trials were not designed to assess the role comorbidities and risk factors may play in efficacy and safety. We are left to provide this information in the PM phase.
Thus, our first recommendation is for a set of population segments to be developed and defined to facilitate future research across data sources. There will be a need to prioritize which segments warrant additional study. A second recommendation is to harmonize the definition and ascertainment of key efficacy and safety endpoints. Efficacy assessments should be able to link back to the trial endpoint definitions and expand upon these as relevant. For safety, common definitions and ascertainment strategies that facilitate integration of knowledge generated across all data sources would be valuable.
Durability of Protection
Durability of protection can be assessed using incident disease of COVID19 as used in the current vaccine trials or using a marker which is associated with protection. Combining data from multiple sources to establish a robust Correlate of Protection (CoP) is the preferred approach [8].
Having a CoP will greatly advance vaccine development. As the incidence of COVID19 drops because of effective vaccines, the next generation of vaccines could be developed based on the CoP in relatively small trials. A CoP can be used to establish duration of immunity and decide if and when a booster is needed.
There are still open questions on the nature of protection beyond just durability. In spite of having several successful highly effective vaccines authorized to date, none of the pivotal trials achieved 100% vaccine effectiveness, none evaluated sterilizing immunity (namely can a successfully vaccinated individual still infect others). Each vaccine would need to be separately evaluated if these limitations are of interest but none are required to achieve full approval from a regulatory authority.
Our third recommendation is for a full set of endpoints related to immunity to be defined (with appropriate descriptive assay information) and available for health systems and future trials to deploy in a structured way so key questions can be addressed.
Safety
The vaccine trials have enrolled 30,000 to 44,000 participants with interim results when median exposure of 2 months had been achieved. Given the usual safety data collection in vaccine trials, characterization of the vaccines’ short-term safety data for common effects (eg rate greater than 0.1%) is reasonable. What remains are rare events (such Guillaume Barre Syndrome which occurs at a rate of 0.0001%) and long-term safety events not immediately recognized as vaccine related (eg pregnancy outcomes). Since new modalities of vaccine constructs are at play, there may be unanticipated safety signals that will require clear and quick adjudication.
There are established mechanisms for PM surveillance in the US and other countries but given the vaccine is to be distributed in an unusual way (eg via pharmacies, places of work, community settings etc.), traditional PM approaches [9] may require rethinking and enhancements. Pharmaceutical sponsors and Regulatory authorities around the world are expanding their approaches which will yield important safety information.
Our fourth recommendation is for a clear communication mechanism to be established both at the country- and global-level for initial signals and for validated findings. All four Primary Recommendations are listed in Table 1.
Modern approaches for evidence generation
These factors suggest a cross-vaccine, cross-regulatory initiative is required for enhancing public confidence and addressing efficacy, durability and safety of vaccines in the worldwide population, and in specific segments. Given the differential between the size of the vaccine trials and the much larger global population, there is no doubt that many residual uncertainties can only be addressed in the PM setting. That said, both traditional and innovative randomized trials as well as real world evidence (RWE) approaches will be required.
Randomized Clinical Trials
Traditional trials or registries and immunobridging studies should be considered prior to full vaccine deployment and initiated quickly for pediatric populations, pregnant women, minority groups, and so on.
In addition to these targeted studies, one could envisage platform trials [10, 11] to compare the efficacy of various vaccines in segments of the population. These may be adaptive so that if several vaccines can be shown to be ‘equivalent’ in a population segment, they can be dropped from that part of the study. Likewise, if one is found to be sub-standard, that arm can be terminated, and the results communicated quickly. These designs would be novel since most platform trials focus on determining superiority of treatments rather than equivalence.
Cluster randomized trials (CRT) could be considered for the direct comparison of vaccines. In such trials, regions or institutions are the unit of randomization though individual subject informed consent is still required [12]. These are easier to implement than traditional trials and given the distribution complexities of individual manufacturer’s vaccines, could offer a substantial advantage. The fundamental unit of randomization may be a nursing home or even a city, but studies must have a reasonable number of clusters. The usual assumption is that within-cluster correlation is small but still needs to be considered in the analysis. CRTs can also be employed to evaluate educational or dose schedule strategies.
Last, one could envisage trials that include randomization to one vaccine or another as described here with follow up from real world data (RWD) sources to supplement the traditional trial data collection. This could include patient reported outcomes for symptom assessments.
Real World Data
In general, RWD sources encompass registries, claims data, health care records (hospital or clinical settings) and data sets that link the underlying data for a patient in multiple environments.
We begin with safety. Well-established channels exist for the evaluation of comparative effectiveness and for monitoring of safety in the PM setting using RWD [13, 14]. These can be leveraged in the case of COVID19 vaccines—and must be augmented given the need for transparent and clear information to allow the public to form its view on the level of confidence it will attach to the vaccines.
RWD for Safety Monitoring
Safety monitoring in the US relies on both manufacturer’s specific PM surveillance proposals and established regulatory data systems such as SENTINEL and the vaccine adverse event reporting system (VAERS). Pfizer and Moderna presented their respective PM approaches at FDA Advisory Committees [5, 6]. FDA has since provided PM surveillance protocols for review via BEST [15, 16], though this must be augmented with process and communication considerations. Alignment with the WHO PV guidance would be helpful for a global approach to safety [17]. Other considerations for the worldwide setting are discussed by Chandler [18]. The example of a unified detection and adjudication process deployed for H1N1 influenza vaccine in 2009 could be used as a foundation, updated with current technologies and tools for harnessing RWD at scale.
Since COVID19 vaccines will be administered to a very large population quickly, the opportunity to enhance our understanding of even very rare events (say 1 in 100,000) is more substantial than for any prior vaccination effort. There are foundational elements required immediately (see Table 2).
RWD sources for evaluation of durability and efficacy in segments of the population
Similar considerations as presented for safety apply for efficacy and durability assessments. Studying these in parallel in the same sources if appropriate allows direct benefit-risk characterization.
Some segments of interest, (e.g., healthcare workers) are especially motivated to volunteer for registries or targeted linked systems [25]. Pregnant women are typically studied in national registries or as a PM requirement. Registries for subjects that are on a specific medication or have a chronic illness that puts them at increased risk for a reaction to a vaccine could provide quick information on such reactions. In all cases, planning what information is gathered and when offers scientists and regulators a tremendous asset. While registries are not randomized, they allow for the study of risk factors and other population covariates, they can establish temporal effects and provide other insights based on cohesive and rich data collection strategies.
Some of the most important open questions including whether one vaccine is preferred over another in a particular segment, e.g., the elderly, could be studied in randomized trials, but it’s more likely that initial observations will come from RWD. Since the likely questions are already established, one could envisage multiple parallel interrogations such as deployed in the OHDSI network [26, 27], and more recently in the Reagan-Udall/Friends of Cancer Research [28] activities. Since gold standard approaches are not yet established for RWE, having a plurality of approaches initially would be informative. In addition, having set venues where RWE researchers debate their protocols prior to analysis, and review results in concert with full transparency will set a good standard for reliability of results [29].
Conclusion
The development of COVID19 vaccines has been an exceptional achievement. The initial randomized controlled trials were designed and executed to yield credible and clear results which enabled emergency use authorization around the world. The important questions that come next will be answered using other data sources.
References
Mehrotra DV, Janes HE, Fleming TR, Annunziato PW, Neuzil KM, Carpp LN, Benkeser D, Brown ER, Carone M, Cho I, Donnell D, Fay MP, Fong Y, Han S, Hirsch I, Huang Y, Huang Y, Hyrien O, Juraska M, Luedtke A, Nason M, Vandebosch A, Zhou H, Cohen MS, Corey L, Hartzel J, Follmann D, Gilbert PB. Clinical endpoints for evaluating efficacy in Covid-19 vaccine trials. Ann Intern Med. 2020. https://doi.org/10.7326/M20-6169.
Polak FP, Thomas SJ, Kitchin N, Absalon J, Gurtman A, Lockhart S, Perez JL, Gonzalo Perez M, Moreira E, Zerbini C, Bailey R, Swanson KA, Roychoudhury S, Koury K, Li P, Kalina WV, Cooper D, Frenck RW, Hammitt LL, Tureci O, Nell H, Schaefer A, Unal S, Tresnan DB, Mather S, Dormitzer PR, Sahin U, Jansen KU, Gruber WC, for the C4591001 Clinical Trial Group. Safety and efficacy of the BNT162b2 mRNA covid-19 vaccine. NEJM. 2020;383:2603–15.
Voysey M, Costa Clemens SA, Madhi SA, Weckx LY, Folegatti PM, Aley PK, Angus B, Baillie VL, Barnabas SL, Bhorat QE, Bibi S, Briner C, Cicconi P, Collins AM, Colin-Jones R, Cutland CL, Darton TC, Dheda K, Duncan CJA, Emary KRW, Ewer KJ, Fairlie L, Faust SN, Feng S, Ferreira DM, Finn A, Goodman AL, Green CM, Green CA, Heath PT, Hill C, Hill H, Hirsch I, Hodgson SHC, Izu A, Jackson S, Jenkin D, Joe CCD, Kerridge S, Koen A, Kwatra G, Lazarus R, Lawrie AM, Lelliott A, Libri V, Lillie PJ, Mallory R, Mendes AVA, Milan EP, Minassian AM, McGregor A, Morrison H, Mujadidi YF, Nana A, O’Reilly PJ, Padayachee PD, Pittella A, Plested E, Pollock KM, Ramasamy MN, Rhead S, Schwarzbold AV, Singh N, Smith A, Song R, Snape MD, Sprinz E, Sutherland RK, Tarrant R, Thomson EC, Török ME, Toshner M, Turner DPJ, Vekemans J, Villafana TL, Watson ME, Williams CJ, Douglas AD, Hill AVS, Lambe T, Gilbert SC, Pollard AJ, on behalf of the Oxford COVID Vaccine Trial Group. Safety and efficacy of the ChAdOx1 nCoV-19 vaccine (AZD1222) against SARS-CoV-2: an interim analysis of four randomised controlled trials in Brazil, South Africa, and the UK. Lancet. 2020. https://doi.org/10.1016/S0140-6736(20)32661-1.
Baden LR, El Sahly HM, Essink B, Kotloff K, Frey S, Novak R, Diemert D, Spector SA, Rouphael N, Creech CB, McGettigan J, Khetan S, Segall N, Solis J, Brosz A, Fierro C, Schwartz H, Neuzil K, Corey L, Gilbert P, Janes H, Follmann D, Marovich M, Mascola J, Polakowski L, Ledgerwood J, Graham BS, Bennett H, Pajon R, Knightly C, Leav B, Deng W, Zhou H, Han S, Ivarsson MJ, Zaks T, for the COVE Study Group. Efficacy and Safety of the MRNA-1273 SARS-CoV-2 vaccine. NEJM. 2020. https://doi.org/10.1056/NEJMoa2035389.
Pfizer. Vaccine and related biological products advisory committee (December 10, 2020) Briefing Document. 2020. fda.gov/media/144246/download. Accessed 29 Dec 2020.
Moderna. Vaccine and related biological products advisory committee (December 17, 2020) Briefing Document. 2020. fda.gov/media/144246/download. Accessed 29 Dec 2020.
Lipsitch M, Dean N. Understanding COVID-19 vaccine efficacy. Science. 2020;370:763–5. https://doi.org/10.1126/science.abe5938A.
Plotkin SA, Gillbert PB. Nomenclature for immune correlates of protection after vaccination. Clin Infect Dis. 2012;54:1615–7.
Marchenko O, Russek-Cohen E, Levenson M, Zink RC, Krukas-Hampel MR, Jiang Q. Sources of safety data and statistical strategies for design and analysis: real world insights. TIRS. 2018;52:170–86.
Woodcock J, Lavange LM. Master protocols to study multiple therapies, multiple diseases or both. NEJM. 2017;377:62–70.
Natanegara F, Zariffa N, Buenconsejo J, Liao R, Cooner F, Lakshminarayanan D, Ghosh S, Schindler JS, Gamalo M. Statistical opportunities to accelerate development for COVID-19 therapeutics. Stat Biopharm Res. 2020. https://doi.org/10.1080/19466315.2020.1865195.
Hayes RJ, Moulton LH. Cluster Randomized Trials. 2nd ed. Boca-Raton: CRC Press; 2017. p. 33487.
Shay DK, Chillarige Y, Kelman J, Forshee RA, Foppa IV, Wernecke M, Lu Y, Ferdinands JM, Iyengar A, Fry AM, Worrall C, Izurieta HS. Comparative effectiveness of high-dose versus standard-dose influenza vaccines among US medicare beneficiaries in preventing postinfluenza deaths during 2012–2013 and 2013–2014. J Inf Dis. 2017;215:510–7.
Sandhu K, Hua W, MaCurdy TE, Franks RL, Avagyan A, Kelman J, Worrall CM, Ball R, Nguyen M. Near real time surveillance for Guillian-Barre syndrome after influenza vaccination among the medicare population 2010/2011 to 2013/2014. Vaccine. 2017;35:2986–92.
FDA. Biologics Effectiveness and Safety System (BEST). 2020. https://www.fda.gov/vaccines-blood-biologics/safety-availability-biologics/cber-biologics-effectiveness-and-safety-best-system. Accessed 18 Jan 2021.
FDA. C19-Vaccine-Safety-Protocol-2020.pdf. 2020. https://www.bestinitiative.org/vaccines-and-allergenics. Accessed 18 Jan 2021.
WHO. Covid-19 Vaccines: Safety Surveillance Manual. 2020. who.int/vaccine_safety/committee/introduction.pdf Accessed 18 Jan 2021.
Chandler RE. Optimizing safety surveillance for COVID-19 vaccines. Nat Rev Immunol. 2020;20:451–2. https://doi.org/10.1038/s41577-020-0372-8.
AMA COVID-19 CPT vaccine and immunization codes https://www.ama-assn.org/practice-management/cpt/covid-19-cpt-vaccine-and-immunization-codes. Accessed 19 Jan 2021.
CDC Vaccine Safety Datalink https://www.cdc.gov/vaccinesafety/ensuringsafety/monitoring/vsd/index.html Accessed 19 Jan 2021.
MEDDRA. 2020 https://www.meddra.org/COVID-19-Related-Terms-MedDRA-23-0-UPDATE-Spreadsheet
Yu W, Zheng C, Xie F, Chen W, Mercado C, Sy LS, Qian L, Glenn S, Tseng HF, Lee G, Duffy J, McNeil MM, Daley MF, Crane B, McLean HQ, Jackson LA, Jacobsen SJ. The use of natural language processing to identify vaccine-related anaphylaxis at five health care systems in the Vaccine Safety Datalink. Pharmacoepi Drug Saf. 2020;29:182–8.
Yih WK, Maro JC, Nguyen M, Baker MA, Balsbaugh C, Cole DV, Dashevsky I, Mba-Jonas A, Kulldorff M. Assessment of Quadrivalent human papillomavirus vaccine safety using the self-controlled tree temporal scan statistic signal detection method in the sentinel system. Am J Epi. 2017;187:1269–76.
World Health Organization. Uppsala Monitoring Centre. 2021. https://www.who-umc.org. Accessed Jan 18 2021.
Duke University. 2020 https://medschool.duke.edu/about-us/news-and-communications/med-school-blog/dcri-verily-and-hero-registry-support-long-term-safety-study-covid-19-vaccine Accessed 18 Jan 2021.
Lane JCE, Weaver J, Kostka K, Duarte-Salles T, Abrahao MTF, Alghoul H, Alser O, Alshammari TM, Biedermann P, Banda JM, Burn E, Casajust P, Conover MM, Culhane C, Davydov A, Duvall SL, Dymshyts D, Fernandez-Bertolin K, Fišter K, Hardin J, Hester L, Hripcsak G, Kaas-Hansen BS, Kent S, Khosla S, Kolovos S, Lambert CG, Lei J, Lynch KE, Makadia R, Margulis AV, Matheny ME, Mehta P, Morales DR, Morgan-Stewart H, Mosseveld M, Newby D, Nyberg F, Ostropolets A, Park RW, Prats-Uribe A, Rao GA, Reich C, Reps J, Rijnbeek P, Sathappan SMK, Schuemie M, Seager S, Sena AG, Shoaibi A, Spotnitz M, Suchard MA, Torre CO, Vizcaya D, Wen H, de Wilde M, Xie J, You SC, Zhang L, Zhuk O, Ryan P, Prieto-Alhambra D, on behalf of the OHDSI-COVID-19 consortium. Risk of hydroxychloroquine alone and in combination with azithromycin in the treatment of rheumatoid arthritis: a multinational, retrospective study. Lancet Rheumatol. 2020;2(11):e698–711.
OHDSI. 2020. https://www.ohdsi.org/. Accessed 18 Jan 2021.
R-U/FOCR. 2020. https://evidenceaccelerator.org/ . Accessed 18 Jan 2021.
von Elm E, Altman DG, Egger M, Pocock SJ, Gotzsche PC, Vandenbroucke JP. The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) Statement: guidelines for reporting observational studies. Ann Intern Med. 2007;147(8):573–7.
Funding
No funding was received for either author in the development of this article.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The views expressed are their own, independent of any other association they have.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Zariffa, N., Russek-Cohen, E. Vaccines After an Emergency Use Authorization (EUA): Modern Evidence Generation Approaches. Ther Innov Regul Sci 55, 866–871 (2021). https://doi.org/10.1007/s43441-021-00290-z
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s43441-021-00290-z