Introduction

The global rollout of vaccines against severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) offers a glimmer of hope toward ending the coronavirus disease 2019 (COVID-19) pandemic. As of the time of writing, there have been with more than 1.7 billion people worldwide received at least one dose of any COVID-19 vaccine, and over 790 million people worldwide are fully vaccinated (Our World in Data 2021).

The phase 3 randomized controlled trial of the BNT162b2 mRNA vaccine against SARS-CoV-2 demonstrated the efficacy of 95% in preventing symptomatic COVID-19, which has led to the emergency conditional approval of the vaccine in many countries (Polack et al. 2020). However, it should be noted that the clinical trial was performed in a highly controlled setting that may not simulate the real-world mass rollout of COVID-19 vaccines.

Therefore, it is imperative to determine the population-level vaccine effectiveness from the mass vaccination campaigns and to report data on the safety aspects of vaccines. This paper aims to summarize through meta-analyses the overall effectiveness of the BNT162b2 mRNA vaccine from large observational studies, which could be important to inform the development of the public health policy related to mass vaccination.

Methods

A systematic literature search with no language restriction was performed in electronic databases, including PubMed, Google Scholar, Scopus, and preprint servers (medRxiv, Research Square, SSRN), to identify eligible studies published up to June 05, 2021. The search strategy was built based on the following keywords and MeSH terms: “BNT162b2”, “Pfizer”, “BioNTech”, “mRNA vaccine”, “mRNA vaccination”, and “effectiveness”. The reference lists of relevant articles were also reviewed to retrieve additional studies. Two investigators (CSK and SSH) independently performed the literature screening to identify eligible studies.

Studies eligible for inclusion were observational studies of any design (case–control, case–cohort, and prospective cohort), which reported the effectiveness of the BNT162b2 mRNA vaccine to prevent reverse transcription–polymerase chain reaction (RT-PCR) confirmed COVID-19 (through comparison between vaccinated and unvaccinated individuals) and adjusted for covariates. For two or more studies which utilized the same data source for their investigations on vaccine effectiveness, we included the one that performed analysis on the latest record. We excluded randomized trials, studies that reported unadjusted effectiveness estimates, studies that reported only non-specific outcomes such as COVID-19-related mortality or COVID-19-related hospitalization, studies where RT-PCR did not use to confirm the diagnosis of COVID-19, and studies that reported vaccine effectiveness against a specific variant(s) of SARS-CoV-2.

Our outcome of interest, namely vaccine effectiveness, is defined as a relative reduction in RT-PCR risk confirmed COVID-19 in vaccinated individuals compared with unvaccinated individuals (Weinberg and Szilagyi 2010). Each included study was independently evaluated by two investigators (CSK and SSH), who also extracted the study characteristics. Study characteristics extracted had the first author’s surname, study design, country, sample population, number of participants, the incidence of COVID-19 in both vaccinated and unvaccinated individuals, and adjusted vaccine effectiveness estimates and covariates adjusted. Two investigators (CSK and SSH) assessed the quality of included observational studies using the Newcastle–Ottawa Scale, with a score of > 7 indicating high quality (Wells et al. 2013).

Meta-analyses with the random-effects model were used to calculate the pooled hazard ratio (HR), pooled incidence rate ratio (IRR), or pooled odds ratio (OR) at 95% confidence intervals, comparing the incidence of RT-PCR confirmed COVID-19 in vaccinated participants relative to unvaccinated participants, when there were three or more studies reporting the same type of effect measure (either HR, IRR, or OR). The vaccine effectiveness was indicated as (pooled HR − 1)/HR, (pooled IRR − 1)/IRR or (pooled OR − 1)/OR, together with a 95% confidence interval. We examined the heterogeneity between studies using the I2 statistics and the χ2 test, with significant heterogeneity set at > 50% and P < 0.10. All analyses were performed using Meta XL, version 5.3 (EpiGear International, Queensland, Australia).

Results

Our literature search yielded 712 abstracts. After deduplication and application of the eligibility criteria, 38 relevant articles were shortlisted for inclusion through full-text examination (Fig. 1). Of these, 19 studies were excluded since they either did not report vaccine effectiveness, reported non-specific outcomes such as COVID-19-related mortality and COVID-19-related hospitalization, or reported unadjusted effectiveness estimates. Therefore, 19 studies (Angel et al. 2021; Björk et al. 2021; Cabezas et al. 2021; Chung et al. 2021; Dagan et al. 2021; Emborg et al. 2021; Fabiani et al. 2021; Glampson et al. 2021; Gras-Valentí et al. 2021; Haas et al. 2021; Hall et al. 2021; Lopez Bernal et al. 2021; Mason et al. 2021; Monge et al. 2021; Pritchard et al. 2021; Regev-Yochay et al. 2021; Shrotri et al. 2021; Swift et al. 2021; Thompson et al. 2021) were included for this meta-analysis; 12 studies (Chung et al. 2021; Dagan et al. 2021; Emborg et al. 2021; Fabiani et al. 2021; Glampson et al. 2021; Gras-Valentí et al. 2021; Haas et al. 2021; Lopez Bernal et al. 2021; Mason et al. 2021; Monge et al. 2021; Regev-Yochay et al. 2021) were retrospective in design with seven database reviews (Dagan et al. 2021; Emborg et al. 2021; Glampson et al. 2021; Haas et al. 2021; Mason et al. 2021; Monge et al. 2021; Swift et al. 2021), three retrospective case–control studies (Chung et al. 2021; Gras-Valentí et al. 2021; Lopez Bernal et al. 2021), and two retrospective cohort studies (Fabiani et al. 2021; Regev-Yochay et al. 2021); the remaining seven studies (Björk et al. 2021; Cabezas et al. 2021; Hall et al. 2021; Menni et al. 2021; Shrotri et al. 2021; Thompson et al. 2021; Pritchard et al. 2021) were prospective cohort studies (n = 6) (Cabezas et al. 2021; Hall et al. 2021; Menni et al. 2021; Shrotri et al. 2021; Thompson et al. 2021; Pritchard et al. 2021) and prospective database review (n = 1) (Björk et al. 2021). The included studies (Björk et al. 2021; Dagan et al. 2021; Fabiani et al. 2021; Glampson et al. 2021; Haas et al. 2021; Hall et al. 2021; Mason et al. 2021; Menni et al. 2021; Monge et al. 2021; Thompson et al. 2021; Pritchard et al. 2021) were originated from 8 countries: the United Kingdom (n = 6) (Glampson et al. 2021; Hall et al. 2021; Lopez Bernal et al. 2021; Mason et al. 2021; Pritchard et al. 2021; Shrotri et al. 2021), the United States (n = 2) (Swift et al. 2021; Thompson et al. 2021), Canada (n = 1) (Chung et al. 2021) Sweden (n = 1) (Björk et al. 2021), Israel (n = 4) (Angel et al. 2021; Dagan et al. 2021; Haas et al. 2021; Regev-Yochay et al. 2021), Italy (n = 1) (Fabiani et al. 2021), Denmark (n = 1) (Emborg et al. 2021), and Spain (n = 3) (Cabezas et al. 2021; Gras-Valentí et al. 2021; Monge et al. 2021). Study characteristics are depicted in Table 1. The included studies (Angel et al. 2021; Björk et al. 2021; Cabezas et al. 2021; Chung et al. 2021; Dagan et al. 2021; Emborg et al. 2021; Fabiani et al. 2021; Glampson et al. 2021; Gras-Valentí et al. 2021; Haas et al. 2021; Hall et al. 2021; Lopez Bernal et al. 2021; Mason et al. 2021; Monge et al. 2021; Pritchard et al. 2021; Regev-Yochay et al. 2021; Shrotri et al. 2021; Swift et al. 2021; Thompson et al. 2021) are deemed moderate-to-good quality with Newcastle–Ottawa Scale ranging from 7 to 8.

Fig. 1
figure 1

PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-analyses) flow diagram of process of study selection

Full size image
Table 1 Characteristics of included studies
Full size table

The meta-analysis of eight studies (Cabezas et al. 2021; Emborg et al. 2021; Fabiani et al. 2021; Glampson et al. 2021; Hall et al. 2021; Monge et al. 2021; Shrotri et al. 2021; Thompson et al. 2021)  which presented effect measure as HR revealed significant protective effect against RT-PCR confirmed COVID-19 14 days or more after the first dose of BNT162b2 mRNA vaccine (pooled HR = 0.58; 95% confidence interval: 0.45–0.75; Fig. 2), where pooled estimate indicates vaccine effectiveness of 42% (95% confidence interval 25%–55%). Similarly, the meta-analysis of five studies (Björk et al. 2021; Dagan et al. 2021; Haas et al. 2021; Mason et al. 2021; Swift et al. 2021) which presented effect measure as IRR revealed significant protective effect against RT-PCR confirmed COVID-19 14 days or more after the first dose of BNT162b2 mRNA vaccine (pooled IRR = 0.47; 95% confidence interval: 0.32–0.68; Fig. 3), where pooled estimate indicates vaccine effectiveness of 53% (95% confidence interval 32%–68%).

Fig. 2
figure 2

Pooled hazard ratio (HR) of the incidence of COVID-19 14 as well as 21 days post first dose of vaccine (A) and 7 as well as 14 days post second dose of vaccine (B) relative to no vaccination

Full size image
Fig. 3
figure 3

Pooled incident rate ratio (IRR) of the incidence of COVID-19 14 as well as 21 days post first dose of vaccine (A) and 7 as well as 14 days post second dose of vaccine (B) relative to no vaccination

Full size image

Even higher vaccine effectiveness was observed 21 days or more after the first dose of BNT162b2 mRNA vaccine, where the meta-analysis of six studies (Emborg et al. 2021; Fabiani et al. 2021; Glampson et al. 2021; Hall et al. 2021; Monge et al. 2021; Shrotri et al. 2021) which presented effect measure as HR reported pooled HR of 0.42 (95% confidence interval: 0.31–0.57; Fig. 2), and thus vaccine effectiveness of 58% (95% confidence interval: 43%–69%). Likewise, the meta-analysis of three studies (Björk et al. 2021; Dagan et al. 2021; Mason et al. 2021) which presented effect measure as IRR reported pooled IRR of 0.41 (95% confidence interval: 0.36–0.47; Fig. 3), and thus vaccine effectiveness of 59% (95% confidence interval: 53–64%).

The recipient of the second dose of the BNT162b2 mRNA vaccine further boosted the vaccine effectiveness. The meta-analysis of three studies (Emborg et al. 2021; Fabiani et al. 2021; Hall et al. 2021) which presented effect measure as HR reported pooled HR of 0.18 (95% confidence interval: 0.16–0.20; Fig. 2) 7 days or more after the second dose, and thus vaccine effectiveness of 82% (95% confidence interval: 80–84%). Similarly, the meta-analysis of five studies (Angel et al. 2021; Björk et al. 2021; Dagan et al. 2021; Haas et al. 2021; Regev-Yochay et al. 2021) which presented effect measure as IRR revealed significant protective effect against RT-PCR confirmed COVID-19 7 days or more after the second dose of BNT162b2 mRNA vaccine (pooled IRR = 0.09; 95% confidence interval: 0.05–0.19; Fig. 3), where pooled estimate indicates vaccine effectiveness of 91% (95% confidence interval 81%–95%). The findings from the meta-analysis of three studies (Chung et al. 2021; Lopez Bernal et al. 2021; Pritchard et al. 2021) which presented effect measure as OR are also consistent (pooled OR = 0.19; 95% confidence interval 0.09–0.40) and show vaccine effectiveness of 81% (95% confidence interval 60%-91%) 7 days or more after the second dose of BNT162b2 mRNA vaccine. The meta-analysis of three studies (Hall et al. 2021; Regev-Yochay et al. 2021; Thompson et al. 2021) which presented effect measure as HR reported pooled HR of 0.12 (95% confidence interval: 0.08–0.16; Fig. 2) 14 days or more after the second dose, and thus vaccine effectiveness of 88% (95% confidence interval: 84%–92%). Likewise, the meta-analysis of three studies (Angel et al. 2021; Haas et al. 2021; Swift et al. 2021) which presented effect measure as IRR revealed significant protective effect against RT-PCR confirmed COVID-19 14 days or more after the second dose of BNT162b2 mRNA vaccine (pooled IRR = 0.04; 95% confidence interval: 0.03–0.05; Fig. 3), where pooled estimate indicates vaccine effectiveness of 96% (95% confidence interval 95–97%).

Discussion

The findings of the meta-analyses align with the phase 3 randomized controlled trial (Polack et al. 2020) of BNT162b2 mRNA vaccine, though with a lower protective rate: 82% after the first dose (versus overall vaccine effectiveness of 48–55% [14–21 days or more] after the first dose in the current study; Fig. 2) and 95% (7 days or more) after the second dose (versus overall vaccine effectiveness of 86–94% [7–14 days or more] after the second dose in the current study; Fig. 3). Variability in the protective rate between clinical trial and real-world studies could stem from the difference in the definition of confirmed COVID-19; confirmed COVID-19 was defined in the clinical trial as the presence of symptoms and positive RT-PCR test for SARS-CoV-2; while the included studies of our meta-analyses, confirmed COVID-19 was defined as positive RT-PCR test for SARS-CoV-2 regardless of the presence of symptoms.

In addition, individuals with comorbidities (e.g., hypertension, diabetes, and obesity) who are predisposed to severe COVID-19 constituted only about one-fifth of the study population in phase 3 randomized controlled trial (Polack et al. 2020) of BNT162b2 mRNA vaccine. Individuals with comorbidities (e.g., hypertension, diabetes, and obesity), especially those with old age, are often prioritized in the real-world mass vaccination campaign. Therefore, this could explain the lack of reproducible vaccine efficacy reported from the highly controlled clinical research settings compared to the real-world settings since these individuals with comorbidities mainly constituted the real-world study population. Indeed, elderly individuals with comorbidities often have diminished immune responses to vaccines (Kwetkat and Heppner 2020).

Nevertheless, with up to 59% of real-world protective rate after the administration of the first dose of the BNT162b2 mRNA vaccine, it seems reasonable to delay the administration of the second dose in an attempt to allow vaccination in a higher proportion of individuals to reduce the risk of transmission of COVID-19 to an acceptable level. Our study was limited by the fact that included studies were originated in only a few countries. Therefore, the generalizability of our findings is unknown, especially to the countries where variants of concern of SARS-CoV-2 are circulating. Future studies should aim to investigate the vaccine effectiveness against different variants of concern of SARS-CoV-2 and with longer follow-ups to determine the duration of protection against COVID-19. Furthermore, the effectiveness of the BNT162b2 mRNA vaccine among immunocompromised individuals as well as individuals who receive treatment with immunosuppressive therapy should also be investigated since they had been excluded from the participation of phase 3 randomized controlled trial (Polack et al. 2020). Despite its effectiveness, reporting vaccine safety data by relevant stakeholders should be encouraged as BNT162b2 mRNA is a new vaccine that has not gained full approval.