Background

Since emerging of coronavirus disease 2019 (COVID-19) caused by a novel severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) in December 2019, more than 452 million cases and 6.0 million deaths have been documented worldwide as of 12 March 2022 [1]. COVID-19 vaccines have been rapidly developed and proved to be highly effective in multiple randomized clinical trials (RCTs) [2,3,4,5] and observational studies [6,7,8]. As of 5 March 2022, more than 10 billion vaccine doses have been administered all over the world, but around 150 thousand new cases are diagnosed each day [1]. Most current vaccines used SARS-CoV-2 spike protein as the key antigenic target based on the originally identified Wuhan lineage virus [9]. The B.1.1.7 (Alpha) variant was first identified from genomic sequencing of samples obtained from COVID-19 patients which accounted for an expanding proportion of cases in England in late 2020 [10]. Subsequently, the emergence of the B.1.351 (Beta) variant in South Africa and the P.1 (Gamma) variant in Brazil increased the COVID-19 pandemic. In December 2020, a novel SARS-CoV-2 variant, the B.1.617.2 (Delta) variant was first detected in India, causing a sharp increase in COVID-19 cases and deaths in India and surrounding countries [11]. Recently, the B.1.1.529 (Omicron) variant emerged in December 2021 contains more than 30 mutations in the spike protein, raising concerns for naturally acquired or vaccinated population [12]. The emerging Alpha, Beta, Gamma, Delta, and Omicron variants were classified as variants of concern (VOC), which were associated with the transmission increasing, more severe disease situation (e.g., increased hospitalizations or deaths), significant reduction in neutralization by antibodies generated during previous infection or vaccination, reduced effectiveness of treatments or vaccines, or diagnostic detection failures [13,14,15,16,17,18]. The importance of vaccination programs and efficient public health measures will be increased if VOC have increased transmissibility or virulence [19]. It was urgent and necessary to synthesize evidence of the vaccine effectiveness (VE) of COVID-19 vaccines against VOC. To our knowledge, there are some studies evaluating the VE of COVID-19 vaccines against VOC [20,21,22,23]. Some relevant systematic review or meta-analysis about COVID-19 vaccines against Delta variant have been published to date [24,25,26], which did not include many recent studies as the most recent retrieval date was October 2021. Therefore, to gain insight in the VE of COVID-19 vaccines against five kinds of VOC, we conducted a comprehensive systematic review and meta-analysis including both RCTs and observational studies. This review of the VE of COVID-19 vaccines against VOC will support global response on public health measures and vaccination programs timely and evidence based.

Methods

Data sources and searches

We conducted this systematic review according to the Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) guidelines [27]; the protocol was registered on PROSPERO (CRD42021273986). We searched for literature published on PubMed, Embase, Cochrane Library, and the ClinicalTrials.gov website on or before 4 March 2022. Keywords including “COVID-19,” “SARS-CoV-2,” “vaccine,” and “variant” were used to search; the detailed search strategy was shown in the Additional file 1 (Appendix S1). Additionally, we identified references by searching the reference lists of included studies and relevant reviews.

Selection of studies

We included randomized controlled trials (RCTs), cohort studies, and case-control studies that evaluated the efficacy or effectiveness of COVID-19 vaccines against VOC including B.1.1.7 (Alpha), B.1.351 (Beta), P.1 (Gamma), B.1.617.2 (Delta), and B.1.1.529 (Omicron). Studies enrolling general population or special populations (e.g., healthcare workers) aged 12 years or older were included. For studies that only reported VE against SARS-CoV-2 infections (without subgroup analysis of VOC), but the specific VOC accounted for 50% or more among positive tests, they were also included in the analysis. We excluded study protocols, editorials, comments, reviews, news, case reports, conference abstracts, animal studies, in vivo experiments, and analysis of antibody neutralization. Searches were limited to English articles. The primary outcome was the VE of full vaccination against VOC; the studies which only reported the VE of partial vaccination were excluded.

Data extraction

Two authors reviewed titles and abstracts independently to identify eligible studies that met pre-specified inclusion criteria and extracted data. When consensus was lacking, a third reviewer was consulted. The journal name, study type, study location, vaccine information, number of participants, characteristics of subjects, and outcomes were extracted from eligible studies. We extracted SARS-CoV-2 infection information if results on both SARS-CoV-2 infection and symptomatic infection were reported. The adjusted VE or estimates of effect size (relative risks, incidence rate ratios, or odds ratios) with corresponding 95% confidence intervals (CIs) were extracted with priority. The risk of bias of RCTs was assessed using the Cochrane Collaboration’s tool [28, 29]. The risk of bias of cohort and case-control studies was assessed using the Newcastle-Ottawa scale (NOS) [30].The NOS contains 8 categories relating to methodological quality, with a maximum of 9 points. A total score of 7–9 points is considered of good quality, while a score of 4–6 points of moderate quality, and a score of 1–3 points of low quality. Two investigators reviewed the studies and judged the risk of bias.

Statistical analysis

Pooled estimates and 95% CIs were calculated using DerSimonian and Laird random-effects meta-analysis [31]. Summary VE was defined as (1-pooled estimate) ×100%. We performed subgroup analysis stratifying by study design, vaccine type, participant, and publication. P for the difference was calculated using random-effects meta-regression, a difference between the estimates of these subgroups was considered significant if pinteraction < 0.10 [32]. Statistical heterogeneity between the studies was assessed with the χ2 test and the I2 statistics. I2 values of 25%, 50%, and 75% have been suggested to be indicators of low, moderate, and high heterogeneity, respectively [33]. All the analyses were performed with STATA 14.

Results

Literature search and study characteristics

This systematic literature search identified 6740 publications; after excluding duplicates and irrelevant papers, 219 published reports were evaluated in full text for eligibility (Additional file 1: Figure S1). Finally, 57 articles were included in the present systematic review [6, 7, 20,21,22,23, 34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63,64,65,66,67,68,69,70,71,72,73,74,75,76,77,78,79,80,81,82,83,84]. There were different study designs for included studies, 11 RCTs (161,388 participants) [20, 22, 23, 34,35,36,37,38,39,40,41], 20 cohort studies (52,782,321 participants) [6, 7, 42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59], and 26 case-control studies (2,584,732 cases) [21, 60,61,62,63,64,65,66,67,68,69,70,71,72,73,74,75,76,77,78,79,80,81,82,83,84]. In total, 11 COVID-19 vaccines (mRNA-1273, BNT162b2, ChAdOx1, Ad26.COV2.S, NVX-CoV2373, BBV152, CoronaVac, BBIBP-CorV, SCB-2019, CVnCoV, and HB02) and 5 VOC (Alpha, Beta, Gamma, Delta, and Omicron) were included in this study. BNT162b2, mRNA-1273, and CVnCoV are mRNA vaccines; CoronaVac, HB02, BBV152, and BBIBP-CorV are inactivated vaccines; Ad26.COV2.S and ChAdOx1 are non-replicating vector vaccines; and NVX-CoV2373 and SCB-2019 are protein subunit vaccines. Only Ad26.COV2.S is a single-dose vaccine; therefore, a one-dose regimen is regarded as full vaccination. Characteristics of individual studies are summarized in Table 1.

Table 1 Study characteristics and participants demographics
Full size table

Risk of Bias

All the RCTs were assessed as some concerns for overall risk-of-bias judgment. Fifteen of 20 cohort studies were judged as good quality, and the remaining 5 studies were moderate quality. For 26 case-control studies, 22 were considered as good quality and 4 were moderate quality. The detailed risk of bias assessment is available in Additional file 1 (Tables S1–S3).

Vaccine effectiveness of COVID-19 vaccines against B.1.1.7 (Alpha) variant

Five RCTs [23, 36, 38,39,40], 9 cohort studies [6, 7, 42, 43, 46, 48, 49, 53, 56], and 10 case-control studies [21, 60, 64,65,66,67,68, 76, 77, 80] had evaluated the VE of COVID-19 vaccines against the Alpha variant. Six COVID-19 vaccines (BNT162b2, mRNA-1273, NVX-CoV2373, ChAdOx1, Ad26.COV2.S and CVnCoV) were included in this analysis. Four studies enrolled healthcare workers [6, 48, 49, 64], one enrolled adults aged 70 or older [76], and the others enrolled the general population. Characteristics of individual studies and VE for Alpha variant are summarized in Fig. 1 and Additional file 1 (Table S4).

Fig. 1
figure 1

Forest plot showing VE of full vaccination against Alpha variant. Abbreviations: VE, vaccine effectiveness; CI, confidence interval; RCT, randomized controlled trial

Full size image

The summary VE of full vaccination against the Alpha variant was 88.0% (95% CI, 83.0–91.5) (Table 2). The VE against any infection and symptomatic infection with the Alpha variant was 89.4% (95% CI, 82.9–93.5) and 90.9% (95% CI, 84.5–94.7), respectively. Subgroup analysis by study design showed that VE was 77.9% (59.3–88.0) in 5 RCTs and 89.3% (95% CI, 84.4–92.6) in 19 real-world settings (case-control or cohort studies) (pinteraction = 0.067). The VE against Alpha variant of real-world evidence seems to be higher than RCTs. Subgroup analysis of vaccine type showed that VE was 90.1% (95% CI, 85.2–93.4) for mRNA vaccines in 20 study groups, 73.9% (95% CI, 69.9–77.4) for non-replicating vector vaccines in 6 study groups, 89.7% (95% CI, 78.8–95.0) for protein subunit vaccine in 2 study groups, and 82.0% (95% CI, 27.0–95.0) for mixed vaccines (BNT162b2/ChAdOx1) in 1 study group (pinteraction = 0.148). And we detected a significant interaction (pinteraction = 0.026) between VE and vaccine type (mRNA vaccines vs. not mRNA vaccines); the VE of mRNA vaccines seemed to be higher than others. The results of subgroup analysis for participant are shown in Table 2.

Table 2 Meta-analysis and subgroup analysis for VE of COVID-19 against VOC
Full size table

Vaccine effectiveness of COVID-19 vaccines against B.1.351 (Beta) and P.1 (Gamma) variants

Four RCTs [20, 22, 40, 41] and 6 case-control studies [60, 65,66,67, 77, 82] had evaluated the VE of COVID-19 vaccines against the Beta variant. Four RCTs [34, 35, 39, 40], 1 cohort study [58], and 4 case-control studies [72, 73, 77, 79] had evaluated the VE of COVID-19 vaccines against the Gamma variant. Both Beta and Gamma have N501Y and E484K mutations, and 2 studies used a combined Beta/Gamma group because of insufficient specimens. Eight COVID-19 vaccines (mRNA-1273, BNT162b2, NVX-CoV2373, ChAdOx1, CVnCoV, SCB-2019, CoronaVac, and Ad26.COV2.S) were included in this analysis. For the study population, 1 study enrolled health care workers [73], 2 studies enrolled older adults [72, 79], 1 studies enrolled participants from long-term care homes [58], and the others enrolled the general population. Characteristics of individual studies and VE for Beta and Gamma variants are summarized in Fig. 2 and Additional file 1 (Table S5).

Fig. 2
figure 2

Forest plot showing VE of full vaccination against Beta/Gamma variants. Abbreviations: VE, vaccine effectiveness; CI, confidence interval; RCT, randomized controlled trial

Full size image

The summary VE of full vaccination against Beta variant was 73.0% (95% CI, 64.3–79.5) (Table 2). Subgroup analysis by study design showed that VE for Beta variant was 45.6% (95% CI, −5.1 to 71.8) in 4 RCTs and 77.9% (95% CI, 72.6–82.2) in 7 real-world settings (case-control studies) (pinteraction = 0.067). The VE against Beta variant of real-world evidence seems to be higher than RCTs. Subgroup analysis of vaccine type showed that VE was 78.6% (95% CI, 73.0–83.0) for mRNA vaccines in 8 study groups, 31.0% (95% CI, 0.0–52.3) for non-replicating vector vaccines in 2 study groups, and 51.1% (95% CI, −0.6 to 76.2) for protein subunit vaccine in 1 study group (pinteraction = 0.057). The summary VE of full vaccination against Gamma variant was 63.0% (95% CI, 47.9–73.7) in 10 study groups; the results of subgroup analysis are shown in Table 2.

When Beta/Gamma variant was treated as one group, the summary VE of full vaccination was 69.3% (95% CI, 60.4–76.2) in 21 study groups. Subgroup analysis of vaccine types showed that VE against Beta/Gamma was 77.2% (95% CI, 71.3–81.8) for mRNA vaccines in 12 study groups and 50.8% (95% CI, 32.6–64.0) for not mRNA vaccines in 9 study groups (pinteraction = 0.006); the VE for mRNA vaccines seemed to be higher than others.

Vaccine effectiveness of COVID-19 vaccines against B.1.617.2 (Delta) variant

Three RCTs [34, 37, 40], 13 cohort studies [42, 44, 45, 47, 50,51,52,53,54,55,56,57, 59], and 17 case-control studies [21, 61,62,63, 66, 69,70,71, 74, 75, 77, 78, 80,81,82,83,84] had evaluated the VE of COVID-19 vaccines against the Delta variant. Ten COVID-19 vaccines (mRNA-1273, BBV152, ChAdOx1, BNT162b2, CoronaVac, Ad26.COV2.S, HB02, CNBG, SCB-2019, and BBIBP-CorV) were included in this analysis. Five studies enrolled adolescents [45, 50, 54, 63, 78], 3 studies enrolled health care workers or frontline workers [42, 44, 52], 1 study enrolled participants in the nursing home [51], and the others enrolled the general population. Three studies reported the VE of booster vaccination against Delta variant [61, 69, 84]. Characteristics of individual studies and VE for Delta variant are summarized in Figs. 3 and 4 and Additional file 1 (Table S6).

Fig. 3
figure 3

Forest plot showing VE of full vaccination against Delta variant. Abbreviations: VE, vaccine effectiveness; CI, confidence interval; RCT, randomized controlled trial

Full size image
Fig. 4
figure 4

Forest plot showing VE of full vaccination against Omicron variant, and VE of booster vaccination against Delta or Omicron variant. Abbreviations: VE, vaccine effectiveness; CI, confidence interval; RCT, randomized controlled trial

Full size image

The summary VE of full vaccination against the Delta variant was 77.8% (95% CI, 72.7–82.0) (Table 2). The VE against any infection and symptomatic infection with the Delta variant were 74.6% (95% CI, 70.0–78.5) and 79.2% (95% CI, 71.9–84.6), respectively. For special populations, the VE was 76.4% (95% CI, 50.3–88.8) in healthcare workers, 59.7% (95% CI, 45.0–70.4) in older adults, and 88.8% (95% CI, 83.5–92.4) in adolescents. Subgroup analysis of study design showed that VE for Delta variant was 59.3% (95% CI, 6.0–82.4) in 3 RCTs and 78.6% (95% CI, 73.4–82.7) in 30 real-world settings (case-control or cohort studies) (pinteraction = 0.168). Subgroup analysis by vaccine type showed that VE was 83.4% (95% CI, 79.6–86.6) for mRNA vaccines in 28 study groups, 65.0% (95% CI, 55.5–72.4) for non-replicating vector vaccines in 12 study groups, 56.7% (95% CI, 47.7–64.2) for inactivated vaccines in 6 study groups, and 78.7% (95% CI, 55.1–89.9) for protein subunit vaccine in 1 study group (pinteraction < 0.001). An interaction (pinteraction < 0.001) between VE and vaccine type (mRNA vaccines vs. not mRNA vaccines) was found; the VE for mRNA vaccines seemed to be higher than others. The summary VE of booster vaccination against the Delta variant was 95.5% (95% CI, 94.2–96.5).

Vaccine effectiveness of COVID-19 vaccines against B.1.1.529 (Omicron) variant

Three case-control studies had evaluated the VE of COVID-19 vaccines against the Omicron variant [69, 74, 84]. Two mRNA vaccines (mRNA-1273 and BNT162b2) were included in this analysis. All three studies enrolled the general population. Characteristics of individual studies and VE for Omicron variant are summarized in Fig. 4 and Additional file 1 (Table S7). The summary VE of full vaccination against the Omicron variant was 55.9% (95% CI, 40.9–67.0), and the VE of booster vaccination against the Omicron variant was 80.8% (95% CI, 58.6–91.1).

Discussion

The VOC have mutations in its spike protein; most breakthrough cases were caused by contemporary variant strains [36]. The VE of current COVID-19 vaccines against VOC is concerning; we conducted this systematic review and meta-analysis to synthesize evidence on this topic during the pandemic. This study has five main findings. First, full vaccination of COVID-19 vaccines was effective against Alpha, Beta, Gamma, Delta, and Omicron variants, with the VE of 88.0%, 73.0%, 63.0%, 77.8%, and 55.9%, respectively. Second, booster vaccination has higher VE against Delta and Omicron variants, with the VE of 95.5% and 80.8%, respectively. Third, mRNA vaccines (BNT162b2 or mRNA-1273) have higher VE against VOC over other vaccines. Fourth, VE against VOC of real-world evidence seemed to be higher than RCTs. Fifth, more evidence was needed to evaluate the VE of COVID-19 against the Omicron variant. To our knowledge, our study is the first comprehensive systematic review and meta-analysis to characterize the VE of COVID-19 vaccines against five kinds of VOC.

The evidence for the Omicron variant was insufficient, which only included three studies evaluating the VE of the mRNA vaccines (BNT162b2 or mRNA-1273). WHO guidelines recommend a lower bound of at least 30% and a vaccine efficacy of at least 50% [85]. The summary VE against Omicron variant was 55.9% of full vaccination and 80.8% of booster vaccination, raising concern for other vaccines. One study showed that Omicron variant extensively but incompletely escaped BNT162b2 neutralization [86]. Owing to multiple spike mutations, over 85% of tested neutralizing antibodies were escaped by Omicron variant, presenting a serious threat to existing therapies and COVID-19 vaccines [12, 87].

The main results in this study were in consistent with a recent meta-analysis for neutralizing antibodies against SARS-CoV-2 variants, which showed that Alpha, Beta, Gamma, and Delta variants significantly escaped natural-infection-mediated neutralization, with an average of 1.4-fold, 4.1-fold, 1.8-fold, and 3.2-fold reduction in live virus neutralization assay [88]. Despite the reduction in neutralization titers against Alpha variant, they remain robust, and there is no evidence of vaccine escape in one study [89]. Escape of Beta variant from neutralization by convalescent plasma and vaccine-induced sera was observed in some studies [13, 90, 91]. Although neutralization titers against Gamma variant are reduced, it is hoped that immunization with vaccines designed against parent strains will protect Gamma variant infection [92]. The Delta variant escapes neutralization by some antibodies that target the receptor-binding domain or N-terminal domain; the neutralization titers against Delta were three to fivefolds than Alpha variant when two-dose of the vaccine administrated [15]. This study also supports the two-dose vaccine regimen recommended by the FDA and EMA, which is consistent with an in vitro study for SARS-CoV-2 variants of concern [93]. Also, booster vaccination demonstrates high VE against Delta infection in our study.

We did not evaluate VE against asymptomatic infection due to poor reporting in included studies. The summary VE against asymptomatic infection was slightly higher than any infection for Alpha, Gamma, and Delta variants, which was consistent with primary studies. The summary VE was higher against any infection with the Beta variant, which was probably confounded by study design and vaccine type. Three of 4 RCTs used symptomatic infection as an outcome, but 5 of 6 case-control studies used any infection. Most studies enrolled general population; only a few studies analyzed the VE in older adults or adolescents. We have performed subgroup analyses for VE against Delta variant stratifying by participants; the VE was 59.7% in older adults which was lower than general population (75.5%), healthcare workers (76.4%), and adolescents (88.8%). Vaccine type may be a confounder for this analysis, because one study showed that the VE of Ad26.COV2.S was much lower than BNT162b2 and mRNA-1273 in adolescents [63]. More evidence is needed for evaluating the VE against VOC in special population.

This review included 3 study designs evaluating 11 COVID-19 vaccines against 5 VOC in different populations. There is high heterogeneity between studies, and high statistical heterogeneity is also observed in most analysis. Other factors like the definition of outcomes (all SARS-CoV-2 or symptomatic infection), days after vaccination, and participant’s characteristics (e.g., age and race) may also contribute to the heterogeneity. Therefore, we mainly performed narrative descriptive synthesis.

This study has some limitations. First, 19% of studies (11 of 57) are nonrandomized. The imbalance between groups in observational studies is a concern, so potential selection bias may be existent. Second, we did not evaluate VE against asymptomatic infection due to poor reporting in included studies. Third, although we performed qualitative analysis by different stratifications, heterogeneity was still high in most quantitative analysis. Fourth, VE against hospitalization or death related to VOC is not included in our analysis. Finally, the evidence of COVID-19 vaccines against Omicron variant is not enough, more research is needed in the future.

Conclusions

Full vaccination of COVID-19 vaccines is highly effective against Alpha variant and moderate effective against Beta, Gamma, and Delta variant. Booster vaccination has more effectiveness against Delta and Omicron variants. mRNA vaccines (BNT162b2 or mRNA-1273) seem to have higher VE against Alpha, Beta, Gamma, or Delta over other vaccines. SARS-CoV-2 Omicron is raising concern for vaccinated individuals, and more evidence is needed to evaluate the VE of COVID-19 vaccines against the Omicron variant.