Background

Influenza is a major public health threat. It is widely accepted that an annual influenza vaccination is the most effective way to prevent influenza [1]. Among adults, for example, the inactivated influenza vaccine was reported to have a vaccine efficacy (VE) of 59% (95% confidence interval (CI) 51 to 67) [2].

Recommendations for influenza vaccine composition currently include two type A influenza strains (A-H3N2 and A-H1N1) and one type B influenza strain, which are updated annually. Strains are selected by scientists convened by the World Health Organization (WHO) [3] in the months before the next epidemic season is expected. This procedure includes consideration of antigenic mismatch between vaccine strains and actual epidemic strains.

Although it is accepted that matched strains provide the best protection, data on the protective efficacy of unmatched strains, or cross protection, are sparse. These data are of particular importance, given that influenza B vaccine strains did not match circulating strains in six influenza seasons between 2000 and 2011 in the USA [4]. Mismatched seasons may lead to reduced uptake of influenza vaccination, reduced VE, and more severe influenza epidemics. Therefore, estimating prevention that can be achieved during mismatched influenza seasons is of prime public health importance. This information cannot be gleaned from previous reviews on the efficacy of the influenza vaccines because they did not specifically examine vaccine efficacy for mismatched seasons [5–7]. As such, in the present work we aimed to determine the cross protection against laboratory-confirmed influenza through vaccination.

Methods

Protocol

The Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) statement was used to guide the reporting and conduct of this review [8]. A systematic review protocol was compiled and circulated to experts in influenza, systematic reviews, and statistics. The protocol was registered with PROSPERO, an international registry for systematic reviews (CRD42012001926) and published in an open-access journal [9].

Eligibility criteria

Studies reporting the incidence of influenza infection after vaccination among healthy individuals were included. The primary outcome was the incidence of laboratory-confirmed influenza verified by polymerase chain reaction (PCR) or viral culture. The secondary outcome was the incidence of laboratory-confirmed influenza verified by at least a fourfold rise in hemagglutinin inhibition (HI) antibody titers at the end of the influenza season versus baseline (serologic assay) or serologic assay in combination with another detection method (for example, viral culture or PCR). Randomized clinical trials (RCTs) and quasi-RCTs (that is, use of non-random methods to allocate patients to the treatment and control groups, such as consecutive enrolment or the last digit of a health card number) comparing any influenza vaccine versus placebo and disseminated in the English language were included.

Study selection process

The eligibility criteria were pilot tested on a random sample of 50 citations and clarified, as needed. Two reviewers subsequently screened titles and abstracts (citations) from the literature search in duplicate. Conflicts were resolved through team discussion. A similar process was followed for screening potentially relevant full-text articles identified through citation screening.

Information sources and search

The full search strategy is reported elsewhere [9]. Briefly, RCTs were identified by searching three Cochrane reviews on influenza vaccines [5–7], MEDLINE, EMBASE, Cochrane Central Register of Controlled Trials, meta-Register (a clinical trials registry), and the references of included trials. The literature searches were conducted by an experienced librarian on 31 January 2012.

Data items and collection process

The data abstracted included study characteristics, participant characteristics, and number of influenza cases per treatment group, confirmed by viral culture, PCR or serologic assay. A data abstraction form was developed and pilot tested. Two reviewers subsequently abstracted all of the data in duplicate. Discrepancies were resolved by discussion. Trial authors were contacted for data clarifications and additional unpublished data was received from six included published RCTs [10–15].

Risk of bias appraisal

The Cochrane Risk of Bias seven-item tool was used to appraise the likelihood that the RCT results were affected by bias [16]. For the selective outcome reporting criterion, trial protocols were obtained and the outcomes reported in the protocol were compared to those reported in the final trial publication. For the other sources of bias criterion, industry-funded RCTs were scored as ‘unclear’, due to the potential for funding bias [17].

Methodological issues

We also assessed other methodological issues identified a priori that are related to influenza and may have influenced the RCT results. These included case definitions used by health care workers to identify influenza-like illness (ILI), use of a surveillance system to monitor influenza cases, and randomization scheme across multisite trials. Case definition of influenza-like illness was defined according to the Centers for Disease Control and Prevention (CDC) as fever >37.8°C (100°F) with cough and/or sore throat [18]. Surveillance system was defined as a system to track ILI in the population and consisted of health care workers contacting participants to monitor ILI symptoms. Randomization scheme referred to the consistent use of an allocation schedule across trial study sites for multisite trials.

Characterization of matched or mismatched vaccine strains

The process of characterizing matched versus mismatched strains was determined a priori and involved the two steps outlined below.

Step 1

Viral strains from influenza cases in the included RCTs were characterized through HI assays using vaccine strains as the reference. Influenza A strains from infected trial participants were matched with the strain in the vaccine if they belonged to the same A subtype (that is, H1N1 or H3N2) and were antigenically similar in the HI assay (that is, if they showed sufficient crossreaction in a HI chessboard table using ferret antisera; for example, with a HI typing quotient <fourfold titer). Influenza A viral strains were considered mismatched by antigenic drift if they were antigenically distinct from influenza A strains contained in the vaccine as per HI typing (for example, HI titer quotient ≥fourfold) or the characterization did not belong to a similar influenza A subtype contained in the vaccine (for example, H1N1 strains circulating but only H3N2 strains contained in the vaccine for bivalent vaccines with one H subtype).

For influenza B, the epidemiological situation is more complex. In recent years, there have been two coexisting phylogenetic influenza B lineages: B/Victoria and B/Yamagata [19, 20]. Influenza B strains from infected trial participants were considered matched if the strain belonged to the same lineage and were antigenically similar to the vaccine strain as per HI typing (for example, HI typing quotient <fourfold titer). For influenza B mismatches, two different forms were considered. Mismatch by antigenic drift refers to strains of the same lineage that were antigenically distinct from influenza B strains contained in the vaccine as per HI typing (for example, HI quotient ≥fourfold titer), whereas mismatch by lineage refers to influenza B strains of different lineages. Whenever the influenza B lineage was not presented in the trial report, categorization was based on the influenza phylogenetic tree and verified by influenza experts on the team.

Step 2

When antigenic characterization of viral strains of infected participants was not reported, data from surveillance systems were used to determine circulating strains that occurred during the time and location of the trial conduct. These included WHO weekly epidemiological records, Mortality and Morbidity Reports Weekly (MMRW), Chinese National Influenza Center, and CDC influenza summary reports. These data were sometimes captured through correspondence with trial authors.

Synthesis of included studies

The relative risk (RR) was calculated for the number of influenza cases per treatment group for included RCTs and 95% confidence intervals (CIs) were derived based on a normal approximation. Meta-analysis was conducted using a random effects model [21] for the live attenuated influenza vaccines (LAIV), trivalent inactivated vaccines (TIV), and other vaccines, separately. A post hoc sensitivity analysis was also conducted to examine the influence of categorizing the vaccines as being either an inactivated influenza vaccine or a live influenza vaccine. VE was derived based on pooled RRs using the following formula: (1 - RR) × 100%. Only influenza infections due to mismatched strains were included in the mismatched analysis, while only influenza infections due to matched strains were included in the matched analysis.

To assess for clinical and methodological heterogeneity, we examined the forest plots from the meta-analysis. The similarity between studies regarding participant and study characteristics was considered and subgroup analysis was conducted. Subgroup analyses determined a priori included age group (children <18 years of age, adults ≥18 years of age, older patients ≥65 years of age) and type of influenza (A versus B). Statistical heterogeneity was examined using the I2 statistic [22]. Funnel plots were performed to identify potential publication bias [23]. All analyses were conducted in SAS version 9.1 (SAS Inc., Cary, NC, USA).

Results

Literature search

The literature search resulted in a total of 1,356 citations, of which 308 were deemed potentially relevant (Figure 1). Reasons for exclusion at the full-text level of screening included not laboratory-confirmed influenza (108/273), no comparison to placebo (82/273), not a RCT (68/273), not a healthy population (11/273), not vaccinated against influenza A or B (3/273), and not disseminated in English (1/273). A total of 34 trials fulfilled the inclusion criteria [10–15, 24–51] plus 1 companion report [52], which was used for supplementary data only (hence a total of 35 RCTs). In all, 32 RCTs [10–15, 25–27, 29–51] and 2 quasi-RCTs [24, 28] provided data on 47 influenza seasons including 94,821 healthy participants.

Figure 1
figure 1

Study flow. This figure displays the flow of titles/abstracts and full-text studies through the systematic review.

Full size image

Trial characteristics

The trials were conducted between 1970 and 2009 (Table 1) in North America (22/34), Europe (3/34), Australia (1/34), Asia (3/34), and a mixture of multisite trials in North America, Europe, South America, Asia, and South Africa (4/34; Additional file 1). A total of 11 trials examined the TIV versus placebo [15, 28, 30, 34, 38, 39, 44–46, 49],[50], 7 examined the LAIV versus placebo [10, 35, 37, 40–43] and 6 examined both TIV and LAIV vaccines versus placebo [11, 13, 14, 31, 36, 47]. Other vaccine types were examined in several RCTs, such as monovalent inactivated [24, 25, 51], bivalent attenuated [26, 27], whole-virus vaccine [29], baculovirus-expressed HI influenza vaccine (rHAO) [12], and a trivalent recombinant HI protein vaccine [48]. One RCT examined both a TIV and bivalent vaccine compared to placebo [32].

Table 1 Antigenic characterization of viral strains
Full size table

In all, 22 RCTs (including information on 34 influenza seasons) provided data on matched influenza strains [10, 11, 13–15, 24, 29, 31, 35, 37],[39–43, 45–51], while 20 RCTs (including data from 33 influenza seasons) provided data on mismatched influenza strains [10–15, 26, 29, 31, 37, 40–47, 49, 50] (Table 1). Influenza infection was confirmed by culture in 20 RCTs [10–12, 15, 24, 26, 29, 31, 35, 37],[39–48], PCR in 3 RCTs [14, 50, 51], and culture or PCR in 2 RCTs [13, 49]. Nine RCTs included serologic assays to determine influenza infection; these were included in the secondary analysis only [25, 27, 28, 30, 32–34, 36, 38].

Participant characteristics

The RCTs included data on a total of 94,821 participants (Table 2). A total of 13 of the RCTs were conducted among children [10, 25, 30, 32, 35, 37, 39–43, 49],[50], and 2 were conducted among seniors [33, 36]. One RCT included children (15%) and adults (85%) [31], and another included adults and seniors [51]; the remaining RCTs were conducted among adults only. The percentage of female participants ranged from 22% [38] to 100% [26].

Table 2 Patient characteristics
Full size table

Risk of bias

Of the included trials, only 26% adequately reported sequence generation and 29% adequately reported allocation concealment (Figure 2). The majority of the RCTs blinded participants and physicians (65%) and addressed incomplete data (62%). Only half of the RCTs were deemed to be free from other sources of bias (44%; 56% scored unclear because they were industry funded) and 23% were free from selective outcome reporting (Table 3).

Figure 2
figure 2

Risk of bias across all studies. This figure represents the risk of bias presented in the included studies. Green refers to a low risk of bias, yellow refers to an unclear risk of bias, and red refers to a high risk of bias.

Full size image
Table 3 Risk of bias
Full size table

The trials were conducted over 40 years and the risk of bias for older studies was compared to newer studies, using 1991 as the midpoint. No trends were observed for sequence generation or allocation sequence and differences between groups were minimal. Blinding of outcome assessors and the reporting of incomplete data improved in recent studies by 17%. Furthermore, selective outcome reporting in recent studies showed an improvement of 42% in the risk of bias. There was also an increase in the proportion of RCTs scoring ‘unclear’ on other risk of bias criterion, as more trials were funded by private industry in recent years.

Methodological issues

Only 15% of trials were free of other methodological issues, which included adequate information on case definition of ILI and surveillance of influenza, as well as details regarding randomization across multiple sites [12, 38, 44, 48, 50] (Additional file 2). Almost half of trials (16 of 34) did not adequately report a case definition of ILI and the remaining studies were unclear in at least 1 of the domains for other methodological concerns.

LAIV versus placebo

The results from all LAIV versus placebo meta-analyses are presented in Additional file 3. For simplicity, we have focused on the results specific to children aged <18 years for the primary outcome.

The LAIV was protective against mismatched strains overall (11 RCTs, VE 60%, 95% CI 44% to 71%; Figure 3; Additional file 3) [10, 11, 13, 14, 31, 37, 40–43, 47], as well as among children aged 6 to 36 months (6 RCTs, VE 54%, 95% CI 28% to 71%) [10, 37, 40–43] when strains were mismatched. Similarly, the LAIV was effective in protecting against infection when circulating strains were well matched overall (12 RCTs, VE 77%, 95% CI 67% to 86%, Figure 4) [10, 11, 13, 14, 31, 35, 37, 40–43],[47]. LAIV provided significant protection among children when circulating strains matched vaccine composition (seven RCTs, VE 83%, 95% CI 75% to 88%) [10, 35, 37, 40–43].

Figure 3
figure 3

Meta-analysis of live attenuated influenza vaccine (LAIV), mismatched. This figure represents the relative risk (RR) of an influenza infection occurring when the circulating strain does not match strains contained in the LAIV.

Full size image
Figure 4
figure 4

Meta-analysis of live attenuated influenza vaccine (LAIV), matched. This figure represents the relative risk (RR) of an influenza infection occurring when the circulating strain matches strains contained in the LAIV.

Full size image

When influenza A strains did not match LAIV composition, protection against infection was statistically significant among children (five RCTs, VE 75%, 95% CI 41% to 90%) [10, 37, 40, 41, 43]. LAIV was also effective in protecting against influenza A when strains were well matched in children (six RCTs, VE 84%, 95% CI 76% to 90%) [10, 35, 40–43].

When influenza B strains were mismatched, LAIV was effective among children (five RCTs, VE 42%, 95% CI 22% to 56%) [10, 40–43]. Furthermore, LAIV was more protective against drifted B strains (two RCTs, VE 62%, 95% CI 21% to 81%) [40, 41] than lineage mismatch for influenza B (five RCTs, VE 34%, 95% CI 4−% to 59%) [10, 40–43] in children. Similarly, LAIV was effective against influenza B when circulating strains matched vaccine strains in children (four RCTs, VE 79%, 95% CI 58% to 90%) [10, 35, 41, 43].

TIV versus placebo

The results from all TIV versus placebo meta-analyses are presented in Additional file 3. For simplicity in the text, we have focused on the results specific to adults aged ≥18 years.

The TIV showed protection against mismatched strains overall (11 RCTs, VE 56%, 95% CI 43% to 66%, Figure 5) [11, 13–15, 31, 44–47, 49, 50] and among adults (9 RCTs, VE 52%, 95% CI 37% to 63%) [11, 13–15, 44–47, 49]. Similarly, TIV vaccines showed protection against matched strains overall (11 RCTs, VE 65%, 95% CI 58% to 72%, Figure 6) [11, 13–15, 31, 39, 45–47, 49, 50] and among adults (8 RCTs, VE 65%, 95% CI 54% to 73%) [11, 13–15, 45–47, 49].

Figure 5
figure 5

Meta-analysis of trivalent inactivated vaccine (TIV), mismatched. This figure represents the relative risk (RR) of an influenza infection occurring when the circulating strain does not match strains contained in the TIV.

Full size image
Figure 6
figure 6

Meta-analysis of trivalent inactivated vaccine (TIV), matched. This figure represents the relative risk (RR) of an influenza infection occurring when the circulating strain does not match strains contained in the TIV.

Full size image

In adults, TIV provided significant protection against mismatched influenza A strains (six RCTs, VE 64%, 95% CI 23% to 82%) [11, 14, 15, 45, 47, 49], and matched influenza A strains (seven RCTs, VE 61%, 95% CI 46% to 73%) [13–15, 45–47, 49]. The TIV also afforded protection against mismatched influenza B strains (eight RCTs, VE 52%, 95% CI 19% to 72%) [11, 13–15, 45–47, 49], as well as matched influenza B strains (four RCTs, VE 77%, 95% CI 18% to 94%) [11, 15, 47, 49] among adults.

Other vaccines compared to placebo

In all, 3 RCTs including 5 influenza seasons and 1,578 participants provided efficacy estimates against mismatched strains for other vaccines [12, 26, 29, 31]. Protection against mismatched strains was statistically significant (three RCTs, VE 56%, 95% CI 23% to 75%) [12, 26, 29].

Other vaccines contributed data on 5 influenza seasons from 4 RCTs including 15,592 participants for matched strains. Other vaccines provided a VE of 54% (four RCTs, 95% CI 5% to 78%) [24, 29, 48, 51].

Post hoc subgroup analysis

Our results did not change after a post hoc sensitivity analysis was conducted to examine the influence of categorizing the vaccines as being either an inactivated influenza vaccine or a live influenza vaccine versus categorizing the vaccines as LAIV, TIV, and other.

Discussion

In the present work, we conducted a systematic review to estimate the protection afforded by mismatched influenza vaccines. Our results show that mismatched vaccines can reduce the risk of PCR or culture-confirmed influenza by 60% for LAIV (95% CI 44% to 71%) and by 56% for TIV (95% CI 43% to 66%). These results suggest a benefit of vaccines in preventing laboratory-confirmed influenza even when there is a mismatch between vaccine composition and circulating strains. For matched influenza, point estimates for VE were slightly higher for both the LAIV (77%, 95% CI 67% to 86%) and the TIV (65%, 95% CI 57% to 72%), versus mismatched point estimates. However, there was substantial overlap in the CIs for matched and mismatched estimates in some instances; the impact of this overlap could not be formally tested via meta-analyses techniques.

A previous systematic review on a similar topic did not report the results separately for matched or mismatched strains [2]. A Cochrane review among healthy adults found a VE for TIV non-WHO matching strains (including when the specific strains are unknown) of 44% (95% CI 23% to 59%) [6]. This result was based on six influenza seasons from four RCTs. A VE of 68% (95% CI 44% to 81%) was observed for LAIV non-WHO matching strains (including when the specific strains are unknown), based on four point estimates from three RCTs [6]. Our review is based on an additional 13 RCTs and we have included more than double the data for both LAIV and TIV compared to the Cochrane review. Furthermore, here we have followed a rigorous process of identifying matched and mismatched data, which has not been attempted in previous reviews. Our results are consistent with those found in a pooled observational study including 5 years of data [53].

We found that the LAIV was more efficacious among children versus adults, which is likely a reflection of the difference in previously acquired influenza infections between age groups and the consequently larger amount of pre-vaccination antibody, which affects the live vaccine. However, we found higher point estimates for adults versus children for mismatched LAIV estimates. This finding might suggest that there might be a possible discrepancy in the degree of matching, which may have impacted our results. Specifically, trials conducted among children may have had a greater degree of mismatch than those conducted among adults. Unfortunately, the current analysis does not allow the degree of mismatch to be examined. For the purposes of our analysis, we dichotomized cross protection but in reality, the degree of mismatch is a continuum. The cross protection inferred by mismatch strains should be analyzed as a continuum in the future.

The results from our secondary outcome analysis were often inconsistent with our primary outcome results. This inconsistency is likely due to differences in sensitivity and specificity of different laboratory tests over time. Indeed, the most reliable diagnostic test for clinical practice is PCR so the results from our primary outcome should be considered the most valid [54, 55].

Our results are generalizable to seasonal influenza, as most of the included studies reported on this type of influenza. Only one of the included RCTs occurred during the influenza pandemic [51]. When this study was removed via sensitivity analysis, we did not observe any differences on our meta-analysis results.

We identified three RCTs reporting data among seniors [33, 36, 51], yet none reported our primary outcome of interest or provided data on mismatched influenza. This finding is consistent with previous influenza reviews, for which few RCTs were identified among this age group [2, 5]. Future RCTs are needed in this area to provide patients, healthcare providers, and health policymakers with guidance related to vaccination among seniors during seasons when the vaccine composition does not match circulating strains [5].

There are a few limitations in the evidence base summarized for this review. Many of the trials had an unclear risk of bias because of poor reporting. However, we did note improvements in risk of bias over time that may be because of uptake of the Consolidated Standards of Reporting Trials (CONSORT Statement), providing guidance on what should be reported in RCTs [56]. Enhanced reporting of outcome definitions might be due to clinical trial registry requirements for trials. We also noted that more trials in this area were funded by industry over time.

Another potential limitation is that although unpublished data was only obtained from six RCTs (Additional file 4), no data were obtained from unpublished RCTs. This finding suggests that we might be missing data from unpublished trials. However, we contacted many trial authors for unpublished data and searched trial registries (for example, meta-Register) to identify potentially relevant unpublished RCTs. Furthermore, our funnel plots did not suggest that publication bias influenced our results [23].

A critical limitation, which may have influenced our results, is the determination of mismatch between circulating strains and those found in the vaccine. Characterizing strains as antigenically similar or distinct using HI assay or ferret antisera might be insensitive, leading to misclassification of strains [57]. Residual misclassification of matching due to the limited discriminatory ability of H1 assays may have also explained some of our findings, and is a limitation of the inferences that can be made. Furthermore, the cut-off values recommended by the CDC to distinguish between match and mismatch strains using the HI assay changed during the study period across the included RCTs, although most of the studies included here used a fourfold quotient HI cut-off (Table 1). As such, some of the data from trials labeled as matched might actually have been mismatched [58], and vice versa.

Conclusions

By summarizing the point estimates and confidence intervals for cross protection, these data can help public health officials anticipate the possible infection and complications during mismatched years. Estimates of protection for mismatched influenza seasons can be used by patients who are contemplating immunization, since the LAIV and TIV have been shown to offer benefit during matched seasons, as well as mismatched seasons.