FormalPara Key Points for Decision Makers

Despite recommendations from the WHO to include herd immunity in vaccine economic evaluations in LMICs, less than one-fifth of published vaccine CEAs do so.

Including herd immunity effects in vaccine CEAs leads to more favorable CEA ratios.

In 42% of cases, including herd immunity in CEAs reduces ICERs enough to cross commonly cited LMIC cost-effectiveness benchmarks.

Vaccine value assessments should investigate cost-effectiveness with and without HI to inform decision-making in LMICs.

1 Background

Mass vaccination campaigns can greatly reduce the spread, morbidity, and mortality of many infectious diseases such as influenza [1,2,3,4], rotavirus [5, 6], and pneumococcal disease [4, 7]. Vaccines not only directly reduce the probability of transmission between susceptible and infected individuals, but they also provide indirect protection for at-risk individuals through herd immunity [8]. Herd immunity is established when a certain proportion of the population is immune to the disease in question, either through vaccination or disease exposure, and transmission is sufficiently reduced so that susceptible individuals avoid infection [9]. The threshold for the proportion of a population that needs to be immune to achieve herd immunity partly depends on the transmissibility of a disease, measured by the basic reproduction number, R0. R0 represents the number of secondary infections generated from the very first infection in the completely susceptible population, which varies from disease to disease [8]. A larger value of R0 requires the higher proportion of immune individuals to achieve herd immunity.

Public health vaccination programs, in particular those in low- and middle-income countries (LMICs), often use economic evaluations to inform funding priorities and maximize health gains. LMICs face multiple competing health challenges, such as other communicable diseases and an increasing burden of chronic diseases [10, 11]. The significance of economic evaluations of vaccine programs may be greater in LMICs because these countries must often choose between vaccination strategies and other health programs given the extremely limited resources available.

The World Health Organization (WHO) recommends that vaccine economic evaluations include herd immunity in the analyses, particularly in situations when ignoring such effects would result in a substantial difference in the estimated cost-effectiveness of the program [12]. Cost-effectiveness analyses (CEAs) typically incorporate herd immunity using two general modeling approaches [13]. The first approach is to simulate the impact of vaccination with a “dynamic model,” which can capture both direct and indirect effects (i.e., herd immunity) of vaccines by simulating pathogen transmission between susceptible individuals. The other approach is to use a “static model,” which does not reflect disease transmission because the model assumes a constant risk of infection. In this case, herd immunity is estimated by adding a fixed percentage improvement in the vaccine effectiveness model output, altering model inputs, or applying a multiplier to vaccine efficacy.

Although herd immunity is a key benefit of vaccination programs, the effects are not routinely included in cost-effectiveness studies. In a systematic review of 625 vaccine economic evaluations worldwide, only 28% included herd immunity [13]. To date, it remains unclear how much cost-effectiveness results may change when herd immunity is included, and how often study conclusions could change given various cost-effectiveness thresholds that are employed in LMICs. In most cases, we expect that including herd immunity effects would lead to more favorable estimates of cost-effectiveness. However, there are rare cases where including herd immunity in a CEA could make an intervention less cost-effective. For example, pneumococcal vaccines (e.g., PCV7, PCV10, PCV13) target at most 13 of the more than 92 pneumococcal serotypes. Achieving herd immunity among those 13 serotypes may lead to serotype replacement, which may in turn increase the non-vaccine serotype prevalence, thus reducing the benefits of vaccination. Beyond these broad illustrations, however, additional research is needed to quantify the impact of herd immunity effects on CEA results, and determine how often there may be implications for policies that are adopted based on these findings.

As the COVID-19 pandemic continues to spread globally, the numbers of confirmed cases are rising rapidly in many LMICs [14]. LMICs are particularly at risk of COVID-19 given their low-resourced health systems and multiple competing health challenges. It will be important to evaluate the economic value of COVID-19 vaccines so that LMIC decision makers will have the right information to make funding prioritization decisions. In fact, pneumococcal vaccine findings may be an important analog for COVID-19 given pneumonia presents as the most severe manifestation of disease.

The objectives of this study were to determine: (1) the frequency with which herd immunity effects have been incorporated into economic evaluations of vaccines in LMICs; (2) the characteristics of CEA studies with versus without herd immunity effects; (3) how the inclusion of herd immunity effects changes CEA results under various cost-effectiveness benchmarks by presenting ICERs with and without these effects; and (4) how the inclusion of herd immunity effects impacts the cost-effectiveness of pneumococcal vaccines, to illustrate potential implications for COVID-19 vaccines.

2 Methods

2.1 Data Source and Study Selection

We searched for economic evaluations of vaccines published between 2000 and 2018 in two distinct data sources: the Tufts Medical Center Cost-effectiveness Analysis Registry (CEAR) and the Global Health CEAR. Both registries are maintained by the Center for the Evaluation of Value and Risk in Health at Tufts Medical Center in Boston, Massachusetts. The CEAR (www.cearegistry.org) contains detailed information on published, English-language cost per quality-adjusted life-year (QALY) studies from 1976 to 2020 and the Global Health CEAR (www.ghcearegistry.org ) summarizes published, English-language cost per disability-adjusted life-year (DALY) studies from 1996 to 2020. The search strategies, inclusion criteria, and review methods for both registries have been published elsewhere [15,16,17,18]. Briefly, the procedure for CEAR begins with an electronic literature database search using the keywords that include “QALYs,” “quality-adjusted life-year,” and “cost-utility.” Following this, abstracts are screened for articles that contain cost per QALY results. Selected studies undergo full paper review by two independent reviewers who extract information for inclusion in the CEAR on study methods, incremental cost-effectiveness ratios (ICERs) and utility weights. The process for the Global Health CEAR is nearly identical, except for the use of “DALYs” and “disability-adjusted life year” in the search strategy.

We searched both databases and applied the following inclusion criteria: (1) CEA studies published between 2000 and 2018; (2) analyses that studied vaccination as an intervention (e.g., vaccination vs. no vaccination, or comparing alternative vaccines); (3) a target population in one or more LMICs based on the World Bank’s country classification [19] (Fig. 1). We chose the years 2000–2018 to focus on more current studies, and because 2018 was the most current year of studies contained in both registries at the time of this analysis.

Fig. 1
figure 1

Literature review flow diagram. HIC high income country

Full size image

2.2 Analysis

We investigated the proportion of vaccine CEAs that incorporated herd immunity effects into their analyses and identified the methods used for quantifying such effects. We compared characteristics (i.e., publication year, funding source, disease of focus, study quality) of vaccine CEAs with versus without herd immunity effects. The funding source was important to investigate because different funders may have different incentives to include or exclude herd immunity effects in CEAs. For example, vaccine manufacturers may prefer to include broader benefits of vaccines to make value arguments, whereas governments may prefer more conservative estimates to prioritize health services using a consistent perspective for resource allocation purposes. We assessed study quality using a subjective rating score assigned by Registry reviewers, ranging from 1 (lowest quality) to 7 (highest quality). Scores are aligned with seven principles of economic evaluation methods and reporting, and studies are given 1 point for each of the following: (1) Methods and results were communicated clearly and transparently to enable easy interpretation; (2) Time horizon was of sufficient length to capture all differences in costs and health benefits and discount rate was specific to the local setting; (3) Detailed disaggregated cost and QALY information was provided and reported ICER was correct; (4) Comprehensive characterization of uncertainty (sensitivity analysis) was undertaken; (5) Explicit reporting of utility weights (includes utility weight value and estimation method); (6) Subgroup analysis performed where relevant; and (7) Non-health effects and/or spillover effects were quantified [20].

In addition, we examined how incorporating herd immunity effects impacted the results of a CEA. When possible within a study, we compared an ICER that included herd immunity effects with a corresponding ICER that did not include such effects. We used herd immunity effects reported in the original papers. Some of these studies did not empirically quantify herd immunity effects but extrapolated from previously published data or assumptions. When those comparisons were not readily available in the paper, but disaggregated costs and effects were, we recalculated the reported ICER to create two ICERs with and without herd immunity effects. We recorded whether including herd immunity effects would increase or decrease the ICER and by how much, and whether the change in ICER was significant enough to cause the results to cross commonly cited CEA benchmarks for LMICs: cost-saving (less costly and more effective); 1 × gross domestic product (GDP) per capita, 3 × GDP per capital, dominated (more costly and less effective) [21, 22]. We further explored the impact of herd immunity in a subset of studies focusing on vaccines against pneumococcal disease. We conducted this case study because these vaccines protect against a form of pneumonia, which is a primary consequence of COVID-19. This case study may help provide context for the consideration of herd immunity when determining the economic value of COVID-19 vaccines. All ICERs and GDP per capita estimates were converted to 2020 US dollars using relevant currency exchange rates and the medical care component of the US consumer price index (https://www.bls.gov/cpi/).

3 Results

3.1 Characteristics of Vaccine Cost-Effectiveness Analyses

Among the 8,067 CEA studies in the CEAR and 709 CEAs in the Global Health CEAR, we excluded 7,994 and 539 studies, respectively, that did not meet our inclusion criteria. Our final analysis included 243 vaccine CEAs, with 73 cost-per-QALY studies and 170 cost-per-DALY studies (Fig. 1). The number of vaccine CEAs increased over time. Almost half of the studies (n = 118; 48%) reported government or foundation funding. The most commonly studied disease areas were rotavirus (22%), followed by HPV (17%) and pneumococcal disease (17%). Approximately 77% of the included studies were given a quality score of 5 or greater (range 1–7).

3.2 Estimating Herd Immunity Effects in Vaccine Cost-effectiveness Analyses

We found that 44 vaccine CEAs (18% of 243) included herd immunity effects, including 16 (22% of 73) cost-per-QALY studies and 28 (16% of 170) cost-per-DALY studies. Among vaccine CEAs that included herd immunity effects, 11% were published in 2000–2010, 32% published in 2011–2014, and 57% published in 2015–2018. Among vaccine CEAs that did not include herd immunity effects, the number of studies published in each period was more evenly distributed. Somewhat surprisingly, studies funded by governments or foundations included herd immunity effects more frequently (55%) than those funded by industry (20%) or other sources (7%). The majority of studies that incorporated herd immunity effects evaluated pneumococcal vaccines (43%), while those that did not include herd immunity effects mainly studied rotavirus (Table 1).

Table 1 Characteristics of vaccine cost-effectiveness analyses (n = 243, all studies)
Full size table

Of the 44 studies that included herd immunity effects, 11 (25%) used dynamic transmission models, while the remainder used static models. For studies that used dynamic models, we were unable to disentangle the impact of herd immunity from other vaccine benefits because herd immunity effects were not reported separately. In the 33 static models, herd immunity was estimated by adding a fixed percentage improvement in the vaccine effectiveness model output in 18 studies (54%) [23,24,25,26,27,28,29,30], altering model inputs (i.e., assuming a percentage decrease in disease incidence) in ten studies (30%) [31,32,33,34,35], or applying a multiplier to vaccine efficacy in three studies (9%) [36]. Two static model studies (6%) did not clearly state how herd immunity effects were estimated [37, 38] (Table 2). Among the 33 static models, 16 reported detailed data or allowed us to recalculate and compare each ICER alternatively with and without the inclusion of herd immunity (n = 48 ratio pairs). Within this subsample, there were eight studies that focused on vaccines for pneumococcal disease.

Table 2 Methods for estimating herd immunity effects
Full size table

3.3 Impact of Herd Immunity Effects on the Reported Incremental Cost-Effectiveness Ratio

The inclusion of herd immunity effects made all 48 ICERs more favorable, including five cases with cost-saving results that remained so after adjustment. In 20 cases (42%), adding herd immunity effects decreased the ICER enough to cross at least one or more commonly cited cost-effectiveness benchmarks for LMICs; in six cases, interventions that were originally cost-increasing became cost-saving (Table 3). After excluding results that became cost-saving with herd-immunity (n = 11), the average ICER without herd immunity was $55,552/QALY (range: $600/QALY–$613,691/QALY); the inclusion of herd immunity on average reduced the ICER to $41,074 (or by 28%). Among DALY-based studies, the mean ICER without herd immunity was $11,211/DALY (range $52/DALY–$69,411/DALY); including herd immunity reduced the average ICER to $6470 (or by 61%) (Table 4).

Table 3 Change in cost-effectiveness analysis value determinations when herd immunity effects are incorporated
Full size table
Table 4 The mean ICERs without herd immunity and the average percentage reduction in ICERs with the inclusion of herd immunity
Full size table

To illustrate how the inclusion of herd immunity effects in a CEA may affect the value of COVID-19 vaccines, we calculated 24 ICERs with and without herd immunity from eight studies that examined the cost-effectiveness of the pneumococcal vaccine compared to no vaccine in a pediatric population. In three cases, the pneumococcal vaccine was estimated to be cost-saving even without herd immunity effects. In eight cases (33%), adding herd immunity effects decreased the ICER enough to cross at least one commonly cited cost-effectiveness benchmark for LMICs; in most of these, the result became cost-saving (Table 3). After excluding interventions that were cost-saving with the inclusion of herd immunity (n = 8), the mean QALY-based ICER without herd immunity was $61,789/QALY (range $600/QALY–$613,691/QALY), and the inclusion of herd immunity on average reduced the ICER to $48,056 (or by 31%). The mean DALY-based ICER without herd immunity was $2,681/DALY (range $87/DALY–$6905-DALY) and was reduced to $1728 (or by 49%) on average with herd immunity (Table 4). The ICER differences with vs. without herd immunity stratified by World Bank’s country income classification (i.e., low-, lower middle-, upper middle-income countries) can be found in the Electronic Supplementary Material (ESM) Table 1. The results stratified by income levels were consistent with the overall findings.

4 Discussion

Our study is the first that we are aware of to demonstrate the impact of herd immunity on the reported cost-effectiveness of vaccines in LMICs. Despite recommendations from WHO to include herd immunity in vaccine CEAs in LMICs [12, 39], our review found that less than one-fifth of vaccine studies in our sample did so. This percentage is lower than the percentage reported in a previous systematic review [13]. The discrepancy may be due to the exclusion of studies focused on high-income countries from our sample that were included in the previous review. We found that studies that included herd immunity effects consistently had more favorable cost-effectiveness results, with ICER reductions ranging from 28 to 61%.

LMICs face severe resource limitations and competing public health priorities. Therefore, studies must provide evidence of the economic and social value of vaccinations in LMICs to help decision-makers appropriately prioritize these interventions. Our data show that in 42% of cases, the reductions in ICERs with the inclusion of herd immunity effects were large enough to cross at least one or more common cost-effectiveness benchmarks, which could lead to important changes in policy decisions on whether to pay for an intervention based on its economic value. The same trends hold true for pneumococcal vaccine CEAs, which may be an important analog for COVID-19 vaccines. Measuring the full benefits that vaccines confer on society through herd immunity provides important information to policy makers and funders as they prioritize the allocation of scarce resources in LMICs.

Among those studies that included herd immunity in their models, one-quarter used dynamic models to simulate disease transmission in the population, while the remainder used static models. Studies using dynamic transmission models may more accurately capture the impact of herd immunity because they simulate pathogen transmission among the susceptible population; however, the reports on the dynamic models in our sample did not provide sufficient information to understand herd immunity effects. In contrast, static models are unable to capture disease transmission among individuals, and there is no standardized or generally accepted method to quantify herd immunity effects using these models. In our sample, the studies with static models approximated herd immunity effects by altering model inputs, such as assuming a percentage decrease in disease incidence, applying a multiplier to vaccine efficacy, or by adding a fixed percentage improvement in vaccine effectiveness outputs. However, the rationale for the specific method used in these static models was generally not explained clearly. Future vaccine CEAs should strive to incorporate and estimate herd immunity effects accurately and transparently, regardless of the method employed.

Currently, there is a lack of international policy agreement on whether herd immunity effects should be included in economic evaluations of vaccinations. The Advisory Committee on Immunization Practices (ACIP) guidelines in the USA consider herd immunity a function of the capabilities and acceptance of the vaccination program rather than the effectiveness of the vaccine itself. Before a new vaccine enters the US market, ACIP requires the base-case CEA results to exclude herd immunity for a more conservative estimation of vaccine benefits and include herd immunity effects in a sensitivity analysis [40]. On the other hand, the WHO and European guidelines suggest including herd immunity in situations where its exclusion would make a substantial difference in the estimated cost-effectiveness of the program [12, 41]. It is important to note the differences among these guidelines, as our study showed that CEA results may in some cases change substantially when herd immunity is considered. Since CEA is a key element in health technology assessment, it is important to show the results both with and without herd immunity to accurately estimate the value of a vaccination under all scenarios, as well as to gauge whether its inclusion makes a material difference.

We used pneumococcal vaccination as a case study to examine potential implications for COVID-19 because pneumonia is a primary symptom of COVID-19 infection. However, it is important to note that there are differences between pneumococcal diseases and COVID-19 that limit their comparability. Even though pneumonia is a primary consequence of COVID-19, current estimates of the basic reproductive number (R0) for COVID-19 is much higher than that for pneumococcal disease. The R0 for pneumococcal disease is approximately two [42], meaning in a fully susceptible population, each pneumonia case on average infects two additional people. In order to achieve herd immunity, 50% of the population would need to be vaccinated to prevent population spread. Precise estimates of the R0 for COVID-19 are still evolving. The initial R0, based on early estimation from the COVID-19 epicenter in Wuhan, China, was estimated to be 2.2–2.7 [43]. However, a recent study published by the Centers for Disease Control and Prevention in the USA showed that the R0 could be as high as 5.7 in some settings [44]. In this case, over 80% of the population would need to be vaccinated to achieve herd immunity. Therefore, herd immunity effects will be easier to achieve and will have more impact on the value of vaccines for pneumococcal diseases than they will for COVID-19 if these new R0 estimates stand.

Despite the need for a significant percentage of the population to be vaccinated against COVID-19 to achieve herd immunity, global vaccine coverage rates remain low [45]. Vaccine hesitancy, the phenomenon in which individuals avoid vaccines because they consider them unsafe and/or unnecessary, is considered a growing threat to the success of vaccination programs [46]. Recent data show that the percentage of people in the USA who are committed to receiving the COVID-19 vaccine remains low (50% as of June 2020 [47, 48], increasing to 69% as of March 2021 [49]).While vaccine hesitancy is of less concern for LMICs [50, 51], the more pressing issue in these countries is limited access to vaccines. Future studies should explore how vaccine access affects the potential to achieve herd immunity for COVID-19 in LMICs.

Our review is limited by the methods and models developed by the authors in the published CEAs included in our study sample. Although dynamic models are considered the gold standard for modeling infectious disease [12, 41], we were unable to include studies with dynamic models to analyze how the inclusion of herd immunity effects changes ICERs, because those studies did not report sufficient information to replicate these estimates. In addition, due to the small sample size, we were unable to tease out how herd immunity effects vary by different types of vaccine CEAs (i.e., a hypothetical vaccine, a replacement vaccine, alternative vaccine strategies, or a different target population). Our study sample was also limited to published and English-language manuscripts. Technical reports from influential immunization organizations such as the National Immunization Technical Advisory Groups (NITAG), ACIP, and other health technology assessment organizations were not included. Finally, we included studies of all quality in the initial study sample, so it is possible that lower quality studies have findings that are not as verifiable. We conducted a sensitivity analysis that excluded low quality studies (i.e., study score ≤ 4). Most results remained the same, except that there were fewer studies funded by industry (10% vs. 16% in the full sample). These results are shown in ESM Table 2. (All studies reporting herd immunity effects were rated as having a quality score greater than 4. Therefore, the effect of including herd immunity was the same as Table 4 in the manuscript.)

5 Conclusion

Although herd immunity is an important benefit of vaccination programs, the majority of vaccine CEAs in LMICs do not incorporate herd immunity in their analysis. As a result, these CEAs may underestimate the potential value of vaccines when herd immunity is achieved. Given the evidence in CEAs of the pneumococcal vaccine, an important analog for COVID-19, that showed that accounting for herd immunity effects led to reductions in incremental cost-effectiveness ratios of 31–49%, COVID-19 vaccination value assessment should aim to produce results both with and without herd immunity effects for decision-making in LMICs.