Abstract
Objective
Our objective was to review economic evaluations of immunization and tuberculosis to determine the extent to which additional unintended consequences were taken into account in the analysis and to describe the methodological approaches used to estimate these, where possible.
Methods
We sourced the vaccine economic evaluations from a previous systematic review by Nymark et al. (2009–2015) and searched PubMed/MEDLINE and Embase from 2015 to 2019 using the same search strategy. For tuberculosis economic evaluations, we extracted studies from 2009 to 2019 that were published in a previous review by Siapka et al. We followed Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidance. Studies were classified according to the categories and subcategories (e.g., herd immunity, non-specific effects, and labor productivity) defined in a framework identifying additional unintended consequences by Nymark and Vassall. Where possible, methods for estimating the additional unintended consequences categories and subcategories were described. We evaluated the reporting quality of included studies according to the Consolidated Health Economic Evaluation Reporting Standards (CHEERS) extraction guideline.
Results
We identified 177 vaccine cost-effectiveness analyses (CEAs) between 2009 and 2019 that met the inclusion criteria. Of these, 98 included unintended consequences. Of the total 98 CEAs, overall health consequence categories were included 73 times; biological categories: herd immunity 43 times; pathogen response: resistance 15 times; and cross-protection 15 times. For health consequences pertaining to the supply-side (health systems) categories, side effects were included five times. On the nonhealth demand side (intrahousehold), labor productivity was included 60 times. We identified 29 tuberculosis CEAs from 2009 to 2019 that met the inclusion criteria. Of these, six articles included labor productivity, four included indirect transmission effects, and one included resistance. Between 2009 and 2019, only 34% of tuberculosis CEAs included additional unintended consequences, compared with 55% of vaccine CEAs.
Conclusions
The inclusion of additional unintended consequences in economic evaluations of immunization and tuberculosis continues to be limited. Additional unintended consequences of economic benefits, such as those examined in this review and especially those that occur outside the health system, offer valuable information to analysts. Further work on appropriate ways to value these additional unintended consequences is still warranted.
Similar content being viewed by others
Avoid common mistakes on your manuscript.
Between 2009 and 2019, only 34% of tuberculosis cost-effectiveness analyses (CEAs) included additional unintended consequences, compared with 55% of vaccine CEAs. |
There is a clear absence of evidence of additional unintended consequences outside the health system for both tuberculosis and vaccine CEAs. |
Further work on appropriate ways to value additional unintended consequences in CEAs is needed. |
1 Introduction
Guidelines for performing economic evaluations of healthcare interventions recommend that all relevant direct and indirect health effects are considered, whereas other guidelines highlight the importance of mapping indirect nonhealth effects into economics frameworks for value assessment [1,2,3,4,5]. Infectious disease-specific guidelines have also made attempts at scoping the inclusion of nondirect effects, particularly for economic evaluation of immunizations [6,7,8].
Given these developments in guidelines, it seems reasonable to expect that the inclusion of indirect health and nonhealth effects would be standard practice. However, the literature indicates that it is still challenging to comprehensively identify which indirect health and nonhealth effects to include in cost-effectiveness analyses (CEAs) [9,10,11]. Several previous reviews of CEAs suggest that indirect health and nonhealth effects are often excluded, even when they may be relevant and significant [12,13,14,15]. Different practices across economic evaluations can mean that cost effectiveness can be difficult to compare across interventions. Therefore, continued attention is required to define, examine, and map out the extent to which all consequences are considered in economic evaluation.
We presented a comprehensive framework elsewhere [16] to assist analysts in identifying and characterizing the additional costs and effects beyond that of the direct health impact that was primarily intended to be influenced by the intervention/technology. We refer to these additional costs and effects hereafter as “additional unintended consequences” [16]. On the whole, the inclusion of additional unintended consequences is relevant in studies using the societal perspective. However, some of the additional unintended consequences can also be relevant to the healthcare system perspective. In this study, we build on the existing literature by assessing the extent to which economic evaluations of vaccines and tuberculosis treatments consider the different types of additional unintended consequences based on our framework. These were chosen to provide a comprehensive summary of the evidence and to explore the consistency of findings (and therefore generalizability) across different disease areas. We also highlight the different methods that were used to measure these additional unintended consequences.
2 Methods
2.1 Search strategy and Data Extraction
We used a combination of previous searches from two separate vaccine and tuberculosis reviews that both followed the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines.
The vaccine review by Nymark et al. [15] searched PubMed/MEDLINE and Embase. It applied a search strategy by Kim and Goldie [17] using free text and medical subject heading (MeSH) terms such as vaccin∗, economic evaluat∗, and humans. It limited the search to the English language and covered articles published between 2009 and 2015. The details of the data included, covering searches, article selection, and data extraction, are presented elsewhere [15]. We identified CEAs for low- and middle-income countries from this review. We merge two reviews in this paper. The review by Siapka et al. [18] only included CEAs for low- and middle-income countries. To account for these differences, we restricted the review by Nymark et al. [15]. The search was updated to identify CEAs of vaccines from 1 August 2015 to 31 December 2019 and included the dengue vaccine, which was not included in the original review. CEAs were eligible for inclusion if the analysis included both costs and health effects and presented a decision-analytic model. Given the importance of model choice to accurately predict several additional unintended consequences, studies needed to present a model. Two reviewers independently screened titles and abstracts and reviewed the full texts to determine inclusion.
To identify tuberculosis CEAs, we included all of the CEAs of tuberculosis treatment that were included in the review by Siapka et al. [18]. The authors mainly searched the UCSR, PubMed, EMBASE, EconLit, Cochrane, NHS EED, and CEA Registry databases using broad searches including economic terms (e.g., cost, economic, or financial), disease-related terms (e.g., TB, tuberculosis, MDR [multi-drug resistant], XDR [extensively drug resistant]) and intervention-specific keywords (e.g., treatment, DOTS [directly observed treatment – short-course], isoniazid preventive therapy, patient cost). The full details of the data included, covering searches, article selection, and data extraction, are presented elsewhere [18]. We retrieved all the studies from 1 January 2009 to 31 December 2019, excluded cost studies, and selected a subset of studies if the analysis included both costs and health effects and presented a decision-analytic model in line with the criteria used in the review by Nymark et al. [15]. However, it should be noted that the review by Siapka et al. [18] only included studies that had primary data collection on costs. This means it missed other tuberculosis CEAs that, despite not having primary cost data collection, may have included transmission effects. Two reviewers independently screened the titles and abstracts and reviewed the full texts to determine inclusion. We also restricted this review to studies presented in English.
2.2 Data Analysis
We presented a comprehensive framework elsewhere that identifies and characterizes additional unintended costs and effects beyond that of the direct health impact primarily intended by the intervention/technology [16]. We briefly present and describe the framework (Fig. 1).
Conceptual framework ‘internal’ and ’external’ consequences
Interventions/technologies may have an impact beyond the intended direct health consequences. These are defined as “internal” consequences that occur within the individual (“internality”) or as “external” consequences that occur outside the individual (“externality”). There are several types of internalities and externalities within health impact. These types can be divided into biological effects, demand-side behavioral consequences, and supply-side behavioral consequences.
The “biological” types describe how cells and molecules within organisms interact and carry out their chemical and physical functions. It describes how these interactions are regulated, for example by control mechanisms and communication between cells. Within the type “biology,” there are three categories of potential additional internalities and externalities: (1) non-specific effects (NSE; impact on other diseases), (2) transmission (infection to others) or herd immunity effects (indirect protection), and (3) pathogen response (the pathogen is resistant). These effects are only relevant for vaccines and infectious diseases. The NSE of an immunization refers to the beneficial impacts of the immunization beyond protection against the pathogen it is directly intended for. Transmission effects are effects that reduce the transmission of an infectious agent from an infected individual to another individual. Herd immunity is one example: a form of indirect protection that occurs when a large enough percentage of the population is vaccinated (and therefore immune to infection) so that the unvaccinated individuals avoid infection. In the case of tuberculosis, interventions reduce the risk of infection to others as treatment reduces transmission; however, most tuberculosis interventions do not confer herd immunity. Pathogen response refers to the immune system’s response when an infectious agent causes disease or illness in its host. Vaccination can lead to serotype replacement, whereby the infectious agent with the serotype targeted by the vaccine is reduced or eliminated, allowing other serotypes the vaccine does not target to replace it. It can also induce cross-protection, which occurs when protection resulting from infection with one strain of a virus prevents infection by another related strain of that virus. Though the framework is specific to infectious diseases/vaccines for the biological type effects, it can apply in a broader sense when applied to other nonbiological impacts.
Demand-side impacts include individual, household, and population health-related consumption. There are two categories related to the health demand-side type: changes in health influencing behavior and changes in health services consumption (internalities). Changes in health influencing behavior refers to changes in a person’s actions, as a result of the intervention, that impact on their health. Changes in health services consumption refers to the changes, as a result of the intervention, in utilization of health services by a person for the purpose of promoting their health and well-being. On the supply-side impacts, the interventions can change the behavior of healthcare providers and impact on other health services or the provision of nonhealth services. Here, we identify one category that falls under the health sector perspective (health systems [external]) and two subcategories (side effects and scientific spill-over effects). Although the term side effect is predominantly used to describe adverse effects, it can also refer to unintended consequences of the use of the intervention. Scientific spill-over effects refer to the knowledge gained from development of a new drug or vaccine that might offer value beyond the drug or vaccine itself. For example, it could lead to further innovations in drug or vaccine development, the development of drugs or vaccines for other diseases, or the development of other health-related technologies.
Finally, there may be internal and external nonhealth consequences affecting the demand and supply side. For the demand side, we present two broad categories: behavior/education/knowledge (internal person) and consumption of nonhealth goods. Categories under this are intrahousehold (subcategories: informal care and change in behavior) and education and labor productivity. For the supply side, we identify two categories under the nonhealth consequences perspective: outside health systems (subcategory: public services) and provision of nonhealth services (subcategory: change in behavior).
We assessed and recorded which of the categories and associated subcategories for both nonhealth and health consequences were included in each of the immunization and tuberculosis CEAs. For each type, we also extracted the methods used to estimate the additional unintended consequences.
2.3 Reporting Quality Appraisal of Included Cost-Effectiveness Analyses (CEAs)
We used the Consolidated Health Economic Evaluation Reporting Standards (CHEERS) statement, consisting of 24 requirements, to appraise the quality of the included CEAs [2]. Data on the 24 items in the CHEERS statement were extracted, such as whether the article included a clear description and justification of the model used, a reference to the choice of health outcomes, mention of the measurement of effectiveness, and estimation of the resources and costs. For each of the 24 items, we assigned a yes/no judgment and then calculated the total number of confirming items (“yes”) to assess the overall reporting quality of each study.
3 Results
3.1 Paper Selection
The PubMed/MEDLINE and Embase database searches for articles published between 2015 and 2019 for vaccines returned 1074 papers after duplicates were removed (Fig. 2a). We screened 239 full-text articles, 163 of which did not meet the inclusion criteria: 32 did not present a clear model based on the model type classification used in the review by Nymark et al. [15], 128 focused on high-income countries, two were in Spanish, and one was a review. We screened 76 CEAs for inclusion of any of the categories included in the framework for additional unintended consequences, and 43 of these were included. Of the 101 articles focusing on low- and middle-income countries between 2009 and 2015 from the review by Nymark et al. [15], 55 articles included one or more categories.
a PRISMA flowchart article selection—vaccines 2009–2019. b PRISMA flowchart article selection—tuberculosis 2009–2019
To identify tuberculosis CEAs, 205 studies from the review by Siapka et al. [18] presenting primary collected tuberculosis cost data in low- and middle-income countries were screened. Of these, 65 articles reporting only costs were excluded (Fig. 2b). We assessed the remaining 140 articles for eligibility based on the criteria used by Nymark et al. [15] and found that 111 did not meet the inclusion criteria: one was a duplicate, three had no clear model classification, and the remaining 107 did not combine costs and effects. We screened 29 articles and found that ten included any of the categories included in the framework for additional unintended consequences.
3.2 Number of CEAs Including Categories of Additional Unintended Consequences
A total of 177 vaccine CEAs from 2009 to 2019 were eligible for inclusion in the review. In total, 98 vaccine CEAs included one or more categories or subcategories of additional unintended consequences (55%) (see Table 1). For health consequences, the category “biology” was included 73 times (41%). This was split between the subcategories cross-protection and resistance, which were included 15 times each (8%), and herd immunity, which was included the highest number of times (43 [24%]). We identified no CEAs with the category “non-specific effects.” Additionally, under health consequences on the supply side, the subcategory side effects was included five times (3%). We identified no CEAs that included health consequences on the demand side. For nonhealth consequences, labor productivity was the only category included on the demand-side (60 times [34%]). We identified no CEAs that included nonhealth consequences on the supply side.
A total of 29 tuberculosis CEAs from between 2009 and 2019 were eligible for inclusion. Of these, ten (34%) included one or more categories or subcategories of additional unintended consequences. Under the health type “biology,” transmission effects were identified in four CEAs (14%), and resistance was identified in one CEA (3%). We did not identify any demand or supply categories under health consequences. For nonhealth consequences, on the demand side, we identified labor productivity in six CEAs (21%). We did not identify any supply categories under nonhealth consequences.
3.3 Methods Used to Measure Additional Unintended Consequences
3.3.1 Health Consequences: Biology
3.3.1.1 Transmission: Indirect protection
We identified 43 vaccine CEAs that included herd immunity.
Pneumococcal: We identified 23 pneumococcal vaccine CEAs that considered herd immunity effects. Vespa et al. [19], Aljunid et al. [20], Nakamura et al. [21], Ayieko et al. [22], Gomez et al. [23], Kulpeng et al. [24], Che et al. [25], Caldwell et al. [26], and Ordonez et al. [27] all estimated herd immunity effects from US surveillance data in unvaccinated people for different age groups. Assumptions about the reduction of incidence of invasive pneumococcal diseases were used to account for herd immunity in the CEAs by Kim et al. [28], Martí et al. [29], Hu et al. [30], Wang et al. [31], Zhou et al. [32], Sundaram et al. [33], and Dorji et al. [34]. Kieninger et al. [35], Komakhidze et al. [36], Mezones-Hilguin et al. [37], Sibak et al. [38], and Mo et al. [39] used a simple multiplier to calculate a percentage increase in health benefits due to herd immunity effects. In sensitivity analyses, Lara et al. [40] included herd immunity but did not state which method was used to estimate these effects. In the CEA by Shen et al. [41], estimates of herd immunity effects were generated by the calculation of the reduction in carriage of vaccine serotypes after vaccine introduction.
Rotavirus: We identified five rotavirus vaccine CEAs that included the category herd immunity. Diop et al. [42], Javanbakht et al. [43], and Sigei et al. [44] estimated herd immunity effects using a multiplier and inflating health benefits to 120% of direct effects in children aged <5 years. In a sensitivity analysis, Atherly et al. [45] assumed that unvaccinated children would receive 15% protection at 50% vaccination coverage These indirect effect scenarios assumed that nonvaccinated children would receive a level of protection proportional to the efficacy in vaccinated children and the level of coverage. Rose et al. [46] used a dynamic model of rotavirus transmission to account for herd immunity effects.
Cholera: We identified three cholera vaccine CEAs that considered herd immunity.
Schaetti et al. [47] estimated herd immunity effects by multiplying the annual incidence of cases without vaccination by the protective efficacy among unvaccinated people. Jeuland et al. [48] used a mathematical equation linking oral cholera vaccine effectiveness to varying coverage rates in the study population. Khan et al. [49] used a dynamic model of cholera transmission to include herd immunity effects.
Haemophilus influenzae type b (Hib): We identified three Hib vaccine CEAs that included herd immunity. The Hib vaccine CEAs by Ning et al. [50], Clark et al. [51], and Griffiths et al. [52] assumed that herd immunity effects would increase the vaccine’s impact by 20%.
Other vaccines: The human papillomavirus (HPV) CEAs by Portnoy et al. [53], and Burger et al. [54] used a disease transmission dynamic approach to account for herd immunity. The typhoid fever CEAs by Antillon et al. [55] and Bilcke et al. [56] used a dynamic model of typhoid transmission to capture herd immunity effects. The two dengue CEAs by Shim [57] and Fitzpatrick et al. [58] used age-dependent dynamic transmission models to account for herd immunity effects. The polio vaccine CEA by Duintjer Tebbens et al. [59] used a dynamic transmission model. The measles vaccine CEA by Levin et al. [60] used a dynamic transmission model to include herd immunity effects. The diphtheria tetanus CEA by Fernandes et al. [61] used a disease dynamic transmission approach to account for herd immunity.
We identified four tuberculosis CEAs that included transmission effects.
Tuberculosis: Winetsky et al. [62] used a dynamic transmission model to account for indirect effects (reduced risk of the infection to others when treating tuberculosis). Wikman-Jorgensen et al. [63] and Mandalakas et al. [64] used Markov models to capture transmission effects. Fitzpatrick et al. [65] considered transmission effects using a mathematical model, but how the secondary cases were included was unclear.
3.3.1.2 Pathogen Response: Resistance
We identified 16 CEAs that included resistance.
Pneumococcal: We identified 15 pneumococcal vaccine CEAs that included resistance (serotype replacement). Nakamura et al. [21], Ayieko et al. [22], Gomez et al. [23], and Kulpeng et al. [24] assumed a US serotype replacement effect and modeled an increase in nonvaccine-type invasive pneumococcal disease following the introduction of 7-valent pneumococcal vaccine (PCV-7). Serotype replacement was assumed as the cause of an increase in acute otitis media disease due to noncovered pneumococcal serotypes for PCV-7 (33% increase) in the CEAs by Kim et al. [28], Martí et al. [29], Wang et al. [31], Zhou et al. [32], and Sundaram et al. [33]. In the CEAs by Kieninger et al. [35], Komakhidze et al. [36], Mezones-Hilguin et al. [37], Sibak et al. [38], and Castaneda-Orjuela et al. [66], a simple multiplier was used to calculate the percentage at which the serotype replacement could impair the indirect protection. Lara et al. [40] did not state which methods were used.
Tuberculosis: We identified one tuberculosis CEA that included resistance (pathogen resistance). Winetsky et al. [62] estimated the impact on resistance itself (i.e., acquired resistance or onwards transmission) using a dynamic transmission model.
3.3.1.3 Pathogen Response: Cross-Protection
We identified 15 vaccine CEAs that included cross-protection.
Pneumococcal: We identified eight pneumococcal vaccine CEAs that included cross-protection. Nakamura et al. [21], Kim et al. [28], Martí et al. [29], Wang et al. [31], Castaneda-Orjuela et al. [66], Mezones-Hilguin et al. [37], Gomez et al. [23], and Marijam et al. [67] assumed cross-protection for pneumococcal polysaccharide protein D-conjugate vaccine against serotype 6A to be equal to that of PCV-7 (76%) on the basis of noninferiority immunogenicity data.
HPV: We identified four HPV vaccine CEAs that included cross-protection. Setiawan et al. [68], Bardach et al. [69], Germar et al. [70], and Van Kriekinge et al. [71] assumed an effect of cross-protection against HPV types at various percentages of vaccine efficacy.
Dengue: We identified two dengue vaccine CEAs that included cross-protection. Shafie et al. [72] and Zeng et al. [73] used a dynamic transmission approach to account for temporary or permanent cross-protection, cross-enhancement, or their combination.
Influenza: We identified one influenza vaccine CEA that included cross-protection. Jamotte et al. [74] assumed an effect of cross-protection against influenza types at various percentages of vaccine efficacy.
3.3.2 Health Consequences: Supply
3.3.2.1 Health Systems: Side effects
We identified five vaccine CEAs that included side effects.
Influenza: We identified two influenza vaccine CEAs that included side effects. In the study by Meeyai et al. [75], side effects were incorporated as disutility values in the disability-adjusted life-year measure. Vo et al. [76] did not state the methods used.
Rotavirus: We identified one rotavirus vaccine CEA, by Wang et al. [77], that included side effects but did not state the methods used.
Varicella: We identified one varicella vaccine CEA, by You et al. [78], that included side effects. In the study, side effects were incorporated as disutility values in the quality-adjusted life-year (QALY) measure.
Pneumococcal: We identified one pneumococcal vaccine CEA, by Aljunid et al. [20], that included side effects but not the methods used.
3.3.3 Nonhealth Consequences: Demand
3.3.3.1 Intrahousehold: Labor Productivity
We identified 66 CEAs that included labor productivity.
Pneumococcal: We identified 14 pneumococcal vaccine CEAs that included labor productivity. Gomez et al. [23], Martí et al. [29], Sartori et al. [79], Che et al. [25], Lara et al. [40], Vespa et al. [19], and De Soarez et al. [80], Zhou et al. [32], Kebede et al. [81], and Sundaram et al. [33] all used the human capital approach to estimate labor productivity loss. Kulpeng et al. [24], Zhao et al. [82], Ayieko et al. [22], and Constenla et al. [83] did not state the methods used.
Rotavirus: We identified 18 rotavirus vaccine CEAs that included labor productivity. Wang et al. [77], Jit et al. [84], and de Blasio et al. [85], Flem et al. [86], Chotivitayatarakorn et al. [87], Ahmeti et al. [88], Kim et al. [89, 90], Wilopo et al. [91], Rheingans et al. [92], Liu et al. [93], and Clark et al. [94] used the human capital approach to estimate labor productivity loss. In the studies by Loganathan et al. [95] and Tu et al. [96], labor productivity losses were embodied in the QALY measure. Alkoshi et al. [97], Atherly et al. [98], and Rose et al. [46, 99] did not state the methods used.
Hepatitis B: We identified six hepatitis B vaccine CEAs that included labor productivity. Lee et al. [100], Wang et al. [101], and Zheng et al. [102] all used the human capital approach to estimate labor productivity loss. In the study by Jia et al. [103], losses in labor productivity were incorporated in health-related quality of life (HRQoL). Tu et al. [104] and Lu et al. [105] did not state the methods used.
Hib: We identified four Hib vaccine CEAs that included labor productivity. Moradi-Lakeh et al. [106], Griffiths et al. [107], and Muangchana et al. [108] used the human capital approach to estimate labor productivity loss. Le et al. [109] used the friction cost method to estimate the indirect cost due to productivity loss.
Influenza: We identified three influenza vaccine CEAs that included labor productivity. Jamotte et al. [74], Vo et al. [76], and Sribhutorn et al. [110] used the human capital approach to estimate labor productivity loss.
Dengue: We identified three dengue vaccine CEAs that included labor productivity. Zeng et al. [73], Shafie et al. [72], and Perera et al. [111] used the human capital approach to estimate labor productivity loss.
HPV: We identified three HPV vaccine CEAs that included labor productivity. Novaes et al. [112], Van Minh et al. [113], and Aguilar et al. [114] used the human capital approach to estimate labor productivity loss.
Other vaccines: The two cholera vaccine CEAs, by Jeuland et al. [48] and Schaetti et al. [47], used the human capital approach to estimate labor productivity loss. The measles vaccine CEA by Bishai et al. [115] used the human capital approach. The polio CEA by Duintjer Tebbens et al. [59] used the human capital approach. The meningococcal vaccine CEA by De Soarez et al. [116] used the human capital approach. The tetanus-diphtheria-acellular pertussis vaccine CEA by Sartori et al. [117] used the human capital approach to estimate labor productivity loss. The typhoid fever vaccine CEA by Lo et al. [118] used the human capital approach to estimate labor productivity loss. The varicella vaccine CEA by Esmaeeli et al. [119] used the human capital approach to estimate labor productivity loss. The hepatitis A vaccine CEA by Carlos et al. [120] used the human capital approach to estimate labor productivity loss.
Tuberculosis: We identified six tuberculosis CEAs that included labor productivity. Sekandi et al. [121], Datiko et al. [122], Steffen et al. [123], Prado et al. [124], Vassall et al. [125], and Wang et al. [126] all used the human capital approach to estimate labor productivity loss.
3.4 Reporting Quality Assessment
In total, 85% of the included studies met 20 or more of the 24 CHEERS checklist criteria. The mean number across the 108 CEAs was 21, with a range between 13 and 24 checklist criteria. Table S1 in the electronic supplementary material provides a complete overview of conforming items for individual CEAs.
4 Discussion
Even though tuberculosis treatments and the vaccines covered in this review have clear indirect effects, only 34% of tuberculosis CEAs included additional unintended consequences, compared with 55% of vaccine CEAs. One factor that may account for the low proportion of studies for tuberculosis is that the review by Siapka et al. [18], which we used to identify tuberculosis CEAs, only included studies that had some sort of primary data collection on costs. This means it missed out other tuberculosis CEAs that, despite not having primary cost data collection, may have included transmission and therefore may have increased the proportion of tuberculosis studies. Tuberculosis has an evidenced impact on labor productivity, but these costs were only included in 21% of the tuberculosis CEAs [127]. Productivity savings are also evident for some vaccines (e.g., Hib and pneumococcal), particularly work days lost by a parent caring for a sick child. In the case of influenza, there is well-established evidence of both household and macro-economic impact, yet these effects were only included in 34% of all vaccine CEAs [128].
Aside from this, other aspects stand out as areas of potential under-inclusion in CEAs. Resistance is a critical issue in both immunization and tuberculosis. We have used the word resistance in a general sense as describing a situation where a pathogen shifts so it no longer reacts to medicines or immunization. For example, in tuberculosis, resistance to several medicines is widespread and described by the World Health Organization as drug-resistant tuberculosis. In the case of vaccines, resistance is an internal biological consequence as serotype replacement is a phenomenon that induces resistance to subtypes of serotypes if the frequency of a subtype of serotype declines because of high levels of immunity, allowing other serotypes to replace it. However, resistance was considered in only in 8% of all vaccine CEAs compared with 3% of all tuberculosis CEAs. Likewise, cross-protection and side effects were also likely to be substantially under-included. Cross-protection is the protection conferred on a host (“internal”) by infection with one strain of a virus that prevents infection by a closely related substrain of that virus. This biological effect is particularly relevant for pneumococcal, HPV, dengue, and influenza vaccinations. However, cross-protection was only included in 8% of all vaccine CEAs. Side effects were only included in 3% of all vaccine CEAs. The biological category “non-specific effects” of vaccines was not included in any CEAs. These are beneficial effects that offer protection beyond specific pathogens and are particularly important for live vaccines. A clear distinction must be made between non-specific effects of vaccines that are due to hypothesized vaccine-induced improvements to vaccinees’ immunity against nontargeted antigens, and non-specific disease outcomes. The latter category is often used in vaccine impact evaluations and CEAs because pathogen-specific disease outcomes (e.g., gastroenteritis due to rotavirus) are difficult to measure and because the etiology of diseases (e.g., diarrheal diseases occurring after measles or circulatory diseases occurring soon after an influenza infection) is often unclear. For instance, nonlive vaccines (e.g., inactivated flu vaccines) can have an impact on non-specific outcomes (e.g., circulatory diseases).
As mentioned previously, the inclusion of additional unintended consequences in economic evaluation is mainly relevant in studies using a social perspective, though some additional unintended consequences can also be relevant to the healthcare system perspective. Whether to include or exclude additional unintended consequences ideally is based on weighting the likely importance, the extent of evidence, and the analytical complexity of doing so. In part, the lack of data, and appropriate methods to include several of the additional unintended consequences covered in this review, may be because of the current novelty of some of these measures. However, this is unlikely to apply to transmission or labor productivity effects. Although dynamic transmission models are often recommended and are important for capturing herd immunity effects, we found a lack of use of this type of model in the CEAs reviewed. Methodologically, we found a strong reliance on the human capital approach to measure the loss of productivity across the disease areas covered in the review. For more minor consequences, comprehensive guidelines for economic evaluations about which of these additional unintended consequences should be reported and how are lacking. Analytically, the complexity of the relationships between internal and external nondirect health and nonhealth impact and how to quantify this requires a range of types of evidence and techniques, but guidance on how to address feedbacks between different consequences is sparse. Furthermore, quantifying and linking changes in nondirect health effects to nonhealth impact (e.g., behavioral outcomes) is complex. Given the difficulty with measuring, it may be challenging to develop measurements capturing all relevant health and economic consequences in immunization, and tuberculosis, respectively.
This is the first review to evaluate the inclusion of additional unintended consequences in economic evaluation studies as presented in the framework by Nymark and Vassall [16]. However, we only included vaccine and tuberculosis CEAs in our search. We acknowledge this as a limitation as it is therefore not possible to generalize the results obtained across several disease areas. We acknowledge that additional unintended consequences may bias results either way. For example, although some unintended consequences, such as reducing susceptibility to other diseases, are likely to be positive, others are likely to be negative. We highlight the latter in the case of tuberculosis as failing to include transmission and productivity may bias results to underestimate cost effectiveness. However, without conducting a formal analysis, we cannot assess this, and the review aims more to identify the extent and nature of the inclusion of these consequences rather than to summarize the extent of their impact on results and conclusions in each study. In addition, the analysis carried out included only papers for low- and middle-income countries. We acknowledge this as a limitation, but to account for the differences between the two reviews used in the analysis, we restricted the review by Nymark et al. [15].
The results of this review should not be taken to suggest that all additional consequences should be included in every economic evaluation but instead should encourage analysts to provide transparency where unintended additional consequences are excluded and to provide the reasons for this. We have previously provided a clear framework that can be used [16].
At the very least, we recommend moving towards reporting against a comprehensive framework of types and categories of additional effects. We hope that further transparency in this aspect of CEA is a feasible first step toward accounting for additional unintended consequences in economic evaluations.
5 Conclusion
The inclusion of additional unintended consequences in economic evaluations of immunization and tuberculosis continues to be limited, even though they offer valuable information to analysts. Only 34% of tuberculosis CEAs included additional unintended consequences, compared with 55% of vaccine CEAs. Further work on appropriate ways to value additional unintended consequences is still warranted, especially for those that occur outside the health system. In particular, work is still needed on how to link changes in internal consequences to external consequences and on combining several additional unintended consequence categories or subcategories in economic evaluations.
References
Drummond MF, Jefferson TO. Guidelines for authors and peer reviewers of economic submissions to the BMJ. The BMJ Economic Evaluation Working Party. BMJ. 1996;313(7052):275–83.
Husereau D, Drummond M, Petrou S, Carswell C, Moher D, Greenberg D, et al. Consolidated Health Economic Evaluation Reporting Standards (CHEERS) statement. Value Health. 2013;16(2):e1-5.
Claxton KRP, Sculpher M, Wilkinson T, Cairns J, Briggs A. The gates reference case for economic evaluation. Seattle: The Bill and Melinda Gates Foundation; 2014.
Sanders GD, Neumann PJ, Basu A, Brock DW, Feeny D, Krahn M, et al. Recommendations for conduct, methodological practices, and reporting of cost-effectiveness analyses: second panel on cost-effectiveness in health and medicine. JAMA. 2016;316(10):1093–103.
Lakdawalla DN, Doshi JA, Garrison LP Jr, Phelps CE, Basu A, Danzon PM. Defining elements of value in health care-a health economics approach: an ISPOR Special Task Force Report [3]. Value Health. 2018;21(2):131–9.
Ultsch B, Damm O, Beutels P, Bilcke J, Bruggenjurgen B, Gerber-Grote A, et al. Methods for health economic evaluation of vaccines and immunization decision frameworks: a consensus framework from a European vaccine economics community. Pharmacoeconomics. 2016;34(3):227–44.
Mauskopf J, Standaert B, Connolly MP, Culyer AJ, Garrison LP, Hutubessy R, et al. Economic analysis of vaccination programs: an ISPOR good practices for outcomes research task force report. Value Health. 2018;21(10):1133–49.
Pitman R, Fisman D, Zaric GS, Postma M, Kretzschmar M, Edmunds J, et al. Dynamic transmission modeling: a report of the ISPOR-SMDM modeling good research practices task force–5. Value Health. 2012;15(6):828–34.
Barnighausen T, Bloom DE, Canning D, Friedman A, Levine OS, O’Brien J, et al. Rethinking the benefits and costs of childhood vaccination: the example of the Haemophilus influenzae type b vaccine. Vaccine. 2011;29(13):2371–80.
Barnighausen T, Bloom DE, Canning D, O’Brien J. Accounting for the full benefits of childhood vaccination in South Africa. S Afr Med J. 2008;98(11):842, 4–6.
Jit M, Demarteau N, Elbasha E, Ginsberg G, Kim J, Praditsitthikorn N, et al. Human papillomavirus vaccine introduction in low-income and middle-income countries: guidance on the use of cost-effectiveness models. BMC Med. 2011;9:54.
Deogaonkar R, Hutubessy R, van der Putten I, Evers S, Jit M. Systematic review of studies evaluating the broader economic impact of vaccination in low and middle income countries. BMC Public Health. 2012;12:878.
Ozawa S, Mirelman A, Stack ML, Walker DG, Levine OS. Cost-effectiveness and economic benefits of vaccines in low- and middle-income countries: a systematic review. Vaccine. 2012;31(1):96–108.
Drake TL, Devine A, Yeung S, Day NP, White LJ, Lubell Y. Dynamic transmission economic evaluation of infectious disease interventions in low- and middle-income countries: a systematic literature review. Health Econ. 2016;25(Suppl 1):124–39.
Nymark LS, Sharma T, Miller A, Enemark U, Griffiths UK. Inclusion of the value of herd immunity in economic evaluations of vaccines. A systematic review of methods used. Vaccine. 2017;35(49 Pt B):6828–41.
Nymark L, Vassall A. A comprehensive framework for considering additional unintended consequences in economic evaluation. Cost Eff Resour Alloc. 2020;18:27.
Kim SY, Goldie SJ. Cost-effectiveness analyses of vaccination programmes: a focused review of modelling approaches. Pharmacoeconomics. 2008;26(3):191–215.
Siapka M, Vassall A, Cunnama L, Pineda C, Cerecero D, Sweeney S, et al. Cost of tuberculosis treatment in low- and middle-income countries: systematic review and meta-regression. Int J Tuberc Lung Dis. 2020;24(8):802–10.
Vespa G, Constenla DO, Pepe C, Safadi MA, Berezin E, de Moraes JC, et al. Estimating the cost-effectiveness of pneumococcal conjugate vaccination in Brazil. Rev Panam Salud Publica. 2009;26(6):518–28.
Aljunid S, Abuduxike G, Ahmed Z, Sulong S, Nur AM, Goh A. Impact of routine PCV7 (Prevenar) vaccination of infants on the clinical and economic burden of pneumococcal disease in Malaysia. BMC Infect Dis. 2011;11:248.
Nakamura MM, Tasslimi A, Lieu TA, Levine O, Knoll MD, Russell LB, et al. Cost effectiveness of child pneumococcal conjugate vaccination in middle-income countries. Int Health. 2011;3(4):270–81.
Ayieko P, Griffiths UK, Ndiritu M, Moisi J, Mugoya IK, Kamau T, et al. Assessment of health benefits and cost-effectiveness of 10-valent and 13-valent pneumococcal conjugate vaccination in Kenyan children. PLoS ONE. 2013;8(6):e67324.
Gomez JA, Tirado JC, Navarro Rojas AA, Castrejon Alba MM, Topachevskyi O. Cost-effectiveness and cost utility analysis of three pneumococcal conjugate vaccines in children of Peru. BMC Public Health. 2013;13:1025.
Kulpeng W, Leelahavarong P, Rattanavipapong W, Sornsrivichai V, Baggett HC, Meeyai A, et al. Cost-utility analysis of 10- and 13-valent pneumococcal conjugate vaccines: protection at what price in the Thai context? Vaccine. 2013;31(26):2839–47.
Che D, Zhou H, He J, Wu B. Modeling the impact of the 7-valent pneumococcal conjugate vaccine in Chinese infants: an economic analysis of a compulsory vaccination. BMC Health Serv Res. 2014;14:56.
Caldwell R, Roberts CS, An Z, Chen CI, Wang B. The health and economic impact of vaccination with 7-valent pneumococcal vaccine (PCV7) during an annual influenza epidemic and influenza pandemic in China. BMC Infect Dis. 2015;15:284.
Ordonez JE, Orozco JJ. Cost-effectiveness analysis of the available pneumococcal conjugated vaccines for children under five years in Colombia. Cost Eff Resour Alloc. 2015;13:6.
Kim SY, Lee G, Goldie SJ. Economic evaluation of pneumococcal conjugate vaccination in the Gambia. BMC Infect Dis. 2010;10:260.
Marti SG, Colantonio L, Bardach A, Galante J, Lopez A, Caporale J, et al. A cost-effectiveness analysis of a 10-valent pneumococcal conjugate vaccine in children in six Latin American countries. Cost Eff Resour Alloc. 2013;11(1):21.
Hu S, Shi Q, Song S, Du L, He J, Chen CI, et al. Estimating the cost-effectiveness of the 7-valent pneumococcal conjugate vaccine in Shanghai. China Value Health Reg Issues. 2014;3:197–204.
Wang XJ, Saha A, Zhang XH. Cost-effectiveness analysis of a universal mass vaccination program with a PHiD-CV 2+1 schedule in Malaysia. Cost Eff Resour Alloc. 2017;15:17.
Zhou H, He J, Wu B, Che D. Cost-effectiveness analysis of routine 13-valent pneumococcal conjugate vaccinations in Chinese infants. Hum Vaccin Immunother. 2018;14(6):1444–52.
Sundaram N, Chen C, Yoong J, Luvsan ME, Fox K, Sarankhuu A, et al. Cost-effectiveness of 13-valent pneumococcal conjugate vaccination in Mongolia. Vaccine. 2017;35(7):1055–63.
Dorji K, Phuntsho S, Kumluang S, Khuntha S, Kulpeng W, et al. Towards the introduction of pneumococcal conjugate vaccines in Bhutan: a cost-utility analysis to determine the optimal policy option. Vaccine. 2018;36(13):1757–65.
Kieninger MP, Caballero EG, Sosa AA, Amarilla CT, Jauregui B, Janusz CB, et al. Cost-effectiveness analysis of pneumococcal conjugate vaccine introduction in Paraguay. Vaccine. 2015;33(Suppl 1):A143–53.
Komakhidze T, Hoestlandt C, Dolakidze T, Shakhnazarova M, Chlikadze R, Kopaleishvili N, et al. Cost-effectiveness of pneumococcal conjugate vaccination in Georgia. Vaccine. 2015;33(Suppl 1):A219–26.
Mezones-Holguin E, Canelo-Aybar C, Clark AD, Janusz CB, Jauregui B, Escobedo-Palza S, et al. Cost-effectiveness analysis of 10- and 13-valent pneumococcal conjugate vaccines in Peru. Vaccine. 2015;33(Suppl 1):A154–66.
Sibak M, Moussa I, El-Tantawy N, Badr S, Chaudhri I, Allam E, et al. Cost-effectiveness analysis of the introduction of the pneumococcal conjugate vaccine (PCV-13) in the Egyptian national immunization program, 2013. Vaccine. 2015;33(Suppl 1):A182–91.
Mo X, Gai Tobe R, Liu X, Mori R. Cost-effectiveness and health benefits of pediatric 23-valent pneumococcal polysaccharide vaccine, 7-valent pneumococcal conjugate vaccine and forecasting 13-valent pneumococcal conjugate vaccine in China. Pediatr Infect Dis J. 2016;35(11):e353–61.
Lara C, De Graeve D, Franco F. Cost-effectiveness analysis of pneumococcal and influenza vaccines administered to children less than 5 years of age in a low-income district of Bogota Colombia. Value Health Reg Issues. 2018;17:21–31.
Shen K, Wasserman M, Liu D, Yang YH, Yang J, Guzauskas GF, et al. Estimating the cost-effectiveness of an infant 13-valent pneumococcal conjugate vaccine national immunization program in China. PLoS ONE. 2018;13(7):e0201245.
Diop A, Atherly D, Faye A, Lamine Sall F, Clark AD, Nadiel L, et al. Estimated impact and cost-effectiveness of rotavirus vaccination in Senegal: a country-led analysis. Vaccine. 2015;33(Suppl 1):A119–25.
Javanbakht M, Moradi-Lakeh M, Yaghoubi M, Esteghamati A, Mansour Ghanaie R, Mahmoudi S, et al. Cost-effectiveness analysis of the introduction of rotavirus vaccine in Iran. Vaccine. 2015;33(Suppl 1):A192-200.
Sigei C, Odaga J, Mvundura M, Madrid Y, Clark AD, Kenya ProVac Technical Working G, et al. Cost-effectiveness of rotavirus vaccination in Kenya and Uganda. Vaccine. 2015;33(Suppl 1):A109–18.
Atherly DE, Lewis KD, Tate J, Parashar UD, Rheingans RD. Projected health and economic impact of rotavirus vaccination in GAVI-eligible countries: 2011–2030. Vaccine. 2012;30(Suppl 1):A7-14.
Rose J, Homa L, Meropol SB, Debanne SM, Bielefeld R, Hoyen C, et al. Health impact and cost-effectiveness of a domestically-produced rotavirus vaccine in India: a model based analysis. PLoS ONE. 2017;12(11):e0187446.
Schaetti C, Weiss MG, Ali SM, Chaignat CL, Khatib AM, Reyburn R, et al. Costs of illness due to cholera, costs of immunization and cost-effectiveness of an oral cholera mass vaccination campaign in Zanzibar. PLoS Negl Trop Dis. 2012;6(10):e1844.
Jeuland M, Cook J, Poulos C, Clemens J, Whittington D, Group DCES. Cost-effectiveness of new-generation oral cholera vaccines: a multisite analysis. Value Health. 2009;12(6):899–908.
Khan AI, Levin A, Chao DL, DeRoeck D, Dimitrov DT, Khan JAM, et al. The impact and cost-effectiveness of controlling cholera through the use of oral cholera vaccines in urban Bangladesh: a disease modeling and economic analysis. PLoS Negl Trop Dis. 2018;12(10):e0006652.
Ning G, Yin Z, Li Y, Wang H, Yang W. Cost-effectiveness of the Haemophilus influenzae type b vaccine for infants in mainland China. Hum Vaccin Immunother. 2018;14(1):36–44.
Clark AD, Griffiths UK, Abbas SS, Rao KD, Privor-Dumm L, Hajjeh R, et al. Impact and cost-effectiveness of Haemophilus influenzae type b conjugate vaccination in India. J Pediatr. 2013;163(1):S60–72.
Griffiths UK, Clark A, Shimanovich V, Glinskaya I, Tursunova D, Kim L, et al. Comparative economic evaluation of Haemophilus influenzae type b vaccination in Belarus and Uzbekistan. PLoS ONE. 2011;6(6):e21472.
Portnoy A, Campos NG, Sy S, Burger EA, Cohen J, Regan C, et al. Impact and cost-effectiveness of human papillomavirus vaccination campaigns. Cancer Epidemiol Biomark Prev. 2020;29(1):22–30.
Burger EA, Campos NG, Sy S, Regan C, Kim JJ. Health and economic benefits of single-dose HPV vaccination in a Gavi-eligible country. Vaccine. 2018;36(32 Pt A):4823–9.
Antillon M, Bilcke J, Paltiel AD, Pitzer VE. Cost-effectiveness analysis of typhoid conjugate vaccines in five endemic low- and middle-income settings. Vaccine. 2017;35(27):3506–14.
Bilcke J, Antillón M, Pieters Z, Kuylen E, Abboud L, Neuzil KM, et al. Cost-effectiveness of routine and campaign use of typhoid Vi-conjugate vaccine in Gavi-eligible countries: a modelling study. Lancet Infect Dis. 2019;19(7):728–39.
Shim E. Cost-effectiveness of dengue vaccination programs in Brazil. Am J Trop Med Hyg. 2017;96(5):1227–34.
Fitzpatrick C, Haines A, Bangert M, Farlow A, Hemingway J, Velayudhan R. An economic evaluation of vector control in the age of a dengue vaccine. PLoS Negl Trop Dis. 2017;11(8):e0005785.
Duintjer Tebbens RJ, Pallansch MA, Cochi SL, Wassilak SG, Linkins J, Sutter RW, et al. Economic analysis of the global polio eradication initiative. Vaccine. 2010;29(2):334–43.
Levin A, Burgess C, Garrison LP Jr, Bauch C, Babigumira J, Simons E, et al. Global eradication of measles: an epidemiologic and economic evaluation. J Infect Dis. 2011;204(Suppl 1):S98-106.
Fernandes EG, Sartori AMC, de Soarez PC, Amaku M, de Azevedo Neto RS, Novaes HMD. Cost-effectiveness analysis of universal adult immunization with tetanus-diphtheria-acellular pertussis vaccine (Tdap) versus current practice in Brazil. Vaccine. 2020;38(1):46–53.
Winetsky DE, Negoescu DM, DeMarchis EH, Almukhamedova O, Dooronbekova A, Pulatov D, et al. Screening and rapid molecular diagnosis of tuberculosis in prisons in Russia and Eastern Europe: a cost-effectiveness analysis. PLoS Med. 2012;9(11):e1001348.
Wikman-Jorgensen PE, Llenas-Garcia J, Perez-Porcuna TM, Hobbins M, Ehmer J, Mussa MA, et al. Microscopic observation drug-susceptibility assay vs. Xpert((R)) MTB/RIF for the diagnosis of tuberculosis in a rural African setting: a cost-utility analysis. Trop Med Int Health. 2017;22(6):734–43.
Mandalakas AM, Hesseling AC, Gie RP, Schaaf HS, Marais BJ, Sinanovic E. Modelling the cost-effectiveness of strategies to prevent tuberculosis in child contacts in a high-burden setting. Thorax. 2013;68(3):247–55.
Fitzpatrick C, Hui Z, Lixia W, Renzhong L, Yunzhou R, Mingting C, et al. Cost-effectiveness of a comprehensive programme for drug-resistant tuberculosis in China. Bull World Health Organ. 2015;93(11):775–84.
Castaneda-Orjuela C, De la Hoz-Restrepo F. How cost effective is switching universal vaccination from PCV10 to PCV13? A case study from a developing country. Vaccine. 2018;36(38):5766–73.
Marijam A, Olbrecht J, Ozakay A, Eken V, Meszaros K. Cost-effectiveness comparison of pneumococcal conjugate vaccines in Turkish children. Value Health Reg Issues. 2019;19:34–44.
Setiawan D, Dolk FC, Suwantika AA, Westra TA, Postma MJ. Cost-utility analysis of human papillomavirus vaccination and cervical screening on cervical cancer patient in Indonesia. Value Health Reg Issues. 2016;9:84–92.
Bardach AE, Garay OU, Calderon M, Pichon-Riviere A, Augustovski F, Marti SG, et al. Health economic evaluation of human papillomavirus vaccines in women from Venezuela by a lifetime Markov cohort model. BMC Public Health. 2017;17(1):152.
Germar MJ, Purugganan C, Bernardino MS, Cuenca B, Chen YC, Li X, et al. Cost-effectiveness analysis of AS04-adjuvanted human papillomavirus 16/18 vaccine compared with human papillomavirus 6/11/16/18 vaccine in the Philippines, with the new 2-dose schedule. Hum Vaccin Immunother. 2017;13(5):1158–66.
Van Kriekinge G, Sohn WY, Aljunid SM, Soon R, Yong CM, Chen J, et al. Comparative cost-effectiveness analysis of two different two-dose human papillomavirus vaccines in Malaysia. Asian Pac J Cancer Prev. 2018;19(4):933–40.
Shafie AA, Yeo HY, Coudeville L, Steinberg L, Gill BS, Jahis R, et al. The potential cost effectiveness of different dengue vaccination programmes in Malaysia: a value-based pricing assessment using dynamic transmission mathematical modelling. Pharmacoeconomics. 2017;35(5):575–89.
Zeng W, Halasa-Rappel YA, Baurin N, Coudeville L, Shepard DS. Cost-effectiveness of dengue vaccination in ten endemic countries. Vaccine. 2018;36(3):413–20.
Jamotte A, Clay E, Macabeo B, Caicedo A, Lopez JG, Bricks L, et al. Public health impact and economic benefits of quadrivalent influenza vaccine in Latin America. Hum Vaccin Immunother. 2017;13(4):877–88.
Meeyai A, Praditsitthikorn N, Kotirum S, Kulpeng W, Putthasri W, Cooper BS, et al. Seasonal influenza vaccination for children in Thailand: a cost-effectiveness analysis. PLoS Med. 2015;12(5):e1001829 (discussion e).
Vo TQ, Chaikledkaew U, Hoang MV, Riewpaiboon A. Modelling the health economic impact of influenza vaccination strategies for high-risk children in Vietnam. J Clin Diagn Res. 2018;12(6):26–32.
Wang XY, Riewpaiboon A, von Seidlein L, Chen XB, Kilgore PE, Ma JC, et al. Potential cost-effectiveness of a rotavirus immunization program in rural China. Clin Infect Dis. 2009;49(8):1202–10.
You JHS, Ming WK, Tsang OT, Chan PK. Optimal gender-specific age for cost-effective vaccination with adjuvanted herpes zoster subunit vaccine in Chinese adults. PLoS ONE. 2019;14(1):e0210005.
Sartori AM, de Soarez PC, Novaes HM. Cost-effectiveness of introducing the 10-valent pneumococcal conjugate vaccine into the universal immunisation of infants in Brazil. J Epidemiol Community Health. 2012;66(3):210–7.
de Soarez PC, Sartori AM, Freitas AC, Nishikawa AM, Novaes HM. Cost-effectiveness analysis of universal vaccination of adults aged 60 years with 23-valent pneumococcal polysaccharide vaccine versus current practice in Brazil. PLoS ONE. 2015;10(6):e0130217.
Kebede TT, Svensson M, Addissie A, Trollfors B, Andersson R. Cost-effectiveness of childhood pneumococcal vaccination program in Ethiopia: results from a quasi-experimental evaluation. BMC Public Health. 2019;19(1):1078.
Zhao D, Gai Tobe R, Cui M, He J, Wu B. Cost-effectiveness of a 23-valent pneumococcal polysaccharide vaccine immunization programme for the elderly in Shanghai, China. Vaccine. 2016;34(50):6158–65.
Constenla DO. Post-introduction economic evaluation of pneumococcal conjugate vaccination in Ecuador, Honduras, and Paraguay. Rev Panam Salud Publica. 2015;38(5):388–95.
Jit M, Yuzbashyan R, Sahakyan G, Avagyan T, Mosina L. The cost-effectiveness of rotavirus vaccination in Armenia. Vaccine. 2011;29(48):9104–11.
de Blasio B, Flem E, Latipov R, Kuatbaeva A, Kristiansen IS. Dynamic modeling of cost-effectiveness of rotavirus vaccination, Kazakhstan. Emerg Infect Dis. 2014;20(1):29–37.
Flem ET, Latipov R, Nurmatov ZS, Xue Y, Kasymbekova KT, Rheingans RD. Costs of diarrheal disease and the cost-effectiveness of a rotavirus vaccination program in Kyrgyzstan. J Infect Dis. 2009;200(Suppl 1):S195-202.
Chotivitayatarakorn P, Chotivitayatarakorn P, Poovorawan Y. Cost-effectiveness of rotavirus vaccination as part of the national immunization program for Thai children. Southeast Asian J Trop Med Public Health. 2010;41(1):114–25.
Ahmeti A, Preza I, Simaku A, Nelaj E, Clark AD, Felix Garcia AG, et al. Cost-effectiveness of rotavirus vaccination in Albania. Vaccine. 2015;33(Suppl 1):A201–8.
Kim SY, Goldie SJ, Salomon JA. Cost-effectiveness of Rotavirus vaccination in Vietnam. BMC Public Health. 2009;9:29.
Kim SY, Sweet S, Slichter D, Goldie SJ. Health and economic impact of rotavirus vaccination in GAVI-eligible countries. BMC Public Health. 2010;10:253.
Wilopo SA, Kilgore P, Kosen S, Soenarto Y, Aminah S, Cahyono A, et al. Economic evaluation of a routine rotavirus vaccination programme in Indonesia. Vaccine. 2009;27(Suppl 5):F67-74.
Rheingans RD, Antil L, Dreibelbis R, Podewils LJ, Bresee JS, Parashar UD. Economic costs of rotavirus gastroenteritis and cost-effectiveness of vaccination in developing countries. J Infect Dis. 2009;200(Suppl 1):S16-27.
Liu N, Yen C, Fang ZY, Tate JE, Jiang B, Parashar UD, et al. Projected health impact and cost-effectiveness of rotavirus vaccination among children <5 years of age in China. Vaccine. 2012;30(48):6940–5.
Clark AD, Walker DG, Mosqueira NR, Penny ME, Lanata CF, Fox-Rushby J, et al. Cost-effectiveness of rotavirus vaccination in Peru. J Infect Dis. 2009;200(Suppl 1):S114–24.
Loganathan T, Ng CW, Lee WS, Hutubessy RCW, Verguet S, Jit M. Thresholds for decision-making: informing the cost-effectiveness and affordability of rotavirus vaccines in Malaysia. Health Policy Plan. 2018;33(2):204–14.
Tu HA, Rozenbaum MH, Coyte PC, Li SC, Woerdenbag HJ, Postma MJ. Health economics of rotavirus immunization in Vietnam: potentials for favorable cost-effectiveness in developing countries. Vaccine. 2012;30(8):1521–8.
Alkoshi S, Maimaiti N, Dahlui M. Cost-effectiveness analysis of rotavirus vaccination among Libyan children using a simple economic model. Libyan J Med. 2014;9:26236.
Atherly D, Dreibelbis R, Parashar UD, Levin C, Wecker J, Rheingans RD. Rotavirus vaccination: cost-effectiveness and impact on child mortality in developing countries. J Infect Dis. 2009;200(Suppl 1):S28-38.
Rose J, Hawthorn RL, Watts B, Singer ME. Public health impact and cost effectiveness of mass vaccination with live attenuated human rotavirus vaccine (RIX4414) in India: model based analysis. BMJ. 2009;339:b3653.
Lee D, Shin HY, Park SM. Cost-effectiveness of antiviral prophylaxis during pregnancy for the prevention of perinatal hepatitis B infection in South Korea. Cost Eff Resour Alloc. 2018;16:6.
Wang Y, Shi JF, Wang L, Yan Y, Yao H, Dai M, et al. Cost-effectiveness analysis of hepatitis B vaccine booster in children born to HBsAg-positive mothers in rural China. Int J Infect Dis. 2019;78:130–9.
Zheng H, Wang FZ, Zhang GM, Cui FQ, Wu ZH, Miao N, et al. An economic analysis of adult hepatitis B vaccination in China. Vaccine. 2015;33(48):6831–9.
Jia Y, Li L, Cui F, Zhang D, Zhang G, Wang F, et al. Cost-effectiveness analysis of a hepatitis B vaccination catch-up program among children in Shandong Province, China. Hum Vaccin Immunother. 2014;10(10):2983–91.
Tu HAT, de Vries R, Woerdenbag HJ, Li SC, Le HH, van Hulst M, et al. Cost-effectiveness analysis of hepatitis B immunization in Vietnam: application of cost-effectiveness affordability curves in health care decision making. Value Health Reg Issues. 2012;1(1):7–14.
Lu SQ, McGhee SM, Xie X, Cheng J, Fielding R. Economic evaluation of universal newborn hepatitis B vaccination in China. Vaccine. 2013;31(14):1864–9.
Moradi-Lakeh M, Shakerian S, Esteghamati A. Immunization against Haemophilus influenzae type b in Iran; cost-utility and cost-benefit analyses. Int J Prev Med. 2012;3:332–40.
Griffiths UK, Clark A, Hajjeh R. Cost-effectiveness of Haemophilus influenzae type b conjugate vaccine in low- and middle-income countries: regional analysis and assessment of major determinants. J Pediatr. 2013;163(1 Suppl):S50-S9 e9.
Muangchana C, Warinsatian P. Incorporation of private demand into cost-benefit analysis of a universal Hib vaccination program in Thailand. Southeast Asian J Trop Med Public Health. 2011;42(2):376–87.
Le P, Griffiths UK, Anh DD, Franzini L, Chan W, Swint JM. Cost-effectiveness of Haemophilus influenzae type b vaccine in Vietnam. Vaccine. 2015;33(36):4639–46.
Sribhutorn A, Phrommintikul A, Wongcharoen W, Chaikledkaew U, Eakanunkul S, Sukonthasarn A. Influenza vaccination in acute coronary syndromes patients in Thailand: the cost-effectiveness analysis of the prevention for cardiovascular events and pneumonia. J Geriatr Cardiol. 2018;15(6):413–21.
Perera S, John D, Senanayaka B. Cost effectiveness of dengue vaccination following pre-vaccination serological screening in Sri Lanka. Int J Technol Assess Health Care. 2019;35(6):427–35.
Novaes HM, de Soarez PC, Silva GA, Ayres A, Itria A, Rama CH, et al. Cost-effectiveness analysis of introducing universal human papillomavirus vaccination of girls aged 11 years into the National Immunization Program in Brazil. Vaccine. 2015;33(Suppl 1):A135–42.
Van Minh H, My NTT, Jit M. Cervical cancer treatment costs and cost-effectiveness analysis of human papillomavirus vaccination in Vietnam: a PRIME modeling study. BMC Health Serv Res. 2017;17(1):353.
Aguilar IB, Mendoza LO, Garcia O, Diaz I, Figueroa J, Duarte RM, et al. Cost-effectiveness analysis of the introduction of the human papillomavirus vaccine in Honduras. Vaccine. 2015;33(Suppl 1):A167–73.
Bishai D, Johns B, Nair D, Nabyonga-Orem J, Fiona-Makmot B, Simons E, et al. The cost-effectiveness of supplementary immunization activities for measles: a stochastic model for Uganda. J Infect Dis. 2011;204(Suppl 1):S107–15.
de Soarez PC, Sartori AM, de Andrade Lagoa Nobrega L, Itria A, Novaes HM. Cost-effectiveness analysis of a universal infant immunization program with meningococcal C conjugate vaccine in Brazil. Value Health. 2011;14(8):1019–27.
Sartori AMC, de Soarez PC, Fernandes EG, Gryninger LCF, Viscondi JYK, Novaes HMD. Cost-effectiveness analysis of universal maternal immunization with tetanus-diphtheria-acellular pertussis (Tdap) vaccine in Brazil. Vaccine. 2016;34(13):1531–9.
Lo NC, Gupta R, Stanaway JD, Garrett DO, Bogoch II, Luby SP, et al. Comparison of strategies and incidence thresholds for Vi conjugate vaccines against typhoid fever: a cost-effectiveness modeling study. J Infect Dis. 2018;218(suppl_4):S232–42.
Esmaeeli S, Yaghoubi M, Nojomi M. Cost-effectiveness of varicella vaccination program in Iran. Int J Prev Med. 2017;8:103.
Carlos F, Gomez JA, Anaya P, Romano-Mazzotti L. Health economic assessment of universal immunization of toddlers against Hepatitis A Virus (HAV) in Mexico. Hum Vaccin Immunother. 2016;12(1):52–63.
Sekandi JN, Dobbin K, Oloya J, Okwera A, Whalen CC, Corso PS. Cost-effectiveness analysis of community active case finding and household contact investigation for tuberculosis case detection in urban Africa. PLoS ONE. 2015;10(2):e0117009.
Datiko DG, Lindtjorn B. Cost and cost-effectiveness of smear-positive tuberculosis treatment by Health Extension Workers in Southern Ethiopia: a community randomized trial. PLoS ONE. 2010;5(2):e9158.
Steffen R, Menzies D, Oxlade O, Pinto M, de Castro AZ, Monteiro P, et al. Patients’ costs and cost-effectiveness of tuberculosis treatment in DOTS and non-DOTS facilities in Rio de Janeiro, Brazil. PLoS ONE. 2010;5(11):e14014.
Prado TN, Wada N, Guidoni LM, Golub JE, Dietze R, Maciel EL. Cost-effectiveness of community health worker versus home-based guardians for directly observed treatment of tuberculosis in Vitoria, Espirito Santo State, Brazil. Cad Saude Publica. 2011;27(5):944–52.
Vassall A, Siapka M, Foster N, Cunnama L, Ramma L, Fielding K, et al. Cost-effectiveness of Xpert MTB/RIF for tuberculosis diagnosis in South Africa: a real-world cost analysis and economic evaluation. Lancet Glob Health. 2017;5(7):e710–9.
Wang WB, Zhang H, Petzold M, Zhao Q, Xu B, Zhao GM. Cost-effectiveness of the Health X Project for tuberculosis control in China. Int J Tuberc Lung Dis. 2014;18(8):939–45.
Ramma L, Cox H, Wilkinson L, Foster N, Cunnama L, Vassall A, et al. Patients’ costs associated with seeking and accessing treatment for drug-resistant tuberculosis in South Africa. Int J Tuberc Lung Dis. 2015;19(12):1513–9.
Smith RD, Keogh-Brown MR. Macroeconomic impact of a mild influenza pandemic and associated policies in Thailand, South Africa and Uganda: a computable general equilibrium analysis. Influenza Other Respir Viruses. 2013;7(6):1400–8.
Acknowledgements
Mariana Siapka provided the tuberculosis cost-effectiveness analyses dataset included in the review.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Funding
Funding was received from the Dutch Ministry of Foreign Affairs (grant number DSO/GA-264/15).
Conflict of interest
Liv Nymark, Alex Miller, and Anna Vassall have no conflicts of interest that are directly relevant to the content of this article.
Ethics approval
Not applicable.
Consent to participate
Not applicable.
Consent for publication
Not applicable.
Availability of data and material
The datasets generated and/or analyzed in the current study are available from the corresponding author on reasonable request.
Code availability
Not applicable.
Author contributions
The study was designed by LN and AV. The search and extraction were carried out by AM. AM and LN conducted second reviews of the data extraction results. Analysis and interpretation of the data was carried out by LN. LN prepared the manuscript. All authors read and approved the final manuscript. LN is the guarantor of this work.
Supplementary Information
Below is the link to the electronic supplementary material.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License, which permits any non-commercial use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by-nc/4.0/.
About this article
Cite this article
Nymark, L.S., Miller, A. & Vassall, A. Inclusion of Additional Unintended Consequences in Economic Evaluation: A Systematic Review of Immunization and Tuberculosis Cost-Effectiveness Analyses. PharmacoEconomics Open 5, 587–603 (2021). https://doi.org/10.1007/s41669-021-00269-4
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s41669-021-00269-4