Introduction

Historically, infectious disease outbreaks have proven to be potentially devastating. A prominent example is the Spanish influenza which may have claimed as many as 50 million lives [1]. The number of outbreaks of infectious diseases has been increasing since 1980, as has the number of unique pathogens [2]. To prevent and effectively combat outbreaks, reporting agreements such as those arranged in the International Health Regulations (IHR) between national governments and international organizations were established [3]. The current IHR require the countries which ratified them to develop a minimum capacity of core functions related to surveillance and response [3]. However, with new threats emerging and given the fragile health systems in many parts of the world, outbreaks still have the potential to occur with potentially severe consequences in multiple countries. Therefore, there is a continuous pressure to improve available detection and response systems, and to increase the possibilities of preventing new threats from doing too much harm.

A recent example that illustrates the relevance of outbreak containment is the Ebola outbreak of 2014. The response to this outbreak received important criticisms, and as a consequence, the World Health Organization reformed, improving its response to infectious threats [4]. Aside from international organizations and non-governmental organizations, under the IHR nations are obliged to have at least a minimum threat handling capacity. However, countries are usually faced with limited healthcare budgets, which require prioritization of what to fund and in which disease areas to invest. Funding of detection and response facilities in case of an outbreak also needs to compete for available resources. Preferably, decisions on how to optimally allocate scarce healthcare resources are informed by sound estimates of potential costs and benefits of various policy scenarios. Assessing the cost-effectiveness of different prevention and treatment strategies is of utmost importance to ensure value for money and optimal health and welfare from the available budgets [5]. However, obtaining sound estimates of both costs and effects of intervention strategies, compared to a relevant comparator (such as the current situation or doing nothing), is not a straightforward task, and one that is full of methodological challenges.

To comprehensively capture the costs and benefits related to an intervention, numerous issues need to be considered, including the costs of the intervention itself, the incurred and avoided health losses, and the incurred and avoided treatment costs. A full analysis may also include elements such as production losses due to illness and premature death from the disease, or even broader economic impacts such as those due to reduced trade and tourism. Clearly, some of these elements may be more difficult to estimate and quantify. Importantly, in applied cost-effectiveness analyses, the decision regarding which costs to include depends on the perspective chosen. The societal perspective aims to capture all relevant costs and effects, regardless of where, when or on whom in society they fall [6]. Narrower perspectives, such as the patient’s perspective or a healthcare perspective, are sometimes used, which limits the scope of the evaluation. Especially for interventions targeted at preventing outbreaks, which can have rather broad impacts, adopting a societal perspective seems warranted [7]. Indeed, the impact of outbreaks is not confined to the healthcare sector and interventions to prevent or mitigate these outbreaks are often not confined to healthcare interventions (or funding). Note that when evaluating pandemics not only a broad range of cost categories in various sectors of the economy need to be considered but also the fact that a pandemic may trigger non-marginal changes in the healthcare sector and possibly the entire economy. Non-marginal changes in the health sector may occur when outbreaks cause capacity problems and displace a large portion of usual care within healthcare and outside the healthcare sector entire industries might be threatened. This suggests that the usual micro-economic perspective which is taken in economic evaluations is insufficient and a more macro-economic perspective might be more [8, 9].

Simulation models are often used to estimate the consequences of preventing or mitigating disease outbreaks [10]. Modeling of infectious diseases is typically done using either so-called static or dynamic transmission models [11]. Static models, such as decision trees and Markov models, assume that the probability of infection between individuals is constant over time. Dynamic models allow for the force of infection to be varied, and can include possible herd immunity effects [12]. Dynamic models are often considered to be more complex, but may be preferred to static models because they are able to take into account a varying transmission rate, which is highly relevant in this context [11]. Both types of models offer the ability to model different scenarios and interventions, and costs and benefits can be estimated using these models by linking them to events and/or states distinguished in the model [11].

An important challenge in infectious disease modeling is to account for behavioral responses that occur when under the threat of an infection [13, 14]. Whether or not individuals themselves take action in the face of an outbreak (threat) may introduce bias in the evaluation of a policy to mitigate an outbreak [15]. For instance, when the actual severity and the perceived severity of an illness diverge, this may complicate forecasts of the impact of interventions. Apart from the challenges in modeling the disease itself, there is also room for improvement in other parts of infectious outbreak policy evaluation. Previous research indicated that outbreak evaluations are often biased toward high-income settings and that little research is done in low-income regions [14]. High-income and low-income countries may face a different set of challenges, including different resource and capacity constraints, different threats and different living environments. Such differences need to be accounted for in evaluations and when attempting to translate results of interventions across settings. Furthermore, it should be acknowledged that an intervention, like setting up a surveillance system or response protocol, targeted at one specific disease may strengthen the healthcare system more generally. This means that the effects of such a measure could go beyond preventing and mitigating one particular type of outbreak. Such “policy spill-over effects” are rarely included [16].

The aim of this study is to review cost-effectiveness studies of major outbreak threats, based on WHO publications [17]. The focus of this review will be on investigating the methodological approaches used to estimate costs and (health) benefits, with the aim of improving our understanding of how evaluations of interventions related to outbreaks are currently conducted. This is key, because if decisions are to be based on available evidence, the evidence itself should preferably be comparable, valid and broad enough for policymakers to consider all relevant elements in the decision-making process.

Methods

To determine how costs and benefits in economic evaluations of interventions aimed at (potential) outbreaks are estimated, we first compiled a list of major outbreak threats of the 21st century. We based this on publications of the WHO which were produced for the meeting ‘’Anticipating Emerging Infectious Disease Epidemics’ [17]. The aim of selecting diseases based on this list was not to capture the most severe diseases or those that, in retrospect, turned out to be found the most costly outbreaks, rather we aimed to collect a broad sample of diseases that have the potential of causing large-scale health and economic damage. Future major outbreaks may have similar characteristics to their predecessors, implying that policy decisions regarding preventing or countering them will (need to) be based on similar information as found in the economic evaluations included here. In this review, we extracted information on study outcomes and methods, using a pre-determined protocol.

Data

We searched PubMed and SCOPUS in April 2018 for the following major outbreaks in the 21st century; SARS in 2003, H5N1 in 2003, H1N1 in 2009, Cholera in Haiti in 2010, MERS-CoV in 2013, H7N9 in 2013 and the West African Ebola outbreak in 2014. For this search, we constructed three blocks, which we used in combination and all terms were searched for in title and/or abstract. The full syntax for both Pubmed and SCOPUS is available in Appendix 1. The first block was the list of the relevant diseases in various combinations: Middle East respiratory syndrome coronavirus OR SARS OR H5N1OR H1N1 OR Cholera OR MERS-CoV OR H7N9 OR Ebola. The second block defined the study type: economic OR cost* OR costing. The third block complemented the second: benefits OR effectiveness OR cost-effectiveness OR cost–benefit OR cost-utility. Last, filters were applied to include studies from 2003 and onward and exclude studies with only animal subjects. We only considered articles published from 2003, given that we focused on the outbreaks of 2003 and later. We assumed that no articles had been published on the relevant outbreaks before their occurrence.

Study selection

We performed two screening rounds. In the first round, we screened articles based on title and abstract. In the second round, we screened full-text articles. Studies reviewed in full-text, but subsequently excluded, are shown with a justification for their exclusion in Appendix 2. We included peer-reviewed studies that conducted a quantitative economic evaluation of any form (cost-minimization, cost-effectiveness, cost-utility, or cost–benefit evaluations) with one or more comparators, and evaluated one or more interventions within the context of the outbreaks previously mentioned. We not only included studies based on actual reported case data but also included studies using measures of how infectious a disease is based on observations to model the outbreak, for example force of infection. We excluded review papers and only included studies written in English.

Data extraction and analysis

The in-depth reviewing of the selected studies focused on characteristics of the study setting (target disease, country, interventions evaluated), issues related to modeling, and finally, the included costs and health gains. We will elaborate on the latter two.

We extracted information about what type of model (dynamic or static) was used in the included studies, and how the studies dealt with uncertainty around estimates. Some models, such as microsimulations, are stochastic by definition while other models may employ various types of sensitivity analyses. Sensitivity analyses may be used not only to test uncertainties, but also to test different assumptions of the transmission model and the economic model. Such analyses may involve varying assumptions and parameters related to the specific setting of a study, which can inform the generalizability of the results to other settings, for instance other drug prices or intervention efficacies [18]. Thus, we also extracted information about the setting of the included studies and grouped these settings according to the World Bank Country and Lending Groups [19].

We divided costs into two categories: (1) costs that occur within the healthcare sector and (2) costs that occur outside of the healthcare sector. For both categories, we further divided the costs into short-term costs and future costs. We defined short-term cost as the costs that occur during the outbreak, and the future costs as those that occur when life is extended. Short-term costs within the healthcare sector are for example staff, equipment, and current treatment costs. Future costs within the healthcare sector include both future consumption of healthcare related to the specific disease being targeted and also future utilization of healthcare due to other diseases in life years gained [20].

Short-term costs outside the healthcare sector are costs that arise for example for the patient or the caregiver of a patient. These costs can be for transportation, time off from work to undergo treatment in a healthcare facility, or out-of-pocket expenses. Future costs outside the healthcare sector include productivity losses due to disability and premature mortality. Productivity losses are often estimated by methods such as the Human capital approach or the Friction cost method. The human capital approach quantifies the remaining productivity that would have occurred during all life years lost [21]. The friction cost method quantifies the time required to replace a worker by someone else, like a formerly unemployed person [22].

There is currently an ongoing debate on which future costs to include in health economic evaluations [23]. This particularly relates to costs in gained life years (i.e., those years that patients would not have lived without the intervention, but do with). If the aim is to comprehensively capture all impacts of an intervention, future costs and benefits, related to consumption and production, cannot be excluded from an analysis [20, 24].

For all cost categories distinguished we extracted information regarding the measurement and valuation of these costs and categorized them according to a micro-costing or a gross-costing approach. Micro-costing refers to the approach of costs’ estimation where the unit cost is multiplied by the used quantity of the referred unit, gross-costing; on the other hand, is when a budget is divided into sectors of usage [25]. Micro-costing is considered a more precise estimation of cost but may be more demanding in terms of data availability, and the sum may even exceed the total budget [25]. Gross costing is less data demanding but may misclassify costs between sectors. Finally, we checked whether studies took account of more disruptive effects on the healthcare sector and the wider economics to account for non-marginal impacts of a pandemic.

To fully account for all the relevant effects, the time horizon should be long enough to capture all costs and benefits of the intervention. Therefore, we extracted this information from the included articles. In addition, we extracted information about discounting of cost and health effects. Discounting is common in economic evaluations as the effects that occur in the present are valued higher than similar effects occurring in the future. The WHO-CHOICE uses an annual discount rate of 3% for both health effects and costs, but national guidelines may recommend different rate(s) [26].

Results

The literature search resulted in 298 records, of which 76 met the inclusion criteria and were assessed in full-text. Of the 76 records, 34 were considered eligible for inclusion in our study. The 42 excluded records were excluded due to not conducting any form of economic evaluation (10 records), methodology paper (6 records), not based on relevant outbreaks (4 records), effectiveness study (3 records), not in English (3 records), studying animal subjects (3 records), not quantifying the impact of an intervention against outbreak (3 records), reviews (2 records), not comparing intervention against baseline (1 record), being a preliminary study to an already included study (1 record), budget impact analysis (1 record), and not able to access (5 records) (Fig. 1).

Fig. 1
figure 1

Schematic flowchart of study selection process

Full size image

As shown in Table 1, H1N1 was the most frequently studied outbreak, with 29 of the included studies. Few studies compared more than two interventions. Pharmaceutical interventions (vaccinations and antivirals) were studied in 23 included studies. Vaccinations were most commonly studied, followed by school closure. Evaluated non-pharmaceutical interventions mostly consisted of strategies aimed at decreasing contact between infected and susceptible individuals. Only four studies compared pharmaceutical interventions with non-pharmaceutical interventions.

Table 1 Sample descriptive
Full size table

Of the included studies, 17 were cost-effectiveness analyses [27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42]. Cost-utility analyses were performed in 13 studies [43,44,45,46,47,48,49,50,51,52,53,54,55], and four studies performed cost–benefit analyses [56,57,58,59]. 29 studies were conducted in a high-income setting, 4 were conducted in an ‘upper-middle’ income setting and only one was conducted in a low-income setting. Of the high-income studies, a majority (i.e., 16 out of 29) were situated in the US (Table 2).

Table 2 Overview of included articles
Full size table

A dynamic model was used in 19 studies, while 11 studies used a static model. Four studies, all evaluating interventions against H1N1, did not use a transmission model and instead used trial data. One study evaluated the impact of individuals taking own initiative to have less contact with others, thereby aiming to reduce the risk of contracting the disease, in a sensitivity analysis [51].

Of all included studies, 30 conducted at least some sort of sensitivity analysis by varying parameter values. A univariate analysis was conducted in 19 studies, a probabilistic in 10 studies and a multivariate sensitivity analysis in one study [37]. For dynamic models, in which probabilistic sensitivity analysis is inherently difficult due to the parameters in the model being highly inter-dependent, univariate sensitivity analyses on key or all parameters were performed. Only 11 out of the 34 included studies discounted both costs and health benefits.

Nine studies did not mention the perspective used; however, several of those studies did include costs outside the healthcare perspective suggesting the use of a societal perspective. Fourteen studies used a societal perspective and six studies a healthcare perspective. Four studies assessed the costs and benefits from both a healthcare perspective and the societal perspective. One study used a patient perspective [27]. Of the studies stating a lifetime horizon, two included some types of future costs [51, 54].

Among the cost-effectiveness studies the outcome measure varied greatly: five used cases averted as outcome measure, four estimated the reduced attack rates, and two assessed life years lost [30, 42]. The remaining studies all used different outcome measures, including deaths averted [37], averted admissions [36], care quality indicators (such as turn-around time and emergency department recidivism) [29], proportion vaccinated [32], or days of sick leave per 100 healthcare workers [34].

All but two studies included treatment costs within the healthcare sector. Both of the studies that did not include these costs assessed the cost-effectiveness of school closures [42, 43]. Other included healthcare costs were administration costs (19 studies), equipment (two studies) [36, 56], co-payments (one study) [28], and costs due to days of sick leave of healthcare workers (one study) [34]. One study mentioned healthcare costs but subsequently did not define the costs explicitly [37]. Only one study included future non-related healthcare costs [51]. With respect to costs outside the healthcare sector, 24 studies included productivity losses due to short-term absenteeism, transportation (two studies) [39, 45], administration (one study) [41], treatment (one study) [39], presenteeism (one study) [49],and energy savings (one study) [45].

Ten studies included some form of future costs. Eight of these included future productivity losses, one included non-related medical costs [51] and one included related medical costs [54]. No study included more than one type of future costs. The studies that included productivity losses all used the human capital approach, basing calculations on wages and remaining life expectancy. One study included future related medical costs in the form of lifetime disability caused by the illness [54]. Another study included future non-related medical consumption by age based on insurance data in the US [51]. Four of the ten studies including future costs did not discount these costs.

When possible, we assessed the most likely costing method used, based on the (sometimes limited) information provided in the manuscripts. We refrained from labeling the costing method in two studies as the data used for costing were not described. The most common method found was micro-costing, which was used in 27 of the studies. Mixed costing methods using both micro- and gross-costing were the second most frequently used, while gross-costing was third. None of the studies took into account macro-economic effects of a pandemic.

Discussion

This study identified a substantial number of studies evaluating intervention strategies for important recent major outbreaks in terms of costs and benefits. We found a strong focus on the H1N1 outbreak and a clear bias toward high-income settings. We also found a discrepancy between pharmaceutical and non-pharmaceutical interventions being evaluated. The majority of the studies adopted a societal perspective but its operationalization varied substantially between studies, also in terms of which costs were included in the evaluation. Furthermore, although many studies modeled future health gains, the inclusion of future costs was limited. Also, none of the included studies included non-marginal effects that outbreaks might have on the healthcare sector and the wider economy.

In this study, we presented an overview of economic evaluations in multiple settings without restrictions to certain interventions. This allowed us to create an overview of the methods used in these economic evaluations of strategies to prevent or mitigate the consequences of major outbreaks. Our focus was on the economic aspects, rendering a comprehensive appraisal of the disease and transmission models used beyond the scope of this study. Still, we emphasize the need for high-quality transmission models in producing reliable economic estimations. In our search of the literature we did not find any studies that took into account more disruptive non-marginal effects of pandemics on the healthcare sector and the wider economy. This suggests that there is a gap between the research on the ex-post evaluation of a pandemic taking a macro-economic perspective and ex-post economic evaluations that estimate the impact of specific interventions.

Some limitations of our study need mentioning. First, our search strategy was broad, but may have missed specific studies. It seems unlikely this would have changed our results. Indeed, we believe that the included studies are relevant and form a sample large enough to base our conclusions on. Second, we searched for economic evaluations in relation to specific outbreaks. In particular, the sample of studies included in this review represents outbreaks that were identified as being potentially large threats. Other criteria could have been used for selecting outbreaks and interventions, which would have resulted in a different sample of studies. We cannot generalize to economic evaluations of interventions targeted at other outbreaks. For example, outbreaks, that may have or have had an even larger impact on health and society than the ones included here, may have been evaluated more extensively, potentially leading to different conclusions. Third, included articles were primarily screened by one researcher (KK). Having a second reviewer for all studies would have been more appropriate. Fourth, we encountered some difficulties in extracting the methods used and assumptions made in some studies. Given the level of information provided in those studies, we cannot rule out that some studies or methods were misclassified in this review. A more detailed presentation of the included elements, methods used and the data sources would facilitate the interpretation of the results and add to the transparency as well as the ability to replicate and compare studies.

To the best of our knowledge, there are no previous studies with a similar scope as ours. Previous reviews often applied a narrower scope by either restricting the search for a specific disease or to a specific setting. Pérez Velasco et al. [62] reviewed the strategies against influenza pandemics. Consistent with our results they found an overrepresentation of pharmaceutical interventions in high-income countries. Pérez Velasco et al. also assessed the quality of the included articles in their study, but focused less on variation in methods. A systematic review by Drake et al. [63], focusing on dynamic transmission economic evaluations of infectious disease interventions in low- and middle-income countries, highlighted the lack of reporting parameter values. This was also the case in our review. Drake et al. emphasized the lack in highlighting the uncertainty surrounding cost estimates in modeling studies. In our sample, we found a vast majority of studies using secondary cost data, with a large number of the studies performing a sensitivity analysis of the cost data. Specifically, many studies addressed uncertainty regarding parameters influencing prices or volumes either using uncertainty applied as a proportion of the mean price estimate or uncertainty regarding the mean cost estimates directly obtained. The number of parameters varied in the sensitivity analyses ranged substantially, from all too just a few. A possible explanation for this difference with the findings from the study by Drake et al. is that in our sample the studies mostly originated from high-income settings where the availability of data might be better. Drake et al. [63] proposed a value of information (VOI) framework to address the indicated shortcomings. This was also suggested by Pérez Velasco et al. [62]. VOI analysis may provide insights about potential beneficial areas to conduct further investigation. In addition, other topics could be addressed such as capacity constraints of the healthcare providers, especially in extra resource constrained or vulnerable settings [64]. A major outbreak with a large number of cases will require large efforts in any setting, which may affect the provision of other healthcare services when resources are diverted.

Our results show that there are large differences in the methods used to estimate the costs and benefits of different interventions. These differences can only very partially be explained by differences in the perspective adopted in the studies, as we found large differences within perspectives as well. Therefore, we conclude that there is a need to standardize which costs to include in economic evaluations in this context. Differences in the inclusion of costs will lead to difficulties comparing studies and their results. Moreover, excluding certain cost categories might create biases in results of economic evaluations and can be done strategically. By ignoring real costs, one also risks unwanted or unexpected effects when the intervention is actually implemented.

Another recommendation is to adopt a lifetime time horizon and to include all relevant benefits and costs during that period. This also implies that future costs need to be included in the evaluation. If life is prolonged due to an intervention, the life years gained can not only result in additional contributions to society (e.g., productivity) but may also result in additional costs, such as healthcare consumption and other consumption. Using long time horizons also increases the importance of discounting, which was not performed in all studies including costs beyond the outbreak duration. Not discounting future costs and effects may lead to biases in the results of an economic evaluation and its influence may be profound [65]. As no global standards exist on which costs to include and which rates to use for discounting costs and effects and whether these should be identical presentation of results with and without discounting (at varying rates) and with and without future costs would be a practical approach [66, 67].

The lack of evaluations from non-high-income countries and regions creates difficulties in generalizing the results to other countries and regions. The importance of this issue is emphasized by the fact that most of the burden of communicable diseases still occurs in low- and middle-income settings. The current bias may therefore leave exactly those policy makers who stand to gain most from better evidence on these matters without it.

Previous studies have addressed the challenge of incorporating behavioral aspects into infectious disease models [13, 68]. In the studies we selected, only one performed a sensitivity analysis in which the effect of individuals limiting their contact with others on their own initiative was explored [51]. This is a topic on which further research is needed, including aimed at standardization of how to include such behavioral changes in economic evaluations. Another topic which needs further research is the impact of outbreaks on the broader economy: the so-called disruptive effects. None of the included studies attempted to incorporate these effects, while they may have a substantial effect on the estimated cost-effectiveness of interventions. For instance, Prager et al. [69] estimated the economic costs of a pandemic influenza to amount to a possible $25 billion in the US. When incorporating avoidance and resilience behavior the potential loss grew to $43 billion. Further research is needed to link the outcomes of such studies to economic evaluations focusing on specific interventions. Based on our findings, we suggest that studies should strive toward more comprehensiveness in what they include and more standardization in terms of how to include relevant costs and (health) benefits. Future costs and productivity costs are two areas in which standardization is clearly required. We also emphasize the need for a presentation of all elements of costs and health effects in future studies in a manner that allows readers to scrutinize the data and methods used, and facilitates transferability of results. Adopting reporting standards such as Consolidated Health Economic Evaluation Reporting Standards (CHEERS) statement would be an improvement in this regard [70].

Conclusions

We note that inclusion of particular costs and benefits may have distributional consequences, also in the context of deciding on interventions aimed at the prevention and mitigation of potential outbreaks. For instance, including productivity losses in the evaluation of an intervention may favor interventions saving or targeted at younger, productive individuals, who participate in the paid labor force. Such distributional consequences should receive due attention, but are not solved by simply ignoring real costs like productivity costs. The increased costs of prolonging life also deserve mentioning in this context. These costs entail not only both costs of consuming healthcare in added life year but also the consumption of non-medical goods. It should be noted that these costs currently often are not included in economic evaluations [71].

Overall, this paper concludes that the evidence base regarding the cost-effectiveness of interventions targeted at preventing or mitigating the effects of major outbreaks at this stage is biased toward specific settings and outbreaks and methodologically diverse. Given the importance of the issue, effort should be taken to improve this.