Identified Studies
The flow chart in Fig. 2 describes the identification process of the relevant studies. Search branch ‘methods’ (branch 1 and 2) resulted in the identification of 42 [16–57] studies. Search branch ‘decision making’ (branch 3) identified 26 studies [9, 21, 44, 58–80].
Identified Aspects for Discussion
Based on the identified studies and experts’ feedback, 17 aspects were selected for discussion in the workshop. The results of expert discussion on each aspect are presented in Sects. 3.3.1–3.3.13 and 3.4.1–3.4.4.
Modelling Methods and Health Economics
Model Choice
Background [17, 18, 20, 22–25, 27, 28, 30, 32, 34, 37, 42, 44, 46, 47, 54]:
There are several types of models:
-
i.
Static cohort models;
-
1.
Decision tree,
-
2.
Markov model.
-
ii.
Population models;
-
a.
Static model,
-
b.
Dynamic model:
-
1.
Compartmental transmission dynamic models,
-
2.
Agent-based models,
-
3.
Discrete-event models.
Models from the category (ii-b) can intrinsically account for transmission of pathogens between individuals or population fractions that influence diseases transmission by indirect effects—dynamic models. Models from categories (i) and most of (ii) are often of a deterministic nature. Input parameters are set deterministically, and base-case results and uncertainty analyses are fully replicable. Models from category (ii-b-2 and ii-b-3) usually use a stochastic approach. For example, a study from Spain analysed the health economic effects of a seasonal influenza vaccination by using both a static and a dynamic model approach [46]. In contrast to the dynamic model’s results, the static model, neglecting indirect effects (such as herd protection), could not show that the influenza vaccination is cost saving. Hence, using a static model might underestimate a vaccine’s value [46].
Expert consensus:
-
For infectious disease modelling, the sole use of static models should always be justified. Static models can be used as a conservative estimate when there is no evidence for harm (e.g. age shifts with adverse effects) if indirect effects are ignored [4, 30, 34, 81, 82]. WHO has developed a flow chart that provides assistance when choosing an adequate type of model (Fig. 3) [4].
-
A challenge is to handle realistic demographic predictions in models (with a long time horizon) because of migration, demographic changes and scarcity of contact studies in (special) populations.
-
Stochastic models;
Advantages:
-
Simulate a more realistic world.
-
Can follow an individual’s life course, which is easier for decision-makers to understand.
-
In stochastic models, the randomness is of first-order uncertainty; therefore, they provide an alternative to account for heterogeneity (if events are not rare) in subgroups as it is done in a deterministic model [81].
Disadvantages:
-
Model calibration and the probabilistic sensitivity analysis (PSA) become more challenging and computationally intensive, hence, transparency might suffer.
-
Data sources may not be accessible, as more fine-grained (non-aggregated) data are needed.
-
A remaining challenge is to find adequate ways to conduct efficient uncertainty analyses on stochastic models [83].
Time Horizon of Models
Background [18, 27, 34, 37]:
Outcomes of static cohort models are typically estimated over a single cohort’s lifetime or a specific time horizon in which the disease typically occurs that is set as the time horizon of the model. In dynamic population-based models, the entire population is modelled, and the time horizon of those models has three phases:
-
1.
Run-in phase Dynamic models need a run-in phase to model the epidemiological conditions in the pre-vaccination era. This is important for a realistic implementation of the respective vaccination. The duration of the run-in phase impacts the model’s results.
-
2.
Evaluation phase This phase starts with the implementation of the vaccination. The duration of the phase should be set in the model as long as necessary to cover indirect (positive or negative) effects caused by the vaccination in the population.
-
3.
Steady-state After a certain duration of the evaluation, an epidemiological equilibrium is reached—the steady state—where the epidemiological variation terminates.
Combining model choice with time horizon, Mauskopf et al. [37] identified four categories of evaluation strategies (see Table 2). For example, a cost-effectiveness analysis of human papillomavirus (HPV) vaccination for Austria showed in sensitivity analyses that an extension of the model’s time horizon by almost three decades decreased the ICER from €50,000/life-years gained to <€0/life-years gained [84].
Table 2 Evaluation strategies of health economic models [37]
Expert consensus:
-
The time horizon of a dynamic model should last until the steady state is achieved in order to deliver valuable results. Hence, the model’s time horizon should not be defined prior to the analysis.
-
Dynamic population models should be evaluated by category 1 (Table 2). ICERs should be projected over time until their cumulative value stabilizes. However, if this takes more than two decades, cumulative ICERs should be presented for different time horizons (e.g. Kim and Goldie [34]).
-
Future research must analyse the impact of these different strategies.
-
A model usually should be run to epidemiological equilibrium, but it ideally should reproduce historical epidemiological (and demographic) values that may not be in equilibrium. There is a need for validation of pre-vaccination as well as post-vaccination epidemiology (whatever is available/applicable). However, this is often technically challenging and may suffer from lack of data.
-
A surveillance system should be implemented to monitor the impact of a vaccination programme. The results can be used to compare the real impact with the impact predicted in the model that was developed before the vaccine was implemented and to evaluate the model.
Natural History of Disease
Background [30, 52]:
Pathogen-specific naturally acquired immunity (including its waning) is usually an important feature in dynamic models. The way this is modelled has an influence on the results [exemplary model structures: ‘susceptible-infectious-susceptible’ (SIS), ‘susceptible-infectious-recovered’ (SIR), ‘susceptible-infectious-recovered-susceptible’ (SIRS), ‘susceptible-exposed-infectious-recovered-susceptible’ (SEIRS)]. In principle, the structure of a model should be developed on the base of the characteristics of the specific target disease, the vaccine of interest and the respective research question. However, for the same vaccine-preventable disease different compartment structures have been used in published studies. For example, a study analysing the health economic effects of seasonal influenza vaccination in Spain used an SIR approach, whereas a study considering the same disease in England used an SEIR structure [46, 85]. Particularly in health economic models for HPV vaccination, the use of an SIR and the use of an SIS structure are common, and the use of both structures can be justified [52, 86].
Expert consensus:
-
The model’s structure should represent the state of knowledge on the specific disease.
-
If uncertainty exists, structural uncertainty analysis should be performed [40, 87].
-
Better (sero-) epidemiological and immunological data are needed to inform such models.
Measures of Vaccine-Induced Protection
Background [16, 29, 42, 45, 54]:
Different vaccine efficacy (VE) measures can be considered in HEEs to account for protection against different outcomes such as infection, symptomatic illness/complication, and/or infectiousness.
If an infection does not cause any symptoms at all at any stage, and thereby does not require treatment and/or prophylaxis costs, there is no need to consider this in an economic assessment. For example, based on data from influenza challenge studies, Basta et al. [16] estimated absolute VEs for different endpoints. For example, vaccine efficacy for seasonal live-attenuated influenza vaccines in homologous seasons ranged between 40 and 90 % (VE for susceptibility 40 %; VE for infectiousness 50 %; VE for illness given infection 83 %; VE for infection-confirmed influenza illness 90 %).
When developing a model, the hierarchy of disease states/endpoints must be incorporated to consider the relevant type of VE.
-
There is a ‘sequential’ [targeting the first endpoint only, e.g. VE protecting against herpes zoster (HZ) only]
-
A ‘non-sequential’ [targeting all VE-relevant endpoints independently, e.g. VE protecting against HZ and also against HZ complication post-herpetic neuralgia (PHN)] approach
A study evaluating the cost effectiveness of a vaccine preventing HZ in the USA used, for instance, a sequential approach [88]. Hence, only VE against HZ was considered, and VE against the complication PHN was neglected. However, this approach was criticized by Brisson and colleagues [89, 90], who used a non-sequential approach when evaluating the cost effectiveness of an HZ vaccine in Canada. In this study, the model was informed with both VE against HZ and VE against PHN, and delivered results more in favour for the vaccine [90].
In clinical trials, two different approaches of analysis are often used to measure VE:
PP usually produces more favourable VE results for the intervention/vaccine than ITT.
Furthermore, in models, the degree of protection and take can be distinguished.
-
The degree of protection (or leaky protection) is the percentage of (partial) protection in successfully vaccinated individuals (e.g. 100 % of vaccinated individuals have a protection of 50 %).
-
Take (or ‘all or nothing’) describes the percentage of successfully vaccinated individuals with full protection (e.g. 50 % of vaccinated individuals have a protection of 100 %).
A mathematical model, for instance, calculated the impacts of fictional HIV vaccines on seroprevalence, depending on whether the vaccine leads to a take or a degree protection. The seroprevalence proportion was, after several decades, lower when using a take protection than a degree protection [91]. This difference can impact the overall results of a health economic model [92].
Finally, a major challenge is a lack of clinical endpoints in clinical trials, especially when immunogenicity (or other surrogate of protection) is the outcome considered for licensure. The preferences, whether to use efficacy or effectiveness data in HEEs, are rather diverse in European guidelines [14].
Expert consensus:
-
The model structure should account for the type of VE measure incorporated. VE in terms of reducing susceptibility to infection is fundamentally different to VE reducing infectiousness. These different aspects of VE have a differential impact on the results. Modelling can be used to estimate unknown parameters including VE estimates by using, for example, a Bayesian framework utilizing Markov Chain Monte Carlo inference [93]. More studies assessing VE against infectiousness are warranted (e.g. challenge studies).
-
Model structure and the decision-maker’s research question determine the use of a sequential or non-sequential approach.
-
ITT data, when available, should be taken for base-case analyses and PP data for uncertainty analyses. However, the use of PP data for the base-case is sufficient when the difference between ITT and PP data is completely explained by the different proportions of susceptible individuals in the study population, since this is ideally incorporated in a model. PP data should be preferably chosen if a specific result on vaccine-dose compliance and/or completion of a vaccine course is of relevance.
-
The choice of representing VE with degree of protection versus take depends on the type of protection conferred by the vaccine of interest. When there is no evidence on whether the vaccine confers a leaky or an all-or-nothing protection, different approaches to account for vaccine efficacy (leaky or all-or-nothing or combination) should be used.
-
The quantitative relationship between immune response and the degree of vaccine-induced protection against clinical disease is often unclear. Validated surrogates can be considered if no clinical endpoints are available.
-
The impact of negative vaccine effects, both at an individual level (i.e. adverse events) and at a population level (i.e. replacement or age shift) needs to be considered. Cases of vaccine-preventable diseases and cases of adverse events are equally relevant outcomes.
-
Vaccine manufacturers currently have little incentive to collect some specific clinical data (e.g. head-to-head comparisons of different vaccine products); however, these are relevant for modelling and public health. For comparison of different studies, standardized case definitions for clinical outcomes are needed, and methods for the implementation of such indirect comparisons into models should be developed/standardized.
Duration of Vaccine-Induced Protection
Background [30, 52]:
The waning of vaccine-induced protection plays a major role when modelling vaccine effects. Clinical trials are often too short to generate robust data on the duration of vaccine protection. In a model, either a lifetime duration (neglecting waning) is assumed or waning is incorporated. Waning can be designed to start right after vaccination or after a delay period, and to decay in different ways (e.g. exponential, stepwise). With the example of HZ vaccination, it can be shown that different assumptions on the VE waning rate can impact the health economic results: in a study from Germany, vaccination at the age of 60 years resulted in an ICER of €21,565/quality-adjusted life-year (QALY) gained or €34,606/QALY gained assuming an annual waning rate of 1 or 20 %, respectively [94]. In this model, when vaccinating individuals at the age of 50 years, the same waning rates led to €18,486 and €43,701/QALY gained, respectively. Especially in health economic models on HPV vaccination, the waning of vaccine-induced protection is an influential determinant of overall results [52, 86, 95].
Expert consensus:
-
If vaccine waning is not well understood, then different waning scenarios should be considered in an uncertainty analysis and their impact compared. Immunological memory can be integrated.
-
The availability of detailed trial data on vaccine-induced protection and waning at the patient level would enable more rapid and less uncertain economic evaluations following the marketing of a new vaccine.
Indirect Effects Apart from Herd Protection
Background [18, 20, 27, 30, 37]:
Vaccination-specific negative externalities such as age shift of peak incidence, serotype replacement or impact on antibiotic resistance might be relevant. The cost effectiveness of pneumococcal vaccination, for example, is strongly influenced by the degree of serotype replacement that is expected after vaccine introduction. This influence was shown by van Hoek et al. [96] in a study on the cost effectiveness of a 13-valent pneumococcal conjugate vaccination for infants in England. Brisson and Edmunds [97, 98] illustrated that the routine infant vaccination against varicella is expected to increase the average age at infection. This age shift is caused by two effects: the cohort effect and the herd protection effect. If the disease severity increases with age at infection, this age shift negatively impacts the overall health-economic results.
Expert consensus:
-
Ecological effects such as intra-population immune boosting following exposure to a pathogen (cf. varicella-zoster virus), replacement of pathogen strains covered by a vaccine by strains not covered (e.g. pneumococcal serotypes) and antibiotic resistance should be part of uncertainty analyses whenever they are possibilities.
-
Ideally, pathogen replacement, eradication, genetic selection in host, changes in behaviour (e.g. screening uptake, risk behaviour such as unprotected sex or social mixing), weakening of maternal immunity, and using vaccination as a platform for adding other interventions should also be considered wherever relevant.
Target Population
Background [20, 23, 29, 30, 42, 43]:
Besides the question of whom to vaccinate (e.g. total population at a certain age versus risk groups) it is necessary to consider how populations/groups mix with each other. In dynamic models, contact patterns and the mixing matrix influence the model outcome. The literature offers data gathered by different approaches, e.g. survey-based (e.g. POLYMOD) and synthetic (social demographic data) contact matrices. For example, to understand contact patterns of infants, a study group sent contact diaries to a representative sample of mothers in the UK. Thereby, the average number, the type, and the duration of daily contacts of infants were documented. These data can serve as important input parameters for dynamic models evaluating the impact of vaccines [99]. Since observational data on contact patterns in the form of age-specific contact matrices are difficult to gather and are currently available only for few countries, Fumanelli et al. [100] presented a computational approach. Based on the simulation of a virtual society of agents based on data collected in eight European countries, contact patterns by age can be estimated for 26 European countries. This approach might serve as a valuable alternative to observational data [100]. Such synthetic contact matrices have been used by Poletti et al. [101] in a varicella-zoster virus study.
Expert consensus:
-
Survey-based mixing data are considered most adequate and should be used wherever possible. Synthetic methods need to be validated against survey data where available.
-
Future research should better measure contact patterns for children and parents, and evaluate what kind of contact is relevant. Knowledge is still lacking about how well contact patterns represent occasions for transmission for specific infections. It is therefore important to assess which subset of mixing data (e.g. contacts involving touching) provides the best fit to relevant observational data for the disease in question (e.g. seroprevalence data). Research funders need to understand that contact matrices are important for dynamic model projections, and that these differ between countries or regions.
Model Calibration and Validation
Background [18, 23, 34, 40, 42, 50, 51, 53, 55]:
Calibration means estimating and adjusting a model’s parameters to generate the expected outcomes observed in real life. The literature reports several approaches such as manual, random (e.g. Monte Carlo) or optimized (e.g. Nelder-Mead [102]) approaches. External validation compares the results of the model with the best available evidence. For example, Kim et al. [103] present calibration methods on how to use real-world data to develop a comprehensive natural history model of HPV. Hence, a calibrated model that fits to real-world data tends to generate more reliable results [103]. However, Basu and Galvani [104] claim that a Bayesian approach provides several advantages compared with the approaches presented by Kim et al. [103].
Expert consensus:
-
The manual calibration approach should be based on a structured process, and the algorithm should be reported. However, the random or optimized approach is considered to be more adequate [53, 105–107]. A random calibration approach may have an identifiability issue. Hence, researchers should make sure that the shape and range of the posterior distribution is plausible.
-
A plain visual validation is not considered sufficient. Instead, the use of goodness-of-fit criteria is recommended.
-
The dataset used for validation should be independent from that used for calibration (ideally even with different endpoints). An alternative is to hold back a portion of the calibration data (e.g. test/training datasets). An alternative option is a cross-model validation approach in which the same data are used on different models. Lack of data points might be a limitation.
Handling Uncertainty
Background [18, 22, 26, 30, 34, 37, 42, 45, 48, 54]:
Structural (or model) uncertainty can be handled by scenario analysis (i.e. presenting results for different models), model averaging or parameterizing the structural uncertainty [108–110]. Parameter uncertainty is quantified with PSA. However, PSAs are often not performed in dynamic models because they are computationally difficult (especially to include the uncertainty of parameters affecting transmission). Alternatively, parameters affecting transmission are excluded from PSA, or two models are developed: an economic sub-model including PSA and a dynamic sub-model focusing on transmission-specific issues. Furthermore, there are many vaccination-specific key aspects for uncertainty analysis, such as duration of vaccine protection, vaccination coverage in dynamic models, time horizon, boosting, contact patterns and targeted age/risk groups. For example, to consider both indirect effects caused by vaccination and PSA, Christensen et al. [111] developed two models (static and dynamic) to evaluate the potential impact of introducing vaccination against serogroup B meningococcal disease in the UK. The static model without parameters affecting the transmission was used to perform PSAs. The dynamic model including parameters affecting the transmission was used to account for indirect effects [111].
Expert consensus:
-
All identifiable sources of uncertainty should be accounted for, if not by PSA then by other analyses. The parameter distributions used in PSA need to be justified. Transparency is important because dynamic models can have many ‘deep parameters’ (i.e. parameters that are not directly observable, such as the probability of infection transmission per contact event). Calibrated parameters also need to capture information about uncertainty [87, 112].
-
Decision-makers need to understand the relevance of uncertainty analyses. PSA can help to identify future research priorities. However, uncertainty measures in calibrated parameters remain challenging.
-
Structural uncertainty should be parameterized where possible, but uncertainty in normative aspects such as perspective, vaccine price and discounting should not be analysed in PSA [87, 112]. Vaccine coverage should be varied between desirable and undesirable levels. Uncertainty in contact patterns should be parameterized wherever possible (see Table 3).
Table 3 Definition of uncertainty analyses based on expert consensus and literature [22, 30, 42, 55, 87, 112]
Discounting
Background [1, 18, 19, 25, 35, 38, 39, 41, 42, 44, 45, 49, 54, 56, 57, 113]:
Uniform discounting (using the same rates for costs and health effects) is most commonly applied in HEEs. However, this approach might considerably influence the long-term ICERs of vaccination in particular, since costs and benefits usually occur at different points of time. Differential discounting (with lower rates for health effects) is used, for example, in Belgium, the Netherlands, Poland and, until 2004, the UK, for the assessment of all health technologies, including vaccines. Differential discounting seems to be technically feasible and fairer from an ‘inter-generational’ perspective [114]. Different recommendations on discounting approaches, for example, drugs and vaccines within one country, do not exist. If lower discount rates for health effects are considered, the level of discount rate for health effects needs to be set separately. Another issue in this context is whether a constant or a changing discount rate over time is used. Constant rates are more widely used, possibly because of pragmatism and ease. In France, the first 30 years are discounted with a uniform rate of 4 %, and years thereafter at 2 %. This ‘slow’ discounting procedure is also recommended by WHO if the effects of vaccination begin only long after the intervention (e.g. vaccination against HPV) [4]. The type of decline can vary (e.g. stepwise, linear, or exponential). For example, Westra et al. [57] used nine different discounting approaches and rates in a model evaluating the cost effectiveness of the HPV vaccination in the Netherlands. Ceteris paribus, the ICERs ranged between €7,600/QALY gained and €165,400/QALY gained.
Expert opinionFootnote 1:
-
The majority of, but not all, experts recommended differential discount rates for costs and effects in HHEs (exclusively in cost-utility and cost-effectiveness analysis) if the model’s time horizon is long (e.g. >20 years).
-
The discount rate of health effects could be around 50 % of the discount rate for costs. However, for a more evidence-based recommendation, empirical research has to be conducted [39].
-
Constant discount rates over time should not be applied in models with a long time horizon (e.g. >20 years) according to a majority of experts.
-
Further research on the, to date rarely applied, approach called time-shifted discounting approach is needed [39, 57, 115].
-
Since discount rates and discount approaches usually have a major impact on results of HEE of vaccines, the variation of these aspects need to be analysed (see Sect. 3.3.9) and explained to decision-makers.
Health-Related Quality of Life and Quality-Adjusted Life-Years as Outcome Measures
Background [20, 25, 32]:
QALYs are a common outcome measure used in cost-utility analyses and are accepted in several countries in Europe [14]. Since vaccine-preventable diseases often affect children, the impact on health-related quality of life (HR-QOL) of carers measured as QALYs is important in HEEs of vaccines. Another prevention-specific issue is utility in anticipation, measured as QALYs. Vaccinated individuals might experience a higher HR-QOL, because they feel ‘protected’ after vaccination. As counterpart, fear of adverse events measured as QALYs also needs to be mentioned. That might lead to an HR-QOL decrease, but on a rather short time horizon. For example, Weinke et al. [116] used a survey to assess the impact of HZ and PHN on the patients’ life but also on life of the family members who cared for them. Most family members (69 % children; 80 % life partners) of patients with HZ or PHN reported that caring for the patient caused a moderate to severe impact on their life. Hence, these impacts might have an effect on overall health economics results [116].
Expert consensus:
-
HR-QOL of carers should routinely be considered in uncertainty analysis in both payers’ and societal perspectives. However, input data might be scarce.
-
Utility in anticipation and fear of adverse events are also difficult to consider due to limited data. When appropriate data are available, they might be considered in uncertainty analyses.
Cost Components
Background [18, 32, 36]:
Even though the literature and national guidelines on (direct or indirect) cost components seem very comprehensive, some factors are vaccine specific and require attention. Traditionally, vaccines mainly protect against diseases occurring in childhood. Since sick children need care, indirect costs of carers (e.g. productivity loss) occur. Set-up costs (e.g. of vaccine campaigns) might also play an important role in immunization programmes. Depending on the disease and the target group, campaigns are needed to reach as many individuals as possible. Respective national guidelines often neglect such cost components. For example, in the model by Hornberger and Robertus [117], that evaluated the cost effectiveness of HZ vaccination in the USA, the price for the vaccine included the unit vaccine cost, a public awareness campaign, administration costs, patient travel time and time receiving vaccine as well as the cost of treating adverse events. Implementing $US500 per vaccine dose caused high ICERs of between $US280,000 and $US560,000/QALY gained [117].
Expert consensus:
-
Indirect costs of carers should be considered for both perspectives. From a payers’ perspective, they are considered as sick pay (if the payer has to cover these costs) and from a societal perspective as productivity loss.
-
If set-up costs (e.g. for campaigns) are not included in the vaccine price (i.e. promotion and distribution of a vaccine is not done by its manufacturer), they should be considered in the perspective that covers these costs.
Perspectives
Background [18–20, 32, 54]:
Some prevailing methodological primers recommend using the societal perspective when assessing vaccination programmes designed to improve public health. A recent systematic literature review on varicella and HZ vaccination concluded that the varicella vaccination (neglecting the impact on HZ) becomes cost saving when switching from a payer to a societal perspective [118]. Hence, indirect costs especially of carers when considering infant vaccination, tend to have a major impact on overall health economic results [118]. However, for example, if costs for vaccination are higher from a societal perspective due to co-payments, the overall results from the societal perspective can become less cost effective when compared with those from the payer perspective [119, 120].
Expert consensus:
Decision Making
Background [9, 24, 41, 58–70, 72–79]:
Purposes of Health Economic Evaluations in Decision Making
From the decision-makers’ perspective, results from HEE of vaccines can be utilized to identify the most efficient vaccination strategies (e.g. targeting the total population or specific age or risk groups—technical efficiency), to support yes/no decisions for vaccine introduction, and/or to support price negotiations with manufacturers (allocative efficiency). Furthermore, the introduction of a new vaccine usually substitutes for curative treatment and/or screening measures in a healthcare system. Those substitutions are also of relevance for decision-makers and are usually considered in HEEs. Furthermore, the experts clearly stated that additional budget impact analyses (BIA) can be useful, especially when a costs of a prevention/intervention measure is extremely high and affordability is unclear.
Integration of Health Economic Results in Decision-Making Processes
The introduction of a new vaccine into a healthcare system affects several aspects of the system. Hence, many questions, such as acceptance, practicability or equity in access, must be addressed. Erickson et al. [62] provide a comprehensive view on all factors that could be considered in immunization decision making. A broad micro- and macro-economic view might be useful. Many but not all questions can be addressed within a health economic model, but should also be considered within a broader appraisal. However, details of a broader appraisal were not subject to this framework. Furthermore, other health economic approaches, such as cost-benefit analyses or BIAs, may also serve as a basis for decision making. However, which or how many approach(es) are considered for decision making is a rather normative question that needs to be address by relevant stakeholders at the national level. Therefore, experts concentrated in this framework on incremental cost-effectiveness and cost-utility analyses, that are the most commonly used forms of HEEs for national decision making in Europe [14]. Yet, the experts acknowledged the usefulness of broader and alternate approaches in the economic analysis of vaccines.
In general, the experts cautioned that lack of transparency and high complexity of evaluations leads to results from HEEs appearing like a black box to decision-makers. Based on a decision making continuum, experts stated pros and contras of three decision making approaches: pure threshold, multi-criteria decision analysis (MCDA), and informal judgment (Fig. 4). Decision making based on a pure threshold is transparent. Especially if the budget is very constrained, a fair use of public resources is possible and it is a useful tool for price negotiations with manufacturers. However, pure threshold does not consider clinical/epidemiological severity or budget impact. Furthermore, the choice of the level or range of the threshold sometimes seems rather arbitrary, and in some countries such willingness-to-pay thresholds are not accepted. MCDA is also transparent and applicable without a threshold. However, MCDA is very complex, needs a well-developed design and requires proportional weights of each criterion. Clearly, in terms of MCDA, more research is needed. In terms of informal judgment, experts recognized advantages such as applicability without a threshold and comprehensive summary of many parameters related to the respective vaccine and disease, including issues that are difficult to quantify (e.g. implementation issues or acceptability of a vaccine).
Key Parameters that Should be Varied in Uncertainty Analyses
The experts made clear that parameter uncertainty should be considered in PSA (see Table 3; Sect. 3.3.9). Additionally, the experts recommended including the variation of age and risk group, vaccination schedule, herd protection, booster or catch-up vaccination, delivery strategy and vaccination coverage in scenario/uncertainty analyses.
Vaccination-Specific Aspects of Reporting Results
Experts listed aspects that are essential to be reported in HEEs:
-
Discounted and also undiscounted results should be presented.
-
Cumulative results should be reported at various time points over a model’s construed decision horizon, including a longitudinal view up to the end of the defined time horizon of the model.
-
Results from various relevant perspectives.
-
Cost-effectiveness acceptability curves (CEACs).
-
Best- and worst-case scenarios.
-
Absolute values and ICERs for all disease-specific outcomes.
-
A report of an HEE should describe the validation/calibration process, the strength of evidence behind the input data, and should discuss the potential variation of results in uncertainty analyses.
-
The most recent questionnaire [71] assessing the credibility of a modelling study needs at least one infectious disease-specific addition: “If applicable, why was a dynamic model not used?”