Meningococcal disease is a potentially fatal bacterial infection of the membranes and cerebrospinal fluid surrounding the brain and spinal cord. Complications of the disease include permanent brain damage, blindness, hearing loss, kidney failure, amputation and chronic neurological disorders. In developed countries, most cases are caused by Neisseria meningitidis and Streptococcus pneumoniae, in large part due to routine vaccination against Haemophilus influenzae type b (Hib).

The evolution of meningococcal vaccines is fairly typical of many vaccines. Meningococcal vaccines were first developed in the 1960s, and polysaccharide vaccines active against multiple serogroups have been available since the 1970s.[1] The polysaccharide vaccines saw limited (non-routine) use due to their poor immunogenicity, particularly in young children. In the 1990s, monovalent and then polyvalent conjugate vaccines were introduced. The conjugate vaccine produces a stronger, T-cell-dependent response that is more durable than the immune response generated by the polysaccharide vaccines, particularly in older children.[1] Evidence from mass immunization campaigns in Canada and the UK suggests that meningococcal vaccination programmes are extremely effective.[2,3] Vaccines against additional serogroups (A and B) and in combination with other agents such as Hib and S. pneumoniae are in development or have already been introduced. Other improvements might include reductions in the number of doses necessary to confer adequate immunity among children. Innovations that come at a cost clearly have relevance for continued economic evaluation.

Furthermore, multiple competing vaccination programme strategies must also be considered, and this has certainly been the case with meningococcal vaccines. The economic evaluation of meningococcal vaccines thus reflects the interaction between the intrinsic value of the vaccine in preventing disease and the use of that vaccine as part of a specific vaccination programme. While the same might be said of many medical interventions, the difference with vaccines is that the number of alternative strategies can be significant. For example, an economic evaluation of two drugs to treat a chronic disease would typically consider one or two dosing options for each drug. While most economic evaluations of meningococcal vaccines do consider only two or three options at a time, there is a rather large number of possible combinations in terms of the number and timing of doses given, the age group(s) targeted for vaccination, whether the vaccine is given routinely or as part of a mass immunization campaign, or whether catch-up vaccination is included.[4]

Vaccines against infectious diseases are unique among medical interventions in at least two other ways. First, vaccines, more so than other medical interventions, have great potential to produce cost savings, even when only direct medical costs are considered.[5,6] For newer, increasingly expensive vaccines, direct medical cost savings may not be enough to offset programme costs. In other words, some vaccine programmes will add to health expenditures. That is not to say that those programmes do not represent good value for money, but, rather, like most medical interventions, they do not generate cost savings unless offsets among indirect costs such as patient time costs, morbidity- and mortality-related productivity effects and the benefits of herd immunity are considered. As the cost of vaccines and vaccine programmes increases, estimates of direct medical cost savings alone may be insufficient to demonstrate the economic efficiency.

Second, the process of vaccination creates positive externalities in the form of herd immunity, which serves to expand the health benefits of vaccines beyond those individuals who are vaccinated. Modelling and incorporating this latter aspect of vaccines into the technology assessment process is challenging but may be necessary in order to demonstrate the efficiency of vaccines targeting less prevalent and/or less serious diseases. For older vaccines that targeted highly prevalent conditions such as those against pertussis and measles, mumps and rubella, the direct benefits of vaccination were enough to support their routine use.[6–8]

Improvements to existing meningococcal vaccines and the development of new vaccination strategies require continued assessments of efficiency. Several reviewers have recommended empirical studies of existing vaccination programmes to help identify cost-effective implementation strategies.[9–11] Such analyses, by definition, are available only after a programme has been chosen and instituted. Thus, the need for economic evaluations to guide policy decisions remains, and those evaluations will necessarily include at least some modelling. This article reviews existing economic evaluations of meningococcal vaccine and proposes a ‘minimum analysis profile’ for the reporting of methods and results, with a focus on methods of evaluation and the purpose of informing future evaluations.

1. Literature Search and Retrieval

A search of the published medical literature was conducted using the PubMed database of the US National Library of Medicine and the National Institutes of Health. A search of (meningococcal OR meningitis) AND (economic evaluation OR cost effective* OR cost benefit), the same search as that used by Drummond et al.,[10] yielded 346 citations. After limiting the search to English-language articles pertaining to humans, 297 titles remained. Full-length articles that contained an economic evaluation of one or more meningococcal vaccines or vaccine strategies in developed countries were selected for review (n = 16).[3,4,12–25] Bibliographies of selected articles were reviewed to locate additional related work; no further economic evaluations were identified.

Each study was carefully reviewed, with particular attention paid to the following aspects of the analysis: the vaccines and/or vaccine strategies being compared; methodological approach (model-based or empirical); modelling approach (static or dynamic, for modelling studies); inclusion or exclusion of indirect (herd immunity) benefits; indirect (productivity) costs; quality of life (QOL) considerations; and reporting of results. The adequacy of sensitivity analysis and assumptions around key parameters such as vaccine uptake were also assessed.

Several authors have compiled reviews of both published and unpublished evaluations of meningococcal vaccine use in developed countries,[10,11,26–28] the most recent in 2008.[27] While a key objective of each review was to summarize the existing economic evidence of vaccine cost effectiveness, the overall purpose of the reviews varied somewhat. The earliest review, by Getsios et al.,[11] was conducted to assess the adequacy of existing cost-effectiveness evidence for meningococcal vaccine. They concluded that more research was needed, particularly in the area of dynamic modelling and the incorporation of herd immunity. Subsequent reviews[10,26,28] also noted the need for dynamic modelling, and now several dynamic models — not all of them economic — have been developed.[4,17,29] However, all reviews noted that the disparity in approaches to the evaluation of vaccine programmes limited the ability to make cross-study comparisons and definitive conclusions about the cost effectiveness of most meningococcal vaccination strategies.

In their investigation of the role of economic evaluation in the development of meningococcal vaccine policy, Welte et al.[28] specifically noted a need for national and international modelling guidelines to facilitate comparisons across economic evaluations. Four analyses have been published since the earliest review.[4,13,15,24] Two studies used the same model; the more complete report and the other two studies are briefly summarized in table I. The discussion of methodological considerations that follows is based on a review of all the studies, although the focus is on the recent studies that have not been included in previous reviews.

Table I
figure Tab1

Summary of recent economic evaluations of meningococcal vaccines

Full size image

2. Methodological Issues

2.1 Direct Costs

The calculation of direct medical costs associated with diagnosis and treatment of disease is relatively straightforward. While the unit costs utilized in a particular study may not generalize to other environments, the widespread use of proxies for the real opportunity cost of health resources at least avoids the distortions associated with the use of list prices or charges. Of greater concern is the inclusion of costs that are likely to impact the analysis.

One item that is often neglected is the added cost of emergency control measures that may be taken in the event of an outbreak.[11] Assuming that (i) the purpose of a vaccination programme is to avoid outbreaks; and (ii) the cost of managing outbreaks is large, the failure to include the costs of managing outbreaks will bias the evaluation in favour of no vaccination. However, it may be difficult, particularly under a static modelling approach, to reliably estimate the frequency and magnitude of disease outbreaks in the presence of a (presumably routine) vaccination programme. In such cases, investigators should consider a base case that excludes such costs. If the base-case analysis supports the cost effectiveness of the programme, including outbreak control costs will not impact the analysis and need not be considered. When the conclusion is not straightforward, sensitivity or scenario analyses should be used to quantify the possible impact of cost savings associated with reduced outbreak frequency. Furthermore, investigators should consider that emergency resources utilized during outbreak control may have higher opportunity costs than similar resources used during routine medical care (e.g. if a shortage of healthcare providers is created or if specialized care is required).

Most published analyses have utilized a societal perspective in which both direct medical costs and at least some indirect costs were considered. Depending on the environment, some or all indirect costs may be irrelevant from the perspective of the decision maker. Or, even if relevant, the decision maker may desire information on the individual effect of indirect costs on the analysis to help guide policy making. In an effort to improve comparability across analyses and enhance the usefulness of the information to decision makers, direct medical costs and indirect costs (when considered) should be considered separately.

2.2 Indirect Costs

Many early evaluations of vaccines considered only the savings in direct medical costs associated with avoided cases of disease when estimating cost effectiveness (or cost benefit), and those savings were enough to justify programme implementation. Over time, direct medical cost savings were insufficient to offset vaccination costs, and investigators began to consider the impact of vaccines in avoiding indirect costs as well (Lieu et al.[30] provides an excellent example). When direct medical cost savings are not enough to offset programme costs, the implementation of vaccination programmes will add to health expenditures. That is not to say that those programmes do not represent good value for money. Rather, like most medical interventions, they may not generate cost savings in the conventional sense. The inclusion of indirect costs dramatically improves the chances of demonstrating overall cost savings, since indirect costs related to patient productivity losses may exceed direct medical costs by a factor of two or more when calculated via the human capital approach.[15,16]

Before proceeding, it is necessary and useful to define precisely what is meant by the term ‘indirect costs’. In the context of vaccine evaluation, indirect costs generally refer to one or more of the following: patient and/or parental time involved in receiving vaccination; parental time associated with caring for children in cases of disease; reduced patient productivity in the event of long-term disease sequelae; and productivity losses associated with death. Lost time need not be restricted to productivity losses, although lost leisure time has rarely been considered.[9]

Since meningococcal vaccine strategies tend to focus on children or adolescents, indirect costs associated with lost time due to care-giving — when included in the analysis — are generally restricted to parents, and calculated using average wage rates.[12,13] The handling of reduced productivity due to morbidity or mortality is subject to more variation. With the friction cost approach, there are some costs associated with the replacement of lost labour,[31] but these are generally assumed away in the analysis.[12,25,32] The alternative to friction costs is the human capital approach, and the quantitative difference between the two calculations is considerable.[31] Much controversy remains on the appropriate means of measuring productivity losses, but the larger issue is the possibility of double counting.

Double counting occurs when the value of lost time due to morbidity and/or mortality is considered both in the denominator of the cost-effectiveness ratio (as QALYs or life-years) and in the numerator.[33] In the case of morbidity losses, the problem arises when the utility weight used to calculate QALYs reflects an assessment of the linkage between productivity and income. This case is discussed in detail in section 2.4.

A more straightforward issue is the monetization of productivity losses due to death, and their inclusion in the numerator. Since a denominator of QALYs or life-years already captures the ‘value’ of premature or avoidable death, inclusion of a monetary value of lost life in the numerator effectively means inclusion of that value twice. Thus, regardless of the economic philosophy regarding the use of friction costs or human capital (or some other approach), costs associated with lost years of life should not be included in the numerator. Doing so effectively transforms the cost-effectiveness analysis into a cost-benefit analysis, in which case the interpretation of the cost-effectiveness ratio in relation to the decision-maker’s threshold value is substantially altered. Analysts may wish to report such costs as a service to decision makers, but they should not be included in the cost-effectiveness calculation.

2.3 Indirect Benefits of Vaccination

Vaccination programmes may provide two indirect benefits. First, individuals in the target population who are not vaccinated may receive protection from the disease via the immunization of others in their cohort. Second, the (unvaccinated) contacts of immunized individuals outside the target vaccination cohort (e.g. household relatives of immunized children) benefit from the reduced infectivity.

These phenomena, collectively described by the term ‘herd immunity’, reflect the economic notion of an externality; that is, a cost or benefit that accrues beyond the decision maker. With respect to vaccines, herd immunity is generally regarded as a positive externality (certainly in the case of meningococcal vaccines),[10] but herd immunity could have negative consequences.[11] Getsios et al.[11] noted the example of varicella, where herd immunity could delay infection in unvaccinated individuals to an age where the complications of the disease are more serious.

Evidence from large-scale vaccination programmes in Canada, Spain and the UK suggest that there may be significant (beneficial) herd immunity effects associated with meningococcal vaccination, at least in some populations.[11] In a review of economic evaluations of adolescent vaccines in US settings (including but not limited to meningococcal vaccine), Ortega-Sanchez et al.[27] found that considerations of herd immunity improved cost effectiveness by 17–51%. However, for adolescent meningococcal vaccination strategies specifically, herd immunity may be overestimated since the benefits of herd immunity depend on achieving sufficient vaccination coverage, which has been difficult in adolescent populations.[27]

A minority of economic evaluations of meningococcal vaccine have considered herd immunity. For evaluations being conducted near the time of vaccine introduction, good information on the degree of herd immunity offered by any particular vaccination strategy is likely to be low. When such information is available, sophisticated modelling is generally required to simulate the dynamic process of disease transmission, including carriage and age-specific incidence.[1] The impact of vaccination on carriage rates is a particularly important consideration, since the ability to prevent carriage will influence whether vaccinating individuals with the highest incidence of disease or the highest carriage will be the more cost-effective strategy.[1]

Dynamic models of meningococcal vaccine that consider herd immunity are beginning to appear in the literature, although the manner in which herd immunity is incorporated within the models differs.[4,15,17,24,29] In some cases, herd immunity is considered indirectly, as a function of age-specific rates of vaccine uptake, seroconversion and waning immunogenicity.[15,17] Others have modelled the various aspects of herd immunity directly, by considering age-assortive mixing patterns between vaccinated and unvaccinated individuals.[4,24,29] These models also allow one to consider cost effectiveness across multiple cohorts. The data requirements for both types of models are substantial,[11] but the necessity of the added complexity of the mixing models is difficult to assess.

Both Ortega-Sanchez et al.[17] and Caro et al.[29] used simulation models of meningococcal vaccine herd immunity that are based on estimates from the UK experience, with the latter model reflecting population mixing patterns. In the first model, considerations of herd immunity increased the expected number of cases prevented by 48%.[17] By contrast, the number of prevented cases attributable to herd immunity in the fully dynamic, mixing model exceeded the number of directly prevented cases by an order of three.[29] Yet, it is impossible to determine with certainty whether one or the other model is over- or under-estimating the impact of herd immunity since they evaluated different scenarios and assumed different coverage rates.

Thus, while it seems clear that herd immunity should be included in the evaluation of meningococcal vaccines, the most efficient method for doing so remains elusive. When vaccine uptake is high or is not expected to significantly alter the risk of infection in unvaccinated individuals, a simple static model may suffice.[4,9] Simulation models that consider herd immunity indirectly may facilitate an examination of how large an impact on cost effectiveness herd immunity may have and therefore inform the need for more dynamic modelling.

Dynamic models are not without serious limitations. If the infection process is very poorly understood, the high levels of uncertainty within a dynamic model may outstrip the advantages of the approach.[4] The added complexities of fully dynamic models also complicates the ability to conduct probabilistic sensitivity analysis.[4] Dynamic models are also particularly dependent on assumptions about vaccine coverage[9] and the relationship between coverage and herd immunity.[29] For example, if indirect effects are considered as a linear function of vaccine coverage, the amount of herd immunity associated with a given increase in coverage will be the same no matter if current coverage is high or low. Since herd effects are most likely not linear with coverage, assumptions of linearity may be appropriate only when relatively small increases in coverage are expected. Finally, Trotter and Maiden[1] note the dangers of extrapolating data from one country to another, since observed indirect effects will vary based on the dominant circulating strains, changes in the transmissibility of strains over time, and the nature of the epidemic, among other factors (e.g. differences in social mixing patterns).

2.4 Quality of Life Considerations

The adjustment of health outcomes for QOL is particularly important for meningococcal and other vaccines that aim to prevent life-long complications of disease,[9] and QALYs are nearly universally used as the primary measure of vaccine effectiveness in published studies (excluding cost-benefit analyses). As a consequence of the lack of good utility information in this area and because a few research teams are responsible for the bulk of the published economic evaluations of meningococcal vaccines, most studies utilize one of two sources of utility information. One source is a study of survivors of meningococcal disease in Canada who completed the EQ-5D, which was part of a larger economic evaluation.[2] Those weights were then used by others.[4] The other major source of QALY weights was developed from a panel of Dutch physicians, again using the EQ-5D;[34] three evaluations used the weights from that study.[12,25,32] Several papers used a variety of sources[17,19,21] or did not specifically report the source(s) from which QALYs were generated.[14,35] Many papers only provided citations for utility information without reporting the specific values being used, diminishing both transparency and reproducibility.

Existing approaches to the incorporation of utility information in economic evaluations of meningococcal disease suffer from several limitations. In addition to appreciating how the population providing the utility information (e.g. patients vs community members) may affect health-state utility assessments, future evaluations should also consider that several sources of utility information cited by authors date from the mid-1990s. To the extent that treatment options have changed over the past 15 years, the older estimates may not reflect the preferences of individuals across the same health states today. Most studies also excluded QOL considerations associated with acute forms of disease or minor vaccine-related adverse events, since such health states are short lived and techniques for measuring the utility (or disutility) of acute events remain undeveloped, especially in paediatric populations. As Welte et al.[28] wryly noted, the measurement of QOL in children “… is still in its infancy” and deserves further investigation.[36]

Perhaps a more important issue related to QOL in economic evaluation of meningococcal vaccine involves the double-counting issue alluded to in section 2.2. Depending on the manner in which utilities are elicited, some analyses that separately calculate the costs associated with morbidity-related productivity losses and include those costs in the numerator of the incremental cost-effectiveness ratio (ICER) may be double counting.[33] When investigators use multi-attribute instruments such as the EQ-5D to define and value health states, the likelihood that productivity effects are captured in the denominator of the ICER via the QALY weights is small. Thus, the greatest concern comes when utilities are directly elicited from individuals whose valuations may include productivity effects, although the extent to which individuals consider productivity losses as part of the valuation process — even when specifically instructed to do so — is not known.[33,37]

Potential double counting was identified in four published studies in which indirect (productivity) costs were calculated via the human capital approach, health benefits were measured as QALYs, and the method by which health-state utilities or quality adjustments were made was not immediately evident[14,35] and/or may have introduced productivity considerations.[17,21] It should be noted that these studies were identified as having only the potential for double counting. The latter two studies by Ortega-Sanchez et al.[17] and Shepard et al.[21] utilized a variety of sources that estimated QOL effects using generic and disease-specific QOL instruments and the Health Utilities Index (HUI). Assuming that patients were not specifically instructed to consider the value of lost time and productivity when completing those instruments, the effect of any double counting is likely to be small.[37] The remaining economic evaluations that utilized QALYs or disability-adjusted life-years either excluded indirect costs,[4,19,22] used the friction cost method (which added zero indirect costs)[12,13,25] or used a utility measure such as the EQ-5D, for which productivity considerations are unlikely be relevant.[2,15] As with productivity losses associated with mortality, care should be taken with the inclusion of morbidity-related losses to ensure that such effects are not being considered in both the numerator and denominator of the cost-effectiveness ratio.

QALYs in general are associated with other limitations, a complete discussion of which is beyond the scope of this article. Some limitations are particularly relevant to the consideration of vaccines and so are worth noting here.

First, QALYs may not encompass the full benefit of vaccination to individuals.[9] These benefits might include peace-of-mind or ‘utility in anticipation’ effects associated with reduced risk of disease or, conversely, anxiety about vaccine-related adverse effects.[9,10] Parents of vaccinated children may also value a reduction in the likelihood of disruptions to work or family routines associated with a child’s illness (beyond the monetary value of those disruptions).[9,10]

Second, in terms of informing public policy, cost-effectiveness analysis cannot readily incorporate ethical concerns that may play an important role in policy decisions. For example, decision makers may view interventions that benefit children as especially desirable. Such preferences are difficult to incorporate into cost-effectiveness analyses.

2.5 Choice of Comparison Programmes

On both the cost and the benefit sides, great care must be taken to reflect the incremental costs and benefits of a new vaccine programme administered in concert with any existing programmes.[9] For example, if a new vaccine was to be incorporated into the recommended vaccine schedule, the administrative (variable) costs would be relatively low. Conversely, a vaccine that is to be given outside the existing schedule (i.e. not at a regularly scheduled vaccine visit) will incur higher costs. In some cases, the incremental benefit of a new vaccine may also depend on the availability of existing vaccines. This is particularly true for meningococcal vaccines, since the incidence of meningitis has been greatly reduced through the use of the Hib vaccine in some areas.

It must be remembered that cost-effectiveness analysis is a comparative exercise and thus the cost effectiveness of any vaccine programme will be greatly influenced by the choice of the alternative programme(s) being evaluated. Additionally, investigators are cautioned to consider the possibility that any existing programme(s) may be inefficient. If the programme with which a new strategy is being compared is not cost effective, the ICER of the new strategy will be biased downward; that is, the new strategy will appear more cost effective than it really is.[38]

For new vaccines, a ‘no vaccination programme’ represents the natural comparison to be made. However, as vaccine strategies evolve, the tendency will be to compare new strategies with existing ones (assuming an existing programme is in place). Of the 16 published economic evaluations reviewed for this paper, four did not consider a ‘no vaccination programme’;[4,15,18,24] however, in only one of these were the results reported in sufficient detail as to estimate the inherent cost effectiveness of the existing (comparison) programme.

De Wals et al.[15] compared two strategies of a booster dose at 12 years with the current standard practice in Canada of vaccination at 1 year of age. Their analysis considered both direct medical and indirect costs (valued as human capital) and included a herd immunity effect. The authors found that a two-dose schedule of meningococcal serogroup C conjugate vaccine (MCC) was cost saving compared with the existing one-dose MCC schedule. However, the two-dose schedule looked particularly efficient because the strategy with which it was being compared was relatively inefficient. Although difficult to calculate from the information provided, the ICER of the one-dose schedule, compared with no vaccination, may exceed $Can150 000 per QALY (year 2004 values). If that is in fact the case, then the one-dose vaccination programme would be weakly dominated by the two-dose schedule. Excluding the one-dose schedule from consideration, the actual cost effectiveness of the two-dose schedule is closer to $Can13 000 per QALY. The two-dose schedule would most likely still be considered cost effective by Canadian standards,[39] but it is probably not cost saving (at least, given the parameters of the study).

In some analyses, failure to recognize the inefficiency of a comparator strategy may lead to inaccurate and overly optimistic estimates of cost effectiveness. For that reason, it is recommended that analysts include the costs and benefits of a ‘no vaccination’ option, and that cost effectiveness of that option be explicitly calculated. In the case above, the adjusted cost effectiveness was difficult to estimate because the effectiveness results were reported as cases prevented and QALYs lost rather than the total cases and QALYs associated with each strategy.[15]

2.6 Sensitivity Analysis

Many of the parameters utilized in the evaluation of healthcare technologies are subject to uncertainty, and this is especially so with vaccine technologies. As modelling capabilities expand to include detailed dynamic models of vaccination and disease transmission, the effect of both parameter and model uncertainty is magnified. Thus, it is imperative that investigators conduct extensive sensitivity analysis across all parameters. At a minimum, authors should conduct one-way analyses of key parameters, including disease incidence and prevalence, discount rates and vaccine price, and present any critical parameter (threshold) values at which the intervention is clearly beneficial or not.

For static simulation models, probabilistic sensitivity analysis is useful for generating confidence intervals around the ICER.[40] Again, without being too proscriptive, investigators should carefully consider the appropriate density function for each parameter included in the analysis so as to avoid introducing additional (irrelevant) sources of uncertainty. When deriving confidence intervals from probabilistic sensitivity analysis, investigators should be conscious of the possibility of negative ICERs. The presence of a negative ratio could indicate cost savings (greater effectiveness and lower costs than the comparator) or inefficiency (lower effectiveness and greater costs than the comparator). When multiple vaccine strategies are being compared, the potential to generate both types of negative ratios is especially high. As an alternative, investigators could present results in the form of a cost-effectiveness acceptability curve, or convert the ICERs to net monetary benefits (or both).[41,42]

When the complexity of a model precludes probabilistic sensitivity analysis, investigators should consider conducting best- and worst-case scenarios. While a comparison of the best and worst cases will overestimate the overall (parameter) uncertainty,[40] the use of scenario analysis may be informative to decision makers. Certainly, scenarios that reflect important analytical assumptions, such as the frequency and magnitude of disease outbreaks, should be included.

2.7 Reporting of Methods and Results

As noted above, much of the manner in which the results of economic evaluations of meningococcal vaccines are reported makes it difficult, if not impossible, to determine the precise impact of specific analysis features on cost effectiveness. The adoption of minimum reporting requirements is an important first step in facilitating comparisons across studies, and Szucs[43] noted the usefulness of guidelines defined by the British Medical Journal in that regard. Specific features of those guidelines that relate to the methodological challenges described here include the following: (i) a description of the methods used to value health states and other benefits, such as indirect costs; (ii) separate reporting of productivity changes, if included; (iii) comparison of relevant alternatives (adequately described); (iv) use of incremental analysis; and (v) presentation of major outcomes in both disaggregated and aggregated form.[43]

The separate reporting of direct and indirect costs (item ii) is particularly important when those costs have been assessed by the human capital method, since doing so may bias cost-effectiveness results in favour of vaccination strategies that target children as opposed to adolescents.[27] Similarly, direct and indirect benefits of vaccination also should be reported separately. Both cost and benefit outcomes should be reported as the totals estimated for each alternative considered, not solely as increases or decreases compared with a reference case. Reporting of analyses as encouraged here will allow for comparisons of studies not only within meningococcal vaccines, but also across interventions.

3. A Recommended Minimum Analysis Profile

The economic evaluation of vaccines is an inherently complicated endeavour, and many barriers to the creation (or at least the use) of standardized guidelines exist. As Ess and Szucs[44] noted, “The precise basis for a decision based on economic evaluations will depend … on the context of the evaluation.” For instance, the structure of existing vaccination strategies will drive the choice of comparison, as discussed above. Another difficulty in putting forth a set of guidelines by which meningococcal vaccine programmes should be evaluated is that many of the factors identified earlier are inter-related. For example, the manner in which indirect costs have been measured is irrelevant when the perspective taken is that of the healthcare system. Similarly, discount rates may vary by the perspective of the study (i.e. the perspective of the decision maker). Some relationships are more subtle. Suppose a consensus statement called for vaccine effectiveness to be measured in QALYs and, furthermore, that indirect costs be included in the analysis. Such a statement would also need to specify the manner in which utilities were to be elicited so as to avoid double counting.[33]

Ess and Szucs[44] also noted that the nature of the policy decision being made (e.g. as an allocation within a fixed budget versus a more general coverage decision) may dictate the overall analytical approach to the evaluation. The use of cost-effectiveness or cost-utility analysis is particularly suited to the prioritization of programmes within fixed budgets, but cost-benefit analysis may be preferred when policy decisions are more broadly considered. Thus, relegating all evaluations to a single set of ideal or standard methods is probably unrealistic and perhaps inefficient.

As an alternative to a set of analytical guidelines, a minimum (rather conservative) analysis profile is advocated here. This minimum profile is meant to form a subset of considerations that are common to the vast majority of analyses: analysis of direct medical costs separately from any indirect cost considerations; inclusion of indirect benefits and methods for doing so; use of utility weights and methods for their elicitation; explicit consideration of a ‘no vaccination programme’ option; conduct of threshold sensitivity analyses; and the disaggregated reporting of results. These factors are described in table II.

Table II
figure Tab2

Proposed minimum analysis profile (MAP) for cost-effectiveness analyses of meningococcal vaccines

Full size image

While the reporting of cost-effectiveness analyses has improved over the years, many gaps remain, particularly with respect to parameters that are derived from outside sources. It is not sufficient to simply cite a reference source; authors should address the applicability of source information to their own analytical objectives. If the results of an analysis are reported in disaggregated form, as recommended here and elsewhere, but fundamental details of the model inputs are not available, the usefulness of the entire analysis to both decision makers and other analysts is greatly diminished.

4. Discussion

A complete understanding of the efficiency of meningococcal vaccines is hampered by the diversity of approaches found in the economic literature. Chief among these differences are considerations of herd immunity, inclusion of and approaches to valuation of indirect costs, and inconsistencies in the reporting of results. More critically, some analyses may have incorrectly considered productivity losses associated with death (double counting) or failed to consider the underlying efficiency of comparison programmes.[4,15,24]

Some efforts have been made to improve comparability across vaccine evaluations, while recognizing the infeasibility of dictating specific analytical requirements.[9,43] Among the recommendations are the adoption of minimum reporting requirements for published economic evaluations.[43] Clearly the first step to improving comparability is ensuring transparency. However, as models become more complicated,[9,11] providing transparency will be increasingly challenging for authors (and publishers).

It should be noted that, while the intent of this review was not to focus solely on cost-effectiveness analyses, no examples of cost-benefit analysis were identified. The use of cost-benefit analysis could address some of the concerns noted above, particularly with respect to the problem of double counting when both indirect costs and QALYs are included in the analysis. Willingness to pay (WTP) methodologies, at least theoretically, can capture all the benefits that individuals attach to vaccination, as well as equity considerations. As Beutels et al.[9] noted, the use of discrete-choice experiment (DCE) methods may be particularly useful in this regard. Compared with contingent valuation methods, DCE forces individuals to make trade-offs across attributes, reducing both the tendency to accept the programme that is offered and protest responses.

Furthermore, since the attributes and attribute levels can be hypothetical, investigators can explore a broad range of vaccination strategies. The use of a DCE by Bishai et al.[45] to estimate French and German citizens’ WTP across a variety of meningococcal vaccine programmes (the results of which paralleled the experience of the US market) is a promising first step towards the assessment of WTP in this area. While cost-benefit analysis and WTP methodologies are not without their own limitations, further research in this area would be welcomed.

5. Conclusion

Inconsistencies in economic evaluations have hampered a complete understanding of the efficiency of meningococcal vaccines, including variability in the consideration of herd immunity and indirect costs, reporting of results, potential for double counting, and adequacy of comparison programmes. The purpose of this article has been less to advocate a specific approach to the economic evaluation of meningococcal vaccines and more to highlight potential problems that should be avoided in future analyses. That said, a minimum analysis profile that includes only costs and benefits relevant to the healthcare system perspective (a subset of the societal perspective) and for which any considerations of indirect costs and benefits are reported separately would provide a conservative assessment of vaccine effectiveness and aid the comparison of meningococcal and other vaccine programmes.