Background

Various countries are currently in the process of evaluating whether or not to include rotavirus vaccination in their national immunization programs. Two rotavirus vaccines are currently available: Rotarix® and Rotateq™, marketed by GlaxoSmithKline (GSK) and Sanofi Pasteur MSD (SPMSD), respectively. Rotavirus vaccines have proven to be efficacious in preventing rotavirus-related disease, gastrointestinal disease and health-care use (general practitioner (GP) visits and hospitalizations) in infants and toddlers [1–3].

Health-economic properties of these new vaccines present one characteristic to be analyzed with respect to their inclusion in National Immunization Programs (NIPs). Numerous health-economic analyses already exist regarding rotavirus vaccination, mostly in high-income country settings, some being single-country analyses but a few multi-country analyses [4–6]. Also, one critical review of these existing models has already been performed [7]. Recently, some studies have become available targeted at situations in low- and middle-income countries, involving all regions of the world (for example, Vietnam, China, Kenya and Colombia [8–11]). A review of these studies in Asia, Africa and South America is in preparation [12]. Based on the favorable cost-effectiveness profile of many of the models applied in low- and middle-income countries, the World Health Organization (WHO) advises inclusion of rotavirus vaccines in the NIPs [13]. However, in general, it is still not always clear which aspects these individual models differ on, what impact such differences have on the cost-effectiveness outcomes and which model might be preferred over the others in continental, regional or country specific situations [14].

This study aims to critically review some of these available cost-effectiveness models for rotavirus vaccination, compare their designs using a standardized approach and compare similarities and differences in cost effectiveness outcomes using a uniform set of input parameters. WHO has initiated this study to enable providing guidance to low- and middle-income countries if requested, on the strengths and weaknesses of existing health-economic models for rotavirus vaccination as a basis for decisions about whether or not to build their own models or to adapt existing models to local situations. In particular, our study intends to help WHO to guide individual countries on rotavirus modelling, specifically national decision makers who may have the interest, research capacity and resources to conduct their own cost-effectiveness analyses in generating evidence for decision making on whether or not to introduce rotavirus vaccination. Therefore, our analysis might be most suited for lower middle income countries that have some capacity to attract global partners for model adaptation, whereas most low income countries may not have sufficient technical capacity for building or adapting existing models. Notably, our goal is neither to advocate for the use of specific models nor to recommend individual modelling groups over others.

Methods

Review of models

Initially, we searched for existing health-economic models in the literature using PubMed, Embase and Web of Science. Although the search was not limited to the English language, the relevant papers that emerged from the initial search and crude selection were all in English. A further selection was made for those models to be included in the detailed comparison based on various criteria related to our goals of achieving diversity in terms of provenance (public versus private), methods (single-cohort models, multi-cohort models and so on) and specific vaccines to be incorporated by the model (that is, two individual vaccines exist that are slightly different). Modelling groups were selected and contacted, explicitly based on these criteria. However, availability, ease of access, complexity and time investment of the individual research groups appeared to be the strongest criteria in practice for inclusion in the final comparison. In particular for those models with a high degree of complexity, the time investment required appeared to prevent the groups from becoming involved in our comparison. Also, for these complex models resource requirements for this analysis presented a strong limitation, which could not easily be overcome given the limited funds available by WHO for the endeavor.

Comparative framework

As described, various cost-effectiveness models were identified by the process described above. Their developers were contacted by a WHO officer (RH) to invite them to participate in the model comparison. Subsequently, the model inclusion was co-ordinated by two authors (RH and MP), while one author (MP) included his own group's model into the comparison. The process resulted in three models provided to us, including analyses using the standardized dataset specified below (Table 1). From all modelling groups at least one co-author is included in the author list of this paper. These models represented a balanced public-private mix involving one designed by the pharmaceutical industry (Roxanne Rotarix™ Analyses of Economics from GSK), one developed by public financing within a European-Union project (POLYMOD) and one privately financed (Sanofi Pasteur MSD) but developed by the University of Groningen within the context of an unrestricted grant (CoRoVa Consensus Rotavirus model Vaccination). Models included could be applied to modelling use of either vaccine (Rotarix or RotaTeq), hence ensuring that the unique features of the two vaccines were adequately represented in our analysis. In addition, both manufacturers were directly (Roxanne) or indirectly (CoRoVa) involved in the models. Only static models were compared; this posed both a disadvantage and an advantage. In particular, comparison with a dynamic model, which could explicitly analyze the effect of vaccination on the spread of rotavirus infection using mathematical modelling, would be extremely valuable. However, our selection of only static models for comparison enhances comparability of the individual models and facilitates understanding the differences that still remain between the models' outcomes.

Table 1 Basic characteristics of the models investigateda
Full size table

The owners of the models provided access to their models through physical transfer of the software - accompanied with user guides and/or publications - and explained various concepts and characteristics of these models face-to-face, through e-mail contacts and telephone calls during 2009. Additionally, during a one-day consultation in 2009 at the WHO headquarters in Geneva, concepts and draft results were discussed with a large group of experts in epidemiology, immunology, vaccinology, political sciences and health economics. In the end, final calculations were performed by the modellers themselves using the most current model versions at the end of 2010 and extensively discussed during the winter of 2010/2011. To summarize the way the three models were included in this analysis, we note that all the software for the models was physically available to the coordinating team (MP and RH), at least two face-to-face meetings between member(s) of the coordinating team and each research group were organized and final calculations were checked by the coordinating team and cross-checked for face validity by all three individual research groups.

The standardized approach in comparing these models involved stepwise analysis of the structure, the input parameters required and specific assumptions underlying the models.

Details on available models

As mentioned, during the model selection process three models with corresponding results for the standardized input parameters became available to us. These were the POLYMOD model [4, 15, 16], Roxanne [17–20], and CoRoVa [21–23].

The POLYMOD-model was developed in the context of a European Union (EU)-funded project with the same acronym [4]. RVGE Rotavirus GastroEnteritis was modelled using an age-structured cohort model that followed cohorts of vaccinated and unvaccinated individuals (Table 1). For the first year of life, the cohort was stratified into monthly age groups, with one-year age bands applied beyond (one to five years old). The model was initially a single-cohort model; however, it was adapted to a multi-cohort model for the purpose of this exercise to allow a step-up in vaccination coverage. RVGE was stratified into mild disease with home treatment only, moderate disease with primary care visits (GP and/or hospital outpatient), severe disease with hospital emergency room (ER)-visits and/or admissions, nosocomial infections and death. QALY losses were incurred both by the index infants and their care givers. QALYs were taken from one specific study on the topic [23], which were quite comparable to those used in some other health-economic studies [5]. The model was designed by modellers based in England's Health Protection Agency (HPA) with input from modellers based in public health institutions in Belgium, Finland, France and the Netherlands and applied to model rotavirus epidemiology and cost-effectiveness of vaccination in the five countries of Belgium, England and Wales, Finland, France and the Netherlands. Unlike most static rotavirus models, waning vaccine immunity was explicitly incorporated into the model structure. Further country-specific models - based on this multi-cohort, multi-country model - have been published, for example, for the Netherlands [16].

Roxanne was developed as a Markov cohort process tree [17]. It is programmed in Microsoft Excel 2007 using Visual Basic and contains both cost-effectiveness and budget impact modules. The model was initially parameterized with data from France [18], but allows data from any country to be used [19]. A precursor of the model has been used to estimate the cost-effectiveness of Rotarix® in the Netherlands [20]. Besides a comparison of vaccination versus no vaccination, the model was designed to additionally allow explicit comparison of two- and three-dose vaccination strategies. Obviously, outcomes in such analyses crucially depend on the exact characteristics and properties applied to the two- and three-dose vaccination schedules. Finally, Roxanne allows extensive probabilistic sensitivity analysis, using the At Risk add-in for Excel. A special feature of Roxanne involves the explicit inclusion of the modelling of maternal protection from (severe) infection through breastfeeding.

CoRoVa was initially developed for the specific Dutch situation and aimed at achieving consensus among various Dutch modelling groups that had previously worked on the cost-effectiveness of rotavirus vaccination [16, 20]. An age-structured cohort model was developed in Excel applying a time horizon of five years with time cycles of one month for children less than one year of age and annual thereafter [21–23]. Outcomes in the model were classified by severity and included home-treated community-acquired diarrhea and rotavirus infection leading to GP consultations and/or hospital admissions (including emergency department visits), nosocomial infections and death. Specific characteristics of the model are the ability to take waning immunity, maternal protection against infection through breastfeeding, and herd protection into account. However, the model is not a transmission dynamic model because herd protection is incorporated by straightforward calculus only, using a static approximation based on a fixed fraction of the direct effects. In the base-case analysis for the Netherlands, QALY losses of caregivers were not included and the QALY decrement for children was based on a combination of two published studies performed in Canada and the United Kingdom [24, 25]. Similar to the Roxanne-model CoRoVa also used the At Risk software for (probabilistic) sensitivity analyses.

Standardized input parameters

For this study, analyses of the models were provided regarding their structure and outcomes for four hypothetical countries, representative of different continents and income levels (low, middle or high), respectively classified into the WHO geographical regions and mortality strata [14]. In particular, sets of standardized input parameters were provided to the modellers and analyzed for a hypothetical Afr E country representing the African region with high child and very high infant mortality; a hypothetical Sear D country representing the South-east Asian region with high child and adult mortality; a hypothetical Amr D country representing the South and Middle American regions with again high child and adult mortality; and a hypothetical Eur A country representing developed countries in the European setting. Tables 2, 3, 4 and 5 summarize the set of standardized input parameters. Rectangular age distributions were assumed, implying that life expectancy decreases with one year exactly for every one-year increase in the age of infants and toddlers considered. (For example, if life expectancy is 70 at birth, it would be 69 at the age of 1, 68 at the age of 2 and so on). Simplifying assumptions were justified as our interest concerned the comparison between models rather than the exact representation of country-specific demographic, epidemiological and economic impacts.

Table 2 Standardized dataset for the cost-effectiveness models in rotavirus vaccination: demography and incidence (Sources: [4, 5, 7, 14, 15, 20, 26, 27])
Full size table
Table 3 Standardized dataset for the cost-effectiveness models in rotavirus vaccination: vaccine characteristics (Sources: [1–5, 7, 15, 20, 28], expert opinions)
Full size table
Table 4 Standardized dataset for the cost-effectiveness models in rotavirus vaccination: health-care use and costs
Full size table
Table 5 Utility losses and some remaining issues [4, 5, 7, 20, 27]
Full size table

Although the time horizon in the single- and multi-cohort models was lifetime, this effectively produced a time horizon of five years after the birth of the last cohort, since it is assumed that no rotavirus gastroenteritis occurs beyond the age of five years and vaccination is only investigated in infants in their first year of life. Various sources were utilized for parameter estimates as indicated in Tables 2, 3, 4 and 5[1–5, 7, 14, 15, 20, 26–31]. However, as the objective was comparative rather than to exactly mimic the situations of specific countries often plausible assumptions were made rather than exact replications of individual sources. Plausibility of assumptions was primarily based on the expert opinions of two of the authors (MP and MJ).

Key parameters for all analyses were the incidence of rotavirus gastroenteritis (RVGE), the corresponding risks of rotavirus-related health-care use, as well as corresponding costs. All cases with RVGE were assumed to be treated at home. In particular, moderate cases were assumed to involve one additional outpatient visit (GP, health center or hospital) and severe cases were assumed to involve: home treatment, an outpatient visit and a hospitalization. Some models had the additional option of severe cases not being hospitalized. Based on data from the clinical trials, vaccine efficacy could be specified for different outcomes and for high-income versus other countries.

Alternatively to what is expressed in Table 1, incidence could be expressed as average annual risks over the same five years considered in the data and modelling. In particular, the average annual risks at population level are 10,440, 4,040 and 320 per 100,000 people in all developing regions for mild, moderate and severe, respectively. Correspondingly, figures for the Eur region are 10,440; 3,460; and 419 per 100,000 people. Similarly, annual mortality risks are 60, 40, 20 and 0.21 per 100,000 population for the Afr, Sear, Amr and Eur regions, respectively.

Standardized output

Model developers were requested to present a standardized set of output variables for one single cohort (for the multi-cohort model, the coordinating center estimated the results for one cohort themselves). In particular, these were: total number of persons and (person-years if available) followed in the model; undiscounted number of mild cases of RVGE; undiscounted number of moderate cases of RVGE; undiscounted number of severe cases of RVGE; undiscounted number of outpatient visits (GP, outpatient or health center; typically this would equal the number of moderate and severe cases); undiscounted number of hospitalizations (typically equal to the number of severe cases); undiscounted number of nosocomial cases (equal to 1/4 or 1/3 of the number of hospitalizations, in the Eur and other regions, respectively); undiscounted number of deaths; discounted direct outpatient costs; discounted direct inpatient costs; discounted vaccination costs; discounted and undiscounted QALYs due to deaths (difference between vaccination and no vaccination is equal to the number of life-years gained); discounted and undiscounted QALYs due to morbidity; cost/QALY from the health-care perspective; cost/QALY from the societal perspective; and finally, a sensitivity analysis was requested by varying parameters values through halving and doubling their base case values, except the discount rate which was investigated for alternative values of 0% and 4%.

In particular, cost-effectiveness was expressed in net costs per QALY gained by subtracting discounted savings from the reduced need for RVGE treatments from (discounted in the multi-cohort model) vaccination costs to provide the numerator and dividing by the QALYs gained (the denominator).

Results

Results of using the standardized dataset for the various regions in different models are shown in Tables 6, 7, 8, 9 and 10 for all three models. In general, cost-effectiveness results are broadly similar and comparable.

Table 6 Comparative analysis on costs per QALY
Full size table
Table 7 Outcomes predicted by models for one birth cohort in Afr-region
Full size table
Table 8 Outcomes predicted by models for one birth cohort in Sear-region
Full size table
Table 9 Outcomes predicted by models for one birth cohort in Amr- region
Full size table
Table 10 Outcomes predicted by models for one birth cohort in Eur-region
Full size table

However, there are differences in the building blocks of these cost-effectiveness ratios, for example, regarding estimations of QALY losses related to morbidity in the three models. The results for the CoRoVa model were generally in between those for the other two. The Roxanne model generally predicts an overall higher number of cases, most notably for hospitalized cases. This translates into relatively high inpatient savings that are however lowered by discounting and overall dominated by vaccination costs to render similar net discounted costs for all three models investigated. However, the POLYMOD model gave consistently lower results for these discounted net costs; this might be due to the approximation from results for the multiple cohorts to just one. Also, higher vaccination costs in later years in the multi-cohort model were offset by a higher number of cases averted in the single-cohort model, resulting in the comparable cost-effectiveness ratios between the two types of models. Results suggest that rotavirus vaccination could potentially be cost-effective in all regions, particularly in low and middle income countries. However, the standardized data set is highly generalized and not specific to any individual country, so conclusions to support policy making should not be drawn in the absence of country-specific analyses.

Figure 1 shows the sensitivity analysis performed on the model results, with the example of CoRoVa shown here (other models showed similar patterns). From previous work on cost-effectiveness of rotavirus vaccination [5, 7, 15, 19], it is well-known that generally these results are highly sensitive to the vaccine price, rotavirus-associated mortality and the discount rate. Our analysis shows a similar pattern for all regions investigated. However, there is an interesting shift observed in variables influencing the cost-effectiveness results between EU and non-EU countries. In the EU region the incidence of the disease is a driver followed by utility weights whereas in the non-EU region mortality is essentially driving the result.

Figure 1
figure 1

Sensitivity analyses on the base case cost effectiveness ratio ($ per QALY, except in D where € 's *1000 were used) using the CoRoVa model for Afr (A), Sear (B), Amr (D), and Eur (D) regions. Parameters were varied through halving and doubling, except for the discount rates which were 0% and 4% for both costs and health effects. Dark bars show the incremental cost effectiveness ratio after a 100% decrease in the parameter, whereas light bars show the incremental cost effectiveness ratio after a 100% increase (note that an increase in the incidence or the costs resulted in negative cost effectiveness ratios in the Eur region [D]). Note that when the incidence of RVGE was increased or decreased, the total number of deaths was kept constant to identify the sole effect of incidence. QALY, quality adjusted life year; RVGE, rotavirus gastroenteritis.

Full size image

Discussion

We identified various models used to estimate the cost-effectiveness of rotavirus vaccination. From these, results using the standardized dataset could be obtained for three specific applications (the POLYMOD, Roxanne and CoRoVa models). Despite differences in the approaches, cost-effectiveness results of the models were quite similar. Differences between the outcomes of the specific building blocks of the cost-effectiveness (that is, vaccination costs, savings and QALYs gained) of the models currently investigated seem to relate to five aspects of the models: the multi-cohort nature of the POLYMOD model which assumed a step up in vaccination coverage (and hence in vaccination costs as well as cases prevented); the exact timing of the waning in the models and in particular the exact modulation of between-dose efficacies in the CoRoVa model; assumptions about the distribution of cases of different severity levels within the one-year age groups provided (for example, assuming a Weibull distribution in the Roxanne model); and the possibility in the models of experiencing subsequent episodes of rotavirus infections and/or experiencing episodes with multiple manifestations (for example, first moderate progressing to severe) and types of health-care use (for example, inpatient and outpatient, rather than just one of both).

Ergo, differences between the models on the individual components of cost-effectiveness could be related to some specific generic features of the models with regards to representing vaccine uptake and pricing, within age-group distributions, waning and between-dose efficacies and inclusion of additional groups and episodes in the general design of these models. Sensitivity analysis revealed that cost-effectiveness of rotavirus vaccination is highly sensitive to vaccine prices, rotavirus-associated mortality and discount rates, in particular that for effects. This is fully in line with other authors' findings [32].

Unfortunately, we were not able to include a model with a transmission dynamic approach in our model comparison, instead of the cohort approach followed in the three models analyzed. Emerging evidence that herd immunity effects might be relevant for rotavirus transmission and vaccination enhances the relevance of considering populations and transmissions between cohorts [33–35]. Inability to include these models was often related to the complexity of these models and the difficulties to adequately grasp these complexities in the standardized framework provided on inputs and outputs. However, for further work it is important to also analyze such dynamic models given their major advantage of incorporating infection dynamics including herd immunity effects and potential age shifts in epidemiology [36]. Also, differences in uptake between high-, middle-, and low-income countries should be analyzed using dynamic models given the different impacts of coverage levels on herd immunity.

The three models selected for our analysis were all basically developed for high-income countries. For reasons stated previously, other models - inclusive of models developed initially for low- and middle-income settings - could unfortunately not be considered in our comparison. As one consequence, all the publications arising from these models involved costs per QALY gained rather than costs per DALY Disability Adjusted Life Year averted. Although not undisputed, for low- and middle-income settings DALYs rather than QALYs are the common metric used [27, 37]. We do note that one specific study showed that results differ only slightly if DALYs are used instead of QALYs; in particular, it showed slightly more favorable cost-effectiveness for DALYs as the outcome [16]. However, it has been demonstrated that the decision about whether or not to include caregiver QALYs has a major impact on results [4, 7]. The appropriateness of including QALYs beyond the index case of disease is being debated; it could be argued that caregivers' QALYs are particularly important for rotavirus as they can be measured directly, and hence may be more valid than QALYs in small children with RVGE, where proxy measurements have to be used [7, 21, 22].

Our findings for the regions Afr, Sear, Amr and Eur should not be considered as exact representative results and policy making should not directly be based on this. For example, it is very unlikely that the similarities assumed for the proportion of RVGE cases that are mild, moderate and severe are valid in real world. This simplifying assumption was made in order to test the models generically and consistently. Nevertheless, our results clearly indicate a general trend of increasingly more favorable cost-effectiveness when going from high- to middle- and on to low-income countries, respectively. As the sensitivity analysis shows, this is obviously primarily related to vaccine pricing and the QALY-impact of averted mortality due to rotavirus infection. However for actual policy making, countries will need to either further consider the results from existing studies and reviews, or initiate country-specific cost-effectiveness analyses. For countries that have the capacity and resources to model the cost-effectiveness of rotavirus vaccine, our comparative analysis can help inform the design of new models or selection of existing models to support national-level decision making.

Hence, although our analysis is not meant to directly inform policy making, it offers considerable guidance for design and/or selection of a model for adaptation to individual (low-income) countries that want to conduct cost-effectiveness analyses. Scarce resources in these countries may direct the choice towards adapting an existing model rather than initiating the development of a new approach. Reassuringly, our analysis suggests that different models produce similar cost-effectiveness estimates, illustrating that the exact choice of which model to adapt may not be as crucial as the choice of assumptions and parameter values to incorporate in the model.

Conclusions

We conclude that our approach is helpful on two specific levels. Firstly, the comparative approach followed here is helpful in understanding the various models selected and will thus benefit (low-income) countries in designing their own cost-effectiveness analyses using new or adapted existing models. Secondly, we find that the communication between the individual researchers involved in the three models was helpful in the further development of these individual models and will be so in the future. Therefore, we recommend that this kind of comparative study be extended to other areas of vaccination and even other infectious disease interventions, beyond the three areas that have been explored by WHO (pneumococcal vaccination, human papilloma virus vaccines and (here) rotavirus) [38–40].

Finally, the models reviewed in the exercise gave similar and comparable results which appear to have face validity. Hence it appears possible to recommend their use in policy settings, at least for high income countries. However, potential users of the models need to consider the specific building blocks of the cost-effectiveness models including the nature, scope, design and assumptions made and how they affect outcomes. Potential users of the models in low and middle income countries need to consider results from existing studies and reviews. There will be a need for contextualization including the use of country specific data inputs. However, given that the underlying biological and epidemiological mechanisms do not change between countries, users are likely to be able to adapt existing model designs rather than developing completely new approaches. Also, transmission dynamic effects are likely to be important, particularly when considering the effect of vaccination (since vaccination can affect other cohorts besides those vaccinated). Hence, we would recommend that future cost-effectiveness tool comparison exercises include dynamic models.