To model or not to model: lessons from two vaccinations
- 507 Downloads
Statistical models have a wide range of application, such as to study the spread of infectious diseases, evaluate strategies for preventing and managing chronic diseases, and develop guidelines and post-marketing studies . Modelling is also increasingly used to estimate the cost-effectiveness of health care interventions. Models offer a way to manage the uncertainty deriving from the lack of clinical data on the effects of new health care practices and are mainly used to estimate cost-effectiveness over a long-time horizon when only short-term experimental data are available .
Here, we report our research on the widespread use of modelling for two very different vaccinations—human papillomavirus (HPV) and influenza—in order to draw some lessons on behalf of health authorities.
HPV vaccination is a typical field where economic evaluations (EEs) have been built on long-term modelling . HPV infection is the strongest causal agent of cervical cancer, the second most common female cancer. Cervical cancer arises over many years in around 5% of women with persistent HPV infection . Recently, EMA granted marketing authorisation for two vaccines, the quadrivalent ®Gardasil and the bivalent ®Cervarix. Clinical trials have demonstrated the efficacy of both these vaccines in eradicating the two most dangerous genotypes (HPV 16 and 18) correlated with the cancer. However, these trials are still too short to permit any firm inference on their real eventual efficacy in cancer prevention.
Although many European Union (EU) countries have promptly adopted HPV vaccination, several EEs based on modelling have been meantime conducted to estimate its long-term consequences in Europe. We recently reviewed the EEs for HPV vaccination conducted in the EU , investigating the methodology and the potential influence of sponsorship on performance and results of economic studies. All the studies reviewed used models and almost all concluded that vaccination was a cost-effective strategy.
The main limits of the EEs reviewed concerned the reliability of the epidemiological and economic parameters included in the models. Most authors reported very optimistic estimates for screening and vaccination coverage; costs were built on the basis of assumptions that generated different results even within the same country; most studies adopted North American utility scores, although they were conducted in European settings; finally, results were very sensitive to uncertain variables like discount rates, duration of protection and vaccine price.
The studies funded by public agencies and those sponsored by manufacturers did not differ significantly in their conclusions, although the former were more cautious in how they commented favourable results. Sponsored studies gave the impression of aiming at supporting the marketing strategies of the two manufacturers, being focused solely on their specific vaccine and trying to highlight an advantage for their product.
The Dutch experience is worth mentioning, where out of three studies sponsored by the bivalent vaccine manufacturer, one did not conclude in favour of the manufacturer, a very rare event in the literature  leading us to speculate that the large number of studies sponsored by the same manufacturer in Holland might reflect some flaws in its marketing strategies.
The second application of economic models we studied was for seasonal influenza vaccine. A relevant number of studies were focussed on the economic sense of extending public coverage of this vaccine to healthy adult workers aged 50–64 years . Immunization of this target population should primarily interrupt the transmission of the disease and reduce its morbidity and productivity losses , more than reducing serious events like complications and death.
EEs on influenza vaccination were designed on decision tree models, based on parameters referring to disease epidemiology and resource consumption related to previous influenza seasons. It is worth noting that influenza vaccine efficacy (IVE) estimates for any 1 year can hardly be applied to the next, because of the virus’ seasonal mutability. IVE mainly depends on the infectiousness of the virus and the antigenic correspondence between the vaccine and the circulating strain, both of which are hard to predict. Further intrinsic uncertainty in IVE estimates stems from using mainly the generic influenza-like illness (ILI) outcome to evaluate influenza, instead of laboratory-confirmed disease—hardly feasible in community care and very expensive too.
All the studies reviewed concluded in favour of the extension strategy, thanks to an estimated substantial drop in indirect costs . Their major limits concerned important aspects, such as the clinical definition of influenza and the time frame to evaluate the benefits of vaccination. Although mortality should not be a crucial factor for this population target, most studies questionably combined the lifetime horizon adopted for assessing the benefits of preventing influenza mortality with costs incurred within a 1-year time frame in the incremental cost-effectiveness ratio, making the extension strategy more likely to appear cost-effective.
The conclusions of sponsored and not-sponsored studies did not differ substantially in this field. Unlike HPV, influenza vaccines are marketed by many manufacturers. Since none of the EEs reviewed concentrated on any single vaccine, we might speculate that sponsored studies were aimed at encouraging market expansion by extending the coverage to not-at-risk groups, rather than promoting specific vaccines.
Major limitations associated with modelling arise from these two apparently very different examples. HPV vaccination seems to be a field where EEs built on modelling display their main limits in contributing to decision-making . First, models consider generally a lifetime horizon, which should boost the chances of showing a favourable cost-effectiveness for vaccination. Second, recourse to many uncertain assumptions seems an intrinsic limit of these long-term models, whose final results are influenced by their authors’ choices.
EEs built on models also show broad limits when it comes to vaccination of healthy adults against influenza. The general uncertainty about seasonal IVE estimates undermines the credibility of optimism about the economic benefits of vaccination. Moreover, since production costs are much more important than health care costs, the wide methodological discretion in measuring indirect costs can lead to very different results , and the analysis viewpoint plays a large role in determining the cost-effectiveness ratio of vaccination for this target population.
A common economic limit of these two examples is the widespread application of ex-factory vaccine prices instead of dispensing prices as the third party payer’s perspective would have required. This is a major drawback, particularly for studies conducted in countries where vaccines are purchased through public procurement that can significantly reduce the third payers’ actual prices. HPV vaccines in Italy provide an interesting example : the ex-factory price negotiated by the Italian Agency for Medicines (AIFA) at central level was more than halved after 2 years, thanks to regional tenders. Vaccine prices are often a further source of uncertainty in EEs, differently from drug prices in most pharmacoeconomic studies.
Another feature common to the two examples is that sponsorship did not seem to significantly affect the final results of the EEs reviewed, unlike what usually happens in pharmacoeconomic studies , since most sponsored and not-sponsored studies showed the favourable cost-effectiveness of vaccination strategies. A tentative explanation might be that models, thanks to their flexibility, can be used to obtain the results authors want. This has encouraged companies to develop general stochastic models, then adapting them to more than one country by tailoring and customizing epidemiologic and cost data. Another explanation for widespread positive results in not-sponsored studies on vaccinations might be that many authors are willing to back new vaccines or new vaccination programs, in line with prevention services aim to eradicate diseases through preventive care.
While companies seem fully aware of the possibilities of exploiting modelling to search convenient conclusions, health authorities could still hone their skills in evaluating the limits of EEs based on modelling, in order to provide credible studies for health care services.
In general, the uncertainty about results generated by modelling casts doubts on the Health Technology Assessment (HTA) approach. First, to avoid confounding factors and become a credible tool in health care decision-making, HTA has to be backed and managed by health authorities, transparently distinguishing assessors and manufacturers . However, health authorities must be fully aware of the intrinsic limits of long-term models, in order to avoid unjustified enthusiasm about their results. In particular, when clinical effectiveness and prices are both uncertain, like in our two examples, decision-makers might benefit more from EEs designed to indicate sustainable prices and using realistic estimates of the main variables like coverage rates, rather than recurring to optimistic assumptions in order to show acceptable cost-effectiveness.
- 5.Koleva, D., De Compadri, P., Casadei, G., Garattini, L.: Economic evaluation of human papilloma virus vaccination in the European union: a critical review. Intern Emerg Med (in press)Google Scholar
- 8.Centers for Disease Control and Prevention. CDC’s Advisory Committee on Immunization Practices (ACIP) Recommends Universal Annual Influenza Vaccination. http://www.cdc.gov/media/pressrel/2010/r100224.htm. Accessed 13 April 2010
- 9.Garattini, L., Koleva, D.: Influenza vaccine for healthy adult workers: an issue for health authorities or employers? Health Policy (in press)Google Scholar
- 11.Garattini, L., Tediosi, F., Ghislandi, S., et al.: How do Italian pharmacoeconomists evaluate indirect costs? Value Health 3(4), 270–276 (2000)Google Scholar
- 12.Garattini, L., Casadei, G.: Cure H: far bene la gara fa bene alla spesa. Sole 24 ore Sanità, 26 gennaio 2010Google Scholar
- 13.Garattini, L., Casadei, G.: Health technology assessment: for whom the bell tolls? Eur. J. Health Econ. 9, 311–312 (2008)Google Scholar