Avoid common mistakes on your manuscript.
In a recent article, Wilson et al. estimated the health and economic impact of switching the pediatric immunization program from the 13-valent (PCV-13) to the 10-valent (PHiD-CV) pneumococcal conjugate vaccine in Canada [1]. In this letter, we would like to highlight a number of methodologic issues that hamper the results that were not included in the original article.
Wilson and colleagues utilized regional pneumococcal incidence data from the Toronto Invasive Bacterial Diseases Network (TIBDN) as the basis of their analysis. The question is whether this is representative for the whole of Canada and why none of the available national data sets were used that would have provided a more robust foundation for the analysis. These include the Canadian Notifiable Disease Surveillance System (CNDSS), the Immunization Monitoring Program, ACTive (IMPACT) network, and the National Enhanced Invasive Pneumococcal Disease Surveillance System (eIPD) pilot [2]. The Public Health Agency of Canada also publishes annual reports of the National Laboratory Surveillance from invasive pneumococcal diseases (IPD) in Canada [3]. These are all sources of data with broader national reach. Differences in serotype distribution as well as immunization rates vary between provinces and would potentially affect serotype response [4]. Furthermore, the incidence figures presented in Wilson et al. are not representative of the most recent IPD incidence rates in the National Laboratory Surveillance [3], suggesting that such arbitrary use of regional data is not appropriate for a Canada-wide analysis.
The regression-based decision-analytic modeling approach used by Wilson and colleagues is not an approach recommended by the International Society for Pharmacoeconomics and Outcomes Research (ISPOR) or the Society for Medical Decision Making (SMDM) [5]. ISPOR and SMDM drafted a series of articles on best practices in the conduct and reporting of modeling studies to ensure reliable and valid results [5]. State-transition models, discrete event simulations, and dynamic transmission models were identified as best-practice modeling techniques subject to the original decision problem. The outcomes when extrapolated over an additional 10-year period may result in uncertainties being grossly magnified; such extrapolation distortions compromise data integrity.
Wilson and colleagues performed age- and serotype-specific regression analyses, resulting in many underlying regression outcomes for which the fit cannot be evaluated. While the authors provided readers with the R2 values for each regression equation [ranging from very weak (0.0091) to good (0.9990)], there was no validation of the predicted versus the observed historical data. Additionally, R2 values alone cannot determine whether the predictions are biased nor can they indicate the fit of a model (for this, one must consult a goodness-of-fit statistic). Thus, the underlying analysis is built upon multiple fitted parameters for which one cannot determine the fit or bias. Using this approach as a basis for an economic evaluation can never result in confidence of the outcomes. ISPOR guidelines also state that vital model parameters be tested in uncertainty analyses [6]. It is concerning that the authors have not included the slope parameters of any of the covered/uncovered serotypes into any of the sensitivity analyses. As can be seen from the scenario analysis results (Table 4), the source of the data used for the trend analyses significantly impacts the resulting incremental cost and quality-adjusted life-years (QALYs).
Wilson and colleagues have drawn conclusions from this study that are inconsistent with those drawn by the Pan American Health Organization and the World Health Organization on the comparability of the different pneumococcal conjugate vaccine (PCV) products [7, 8]. Particularly, studies have shown that PCVs of differing valencies afford similar levels of protection against all-cause pneumonia and IPD. Neglecting this body of evidence, the authors have applied the same previously explained incorrectly assumed yearly impact (rate reduction) on IPD to the all-cause pneumonia and acute otitis media (AOM). The incidences of these two diseases are orders of magnitude higher than IPD, and thus the use of this approach vastly distorts the potential impact of any switch.
In Canada, PHiD-CV is indicated for the prevention of diseases caused by Streptococcus pneumoniae serotypes 1, 4, 5, 6B, 7F, 9V, 14, 18C, 19F, 23F, and cross-reactive 19A [9], based on post-marketing effectiveness studies conducted in Quebec, Brazil, and Finland. In these studies, the vaccine effectiveness (VE) for PHiD-CV against 19A IPD cases was 71, 82, and 62%, respectively [10,11,12]. However, the approach used by Wilson et al. did not account for this cross-reactivity against serotype 19A. According to the Canadian product monograph for PHiD-CV, the VE for AOM cases caused by nontypeable Haemophilus influenzae (NTHi) was 15% [9]. The protection against AOM cases caused by NTHi was not included in the analysis by Wilson et al., even though previous cost-effectiveness analyses have shown that when the NTHi effect is included, PHiD-CV is more cost-effective than PCV-13 [13, 14]. By not including the totality of the published literature, the authors have introduced bias into their study leading to a distortion of the impact of a switch from PCV-13 to PHiD-CV in Canada. The results are also inconsistent with health economic analyses conducted by the Comité sur l’immunisation du Québec (CIQ) demonstrating that PHiD-CV was more cost-effective compared with PCV-13 in Quebec [15].
While the results of Wilson et al. suggest that switching from PCV-13 to PHiD-CV would result in increased costs of $500 million over 10 years, the methodology used and the data on which the analysis is based are prone to bias and likely to distort the resulting conclusions. Caution should be used when interpreting the results of this study as the totality of the evidence was not included, the underlying data set was regionally specific and not nationally representative, and the methodology is prone to bias.
References
Wilson M, Wasserman M, Jadavi T, Postma M, Breton MC, Peloquin F, et al. Clinical and economic impact of a potential switch from 13-valent to 10-valent pneumococcal conjugate infant vaccination in Canada. Infect Dis Ther. 2018.
Government of Canada. Invasive Pneumococcal Disease: Surveillance of Invasive Pneumococcal Disease (IPD) 2018 [cited 2018 July 13]. https://www.canada.ca/en/public-health/services/immunization/vaccine-preventable-diseases/invasive-pneumococcal-disease/surveillance.html.
Streptococcus and STI Unit NML, Public Health Agency of Canada. National Laboratory Surveillance of Invasive Streptococcal Disease in Canada. 2016.
Busby C, Chesterley N. A Shot in the arm: how to improve vaccination policy in Canada C.D. Howe Institute. 2015.
Caro JJ, Briggs AH, Siebert U, Kuntz KM. Modeling good research practices–overview: a report of the ISPOR-SMDM Modeling Good Research Practices Task Force-1. Med Decis Making. 2012;32(5):667–77.
Briggs AH, Weinstein MC, Fenwick EA, Karnon J, Sculpher MJ, Paltiel AD, et al. Model parameter estimation and uncertainty: a report of the ISPOR-SMDM Modeling Good Research Practices Task Force-6. Value Health. 2012;15(6):835–42.
de Oliveira LH, Camacho LA, Coutinho ES, Martinez-Silveira MS, Carvalho AF, Ruiz-Matus C, et al. Impact and effectiveness of 10 and 13-valent pneumococcal conjugate vaccines on hospitalization and mortality in children aged less than 5 years in Latin American Countries: a systematic review. PLoS ONE. 2016;11(12):e0166736.
SAGE. Executive Summary: SAGE 2017, Pneumococcal Conjugate Vaccine Session. 2017.
GSK Canada. Synflorix Product Monograph 2017 (cited 2018 Apr 30). https://ca.gsk.com/media/591956/synflorix.pdf.
Deceuninck G, De Serres G, Boulianne N, Lefebvre B, De Wals P. Effectiveness of three pneumococcal conjugate vaccines to prevent invasive pneumococcal disease in Quebec, Canada. Vaccine. 2015;33(23):2684–9.
Domingues CM, Verani JR, Montenegro Renoiner EI, de Cunto Brandileone MC, Flannery B, de Oliveira LH, et al. Effectiveness of ten-valent pneumococcal conjugate vaccine against invasive pneumococcal disease in Brazil: a matched case-control study. Lancet Respir Med. 2014;2(6):464–71.
Jokinen J, Rinta-Kokko H, Siira L, Palmu AA, Virtanen MJ, Nohynek H, et al. Impact of ten-valent pneumococcal conjugate vaccination on invasive pneumococcal disease in Finnish children—a population-based study. PLoS ONE. 2015;10(3):e0120290.
Chuck AW, Jacobs P, Tyrrell G, Kellner JD. Pharmacoeconomic evaluation of 10- and 13-valent pneumococcal conjugate vaccines. Vaccine. 2010;28(33):5485–90.
Newall AT, Creighton P, Philp DJ, Wood JG, MacIntyre CR. The potential cost-effectiveness of infant pneumococcal vaccines in Australia. Vaccine. 2011;29(45):8077–85.
Comité sur l’immunisation du Québec (CIQ). Scientific advisory on the optimal schedule for childhood immunization against pneumococcal disease in Québec. Quebec: Institut national de santé publique Quebec. 2017.
Acknowledgements
Funding
GlaxoSmithKline Biologicals SA (Rixensart, Belgium) was in charge of all costs associated with the development of this letter.
Editorial Assistance
The authors would like to thank the Business & Decision Life Sciences platform for editorial assistance and manuscript coordination on behalf of GSK, Fabien Debailleul coordinated the manuscript development and provided editorial support.
Authorship
All named authors meet the International Committee of Medical Journal Editors (ICMJE) criteria for authorship for this article, take responsibility for the integrity of the work as a whole, and have given their approval for this version to be published.
Disclosures
Ashleigh McGirr is an employee of the GSK group of companies. Shehzad Iqbal is an employee of the GSK group of companies. Jan Olbrecht is an employee of the GSK group of companies. Lijoy Varghese is an employee of the GSK group of companies.
Compliance with Ethics Guidelines
This article is based on previously conducted studies and does not contain any studies with human participants or animals performed by any of the authors.
Open Access
This article is distributed under the terms of the Creative Commons Attribution-NonCommercial 4.0 International License (http://creativecommons.org/licenses/by-nc/4.0/), which permits any noncommercial use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.
Author information
Authors and Affiliations
Corresponding author
Additional information
Enhanced digital features
To view enhanced digital features for this article go to https://doi.org/10.6084/m9.figshare.7241081.
Rights and permissions
This article is published under an open access license. Please check the 'Copyright Information' section either on this page or in the PDF for details of this license and what re-use is permitted. If your intended use exceeds what is permitted by the license or if you are unable to locate the licence and re-use information, please contact the Rights and Permissions team.
About this article
Cite this article
McGirr, A., Iqbal, S.M., Olbrecht, J. et al. Comment on: ‘Clinical and Economic Impact of a Potential Switch from 13-Valent to 10-Valent Pneumococcal Conjugate Infant Vaccination in Canada’, Wilson et al., 22 June 2018. Infect Dis Ther 7, 535–538 (2018). https://doi.org/10.1007/s40121-018-0220-3
Received:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s40121-018-0220-3