Abstract
Objective: The aim of this study was to examine how calibration uncertainty affects the overall uncertainty of a mathematical model and to evaluate potential drivers of calibration uncertainty.
Methods: A lifetime Markov model of the natural history of human papillomavirus (HPV) infection and cervical disease was developed to assess the cost effectiveness of a hypothetical HPV vaccine. Published data on cervical cancer incidence and mortality and prevalence of pre-cursor lesions were used as endpoints to calibrate the age- and HPV-type-specific transition probabilities between health states using the Nelder-Mead simplex method of calibration. A conventional probabilistic sensitivity analysis (PSA) was performed to assess uncertainty in vaccine efficacy, cost and utility estimates. To quantify the uncertainty around calibrated transition probabilities, a second PSA (calibration PSA) was performed using 25 distinct combinations of objective functions and starting simplexes.
Results: The initial calibration produced an incremental cost-effectiveness ratio (ICER) of $US4300 per QALY for vaccination compared with no vaccination, and the conventional PSA gave a 95% credible interval of dominant to $US9800 around this estimate (2005 values). The 95% credible interval for the ICERs in the calibration PSA ranged from $US1000 to $US37 700.
Conclusions: Compared with a conventional PSA, the calibration PSA results reveal a greater level of uncertainty in cost-effectiveness results. Sensitivity analyses around model calibration should be performed to account for uncertainty arising from the calibration process.
This is a preview of subscription content, log in via an institution to check access.
References
Weinstein MC. Recent developments in decision-analytic modelling for economic evaluation. Pharmacoeconomics 2006; 24 (11): 1043–53
Law AM, Kelton WD. Simulation modeling and analysis. 3rd ed. Boston (MA): McGraw-Hill, 2000
Clarke LD, Plevritis SK, Boer R, et al. A comparative review of CISNET breast models used to analyze U.S. breast cancer incidence and mortality trends. J Natl Cancer Inst Monogr 2006; 36: 96–105
Banks J, editor. Handbook of simulation: principles, methodology, advances, applications, and practice. New York: John Wiley & Sons, Inc., 1998
Vanni T, Karnon J, Madan J, et al. Calibrating models in economic evaluation: a seven-step approach. Pharmacoeconomics 2011 Jan 1; 29 (1): 35–49
Taylor DCA, Pawar V, Kruzikas D, et al. Calibrating longitudinal models to cross-sectional data: the effect of temporal changes in health practices. Value Health 2011 Jul-Aug; 14 (5): 700–4
Kohli M, Ferko N, Martin A, et al. Estimating the long term impact of prophylactic human papillomavirus 16/18 vaccine on the burden of cervical cancer in the UK. Br J Cancer 2007; 96: 143–50
Ries LAG, Melbert D, Krapcho M, et al. SEER cancer statistics review, 1975-2004. Bethesda (MD): National Cancer Institute, 2007 [online]. Available from URL: http://seer.cancer.gov/csr/1975_2004/ [Accessed 2010 Aug 16]
Benard V, Eheman C, Lawson H, et al. Cervical screening in the national breast and cervical cancer early detection program, 1995-2001. Obstet Gynecol 2004; 103 (3): 564–71
Nelder J, Mead R. A simplex method for function minimization. Comput J 1965; 7: 308–13
Taylor DC, Pawar V, Kruzikas D, et al. Methods of model calibration: observations from a mathematical model of cervical cancer. Pharmacoeconomics 2010 Nov 1; 28 (11): 995–1000
Stout NK, Knudsen AB, Kong CY, et al. Calibration methods used in cancer simulation models and suggested reporting guidelines. Pharmacoeconomics 2009; 27 (7): 533–45
Fryback DG, Stout NK, Rosenberg MA, et al. The Wisconsin breast cancer epidemiology simulation model. J Natl Cancer Inst Monogr 2006; 36: 37–47
Urban N, Drescher C, Etzioni R, et al. Use of a stochastic simulation model to identify an efficient protocol for ovarian cancer screening. Control Clin Trials 1997 Jun; 18 (3): 251–70
Berkhof J, de Bruijne MC, Zielinski GD, et al. Natural history and screening model for high-risk human papillomavirus infection, neoplasia and cervical cancer in the Netherlands. Int J Cancer 2005 Jun 10; 115 (2): 268–75
Kim JJ, Kuntz KM, Stout NK, et al. Multi-parameter calibration of a natural history model of cervical cancer. Am J Epidemiol 2007; 166 (2): 137–50
Van de Velde N, Brisson M, Boily M-C. Modeling human papillomavirus vaccine effectiveness: quantifying the impact of parameter uncertainty. Am J Epidemiol 2007 Apr 1; 165 (7): 762–75
Goldhaber-Fiebert JD, Stout NK, Ortendahl J, et al. Modeling human papillomavirus and cervical cancer in the United States for analyses of screening and vaccination. Population Health Metrics 2007 Oct 29; 5: 11
Jit M, Edmunds WJ. Economic evaluation of human papillomavirus vaccination in the United Kingdom. BMJ 2008; 337: a769
Karnon J, Czoski-Murray C, Smith KJ, et al. A hybrid cohort individual sampling natural history model of age-related macular degeneration: assessing the cost-effectiveness of screening using probabilistic calibration. Med Decis Making 2009 May-Jun; 29 (3): 304–16
Rutter CM, Miglioretti DL, Savarino JE. Bayesian calibration of microsimulationmodels. J Am Stat Assoc 2009 Dec 1; 104 (488): 1338–50
Stout NK, Rosenberg MA, Trentham-Dietz A, et al. Retrospective cost-effectiveness analysis of screening mammography. J Natl Cancer Inst 2006; 98: 774–82
Acknowledgements
Funding for this study was provided by GlaxoSmithKline, Philadelphia, PA, US. Denise T. Kruzikas was an employee of GlaxoSmithKline when this work was performed, but is now employed by GE Healthcare. All other authors were paid consultants to GlaxoSmithKline. Vivek Pawar is now an employee of Bayer Healthcare Pharmaceuticals, Inc. Milton C. Weinstein conducts academic research and consulting that makes use of the methodologies described in this paper.
The authors would like to thank Ankur Pandya and Rowan Iskandar for their valuable assistance in developing the calibration tools used in this study.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Taylor, D., Pawar, V., Kruzikas, D.T. et al. Incorporating Calibrated Model Parameters into Sensitivity Analyses. PharmacoEconomics 30, 119–126 (2012). https://doi.org/10.2165/11593360-000000000-00000
Published:
Issue Date:
DOI: https://doi.org/10.2165/11593360-000000000-00000