Probabilistic Sensitivity Analysis in Cost-Effectiveness Models: Determining Model Convergence in Cohort Models
Probabilistic sensitivity analysis (PSA) demonstrates the parameter uncertainty in a decision problem. The technique involves sampling parameters from their respective distributions (rather than simply using mean/median parameter values). Guidance in the literature, and from health technology assessment bodies, on the number of simulations that should be performed suggests a ‘sufficient number’, or until ‘convergence’, which is seldom defined. The objective of this tutorial is to describe possible outcomes from PSA, discuss appropriate levels of accuracy, and present guidance by which an analyst can determine if a sufficient number of simulations have been conducted, such that results are considered to have converged. The proposed approach considers the variance of the outcomes of interest in cost-effectiveness analysis as a function of the number of simulations. A worked example of the technique is presented using results from a published model, with recommendations made on best practice. While the technique presented remains essentially arbitrary, it does give a mechanism for assessing the level of simulation error, and thus represents an advance over current practice of a round number of simulations with no assessment of model convergence.
The content of this manuscript was agreed by AJH, ABu, ABr, MP and MDS. The first draft was prepared by AJH and ABu, and the manuscript was revised by AJH, ABu, ABr, MP and MDS. The method proposed was derived by MDS, ABr, ABu, AJH and MP, and the downloadable workbook was prepared by ABu.
Compliance with Ethical Standards
Conflict of interest
Anthony James Hatswell, Ash Bullement, and Mike Paulden report no conflicts of interest. Andrew Briggs and Matthew D. Stevenson have previously published on probabilistic analysis but have no financial conflicts of interest.
No funding was received for this manuscript.
- 3.Canadian Agency for Drugs and Technologies in Health. Guidelines for the economic evaluation of health technologies: Canada. 4th edn. 2017. https://www.cadth.ca/sites/default/files/pdf/guidelines_for_the_economic_evaluation_of_health_technologies_canada_4th_ed.pdf.
- 4.National Institute for Health and Care Excellence (NICE). Guide to the methods of technology appraisal. NICE; 2013. https://www.nice.org.uk/process/pmg9/chapter/foreword.
- 8.Barton GR, Briggs AH, Fenwick EAL. Optimal cost-effectiveness decisions: the role of the cost-effectiveness acceptability curve (CEAC), the cost-effectiveness acceptability frontier (CEAF), and the expected value of perfection information (EVPI). Value Health. 2008;11:886–97.CrossRefPubMedGoogle Scholar
- 10.Zorginstituut Nederland. Guideline for economic evaluations in healthcare. Guideline—National Health Care Institute [cited 21 May 2017]. 2016. https://english.zorginstituutnederland.nl/publications/reports/2016/06/16/guideline-for-economic-evaluations-in-healthcare.
- 17.National Institute for Health and Care Excellence. Naltrexone-bupropion (prolonged release) for managing overweight and obesity [ID757]. Company evidence submission [cited 16 Sep 2017]. 2017. https://www.nice.org.uk/guidance/gid-tag486/documents/appraisal-consultation-document-2.
- 18.National Institute for Health and Care Excellence. Colorectal cancer (metastatic): trifluridine with tipiracil hydrochloride, after standard therapy [ID876]. Company evidence submission [cited 16 Sep 2017]. 2016. https://www.nice.org.uk/guidance/ta405/documents/committee-papers.
- 19.Scottish Medicines Consortium. Working with SMC—a guide for manufacturers. Glasgow: Scottish Medicines Consortium; 2017. p. 11.Google Scholar
- 21.Jackknife-After-Bootstrap Efron B, Errors Standard, Functions Influence. J R Stat Soc Ser B Methodol. 1992;54:83–127.Google Scholar