Exploring Structural Uncertainty in Model-Based Economic Evaluations
- 372 Downloads
Given the inherent uncertainty in estimates produced by decision analytic models, the assessment of uncertainty in model-based evaluations is an essential part of the decision-making process. Although the impact of uncertainty around the choice of model structure and making incorrect structural assumptions on model predictions is noted, relatively little attention has been paid to characterising this type of uncertainty in guidelines developed by national funding bodies such as the Australian Pharmaceutical Benefits Advisory Committee (PBAC). The absence of a detailed description and evaluation of structural uncertainty can add further uncertainty to the decision-making process, with potential impact on the quality of funding decisions. This paper provides a summary of key elements of structural uncertainty describing why it matters and how it could be characterised. Five alternative approaches to characterising structural uncertainty are discussed, including scenario analysis, model selection, model averaging, parameterization and discrepancy. We argue that the potential effect of structural uncertainty on model predictions should be considered in submissions to national funding bodies; however, the characterisation of structural uncertainty is not well defined within the guidelines of these bodies. There has been little consideration of the forms of structural sensitivity analysis that might best inform applied decision-making processes, and empirical research in this area is required.
KeywordsHealth Technology Assessment Deviance Information Criterion Funding Decision Structural Uncertainty Bayesian Information Criterion
Hossein Haji Ali Afzali and Jonathan Karnon conceptualized the manuscript and prepared the final draft. They share full responsibility for its content. Hossein Haji Ali Afzali is the overall guarantor.
No sources of funding were used to prepare this manuscript.
Conflicts of interest
Hossein Haji Ali Afzali has served as a member of the Protocol Advisory Subcommittee (PASC) of the Medical Services Advisory Committee (MSAC) since 2012. Jonathan Karnon has served as a member of the Economics Subcommittee (ESC) of the PBAC since 2009.
- 6.Committee Pharmaceutical Benefits Advisory. Guidelines for preparing submissions to the Pharmaceutical Benefits Advisory Committee. Canberra: Australian Government; 2013.Google Scholar
- 7.National Institute for Health and Clinical Excellence (NICE). Guide to the methods of technology appraisal 2013. London: NICE; 2013.Google Scholar
- 8.Canadian Agency for Drugs and Technologies in Health. Guidelines for the economic evaluation of health technologies. Ottawa: Canadian Agency for Drugs and Technologies in Health; 2006.Google Scholar
- 13.Frederix GW, van Hasselt JG, Schellens JH, Hovels AM, Raaijmakers JA, Huitema AD, et al. The impact of structural uncertainty on cost-effectiveness models for adjuvant endocrine breast cancer treatments: the need for disease-specific model standardization and improved guidance. Pharmacoeconomics. 2014;32:47–61.CrossRefPubMedGoogle Scholar
- 14.Bojke L, Claxton K, Palmer S, Sculpher M. Defining and characterising structural uncertainty in decision analytic models. York: Centre for Health Economics, University of York; 2006.Google Scholar
- 17.Jones L, Griffin S, Palmer S, Main C, Orton V, Sculpher M, et al. Clinical effectiveness and cost-effectiveness of clopidogrel and modified-release dipyridamole in the secondary prevention of occlusive vascular events: a systematic review and economic evaluation. Health Technol Assess. 2004;8:1–196.Google Scholar
- 19.Haji Ali Afzali H, Karnon J. Specification and implementation of decision analytic model structures for the economic evaluation of health care technologies. In: Culyer AJ, editor. Encyclopaedia of health economics. 1st ed. London: Elsevier; 2014: p. 340–7.Google Scholar
- 20.Burnham KP, Anderson DR. Model selection and multi-model inference: a practical information-theoretic approach. New York: Springer; 2002.Google Scholar
- 22.Draper D. Assessment and propagation of model uncertainty. J R Stat Soc. 1995;57:45–97.Google Scholar
- 25.Laud PW, Ibrahim JG. Predictive model slection. J R Stat Soc. 1995;57:247–62.Google Scholar