Health economic evaluation is a framework for the comparative analysis of the incremental health gains and costs associated with competing decision alternatives. The process of developing health economic models is usually complex, financially expensive and time-consuming. For these reasons, model development is sometimes based on previous model-based analyses; this endeavour is usually referred to as model replication. Such model replication activity may involve the comprehensive reproduction of an existing model or ‘borrowing’ all or part of a previously developed model structure. Generally speaking, the replication of an existing model may require substantially less effort than developing a new de novo model by bypassing, or undertaking in only a perfunctory manner, certain aspects of model development such as the development of a complete conceptual model and/or comprehensive literature searching for model parameters. A further motivation for model replication may be to draw on the credibility or prestige of previous analyses that have been published and/or used to inform decision making. The acceptability and appropriateness of replicating models depends on the decision-making context: there exists a trade-off between the ‘savings’ afforded by model replication and the potential ‘costs’ associated with reduced model credibility due to the omission of certain stages of model development. This paper provides an overview of the different levels of, and motivations for, replicating health economic models, and discusses the advantages, disadvantages and caveats associated with this type of modelling activity. Irrespective of whether replicated models should be considered appropriate or not, complete replicability is generally accepted as a desirable property of health economic models, as reflected in critical appraisal checklists and good practice guidelines. To this end, the feasibility of comprehensive model replication is explored empirically across a small number of recent case studies. Recommendations are put forward for improving reporting standards to enhance comprehensive model replicability.
This is a preview of subscription content, log in to check access.
Briggs A, Claxton K, Sculpher M. Decision modelling for health economic evaluation. 1st ed. New York: Oxford University Press; 2006.
Popper K. The logic of scientific discovery. 1st ed. London: Routledge; 1992.
Weinstein M, O’Brien B, Hornberger J, Jackson J, Johannesson M, McCabe M, et al. Principles of good practice for decision analytic modeling in health-care evaluation: report of the ISPOR task force on good research practices—modeling studies. Value Health. 2003;6(1):9–17.
Husereau D, Drummond M, Petrou S, Carswell C, Moher D, Greenberg D. Consolidated health economic evaluation reporting standards (CHEERS)—explanation and elaboration: a report of the ISPOR health economic evaluation publication guidelines good reporting practices task force. Value Health. 2013;16(2):231–50.
Eddy D. Technology assessment: the role of mathematical modeling assessing medical technology. 1st ed. Washington, DC: National Academy Press; 1985.
Akehurst R, Anderson P, Brazier J, Brennan A, Briggs A, Buxton M, et al. Decision analytic modelling in the economic evaluation of health technologies. Pharmacoeconomics. 2000;17(5):443–4.
Eddy D, Hollingworth W, Caro J, Tsevat J, McDonald K, Wong J. Model transparency and validation: a report of the ISPOR-SMDM modeling good research practices task force-7. Med Decis Marking. 2012;32(5):733–43.
McManus E. How reproducible are economic decision-analytic models? A systematic review and replication study—study outline. 2016. Available at: http://www.clahrcprojects.co.uk/impact/publications/how-reproducible-are-economic-decision-analytic-models-systematic-review-and.
Woods S, Rizzo J. Cost-effectiveness of antidepressant treatment reassessed. Br J Psychiatry. 1997;170:257–63.
Batty A, Hansen R, Bloudek L, Varon S, Hayward E, Pennington B, et al. The cost-effectiveness of onabotulinumtoxinA for the prophylaxis of headache in adults with chronic migraine in the UK. J Med Econ. 2013;16(7):877–87.
Smolen L, Klein T, Kelton K. Replication of a published Markov chronic migraine cost-effectiveness analysis model for purposes of early phase adaptation and expansion. Value Health. 2015;83:A19.
Palmer S, Sculpher M, Philips Z, Robinson M, Ginnelly L, Bakhai A, et al. Management of non-ST-elevation acute coronary syndromes: how cost-effective are glycoprotein IIb/IIIA antagonists in the UK National Health Service? Int J Cardiol. 2005;100(2):229–40.
AstraZeneca Ltd. Ticagrelor for the treatment of acute coronary syndromes: manufacturer’s submission to the National Institute for Health and Care Excellence. 2011. Available at: https://www.nice.org.uk/guidance/TA236/documents/acute-coronary-syndromes-ticagrelor-astrazeneca4.
The Medicines Company. Bivalirudin (Angiox®) for the treatment of ST-segment elevation myocardial infarction: manufacturer’s submission to the National Institute for Health and Care Excellence. 2011. Available at: https://www.nice.org.uk/guidance/TA230/documents/myocardial-infarction-persistent-stsegment-elevation-bivalirudin-the-medicines-company2.
Chilcott J, McCabe C, Tappenden P, O’Hagan A, Cooper N, Abrams K, et al. Modelling the cost effectiveness of interferon beta and glatiramer acetate in the management of multiple sclerosis. Commentary: evaluating disease modifying treatments in multiple sclerosis. Br Med J. 2003;326(7388):522.
Allen F, Montgomery S, Maruszczak M, Kusel J, Adlard N. Convergence yet continued complexity: a systematic review and critique of health economic models of relapsing-remitting multiple sclerosis in the United Kingdom. Value Health. 2015;18(6):925–38.
Biogen Idec Ltd. Natalizumab (Tysabri®) for the treatment of adults with highly active relapsing remitting multiple sclerosis: manufacturer’s submission to the National Institute for Health and Care Excellence. 2007. Available at: https://www.nice.org.uk/guidance/TA127/documents/multiple-sclerosis-natalizumab-manufacturer-submissions-biogen-idec-uk-and-elan-pharma-international-ltd-joint-development-agreement-confidential-information-removed2.
Gani R, Giovannoni G, Bates D, Kemball B, Hughes S, Kerrigan J. Cost-effectiveness analyses of natalizumab (Tysabri) compared with other disease-modifying therapies for people with highly active relapsing-remitting multiple sclerosis in the UK. Pharmacoeconomics. 2008;26(7):617–27.
Genzyme. Teriflunomide for the treatment of relapsing remitting multiple sclerosis in adults: manufacturer’s submission to the National Institute for Health and Care Excellence. 2013. Available at: https://www.nice.org.uk/guidance/TA303/documents/multiple-sclerosis-relapsing-teriflunomide-evaluation-report4.
Novartis Pharmaceuticals UK Ltd. Fingolimod for the treatment of relapsing remitting multiple sclerosis in adults: manufacturer’s submission to the National Institute for Health and Care Excellence. 2011. Available at: https://www.nice.org.uk/guidance/TA254/documents/manufacturer-submission-from-novartis2.
Biogen Idec Ltd. Dimethyl fumarate for the treatment of adult patients with relapsing remitting multiple sclerosis: manufacturer’s submission to the National Institute for Health and Care Excellence. 2013. Available at: https://www.nice.org.uk/guidance/TA320/documents/multiple-sclerosis-relapsingremitting-dimethyl-fumarate-evaluation-report2.
Woolacott N, Hawkins N, Mason A, Kainth A, Khadjesari Z, Bravo Vergel Y, et al. Efalizumab and etanercept for the treatment of psoriasis: assessment report submitted to the National Institute for Health and Care Excellence. 2007. Available at: https://www.nice.org.uk/guidance/TA103/documents/psoriasis-efalizumab-and-etanercept-assessment-report2.
Schering-Plough Ltd. Remicade® in the treatment of psoriasis in England and Wales: manufacturer’s submission to the National Institute for Health and Care Excellence. 2007. Available at: https://www.nice.org.uk/guidance/TA134/documents/psoriasis-infliximab-scheringplough-ltd2.
Abbott Laboratories Ltd. Adalimumab (Humira®) for the treatment of moderate to severe plaque psoriasis: manufacturer’s submission to the National Institute for Health and Care Excellence. 2007. Available at: https://www.nice.org.uk/guidance/TA146/documents/psoriasis-adalimumab-appraisal-consultation-manufacturer-submission-abbott-laboratories-ltd2.
Janssen-Cilag Ltd. Ustekinumab (Stelara) for the treatment of moderate to severe plaque psoriasis in England and Wales: manufacturer’s submission to the National Institute for Health and Care Excellence. 2009. Available at: https://www.nice.org.uk/guidance/TA180/documents/psoriasis-ustekinumab-manufacturer-submission-janssencilag2.
Eggington S, Tappenden P, Pandor A, Paisley S, Saunders M, Seymour M, et al. The clinical and cost-effectiveness of oxaliplatin and capecitabine for the adjuvant treatment of colon cancer: systematic review and economic evaluation. Br J Cancer. 2006;95(9):1195–201.
Tappenden P, Jones R, Paisley S, Carroll C. The cost-effectiveness of bevacizumab in the first-line treatment of metastatic colorectal cancer in England and Wales. Eur J Cancer. 2007;43(17):2487–94.
Kearns B, Rafia R, Leaviss J, Preston L, Brazier J, Palmer S, et al. The cost-effectiveness of changes to the care pathway used to identify depression and provide treatment amongst people with diabetes in England: a model-based economic evaluation. BMC Health Serv Res. 2017;17(1):78.
Tosh J, Kearns B, Brennan A, Parry G, Ricketts T, Saxon D, et al. Innovation in health economic modelling of service improvements for longer-term depression: demonstration in a local health community. BMC Health Serv Res. 2013;13:150.
Clarke P, Gray A, Legood R, Briggs A, Holman R. The impact of diabetes-related complications on healthcare costs: results from the United Kingdom Prospective Diabetes Study (UKPDS Study No. 65). Diabetic Med. 2003;20(6):442–50.
Alva M, Gray A, Mihaylova B, Clarke P. The effect of diabetes complications on health-related quality of life: the importance of longitudinal data to address patient heterogeneity. Health Econ. 2014;23(4):487–500.
Hayes A, Leal J, Gray A, Holman R, Clarke P, et al. UKPDS outcomes model 2: a new version of a model to simulate lifetime health outcomes of patients with type 2 diabetes mellitus using data from the 30 year United Kingdom Prospective Diabetes Study: UKPDS 82. Diabetologia. 2013;56(9):1925–33.
National Institute for Health and Care Excellence. Bosutinib for previously treated chronic myeloid leukaemia. 2013. Available at: https://www.nice.org.uk/guidance/ta299.
National Institute for Health and Care Excellence. Dasatinib, high-dose imatinib and nilotinib for the treatment of imatinib-resistant chronic myeloid leukaemia (CML) (part review of NICE technology appraisal guidance 70), and dasatinib and nilotinib for people with CML for whom treatment with imatinib has failed because of intolerance. 2012. Available at: https://www.nice.org.uk/guidance/ta299.
Stevenson M, Archer R, Tosh J, Simpson E, Everson-Hock E, Stevens J, et al. Adalimumab, etanercept, infliximab, certolizumab pegol, golimumab, tocilizumab and abatacept for the treatment of rheumatoid arthritis not previously treated with disease-modifying antirheumatic drugs and after the failure of conventional disease-modifying antirheumatic drugs only: systematic review and economic evaluation. Health Technol Assess. 2016;20(35):1–610.
National Institute for Health and Care Excellence. Adalimumab, etanercept, infliximab, rituximab and abatacept for the treatment of rheumatoid arthritis after the failure of a TNF inhibitor. 2013. Available at: https://www.nice.org.uk/guidance/ta195.
National Institute for Health and Care Excellence. Adalimumab, etanercept and infliximab for the treatment of rheumatoid arthritis. 2013. Available at: https://www.nice.org.uk/guidance/ta130.
Norton S, Sacker A, Dixey J, Done J, Williams P, Young A. Trajectories of functional limitation in early rheumatoid arthritis and their association with mortality. Rheumatology. 2013;52(11):2016–24.
Tappenden P, Chilcott J. Avoiding and identifying errors and other threats to the credibility of health economic models. Pharmacoeconomics. 2014;32(10):967–79.
Elvidge J, Bullement A, Hatswell A. Cost effectiveness of characterised chondrocyte implantation for treatment of cartilage defects of the knee in the UK. Pharmacoeconomics. 2016;34(11):1145–59.
Tappenden P, Ren S, Archer R, Harvey R, Martyn St-James M, Basarir H, et al. A model-based economic evaluation of biologic and non-biologic options for the treatment of adults with moderately-to-severely active ulcerative colitis after the failure of conventional therapy. Pharmacoeconomics. 2016;34(10):1023–38.
Davies M, Glah D, Chubb B, Konidaris G, McEwan P. Cost effectiveness of IDegLira vs. alternative basal insulin intensification therapies in patients with type 2 diabetes mellitus uncontrolled on basal insulin in a UK setting. Pharmacoeconomics. 2016;34(9):953–66.
Oddershede L, Walker S, Stöhr W, Dunn D, Arenas-Pinto A, Paton NI, et al. Cost effectiveness of protease inhibitor monotherapy versus standard triple therapy in the long-term management of HIV patients: analysis using evidence from the PIVOT trial. Pharmacoeconomics. 2016;34(8):795–804.
Versteegh M. Impact on the incremental cost-effectiveness ratio of using alternatives to EQ-5D in a Markov model for multiple sclerosis. Pharmacoeconomics. 2016;34(11):1133–44.
McEwan P, Foos V, Palmer J, Lamotte M, Lloyd A, Grant D, et al. Validation of the IMS CORE diabetes model. Value Health. 2014;17(6):714–24.
Barber J, Thompson S. Multiple regression of cost data: use of generalised linear models. J Health Serv Res Policy. 2004;9(4):197–204.
Dunlop W, Mason N, Kenworthy J, Akehurst R. Benefits, challenges and potential strategies of open source health economic models. Pharmacoeconomics. 2017;35(1):125–8.
IB, PT and JY did not receive any funding to support the drafting of this manuscript and have no conflicts of interest to declare. PT attempted to replicate four of the five models included in the pilot study, IB replicated the remaining model, and JY replicated one of the models. All authors were involved in writing the manuscript.
About this article
Cite this article
Bermejo, I., Tappenden, P. & Youn, J. Replicating Health Economic Models: Firm Foundations or a House of Cards?. PharmacoEconomics 35, 1113–1121 (2017). https://doi.org/10.1007/s40273-017-0553-x