Transparency in Decision Modelling: What, Why, Who and How?

  • Christopher James SampsonEmail author
  • Renée Arnold
  • Stirling Bryan
  • Philip Clarke
  • Sean Ekins
  • Anthony Hatswell
  • Neil Hawkins
  • Sue Langham
  • Deborah Marshall
  • Mohsen Sadatsafavi
  • Will Sullivan
  • Edward C. F. Wilson
  • Tim Wrightson
Review Article


Transparency in decision modelling is an evolving concept. Recently, discussion has moved from reporting standards to open-source implementation of decision analytic models. However, in the debate about the supposed advantages and disadvantages of greater transparency, there is a lack of definition. The purpose of this article is not to present a case for or against transparency, but rather to provide a more nuanced understanding of what transparency means in the context of decision modelling and how it could be addressed. To this end, we review and summarise the discourse to date, drawing on our collective experience. We outline a taxonomy of the different manifestations of transparency, including reporting standards, reference models, collaboration, model registration, peer review and open-source modelling. Further, we map out the role and incentives for the various stakeholders, including industry, research organisations, publishers and decision makers. We outline the anticipated advantages and disadvantages of greater transparency with respect to each manifestation, as well as the perceived barriers and facilitators to greater transparency. These are considered with respect to the different stakeholders and with reference to issues including intellectual property, legality, standards, quality assurance, code integrity, health technology assessment processes, incentives, funding, software, access and deployment options, data protection and stakeholder engagement. For each manifestation of transparency, we discuss the ‘what’, ‘why’, ‘who’ and ‘how’. Specifically, their meaning, why the community might (or might not) wish to embrace them, whose engagement as stakeholders is required and how relevant objectives might be realised. We identify current initiatives aimed to improve transparency to exemplify efforts in current practice and for the future.



SE kindly acknowledges his colleagues at Collaborations Pharmaceuticals.

Author Contributions

All authors were involved in the conception and planning of the manuscript, wrote sections, edited and commented on drafts and approved the submitted version. CS organised and led the write-up and contributed to the writing of all sections. RA wrote parts of Sects. 15. SB wrote parts of Sects. 4 and 5. PC wrote parts of Sects. 2. SE wrote parts of Sects. 2, 4 and 5. AH wrote parts of Sects. 25. NH wrote parts of Sects. 1, 3 and 4. SL wrote parts of Sects. 25. DM wrote parts of Sects. 1, 3 and 5. MS wrote parts of Sects. 25. WS wrote parts of Sects. 15. EW wrote parts of Sects. 14. TW wrote parts of Sects. 25.

Compliance with Ethical Standards

No specific funding was received to support the preparation of this manuscript. CS is an employee of the Office of Health Economics, a registered charity, research organisation and consultancy, which receives funding from a variety of sources including the Association of the British Pharmaceutical Industry and the National Institute for Health Research (NIHR). CS has received honoraria from the NIHR in relation to peer review activities. CS has received travel support from Duke University for attendance at a meeting related to the content of this manuscript. SB receives salary support from the University of British Columbia, Vancouver Coastal Health and the British Columbia Academic Health Science Network. He previously chaired CADTH’s Health Technology Expert Review Panel and provides consultancy advice to CADTH, for which he received honoraria and travel support. All of his decision model project work is supported through public sources, primarily the Canadian Institutes for Health Research (CIHR) and the BC Ministry of Health. PC has received payment for workshops run alongside Mt Hood conferences. SE is CEO and Owner of Collaborations Pharmaceuticals, Inc. SE has received a grant from the National Institutes for Health to build machine learning models and software. DM is employed by the University of Calgary, with a salary funded by Arthur J.E. Child Chair in Rheumatology and Canada Research Chair in Health Systems and Services Research. DM’s research is funded through multiple funding organisations including the CIHR, Arthritis Society, CRA and AAC in peer-reviewed funding competitions, none of which relate to this work. DM has received reimbursement for travel from Illumina and Janssen for attendance and presentation at scientific meetings not related to this work. RA, AH, NH, SL, MS, WS, and EW declare that they have no conflicts of interest. TW is a Co-Editor of PharmacoEconomics, but was not involved in the peer review nor any editorial decisions for this paper.


SE acknowledges funding from NIH/ NIGMS R43GM122196, R44GM122196-02A1.


  1. 1.
    Daniels N, Sabin J. Limits to health care: fair procedures, democratic deliberation, and the legitimacy problem for insurers. Philosophy Public Affairs. 1997;26:303–50.CrossRefPubMedGoogle Scholar
  2. 2.
    Daniels N, Sabin JE. Accountability for reasonableness. Setting limits fairly: can we learn to share medical resources?. Oxford: Oxford University Press; 2002.CrossRefGoogle Scholar
  3. 3.
    Eddy DM, Hollingworth W, Caro JJ, Tsevat J, McDonald KM, Wong JB. Model transparency and validation: a report of the ISPOR-SMDM modeling good research practices task force–7. Med Decis Mak. 2012;32:733–43.CrossRefGoogle Scholar
  4. 4.
    Philips Z, Ginnelly L, Sculpher M, Claxton K, Golder S, Riemsma R, et al. Review of guidelines for good practice in decision-analytic modelling in health technology assessment. Health Technol Assess. 2004;iii:iv (ix–xi, 1–158).Google Scholar
  5. 5.
    Sampson CJ, Wrightson T. Model registration: a call to action. PharmacoEcon Open. 2017;1:73–7.CrossRefPubMedPubMedCentralGoogle Scholar
  6. 6.
    Freemantle N, Mason J. Publication bias in clinical trials and economic analyses. PharmacoEconomics. 1997;12:10–6.CrossRefPubMedGoogle Scholar
  7. 7.
    Sacristán JA, Bolaños E, Hernández JM, Soto J, Galende I. Publication bias in health economic studies. Pharmacoeconomics. 1997;11:289–92.CrossRefPubMedGoogle Scholar
  8. 8.
    Mt Hood Database [Internet]. Mt hood diabetes challenge network. [cited 2019 Feb 18].
  9. 9.
    CISNET Model Registry Home [Internet]. [cited 2019 Feb 18].
  10. 10.
    Global Health CEA—Open-Source Model Clearinghouse [Internet]. [cited 2019 Feb 18].
  11. 11.
    Husereau D, Drummond M, Petrou S, Carswell C, Moher D, Greenberg D, et al. Consolidated health economic evaluation reporting standards (CHEERS) statement. PharmacoEconomics. 2013;31:361–7.CrossRefPubMedGoogle Scholar
  12. 12.
    Caro JJ, Briggs AH, Siebert U, Kuntz KM. Modeling good research practices—overview: a report of the ISPOR-SMDM modeling good research practices task force–1. Med Decis Mak. 2012;32:667–77.CrossRefGoogle Scholar
  13. 13.
    Sanghera S, Frew E, Roberts T. Adapting the CHEERS statement for reporting cost-benefit analysis. PharmacoEconomics. 2015;33:533–4.CrossRefPubMedGoogle Scholar
  14. 14.
    Husereau D, Drummond M, Petrou S, Carswell C, Moher D, Greenberg D, et al. Consolidated health economic evaluation reporting standards (CHEERS)—explanation and elaboration: a report of the ISPOR health economic evaluation publication guidelines good reporting practices task force. Value in Health. 2013;16:231–50.CrossRefPubMedGoogle Scholar
  15. 15.
    Haji Ali Afzali H, Karnon J. Addressing the challenge for well informed and consistent reimbursement decisions: the case for reference models. PharmacoEconomics. 2011;29:823–5.CrossRefPubMedGoogle Scholar
  16. 16.
    Afzali HHA, Karnon J, Merlin T. Improving the accuracy and comparability of model-based economic evaluations of health technologies for reimbursement decisions: a methodological framework for the development of reference models. Med Decis Mak. 2013;33:325–32.CrossRefGoogle Scholar
  17. 17.
    Frederix GWJ, Haji Ali Afzali H, Dasbach EJ, Ward RL. Development and use of disease-specific (reference) models for economic evaluations of health technologies: an overview of key issues and potential solutions. PharmacoEconomics. 2015;33:777–81.CrossRefPubMedGoogle Scholar
  18. 18.
    Lewsey JD, Lawson KD, Ford I, Fox K a. A, Ritchie LD, Tunstall-Pedoe H, et al. A cardiovascular disease policy model that predicts life expectancy taking into account socioeconomic deprivation. Heart. 2015;101:201–8.Google Scholar
  19. 19.
    Rydzak CE, Cotich KL, Sax PE, Hsu HE, Wang B, Losina E, et al. Assessing the performance of a computer-based policy model of HIV and AIDS. PLOS One. 2010;5:e12647.CrossRefPubMedPubMedCentralGoogle Scholar
  20. 20.
    Cancer Intervention and Surveillance Modeling Network (CISNET) [Internet]. [cited 2019 Feb 18].
  21. 21.
    Hennessy DA, Flanagan WM, Tanuseputro P, Bennett C, Tuna M, Kopec J, et al. The Population Health Model (POHEM): an overview of rationale, methods and applications. Popul Health Metr. 2015;13:24.CrossRefPubMedPubMedCentralGoogle Scholar
  22. 22.
    Sadatsafavi M, Ghanbarian S, Adibi A, Johnson K, FitzGerald JM, Flanagan W, et al. Development and validation of the evaluation platform in COPD (EPIC): a population-based outcomes model of COPD for Canada. Med Decis Mak. 2019;39:152–67.CrossRefGoogle Scholar
  23. 23.
    Sullivan W, Hirst M, Beard S, Gladwell D, Fagnani F, López Bastida J, et al. Economic evaluation in chronic pain: a systematic review and de novo flexible economic model. Eur J Health Econ. 2016;17:755–70.CrossRefPubMedGoogle Scholar
  24. 24.
    Incerti D, Curtis JR, Shafrin J, Lakdawalla DN, Jansen JP. A flexible open-source decision model for value assessment of biologic treatment for rheumatoid arthritis. Pharmacoeconomics. 2019;37(6):829–43.CrossRefPubMedGoogle Scholar
  25. 25.
    Global Health CEA—About the Clearinghouse [Internet]. [cited 2019 Feb 14].
  26. 26.
    Open-Source Value Platform [Internet]. Innovation and value initiative. [cited 2019 Feb 14].
  27. 27.
    Priem J, Hemminger BM. Decoupling the scholarly journal. Front Comput Neurosci. 2012;6:19.CrossRefPubMedPubMedCentralGoogle Scholar
  28. 28.
    Arnold RJG, Ekins S. Time for cooperation in health economics among the modelling community. PharmacoEconomics. 2010;28:609–13.CrossRefPubMedGoogle Scholar
  29. 29.
    American Diabetes Association Consensus Panel. Guidelines for computer modeling of diabetes and its complications. Diabetes Care. 2004;27:2262–5.CrossRefGoogle Scholar
  30. 30.
    Hayes AJ, Leal J, Gray AM, Holman RR, Clarke PM. UKPDS outcomes model 2: a new version of a model to simulate lifetime health outcomes of patients with type 2 diabetes mellitus using data from the 30 year United Kingdom Prospective Diabetes Study: UKPDS 82. Diabetologia. 2013;56:1925–33.CrossRefPubMedGoogle Scholar
  31. 31.
    Palmer AJ, Si L, Tew M, Hua X, Willis MS, Asseburg C, et al. Computer modeling of diabetes and its transparency: a report on the eighth mount hood challenge. Value in Health. 2018;21:724–31.CrossRefPubMedPubMedCentralGoogle Scholar
  32. 32.
    Project HERCULES [Internet]. Duchenne UK. [cited 2019 Feb 14].
  33. 33.
    Hoogendoorn M, Feenstra TL, Asukai Y, Briggs AH, Hansen RN, Leidl R, et al. External validation of health economic decision models for chronic obstructive pulmonary disease (COPD): report of the third COPD modeling meeting. Value Health. 2017;20:397–403.CrossRefPubMedGoogle Scholar
  34. 34.
    Mount Hood 4 Modeling Group. Computer modeling of diabetes and its complications: a report on the fourth mount hood challenge meeting. Diabetes Care. 2007;30:1638–46.CrossRefGoogle Scholar
  35. 35.
    Palmer AJ. Computer modeling of diabetes and its complications: a report on the fifth mount hood challenge meeting. Value in Health. 2013;16:670–85.CrossRefPubMedGoogle Scholar
  36. 36.
    About Cochrane | Cochrane Library [Internet]. [cited 2019 Feb 14].
  37. 37.
    Goeree R, Levin L. Building bridges between academic research and policy formulation: the PRUFE framework—an integral part of ontario’s evidence-based HTPA process. PharmacoEconomics. 2006;24:1143–56.CrossRefPubMedGoogle Scholar
  38. 38.
    Drummond M, Sorenson C. Use of pharmacoeconomics in drug reimbursement in Australia, Canada and the UK: what can we learn from international experience? In: Arnold RJG, editor. Pharmacoeconomics: from theory to practice. Boca Raton: CRC Press; 2009. p. 175–96.Google Scholar
  39. 39.
    Cooper NJ, Sutton AJ, Ades AE, Paisley S, Jones DR, on behalf of the working group on the ‘use of evidence in economic decision models’. Use of evidence in economic decision models: practical issues and methodological challenges. Health Economics. 2007;16:1277–86.CrossRefPubMedGoogle Scholar
  40. 40.
    Barton P. Development of the Birmingham rheumatoid arthritis model: past, present and future plans. Rheumatology (Oxford). 2011;50:iv32–8.CrossRefPubMedGoogle Scholar
  41. 41.
    Localio AR, Goodman SN, Meibohm A, Cornell JE, Stack CB, Ross EA, et al. Statistical code to support the scientific story. Ann Internal Med. 2018;168:828.CrossRefGoogle Scholar
  42. 42.
    Christensen G, Miguel E. Transparency, reproducibility, and the credibility of economics research. J Econ Lit. 2018;56:920–80.CrossRefGoogle Scholar
  43. 43.
    Merlo G, Page K, Ratcliffe J, Halton K, Graves N. Bridging the gap: exploring the barriers to using economic evidence in healthcare decision making and strategies for improving uptake. Appl Health Econ Health Policy. 2015;13:303–9.CrossRefPubMedGoogle Scholar
  44. 44.
    Vemer P, Corro Ramos I, van Voorn GAK, Al MJ, Feenstra TL. AdViSHE: a validation-assessment tool of health-economic models for decision makers and model users. PharmacoEconomics. 2016;34:349–61.CrossRefPubMedGoogle Scholar
  45. 45.
    Hua X, Lung TW-C, Palmer A, Si L, Herman WH, Clarke P. How consistent is the relationship between improved glucose control and modelled health outcomes for people with type 2 diabetes mellitus? A systematic review. PharmacoEconomics. 2017;35:319–29.CrossRefPubMedGoogle Scholar
  46. 46.
    Goodacre S. Being economical with the truth: how to make your idea appear cost effective. Emerg Med J. 2002;19:301–4.CrossRefPubMedPubMedCentralGoogle Scholar
  47. 47.
    Dunlop WCN, Mason N, Kenworthy J, Akehurst RL. Benefits, challenges and potential strategies of open source health economic models. PharmacoEconomics. 2017;35:125–8.CrossRefPubMedGoogle Scholar
  48. 48.
    Laine C, Goodman SN, Griswold ME, Sox HC. Reproducible research: moving toward research the public can really trust. Ann Internal Med. 2007;146:450.CrossRefGoogle Scholar
  49. 49.
    Lampe K, Mäkelä M, Garrido MV, Anttila H, Autti-Rämö I, Hicks NJ, et al. The HTA core model: a novel method for producing and reporting health technology assessments. Int J Technol Assess Health Care. 2009;25:9–20.CrossRefPubMedGoogle Scholar
  50. 50.
    EUnetHTA. HTA Core ModelVersion 3.0 [Internet]. 2016 Jan. Report No.: JA2 WP8.  [cited 2019 Feb 14].
  51. 51.
    Academy of Managed Care Pharmacy. The AMCP Format for Formulary Submissions [Internet]. 2012. Report No.: Version 3.1.  [cited 2019 Feb 14].
  52. 52.
    Hatswell AJ, Chandler F. Sharing is caring: the case for company-level collaboration in pharmacoeconomic modelling. PharmacoEconomics. 2017;35:755–7.CrossRefPubMedGoogle Scholar
  53. 53.
    Cole A, Towse A. Legal Barriers to the Better Use of Health Data to Deliver Pharmaceutical Innovation. OHE Consulting Report, London: Office of Health Economics; 2018.
  54. 54.
    R Core Team. R: a language and environment for statistical computing [Internet]. Vienna, Austria: R Foundation for Statistical Computing; 2018 [cited 2019 Feb 14].
  55. 55.
    Python Software Foundation. Python Language Reference [Internet]. 2008. [cited 2019 Feb 14]
  56. 56.
    Frederix GWJ. Check your checklist: the danger of over- and underestimating the quality of economic evaluations. PharmacoEconomics Open. 2019;. Scholar
  57. 57.
    Jalal H, Pechlivanoglou P, Krijkamp E, Alarid-Escudero F, Enns E, Hunink MGM. An overview of R in health decision sciences. Med Decis Mak. 2017;37:735–46.CrossRefGoogle Scholar
  58. 58.
    Chang W, Cheng J, Allaire JJ, Xie Y, McPherson J. Shiny: web application framework for R [Internet]. 2018 [cited 2019 Feb 14].
  59. 59.
    Dabbish L, Stuart C, Tsay J, Herbsleb J. Social coding in github: transparency and collaboration in an open software repository. In: Proceedings of the ACM 2012 Conference on Computer Supported Cooperative Work [Internet]. New York, NY, USA: ACM; 2012 [cited 2019 Feb 19]. p. 1277–1286.
  60. 60.
    Bryan S, Mitton C, Donaldson C. Breaking the addiction to technology adoption. Health Econ. 2014;23:379–83.CrossRefPubMedGoogle Scholar
  61. 61.
    Scotland G, Bryan S. Why do health economists promote technology adoption rather than the search for efficiency? A proposal for a change in our approach to economic evaluation in health care. Med Decis Mak. 2017;37:139–47.CrossRefGoogle Scholar
  62. 62.
    Bertagnolli MM, Sartor O, Chabner BA, Rothenberg ML, Khozin S, Hugh-Jones C, et al. Advantages of a truly open-access data-sharing model. N Engl J Med. 2017;376:1178–81.CrossRefPubMedGoogle Scholar
  63. 63.
    Sotos JG, Huyen Y, Borrelli A. Correspondence: data-sharing models. N Engl J Med. 2017;376:2305–6.CrossRefPubMedGoogle Scholar
  64. 64.
    Ekins S, Arnold RJG. From machine learning in drug discovery to pharmacoeconomics. In: Arnold RJG, editor. Pharmacoeconomics: from theory to practice. 2nd ed. Boca Raton: CRC Press; 2019.Google Scholar
  65. 65.
    Arnold RJG, Ekins S. Ahead of our time: collaboration in modeling then and now. PharmacoEconomics. 2017;35:975–6.CrossRefPubMedGoogle Scholar
  66. 66.
    Government of Canada CI of HR. Strategy for patient-oriented research [Internet]. 2018 [cited 2019 Feb 18].
  67. 67.
    PCORI [Internet]. [cited 2019 Feb 18].
  68. 68.
    INVOLVE | INVOLVE Supporting public involvement in NHS, public health and social care research [Internet]. [cited 2019 Feb 18].
  69. 69.
    van Voorn GAK, Vemer P, Hamerlijnck D, Ramos IC, Teunissen GJ, Al M, et al. The missing stakeholder group: why patients should be involved in health economic modelling. Appl Health Econ Health Policy. 2016;14:129–33.CrossRefPubMedGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.Office of Health EconomicsLondonUK
  2. 2.Arnold Consultancy & Technology, LLCNew YorkUSA
  3. 3.University of British ColumbiaVancouverCanada
  4. 4.University of OxfordOxfordUK
  5. 5.Collaborations Pharmaceuticals Inc.RaleighUSA
  6. 6.Delta HatNottinghamUK
  7. 7.University of GlasgowGlasgowUK
  8. 8.Maverex LimitedSaleUK
  9. 9.University of CalgaryCalgaryCanada
  10. 10.University of British ColumbiaVancouverCanada
  11. 11.BresMed Health SolutionsSheffieldUK
  12. 12.Health Economics Group, Norwich Medical SchoolUniversity of East AngliaNorwichUK
  13. 13.Adis International LimitedAucklandNew Zealand

Personalised recommendations