Skip to main content

Transparency in Decision Modelling: What, Why, Who and How?

PharmacoEconomics volume 37, pages 1355–1369 (2019)Cite this article

Abstract

Transparency in decision modelling is an evolving concept. Recently, discussion has moved from reporting standards to open-source implementation of decision analytic models. However, in the debate about the supposed advantages and disadvantages of greater transparency, there is a lack of definition. The purpose of this article is not to present a case for or against transparency, but rather to provide a more nuanced understanding of what transparency means in the context of decision modelling and how it could be addressed. To this end, we review and summarise the discourse to date, drawing on our collective experience. We outline a taxonomy of the different manifestations of transparency, including reporting standards, reference models, collaboration, model registration, peer review and open-source modelling. Further, we map out the role and incentives for the various stakeholders, including industry, research organisations, publishers and decision makers. We outline the anticipated advantages and disadvantages of greater transparency with respect to each manifestation, as well as the perceived barriers and facilitators to greater transparency. These are considered with respect to the different stakeholders and with reference to issues including intellectual property, legality, standards, quality assurance, code integrity, health technology assessment processes, incentives, funding, software, access and deployment options, data protection and stakeholder engagement. For each manifestation of transparency, we discuss the ‘what’, ‘why’, ‘who’ and ‘how’. Specifically, their meaning, why the community might (or might not) wish to embrace them, whose engagement as stakeholders is required and how relevant objectives might be realised. We identify current initiatives aimed to improve transparency to exemplify efforts in current practice and for the future.

This is a preview of subscription content, access via your institution.

Access options

Buy single article

Instant access to the full article PDF.

49,54 €

Price includes VAT (Finland)
Tax calculation will be finalised during checkout.

Fig. 1
Fig. 2
Fig. 3

References

  1. Daniels N, Sabin J. Limits to health care: fair procedures, democratic deliberation, and the legitimacy problem for insurers. Philosophy Public Affairs. 1997;26:303–50.

    PubMed  Google Scholar 

  2. Daniels N, Sabin JE. Accountability for reasonableness. Setting limits fairly: can we learn to share medical resources?. Oxford: Oxford University Press; 2002.

    Google Scholar 

  3. Eddy DM, Hollingworth W, Caro JJ, Tsevat J, McDonald KM, Wong JB. Model transparency and validation: a report of the ISPOR-SMDM modeling good research practices task force–7. Med Decis Mak. 2012;32:733–43.

    Article  Google Scholar 

  4. Philips Z, Ginnelly L, Sculpher M, Claxton K, Golder S, Riemsma R, et al. Review of guidelines for good practice in decision-analytic modelling in health technology assessment. Health Technol Assess. 2004;iii:iv (ix–xi, 1–158).

    Google Scholar 

  5. Sampson CJ, Wrightson T. Model registration: a call to action. PharmacoEcon Open. 2017;1:73–7.

    PubMed  PubMed Central  Google Scholar 

  6. Freemantle N, Mason J. Publication bias in clinical trials and economic analyses. PharmacoEconomics. 1997;12:10–6.

    CAS  PubMed  Google Scholar 

  7. Sacristán JA, Bolaños E, Hernández JM, Soto J, Galende I. Publication bias in health economic studies. Pharmacoeconomics. 1997;11:289–92.

    PubMed  Google Scholar 

  8. Mt Hood Database [Internet]. Mt hood diabetes challenge network. [cited 2019 Feb 18]. https://www.mthooddiabeteschallenge.com/registry.

  9. CISNET Model Registry Home [Internet]. [cited 2019 Feb 18]. https://resources.cisnet.cancer.gov/registry.

  10. Global Health CEA—Open-Source Model Clearinghouse [Internet]. [cited 2019 Feb 18]. http://healtheconomics.tuftsmedicalcenter.org/orchard/open-source-model-clearinghouse.

  11. Husereau D, Drummond M, Petrou S, Carswell C, Moher D, Greenberg D, et al. Consolidated health economic evaluation reporting standards (CHEERS) statement. PharmacoEconomics. 2013;31:361–7.

    PubMed  Google Scholar 

  12. Caro JJ, Briggs AH, Siebert U, Kuntz KM. Modeling good research practices—overview: a report of the ISPOR-SMDM modeling good research practices task force–1. Med Decis Mak. 2012;32:667–77.

    Google Scholar 

  13. Sanghera S, Frew E, Roberts T. Adapting the CHEERS statement for reporting cost-benefit analysis. PharmacoEconomics. 2015;33:533–4.

    PubMed  Google Scholar 

  14. Husereau D, Drummond M, Petrou S, Carswell C, Moher D, Greenberg D, et al. Consolidated health economic evaluation reporting standards (CHEERS)—explanation and elaboration: a report of the ISPOR health economic evaluation publication guidelines good reporting practices task force. Value in Health. 2013;16:231–50.

    PubMed  Google Scholar 

  15. Haji Ali Afzali H, Karnon J. Addressing the challenge for well informed and consistent reimbursement decisions: the case for reference models. PharmacoEconomics. 2011;29:823–5.

    PubMed  Google Scholar 

  16. Afzali HHA, Karnon J, Merlin T. Improving the accuracy and comparability of model-based economic evaluations of health technologies for reimbursement decisions: a methodological framework for the development of reference models. Med Decis Mak. 2013;33:325–32.

    Google Scholar 

  17. Frederix GWJ, Haji Ali Afzali H, Dasbach EJ, Ward RL. Development and use of disease-specific (reference) models for economic evaluations of health technologies: an overview of key issues and potential solutions. PharmacoEconomics. 2015;33:777–81.

    PubMed  Google Scholar 

  18. Lewsey JD, Lawson KD, Ford I, Fox K a. A, Ritchie LD, Tunstall-Pedoe H, et al. A cardiovascular disease policy model that predicts life expectancy taking into account socioeconomic deprivation. Heart. 2015;101:201–8.

    PubMed  PubMed Central  Google Scholar 

  19. Rydzak CE, Cotich KL, Sax PE, Hsu HE, Wang B, Losina E, et al. Assessing the performance of a computer-based policy model of HIV and AIDS. PLOS One. 2010;5:e12647.

    PubMed  PubMed Central  Google Scholar 

  20. Cancer Intervention and Surveillance Modeling Network (CISNET) [Internet]. [cited 2019 Feb 18]. https://cisnet.cancer.gov/.

  21. Hennessy DA, Flanagan WM, Tanuseputro P, Bennett C, Tuna M, Kopec J, et al. The Population Health Model (POHEM): an overview of rationale, methods and applications. Popul Health Metr. 2015;13:24.

    PubMed  PubMed Central  Google Scholar 

  22. Sadatsafavi M, Ghanbarian S, Adibi A, Johnson K, FitzGerald JM, Flanagan W, et al. Development and validation of the evaluation platform in COPD (EPIC): a population-based outcomes model of COPD for Canada. Med Decis Mak. 2019;39:152–67.

    Google Scholar 

  23. Sullivan W, Hirst M, Beard S, Gladwell D, Fagnani F, López Bastida J, et al. Economic evaluation in chronic pain: a systematic review and de novo flexible economic model. Eur J Health Econ. 2016;17:755–70.

    CAS  PubMed  Google Scholar 

  24. Incerti D, Curtis JR, Shafrin J, Lakdawalla DN, Jansen JP. A flexible open-source decision model for value assessment of biologic treatment for rheumatoid arthritis. Pharmacoeconomics. 2019;37(6):829–43.

    PubMed  Google Scholar 

  25. Global Health CEA—About the Clearinghouse [Internet]. [cited 2019 Feb 14]. http://healtheconomics.tuftsmedicalcenter.org/orchard/about-the-clearinghouse.

  26. Open-Source Value Platform [Internet]. Innovation and value initiative. [cited 2019 Feb 14]. https://www.thevalueinitiative.org/open-source-value-project/.

  27. Priem J, Hemminger BM. Decoupling the scholarly journal. Front Comput Neurosci. 2012;6:19.

    PubMed  PubMed Central  Google Scholar 

  28. Arnold RJG, Ekins S. Time for cooperation in health economics among the modelling community. PharmacoEconomics. 2010;28:609–13.

    PubMed  Google Scholar 

  29. American Diabetes Association Consensus Panel. Guidelines for computer modeling of diabetes and its complications. Diabetes Care. 2004;27:2262–5.

    Google Scholar 

  30. Hayes AJ, Leal J, Gray AM, Holman RR, Clarke PM. UKPDS outcomes model 2: a new version of a model to simulate lifetime health outcomes of patients with type 2 diabetes mellitus using data from the 30 year United Kingdom Prospective Diabetes Study: UKPDS 82. Diabetologia. 2013;56:1925–33.

    CAS  PubMed  Google Scholar 

  31. Palmer AJ, Si L, Tew M, Hua X, Willis MS, Asseburg C, et al. Computer modeling of diabetes and its transparency: a report on the eighth mount hood challenge. Value in Health. 2018;21:724–31.

    PubMed  PubMed Central  Google Scholar 

  32. Project HERCULES [Internet]. Duchenne UK. [cited 2019 Feb 14]. https://www.duchenneuk.org/project-hercules.

  33. Hoogendoorn M, Feenstra TL, Asukai Y, Briggs AH, Hansen RN, Leidl R, et al. External validation of health economic decision models for chronic obstructive pulmonary disease (COPD): report of the third COPD modeling meeting. Value Health. 2017;20:397–403.

    PubMed  Google Scholar 

  34. Mount Hood 4 Modeling Group. Computer modeling of diabetes and its complications: a report on the fourth mount hood challenge meeting. Diabetes Care. 2007;30:1638–46.

    Google Scholar 

  35. Palmer AJ. Computer modeling of diabetes and its complications: a report on the fifth mount hood challenge meeting. Value in Health. 2013;16:670–85.

    PubMed  Google Scholar 

  36. About Cochrane | Cochrane Library [Internet]. [cited 2019 Feb 14]. https://www.cochranelibrary.com/about/about-cochrane.

  37. Goeree R, Levin L. Building bridges between academic research and policy formulation: the PRUFE framework—an integral part of ontario’s evidence-based HTPA process. PharmacoEconomics. 2006;24:1143–56.

    PubMed  Google Scholar 

  38. Drummond M, Sorenson C. Use of pharmacoeconomics in drug reimbursement in Australia, Canada and the UK: what can we learn from international experience? In: Arnold RJG, editor. Pharmacoeconomics: from theory to practice. Boca Raton: CRC Press; 2009. p. 175–96.

    Google Scholar 

  39. Cooper NJ, Sutton AJ, Ades AE, Paisley S, Jones DR, on behalf of the working group on the ‘use of evidence in economic decision models’. Use of evidence in economic decision models: practical issues and methodological challenges. Health Economics. 2007;16:1277–86.

    CAS  PubMed  Google Scholar 

  40. Barton P. Development of the Birmingham rheumatoid arthritis model: past, present and future plans. Rheumatology (Oxford). 2011;50:iv32–8.

    PubMed  Google Scholar 

  41. Localio AR, Goodman SN, Meibohm A, Cornell JE, Stack CB, Ross EA, et al. Statistical code to support the scientific story. Ann Internal Med. 2018;168:828.

    Google Scholar 

  42. Christensen G, Miguel E. Transparency, reproducibility, and the credibility of economics research. J Econ Lit. 2018;56:920–80.

    Google Scholar 

  43. Merlo G, Page K, Ratcliffe J, Halton K, Graves N. Bridging the gap: exploring the barriers to using economic evidence in healthcare decision making and strategies for improving uptake. Appl Health Econ Health Policy. 2015;13:303–9.

    PubMed  Google Scholar 

  44. Vemer P, Corro Ramos I, van Voorn GAK, Al MJ, Feenstra TL. AdViSHE: a validation-assessment tool of health-economic models for decision makers and model users. PharmacoEconomics. 2016;34:349–61.

    CAS  PubMed  Google Scholar 

  45. Hua X, Lung TW-C, Palmer A, Si L, Herman WH, Clarke P. How consistent is the relationship between improved glucose control and modelled health outcomes for people with type 2 diabetes mellitus? A systematic review. PharmacoEconomics. 2017;35:319–29.

    PubMed  Google Scholar 

  46. Goodacre S. Being economical with the truth: how to make your idea appear cost effective. Emerg Med J. 2002;19:301–4.

    CAS  PubMed  PubMed Central  Google Scholar 

  47. Dunlop WCN, Mason N, Kenworthy J, Akehurst RL. Benefits, challenges and potential strategies of open source health economic models. PharmacoEconomics. 2017;35:125–8.

    PubMed  Google Scholar 

  48. Laine C, Goodman SN, Griswold ME, Sox HC. Reproducible research: moving toward research the public can really trust. Ann Internal Med. 2007;146:450.

    Google Scholar 

  49. Lampe K, Mäkelä M, Garrido MV, Anttila H, Autti-Rämö I, Hicks NJ, et al. The HTA core model: a novel method for producing and reporting health technology assessments. Int J Technol Assess Health Care. 2009;25:9–20.

    PubMed  Google Scholar 

  50. EUnetHTA. HTA Core ModelVersion 3.0 [Internet]. 2016 Jan. Report No.: JA2 WP8.  [cited 2019 Feb 14]. https://www.eunethta.eu/wp-content/uploads/2018/03/HTACoreModel3.0-1.pdf.

  51. Academy of Managed Care Pharmacy. The AMCP Format for Formulary Submissions [Internet]. 2012. Report No.: Version 3.1.  [cited 2019 Feb 14]. http://amcp.org/practice-resources/amcp-format-formulary-submisions.pdf.

  52. Hatswell AJ, Chandler F. Sharing is caring: the case for company-level collaboration in pharmacoeconomic modelling. PharmacoEconomics. 2017;35:755–7.

    PubMed  Google Scholar 

  53. Cole A, Towse A. Legal Barriers to the Better Use of Health Data to Deliver Pharmaceutical Innovation. OHE Consulting Report, London: Office of Health Economics; 2018. https://ideas.repec.org/p/ohe/conrep/002096.html.

  54. R Core Team. R: a language and environment for statistical computing [Internet]. Vienna, Austria: R Foundation for Statistical Computing; 2018 [cited 2019 Feb 14]. https://www.R-project.org/.

  55. Python Software Foundation. Python Language Reference [Internet]. 2008. [cited 2019 Feb 14] http://www.python.org.

  56. Frederix GWJ. Check your checklist: the danger of over- and underestimating the quality of economic evaluations. PharmacoEconomics Open. 2019;. https://doi.org/10.1007/s41669-019-0118-3.

    Article  PubMed  Google Scholar 

  57. Jalal H, Pechlivanoglou P, Krijkamp E, Alarid-Escudero F, Enns E, Hunink MGM. An overview of R in health decision sciences. Med Decis Mak. 2017;37:735–46.

    Google Scholar 

  58. Chang W, Cheng J, Allaire JJ, Xie Y, McPherson J. Shiny: web application framework for R [Internet]. 2018 [cited 2019 Feb 14]. https://CRAN.R-project.org/package=shiny.

  59. Dabbish L, Stuart C, Tsay J, Herbsleb J. Social coding in github: transparency and collaboration in an open software repository. In: Proceedings of the ACM 2012 Conference on Computer Supported Cooperative Work [Internet]. New York, NY, USA: ACM; 2012 [cited 2019 Feb 19]. p. 1277–1286. http://doi.acm.org/10.1145/2145204.2145396.

  60. Bryan S, Mitton C, Donaldson C. Breaking the addiction to technology adoption. Health Econ. 2014;23:379–83.

    PubMed  Google Scholar 

  61. Scotland G, Bryan S. Why do health economists promote technology adoption rather than the search for efficiency? A proposal for a change in our approach to economic evaluation in health care. Med Decis Mak. 2017;37:139–47.

    Google Scholar 

  62. Bertagnolli MM, Sartor O, Chabner BA, Rothenberg ML, Khozin S, Hugh-Jones C, et al. Advantages of a truly open-access data-sharing model. N Engl J Med. 2017;376:1178–81.

    PubMed  Google Scholar 

  63. Sotos JG, Huyen Y, Borrelli A. Correspondence: data-sharing models. N Engl J Med. 2017;376:2305–6.

    PubMed  Google Scholar 

  64. Ekins S, Arnold RJG. From machine learning in drug discovery to pharmacoeconomics. In: Arnold RJG, editor. Pharmacoeconomics: from theory to practice. 2nd ed. Boca Raton: CRC Press; 2019.

    Google Scholar 

  65. Arnold RJG, Ekins S. Ahead of our time: collaboration in modeling then and now. PharmacoEconomics. 2017;35:975–6.

    PubMed  Google Scholar 

  66. Government of Canada CI of HR. Strategy for patient-oriented research [Internet]. 2018 [cited 2019 Feb 18]. http://www.cihr-irsc.gc.ca/e/41204.html.

  67. PCORI [Internet]. [cited 2019 Feb 18]. https://www.pcori.org/.

  68. INVOLVE | INVOLVE Supporting public involvement in NHS, public health and social care research [Internet]. [cited 2019 Feb 18]. https://www.invo.org.uk/.

  69. van Voorn GAK, Vemer P, Hamerlijnck D, Ramos IC, Teunissen GJ, Al M, et al. The missing stakeholder group: why patients should be involved in health economic modelling. Appl Health Econ Health Policy. 2016;14:129–33.

    PubMed  Google Scholar 

Download references

Acknowledgements

SE kindly acknowledges his colleagues at Collaborations Pharmaceuticals.

Author information

Affiliations

  1. Office of Health Economics, 7th Floor, Southside, 105 Victoria Street, London, SW1E 6QT, UK

    Christopher James Sampson

  2. Arnold Consultancy & Technology, LLC, 15 West 72nd Street-23rd Floor, New York, NY, 10023-3458, USA

    Renée Arnold

  3. University of British Columbia, 701-828 West 10th Avenue, Research Pavilion, Vancouver, BC, V5Z 1M9, Canada

    Stirling Bryan

  4. University of Oxford, Richard Doll Building, Old Road Campus, Oxford, OX3 7LF, UK

    Philip Clarke

  5. Collaborations Pharmaceuticals Inc., 840 Main Campus Drive, Lab 3510, Raleigh, NC, 27606, USA

    Sean Ekins

  6. Delta Hat, 212 Tamworth Road, Nottingham, NG10 3GS, UK

    Anthony Hatswell

  7. University of Glasgow, Lilybank Gardens 1, Glasgow, G12 8RZ, UK

    Neil Hawkins

  8. Maverex Limited, 5 Brooklands Place, Brooklands Road, Sale, Cheshire, M33 3SD, UK

    Sue Langham

  9. University of Calgary, 3280 Hospital Drive NW, Calgary, AB, T2N 4Z6, Canada

    Deborah Marshall

  10. University of British Columbia, 2405 Wesbrook Mall, Vancouver, BC, V6T1Z3, Canada

    Mohsen Sadatsafavi

  11. BresMed Health Solutions, Steel City House, West Street, Sheffield, S1 2GQ, UK

    Will Sullivan

  12. Health Economics Group, Norwich Medical School, University of East Anglia, Norwich, NR4 7TJ, UK

    Edward C. F. Wilson

  13. Adis International Limited, 5 The Warehouse Way, Northcote, 0627, Auckland, New Zealand

    Tim Wrightson

Authors
  1. Christopher James Sampson
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Renée Arnold
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Stirling Bryan
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Philip Clarke
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Sean Ekins
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Anthony Hatswell
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Neil Hawkins
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Sue Langham
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Deborah Marshall
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Mohsen Sadatsafavi
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. Will Sullivan
    View author publications

    You can also search for this author in PubMed Google Scholar

  12. Edward C. F. Wilson
    View author publications

    You can also search for this author in PubMed Google Scholar

  13. Tim Wrightson
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

All authors were involved in the conception and planning of the manuscript, wrote sections, edited and commented on drafts and approved the submitted version. CS organised and led the write-up and contributed to the writing of all sections. RA wrote parts of Sects. 1–5. SB wrote parts of Sects. 4 and 5. PC wrote parts of Sects. 2. SE wrote parts of Sects. 2, 4 and 5. AH wrote parts of Sects. 2–5. NH wrote parts of Sects. 1, 3 and 4. SL wrote parts of Sects. 2–5. DM wrote parts of Sects. 1, 3 and 5. MS wrote parts of Sects. 2–5. WS wrote parts of Sects. 1–5. EW wrote parts of Sects. 1–4. TW wrote parts of Sects. 2–5.

Corresponding author

Correspondence to Christopher James Sampson.

Ethics declarations

No specific funding was received to support the preparation of this manuscript. CS is an employee of the Office of Health Economics, a registered charity, research organisation and consultancy, which receives funding from a variety of sources including the Association of the British Pharmaceutical Industry and the National Institute for Health Research (NIHR). CS has received honoraria from the NIHR in relation to peer review activities. CS has received travel support from Duke University for attendance at a meeting related to the content of this manuscript. SB receives salary support from the University of British Columbia, Vancouver Coastal Health and the British Columbia Academic Health Science Network. He previously chaired CADTH’s Health Technology Expert Review Panel and provides consultancy advice to CADTH, for which he received honoraria and travel support. All of his decision model project work is supported through public sources, primarily the Canadian Institutes for Health Research (CIHR) and the BC Ministry of Health. PC has received payment for workshops run alongside Mt Hood conferences. SE is CEO and Owner of Collaborations Pharmaceuticals, Inc. SE has received a grant from the National Institutes for Health to build machine learning models and software. DM is employed by the University of Calgary, with a salary funded by Arthur J.E. Child Chair in Rheumatology and Canada Research Chair in Health Systems and Services Research. DM’s research is funded through multiple funding organisations including the CIHR, Arthritis Society, CRA and AAC in peer-reviewed funding competitions, none of which relate to this work. DM has received reimbursement for travel from Illumina and Janssen for attendance and presentation at scientific meetings not related to this work. RA, AH, NH, SL, MS, WS, and EW declare that they have no conflicts of interest. TW is a Co-Editor of PharmacoEconomics, but was not involved in the peer review nor any editorial decisions for this paper.

Funding

SE acknowledges funding from NIH/ NIGMS R43GM122196, R44GM122196-02A1.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Sampson, C.J., Arnold, R., Bryan, S. et al. Transparency in Decision Modelling: What, Why, Who and How?. PharmacoEconomics 37, 1355–1369 (2019). https://doi.org/10.1007/s40273-019-00819-z

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s40273-019-00819-z