Transparency in Health Economic Modeling: Options, Issues and Potential Solutions
Economic models are increasingly being used by health economists to assess the value of health technologies and inform healthcare decision making. However, most published economic models represent a kind of black box, with known inputs and outputs but undisclosed internal calculations and assumptions. This lack of transparency makes the evaluation of the model results challenging, complicates comparisons between models, and limits the reproducibility of the models. Here, we aim to provide an overview of the possible steps that could be undertaken to make economic models more transparent and encourage model developers to share more detailed calculations and assumptions with their peers. Scenarios with different levels of transparency (i.e., how much information is disclosed) and reach of transparency (i.e., who has access to the disclosed information) are discussed, and five key concerns (copyrights, model misuse, confidential data, software, and time/resources) pertaining to model transparency are presented, along with possible solutions. While a shift toward open-source models is underway in health economics, as has happened before in other research fields, the challenges ahead should not be underestimated. Importantly, there is a pressing need to find an acceptable trade-off between the added value of model transparency and the time and resources needed to achieve such transparency. To this end, it will be crucial to set incentives at different stakeholder levels. Despite the many challenges, the many benefits of publicly sharing economic models make increased transparency a goal worth pursuing.
All authors contributed to manuscript ideation and development.
Compliance with Ethical Standards
Conflict of interest
Eric Wu, Zheng-Yi Zhou, Jipan Xie, Cinzia Metallo, Praveen Thokala have no conflicts of interest that are directly relevant to the content of this review/study. Eric Wu, Zheng-Yi Zhou, and Jipan Xie are employees of Analysis Group, a strategy consulting firm, where they advise clients on matters related to health economics and outcomes research (HEOR), including economic models. At the time of manuscript development, Cinzia Metallo was also an employee of Analysis Group. Praveen Thokala is a Senior Research Fellow at the University of Sheffield; his research focuses on health economic modeling.
No funding was received for the development of this review article.
- 8.Dragicevic P, Jansen Y, Sarma A, Kay M, Chevalier F. Increasing the transparency of research papers with explorable multiverse analyses. In: Proceedings of the 2019 CHI conference on human factors in computing systems. Glasgow, Scotland UK: ACM; 2019. pp. 1–15.Google Scholar
- 14.Incerti D, Jansen JA. A description of the IVI-NSCLC model v1.0; last updated January 31, 2019. 2019. https://innovationvalueinitiative.github.io/IVI-NSCLC/modeldoc/model-doc.pdf. Cited 1 Aug 2019.
- 15.Global Health Cost Effectiveness Analysis (GHCEA) Registry. Open-source model clearinghouse. 2019. http://healtheconomics.tuftsmedicalcenter.org/orchard/open-source-model-clearinghouse. Accessed Feb 2019.
- 16.Snowsill TM. Immunosuppressive therapy for kidney transplantation in adults: economic model (dataset). University of Exeter. 2019. https://ore.exeter.ac.uk/repository/handle/10871/25885. Accessed Aug 2019.
- 19.SenseLab. ModelDB. 2019. https://senselab.med.yale.edu/modeldb/. Accessed Feb 2019.
- 20.GitHub Inc. GitHub 2019. https://github.com/. Accessed Feb 2019.
- 21.Introduction of International Copyright Law. 2019. https://www.copyrightlaws.com/introduction-international-copyright-law/. Accessed 1 Aug 2019.
- 22.Creative Commons. https://creativecommons.org/licenses/. Accessed Aug 2019.
- 23.ClinicalTrials.gov ClinicalTrials.gov Database. 2019. https://clinicaltrials.gov/. Accessed Feb 2019.
- 26.US Census Bureau. Synthetic longitudinal business database (SynLBD). 2012. https://www.census.gov/ces/dataproducts/synlbd/. Accessed Feb 2019.