Advertisement

Transparency in Health Economic Modeling: Options, Issues and Potential Solutions

  • Eric Q. WuEmail author
  • Zheng-Yi Zhou
  • Jipan Xie
  • Cinzia Metallo
  • Praveen Thokala
Practical Application

Abstract

Economic models are increasingly being used by health economists to assess the value of health technologies and inform healthcare decision making. However, most published economic models represent a kind of black box, with known inputs and outputs but undisclosed internal calculations and assumptions. This lack of transparency makes the evaluation of the model results challenging, complicates comparisons between models, and limits the reproducibility of the models. Here, we aim to provide an overview of the possible steps that could be undertaken to make economic models more transparent and encourage model developers to share more detailed calculations and assumptions with their peers. Scenarios with different levels of transparency (i.e., how much information is disclosed) and reach of transparency (i.e., who has access to the disclosed information) are discussed, and five key concerns (copyrights, model misuse, confidential data, software, and time/resources) pertaining to model transparency are presented, along with possible solutions. While a shift toward open-source models is underway in health economics, as has happened before in other research fields, the challenges ahead should not be underestimated. Importantly, there is a pressing need to find an acceptable trade-off between the added value of model transparency and the time and resources needed to achieve such transparency. To this end, it will be crucial to set incentives at different stakeholder levels. Despite the many challenges, the many benefits of publicly sharing economic models make increased transparency a goal worth pursuing.

Notes

Author contributions

All authors contributed to manuscript ideation and development.

Compliance with Ethical Standards

Conflict of interest

Eric Wu, Zheng-Yi Zhou, Jipan Xie, Cinzia Metallo, Praveen Thokala have no conflicts of interest that are directly relevant to the content of this review/study. Eric Wu, Zheng-Yi Zhou, and Jipan Xie are employees of Analysis Group, a strategy consulting firm, where they advise clients on matters related to health economics and outcomes research (HEOR), including economic models. At the time of manuscript development, Cinzia Metallo was also an employee of Analysis Group. Praveen Thokala is a Senior Research Fellow at the University of Sheffield; his research focuses on health economic modeling.

Funding

No funding was received for the development of this review article.

References

  1. 1.
    Bell CF. The pursuit of transparency and quality improvement in cost-effectiveness analysis—a case study in disease-modifying drugs for the treatment of multiple sclerosis. J Manag Care Pharm. 2011;17(6):463–8.PubMedGoogle Scholar
  2. 2.
    Dunlop WCN, Mason N, Kenworthy J, Akehurst RL. Benefits, challenges and potential strategies of open source health economic models. PharmacoEconomics. 2017;35(1):125–8.CrossRefGoogle Scholar
  3. 3.
    Eddy DM, Hollingworth W, Caro JJ, Tsevat J, McDonald KM, Wong JB, et al. Model transparency and validation: a report of the ISPOR-SMDM Modeling Good Research Practices Task Force—7. Value Health. 2012;15(6):843–50.CrossRefGoogle Scholar
  4. 4.
    Cohen JT, Wong JB. Can economic model transparency improve provider interpretation of cost-effectiveness analysis? A Response. Med Care. 2017;55(11):912–4.CrossRefGoogle Scholar
  5. 5.
    Padula WV, McQueen RB, Pronovost PJ. Finding resolution for the responsible transparency of economic models in health and medicine. Med Care. 2017;55(11):915–7.CrossRefGoogle Scholar
  6. 6.
    Cohen JT, Neumann PJ, Wong JB. A call for open-source cost-effectiveness analysis. Ann Intern Med. 2017;167(6):432–3.CrossRefGoogle Scholar
  7. 7.
    Padula WV, McQueen RB, Pronovost PJ. Can economic model transparency improve provider interpretation of cost-effectiveness analysis? Evaluating tradeoffs presented by the second panel on cost-effectiveness in health and medicine. Med Care. 2017;55(11):909–11.CrossRefGoogle Scholar
  8. 8.
    Dragicevic P, Jansen Y, Sarma A, Kay M, Chevalier F. Increasing the transparency of research papers with explorable multiverse analyses. In: Proceedings of the 2019 CHI conference on human factors in computing systems. Glasgow, Scotland UK: ACM; 2019. pp. 1–15.Google Scholar
  9. 9.
    Poole C, Agrawal S, Currie CJ. Let cost effectiveness models be open to scrutiny. BMJ (Clin Res Ed). 2007;335(7623):735.CrossRefGoogle Scholar
  10. 10.
    Eddy DM, Hollingworth W, Caro JJ, Tsevat J, McDonald KM, Wong JB. Model transparency and validation: a report of the ISPOR-SMDM Modeling Good Research Practices Task Force-7. Med decis Mak. 2012;32(5):733–43.CrossRefGoogle Scholar
  11. 11.
    Caro JJ, Briggs AH, Siebert U, Kuntz KM. Modeling good research practices—overview: a report of the ISPOR-SMDM Modeling Good Research Practices Task Force-1. Med Decis Mak. 2012;32(5):667–77.CrossRefGoogle Scholar
  12. 12.
    Sullivan W, Hirst M, Beard S, Gladwell D, Fagnani F, Lopez Bastida J, et al. Economic evaluation in chronic pain: a systematic review and de novo flexible economic model. Eur J Health Econ. 2016;17(6):755–70.CrossRefGoogle Scholar
  13. 13.
    Incerti D, Curtis JR, Shafrin J, Lakdawalla DN, Jansen JP. A flexible open-source decision model for value assessment of biologic treatment for rheumatoid arthritis. Pharmacoeconomics. 2019;37(6):829–43.CrossRefGoogle Scholar
  14. 14.
    Incerti D, Jansen JA. A description of the IVI-NSCLC model v1.0; last updated January 31, 2019. 2019. https://innovationvalueinitiative.github.io/IVI-NSCLC/modeldoc/model-doc.pdf. Cited 1 Aug 2019.
  15. 15.
    Global Health Cost Effectiveness Analysis (GHCEA) Registry. Open-source model clearinghouse. 2019. http://healtheconomics.tuftsmedicalcenter.org/orchard/open-source-model-clearinghouse. Accessed Feb 2019.
  16. 16.
    Snowsill TM. Immunosuppressive therapy for kidney transplantation in adults: economic model (dataset). University of Exeter. 2019. https://ore.exeter.ac.uk/repository/handle/10871/25885. Accessed Aug 2019.
  17. 17.
    Bierer BE, Crosas M, Pierce HH. Data authorship as an incentive to data sharing. N Engl J Med. 2017;376(17):1684–7.CrossRefGoogle Scholar
  18. 18.
    McDougal RA, Bulanova AS, Lytton WW. Reproducibility in computational neuroscience models and simulations. IEEE Trans Biomed Eng. 2016;63(10):2021–35.CrossRefGoogle Scholar
  19. 19.
    SenseLab. ModelDB. 2019. https://senselab.med.yale.edu/modeldb/. Accessed Feb 2019.
  20. 20.
    GitHub Inc. GitHub 2019. https://github.com/. Accessed Feb 2019.
  21. 21.
    Introduction of International Copyright Law. 2019. https://www.copyrightlaws.com/introduction-international-copyright-law/. Accessed 1 Aug 2019.
  22. 22.
    Creative Commons. https://creativecommons.org/licenses/. Accessed Aug 2019.
  23. 23.
    ClinicalTrials.gov ClinicalTrials.gov Database. 2019. https://clinicaltrials.gov/. Accessed Feb 2019.
  24. 24.
    Taichman DB, Sahni P, Pinborg A, Peiperl L, Laine C, James A, et al. Data sharing statements for clinical trials: a requirement of the international committee of medical journal editors. JAMA. 2017;317(24):2491–2.CrossRefGoogle Scholar
  25. 25.
    Haji Ali Afzali H, Karnon J. Addressing the challenge for well informed and consistent reimbursement decisions: the case for reference models. PharmacoEconomics. 2011;29(10):823–5.CrossRefGoogle Scholar
  26. 26.
    US Census Bureau. Synthetic longitudinal business database (SynLBD). 2012. https://www.census.gov/ces/dataproducts/synlbd/. Accessed Feb 2019.

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Eric Q. Wu
    • 1
    Email author
  • Zheng-Yi Zhou
    • 2
  • Jipan Xie
    • 3
  • Cinzia Metallo
    • 1
  • Praveen Thokala
    • 4
  1. 1.Analysis GroupBostonUSA
  2. 2.Analysis GroupLondonUK
  3. 3.Analysis GroupLos AngelesUSA
  4. 4.University of SheffieldSheffieldUK

Personalised recommendations