A Flexible Open-Source Decision Model for Value Assessment of Biologic Treatment for Rheumatoid Arthritis

  • Devin Incerti
  • Jeffrey R. Curtis
  • Jason Shafrin
  • Darius N. Lakdawalla
  • Jeroen P. JansenEmail author
Original Research Article



The nature of model-based cost-effectiveness analysis can lead to disputes in the scientific community. We propose an iterative and collaborative approach to model development by presenting a flexible open-source simulation model for rheumatoid arthritis (RA), accessible to both technical and non-technical end-users.


The RA model is a discrete-time individual patient simulation with 6-month cycles. Model input parameters were estimated based on currently available evidence and treatment effects were obtained with Bayesian network meta-analysis techniques. The model contains 384 possible model structures informed by previously published models. The model consists of the following components: (i) modifiable R and C++ source code available in a GitHub repository; (ii) an R package to run the model for custom analyses; (iii) detailed model documentation; (iv) a web-based user interface for full control over the model without the need to be well-versed in the programming languages; and (v) a general audience web-application allowing those who are not experts in modeling or health economics to interact with the model and contribute to value assessment discussions.


A primary function of the initial version of RA model is to help understand and quantify the impact of parameter uncertainty (with probabilistic sensitivity analysis), structural uncertainty (with multiple competing model structures), the decision framework (cost-effectiveness analysis or multi-criteria decision analysis), and perspective (healthcare or limited societal) on estimates of value.


In order for a decision model to remain relevant over time it needs to evolve along with its supporting body of clinical evidence and scientific insight. Multiple clinical and methodological experts can modify or contribute to the RA model at any time due to its open-source nature.


Author Contributions

DI and JPJ designed the study, developed the model, and wrote the manuscript. JRC provided clinical input on model design and contributed to the writing of the manuscript; JS and DNL provided economic input on model design and contributed to writing of the manuscript.

Compliance with Ethical Standards


This research was funded through the Innovation and Value Initiative (IVI), a multi-stakeholder research initiative.

Conflicts of interest

Devin Incerti, Jason Shafrin, and Jeroen Jansen are salaried employees of Precision Medicine Group. Darius Lakdawalla and Jeroen Jansen are shareholders of Precision Medicine Group, the parent company of Precision Health Economics (PHE), and Darius Lakdawalla is also a paid consultant to PHE. Jeffrey Curtis is a paid consultant to IVI. At the time of the current study, IVI was part of PHE and partly funded by different pharmaceutical companies.

Data Availability

Source code and data for the model are available at: A webpage with links to all components of the model (R package, tutorial, supplemental documentation, and web-interfaces) can be found at:

Supplementary material

40273_2018_765_MOESM1_ESM.docx (176 kb)
Supplementary material 1 (DOCX 175 kb)


  1. 1.
    Dunlop WC, Mason N, Kenworthy J, Akehurst RL. Benefits, challenges and potential strategies of open source health economic models. Pharmacoeconomics. 2017;35(1):125–8.Google Scholar
  2. 2.
    Schramm W, Sailer F, Pobiruchin M, Weiss C. PROSIT open source disease models for diabetes mellitus. Stud Health Technol Inform. 2016;226:115–8.Google Scholar
  3. 3.
    Cohen JT, Neumann PJ, Wong JB. A call for open-source cost-effectiveness analysis. Ann Intern Med. 2018;168(7):529.Google Scholar
  4. 4.
    Sanders GD, Neumann PJ, Basu A, Brock DW, Feeny D, Krahn M, et al. Recommendations for conduct, methodological practices, and reporting of cost-effectiveness analyses: second panel on cost-effectiveness in health and medicine. JAMA. 2016;316(10):1093–103.Google Scholar
  5. 5.
    Jackson CH, Sharples LD, Thompson SG. Structural and parameter uncertainty in Bayesian cost-effectiveness models. J Roy Stat Soc Ser C (Appl Stat). 2010;59(2):233–53.Google Scholar
  6. 6.
    Incerti D, Jansen JP. A description of the IVI-RA model. 2017.
  7. 7.
    Helmick CG, Felson DT, Lawrence RC, Gabriel S, Hirsch R, Kwoh CK, et al. Estimates of the prevalence of arthritis and other rheumatic conditions in the United States. Part I. Arthritis Rheum. 2008;58(1):15–25.Google Scholar
  8. 8.
    da Rocha Castelar Pinheiro G, Khandker R, Sato R, Rose A, Piercy J. Impact of rheumatoid arthritis on quality of life, work productivity and resource utilisation: an observational, cross-sectional study in Brazil. Clin Exp Rheumatol. 2013;31(3):334–40.Google Scholar
  9. 9.
    Birnbaum H, Pike C, Kaufman R, Marynchenko M, Kidolezi Y, Cifaldi M. Societal cost of rheumatoid arthritis patients in the US. Curr Med Res Opin. 2010;26(1):77–90.Google Scholar
  10. 10.
    Lundkvist J, Kastäng F, Kobelt G. The burden of rheumatoid arthritis and access to treatment: health burden and costs. Eur J Health Econ. 2008;8(2):49–60.Google Scholar
  11. 11.
    Institute for Clinical and Economic Review. Targeted immune modulators for rheumatoid arthritis: effectiveness & value. Institute for Clinical and Economic Review; 2017.Google Scholar
  12. 12.
    Madan J, Ades AE, Welton NJ. An overview of models used in economic analyses of biologic therapies for arthritis—from current diversity to future consensus. Rheumatology. 2011;50 Suppl 4:iv10–iv8.Google Scholar
  13. 13.
    Brennan A, Bansback N, Reynolds A, Conway P. Modelling the cost-effectiveness of etanercept in adults with rheumatoid arthritis in the UK. Rheumatology (Oxford). 2004;43(1):62–72.Google Scholar
  14. 14.
    Tosh J, Brennan A, Wailoo A, Bansback N. The Sheffield rheumatoid arthritis health economic model. Rheumatology. 2011;50(Suppl 4):iv26–iv31.Google Scholar
  15. 15.
    Wailoo AJ, Bansback N, Brennan A, Michaud K, Nixon RM, Wolfe F. Biologic drugs for rheumatoid arthritis in the Medicare program: a cost-effectiveness analysis. Arthritis Rheum. 2008;58(4):939–46.Google Scholar
  16. 16.
    Carlson JJ, Ogale S, Dejonckheere F, Sullivan SD. Economic evaluation of tocilizumab monotherapy compared to adalimumab monotherapy in the treatment of severe active rheumatoid arthritis. Value Health. 2015;18(2):173–9.Google Scholar
  17. 17.
    Stephens S, Botteman MF, Cifaldi MA, van Hout BA. Modelling the cost-effectiveness of combination therapy for early, rapidly progressing rheumatoid arthritis by simulating the reversible and irreversible effects of the disease. BMJ Open. 2015;5(6):e006560.Google Scholar
  18. 18.
    Athanasakis K, Tarantilis F, Tsalapati K, Konstantopoulou T, Vritzali E, Kyriopoulos J. Cost-utility analysis of tocilizumab monotherapy in first line versus standard of care for the treatment of rheumatoid arthritis in Greece. Rheumatol Int. 2015;35(9):1489–95.Google Scholar
  19. 19.
    Stevenson M, Archer R, Tosh J, Simpson E, Everson-Hock E, Stevens J, et al. Adalimumab, etanercept, infliximab, certolizumab pegol, golimumab, tocilizumab and abatacept for the treatment of rheumatoid arthritis not previously treated with disease-modifying antirheumatic drugs and after the failure of conventional disease-modifying antirheumatic drugs only: systematic review and economic evaluation. Health Technol Assess. 2016;20(35):1–610.Google Scholar
  20. 20.
    Stevenson MD, Wailoo AJ, Tosh JC, Hernandez-Alava M, Gibson LA, Stevens JW, et al. The cost-effectiveness of sequences of biological disease-modifying antirheumatic drug treatment in england for patients with rheumatoid arthritis who can tolerate methotrexate. J Rheumatol. 2017;44(7):973–80.Google Scholar
  21. 21.
    Diamantopoulos A, Finckh A, Huizinga T, Sungher D, Sawyer L, Neto D, et al. Tocilizumab in the treatment of rheumatoid arthritis: a cost-effectiveness analysis in the UK. Pharmacoeconomics. 2014;32(8):775–87.Google Scholar
  22. 22.
    Jalal H, Pechlivanoglou P, Krijkamp E, Alarid-Escudero F, Enns E, Hunink MGM. An overview of R in health decision sciences. Med Decis Making. 2017;37(7):735–46.Google Scholar
  23. 23.
    Thokala P, Devlin N, Marsh K, Baltussen R, Boysen M, Kalo Z, et al. Multiple criteria decision analysis for health care decision making—an introduction: report 1 of the ISPOR MCDA Emerging Good Practices Task Force. Value Health. 2016;19(1):1–13.Google Scholar
  24. 24.
    Briggs AH, Claxton K, Sculpher MJ. Decision modelling for health economic evaluation. Oxford: Oxford University Press; 2006.Google Scholar
  25. 25.
    Meltzer DO, Smith PC. Theoretical issues relevant to the economic evaluation of health technologies. Handb Health Econ. 2011;2:433–69.Google Scholar
  26. 26.
    Drummond MF, Sculpher MJ, Claxton K, Stoddart GL, Torrance GW. Methods for the economic evaluation of health care programmes. Oxford: Oxford University Press; 2015.Google Scholar
  27. 27.
    Lakdawalla D, Malani A, Reif J. The insurance value of medical innovation. J Public Econ. 2017;145:94–102.Google Scholar
  28. 28.
    Garrison LP, Kamal-Bahl S, Towse A. Toward a broader concept of value: identifying and defining elements for an expanded cost-effectiveness analysis. Value Health. 2017;20(2):213–6.Google Scholar
  29. 29.
    Singh JA, Saag KG, Bridges SL, Akl EA, Bannuru RR, Sullivan MC, et al. 2015 American college of rheumatology guideline for the treatment of rheumatoid arthritis. Arthritis Care Res (Hoboken). 2016;68(1):1–25.Google Scholar
  30. 30.
    Anderson J, Caplan L, Yazdany J, Robbins ML, Neogi T, Michaud K, et al. Rheumatoid arthritis disease activity measures: American College of Rheumatology recommendations for use in clinical practice. Arthritis Care Res (Hoboken). 2012;64(5):640–7.Google Scholar
  31. 31.
    Aletaha D, Ward MM, Machold KP, Nell VP, Stamm T, Smolen JS. Remission and active disease in rheumatoid arthritis: defining criteria for disease activity states. Arthritis Rheum. 2005;52(9):2625–36.Google Scholar
  32. 32.
    Prevoo ML, van’t Hof MA, Kuper HH, van Leeuwen MA, van de Putte LB, van Riel PL. Modified disease activity scores that include twenty-eight-joint counts. Development and validation in a prospective longitudinal study of patients with rheumatoid arthritis. Arthritis Rheum. 1995;38(1):44–8.Google Scholar
  33. 33.
    Smolen JS, Breedveld FC, Schiff MH, Kalden JR, Emery P, Eberl G, et al. A simplified disease activity index for rheumatoid arthritis for use in clinical practice. Rheumatology (Oxford). 2003;42(2):244–57.Google Scholar
  34. 34.
    Aletaha D, Nell VP, Stamm T, Uffmann M, Pflugbeil S, Machold K, et al. Acute phase reactants add little to composite disease activity indices for rheumatoid arthritis: validation of a clinical activity score. Arthritis Res Ther. 2005;7(4):R796–806.Google Scholar
  35. 35.
    Aletaha D, Smolen J. The Simplified Disease Activity Index (SDAI) and the Clinical Disease Activity Index (CDAI): a review of their usefulness and validity in rheumatoid arthritis. Clin Exp Rheumatol. 2005;23(5 Suppl 39):S100–8.Google Scholar
  36. 36.
    Wolfe F, Michaud K. The loss of health status in rheumatoid arthritis and the effect of biologic therapy: a longitudinal observational study. Arthritis Res Ther. 2010;12(2):R35.Google Scholar
  37. 37.
    Michaud K, Wallenstein G, Wolfe F. Treatment and nontreatment predictors of health assessment questionnaire disability progression in rheumatoid arthritis: a longitudinal study of 18,485 patients. Arthritis Care Res (Hoboken). 2011;63(3):366–72.Google Scholar
  38. 38.
    Gibson L, Alava MH, Wailoo A. Progression of disease in people with rheumatoid arthritis treated with non-biologic therapies. Sheffield: School of Health and Related Research, University of Sheffield; 2015.Google Scholar
  39. 39.
    Norton S, Fu B, Scott DL, Deighton C, Symmons DP, Wailoo AJ, et al. Health Assessment Questionnaire disability progression in early rheumatoid arthritis: systematic review and analysis of two inception cohorts. Semin Arthritis Rheum. 2014;44(2):131–44.Google Scholar
  40. 40.
    Strand V, Williams S, Miller P, Saunders K, Grant S, Kremer J. OP0064 discontinuation of biologic therapy in rheumatoid arthritis (RA): analysis from the Consortium of Rheumatology Researchers of North America (CORRONA) database. Ann Rheum Dis. 2013;72(Suppl 3):A71–2.Google Scholar
  41. 41.
    Zhang J, Shan Y, Reed G, Kremer J, Greenberg JD, Baumgartner S, et al. Thresholds in disease activity for switching biologics in rheumatoid arthritis patients: experience from a large US cohort. Arthritis Care Res. 2011;63(12):1672–9.Google Scholar
  42. 42.
    Ramiro S, Sepriano A, Chatzidionysiou K, Nam JL, Smolen JS, van der Heijde D, et al. Safety of synthetic and biological DMARDs: a systematic literature review informing the 2016 update of the EULAR recommendations for management of rheumatoid arthritis. Ann Rheum Dis. 2017;76(6):1101–36.Google Scholar
  43. 43.
    Singh JA, Wells GA, Christensen R, Tanjong Ghogomu E, Maxwell LJ, MacDonald JK, et al. Adverse effects of biologics: a network meta-analysis and Cochrane overview. London: The Cochrane Library; 2011.Google Scholar
  44. 44.
    Arias E. United States life tables, 2011. Natl Vital Stat Rep. 2015;64(11):1–63.Google Scholar
  45. 45.
    Wolfe F, Michaud K, Gefeller O, Choi HK. Predicting mortality in patients with rheumatoid arthritis. Arthritis Rheum. 2003;48(6):1530–42.Google Scholar
  46. 46.
    Michaud K, Vera-Llonch M, Oster G. Mortality risk by functional status and health-related quality of life in patients with rheumatoid arthritis. J Rheumatol. 2012;39(1):54–9.Google Scholar
  47. 47.
    Hernández Alava M, Wailoo A, Wolfe F, Michaud K. The relationship between EQ-5D, HAQ and pain in patients with rheumatoid arthritis. Rheumatology (Oxford). 2013;52(5):944–50.Google Scholar
  48. 48.
    Wailoo A, Brennan A, Bansback N, Nixon R, Wolfe F, Michaud K. Modeling the cost effectiveness of etanercept, adalimumab and anakinra compared to infliximab in the treatment of patients with rheumatoid arthritis in the Medicare program. Rockville: Agency for Healthcare Research and Quality; 2006.Google Scholar
  49. 49.
    Oppong R, Kaambwa B, Nuttall J, Hood K, Smith RD, Coast J. The impact of using different tariffs to value EQ-5D health state descriptions: an example from a study of acute cough/lower respiratory tract infections in seven countries. Eur J Health Econ. 2013;14(2):197–209.Google Scholar
  50. 50.
    Claxton K, Sculpher M, McCabe C, Briggs A, Akehurst R, Buxton M, et al. Probabilistic sensitivity analysis for NICE technology assessment: not an optional extra. Health Econ. 2005;14(4):339–47.Google Scholar
  51. 51.
    Claxton L, Jenks M, Taylor M, Wallenstein G, Mendelsohn AM, Bourret JA, et al. An Economic evaluation of tofacitinib treatment in rheumatoid arthritis: modeling the cost of treatment strategies in the united states. J Manag Care Spec Pharm. 2016;22(9):1088–102.Google Scholar
  52. 52.
    Baio G, Dawid AP. Probabilistic sensitivity analysis in health economics. Stat Methods Med Res. 2015;24(6):615–34.Google Scholar
  53. 53.
    Vemer P, Corro Ramos I, van Voorn GA, Al MJ, Feenstra TL. AdViSHE: a validation-assessment tool of health-economic models for decision makers and model users. Pharmacoeconomics. 2016;34(4):349–61.Google Scholar
  54. 54.
    Gonzalez A, Maradit Kremers H, Crowson CS, Nicola PJ, Davis JM, Therneau TM, et al. The widening mortality gap between rheumatoid arthritis patients and the general population. Arthritis Rheum. 2007;56(11):3583–7.Google Scholar
  55. 55.
    Radovits BJ, Fransen J, Al Shamma S, Eijsbouts AM, van Riel PL, Laan RF. Excess mortality emerges after 10 years in an inception cohort of early rheumatoid arthritis. Arthritis Care Res (Hoboken). 2010;62(3):362–70.Google Scholar
  56. 56.
    How is lifespan affected by RA. National Rheumatoid Arthritis Society; 2016. Accessed July 2017.
  57. 57.
    Hoeting JA, Madigan D, Raftery AE, Volinsky CT. Bayesian model averaging: a tutorial. Stat Sci. 1999:382–401.Google Scholar
  58. 58.
    Jalal H, Dowd B, Sainfort F, Kuntz KM. Linear regression metamodeling as a tool to summarize and present simulation model results. Med Decis Making. 2013;33(7):880–90.Google Scholar
  59. 59.
    Heath A, Manolopoulou I, Baio G. Estimating the expected value of partial perfect information in health economic evaluations using integrated nested Laplace approximation. Stat Med. 2016;35(23):4264–80.Google Scholar
  60. 60.
    Curtis JR, Jain A, Askling J, Bridges SL Jr, Carmona L, Dixon W, et al. (eds). A Comparison of Patient Characteristics and Outcomes in Selected European and US Rheumatoid Arthritis Registries. Seminars in arthritis and rheumatism; 2010;40(1):2–14.e1.Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Devin Incerti
    • 1
  • Jeffrey R. Curtis
    • 2
  • Jason Shafrin
    • 1
  • Darius N. Lakdawalla
    • 3
  • Jeroen P. Jansen
    • 1
    • 4
    Email author
  1. 1.Innovation and Value InitiativeLos AngelesUSA
  2. 2.Division of Clinical Immunology and RheumatologyUniversity of Alabama at BirminghamBirminghamUSA
  3. 3.Schaeffer Center for Health Policy and EconomicsUniversity of Southern CaliforniaLos AngelesUSA
  4. 4.Department of Health Research and Policy (Epidemiology)Stanford University School of MedicineStanfordUSA

Personalised recommendations