Health Care Management Science

, Volume 21, Issue 1, pp 105–118 | Cite as

Optimal healthcare decision making under multiple mathematical models: application in prostate cancer screening

  • Dimitris Bertsimas
  • John SilberholzEmail author
  • Thomas Trikalinos


Important decisions related to human health, such as screening strategies for cancer, need to be made without a satisfactory understanding of the underlying biological and other processes. Rather, they are often informed by mathematical models that approximate reality. Often multiple models have been made to study the same phenomenon, which may lead to conflicting decisions. It is natural to seek a decision making process that identifies decisions that all models find to be effective, and we propose such a framework in this work. We apply the framework in prostate cancer screening to identify prostate-specific antigen (PSA)-based strategies that perform well under all considered models. We use heuristic search to identify strategies that trade off between optimizing the average across all models’ assessments and being “conservative” by optimizing the most pessimistic model assessment. We identified three recently published mathematical models that can estimate quality-adjusted life expectancy (QALE) of PSA-based screening strategies and identified 64 strategies that trade off between maximizing the average and the most pessimistic model assessments. All prescribe PSA thresholds that increase with age, and 57 involve biennial screening. Strategies with higher assessments with the pessimistic model start screening later, stop screening earlier, and use higher PSA thresholds at earlier ages. The 64 strategies outperform 22 previously published expert-generated strategies. The 41 most “conservative” ones remained better than no screening with all models in extensive sensitivity analyses. We augment current comparative modeling approaches by identifying strategies that perform well under all models, for various degrees of decision makers’ conservativeness.


Comparative modeling Decision analysis Sensitivity analysis Model averaging Optimization Prostate cancer screening Simulation modeling 


  1. 1.
    Albertsen P, Hanley J, Fine J (2005) 20-year outcomes following conservative management of clinically localized prostate cancer. JAMA 293(17):2095–2101CrossRefGoogle Scholar
  2. 2.
    Andriole GL, Crawford ED, Grubb RL, Buys SS, Chia D, Church TR, Fouad MN, Gelmann EP, Kvale PA, Reding DJ, Weissfeld JL, Yokochi LA, O’Brien B, Clapp JD, Rathmell JM, Riley TL, Hayes RB, Kramer BS, Izmirlian G, Miller AB, Pinsky PF, Prorok PC, Gohagan JK, Berg CD (2009) Mortality results from a randomized prostate-cancer screening trial. N Engl J Med 360(13):1310–1319CrossRefGoogle Scholar
  3. 3.
    Arias E (2010) United States life tables, 2006. Natl Vital Stat Rep 58(21):1–40Google Scholar
  4. 4.
    Aus G, Robinson D, Rosell J, Sandblom G, Varenhorst E (2005) Survival in prostate carcinoma—outcomes from a prospective, population-based cohort of 8887 men with up to 15 years of follow-up. Cancer 103(5):943–951CrossRefGoogle Scholar
  5. 5.
    Bertsimas D, Tsitsiklis J (1997) Introduction to linear optimization. Athena ScientificGoogle Scholar
  6. 6.
    Bojke L, Claxton K, Sculpher M, Palmer S (2009) Characterizing structural uncertainty in decision analytic models: a review and application of methods. Value Health 12(5):739–749CrossRefGoogle Scholar
  7. 7.
    Briggs AH, Weinstein MC, Fenwick EA, Karnon J, Sculpher MJ, Paltiel AD (2012) Model parameter estimation and uncertainty analysis: a report of the ISPOR-SMDM Modeling Good Research Practices Task Force Working Group-6. Med Decis Making 32(5):722–732Google Scholar
  8. 8.
    Bubendorf L, Schöpfer A, Wagner U, Sauter G, Moch H, Willi N, Gasser T, Mihatsch M (2000) Metastatic patterns of prostate cancer: an autopsy study of 1,589 patients. Hum Pathol 31(5):578–583CrossRefGoogle Scholar
  9. 9.
    U.S. Cancer Statistics Working Group (2016) United States Cancer Statistics: 1999 – 2013 Incidence and Mortality Web-based Report. Accessed on July 31, 2016
  10. 10.
    Cuzick J, Thorat MA, Andriole G, Brawley OW, Brown PH, Culig Z, Eeles RA, Ford LG, Hamdy FC, Holmberg L, Ilic D, Key TJ, La Vecchia C, Lilja H, MarbergerM,Meyskens FL, Minasian LM, Parker C, Parnes HL, Perner S, Rittenhouse H, Schalken J, Schmid HP, Schmitz-Dräger BJ, Schröder FH, Stenzl A, Tombal B, Wilt TJ, Wolk A (2014) Prevention and early detection of prostate cancer. Lancet Oncol 15(11):E484–E492Google Scholar
  11. 11.
    Dowdy DW, Houben R, Cohen T, Pai M, Cobelens F, Vassall A, Menzies NA, Gomez GB, Langley I, Squire SB, White R (2014) Impact and cost-effectiveness of current and future tuberculosis diagnostics: the contribution of modelling. Int J Tuberc Lung Dis 18(9):1012–1018CrossRefGoogle Scholar
  12. 12.
    Draisma G, Etzioni R, Tsodikov A, Mariotto A, Wever E, Gulati R, Feuer E, de Koning H (2009) Lead time and overdiagnosis in prostate-specific antigen screening: Importance of methods and context. J Natl Cancer Inst 101(6):374–383CrossRefGoogle Scholar
  13. 13.
    Draper D (1995) Assessment and propagation of model uncertainty. J R Stat Soc Series B Stat Methodol 57(1):45–97Google Scholar
  14. 14.
    Eaton JW, Menzies NA, Stover J, Cambiano V, Chindelevitch L, Cori A, Hontelez JA, Humair S, Kerr CC, Klein DJ, Mishra S, Mitchell KM, Nichols BE, Vickerman P, Bakker R, Bärnighausen T, Bershteyn A, Bloom DE, Boily MC, Chang ST, Cohen T, Dodd PJ, Fraser C, Gopalappa C, Lundgren J, Martin NK, Mikkelsen E, Mountain E, Pham QD, Pickles M, Phillips A, Platt L, Pretorius C, Prudden HJ, Salomon JA, van de Vijver DA, de Vlas SJ, Wagner BG, White RG, Wilson DP, Zhang L, Blandford J, Meyer-Rath G, Remme M, Revill P, Sangrujee N, Terris-Prestholt F, Doherty M, Shaffer N, Easterbrook PJ, Hirnschall G, Hallett TB (2014) Health benefits, costs, and cost-effectiveness of earlier eligibility for adult antiretroviral therapy and expanded treatment coverage: a combined analysis of 12 mathematical models. Lancet Glob Health 2(1):e23–34CrossRefGoogle Scholar
  15. 15.
    Eddy DM (1980) Screening for cancer: theory, analysis, and design. Prentice HallGoogle Scholar
  16. 16.
    Eddy DM, Hollingworth W, Caro JJ, Tsevat J, McDonald KM, Wong JB (2012) Model transparency and validation: A report of the ISPOR-SMDM Modeling Good Research Practices Task Force-7. Value Health 15(6):843–850Google Scholar
  17. 17.
    Etzioni R, Gulati R (2013) Response: Reading between the lines of cancer screening trials: Using modeling to understand the evidence. Med Care 51(4):304–306CrossRefGoogle Scholar
  18. 18.
    Etzioni R, Tsodikov A, Mariotto A, Szabo A, Falcon S, Wegelin J, diTommaso D, Karnofski K, Gulati R, Penson DF, Feuer E (2008) Quantifying the role of PSA screening in the US prostate cancer mortality decline. Cancer Causes Control 19(2):175–181CrossRefGoogle Scholar
  19. 19.
    Ferlay J, Soerjomataram I, Ervik M, Dikshit R, Eser S, Mathers C, Rebelo M, Parkin DM, Forman D, Bray F (2014) GLOBOCAN 2012 v1.1, Cancer Incidence and Mortality Worldwide: IARC CancerBase No. 11. Lyon, France: International Agency for Research on Cancer. Accessed on July 31, 2016
  20. 20.
    Greco S, Ehrgott M, Figueira JR, (eds.) (2014) Multiple criteria decision analysis. State of the art surveys. SpringerGoogle Scholar
  21. 21.
    Gilboa I, Schmeidler D (1989) Maxmin expected utility with a non-unique prior. J Math Econom 18(2):141–153Google Scholar
  22. 22.
    Ghani KR, Grigor K, Tulloch DN, Bollina PR, McNeill SA (2005) Trends in reporting gleason score 1991 to 2001: changes in the pathologist’s practice. Eur Urol 47(2):196–201CrossRefGoogle Scholar
  23. 23.
    Ghirardato P, Maccheroni F, Marinacci M (2004) Differentiating ambiguity and ambiguity attitude. J Econ Theory 118(2):133–173CrossRefGoogle Scholar
  24. 24.
    Gulati R, Gore JL, Etzioni R (2013) Comparative effectiveness of alternative PSA-based prostate cancer screening strategies. Ann Intern Med 158(3):145–153CrossRefGoogle Scholar
  25. 25.
    Gulati R, Tsodikov A, Wever EM, Mariotto AB, Heijnsdijk EAM, Katcher J, de Koning HJ, Etzioni R (2012) The impact of PLCO control arm contamination on perceived PSA screening efficacy. Cancer Causes Control 23(6):827–835CrossRefGoogle Scholar
  26. 26.
    Haas GP, Delongchamps NB, Jones RF, Chandan V, Serio AM, Vickers AJ, Jumbelic M, Threatte G, Korets R, Lilja H, de la Roza G (2007) Needle biopsies on autopsy prostates: Sensitivity of cancer detection based on true prevalence. J Natl Cancer Inst 99(19):1484–1489Google Scholar
  27. 27.
    Habbema JDF, Schechter CB, Cronin KA, Clarke LD, Feuer EJ (2006) Modeling cancer natural history, epidemiology, and control: reflections on the CISNET breast group experience. J Natl Cancer Inst Monogr 2006(36):122–126CrossRefGoogle Scholar
  28. 28.
    Center for the Evaluation of Value and Risk in Health (2015) The Cost-Effectiveness Analysis Registry. Institute for Clinical Research and Health Policy Studies, Tufts Medical Center, Boston, MA, US. Accessed on October 03, 2015Google Scholar
  29. 29.
    Heijnsdijk EA, Wever EM, Auvinen A, Hugosson J, Ciatto S, Nelen V, Kwiatkowski M, Villers A, Páez A, Moss SM, Zappa M, Tammela TL, Mäkinen T, Carlsson S, Korfage IJ, Essink-Bot ML, Otto SJ, Draisma G, Bangma CH, Roobol MJ, Schröder FH, de Koning HJ (2012) Quality-of-life effects of prostate-specific antigen screening. N Engl J Med 367(7):595–605CrossRefGoogle Scholar
  30. 30.
    Ilic D, Neuberger MM, Djulbegovic M, Dahm P (2013) Screening for prostate cancer. Cochrane Database Syst Rev (1):CD004720Google Scholar
  31. 31.
    Kjellman A, Akre O, Norming U, Törnblom M, Gustafsson O (2009) 15-year followup of a population based prostate cancer screening study. J Urol 181(4):1615–1621CrossRefGoogle Scholar
  32. 32.
    Kobayashi T, Goto R, Ito K, Mitsumori K (2007) Prostate cancer screening strategies with re-screening interval determined by individual baseline prostate-specific antigen values are cost-effective. Eur J Surg Oncol 33 (6):783–789CrossRefGoogle Scholar
  33. 33.
    Kong CY, Kroep S, Curtius K, Hazelton WD, Jeon J, Meza R, Heberle CR, Miller MC, Choi SE, Lansdorp-Vogelaar I, van Ballegooijen M, Feuer EJ, Inadomi JM, Hur C, Luebeck EG (2014) Exploring the recent trend in esophageal adenocarcinoma incidence and mortality using comparative simulation modeling. Cancer Epidemiol Biomarkers Prev 23(6):997–1006CrossRefGoogle Scholar
  34. 34.
    de Koning HJ, Meza R, Plevritis SK, ten Haaf K, Munshi VN, Jeon J, Erdogan SA, Kong CY, Han SS, van Rosmalen J, Choi SE, Pinsky PF, de Gonzalez AB, Berg CD, Black WC, Tammemägi MC, Hazelton WD, Feuer EJ, McMahon PM (2014) Benefits and harms of computed tomography lung cancer screening strategies: A comparative modeling study for the U.S. Preventive Services Task Force. Ann Intern Med 160(5):311–320CrossRefGoogle Scholar
  35. 35.
    Krahn MD, Mahoney JE, Eckman MH, Trachtenberg J, Pauker SG, Detsky AS (1994) Screening for prostate cancer: a decision analytic view. JAMA 272(10):773–780Google Scholar
  36. 36.
    Kuntz KM, Lansdorp-Vogelaar I, Rutter CM, Knudsen AB, van Ballegooijen M, Savarino JE, Feuer EJ, Zauber AG (2011) A systematic comparison of microsimulation models of colorectal cancer: the role of assumptions about adenoma progression. Med Decis Making 31(4):530–539CrossRefGoogle Scholar
  37. 37.
    Labrie F, Candas B, Cusan L, Gomez JL, Bélanger A, Brousseau G, Chevrette E, Lévesque J (2004) Screening decreases prostate cancer mortality: 11-year follow-up of the 1988 Quebec prospective randomized controlled trial. Prostate 59(3):311–318CrossRefGoogle Scholar
  38. 38.
    Lee SJ, Zelen M (2002) Statistical models for screening: planning public health programs. In: Beam C (ed) Biostatistical applications in cancer research. Springer, US, pp 19–36CrossRefGoogle Scholar
  39. 39.
    Mandelblatt JS, Cronin KA, Bailey S, Berry DA, de Koning HJ, Draisma G, Huang H, Lee SJ, Munsell M, Plevritis SK, Ravdin P, Schechter CB, Sigal B, Stoto MA, Stout NK, van Ravesteyn NT, Venier J, Zelen M, Feuer EJ (2009) Effects of mammography screening under different screening schedules: Model estimates of potential benefits and harms. Ann Intern Med 151(10):738–747CrossRefGoogle Scholar
  40. 40.
    Messing EM, Manola J, Yao J, Kiernan M, Crawford D, Wilding G, di’SantAgnese PA, Trump D (2006) Immediate versus deferred androgen deprivation treatment in patients with node-positive prostate cancer after radical prostatectomy and pelvic lymphadenectomy. Lancet Oncol 7(6):472–479CrossRefGoogle Scholar
  41. 41.
    National Academies of Science (2012) Assessing the reliability of complex models: Mathematical and statistical foundations of verification, validation and uncertainty quantification.
  42. 42.
    National Cancer Institute (2008) Surveillance epidemiology and end results.
  43. 43.
    Oesterling JE, Jacobsen SJ, Chute CG, Guess HA, Girman CJ, Panser LA, Lieber MM (1993) Serum prostate-specific antigen in a community-based population of healthy men. Establishment of age-specific reference ranges. JAMA 270(7):860–864CrossRefGoogle Scholar
  44. 44.
    Ross KS, Carter HB, Pearson JD, Guess HA (2000) Comparative efficiency of prostate-specific antigen screening strategies for prostate cancer detection. JAMA 284(11):1399–1405CrossRefGoogle Scholar
  45. 45.
    Sandblom G, Varenhorst E, Rosell J, Löfman O, Carlsson P (2011) Randomised prostate cancer screening trial: 20 year follow-up. BMJ 342:d1539CrossRefGoogle Scholar
  46. 46.
    Scardino PT, Beck JR, Miles BJ (1994) Conservative management of prostate cancer. N Engl J Med 330(25):1831Google Scholar
  47. 47.
    Schröder FH, Hugosson J, Roobol MJ, Tammela TL, Ciatto S, Nelen V, Kwiatkowski M, Lujan M, Lilja H, Zappa M, Denis LJ, Recker F, Berenguer A, Määttänen L, Bangma CH, Aus G, Villers A, Rebillard X, van der Kwast T, Blijenberg BG, Moss SM, de Koning HJ, Auvinen A (2009) Screening and prostate-cancer mortality in a randomized European study. N Engl J Med 360(13):1320–1328CrossRefGoogle Scholar
  48. 48.
    Tsodikov A, Szabo A, Wegelin J (2006) A population model of prostate cancer incidence. Stat Med 25 (16):2846–2866CrossRefGoogle Scholar
  49. 49.
    Underwood DJ, Zhang J, Denton BT, Shah ND, Inman BA (2012) Simulation optimization of PSA-threshold based prostate cancer screening policies. Health Care Manag Sci 15(4):293–309CrossRefGoogle Scholar
  50. 50.
    Weinstein MC, O’Brien B, Hornberger J, Jackson J, Johannesson M, McCabe C, Luce BR (2003) Principles of good practice for decision analytic modeling in health-care evaluation: report of the ISPOR Task Force on Good Research Practices–Modeling Studies. Value Health 6(1):9–17Google Scholar
  51. 51.
    Zauber AG, Lansdorp-Vogelaar I, Knudsen AB, Wilschut J, van Ballegooijen M, Kuntz KM (2008) Evaluating test strategies for colorectal cancer screening: A decision analysis for the U.S. Preventive Services Task Force. Ann Intern Med 149(9):659–669CrossRefGoogle Scholar
  52. 52.
    Zelen M, Feinleib M (1969) On the theory of screening for chronic diseases. Biometrika 56(3):601–614CrossRefGoogle Scholar
  53. 53.
    Zhang J, Denton B, Balasubramanian H, Shah N, Inman B (2012) Optimization of PSA screening policies: a comparison of the patient and societal perspectives. Med Decis Making 32(2):337–349CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2016

Authors and Affiliations

  • Dimitris Bertsimas
    • 1
  • John Silberholz
    • 2
    Email author
  • Thomas Trikalinos
    • 3
  1. 1.MIT Sloan School of Management and Operations Research CenterCambridgeUSA
  2. 2.MIT Sloan School of Management and Operations Research CenterCambridgeUSA
  3. 3.Department of Health Services, Policy & Practice and Center for Evidence-based MedicineBrown University School of Public HealthProvidenceUSA

Personalised recommendations