Skip to main content

Calibration Methods Used in Cancer Simulation Models and Suggested Reporting Guidelines

PharmacoEconomics volume 27, pages 533–545 (2009)Cite this article

Abstract

Increasingly, computer simulation models are used for economic and policy evaluation in cancer prevention and control. A model’s predictions of key outcomes, such as screening effectiveness, depend on the values of unobservable natural history parameters. Calibration is the process of determining the values of unobservable parameters by constraining model output to replicate observed data. Because there are many approaches for model calibration and little consensus on best practices, we surveyed the literature to catalogue the use and reporting of these methods in cancer simulation models.

We conducted a MEDLINE search (1980 through 2006) for articles on cancer-screening models and supplemented search results with articles from our personal reference databases. For each article, two authors independently abstracted pre-determined items using a standard form. Data items included cancer site, model type, methods used for determination of unobservable parameter values and description of any calibration protocol. All authors reached consensus on items of disagreement. Reviews and non-cancer models were excluded. Articles describing analytical models, which estimate parameters with statistical approaches (e.g. maximum likelihood) were catalogued separately.Models that included unobservable parameters were analysed and classified by whether calibration methods were reported and if so, the methods used.

The review process yielded 154 articles that met our inclusion criteria and, of these, we concluded that 131 may have used calibration methods to determine model parameters. Although the term ‘calibration’ was not always used, descriptions of calibration or ‘model fitting’ were found in 50% (n = 66) of the articles, with an additional 16% (n = 21) providing a reference to methods. Calibration target data were identified in nearly all of these articles. Other methodological details, such as the goodness-of-fit metric, were discussed in 54% (n = 47 of 87) of the articles reporting calibration methods, while few details were provided on the algorithms used to search the parameter space.

Our review shows that the use of cancer simulation modelling is increasing, although thorough descriptions of calibration procedures are rare in the published literature for these models. Calibration is a key component of model development and is central to the validity and credibility of subsequent analyses and inferences drawn from model predictions. To aid peer-review and facilitate discussion of modelling methods, we propose a standardized Calibration Reporting Checklist for model documentation.

This is a preview of subscription content, access via your institution.

Access options

Buy single article

Instant access to the full article PDF.

USD 49.95

Price includes VAT (USA)
Tax calculation will be finalised during checkout.

Table I
Fig. 1
Table II
Fig. 2

References

  1. Ramsey SD, McIntosh M, Etzioni RD, et al. Simulation modeling of outcomes and cost-effectiveness. Hematol Oncol Clin North Am 2000 Aug; 14 (4): 925–38

    PubMed  CAS  Article  Google Scholar 

  2. Knudsen AB, McMahon PM, Gazelle GS. Use of modeling to evaluate the cost-effectiveness of cancer screening programs. J Clin Oncol 2007; 25: 203–8

    PubMed  Article  Google Scholar 

  3. Feuer EJ, Etzioni RD, Cronin KA, et al. The use of modeling to understand the impact of screening on US mortality: examples from mammography and PSA testing. Stat Methods Med Res 2004 Dec; 13: 421–42

    PubMed  Article  Google Scholar 

  4. Goldie SJ. Chapter 15: public health policy and cost-effectiveness analysis. J Natl Cancer Inst Monogr 2003; 31: 102–10

    PubMed  Article  Google Scholar 

  5. Goldie SJ, Goldhaber-Fiebert JD, Garnett G. Chapter 18: public health policy for cervical cancer prevention. The role of decision science, economic evaluation, and mathematical modeling. Vaccine 2006 Aug 31; 24 (S3): 155–63

    Article  Google Scholar 

  6. Weinstein MC. Recent developments in decision-analytic modelling for economic evaluation. Pharmacoeconomics 2006; 24 (11): 1043–53

    PubMed  Article  Google Scholar 

  7. Goldie SJ, Grima D, Kohli M, et al. A comprehensive natural history model of HPV infection and cervical cancer to estimate the clinical impact of a prophylactic HPV-16/18 vaccine. Int J Cancer 2003; 106: 896–904

    PubMed  CAS  Article  Google Scholar 

  8. Yeh JM, Kuntz KM, Ezzati M, et al. Development of an empirically calibrated model of gastric cancer in two high-risk countries. Cancer Epidemiol Biomarkers Prev 2008 May 1; 17 (5): 1179–87

    PubMed  Article  Google Scholar 

  9. Mandelblatt JS, Lawrence WF, Womack SM, et al. Benefits and costs of using HPV testing to screen for cervical cancer. JAMA 2002 May 8; 287 (18): 2372–81

    PubMed  Article  Google Scholar 

  10. Urban N, Drescher C, Etzioni R, et al. Use of a stochastic simulation model to identify an efficient protocol for ovarian cancer screening. Control Clin Trials 1997 Jun; 18 (3): 251–70

    PubMed  CAS  Article  Google Scholar 

  11. Myers ER, McCrory DC, Nanda K, et al. Mathematical model for the natural history of human papillomavirus infection and cervical carcinogenesis. Am J Epidemiol 2000 June 15, 2000; 151 (12): 1158–71

    CAS  Google Scholar 

  12. Law AM, Kelton WD. Simulation modeling and analysis. 3rd ed. Boston (MA): McGraw-Hill, 2000

    Google Scholar 

  13. Clarke LD, Plevritis SK, Boer R, et al. A comparative review of CISNET breast models used to analyze U.S. breast cancer incidence and mortality trends. J Natl Cancer Inst Monogr 2006; 36: 96–105

    PubMed  Article  Google Scholar 

  14. Banks J, editor. Handbook of simulation: principles, methodology, advances, applications, and practice. New York: John Wiley & Sons. Inc., 1998

    Google Scholar 

  15. Bruning JL, Kintz BL. Computational handbook of statistics. 4th ed. Boston (MA): Allyn & Bacon, 1997

    Google Scholar 

  16. Nelder JA, Mead R. A simplex method for function minimization. Comput J 1965; 7 (4): 308–13

    Article  Google Scholar 

  17. Kirkpatrick S, Gelatt CD, Vecchi MP. Optimization by simulated annealing. Science 1983; 220 (4598): 671–80

    PubMed  CAS  Article  Google Scholar 

  18. Press WH, Teukolsky SA, Vetterling WT, et al. Numerical recipes in C++. 2nd ed. New York (NY): Cambridge University Press, 2002

    Google Scholar 

  19. Wong DF, Leong HW, Liu CL. Simulated annealing for VLSI design. Boston (MA): Kluwer Academic Publishers, 1988

    Book  Google Scholar 

  20. Holland JH. Adaptation in natural and artificial systems. Ann Arbor (MI): University of Michigan Press, 1975

    Google Scholar 

  21. Goldberg DE. Genetic algorithms in search, optimization and machine learning. Boston (MA): Kluwer Academic Publishers, 1989

    Google Scholar 

  22. Manikas TW, Cain JT. Genetic algorithms vs. simulated annealing: a comparison of approaches for solving the circuit partitioning problem. Pittsburgh (PA): University of Pittsburgh, 1996

    Google Scholar 

  23. Ingber L, Rosen B. Genetic algorithms and very fast simulated annealing: a comparison. Math Comput Model 1992; 16 (11): 87–100

    Article  Google Scholar 

  24. Glover F. Tabu search: part I. ORSA J Computing 1989; 1 (3): 190–206

    Article  Google Scholar 

  25. Glover F. Tabu search: part II. ORSA J Computing 1990; 2 (1): 4–32

    Article  Google Scholar 

  26. Kong CY, McMahon PM, Gazelle GS. Calibration of disease simulation models using an engineering approach. Value Health 2009; 12 (4): 521–9

    PubMed  Article  Google Scholar 

  27. Baker RD. Use of a mathematical model to evaluate breast cancer screening policy. Health Care Manage Sci 1998 Oct; 1 (2): 103–13

    CAS  Article  Google Scholar 

  28. Beckett JR, Kotre CJ, Michaelson JS. Analysis of benefit: risk ratio and mortality reduction for the UK Breast Screening Programme. Br J Radiol 2003 May; 76 (905): 309–20

    PubMed  CAS  Article  Google Scholar 

  29. Beemsterboer PM, Warmerdam PG, Boer R, et al. Radiation risk of mammography related to benefit in screening programmes: a favourable balance? J Med Screen 1998; 5 (2): 81–7

    PubMed  CAS  Google Scholar 

  30. Berry DA, Cronin KA, Plevritis SK, et al. Effect of screening and adjuvant treatment on mortality from breast cancer. N Engl J Med 2005 Oct 27; 353 (17): 1784–92

    PubMed  CAS  Article  Google Scholar 

  31. Berry DA, Inoue L, Shen Y, et al. Modeling the impact of treatment and screening on U.S. breast cancer mortality: a Bayesian approach. J Natl Cancer Inst Monogr 2006; 36: 30–6

    PubMed  Article  Google Scholar 

  32. Blanchard K, Colbert JA, Puri D, et al. Mammographic screening: patterns of use and estimated impact on breast carcinoma survival. Cancer 2004 Aug 1; 101 (3): 495–507

    PubMed  Article  Google Scholar 

  33. Boer R, de Koning H, Threlfall A, et al. Cost effectiveness of shortening screening interval or extending age range of NHS breast screening programme: computer simulation study. BMJ 1998 Aug 8; 317 (7155): 376–9

    PubMed  CAS  Article  Google Scholar 

  34. Boer R, de Koning H, van Oortmarssen G, et al. Stage distribution at first and repeat examinations in breast cancer screening. J Med Screen 1999; 6 (3): 132–8

    PubMed  CAS  Google Scholar 

  35. Carter R, Glasziou P, van Oortmarssen G, et al. Cost-effectiveness of mammographic screening in Australia. Aust J Public Health 1993 Mar; 17 (1): 42–50

    PubMed  CAS  Article  Google Scholar 

  36. Carter KJ, Castro F, Kessler E, et al. A computer model for the study of breast cancer. Comput Biol Med 2003 Jul; 33: 345–60

    PubMed  Article  Google Scholar 

  37. Carter KJ, Castro F, Kessler E, et al. Simulation of breast cancer screening: quality assessment of two protocols. J Healthc Qual 2004 Nov-Dec; 26 (6): 31–8

    PubMed  Article  Google Scholar 

  38. Castro F, Carter KJ, Kessler E, et al. The relation of breast cancer staging to screening protocol compliance: a computer simulation study. Comput Biol Med 2005 Feb; 35 (2): 91–101

    PubMed  Article  Google Scholar 

  39. Chia YL, Salzman P, Plevritis SK, et al. Simulation-based parameter estimation for complex models: a breast cancer natural history modelling illustration. Stat Methods Med Res 2004 Dec; 13 (6): 507–24

    PubMed  Article  Google Scholar 

  40. Cong XJ, Shen Y, Miller AB. Estimation of age-specific sensitivity and sojourn time in breast cancer screening studies. Stat Med 2005 Oct 30; 24 (20): 3123–38

    PubMed  Article  Google Scholar 

  41. Connor RJ, Boer R, Prorok PC, et al. Investigation of design and bias issues in case-control studies of cancer screening using microsimulation. Am J Epidemiol 2000 May 15; 151 (10): 991–8

    PubMed  CAS  Article  Google Scholar 

  42. de Koning HJ, van Ineveld BM, van Oortmarssen GJ, et al. Breast cancer screening and cost-effectiveness: policy alternatives, quality of life considerations and the possible impact of uncertain factors. Int J Cancer 1991 Oct 21; 49: 531–7

    Article  Google Scholar 

  43. de Koning HJ, Boer R, Warmerdam PG, et al. Quantitative interpretation of age-specific mortality reductions from the Swedish breast cancer-screening trials. J Natl Cancer Inst 1995 Aug 16; 87 (16): 1217–23

    PubMed  Article  Google Scholar 

  44. Eddy DM. Screening for breast cancer. Ann Intern Med 1989; 111 (5): 389–99

    PubMed  CAS  Google Scholar 

  45. Feldstein M, Zelen M. Inferring the natural time history of breast cancer: implications for tumor growth rate and early detection. Breast Cancer Res Treat 1984; 4: 3–10

    PubMed  CAS  Article  Google Scholar 

  46. Fett MJ. Computer modelling of the Swedish two county trial of mammographic screening and trade offs between participation and screening interval. J Med Screen 2001; 8 (1): 39–45

    PubMed  CAS  Article  Google Scholar 

  47. Fracheboud J, Groenewoud JH, Boer R, et al. Seventy-five years is an appropriate upper age limit for population-based mammography screening. Int J Cancer 2006 Apr 15; 118 (8): 2020–5

    PubMed  CAS  Article  Google Scholar 

  48. Fryback DG, Stout NK, Rosenberg MA, et al. The Wisconsin breast cancer epidemiology simulation model. J Natl Cancer Inst Monogr 2006; 36: 37–47

    PubMed  Article  Google Scholar 

  49. Hanin LG, Miller A, Zorin AV, et al. The University of Rochester model of breast cancer detection and survival. J Natl Cancer Inst Monogr 2006; 36: 66–78

    PubMed  Article  Google Scholar 

  50. Hsieh HJ, Chen TH, Chang SH. Assessing chronic disease progression using non-homogeneous exponential regression Markov models: an illustration using a selective breast cancer screening in Taiwan. Stat Med 2002 Nov 30; 21 (22): 3369–82

    PubMed  Article  Google Scholar 

  51. Hunter DJ, Drake SM, Shortt SE, et al. Simulation modeling of change to breast cancer detection age eligibility recommendations in Ontario, 2002–2021. Cancer Detect Prev 2004; 28 (6): 453–60

    PubMed  Article  Google Scholar 

  52. Jacobi CE, Jonker MA, Nagelkerke NJ, et al. Prevalence of family histories of breast cancer in the general population and the incidence of related seeking of health care [letter]. J Med Genet 2003 Jul; 40 (7): e83

    Article  Google Scholar 

  53. Jacobi CE, Nagelkerke NJ, van Houwelingen JH, et al. Breast cancer screening, outside the population-screening program, of women from breast cancer families without proven BRCA1/BRCA2 mutations: a simulation study. Cancer Epidemiol Biomarkers Prev 2006 Mar; 15 (3): 429–36

    PubMed  Article  Google Scholar 

  54. Jansen JT, Zoetelief J. MBS: a model for risk benefit analysis of breast cancer screening. Br J Radiol 1995 Feb; 68 (806): 141–9

    PubMed  CAS  Article  Google Scholar 

  55. Jansen JT, Zoetelief J. Assessment of lifetime gained as a result of mammographic breast cancer screening using a computer model. Br J Radiol 1997 Jun; 70 (834): 619–28

    PubMed  CAS  Google Scholar 

  56. Jansen JT, Zoetelief J. Optimisation of mammographic breast cancer screening using a computer simulation model. Eur J Radiol 1997 Feb; 24 (2): 137–44

    PubMed  CAS  Article  Google Scholar 

  57. Knox EG. Evaluation of a proposed breast cancer screening regimen. BMJ 1988 Sep 10; 297 (6649): 650–4

    PubMed  CAS  Article  Google Scholar 

  58. Koscielny S, Tubiana M, Valleron AJ. A simulation model of the natural history of human breast cancer. Br J Cancer 1985 Oct; 52 (4): 515–24

    PubMed  CAS  Article  Google Scholar 

  59. Lee S, Zelen M. A stochastic model for predicting the mortality of breast cancer. J Natl Cancer Inst Monogr 2006; 36: 79–86

    PubMed  Article  Google Scholar 

  60. Mandelblatt JS, Schechter CB, Yabroff KR, et al. Benefits and costs of interventions to improve breast cancer outcomes in African American women. J Clin Oncol 2004 Jul 1; 22 (13): 2554–66

    PubMed  Article  Google Scholar 

  61. Mandelblatt J, Schechter CB, Lawrence W, et al. The SPECTRUM population model of the impact of screening and treatment on U.S. breast cancer trends from 1975 to 2000: principles and practice of the model methods. J Natl Cancer Inst Monogr 2006; 36: 47–55

    PubMed  Article  Google Scholar 

  62. Manton KG, Stallard E. Demographics (1950–1987) of breast cancer in birth cohorts of older women. J Gerontol 1992 Nov; 47 Spec No: 32–42

    PubMed  Article  Google Scholar 

  63. Michaelson JS, Halpern E, Kopans DB. Breast cancer: computer simulation method for estimating optimal intervals for screening. Radiology 1999 Aug; 212 (2): 551–60

    PubMed  CAS  Google Scholar 

  64. Michaelson JS, Satija S, Moore R, et al. Estimates of breast cancer growth rate and sojourn time from screening database information. J Wom Imag 2003 Feb; 5 (1): 11–9

    Article  Google Scholar 

  65. Michaelson JS, Satija S, Moore R, et al. Estimates of the sizes at which breast cancers become detectable on mammographic and clinical grounds. J Wom Imag 2003 Feb; 5 (1): 3–10

    Article  Google Scholar 

  66. Myles JP, Nixon RM, Duffy SW, et al. Bayesian evaluation of breast cancer screening using data from two studies. Stat Med 2003 May 30; 22 (10): 1661–74

    PubMed  Article  Google Scholar 

  67. Okubo I, Glick H, Frumkin H, et al. Cost-effectiveness analysis of mass screening for breast cancer in Japan. Cancer 1991 Apr 15; 67 (8): 2021–9

    PubMed  CAS  Article  Google Scholar 

  68. Paci E, Boer R, Zappa M, et al. A model-based prediction of the impact on reduction in mortality by a breast cancer screening programme in the city of Florence, Italy. Eur J Cancer 1995; 31A (3): 348–53

    Article  Google Scholar 

  69. Plevritis SK. A mathematical algorithm that computes breast cancer sizes and doubling times detected by screening. Math Biosci 2001 Jun; 171 (2): 155–78

    PubMed  CAS  Article  Google Scholar 

  70. Plevritis SK, Kurian AW, Sigal BM, et al. Cost-effectiveness of screening BRCA1/2 mutation carriers with breast magnetic resonance imaging. JAMA 2006 May 24; 295 (20): 2374–84

    PubMed  CAS  Article  Google Scholar 

  71. Plevritis SK, Sigal BM, Salzman P, et al. A stochastic simulation model of U.S. breast cancer mortality trends from 1975 to 2000. J Natl Cancer Inst Monogr 2006; 36: 86–95

    PubMed  Article  Google Scholar 

  72. Plevritis SK, Salzman P, Sigal BM, et al. A natural history model of stage progression applied to breast cancer. Stat Med 2006; 26 (3): 581–95

    Article  Google Scholar 

  73. Shen Y, Huang X. Nonparametric estimation of asymptomatic duration from a randomized prospective cancer screening trial. Biometrics 2005 Dec; 61 (4): 992–9

    PubMed  Article  Google Scholar 

  74. Shen Y, Zelen M. Robust modeling in screening studies: estimation of sensitivity and preclinical sojourn time distribution. Biostatistics 2005 Oct; 6 (4): 604–14

    PubMed  Article  Google Scholar 

  75. Stout NK, Rosenberg MA, Trentham-Dietz A, et al. Retrospective cost-effectiveness analysis of screening mammography. J Natl Cancer Inst 2006 Jun 7; 98 (11): 774–82

    PubMed  Article  Google Scholar 

  76. Szeto KL, Devlin NJ. The cost-effectiveness of mammography screening: evidence from a microsimulation model for New Zealand. Health Policy 1996 Nov; 38 (2): 101–15

    PubMed  CAS  Article  Google Scholar 

  77. Tan SYGL, van Oortmarssen GJ, de Koning HJ, et al. The MISCAN-Fadia continuous tumor growth model for breast cancer. J Natl Cancer Inst Monogr 2006; 36: 56–65

    PubMed  Article  Google Scholar 

  78. Tubiana M, Koscielny S. The natural history of breast cancer: implications for a screening strategy. Int J Radiat Oncol Biol Phys 1990 Nov; 19 (5): 1117–20

    PubMed  CAS  Article  Google Scholar 

  79. van der Maas PJ, de Koning HJ, van Ineveld BM, et al. The cost-effectiveness of breast cancer screening. Int J Cancer 1989 Jun 15; 43 (6): 1055–60

    PubMed  Article  Google Scholar 

  80. van Oortmarssen GJ, Habbema JD, Lubbe JT, et al. A model-based analysis of the HIP project for breast cancer screening. Int J Cancer 1990 Aug 15; 46 (2): 207–13

    PubMed  Article  Google Scholar 

  81. van Oortmarssen GJ, Habbema JD, van der Maas PJ, et al. A model for breast cancer screening. Cancer 1990 Oct 1; 66 (7): 1601–12

    PubMed  Article  Google Scholar 

  82. Vervoort MM, Draisma G, Fracheboud J, et al. Trends in the usage of adjuvant systemic therapy for breast cancer in the Netherlands and its effect on mortality. Br J Cancer 2004 Jul 19; 91 (2): 242–7

    PubMed  CAS  Google Scholar 

  83. Berkhof J, de Bruijne MC, Zielinski GD, et al. Natural history and screening model for high-risk human papillomavirus infection, neoplasia and cervical cancer in the Netherlands. Int J Cancer 2005 Jun 10; 115 (2): 268–75

    PubMed  CAS  Article  Google Scholar 

  84. Berkhof J, de Bruijne MC, Zielinski GD, et al. Evaluation of cervical screening strategies with adjunct high-risk human papillomavirus testing for women with borderline or mild dyskaryosis. Int J Cancer 2006 Apr 1; 118 (7): 1759–68

    PubMed  CAS  Article  Google Scholar 

  85. Eddy DM. The frequency of cervical cancer screening: comparison of a mathematical model with empirical data. Cancer 1987 Sep 1; 60 (5): 1117–22

    PubMed  CAS  Article  Google Scholar 

  86. Eddy DM. Screening for cervical cancer. Ann Intern Med 1990 Aug 1; 113 (3): 214–26

    PubMed  CAS  Google Scholar 

  87. Goldie SJ, Weinstein MC, Kuntz KM, et al. The costs, clinical benefits, and cost-effectiveness of screening for cervical cancer in HIV-infected women. Ann Intern Med 1999 Jan 19; 130 (2): 97–107

    PubMed  CAS  Google Scholar 

  88. Goldie SJ, Kuhn L, Denny L, et al. Policy analysis of cervical cancer screening strategies in low-resource settings: clinical benefits and cost-effectiveness. JAMA 2001 Jun 27; 285 (24): 3107–15

    PubMed  CAS  Article  Google Scholar 

  89. Goldie SJ, Kim JJ, Wright TCJ. Cost-effectiveness of human papillomavirus DNA testing for cervical cancer screening in women aged 30 years or more. Obstet Gynecol 2004 Apr; 103 (4): 619–31

    PubMed  Article  Google Scholar 

  90. Goldie SJ, Kohli M, Grima D, et al. Projected clinical benefits and cost-effectiveness of a human papillomavirus 16/18 vaccine. J Natl Cancer Inst 2004 Apr 21; 96 (8): 604–15

    PubMed  Article  Google Scholar 

  91. Goldie SJ, Gaffikin L, Goldhaber-Fiebert JD, et al. Cost-effectiveness of cervical-cancer screening in five developing countries. N Engl J Med 2005 Nov 17; 353 (20): 2158–68

    PubMed  CAS  Article  Google Scholar 

  92. Gustafsson L, Adami HO. Natural history of cervical neoplasia: consistent results obtained by an identification technique. Br J Cancer 1989 Jul; 60 (1): 132–41

    PubMed  CAS  Article  Google Scholar 

  93. Gustafsson L, Adami HO. Cytologic screening for cancer of the uterine cervix in Sweden evaluated by identification and simulation. Br J Cancer 1990 Jun; 61 (6): 903–8

    PubMed  CAS  Article  Google Scholar 

  94. Gustafsson L, Adami HO. Optimization of cervical cancer screening. Cancer Causes Control 1992 Mar; 3 (2): 125–36

    PubMed  CAS  Article  Google Scholar 

  95. Gyrd-Hansen D, Holund B, Andersen P. A cost-effectiveness analysis of cervical cancer screening: health policy implications. Health Policy 1995 Oct; 34 (1): 35–51

    PubMed  CAS  Article  Google Scholar 

  96. Habbema JD, van Oortmarssen GJ, Lubbe JT, et al. Model building on the basis of Dutch cervical cancer screening data. Maturitas 1985 May; 7 (1): 11–20

    PubMed  CAS  Article  Google Scholar 

  97. Habbema JDF, Lubbe JTN, van Oortmarssen GJ, et al. A simulation approach to cost-effectiveness and cost-benefit calculations of screening for the early detection of disease. Eur J Oper Res 1987 May; 29 (2): 159–66

    Article  Google Scholar 

  98. Helfand M, O’Connor GT, Zimmer-Gembeck M, et al. Effect of the Clinical Laboratory Improvement Amendments of 1988 (CLIA ‘88) on the incidence of invasive cervical cancer. Med Care 1992 Dec; 30 (12): 1067–82

    PubMed  CAS  Article  Google Scholar 

  99. Kim JJ, Wright TCJ, Goldie SJ. Cost-effectiveness of alternative triage strategies for atypical squamous cells of undetermined significance. JAMA 2002 May 8; 287 (18): 2382–90

    PubMed  Article  Google Scholar 

  100. Kim JJ, Leung GM, Woo PP, et al. Cost-effectiveness of organized versus opportunistic cervical cytology screening in Hong Kong. J Public Health (Oxf) 2004 Jun; 26 (2): 130–7

    Article  Google Scholar 

  101. Knox EG. A simulation system for screening procedures. In: McLachlan G, editor. Future and present indicatives, problems and progress in medical care. 9th rev. Oxford: Nuffield Provincial Hospitals Trust, Oxford University Press, 1973: 17–55

    Google Scholar 

  102. Koong SL, Yen AM, Chen TH. Efficacy and cost-effectiveness of nationwide cervical cancer screening in Taiwan. J Med Screen 2006; 13 Suppl. 1: S44–7

    Google Scholar 

  103. Koopmanschap MA, Lubbe KT, van Oortmarssen GJ, et al. Economic aspects of cervical cancer screening. Soc Sci Med 1990; 30 (10): 1081–7

    PubMed  CAS  Article  Google Scholar 

  104. Kulasingam SL, Myers ER, Lawson HW, et al. Cost-effectiveness of extending cervical cancer screening intervals among women with prior normal pap tests. Obstet Gynecol 2006 Feb; 107 (2 Pt 1): 321–8

    PubMed  Article  Google Scholar 

  105. Mandelblatt JS, Lawrence WF, Gaffikin L, et al. Costs and benefits of different strategies to screen for cervical cancer in less-developed countries. J Natl Cancer Inst 2002 Oct 2; 94 (19): 1469–83

    PubMed  Article  Google Scholar 

  106. Mandelblatt J, Lawrence W, Yi B, et al. The balance of harms, benefits, and costs of screening for cervical cancer in older women. Arch Intern Med 2004 Feb 9; 164: 245–7

    Article  Google Scholar 

  107. Matsunaga G, Tsuji I, Sato S, et al. Cost-effective analysis of mass screening for cervical cancer in Japan. J Epidemiol 1997 Sep; 7 (3): 135–41

    PubMed  CAS  Article  Google Scholar 

  108. Office of Technology Assessment. The costs and effectiveness of screening for cervical cancer in elderly women: background paper, OTA-BP-H-65. Washington, DC: U.S. Congress, 1990

    Google Scholar 

  109. Parkin DM. A computer simulation model for the practical planning of cervical cancer screening programmes. Br J Cancer 1985 Apr; 51 (4): 551–68

    PubMed  CAS  Article  Google Scholar 

  110. Parkin DM, Moss SM. An evaluation of screening policies for cervical cancer in England and Wales using a computer simulation model. J Epidemiol Community Health 1986 Jun; 40 (2): 143–53

    PubMed  CAS  Article  Google Scholar 

  111. Radensky PW, Mango LJ. Interactive neural-network-assisted screening: an economic assessment. Acta Cytol 1998 Jan-Feb; 42 (1): 246–52

    PubMed  CAS  Article  Google Scholar 

  112. Sato S, Matunaga G, Tsuji I, et al. Determining the cost-effectiveness of mass screening for cervical cancer using common analytic models. Acta Cytol 1999 Nov-Dec; 43 (6): 1006–14

    PubMed  CAS  Article  Google Scholar 

  113. Sawaya GF, McConnell KJ, Kulasingam SL, et al. Risk of cervical cancer associated with extending the interval between cervical-cancer screenings. N Engl J Med 2003 Oct 16; 349 (16): 1501–9

    PubMed  CAS  Article  Google Scholar 

  114. Schechter CB. Cost-effectiveness of rescreening conventionally prepared cervical smears by PAPNET testing. Acta Cytol 1996 Nov-Dec; 40 (6): 1272–82

    PubMed  CAS  Article  Google Scholar 

  115. Sherlaw-Johnson C, Gallivan S, Jenkins D. Withdrawing low risk women from cervical screening programmes: mathematical modelling study. BMJ 1999 Feb 6; 318 (7180): 356–60

    PubMed  CAS  Article  Google Scholar 

  116. Sherlaw-Johnson C, Philips Z. An evaluation of liquid-based cytology and human papillomavirus testing within the UK Cervical Cancer Screening Programme. Br J Cancer 2004 Jul 5; 91 (1): 84–91

    PubMed  CAS  Article  Google Scholar 

  117. Shun-Zhang Y, Miller AB, Sherman GJ. Optimising the age, number of tests, and test interval for cervical screening in Canada. J Epidemiol Community Health 1982 Mar; 36 (1): 1–10

    PubMed  CAS  Article  Google Scholar 

  118. Sreenivas V, Prabhakar AK, Ravi R, et al. A simulation approach for estimating the loss of woman years due to cervical cancer and probability of developing cervical cancer. Neoplasma 1989; 36 (5): 623–7

    PubMed  CAS  Google Scholar 

  119. van Ballegooijen M, van den Akker-van Marle E, Patnick J, et al. Overview of important cervical cancer screening process values in European Union (EU) countries, and tentative predictions of the corresponding effectiveness and cost-effectiveness. Eur J Cancer 2000 Nov; 36 (17): 2177–88

    PubMed  Article  Google Scholar 

  120. van den Akker-van Marle ME, van Ballegooijen M, van Oortmarssen GJ, et al. Cost-effectiveness of cervical cancer screening: comparison of screening policies. J Natl Cancer Inst 2002 Feb 6; 94 (3): 193–204

    PubMed  Article  Google Scholar 

  121. Clemen RT, Lacke CJ. Analysis of colorectal cancer screening regimens. Health Care Manage Sci 2001 Dec; 4 (4): 257–67

    CAS  Article  Google Scholar 

  122. Frazier AL, Colditz GA, Fuchs CS, et al. Cost-effectiveness of screening for colorectal cancer in the general population. JAMA 2000 Oct 18; 284 (15): 1954–61

    PubMed  CAS  Article  Google Scholar 

  123. Gyrd-Hansen D, Søgaard J, Kronborg O. Analysis of screening data: colorectal cancer. Int J Epidemiol 1997 Dec; 26 (6): 1172–81

    PubMed  CAS  Article  Google Scholar 

  124. Gyrd-Hansen D, Søgaard J, Kronborg O. Colorectal cancer screening: efficiency and effectiveness. Health Econ 1998 Feb; 7 (1): 9–20

    PubMed  CAS  Article  Google Scholar 

  125. Haug U, Brenner H. A simulation model for colorectal cancer screening: potential of stool tests with various performance characteristics compared with screening colonoscopy. Cancer Epidemiol Biomarkers Prev 2005 Feb; 14 (2): 422–8

    PubMed  Article  Google Scholar 

  126. Khandker RK, Dulski JD, Kilpatrick JB, et al. A decision model and cost-effectiveness analysis of colorectal cancer screening and surveillance guidelines for average-risk adults. Int J Technol Assess Health Care 2000 Summer; 16 (3): 799–810

    PubMed  CAS  Article  Google Scholar 

  127. Ladabaum U, Chopra CL, Huang G, et al. Aspirin as an adjunct to screening for prevention of sporadic colorectal cancer: a cost-effectiveness analysis. Ann Intern Med 2001 Nov 6; 135 (9): 769–81

    PubMed  CAS  Google Scholar 

  128. Ladabaum U, Scheiman JM, Fendrick AM. Potential effect of cyclooxygenase-2-specific inhibitors on the prevention of colorectal cancer: a cost-effectiveness analysis. Am J Med 2003 May; 114 (7): 546–54

    PubMed  CAS  Article  Google Scholar 

  129. Lejeune C, Arveux P, Dancourt V, et al. A simulation model for evaluating the medical and economic outcomes of screening strategies for colorectal cancer. Eur J Cancer Prev 2003 Feb; 12 (1): 77–84

    PubMed  CAS  Article  Google Scholar 

  130. Lejeune C, Arveux P, Dancourt V, et al. Cost-effectiveness analysis of fecal occult blood screening for colorectal cancer. Int J Technol Assess Health Care 2004 Fall; 20 (4): 434–9

    PubMed  Article  Google Scholar 

  131. Loeve F, Boer R, van Oortmarssen GJ, et al. The MISCAN-COLON simulation model for the evaluation of colorectal cancer screening. Comput Biomed Res 1999 Feb; 32 (1): 13–33

    PubMed  CAS  Article  Google Scholar 

  132. Loeve F, Brown ML, Boer R, et al. Endoscopic colorectal cancer screening: a cost-saving analysis. J Natl Cancer Inst 2000 Apr 5; 92 (7): 557–63

    PubMed  CAS  Article  Google Scholar 

  133. Loeve F, Boer R, van Oortmarssen GJ, et al. Impact of systematic false-negative test results on the performance of faecal occult blood screening. Eur J Cancer 2001 May; 37 (7): 912–7

    PubMed  CAS  Article  Google Scholar 

  134. Neilson AR, Whynes DK. Cost-effectiveness of screening for colorectal cancer: a simulation model. IMA J Math Appl Med Biol 1995 Sep-Dec; 12 (3–4): 355–67

    PubMed  CAS  Article  Google Scholar 

  135. Ness RM, Holmes AM, Klein R, et al. Cost-utility of one-time colonoscopic screening for colorectal cancer at various ages. Am J Gastroenterol 2000 Jul; 95 (7): 1800–11

    PubMed  CAS  Article  Google Scholar 

  136. Song K, Fendrick AM, Ladabaum U. Fecal DNA testing compared with conventional colorectal cancer screening methods: a decision analysis. Gastroenterology 2004 May; 126 (5): 1270–9

    PubMed  Article  Google Scholar 

  137. Vijan S, Hwang EW, Hofer TP, et al. Which colon cancer screening test? A comparison of costs, effectiveness, and compliance. Am J Med 2001 Dec 1; 111 (8): 593–601

    PubMed  CAS  Article  Google Scholar 

  138. Wagner JL, Herdman RC, Wadhwa S. Cost effectiveness of colorectal cancer screening in the elderly. Ann Intern Med 1991 Nov 15; 115 (10): 807–17

    PubMed  CAS  Google Scholar 

  139. Wagner JL, Tunis S, Brown M, et al. Cost-effectiveness of colorectal cancer screening in average-risk adults. In: Young G, Rosen P, Levin B, editors. Prevention and early detection of colorectal cancer. Philadelphia (PA): Saunders, 1996: 321–56

    Google Scholar 

  140. Whynes DK, Neilson AR, Walker AR, et al. Faecal occult blood screening for colorectal cancer: is it cost-effective? Health Econ 1998 Feb; 7 (1): 21–9

    PubMed  CAS  Article  Google Scholar 

  141. Wong JM, Yen MF, Lai MS, et al. Progression rates of colorectal cancer by Dukes’ stage in a high-risk group: analysis of selective colorectal cancer screening. Cancer J 2004 May-Jun; 10 (3): 160–9

    PubMed  Article  Google Scholar 

  142. Yang KC, Liao CS, Chiu YH, et al. Colorectal cancer screening with faecal occult blood test within a multiple disease screening programme: an experience from Keelung, Taiwan. J Med Screen 2006; 13 Suppl. 1: S8–13

    CAS  Article  Google Scholar 

  143. Garside R, Pitt M, Somerville M, et al. Surveillance of Barrett’s oesophagus: exploring the uncertainty through systematic review, expert workshop and economic modelling. Health Technol Assess 2006 Mar; 10 (8): 1–142

    PubMed  CAS  Google Scholar 

  144. Davies R, Crabbe D, Roderick P, et al. A simulation to evaluate screening for Helicobacter pylori infection in the prevention of peptic ulcers and gastric cancers. Health Care Manage Sci 2002 Nov; 5 (4): 249–58

    Article  Google Scholar 

  145. Fendrick AM, Chernew ME, Hirth RA, et al. Clinical and economic effects of population-based Helicobacter pylori screening to prevent gastric cancer. Arch Intern Med 1999 Jan 25; 159 (2): 142–8

    PubMed  CAS  Article  Google Scholar 

  146. Roderick P, Davies R, Raftery J, et al. The cost-effectiveness of screening for Helicobacter pylori to reduce mortality and morbidity from gastric cancer and peptic ulcer disease: a discrete-event simulation model. Health Technol Assess 2003; 7 (6): 1–86

    PubMed  CAS  Google Scholar 

  147. Roderick P, Davies R, Raftery J, et al. Cost-effectiveness of population screening for Helicobacter pylori in preventing gastric cancer and peptic ulcer disease, using simulation. J Med Screen 2003; 10 (3): 148–56

    PubMed  CAS  Article  Google Scholar 

  148. Das P, Ng AK, Earle CC, et al. Computed tomography screening for lung cancer in Hodgkin’s lymphoma survivors: decision analysis and cost-effectiveness analysis. Ann Oncol 2006 May; 17 (5): 785–93

    PubMed  CAS  Article  Google Scholar 

  149. Flehinger BJ, Kimmel M. The natural history of lung cancer in a periodically screened population. Biometrics 1987 Mar; 43 (1): 127–44

    PubMed  CAS  Article  Google Scholar 

  150. Flehinger BJ, Kimmel M, Melamed MR. Natural history of adenocarcinoma-large cell carcinoma of the lung: conclusions from screening programs in New York and Baltimore. J Natl Cancer Inst 1988 May 4; 80 (5): 337–44

    PubMed  CAS  Article  Google Scholar 

  151. Flehinger BJ, Kimmel M, Polyak T, et al. Screening for lung cancer: the Mayo Lung Project revisited. Cancer 1993 Sep 1; 72 (5): 1573–80

    PubMed  CAS  Article  Google Scholar 

  152. Gorlova OY, Kimmel M, Henschke C. Modeling of long-term screening for lung carcinoma. Cancer 2001 Sep 15; 92 (6): 1531–40

    PubMed  CAS  Article  Google Scholar 

  153. Mahadevia PJ, Fleisher LA, Frick KD, et al. Lung cancer screening with helical computed tomography in older adult smokers: a decision and cost-effectiveness analysis. JAMA 2003 Jan 15; 289 (3): 313–22

    PubMed  Article  Google Scholar 

  154. Marshall D, Simpson KN, Earle CC, et al. Potential cost-effectiveness of one-time screening for lung cancer (LC) in a high risk cohort. Lung Cancer 2001 Jun; 32 (3): 227–36

    PubMed  CAS  Article  Google Scholar 

  155. Yamaguchi N, Tamura Y, Sobue T, et al. Evaluation of cancer prevention strategies by computerized simulation model: an approach to lung cancer. Cancer Causes Control 1991 May; 2 (3): 147–55

    PubMed  CAS  Article  Google Scholar 

  156. Yamaguchi N, Mizuno S, Akiba S, et al. A 50-year projection of lung cancer deaths among Japanese males and potential impact evaluation of anti-smoking measures and screening using a computerized simulation model. Jpn J Cancer Res 1992 Mar; 83 (3): 251–7

    PubMed  CAS  Article  Google Scholar 

  157. Yamaguchi N, Tamura Y, Sobue T, et al. Evaluation of cancer prevention strategies by computerized simulation model: methodological issues. Environ Health Perspect 1994 Nov; 102 Suppl. 8: 67–71

    PubMed  Article  Google Scholar 

  158. Girgis A, Clarke P, Burton RC, et al. Screening for melanoma by primary health care physicians: a cost-effectiveness analysis. J Med Screen 1996; 3 (1): 47–53

    PubMed  CAS  Google Scholar 

  159. Downer MC, Jullien JA, Speight PM. An interim determination of health gain from oral cancer and precancer screening: 2. Developing a model of population screening. Community Dent Health 1997 Dec; 14 (4): 227–32

    PubMed  CAS  Google Scholar 

  160. Myers ER, Havrilesky LJ, Kulasingam SL, et al. Genomic tests for ovarian cancer detection and management. Evid Rep Technol Assess (Full Rep) 2006 Oct; (145): 1–100

    Google Scholar 

  161. Skates SJ, Singer DE. Quantifying the potential benefit of CA 125 screening for ovarian cancer. J Clin Epidemiol 1991; 44 (4–5): 365–80

    PubMed  CAS  Article  Google Scholar 

  162. Skates SJ, Pauler DK, Jacobs IJ. Screening based on the risk of cancer calculation from Bayesian hierarchical change point and mixture models of longitudinal markers. J Am Stat Assoc 2001; 96 (454): 429–39

    Article  Google Scholar 

  163. Cowen ME, Chartrand M, Weitzel WF. A Markov model of the natural history of prostate cancer. J Clin Epidemiol 1994 Jan; 47 (1): 3–21

    PubMed  CAS  Article  Google Scholar 

  164. Draisma G, Boer R, Otto SJ, et al. Lead times and overdetection due to prostate-specific antigen screening: estimates from the European Randomized Study of Screening for Prostate Cancer. J Natl Cancer Inst 2003 Jun 18; 95 (12): 868–78

    PubMed  Article  Google Scholar 

  165. Draisma G, De Koning HJ. MISCAN: estimating lead-time and over-detection by simulation. BJU Int 2003 Dec; 92 Suppl. 2: 106–11

    PubMed  Article  Google Scholar 

  166. Draisma G, Postma R, Schroder FH, et al. Gleason score, age and screening: modeling dedifferentiation in prostate cancer. Int J Cancer 2006 Nov 15; 119 (10): 2366–71

    PubMed  CAS  Article  Google Scholar 

  167. Etzioni R, Cha R, Cowen ME. Serial prostate specific antigen screening for prostate cancer: a computer model evaluates competing strategies. J Urol 1999 Sep; 162 (3 Pt 1): 741–8

    PubMed  CAS  Article  Google Scholar 

  168. Etzioni R, Legler JM, Feuer EJ, et al. Cancer surveillance series: interpreting trends in prostate cancer. Part III: quantifying the link between population prostate-specific antigen testing and recent declines in prostate cancer mortality. J Natl Cancer Inst 1999 Jun 16; 91 (12): 1033–9

    PubMed  CAS  Article  Google Scholar 

  169. Etzioni R, Penson DF, Legler JM, et al. Overdiagnosis due to prostate-specific antigen screening: lessons from U.S. prostate cancer incidence trends. J Natl Cancer Inst 2002 Jul 3; 94 (13): 981–90

    PubMed  Article  Google Scholar 

  170. Krahn MD, Mahoney JE, Eckman MH, et al. Screening for prostate cancer: a decision analytic view. JAMA 1994 Sep 14; 272 (10): 773–80

    PubMed  CAS  Article  Google Scholar 

  171. Parker C, Muston D, Melia J, et al. A model of the natural history of screen-detected prostate cancer, and the effect of radical treatment on overall survival. Br J Cancer 2006 May 22; 94 (10): 1361–8

    PubMed  CAS  Article  Google Scholar 

  172. Ross KS, Carter HB, Pearson JD, et al. Comparative efficiency of prostate-specific antigen screening strategies for prostate cancer detection. JAMA 2000 Sep 20; 284 (11): 1399–405

    PubMed  CAS  Article  Google Scholar 

  173. Ross KS, Guess HA, Carter HB. Estimation of treatment benefits when PSA screening for prostate cancer is discontinued at different ages. Urology 2005 Nov; 66 (5): 1038–42

    PubMed  Article  Google Scholar 

  174. Tsodikov A, Szabo A, Wegelin J. A population model of prostate cancer incidence. Stat Med 2006 Aug 30; 25 (16): 2846–66

    PubMed  CAS  Article  Google Scholar 

  175. Kimmel M, Flehinger BJ. Nonparametric estimation of the size-metastasis relationship in solid cancers. Biometrics 1991 Sep; 47 (3): 987–1004

    PubMed  CAS  Article  Google Scholar 

  176. Wang PE, Wang TT, Chiu YH, et al. Evolution of multiple disease screening in Keelung: a model for community involvement in health interventions? J Med Screen 2006; 13 Suppl. 1: S54–8

    CAS  Article  Google Scholar 

  177. Garrison LP. The ISPOR good practice modeling principles: a sensible approach. Be transparent, be reasonable. Value Health 2003 Jan-Feb; 6 (1): 6–8

    PubMed  Article  Google Scholar 

  178. Kim JJ, Kuntz KM, Stout NK, et al. Multiparameter calibration of a natural history model of cervical cancer. Am J Epidemiol 2007 July 15, 2007; 166 (2): 137–50

    Article  Google Scholar 

  179. Goldhaber-Fiebert JD, Stout NK, Ortendahl J, et al. Modeling human papillomavirus and cervical cancer in the United States for analyses of screening and vaccination. Popul Health Metr 2007; 5 (1): 11 [online]. Available from URL: http://www.pophealthmetrics.com/content/5/1/11 [Accessed 2009 Jul 1]

    PubMed  Article  Google Scholar 

  180. Weinstein MC, O’Brien B, Hornberger J, et al. Principles of good practice for decision analytic modeling in health-care evaluation. Report of the ISPOR task force on good research practices: modeling studies. Value Health 2003 Jan-Feb; 6 (1): 9–17

    PubMed  Article  Google Scholar 

Download references

Acknowledgements

The authors gratefully acknowledge the support of Drs Eric Feuer and Karen Kuntz and members of the NCI Cancer Intervention and Surveillance Modeling Network. This work was supported in part by grants from the National Cancer Institute: F32 CA125984 (Natasha K. Stout), R25 CA92203 (Amy B. Knudsen), K99 126147 (Pamela M. McMahon, Chung Yin Kong) and R01 97337 (G. Scott Gazelle, Pamela M. McMahon, Chung Yin Kong). The funding agreements ensured the authors’ independence in designing the study, collecting, analysing and interpreting the data and writing and publishing the report. An earlier version of this work was presented at the 2007 Society for Medical Decision Making Annual Meeting.

Author information

Authors and Affiliations

  1. Department of Ambulatory Care and Prevention, Harvard Medical School/Harvard Pilgrim Health Care, 133 Brookline Ave, 6th Floor, Boston, Massachusetts, 02215, USA

    Natasha K. Stout

  2. Program in Health Decision Science, Department of Health Policy and Management, Harvard School of Public Health, Boston, Massachusetts, USA

    Natasha K. Stout & G. Scott Gazelle

  3. Institute for Technology Assessment, Massachusetts General Hospital, Boston, Massachusetts, USA

    Amy B. Knudsen, Chung Yin Kong, Pamela M. McMahon & G. Scott Gazelle

  4. Department of Radiology, Massachusetts General Hospital, Boston, Massachusetts, USA

    Amy B. Knudsen, Chung Yin Kong, Pamela M. McMahon & G. Scott Gazelle

Authors
  1. Natasha K. Stout
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Amy B. Knudsen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Chung Yin Kong
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Pamela M. McMahon
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. G. Scott Gazelle
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Natasha K. Stout.

Electronic supplementary material

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Stout, N.K., Knudsen, A.B., Kong, C.Y. et al. Calibration Methods Used in Cancer Simulation Models and Suggested Reporting Guidelines. Pharmacoeconomics 27, 533–545 (2009). https://doi.org/10.2165/11314830-000000000-00000

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.2165/11314830-000000000-00000

Keywords

  • Calibration Method
  • Comparative Effectiveness Research
  • Calibration Target
  • Calibration Protocol
  • Unobservable Parameter