Skip to main content
SpringerLink
Log in

Bayesian Calibration, Validation and Uncertainty Quantification for Predictive Modelling of Tumour Growth: A Tutorial

  • Research Methods Article
  • Published:
Bulletin of Mathematical Biology Aims and scope Submit manuscript

Abstract

In this work, we present a pedagogical tumour growth example, in which we apply calibration and validation techniques to an uncertain, Gompertzian model of tumour spheroid growth. The key contribution of this article is the discussion and application of these methods (that are not commonly employed in the field of cancer modelling) in the context of a simple model, whose deterministic analogue is widely known within the community. In the course of the example, we calibrate the model against experimental data that are subject to measurement errors, and then validate the resulting uncertain model predictions. We then analyse the sensitivity of the model predictions to the underlying measurement model. Finally, we propose an elementary learning approach for tuning a threshold parameter in the validation procedure in order to maximize predictive accuracy of our validated model.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15

Similar content being viewed by others

Notes

  1. That is, we model observable outcomes conditionally on parameters which themselves are given a probabilistic description in terms of further parameters known as hyperparameters.

  2. While repeated experiments are not available in the patient-specific clinical setting, we may view this as data from multiple individual patients in a similar population.

References

  • Achilleos A, Loizides C, Stylianopoulos T, Mitsis GD (2013) Multi-process dynamic modeling of tumor-specific evolution. In: 13th IEEE conference on bioinformatics and bioengineering, pp. 1–4. doi:10.1109/BIBE.2013.6701614

  • Achilleos A, Loizides C, Hadjiandreou M, Stylianopoulos T, Mitsis GD (2014) Multiprocess dynamic modeling of tumor evolution with Bayesian tumor-specific predictions. Ann Biomed Eng 42(5):1095–1111. doi:10.1007/s10439-014-0975-y

    Article  Google Scholar 

  • Aguilar O, Allmaras M, Bangerth W, Tenorio L (2015) Statistics of parameter estimates: a concrete example. SIAM Rev 57(1):131–149

    Article  MathSciNet  MATH  Google Scholar 

  • Akaike H (1973) Information theory and an extension of the maximum likelihood principle. In: Petrov BN, Caski F (eds) Proceedings of the second international symposium on information theory. Akademiai Kiado, Budapest

  • Allmaras M, Bangerth W, Linhart JM, Polanco J, Wang F, Wang K, Webster J, Zedler S (2013) Estimating parameters in physical models through Bayesian inversion: a complete example. SIAM Rev 55(1):149–167

    Article  MathSciNet  MATH  Google Scholar 

  • Andrieu C, Thoms J (2008) A tutorial on adaptive MCMC. Stat Comput 18(4):343–373. doi:10.1007/s11222-008-9110-y

    Article  MathSciNet  Google Scholar 

  • Arlot S, Celisse A (2010) A survey of cross-validation procedures for model selection. Stat Surv 4:40–79. doi:10.1214/09-SS054

    Article  MathSciNet  MATH  Google Scholar 

  • ASME (2006) ASME V&V 10-2006: guide for verification and validation in computational solid mechanics. American Society of Mechanical Engineers, New York

  • ASME (2009) ASME V&V 20-2009: standard for verification and validation in computational fluid dynamics. American Society of Mechanical Engineers, New York

  • ASME (2012) ASME V&V 10.1-2012: an illustration of the concepts of verification and validation in computational solid mechanics. American Society of Mechanical Engineers, New York

  • Babuška I, Tempone R, Zouraris GE (2004) Galerkin finite element approximations of stochastic elliptic partial differential equations. SIAM J Numer Anal 42(2):800–825

    Article  MathSciNet  MATH  Google Scholar 

  • Babuška I, Nobile F, Tempone R (2007) A stochastic collocation method for elliptic partial differential equations with random input data. SIAM J Numer Anal 45(3):1005–1034

    Article  MathSciNet  MATH  Google Scholar 

  • Babuška I, Nobile F, Tempone R (2008) A systematic approach to model validation based on Bayesian updates and prediction related rejection criteria. Comput Methods Appl Mech Eng 197:2517–2539

    Article  MATH  Google Scholar 

  • Baldock AL, Rockne RC, Boone AD, Neal ML, Hawkins-Daarud A, Corwin DM, Bridge CA, Guyman LA, Trister AD, Mrugala MM, Rockhill JK, Swanson KR (2013) From patient-specific mathematical neuro-oncology to precision medicine. Front Oncol 3:62

    Article  Google Scholar 

  • Bayarri MJ, Berger J, Paulo R, Sacks J, Cafeo JA, Cavendish JC, Lin C, Tu J (2007) A framework for validation of computer models. Technometrics 49:138–154

    Article  MathSciNet  Google Scholar 

  • Bellman R, Åström KJ (1970) On structural identifiability. Math Biosci 7(3):329–339. ISSN 0025-5564. doi:10.1016/0025-5564(70)90132-X

  • Berger JO (1984) The robust Bayesian viewpoint (with discussion). In: Kadane JB (ed) Robustness of Bayesian analyses. North-Holland, Amsterdam, pp 63–144

    Google Scholar 

  • Chen W, Wong C, Vosburgh E, Levine AJ, Foran DJ, Xu EY (2014) High-throughput image analysis of tumor spheroids: a user-friendly software application to measure the size of spheroids automatically and accurately. J Vis Exp (89). doi:10.3791/51639. http://www.jove.com/video/51639/high-throughput-image-analysis-tumor-spheroids-user-friendly-software

  • Chib S, Greenberg E (1995) Understanding the Metropolis–Hastings algorithm. Am Stat 49(4):327–335

    Google Scholar 

  • Cobelli C, DiStefano JJ (1980) Parameter and structural identifiability concepts and ambiguities: a critical review and analysis. Am J Physiol Regul Integr Comp Physiol 239(1):R7–R24, ISSN 0363-6119

  • Connor AJ (2016) Calibration, validation and uncertainty quantification. doi:10.6084/m9.figshare.3406876. Supporting data and code

  • Gammon K (2012) Mathematical modelling: forecasting cancer. Nature 491:66–67

    Article  Google Scholar 

  • Gelman A, Carlin JB, Stern HS, Dunson DB, Vehtari A, Rubin DB (2014a) Bayesian data analysis, 3rd edn. CRC Press, Boca Raton

    MATH  Google Scholar 

  • Gelman A, Hwang J, Vehtari A (2014b) Understanding predictive information criteria for Bayesian models. Stat Comput 24(6):997–1016. doi:10.1007/s11222-013-9416-2

    Article  MathSciNet  MATH  Google Scholar 

  • Ghanem R (1999) Stochastic finite elements with multiple random non-Gaussian properties. ASCE J Eng Math 125(1):26–40

    Article  Google Scholar 

  • Ghanem R, Red-Horse J (1999) Propogation of probabilistic uncertainity in complex physical systems using a stochastic finite element approach. Phys D 133:137–144

    Article  MathSciNet  MATH  Google Scholar 

  • Ghanem RG, Spanos PD (1991) Stochastic finite elements: a spectral approach. Springer, Berlin

    Book  MATH  Google Scholar 

  • Gilks WR, Richardson S, Spiegelhalter DJ (eds) (1996) Practical Markov chain Monte Carlo. Chapman and Hall, London

    MATH  Google Scholar 

  • Gompertz G (1825) On the nature of the function expressive of the law of human mortality, and on the new mode of determining the value of life contingencies. Philos Trans R Soc Lond 115:513–585

    Article  Google Scholar 

  • Hastings WK (1970) Monte Carlo sampling methods using Markov chains and their applications. Biometrika 57:97–109

    Article  MathSciNet  MATH  Google Scholar 

  • Hawkins-Daarud A, Prudhomme S, van der Zee KG, Oden JT (2013) Bayesian calibration, validation, and uncertainty quantification of diffuse interface models of tumour growth. J Math Biol 67:1457–1485

    Article  MathSciNet  MATH  Google Scholar 

  • Higdon D, Kennedy M, Cavendish JC, Cafeo JA, Ryne RD (2005) Combining field data and computer simulations for calibration and prediction. SIAM J Sci Comput 2(26):448–466

    MathSciNet  MATH  Google Scholar 

  • Insua DR, Ruggeri F (eds) (2000) Robust Bayesian analysis, volume 152 of lecture notes in statistics. Springer, New York

    Google Scholar 

  • Kaipio J, Somersalo E (2006) Statistical and computational inverse problems. Springer, New York

    MATH  Google Scholar 

  • Kennedy MC, O’Hagan A (2001) Bayesian calibration of computer models. J R Stat Soc Ser B (Stat Methodol) 63(4):425–464. doi:10.1111/1467-9868.00294

    Article  MathSciNet  MATH  Google Scholar 

  • Kullback S, Leibler RA (1951) On information and sufficiency. Ann Math Stat 22(1):79–86. doi:10.1214/aoms/1177729694

    Article  MathSciNet  MATH  Google Scholar 

  • Laird AK (1964) Dynamics of tumor growth. Br J Cancer 18:490–502

    Article  Google Scholar 

  • Lopes HF, Tobias JL (2011) Confronting prior convictions: on issues of prior sensitivity and likelihood robustness in Bayesian analysis. Annu Rev Econ 3:107–131. doi:10.1146/annurev-economics-111809-125134

    Article  Google Scholar 

  • Massey FJ (1951) The Kolmogorov–Smirnov test for goodness of fit. J Am Stat Assoc 46(253):68–78

    Article  MATH  Google Scholar 

  • Metropolis N, Ulam S (1949) The Monte Carlo method. J Am Stat Assoc 44:335–341

    Article  MATH  Google Scholar 

  • Metropolis N, Rosenbluth AW, Rosenbluth MN, Teller AH, Teller E (1953) Equation of state calculations by fast computing machines. J Chem Phys 21:1087–1092

    Article  Google Scholar 

  • Miller LH (1956) Table of percentage points of Kolmogorov statistics. J Am Stat Assoc 51(273):111–121

    Article  MATH  Google Scholar 

  • Najm HN (2009) Uncertainty quantification and polynomial chaos techniques in computational fluid dynamics. Annu Rev Fluid Mech 41:35–52

    Article  MathSciNet  MATH  Google Scholar 

  • Nobile F, Tempone R, Webster C (2008a) A sparse grid stochastic collocation method for elliptic partial differential equations with random input data. SIAM J Numer Anal 46(5):2309–2345

    Article  MathSciNet  MATH  Google Scholar 

  • Nobile F, Tempone R, Webster C (2008b) An anisotropic sparse grid stochastic collocation method for elliptic partial differential equations with random input data. SIAM J Numer Anal 46(5):2411–2442

    Article  MathSciNet  MATH  Google Scholar 

  • Norris JR (1998) Markov chains. Number no. 2008 in Cambridge Series in Statistical and Probabilistic Mathematics. Cambridge University Press. ISBN 9780521633963

  • NRC (2012) Assessing the reliability of complex models: mathematical and statistical foundations of verification, validation, and uncertainty quantification. The National Academies Press. ISBN 9780309256346

  • Oberkampf WL, Barone MF (2006) Measures of agreement between computation and experiment: validation metrics. J Comput Phys 217(1):5–36. doi:10.1016/j.jcp.2006.03.037

    Article  MATH  Google Scholar 

  • Oberkampf WL, Roy CJ (2010) Verification and validation in scientific computing. Cambridge University Press, Cambridge

    Book  MATH  Google Scholar 

  • Oberkampf WL, Trucano TG, Hirsch C (2004) Verification, validation, and predictive capability in computational engineering and physics. Appl Mech Rev 57(5):345–384. doi:10.1115/1.1767847

    Article  Google Scholar 

  • Oden JT, Moser R, Ghattas O (2010a) Computer predictions with quantified uncertainty, part i. SIAM News 43(9):1–3

    Google Scholar 

  • Oden JT, Moser R, Ghattas O (2010b) Computer predictions with quantified uncertainty, part ii. SIAM News 43(10):1–3

    Google Scholar 

  • Pathmanathan P, Gray RA (2013) Ensuring reliability of safety-critical clinical applications of computational cardiac models. Front Physiol. doi:10.3389/fphys.2013.00358

    Google Scholar 

  • Pericchi LR, Peréz ME (1994) Posterior robustness with more than one sampling model. J Stat Plan Inference 40:279–294

    Article  MathSciNet  MATH  Google Scholar 

  • Raue A, Kreutz C, Maiwald T, Bachmann J, Schilling M, Klingmüller U, Timmer J (2009) Structural and practical identifiability analysis of partially observed dynamical models by exploiting the profile likelihood. Bioinformatics 25(15):1923–1929. doi:10.1093/bioinformatics/btp358

    Article  Google Scholar 

  • Roache PJ (2009) Fundamentals of verification and validation, 2nd edn. Hermosa Publishers, New Mexico

    Google Scholar 

  • Savage N (2012) Modelling: computing cancer. Nature 491:62–63

    Article  Google Scholar 

  • Schervish MJ (1995) Theory of statistics. Springer series in statistics. Springer, New York

    MATH  Google Scholar 

  • Simpson DP, Rue H, Martins TG, Riebler A, Sørbye SH (2014) Penalising model component complexity: a principled, practical approach to constructing priors. In arXiv preprint arXiv:1403.4630 [stat.ME]

  • Spiegelhalter DJ, Best NG, Carlin BP, van der Linde A (2002) Bayesian measures of model complexity and fit (with discussion). J R Stat Soc B 64(4):583–639

    Article  MathSciNet  MATH  Google Scholar 

  • Tarantola A (2005) Inverse problem theory and methods for model parameter estimation. SIAM, Philadelphia

    Book  MATH  Google Scholar 

  • Tatang MA, Pan W, Prinn RG, McRae GJ (1997) An efficient method for parametric uncertainity analysis of numerical geophysical models. J Geophys Res 102(D18):21925–21932

    Article  Google Scholar 

  • Watanabe S (2010) Asymptotic equivalence of Bayes cross validation and widely applicable information criterion in singular learning theory. J Mach Learn Res 11:3571–3594

    MathSciNet  MATH  Google Scholar 

  • Xiu D, Hesthaven JS (2005) High-order collocation methods for differential equations with random inputs. SIAM J Sci Comput 27(3):1118–1139

    Article  MathSciNet  MATH  Google Scholar 

  • Xiu D, Karniadakis GE (2002) Modeling uncertainty in steady state diffusion problems via generalized polynomial chaos. Comput Methods Appl Mech Eng 191:4927–4948

    Article  MathSciNet  MATH  Google Scholar 

  • Xiu D, Karniadakis GE (2003) Modeling uncertainty in flow simulations via generalized polynomial chaos. J Comput Phys 187:137–167

    Article  MathSciNet  MATH  Google Scholar 

  • Yankeelov TE, Atuegwu N, Hormuth D, WeisJA, Barnes SL, Miga MI, Rericha EC, Quaranta V (2013) Clinically relevant modeling of tumor growth and treatment response. Sci Transl Med 5(187). ISSN 1946-6234. doi:10.1126/scitranslmed.3005686

Download references

Acknowledgements

J. Collis and M. E. Hubbard acknowledge the support of EPSRC Grant Number EP/K039342/1. This project has received funding from the European Unions Seventh Framework Programme for research, technological development and demonstration under Grant Agreement No. 600841.

Author information

Author notes

    Authors and Affiliations

    1. School of Mathematical Sciences, University of Nottingham, Nottingham, UK

      Joe Collis & Matthew E. Hubbard

    2. Mathematical Institute, University of Oxford, Oxford, UK

      Anthony J. Connor, Marcin Paczkowski & Helen M. Byrne

    3. Gray Institute for Radiation Oncology and Biology, University of Oxford, Oxford, UK

      Pavitra Kannan

    4. Department of Computer Science, University of Oxford, Oxford, UK

      Joe Pitt-Francis

    Authors
    1. Joe Collis
      View author publications

      You can also search for this author in PubMed Google Scholar

    2. Anthony J. Connor
      View author publications

      You can also search for this author in PubMed Google Scholar

    3. Marcin Paczkowski
      View author publications

      You can also search for this author in PubMed Google Scholar

    4. Pavitra Kannan
      View author publications

      You can also search for this author in PubMed Google Scholar

    5. Joe Pitt-Francis
      View author publications

      You can also search for this author in PubMed Google Scholar

    6. Helen M. Byrne
      View author publications

      You can also search for this author in PubMed Google Scholar

    7. Matthew E. Hubbard
      View author publications

      You can also search for this author in PubMed Google Scholar

    Corresponding author

    Correspondence to Joe Collis.

    Additional information

    Joe Collis and Anthony J. Connor have contributed equally to this work.

    Appendix: Measurement Times

    Appendix: Measurement Times

    The times at which measurements were taken in the experiments described in Sect. 2.2 are given in Table 6.

    Table 6 Times at which measurements of the spheroids were taken measured after an initial seed of 2000 tumour cells per spheroid were implanted at \(t=0\)
    Full size table

    Rights and permissions

    Reprints and permissions

    About this article

    Check for updates. Verify currency and authenticity via CrossMark

    Cite this article

    Collis, J., Connor, A.J., Paczkowski, M. et al. Bayesian Calibration, Validation and Uncertainty Quantification for Predictive Modelling of Tumour Growth: A Tutorial. Bull Math Biol 79, 939–974 (2017). https://doi.org/10.1007/s11538-017-0258-5

    Download citation

    • Received:

    • Accepted:

    • Published:

    • Issue Date:

    • DOI: https://doi.org/10.1007/s11538-017-0258-5

    Keywords

    Search

    Navigation