Bayesian Calibration, Validation and Uncertainty Quantification for Predictive Modelling of Tumour Growth: A Tutorial

Abstract

In this work, we present a pedagogical tumour growth example, in which we apply calibration and validation techniques to an uncertain, Gompertzian model of tumour spheroid growth. The key contribution of this article is the discussion and application of these methods (that are not commonly employed in the field of cancer modelling) in the context of a simple model, whose deterministic analogue is widely known within the community. In the course of the example, we calibrate the model against experimental data that are subject to measurement errors, and then validate the resulting uncertain model predictions. We then analyse the sensitivity of the model predictions to the underlying measurement model. Finally, we propose an elementary learning approach for tuning a threshold parameter in the validation procedure in order to maximize predictive accuracy of our validated model.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15

Notes

  1. 1.

    That is, we model observable outcomes conditionally on parameters which themselves are given a probabilistic description in terms of further parameters known as hyperparameters.

  2. 2.

    While repeated experiments are not available in the patient-specific clinical setting, we may view this as data from multiple individual patients in a similar population.

References

  1. Achilleos A, Loizides C, Stylianopoulos T, Mitsis GD (2013) Multi-process dynamic modeling of tumor-specific evolution. In: 13th IEEE conference on bioinformatics and bioengineering, pp. 1–4. doi:10.1109/BIBE.2013.6701614

  2. Achilleos A, Loizides C, Hadjiandreou M, Stylianopoulos T, Mitsis GD (2014) Multiprocess dynamic modeling of tumor evolution with Bayesian tumor-specific predictions. Ann Biomed Eng 42(5):1095–1111. doi:10.1007/s10439-014-0975-y

    Article  Google Scholar 

  3. Aguilar O, Allmaras M, Bangerth W, Tenorio L (2015) Statistics of parameter estimates: a concrete example. SIAM Rev 57(1):131–149

    MathSciNet  Article  MATH  Google Scholar 

  4. Akaike H (1973) Information theory and an extension of the maximum likelihood principle. In: Petrov BN, Caski F (eds) Proceedings of the second international symposium on information theory. Akademiai Kiado, Budapest

  5. Allmaras M, Bangerth W, Linhart JM, Polanco J, Wang F, Wang K, Webster J, Zedler S (2013) Estimating parameters in physical models through Bayesian inversion: a complete example. SIAM Rev 55(1):149–167

    MathSciNet  Article  MATH  Google Scholar 

  6. Andrieu C, Thoms J (2008) A tutorial on adaptive MCMC. Stat Comput 18(4):343–373. doi:10.1007/s11222-008-9110-y

    MathSciNet  Article  Google Scholar 

  7. Arlot S, Celisse A (2010) A survey of cross-validation procedures for model selection. Stat Surv 4:40–79. doi:10.1214/09-SS054

    MathSciNet  Article  MATH  Google Scholar 

  8. ASME (2006) ASME V&V 10-2006: guide for verification and validation in computational solid mechanics. American Society of Mechanical Engineers, New York

  9. ASME (2009) ASME V&V 20-2009: standard for verification and validation in computational fluid dynamics. American Society of Mechanical Engineers, New York

  10. ASME (2012) ASME V&V 10.1-2012: an illustration of the concepts of verification and validation in computational solid mechanics. American Society of Mechanical Engineers, New York

  11. Babuška I, Tempone R, Zouraris GE (2004) Galerkin finite element approximations of stochastic elliptic partial differential equations. SIAM J Numer Anal 42(2):800–825

    MathSciNet  Article  MATH  Google Scholar 

  12. Babuška I, Nobile F, Tempone R (2007) A stochastic collocation method for elliptic partial differential equations with random input data. SIAM J Numer Anal 45(3):1005–1034

    MathSciNet  Article  MATH  Google Scholar 

  13. Babuška I, Nobile F, Tempone R (2008) A systematic approach to model validation based on Bayesian updates and prediction related rejection criteria. Comput Methods Appl Mech Eng 197:2517–2539

    Article  MATH  Google Scholar 

  14. Baldock AL, Rockne RC, Boone AD, Neal ML, Hawkins-Daarud A, Corwin DM, Bridge CA, Guyman LA, Trister AD, Mrugala MM, Rockhill JK, Swanson KR (2013) From patient-specific mathematical neuro-oncology to precision medicine. Front Oncol 3:62

    Article  Google Scholar 

  15. Bayarri MJ, Berger J, Paulo R, Sacks J, Cafeo JA, Cavendish JC, Lin C, Tu J (2007) A framework for validation of computer models. Technometrics 49:138–154

    MathSciNet  Article  Google Scholar 

  16. Bellman R, Åström KJ (1970) On structural identifiability. Math Biosci 7(3):329–339. ISSN 0025-5564. doi:10.1016/0025-5564(70)90132-X

  17. Berger JO (1984) The robust Bayesian viewpoint (with discussion). In: Kadane JB (ed) Robustness of Bayesian analyses. North-Holland, Amsterdam, pp 63–144

    Google Scholar 

  18. Chen W, Wong C, Vosburgh E, Levine AJ, Foran DJ, Xu EY (2014) High-throughput image analysis of tumor spheroids: a user-friendly software application to measure the size of spheroids automatically and accurately. J Vis Exp (89). doi:10.3791/51639. http://www.jove.com/video/51639/high-throughput-image-analysis-tumor-spheroids-user-friendly-software

  19. Chib S, Greenberg E (1995) Understanding the Metropolis–Hastings algorithm. Am Stat 49(4):327–335

    Google Scholar 

  20. Cobelli C, DiStefano JJ (1980) Parameter and structural identifiability concepts and ambiguities: a critical review and analysis. Am J Physiol Regul Integr Comp Physiol 239(1):R7–R24, ISSN 0363-6119

  21. Connor AJ (2016) Calibration, validation and uncertainty quantification. doi:10.6084/m9.figshare.3406876. Supporting data and code

  22. Gammon K (2012) Mathematical modelling: forecasting cancer. Nature 491:66–67

    Article  Google Scholar 

  23. Gelman A, Carlin JB, Stern HS, Dunson DB, Vehtari A, Rubin DB (2014a) Bayesian data analysis, 3rd edn. CRC Press, Boca Raton

    Google Scholar 

  24. Gelman A, Hwang J, Vehtari A (2014b) Understanding predictive information criteria for Bayesian models. Stat Comput 24(6):997–1016. doi:10.1007/s11222-013-9416-2

    MathSciNet  Article  MATH  Google Scholar 

  25. Ghanem R (1999) Stochastic finite elements with multiple random non-Gaussian properties. ASCE J Eng Math 125(1):26–40

    Article  Google Scholar 

  26. Ghanem R, Red-Horse J (1999) Propogation of probabilistic uncertainity in complex physical systems using a stochastic finite element approach. Phys D 133:137–144

    MathSciNet  Article  MATH  Google Scholar 

  27. Ghanem RG, Spanos PD (1991) Stochastic finite elements: a spectral approach. Springer, Berlin

    Google Scholar 

  28. Gilks WR, Richardson S, Spiegelhalter DJ (eds) (1996) Practical Markov chain Monte Carlo. Chapman and Hall, London

    Google Scholar 

  29. Gompertz G (1825) On the nature of the function expressive of the law of human mortality, and on the new mode of determining the value of life contingencies. Philos Trans R Soc Lond 115:513–585

    Article  Google Scholar 

  30. Hastings WK (1970) Monte Carlo sampling methods using Markov chains and their applications. Biometrika 57:97–109

    MathSciNet  Article  MATH  Google Scholar 

  31. Hawkins-Daarud A, Prudhomme S, van der Zee KG, Oden JT (2013) Bayesian calibration, validation, and uncertainty quantification of diffuse interface models of tumour growth. J Math Biol 67:1457–1485

    MathSciNet  Article  MATH  Google Scholar 

  32. Higdon D, Kennedy M, Cavendish JC, Cafeo JA, Ryne RD (2005) Combining field data and computer simulations for calibration and prediction. SIAM J Sci Comput 2(26):448–466

    MathSciNet  MATH  Google Scholar 

  33. Insua DR, Ruggeri F (eds) (2000) Robust Bayesian analysis, volume 152 of lecture notes in statistics. Springer, New York

    Google Scholar 

  34. Kaipio J, Somersalo E (2006) Statistical and computational inverse problems. Springer, New York

    Google Scholar 

  35. Kennedy MC, O’Hagan A (2001) Bayesian calibration of computer models. J R Stat Soc Ser B (Stat Methodol) 63(4):425–464. doi:10.1111/1467-9868.00294

    MathSciNet  Article  MATH  Google Scholar 

  36. Kullback S, Leibler RA (1951) On information and sufficiency. Ann Math Stat 22(1):79–86. doi:10.1214/aoms/1177729694

    MathSciNet  Article  MATH  Google Scholar 

  37. Laird AK (1964) Dynamics of tumor growth. Br J Cancer 18:490–502

    Article  Google Scholar 

  38. Lopes HF, Tobias JL (2011) Confronting prior convictions: on issues of prior sensitivity and likelihood robustness in Bayesian analysis. Annu Rev Econ 3:107–131. doi:10.1146/annurev-economics-111809-125134

    Article  Google Scholar 

  39. Massey FJ (1951) The Kolmogorov–Smirnov test for goodness of fit. J Am Stat Assoc 46(253):68–78

    Article  MATH  Google Scholar 

  40. Metropolis N, Ulam S (1949) The Monte Carlo method. J Am Stat Assoc 44:335–341

    Article  MATH  Google Scholar 

  41. Metropolis N, Rosenbluth AW, Rosenbluth MN, Teller AH, Teller E (1953) Equation of state calculations by fast computing machines. J Chem Phys 21:1087–1092

    Article  Google Scholar 

  42. Miller LH (1956) Table of percentage points of Kolmogorov statistics. J Am Stat Assoc 51(273):111–121

    Article  MATH  Google Scholar 

  43. Najm HN (2009) Uncertainty quantification and polynomial chaos techniques in computational fluid dynamics. Annu Rev Fluid Mech 41:35–52

    MathSciNet  Article  MATH  Google Scholar 

  44. Nobile F, Tempone R, Webster C (2008a) A sparse grid stochastic collocation method for elliptic partial differential equations with random input data. SIAM J Numer Anal 46(5):2309–2345

    MathSciNet  Article  MATH  Google Scholar 

  45. Nobile F, Tempone R, Webster C (2008b) An anisotropic sparse grid stochastic collocation method for elliptic partial differential equations with random input data. SIAM J Numer Anal 46(5):2411–2442

    MathSciNet  Article  MATH  Google Scholar 

  46. Norris JR (1998) Markov chains. Number no. 2008 in Cambridge Series in Statistical and Probabilistic Mathematics. Cambridge University Press. ISBN 9780521633963

  47. NRC (2012) Assessing the reliability of complex models: mathematical and statistical foundations of verification, validation, and uncertainty quantification. The National Academies Press. ISBN 9780309256346

  48. Oberkampf WL, Barone MF (2006) Measures of agreement between computation and experiment: validation metrics. J Comput Phys 217(1):5–36. doi:10.1016/j.jcp.2006.03.037

    Article  MATH  Google Scholar 

  49. Oberkampf WL, Roy CJ (2010) Verification and validation in scientific computing. Cambridge University Press, Cambridge

    Google Scholar 

  50. Oberkampf WL, Trucano TG, Hirsch C (2004) Verification, validation, and predictive capability in computational engineering and physics. Appl Mech Rev 57(5):345–384. doi:10.1115/1.1767847

    Article  Google Scholar 

  51. Oden JT, Moser R, Ghattas O (2010a) Computer predictions with quantified uncertainty, part i. SIAM News 43(9):1–3

    Google Scholar 

  52. Oden JT, Moser R, Ghattas O (2010b) Computer predictions with quantified uncertainty, part ii. SIAM News 43(10):1–3

    Google Scholar 

  53. Pathmanathan P, Gray RA (2013) Ensuring reliability of safety-critical clinical applications of computational cardiac models. Front Physiol. doi:10.3389/fphys.2013.00358

    Google Scholar 

  54. Pericchi LR, Peréz ME (1994) Posterior robustness with more than one sampling model. J Stat Plan Inference 40:279–294

    MathSciNet  Article  MATH  Google Scholar 

  55. Raue A, Kreutz C, Maiwald T, Bachmann J, Schilling M, Klingmüller U, Timmer J (2009) Structural and practical identifiability analysis of partially observed dynamical models by exploiting the profile likelihood. Bioinformatics 25(15):1923–1929. doi:10.1093/bioinformatics/btp358

    Article  Google Scholar 

  56. Roache PJ (2009) Fundamentals of verification and validation, 2nd edn. Hermosa Publishers, New Mexico

    Google Scholar 

  57. Savage N (2012) Modelling: computing cancer. Nature 491:62–63

    Article  Google Scholar 

  58. Schervish MJ (1995) Theory of statistics. Springer series in statistics. Springer, New York

    Google Scholar 

  59. Simpson DP, Rue H, Martins TG, Riebler A, Sørbye SH (2014) Penalising model component complexity: a principled, practical approach to constructing priors. In arXiv preprint arXiv:1403.4630 [stat.ME]

  60. Spiegelhalter DJ, Best NG, Carlin BP, van der Linde A (2002) Bayesian measures of model complexity and fit (with discussion). J R Stat Soc B 64(4):583–639

    MathSciNet  Article  MATH  Google Scholar 

  61. Tarantola A (2005) Inverse problem theory and methods for model parameter estimation. SIAM, Philadelphia

    Google Scholar 

  62. Tatang MA, Pan W, Prinn RG, McRae GJ (1997) An efficient method for parametric uncertainity analysis of numerical geophysical models. J Geophys Res 102(D18):21925–21932

    Article  Google Scholar 

  63. Watanabe S (2010) Asymptotic equivalence of Bayes cross validation and widely applicable information criterion in singular learning theory. J Mach Learn Res 11:3571–3594

    MathSciNet  MATH  Google Scholar 

  64. Xiu D, Hesthaven JS (2005) High-order collocation methods for differential equations with random inputs. SIAM J Sci Comput 27(3):1118–1139

    MathSciNet  Article  MATH  Google Scholar 

  65. Xiu D, Karniadakis GE (2002) Modeling uncertainty in steady state diffusion problems via generalized polynomial chaos. Comput Methods Appl Mech Eng 191:4927–4948

    MathSciNet  Article  MATH  Google Scholar 

  66. Xiu D, Karniadakis GE (2003) Modeling uncertainty in flow simulations via generalized polynomial chaos. J Comput Phys 187:137–167

    MathSciNet  Article  MATH  Google Scholar 

  67. Yankeelov TE, Atuegwu N, Hormuth D, WeisJA, Barnes SL, Miga MI, Rericha EC, Quaranta V (2013) Clinically relevant modeling of tumor growth and treatment response. Sci Transl Med 5(187). ISSN 1946-6234. doi:10.1126/scitranslmed.3005686

Download references

Acknowledgements

J. Collis and M. E. Hubbard acknowledge the support of EPSRC Grant Number EP/K039342/1. This project has received funding from the European Unions Seventh Framework Programme for research, technological development and demonstration under Grant Agreement No. 600841.

Author information

Affiliations

Authors

Corresponding author

Correspondence to Joe Collis.

Additional information

Joe Collis and Anthony J. Connor have contributed equally to this work.

Appendix: Measurement Times

Appendix: Measurement Times

The times at which measurements were taken in the experiments described in Sect. 2.2 are given in Table 6.

Table 6 Times at which measurements of the spheroids were taken measured after an initial seed of 2000 tumour cells per spheroid were implanted at \(t=0\)

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Collis, J., Connor, A.J., Paczkowski, M. et al. Bayesian Calibration, Validation and Uncertainty Quantification for Predictive Modelling of Tumour Growth: A Tutorial. Bull Math Biol 79, 939–974 (2017). https://doi.org/10.1007/s11538-017-0258-5

Download citation

Keywords

  • Bayesian calibration
  • Tumour growth
  • Model validation