Mathematical Modeling of the Effects of Tumor Heterogeneity on the Efficiency of Radiation Treatment Schedule


Radiotherapy uses high doses of energy to eradicate cancer cells and control tumors. Various treatment schedules have been developed and tried in clinical trials, yet significant obstacles remain to improving the radiotherapy fractionation. Genetic and non-genetic cellular diversity within tumors can lead to different radiosensitivity among cancer cells that can affect radiation treatment outcome. We propose a minimal mathematical model to study the effect of tumor heterogeneity and repair in different radiation treatment schedules. We perform stochastic and deterministic simulations to estimate model parameters using available experimental data. Our results suggest that gross tumor volume reduction is insufficient to control the disease if a fraction of radioresistant cells survives therapy. If cure cannot be achieved, protocols should balance volume reduction with minimal selection for radioresistant cells. We show that the most efficient treatment schedule is dependent on biology and model parameter values and, therefore, emphasize the need for careful tumor-specific model calibration before clinically actionable conclusions can be drawn.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3
Fig. 4


  1. Al-Hajj Muhammad, Wicha Max S, Benito-Hernandez Adalberto, Morrison Sean J, Clarke Michael F (2003) Prospective identification of tumorigenic breast cancer cells. Proc Nat Acad Sci 100(7):983–3988

    Article  Google Scholar 

  2. Bao Shideng, Qiulian Wu, McLendon Roger E, Hao Yueling, Shi Qing, Hjelmeland Anita B, Dewhirst Mark W, Bigner Darell D, Rich Jeremy N (2006) Glioma stem cells promote radioresistance by preferential activation of the DNA damage response. Nature 444:756–760

    Article  Google Scholar 

  3. Baumann Michael, Krause Mechthild, Hill Richard (2008) Exploring the role of cancer stem cells in radioresistance. Nat Rev Cancer 8:545–554

    Article  Google Scholar 

  4. Conforti Domenico, Guerriero Francesca, Guido Rosita (2008) Optimization models for radiotherapy patient scheduling. 4OR 6:263–278

    MathSciNet  Article  MATH  Google Scholar 

  5. Dhawan Andrew, Kohandel Mohammad, Hill Richard, Sivaloganathan Sivabal (2014) Tumour control probability in cancer stem cells hypothesis. PLoS ONE 9:e96093

    Article  Google Scholar 

  6. Dionysiou Dimitra D, Stamatakos Georgios S, Uzunoglu Nikolaos K, Nikita Konstantina S, Marioli Antigoni (2004) A four-dimensional simulation model of tumour response to radiotherapy in vivo: parametric validation considering radiosensitivity, genetic profile and fractionation. J Theor Biol 230:1–20

    Article  Google Scholar 

  7. Enderling Heiko, Park Derek, Hlatky Lynn, Hahnfeldt Philip (2009) The importance of spatial distribution of stemness and proliferation state in determining tumor radioresponse. Math Model Nat Phenom 4:117–133

    MathSciNet  Article  MATH  Google Scholar 

  8. Enderling Heiko, Anderson Alexander RA, Chaplain Mark AJ, Beheshti Afshin, Hlatky Lynn, Hahnfeldt Philip (2009) Paradoxical dependencies of tumor dormancy and progression on basic cell kinetics. Cancer Res 69(22):8814–8821

    Article  Google Scholar 

  9. Fillmore Christine M, Kuperwasser Charlotte (2008) Human breast cancer cell lines contain stem-like cells that self-renew, give rise to phenotypically diverse progeny and survive chemotherapy. Breast Cancer Res 10(2):1

    Article  Google Scholar 

  10. Fowler John F (1989) The linear-quadratic formula and progress in fractionated radiotherapy. Br Jo Radiol 62:679–694

    Article  Google Scholar 

  11. Gao Xuefeng, McDonald John T, Hlatky Lynn, Enderling Heiko (2013) Acute and fractionated irradiation differentially modulate glioma stem cell division kinetics. Cancer Res 73(5):1481–1490

    Article  Google Scholar 

  12. Gupta Piyush B, Fillmore Christine M, Jiang Guozhi, Shapira Sagi D, Tao Kai, Kuperwasser Charlotte, Lander Eric S (2011) Stochastic state transitions give rise to phenotypic equilibrium in populations of cancer cells. Cell 146:633–644

    Article  Google Scholar 

  13. Hall Eric J, Giaccia Amato J (2006) Radiobiology for the radiologist. Lippincott Williams and Wilkins, Philadelphia

    Google Scholar 

  14. Haviland Joanne S, Owen J Roger, Dewar John A, Agrawal Rajiv K, Barrett Jane, Barrett-Lee Peter J, Dobbs H Jane et al (2013) The UK Standardisation of Breast Radiotherapy (START) trials of radiotherapy hypofractionation for treatment of early breast cancer: 10-year follow-up results of two randomised controlled trials. Lancet Oncol 14:1086–1094

    Article  Google Scholar 

  15. Lagadec Chann, Vlashi Erina, Della Donna Lorenza, Meng YongHong, Dekmezian Carmen, Kim Kwanghee, Pajonk Frank (2010) Survival and self-renewing capacity of breast cancer initiating cells during fractionated radiation treatment. Breast Cancer Res 12:R13

    Article  Google Scholar 

  16. Leder Kevin, Pitter Ken, LaPlant Quincey, Hambardzumyan Dolores, Ross Brian D, Chan Timothy A, Holland Eric C, Michor Franziska (2014) Mathematical modeling of PDGF-driven glioblastoma reveals optimized radiation dosing. Cell 156:603–616

    Article  Google Scholar 

  17. Mathews Lesley A, Cabarcas Stephanie M, Hurt Elaine M (2013) DNA repair of cancer stem cells. Springer, Berlin

    Book  Google Scholar 

  18. Marjanovic Nemanja D, Weinberg Robert A, Chaffer Christine L (2013) Cell plasticity and heterogeneity in cancer. Clin Chem 59:168–179

    Article  Google Scholar 

  19. Phillips Tiffany M, McBride William H, Pajonk Frank (2006) The response of CD24-/low/CD44+ breast cancer initiating cells to radiation. J Natl Cancer Inst 98:1777–1785

    Article  Google Scholar 

  20. Reya Tannishtha, Morrison Sean J, Clarke Michael F, Weissman Irving L (2001) Stem cells, cancer, and cancer stem cells. Nature 414:105–111

    Article  Google Scholar 

  21. Sarcar Bhaswati et al (2011) Targeting radiation-induced G2 checkpoint activation with the Wee-1 inhibitor MK-1775 in glioblastoma cell lines. Mol Cancer Ther 10:2405–2414

    Article  Google Scholar 

  22. Stamatakos GS, Antipas VP, Uzunoglu NK, Dale RG (2014) A four-dimensional computer simulation model of the in vivo response to radiotherapy of glioblastoma multiforme: studies on the effect of clonogenic cell density. Br J Radiol 79:389

    Article  Google Scholar 

  23. Shackleton Mark, Quintana Elsa, Fearon Eric R, Morrison Sean J (2009) Heterogeneity in cancer: cancer stem cells versus clonal evolution. Cell 138:822–829

    Article  Google Scholar 

  24. Trialists’ Group (2008) The START, The UK Standardisation of Breast Radiotherapy (START) Trial B of radiotherapy hypofractionation for treatment of early breast cancer: a randomised trial. Lancet 371:1098–1107

  25. Wein Lawrence M, Cohen Jonathan E, Wu Joseph T (2000) Dynamic optimization of a linear-quadratic model with incomplete repair and volume-dependent sensitivity and repopulation. Int J Radiat Oncol Biol Phys 47:1073–1083

    Article  Google Scholar 

  26. Withers HR (1992) Biological basis of radiation therapy for cancer. Lancet 339(8786):156–9

    Article  Google Scholar 

Download references


Financial support by the Natural Sciences and Engineering Research Council of Canada (NSERC) (MK) is gratefully acknowledged.

Author information



Corresponding authors

Correspondence to Farinaz Forouzannia or Mohammad Kohandel.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (pdf 2379 KB)

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Forouzannia, F., Enderling, H. & Kohandel, M. Mathematical Modeling of the Effects of Tumor Heterogeneity on the Efficiency of Radiation Treatment Schedule. Bull Math Biol 80, 283–293 (2018).

Download citation


  • Cancer stem cell
  • Fractionation
  • Tumor control