Deconstructing the Contributions of Heterogeneity to Combination Treatment of Hormone-Sensitive Breast Cancer

Abstract

Combination therapies are fundamental to cancer treatments, including in breast cancer, which is the most common invasive malignancy in women. Breast cancer treatment is determined based on molecular subtypes. Since 2016, combination palbociclib and fulvestrant has been used to treat hormone receptor-positive breast cancer. However, the impact of heterogeneity of the tumor landscape and tumor composition dynamics on scheduling decisions remains poorly understood. To elucidate the contributions of variability at multiple scales to treatment outcomes in hormone receptor-positive breast cancer, we developed a simple mathematical model of two unique estrogen receptor-positive (ER$+$) breast cancer cell types and their responses to combination treatment with palbociclib and fulvestrant. We used this model to understand how the initial fraction of either cell type may impact the fraction remaining after treatment and examined how heterogeneity in pharmacokinetics and pharmacodynamics results in a large distribution of outcomes. Our results suggest that the pharmacokinetics and pharmacodynamics of fulvestrant were the major drivers of final tumor size and composition. We then leveraged our model to guide therapeutic scheduling of combination palbociclib and fulvestrant, demonstrating the use of mathematical modeling to improve our understanding of cancer biology and treatments.

1 Introduction

Breast cancer is the most common invasive malignancy in women; a woman has a one in eight chance of developing breast cancer in her lifetime [1, 2]. The treatment of breast cancer requires a multifaceted approach combining surgery, radiation, neoadjuvant, and adjuvant treatments [3]. There are five molecular subtypes of breast cancer (luminal A, luminal B, HER-2, basal, and normal)—each with a different combination of cancer cells that over- or under-express progesterone receptor (PR$+/-$), estrogen receptor (ER$+/-$), and human epidermal growth factor receptor 2 (HER2$+/-$) [4]. Effective treatment of these varying subtypes of breast cancer requires a deep understanding of heterogeneity in their responses to the different treatment types; unfortunately, there is still no completely curative treatment for any subtype.

Combination therapy, i.e., combining two or more therapeutic agents, is a cornerstone of cancer therapy [5]. The major goal of combination therapy in oncology is to enhance the therapeutic efficacy of a single anti-cancer drug through co-administration with a synergistic or additive drug that targets key pathways [5]. For example, metformin, an agent used to treat type 2 diabetes, was found to increase the susceptibility of a p53 breast cancer cell line to therapeutic molecule tumor necrosis factor-related apoptosis inducing ligand (TRAIL) [6]. To that end, a combination therapy in breast cancer with recognized potential is palbociclib combined with fulvestrant [7, 8], which was approved in early 2016 by the FDA to treat hormone receptor-positive breast cancer [9].

Palbociclib (brand name Ibrance) is an orally available, highly selective inhibitor of cyclin-dependent kinase 4 and 6 (CDK4 and CDK6) [10,11,12]. CDK4/6 are critical mediators of the cellular transition into the S phase and are crucial for the initiation, growth, and survival of many cancer types [13]. As such, pharmacological inhibitors of CDK4/6 are rapidly becoming a new standard of care for patients with advanced hormone receptor-positive breast cancer. Palbociclib is an inhibitor of CDK4/6 and thus forces cells to stay in the G1 phase in lieu of undergoing cell division (Fig. 1). Importantly, palbociclib does not induce apoptosis but instead halts cellular division. According to the US National Institutes of Health (NCT03007979), patients have historically been on a 21-day-on, 7-day-off palbociclib schedule, though there were concerns that the off days were the reason behind worse patient outcomes [14]. A 5-day-on, 2-day-off schedule has thus far shown better health outcomes for the treatment of ER+ breast cancer [14, 15], though this study is still ongoing.

Fig. 1

Schematic summarizing the mathematical model of combination therapy on breast cancer cells. (a) Our model consists of pharmacokinetics and pharmacodynamics (PK/PD) of two drugs (palbociclib and fulvestrant), each with different mechanisms of action. Palbociclib targets and arrests the cell in the cell cycle, and fulvestrant degrades the estrogen receptor on cells, essentially causing cell death. To examine the impact of heterogeneity on tumor composition prior to, during, and at the end of treatment with this combination, we considered heterogeneous tumors composed of less aggressive and more aggressive cells. We parameterized the model’s parameters to in vitro data from two cell lines: MCF7 and T47D. (b) Schematic overview of the mechanism of action of palbociclib on cell cycle arrest—inhibiting the cell cycle transition from G1 to S phase

Fulvestrant is a novel endocrine therapy for breast cancer that binds, blocks, and degrades the estrogen receptor, leading to complete inhibition of estrogen signaling through the ER [16, 17]. Through extensive preclinical and clinical trials, fulvestrant has demonstrated improved clinical efficacy compared to established endocrine agents [17]. Fulvestrant has been combined with several different classes of therapeutics, in particular, CDK4/6 inhibitors [16]. The PALOMA-3 study investigated fulvestrant with palbociclib or placebo in both pre- and postmenopausal patients who had progressed on previous endocrine treatment [18, 19]. The trial demonstrated a substantial increase in progression-free survival from 4.6 months to 9.5 months in the placebo compared to palbociclib arms [16]. The FLIPPER trial was a phase II study comparing fulvestrant and palbociclib with fulvestrant and placebo in the first-line metastatic setting [20].

While it can be challenging to fully capture the effects of heterogeneity on treatment outcomes experimentally and clinically, mathematical modeling is well-placed to provide insight into how cancer treatments are affected by multiple scales of heterogeneity. Previously, groups have used deterministic mathematical models to examine the combined treatment of breast cancer using palbociclib and AZD9496 [21]. For example, He et al. [22] used a mathematical model that captured the cell cycle and signaling pathways in response to endocrine therapy and CDK4/6 inhibition. Their model successfully predicted the combined effects of estrogen deprivation and palbociclib and was used to explore combination scheduling. Mathematical modeling studies can also be extended to a virtual or in silico clinical trial setting to account for variations in patient characteristics and comprehensively explore dosing regimens in ways that are clinically unfeasible [23,24,25,26]. There are many examples of virtual clinical trials employed in cancer therapies [27,28,29,30,31] to this end, and this approach continues to gain traction within pharmaceutical and other applications [32, 33].

As heterogeneity can impact combination strategies aimed at CDK4/6 inhibition at multiple levels, we developed a simple mathematical model of two unique ER$+$ breast cancer cell types and their responses to combination treatment with palbociclib and fulvestrant to understand how different sources of variation impact this therapeutic approach. We examined in situ how co-culturing of heterogeneous cell types, specifically two commonly used breast cancer cell lines exhibiting different degrees of aggressivity, affects their responses to treatment. Using our model, we next explored how interindividual variability in PK within a virtual breast cancer patient cohort affects treatment outcomes. Lastly, we used our integrated framework to establish how therapeutic scheduling determines treatment responses, providing insight into effective regimens using this combination treatment.

2 Methods

2.1 Mathematical Model of Breast Cancer Co-cultures and Combination Therapy

In this section, we detail the development of a mathematical model to capture the action of palbociclib and fulvestrant on a heterogeneous population of breast cancer cells. This type of model of the effects of the drugs on the target tissue in the body is known as a pharmacodynamic (PD) model. To capture and understand how intrinsic cell characteristics affect combination palbociclib and fulvestrant treatment, we considered co-cultures of MCF7 and T47D cell lines—two commonly used breast cancer cell lines that display different sensitivities to each drug, with T47D thought to be more aggressive (i.e., exhibit stronger/faster growth) than MCF7. A schematic overview of our model is provided in Fig. 1.

2.1.1 Palbociclib’s Impact on Cell Growth

We first constructed a mathematical PD model describing the growth of a cell population under treatment by palbociclib, a drug whose effects were assumed to inhibit the cell cycle. To model these effects, we adopted a general inhibitory effects model given by

$$\displaystyle \begin{aligned} {} E_i = E_{0,i} - \frac{E_{0,i} I_{max,i} [P]^{h_i}}{[P]^{h_i}+[IC_{50,i}]^{h_i}}, \end{aligned} $$

(1)

where $E_i$ with $i=M,T$ specifies the effect on cell line MCF7 and T47D, respectively, P is the concentration of palbociclib at the tumor site, $E_{0,i}$ denotes the baseline effect of the drug palbociclib on cell type i, $I_{max,i}$ represents the maximal effect of the drug at high concentrations, $h_i$ is the Hill coefficient measuring the slope of the inhibitory curve for cell type i, and $IC_{50,i}$ represents the drug concentration eliciting 50% of the maximal inhibition. This model formulation is regularly used to capture the effect of a drug on inhibiting a cell population [34].

As palbociclib arrests cells in the cell cycle, the general growth inhibition model for a population of cells of type i, $C_i(t)$, inhibited by palbociclib is given by

$$\displaystyle \begin{aligned} \frac{dC_i}{dt} = \lambda (C_i)E_iC_i, \end{aligned}$$

where $\lambda (C_i)$ is a function describing cell population growth in the absence of treatment (see Fig. 1). As in vitro tumor growth is constrained by the availability of nutrients, space, etc., we modeled cell growth using the logistic growth law

$$\displaystyle \begin{aligned} \lambda(C_i)=r_i\left(1-\frac{C_i}{K}\right), {} \end{aligned} $$

(2)

where $r_i$ is the cell line-specific proportionality constant and K is the total cell population carrying capacity in a given space. We chose logistic growth as this provided the most accurate fit to cell count measurements; however, Gompertzian tumor growth also provided a close (but less accurate) fit to our data (Figs. 10 and 11). Thus, the complete model of monoculture growth under treatment with palbociclib is given by

$$\displaystyle \begin{aligned} \frac{dC_i}{dt} = C_i r_i\left(1-\frac{C_i}{K}\right) \left(E_{0,i}-\frac{E_{0,i} I_{max,i} [P]^{h_i}}{[P]^{h_i}+[I_{50,i}]^{h_i}} \right). \end{aligned}$$

Typically, tumors are not homogeneous in nature and are comprised of a variety of different cell types. We accounted for this phenotypic heterogeneity by modeling both MCF7 and T47D cell types within a single tumor environment in co-culture. As mentioned above, while both MCF7 and T47D are ER+, they differ in their responses to treatment. We, therefore, considered each to have separate parameters, with co-culture growth rates affected by the available space in the domain. In addition, since the effect parameters will depend on the drug being applied, we now update our PD variables to be drug specific. In other words, since parameters are cell line and drug specific, the specific combination is represented in their subscript as $X_{CELL^{drug}}$, where drug is denoted by either p for palbociclib or f for fulvestrant and the cell line is denoted either by M for MCF7 or by T for T47D.

For simplicity, and owing to the absence of data, we did not consider switching between tolerant and resistant types [35]. We assumed that the carrying capacity and growth of each cell type is affected by the presence of the other cell type in the dish, thus modifying our logistic growth model. Therefore, to account for the impact of variable growth between each cell type, we included cell-specific carrying capacities in our model of cell growth in Eq. (2). Our final model describing the change in population of two, indirectly interacting, cell types (MCF7 and T47D) is given by

$$\displaystyle \begin{aligned} \frac{dC_M}{dt} &= C_M r_M\left(1-\frac{C_M+C_T}{K_M\phi_M+K_T\phi_T}\right) \left(E_{0,M^p}-\frac{E_{0,M^p} I_{max,M^p} [P]^{h_{M^p}}}{[P]^{h_{M^p}}+[I_{50,M^p}]^{h_{M^p}}} \right) {} \end{aligned} $$

(3)

$$\displaystyle \begin{aligned} \frac{dC_T}{dt} &= C_T r_T\left(1-\frac{C_M+C_T}{K_M\phi_M+K_T\phi_T}\right) \left(E_{0,T^p}-\frac{E_{0,T^p} I_{max,T^p} [P]^{h_{T^p}}}{[P]^{h_{T^p}}+[I_{50,T^p}]^{h_{T^p}}} \right), {} \end{aligned} $$

(4)

where $\phi _i$ is the volume fraction of cell type i for either type MCF7 or T47D, respectively, in the domain and is calculated by $\phi _i= C_i/(C_M+C_T)$ with $\phi _M+\phi _T=1$. The global carrying capacity in the domain is given by $K=K_M \phi _M+K_T \phi _T$, where $K_i$ is the individual carrying capacity for each cell type, MCF7 and T47D, respectively.

2.1.2 Modeling the Effects of Fulvestrant on a Heterogenous Tumor

As fulvestrant degrades cells, we modeled its effect on the rates of decay for both the MCF7 and T47D cells. We updated the ordinary differential equation (ODE) system for the effect of palbociclib in Eqs. (3) and (4) to include a decay term for both cell populations, $d_i$, that is affected by the concentration of fulvestrant (F), using a modified version of the effect function in Eq. (1):

$$\displaystyle \begin{aligned} \frac{dC_M}{dt} = C_M r_M &\left(1-\frac{C_M+C_T}{K_M\phi_M+K_T\phi_T}\right) \left(E_{0,M^p}-\frac{E_{0,M^p} I_{max,M^p} [P]^{h_{M^p}}}{[P]^{h_{M^p}}+[I_{50,M^p}]^{h_{M^p}}} \right)\\ &-C_Md_M\left(\frac{E_{0,M^f} I_{max,M^f} [F]^{h_{M^f}}}{[F]^{h_{M^f}}+[IC_{50,M^f}]^{h_{M^f}}}\right) {} \end{aligned} $$

(5)

$$\displaystyle \begin{aligned} \frac{dC_T}{dt} = C_T r_T &\left(1-\frac{C_M+C_T}{K_M\phi_M+K_T\phi_T}\right) \left(E_{0,T^p}-\frac{E_{0,T^p} I_{max,T^p} [P]^{h_{T^p}}}{[P]^{h_{T^p}}+[I_{50,T^p}]^{h_{T^p}}} \right) \\ &-C_Td_T\left(\frac{E_{0,T^f} I_{max,T^f} [F]^{h_{T^f}}}{[F]^{h_{T^f}}+[IC_{50,T^f}]^{h_{T^f}}}\right), {} \end{aligned} $$

(6)

where $E_{0,i^f}$ is the basal effect of fulvestrant on cell type i, $I_{max,i^f}$ is the maximum effect of fulvestrant, $h_{i^f}$ is the Hill coefficient for fulvestrant, and $IC_{50,i^f}$ is the half-effect of fulvestrant, given i represents either M or T for cell type MCF7 or T47D, respectively. Note that this modified effect function for fulvestrant aims to capture the death rate increase that results from fulvestrant concentration F increase. To determine the concentration of palbociclib and fulvestrant after administration, we introduced pharmacokinetic (PK) models parameterized from clinical PK studies for both drugs.

2.1.3 Palbociclib and Fulvestrant Pharmacokinetic Models

We used a linear two-compartment PK model with first-order absorption and absorption lag to model the dynamics of orally administered palbociclib,

$$\displaystyle \begin{aligned} \frac{dM_0}{dt} &= -k_aM_0,{} \end{aligned} $$

(7)

$$\displaystyle \begin{aligned} \frac{dM_1}{dt} & = k_aM_0-M_1\left(\frac{k_e+k_{el}}{V_C}\right)+M_2\left(\frac{k_e}{V_P}\right), {} \end{aligned} $$

(8)

$$\displaystyle \begin{aligned} \frac{dM_2}{dt} &= -M_2\left(\frac{k_e}{V_P}\right)+M_1\left(\frac{k_e}{V_C}\right), {} \end{aligned} $$

(9)

where $M_0$, $M_1$, and $M_2$ are the palbociclib concentrations pre-absorption, in plasma, and in peripheral tissue, respectively. Furthermore, $k_a$ is the rate of absorption into plasma, $k_{el}$ is the rate of linear elimination, $k_e$ is the exchange rate between plasma and tissue, and $V_C$ and $V_P$ are the apparent plasma and peripheral tissue volumes, respectively. The concentration of palbociclib at the tumor site is then calculated by $P(t)=M_1 (t)/V_C$ in Eqs. (5) and (6).

Based on data from 38 postmenopausal women with advanced breast cancer who received 250 mg doses of extended-release fulvestrant (in a single 5 mL intramuscular (IM) injection or two 2.5 mL IM injections [36]), a two-compartment PK model with zero-order administration and linear elimination was developed. The fulvestrant PK model is given by

$$\displaystyle \begin{aligned} \frac{dF_1}{dt} &= In-k_{el}F_1+k_{21}F_2-k_{12}F_1, {} \end{aligned} $$

(10)

$$\displaystyle \begin{aligned} \frac{dF_2}{dt} & = -k_{21}F_2 +k_{12}F_1,{} \end{aligned} $$

(11)

$$\displaystyle \begin{aligned} In &= \frac{D}{T_{k0}V},{} \end{aligned} $$

(12)

where $F_1$ and $F_2$ denote fulvestrant concentrations in the plasma and tissues, respectively, In represents the administered dose (here taken to be an IM administration), $k_{el}$ is the rate of linear elimination, $k_{12}$ and $k_{21}$ are transit rates between the plasma and tissue compartments, D represents the IM dose, $T_{k0}$ is the time for absorption, and V  is the volume of distribution. The concentration of fulvestrant at the tumor site is set as $F(t)=F_1 (t)$ in Eqs. (5) and (6).

2.2 Parameter Estimation

2.2.1 Estimating Tumor Growth Parameters

Cell counting was performed in breast cancer cell lines MCF7 and T47D by Vijayaraghavan et al. [37]. Cells were plated in six-well plates and treated with indicated agents for 10 days. The medium was replaced every other day over the course of the experiment. Cells were then collected and counted using BioRad TC20 Automated Cell Counter on days 0, 3, 6, and 10 (see data in Figs. 10 and 11). We estimated parameters governing cell growth by setting all drug concentrations to zero in our model, Eqs. (5) and (6), and fitting the proliferation rate $r_i$ and carrying capacity $K_i$ to cell type i count data. Fitting was performed in MATLAB using the nonlinear least-squares fitting function lsqnonlin; the trust-region-reflective algorithm with 1000 maximum function evaluations was chosen. The model was solved using ode45.

2.2.2 Estimating Drug Effect Parameters from Cell Viability Assays

Cell viability measurements for MCF7 and T47D with palbociclib were measured by Vijayaraghavan et al. [37]. For these dose–response studies, cells were plated on a 96-well plate and treated with increasing concentrations (0.01–12 $\mu$M) of palbociclib for 1, 2, 4, 6, or 8 days. The medium was replaced with drug-containing medium every other day. At the completion of drug treatment, cultures were continued in drug-free medium until day 12 after which they were stained with 0.5% crystal violet solution. Values were normalized to those of their no treatment controls. We assumed that after 8 days of drug exposure, the drug effects were saturated. We fit the 8-day data (see Fig. 12) and estimated the values of $E_{0,i^p},I_{max,i^p}$, and $IC_{50,i^p}$ for each cell type i in Eqs. (5) and (6) by minimizing the least-squares error between the data and the inhibitory growth model using lsqnonlin in MATLAB. We additionally estimated the 95% confidence intervals for the parameters using the Jacobian returned from the lsqnonlin fit. All fitted parameters and their bounds are given in Fig. 12 for both MCF7 and T47D cell lines.

In similar experiments, Nukatsuka et al. [38] measured MCF7 cell growth under varying fulvestrant concentrations. Measurements were calculated as means and standard deviation of cell growth relative to that of the control for three independent experiments. Lewis-Wambi et al. [39] measured DNA ($\mu$g/well) from T47D cells after treatment with fulvestrant. Cells were seeded in 24-well dishes and after 24h were treated with varying drug concentrations for 7 days. At the conclusion of the experiment, cells were harvested, and proliferation was assessed as cellular DNA mass ($\mu$g/well). We assumed this as a proxy for cell viability relative to control. As with the palbociclib experiments from Vijayaraghavan et al. [37], we estimated the PD parameters in Eqs. (5) and (6) by minimizing the least-squares error between the data and the inhibitory growth model using lsqnonlin in MATLAB (see Fig. 12).

2.2.3 Estimating Pharmacokinetic Parameters

We used a nonlinear mixed effects model in Monolix to estimate parameters of the fulvestrant PK model in Eqs. (10)–(12). As the data reported in Robertson et al. [36] were pooled, we extracted the reported mean and lower and upper bounds to estimate interindividual variability (IIV). We then fit the model in Eqs. (10)–(12) to this data assuming lognormal distributions on parameters (Fig. 13) subject to IIV according to

$$\displaystyle \begin{aligned} \rho_i = \theta_j \mbox{ exp}(\eta_{ji}), \ \ \ \ \ \ \eta_{ji}\sim \mbox{N}(0,\omega_j^2), {} \end{aligned}$$

where $\rho _i$ is the value of a given model parameter (e.g., $k_{el}$, $k_{12}$, etc.) for subject i, $\theta _j$ is the population mean, and $\eta _{ji}$ represents the deviation from the mean (i.e., IIV) for the i-th individual. Estimated model parameters are presented in Tables 1 and 2.

Table 1 Estimated parameter values for the palbociclib population pharmacokinetic model in Eqs. (7)–(9)

Table 2 Estimated parameter values for the fixed effects of the fulvestrant population pharmacokinetic model in Eqs. (10)–(12) and other terms used in Monolix

The model in Eqs. (7)–(9) is based on the clinical and theoretical work of Yu et al. [40], which described data from 26 advanced breast cancer patients who received palbociclib and letrozole on a 3-weeks-on, 1-week-off treatment regimen. The palbociclib PK model parameters were taken from Yu et al. and were used to simulate patient populations. We assumed lognormal distributions on parameters subject to interindividual variability using $\rho _i$ from Eq. (??). Parameter values for fulvestrant were taken directly from Robertson et al. [36].

2.3 Generating Heterogeneous Pharmacokinetics and Pharmacodynamics

2.3.1 Pharmacokinetic Parameters

We investigated palbociclib and fulvestrant individually to quantify each of their contributions to the effects of PK IIV on tumor growth. For palbociclib, we sampled $V_C$, $V_P$, $k_{el}$, $k_e$, and $k_a$ from lognormal distributions according to the parameters in Table 1 to produce a virtual patient population. Similarly, for fulvestrant, we sampled $T_{k0}$, V , $k_{el}$, $k_{12}$, and $k_{21}$ from lognormal distributions according to the best-fit nonlinear mixed effects model determined by our parameter fitting (see Table 2) to generate virtual patients. In the case of each drug, by simulating the full model (with all other components’ parameters set to their average values), we selected only those virtual patients whose predicted trajectories were realistic (as confirmed by visual predictive check of their concentration time courses) before accepting them into our cohort. This left 500 virtual patients in the case of palbociclib and 438 for fulvestrant.

2.3.2 Pharmacodynamic Parameters

To investigate the effect of heterogeneity of the PD of palbociclib and fulvestrant, we generated 400 sets of parameter values by sampling $E_0$, $I_{max}$, h, and $IC_{50}$ for each cell type–drug combination from the ranges established during parameter fitting (Fig. 14). As each of these four parameters is drug and cell type specific, this gave 16 parameters to sample:

$$\displaystyle \begin{aligned} \hat{p} = [&E_{0,M^p}, I_{max,M^p}, h_{M^p}, IC_{50,M^p}, E_{0,T^p}, I_{max,T^p}, h_{T^p}, IC_{50,T^p}, \ldots \\ & E_{0,M^f}, I_{max,M^f}, h_{M^f}, IC_{50,M^f}, E_{0,T^f}, I_{max,T^f}, h_{T^f}, IC_{50,T^f}]. {} \end{aligned} $$

(13)

Parameters were sampled from a multivariate normal distribution with mean $\mu$ set to the fitted values in Table 3 for $\hat {p}$ and standard deviation $\sigma$ determined from the confidence intervals (CIs) returned for the fitted parameters and the formula

$$\displaystyle \begin{aligned} \frac{CI-\mu}{1.96} = \sigma, \end{aligned}$$

Table 3 Fitting parameter values obtained by fits in Figs. 10 and 11 to the pharmacodynamics model in Eqs. (5) and (6)

where 1.96 was chosen to return values in the 95% confidence interval. Any samples resulting in negative parameter values were discarded. The resulting distributions of parameters are provided in Fig. 14.

3 Results

3.1 Shorter Treatment Cycle Reduces Aggressive Cell Viabilities as Compared to Conventional Schedule

We first set out to predict whether a shortened treatment cycle (i.e., 5 days on of palbociclib followed by 2 days of rest, repeated for 28 days) was a viable strategy as compared to a conventional (21 days on of palbociclib followed by 7 days of rest) schedule. Both protocols included combination therapy with 125 mg of fulvestrant on days 1 and 15. For this, we simulated the complete model with mean values for both the palbociclib and fulvestrant PK models (Tables 1 and 2 for Eqs. (7)–(12)) and PD effects model (Table 3 for Eqs. (5) and (6)). We considered only the case where the two cell types were present in equal fractions (i.e., $\phi _i=0.5$) with a total cell count of $C_{M,i}+C_{T,i}=7\times 10^4$ cells. We called this an “average patient.” Left untreated over the course of 28 days, unsurprisingly both cell lines were predicted to grow to the global carrying capacity of $K=K_M \phi _M+K_T \phi _T$ (see Fig. 15).

We then introduced treatment to this average patient. We first simulated 125 mg of palbociclib daily for 21 days followed by a period of rest for 7 days, with 125 mg of fulvestrant on days 1 and 15, consistent with current treatment schedules [40] (Fig. 2). Our model predicted the resulting viabilities at the end of treatment to be 0.39 (MCF7) and 0.51 (T47D). Here, cell viability was determined by comparing treatment outcomes to the untreated scenario for the same parameters, i.e., the viability of each cell line was calculated by comparing the total cells at the end of treatment to the total number of cells under no treatment. Repeating this strategy for a treatment course of 125 mg of palbociclib for 5 days followed by 2 days of rest repeated over a period of 28 days, with 125 mg of fulvestrant administered on days 1 and 15, we found viabilities after 28 days of 0.38 (MCF7) and 0.47 (T47D), respectively (Fig. 2). Notably, this change in treatment schedule was predicted to somewhat lower the viability of T47D, which is the more aggressive cell type.

Fig. 2

Comparison of alternate protocols for combination therapy. (a) Two established protocols are considered for combination palbociclib and fulvestrant treatment denoted by this schematic: (left) conventional treatment with 21 days on of palbociclib followed by 7 days of rest, and (right) shortened treatment with 5 days on of palbociclib followed by 2 days of rest, repeated for 28 days. Both protocols include combination therapy with 125 mg of fulvestrant on days 1 and 15. (b)–(d) Tumor growth dynamics on conventional treatment. (e)–(g) Tumor growth dynamics on shortened treatment. (b) and (e) Fulvestrant pharmacokinetic model (Eqs. (10)–(12)). (c) and (f) Palbociclib pharmacokinetic model (Eqs. (7)–(9)). (d) and (g) Tumor response to treatment by pharmacodynamic model (Eqs. (5) and (6)). For (d) by comparing the total number of MCF7 and T47D cells at the end of treatment to the trial that did not receive treatment (Fig. 14), the cell viability was calculated as 0.39 and 0.51 for MCF7 and T47D cells, respectively. For (g) by comparing the total number of MCF7 and T47D cells at the end of treatment to the trial that did not receive treatment (Fig. 15), the cell viability was calculated as 0.38 and 0.47 for MCF7 and T47D cells, respectively. Figure 16 shows plots of the corresponding effect function values over time

Fig. 3

Results for varying initial tumor composition and total initial cell count, conventional treatment (i.e., 21 days on followed by 7 days off for palbociclib, Fig. 2a). Initial fraction of MCF7 cell line ($\phi _M$) and the total number of cells ($C_M+C_T$) are varied over $0 < \phi _M < 1$ and $101 < C_M+C_T < 105$. (a) Viability of the MCF7 line for conventional treatment over varied $\phi _M$ and $C_M+C_T$. Viability is calculated by comparing the total number of MCF7 cells with treatment compared to the total number of MCF7 cells without treatment after 28 days; both trials have the same initial conditions and only differ in whether treatment is administered. (b) Viability of the T47D line for conventional treatment over varied $\phi _M$ and $C_M+C_T$. Viability for T47D is larger than that of MCF7. (c) The final fraction of MCF7 cell line ($\phi _M$) after the 28 days of treatment. (d) The final fraction of MCF7 cell line ($\phi _M$) after the 28 days of treatment compared to the initial fraction. Note the differences in the color bars between panels

Fig. 4

Predicted outcomes on conventional regimen in palbociclib virtual patient cohort. (a) MCF7 and (b) T47D cell counts over the course of the conventional treatment regimen with variation in palbociclib pharmacokinetic parameters summarized in Fig. 18. Color bar: viability of T47D cells

Fig. 5

Some palbociclib pharmacokinetic parameters differ between responders and non-responders. Virtual patients were classified as responders or non-responders based on the predicted terminal T47D cell count of each virtual patient. Upon performing a two-sided Kolmogorov–Smirnov test for each parameter between these two subcohorts, significant differences were found in (a) the elimination rate ($k_{el}$) and (d) the absorption rate ($k_a$); no significant differences were observed in (b) the central volume ($V_C$) and (c) the intercompartmental clearance rate ($k_e$)

Fig. 6

Spaghetti plots for fulvestrant virtual patients. Predicted dynamics for 438 patients in fulvestrant virtual patient cohort after treatment with 125 mg of fulvestrant on days 1 and 15. (a) MCF7 cells, (b) T47D cells, and (c) fulvestrant concentrations. In all, color bar indicates total tumor viability

Fig. 7

Fulvestrant pharmacokinetic parameters differ between virtual patients with high terminal fulvestrant concentrations and those with low terminal concentrations. We classified virtual patients as “high concentration” or “low concentration” based on the predicted terminal fulvestrant concentration of each virtual patient (Fig. 6c). Using a Kolmogorov–Smirnoff test for differences in distributions, significant differences were found in (a) absorption delay ($t_{k0}$), (b) central volume of distribution ($V_D$), (c) rate of elimination ($k_{el}$), (d) rate of transfer from central to peripheral compartment ($k_{12}$), and (e) rate of transfer from peripheral to central compartment ($k_{21}$). In legends, “high conc” corresponds to those virtual patients with high terminal fulvestrant concentrations above $-1.4$ log($\mu$M) and “low conc” to those with low terminal fulvestrant concentrations below $-3.97$ $\log (\mu \mathrm {M})$

Fig. 8

Virtual cohort investigation into the effect of pharmacodynamics on combination treatment. 400 virtual patients were generated with varying pharmacodynamic parameters (see $\hat {p}$ in Eq. (13)). The shortened 5-day-on, 2-day-off palbociclib regimen combined with two fulvestrant dosages was considered. (a) Cell counts for MCF7 ($C_M$) and T47D ($C_T$) cells over time plotted as mean and standard deviation of patient cohort. (b) Individual patient trajectories for total cell count $C_M+C_T$. (c) A scatter plot of the final number of each cell type after 28 days of treatment, colored by the corresponding total number of cells after treatment. (d)–(e) Waterfall plots for patient specific $E_{0,T^f}$ and $I_{max,T^f}$ against the total cells relative to the cohort average. Color bar corresponds to the value of each patient’s parameter normalized to a range between 0 and 1

3.2 Initial Tumor Composition Has Little Impact on Treatment Outcomes

Given that our model predicted a slight reduction in T47D viability under shortened schedules for an average patient, we next interrogated how various levels of heterogeneity (e.g., intrinsic to the tumor population, PK, PD, and treatment scheduling) would affect outcomes. First, we explored the effects of the initial tumor composition and initial total cancer cell count on the outcomes of different treatment regimens (Figs. 3 and 17).

Fig. 9

Investigating long-term dynamics of Regimens 1, 2, and 3. (a), (c), and (e) cancer cell ratios after treatment with Regimens 1, 2, and 3, respectively. Blue curves: ratio of cancer cell line MCF7 given by $C_M/(C_M+C_T)$, and red curves: ratio of cancer cell T47D given by $C_T/(C_M+C_T)$. (b), (d), and (f) comparison of the total cell load of both cell lines after treatment with Regimens 1, 2, and 3, respectively

To isolate the effect of the initial tumor composition, we set all model parameters in both the palbociclib and fulvestrant PK models and their PD models to be their mean values (Tables 1, 2, and 3), as in the previous section. We then varied the initial fraction of MCF7 cells ($\phi _M$), i.e., the less aggressive cell type, from 0 to 1 and the total initial cell count ($C_M+C_T$) from $10^1$ to $10^5$ cells. To accurately capture the effect of changing these parameters, we chose values of $\phi _M$ and $C_M+C_T$ in these ranges evenly spaced apart to give 440 unique parameter combinations.

We found that the cell viability—defined as stated earlier by comparing the total cells at the end of treatment to the total number of cells under no treatment—and final fraction ($\phi _M$ after 28 days of treatment) over these varying initial conditions showed decreased T47D viability for the shortened treatment (i.e., shortened vs. conventional, see Figs. 3 and 17). At lower initial cell counts, our model predicted that the more aggressive T47D cells were more likely to dominate at the end of treatment (Fig. 3). Our predictions show that as the initial number of cells increased, so too did the cell viability. This implies that with more cells, the drug combination becomes less effective. In both regimens, there appeared to be a switching point for the initial cell fraction above which MCF7 cells can dominate ($\sim$0.75). Though T47D viability did decrease with the shortened treatment, there was not an exceptional difference between the shortened and conventional treatment, indicating that the dosing schedules are more dependent on other factors of variability, i.e., PK and/or PD.

3.3 The Effects of Pharmacokinetic Variability Are Determined Uniquely Through Fulvestrant Interindividual Variability

To quantify the effects of interindividual variability in PK parameters on the outcomes of both MCF7 and T47D cells, we simulated the conventional dosing regimen of each drug in the population of virtual patients defined by our estimated population PK models (see Figs. 4 and 18) using the methods described in Sect. 2.3.1.

Fig. 10

Fit of logistic growth to cell count measurements for MCF7 and T47D from Vijayaraghavan et al. [37]. A logistic growth curve (Eq. (2)) was fit to the cell count measurements to obtain a cell growth rate $r_i$ and a cell carrying capacity $K_i$, where i represents either M or T for cell type MCF7 or T47D, respectively. The fit is given as a solid curve with a shaded 95% confidence interval. The data are represented as solid points. The resulting parameters are in Table 3

Fig. 11

Comparative analysis of model selection to MCF7 and T47D cell count data. We compared the least-squares fit for (a, d) exponential growth, (b, e) logistic growth, and (c, f) Gompertzian growth. We calculated the corrected Akaike information criterion for each figure, which returned (a) 134.3, (b) 131.1, (c) 134.5, (d) 131.7, (e) 116.5, and (f) 115.0. We also considered the confidence intervals plotted for each model fit. Given that the Gompertzian growth has wider confidence intervals compared to logistic growth, we concluded that logistic growth was a good model choice for tumor growth

For palbociclib, our results suggest that variability in the PK parameters has a negligible influence on tumor growth outcomes for both cell types (Figs. 4 and 19). Interestingly, examining the palbociclib parameters by classifying patients as either responders or non-responders based on their predicted terminal T47D cell count, we see a clear distinction in the cohort’s value for $k_{el}$, which is high for those with high terminal concentrations and low for those without (Fig. 5). Upon performing a two-sided Kolmogorov–Smirnov test for each parameter between the responder and non-responder subcohorts, we found significant differences in the elimination rate ($k_{el}$) and the absorption rate ($k_a$) (Fig. 5).

Fig. 12

Fit of drug effect parameters in the pharmacodynamics model in Eqs. (5) and (6) to cell viability measurements for fulvestrant and palbociclib on MCF7 and T47D. Cell viability measurements for (a) fulvestrant on MCF7 cells [38] and (b) fulvestrant on T47D cells [39]. Cell viability measurements for (c) palbociclib on MCF7 and (b) palbociclib on T47D by Vijayaraghavan et al. [37]. The resulting parameters are in Table 3. In (a) the inset zooms in on the confidence intervals surrounding the data fit

Fig. 13

Fulvestrant pharmacokinetic parameter estimation fits. Population pharmacokinetic data from Robertson et al. [36] was pooled to extract the mean and lower and upper bounds of the data. Pharmacokinetic parameters were estimated for Eqs. (10)–(12) to these data assuming lognormal distributions on parameters subject to interindividual variability using a standard nonlinear mixed effects model in Monolix

Fig. 14

Virtual patient parameter values for investigation varying the drug effect pharmacodynamic parameters. Parameters relating the effect of fulvestrant and palbociclib on MCF7 and T47D were sampled from normal distributions as described in Sect. 2.3.2 to obtain 400 unique parameter combinations corresponding to 400 virtual patients

In contrast, our results suggest that fulvestrant PK variability has a significant impact on tumor growth outcomes for both cell types (Fig. 6a, b). Distributions of fulvestrant PK parameters in the virtual patient cohort are provided in Fig. 20. We observed that virtual patients who sustained high concentrations of fulvestrant over the treatment period have significantly and consistently lower tumor growth as compared to virtual patients who more rapidly cleared the drug (Fig. 6c).

Fig. 15

Tumor growth dynamics under no treatment. The total number of MCF7 and T47D cells, $C_M$ and $C_T$, respectively, was simulated in the absence of palbociclib and fulvestrant, i.e., $F=P=0$

Given the clear relationship between terminal fulvestrant concentrations and outcomes in our virtual patients, we defined virtual patients with “high concentration” to be those with terminal fulvestrant concentrations above $-1.4$ log($\mu$M) and those with “low concentration” as those with concentrations below $-3.97$ $\log (\mu \mathrm {M})$ (Fig. 7). Using a two-sided Kolmogorov–Smirnoff test to test for statistically significant differences in distributions between these two subcohorts, we found significant differences in all fulvestrant PK parameters between these groups. This suggests that not only is fulvestrant the key driver of PK heterogeneity (as compared to palbociclib), but that differences in final tumor viability were related to higher $t_{k0}, V_D, k_{el}$, and $k_{21}$ values and lower $k_{12}$ values than those virtual patients who were predicted to have strong responses to fulvestrant treatments (Fig. 7).

Fig. 16

The effect functions corresponding to the simulation in Fig. 2. The effect function Eq. (1) for fulvestrant (a)–(b) and palbociclib (c)–(d) for the two dosage protocols considered in Fig. 2: (a), (c) shortened treatment with palbociclib 5 days on and 2 days off and (b), (d) conventional treatment with palbociclib 21 days on and 7 days off. Both protocols include combination therapy with 125 mg of fulvestrant on days 1 and 15

Fig. 17

Results for varying initial tumor composition and total initial cell count, shortened treatment (i.e., 5 days on followed by 2 days off, repeated for 28 days, for palbociclib, Fig. 2a). Initial fraction of MCF7 cell line ($\phi _M$) and the total number of cells ($C_M+C_T$) are varied over $0 < \phi _M < 1$ and $101 < C_M+C_T < 105$. (a) Viability of the MCF7 line for shortened treatment over varied $\phi _M$ and $C_M+C_T$. Viability is calculated by comparing the total number of MCF7 cells with treatment compared to the total number of MCF7 cells without treatment after 28 days; both trials have the same initial conditions and only differ in whether treatment is administered. The overall viability of MCF7 is less for the shortened treatment, compared to the conventional treatment. (b) Viability of the T47D line for shortened treatment over varied $\phi _M$ and $C_M+C_T$. The overall viability of T47D is less for the shortened treatment, compared to the conventional treatment. (c) The final fraction of MCF7 cell line ($\phi _M$) after the 28 days of treatment. (d) The final fraction of MCF7 cell line ($\phi _M$) after the 28 days of treatment compared to the initial fraction. At lower initial total cell numbers, T47D has a greater propensity to overtake the cancer tumor and take up a greater fraction of the total tumor. Note the differences in the color bars between panels

3.4 Variability in Each Drug’s Maximal Effect Drives Heterogeneity in Outcomes

To next explore the effect of PD variability on treatment outcomes, we fixed all model parameters to be that of an average patient (see Tables 1, 2, and 3) except the PD parameters in Eqs. (5) and (6) noted in $\hat {p}$ from Eq. (13). We generated 400 parameter sets within this range as described in Sect. 2.3.2. We then examined whether we could discern a relationship between each individual’s response to treatment and their inherent PD response. For each virtual patient, the shortened treatment protocol was simulated, and the corresponding MCF7 and TD47 cell counts were recorded (Fig. 8a) along with the total number of tumor cells (Fig. 8b). We observed large variance in counts of both cell types across the cohort, but our model did not predict tumor eradication for any patient (Fig. 21).

Fig. 18

Distributions of palbociclib virtual patient pharmacokinetic parameters: (a) elimination rate ($k_{el}$), (b) central volume ($V_C$), (c) intercompartmental clearance rate ($k_e$), and (d) absorption rate of palbociclib ($k_a$) constructed from 500 virtual patients sampled as described in Sect. 2.3.1. Experimental data suggests negligible variation in the parameter $V_P$. This parameter is thus taken to be constant, hence the lack of distribution. The virtual patient population randomly generated to produce these distributions is maintained in Fig. 5

To examine the correlation between the final numbers of MCF7 and T47D cells at the end of the treatment, we next plotted the total number of MCF7 and T47D cells at the end of treatment (Fig. 8c) for each generated parameter set in our ensemble. Our results clearly show that as the final amount of MCF7 cells decreased, there was a corresponding increase in the final T47D cell count and vice versa. In other words, final MCF7 and T47D were predicted to have an inverse linear relationship when PD variability was considered. We also correlated the final MCF7 and T47D cell counts with the final tumor size and found that the largest tumors are those that are predominately made up of T47D, whereas the smallest tumors are a mixture of both cell types.

We then examined which PD characteristics were the major drivers of final tumor size. We found that $E_0$ and $I_{max}$ for T47D (i.e., $E_{0,T^f}$,$I_{max,T^f}$) were most correlated with the final tumor size (Fig. 8d, e), as we predicted that small values of either parameter corresponded to large tumor sizes relative to the average (Fig. 22). All other parameters were not found to contribute significantly to the final tumor size.

3.5 Examining the Long-Term Effect of Variation on the Combination Protocol

Finally, with our understanding of the effects of cell-intrinsic, PK, and PD variability on combination palbociclib and fulvestrant therapy, we explored alternative treatment regimens to study whether we could improve upon the conventional and investigational schedules using virtual clinical trials of three different dosing regimens. On the current standard-of-care conventional 3-weeks-on/1-week-off dosing schedule of palbociclib, numerous patients have been reported to develop grade 3 or higher degree of neutropenia [15]. This adverse event could result in dose reduction or treatment discontinuation [15]. Furthermore, it has been hypothesized that the one week off-drug in the conventional combination schedule could potentially lead to an increase in the retinoblastoma tumor suppressor protein (Rb) [15].

Therefore, we explored alternative schedules with the aim of reducing the time off-drug (as compared to the conventional regimen) and limiting dose intensity to minimize off-target effects (Regimen 1). Additionally, clinical reports suggest that fulvestrant is most likely to cause acute liver injury [41, 42]. To reduce the risk of hepatotoxicity, we virtually reduce the dose level of fulvestrant and test the effectiveness of this dual-agent combination therapy in Regimens 2 and 3. Thus, we compared the following three schedules (Fig. 9):

Regimen 1:

125 mg of oral palbociclib administered once daily for 5 consecutive days, followed by 2 days off, plus 500 mg of intramuscular (IM) fulvestrant administered every 14 days for the first three injections and then every 28 days.

Fig. 19

Parameter sensitivity of the palbociclib pharmacokinetic parameters: (a) elimination rate ($k_el$), (b) central volume ($V_C$), (c) intercompartmental clearance rate ($k_e$), and (d) absorption rate of palbociclib ($k_a$). Curves convey changes in cell counts resulting from varying each parameter uniformly within three standard deviations of its mean (Table 1). Numerical values on the axes correspond to the fractional deviation of an end-of-treatment cell count from its mean scaled by the fractional deviation of one standard deviation of a parameter from that parameter mean. Qualitatively, the distribution of a parameter non-negligibly influences final cell count if the curves have coordinate values far from zero and close to one. Thus, our results show that all the parameter distributions have a negligible influence on the final cell counts

Fig. 20

Distributions of fulvestrant virtual patient pharmacokinetic parameters: (a) absorption delay ($t_{k0}$), (b) central volume of distribution ($V_D$), (c) rate of elimination ($k_{el}$), and (d)–(e) rates of transit between central and peripheral compartments ($k_{12}$ and $k_{21}$). Distributions describe the 438 virtual patients in the fulvestrant virtual patient cohort, generated as described in Sect. 2.3.1

Fig. 21

Results of simulating the virtual cohort with varying pharmacodynamics. (a)–(c) the mean and standard deviation for the 400 virtual patients’ cohort simulated with varying effect parameters (Fig. 12). (d)–(f) the corresponding individual patient trajectories

Regimen 2:

125 mg of oral palbociclib administered once daily for 5 consecutive days, followed by 2 days off, plus 250 mg IM of fulvestrant administered every 7 days for the first five injections and then every 14 days.

Regimen 3:

500 mg of oral palbociclib administered once daily for 5 consecutive days, followed by 2 days off, plus 250 mg IM of fulvestrant administered every 7 days for the first five injections and then every 14 days.

Simulations of Regimen 1 suggest that this schedule leads to the competitive exclusion of aggressive T47D cells. Selective killing of the therapy sensitive cells removes competitive restriction of MCF7 cells (Fig. 9a). The troughs of the fluctuating total cell loads come down to 3$\times 10^7$, while the peaks still reach a high level (Fig. 9b). In contrast, by reducing the dose of fulvestrant, Regimen 2 did not lead to competitive exclusion of the T47D cells (Fig. 9c) but resulted in an overall significant decrease in the total number of tumor cells (Fig. 9d). We further found that Regimen 3 did not lead to competitive exclusion of the T47D cells (Fig. 9e), and our model predictions suggest that the total number of cancer cells from both lines would continue to decrease (Fig. 9f).

Overall, we found that increasing the dose level of palbociclib within acceptable toxicity levels could achieve a lower level of total cancer cell load. Importantly, based on the simulations of our three dosing regimens, it is possible that higher doses/concentrations of fulvestrant could cause competitive exclusion of the T47D cell line. As a result, the relative strength of the less aggressive MCF7 cells may in fact inhibit the efficacy of the palbociclib-fulvestrant combination therapy.

4 Discussion

Heterogeneity is a key factor in cancer therapeutic planning, particularly when considering combination therapies that may have overlapping and interacting factors driving treatment responses. The interest in establishing different treatment regimens for palbociclib plus fulvestrant for the treatment of hormone-sensitive breast cancers gives rise to a number of questions relating to optimal scheduling. These include various scales of heterogeneity and their impact on combination palbociclib and fulvestrant, i.e., cell-intrinsic, PK, and PD heterogeneities. Understanding the contributions of each of these elements to tumor responses helps to establish new, and perhaps more potent and less toxic, therapeutic regimens. In this work, we used a simple model of interacting cells to quantify these contributions to help guide preclinical studies of palbociclib plus fulvestrant.

Considering a tumor composed of lesser and more aggressive cells (i.e., MCF7 and T47D cell lines), each type sensitive to a different degree to each drug, we predicted the overall tumor cell population and composition after treatment under variable initial fractions. Our model’s predictions showed that the initial cell fractions have little impact on the final tumor composition after treatment on either the shortened (i.e., 5 days on of palbociclib followed by 2 days of rest, repeated for 28 days) or conventional (21 days on of palbociclib followed by 7 days of rest) schedule. This is encouraging, in the sense that it suggests that it is primarily PK/PD variability controlling outcomes, and these can be more easily modulated to provide better outcomes.

When considering both PK and PD heterogeneity through the generation of virtual patients, we found that palbociclib PK variability alone had little impact on outcomes, whereas the PK of fulvestrant (as a cytotoxic agent) was a strong determinant of final tumor compositions. This is perhaps expected, as palbociclib acts to freeze the cell cycle rather than to induce apoptosis. Our results further show that palbociclib and fulvestrant are truly synergistic when given in combination, with each being less effective on its own.

Lastly, we used our investigations of the impact of various scales of heterogeneity to propose three alternative regimens to conventional and shortened. These regimens were designed to account for the undesired side effects of each drug through dose fractionation. Our model predictions suggest that it is possible that fulvestrant could cause competitive exclusion of the MCF7 (i.e., less aggressive) cells composing the tumors in our study. Indeed, our results showed that the more aggressive T47D cells act to inhibit the efficacy of the palbociclib-fulvestrant combination therapy, acting similarly to drug tolerant cells despite us not considering resistance in our study. Moreover, within acceptable toxicity levels, increasing the dose level of palbociclib could achieve better outcomes with respect to final tumor size.

In our model, we implemented a logistic growth function to model tumor growth. While Gompertzian tumor growth returned a similar Akaike information criterion and was able to capture the data (Fig. 11), we do not anticipate a large difference in our predictions between the two growth models. This is largely due to the fact both exhibit sigmodal style growth to a carrying capacity.

There are limitations to our approach. The lack of robust clinical data measuring this combination therapy presents a limitation in the reliability of our predictions. Given this is an exploratory study focused primarily on the effects of different sources of heterogeneity, we believe our results support further experiments into combination therapy that may be used in the future to validate our model predictions. Our parameterization could be further validated with in vivo experiments. Future work could also look to refine the number of parameters in the model and introduce simpler terms to model the effect of the drugs on the cancer cell population or simplify the cancer growth function.

Though we considered heterogeneity in tumor composition, we did not include the mammary stem cell cascade [43, 44], nor does our model include the actions of the immune system. We opted for the simpler model studied herein to provide a straightforward initial (more in vitro-focused) conceptualization of the impact of many scales of heterogeneity affecting treatment outcomes under combination palbociclib and fulvestrant; future studies will include key in vivo factors impacting therapies. As seen in Fig. 12a, there is a loss of fidelity in the MCF7 fits for high fulvestrant concentrations. This is a byproduct of the standard nonlinear effect function in Eq. (1) used here. As can be seen in the data, after an initial plateau beginning around $10^{-2}\,\mu$M, there is a second dip in the observed viability around $1\,\mu$M. Unfortunately, our effect function is unable to capture this second decrease, resulting in a higher predicted maximal effect than what is suggested in the data. To confirm the asymptotic behavior of MCF7 cells under treatment with fulvestrant, it would be necessary to have more viability data above $1\mu$M. Nonetheless, as our maximal effect is potentially slightly higher than in the data, our predictions are more conservative with respect to the overall treatment response to fulvestrant. We also did not consider the ways in which each degree of heterogeneity interacts with one another, opting instead to study each individually. This can be incorporated in subsequent iterations of our work.

Our study provides a roadmap for the continued study of CDK4/6 inhibitors and combination therapies in anti-cancer treatments more broadly. Despite using a simple model of tumor growth, our model’s predictions showed important, perhaps unexpected behaviors, including how competition between less and more aggressive cells in a heterogeneous tumor impacts treatment scheduling. Ultimately, this work demonstrates the importance of merging mathematical modeling within preclinical studies to improve drug development considerations.

Supplementary Information

See Figs. 10–22.

Fig. 22

Results of simulating the virtual cohort with varying pharmacodynamics. Each column corresponds to a virtual patient where the patients are ordered by the number of cells (left: T47D and right: MCF7) at the end of the treatment. The rows correspond to the parameter value where the column is the normalized value for that parameter. The color bar corresponds to the value of the parameter normalized between 0 and 1

1.1 Analysis of the Two Cell Line Model Dynamics

To understand the dynamics of the system of ODEs used in our study, we carried out a linear steady-state analysis to determine long-term stability of a simplified version of the system. We took the one-drug model in Eqs. (3) and (4) and considered only the effects of palbociclib on MCF7 and T47D cells.

We assumed both cell lines share the same carrying capacity due to them co-existing in the same spatial location and hence having the same spatial limitations, that is, $K=K_M=K_T$. For convenience for this analysis, we also assume that all PD parameters are equivalent for the two cell types, i.e., $I_{50,M^p}=I_{50,T^p} =IC_{50}$, $h_{M^p}= h_{T^p}=h$, $I_{max,M^p}=I_{max,T^p}=I_{max}$, and $E_0=E_{0,M^p}=E_{0,T^p}$, etc. For convenience, we denote $r_M E_0 = \tilde {r}_M$ and $r_T E_0=\tilde {r}_T$.

The model has three isolated equilibrium points

$$\displaystyle \begin{aligned} (C^*_M,C^*_T)=(0,0), \ \ \ (C_M^*, C_T^*) = (0,K), \ \ \ (C_M^*, C_T^*) = (K,0) \end{aligned}$$

and a line containing infinite number of equilibrium points:

$$\displaystyle \begin{aligned} C_M+C_T-K = 0. \end{aligned}$$

Linear stability analysis shows that $(C_M^*, C_T^*) = (0,0)$ has two positive eigenvalues

$$\displaystyle \begin{aligned} & \lambda_{01} = \left[ IC_{50}^h-(I_{max}-1)P^h\right]\tilde{r}_TK>0~~\mbox{and} \\ & \quad \lambda_{02} = \left[ IC_{50}^h-(I_{max}-1)P^h\right]\tilde{r}_MK>0, \end{aligned} $$

with corresponding eigenvectors

$$\displaystyle \begin{aligned} v_{01} = [1,0]^T, \ \ \ \ v_{02} = [0,1]^T. \end{aligned}$$

The equilibrium $(C_M^*, C_T^*) = (0,K)$ has eigenvalues:

$$\displaystyle \begin{aligned} \lambda_{11} =\left [ (I_{max}-1)P^h-IC_{50}^h \right]\tilde{r}_TK<0 \ \ \ \mbox{and} \ \ \lambda_{12} = 0, \end{aligned}$$

with corresponding eigenvectors

$$\displaystyle \begin{aligned} v_{11}=[0,1]^T, \ \ \ v_{12} = [-1,1]^T . \end{aligned}$$

Lastly, the equilibrium $(C_M^*, C_T^*) = (K,0)$ has two eigenvalues

$$\displaystyle \begin{aligned} \lambda_{21} = 0 \ \ \ \mbox{and} \ \ \ \lambda_{22} = \left[(I_{max}-1)P^h-IC_{50}^h\right]<0\tilde{r}_MK<0, \end{aligned}$$

with corresponding eigenvectors

$$\displaystyle \begin{aligned} v_{21} = [-1,1]^T, \ \ \ v_{22} = [1,0]^T. \end{aligned}$$

The line of infinite equilibria $C_M+C_T-K = 0$ has two eigenvalues

$$\displaystyle \begin{aligned} & \lambda_1 = 0, \ \ \mbox{and} \ \ \ \lambda_2 = \left[(I_{max}-1)P^h-IC_{50}^h \right](K-C_M)\tilde{r}_M \\ & \quad +\tilde{r}_TC_T\left[(I_{max}-1)P^h-IC_{50}^h\right]<0, \end{aligned} $$

with corresponding eigenvectors

$$\displaystyle \begin{aligned} v_1 = [-1,1]^T, \ \ \ v_2 = \left[ \frac{(K-C_T)(P^h(I_{max}-1)-IC_{50}^h)\tilde{r}_M}{C_T\tilde{r}_T(P^h(I_{max}-1)-IC_{50}^h)},1 \right]. \end{aligned}$$

Flow along nonzero eigenvalues is much faster than flow along zero eigenvalues. Therefore, in the fast timescale, cell trajectories are repelled from equilibrium $(C_M^*,C_T^* )=(0,0)$ and quickly converge to the neighborhood of the eigenvector associated with zero eigenvalue. Moreover, the eigenvector associated with zero eigenvalue $v_{12}=v_{21}=v_1=[-1,1]^T$ collides with the equilibrium line $C_M+C_T-K=0$, which is the slow manifold. Therefore, in the slow timescales, the flow on the slow manifold has no movement. It implies that the cancer population will eventually converge to the carrying capacity K but will have a different proportion depending on the initial fractions. A simulated vector field under the assumption $K_M=K_T=K$ is shown in Fig. 23.

Fig. 23

Simulated vector field. Three equilibria $(0,0)$, $(0,K)$, and $(K,0)$ are denoted in red circles. The line with infinite equilibria, $C_M+C_T-K=0$, is plotted in red. The eigenvector corresponding to the negative eigenvalues (i.e., $\lambda _{11}$, $\lambda _{22}$, and $\lambda _2$) is denoted by the green lines. The eigenvector corresponding to the zero eigenvalue is represented as a pink line and collapses with the equilibrium line, $C_M+C_T-K=0$. As a result, the vector field (blue arrows) points toward the red equilibrium line. Moreover, there is no movement in the red line