1 Introduction

According to the National Cancer Registry, cancer kills more people than tuberculosis, AIDS, and malaria combined. Statistics show that cancer mortality is estimated at 13 million deaths by 2030 [13]. Breast cancer is the most widely recognized and obstructive disease in females around the world. Breast cancer is a disease that affects the breast’s cells and causes an uncontrolled division of abnormal cells that can be malignant in the breast tissues [4]. Some of the breast cancer risk factors are hormonal imbalance, genetics, and environmental; however, there is still little information on this disease as regards the leading cause of a malignant form [5, 6]. Various mathematical and analytical approaches have been developed to understand the interaction between the tumor cells and the immune response during the treatment process that was mainly established as integer-order differential equations (IDEs) [79].

However, it is seen that many problems in biology, as well as in other fields such as engineering, finance, economics, and neural networks, can be formulated successfully by fractional-order differential equations (FDEs); see, for instance, Refs. [1015]. The nonlocal property of models of FDEs is not only depending on the current state but also provides an adequate description for the historical ones. The transformation of models governed by IDES into models of FDE needs to be precise due to the peculiarity of the order of differentiation α, where a small change in α may cause a significant change in the result. FDEs can model certain phenomena that cannot be modeled by IDEs. Thus, FDEs mainly use biological models since they are relevant to memory and the hereditary feature [1620].

2 Mathematical model

Over the years, modeling breast cancer has become a valuable tool to understand tumor growth’s dynamical behavior in the treatment process. D’Onofrio et al. explored the role of mathematical modeling in combination therapy for tumors [21]. Studies of Kermack and McKendrick [22, 23] and some other investigations in [2430] have shown that modeling mathematically the biological phenomena is a useful tool to solve epidemical problems.

However, these studies did not include a nutritional diet (ketogenic diet) in their mathematical model. Oke et al. improved the model of Mudufza [28] in [3]. They incorporated the control parameters such as ketogenic diet, immune booster, and anti-cancer drug to emphasize the point that there is an interaction between the cells due to the mutation in the tumor cell’s DNA.

In this paper, we establish a system of FDEs in the form

$$ \textstyle\begin{cases} D^{\alpha } N ( t ) =N ( t ) r_{1} ( K_{1} - \mu _{1} N ( t ) - \beta _{1} T ( t ) ) - ( 1-k ) \phi _{1} N ( t ) E ( t ), \\ D^{\alpha } T ( t ) =T ( t ) r_{2} ( K_{2} d- \mu _{2} T ( t ) - \beta _{2} I ( t ) ) -\delta T(t)\\ \hphantom{D^{\alpha } T ( t ) =}{}+ ( 1-k ) \phi _{1} N ( t ) E ( t ) T(t),\\ D^{\alpha } I ( t ) =\rho \gamma +I ( t ) ( K_{3} - \mu _{3} - \beta _{3} T ( t ) ) - ( 1-k ) \phi _{2} I ( t ) E ( t ), \\ D^{\alpha } E ( t ) = ( 1-k ) \varepsilon - \beta _{4} E ( t ), \end{cases} $$
(2.1)

where the parameters denote positive real numbers.

The first equation in system (2.1) represents the normal cell population \(N ( t )\). \(r_{1}\) is the growth rate, while \(K_{1}\) is the carrying capacity of the population. \(\mu _{1}\) represents the logistic rate and \(\beta _{1}\) denotes the inhibition rate of \(N(t)\). \(\phi _{1}\) is the tumor formation rate resulting from DNA mutation caused by the presence of excess estrogen, while \(( 1-k )\) represents the effectiveness of anti-cancer drugs.

In the second equation of (2.1), we show the luminal type (having estrogen receptors, ESR1 + ve) tumor cells that are denoted by \(T ( t )\). \(K_{2}\) is the carrying capacity of the population, and it depends on the rate of d, which is the ketogenic diet. \(r_{2}\) is the growth rate of the \(T ( t )\) population. Any mutation in DNA that is caused by excess estrogen repopulate the tumor cells by \(\phi _{1} N ( t ) E ( t )\). \(\mu _{2}\) represents the logistic rate of the tumor cell population. \(\beta _{2}\) is the rate of the effectiveness of the immune system to the tumor cells, δ is the result of the tumor starvation nutrients during the ketogenic diet.

The third equation in (2.1) shows the class of immune response as \(I ( t ) \), where ρ denotes the source rate of immune response fully infused in the body daily. The immune booster γ assists the immune response whenever tumor cells overpower immune cells to activate the immune response and fight against the tumor cells. \(\mu _{3}\) represents the logistic rate of the immune cell population, while \(\phi _{2}\) is the immune suppression rate. \(K_{3}\) is the carrying capacity and \(\beta _{3}\) is the rate of interaction between \(T(t)\) and \(I(t)\).

Finally, the last equation of system (2.1) is the estrogen compartment, which is denoted by \(E ( t ) \). It should be known that an increase in estrogen level can lead to the growth of the tumor cells. The process of constantly replenishing excess estrogen is denoted by ε. We assume that the majority of cancer cells are estrogen-receptor positive and only a small proportion of epithelial cells are estrogen-receptor positive, which can only be blocked by the anti-cancer drug \(( 1-k )\) tamoxifen. \(\beta _{4}\) is the rate at which estrogen is being washed out from the body [3, 22]. According to the information on the parameters, all parametric values are stated in Table 1.

Table 1 The description of parameters

Definition 2.1

([31])

Given a function \(\varphi (x)\), the fractional integral with order \(\alpha >0\) is given by Abdel’s formula by

$$ I_{\alpha } \varphi ( x ) = \frac{1}{\Gamma ( \alpha )} \int _{0}^{x} \varphi ( t ) ( x-t )^{\alpha -1} f ( t ) \,dt,\quad x>0. $$
(2.2)

Definition 2.2

([31])

Let \(\varphi : R^{+} \rightarrow R\) be a continuous function. The Caputo fractional derivative of order \(\alpha \in ( n-1, n )\), where n is a positive integer is defined as

$$ D_{0}^{\alpha } \varphi ( x ) = \frac{1}{\Gamma ( n-\alpha )} \int _{0}^{x} \frac{\varphi ^{(n)} ( t )}{ ( x-t )^{\alpha +1-n}} \,dt. $$
(2.3)

When \(\alpha =n\), the derivatives are defined to be the usual nth order derivatives.

Definition 2.3

([32])

The Mittag-Leffler function of one variable is

$$ E_{\alpha } ( \lambda ,z ) = E_{\alpha } \bigl( \lambda z^{\alpha } \bigr) = \sum_{k=0}^{\infty } \frac{\lambda ^{k} z^{\alpha k}}{\Gamma ( 1+\alpha k )}\quad \bigl( \lambda \neq 0, z \in \mathbb{C:} \operatorname{Re} ( \alpha ) >0 \bigr). $$
(2.4)

3 The equilibrium points

Consider the system

$$ \textstyle\begin{cases} D^{\alpha } N ( t ) =f ( N ( t ),T ( t ),I ( t ),E ( t ) ) \\ \hphantom{D^{\alpha } N ( t )}=N ( t ) r_{1} ( K_{1} - \mu _{1} N ( t ) - \beta _{1} T ( t ) ) - ( 1-k ) \phi _{1} N ( t ) E ( t ),\\ D^{\alpha } T ( t ) =g ( N ( t ),T ( t ),I ( t ),E ( t ) ), \\ \hphantom{D^{\alpha } T ( t )}=T ( t ) r_{2} ( K_{2} d- \mu _{2} T ( t ) - \beta _{2} I ( t ) ) -\delta T(t)+ ( 1-k ) \phi _{1} N ( t ) E ( t ) T(t),\\ D^{\alpha } I ( t ) =h ( N ( t ),T ( t ),I ( t ),E ( t ) ) \\ \hphantom{D^{\alpha } I ( t )}=\rho \gamma +I ( t ) ( K_{3} - \mu _{3} - \beta _{3} T ( t ) ) - ( 1-k ) \phi _{2} I ( t ) E ( t ), \\ D^{\alpha } E ( t ) =j ( N ( t ),T ( t ),I ( t ),E ( t ) ) = ( 1-k ) \varepsilon - \beta _{4} E ( t ). \end{cases} $$
(3.1)

To analyze the stability of the system (3.1), we perturb the equilibrium point by adding \(\varepsilon _{i} ( t ) >0\), \(i=1, 2, 3, 4\), that is,

$$ \begin{gathered} N ( t ) - \overline{N} = \varepsilon _{1} ( t ), \qquad T ( t ) - \overline{T} = \varepsilon _{2} ( t ), \\ I ( t ) - \overline{I} = \varepsilon _{3} ( t )\quad \text{and}\quad E ( t ) - \overline{E} = \varepsilon _{4} ( t ). \end{gathered} $$
(3.2)

Thus, we have

$$\begin{aligned}& \begin{aligned} D^{\alpha } \bigl( \varepsilon _{1} ( t ) \bigr) \simeq{}& f ( \overline{N}, \overline{T}, \overline{I}, \overline{E} ) + \frac{\partial f ( \overline{N}, \overline{T}, \overline{I}, \overline{E} )}{\partial N} \varepsilon _{1} ( t ) + \frac{\partial f ( \overline{N}, \overline{T}, \overline{I}, \overline{E} )}{\partial T} \varepsilon _{2} ( t ) \\ &{}+ \frac{\partial f ( \overline{N}, \overline{T}, \overline{I}, \overline{E} )}{\partial I} \varepsilon _{3} ( t ) + \frac{\partial f ( \overline{N}, \overline{T}, \overline{I}, \overline{E} )}{\partial j} \varepsilon _{4} ( t ), \end{aligned} \\& \begin{aligned} D^{\alpha } \bigl( \varepsilon _{2} ( t ) \bigr) \simeq{}& g ( \overline{N}, \overline{T}, \overline{I}, \overline{E} ) + \frac{\partial g ( \overline{N}, \overline{T}, \overline{I}, \overline{E} )}{\partial N} \varepsilon _{1} ( t ) + \frac{\partial g ( \overline{N}, \overline{T}, \overline{I}, \overline{E} )}{\partial T} \varepsilon _{2} ( t ) \\ &{}+ \frac{\partial g ( \overline{N}, \overline{T}, \overline{I}, \overline{E} )}{\partial I} \varepsilon _{3} ( t ) + \frac{\partial g ( \overline{N}, \overline{T}, \overline{I}, \overline{E} )}{\partial j} \varepsilon _{4} ( t ), \end{aligned} \\& \begin{aligned} D^{\alpha } \bigl( \varepsilon _{3} ( t ) \bigr) \simeq{}& h ( \overline{N}, \overline{T}, \overline{I}, \overline{E} ) + \frac{\partial h ( \overline{N}, \overline{T}, \overline{I}, \overline{E} )}{\partial N} \varepsilon _{1} ( t ) + \frac{\partial h ( \overline{N}, \overline{T}, \overline{I}, \overline{E} )}{\partial T} \varepsilon _{2} ( t ) \\ &{} + \frac{\partial h ( \overline{N}, \overline{T}, \overline{I}, \overline{E} )}{\partial I} \varepsilon _{3} ( t ) + \frac{\partial h ( \overline{N}, \overline{T}, \overline{I}, \overline{E} )}{\partial j} \varepsilon _{4} ( t ) \end{aligned} \end{aligned}$$

and

$$\begin{aligned} D^{\alpha } \bigl( \varepsilon _{4} ( t ) \bigr) \simeq& j ( \overline{N}, \overline{T}, \overline{I}, \overline{E} ) + \frac{\partial j ( \overline{N}, \overline{T}, \overline{I}, \overline{E} )}{\partial N} \varepsilon _{1} ( t ) + \frac{\partial j ( \overline{N}, \overline{T}, \overline{I}, \overline{E} )}{\partial T} \varepsilon _{2} ( t ) \\ &{} + \frac{\partial j ( \overline{N}, \overline{T}, \overline{I}, \overline{E} )}{\partial I} \varepsilon _{3} ( t ) + \frac{\partial j ( \overline{N}, \overline{T}, \overline{I}, \overline{E} )}{\partial j} \varepsilon _{4} ( t ). \end{aligned}$$

This perturbation around the equilibrium point is to linearize the system since we established a nonlinear fractional-order system based on the Lotka-Volterra logistic equation. Moreover, since we are working with the Caputo derivative, the constant function is here zero.

We use the fact that \(f ( \overline{N}, \overline{T}, \overline{I}, \overline{E} ) =g ( \overline{N}, \overline{T}, \overline{I}, \overline{E} ) =h ( \overline{N}, \overline{T}, \overline{I}, \overline{E} ) =j ( \overline{N}, \overline{T}, \overline{I}, \overline{E} ) =0\) and obtain, therefore, a linearized system about the equilibrium point such as

$$ D^{\alpha } Z=JZ, $$
(3.3)

where \(Z= ( \varepsilon _{1} ( t ), \varepsilon _{2} ( t ), \varepsilon _{3} ( t ), \varepsilon _{4} ( t ) )\). Moreover, J is the Jacobian matrix at equilibrium:

$$ J= \left . \begin{pmatrix} \frac{\partial f ( \overline{N}, \overline{T}, \overline{I},\overline{E} )}{\partial N}&\frac{\partial f ( \overline{N}, \overline{T}, \overline{I},\overline{E} )}{\partial T}&\frac{\partial f ( \overline{N}, \overline{T}, \overline{I},\overline{E} )}{\partial I}&\frac{\partial f ( \overline{N}, \overline{T}, \overline{I},\overline{E} )}{\partial j}\\ \frac{\partial g ( \overline{N}, \overline{T}, \overline{I},\overline{E} )}{\partial N}&\frac{\partial g ( \overline{N}, \overline{T}, \overline{I},\overline{E} )}{\partial T}&\frac{\partial g ( \overline{N}, \overline{T}, \overline{I},\overline{E} )}{\partial I}&\frac{\partial g ( \overline{N}, \overline{T}, \overline{I},\overline{E} )}{\partial j}\\ \frac{\partial h ( \overline{N}, \overline{T}, \overline{I},\overline{E} )}{\partial N}&\frac{\partial h ( \overline{N}, \overline{T}, \overline{I},\overline{E} )}{\partial T}&\frac{\partial h ( \overline{N}, \overline{T}, \overline{I},\overline{E} )}{\partial I}&\frac{\partial h ( \overline{N}, \overline{T}, \overline{I},\overline{E} )}{\partial j}\\ \frac{\partial j ( \overline{N}, \overline{T}, \overline{I},\overline{E} )}{\partial N}&\frac{\partial j ( \overline{N}, \overline{T}, \overline{I},\overline{E} )}{\partial T}&\frac{\partial j ( \overline{N}, \overline{T}, \overline{I},\overline{E} )}{\partial I}&\frac{\partial j ( \overline{N}, \overline{T}, \overline{I},\overline{E} )}{\partial j} \end{pmatrix} \right \vert _{ ( N ( t ),T ( t ), I ( t ), E(t) ) = ( \overline{N}, \overline{T}, \overline{I}, \overline{E} )}. $$
(3.4)

We have \(B^{-1} JB=C\), where C is the diagonal matrix of the eigenvalues \(\lambda _{i}\) (\(i=1, 2, 3, 4 \)) and B, the eigenvectors of J. Therefore, we get

$$ \textstyle\begin{cases} D_{0}^{\alpha } \eta _{1} = \lambda _{1} \eta _{1},\\ D_{0}^{\alpha } \eta _{2} = \lambda _{2} \eta _{2},\\ D_{0}^{\alpha } \eta _{3} = \lambda _{3} \eta _{3},\\ D_{0}^{\alpha } \eta _{4} = \lambda _{4} \eta _{4}, \end{cases}\displaystyle \quad \text{where } \eta = \begin{pmatrix} \eta _{1}\\ \eta _{2}\\ \eta _{3}\\ \eta _{4} \end{pmatrix} \text{ and }\eta = B^{-1} Z, $$
(3.5)

whose solutions are given by the Mittag-Leffler functions,

$$\begin{aligned}& \eta _{1} ( t ) = \sum_{n=0}^{\infty } \frac{ ( \lambda _{1} )^{n} t^{n\alpha }}{\Gamma ( n\alpha +1 )} \eta _{1} ( 0 ) = E_{\alpha } \bigl( \lambda _{1} t^{\alpha } \bigr) \eta _{1} ( 0 ), \end{aligned}$$
(3.6)
$$\begin{aligned}& \eta _{2} ( t ) = \sum_{n=0}^{\infty } \frac{ ( \lambda _{2} )^{n} t^{n\alpha }}{\Gamma ( n\alpha +1 )} \eta _{2} ( 0 ) = E_{\alpha } \bigl( \lambda _{2} t^{\alpha } \bigr) \eta _{2} ( 0 ), \end{aligned}$$
(3.7)
$$\begin{aligned}& \eta _{3} ( t ) = \sum_{n=0}^{\infty } \frac{ ( \lambda _{3} )^{n} t^{n\alpha }}{\Gamma ( n\alpha +1 )} \eta _{3} ( 0 ) = E_{\alpha } \bigl( \lambda _{3} t^{\alpha } \bigr) \eta _{3} ( 0 ) \end{aligned}$$
(3.8)

and

$$ \eta _{4} ( t ) = \sum_{n=0}^{\infty } \frac{ ( \lambda _{4} )^{n} t^{n\alpha }}{\Gamma ( n\alpha +1 )} \eta _{4} ( 0 ) = E_{\alpha } \bigl( \lambda _{4} t^{\alpha } \bigr) \eta _{4} ( 0 ). $$
(3.9)

Matington studied in [33], a fractional differential system involving the Caputo derivative

$$ D_{0}^{\alpha } x ( t ) =Ax(t) $$
(3.10)

with initial values \(x ( 0 ) = x_{0} = ( x_{10}, x_{20},\dots , x_{n0} )^{T}\), where \(x= ( x_{1}, x_{2},\dots , x_{n} )^{T}\), \(\alpha \in ( 0,1 )\) and \(A \in \mathbb{R}^{n\times n}\). The stability of system (3.10) was defined by Matington as follows.

Definition 3.1

The autonomous system (3.10) is said to be

  1. (i)

    stable if and only if for any \(x_{0}\), there exists \(\varepsilon >0\) such that \(\Vert x \Vert \leq \varepsilon \) for \(t \geq 0\),

  2. (ii)

    asymptotically stable if and only if \(\lim_{t \rightarrow \infty } \Vert x \Vert =0\).

Theorem A

([34])

The autonomous system (3.10) is

  1. (i)

    asymptotically stable if and only if if \(\vert \arg ( \lambda _{i} ) \vert > \frac{\alpha \pi }{2}\), in this case, the components of the state decay towards 0 like \(t^{-\alpha } \),

  2. (ii)

    stable if and only if either it is asymptotically stable, or those critical values which satisfy \(\vert \arg ( \lambda _{i} ) \vert = \frac{\alpha \pi }{2}\) have geometric multiplicity one.

(Here, \(\arg ( \lambda _{i} )\) denotes the arguments of the eigenvalues of the square matrix A.)

The proof of Theorem A was sketched by Matington in [33], while Zeng et al. proved the theorem in detail by using the Mittag-Leffler functions in [35]. Therefore, considering system (3.3) we can say that if \(\vert \arg ( \lambda _{i} ) \vert > \frac{\alpha \pi }{2}\) (\(i=1, 2, 3, 4 \)), then \(\eta _{i} ( t ) \) (\(i=1, 2, 3, 4 \)) are decreasing and therefore also \(\varepsilon _{i} ( t ) \) (\(i=1, 2, 3, 4 \)) are decreasing. Thus, let the solution \(( \varepsilon _{1} ( t ), \varepsilon _{2} ( t ), \varepsilon _{3} ( t ), \varepsilon _{4} ( t ) )\) of (3.3) exist. If the solution of (3.3) is increasing, then \(( \overline{N}, \overline{T}, \overline{I}, \overline{E} )\) is unstable and if \(( \varepsilon _{1} ( t ), \varepsilon _{2} ( t ), \varepsilon _{3} ( t ), \varepsilon _{4} ( t ) )\) is decreasing, then \(( \overline{N}, \overline{T}, \overline{I}, \overline{E} )\) is locally asymptotically stable.

There are four critical points; two of them are the dead critical points, one is a tumor-free critical point, and the other one is the co-existing critical point. These are given as follows:

Tumor-free critical point: \(\Lambda _{1} = ( \overline{N}_{1},0, \overline{I}_{1}, \overline{E}_{1} ) = ( \frac{r_{1} K_{1} \beta _{4} - ( 1-k )^{2} \phi _{1} \varepsilon }{\mu _{1} r_{1} \beta _{4}},0, \frac{\rho \gamma \beta _{4}}{ ( 1-k )^{2} \phi _{2} \varepsilon - \beta _{4} ( K_{3} -\delta )}, \frac{ ( 1-k ) \varepsilon }{\beta _{4}} )\), where \(\phi _{1} < \frac{K_{1} r_{1} \beta _{4}}{\varepsilon ( 1-k )^{2}}\) and \(\phi _{2} > \frac{ ( K_{3} -\delta ) \beta _{4}}{\varepsilon ( 1-k )^{2}}\) for \(K_{3} >\delta \).

Dead critical point 1: \(\Lambda _{2} = ( 0,0, \overline{I}_{2}, \overline{E}_{2} ) = ( 0, 0, \frac{\rho \gamma \beta _{4}}{ ( 1-k )^{2} \phi _{2} \varepsilon - \beta _{4} ( K_{3} -\delta )}, \frac{ ( 1-k ) \varepsilon }{\beta _{4}} )\), where \(\phi _{2} > \frac{ ( K_{3} -\delta ) \beta _{4}}{\varepsilon ( 1-k )^{2}}\) for \(K_{3} >\delta \).

Dead critical point 2: \(\Lambda _{3} = ( 0, \overline{T}_{3}, \overline{I}_{3}, \overline{E}_{3} ) = ( 0, \overline{T}_{3}, \overline{I}_{3}, \frac{ ( 1-k ) \varepsilon }{\beta _{4}} )\), where \(\phi _{2} > \frac{ ( K_{3} -\delta ) \beta _{4}}{\varepsilon ( 1-k )^{2}}\) for \(K_{3} >\delta \).

Co-existing critical point: \(\Lambda _{4} = ( \overline{N}_{4}, \overline{T}_{4}, \overline{I}_{4}, \overline{E}_{4} )\), where all classes are positive, where \(\phi _{1} < \frac{K_{1} r_{1} \beta _{4}}{\varepsilon ( 1-k )^{2}}\) and \(\phi _{2} > \frac{ ( K_{3} -\delta ) \beta _{4}}{\varepsilon ( 1-k )^{2}}\) for \(K_{3} >\delta \).

4 Stability, existence, and uniqueness

This section analyzes the local stability of (3.1) around each obtained critical point. The Jacobian matrix of the tumor-free critical point \(\Lambda _{1} = ( \overline{N}_{1},0, \overline{I}_{1}, \overline{E}_{1} )\) is given as

$$ J ( \Lambda _{1} ) = \begin{pmatrix} a_{11} & a_{12} & 0 &a_{14}\\ 0 & a_{22} & 0 &0\\ 0 & a_{32} & a_{33} &a_{34}\\ 0 & 0 & 0&a_{44} \end{pmatrix}, $$
(4.1)

where

$$\begin{aligned}& a_{11} = r_{1} K_{1} -2 \mu _{1} \overline{N}_{1} - \frac{ ( 1-k )^{2} \phi _{1} \varepsilon }{\beta _{4}},\qquad a_{12} =- \beta _{1} r_{1} \overline{N}_{1},\qquad a_{14} =- ( 1-k ) \phi _{1} \overline{N}_{1}, \\& a_{22} = r_{2} K_{2} d-\delta - \beta _{2} r_{2} \overline{I}_{1} + \frac{ ( 1-k )^{2} \phi _{1} \varepsilon }{\beta _{4}} \overline{N}_{1},\qquad a_{32} =- \beta _{3} \overline{I}_{1}, \\& a_{33} =- \frac{ ( ( 1-k )^{2} \phi _{2} \varepsilon -\beta _{4} ( K_{3} - \mu _{3} ) )}{\beta _{4}},\qquad a_{34} =- ( 1-k ) \phi _{2} \overline{I}_{1},\qquad a_{44} =- \beta _{4}. \end{aligned}$$

On the other hand, the characteristic equation of \(\Lambda _{1}\) is given by

$$ ( a_{11} -\lambda ) ( a_{22} -\lambda ) ( a_{33} -\lambda ) ( a_{44} -\lambda ) =0. $$
(4.2)

Theorem 4.1

Let \(\Lambda _{1}\) be the tumor-free critical point of system (3.1) and assume that \(\phi _{1} < \frac{K_{1} r_{1} \beta _{4}}{\varepsilon ( 1-k )^{2}}\) and \(\phi _{2} > \frac{ ( K_{3} -\delta ) \beta _{4}}{\varepsilon ( 1-k )^{2}}\) hold. Then \(\Lambda _{1}\) is stable local asymptotic if and only if

$$ \overline{N}_{1} < \frac{ ( \delta - r_{2} K_{2} d ) \beta _{4}}{ ( 1-k )^{2} \phi _{1} \varepsilon }, $$
(4.3)

where \(r_{2} < \frac{\delta }{K_{2} d}\).

Proof

From (4.2), it follows that

  1. (i)

    \(\lambda _{1} = r_{1} K_{1} -2 \mu _{1} \overline{N}_{1} - \frac{ ( 1-k )^{2} \phi _{1} \varepsilon }{\beta _{4}} <0\) since \(\phi _{1} < \frac{K_{1} r_{1} \beta _{4}}{\varepsilon ( 1-k )^{2}}\),

  2. (ii)

    \(\lambda _{2} = r_{2} K_{2} d-\delta - \beta _{2} r_{2} \overline{I}_{1} + \frac{ ( 1-k )^{2} \phi _{1} \varepsilon }{\beta _{4}} \overline{N}_{1} <0\Rightarrow \overline{N}_{1} < \frac{ ( \delta - r_{2} K_{2} d ) \beta _{4}}{ ( 1-k )^{2} \phi _{1} \varepsilon }\), where \(r_{2} < \frac{\delta }{K_{2} d}\),

  3. (iii)

    \(\lambda _{3} =- \frac{ ( ( 1-k )^{2} \phi _{2} \varepsilon -\beta _{4} ( K_{3} - \mu _{3} ) )}{\beta _{4}} <0\), since \(\phi _{2} > \frac{ ( K_{3} -\delta ) \beta _{4}}{\varepsilon ( 1-k )^{2}}\),

  4. (iv)

    \(\lambda _{4} =- \beta _{4} <0\).

 □

Remark 4.1

In Theorem 4.1. it is seen that any increase in the ketogenic rate affects the growth of the tumor population. It is assumed here that the invasion of the tumor cells into the normal cells is small. Thus, the tumor cell population can be eliminated from the breast tissues, since \(\Lambda _{1}\) depends on the immune response and the estrogen level.

The Jacobian matrix of the dead critical point \(\Lambda _{2} = ( 0, 0, \frac{\rho \gamma \beta _{4}}{ ( 1-k )^{2} \phi _{2} \varepsilon - \beta _{4} ( K_{3} -\delta )}, \frac{ ( 1-k ) \varepsilon }{\beta _{4}} )\) is given as

$$ J ( \Lambda _{2} ) = \begin{pmatrix} a_{11} & 0 & 0 &0 \\ 0 & a_{22} & 0 &0 \\ 0 & a_{32} & a_{33}&a_{34} \\ 0 & 0 & 0 & a_{44} \end{pmatrix}, $$
(4.4)

where

$$\begin{aligned}& a_{11} = r_{1} K_{1} - \frac{ ( 1-k )^{2} \phi _{1} \varepsilon }{\beta _{4}}, \qquad a_{22} = r_{2} K_{2} d-\delta - \beta _{2} r_{2} \overline{I}_{2},\qquad a_{32} =- \beta _{3} \overline{I}_{2}, \\& a_{33} =- \frac{ ( ( 1-k )^{2} \phi _{2} \varepsilon -\beta _{4} ( K_{3} - \mu _{3} ) )}{\beta _{4}},\qquad a_{34} =- ( 1-k ) \phi _{2} \overline{I}_{2},\qquad a_{44} =- \beta _{4}. \end{aligned}$$

Theorem 4.2

Assume that \(\phi _{1} > \frac{K_{1} r_{1} \beta _{4}}{\varepsilon ( 1-k )^{2}}\) and \(\phi _{2} > \frac{ ( K_{3} -\delta ) \beta _{4}}{\varepsilon ( 1-k )^{2}}\) holds. The dead critical point \(\Lambda _{2}\) of system (3.1) is stable local asymptotic if and only if

$$ \overline{I}_{2} > \frac{ ( r_{2} K_{2} d-\delta ) \beta _{4}}{ ( 1-k )^{2} \phi _{1} \varepsilon }, $$
(4.5)

where \(r_{2} > \frac{\delta }{K_{2} d}\).

Proof

From (4.4), we obtain the following:

  1. (i)

    \(\lambda _{1} = r_{1} K_{1} - \frac{ ( 1-k )^{2} \phi _{1} \varepsilon }{\beta _{4}} <0\) if \(\phi _{1} > \frac{K_{1} r_{1} \beta _{4}}{\varepsilon ( 1-k )^{2}}\),

  2. (ii)

    \(\lambda _{2} = r_{2} K_{2} d-\delta - \beta _{2} r_{2} \overline{I}_{2} <0\Rightarrow \overline{I}_{2} > \frac{ ( r_{2} K_{2} d-\delta ) \beta _{4}}{ ( 1-k )^{2} \phi _{1} \varepsilon }\), where \(r_{2} > \frac{\delta }{K_{2} d}\),

  3. (iii)

    \(\lambda _{3} =- \frac{ ( ( 1-k )^{2} \phi _{2} \varepsilon -\beta _{4} ( K_{3} - \mu _{3} ) )}{\beta _{4}} <0\), since \(\phi _{2} > \frac{ ( K_{3} -\delta ) \beta _{4}}{\varepsilon ( 1-k )^{2}}\),

  4. (iv)

    \(\lambda _{4} =- \beta _{4} <0\).

 □

Remark 4.2

In Theorem 4.2. it is shown that applying a low ketogenic diet increases the growth of the tumor population. In this scenario, the damage of DNA causes a malignant class in the breast tissues. The competition between the tumor cells and the immune system is intense, and the effect of the anti-cancer drug tamoxifen decreases the tumor population. However, the increase of the estrogen level also affected the healthy cell population to extinct as well.

Now, we consider the Jacobian matrix of dead critical point 2; \(\Lambda _{3} = ( 0, \overline{T}_{3}, \overline{I}_{3}, \frac{ ( 1-k ) \varepsilon }{\beta _{4}} )\), now we have

$$ J ( \Lambda _{1} ) = \begin{pmatrix} a_{11} & 0 & 0 &0\\ a_{21} & a_{22} & a_{23} &0\\ 0 & a_{32} & a_{33} &a_{34} \\ 0 & 0 & 0 & a_{44} \end{pmatrix}, $$
(4.6)

where

$$\begin{aligned}& a_{11} = r_{1} K_{1} - \beta _{1} r_{1} \overline{T}_{3} - \frac{ ( 1-k )^{2} \phi _{1} \varepsilon }{\beta _{4}},\qquad a_{21} = \frac{ ( 1-k )^{2} \phi _{1} \varepsilon }{\beta _{4}} \overline{T}_{3}, \\& a_{22} = ( K_{2} d r_{2} -\delta ) -2 r_{2} \mu _{2} \overline{T}_{3} - \beta _{2} r_{2} \overline{I}_{3}, \qquad a_{23} =- \beta _{2} r_{2} \overline{I}_{3}, \qquad a_{32} =- \beta _{3} \overline{I}_{3}, \\& a_{33} = ( K_{3} - \mu _{3} - \beta _{3} \overline{T}_{3} ) - \frac{ ( 1-k )^{2} \phi _{2} \varepsilon }{\beta _{4}},\qquad a_{34} =- ( 1-k ) \phi _{2} \overline{I}_{3}, \qquad a_{44} =- \beta _{4}. \end{aligned}$$

The characteristic equation of the dead critical point 2 is given by

$$ ( a_{11} -\lambda ) ( a_{44} -\lambda ) \bigl\{ ( a_{22} -\lambda ) ( a_{33} -\lambda ) - a_{23} a_{32} \bigr\} = 0. $$
(4.7)

Thus, the local stability for \(\Lambda _{3} = ( 0, \overline{T}_{3}, \overline{I}_{3}, \overline{E}_{3} )\) is obtained in Theorem 4.3 as follows.

Theorem 4.3

Let \(\Lambda _{3}\) be the critical point of system (3.1) and assume that \(\phi _{1} > \frac{K_{1} r_{1} \beta _{4}}{\varepsilon ( 1-k )^{2}} \). Then the following statements are true:

(i) Let \(r_{2} > \frac{\delta }{K_{2} d}\) and \(R_{0} <1\). If

$$\begin{aligned}& \overline{T}_{3} \in \biggl( \frac{ ( K_{3} - \mu _{3} ) \beta _{4} - ( 1-k )^{2} \phi _{2} \varepsilon }{\beta _{4} \beta _{3}}, \frac{K_{2} d r_{2} -\delta }{2 r_{2} \mu _{2}} \biggr), \\& \overline{I}_{3} \in \biggl( \frac{ \{ ( K_{2} d r_{2} + K_{3} ) \beta _{4} - ( ( \mu _{3} +\delta ) \beta _{4} + ( 1-k )^{2} \phi _{2} \varepsilon ) \} - ( 2 r_{2} \mu _{2} + \beta _{3} ) \beta _{4} \overline{T}_{3}}{\beta _{4} \beta _{2} r_{2}}, \\& \hphantom{\overline{I}_{3} \in\ } \frac{ ( K_{2} d r_{2} -\delta ) -2 r_{2} \mu _{2} \overline{T}_{3}}{\beta _{2} r_{2}} \biggr) \end{aligned}$$

and

$$ \overline{E}_{4} \in \biggl( \frac{2 r_{2} \mu _{2} ( K_{3} - \mu _{3} ) - ( K_{2} d r_{2} -\delta ) \beta _{3}}{2 r_{2} \mu _{2} ( 1-k ) \phi _{2}}, \frac{ ( K_{2} d r_{2} + K_{3} - ( \mu _{3} +\delta ) )}{ ( 1-k ) \phi _{2}} \biggr) $$

then both roots are real or complex conjugates with negative real parts and \(\vert \arg ( \lambda _{i} ) \vert > \frac{\alpha \pi }{2}\) (\(i=1, 2, 3, 4 \)) is equivalent to the Routh-Hurwitz criteria. This implies that \(\Lambda _{3}\) is locally asymptotically stable.

(ii) Let \(r_{2} > \frac{\delta }{K_{2} d}\) and \(R_{0} <1\). If

$$ \overline{T}_{3} \in \biggl( 0, \frac{ ( K_{3} - \mu _{3} ) \beta _{4} - ( 1-k )^{2} \phi _{2} \varepsilon }{\beta _{4} \beta _{3}} \biggr), \qquad \overline{I}_{3} \in \biggl( \frac{ ( K_{2} d r_{2} -\delta ) -2 r_{2} \mu _{2} \overline{T}_{3}}{\beta _{2} r_{2}}, \infty \biggr) $$

and

$$ \overline{E}_{4} \in \biggl( \frac{2 r_{2} \mu _{2} ( K_{3} - \mu _{3} ) - ( K_{2} d r_{2} -\delta ) \beta _{3}}{2 r_{2} \mu _{2} ( 1-k ) \phi _{2}}, \frac{ ( K_{2} d r_{2} + K_{3} - ( \mu _{3} +\delta ) )}{ ( 1-k ) \phi _{2}} \biggr), $$

then both roots are complex conjugate with positive real parts and

$$\begin{aligned}& \biggl\vert \tan ^{-1} \biggl( - \biggl(4 \bigl( ( K_{2} d r_{2} -\delta ) -2 r_{2} \mu _{2} \overline{T}_{3} - \beta _{2} r_{2} \overline{I}_{3} \bigr)\\& \qquad {}\times \biggl( ( K_{3} - \mu _{3} - \beta _{3} \overline{T}_{3} ) - \frac{ ( 1-k )^{2} \phi _{2} \varepsilon }{\beta _{4}} \biggr)( 1- R_{0} ) \\& \qquad {}- \biggl( ( K_{2} d r_{2} -\delta ) -2 r_{2} \mu _{2} \overline{T}_{3} - \beta _{2} r_{2} \overline{I}_{3} + ( K_{3} - \mu _{3} - \beta _{3} \overline{T}_{3} ) - \frac{ ( 1-k )^{2} \phi _{2} \varepsilon }{\beta _{4}} \biggr)^{2} \biggr)^{\frac{1}{2}} \\& \qquad {}\times \biggl({ \biggl( ( K_{2} d r_{2} - \delta ) -2 r_{2} \mu _{2} \overline{T}_{3} - \beta _{2} r_{2} \overline{I}_{3} + ( K_{3} - \mu _{3} - \beta _{3} \overline{T}_{3} ) - \frac{ ( 1-k )^{2} \phi _{2} \varepsilon }{\beta _{4}} \biggr)} \biggr)^{-1} \biggr) \biggr\vert \\& \quad > \frac{\alpha \pi }{2}. \end{aligned}$$

This implies that \(\Lambda _{3}\) is locally asymptotically stable.

Proof

From (4.7), we obtain

$$ \lambda _{1} = r_{1} K_{1} - \frac{ ( 1-k )^{2} \phi _{1} \varepsilon }{\beta _{4}} < 0\quad \text{if } \phi _{1} > \frac{K_{1} r_{1} \beta _{4}}{\varepsilon ( 1-k )^{2}} $$
(4.8)

and \(\lambda _{4} =- \beta _{4} <0 \). In this case, we need to consider only the characteristic equation

$$ \lambda ^{2} - ( a_{22} + a_{33} ) \lambda + a_{22} a_{33} - a_{23} a_{32} =0. $$
(4.9)

Equation (4.9) shows the basic reproductive number, which is

$$ R_{0} = \frac{a_{23} a_{32}}{a_{22} a_{33}} $$
(4.10)

for the characteristic equation

$$ \lambda ^{2} - ( a_{22} + a_{33} ) \lambda + a_{22} a_{33} \biggl( 1- \frac{a_{23} a_{32}}{a_{22} a_{33}} \biggr) =0, $$

which implies

$$ \lambda ^{2} - \sum_{i=2}^{3} a_{ii} \lambda + \prod_{i=2}^{3} a_{ii} ( 1- R_{0} ) =0. $$
(4.11)

(i) Let us consider the case where \(\Delta = ( a_{22} + a_{33} )^{2} -4 a_{22} a_{33} ( 1- R_{0} ) >0\). For \(R_{0} <1\) and \(r_{2} > \frac{\delta }{K_{2} d}\), we have

$$ \begin{gathered} \frac{ ( K_{3} - \mu _{3} ) \beta _{4} - ( 1-k )^{2} \phi _{2} \varepsilon }{\beta _{4} \beta _{3}} < \overline{T}_{3} < \frac{K_{2} d r_{2} -\delta }{2 r_{2} \mu _{2}} \quad \text{and} \\ \overline{I}_{3} < \frac{ ( K_{2} d r_{2} -\delta ) -2 r_{2} \mu _{2} \overline{T}_{3}}{\beta _{2} r_{2}}, \end{gathered} $$
(4.12)

which implies that \(\Delta >0\). Moreover, computations show that

$$\begin{aligned}& a_{22} + a_{33} < 0 \\& \quad \Rightarrow \quad \frac{ \{ ( K_{2} d r_{2} + K_{3} ) \beta _{4} - ( ( \mu _{3} +\delta ) \beta _{4} + ( 1-k )^{2} \phi _{2} \varepsilon ) \} - ( 2 r_{2} \mu _{2} + \beta _{3} ) \beta _{4} \overline{T}_{3}}{\beta _{4} \beta _{2} r_{2}} < \overline{I}_{3}, \end{aligned}$$
(4.13)

where

$$ \overline{T}_{3} < \frac{ ( K_{2} d r_{2} + K_{3} ) \beta _{4} - ( ( \mu _{3} +\delta ) \beta _{4} + ( 1-k )^{2} \phi _{2} \varepsilon )}{ ( 2 r_{2} \mu _{2} + \beta _{3} ) \beta _{4}} $$
(4.14)

and

$$ \overline{E}_{4} < \frac{ ( K_{2} d r_{2} + K_{3} - ( \mu _{3} +\delta ) )}{ ( 1-k ) \phi _{2}}. $$
(4.15)

Considering (4.12)–(4.14), we obtain

$$\begin{aligned}& \frac{ \{ ( K_{2} d r_{2} + K_{3} ) \beta _{4} - ( ( \mu _{3} +\delta ) \beta _{4} + ( 1-k )^{2} \phi _{2} \varepsilon ) \} - ( 2 r_{2} \mu _{2} + \beta _{3} ) \beta _{4} \overline{T}_{3}}{\beta _{4} \beta _{2} r_{2}} \\& \quad < \overline{I}_{3} < \frac{ ( K_{2} d r_{2} -\delta ) -2 r_{2} \mu _{2} \overline{T}_{3}}{\beta _{2} r_{2}} \end{aligned}$$
(4.16)

and

$$\begin{aligned}& \frac{ ( K_{3} - \mu _{3} ) \beta _{4} - ( 1-k )^{2} \phi _{2} \varepsilon }{\beta _{4} \beta _{3}} \\& \quad < \overline{T}_{3} < \frac{K_{2} d r_{2} -\delta }{2 r_{2} \mu _{2}} < \frac{ ( K_{2} d r_{2} + K_{3} ) \beta _{4} - ( ( \mu _{3} +\delta ) \beta _{4} + ( 1-k )^{2} \phi _{2} \varepsilon )}{ ( 2 r_{2} \mu _{2} + \beta _{3} ) \beta _{4}}, \end{aligned}$$
(4.17)

where

$$ \overline{E}_{4} > \frac{2 r_{2} \mu _{2} ( K_{3} - \mu _{3} ) - ( K_{2} d r_{2} -\delta ) \beta _{3}}{2 r_{2} \mu _{2} ( 1-k ) \phi _{2}}. $$
(4.18)

From (4.15) and (4.18), we get

$$ \frac{2 r_{2} \mu _{2} ( K_{3} - \mu _{3} ) - ( K_{2} d r_{2} -\delta ) \beta _{3}}{2 r_{2} \mu _{2} ( 1-k ) \phi _{2}} < \overline{E}_{4} < \frac{ ( K_{2} d r_{2} + K_{3} - ( \mu _{3} +\delta ) )}{ ( 1-k ) \phi _{2}} \quad \text{for } r_{2} > \frac{\delta }{K_{2} d}. $$
(4.19)

(ii) Let us consider the case of complex roots with positive real parts. First of all, if

$$ a_{22} + a_{33} >0 $$
(4.20)

then we obtain positive real parts. Thus, for the inequality

$$\begin{aligned}& a_{22} + a_{33} >0 \\& \quad \Rightarrow \quad \frac{ \{ ( K_{2} d r_{2} + K_{3} ) \beta _{4} - ( ( \mu _{3} +\delta ) \beta _{4} + ( 1-k )^{2} \phi _{2} \varepsilon ) \} - ( 2 r_{2} \mu _{2} + \beta _{3} ) \beta _{4} \overline{T}_{3}}{\beta _{4} \beta _{2} r_{2}} > \overline{I}_{3}, \end{aligned}$$
(4.21)

where

$$ \overline{T}_{3} < \frac{ ( K_{2} d r_{2} + K_{3} ) \beta _{4} - ( ( \mu _{3} +\delta ) \beta _{4} + ( 1-k )^{2} \phi _{2} \varepsilon )}{ ( 2 r_{2} \mu _{2} + \beta _{3} ) \beta _{4}} $$
(4.22)

and

$$ \overline{E}_{4} < \frac{ ( K_{2} d r_{2} + K_{3} - ( \mu _{3} +\delta ) )}{ ( 1-k ) \phi _{2}}, $$
(4.23)

we end up with (4.20).

Considering the condition \(\Delta = ( a_{22} + a_{33} )^{2} -4 a_{22} a_{33} ( 1- R_{0} ) <0\), we have

$$ R_{0} < \frac{4 a_{22} a_{33} - ( a_{22} + a_{33} )^{2}}{4 a_{22} a_{33}} < 1. $$
(4.24)

Moreover, computations reveal that \(4 a_{22} a_{33} > ( a_{22} + a_{33} )^{2}\), if

$$ \overline{T}_{3} < \frac{ ( K_{3} - \mu _{3} ) \beta _{4} - ( 1-k )^{2} \phi _{2} \varepsilon }{\beta _{4} \beta _{3}} $$
(4.25)

and

$$ \overline{I}_{3} > \frac{ ( K_{2} d r_{2} -\delta ) -2 r_{2} \mu _{2} \overline{T}_{3}}{\beta _{2} r_{2}}, $$
(4.26)

where

$$ \overline{T}_{3} < \frac{K_{2} d r_{2} -\delta }{2 r_{2} \mu _{2}}. $$
(4.27)

From (4.22), (4.25), and (4.27), we get

$$ \overline{T}_{3} < \frac{ ( K_{3} - \mu _{3} ) \beta _{4} - ( 1-k )^{2} \phi _{2} \varepsilon }{\beta _{4} \beta _{3}},\qquad \overline{I}_{3} > \frac{ ( K_{2} d r_{2} -\delta ) -2 r_{2} \mu _{2} \overline{T}_{3}}{\beta _{2} r_{2}} $$
(4.28)

and

$$ \frac{2 r_{2} \mu _{2} ( K_{3} - \mu _{3} ) - ( K_{2} d r_{2} -\delta ) \beta _{3}}{2 r_{2} \mu _{2} ( 1-k ) \phi _{2}} < \overline{E}_{4} < \frac{ ( K_{2} d r_{2} + K_{3} - ( \mu _{3} +\delta ) )}{ ( 1-k ) \phi _{2}} \quad \text{for } r_{2} > \frac{\delta }{K_{2} d}. $$
(4.29)

 □

Remark 4.3

It is shown in Theorem 4.3 that the reproduction number has a dominant role to play in the dynamical system’s stability. In this scenario, we assumed a weak immune response with low ketogenic support. It is seen that without the control parameters, the immune system is not strong enough to defend the tissues and the human body from the invasion of the malignant population.

Let us consider the Jacobian matrix of the co-existing critical point: \(\Lambda _{4} = ( \overline{N}_{4}, \overline{T}_{4}, \overline{I}_{4}, \overline{E}_{4} )\), which has the form

$$ J ( \Lambda _{4} ) = \begin{pmatrix} a_{11} & a_{12} & 0 &a_{14}\\ a_{21} & a_{22} & a_{23} &a_{24}\\ 0 & a_{32} & a_{33} &a_{34}\\ 0 & 0 & 0 & a_{44} \end{pmatrix}, $$
(4.30)

where

$$\begin{aligned}& a_{11} = r_{1} K_{1} -2 r_{1} \mu _{1} \overline{N}_{4} - r_{1} \beta _{1} \overline{T}_{4} - \frac{ ( 1-k )^{2} \phi _{1} \varepsilon }{\beta _{4}},\qquad a_{12} =- \beta _{1} r_{1} \overline{N}_{4}, \\& a_{14} =- ( 1-k ) \phi _{1} \overline{N}_{4},\qquad a_{21} = \frac{ ( 1-k )^{2} \phi _{1} \varepsilon \overline{T}_{4}}{\beta _{4}},\\& a_{22} = ( K_{2} d r_{2} -\delta ) -2 r_{2} \mu _{2} \overline{T}_{4} - \beta _{2} r_{2} \overline{I}_{4} + \frac{ ( 1-k )^{2} \phi _{1} \varepsilon \overline{N}_{4}}{\beta _{4}}, \\& a_{23} =- \beta _{2} r_{2} \overline{T}_{4},\qquad a_{24} = ( 1-k ) \phi _{1} \overline{N}_{4} \overline{T}_{4},\qquad a_{32} =- \beta _{3} \overline{I}_{4},\\& a_{33} = ( K_{3} - \mu _{3} - \beta _{3} \overline{T}_{4} ) - \frac{ ( 1-k )^{2} \phi _{2} \varepsilon }{\beta _{4}},\\& a_{34} =- ( 1-k ) \phi _{2} \overline{I}_{4},\qquad a_{44} =- \beta _{4}. \end{aligned}$$

The characteristic equation of (4.30) around \(\Lambda _{4}\) is given by

$$ ( a_{11} -\lambda ) \bigl\{ \lambda ^{2} - ( a_{22} + a_{33} ) \lambda + a_{22} a_{33} ( 1- R_{0} ) \bigr\} - ( a_{33} - \lambda ) a_{12} a_{21} =0 $$
(4.31)

and

$$ \lambda _{4} =- \beta _{4} < 0, $$
(4.32)

while (4.31) is a cubic equation of the form

$$\begin{aligned}& \lambda ^{3} + ( - a_{11} - a_{22} - a_{33} ) \lambda ^{2} + \bigl( a_{11} a_{22} + a_{11} a_{33} - a_{12} a_{21} + a_{23} a_{32} ( 1- R_{0} ) \bigr) \lambda \\& \quad {}+ \bigl( a_{33} a_{12} a_{21} - a_{11} a_{22} a_{33} ( 1- R_{0} ) \bigr) =0. \end{aligned}$$
(4.33)

Theorem 4.4

Let \(\Lambda _{4}\) be the equilibrium point of system (3.1) and assume that \(R_{0} <1\), \(\phi _{1} < \frac{K_{1} r_{1} \beta _{4}}{\varepsilon ( 1-k )^{2}}\) and \(\frac{ ( K_{3} -\delta ) \beta _{4}}{\varepsilon ( 1-k )^{2}} < \phi _{2} < \frac{ ( K_{3} - \mu _{3} ) \beta _{4}}{\varepsilon ( 1-k )^{2}}\). Then the following statements are true:

  1. (i)

    Let \(r_{1} > \frac{ ( 1-k )^{2} \phi _{1} \varepsilon }{2 \beta _{4} \mu _{1}}\) and \(r_{2} > \frac{\delta }{K_{2} d} \). If

    $$ \overline{T}_{4} > \frac{r_{1} K_{1} \beta _{4} - ( 1-k )^{2} \phi _{1} \varepsilon }{ ( r_{1} \beta _{1} +2 r_{2} \mu _{2} ) \beta _{4}}\quad \textit{and}\quad \overline{I}_{4} > \frac{K_{2} d r_{2} -\delta }{\beta _{2} r_{2}}, $$

    where \(\beta _{3} > \frac{ ( r_{1} \beta _{1} +2 r_{2} \mu _{2} ) ( ( K_{3} - \mu _{3} ) \beta _{4} - ( 1-k )^{2} \phi _{2} \varepsilon )}{r_{1} K_{1} \beta _{4} - ( 1-k )^{2} \phi _{1} \varepsilon }\) then all the roots of (4.33) are real. This implies that \(\Lambda _{4}\) is locally asymptotically stable.

  2. (ii)

    Let \(r_{1} \geq \frac{ ( 1-k )^{2} \phi _{1} \varepsilon }{2 \beta _{4} \mu _{1}}\) and \(r_{2} > \frac{\delta }{K_{2} d} \). If

    $$\begin{aligned}& \overline{N}_{4} \in \biggl( \frac{ ( r_{1} K_{1} - r_{1} \beta _{1} \overline{T}_{4} ) \beta _{4} - ( 1-k )^{2} \phi _{1} \varepsilon }{\beta _{4} r_{1} ( 2 \mu _{1} + \beta _{1} )}, \infty \biggr), \\& \overline{T}_{4} \in \biggl[ \frac{r_{1} K_{1} \beta _{4} - ( 1-k )^{2} \phi _{1} \varepsilon }{ ( r_{1} \beta _{1} +2 r_{2} \mu _{2} ) \beta _{4}} , \frac{r_{1} K_{1} \beta _{4} - ( 1-k )^{2} \phi _{1} \varepsilon }{r_{1} \beta _{1} \beta _{4}} \biggr),\\& \overline{I}_{4} \in \biggl[ \frac{K_{2} d r_{2} -\delta }{\beta _{2} r_{2}} , \infty \biggr) \end{aligned}$$

    and

    $$ \biggl( \frac{ ( 1-k ) \phi _{1} \overline{E}_{4} +2 r_{2} \mu _{2}}{ ( 1-k ) \phi _{1} \overline{E}_{4}} \biggr) > \frac{\overline{N}_{4}}{\overline{T}_{4}}, $$

    where \(\beta _{3} = \frac{ ( r_{1} \beta _{1} +2 r_{2} \mu _{2} ) ( ( K_{3} - \mu _{3} ) \beta _{4} - ( 1-k )^{2} \phi _{2} \varepsilon )}{r_{1} K_{1} \beta _{4} - ( 1-k )^{2} \phi _{1} \varepsilon } \), then there is one real root and one complex root with its complex conjugate, which implies that \(\Lambda _{4}\) is locally asymptotically stable.

Proof

(i) If the discriminant of (4.33) is positive, then the Routh-Hurwitz conditions are necessary and sufficient conditions for locally asymptotically stability of \(\Lambda _{4} \). Thus, we first consider the conditions for a positive discriminant of the cubic polynomial (4.33):

If

$$\begin{aligned}& a_{11} + a_{22} + a_{33} < 0, \end{aligned}$$
(4.34)
$$\begin{aligned}& a_{33} a_{12} a_{21} - a_{11} a_{22} a_{33} ( 1- R_{0} ) >0, \end{aligned}$$
(4.35)
$$\begin{aligned}& a_{11} a_{22} + a_{11} a_{33} - a_{12} a_{21} + a_{23} a_{32} ( 1- R_{0} ) >0 \end{aligned}$$
(4.36)

and

$$\begin{aligned}& ( - a_{11} - a_{22} - a_{33} ) \bigl( a_{11} a_{22} + a_{11} a_{33} - a_{12} a_{21} + a_{23} a_{32} ( 1- R_{0} ) \bigr) \\& \quad > a_{33} a_{12} a_{21} - a_{11} a_{22} a_{33} ( 1- R_{0} ), \end{aligned}$$
(4.37)

then the discriminant of (4.33) is positive.

Because of (4.34), we have

$$\begin{aligned}& \biggl( \frac{ ( 1-k )^{2} \phi _{1} \varepsilon }{\beta _{4}} -2 r_{1} \mu _{1} \biggr) \overline{N}_{4} + \biggl( r_{1} K_{1} - ( r_{1} \beta _{1} +2 r_{2} \mu _{2} ) \overline{T}_{4} - \frac{ ( 1-k )^{2} \phi _{1} \varepsilon }{\beta _{4}} \biggr) \\& \quad {}+ ( K_{2} d r_{2} -\delta - \beta _{2} r_{2} \overline{I}_{4} ) + \biggl( K_{3} - \mu _{3} - \frac{ ( 1-k )^{2} \phi _{2} \varepsilon }{\beta _{4}} - \beta _{3} \overline{T}_{4} \biggr) < 0, \end{aligned}$$

where we obtain

$$\begin{aligned}& \frac{ ( 1-k )^{2} \phi _{1} \varepsilon }{\beta _{4}} -2 r_{1} \mu _{1} < 0\quad \Rightarrow\quad r_{1} > \frac{ ( 1-k )^{2} \phi _{1} \varepsilon }{2 \beta _{4} \mu _{1}}, \end{aligned}$$
(4.38)
$$\begin{aligned}& r_{1} K_{1} - ( r_{1} \beta _{1} +2 r_{2} \mu _{2} ) \overline{T}_{4} - \frac{ ( 1-k )^{2} \phi _{1} \varepsilon }{\beta _{4}} < 0 \\& \quad \Rightarrow\quad \overline{T}_{4} > \frac{r_{1} K_{1} \beta _{4} - ( 1-k )^{2} \phi _{1} \varepsilon }{ ( r_{1} \beta _{1} +2 r_{2} \mu _{2} ) \beta _{4}}\quad \text{for } \phi _{1} < \frac{K_{1} r_{1} \beta _{4}}{\varepsilon ( 1-k )^{2}}, \end{aligned}$$
(4.39)
$$\begin{aligned}& K_{2} d r_{2} -\delta - \beta _{2} r_{2} \overline{I}_{4} < 0\quad \Rightarrow\quad \overline{I}_{4} > \frac{K_{2} d r_{2} -\delta }{\beta _{2} r_{2}}\quad \text{for } r_{2} > \frac{\delta }{K_{2} d} \end{aligned}$$
(4.40)

and

$$\begin{aligned}& K_{3} - \mu _{3} - \frac{ ( 1-k )^{2} \phi _{2} \varepsilon }{\beta _{4}} - \beta _{3} \overline{T}_{4} < 0 \\& \quad \Rightarrow\quad \overline{T}_{4} > \frac{ ( K_{3} - \mu _{3} ) \beta _{4} - ( 1-k )^{2} \phi _{2} \varepsilon }{\beta _{4} \beta _{3}}\quad \text{for } \phi _{2} < \frac{ ( K_{3} - \mu _{3} ) \beta _{4}}{\varepsilon ( 1-k )^{2}}. \end{aligned}$$
(4.41)

Since \(\Lambda _{4}\) is a positive critical point for \(\phi _{2} > \frac{ ( K_{3} -\delta ) \beta _{4}}{\varepsilon ( 1-k )^{2}}\), we see from (4.41) that \(\mu _{3} <\delta \). Considering both (4.39) and (4.41), we have

$$ \overline{T}_{4} > \frac{r_{1} K_{1} \beta _{4} - ( 1-k )^{2} \phi _{1} \varepsilon }{ ( r_{1} \beta _{1} +2 r_{2} \mu _{2} ) \beta _{4}} > \frac{ ( K_{3} - \mu _{3} ) \beta _{4} - ( 1-k )^{2} \phi _{2} \varepsilon }{\beta _{4} \beta _{3}}, $$
(4.42)

where \(\beta _{3} > \frac{ ( r_{1} \beta _{1} +2 r_{2} \mu _{2} ) ( ( K_{3} - \mu _{3} ) \beta _{4} - ( 1-k )^{2} \phi _{2} \varepsilon )}{r_{1} K_{1} \beta _{4} - ( 1-k )^{2} \phi _{1} \varepsilon } \).

On the other side, it is evident that (4.35) holds, since \(a_{33} <0\). Moreover, considering (4.36) and (4.37), we obtain the result that it is satisfied with the conditions in (4.38)–(4.42). This completes the proof of (i).

(ii) If the discriminant of (4.33) is negative, then there is only one real root and one complex root with its complex conjugate. Let us assume that

$$\begin{aligned} P ( \lambda ) =& ( \lambda +a ) ( \lambda -b-i\omega ) ( \lambda -b+i\omega ) \\ =&\lambda ^{3} + ( - a_{11} - a_{22} - a_{33} ) \lambda ^{2} + \bigl( a_{11} a_{22} + a_{11} a_{33} - a_{12} a_{21} + a_{23} a_{32} ( 1- R_{0} ) \bigr) \lambda \\ &{}+ \bigl( a_{33} a_{12} a_{21} - a_{11} a_{22} a_{33} ( 1- R_{0} ) \bigr) =0. \end{aligned}$$
(4.43)

Then

$$\begin{aligned}& a-2b=- a_{11} - a_{22} - a_{33}, \\& b^{2} + \omega ^{2} -2ab= a_{11} a_{22} + a_{11} a_{33} - a_{12} a_{21} + a_{23} a_{32} ( 1- R_{0} ) \end{aligned}$$

and

$$ a \bigl( b^{2} + \omega ^{2} \bigr) = a_{33} a_{12} a_{21} - a_{11} a_{22} a_{33} ( 1- R_{0} ). $$

By the result in [36], since \(a-2b>0\) and \(b \geq 0\),

$$ a_{11} + a_{22} + a_{33} \leq 0, $$
(4.44)

which holds for

$$\begin{aligned}& \frac{ ( 1-k )^{2} \phi _{1} \varepsilon }{\beta _{4}} -2 r_{1} \mu _{1} \leq 0 \quad \Rightarrow\quad r_{1} \geq \frac{ ( 1-k )^{2} \phi _{1} \varepsilon }{2 \beta _{4} \mu _{1}}, \end{aligned}$$
(4.45)
$$\begin{aligned}& r_{1} K_{1} - ( r_{1} \beta _{1} +2 r_{2} \mu _{2} ) \overline{T}_{4} - \frac{ ( 1-k )^{2} \phi _{1} \varepsilon }{\beta _{4}} \leq 0 \\& \quad \Rightarrow\quad \overline{T}_{4} \geq \frac{r_{1} K_{1} \beta _{4} - ( 1-k )^{2} \phi _{1} \varepsilon }{ ( r_{1} \beta _{1} +2 r_{2} \mu _{2} ) \beta _{4}}\quad \text{for } \phi _{1} < \frac{K_{1} r_{1} \beta _{4}}{\varepsilon ( 1-k )^{2}}, \end{aligned}$$
(4.46)
$$\begin{aligned}& K_{2} d r_{2} -\delta - \beta _{2} r_{2} \overline{I}_{4} \leq 0\quad \Rightarrow\quad \overline{I}_{4} \geq \frac{K_{2} d r_{2} -\delta }{\beta _{2} r_{2}}\quad \text{for } r_{2} > \frac{\delta }{K_{2} d} \end{aligned}$$
(4.47)

and

$$ K_{3} - \mu _{3} - \frac{ ( 1-k )^{2} \phi _{2} \varepsilon }{\beta _{4}} - \beta _{3} \overline{T}_{4} \leq 0\quad \Rightarrow\quad \overline{T}_{4} \geq \frac{ ( K_{3} - \mu _{3} ) \beta _{4} - ( 1-k )^{2} \phi _{2} \varepsilon }{\beta _{4} \beta _{3}} $$
(4.48)

for \(\frac{ ( K_{3} -\delta ) \beta _{4}}{\varepsilon ( 1-k )^{2}} < \phi _{2} < \frac{ ( K_{3} - \mu _{3} ) \beta _{4}}{\varepsilon ( 1-k )^{2}}\) where \(\mu _{3} <\delta \). Considering both (4.46) and (4.48), we have

$$ \overline{T}_{4} \geq \frac{r_{1} K_{1} \beta _{4} - ( 1-k )^{2} \phi _{1} \varepsilon }{ ( r_{1} \beta _{1} +2 r_{2} \mu _{2} ) \beta _{4}}, $$
(4.49)

where \(\beta _{3} = \frac{ ( r_{1} \beta _{1} +2 r_{2} \mu _{2} ) ( ( K_{3} - \mu _{3} ) \beta _{4} - ( 1-k )^{2} \phi _{2} \varepsilon )}{r_{1} K_{1} \beta _{4} - ( 1-k )^{2} \phi _{1} \varepsilon } \).

On the other side, from [36] we have

$$ b^{2} + \omega ^{2} -2ab \geq 0\quad \Rightarrow\quad b^{2} + \omega ^{2} \geq 2ab\quad \Rightarrow\quad b^{2} \sec ^{2} \theta \geq 2ab \geq 4 b^{2}, $$
(4.50)

which implies that \(\vert \arg ( \lambda ) \vert =\theta \geq \frac{\pi }{3}\) and from \(\vert \arg ( \lambda ) \vert > \frac{\alpha \pi }{2}\), we have \(\alpha < \frac{2}{3} \). Thus,

$$ a_{11} a_{22} + a_{11} a_{33} - a_{12} a_{21} + a_{23} a_{32} ( 1- R_{0} ) \geq 0, $$
(4.51)

which holds for the conditions in (4.45)–(4.49).

At last, we consider

$$ a_{33} a_{12} a_{21} - a_{11} a_{22} a_{33} ( 1- R_{0} ) >0, $$

which implies

$$\begin{aligned}& \beta _{1} r_{1} \overline{N}_{4} \overline{T}_{4} \frac{ ( 1-k )^{2} \phi _{1} \varepsilon }{\beta _{4}} \\& \quad >- \biggl( r_{1} K_{1} -2 r_{1} \mu _{1} \overline{N}_{4} - r_{1} \beta _{1} \overline{T}_{4} - \frac{ ( 1-k )^{2} \phi _{1} \varepsilon }{\beta _{4}} \biggr) \\& \qquad {}\times\biggl( ( K_{2} d r_{2} -\delta ) -2 r_{2} \mu _{2} \overline{T}_{4} - \beta _{2} r_{2} \overline{I}_{4} + \frac{ ( 1-k )^{2} \phi _{1} \varepsilon \overline{N}_{4}}{\beta _{4}} \biggr) ( 1- R_{0} ) \end{aligned}$$
(4.52)

we have

$$ \overline{N}_{4} > \frac{ ( r_{1} K_{1} - r_{1} \beta _{1} \overline{T}_{4} ) \beta _{4} - ( 1-k )^{2} \phi _{1} \varepsilon }{\beta _{4} r_{1} ( 2 \mu _{1} + \beta _{1} )}, $$
(4.53)

where

$$ \overline{T}_{4} < \frac{r_{1} K_{1} \beta _{4} - ( 1-k )^{2} \phi _{1} \varepsilon }{r_{1} \beta _{1} \beta _{4}} $$
(4.54)

and

$$ \biggl( \frac{ ( 1-k ) \phi _{1} \overline{E}_{4} +2 r_{2} \mu _{2}}{ ( 1-k ) \phi _{1} \overline{E}_{4}} \biggr) > \frac{\overline{N}_{4}}{\overline{T}_{4}}. $$
(4.55)

 □

Remark 4.4

In Theorem 4.4, it is shown that, if the source rate of estrogen is low, then the interaction between the normal cells and tumor cells continues. At the same time, the growth of both populations has an inverse relation. According to the ketogenic assistance and supplements of an immune booster, a robust immune response is expected in this competition.

Example 4.1

In this example, the theoretical results are demonstrated by using numerical simulations. For this purpose, we wrote code and ran it using the MATLAB version 2019. The initial values of (3.1) are \(N ( 0 ) =200\), \(T ( 0 ) =50\), \(I ( 0 ) =50\), \(E ( 0 ) =2\).

Figure 1 shows the limited competition between cancer and normal cell populations. After the cancer cells appear, some immune booster supplements support the normal cells in the interaction. The red graph is the cancer cell population \(T ( t )\), while the blue graph shows the normal cell population \(N ( t ) \). It is seen that after a specific time, the tumor population becomes so strong that the immune system needs additional support from the control parameters. Therefore, to stabilize only the immune system is not sufficient for interaction against the malignant tumor.

Figure 1
figure 1

Bifurcation diagram of \(T ( t )\) and \(N ( t )\) with only one control parameter, the immune booster

In Figs. 2 and 3, we notice that introducing a ketogenic diet reduces the cancer cells. In this case, applying for the ketogenic program during the mixed-immunotherapy shows an effect on the per capita growth of the cancer cells and stabilizes the treatment to support the normal cells.

Figure 2
figure 2

Bifurcation diagram of \(T ( t )\) and \(N ( t )\) with a mixed therapy including the ketogenic diet

Figure 3
figure 3

Effect of the ketogenic diet to the cell populations \(T ( t )\) and \(N ( t ) \)

However, it is also known that increasing the rate of the ketogenic diet leads to keto-acidosis. Keto-acidosis is a composition of ketosis and acidosis. Ketosis is a substance known as ketone bodies, and acidosis is the acid of the blood, causing frequent polyuria, poor appetite, and loss of consciousness. Therefore, a rate at \(d=0.3\) is reasonable, and the interaction can be supported by anti-cancer drugs such as tamoxifen. The tumor cell is given in red, while the normal cells are denoted in blue.

5 Existence and uniqueness

Considering system (3.1) with the initial conditions \(N ( 0 ) >0\), \(T ( 0 ) >0\), \(I ( 0 ) >0\) and \(E ( 0 ) >0\), the IVP can be written in matrix form as

$$ \textstyle\begin{cases} D^{\alpha } U ( t ) =M+AU ( t ) +N ( t ) BU ( t ) +T ( t ) CU ( t )\\ \hphantom{D^{\alpha } U ( t ) =}{} +I ( t ) DU ( t ) +E(t)GU(t),\quad t\in ( 0, T],\\ U ( 0 ) = U_{0}, \end{cases} $$
(5.1)

where U(t)= [ N ( t ) T ( t ) I ( t ) E ( t ) ] and U(0)= [ N ( 0 ) T ( 0 ) I ( 0 ) E ( 0 ) ] and M= [ 0 0 ρ γ ( 1 k ) ε ] .

Let us assume that \(N ( 0 ) \leq \upsilon _{1}\), and \(T ( 0 ) >0\), \(I ( 0 ) >0\), \(E ( 0 ) >0\), when \(t>\sigma \geq 0\).

In this case, the following definitions can be applied to the main theorems in this section.

Definition 5.1

Let \(C^{*} [0, T]\) be the class of continuous column vector \(U(t)\) whose components \(N ( t ), T ( t ), I ( t ), E(t)\in C[0, T]\) are the class of continuous functions on the interval \([ 0, T ] \). The norm of \(U\in C^{*} [0, T]\) is given by

$$ \Vert U \Vert = \sup_{t} \bigl\vert e^{-Wt} N(t) \bigr\vert + \sup_{t} \bigl\vert e^{-Wt} T(t) \bigr\vert + \sup_{t} \bigl\vert e^{-Wt} I(t) \bigr\vert + \sup_{t} \bigl\vert e^{-Wt} E(t) \bigr\vert . $$

When \(t>\sigma \geq 0\), we write \(C_{\sigma }^{*} [0, T]\) and \(C_{\sigma } [0, T]\).

Definition 5.2

Let the IVP of (5.1) have a solution given by \(U\in C^{*} [0, T]\), if

  1. (i)

    \(( t, U(t) ) \in D\), \(t\in [ 0, T ]\) where \(D= [ 0,T ] \times K\) and

    $$ K= \bigl\{ \bigl( N ( t ), T ( t ),I ( t ),E(t) \bigr): \bigl\vert N ( t ) \bigr\vert \leq \upsilon _{1}, \bigl\vert T ( t ) \bigr\vert \leq \upsilon _{2}, \bigl\vert I ( t ) \bigr\vert \leq \upsilon _{3}, \bigl\vert E ( t ) \bigr\vert \leq \upsilon _{4} \bigr\} . $$
  2. (ii)

    \(U(t)\) satisfies (5.1).

Theorem 5.1

The IVP of (5.1) has a unique solution \(U\in C^{*} [ 0, T ] \).

Proof

Let us write

$$ I^{1-\alpha }\, \frac{d}{dt} U ( t ) =M+AU ( t ) +N ( t ) BU ( t ) +T ( t ) CU ( t ) +I ( t ) DU ( t ) +E(t)GU(t). $$
(5.2)

Operating with \(I^{\alpha } \), we obtain

$$ U ( t ) =U ( 0 ) + I^{\alpha } \bigl( M+AU ( t ) +N ( t ) BU ( t ) +T ( t ) CU ( t ) +I ( t ) DU ( t ) +E(t)GU(t) \bigr). $$
(5.3)

Now let \(F: C^{*} [0, T]\rightarrow C^{*} [0, T]\) be defined by

$$ FU ( t ) =U ( 0 ) + I^{\alpha } \bigl( M+AU ( t ) +N ( t ) BU ( t ) +T ( t ) CU ( t ) +I ( t ) DU ( t ) +E(t)GU(t) \bigr). $$
(5.4)

Then we get

$$\begin{aligned}& e^{-Wt} \Vert FU-FV \Vert \\& \quad = e^{-Wt} I^{\alpha } \bigl( A \bigl( U ( t ) -V ( t ) \bigr) +N ( t ) B \bigl( U ( t ) -V ( t ) \bigr) +T ( t ) C \bigl( U ( t ) -V ( t ) \bigr) \\& \qquad {}+I ( t ) D \bigl( U ( t ) -V(t) \bigr) +E ( t ) G \bigl( U ( t ) -V(t) \bigr) \bigr) \\& \quad \leq \frac{1}{\Gamma ( \alpha )} \int _{0}^{t} ( t-s )^{\alpha -1} e^{-W ( t-s )} \bigl( U ( s ) -V(s) \bigr) e^{-Ws} ( A+ \upsilon _{1} B+ \upsilon _{2} C+ \upsilon _{3} D+ \upsilon _{4} G ) \,ds \\& \quad \leq \frac{ ( A+ \upsilon _{1} B+ \upsilon _{2} C+ \upsilon _{3} D+ \upsilon _{4} G )}{W^{\alpha }} \Vert U-V \Vert \int _{0}^{t} \frac{s^{\alpha -1}}{\Gamma ( \alpha )} \,ds. \end{aligned}$$

This implies that \(\Vert FU-FV \Vert \leq \frac{ ( A+ \upsilon _{1} B+ \upsilon _{2} C+ \upsilon _{3} D+ \upsilon _{4} G )}{W^{\alpha }} \Vert U-V \Vert \). If we choose W such that \(W^{\alpha } >A+ \upsilon _{1} B+ \upsilon _{2} C+ \upsilon _{3} D+ \upsilon _{4} G\), then we obtain \(\Vert FU-FV \Vert \leq \Vert U-V \Vert \). Moreover, the operator F given by (4.4) has a unique fixed point.

Consequently, (5.3) has a unique solution \(U\in C^{*} [ 0, T ] \). From (5.3), we have

$$\begin{aligned} U ( t ) =&U ( 0 ) + \biggl( \frac{t^{\alpha }}{\Gamma ( \alpha +1 )} \bigl( M+AU ( t ) +N ( t ) BU ( t ) +T ( t ) CU ( t ) \\ &{}+I ( t ) DU ( t ) +E(t)GU(t) \bigr) \biggr) \\ &{}+ I^{\alpha +1} \bigl( A U ' ( t ) + N ' ( t ) BU ( t ) +N ( t ) B U ' ( t ) + T ' ( t ) CU ( t ) +T ( t ) C U ' ( t ) \\ &{}+ I ' ( t ) DU ( t ) +I ( t ) D U ' ( t ) + E ' ( t ) GU ( t ) +E ( t ) G U ' ( t ) \bigr) \end{aligned}$$

and

$$\begin{aligned} \frac{U(t)}{dt} =& \frac{t^{\alpha -1}}{\Gamma ( \alpha )} \bigl( M+AU ( 0 ) +N ( 0 ) BU ( 0 ) +T ( 0 ) CU ( 0 ) +I ( 0 ) DU ( 0 ) +E(0)GU(0) \bigr) \\ &{}+ I^{\alpha } \bigl( A U ' ( t ) + N ' ( t ) BU ( t ) +N ( t ) B U ' ( t ) + T ' ( t ) CU ( t ) +T ( t ) C U ' ( t ) \\ &{}+ I ' ( t ) DU ( t ) +I ( t ) D U ' ( t ) + E ' ( t ) GU ( t ) +E ( t ) G U ' ( t ) \bigr), \end{aligned}$$

which implies

$$\begin{aligned}& e^{-Nt} \biggl( \frac{U ( t )}{dt} \biggr) \\& \quad = e^{-Nt} \biggl( \frac{t^{\alpha -1}}{\Gamma ( \alpha )} \bigl( M+AU ( 0 ) +N ( 0 ) BU ( 0 ) +T ( 0 ) CU ( 0 ) +I ( 0 ) DU ( 0 ) +E(0)GU(0) \bigr) \\& \qquad {}+ I^{\alpha } \bigl( A U ' ( t ) + N ' ( t ) BU ( t ) +N ( t ) B U ' ( t ) + T ' ( t ) CU ( t ) +T ( t ) C U ' ( t )\\& \qquad {} + I ' ( t ) DU ( t ) +I ( t ) D U ' ( t ) + E ' ( t ) GU ( t ) +E ( t ) G U ' ( t ) \bigr) \biggr) \end{aligned}$$

from which we can deduce that \(U ' \in C_{\sigma }^{*} [ 0, T ] \). Thus, we have

$$ \frac{dU ( t )}{dt} = \frac{d}{dt} I^{\alpha } \bigl( M+AU ( t ) +N ( t ) BU ( t ) +T ( t ) CU ( t ) +I ( t ) DU ( t ) +E(t)GU(t) \bigr), $$

which implies

$$ I^{1-\alpha } \,\frac{dU ( t )}{dt} = I^{1-\alpha } \, \frac{d}{dt} I^{\alpha } \bigl( M+AU ( t ) +N ( t ) BU ( t ) +T ( t ) CU ( t ) +I ( t ) DU ( t ) +E(t)GU(t) \bigr) $$

or

$$ D^{\alpha } U ( t ) =M+AU ( t ) +N ( t ) BU ( t ) +T ( t ) CU ( t ) +I ( t ) DU ( t ) +E(t)GU(t) $$

and

$$ U ( 0 ) = U_{0} + I^{\alpha } \bigl( M+AU ( 0 ) +N ( 0 ) BU ( 0 ) +T ( 0 ) CU ( 0 ) +I ( 0 ) DU ( 0 ) +E(0)GU(0) \bigr) = U_{0}. $$

Therefore, this IVP is equivalent to (5.1), which completes the proof. □

6 An analysis of the tumor growth at low density

In 1838, Verhulst considered the logistic growth function to explain mono-species growth. Later on, it was demonstrated that the logistic equation needs modifications to explain the growth of the population in low density-size, which is known as the Allee effect.

The Allee effect can be divided into two main types:

  • strong Allee effect and

  • weak Allee effect.

A population with a strong Allee effect will have a critical population size, the population’s threshold. Any size that is less than the threshold will go to extinction without any further aid. However, a population with a weak Allee effect will reduce the per capita growth rate at a lower population density or size [3741].

Based on the above discussion, we apply the weak Allee function at time t to the system (3.1), such as

$$ \textstyle\begin{cases} D^{\alpha } N ( t ) =N ( t ) r_{1} ( K_{1} - \mu _{1} N ( t ) - \beta _{1} T ( t ) ) - ( 1-k ) \phi _{1} N ( t ) E ( t ), \\ D^{\alpha } T ( t ) =a ( T(t) ) \cdot T ( t ) \{ r_{2} ( K_{2} d- \mu _{2} T ( t ) - \beta _{2} I ( t ) ) -\delta + ( 1-k ) \phi _{1} N ( t ) E ( t ) \} , \\ D^{\alpha } I ( t ) =\rho \gamma +I ( t ) ( K_{3} - \mu _{3} - \beta _{3} T ( t ) ) - ( 1-k ) \phi _{2} I ( t ) E ( t ), \\ D^{\alpha } E ( t ) = ( 1-k ) \varepsilon - \beta _{4} E ( t ), \end{cases} $$
(6.1)

where \(t>0\) and \(( N ( 0 ), T ( 0 ),I ( 0 ),E(0) ) = ( N_{0}, T_{0}, I_{0}, E_{0} ) \). Moreover, we define \(\varpi >0\) as the Allee coefficient of the tumor population and \(a ( T(t) ) = \frac{T(t)}{\varpi +T(t)}\) is the Allee function.

Considering the density of class \(T(t)\), we can see that the Allee function is useful for the malignant class if

$$ T ( t ) \in \biggl( 0,-\varpi + \sqrt{\frac{\varpi ( \mu _{2} r_{2} \varpi + K_{2} d r_{2} -\delta - \beta _{2} r_{2} I ( t ) + ( 1-k ) \phi _{1} N ( t ) E ( t ) )}{\mu _{2} r_{2}}} \biggr), $$
(6.2)

where the immune system is also dependent on the Allee coefficient,

$$ I ( t ) < \frac{\mu _{2} r_{2} \varpi + K_{2} d r_{2} -\delta }{\beta _{2} r_{2}}\quad \text{for } r_{2} > \frac{\delta }{K_{2} d}. $$
(6.3)

For a low population size of the tumor population, let us consider the stability conditions around \(\Lambda _{4}\). The characteristic equation is given by

$$ ( a_{11} -\lambda ) \bigl\{ \lambda ^{2} - ( a_{22} + a_{33} ) \lambda + a_{22} a_{33} ( 1- R_{0} ) \bigr\} - ( a_{33} - \lambda ) a_{12} a_{21} =0 $$
(6.4)

and

$$ \lambda _{4} =- \beta _{4} < 0, $$
(6.5)

while (6.4) is a cubic equation of the form

$$\begin{aligned}& \lambda ^{3} + ( - a_{11} - a_{22} - a_{33} ) \lambda ^{2} + \bigl( a_{11} a_{22} + a_{11} a_{33} - a_{12} a_{21} + a_{23} a_{32} ( 1- R_{0} ) \bigr) \lambda \\& \quad {}+ \bigl( a_{33} a_{12} a_{21} - a_{11} a_{22} a_{33} ( 1- R_{0} ) \bigr) =0 \end{aligned}$$
(6.6)

and

$$\begin{aligned}& a_{11} = r_{1} K_{1} -2 r_{1} \mu _{1} \overline{N}_{4} - r_{1} \beta _{1} \overline{T}_{4} - \frac{ ( 1-k )^{2} \phi _{1} \varepsilon }{\beta _{4}},\qquad a_{12} =- \beta _{1} r_{1} \overline{N}_{4},\\& a_{14} =- ( 1-k ) \phi _{1} \overline{N}_{4}, \qquad a_{21} = \frac{ ( 1-k )^{2} \phi _{1} \varepsilon \overline{T}_{4} a ( \overline{T}_{4} )}{\beta _{4}},\\& a_{22} =a ( \overline{T}_{4} ) \biggl( ( K_{2} d r_{2} - \delta ) -2 r_{2} \mu _{2} \overline{T}_{4} - \beta _{2} r_{2} \overline{I}_{4} + \frac{ ( 1-k )^{2} \phi _{1} \varepsilon \overline{N}_{4}}{\beta _{4}} \biggr), \\& a_{23} =- \beta _{2} r_{2} \overline{T}_{4} a ( \overline{T}_{4} ),\qquad a_{24} = ( 1-k ) \phi _{1} \overline{N}_{4} \overline{T}_{4} a ( \overline{T}_{4} ),\qquad a_{32} =- \beta _{3} \overline{I}_{4}, \\& a_{33} = ( K_{3} - \mu _{3} - \beta _{3} \overline{T}_{4} ) - \frac{ ( 1-k )^{2} \phi _{2} \varepsilon }{\beta _{4}},\qquad a_{34} =- ( 1-k ) \phi _{2} \overline{I}_{4}, \qquad a_{44} =- \beta _{4}. \end{aligned}$$

Theorem 6.1

Let \(\Lambda _{4}\) be the positive equilibrium point of system (6.1) and assume that \(R_{0} <1\), \(\phi _{1} < \frac{K_{1} r_{1} \beta _{4}}{\varepsilon ( 1-k )^{2}}\) and \(\frac{ ( K_{3} -\delta ) \beta _{4}}{\varepsilon ( 1-k )^{2}} < \phi _{2} < \frac{ ( K_{3} - \mu _{3} ) \beta _{4}}{\varepsilon ( 1-k )^{2}}\). Then the following statements are true:

  1. (i)

    Let \(r_{1} > \frac{ ( 1-k )^{2} \phi _{1} \varepsilon a ( \overline{T}_{4} )}{2 \beta _{4} \mu _{1}}\) and \(r_{2} > \frac{\delta }{K_{2} d} \). If

    $$ \overline{T}_{4} > \frac{r_{1} K_{1} \beta _{4} - ( 1-k )^{2} \phi _{1} \varepsilon }{ ( r_{1} \beta _{1} +2 r_{2} \mu _{2} a ( \overline{T}_{4} ) ) \beta _{4}}\quad \textit{and} \quad \overline{I}_{4} > \frac{K_{2} d r_{2} -\delta }{\beta _{2} r_{2}}, $$

    where \(\beta _{3} > \frac{ ( r_{1} \beta _{1} +2 r_{2} \mu _{2} a ( \overline{T}_{4} ) ) ( ( K_{3} - \mu _{3} ) \beta _{4} - ( 1-k )^{2} \phi _{2} \varepsilon )}{r_{1} K_{1} \beta _{4} - ( 1-k )^{2} \phi _{1} \varepsilon } \) then all the roots are real. This implies that \(\Lambda _{4}\) is locally asymptotically stable.

  2. (ii)

    Let \(r_{1} \geq \frac{ ( 1-k )^{2} \phi _{1} \varepsilon a ( \overline{T}_{4} )}{2 \beta _{4} \mu _{1}}\) and \(r_{2} > \frac{\delta }{K_{2} d} \). If

    $$\begin{aligned}& \overline{N}_{4} \in \biggl( \frac{ ( r_{1} K_{1} - r_{1} \beta _{1} \overline{T}_{4} ) \beta _{4} - ( 1-k )^{2} \phi _{1} \varepsilon }{\beta _{4} r_{1} ( 2 \mu _{1} + \beta _{1} )}, \infty \biggr), \\& \overline{T}_{4} \in \biggl[ \frac{r_{1} K_{1} \beta _{4} - ( 1-k )^{2} \phi _{1} \varepsilon }{ ( r_{1} \beta _{1} +2 r_{2} \mu _{2} a ( \overline{T}_{4} ) ) \beta _{4}} , \frac{r_{1} K_{1} \beta _{4} - ( 1-k )^{2} \phi _{1} \varepsilon }{r_{1} \beta _{1} \beta _{4}} \biggr),\\& \overline{I}_{4} \in \biggl[ \frac{K_{2} d r_{2} -\delta }{\beta _{2} r_{2}} , \infty \biggr) \end{aligned}$$

    and

    $$ \biggl( \frac{ ( 1-k ) \phi _{1} \overline{E}_{4} +2 r_{2} \mu _{2} a ( \overline{T}_{4} )}{ ( 1-k ) \phi _{1} \overline{E}_{4}} \biggr) > \frac{\overline{N}_{4}}{\overline{T}_{4}}, $$

    where \(\beta _{3} = \frac{ ( r_{1} \beta _{1} +2 r_{2} \mu _{2} a ( \overline{T}_{4} ) ) ( ( K_{3} - \mu _{3} ) \beta _{4} - ( 1-k )^{2} \phi _{2} \varepsilon )}{r_{1} K_{1} \beta _{4} - ( 1-k )^{2} \phi _{1} \varepsilon } \), then there are only one real root and one complex root with its complex conjugate, which imply that \(\Lambda _{4}\) is locally asymptotically stable.

Proof

(i) Let us consider the conditions for a positive discriminant of (6.6). Thus,

$$\begin{aligned}& + \biggl( \frac{ ( 1-k )^{2} \phi _{1} \varepsilon a ( \overline{T}_{4} )}{\beta _{4}} -2 r_{1} \mu _{1} \biggr) \overline{N}_{4} + \biggl( r_{1} K_{1} - \bigl( r_{1} \beta _{1} +2 r_{2} \mu _{2} a ( \overline{T}_{4} ) \bigr) \overline{T}_{4} - \frac{ ( 1-k )^{2} \phi _{1} \varepsilon }{\beta _{4}} \biggr) \\& \quad {}+a ( \overline{T}_{4} ) ( K_{2} d r_{2} -\delta - \beta _{2} r_{2} \overline{I}_{4} ) + \biggl( K_{3} - \mu _{3} - \beta _{3} \overline{T}_{4} - \frac{ ( 1-k )^{2} \phi _{2} \varepsilon }{\beta _{4}} \biggr) < 0, \end{aligned}$$
(6.7)

where we obtain

$$\begin{aligned}& \frac{ ( 1-k )^{2} \phi _{1} \varepsilon a ( \overline{T}_{4} )}{\beta _{4}} -2 r_{1} \mu _{1} < 0\quad \Rightarrow\quad r_{1} > \frac{ ( 1-k )^{2} \phi _{1} \varepsilon a ( \overline{T}_{4} )}{2 \beta _{4} \mu _{1}}, \end{aligned}$$
(6.8)
$$\begin{aligned}& r_{1} K_{1} - \bigl( r_{1} \beta _{1} +2 r_{2} \mu _{2} a ( \overline{T}_{4} ) \bigr) \overline{T}_{4} - \frac{ ( 1-k )^{2} \phi _{1} \varepsilon }{\beta _{4}} < 0 \\& \quad \Rightarrow\quad \overline{T}_{4} > \frac{r_{1} K_{1} \beta _{4} - ( 1-k )^{2} \phi _{1} \varepsilon }{ ( r_{1} \beta _{1} +2 r_{2} \mu _{2} a ( \overline{T}_{4} ) ) \beta _{4}}\quad \text{for } \phi _{1} < \frac{K_{1} r_{1} \beta _{4}}{\varepsilon ( 1-k )^{2}}, \end{aligned}$$
(6.9)
$$\begin{aligned}& K_{2} d r_{2} -\delta - \beta _{2} r_{2} \overline{I}_{4} < 0\quad \Rightarrow\quad \overline{I}_{4} > \frac{K_{2} d r_{2} -\delta }{\beta _{2} r_{2}}\quad \text{for } r_{2} > \frac{\delta }{K_{2} d} \end{aligned}$$
(6.10)

and

$$\begin{aligned}& K_{3} - \mu _{3} - \frac{ ( 1-k )^{2} \phi _{2} \varepsilon }{\beta _{4}} - \beta _{3} \overline{T}_{4} < 0 \\& \quad \Rightarrow\quad \overline{T}_{4} > \frac{ ( K_{3} - \mu _{3} ) \beta _{4} - ( 1-k )^{2} \phi _{2} \varepsilon }{\beta _{4} \beta _{3}}\quad \text{for } \phi _{2} < \frac{ ( K_{3} - \mu _{3} ) \beta _{4}}{\varepsilon ( 1-k )^{2}}. \end{aligned}$$
(6.11)

Considering both (6.9) and (6.11), we have

$$ \overline{T}_{4} > \frac{r_{1} K_{1} \beta _{4} - ( 1-k )^{2} \phi _{1} \varepsilon }{ ( r_{1} \beta _{1} +2 r_{2} \mu _{2} a ( \overline{T}_{4} ) ) \beta _{4}} > \frac{ ( K_{3} - \mu _{3} ) \beta _{4} - ( 1-k )^{2} \phi _{2} \varepsilon }{\beta _{4} \beta _{3}}, $$
(6.12)

where \(\beta _{3} > \frac{ ( r_{1} \beta _{1} +2 r_{2} \mu _{2} a ( \overline{T}_{4} ) ) ( ( K_{3} - \mu _{3} ) \beta _{4} - ( 1-k )^{2} \phi _{2} \varepsilon )}{r_{1} K_{1} \beta _{4} - ( 1-k )^{2} \phi _{1} \varepsilon } \). This completes the proof of (i).

Since Theorem 4.4/(ii) is similar to (i), it is given without proof. □

Remark 6.1

Theorem 6.1 shows that the ketogenic diet affects the growth rate of the tumor population. Decreasing the ketogenic rate is increasing the tumor growth rate, which leads to a weak immune response. On the other side, increasing the ketogenic program in continuous time restricts the tumor population’s capacity, supporting the immune system. Additionally, we noticed that other complement control parameters such as anti-cancer drugs decrease the estrogen level caused by damage to the DNA.

Example 6.1

In this example, we wrote code and ran it using the MATLAB version 2019. The initial values of (3.1) are \(N ( 0 ) =200\), \(T ( 0 ) =50\), \(I ( 0 ) =50\), \(E ( 0 ) =2\). Figure 4 shows the graph of the tumor cells and the normal cells, where \(T ( t )\) is given in red and \(N ( t )\) in blue for mixed therapy in the case of early detection. All control parameters are included in the treatment with low dosages, which drive the tumor density extinct in a short discrete time. This simulation demonstrates the weak Allee effect, where the Allee coefficient is \(\varpi =0.4\). Therefore, it is seen that the ketogenic assistance and supplements of an immune booster are essential in all stages of cancer treatment.

Figure 4
figure 4

Bifurcation diagram of \(T ( t )\) and \(N ( t )\) with a mixed therapy with Allee effect

7 Conclusions

This study analyzed a mathematical model of breast cancer as a fractional-order system to analyze tumor growth under chemotherapeutic treatment and immune response. We incorporated sensitive control parameters (ketogenic diet, immune booster, and anti-cancer drugs). Further, we assumed in the model that the majority of the malignant cells are ESR1 + ve.

In Sect. 2, we established the model, and in Sect. 3, we obtained four essential equilibrium points: the tumor-free equilibrium point, the death equilibrium point 1-2, and the co-existing equilibrium point. We proved in Sect. 4 that if the tumor cells’ invasion of the healthy cells is small, then the malignant tumor size can be eliminated from the breast tissues because of the immune response and estrogen level seen for the tumor-free equilibrium point. We found that the ketogenic diet rate supports the immune response to fight against tumor growth but cannot stop the estrogen level for the death equilibrium points. For the co-existing positive equilibrium point, it is shown that if we have a low rate of the estrogen level, then the normal cells and the cancer cells are interactional stables. This occurs with a robust immune system relevant to the support of ketogenic assistance and the immune booster’s supplements. After that, we considered the existence and uniqueness of the initial value problem in Sect. 5.

Finally, in Sect. 6, we considered the early detection case of the tumor population by applying the weak Allee effect at a discrete-time of t. We show that the Allee coefficient affects the normal cell population growth. We emphasized that the DNA damage caused by the estrogen level is still low in the early stages of the malignant population. The essential point was that the ketogenic diet rate was not related to the malignant class density, while the immune response was concerning the tumor growth and the Allee coefficient.