Controlling IL-7 Injections in HIV-Infected Patients

Abstract

Immune interventions consisting in repeated injections are broadly used as they are thought to improve the quantity and the quality of the immune response. However, they also raise several questions that remain unanswered, in particular the number of injections to make or the delay to respect between different injections to achieve this goal. Practical and financial considerations add constraints to these questions, especially in the framework of human studies. We specifically focus here on the use of interleukin-7 (IL-7) injections in HIV-infected patients under antiretroviral treatment, but still unable to restore normal levels of $\hbox {CD}4^{+}$ T lymphocytes. Clinical trials have already shown that repeated cycles of injections of IL-7 could help maintaining $\hbox {CD}4^{+}$ T lymphocytes levels over the limit of 500 cells/$\upmu$L, by affecting proliferation and survival of $\hbox {CD}4^{+}$ T cells. We then aim at answering the question: how to maintain a patients level of $\hbox {CD}4^{+}$ T lymphocytes by using a minimum number of injections (i.e., optimizing the strategy of injections)? Based on mechanistic models that were previously developed for the dynamics of $\hbox {CD}4^{+}$ T lymphocytes in this context, we model the process by a piecewise deterministic Markov model. We then address the question by using some recently established theory on impulse control problem in order to develop a numerical tool determining the optimal strategy. Results are obtained on a reduced model, as a proof of concept: the method allows to define an optimal strategy for a given patient. This method could be applied to optimize injections schedules in clinical trials.

Appendices

Optimal Control: Application

We defined the process describing the effect of IL-7 on $\hbox {CD}4^{+}$ T lymphocytes dynamics by its characteristics $\phi$, $\eta$ and Q, boundaries and possible actions in Sect. 2.3. We also defined both gradual cost on the trajectory and impulse cost in that section. As we aim at applying the results from Theorem 1 to determine the optimal cost and an optimal strategy by dynamic programming, we need to determine how to compute numerically the function $\mathfrak {B}$ to iterate the sequence $\{W_q\}_{q \in \mathbb {N}}$ defined in Eq. 4. As a reminder, $\mathfrak {B}$ is defined in Costa et al. (2016) by:

$$\begin{aligned} \mathfrak {B}V(y)= \int _{[0,t^{*}(y)[} e^{-(K+\alpha )t} \mathfrak {R}V(\phi (y,t)) \text {d}t + e^{-(K+\alpha )t^{*}(y)} \mathfrak {T}V(\phi (y,t^{*}(y))) \end{aligned}$$

We will first detail the computation of $\mathfrak {R}$ then $\mathfrak {T}$, and we will finally show how to compute $\mathfrak {B}$.

1.1 Computation of $\mathfrak {R}$

For $x=(\gamma ,n,\sigma ,\theta ,p,r) \in X$, and function $V: \overline{X} \rightarrow \mathbb {R}$, $\mathfrak {R}$ is defined as:

$$\begin{aligned} \mathfrak {R}V(x) = C^g(x) + qV(x) + \eta V(x) \end{aligned}$$

with q computing the difference between the states before and after the spontaneous jump. As Q depends on the action only when the process hits the active boundary,

$$\begin{aligned} \begin{array}{rll} q(\text {d}y|x,d)&{} =&{}\eta (x)[Q(\text {d}y|x)-\delta _x(\text {d}y)]\\ &{}=&{}\mathbf {1}_{\{\gamma >1\}}\eta [\delta _{(1,n,\sigma ,\theta ,p,r)}(\text {d}y)-\delta _{(\gamma ,n,\sigma ,\theta ,p,r)}(\text {d}y)] \end{array} \end{aligned}$$

then for every function V, and as $K=\eta$:

$$\begin{aligned} \begin{array}{rll} qV(x) &{}=&{} \displaystyle \int V(y)q(\text {d}y|x) \nonumber \\ &{}=&{} \mathbf {1}_{\{\gamma >1\}}K[V(1,n,\sigma ,\theta ,p,r)-V(\gamma ,n,\sigma ,\theta ,p,r)] \end{array} \end{aligned}$$

Then

$$\begin{aligned} \begin{array}{rll} \mathfrak {R}V(x)&{}=&{} \displaystyle \frac{1}{30}\mathbf {1}_{\{p+r\le 500\}} + qV(x) + K V(x) \\ &{}= &{} \displaystyle \frac{1}{30}\mathbf {1}_{\{p+r\le 500\}} +K V(1,n,\sigma ,\theta ,p,r) \mathbf {1}_{\{\gamma >1\}} + K V(x) \mathbf {1}_{\{\gamma =1\}} \\ \end{array} \end{aligned}$$

Finally,

$$\begin{aligned} \begin{array}{rll} \mathfrak {R}V(x) &{}=&{} \displaystyle \frac{1}{30} \mathbf {1}_{\{p+r\le 500\}} + K V(1,n,\sigma ,\theta ,p,r) \\ \mathfrak {R}V(\varDelta ) &{}=&{} KV(\varDelta ) \end{array} \end{aligned}$$

(10)

1.2 Computation of $\varvec{\mathfrak {T}}$

For $x \in \varXi$, and function $V: \overline{X} \rightarrow \mathbb {R}$, $\mathfrak {T}$ is defined as:

$$\begin{aligned} \begin{array}{rll} \mathfrak {T}V(x) &{}=&{} \displaystyle \inf _{d \in A(x)} \Big \{C^i(x,d) + QV(x,d) \Big \} \\ &{}=&{}\displaystyle \inf _{d \in A(x)} \Big \{\mathbf {1}_{x \in \varXi _1 \cup \varXi _4} + \mathbf {1}_{d \ne 0}\mathbf {}{1}_{x \in \varXi _3} + \int V(y)\Big [\delta _{(\gamma (d),1,0,1,P_c,R_c)}(\text {d}y)\mathbf {1}_{\{x \in \varXi _1\}} \\ &{}&{}+\, \delta _{\varDelta }(\text {d}y)\mathbf {1}_{\{x \in \varXi _2\}} + \delta _{(\gamma (d),n+1,0,\theta ,p,r)}(\text {d}y)\mathbf {1}_{\{x \in \varXi _3\}} \\ &{}&{}+\, \delta _{(\gamma (d),1,0,\theta ,p,r)}(\text {d}y)\mathbf {1}_{\{x \in \varXi _4\}} \\ &{}&{}+\,\delta _{(1,n,\sigma ,\theta ,p,r)}(\text {d}y)\mathbf {1}_{\{x \in \varXi _5\}}\Big ] \Big \} \end{array} \end{aligned}$$

Finally,

$$\begin{aligned} \begin{array}{rll} \mathfrak {T}V(x) &{}=&{} \displaystyle \inf _{d \in A(x)} \Big \{ [1+V(\gamma (d),1,0,1,P_c,R_c)] \mathbf {1}_{x \in \varXi _1} \\ &{}&{}+\,[\mathbf {1}_{d \ne 0}+V(\gamma (d),n+1,0,\theta ,p,r)] \mathbf {1}_{x \in \varXi _3} \\ &{}&{}+\, [1+V(\gamma (d),1,0,\theta ,p,r)] \mathbf {1}_{x \in \varXi _4} \Big \} + V(\varDelta ) \mathbf {1}_{x \in \varXi _2} \\ &{}&{}+\, V(1,n,\sigma ,\theta ,p,r) \mathbf {1}_{x \in \varXi _5} \\ \mathfrak {T}V(\varDelta )&{}=&{}V(\varDelta ) \end{array} \end{aligned}$$

(11)

1.3 Computation of $\varvec{\mathfrak {B}}$

Now, for $Y \in \overline{X}$, and function $V: \overline{X} \rightarrow \mathbb {R}$, we need to compute:

$$\begin{aligned} \mathfrak {B}V(y)= \int _{[0,t^{*}(y)[} e^{-(K+\alpha )t} \mathfrak {R}V(\phi (y,t)) \text {d}t + e^{-(K+\alpha )t^{*}(y)} \mathfrak {T}V(\phi (y,t^{*}(y))) \end{aligned}$$

As we cannot make an exact computation of $\mathfrak {B}V$ on $\overline{X}$, we need to approximate this computation on a grid of the state space. In order to detail the approximation of the computation, we define

$$\begin{aligned} G(V,y)=\int _{[0,t^*(y)[} e^{-(K+\alpha )t}\mathfrak {R}V(\phi (y,t))\text {d}t \end{aligned}$$

and

$$\begin{aligned} H(V,y)=e^{-(K+\alpha )t^*(y)}\mathfrak {T}V(\phi (y,t^*(y)) \end{aligned}$$

as in Eqs. 7 and 8. We define a time interval $\varDelta t$ (in practice equal to 1 day) and for every $y=(\gamma ,n,\sigma ,\theta ,p,r) \in \tilde{X}$, we note

$$\begin{aligned} n^*(y)=\left\lfloor \dfrac{t^*(y)}{\varDelta t} \right\rfloor \end{aligned}$$

For every $j \in \{0..n^*(y)-1\}$, we note $\phi _j(y,t)=\phi (y,j\varDelta t)$ and $\phi (y,t^*(y))=(\gamma ,n,\sigma +t^*(y),\theta +t^*(y),p^*(y),r^*(y))$. The integral defined in Eq. 7 is computed by approximation using the classic trapezoidal rule using the $j\varDelta t$ nodes:

$$\begin{aligned} \begin{array}{rll} G(V,y) &{}\simeq &{}\displaystyle \frac{\varDelta t}{2}\mathfrak {R}V(y)+\frac{\varDelta t}{2}e^{-(K+\alpha )t^*(y)}\mathfrak {R}V(\phi (y,t^*(y))) \\ &{}&{}+\, \displaystyle \sum _{j=1}^{n^*(y)-2} \varDelta t e^{-(K+\alpha )j \varDelta t} \mathfrak {R}V(\phi _j(y,t)) \end{array} \end{aligned}$$

with $\mathfrak {R}V(x) = \displaystyle \frac{1}{30} \mathbf {1}_{\{p+r\le 500\}} + K V(1,n,\sigma ,\theta ,p,r)$, as computed in Eq. 10. Then we obtain the following for every $y=(\gamma ,n,\sigma ,\theta ,p,r) \in \tilde{X}$:

$$\begin{aligned} \begin{array}{rll} G(V,y)&{}=&{}\displaystyle \frac{\varDelta t}{2}\left( \displaystyle \frac{1}{30} \mathbf {1}_{\{p+r<500\}} + K V(1,n,\sigma ,\theta ,p,r)\right) \\ &{}&{}+\,\displaystyle \frac{\varDelta t}{2}e^{-(K+\alpha )t^*(y)}\Bigg (\displaystyle \frac{1}{30}\mathbf {1}_{\{p^*+r^*<500\}}\\ &{}&{}+\, K V(1,n,\sigma +t^*,\theta +t^*,p^*(y),r^*(y))\Bigg ) \\ &{}&{}+\, \varDelta t \displaystyle \sum _{j=1}^{n^*(y)-2} e^{-(K+\alpha )j \varDelta t} \Bigg (\displaystyle \frac{1}{30}\mathbf {1}_{\{p_j+r_j<500\}}\\ &{}&{}+\,K V(1,n,\sigma + j\varDelta t,\theta +j\varDelta t,p_j,r_j)\Bigg ) \end{array} \end{aligned}$$

Now, we need to compute H as defined in Eq. 8: it depends on $\mathfrak {T}V(\phi (y,t^*(y)))$, which takes different values according to the boundary reached in that point, as written in Eq. 11. Moreover, as we know the flow, we can give conditions on $y=(\gamma ,n,\sigma ,\theta ,p,r)$ to reach a given boundary in $\phi (y,t^*(y))$. Then:

• if $\phi (y,t^*(y)) \in \varXi _1$ ($\theta \le 1$) then

  $$\begin{aligned} H(V,y)= \inf _{d \in [d_1,..d_{m_d}]} \Big \{ e^{-(K+\alpha )t^*(y)}\Big [1+V(\gamma (d),1,0,1,P_{c},R_{c}) \Big ]\Big \} \end{aligned}$$

• if $\phi (y,t^*(y)) \in \varXi _2$ ($\theta + t^*(y) \ge T_h$) then

  $$\begin{aligned} H(V,y)= e^{-(K+\alpha )t^*(y)} V(\varDelta ) \end{aligned}$$

• if $\phi (y,t^*(y)) \in \varXi _3$ ($n< n_{\text {inj}}, \theta + t^*(y) < T_h$) then

  $$\begin{aligned} \begin{array}{rll} H(V,y)&{}=&{} \inf _{d \in [0,d_1,..d_{m_d}]} \Big \{ e^{-(K+\alpha )t^*(y)}\Big [\mathbf {1}_{\{d \ne 0\}}\\ &{}&{}+\,V(\gamma (d),n+1,0,\theta +t^*(y),p^*(y),r^*(y)) \Big ]\Big \} \end{array} \end{aligned}$$

• if $\phi (y,t^*(y)) \in \varXi _4$ ($n = n_{\text {inj}}, \gamma =1, \theta + t^*(y) < T_h$) then

  $$\begin{aligned} H(V,y)= & {} \inf _{d \in [d_1,..d_{m_d}]} \Big \{ e^{-(K+\alpha )t^*(y)}\Big [1+V(\gamma (d),1,0,\theta \\&+\,t^*(y),p^*(y),r^*(y)) \Big ]\Big \} \end{aligned}$$

• if $\phi (y,t^*(y)) \in \varXi _5$ ($n = n_{\text {inj}}, \gamma >1, \theta + t^*(y) < T_h$) then

  $$\begin{aligned} H(V,y)= e^{-(K+\alpha )t^*(y)} V(1,n,\sigma +t^*(y),\theta +t^*(y),p^*(y),r^*(y)) \end{aligned}$$

Finally, for every $y=(\gamma ,n,\sigma ,\theta ,p,r)\in \tilde{X}$:

$$\begin{aligned} \mathfrak {B}V(y)= & {} \displaystyle \frac{\varDelta t}{2}\left( \displaystyle \frac{1}{30}\mathbf {1}_{\{p+r<500\}} + K V(1,n,\sigma ,\theta ,p,r)\right) \nonumber \\&+\,\frac{\varDelta t}{2}e^{-(K+\alpha )t^*(y)}\Big [\displaystyle \frac{1}{30}\mathbf {1}_{\{p^*+r^*<500\}}+ K V(1,n,\sigma \nonumber \\&+\,t^*,\theta +t^*,p^*(y),r^*(y))\Big ] \nonumber \\&+\,\varDelta t \displaystyle \sum _{j=1}^{n^*(y)-2} e^{-(K+\alpha )j \varDelta t} \Big [\displaystyle \frac{1}{30}\mathbf {1}_{\{p_j+r_j<500\}}\nonumber \\&+\,K V(1,n,\sigma + j\varDelta t,\theta +j\varDelta t,p_j,r_j)\Big ] \nonumber \\&+\,\displaystyle \inf _{d \in [d_1,..d_{m_d}]} \Big \{ e^{-(K+\alpha )t^*(y)}\Big [1+V(\gamma (d),1,0,1,P_{c},R_{c}) \Big ]\Big \} \mathbf {1}_{\{ \theta \le 1 \}} \nonumber \\&+\,e^{-(K+\alpha )t^*(y)} V(\varDelta ) \mathbf {1}_{\{ \theta + t^*(y) \ge T_h\} }\nonumber \\&+\, \displaystyle \inf _{d \in [0,d_1,..d_{m_d}]} \Big \{ e^{-(K+\alpha )t^*(y)}\Big [\mathbf {1}_{\{d \ne 0\}}\nonumber \\&+\,V(\gamma (d),n+1,0,\theta +t^*(y),p^*(y),r^*(y)) \Big ]\Big \} \mathbf {1}_{\{ n< n_{\text {inj}}, \theta + t^*(y)< T_h\}} \nonumber \\&+\,\displaystyle \inf _{d \in [d_1,..d_{m_d}]} \Big \{ e^{-(K+\alpha )t^*(y)}\Big [1+V(\gamma (d),1,0,\theta +t^*(y),p^*(y),r^*(y)) \Big ]\Big \}\nonumber \\&\times \mathbf {1}_{\{ n = n_{\text {inj}}, \gamma =1, \theta + t^*(y)< T_h\}} \nonumber \\&+\, e^{-(K+\alpha )t^*(y)} V(1,n,\sigma +t^*(y),\theta +t^*(y),p^*(y),r^*(y))\nonumber \\&\times \mathbf {1}_{\{n = n_{\text {inj}}, \gamma >1, \theta + t^*(y) < T_h\}} \end{aligned}$$

(12)

and

$$\begin{aligned} \mathfrak {B}V(\varDelta )=\int _{[0,+\infty )} e^{-(K+\alpha )t}KV(\varDelta )\text {d}t = \frac{K}{K+\alpha } V(\varDelta ) \end{aligned}$$

Fig. 4

Structure of the code and its subroutines

Structure of the Code

Structure of the code and its subroutines are shown in Fig. 4. Application in the results section requires the following grid:

• $\gamma \in \{1..3\}$

• n $\in \{1..3\}$

• $\sigma \in \{0..351\}$

• $\theta \in \{1..365\}$

• $p \in \{2..110\}$ depending on the patient

• $r \in \{100..1500\}$ depending on the patient

The grid of the state space created in Matlab contains 67,614 lines and 7755 columns. For a given patient, the computation of 40 iterations of the sequence (convergence is reached between 35 and 45 iterations) requires between 5 and 6 days.

Table 5 Pairs of ($\lambda$,$\rho$), associated CD4 values and mean cost for protocols P1 to P5

Sensitivity Analysis of the Method

To evaluate how the uncertainty on individual parameters estimation could impact the determination of the optimal strategy, we have realized a sensitivity analysis. For a given patient, we suppose a normal distribution of parameters $\lambda$ and $\rho$. We generate $L=500$ pairs of parameters ($\lambda$, $\rho$) from this joint distribution. Each pair corresponds to an initial value of lymphocytes T $\text {CD4}_0$. We determine the empirical quartiles of the distribution of the $\text {CD4}_0$ and focus on the pairs inducing values close the first and the third quartiles. Then, for each pair, we simulate the five possible protocols P1 to P5 and compare them to the optimal strategy determined on the mean value of ($\lambda$,$\rho$). In practice, values of pairs and associated values of CD4 are displayed in Table 5. For the mean value of ($\lambda$,$\rho$), we determined the optimal strategy to be a first cycle of 2 injections and then cycles of 1 injection, which corresponds to protocol P4. We show in Table 5 the cost of each protocol for each pair of ($\lambda$,$\rho$), and we put in bold the minimum cost over the five protocols. We can see that protocol P4 achieves the minimum cost for all pairs inducing CD4 values at the first quartile. For pairs inducing CD4 values at the third quartile, the protocol achieving the minimum cost is P5. However, the difference of cost is not huge and P4 actually induces more time spent over the 500 threshold and less than one more injection than P5 on average, which is still acceptable. Overall, this shows that even with some error on the estimation on $\lambda$, $\rho$ we would be able to determine a strategy achieving a good balance between clinical criteria.

Trajectories of Patients B and C

See Figs. 5, 6.

Fig. 5

Dynamics of $\hbox {CD}4^{+}$ T lymphocytes in patient B. Straight line corresponds to the “best” outcome, i.e., when the effect of all injections lasts 7 days. Dashed line corresponds to other possible trajectories, when this effect can last less than 7 days. a Dynamics of $\hbox {CD}4^{+}$ T lymphocytes in patient B under P3, a 2-injections cycles protocol (dose 20). b Dynamics of $\hbox {CD}4^{+}$ T lymphocytes in patient B under the determined optimal strategy

Fig. 6

Dynamics of $\hbox {CD}4^{+}$ T lymphocytes in patient C. Straight line corresponds to the “best” outcome, i.e., when the effect of all injections lasts 7 days. Dashed line corresponds to other possible trajectories, when this effect can last less than 7 days. a Dynamics of $\hbox {CD}4^{+}$ T lymphocytes in patient C under P3, a 2-injections cycles protocol (dose 20). b Dynamics of $\hbox {CD}4^{+}$ T lymphocytes in patient C under the determined optimal strategy