Annals of Biomedical Engineering

, Volume 37, Issue 6, pp 1251–1261

Optimal Control of HIV-Virus Dynamics

Authors

    • Department of Electrical and Computer EngineeringRutgers University
Article

DOI: 10.1007/s10439-009-9672-7

Cite this article as:
Radisavljevic-Gajic, V. Ann Biomed Eng (2009) 37: 1251. doi:10.1007/s10439-009-9672-7

Abstract

In this paper we consider a mathematical model of HIV-virus dynamics and propose an efficient control strategy to keep the number of HIV virons under a pre-specified level and to reduce the total amount of medications that patients receive. The model considered is a nonlinear third-order model. The third-order model describes dynamics of three most dominant variables: number of healthy white blood cells (T-cells), number of infected T-cells, and number of virus particles. There are two control variables in this model corresponding to two categories of antiviral drugs: reverse transcriptase inhibitors (RTI) and protease inhibitors (PI). The proposed strategy is based on linearization of the nonlinear model at the equilibrium point (steady state). The corresponding controller has two components: the first one that keeps the system state variables at the desired equilibrium (set-point controller) and the second-one that reduces in an optimal way deviations of the system state variables from their desired equilibrium values. The second controller is based on minimization of the square of the error between the actual and desired (equilibrium) values for the linearized system (linear-quadratic optimal controller). The obtained control strategy recommends to HIV researchers and experimentalists that the constant dosages of drugs have to be administrated at all times (set point controller, open-loop controller) and that the variable dosages of drugs have to be administrated on a daily basis (closed-loop controller, feedback controller).

Keywords

Nonlinear controlLinearizationOptimal controlHIV dynamic model

Introduction

The human immunodeficiency virus (HIV) causes the acquired immune deficiency syndrome, knows as AIDS. Mathematical models of HIV dynamics were derived several years ago.7,8,16,19 Controlling dynamics of HIV has been an interesting and challenging research problem.2,3,5,6,10,12,14,21 In this study, we consider the third-order model of HIV dynamics that captures the time rate of healthy and infected white blood cells (T-cells) and the number of HIV viruses. This model has two control variables corresponding to two major categories of antiretroviral drugs: reverse transcriptase inhibitors (RTI) and protease inhibitors (PI). There are more complex models of HIV dynamics that can be found in the literature.2,7,8,15,18 The methodology presented in this paper can be applied with minor modifications to the other models of HIV dynamics. However, as indicated by an anonymous reviewer of the manuscript, it should be emphasized that the presented analysis is limited in the sense that it “can not take into the account patient factors as physiological/genetic level, physicochemical factors at cell–protein–viral interactions level, and viral factors that relate to the various HIV strains and clades.”

Consider a nonlinear HIV-dynamics model whose state space form with the corresponding numerical data is given by5,13,23
$$ \begin{gathered} \frac{{dx_{1} (t)}}{dt} = - dx_{1} (t) - \beta x_{1} (t)x_{3} (t) + s \hfill \\ \frac{{dx_{2} (t)}}{dt} = - \mu_{2} x_{2} (t) + \beta x_{1} (t)x_{3} (t) \hfill \\ \frac{{dx_{3} (t)}}{dt} = kx_{2} (t) - \mu_{1} x_{3} (t) \hfill \\ \end{gathered} $$
(1)
where x1(t) denotes the number of healthy white blood cells (known as T-cells), x2(t) denotes the number of infected T-cells, and x3(t) represents free virus particles (viruses are not classified as living organisms because they can not replicate without a help of a host cell). The constant parameters in (1) are
  • s = 10 per mm3 per day is the constant source of healthy T-cells (thymus);

  • d = 0.02 per day represents the death rate of healthy T-cells;

  • μ1 = 0.24 per day represents the death rate of viruses;

  • μ2 = 2.4 per day represents the death rate of infected T-cells;

  • β = 2.4 × 10−5 per (mm3 × day) is the infectivity rate of free viruses;

  • k = 100 per cell is the rate of virons (free virous particles) produced per infected T-cell.

These parameters can be estimated using an adaptive observer.23 When the number of T-cells, which is normally around 1000 per mm3 is under 200 per mm3, a HIV-infected patient is classified as having AIDS.

System Analysis

The equilibrium point of the system defined in (1) can be obtained by setting the time derivatives to zero and solving the corresponding system of algebraic equations. The equilibrium points of (1), and their numerical values, are given by
$$ \left( {x_{{1{\text{a}}}}^{\text{ss}} ,\;x_{{2{\text{a}}}}^{\text{ss}} ,\;x_{{3{\text{a}}}}^{\text{ss}} } \right) = \left( {\frac{s}{d},\;0,\;0} \right) = (500,\;0,\;0) $$
(2a)
$$ \left( {x_{{1{\text{b}}}}^{\text{ss}} ,\;x_{{2{\text{b}}}}^{\text{ss}} ,\;x_{{3{\text{b}}}}^{\text{ss}} } \right) = \left( {\frac{{\mu_{1} \mu_{2} }}{\beta k},\;\frac{s}{{\mu_{2} }} - \frac{{d\mu_{1} }}{\beta k},\;\frac{ks}{{\mu_{1} \mu_{2} }} - \frac{d}{\beta }} \right)\; = \;\left( {240,\;21.67,\;902.78} \right) $$
(2b)
The first equilibrium point corresponds to the situation with no virons and no infected T cells. This case does not correspond to physical reality of HIV infected patients, and it is of less interest for us. Even more, by finding the system Jacobian matrix (the matrix of partial derivatives of the right-hand side of (1)), that is
$$ J = \left[ {\begin{array}{*{20}l} { - d - \beta x_{3} } & 0 & { - \beta x_{1} } \\ {\beta x_{3} } & { - \mu_{2} } & {\beta x_{1} } \\ 0 & k & { - \mu_{1} } \\ \end{array} } \right] $$
(3)
we can show that the first equilibrium point is unstable (by the first method of Lyapunov13) since not all eigenvalues of the Jacobin matrix, evaluated at this equilibrium point, are in the left half complex plane, that is, −0.02, 0.2183, −2.8583. It is interesting to observe that this equilibrium point, even though unstable, can be arbitrary perturbed with respect to x1a(0) and still remain at the equilibrium (x1a(0), 0, 0) as long as there are no perturbations in the directions of x2a(0) and x3a(0). However, only tiny perturbations of x2a(0) = 0 and x3a(0) = 0 will cause the equilibrium point instability. The second equilibrium point is stable since the corresponding Jacobian matrix eigenvalues are given by −0.0199 ± i0.6658, −2.6418. This means that any motion starting in the neighborhood of this equilibrium point will tend in time to the equilibrium point. Our goal in this paper will be to achieve convergence to this stable equilibrium in an optimal manner, quickly and with the minimum dosage of drugs. Even more, once we add control signals, we will see that they will also impact the equilibrium point values so that we will have a chance to keep the number of infected cells and the viral load under pre-specified limits recommended in U.S. Department of Health and Human Services Guidelines.22
The MATLAB/Simulink block diagram for the HIV dynamic model presented in (1) is given in Fig. 1. Assuming that the system (1) initial conditions are (x1(0) ≠ 0, 0, 0), the system goes to the first equilibrium, in other words with no initial viruses present, no T-cells will be affected and all T-cells remain healthy. However, by introducing only one virous particle per mm3, that is, by choosing the initial condition as (1000, 0, 1), extremely quickly, within 25 days or so, the number of virous particles explodes to the order of 104 per mm3, Fig. 2. Similarly, the number of infected T-cells increases very rapidly during this initial time interval, see Fig. 3. However, due to the reaction of the body’s immune system, these numbers will sharply decline roughly within 50 days and reach their steady state values after 200 days. On the other hand, the number of the healthy T-cells drastically declines during the initial infection time period, in our case from the initial value of x1(0) = 1000 per mm3 to below 100 per mm3 after only 30 days, and then it stabilizes around 240 per mm3 after 200 or so days, see Fig. 4. The simulation results presented in Figs. 24 are consistent with what is known about the HIV “dynamics in vivo”.3,11,12,17,19
https://static-content.springer.com/image/art%3A10.1007%2Fs10439-009-9672-7/MediaObjects/10439_2009_9672_Fig1_HTML.gif
Figure 1

MATLAB/Simulink block diagram for HIV-virus dynamics. The parameters are k = 100, μ1 = 0.24, μ2 = 2.4, d = 0.02, β = 2.4 × 10−5. The units of these parameters are defined in the text under formula (1)

https://static-content.springer.com/image/art%3A10.1007%2Fs10439-009-9672-7/MediaObjects/10439_2009_9672_Fig2_HTML.gif
Figure 2

The number of HIV viruses per day obtained by introducing on the initial day only one virous particle per mm3, that is, by choosing the initial condition for the number of healthy white blood cells, infected T-cells, and free virus particles, respectively, as (1000, 0, 1). It can be observed that very quickly, within 25 days or so, the number of virous particles explodes to more than 11000 per mm3. Due to the reaction of the body’s immune system, this number sharply declines roughly within 50 days, and it reaches its steady state value after 200 days close to 1000 virous particles per mm3

https://static-content.springer.com/image/art%3A10.1007%2Fs10439-009-9672-7/MediaObjects/10439_2009_9672_Fig3_HTML.gif
Figure 3

The number of infected T-cells per day obtained by introducing on the initial day only one virous particle per mm3, that is, by choosing the initial condition for the number of healthy white blood cells, infected T-cells, and free virus particles, respectively, as (1000, 0, 1). It can be observed that very quickly, within 25 days or so, the number of infected T-cells increases to more than 250 per mm3

https://static-content.springer.com/image/art%3A10.1007%2Fs10439-009-9672-7/MediaObjects/10439_2009_9672_Fig4_HTML.gif
Figure 4

The number of healthy T-cells per day obtained by introducing on the initial day only one virous particle per mm3, that is, by choosing the initial condition for the number of healthy white blood cells, infected T-cells, and free virus particles, respectively, as (1000, 0, 1). It can be observed that very quickly, within 30 days or so, the number of healthy T-cells drops under 100 per mm3 and that after 200 days the number of healthy T-cells stabilizes around 230 per mm3

HIV patients can be well treated with a combination of drugs over a long period of time. One of the reasons for the negative outcome of the treatment is the high dosage of drugs that patients receive over such a long time period. In the next section, we will discuss control theory strategies that minimize the amount of drugs needed, while keeping the patient at the stable “quasi steady state” equilibrium for long time.

Open- and Closed-Loop (Feedback) Control

The development of controllers for HIV virus dynamics has started recently.2,3,5,6,10,13,15,23 Such a controller3 has been derived, where it was assumed that an external control agent (an antiviral drug agent) is additively added to the third equation of (1), that is
$$ \frac{{dx_{3} (t)}}{dt} = kx_{2} (t) - \mu_{1} x_{3} (t) - U $$
(4)
so that the introduced control signal additively reduces the time rate of the viral load.
In this paper we will adopt the model that assumes that control agents are introduced multiplicatively into the system.13 The HIV dynamic model can be controlled by reducing the parameter β (virus infectivity rate) and/or parameter k (infected T-cell productivity of free virous particles). In practice, this is achieved by using two major categories of antiretroviral drugs: reverse transcriptase inhibitors (RTI) acting on β and protease inhibitors (PI) acting on k. The corresponding drug controlled system can be mathematically represented by
$$ \begin{gathered} \frac{{dx_{1} (t)}}{dt} = - dx_{1} (t) - \left( {1 - u_{1} (t)} \right)\beta x_{1} (t)x_{3} (t) + s \hfill \\ \frac{{dx_{2} (t)}}{dt} = - \mu_{2} x_{2} (t) + \left( {1 - u_{1} (t)} \right)\beta x_{1} (t)x_{3} (t) \hfill \\ \frac{{dx_{3} (t)}}{dt} = \left( {1 - u_{2} (t)} \right)kx_{2} (t) - \mu_{1} x_{3} (t) \hfill \\ \end{gathered} $$
(5)
with u1(t) and u2(t) representing control variables. The control variables are normalized to the range 0 ≤ ui(t) ≤ 1, i = 1,2, with 1 corresponding to the maximal dosage and 0 corresponding to the situation when the drug is not administrated.

The controlled dynamic system (5) does not reflect different chemical structures of the RTI and PI drugs and the fact that they target different proteins in human cells. At the present state of knowledge, no mathematical dynamic models for the RTI and PI drugs and the corresponding proteins exist in the scientific literature. However, a new rapidly developing field, named systems biology16,20,21 (developed within bioinformatics), studies dynamics of chemical processes in human cells and develops corresponding mathematical models described by controlled differential equations. Hopefully, such models will be obtained in not very distant future. Once such models become available, it will be very interesting and very challenging to couple those models with the existing mathematical models for HIV dynamic and controls and perform the corresponding analysis at a much more sophisticated level. In our study, we have a simplified model in which the RTI drug affects multiplicatively the virus infectivity rate and the PI drug affects multiplicatively the infected T-cell productivity of free virous particles. Since model (5) uses the normalized values of control variables (RTI and PI drugs) our analysis will have a simplified goal to determine (via optimal feedback control theory) what percentage values of these drugs should be administrated daily over a long period of time.

The concept of system controllability4 was used to make a decision when to initiate HIV therapy13 for the HIV dynamic model very similar to (5). The conclusion reached states: “the therapy is best initiated when the viral load is easier to control because this implies the use of lower drug doses and consequently bearable side effects.”13 This conclusion was achieved by finding the singular values of the corresponding controllability Grammian,4 “a higher singular value indicates an easier to control viral load in the sense that the transition to the treatment steady state is faster. A lower singular value indicates a more difficult to control viral load characterized by a slow transition to the final state, given the same control effort.” Even more, the paper finds that the controllability profiles of both RTI and PI drugs are similar. Several types of controller for HIV dynamics can be found in the literature: classical PID controller1,6 controller based on linear state feedback.5 In addition, a controller based on the backstepping14 technique of nonlinear control technique that can drive state variables to their desired values and assure non-negativity of all states in the closed-loop (feedback) system was considered.10 An optimal controller15 based on dynamic programming was recommended “to reduce medication and establish long-term immune response against HIV-infection.” That paper also studies the controllability property of the HIV model used. A nonlinear model of HIV was controlled via the state-dependent Riccati equation2 approach. That paper considers both the full and partial state feedback cases, and in the case when all state variables are not available for feedback, an observer is designed to reconstruct all state space variables.

Equilibrium Points of the Controlled HIV Model and the Set-Point Controller

By setting all derivatives in (5) to zero we get a set of algebraic equations that determine the equilibrium points (steady state values) in terms of the control inputs. The corresponding control inputs are assumed to be of the form
$$ u_{i} (t) = u_{{i{\text{ss}}}} + \Updelta u_{i} (t),\quad i = 1,2 $$
(6)
where uiss are constant steady state components of the control inputs whose roles are to keep the system at the constant equilibrium point, and Δui(t) are small time varying components that will be determined in an optimal manner (using the optimal control theory). Similarly, we assume that the state variables of the considered HIV dynamic model satisfy
$$ x_{i} (t) = x_{{i{\text{ss}}}} + \Updelta x_{i} (t),\quad i = 1,2,3 $$
(7)
with xiss being constant and Δxi(t) being time varying. Assuming that the control is applied as soon as the system leaves its steady state, the assumption that both Δxi(t) and Δui(t) are small is realistic (small efforts are needed to correct small deviations from the equilibrium). Hence, if Δui(t) is implemented on a daily basis, small amounts of additional drugs will be needed to bring the system (model) back to its equilibrium. It can be shown, that the small deviations of the state variables approximately satisfy a linear system given by
$$ \begin{gathered} \left[ {\begin{array}{*{20}l} {\Updelta \dot{x}_{1} (t)} \\ {\Updelta \dot{x}_{2} (t)} \\ {\Updelta \dot{x}_{3} (t)} \\ \end{array} } \right] = \left[ {\begin{array}{*{20}c} { - d - \left( {1 - u_{{1{\text{ss}}}} } \right)\beta x_{{3{\text{ss}}}}^{\text{tar}} } & 0 & { - \left( {1 - u_{{1{\text{ss}}}} } \right)\beta x_{{1{\text{ss}}}}^{\text{tar}} } \\ {\left( {1 - u_{{1{\text{ss}}}} } \right)\beta x_{{3{\text{ss}}}}^{\text{tar}} } & { - \mu_{2} } & {\left( {1 - u_{{1{\text{ss}}}} } \right)\beta x_{{1{\text{ss}}}}^{\text{tar}} } \\ 0 & {\left( {1 - u_{{2{\text{ss}}}} } \right)k} & { - \mu_{1} } \\ \end{array} } \right]\;\left[ {\begin{array}{*{20}c} {\Updelta x_{1} (t)} \\ {\Updelta x_{2} (t)} \\ {\Updelta x_{3} (t)} \\ \end{array} } \right] \hfill \\ \quad \quad \quad \quad \quad \quad \quad + \left[ {\begin{array}{*{20}c} {\beta x_{{1{\text{ss}}}}^{\text{tar}} x_{{3{\text{ss}}}}^{\text{tar}} } \\ { - \beta x_{{1{\text{ss}}}}^{\text{tar}} x_{{3{\text{ss}}}}^{\text{tar}} } \\ 0 \\ \end{array} \begin{array}{*{20}c} 0 \\ 0 \\ { - kx_{{2{\text{ss}}}}^{\text{tar}} } \\ \end{array} } \right]\;\left[ {\begin{array}{*{20}c} {\Updelta u_{1} (t)} \\ {\Updelta u_{2} (t)} \\ \end{array} } \right] = A\Updelta x(t) + B\Updelta u(t) \hfill \\ \end{gathered} $$
(8)
Note that the system matrix of the linearized model is equal to the Jacobian of the right hand-side of (5) with respect to the state variables evaluated at the steady state target equilibrium point. Also, the control matrix of the linearized model is the Jacobian of the right-hand side of (5) with respect to control variables evaluated at the steady state target equilibrium point.
The equilibrium points under control inputs are obtained from
$$ \begin{gathered} 0 = - dx_{{ 1 {\text{ss}}}} - \left( {1 - u_{{1{\text{ss}}}} } \right)\beta x_{{1{\text{ss}}}} x_{{3{\text{ss}}}} + s \hfill \\ 0 = - \mu_{2} x_{{2{\text{ss}}}} + \left( {1 - u_{{1{\text{ss}}}} } \right)\beta x_{{1{\text{ss}}}} x_{{3{\text{ss}}}} \hfill \\ 0 = \left( {1 - u_{{2{\text{ss}}}} } \right)kx_{{2{\text{ss}}}} - \mu_{1} x_{{3{\text{ss}}}} \hfill \\ \end{gathered} $$
(9)
It can be seen from the first and second equations in (9) that the following relationship exists between x1ss and x2ss
$$ x_{{ 1 {\text{ss}}}} = \frac{s}{d} - \frac{{\mu_{2} }}{d}x_{{ 2 {\text{ss}}}} \Rightarrow x_{{ 2 {\text{ss}}}} = \frac{{s - dx_{{ 1 {\text{ss}}}} }}{{\mu_{2} }} $$
(10)
Hence, we have two independent state variables in (9). Assuming that x2ss and x3ss are required to be at all times below certain pre-specified target values, x2sstar and x3sstar then the target value x1sstar is evaluated from (10). Having defined the target values for the number of viruses, number of infected (and number of healthy) cells, we can evaluate from the second and third equations of (9) the corresponding steady state control inputs that keep the system at the desired, target, equilibrium point. It can be evaluated from these equations that the steady state target control values satisfy
$$ 1 - u_{{ 1 {\text{ss}}}}^{\text{tar}} = \frac{{\mu_{2} x_{{ 2 {\text{ss}}}}^{\text{tar}} }}{{\beta x_{{ 1 {\text{ss}}}}^{\text{tar}} x_{{ 3 {\text{ss}}}}^{\text{tar}} }},\quad 1 - u_{{ 2 {\text{ss}}}}^{\text{tar}} = \frac{{\mu_{1} x_{{ 3 {\text{ss}}}}^{\text{tar}} }}{{kx_{{ 2 {\text{ss}}}}^{\text{tar}} }} $$
(11)
In our implementation of the optimal controller we will need also expressions for the state variables steady state target values in terms of input signals (required constant drug dosages, set point controllers). It can be evaluated from (10) to (11), that for the given constant drug dosages u1sstar and u2sstar, the target state variable values are given by
$$ \begin{gathered} x_{{ 1 {\text{ss}}}}^{\text{tar}} = \frac{{\mu_{1} \mu_{2} }}{\beta k}\frac{1}{{\left( {1 - u_{{ 1 {\text{ss}}}}^{\text{tar}} } \right)\left( {1 - u_{{ 2 {\text{ss}}}}^{\text{tar}} } \right)}} \hfill \\ x_{{ 2 {\text{ss}}}}^{\text{tar}} = \frac{s}{{\mu_{2} }} - \frac{{d\mu_{1} }}{\beta k}\frac{1}{{\left( {1 - u_{{ 1 {\text{ss}}}}^{\text{tar}} } \right)\left( {1 - u_{{ 2 {\text{ss}}}}^{\text{tar}} } \right)}} \hfill \\ x_{{ 3 {\text{ss}}}}^{\text{tar}} = \frac{ks}{{\mu_{1} \mu_{2} }}\left( {1 - u_{{ 2 {\text{ss}}}}^{\text{tar}} } \right) - \frac{d}{\beta }\frac{1}{{\left( {1 - u_{{ 1 {\text{ss}}}}^{\text{tar}} } \right)}} \hfill \\ \end{gathered} $$
(12)
In Figs. 5 and 6, we present graphs that show acceptable values for the steady target control inputs (amount of the required RTI and PI drugs) and the desired steady state target values for the number of healthy T-cells and the number of viruses. It can be seen from these figures that the range of steady state values of RTI and PI drugs is quite restricted, and in some regions they exclude each other. The presented range of values for x1sstar and x3sstar will be of interest for further analysis.
https://static-content.springer.com/image/art%3A10.1007%2Fs10439-009-9672-7/MediaObjects/10439_2009_9672_Fig5_HTML.gif
Figure 5

The required relative steady state amount of the RTI drug obtained from formula (11) in terms of the target steady values for the number of healthy T-cells x1 and the number of viruses x3. The variable RTI is normalized to the range [0,1], with 1 corresponding to the maximal dosage and 0 corresponding to the situation when the drug is not administrated

https://static-content.springer.com/image/art%3A10.1007%2Fs10439-009-9672-7/MediaObjects/10439_2009_9672_Fig6_HTML.gif
Figure 6

The required relative steady state amount of the PI drug obtained from formulas (11) to (12) in terms of the target steady state values for the number of healthy T-cells x1 and the number of viruses x3. The variable PI is normalized to the range [0,1], with 1 corresponding to the maximal dosage and 0 corresponding to the situation when the drug is not administrated

Optimal Feedback Controller

Since the viral drugs are administrated based on measurements of healthy T-cells, x1(t), and viruses x3(t), the system output equation can be taken as y1(t) = x1(t) and y2(t) = x3(t). Note that the number of infected T-cells can be determined using an observer.6 Hence, we can assume that all state variables are available for feedback so that information about xi(t) − xiss = Δxi(t), i = 1,2,3, is known.

The simplest way to reduce Δxi(t) to zero is to use the nonlinear integrator technique from Khalil14 that requires that
$$ \Updelta u_{j} (t) = \sum\limits_{i = 1}^{3} {K_{ji} \int\limits_{0}^{t} {\Updelta x_{i} (t)dt} } ,\quad i = 1,2,3,\quad j = 1,2 $$
(13)
with gains Kji appropriately chosen to preserve system stability and provide fast convergence to the equilibrium. Note that at the equilibrium Δxi(t) = 0, i = 1,2,3. However, there is no systematic procedure how to choose Kji for nonlinear systems. This choice is system dependent. One may try to use the general results for tuning PID controllers.1 Even more, our goal is also to minimize Δuj(t), i = 1,2, the required amounts of additional drugs to be given to the patient.
The technique that takes care of both minimization of Δui(t) and optimal convergence to the equilibrium (desired steady state) point is known as control of nonlinear systems via linearization.9 We have already linearized our nonlinear system and derived relations among the system state variables and control signals, (10)–(12). From these equations we can find the desired control strategies (amount of drugs needed), ujsstar, j = 1,2, that keep the system at the desired equilibrium point. Δuj(t) is obtained via linear-quadratic optimal control theory: the linear dynamic system for Δxi(t) is given in (8), and the quadratic performance criterion is
$$ J = \int\limits_{0}^{\infty } {[\Updelta x^{\text{T}} (t)W_{1} \Updelta x(t) + \Updelta u^{\text{T}} (t)W_{2} \Updelta u(t)]dt} ,\quad \Updelta x(t) = \left[ {\begin{array}{*{20}c} {\Updelta x_{1} (t)} \\ {\Updelta x_{2} (t)} \\ {\Updelta x_{3} (t)} \\ \end{array} } \right],\quad \Updelta u(t) = \left[ {\begin{array}{*{20}c} {\Updelta u_{1} (t)} \\ {\Updelta u_{2} (t)} \\ \end{array} } \right] $$
(14)
where W1 ≥ and W2 > 0 are corresponding weighting matrices. It is known that the optimal control that minimizes the above performance criterion9 along trajectories of (8) is
$$ \Updelta u^{\text{opt}} (t) = - F^{\text{opt}} \Updelta x(t) $$
(15)
with the optimal gain satisfying
$$ F^{\text{opt}} = W_{2}^{ - 1} B^{\text{T}} P,\quad PA + A^{\text{T}} P + W_{1} - PBW_{2}^{ - 1} B^{\text{T}} P = 0 $$
(16)
The algebraic equation introduced in (16) is well known Riccati algebraic equation. It can be easily solved using MATLAB.
The corresponding MATLAB/Simulink block diagram of the proposed controller is given in Fig. 7. The block diagram contains two limiters whose limit values are set to zero and one, due to the fact that the control input signals in the considered model (5) are normalized to the interval 0 ≤ uj(t) ≤ 1, j = 1,2.
https://static-content.springer.com/image/art%3A10.1007%2Fs10439-009-9672-7/MediaObjects/10439_2009_9672_Fig7_HTML.gif
Figure 7

Simulink block diagram for the proposed optimal feedback control strategy. The HIV model is defined in formula (1) and represented by the block diagram in Fig. 1. The set point controller is defined by formulas (11) and (12). The optimal feedback gain Fopt is defined in formula (10)

Simulation Results

Let us assume that the desired steady state values are determined by the following data x1sstar = 490 per mm3, x3sstar = 30 per mm3 (this meets the U.S. Department of Health and Human Services HIV Therapy Guidelines22 that the number of virons should be suppressed under 50 per mm3). The remaining steady state quantities x2sstar, u1sstar, x2sstar can be evaluated, respectively, from formulas (10) and (11). For simplicity, we take the weights in the performance criterion to be equal to identity matrices, that is, W1 = I3, W2 = I2 (it will be interesting to study the impact of different weights on the results obtained). The corresponding simulation results for the optimal feedback (closed-loop) strategy are presented in Figs. 813. We have assumed that at some point in time (assumed for simplicity to be at the origin) the number of viruses is out of the desired range and equal to x3(0) = 60 per mm3. The remaining two initial conditions (needed to run simulation) are chosen as x1(0) = 560 per mm3 with x2(0) evaluated from (10).
https://static-content.springer.com/image/art%3A10.1007%2Fs10439-009-9672-7/MediaObjects/10439_2009_9672_Fig8_HTML.gif
Figure 8

Dynamics of healthy T-cells x1(t) for the proposed optimal feedback (closed-loop) strategy and the initial conditions x1(0) = 560 per mm3, x3(0) = 60 per mm3 with x2(0) evaluated from (10). It can be seen that the number of healthy T-cells decays slowly from 560 to 490 per mm3, reaching the target value after roughly 110 days

https://static-content.springer.com/image/art%3A10.1007%2Fs10439-009-9672-7/MediaObjects/10439_2009_9672_Fig9_HTML.gif
Figure 9

Dynamics of infected T-cells x2(t) for the proposed optimal feedback (closed-loop) strategy and the initial conditions x1(0) = 560 per mm3, x3(0) = 60 per mm3 with x2(0) evaluated from (10)

https://static-content.springer.com/image/art%3A10.1007%2Fs10439-009-9672-7/MediaObjects/10439_2009_9672_Fig10_HTML.gif
Figure 10

Dynamics of HIV virons x3(t) over the time interval of five hundred days for the proposed optimal feedback (closed-loop) strategy and the initial conditions x1(0) = 60 per mm3, x3(0) = 60 per mm3 with x2(0) evaluated from (10). It can be seen that the number of viruses decays very rapidly and in less than one day reaches its desired target value of 30 per mm3

https://static-content.springer.com/image/art%3A10.1007%2Fs10439-009-9672-7/MediaObjects/10439_2009_9672_Fig11_HTML.gif
Figure 11

Dynamics of HIV virons x3(t) over the time interval of ten days for the proposed optimal feedback (closed-loop) strategy and the initial conditions x1(0) = 560 per mm3, x3(0) = 60 per mm3 with x2(0) evaluated from (10). It can be seen that the number viruses decays very rapidly and in less than one day reaches its desired target value of 30 per mm3, and it remains there during the entire time interval of simulation equal to 500 days

https://static-content.springer.com/image/art%3A10.1007%2Fs10439-009-9672-7/MediaObjects/10439_2009_9672_Fig12_HTML.gif
Figure 12

Relative dosage of RTI drugs over 500 days for the proposed optimal feedback (closed-loop) strategy and the initial conditions x1(0) = 560 per mm3, x3(0) = 60 per mm3 with x2(0) evaluated from (10). It can be seen that during the first 110 days the drug RTI should not be administrated and after that period, it should (in a matter of days) be increased to the relative value of 0.43

https://static-content.springer.com/image/art%3A10.1007%2Fs10439-009-9672-7/MediaObjects/10439_2009_9672_Fig13_HTML.gif
Figure 13

Relative dosage of PI drugs over 500 days for the proposed optimal feedback (closed-loop) strategy and the initial conditions x1(0) = 560 per mm3, x3(0) = 60 per mm3 with x2(0) evaluated from (10). The results presented indicate that the drug PI should be given during the initial therapy. Its dosage should very quickly go to 0.54 and after 110 days it should be gradually reduced such that after 150 days its dosage is equal to 0.14. That dosage should remain constant during the time interval from 150 to 500 days

It can be observed from Fig. 8 that for the optimal feedback strategy, the number of healthy T-cells decays slowly from 560 to 490 per mm3, reaching the target value after roughly 110 days. On the other hand, it can be seen from Fig. 11 that the number of viruses decays very rapidly and in less than one day reaches its desired target value of 30 per mm3. Figure 10 shows that the number of viruses remains in its steady state desired target value of 30 per mm3 for the entire time interval of simulation equal to 500 days. Figure 9 indicates that the number of infected T-cells at steady state is only one per mm3. The proposed optimal feedback strategies are presented in Figs. 12 and 13. The results obtained for the drug treatment schedule are very interesting. They indicate that during the first 110 days the drug RTI should not be administrated at all and after that period it should in a matter of days be increased to the relative value of 0.43. The drug PI should be the only one given during the initial therapy. Its dosage should very quickly go to 0.54 and after 110 days it should be gradually reduced such that after 150 days its dosage is equal to 0.14. That dosage should remain constant during the time interval from 150 to 500 days.

Note that the proposed treatment strategy has two components: the first one that comes from the constant feed-forward set-point controller (open-loop controller) and the second one comes from the time varying optimal feedback strategy (closed-loop controller), see Fig. 7, where these two components are added in the summing block to form the proposed control signal u(t). It is interested to see what the results will look like when the feedback loop is not present and the drug administration is based only on the feedforward controller (open-loop strategy). The feed-forward controller is designed such that the desired target values are reached at the steady state. However, this controller is not able to take care about the transient behavior, that is, it is unable to control the system passage from the given initial state to the desired steady state.

Open-Loop Drug Treatment

The results when only the quasi-steady state target strategies are used, open-loop control, obtained for Fopt = 0, are presented in Figs. 1417. In this case only the feed-forward controller is active producing at all times constant values for both drugs, that is, RTI = 0.43 and PI = 0.14 (note that these are relative values indicating the percentage of the maximal values of these two drugs). It can be seen from Figs. 1417 that the open-loop drug administration strategy produces much worse results than the previously presented the closed-loop strategy. Comparing Figs. 10, 11 and 14, 15, we see that in the case of the open-loop strategy the number of viruses is far above the number of viruses in the case of closed-loop strategies (30 per mm3 after the first day of treatment) and that it takes more than 500 days for the open-loop strategy to reach the steady state target value of 30 per mm3. In addition, it can be seen from Fig. 17 that the number of healthy T-cells goes faster (80 days) to the level below 490 per mm3 and remains under that level for more than 400 days before reaching the desired steady state value of 490 per mm3. Similarly, Fig. 16 indicates that in the case of the open-loop strategy the number of infected T-cells is above the number of infected T-cells when the closed-loop strategy is used.
https://static-content.springer.com/image/art%3A10.1007%2Fs10439-009-9672-7/MediaObjects/10439_2009_9672_Fig14_HTML.gif
Figure 14

Dynamics of HIV-virons for the time interval of 500 days under the open-loop strategy. Comparing this figure with Fig. 10 , it can be seen that in the case of the open-loop strategy the number of viruses is far above the number of viruses in the case of closed-loop strategies (30 per mm3 after the first day of treatment) and that it takes more than 500 days for the open-loop strategy to reach the steady state target value of 30 per mm3

https://static-content.springer.com/image/art%3A10.1007%2Fs10439-009-9672-7/MediaObjects/10439_2009_9672_Fig15_HTML.gif
Figure 15

Dynamics HIV-virons for the time interval of 10 days under the open-loop control strategy (see also Fig. 14 for the time scale from 0 to 150 days)

https://static-content.springer.com/image/art%3A10.1007%2Fs10439-009-9672-7/MediaObjects/10439_2009_9672_Fig16_HTML.gif
Figure 16

Dynamics of infected T-cells under the open-loop control strategy. This figure indicates that in the case of the open-loop control strategy, the number of infected T-cells is above the number of infected T-cells when the closed-loop strategy is used, see Fig. 9

https://static-content.springer.com/image/art%3A10.1007%2Fs10439-009-9672-7/MediaObjects/10439_2009_9672_Fig17_HTML.gif
Figure 17

Dynamics of healthy T-cells under the open-loop control strategy. It can be seen that the number of healthy T-cells drops faster (80 days) to the level below 490 per mm3 and remains under that level for more than 400 days before reaching the desired steady state value of 490 per mm3. It can be concluded that the corresponding closed-loop result presented in Fig. 8 is superior

Conclusions

We have presented a drug treatment strategy based on the optimal control theory and the control of nonlinear system technique via linearization. It has been shown that such a feedback (closed-loop) strategy regulates the HIV dynamics (represented by the considered model) much more efficiently than the open-loop strategy. Open-loop drug administration is common in medical practice (it prescribes constant levels of medications, for example 10 mg, 20 mg, and patients are checked every month or three or six months to eventually receive new dosages). The closed-loop drug administration requires that dosages take continuum of values and that the state of a patient be monitored at all times, every day. This might be a difficult requirement, but the advantages of the closed-loop over the open-loop drug administration, as shown in our simulation results, are enormous.

Copyright information

© Biomedical Engineering Society 2009