# Optimal Control of HIV-Virus Dynamics

## Authors

- First Online:

- Received:
- Accepted:

DOI: 10.1007/s10439-009-9672-7

- Cite this article as:
- Radisavljevic-Gajic, V. Ann Biomed Eng (2009) 37: 1251. doi:10.1007/s10439-009-9672-7

- 5 Citations
- 196 Views

## Abstract

In this paper we consider a mathematical model of HIV-virus dynamics and propose an efficient control strategy to keep the number of HIV virons under a pre-specified level and to reduce the total amount of medications that patients receive. The model considered is a nonlinear third-order model. The third-order model describes dynamics of three most dominant variables: number of healthy white blood cells (T-cells), number of infected T-cells, and number of virus particles. There are two control variables in this model corresponding to two categories of antiviral drugs: reverse transcriptase inhibitors (RTI) and protease inhibitors (PI). The proposed strategy is based on linearization of the nonlinear model at the equilibrium point (steady state). The corresponding controller has two components: the first one that keeps the system state variables at the desired equilibrium (set-point controller) and the second-one that reduces in an optimal way deviations of the system state variables from their desired equilibrium values. The second controller is based on minimization of the square of the error between the actual and desired (equilibrium) values for the linearized system (linear-quadratic optimal controller). The obtained control strategy recommends to HIV researchers and experimentalists that the constant dosages of drugs have to be administrated at all times (set point controller, open-loop controller) and that the variable dosages of drugs have to be administrated on a daily basis (closed-loop controller, feedback controller).

### Keywords

Nonlinear controlLinearizationOptimal controlHIV dynamic model## Introduction

The human immunodeficiency virus (HIV) causes the acquired immune deficiency syndrome, knows as AIDS. Mathematical models of HIV dynamics were derived several years ago.7,8,16,19 Controlling dynamics of HIV has been an interesting and challenging research problem.2,3,5,6,10,12,14,21 In this study, we consider the third-order model of HIV dynamics that captures the time rate of healthy and infected white blood cells (T-cells) and the number of HIV viruses. This model has two control variables corresponding to two major categories of antiretroviral drugs: reverse transcriptase inhibitors (RTI) and protease inhibitors (PI). There are more complex models of HIV dynamics that can be found in the literature.2,7,8,15,18 The methodology presented in this paper can be applied with minor modifications to the other models of HIV dynamics. However, as indicated by an anonymous reviewer of the manuscript, it should be emphasized that the presented analysis is limited in the sense that it “can not take into the account patient factors as physiological/genetic level, physicochemical factors at cell–protein–viral interactions level, and viral factors that relate to the various HIV strains and clades.”

*x*

_{1}(

*t*) denotes the number of healthy white blood cells (known as T-cells),

*x*

_{2}(

*t*) denotes the number of infected T-cells, and

*x*

_{3}(

*t*) represents free virus particles (viruses are not classified as living organisms because they can not replicate without a help of a host cell). The constant parameters in (1) are

*s*= 10 per mm^{3}per day is the constant source of healthy T-cells (thymus);*d*= 0.02 per day represents the death rate of healthy T-cells;*μ*_{1}= 0.24 per day represents the death rate of viruses;*μ*_{2}= 2.4 per day represents the death rate of infected T-cells;*β*= 2.4 × 10^{−5}per (mm^{3}× day) is the infectivity rate of free viruses;*k*= 100 per cell is the rate of virons (free virous particles) produced per infected T-cell.

These parameters can be estimated using an adaptive observer.23 When the number of T-cells, which is normally around 1000 per mm^{3} is under 200 per mm^{3}, a HIV-infected patient is classified as having AIDS.

## System Analysis

*x*

_{1a}(0) and still remain at the equilibrium (

*x*

_{1a}(0), 0, 0) as long as there are no perturbations in the directions of

*x*

_{2a}(0) and

*x*

_{3a}(0). However, only tiny perturbations of

*x*

_{2a}(0) = 0 and

*x*

_{3a}(0) = 0 will cause the equilibrium point instability. The second equilibrium point is stable since the corresponding Jacobian matrix eigenvalues are given by −0.0199 ±

*i*0.6658, −2.6418. This means that any motion starting in the neighborhood of this equilibrium point will tend in time to the equilibrium point. Our goal in this paper will be to achieve convergence to this stable equilibrium in an optimal manner, quickly and with the minimum dosage of drugs. Even more, once we add control signals, we will see that they will also impact the equilibrium point values so that we will have a chance to keep the number of infected cells and the viral load under pre-specified limits recommended in U.S. Department of Health and Human Services Guidelines.22

*x*

_{1}(0) ≠ 0, 0, 0), the system goes to the first equilibrium, in other words with no initial viruses present, no T-cells will be affected and all T-cells remain healthy. However, by introducing only one virous particle per mm

^{3}, that is, by choosing the initial condition as (1000, 0, 1), extremely quickly, within 25 days or so, the number of virous particles explodes to the order of 10

^{4}per mm

^{3}, Fig. 2. Similarly, the number of infected T-cells increases very rapidly during this initial time interval, see Fig. 3. However, due to the reaction of the body’s immune system, these numbers will sharply decline roughly within 50 days and reach their steady state values after 200 days. On the other hand, the number of the healthy T-cells drastically declines during the initial infection time period, in our case from the initial value of

*x*

_{1}(0) = 1000 per mm

^{3}to below 100 per mm

^{3}after only 30 days, and then it stabilizes around 240 per mm

^{3}after 200 or so days, see Fig. 4. The simulation results presented in Figs. 2–4 are consistent with what is known about the HIV “dynamics in vivo”.3,11,12,17,19

HIV patients can be well treated with a combination of drugs over a long period of time. One of the reasons for the negative outcome of the treatment is the high dosage of drugs that patients receive over such a long time period. In the next section, we will discuss control theory strategies that minimize the amount of drugs needed, while keeping the patient at the stable “quasi steady state” equilibrium for long time.

## Open- and Closed-Loop (Feedback) Control

*k*(infected T-cell productivity of free virous particles). In practice, this is achieved by using two major categories of antiretroviral drugs: reverse transcriptase inhibitors (RTI) acting on β and protease inhibitors (PI) acting on

*k*. The corresponding drug controlled system can be mathematically represented by

*u*

_{1}(

*t*) and

*u*

_{2}(

*t*) representing control variables. The control variables are normalized to the range 0 ≤

*u*

_{i}(

*t*) ≤ 1,

*i*= 1,2, with 1 corresponding to the maximal dosage and 0 corresponding to the situation when the drug is not administrated.

The controlled dynamic system (5) does not reflect different chemical structures of the RTI and PI drugs and the fact that they target different proteins in human cells. At the present state of knowledge, no mathematical dynamic models for the RTI and PI drugs and the corresponding proteins exist in the scientific literature. However, a new rapidly developing field, named systems biology16,20,21 (developed within bioinformatics), studies dynamics of chemical processes in human cells and develops corresponding mathematical models described by controlled differential equations. Hopefully, such models will be obtained in not very distant future. Once such models become available, it will be very interesting and very challenging to couple those models with the existing mathematical models for HIV dynamic and controls and perform the corresponding analysis at a much more sophisticated level. In our study, we have a simplified model in which the RTI drug affects multiplicatively the virus infectivity rate and the PI drug affects multiplicatively the infected T-cell productivity of free virous particles. Since model (5) uses the normalized values of control variables (RTI and PI drugs) our analysis will have a simplified goal to determine (via optimal feedback control theory) what percentage values of these drugs should be administrated daily over a long period of time.

The concept of system controllability4 was used to make a decision when to initiate HIV therapy13 for the HIV dynamic model very similar to (5). The conclusion reached states: “the therapy is best initiated when the viral load is easier to control because this implies the use of lower drug doses and consequently bearable side effects.”13 This conclusion was achieved by finding the singular values of the corresponding controllability Grammian,4 “a higher singular value indicates an easier to control viral load in the sense that the transition to the treatment steady state is faster. A lower singular value indicates a more difficult to control viral load characterized by a slow transition to the final state, given the same control effort.” Even more, the paper finds that the controllability profiles of both RTI and PI drugs are similar. Several types of controller for HIV dynamics can be found in the literature: classical PID controller1,6 controller based on linear state feedback.5 In addition, a controller based on the backstepping14 technique of nonlinear control technique that can drive state variables to their desired values and assure non-negativity of all states in the closed-loop (feedback) system was considered.10 An optimal controller15 based on dynamic programming was recommended “to reduce medication and establish long-term immune response against HIV-infection.” That paper also studies the controllability property of the HIV model used. A nonlinear model of HIV was controlled via the state-dependent Riccati equation2 approach. That paper considers both the full and partial state feedback cases, and in the case when all state variables are not available for feedback, an observer is designed to reconstruct all state space variables.

### Equilibrium Points of the Controlled HIV Model and the Set-Point Controller

*u*

_{iss}are constant steady state components of the control inputs whose roles are to keep the system at the constant equilibrium point, and Δ

*u*

_{i}(

*t*) are small time varying components that will be determined in an optimal manner (using the optimal control theory). Similarly, we assume that the state variables of the considered HIV dynamic model satisfy

*x*

_{iss}being constant and Δ

*x*

_{i}(

*t*) being time varying. Assuming that the control is applied as soon as the system leaves its steady state, the assumption that both Δ

*x*

_{i}(

*t*) and Δ

*u*

_{i}(

*t*) are small is realistic (small efforts are needed to correct small deviations from the equilibrium). Hence, if Δ

*u*

_{i}(

*t*) is implemented on a daily basis, small amounts of additional drugs will be needed to bring the system (model) back to its equilibrium. It can be shown, that the small deviations of the state variables approximately satisfy a linear system given by

*x*

_{1ss}and

*x*

_{2ss}

*x*

_{2ss}and

*x*

_{3ss}are required to be at all times below certain pre-specified target values,

*x*

_{2ss}

^{tar}and

*x*

_{3ss}

^{tar}then the target value

*x*

_{1ss}

^{tar}is evaluated from (10). Having defined the target values for the number of viruses, number of infected (and number of healthy) cells, we can evaluate from the second and third equations of (9) the corresponding steady state control inputs that keep the system at the desired, target, equilibrium point. It can be evaluated from these equations that the steady state target control values satisfy

*u*

_{1ss}

^{tar}and

*u*

_{2ss}

^{tar}, the target state variable values are given by

*x*

_{1ss}

^{tar}and

*x*

_{3ss}

^{tar}will be of interest for further analysis.

### Optimal Feedback Controller

Since the viral drugs are administrated based on measurements of healthy T-cells, *x*_{1}(*t*), and viruses *x*_{3}(*t*), the system output equation can be taken as *y*_{1}(*t*) = *x*_{1}(*t*) and *y*_{2}(*t*) = *x*_{3}(*t*). Note that the number of infected T-cells can be determined using an observer.6 Hence, we can assume that all state variables are available for feedback so that information about *x*_{i}(*t*) − *x*_{iss} = Δ*x*_{i}(*t*), *i* = 1,2,3, is known.

*x*

_{i}(

*t*) to zero is to use the nonlinear integrator technique from Khalil14 that requires that

*K*

_{ji}appropriately chosen to preserve system stability and provide fast convergence to the equilibrium. Note that at the equilibrium Δ

*x*

_{i}(

*t*) = 0,

*i*= 1,2,3. However, there is no systematic procedure how to choose

*K*

_{ji}for nonlinear systems. This choice is system dependent. One may try to use the general results for tuning PID controllers.1 Even more, our goal is also to minimize Δ

*u*

_{j}(

*t*),

*i*= 1,2, the required amounts of additional drugs to be given to the patient.

*u*

_{i}(

*t*) and optimal convergence to the equilibrium (desired steady state) point is known as control of nonlinear systems via linearization.9 We have already linearized our nonlinear system and derived relations among the system state variables and control signals, (10)–(12). From these equations we can find the desired control strategies (amount of drugs needed),

*u*

_{jss}

^{tar},

*j*= 1,2, that keep the system at the desired equilibrium point. Δ

*u*

_{j}(

*t*) is obtained via linear-quadratic optimal control theory: the linear dynamic system for Δ

*x*

_{i}(

*t*) is given in (8), and the quadratic performance criterion is

*W*

_{1}≥ and

*W*

_{2}> 0 are corresponding weighting matrices. It is known that the optimal control that minimizes the above performance criterion9 along trajectories of (8) is

*u*

_{j}(

*t*) ≤ 1,

*j*= 1,2.

### Simulation Results

*x*

_{1ss}

^{tar}= 490 per mm

^{3},

*x*

_{3ss}

^{tar}= 30 per mm

^{3}(this meets the U.S. Department of Health and Human Services HIV Therapy Guidelines22 that the number of virons should be suppressed under 50 per mm

^{3}). The remaining steady state quantities

*x*

_{2ss}

^{tar},

*u*

_{1ss}

^{tar},

*x*

_{2ss}

^{tar}can be evaluated, respectively, from formulas (10) and (11). For simplicity, we take the weights in the performance criterion to be equal to identity matrices, that is,

*W*

_{1}=

*I*

_{3},

*W*

_{2}=

*I*

_{2}(it will be interesting to study the impact of different weights on the results obtained). The corresponding simulation results for the optimal feedback (closed-loop) strategy are presented in Figs. 8–13. We have assumed that at some point in time (assumed for simplicity to be at the origin) the number of viruses is out of the desired range and equal to

*x*

_{3}(0) = 60 per mm

^{3}. The remaining two initial conditions (needed to run simulation) are chosen as

*x*

_{1}(0) = 560 per mm

^{3}with

*x*

_{2}(0) evaluated from (10).

It can be observed from Fig. 8 that for the optimal feedback strategy, the number of healthy T-cells decays slowly from 560 to 490 per mm^{3}, reaching the target value after roughly 110 days. On the other hand, it can be seen from Fig. 11 that the number of viruses decays very rapidly and in less than one day reaches its desired target value of 30 per mm^{3}. Figure 10 shows that the number of viruses remains in its steady state desired target value of 30 per mm^{3} for the entire time interval of simulation equal to 500 days. Figure 9 indicates that the number of infected T-cells at steady state is only one per mm^{3}. The proposed optimal feedback strategies are presented in Figs. 12 and 13. The results obtained for the drug treatment schedule are very interesting. They indicate that during the first 110 days the drug RTI should not be administrated at all and after that period it should in a matter of days be increased to the relative value of 0.43. The drug PI should be the only one given during the initial therapy. Its dosage should very quickly go to 0.54 and after 110 days it should be gradually reduced such that after 150 days its dosage is equal to 0.14. That dosage should remain constant during the time interval from 150 to 500 days.

Note that the proposed treatment strategy has two components: the first one that comes from the constant feed-forward set-point controller (*open-loop controller*) and the second one comes from the *time varying optimal* feedback strategy (*closed-loop controller*), see Fig. 7, where these two components are added in the summing block to form the proposed control signal *u*(*t*). It is interested to see what the results will look like when the feedback loop is not present and the drug administration is based only on the feedforward controller (open-loop strategy). The feed-forward controller is designed such that the desired target values are reached at the steady state. However, this controller is not able to take care about the transient behavior, that is, it is unable to control the system passage from the given initial state to the desired steady state.

### Open-Loop Drug Treatment

*F*

^{opt}= 0, are presented in Figs. 14–17. In this case only the feed-forward controller is active producing at all times constant values for both drugs, that is, RTI = 0.43 and PI = 0.14 (note that these are relative values indicating the percentage of the maximal values of these two drugs). It can be seen from Figs. 14–17 that the

*open-loop drug administration strategy produces much worse results than the previously presented the closed-loop strategy*. Comparing Figs. 10, 11 and 14, 15, we see that in the case of the open-loop strategy the number of viruses is far above the number of viruses in the case of closed-loop strategies (30 per mm

^{3}after the first day of treatment) and that it takes more than 500 days for the open-loop strategy to reach the steady state target value of 30 per mm

^{3}. In addition, it can be seen from Fig. 17 that the number of healthy T-cells goes faster (80 days) to the level below 490 per mm

^{3}and remains under that level for more than 400 days before reaching the desired steady state value of 490 per mm

^{3}. Similarly, Fig. 16 indicates that in the case of the open-loop strategy the number of infected T-cells is above the number of infected T-cells when the closed-loop strategy is used.

## Conclusions

We have presented a drug treatment strategy based on the optimal control theory and the control of nonlinear system technique via linearization. It has been shown that such a feedback (closed-loop) strategy regulates the HIV dynamics (represented by the considered model) much more efficiently than the open-loop strategy. Open-loop drug administration is common in medical practice (it prescribes constant levels of medications, for example 10 mg, 20 mg, and patients are checked every month or three or six months to eventually receive new dosages). The closed-loop drug administration requires that dosages take continuum of values and that the state of a patient be monitored at all times, every day. This might be a difficult requirement, but the advantages of the closed-loop over the open-loop drug administration, as shown in our simulation results, are enormous.