Skip to main content

Advertisement

SpringerLink
Log in

Dynamic programming approach to the numerical solution of optimal control with paradigm by a mathematical model for drug therapies of HIV/AIDS

  • Published:
Optimization and Engineering Aims and scope Submit manuscript

Abstract

In this paper, we present a new numerical algorithm to find the optimal control for the general nonlinear lumped systems without state constraints. The dynamic programming-viscosity solution (DPVS) approach is developed and the numerical solutions of both approximate optimal control and trajectory are produced. To show the effectiveness and efficiency of new algorithm, we apply it to an optimal control problem of two types of drug therapies for human immunodeficiency virus (HIV)/acquired immune deficiency syndrome (AIDS). The quality of the obtained optimal control and the trajectory pair is checked through comparison with the costs under the arbitrarily selected different controls. The results illustrate the effectiveness of the algorithm.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

References

  • Adams BM, Banks HT, Kwon H-D, Tran HT (2004) Dynamic multidrug therapies for HIV: optimal and STI control approaches. Math Biosci Eng 1(2):223–241

    Article  MATH  MathSciNet  Google Scholar 

  • Adams BM, Banks HT, Davidian M, Kwon H-D, Tran HT, Wynne SN, Rosenberg ES (2005) HIV dynamics: modeling, data analysis, and optimal treatment protocols. J Comput Appl Math 184(1):10–49

    Article  MATH  MathSciNet  Google Scholar 

  • Adams BM, Banks HT, Davidian M, Rosenberg ES (2007) Estimation and prediction with HIV-treatment interruption data. Bull Math Biol 69(2):563–584

    Article  MATH  Google Scholar 

  • Bardi M, Capuzzo-Dolcetta I (1997) Optimal control and viscosity solutions of Hamilton-Jacobi-Bellman equations. Birkhäuser, Boston

    Book  MATH  Google Scholar 

  • Boltyanskii VG (1971) Mathematical methods of optimal control. Balskrishnan-Neustadt series. Holt, Rinehart and Winston, New York-Montreal-London. Translated from the Russian by K.N. Trirogoff. Edited by Ivin Tarnove

    Google Scholar 

  • Brandt ME, Chen GR (2001) Feedback control of a biodynamical model of HIV-1. IEEE Trans Biomed Eng 48(7):754–759

    Article  Google Scholar 

  • Bryson AE Jr (1996) Optimal control—1950 to 1985. IEEE Control Syst 16:26–33

    Article  Google Scholar 

  • Butler S, Kirschner D, Lenhart S (1997) Optimal control of chemotherapy affecting the infectivity of HIV. In: Advances in mathematical population dynamics—molecules, cells and man. Series in mathematical biology and medicine, vol 6. World Scientific, River Edge, pp 557–569

    Google Scholar 

  • Craig IK, Xia X (2001) Can HIV/AIDS be controlled? Applying control engineering concepts outside traditional fields. IEEE Control Syst Mag 25(1):80–83

    Article  Google Scholar 

  • Craig IK, Xia X, Venter JW (2004) Introducing HIV/AIDS education into the electrical engineering curriculum at the University of Pretoria. IEEE Trans Ed 47(1):65–73

    Article  Google Scholar 

  • Crandall MG, Lions PL (1984) Two approximations of solutions of Hamilton-Jacobi equations. Math Comput 43:1–19

    Article  MATH  MathSciNet  Google Scholar 

  • David J, Tran H, Banks HT (2011) Receding horizon control of HIV. Optim Control Appl Methods 32(6):681–699

    Article  MATH  MathSciNet  Google Scholar 

  • Ding Y, Wang Z, Ye H (2012) Optimal control of a fractional-order HIV-immune system with memory. IEEE Trans Control Syst Technol 20(3):763–769

    Article  Google Scholar 

  • Falcone M, Lanucara P, Seghini A (1994) A splitting algorithm for Hamilton-Jacobi-Bellman equations. Appl Numer Math 15:207–218

    Article  MATH  MathSciNet  Google Scholar 

  • Felippe de Souza JAM, Caetano MAL, Yoneyama T (2000) Optimal control theory applied to the anti-viral treatment of AIDS. In: Proceedings of the 39th IEEE conference on decision and control, Sydney, Australia, December 2000, pp 4839–4844

    Google Scholar 

  • Fleming WH, Rishel RW (1975) Deterministic and stochastic optimal control. Springer, Berlin-New York

    Book  MATH  Google Scholar 

  • Fleming WH, Soner HM (1993) Controlled Markov processes and viscosity solutions. Springer, New York

    MATH  Google Scholar 

  • Guo BZ, Sun B (2005) Numerical solution to the optimal birth feedback control of a population dynamics: a viscosity solution approach. Optim Control Appl Methods 26:229–254

    Article  MathSciNet  Google Scholar 

  • Guo BZ, Sun B (2007) Numerical solution to the optimal feedback control of continuous casting process. J Glob Optim 39:171–195

    Article  MATH  MathSciNet  Google Scholar 

  • Hackbusch W (1978) A numerical method for solving parabolic equations with opposite orientations. Computing 20(3):229–240

    Article  MATH  MathSciNet  Google Scholar 

  • Ho YC (2005) On centralized optimal control. IEEE Trans Autom Control 50(4):537–538

    Article  Google Scholar 

  • Huang CS, Wang S, Teo KL (2000) Solving Hamilton-Jacobi-Bellman equations by a modified method of characteristics. Nonlinear Anal 40:279–293

    Article  MATH  MathSciNet  Google Scholar 

  • Huang CS, Wang S, Teo KL (2004) On application of an alternating direction method to Hamilton-Jacobi-Bellman equations. J Comput Appl Math 166:153–166

    Article  MATH  MathSciNet  Google Scholar 

  • Jacobson DH, Mayne DQ (1970) Differential dynamic programming. Elsevier, New York

    MATH  Google Scholar 

  • Jeffrey AM, Xia X, Craig IK (2003) When to initiate HIV therapy: a control theoretic approach. IEEE Trans Biomed Eng 50(11):1213–1220

    Article  Google Scholar 

  • Joshi HR (2002) Optimal control of an HIV immunology model. Optim Control Appl Methods 23(4):199–213

    Article  MATH  Google Scholar 

  • Kirschner D (1996) Using mathematics to understand HIV immune dynamics. Not Am Math Soc 43(2):191–202

    MATH  MathSciNet  Google Scholar 

  • Kirschner D, Webb GF (1998) Immunotherapy of HIV-1 infection. J Biol Syst 6(1):71–83

    Article  MATH  Google Scholar 

  • Kirschner D, Lenhart S, Serbin S (1997) Optimal control of the chemotherapy of HIV. J Math Biol 35:775–792

    Article  MATH  MathSciNet  Google Scholar 

  • Lenhart S, Workman JT (2007) Optimal control applied to biological models. Chapman & Hall/CRC mathematical and computational biology series. Chapman & Hall/CRC, Boca Raton

    MATH  Google Scholar 

  • Lin TC, Arora JS (1994) Differential dynamic programming technique for optimal control. Optim Control Appl Methods 15(2):77–100

    Article  MATH  MathSciNet  Google Scholar 

  • Mayne DQ, Polak E (1975) First-order strong variation algorithms for optimal control. J Optim Theory Appl 16(3/4):277–301

    Article  MATH  MathSciNet  Google Scholar 

  • Miricǎ S (2008) MR2316829 (2008f:49001). Mathematical review of Lenhart and Workman (2007). Available at http://www.ams.org/mathscinet/

  • Nowak MA, May RM (2000) Virus dynamics: mathematical principle of immunology and virology. Oxford University Press, Oxford

    Google Scholar 

  • Perelson AS, Nelson PW (1999) Mathematical analysis of HIV-1 dynamics in vivo. SIAM Rev 41(1):3–44

    Article  MATH  MathSciNet  Google Scholar 

  • Press WH, Teukolsky SA, Vetterling WT, Flannery BP (2002) Numerical recipes in C++: the art of scientific computing, 2nd edn. Cambridge University Press, Cambridge

    Google Scholar 

  • Radisavljevic-Gajic V (2009) Optimal control of HIV-virus dynamics. Ann Biomed Eng 37(6):1251–1261

    Article  Google Scholar 

  • Richardson S, Wang S (2006) Numerical solution of Hamilton-Jacobi-Bellman equations by an exponentially fitted finite volume method. Optimization 55(1–2):121–140

    Article  MATH  MathSciNet  Google Scholar 

  • Rong L, Perelson AS (2009) Modeling HIV persistence, the latent reservoir, and viral blips. J Theor Biol 260(2):308–331

    Article  MathSciNet  Google Scholar 

  • Rong L, Feng Z, Perelson AS (2007) Emergence of HIV-1 drug resistance during antiretroviral treatment. Bull Math Biol 69(6):2027–2060

    Article  MATH  MathSciNet  Google Scholar 

  • Rubio JE (1986) Control and optimization: the linear treatment of nonlinear problems, nonlinear science: theory and applications. Manchester University Press, Manchester

    Google Scholar 

  • Sargent RWH (2000) Optimal control. J Comput Appl Math 124:361–371

    Article  MATH  MathSciNet  Google Scholar 

  • Stengel RF, Ghigliazza RM, Kulkarni NV (2002) Optimal enhancement of immune response. Bioinformatics 18(9):1227–1235

    Article  Google Scholar 

  • Stoer J, Bulirsch R (1993) Introduction to numerical analysis. Springer, New York

    Book  MATH  Google Scholar 

  • The INSIGHT-ESPRIT Study Group and SILCAAT Scientific Committee (2009) Interleukin-2 therapy in patients with HIV infection. N Engl J Med 361(16):1548–1559

    Article  Google Scholar 

  • von Stryk O, Bulirsch R (1992) Direct and indirect methods for trajectory optimization. Ann Oper Res 37:357–373

    Article  MATH  MathSciNet  Google Scholar 

  • Wang S, Gao F, Teo KL (2000) An upwind finite-difference method for the approximation of viscosity solutions to Hamilton-Jacobi-Bellman equations. IMA J Math Control Inf 17:167–178

    Article  MATH  MathSciNet  Google Scholar 

  • Wang S, Jennings LS, Teo KL (2003) Numerical solution of Hamilton-Jacobi-Bellman equations by an upwind finite volume method. J Glob Optim 27:177–192

    Article  MATH  MathSciNet  Google Scholar 

  • Wein LM, Zenios SA, Nowak M (1997) Dynamic multidrug therapies for HIV: a control theoretic approach. J Theor Biol 185:15–29

    Article  Google Scholar 

  • Zurakowski R, Teel AR (2006) A model predictive control based scheduling method for HIV therapy. J Theor Biol 238(2):368–382

    Article  MathSciNet  Google Scholar 

Download references

Acknowledgements

The authors would like to thank the Editor and the anonymous reviewers for their valuable comments and suggestions that improve the paper substantially.

Author information

Authors and Affiliations

  1. Academy of Mathematics and Systems Science, Academia Sinica, Beijing, 100190, P.R. China

    Bao-Zhu Guo

  2. School of Mathematics, Beijing Institute of Technology, Beijing, 100081, P.R. China

    Bing Sun

  3. School of Computational and Applied Mathematics, University of the Witwatersrand, Private Bag 3 Wits 2050, Johannesburg, South Africa

    Bao-Zhu Guo & Bing Sun

Authors
  1. Bao-Zhu Guo
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Bing Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Bing Sun.

Additional information

This work was supported by the National Natural Science Foundation of China (11001012, 61273129, 11171011), the National Basic Research Program of China (2011CB808002), the Natural Science Foundation of Beijing (1102010), and the National Research Foundation of South Africa.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Guo, BZ., Sun, B. Dynamic programming approach to the numerical solution of optimal control with paradigm by a mathematical model for drug therapies of HIV/AIDS. Optim Eng 15, 119–136 (2014). https://doi.org/10.1007/s11081-012-9204-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11081-012-9204-4

Keywords

Search

Navigation