Optimization and Engineering

, Volume 15, Issue 1, pp 119–136 | Cite as

Dynamic programming approach to the numerical solution of optimal control with paradigm by a mathematical model for drug therapies of HIV/AIDS



In this paper, we present a new numerical algorithm to find the optimal control for the general nonlinear lumped systems without state constraints. The dynamic programming-viscosity solution (DPVS) approach is developed and the numerical solutions of both approximate optimal control and trajectory are produced. To show the effectiveness and efficiency of new algorithm, we apply it to an optimal control problem of two types of drug therapies for human immunodeficiency virus (HIV)/acquired immune deficiency syndrome (AIDS). The quality of the obtained optimal control and the trajectory pair is checked through comparison with the costs under the arbitrarily selected different controls. The results illustrate the effectiveness of the algorithm.


Optimal control Viscosity solution Dynamic programming Numerical solution 


  1. Adams BM, Banks HT, Kwon H-D, Tran HT (2004) Dynamic multidrug therapies for HIV: optimal and STI control approaches. Math Biosci Eng 1(2):223–241 CrossRefMATHMathSciNetGoogle Scholar
  2. Adams BM, Banks HT, Davidian M, Kwon H-D, Tran HT, Wynne SN, Rosenberg ES (2005) HIV dynamics: modeling, data analysis, and optimal treatment protocols. J Comput Appl Math 184(1):10–49 CrossRefMATHMathSciNetGoogle Scholar
  3. Adams BM, Banks HT, Davidian M, Rosenberg ES (2007) Estimation and prediction with HIV-treatment interruption data. Bull Math Biol 69(2):563–584 CrossRefMATHGoogle Scholar
  4. Bardi M, Capuzzo-Dolcetta I (1997) Optimal control and viscosity solutions of Hamilton-Jacobi-Bellman equations. Birkhäuser, Boston CrossRefMATHGoogle Scholar
  5. Boltyanskii VG (1971) Mathematical methods of optimal control. Balskrishnan-Neustadt series. Holt, Rinehart and Winston, New York-Montreal-London. Translated from the Russian by K.N. Trirogoff. Edited by Ivin Tarnove Google Scholar
  6. Brandt ME, Chen GR (2001) Feedback control of a biodynamical model of HIV-1. IEEE Trans Biomed Eng 48(7):754–759 CrossRefGoogle Scholar
  7. Bryson AE Jr (1996) Optimal control—1950 to 1985. IEEE Control Syst 16:26–33 CrossRefGoogle Scholar
  8. Butler S, Kirschner D, Lenhart S (1997) Optimal control of chemotherapy affecting the infectivity of HIV. In: Advances in mathematical population dynamics—molecules, cells and man. Series in mathematical biology and medicine, vol 6. World Scientific, River Edge, pp 557–569 Google Scholar
  9. Craig IK, Xia X (2001) Can HIV/AIDS be controlled? Applying control engineering concepts outside traditional fields. IEEE Control Syst Mag 25(1):80–83 CrossRefGoogle Scholar
  10. Craig IK, Xia X, Venter JW (2004) Introducing HIV/AIDS education into the electrical engineering curriculum at the University of Pretoria. IEEE Trans Ed 47(1):65–73 CrossRefGoogle Scholar
  11. Crandall MG, Lions PL (1984) Two approximations of solutions of Hamilton-Jacobi equations. Math Comput 43:1–19 CrossRefMATHMathSciNetGoogle Scholar
  12. David J, Tran H, Banks HT (2011) Receding horizon control of HIV. Optim Control Appl Methods 32(6):681–699 CrossRefMATHMathSciNetGoogle Scholar
  13. Ding Y, Wang Z, Ye H (2012) Optimal control of a fractional-order HIV-immune system with memory. IEEE Trans Control Syst Technol 20(3):763–769 CrossRefGoogle Scholar
  14. Falcone M, Lanucara P, Seghini A (1994) A splitting algorithm for Hamilton-Jacobi-Bellman equations. Appl Numer Math 15:207–218 CrossRefMATHMathSciNetGoogle Scholar
  15. Felippe de Souza JAM, Caetano MAL, Yoneyama T (2000) Optimal control theory applied to the anti-viral treatment of AIDS. In: Proceedings of the 39th IEEE conference on decision and control, Sydney, Australia, December 2000, pp 4839–4844 Google Scholar
  16. Fleming WH, Rishel RW (1975) Deterministic and stochastic optimal control. Springer, Berlin-New York CrossRefMATHGoogle Scholar
  17. Fleming WH, Soner HM (1993) Controlled Markov processes and viscosity solutions. Springer, New York MATHGoogle Scholar
  18. Guo BZ, Sun B (2005) Numerical solution to the optimal birth feedback control of a population dynamics: a viscosity solution approach. Optim Control Appl Methods 26:229–254 CrossRefMathSciNetGoogle Scholar
  19. Guo BZ, Sun B (2007) Numerical solution to the optimal feedback control of continuous casting process. J Glob Optim 39:171–195 CrossRefMATHMathSciNetGoogle Scholar
  20. Hackbusch W (1978) A numerical method for solving parabolic equations with opposite orientations. Computing 20(3):229–240 CrossRefMATHMathSciNetGoogle Scholar
  21. Ho YC (2005) On centralized optimal control. IEEE Trans Autom Control 50(4):537–538 CrossRefGoogle Scholar
  22. Huang CS, Wang S, Teo KL (2000) Solving Hamilton-Jacobi-Bellman equations by a modified method of characteristics. Nonlinear Anal 40:279–293 CrossRefMATHMathSciNetGoogle Scholar
  23. Huang CS, Wang S, Teo KL (2004) On application of an alternating direction method to Hamilton-Jacobi-Bellman equations. J Comput Appl Math 166:153–166 CrossRefMATHMathSciNetGoogle Scholar
  24. Jacobson DH, Mayne DQ (1970) Differential dynamic programming. Elsevier, New York MATHGoogle Scholar
  25. Jeffrey AM, Xia X, Craig IK (2003) When to initiate HIV therapy: a control theoretic approach. IEEE Trans Biomed Eng 50(11):1213–1220 CrossRefGoogle Scholar
  26. Joshi HR (2002) Optimal control of an HIV immunology model. Optim Control Appl Methods 23(4):199–213 CrossRefMATHGoogle Scholar
  27. Kirschner D (1996) Using mathematics to understand HIV immune dynamics. Not Am Math Soc 43(2):191–202 MATHMathSciNetGoogle Scholar
  28. Kirschner D, Webb GF (1998) Immunotherapy of HIV-1 infection. J Biol Syst 6(1):71–83 CrossRefMATHGoogle Scholar
  29. Kirschner D, Lenhart S, Serbin S (1997) Optimal control of the chemotherapy of HIV. J Math Biol 35:775–792 CrossRefMATHMathSciNetGoogle Scholar
  30. Lenhart S, Workman JT (2007) Optimal control applied to biological models. Chapman & Hall/CRC mathematical and computational biology series. Chapman & Hall/CRC, Boca Raton MATHGoogle Scholar
  31. Lin TC, Arora JS (1994) Differential dynamic programming technique for optimal control. Optim Control Appl Methods 15(2):77–100 CrossRefMATHMathSciNetGoogle Scholar
  32. Mayne DQ, Polak E (1975) First-order strong variation algorithms for optimal control. J Optim Theory Appl 16(3/4):277–301 CrossRefMATHMathSciNetGoogle Scholar
  33. Miricǎ S (2008) MR2316829 (2008f:49001). Mathematical review of Lenhart and Workman (2007). Available at http://www.ams.org/mathscinet/
  34. Nowak MA, May RM (2000) Virus dynamics: mathematical principle of immunology and virology. Oxford University Press, Oxford Google Scholar
  35. Perelson AS, Nelson PW (1999) Mathematical analysis of HIV-1 dynamics in vivo. SIAM Rev 41(1):3–44 CrossRefMATHMathSciNetGoogle Scholar
  36. Press WH, Teukolsky SA, Vetterling WT, Flannery BP (2002) Numerical recipes in C++: the art of scientific computing, 2nd edn. Cambridge University Press, Cambridge Google Scholar
  37. Radisavljevic-Gajic V (2009) Optimal control of HIV-virus dynamics. Ann Biomed Eng 37(6):1251–1261 CrossRefGoogle Scholar
  38. Richardson S, Wang S (2006) Numerical solution of Hamilton-Jacobi-Bellman equations by an exponentially fitted finite volume method. Optimization 55(1–2):121–140 CrossRefMATHMathSciNetGoogle Scholar
  39. Rong L, Perelson AS (2009) Modeling HIV persistence, the latent reservoir, and viral blips. J Theor Biol 260(2):308–331 CrossRefMathSciNetGoogle Scholar
  40. Rong L, Feng Z, Perelson AS (2007) Emergence of HIV-1 drug resistance during antiretroviral treatment. Bull Math Biol 69(6):2027–2060 CrossRefMATHMathSciNetGoogle Scholar
  41. Rubio JE (1986) Control and optimization: the linear treatment of nonlinear problems, nonlinear science: theory and applications. Manchester University Press, Manchester Google Scholar
  42. Sargent RWH (2000) Optimal control. J Comput Appl Math 124:361–371 CrossRefMATHMathSciNetGoogle Scholar
  43. Stengel RF, Ghigliazza RM, Kulkarni NV (2002) Optimal enhancement of immune response. Bioinformatics 18(9):1227–1235 CrossRefGoogle Scholar
  44. Stoer J, Bulirsch R (1993) Introduction to numerical analysis. Springer, New York CrossRefMATHGoogle Scholar
  45. The INSIGHT-ESPRIT Study Group and SILCAAT Scientific Committee (2009) Interleukin-2 therapy in patients with HIV infection. N Engl J Med 361(16):1548–1559 CrossRefGoogle Scholar
  46. von Stryk O, Bulirsch R (1992) Direct and indirect methods for trajectory optimization. Ann Oper Res 37:357–373 CrossRefMATHMathSciNetGoogle Scholar
  47. Wang S, Gao F, Teo KL (2000) An upwind finite-difference method for the approximation of viscosity solutions to Hamilton-Jacobi-Bellman equations. IMA J Math Control Inf 17:167–178 CrossRefMATHMathSciNetGoogle Scholar
  48. Wang S, Jennings LS, Teo KL (2003) Numerical solution of Hamilton-Jacobi-Bellman equations by an upwind finite volume method. J Glob Optim 27:177–192 CrossRefMATHMathSciNetGoogle Scholar
  49. Wein LM, Zenios SA, Nowak M (1997) Dynamic multidrug therapies for HIV: a control theoretic approach. J Theor Biol 185:15–29 CrossRefGoogle Scholar
  50. Zurakowski R, Teel AR (2006) A model predictive control based scheduling method for HIV therapy. J Theor Biol 238(2):368–382 CrossRefMathSciNetGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2012

Authors and Affiliations

  1. 1.Academy of Mathematics and Systems ScienceAcademia SinicaBeijingP.R. China
  2. 2.School of MathematicsBeijing Institute of TechnologyBeijingP.R. China
  3. 3.School of Computational and Applied MathematicsUniversity of the WitwatersrandJohannesburgSouth Africa

Personalised recommendations