Journal of Global Optimization

, Volume 27, Issue 2–3, pp 177–192 | Cite as

Numerical Solution of Hamilton-Jacobi-Bellman Equations by an Upwind Finite Volume Method

  • S. Wang
  • L.S. Jennings
  • K.L. Teo


In this paper we present a finite volume method for solving Hamilton-Jacobi-Bellman(HJB) equations governing a class of optimal feedback control problems. This method is based on a finite volume discretization in state space coupled with an upwind finite difference technique, and on an implicit backward Euler finite differencing in time, which is absolutely stable. It is shown that the system matrix of the resulting discrete equation is an M-matrix. To show the effectiveness of this approach, numerical experiments on test problems with up to three states and two control variables were performed. The numerical results show that the method yields accurate approximate solutions to both the control and the state variables.

Optimal feedback control Hamilton-Jacobi-Bellman equation finite volume method Viscosity solution upwind finite difference 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    M. Bardi and I. Capuzzo-Dolcetta, Optimal Control and Viscosity Solutions of Hamilton-Jacobi-Bellman equations, Birlhäuser, Boston-Basel-Berlin (1997).Google Scholar
  2. 2.
    M. Bardi, M. Falcone, P. Soravia. Numerical methods for pursuit-evasion games via viscosity solutions, in Stochastic and differential games: theory and numerical methods eds. M. Bardi, T. Parthasarathy, T.E.S. Raghavan, Birkhäuser, Boston (1999) 105-175.Google Scholar
  3. 3.
    M.G. Crandall and P.L. Lions. Viscosity solutions of Hamilton-Jacobi equations. Trans. AMS, 277, No.1 (1983) 1-42.Google Scholar
  4. 4.
    M.G. Crandall, L.C. Evans and P.L. Lions. Some properties of viscosity solutions of Hamilton-Jacobi equations, Trans. AMS, 282, No.2 (1984) 487-502.Google Scholar
  5. 5.
    M.G. Crandall and P.L. Lions, Two approximations of solutions of Hamilton-Jacobi equations. Math. Comp., 43 (1984) 1-19.Google Scholar
  6. 6.
    M.G. Crandall, H. Ishii and P.L. Lions. User’s guide to viscosity solutions of second order partial differential equations. Bull. Amer. Math. Soc., 27 (1992) 1-67.Google Scholar
  7. 7.
    M. Falcone. Numerical solution of dynamic programming equations. Appendix in [1].Google Scholar
  8. 8.
    M. Falcone and T. Giorgo. An approximation scheme for evolutive Hamilton-Jacobi equations. Stochastic analysis, control, optimization and applications eds W.M. McEneaney, G. George Y. and Q. Zhang, Birkhüser Boston, Boston, MA (1999) 289-303.Google Scholar
  9. 9.
    C.-S. Huang, S. Wang and K.L. Teo. Solving Hamilton-Jacobi-Bellman equations by a modified method of characteristic, Nonlinear Analysis, TMA, 40 (2000) 279-293.Google Scholar
  10. 10.
    H.J. Kushner and P.G. Dupuis. Numerical Methods for Stochastic Control Problems in Continuous Time. Springer-Verlag, New York (1992).Google Scholar
  11. 11.
    P.L. Lions. Generalised solutions of Hamilton-Jacobi equations. Pitman Research Notes in Mathematics, 69 (1982).Google Scholar
  12. 12.
    P. Sonneveld. CGS, a fast Lanczos-type solver for nonsymmetric linear systems. SIAM J. Sci. Statist. Comput., 10 (1989) 36-52.Google Scholar
  13. 13.
    S. Wang, F. Gao and K.L. Teo. An upwind finite difference method for the approximation of viscosity solutions to Hamilton-Jacobi-Bellman equations. IMA J. Math. Control & Its Appl., 17 (2000) 167-178.Google Scholar
  14. 14.
    J.L. Zhou, A.L. Tits and C.T. Lawrence. User’s Guide for FFSQP Version 3.7: A Fortran Code for solving Constrained Optimization Problems, Generating Iterates Satisfying All Inequality and Linear Constraints. Technical Report TR92-107-r2, System Research Center, Univ. of Maryland (1997).Google Scholar
  15. 15.
    H.A. van der vorst. Bi-CGSTAB: a fast and smoothly converging variant of Bi-CG for the solution of nonsymmetric linear systems. SIAM J. Sci. Statist. Comput., 13 (1992) 631-644Google Scholar
  16. 16.
    R.S. Varga. Matrix Iterative Analysis. Prentice-Hall, Englewood Cliffs, NJ (1962).Google Scholar

Copyright information

© Kluwer Academic Publishers 2003

Authors and Affiliations

  • S. Wang
    • 1
  • L.S. Jennings
    • 1
  • K.L. Teo
    • 2
  1. 1.Centre for Applied Dynamics and Optimization, Department of Mathematics and StatisticsThe University of Western AustraliaCrawleyAustralia
  2. 2.Department of Applied MathematicsHong Kong Polytechnic UniversityHung Hom, KowloonHong Kong

Personalised recommendations