Abstract
In this paper we present a finite volume method for solving Hamilton-Jacobi-Bellman(HJB) equations governing a class of optimal feedback control problems. This method is based on a finite volume discretization in state space coupled with an upwind finite difference technique, and on an implicit backward Euler finite differencing in time, which is absolutely stable. It is shown that the system matrix of the resulting discrete equation is an M-matrix. To show the effectiveness of this approach, numerical experiments on test problems with up to three states and two control variables were performed. The numerical results show that the method yields accurate approximate solutions to both the control and the state variables.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
M. Bardi and I. Capuzzo-Dolcetta, Optimal Control and Viscosity Solutions of Hamilton-Jacobi-Bellman equations, Birlhäuser, Boston-Basel-Berlin (1997).
M. Bardi, M. Falcone, P. Soravia. Numerical methods for pursuit-evasion games via viscosity solutions, in Stochastic and differential games: theory and numerical methods eds. M. Bardi, T. Parthasarathy, T.E.S. Raghavan, Birkhäuser, Boston (1999) 105-175.
M.G. Crandall and P.L. Lions. Viscosity solutions of Hamilton-Jacobi equations. Trans. AMS, 277, No.1 (1983) 1-42.
M.G. Crandall, L.C. Evans and P.L. Lions. Some properties of viscosity solutions of Hamilton-Jacobi equations, Trans. AMS, 282, No.2 (1984) 487-502.
M.G. Crandall and P.L. Lions, Two approximations of solutions of Hamilton-Jacobi equations. Math. Comp., 43 (1984) 1-19.
M.G. Crandall, H. Ishii and P.L. Lions. User’s guide to viscosity solutions of second order partial differential equations. Bull. Amer. Math. Soc., 27 (1992) 1-67.
M. Falcone. Numerical solution of dynamic programming equations. Appendix in [1].
M. Falcone and T. Giorgo. An approximation scheme for evolutive Hamilton-Jacobi equations. Stochastic analysis, control, optimization and applications eds W.M. McEneaney, G. George Y. and Q. Zhang, Birkhüser Boston, Boston, MA (1999) 289-303.
C.-S. Huang, S. Wang and K.L. Teo. Solving Hamilton-Jacobi-Bellman equations by a modified method of characteristic, Nonlinear Analysis, TMA, 40 (2000) 279-293.
H.J. Kushner and P.G. Dupuis. Numerical Methods for Stochastic Control Problems in Continuous Time. Springer-Verlag, New York (1992).
P.L. Lions. Generalised solutions of Hamilton-Jacobi equations. Pitman Research Notes in Mathematics, 69 (1982).
P. Sonneveld. CGS, a fast Lanczos-type solver for nonsymmetric linear systems. SIAM J. Sci. Statist. Comput., 10 (1989) 36-52.
S. Wang, F. Gao and K.L. Teo. An upwind finite difference method for the approximation of viscosity solutions to Hamilton-Jacobi-Bellman equations. IMA J. Math. Control & Its Appl., 17 (2000) 167-178.
J.L. Zhou, A.L. Tits and C.T. Lawrence. User’s Guide for FFSQP Version 3.7: A Fortran Code for solving Constrained Optimization Problems, Generating Iterates Satisfying All Inequality and Linear Constraints. Technical Report TR92-107-r2, System Research Center, Univ. of Maryland (1997).
H.A. van der vorst. Bi-CGSTAB: a fast and smoothly converging variant of Bi-CG for the solution of nonsymmetric linear systems. SIAM J. Sci. Statist. Comput., 13 (1992) 631-644
R.S. Varga. Matrix Iterative Analysis. Prentice-Hall, Englewood Cliffs, NJ (1962).
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Wang, S., Jennings, L. & Teo, K. Numerical Solution of Hamilton-Jacobi-Bellman Equations by an Upwind Finite Volume Method. Journal of Global Optimization 27, 177–192 (2003). https://doi.org/10.1023/A:1024980623095
Issue Date:
DOI: https://doi.org/10.1023/A:1024980623095