Numerical Solution of Hamilton-Jacobi-Bellman Equations by an Upwind Finite Volume Method
Rent the article at a discountRent now
* Final gross prices may vary according to local VAT.Get Access
In this paper we present a finite volume method for solving Hamilton-Jacobi-Bellman(HJB) equations governing a class of optimal feedback control problems. This method is based on a finite volume discretization in state space coupled with an upwind finite difference technique, and on an implicit backward Euler finite differencing in time, which is absolutely stable. It is shown that the system matrix of the resulting discrete equation is an M-matrix. To show the effectiveness of this approach, numerical experiments on test problems with up to three states and two control variables were performed. The numerical results show that the method yields accurate approximate solutions to both the control and the state variables.
- M. Bardi and I. Capuzzo-Dolcetta, Optimal Control and Viscosity Solutions of Hamilton-Jacobi-Bellman equations, Birlhäuser, Boston-Basel-Berlin (1997).
- M. Bardi, M. Falcone, P. Soravia. Numerical methods for pursuit-evasion games via viscosity solutions, in Stochastic and differential games: theory and numerical methods eds. M. Bardi, T. Parthasarathy, T.E.S. Raghavan, Birkhäuser, Boston (1999) 105-175.
- M.G. Crandall and P.L. Lions. Viscosity solutions of Hamilton-Jacobi equations. Trans. AMS, 277, No.1 (1983) 1-42.
- M.G. Crandall, L.C. Evans and P.L. Lions. Some properties of viscosity solutions of Hamilton-Jacobi equations, Trans. AMS, 282, No.2 (1984) 487-502.
- M.G. Crandall and P.L. Lions, Two approximations of solutions of Hamilton-Jacobi equations. Math. Comp., 43 (1984) 1-19.
- M.G. Crandall, H. Ishii and P.L. Lions. User’s guide to viscosity solutions of second order partial differential equations. Bull. Amer. Math. Soc., 27 (1992) 1-67.
- M. Falcone. Numerical solution of dynamic programming equations. Appendix in .
- M. Falcone and T. Giorgo. An approximation scheme for evolutive Hamilton-Jacobi equations. Stochastic analysis, control, optimization and applications eds W.M. McEneaney, G. George Y. and Q. Zhang, Birkhüser Boston, Boston, MA (1999) 289-303.
- C.-S. Huang, S. Wang and K.L. Teo. Solving Hamilton-Jacobi-Bellman equations by a modified method of characteristic, Nonlinear Analysis, TMA, 40 (2000) 279-293.
- H.J. Kushner and P.G. Dupuis. Numerical Methods for Stochastic Control Problems in Continuous Time. Springer-Verlag, New York (1992).
- P.L. Lions. Generalised solutions of Hamilton-Jacobi equations. Pitman Research Notes in Mathematics, 69 (1982).
- P. Sonneveld. CGS, a fast Lanczos-type solver for nonsymmetric linear systems. SIAM J. Sci. Statist. Comput., 10 (1989) 36-52.
- S. Wang, F. Gao and K.L. Teo. An upwind finite difference method for the approximation of viscosity solutions to Hamilton-Jacobi-Bellman equations. IMA J. Math. Control & Its Appl., 17 (2000) 167-178.
- J.L. Zhou, A.L. Tits and C.T. Lawrence. User’s Guide for FFSQP Version 3.7: A Fortran Code for solving Constrained Optimization Problems, Generating Iterates Satisfying All Inequality and Linear Constraints. Technical Report TR92-107-r2, System Research Center, Univ. of Maryland (1997).
- H.A. van der vorst. Bi-CGSTAB: a fast and smoothly converging variant of Bi-CG for the solution of nonsymmetric linear systems. SIAM J. Sci. Statist. Comput., 13 (1992) 631-644
- R.S. Varga. Matrix Iterative Analysis. Prentice-Hall, Englewood Cliffs, NJ (1962).
- Numerical Solution of Hamilton-Jacobi-Bellman Equations by an Upwind Finite Volume Method
Journal of Global Optimization
Volume 27, Issue 2-3 , pp 177-192
- Cover Date
- Print ISSN
- Online ISSN
- Kluwer Academic Publishers
- Additional Links
- Optimal feedback control
- Hamilton-Jacobi-Bellman equation
- finite volume method
- Viscosity solution
- upwind finite difference
- Industry Sectors
- Author Affiliations
- 1. Centre for Applied Dynamics and Optimization, Department of Mathematics and Statistics, The University of Western Australia, 35 Stirling Highway, Crawley, WA, 6009, Australia
- 2. Department of Applied Mathematics, Hong Kong Polytechnic University, Hung Hom, Kowloon, Hong Kong