Abstract
We focus on numerically solving a typical type of Hamilton-Jacobi-Bellman (HJB) equations arising from a class of optimal controls with a standard multidimensional diffusion model. Solving such an equation results in the value function and an optimal feedback control law. The Bellman's curse of dimensionality seems to be the main obstacle to applicability of most numerical algorithms for solving HJB. We decompose HJB into a number of lower-dimensional problems, and discuss how the usual alternating direction method can be extended for solving HJB. We present some convergence results, as well as preliminary experimental outcomes.
Similar content being viewed by others
References
Bartels RH, Stewart GW (1972) Algorithm 432, solution of the matrix equation AX+XB=C, Comm. ACM 15:820–826
Bellman R (1961) Adaptive Control Processes: A Guided Tour, Princeton University Press, New Jersey
Douglas J, Rachford HH (1956) On the numerical solution of heat conduction problems in two or three space variables, Trans. Amer. Math. Soc. 82:421–439
Duff IS, Erisman A, Reid J (1986) Direct Methods for Sparse Matrices, Clarendon Press, Oxford
Kellog RB (1969) A nonlinear alternating direction method, Math. Comp. 23:23–27
Kushner HJ, Dupuis PG (1992) Numerical Methods for Stochastic Control Problems in Continuous Time, Springer-Verlag, New York
Ortega JM, Rheinboldt WC (1970) Iterative Solution of Nonlinear Equations in Several Variables, Academic Press, New York
Peaceman DW, Rachford HH (1955) The numerical solution of parabolic and elliptic differential equations, J. SIAM 3:28–41
Smith RA (1968) Matrix equation XA+BX=C, SIAM J. Applied Math. 16:198–201
Sun M (1990) Numerical solutions of singular stochastic control problems in bounded intervals, Lectures in Applied Mathematics, AMS, 26:619–643
Sun M (1990) Iterative algorithms for solving undiscounted Bellman equations, Numer. Func. Anal. Optim. 11:149–166
Sun M (1993) Domain decomposition algorithms for solving Hamilton-Jacobi-Bellman equations, Numer. Func. Anal. Optim. 14:145–166
Tewarson R (1973) Sparse Matrices, Academic Press, New York
Varga RS (1962) Matrix Iterative Analysis, Prentice-Hall, Englewood Cliffs, NJ
Author information
Authors and Affiliations
Additional information
This research was funded in part by an RGC grant from the University of Alabama.
Rights and permissions
About this article
Cite this article
Sun, M. Alternating direction algorithms for solving Hamilton-Jacobi-Bellman equations. Appl Math Optim 34, 267–277 (1996). https://doi.org/10.1007/BF01182626
Accepted:
Issue Date:
DOI: https://doi.org/10.1007/BF01182626
Key words
- Optimal controls
- Dynamic programming
- Hamilton-Jacobi-Bellman equations
- Alternating direction methods
- Convergence