Abstract
The centerpiece of the theory of dynamic programming is the HamiltonJacobi-Bellman (HJB) equation, which can be used to solve for the optimal cost functional Vo for a nonlinear optimal control problem, while one can solve a second partial differential equation for the corresponding optimal control law ko.Although the direct solution of the HJB equation is computationally untenable, the HJB equation and the relationship between Vo and koserves as the basis for the adaptive dynamic programming algorithm. Here, one starts with an initial cost functional and stabilizing control law pair (Vo, k0) and constructs a sequence of cost functional/control law pairs (Vi, ki) in real time, which are stepwise stable and converge to the optimal cost functional/control law pair, for a prescribed nonlinear optimal control problem with unknown input affine state dynamics.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
S. Barnett The Matrices of Control Theory Van Nostrand Reinhold, New York, 1971.
R. E. Bellman Dynamic Programming Princeton University Press, Princeton, NJ, 1957.
D. P. Bertsekas Dynamic Programming: Deterministic and Stochastic Models Prentice-Hall, Englewood Cliffs, NJ, 1987.
A. Devinatz and J. L. Kaplan, “Asymptotic estimates for solutions of linear systems of ordinary differential equations having multiple characteristics roots,” Indiana University Math J., vol. 22, p. 335, 1972.
J. Dieudonne Foundations of Mathematical Analysis Academic Press, New York, 1960.
A. Halanay and V. Rasvan Applications of Liapunov Methods in Stability Kluwer, Dordrecht, 1993.
D. G. Luenberger Introduction to Dynamic Systems: Theory Models and Applications John Wiley, New York, 1979.
J. Murray, C. Cox, G. Lendaris, and R. Saeks, “Adaptive dynamic programming,” IEEE Trans. Systems, Man and Cybernetics: Part C vol. 32, pp. 140–153, 2002.
R. Saeks and C. Cox, “Adaptive critic control and functional link networks,” Proc. 1998 IEEE Conference on Systems Man and Cybernetics San Diego, CA, pp. 1652–1657, 1998.
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2003 Springer Science+Business Media New York
About this chapter
Cite this chapter
Murray, J.J., Cox, C.J., Saeks, R.E. (2003). The Adaptive Dynamic Programming Theorem. In: Liu, D., Antsaklis, P.J. (eds) Stability and Control of Dynamical Systems with Applications. Control Engineering. Birkhäuser, Boston, MA. https://doi.org/10.1007/978-1-4612-0037-6_19
Download citation
DOI: https://doi.org/10.1007/978-1-4612-0037-6_19
Publisher Name: Birkhäuser, Boston, MA
Print ISBN: 978-1-4612-6583-2
Online ISBN: 978-1-4612-0037-6
eBook Packages: Springer Book Archive