Abstract
Adaptive control and identification theory for stochastic systems was developed in the last few decades and is now very mature. Many excellent textbooks exist, see e.g., [9, 165, 192, 193, 206]. There has been a continuing discussion of what adaptive control is. In general, the problems studied in this area involve systems whose structures and/or parameters are unknown and/or are time-varying, However, to precisely define adaptive control is not an easy job [9, 206].
Never follow the beaten track, it leads only where others have been before.
Alexander Graham Bell, American (Scottish-born) scientist and inventor, (1847 – 1922)
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
H. Kaufman, I. Bar-Kana, and K. Sobel, Direct Adaptive Control Algorithms - Theory and Applications, Springer-Verlag, Noew York, 1994.
L. Ljung and T. Söderström, Theory and Practice of Recursive Identification, MIT Press, Cambridge, Massachusetts, 1983.
L. Ljung, System Identification - Theory for the User, PTR Prentice Hall, 1999.
K. S. Narendra and A. M. Annaswamy, Stable Adaptive Systems, Prentice Hall, Englewood Cliffs, New Jersey, 1989.
K. J. Åström and B. Wittenmark, Adaptive Control, Addison-Wesley, Reading, Massachusetts, 1989.
A. Al-Tamimi, F. L. Lewis and M. Abu-Khalaf, “Model-Free Q-Learning Designs for Linear Discrete-Time Zero-Sum Games with Application to H-Infinity Control,” Automatica, Vol. 43, 473-481, 2007.
S. J. Bradtke, B. E. Ydestie and A. G. Barto, “Adaptive Linear Quadratic Control Using Policy Iteration,” Proceedings of the American Control Conference, Baltimore, Maryland, U.S.A, 3475-3479, 1994.
O. L. V. Costa and J. C. C. Aya, “Monte Carlo TD(λ)-Methods for the Optimal Control of Discrete-Time Markovian Jump Linear Systems,” Automatica, Vol. 38, 217-225, 2002.
S. Hagen and B. Krose, “Linear Quadratic Regulation Using Reinforcement Learning,” Proceedings of 8th Belgian-Dutch Conference on Machine Learning, Wageningen, The Netherlands, 39-46, 1998.
P. J. Werbos, “Consistency of HDP applied to a simple reinforcement learning problem,” Neural Networks, Vol. 3, 179-189, 1990.
O. Hernández-Lerma and J. B. Lasserre, Discrete-Time Markov Control Processes: Basic Optimality Criteria, Springer-Verlag, New York, 1996.
D. P. Bertsekas and S. E. Shreve, Stochastic Optimal Control: The Discrete Time Case, Academic Press, New York, 1978.
O. Hernández-Lerma and J. B. Lasserre, “Policy Iteration for Average Cost Markov Control Processes on Borel Spaces,” Acta Appliandae Mathematicae, Vol. 47, 125-154, 1997.
S. P. Meyn and R. L. Tweedie, Markov Chains and Stochastic Stability, Springer-Verlag, London, 1993.
A. E. Bryson and Y. C. Ho, Applied Optimal Control: Optimization, Estimation, and Control, Blaisdell, Waltham, Massachusetts, 1969.
K. J. Zhang, Y. K. Xu, X. Chen and X. R. Cao, “Policy iteration based feedback control,” submmited to Automatica.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
Copyright information
© 2007 Springer Science+Business Media, LLC
About this chapter
Cite this chapter
Cao, XR. (2007). Adaptive Control Problems as MDPs. In: Stochastic Learning and Optimization. Springer, Boston, MA. https://doi.org/10.1007/978-0-387-69082-7_7
Download citation
DOI: https://doi.org/10.1007/978-0-387-69082-7_7
Publisher Name: Springer, Boston, MA
Print ISBN: 978-0-387-36787-3
Online ISBN: 978-0-387-69082-7
eBook Packages: Computer ScienceComputer Science (R0)