Abstract
In this paper, we develop an online learning control for a class of unknown nonaffine nonlinear discrete-time systems with unknown bounded disturbances. Under the framework of reinforcement learning, we employ two neural networks (NNs): an action NN is used to generate the control signal, and a critic NN is utilized to estimate the prescribed cost function. By using Lyapunov’s direct method, we prove the stability of the closed-loop system. Moreover, based on the developed adaptive scheme, we show that all signals involved are uniformly ultimately bounded. Finally, we provide an example to demonstrate the effectiveness and applicability of the present approach.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Sutton, R.S., Barto, A.G.: Reinforcement Learning-An Introduction. MIT Press, Cambridge (1998)
Du, K., Swamy, M.N.S.: Neural Networks and Statistical Learning. Springer, London (2014)
Werbos, P.J.: Neural networks and the experience and cultivation of mind. Neural Networks 32, 86–95 (2012)
Lewis, F.L., Vrabie, D., Vamvoudakis, K.G.: Reinforcement learning and feedback control: Using natural decision methods to design optimal adaptive controllers. IEEE Control Systems 32(6), 76–105 (2012)
Lewis, F.L., Liu, D.: Reinforcement Learning and Approximate Dynamic Programming for Feedback Control. Wiley, New Jersey (2012)
He, P., Jagannathan, S.: Reinforcement learning neural-network-based controller for nonlinear discrete-time systems with input constraints. IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics 37(2), 425–436 (2007)
Liu, D., Yang, X., Li, H.: Adaptive optimal control for a class of continuous-time affine nonlinear systems with unknown internal dynamics. Neural Computing and Applications 23(7–8), 1843–1850 (2013)
Liu, D., Wei, Q.: Finite-approximation-error-based optimal control approach for discrete-time nonlinear systems. IEEE Transactions on Cybernetics 43(2), 779–789 (2013)
Yang, Q., Jagannathan, S.: Reinforcement learning controller design for affine nonlinear discrete-time systems using online approximators. IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics 42(2), 377–390 (2012)
Yang, X., Liu, D., Wang, D.: Reinforcement learning for adaptive optimal control of unknown continuous-time nonlinear systems with input constraints. International Journal of Control 87(3), 553–566 (2014)
Yang, X., Liu, D., Wang, D.: Observer-based adaptive output feedback control for nonaffine nonlinear discrete-time systems using reinforcement learning. In: 20th International Conference on Neural Information Processing, Daegu, Korea, pp. 631–638 (2013)
Yang, Q., Vance, J.B., Jagannathan, S.: Control of nonaffine nonlinear discrete-time systems using reiforcement-learning-based linearly parameterized neural networks. IEEE Transactions on Systems, Man, Cybernetics, Part B, Cybernetics 38(4), 994–1001 (2008)
Xu, B., Yang, C., Shi, Z.: Reinforcement learning output feedback NN control using deterministic learning technique. IEEE Transactions on Neural Networks and Learning Systems 25(3), 635–641 (2014)
Lewis, F.L., Jagannathan, S., Yesildirek, A.: Neural Network Control of Robot Manipulators and Nonlinear Systems. Taylor & Francis, London (1999)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer International Publishing Switzerland
About this paper
Cite this paper
Yang, X., Liu, D., Wei, Q. (2014). Reinforcement-Learning-Based Controller Design for Nonaffine Nonlinear Systems. In: Zeng, Z., Li, Y., King, I. (eds) Advances in Neural Networks – ISNN 2014. ISNN 2014. Lecture Notes in Computer Science(), vol 8866. Springer, Cham. https://doi.org/10.1007/978-3-319-12436-0_7
Download citation
DOI: https://doi.org/10.1007/978-3-319-12436-0_7
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-12435-3
Online ISBN: 978-3-319-12436-0
eBook Packages: Computer ScienceComputer Science (R0)