Abstract
A novel observer-based online control strategy is proposed for a class of uncertain continuous-time nonlinear systems based on solving the HJB equation. Due to the dynamics complexity, the approximate optimal control for affine uncertain continuous-time nonlinear systems is pursued by policy iteration algorithm. Considering that only output variables can be measured in control practice, an observer is designed to reconstruct all system states by relying on output information and then is used to develop the policy iteration control scheme. The observer-based policy iteration algorithm can approximately solve the HJB equation within the ADP framework, where a critic neural network is constructed to approximate the optimal cost function. Then, the approximate expression of the optimal control policy can be directly derived from solving the HJB equation. Additionally, the stability of the closed-loop system is provided based on the Lyapunov theory. Two simulation examples are presented to verify the effectiveness of the proposed control approach.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Abdollahi, F., Talebi, H.A., Patel, R.V.: A stable neural network-based observer with application to flexible-joint manipulators. IEEE Trans. Neural Netw. 17(1), 118–129 (2006)
Abu-Khalaf, M., Lewis, F.L.: Nearly optimal control laws for nonlinear systems with saturating actuators using a neural network HJB approach. Automatica 41(5), 779–791 (2005)
Bellman, R.E.: Dynamic Programming. Princeton University Press, Princeton (1957)
Chen, C., Liu, Z., Zhang, Y., Chen, C.L.P., Xie, S.: Asymptotic fuzzy tracking control for a class of stochastic strict-feedback systems. IEEE Trans. Fuzzy Syst. 25(3), 556–568 (2017)
Chen, C.L.P., Wen, G.X., Liu, Y.J., Liu, Z.: Observer-based adaptive backstepping consensus tracking control for high-order nonlinear semi-strict-feedback multiagent systems. IEEE Trans. Cybern. 46(7), 1591–1601 (2016)
Chen, Z., Li, Z., Chen, C.L.P.: Adaptive neural control of uncertain MIMO nonlinear systems with state and input constraints. IEEE Trans. Neural Netw. Learn. Syst. 28(6), 1318–1330 (2017)
Cui, L., Zhang, H., Chen, B., Zhang, Q.: Asymptotic tracking control scheme for mechanical systems with external disturbances and friction. Neurocomputing 73(7–9), 1293–1302 (2010)
Dierks, T., Jagannathan, S.: Online optimal control of affine nonlinear discrete-time systems with unknown internal dynamics by using time-based policy update. IEEE Trans. Neural Netw. Learn. Syst. 23(7), 1118–1129 (2012)
Hanselmann, T., Noakes, L., Zaknich, A.: Continuous-time adaptive critics. IEEE Trans. Neural Netw. 18(3), 631–647 (2007)
He, H., Ni, Z., Fu, J.: A three-network architecture for on-line learning and optimization based on adaptive dynamic programming. Neurocomputing 78(1), 3–13 (2012)
He, W., Dong, Y., Sun, C.: Adaptive neural impedance control of a robotic manipulator with input saturation. IEEE Trans. Syst. Man Cybern.: Syst. 46(3), 334–344 (2016)
Hornik, K., Stinchcombe, M., White, H.: Universal approximation of an unknown mapping and its derivatives using multilayer feedforward networks. Neural Netw. 3(5), 551–560 (1990)
Jiang, Y., Jiang, Z.P.: Computational adaptive optimal control for continuous-time linear systems with completely unknown dynamics. Automatica 48(10), 2699–2704 (2012)
Jiang, Y., Jiang, Z.P.: Robust adaptive dynamic programming for large-scale systems with an application to multimachine power systems. IEEE Trans. Circuits Syst.-II: Express Briefs 59(10), 693–697 (2012)
Jiang, Y., Jiang, Z.P.: Robust adaptive dynamic programming and feedback stabilization of nonlinear systems. IEEE Trans. Neural Netw. Learn. Syst. 25(5), 882–893 (2014)
Kim, Y.H., Lewis, F.L., Abdallah, C.T.: A dynamic recurrent neural-network-based adaptive observer for a class of nonlinear systems. Automatica 33(8), 1539–1543 (1997)
Lewis, F.L., Syrmos, V.L.: Optimal Control. Wiley, New York (1995)
Lin, W.S., Sheu, J.W.: Optimization of train regulation and energy usage of metro lines using an adaptive-optimal-control algorithm. IEEE Trans. Autom. Sci. Eng. 8(4), 855–864 (2011)
Liu, D., Huang, Y., Wang, D., Wei, Q.: Neural-network-observer-based optimal control for unknown nonlinear systems using adaptive dynamic programming. Int. J. Control 86(9), 1554–1566 (2013)
Liu, Y., Li, J., Tong, S., Chen, C.L.P.: Neural network control-based adaptive learning design for nonlinear systems with full-state constraints. IEEE Trans. Neural Netw. Learn. Syst. 27(7), 1562–1571 (2016)
Lv, Y., Na, J., Yang, Q., Wu, X., Guo, Y.: Online adaptive optimal control for continuous-time nonlinear systems with completely unknown dynamics. Int. J. Control 89(1), 99–112 (2016)
Modares, H., Lewis, F.L., Naghibi-Sistani, M.B.: Adaptive optimal control of unknown constrained-input systems using policy iteration and neural networks. IEEE Trans. Neural Netw. Learn. Syst. 24(10), 1513–1525 (2013)
Mu, C., Ni, Z., Sun, C., He, H.: Air-breathing hypersonic vehicle tracking control based on adaptive dynamic programming. IEEE Trans. Neural Netw. Learn. Syst. 28(3), 584–598 (2017)
Mu, C., Ni, Z., Sun, C., He, H.: Data-driven tracking control with adaptive dynamic programming for a class of continuous-time nonlinear systems. IEEE Trans. Cybern. 47(6), 1460–1470 (2017)
Mu, C.: Observer-based online adaptive control for a class of uncertain continuous-time nonlinear systems. In preparation (2018)
Ni, Z., He, H., Wen, J., Xu, X.: Goal representation heuristic dynamic programming on maze navigation. IEEE Trans. Neural Netw. Learn. Syst. 24(12), 2038–2050 (2013)
Prokhorov, D.V., Santiago, R.A., Wunsch, D.C.: Adaptive critic designs: a case study for neurocontrol. Neural Netw. 8(9), 1367–1372 (1995)
Rudin, W.: Principles of Mathematical Analysis. McGraw-Hill, New York (1976)
Si, J., Wang, Y.T.: On-line learning control by association and reinforcement. IEEE Trans. Neural Netw. 12(2), 264–276 (2001)
Tong, S., Zhang, L., Li, Y.: Observed-based adaptive fuzzy decentralized tracking control for switched uncertain nonlinear large-scale systems with dead zones. IEEE Trans. Syst. Man Cybern.: Syst. 46(1), 37–47 (2016)
Vamvoudakis, K.G., Lewis, F.L.: Online actor-critic algorithm to solve the continuous-time infinite horizon optimal control problem. Automatica 46(5), 878–888 (2010)
Vrabie, D., Lewis, F.L.: Neural network approach to continuous-time direct adaptive optimal control for partially unknown nonlinear systems. Neural Netw. 22(3), 237–246 (2009)
Wang, D., Liu, D., Li, H.: Policy iteration algorithm for online design of robust control for a class of continuous-time nonlinear systems. IEEE Trans. Autom. Sci. Eng. 11(2), 627–632 (2014)
Wang, D., Liu, D., Li, H., Ma, H.: Neural-network-based robust optimal control design for a class of uncertain nonlinear systems via adaptive dynamic programming. Inf. Sci. 282, 167–179 (2014)
Wang, D., Liu, D., Zhang, Q., Zhao, D.: Data-based adaptive critic designs for nonlinear robust optimal control with uncertain dynamics. IEEE Trans. Syst. Man Cybern.: Syst. 46(11), 1544–1555 (2016)
Wang, D., Mu, C., He, H., Liu, D.: Event-driven adaptive robust control of nonlinear systems with uncertainties through NDP strategy. IEEE Trans. Syst. Man Cybern.: Syst. 47(7), 1358–1370 (2017)
Wang, D., Mu, C., Yang, X., Liu, D.: Event-based constrained robust control of affine systems incorporating adaptive critic mechanism. IEEE Trans. Syst. Man Cybern.: Syst. 47(7), 1602–1612 (2017)
Werbos, P.J.: Approximate dynamic programming for real-time control and neural modeling. In: Handbook of Intelligent Control: Neural, Fuzzy, and Adaptive Approaches, pp. 493–526 (1992)
Yang, X., Liu, D., Ma, H., Xu, Y.: Online approximate solution of HJI equation for unknown constrained-input nonlinear continuous-time systems. Inf. Sci. 328, 435–454 (2016)
Zhang, H., Cui, L., Luo, Y.: Near-optimal control for nonzero-sum differential games of continuous-time nonlinear systems using single-network ADP. IEEE Trans. Cybern. 43(1), 206–216 (2013)
Zhang, H., Liu, D., Luo, Y., Wang, D.: Adaptive Dynamic Programming for Control: Algorithms and Stability. Springer, London (2013)
Zhao, D., Zhang, Q., Wang, D., Zhu, Y.: Experience replay for optimal control of nonzero-sum game systems with unknown dynamics. IEEE Trans. Cybern. 46(3), 854–865 (2016)
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
Copyright information
© 2019 Springer Nature Singapore Pte Ltd.
About this chapter
Cite this chapter
Wang, D., Mu, C. (2019). Observer-Based Online Adaptive Regulation for a Class of Uncertain Nonlinear Systems. In: Adaptive Critic Control with Robust Stabilization for Uncertain Nonlinear Systems. Studies in Systems, Decision and Control, vol 167. Springer, Singapore. https://doi.org/10.1007/978-981-13-1253-3_3
Download citation
DOI: https://doi.org/10.1007/978-981-13-1253-3_3
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-13-1252-6
Online ISBN: 978-981-13-1253-3
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)