Abstract
In this chapter, the optimal state feedback control problem is studied based on ADP for both infinite horizon and finite horizon. Three different structures of ADP are utilized to solve the optimal state feedback control strategies, respectively. First, considering a class of affine constrained systems, a new DHP method is developed to stabilize the system with convergence proof. Then, due to the special advantages of GDHP structure, a new optimal control scheme is developed with discounted cost functional. Moreover, based on a least-square successive approximation method, a series of GHJB equations are solved to obtain the optimal control solutions. Finally, a novel finite-horizon optimal control scheme is developed to obtain the suboptimal control solutions within a fixed finite number of control steps. Compared with the existing results in the infinite-horizon case, the present finite-horizon optimal controller is preferred in real-world applications.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Al-Tamimi A, Lewis FL (2007) Discrete-time nonlinear HJB solution using approximate dynamic programming: convergence proof. In: Proceedings of IEEE international symposium on approximate dynamic programming and reinforcement learning, Honolulu, HI, pp 38–43
Bagnell J, Kakade S, Ng A, Schneider J (2003) Policy search by dynamic programming. In: Proceedings of 17th annual conference on neural information processing systems, Vancouver, Canada, vol 16, pp 831–838
Bryson AE, Ho YC (1975) Applied optimal control: optimization, estimation, and control. Hemisphere–Wiley, New York
Burk F (1998) Lebesgue measure and integration. Wiley, New York
Chen Z, Jagannathan S (2008) Generalized Hamilton–Jacobi–Bellman formulation-based neural network control of affine nonlinear discrete-time systems. IEEE Trans Neural Netw 19(1):90–106
Cui LL, Zhang HG, Liu D, Kim YS (2009) Constrained optimal control of affine nonlinear discrete-time systems using GHJB method. In: Proceedings of IEEE international symposium on adaptive dynamic programming and reinforcement learning, Nashville, USA, pp 16–21
Han D, Balakrishnan SN (2002) State-constrained agile missile control with adaptive-critic-based neural networks. IEEE Trans Control Syst Technol 10(4):481–489
Haykin S (1999) Neural networks: a comprehensive foundation. Prentice Hall, Upper Saddle River
Jin N, Liu D, Huang T, Pang Z (2007) Discrete-time adaptive dynamic programming using wavelet basis function neural networks. In: Proceedings of the IEEE symposium on approximate dynamic programming and reinforcement learning, Honolulu, HI, pp 135–142
Liu D, Wang D, Zhao D, Wei Q, Jin N (2012) Neural-network-based optimal control for a class of unknown discrete-time nonlinear systems using globalized dual heuristic programming. IEEE Trans Autom Sci Eng 9(3):628–634
Plumer ES (1996) Optimal control of terminal processes using neural networks. IEEE Trans Neural Netw 7(2):408–418
Si J, Wang YT (2001) On-line learning control by association and reinforcement. IEEE Trans Neural Netw 12(2):264–276
Wang FY, Zhang HG, Liu D (2009) Adaptive dynamic programming: an introduction. IEEE Comput Intell Mag 4(2):39–47
Wang FY, Jin N, Liu D, Wei Q (2011) Adaptive dynamic programming for finite-horizon optimal control of discrete-time nonlinear systems with ε-error bound. IEEE Trans Neural Netw 22(1):24–36
Zhang HG, Xie X (2011) Relaxed stability conditions for continuous-time T-S fuzzy-control systems via augmented multi-indexed matrix approach. IEEE Trans Fuzzy Syst 19(3):478–492
Zhang HG, Wei QL, Luo YH (2008) A novel infinite-time optimal tracking control scheme for a class of discrete-time nonlinear systems via the greedy HDP iteration algorithm. IEEE Trans Syst Man Cybern, Part B, Cybern 38(4):937–942
Zhang HG, Luo YH, Liu D (2009) Neural-network-based near-optimal control for a class of discrete-time affine nonlinear systems with control constraints. IEEE Trans Neural Netw 20(9):1490–1503
Author information
Authors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer-Verlag London
About this chapter
Cite this chapter
Zhang, H., Liu, D., Luo, Y., Wang, D. (2013). Optimal State Feedback Control for Discrete-Time Systems. In: Adaptive Dynamic Programming for Control. Communications and Control Engineering. Springer, London. https://doi.org/10.1007/978-1-4471-4757-2_2
Download citation
DOI: https://doi.org/10.1007/978-1-4471-4757-2_2
Publisher Name: Springer, London
Print ISBN: 978-1-4471-4756-5
Online ISBN: 978-1-4471-4757-2
eBook Packages: EngineeringEngineering (R0)