Optimal State Feedback Control for Discrete-Time Systems

  • Huaguang Zhang
  • Derong Liu
  • Yanhong Luo
  • Ding Wang
Part of the Communications and Control Engineering book series (CCE)


In this chapter, the optimal state feedback control problem is studied based on ADP for both infinite horizon and finite horizon. Three different structures of ADP are utilized to solve the optimal state feedback control strategies, respectively. First, considering a class of affine constrained systems, a new DHP method is developed to stabilize the system with convergence proof. Then, due to the special advantages of GDHP structure, a new optimal control scheme is developed with discounted cost functional. Moreover, based on a least-square successive approximation method, a series of GHJB equations are solved to obtain the optimal control solutions. Finally, a novel finite-horizon optimal control scheme is developed to obtain the suboptimal control solutions within a fixed finite number of control steps. Compared with the existing results in the infinite-horizon case, the present finite-horizon optimal controller is preferred in real-world applications.


  1. 1.
    Al-Tamimi A, Lewis FL (2007) Discrete-time nonlinear HJB solution using approximate dynamic programming: convergence proof. In: Proceedings of IEEE international symposium on approximate dynamic programming and reinforcement learning, Honolulu, HI, pp 38–43 CrossRefGoogle Scholar
  2. 2.
    Bagnell J, Kakade S, Ng A, Schneider J (2003) Policy search by dynamic programming. In: Proceedings of 17th annual conference on neural information processing systems, Vancouver, Canada, vol 16, pp 831–838 Google Scholar
  3. 3.
    Bryson AE, Ho YC (1975) Applied optimal control: optimization, estimation, and control. Hemisphere–Wiley, New York Google Scholar
  4. 4.
    Burk F (1998) Lebesgue measure and integration. Wiley, New York zbMATHGoogle Scholar
  5. 5.
    Chen Z, Jagannathan S (2008) Generalized Hamilton–Jacobi–Bellman formulation-based neural network control of affine nonlinear discrete-time systems. IEEE Trans Neural Netw 19(1):90–106 CrossRefGoogle Scholar
  6. 6.
    Cui LL, Zhang HG, Liu D, Kim YS (2009) Constrained optimal control of affine nonlinear discrete-time systems using GHJB method. In: Proceedings of IEEE international symposium on adaptive dynamic programming and reinforcement learning, Nashville, USA, pp 16–21 Google Scholar
  7. 7.
    Han D, Balakrishnan SN (2002) State-constrained agile missile control with adaptive-critic-based neural networks. IEEE Trans Control Syst Technol 10(4):481–489 CrossRefGoogle Scholar
  8. 8.
    Haykin S (1999) Neural networks: a comprehensive foundation. Prentice Hall, Upper Saddle River zbMATHGoogle Scholar
  9. 9.
    Jin N, Liu D, Huang T, Pang Z (2007) Discrete-time adaptive dynamic programming using wavelet basis function neural networks. In: Proceedings of the IEEE symposium on approximate dynamic programming and reinforcement learning, Honolulu, HI, pp 135–142 CrossRefGoogle Scholar
  10. 10.
    Liu D, Wang D, Zhao D, Wei Q, Jin N (2012) Neural-network-based optimal control for a class of unknown discrete-time nonlinear systems using globalized dual heuristic programming. IEEE Trans Autom Sci Eng 9(3):628–634 CrossRefGoogle Scholar
  11. 11.
    Plumer ES (1996) Optimal control of terminal processes using neural networks. IEEE Trans Neural Netw 7(2):408–418 CrossRefGoogle Scholar
  12. 12.
    Si J, Wang YT (2001) On-line learning control by association and reinforcement. IEEE Trans Neural Netw 12(2):264–276 MathSciNetCrossRefGoogle Scholar
  13. 13.
    Wang FY, Zhang HG, Liu D (2009) Adaptive dynamic programming: an introduction. IEEE Comput Intell Mag 4(2):39–47 CrossRefGoogle Scholar
  14. 14.
    Wang FY, Jin N, Liu D, Wei Q (2011) Adaptive dynamic programming for finite-horizon optimal control of discrete-time nonlinear systems with ε-error bound. IEEE Trans Neural Netw 22(1):24–36 CrossRefGoogle Scholar
  15. 15.
    Zhang HG, Xie X (2011) Relaxed stability conditions for continuous-time T-S fuzzy-control systems via augmented multi-indexed matrix approach. IEEE Trans Fuzzy Syst 19(3):478–492 CrossRefGoogle Scholar
  16. 16.
    Zhang HG, Wei QL, Luo YH (2008) A novel infinite-time optimal tracking control scheme for a class of discrete-time nonlinear systems via the greedy HDP iteration algorithm. IEEE Trans Syst Man Cybern, Part B, Cybern 38(4):937–942 CrossRefGoogle Scholar
  17. 17.
    Zhang HG, Luo YH, Liu D (2009) Neural-network-based near-optimal control for a class of discrete-time affine nonlinear systems with control constraints. IEEE Trans Neural Netw 20(9):1490–1503 CrossRefGoogle Scholar

Copyright information

© Springer-Verlag London 2013

Authors and Affiliations

  • Huaguang Zhang
    • 1
  • Derong Liu
    • 2
  • Yanhong Luo
    • 1
  • Ding Wang
    • 2
  1. 1.College of Information Science Engin.Northeastern UniversityShenyangPeople’s Republic of China
  2. 2.Institute of Automation, Laboratory of Complex SystemsChinese Academy of SciencesBeijingPeople’s Republic of China

Personalised recommendations