Several Special Optimal Feedback Control Designs Based on ADP

  • Huaguang Zhang
  • Derong Liu
  • Yanhong Luo
  • Ding Wang
Part of the Communications and Control Engineering book series (CCE)


In this chapter, several special optimal feedback control schemes will be investigated. In the first part, the optimal feedback control problem of affine nonlinear switched systems is studied. To seek optimal solutions, a novel two-stage adaptive dynamic programming (TSADP) method is developed. The algorithm can be divided into two stages: first, for each possible mode, calculate the associated value function, and then select the optimal mode for each state. In the second and third parts, the near-optimal controllers for nonlinear descriptor systems and singularly perturbed systems are solved by iterative DHP and HDP algorithms, respectively. In the fourth part, the near-optimal state-feedback control problem of nonlinear constrained discrete-time systems is solved via a single network ADP algorithm. At each step of the iterative algorithm, a neural network is utilized to approximate the costate function, and then the optimal control policy of the system can be computed directly according to the costate function, which removes the action network appearing in the ordinary ADP method.


  1. 1.
    Al-Tamimi A, Lewis F, Abu-Khalaf M (2008) Discrete-time nonlinear HJB solution using approximate dynamic programming: convergence proof. IEEE Trans Syst Man Cybern, Part B, Cybern 38:943–949 CrossRefGoogle Scholar
  2. 2.
    Beard R (1995) Improving the closed-loop performance of nonlinear systems. PhD dissertation, Rensselaer Polytechnic Institute, Troy, NY Google Scholar
  3. 3.
    Cao N, Zhang HG, Luo YH, Feng DZ, Liu Y (2011) Suboptimal control of a class of nonlinear singularly perturbed systems. Control Theory Appl 28(5):688–692 Google Scholar
  4. 4.
    Cao N, Zhang HG, Luo YH, Feng DZ (2012) Infinite horizon optimal control of affine nonlinear discrete switched systems using two-stage approximate dynamic programming. Int J Syst Sci 43(9):1673–1682 CrossRefGoogle Scholar
  5. 5.
    Lincoln B, Rantzer A (2006) Relaxing dynamic programming. IEEE Trans Autom Control 51:1249–1260 MathSciNetCrossRefGoogle Scholar
  6. 6.
    Luo YH, Zhang HG, Cao N, Chen B (2009) Near-optimal stabilization for a class of nonlinear systems with control constraint based on single network greedy iterative DHP algorithm. Acta Autom Sin 35(11):1436–1445 MathSciNetzbMATHGoogle Scholar
  7. 7.
    Luo YH, Liu Z, Yang D (2010) Greedy iterative DHP algorithm-based near-optimal control for a class of nonlinear descriptor systems with actuator saturating. In: Proceedings of the 9th IEEE international conference on cognitive informatics, pp 788–793 CrossRefGoogle Scholar
  8. 8.
    Lyshevski SE (1998) Nonlinear discrete-time systems: constrained optimization and application of nonquadratic costs. In: Proceedings of the American control conference, Philadelphia, USA, pp 3699–3703 Google Scholar
  9. 9.
    Padhi R, Unnikrishnan N, Wang X, Balakrishnan SN (2006) A single network adaptive critic (SNAC) architecture for optimal control synthesis for a class of nonlinear systems. Neural Netw 19(10):1648–1660 zbMATHCrossRefGoogle Scholar
  10. 10.
    Rantzer A (2005) On approximate dynamic programming in switching systems. In: Proceeding of the IEEE conference on decision and control and the European control conference, Seville, Spain, pp 1391–1396 CrossRefGoogle Scholar
  11. 11.
    Seatzu C, Corona D, Giua A, Bempoard A (2006) Optimal control of continuous time switched affine systems. IEEE Trans Autom Control 51:726–741 CrossRefGoogle Scholar
  12. 12.
    Xu XP, Antsaklis PJ (2000) Optimal control of switched systems: new results and open problems. In: Proceeding of the American control conference, Chicago, Illinois, pp 2683–2687 Google Scholar
  13. 13.
    Xu XP, Antsaklis PJ (2003) Results and perspectives on computational methods for optimal control of switched systems. Hybrid systems: computation and control (HSCC). Springer, Berlin, pp 540–555 Google Scholar
  14. 14.
    Yang H, Jiang B, Cocquempot V, Zhang HG (2011) Stabilization of switched nonlinear systems with all unstable modes: application to multi-agent systems. IEEE Trans Autom Control 56(9):2230–2235 MathSciNetCrossRefGoogle Scholar
  15. 15.
    Zhang W, Hu J, Abate A (2009) On the value functions of the discrete-time switched LQR problem. IEEE Trans Autom Control 54:2669–2674 MathSciNetCrossRefGoogle Scholar
  16. 16.
    Zhang HG, Liu Z, Huang GB (2010) Novel delay-dependent robust stability analysis for switched neutral-type neural network with time-varying delays via SC technique. IEEE Trans Syst Man Cybern, Part B, Cybern 40(6):1480–1491 CrossRefGoogle Scholar

Copyright information

© Springer-Verlag London 2013

Authors and Affiliations

  • Huaguang Zhang
    • 1
  • Derong Liu
    • 2
  • Yanhong Luo
    • 1
  • Ding Wang
    • 2
  1. 1.College of Information Science Engin.Northeastern UniversityShenyangPeople’s Republic of China
  2. 2.Institute of Automation, Laboratory of Complex SystemsChinese Academy of SciencesBeijingPeople’s Republic of China

Personalised recommendations