Optimal Feedback Control for Continuous-Time Systems via ADP

  • Huaguang Zhang
  • Derong Liu
  • Yanhong Luo
  • Ding Wang
Part of the Communications and Control Engineering book series (CCE)


In this chapter, we focus on the design of controllers for continuous-time systems via the ADP approach. Although many ADP methods have been proposed for continuous-time systems, a suitable framework in which the optimal controller can be designed for a class of general unknown continuous-time systems still has not been developed. Therefore, in the first part of this chapter, we develop a new scheme to design the optimal robust tracking controllers for unknown general continuous-time nonlinear systems. The merit of the present method is that we require only the availability of input/output data instead of an exact system model. The obtained control input can be guaranteed to be close to the optimal control input within a small bound. In the second part of this chapter, a novel ADP-based robust neural network controller is developed for a class of continuous-time nonaffine nonlinear systems, which is the first attempt to extend the ADP approach to continuous-time nonaffine nonlinear systems. Numerical simulations have shown that the present methods are effective and can be used for a quite wide class of continuous-time nonlinear systems.


  1. 1.
    Al-Tamimi A, Lewis FL, Abu-Khalaf M (2007) Model-free Q-learning designs for linear discrete-time zero-sum games with application to H control. Automatica 43:473–481 MathSciNetzbMATHCrossRefGoogle Scholar
  2. 2.
    Calise AJ, Hovakimyan N, Idan M (2001) Adaptive output feedback control of nonlinear systems using neural networks. Automatica 37:1201–1211 MathSciNetzbMATHCrossRefGoogle Scholar
  3. 3.
    Chellaboina V, Haddad WM (2002) A unification between partial stability and stability theory for time-varying systems. IEEE Control Syst Mag 22:66–75 CrossRefGoogle Scholar
  4. 4.
    Cui LL, Luo YH, Zhang HG (2011) Adaptive critic design based robust neural network control for a class of continuous-time nonaffine nonlinear system. In: Proceedings of international conference on modelling, identification and control, Shanghai, pp 26–29 Google Scholar
  5. 5.
    Ge S, Zhang J (2003) Neural-network control of nonaffine nonlinear system with zero dynamics by state and output feedback. IEEE Trans Neural Netw 14:900–918 CrossRefGoogle Scholar
  6. 6.
    Hanselmann T, Noakes L, Zaknich A (2007) Continuous-time adaptive critics. IEEE Trans Neural Netw 3:631–647 CrossRefGoogle Scholar
  7. 7.
    Khalil HK (2002) Nonlinear system. Prentice Hall, Englewood Cliffs Google Scholar
  8. 8.
    Kim N, Calise AJ (2007) Several extensions in methods for adaptive output feedback control. IEEE Trans Neural Netw 18:482–494 CrossRefGoogle Scholar
  9. 9.
    Kim YH, Lewis FL (2000) Reinforcement adaptive learning neural-net-based friction compensation control for high speed and precision. IEEE Trans Control Syst Technol 8:118–126 CrossRefGoogle Scholar
  10. 10.
    Kuljaca O, Lewis FL (2003) Adaptive critic design using non-linear network structure. Int J Adapt Control Signal Prog 17:431–445 zbMATHCrossRefGoogle Scholar
  11. 11.
    Lewis FL, Jagannathan S, Yesildirek A (1999) Neural network control of robot manipulators and nonlinear systems. Taylor & Francis, London Google Scholar
  12. 12.
    Lin CK (2005) Adaptive critic autopilot design of bank-to-turn missiles using fuzzy basis function networks. IEEE Trans Syst Man Cybern, Part B, Cybern 35:197–206 CrossRefGoogle Scholar
  13. 13.
    Lin CK (2009) H reinforcement learning control of robot manipulators using fuzzy wavelet networks. Fuzzy Sets Syst 160:1765–1786 zbMATHCrossRefGoogle Scholar
  14. 14.
    Liu ZW, Zhang HG, Zhang QL (2010) Novel stability analysis for recurrent neural networks with multiple delays via line integral-type L-K functional. IEEE Trans Neural Netw 21:1710–1718 CrossRefGoogle Scholar
  15. 15.
    Murray JJ, Cox CJ, Lendaris GG, Saeks R (2002) Adaptive dynamic programming. IEEE Trans Syst Man Cybern, Part B, Cybern 32:140–153 CrossRefGoogle Scholar
  16. 16.
    Rubio JDJ, Yu W (2007) Stability analysis of nonlinear system identification via delayed neural networks. IEEE Trans Circuits Syst II, Express Briefs 54:161–195 CrossRefGoogle Scholar
  17. 17.
    Vamvoudakis KG, Lewis FL (2010) Online actor-critic algorithm to solve the continuous-time infinite horizon optimal control problem. Automatica 46:878–888 MathSciNetzbMATHCrossRefGoogle Scholar
  18. 18.
    Vrabie D, Lewis FL (2009) Neural network approach to continuous-time direct adaptive optimal control for partially unknown nonlinear system. Neural Netw 22:237–246 CrossRefGoogle Scholar
  19. 19.
    Vrabie D, Pastravanu O, Abu-Khalaf M, Lewis FL (2009) Adaptive optimal control for continuous-time linear systems based on policy iteration. Automatica 45:477–484 MathSciNetzbMATHCrossRefGoogle Scholar
  20. 20.
    Wang ZS, Zhang HG, Yu W (2009) Robust stability of Cohen–Grossberg neural networks via state transmission matrix. IEEE Trans Neural Netw 20:169–174 CrossRefGoogle Scholar
  21. 21.
    Wang ZS, Zhang HG, Jiang B (2011) LMI-based approach for global asymptotic stability analysis of recurrent neural networks with various delays and structures. IEEE Trans Neural Netw 22:1032–1045 CrossRefGoogle Scholar
  22. 22.
    Zhang HG, Wang ZS, Liu DR (2008) Global asymptotic stability of recurrent neural networks with multiple time-varying delays. IEEE Trans Neural Netw 19:855–873 CrossRefGoogle Scholar
  23. 23.
    Zhang HG, Li M, Yang J (2009) Fuzzy model-based robust networked control for a class of nonlinear systems. IEEE Trans Syst Man Cybern, Part A, Syst Hum 39:437–447 CrossRefGoogle Scholar
  24. 24.
    Zhang HG, Liu ZW, Huang GB (2010) Novel weighting-delay-based stability criteria for recurrent neural networks with time-varying delay. IEEE Trans Neural Netw 21:91–106 CrossRefGoogle Scholar
  25. 25.
    Zhang HG, Cui LL, Zhang X, Luo YH (2011) Data-driven robust approximate optimal tracking control for unknown general nonlinear systems using adaptive dynamic programming method. IEEE Trans Neural Netw 22(12):2226–2236 CrossRefGoogle Scholar
  26. 26.
    Zhang HG, Xie XP, Tong SC (2011) Homogeneous polynomially parameter-dependent H filter designs of discrete-time fuzzy systems. IEEE Trans Syst Man Cybern, Part B, Cybern 41:1313–1322 CrossRefGoogle Scholar

Copyright information

© Springer-Verlag London 2013

Authors and Affiliations

  • Huaguang Zhang
    • 1
  • Derong Liu
    • 2
  • Yanhong Luo
    • 1
  • Ding Wang
    • 2
  1. 1.College of Information Science Engin.Northeastern UniversityShenyangPeople’s Republic of China
  2. 2.Institute of Automation, Laboratory of Complex SystemsChinese Academy of SciencesBeijingPeople’s Republic of China

Personalised recommendations