Zero-Sum Games for Discrete-Time Systems Based on Model-Free ADP

  • Huaguang Zhang
  • Derong Liu
  • Yanhong Luo
  • Ding Wang
Part of the Communications and Control Engineering book series (CCE)


In this chapter, zero-sum games are investigated for discrete-time systems based on the model-free ADP method. First, an effective data-based optimal control scheme is developed via the iterative ADP algorithm to find the optimal controller of a class of discrete-time zero-sum games for Roesser type 2-D systems. Since the exact models of many 2-D systems cannot be obtained inherently, the iterative ADP method is expected to avoid the requirement of exact system models. Second, a data-based optimal output feedback controller is developed for solving the zero-sum games of a class of discrete-time systems, whose merit is that not only knowledge of the system model is not required, but neither is information of the system states. Theoretical analysis and a simulation study show the validity of the methods presented.


  1. 1.
    Aangenent W, Kostic D, de Jager B, Van de Molengraft R, Steinbuch M (2005) Data-based optimal control. In: Proceedings of American control conference, Portland, pp 1460–1465 CrossRefGoogle Scholar
  2. 2.
    Abu-Khalaf M, Lewis FL (2008) Neurodynamic programming and zero-sum games for constrained control systems. IEEE Trans Neural Netw 19:1243–1252 CrossRefGoogle Scholar
  3. 3.
    Abu-Khalaf M, Lewis FL, Huang J (2006) Policy iterations on the Hamilton–Jacobi–Isaacs equation for H state feedback control with input saturation. IEEE Trans Autom Control 51:1989–1995 MathSciNetCrossRefGoogle Scholar
  4. 4.
    Al-Tamimi A, Abu-Khalaf M, Lewis FL (2007) Adaptive critic designs for discrete-time zero-sum games with application to H control. IEEE Trans Syst Man Cybern, Part B, Cybern 37:240–247 CrossRefGoogle Scholar
  5. 5.
    Al-Tamimi A, Lewis FL, Abu-Khalaf M (2007) Model-free q-learning designs for linear discrete-time zero-sum games with application to H control. Automatica 43:473–481 MathSciNetzbMATHCrossRefGoogle Scholar
  6. 6.
    Al-Tamimi A, Lewis FL, Abu-Khalaf M (2007) Model-free Q-learning designs for linear discrete-time zero-sum games with application to H-infinity control. Automatica 43:473–481 MathSciNetzbMATHCrossRefGoogle Scholar
  7. 7.
    Basar T, Bernhard P (1995) H optimal control and related minimax design problems. Birkhauser, Basel zbMATHCrossRefGoogle Scholar
  8. 8.
    Basar T, Olsder GJ (1982) Dynamic noncooperative game theory. Academic Press, New York zbMATHGoogle Scholar
  9. 9.
    Bertsekas DP (2003) Convex analysis and optimization. Athena Scientific, Boston zbMATHGoogle Scholar
  10. 10.
    Cui LL, Zhang HG, Zhang X, Luo YH (2011) Adaptive critic design based output feedback control for discrete-time zero-sum games. In: Proceedings of IEEE symposium on adaptive dynamic programming and reinforcement learning, France, pp 190–195 CrossRefGoogle Scholar
  11. 11.
    Hua X, Mizukami K (1994) Linear-quadratic zero-sum differential games for generalized state space systems. IEEE Trans Autom Control 39:143–147 zbMATHCrossRefGoogle Scholar
  12. 12.
    Li CJ, Fadali MS (1991) Optimal control of 2-D systems. IEEE Trans Autom Control 36:223–228 MathSciNetzbMATHCrossRefGoogle Scholar
  13. 13.
    Luenberger DG (1969) Optimization by vector space methods. Wiley, New York zbMATHGoogle Scholar
  14. 14.
    Tsai JS, Li JS, Shieh LS (2002) Discretized quadratic optimal control for continuous-time two-dimensional systems. IEEE Trans Circuits Syst I, Fundam Theory Appl 49:116–125 MathSciNetCrossRefGoogle Scholar
  15. 15.
    Uetake Y (1992) Optimal smoothing for noncausal 2-D systems based on a descriptor model. IEEE Trans Autom Control 37:1840–1845 MathSciNetzbMATHCrossRefGoogle Scholar
  16. 16.
    Wei QL, Zhang HG, Cui LL (2009) Data-based optimal control for discrete-time zero-sum games of 2-D systems using adaptive critic designs. Acta Autom Sin 35:682–692 MathSciNetzbMATHCrossRefGoogle Scholar

Copyright information

© Springer-Verlag London 2013

Authors and Affiliations

  • Huaguang Zhang
    • 1
  • Derong Liu
    • 2
  • Yanhong Luo
    • 1
  • Ding Wang
    • 2
  1. 1.College of Information Science Engin.Northeastern UniversityShenyangPeople’s Republic of China
  2. 2.Institute of Automation, Laboratory of Complex SystemsChinese Academy of SciencesBeijingPeople’s Republic of China

Personalised recommendations