Stochastic Nash Equilibrium Seeking for Games with General Nonlinear Payoffs

  • Shu-Jun Liu
  • Miroslav KrsticEmail author
Part of the Communications and Control Engineering book series (CCE)


Extremum seeking is extended from standard multi-input optimization problems to multi-player non-cooperative games. The players do not have knowledge of the payoff functions and only measure their own payoff values. The players are also unaware of the other players’ actions. Extremum seeking is shown to achieve convergence to the Nash equilibrium of the underlying static game.


Nash Equilibrium Payoff Function Invariant Distribution Noncooperative Game Nash Game 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 4.
    Altman E, Başar T, Srikant R (2002) Nash equilibria for combined flow control and routing in networks: asymptotic behavior for a large number of users. IEEE Trans Autom Control 47:917–930 CrossRefGoogle Scholar
  2. 10.
    Başar T (2007) Control and game-theoretic tools for communication networks (overview). Appl Comput Math 6:104–125 MathSciNetzbMATHGoogle Scholar
  3. 11.
    Başar T, Olsder GJ (1999) Dynamic noncooperative game theory, 2nd edn. SIAM, Philadelphia zbMATHGoogle Scholar
  4. 12.
    Bauso D, Giarre L, Pesenti R (2008) Consensus in noncooperative dynamic games: a multiretailer inventory application. IEEE Trans Autom Control 53:998–1003 MathSciNetCrossRefGoogle Scholar
  5. 26.
    Cesa-Bianchi N, Lugosi G (2006) Prediction, learning, and games. Cambridge University Press, New York zbMATHCrossRefGoogle Scholar
  6. 37.
    Foster DP, Young HP (2003) Learning, hypothesis testing, and Nash equilibrium. Games Econ Behav 45:73–96 MathSciNetzbMATHCrossRefGoogle Scholar
  7. 38.
    Foster DP, Young HP (2006) Regret testing: learning to play Nash equilibrium without knowing you have an opponent. Theor Econ 1:341–367 Google Scholar
  8. 41.
    Frihauf P, Krstic M, Başar T (2012) Nash equilibrium seeking in noncooperative games. IEEE Trans Autom Control 57:1192–1207 CrossRefGoogle Scholar
  9. 42.
    Fudenberg D, Levine DK (1998) The theory of learning in games. MIT Press, Cambridge zbMATHGoogle Scholar
  10. 54.
    Jafari A, Greenwald A, Gondek D, Ercal G (2001) On no-regret learning, fictitious play, and Nash equilibrium. In: Proceedings of the 18th international conference on machine learning Google Scholar
  11. 72.
    Krstic M, Frihauf P, Krieger J, Başar T (2010) Nash eqilibrium seeking with finitely- and infinitely-many players. In: Proceedings of the 8th IFAC symposium on nonlinear control systems, Bologna, Italy Google Scholar
  12. 74.
    Kubica BJ, Wozniak A (2010) An interval method for seeking the Nash equilibrium of non-cooperative games. In: Parallel processing and applied mathematics. Lecture notes in computer science, vol. 6068. Springer, Berlin, pp 446–455 CrossRefGoogle Scholar
  13. 83.
    Li S, Başar T (1987) Distributed algorithms for the computation of noncooperative equilibria. Automatica 23:523–533 zbMATHCrossRefGoogle Scholar
  14. 89.
    Liu S-J, Krstic M (2010) Stochastic averaging in continuous time and its applications to extremum seeking. IEEE Trans Autom Control 55(10):2235–2250 MathSciNetCrossRefGoogle Scholar
  15. 91.
    Liu S-J, Krstic M (2011) Stochastic Nash equilibrium seeking for games with general nonlinear payoffs. SIAM J Control Optim 49:1659–1679 MathSciNetzbMATHCrossRefGoogle Scholar
  16. 97.
    MacKenzie AB, Wicker SB (2001) Game theory and the design of self-configuring, adaptive wireless networks. IEEE Commun Mag 39:126–131 CrossRefGoogle Scholar
  17. 99.
    Marden JR, Arslon G, Shamma JS (2009) Cooperative control and potential games. IEEE Trans Syst Man Cybern, Part B, Cybern 39:1393–1407 CrossRefGoogle Scholar
  18. 114.
    Rao SS, Venkayya VB, Khot NS (1988) Game theory approach for the integrated design of structures and controls. AIAA J 26:463–469 MathSciNetzbMATHCrossRefGoogle Scholar
  19. 118.
    Rosen JB (1965) Existence and uniqueness of equilibrium points for concave N-person games. Econometrica 33:520–534 MathSciNetzbMATHCrossRefGoogle Scholar
  20. 124.
    Scutari G, Palomar DP, Barbarossa S (2009) The MIMO iterative waterfilling algorithm. IEEE Trans Signal Process 57:1917–1935 CrossRefGoogle Scholar
  21. 125.
    Semsar-Kazerooni E, Khorasani K (2009) Multi-agent team cooperation: a game theory approach. Automatica 45:2205–2213 MathSciNetzbMATHCrossRefGoogle Scholar
  22. 126.
    Shamma JS, Arslan G (2005) Dynamic fictitious play, dynamic gradient play, and distributed convergence to Nash equilibria. IEEE Trans Autom Control 53(3):312–327 MathSciNetCrossRefGoogle Scholar
  23. 127.
    Sharma R, Gopal M (2010) Synergizing reinforcement learning and game theory—a new direction for control. Appl Soft Comput 10(3):675–688 CrossRefGoogle Scholar
  24. 136.
    Stanković MS, Johansson KH, Stipanović DM (2010) Distributed seeking of Nash equilibrium in mobile sensor networks. In: Proceedings of IEEE conference on decision and control, Atlanta, GA, USA, December 15–17, pp 5598–5603 Google Scholar
  25. 137.
    Tan Y, Nešić D, Mareels IMY (2006) On non-local stability properties of extremum seeking controllers. Automatica 42:889–903 zbMATHCrossRefGoogle Scholar
  26. 150.
    Zhu M, Martinez S (2010) Distributed coverage games for mobile visual sensor networks. SIAM J Control Optim (submitted). Available at

Copyright information

© Springer-Verlag London 2012

Authors and Affiliations

  1. 1.Department of MathematicsSoutheast UniversityNanjingPeople’s Republic of China
  2. 2.Department Mechanical & Aerospace EngineeringUniversity of California, San DiegoLa JollaUSA

Personalised recommendations