Nash Equilibrium for Upward-Closed Objectives

  • Krishnendu Chatterjee
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4207)


We study infinite stochastic games played by n-players on a finite graph with goals specified by sets of infinite traces. The games are concurrent (each player simultaneously and independently chooses an action at each round), stochastic (the next state is determined by a probability distribution depending on the current state and the chosen actions), infinite (the game continues for an infinite number of rounds), nonzero-sum (the players’ goals are not necessarily conflicting), and undiscounted. We show that if each player has an upward-closed objective, then there exists an ε-Nash equilibrium in memoryless strategies, for every ε>0; and exact Nash equilibria need not exist. Upward-closure of an objective means that if a set Z of infinitely repeating states is winning, then all supersets of Z of infinitely repeating states are also winning. Memoryless strategies are strategies that are independent of history of plays and depend only on the current state. We also study the complexity of finding values (payoff profile) of an ε-Nash equilibrium. We show that the values of an ε-Nash equilibrium in nonzero-sum concurrent games with upward-closed objectives for all players can be computed by computing ε-Nash equilibrium values of nonzero-sum concurrent games with reachability objectives for all players and a polynomial procedure. As a consequence we establish that values of an ε-Nash equilibrium can be computed in TFNP (total functional NP), and hence in EXPTIME.


Markov Chain Nash Equilibrium Markov Decision Process Stochastic Game Game Graph 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Chatterjee, K.: Two-player nonzero-sum ω-regular games. In: Abadi, M., de Alfaro, L. (eds.) CONCUR 2005. LNCS, vol. 3653, pp. 413–427. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  2. 2.
    Chatterjee, K., Majumdar, R., Jurdziński, M.: On Nash equilibria in stochastic games. In: Marcinkowski, J., Tarlecki, A. (eds.) CSL 2004. LNCS, vol. 3210, pp. 26–40. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  3. 3.
    Courcoubetis, C., Yannakakis, M.: The complexity of probabilistic verification. JACM 42(4), 857–907 (1995)MATHCrossRefMathSciNetGoogle Scholar
  4. 4.
    de Alfaro, L.: Formal Verification of Probabilistic Systems. PhD thesis, Stanford University (1997)Google Scholar
  5. 5.
    de Alfaro, L.: Computing minimum and maximum reachability times in probabilistic systems. In: Baeten, J.C.M., Mauw, S. (eds.) CONCUR 1999. LNCS, vol. 1664, pp. 66–81. Springer, Heidelberg (1999)CrossRefGoogle Scholar
  6. 6.
    Filar, J., Vrieze, K.: Competitive Markov Decision Processes. Springer, Heidelberg (1997)MATHGoogle Scholar
  7. 7.
    Fink, A.M.: Equilibrium in a stochastic n-person game. Journal of Science of Hiroshima University 28, 89–93 (1964)MATHMathSciNetGoogle Scholar
  8. 8.
    Nash Jr., J.F.: Equilibrium points in n-person games. Proceedings of the National Academny of Sciences USA 36, 48–49 (1950)MATHCrossRefMathSciNetGoogle Scholar
  9. 9.
    Martin, D.A.: The determinacy of Blackwell games. Journal of Symbolic Logic 63(4), 1565–1581 (1998)MATHCrossRefMathSciNetGoogle Scholar
  10. 10.
    Owen, G.: Game Theory. Academic Press, London (1995)Google Scholar
  11. 11.
    Papadimitriou, C.H.: On the complexity of the parity argument and other inefficient proofs of existence. JCSS 48(3), 498–532 (1994)MATHMathSciNetGoogle Scholar
  12. 12.
    Secchi, P., Sudderth, W.D.: Stay-in-a-set games. International Journal of Game Theory 30, 479–490 (2001)MATHCrossRefMathSciNetGoogle Scholar
  13. 13.
    Shapley, L.S.: Stochastic games. Proc. Nat. Acad. Sci. USA 39, 1095–1100 (1953)MATHCrossRefMathSciNetGoogle Scholar
  14. 14.
    Thomas, W.: Languages, automata, and logic. In: Handbook of Formal Languages, vol. 3, ch. 7, pp. 389–455. Springer, Heidelberg (1997)Google Scholar
  15. 15.
    Vieille, N.: Two player stochastic games I: a reduction. Israel Journal of Mathematics 119, 55–91 (2000)MATHCrossRefMathSciNetGoogle Scholar
  16. 16.
    Vieille, N.: Two player stochastic games II: the case of recursive games. Israel Journal of Mathematics 119, 93–126 (2000)MATHCrossRefMathSciNetGoogle Scholar
  17. 17.
    von Neumann, J., Morgenstern, O.: Theory of games and economic behavior. Princeton University Press, Princeton (1947)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Krishnendu Chatterjee
    • 1
  1. 1.EECSUniversity of CaliforniaBerkeleyUSA

Personalised recommendations