Dynamic Games and Applications

, Volume 3, Issue 2, pp 172–186 | Cite as

A Zero-Sum Stochastic Game with Compact Action Sets and no Asymptotic Value

  • Guillaume VigeralEmail author


We give an example of a zero-sum stochastic game with four states, compact action sets for each player, and continuous payoff and transition functions, such that the discounted value does not converge as the discount factor tends to 0, and the value of the n-stage game does not converge as n goes to infinity.


Zero sum stochastic games Asymptotic behavior Uniform value Compact action sets 



This research was supported by grant ANR-10-BLAN 0112 (France).

This paper owes a lot to Sylvain Sorin. I pleasantly remember countless discussions about compact games and why they should have an asymptotic value or not, as well as devising with him a number of “almost proofs” of convergence. This was decisive to understand the right direction to go to stumble upon this counterexample.

I also would like to thank Jérome Bolte for being the first to warn me about non-semialgebraic functions, Jérôme Renault for raising several interesting questions while I was writing this paper, as well as Andrzej S. Nowak for useful references.


  1. 1.
    Aumann RJ, Maschler M, with the collaboration Stearns RE (1995) Repeated games with incomplete information. MIT Press, Cambridge zbMATHGoogle Scholar
  2. 2.
    Bather J (1973) Optimal decision procedures for finite Markov chains. Part I: Examples. Adv Appl Probab 5:328–339 MathSciNetzbMATHCrossRefGoogle Scholar
  3. 3.
    Bather J (1973) Optimal decision procedures for finite Markov chains. Part II: Communicating systems. Adv Appl Probab 5:521–540 MathSciNetzbMATHCrossRefGoogle Scholar
  4. 4.
    Bather J (1973) Optimal decision procedures for finite Markov chains. Part III: General convex systems. Adv Appl Probab 5:541–553 MathSciNetzbMATHCrossRefGoogle Scholar
  5. 5.
    Bewley T, Kohlberg E (1976) The asymptotic theory of stochastic games. Math Oper Res 1:197–208 MathSciNetzbMATHCrossRefGoogle Scholar
  6. 6.
    Bewley T, Kohlberg E (1976) The asymptotic solution of a recursion equation occurring in stochastic games. Math Oper Res 1:321–336 MathSciNetzbMATHCrossRefGoogle Scholar
  7. 7.
    Bewley T, Kohlberg E (1978) On stochastic games with stationary optimal strategies. Math Oper Res 3:104–125 MathSciNetzbMATHCrossRefGoogle Scholar
  8. 8.
    Bolte J, Gaubert S, Vigeral G (2012) Definable zero-sum stochastic games. Preprint Google Scholar
  9. 9.
    Cardaliaguet P, Laraki R, Sorin S (2012) A continuous time approach for the asymptotic value in two-person zero-sum repeated games. SIAM J Control Optim 50:1573–1596 MathSciNetzbMATHCrossRefGoogle Scholar
  10. 10.
    Dynkin E, Yushkevich A (1979) Controlled Markov processes. Controlled Markov processes. Springer, Berlin CrossRefGoogle Scholar
  11. 11.
    Everett H (1957) Recursive games. In: Kuhn HW, Tucker AW (eds) Contributions to the theory of games, III. Annals of mathematical studies, vol 39. Princeton University Press, Princeton, pp 47–78 Google Scholar
  12. 12.
    Kohlberg E (1974) Repeated games with absorbing states. Ann Stat 2:724–738 MathSciNetzbMATHCrossRefGoogle Scholar
  13. 13.
    Kohlberg E, Neyman A (1981) Asymptotic behavior of nonexpansive mappings in normed linear spaces. Isr J Math 38:269–275 MathSciNetzbMATHCrossRefGoogle Scholar
  14. 14.
    Maitra A, Parthasarathy T (1970) On stochastic games. J Optim Theory Appl 5:289–300 MathSciNetzbMATHCrossRefGoogle Scholar
  15. 15.
    Mertens J-F, Zamir S (1971) The value of two player zero sum repeated games with lack of information on both sides. Int J Game Theory 1:39–64 MathSciNetzbMATHCrossRefGoogle Scholar
  16. 16.
    Mertens J-F, Neyman A, Rosenberg D (2009) Absorbing games with compact action spaces. Math Oper Res 34:257–262 MathSciNetzbMATHCrossRefGoogle Scholar
  17. 17.
    von Neumann J (1928) Zur Theorie der Gesellschaftsspiele. Math Ann 100:295–320 MathSciNetzbMATHCrossRefGoogle Scholar
  18. 18.
    Neyman A (2003) Stochastic games and nonexpansive maps. In: Neyman A, Sorin S (eds) Stochastic games and applications. Kluwer Academic, Dordrecht CrossRefGoogle Scholar
  19. 19.
    Oliu-Barton M (2012) The asymptotic value in finite stochastic games. Preprint Google Scholar
  20. 20.
    Renault J (2006) The value of Markov chain games with lack of information on one side. Math Oper Res 31:490–512 MathSciNetzbMATHCrossRefGoogle Scholar
  21. 21.
    Renault J (2011) Uniform value in dynamic programming. J Eur Math Soc 13:309–330 MathSciNetzbMATHCrossRefGoogle Scholar
  22. 22.
    Renault J (2012) The value of repeated games with an informed controller. Math Oper Res 37:309–330 MathSciNetCrossRefGoogle Scholar
  23. 23.
    Rosenberg D (2000) Zero-sum absorbing games with incomplete information on one side: asymptotic analysis. SIAM J Control Optim 39:208–225 MathSciNetzbMATHCrossRefGoogle Scholar
  24. 24.
    Rosenberg D, Sorin S (2001) An operator approach to zero-sum repeated games. Isr J Math 121:221–246 MathSciNetzbMATHCrossRefGoogle Scholar
  25. 25.
    Rosenberg D, Vieille N (2000) The maxmin of recursive games with lack of information on one side. Math Oper Res 25:23–35 MathSciNetzbMATHCrossRefGoogle Scholar
  26. 26.
    Shapley LS (1953) Stochastic games. Proc Natl Acad Sci USA 39:1095–1100 MathSciNetzbMATHCrossRefGoogle Scholar
  27. 27.
    Sion M (1958) On general minimax theorems. Pac J Math 8:171–176 MathSciNetzbMATHCrossRefGoogle Scholar
  28. 28.
    Sorin S (2002) A first course on zero-sum repeated games. Springer, Berlin zbMATHGoogle Scholar
  29. 29.
    Sorin S (2003) The operator approach to zero-sum stochastic games. In: Neyman A, Sorin S (eds) Stochastic games and applications. Kluwer Academic, Dordrecht Google Scholar
  30. 30.
    Sorin S (2004) Asymptotic properties of monotonic nonexpansive mappings. Discrete Event Dyn Syst 14:109–122 MathSciNetzbMATHCrossRefGoogle Scholar
  31. 31.
    Sorin S, Vigeral G (2013) Existence of the limit value of two person zero-sum discounted repeated games via comparison theorems. J Optim Theory Appl. doi: 10.1007/s10957-012-0193-4 MathSciNetGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2013

Authors and Affiliations

  1. 1.CEREMADEUniversité Paris-DauphineParis cedex 16France

Personalised recommendations