Stochastic Games with Average Cost Constraints

  • Nahum Shimkin
Part of the Annals of the International Society of Dynamic Games book series (AISDG, volume 1)


The paper introduces the model of undiscounted stochastic games augmented by side constraints on the levels of a player’s average costs. Several variants of this problem are suggested, and the special case of zero-sum games with constraints on one side is analyzed; under certain recurrence conditions, existence of the value and (non-stationary) optimal strategies are established.


Nash Equilibrium Average Cost Stationary Strategy Reward Function Stochastic Game 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. [1]
    E. Altman and A. Shwartz, “Markov decision problems and state-action frequencies,” SIAM J. Control and Optimization 29 No. 4, July 1991.Google Scholar
  2. [2]
    F. J. Beutler and K. W. Ross, “Optimal policies for controlled Markov chain with a constraint,” J. Math. Anal. Appl. 12, pp. 236–256, 1985.MathSciNetCrossRefGoogle Scholar
  3. [3]
    T. Bewly and E. Kohlberg, “On stochastic games with stationary optimal strategies,” Mathematics of Operations Research 3, pp. 104–125, 1978.MathSciNetCrossRefGoogle Scholar
  4. [4]
    D. Blackwell, “An analogue for the minimax theorem for vector payoffs,” Pacific J. Math. 6, pp. 1–8, 1956.MathSciNetMATHGoogle Scholar
  5. [5]
    V. S. Borkar, “Controlled Markov chains with constraints,” Proceedings of the Workshop on Recent Advances in Modeling and Control of Stochastic Systems, Bangalore, Jan. 1991.Google Scholar
  6. [6]
    G. Debreu, “A social equilibrium existence theorem,” Proceedings of the National Academy of Science 38, pp. 886–893, 1952.MathSciNetMATHCrossRefGoogle Scholar
  7. [7]
    C. Derman, Finite State Markovian Decision Processes, Academic Press, New-York, 1970.MATHGoogle Scholar
  8. [8]
    A. Federgruen, “On n-person stochastic games with denumerable state space,” Adv. Appl. Prob. 10, pp. 452–471, 1978.MathSciNetMATHCrossRefGoogle Scholar
  9. [9]
    A. Hordijk and L. C. M. Kallenberg, “Constrained undiscounted stochastic dynamic programming,” Mathematics of Operations Research 9, pp. 276–289, 1984.MathSciNetMATHCrossRefGoogle Scholar
  10. [10]
    D.-J. Ma, A. M. Makowski and A. Shwartz, “Estimation and optimal control for constrained Markov chains,” Proc. 25th IEEE Conf. on Decision and Control, pp. 994–999, 1986.Google Scholar
  11. [11]
    H. Meister, The Purification Problem for Constrained Games with Incomplete Information, (Lecture notes in economics and mathematical systems; 295) Springer, Berlin, 1987.CrossRefGoogle Scholar
  12. [12]
    J. B. Rosen, “Existence and uniqueness of equilibrium points for concave n-person games,” Econometrica 33, pp. 520–534, 1965.MathSciNetMATHCrossRefGoogle Scholar
  13. [13]
    R. W. Rosenthal and A. Rubinstein, “Repeated two-player games with ruin,” International Journal of Game Theory 13, pp. 155–177, 1984.MathSciNetMATHCrossRefGoogle Scholar
  14. [14]
    K. W. Ross and R. Varadarajan, “Markov decision processes with sample path constraints: the communicating case,” Operations Research 37, pp. 780–790, 1989.MathSciNetMATHCrossRefGoogle Scholar
  15. [15]
    N. Shimkin and A. Shwartz, “Guaranteed performance regions in Markovian systems with competing decision makers ,” to appear in IEEE Trans. Automat Contr., Feb. 1993.Google Scholar
  16. [16]
    M. A. Stern, On Stochastic Games with Limiting Average Pay-Off. Ph.D. dissertation, University of Illinois, Circle Campus, Chicago, 1975.Google Scholar

Copyright information

© Springer Science+Business Media New York 1994

Authors and Affiliations

  • Nahum Shimkin
    • 1
  1. 1.Institute for Mathematics and its ApplicationsUniversity of MinnesotaMinneapolisUSA

Personalised recommendations