Denumerable state stochastic games with limiting average payoff

  • V. S. Borkar
  • M. K. Ghosh
Contributed Papers


We study stochastic games with countable state space, compact action spaces, and limiting average payoff. ForN-person games, the existence of an equilibrium in stationary strategies is established under a certain Liapunov stability condition. For two-person zero-sum games, the existence of a value and optimal strategies for both players are established under the same stability condition.

Key Words

Ergodic occupation measure stationary strategies Shapley equation equilibrium 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Gillette, D.,Stochastic Games with Zero Stop Probabilities, Contributions to the Theory of Games, Edited by M. Dresher et al., Princeton University Press, Princeton, New Jersey, pp. 179–188, 1957.Google Scholar
  2. 2.
    Sobel, M. J.,Noncooperative Stochastic Games, Annals of Mathematical Statistics, Vol. 42, pp. 1930–1935, 1971.Google Scholar
  3. 3.
    Bewley, T., andKohlberg, E.,The Asymptotic Theory of Stochastic Games, Mathematics of Operations Research, Vol. 1, pp. 197–208, 1976.Google Scholar
  4. 4.
    Federgruen, A.,On N-Person Stochastic Games with Denumerable State Space, Advances in Applied Probability, Vol. 10, pp. 452–471, 1978.Google Scholar
  5. 5.
    Mertens, J. F., andNeyman, A.,Stochastic Games, International Journal of Game Theory, Vol. 10, pp. 53–66, 1981.Google Scholar
  6. 6.
    Kumar, P. R., andShiau, T. H.,Existence of Value and Randomized Strategies in Zero-Sum Discrete-Time Stochastic Dynamic Games, SIAM Journal on Control and Optimization, Vol. 19, pp. 617–634, 1981.Google Scholar
  7. 7.
    Parthasarathy, T., andSinha, S.,Existence of Stationary Equilibrium Strategies in Nonzero-Sum Discounted Games with Uncountable State Space and State Independent Transitions, International Journal of Game Theory, Vol. 18, pp. 189–194, 1989.Google Scholar
  8. 8.
    Mertens, J. F., andParthasarathy, T.,Equilibria for Discounted Stochastic Games, Discussion Paper 8750, Center for Operations Research and Econometrics, Université Catholique de Louvain, 1989.Google Scholar
  9. 9.
    Blackwell, D., andFerguson, T. S.,The Big Match, Annals of Mathematical Statistics, Vol. 39, pp. 159–163, 1968.Google Scholar
  10. 10.
    Borkar, V. S.,Control of Markov Chains with Long-Run Average Cost Criterion, Proceedings of the IMA Workshop on Stochastic Differential Systems, with Applications to Electrical/Computer Engineering, Control Theory, and Operations Research, Edited by W. Fleming and P. L. Lions, Springer, New York, New York, pp. 57–77, 1987.Google Scholar
  11. 11.
    Borkar, V. S.,Control of Markov Chains with Long-Run Average Cost Criterion: The Dynamic Programming Equations, SIAM Journal on Control and Optimization, Vol. 27, pp. 642–657, 1989.Google Scholar
  12. 12.
    Borkar, V. S., andGhosh, M. K.,Ergodic and Adaptive Control of Nearest Neighbor Motions, Mathematics of Control, Systems, and Signals, Vol. 4, pp. 81–98, 1990.Google Scholar
  13. 13.
    Borkar, V. S.,A Convex Analytic Approach to Markov Decision Processes, Probability Theory and Related Fields, Vol. 78, pp. 583–602, 1988.Google Scholar
  14. 14.
    Billingsley, P.,Convergence of Probability Measures, Wiley, New York, New York, 1968.Google Scholar
  15. 15.
    Loeve, M.,Probability Theory, Part 1, 4th Edition, Springer, New York, New York, 1977.Google Scholar
  16. 16.
    Fan, K.,Fixed-Point and Minimax Theorems in Locally Convex Topological Linear Spaces, Proceedings of the National Academy of Sciences, USA, Vol. 38, pp. 121–126, 1952.Google Scholar

Copyright information

© Plenum Publishing Corporation 1993

Authors and Affiliations

  • V. S. Borkar
    • 1
  • M. K. Ghosh
    • 2
  1. 1.Department of Electrical EngineeringIndian Institute of ScienceBangaloreIndia
  2. 2.Department of MathematicsIndian Institute of ScienceBangaloreIndia

Personalised recommendations