Skip to main content
SpringerLink
Log in

Constrained Markov Games: Nash Equilibria

  • Conference paper
Advances in Dynamic Games and Applications

Part of the book series: Annals of the International Society of Dynamic Games ((AISDG,volume 5))

Abstract

In this paper we develop the theory of constrained Markov games. We consider the expected average cost as well as discounted cost. We allow different players to have different types of costs. We present sufficient conditions for the existence of stationary Nash equilibrium. Our results are based on the theory of sensitivity analysis of mathematical programs developed by Dantzig, Folkman, and Shapiro [9], which was applied to Markov Decision Processes (MDPs) in [3]. We further characterize all stationary Nash equilibria as fixed points of some coupled Linear Programs.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 129.00
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 169.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 169.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Altman, E. and A. Shwartz. Optimal Priority Assignment: A Time Sharing Approach. IEEE Transactions on Automatic Control, AC-34, No. 10, 1089–1102, 1989.

    MathSciNet  Google Scholar 

  2. Altman, E. and A. Shwartz. Markov Decision Problems and State-Action Frequencies. SIAM Journal of Control and Optimization, 29, No. 4, 786–809, 1991.

    Article  MathSciNet  MATH  Google Scholar 

  3. Altman, E. and A. Shwartz. Sensitivity of Constrained Markov Decision Problems. Annals of Operations Research,32, 1–22, 1991.

    Article  MathSciNet  MATH  Google Scholar 

  4. Altman, E. Denumerable Constrained Markov Decision Processes and Finite Approximations. Mathematics of Operations Research, 19, No. 1, 169–191, 1994.

    Article  MathSciNet  MATH  Google Scholar 

  5. Altman, E. Asymptotic Properties of Constrained Markov Decision Processes. ZOR— Methods and Models in Operations Research, 37, No. 2, 151–170, 1993.

    MathSciNet  MATH  Google Scholar 

  6. Beutler, F. J. and K. W. Ross. Optimal Policies for Controlled Markov Chains with a Constraint. Journal of Mathematical Analysis and Applications, 112,236–252,1985.

    Article  MathSciNet  MATH  Google Scholar 

  7. Beutler, F. J. and K. W. Ross. Time-Average Optimal Constrained Semi-Markov Decision Processes. Advances of Applied Probability, 18, No. 2, 341–359, 1986.

    Article  MathSciNet  MATH  Google Scholar 

  8. Borkar, V. S. Ergodic Control of Markov Chains with Constraints—The General Case. SIAM Journal of Control and Optimization, 32, No. 1, 176–186, 1994.

    Article  MathSciNet  MATH  Google Scholar 

  9. Dantzig, G. B., J. Folkman, and N. Shapiro. On the Continuity of the Minimum Set of a Continuous Function. Journal of Mathematical Analysis and Applications, 17, 519–548, 1967.

    Article  MathSciNet  MATH  Google Scholar 

  10. Derman, C and M. Klein. Some Remarks on Finite Horizon Markovian Decision Models. Operations Research, 13, 272–278, 1965.

    Article  MathSciNet  MATH  Google Scholar 

  11. Feinberg, E. A. Constrained Semi-Markov Decision Processes with Average Rewards. ZOR, 39, 257–288, 1993.

    MathSciNet  Google Scholar 

  12. Feinberg, E. A. and M. I. Reiman. Optimality of Randomized Trunk Reservation. Probability in the Engineering and Informational Sciences, 8,463–489, 1994.

    Article  Google Scholar 

  13. Feinberg, E. A. and A. Shwartz. Constrained Markov Decision Models with Weighted Discounted Rewards. Mathematics of Operations Research, 20, 302–320, 1995.

    Article  MathSciNet  MATH  Google Scholar 

  14. Feinberg, E. A. and A. Shwartz. Constrained Discounted Dynamic Programming. Mathematics of Operations Research, 21, 922–945, 1996.

    Article  MathSciNet  MATH  Google Scholar 

  15. Hordijk, A. and L. C. M. Kallenberg. Constrained Undiscounted Stochastic Dynamic Programming. Mathematics of Operations Research, 9, No. 2, 276–289, 1984.

    Article  MathSciNet  MATH  Google Scholar 

  16. Hordijk, A. and F. Spieksma. Constrained Admission Control to a Queuing System. Advances of Applied Probability, 21,409–431, 1989.

    Article  MathSciNet  MATH  Google Scholar 

  17. Hsiao, M. T. and A. A. Lazar. Optimal Decentralized Flow Control of Markovian queueing Networks with Multiple Controllers. Performance Evaluation, 13,181–204, 1991.

    Article  MathSciNet  MATH  Google Scholar 

  18. Kallenberg, L. C. M. Linear Programming and Finite Markovian Control Problems. Mathematical Centre Tracts 148, Amsterdam, 1983.

    MATH  Google Scholar 

  19. Korilis, Y. A. and A. Lazar. On the Existence of Equilibria in Noncooperative Optimal Flow Control. Journal of the Association for Computing Machinery, 42, No. 3, 584–613, 1995.

    Article  MathSciNet  MATH  Google Scholar 

  20. Lazar, A. Optimal Flow Control of a Class of Queuing Networks in Equilibrium. IEEE Transactions on Automatic Control, 28, No. 11, 1001–1007, 1983.

    Article  MathSciNet  MATH  Google Scholar 

  21. Levi, R. and A. Shwartz. A Theory of ApproachabiHty and Throughput-Cost Tradeoff in a Queue with Impatient Customers. EE Pub. 936, Technion, 1994.

    Google Scholar 

  22. Levi, R. and A. Shwartz. Throughput-Delay Tradeoff with Impatient Arrivals. Proceedings of the 23rd Allerton Conference on Communications, Control and Computing, Allerton, IL, 1994.

    Google Scholar 

  23. Nain, P. and K. W. Ross. Optimal Priority Assignment with hard Constraint. Transactions on Automatic Control, 31, No. 10, 883–888, IEEE 1986.

    Article  MathSciNet  MATH  Google Scholar 

  24. Rosen, J. B. Existence and Uniqueness of Equilibrium Points for Concave n-Person Games. Econometrica, 33, 520–534, 1965.

    Article  MathSciNet  MATH  Google Scholar 

  25. Ross, K. W. Randomized and Past-Dependent Policies for Markov Decision Processes with Multiple Constraints. Operations Research, 37, No. 3, 474–477, 1989.

    Article  MathSciNet  MATH  Google Scholar 

  26. Ross K. W. and B. Chen. Optimal Scheduling of Interactive and Nonlnteractive Traffic in Telecommunication Systems. IEEE Transactions on Automatic Control, 33, No. 3, 261–267, 1988.

    Article  MATH  Google Scholar 

  27. Ross, K. W. and R. Varadarajan. Markov Decision Processes with Sample Path Constraints: The Communicating Case. Operations Research, 37, No. 5, 780–790, 1989.

    Article  MathSciNet  MATH  Google Scholar 

  28. Sennott, L. I. Constrained Discounted Markov Decision Chains. Probability in the Engineering and Informational Sciences, 5,463–475, 1991.

    Article  MathSciNet  MATH  Google Scholar 

  29. Sennott, L. I. Constrained Average Cost Markov Decision Chains, Probability in the Engineering and Informational Sciences 7, 69–83, 1993.

    Article  Google Scholar 

  30. Shimkin, N. Stochastic Games with Average Cost Constraints. Annals of the International Society of Dynamic Games, Vol. 1: Advances in Dynamic Games and Applications (T. Basar and A. Haurie, eds.) Birkhauser, Boston, 1994.

    Google Scholar 

  31. Shimkin, N. and A. Shwartz. Guaranteed Performance Regions for Markovian Systems with Competing Decision Makers, IEEE Transactions on Automatic Control, 38, 84–95, 1993.

    Article  MathSciNet  MATH  Google Scholar 

  32. Spieksma, F. M. Geometrically Ergodic Markov Chains and the Optimal Control of Queues. Ph.D. thesis, University of Leiden, 1990.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. INRIA, B.P. 93, Sophia-Antipolis Cedex, France

    Eitan Altman

  2. Department of Electrical Engineering, Technion-Israel Institute of Technology, Haifa, Israel

    Adam Shwartz

Authors
  1. Eitan Altman
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Adam Shwartz
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Centre for Industrial and Applicable Mathematics School of Mathematics, University of South Australia, 5000, Adelaide, SA, Australia

    Jerzy A. Filar

  2. School of Mathematics, University of South Australia, 5000, Adelaide, SA, Australia

    Vladimir Gaitsgory

  3. Division of Mathematical and Information Sciences, Faculty of Integrated Arts and Sciences, Hiroshima University, 1-7-1, Kagamiyama, 739-8521, Higashi-Hiroshima City, Japan

    Koichi Mizukami

Rights and permissions

Reprints and permissions

Copyright information

© 2000 Springer Science+Business Media New York

About this paper

Cite this paper

Altman, E., Shwartz, A. (2000). Constrained Markov Games: Nash Equilibria. In: Filar, J.A., Gaitsgory, V., Mizukami, K. (eds) Advances in Dynamic Games and Applications. Annals of the International Society of Dynamic Games, vol 5. Birkhäuser, Boston, MA. https://doi.org/10.1007/978-1-4612-1336-9_11

Download citation

  • DOI: https://doi.org/10.1007/978-1-4612-1336-9_11

  • Publisher Name: Birkhäuser, Boston, MA

  • Print ISBN: 978-1-4612-7100-0

  • Online ISBN: 978-1-4612-1336-9

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics

Search

Navigation