Abstract
We show that an undiscounted stochastic game possesses optimal stationary strategies if and only if a global minimum with objective value zero can be found to an appropriate nonlinear program with linear constraints. This nonlinear program arises as a method for solving a certain bilinear system, satisfaction of which is also equivalent to finding a stationary optimal solution for the game. The objective function of the program is a nonnegatively valued quadric polynomial.
This is a preview of subscription content, log in via an institution to check access.
References
T. Bewley and E. Kohlberg, “On stochastic games with stationary optimal strategies“,Mathematics of Operations Research 3 (1978) 104–125.
D. Blackwell and T.S. Ferguson, “The big match“,Annals of Mathematical Statistics 39 (1968) 159–163.
J.A. Filar, “Ordered field property for stochastic games when the player who controls transition probabilities changes from state to state“,Journal of Optimization Theory and Applications 34 (1981) 505–517.
J.A. Filar, “A single loop stochastic game which one player can terminate“,Opsearch 18 (1981) 185–203.
P. Gill, W. Murray, M. Saunders and M. Wright, “User's guide for SOL/NPSOL”, Technical Report SOL 83-12, Department of Operations Research, Stanford University (Stanford, CA 94305, 1983).
A.J. Hoffman and R.M. Karp, “On non-terminating stochastic games“,Management Science 12 (1966) 359–370.
A. Hordijk and L.C.M. Kallenberg, “Linear programming and Markov decision chains“,Management Science 25 (1979) 352–362.
L.C.M. Kallenberg, “Linear programming and finite Markovian control problems”, Doctoral Thesis, University of Leiden (1980), Mathematical Centre Tracts 148, Amsterdam (1983).
T. Parthasarathy and T.E.S. Raghavan, “An orderfield property for stochastic games when one player controls the transition probabilities“,Journal of Optimization Theory and Applications 33 (1981) 375–392.
T. Parthasarathy, S.H. Tijs and O.J. Vrieze, “Stochastic games with state independent transitions and separable rewards”, in: G. Hammer and D. Pallaschke, eds.,Selected topics in operations research and mathematical economics, Springer-Verlag Lecture Notes Series #226 (1984).
T.E.S. Raghavan, S.H. Tijs and O.J. Vrieze, “On stochastic games with additive reward and transition structure”,Journal of Optimization Theory and Applications (to appear).
U.G. Rothblum, “Solving stopping stochastic games by maximizing a linear function subject to quadratic constraints“, in: O. Moeschlin and D. Pallaschke, eds., Game theory and related topics (North-Holland Publishing Company, Amsterdam, 1979) 103–105.
L.S. Shapley, “Stochastic games“,Proceedings of the National Academy of Sciences USA 39 (1953) 1095–1100.
M.J. Sobel, “Noncooperative games“,The Annals of Mathematical Statistics 42 (1971) 1930–1935.
O.J. Vrieze, “Stochastic games with finite state and action spaces”, Ph.D. thesis, Department of Mathematics, Free University of Amsterdam, The Netherlands, 1983.
Author information
Authors and Affiliations
Additional information
This research was supported in part by the National Science Foundation under the grant #ECS-8503440. We wish to thank the referee for many helpful comments and in streamlining the presentation.
Rights and permissions
About this article
Cite this article
Filar, J.A., Schultz, T.A. Nonlinear programming and stationary strategies in stochastic games. Mathematical Programming 34, 243–247 (1986). https://doi.org/10.1007/BF01580590
Received:
Revised:
Issue Date:
DOI: https://doi.org/10.1007/BF01580590