Abstract
The notion of stationary equilibrium is one of the most crucial solution concepts in stochastic games. However, a stochastic game can have multiple stationary equilibria, some of which may be unstable or counterintuitive. As a refinement of stationary equilibrium, we extend the concept of perfect equilibrium in strategic games to stochastic games and formulate the notion of perfect stationary equilibrium (PeSE). To further promote its applications, we develop a differentiable homotopy method to compute such an equilibrium. We incorporate vanishing logarithmic barrier terms into the payoff functions, thereby constituting a logarithmic-barrier stochastic game. As a result of this barrier game, we attain a continuously differentiable homotopy system. To reduce the number of variables in the homotopy system, we eliminate the Bellman equations through a replacement of variables and derive an equivalent system. We use the equivalent system to establish the existence of a smooth path, which starts from an arbitrary total mixed strategy profile and ends at a PeSE. Extensive numerical experiments, including relevant applications like dynamic oligopoly models and dynamic legislative voting, further affirm the effectiveness and efficiency of the method.
Article PDF
Similar content being viewed by others
References
Abbring, J.H., Campbell, J.R., Tilly, J., et al.: Very simple Markov-perfect industry dynamics: theory. Econometrica 86(2), 721–735 (2018)
Acemoglu, D., Egorov, G., Sonin, K.: Equilibrium refinement in dynamic voting games. MIT Department of Economics Working Paper Series, Working Paper 09-26, Massachusetts Institute of Technology (2009)
Allgower, E.L., Georg, K.: Numerical Continuation Methods: An Introduction, vol. 13. Springer, Berlin (2012)
Amir, R., Mertens, J., Neyman, A.: Stochastic games in economics and related fields: an overview. In: Neyman, A., Sorin, S. (eds.) Stochastic Games and Applications. Springer, Dordrecht (2003)
Browder, F.E.: On continuity of fixed points under deformations of continuous mappings. Summa Bras. Math. 4, 183–191 (1960)
Chatterjee, K., Dutta, B., Ray, D., et al.: A noncooperative theory of coalitional bargaining. Rev. Econ. Stud. 60(2), 463–477 (1993)
Chen, Y., Dang, C.: A reformulation-based simplicial homotopy method for approximating perfect equilibria. Comput. Econ. 54(3), 877–891 (2019)
Chen, Y., Dang, C.: A differentiable homotopy method to compute perfect equilibria. Math. Program. 185, 77–109 (2021)
Dang, C., Herings, P.J.J., Li, P.: An interior-point differentiable path-following method to compute stationary equilibria in stochastic games. INFORMS J. Comput. 34(3), 1403–1418 (2022)
Dirkse, S.P., Ferris, M.C.: The path solver: a nommonotone stabilization scheme for mixed complementarity problems. Optim. Methods Softw. 5(2), 123–156 (1995)
Doraszelski, U., Pakes, A.: A framework for applied dynamic analysis in IO. Handb. Ind. Organ. 3, 1887–1966 (2007)
Doraszelski, U., Satterthwaite, M.: Computable Markov-perfect industry dynamics. Rand J. Econ. 41(2), 215–243 (2010)
Duggan, J., Kalandrakis, T.: Dynamic legislative policy making. J. Econ. Theory 147(5), 1653–1688 (2012)
Eaves, B.C., Schmedders, K.: General equilibrium models and homotopy methods. J. Econ. Dyn. Control 23(9–10), 1249–1279 (1999)
Eibelshäuser, S., Poensgen, D.: Markov quantal response equilibrium and a homotopy method for computing and selecting Markov perfect equilibria of dynamic stochastic games. Available at SSRN 3314404 (2019)
Eraslan, H., Evdokimov, K.S., Zápal, J.: Dynamic legislative bargaining. In: Karagozoglu, E., Hyndman, K.B. (eds.) Bargaining: Current Research and Future Directions, pp. 151–175. Palgrave Macmillan, Cham (2022)
Ericson, R., Pakes, A.: Markov-perfect industry dynamics: a framework for empirical work. Rev. Econ. Stud. 62(1), 53–82 (1995)
Fink, A.M.: Equilibrium in a stochastic \( n \)-person game. J. Sci. Hiroshima Univ. Ser. AI (Math.) 28(1), 89–93 (1964)
Goldlücke, S., Kranz, S.: Discounted stochastic games with voluntary transfers. Econ. Theory 66(1), 235–263 (2018)
Gomes, A., Jehiel, P.: Dynamic processes of social and economic interactions: on the persistence of inefficiencies. J. Political Econ. 113(3), 626–667 (2005)
Govindan, S., Wilson, R.: Global Newton method for stochastic games. J. Econ. Theory 144(1), 414–421 (2009)
Hansen, K.A., Miltersen, P.B., Sørensen, T.B.: The computational complexity of trembling hand perfection and other equilibrium refinements. In: International Symposium on Algorithmic Game Theory. Springer, Berlin, pp. 198–209 (2010)
Harsanyi, J.C.: The tracing procedure: a Bayesian approach to defining a solution for \( n\)-person noncooperative games. Int. J. Game Theory 4, 61–94 (1975)
Harsanyi, J.C., Selten, R.: A General Theory of Equilibrium Selection in Games. MIT Press, Cambridge (1988)
He, W.: Discontinuous stochastic games. Econ. Theory 73(4), 827–858 (2022)
Herings, P.J.J.: Two simple proofs of the feasibility of the linear tracing procedure. Econ. Theory 15(2), 485–490 (2000)
Herings, P.J.J., Peeters, R.J.A.P.: A differentiable homotopy to compute Nash equilibria of \(n\)-person games. Econ. Theory 18(1), 159–185 (2001)
Herings, P.J.J., Peeters, R.J.A.P.: Stationary equilibria in stochastic games: structure, selection, and computation. J. Econ. Theory 118, 32–60 (2004)
Herings, P.J.J., Schmedders, K.: Computing equilibria in finance economies with incomplete markets and transaction costs. Econ. Theory 27(3), 493–512 (2006)
Herings, P.J.J., Peeters, R., Schinkel, M.P.: Intertemporal market division: a case of alternating monopoly. Eur. Econ. Rev. 49(5), 1207–1223 (2005)
Li, P., Dang, C.: An arbitrary starting tracing procedure for computing subgame perfect equilibria. J. Optim. Theory Appl. 186(2), 667–687 (2020)
Manea, M.: Intermediation and resale in networks. J. Political Econ. 126(3), 1250–1301 (2018)
Mas-Colell, A.: The Theory of General Economic Equilibrium: A Differentiable Approach, vol. 9. Cambridge University Press, Cambridge (1989)
Maskin, E., Tirole, J.: Markov perfect equilibrium: I. Observable actions. J. Econ. Theory 100(2), 191–219 (2001)
McKelvey, R.D., Palfrey, T.R.: Quantal response equilibria for normal form games. Games Econ. Behav. 10(1), 6–38 (1995)
Mertens, J.F.: Stable equilibria-a reformulation: Part I. Definition and basic properties. Math. Oper. Res. 14(4), 575–625 (1989)
Myerson, R.B.: Refinements of the Nash equilibrium concept. Int. J. Game Theory 7(2), 73–80 (1978)
Okada, A.: Dynamic bargaining with voluntary participation and externalities. Econ. Theory 75(2), 427–452 (2023)
Osborne, M.J., Rubinstein, A.: A Course in Game Theory. MIT Press, Cambridge (1994)
Schanuel, S.H., Simon, L.K., Zame, W.R.: The algebraic geometry of games and the tracing procedure. In: Selten, R. (ed.) Game Equilibrium Models II, pp. 9–43. Springer, Berlin (1991)
Selten, R.: Reexamination of the perfectness concept for equilibrium points in extensive games. Int. J. Game Theory 4(1), 25–55 (1975)
Shapley, L.S.: Stochastic games. Proc. Natl. Acad. Sci. 39(10), 1095–1100 (1953)
Sobel, M.: Noncooperative stochastic games. Ann. Math. Stat. 42(6), 1930–1935 (1971)
Takahashi, M.: Equilibrium points of stochastic non-cooperative \( n \)-person games. J. Sci. Hiroshima Univ. Ser. AI (Math.) 28(1), 95–99 (1964)
van den Elzen, A., Talman, A.J.J.: A procedure for finding Nash equilibria in bi-matrix games. Z. Oper. Res. 35, 27–43 (1991)
von Stengel, B., van den Elzen, A., Talman, D.: Computing normal form perfect equilibria for extensive two-person games. Econometrica 70(2), 693–715 (2002)
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Li, P., Dang, C. & Herings, P.JJ. Computing perfect stationary equilibria in stochastic games. Econ Theory (2024). https://doi.org/10.1007/s00199-024-01565-w
Received:
Accepted:
Published:
DOI: https://doi.org/10.1007/s00199-024-01565-w