Dynamic Games and Applications

, Volume 6, Issue 2, pp 187–208 | Cite as

A Strategic Dynamic Programming Method for Studying Short-Memory Equilibria of Stochastic Games with Uncountable Number of States

  • Łukasz Balbus
  • Łukasz WoźnyEmail author


We study a class of infinite horizon stochastic games with uncountable number of states. We first characterize the set of all (nonstationary) short-term (Markovian) equilibrium values by developing a new (Abreu et al. in Econometrica 58(5):1041–1063, 1990)-type procedure operating in function spaces. This (among others) proves Markov perfect Nash equilibrium (MPNE) existence. Moreover, we present techniques of MPNE value set approximation by a sequence of sets of discretized functions iterated on our approximated APS-type operator. This method is new and has some advantages as compared to Judd et al. (Econometrica 71(4):1239–1254, 2003), Feng et al. (Int Econ Rev 55(1):83–110, 2014), or Sleet and Yeltekin (Dyn Games Appl doi: 10.1007/s13235-015-0139-1, 2015). We show applications of our approach to hyperbolic discounting games and dynamic games with strategic complementarities.


Stochastic games Hyperbolic discounting Supermodular games Short-memory (Markov) equilibria Constructive methods Computation Approximation 

JEL Classification



  1. 1.
    Abreu D, Pearce D, Stacchetti E (1986) Optimal cartel equilibria with imperfect monitoring. J Econ Theory 39(1):251–269MathSciNetCrossRefzbMATHGoogle Scholar
  2. 2.
    Abreu D, Pearce D, Stacchetti E (1990) Toward a theory of discounted repeated games with imperfect monitoring. Econometrica 58(5):1041–1063MathSciNetCrossRefzbMATHGoogle Scholar
  3. 3.
    Aliprantis CD, Border KC (2005) Infinite dimensional analysis. A Hitchhiker’s guide. Springer, HeilbelbergzbMATHGoogle Scholar
  4. 4.
    Amir R (1996) Continuous stochastic games of capital accumulation with convex transitions. Games Econ Behav 15(2):111–131MathSciNetCrossRefzbMATHGoogle Scholar
  5. 5.
    Amir R (2002) Complementarity and diagonal dominance in discounted stochastic games. Ann Oper Res 114(1–4):39–56MathSciNetCrossRefzbMATHGoogle Scholar
  6. 6.
    Amir R (2010) Discounted supermodular stochastic games: theory and applications. MimeoGoogle Scholar
  7. 7.
    Atkeson A (1991) International lending with moral hazard and risk of repudiation. Econometrica 59(4):1069–89MathSciNetCrossRefzbMATHGoogle Scholar
  8. 8.
    Balbus Ł, Jaśkiewicz A, Nowak AS (2014) Robust Markov perfect equilibria in a dynamic choice model with quasi-hyperbolic discounting. In: Haunschmied J, Veliov V, Wrzaczek S (eds) Dynamic games in economics. Dynamic modeling and econometrics in economics and finance, vol 16. Springer, Berlin, pp 1–22Google Scholar
  9. 9.
    Balbus Ł, Jaśkiewicz A, Nowak AS (2015) Existence of stationary Markov perfect equilibria in stochastic altruistic growth economies. J Optim Theory Appl 165(1):295–315MathSciNetCrossRefzbMATHGoogle Scholar
  10. 10.
    Balbus Ł, Nowak AS (2008) Existence of perfect equilibria in a class of multigenerational stochastic games of capital accumulation. Automatica 44(6):1471–1479MathSciNetCrossRefzbMATHGoogle Scholar
  11. 11.
    Balbus Ł, Reffett K, Woźny Ł (2012) Stationary Markovian equilibrium in altruistic stochastic OLG models with limited commitment. J Math Econ 48(2):115–132MathSciNetCrossRefzbMATHGoogle Scholar
  12. 12.
    Balbus Ł, Reffett K, Woźny Ł (2013a) A constructive geometrical approach to the uniqueness of Markov perfect equilibrium in stochastic games of intergenerational altruism. J Econ Dyn Control 37(5):1019–1039CrossRefGoogle Scholar
  13. 13.
    Balbus Ł, Reffett K, Woźny Ł (2013b) Markov stationary equilibria in stochastic supermodular games with imperfect private and public information. Dyn Games Appl 3(2):187–206MathSciNetCrossRefzbMATHGoogle Scholar
  14. 14.
    Balbus Ł, Reffett K, Woźny Ł (2014) A constructive study of Markov equilibria in stochastic games with strategic complementarities. J Econ Theory 150:815–840MathSciNetCrossRefzbMATHGoogle Scholar
  15. 15.
    Balbus Ł, Reffett K, Woźny Ł (2015) Time consistent Markov policies in dynamic economies with quasi-hyperbolic consumers. Int J Game Theory 44(1):83–112MathSciNetCrossRefzbMATHGoogle Scholar
  16. 16.
    Barelli P, Duggan J (2014) A note on semi-Markov perfect equilibria in discounted stochastic games. J Econ Theory 151:596–604MathSciNetCrossRefzbMATHGoogle Scholar
  17. 17.
    Beer G (1980) The approximation of upper semicontinuous multifunction by step multifunction. Pac J Math 87(1):11–19MathSciNetCrossRefzbMATHGoogle Scholar
  18. 18.
    Bernheim BD, Ray D (1983) Altruistic growth economies. I. Existence of bequest equilibria. MimeoGoogle Scholar
  19. 19.
    Bernheim BD, Ray D, Yeltekin S (2015) Poverty and self-control. Econometrica 83(5):1877–1911Google Scholar
  20. 20.
    Billingsley P (1968) Convergence of probability measures. Wiley, New YorkzbMATHGoogle Scholar
  21. 21.
    Caplin A, Leahy J (2006) The recursive approach to time inconsistency. J Econ Theory 131(1):134–156MathSciNetCrossRefzbMATHGoogle Scholar
  22. 22.
    Chade H, Prokopovych P, Smith L (2008) Repeated games with present-biased preferences. J Econ Theory 139(1):157–175MathSciNetCrossRefzbMATHGoogle Scholar
  23. 23.
    Chang R (1998) Credible monetary policy in an infinite horizon model: recursive approaches. J Econ Theory 81(2):431–461MathSciNetCrossRefzbMATHGoogle Scholar
  24. 24.
    Cole HL, Kocherlakota N (2001) Dynamic games with hidden actions and hidden states. J Econ Theory 98(1):114–126MathSciNetCrossRefzbMATHGoogle Scholar
  25. 25.
    Cronshaw MB (1997) Algorithms for finding repeated game equilibria. Comput Econ 10(2):139–168CrossRefzbMATHGoogle Scholar
  26. 26.
    Curtat L (1996) Markov equilibria of stochastic games with complementarities. Games Econ Behav 17(2):177–199MathSciNetCrossRefzbMATHGoogle Scholar
  27. 27.
    Doraszelski U, Escobar JF (2012) Restricted feedback in long term relationships. J Econ Theory 147(1):142–161MathSciNetCrossRefzbMATHGoogle Scholar
  28. 28.
    Duggan J (2012) Noisy stochastic games. Econometrica 80(5):2017–2045MathSciNetCrossRefzbMATHGoogle Scholar
  29. 29.
    Dutta PK, Sundaram R (1992) Markovian equilibrium in a class of stochastic games: existence theorems for discounted and undiscounted models. Econ Theory 2(2):197–214MathSciNetCrossRefzbMATHGoogle Scholar
  30. 30.
    Feng Z, Miao J, Peralta-Alva A, Santos MS (2014) Numerical simulation of nonoptimal dynamic equilibrium models. Int Econ Rev 55(1):83–110MathSciNetCrossRefzbMATHGoogle Scholar
  31. 31.
    Harris C, Laibson D (2001) Dynamic choices of hyperbolic consumers. Econometrica 69(4):935–57MathSciNetCrossRefzbMATHGoogle Scholar
  32. 32.
    Jaśkiewicz A, Nowak AS (2014) Stationary Markov perfect equilibria in risk sensitive stochastic overlapping generations models. J Econ Theory 151:411–447MathSciNetCrossRefzbMATHGoogle Scholar
  33. 33.
    Jaśkiewicz A, Nowak A S (2015) Stationary almost Markov perfect equilibria in discounted stochastic games. Math Oper Res. doi: 10.1287/moor.2015.0734
  34. 34.
    Judd KL, Yeltekin S, Conklin J (2003) Computing supergame equilibria. Econometrica 71(4):1239–1254MathSciNetCrossRefzbMATHGoogle Scholar
  35. 35.
    Karp L (2005) Global warming and hyperbolic discounting. J Polit Econ 89(2–3):261–282Google Scholar
  36. 36.
    Krusell P, Kuruşçu B, Smith AAJ (2002) Equilibrium welfare and government policy with quasi-geometric discounting. J Econ Theory 105(1):42–72MathSciNetCrossRefzbMATHGoogle Scholar
  37. 37.
    Kydland F, Prescott E (1980) Dynamic optimal taxation, rational expectations and optimal control. J Econ DynControl 2(1):79–91Google Scholar
  38. 38.
    Laibson D (1997) Golden eggs and hyperbolic discounting. Q J Econ 112(2):443–477CrossRefzbMATHGoogle Scholar
  39. 39.
    Levy Y, McLennan A (2015) Corrigendum to: discounted stochastic games with no stationary Nash equilibrium: two examples. Econometrica 83(3):1237–1252MathSciNetCrossRefGoogle Scholar
  40. 40.
    Mertens J-F, Parthasarathy T (1987) Equilibria for discounted stochastic games. C.O.R.E. Discussion Paper 8750Google Scholar
  41. 41.
    Mertens J-F, Parthasarathy T (2003) Equilibria for discounted stochastic games. In: Neyman A, Sorin S (eds) Stochastic games and applications. NATO Advanced Science Institutes series D: behavioural and social sciences. Kluwer, BostonGoogle Scholar
  42. 42.
    Mertens J-F, Sorin S, Zamir S (2015) Repeated games. Cambridge University Press, New YorkCrossRefzbMATHGoogle Scholar
  43. 43.
    Milgrom P, Roberts J (1994) Comparing equilibria. Am Econ Rev 84(3):441–459Google Scholar
  44. 44.
    Nowak AS (2003) On a new class of nonzero-sum discounted stochastic games having stationary Nash equilibrium points. Int JGame Theory 32:121–132MathSciNetCrossRefzbMATHGoogle Scholar
  45. 45.
    Nowak AS (2007) On stochastic games in economics. Math Methods Oper Res 66(3):513–530MathSciNetCrossRefzbMATHGoogle Scholar
  46. 46.
    Nowak AS (2010) On a noncooperative stochastic game played by internally cooperating generations. J Optim Theory Appl 144(1):88–106MathSciNetCrossRefzbMATHGoogle Scholar
  47. 47.
    Nowak AS, Raghavan T (1992) Existence of stationary correlated equilibria with symmetric information for discounted stochastic games. Math Oper Res 17(3):519–526MathSciNetCrossRefzbMATHGoogle Scholar
  48. 48.
    Pearce D, Stacchetti E (1997) Time consistent taxation by a government with redistributive goals. J Econ Theory 72(2):282–305MathSciNetCrossRefzbMATHGoogle Scholar
  49. 49.
    Phelan C, Stacchetti E (2001) Sequential equilibria in a Ramsey tax model. Econometrica 69(6):1491–1518MathSciNetCrossRefzbMATHGoogle Scholar
  50. 50.
    Shapley L (1953) Stochastic games. Proc Natl Acad Sci USA 39(10):1095–1100MathSciNetCrossRefzbMATHGoogle Scholar
  51. 51.
    Sleet C, Yeltekin S (2015) On the computation of value correspondences. Dyn Games Appl doi: 10.1007/s13235-015-0139-1
  52. 52.
    Stokey NL (1991) Credible public policy. J Econ DynControl 15(4):627–656zbMATHGoogle Scholar
  53. 53.
    Strotz RH (1955) Myopia and inconsistency in dynamic utility maximization. Rev Econ Stud 23(3):165–180CrossRefGoogle Scholar
  54. 54.
    Topkis DM (1978) Minimizing a submodular function on a lattice. Oper Res 26(2):305–321MathSciNetCrossRefzbMATHGoogle Scholar
  55. 55.
    Topkis DM (1979) Equilibrium points in nonzero-sum n-person submodular games. SIAM J Control Optim 17(6):773–787MathSciNetCrossRefzbMATHGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2015

Authors and Affiliations

  1. 1.Faculty of Mathematics, Computer Sciences and EconometricsUniversity of Zielona GóraZielona GoraPoland
  2. 2.Department of Quantitative EconomicsWarsaw School of EconomicsWarsawPoland

Personalised recommendations