Dynamic Games and Applications

, Volume 6, Issue 2, pp 187–208 | Cite as

A Strategic Dynamic Programming Method for Studying Short-Memory Equilibria of Stochastic Games with Uncountable Number of States

Article

Abstract

We study a class of infinite horizon stochastic games with uncountable number of states. We first characterize the set of all (nonstationary) short-term (Markovian) equilibrium values by developing a new (Abreu et al. in Econometrica 58(5):1041–1063, 1990)-type procedure operating in function spaces. This (among others) proves Markov perfect Nash equilibrium (MPNE) existence. Moreover, we present techniques of MPNE value set approximation by a sequence of sets of discretized functions iterated on our approximated APS-type operator. This method is new and has some advantages as compared to Judd et al. (Econometrica 71(4):1239–1254, 2003), Feng et al. (Int Econ Rev 55(1):83–110, 2014), or Sleet and Yeltekin (Dyn Games Appl doi:10.1007/s13235-015-0139-1, 2015). We show applications of our approach to hyperbolic discounting games and dynamic games with strategic complementarities.

Keywords

Stochastic games Hyperbolic discounting Supermodular games Short-memory (Markov) equilibria Constructive methods Computation Approximation 

JEL Classification

C72 

References

  1. 1.
    Abreu D, Pearce D, Stacchetti E (1986) Optimal cartel equilibria with imperfect monitoring. J Econ Theory 39(1):251–269MathSciNetCrossRefMATHGoogle Scholar
  2. 2.
    Abreu D, Pearce D, Stacchetti E (1990) Toward a theory of discounted repeated games with imperfect monitoring. Econometrica 58(5):1041–1063MathSciNetCrossRefMATHGoogle Scholar
  3. 3.
    Aliprantis CD, Border KC (2005) Infinite dimensional analysis. A Hitchhiker’s guide. Springer, HeilbelbergMATHGoogle Scholar
  4. 4.
    Amir R (1996) Continuous stochastic games of capital accumulation with convex transitions. Games Econ Behav 15(2):111–131MathSciNetCrossRefMATHGoogle Scholar
  5. 5.
    Amir R (2002) Complementarity and diagonal dominance in discounted stochastic games. Ann Oper Res 114(1–4):39–56MathSciNetCrossRefMATHGoogle Scholar
  6. 6.
    Amir R (2010) Discounted supermodular stochastic games: theory and applications. MimeoGoogle Scholar
  7. 7.
    Atkeson A (1991) International lending with moral hazard and risk of repudiation. Econometrica 59(4):1069–89MathSciNetCrossRefMATHGoogle Scholar
  8. 8.
    Balbus Ł, Jaśkiewicz A, Nowak AS (2014) Robust Markov perfect equilibria in a dynamic choice model with quasi-hyperbolic discounting. In: Haunschmied J, Veliov V, Wrzaczek S (eds) Dynamic games in economics. Dynamic modeling and econometrics in economics and finance, vol 16. Springer, Berlin, pp 1–22Google Scholar
  9. 9.
    Balbus Ł, Jaśkiewicz A, Nowak AS (2015) Existence of stationary Markov perfect equilibria in stochastic altruistic growth economies. J Optim Theory Appl 165(1):295–315MathSciNetCrossRefMATHGoogle Scholar
  10. 10.
    Balbus Ł, Nowak AS (2008) Existence of perfect equilibria in a class of multigenerational stochastic games of capital accumulation. Automatica 44(6):1471–1479MathSciNetCrossRefMATHGoogle Scholar
  11. 11.
    Balbus Ł, Reffett K, Woźny Ł (2012) Stationary Markovian equilibrium in altruistic stochastic OLG models with limited commitment. J Math Econ 48(2):115–132MathSciNetCrossRefMATHGoogle Scholar
  12. 12.
    Balbus Ł, Reffett K, Woźny Ł (2013a) A constructive geometrical approach to the uniqueness of Markov perfect equilibrium in stochastic games of intergenerational altruism. J Econ Dyn Control 37(5):1019–1039CrossRefGoogle Scholar
  13. 13.
    Balbus Ł, Reffett K, Woźny Ł (2013b) Markov stationary equilibria in stochastic supermodular games with imperfect private and public information. Dyn Games Appl 3(2):187–206MathSciNetCrossRefMATHGoogle Scholar
  14. 14.
    Balbus Ł, Reffett K, Woźny Ł (2014) A constructive study of Markov equilibria in stochastic games with strategic complementarities. J Econ Theory 150:815–840MathSciNetCrossRefMATHGoogle Scholar
  15. 15.
    Balbus Ł, Reffett K, Woźny Ł (2015) Time consistent Markov policies in dynamic economies with quasi-hyperbolic consumers. Int J Game Theory 44(1):83–112MathSciNetCrossRefMATHGoogle Scholar
  16. 16.
    Barelli P, Duggan J (2014) A note on semi-Markov perfect equilibria in discounted stochastic games. J Econ Theory 151:596–604MathSciNetCrossRefMATHGoogle Scholar
  17. 17.
    Beer G (1980) The approximation of upper semicontinuous multifunction by step multifunction. Pac J Math 87(1):11–19MathSciNetCrossRefMATHGoogle Scholar
  18. 18.
    Bernheim BD, Ray D (1983) Altruistic growth economies. I. Existence of bequest equilibria. MimeoGoogle Scholar
  19. 19.
    Bernheim BD, Ray D, Yeltekin S (2015) Poverty and self-control. Econometrica 83(5):1877–1911Google Scholar
  20. 20.
    Billingsley P (1968) Convergence of probability measures. Wiley, New YorkMATHGoogle Scholar
  21. 21.
    Caplin A, Leahy J (2006) The recursive approach to time inconsistency. J Econ Theory 131(1):134–156MathSciNetCrossRefMATHGoogle Scholar
  22. 22.
    Chade H, Prokopovych P, Smith L (2008) Repeated games with present-biased preferences. J Econ Theory 139(1):157–175MathSciNetCrossRefMATHGoogle Scholar
  23. 23.
    Chang R (1998) Credible monetary policy in an infinite horizon model: recursive approaches. J Econ Theory 81(2):431–461MathSciNetCrossRefMATHGoogle Scholar
  24. 24.
    Cole HL, Kocherlakota N (2001) Dynamic games with hidden actions and hidden states. J Econ Theory 98(1):114–126MathSciNetCrossRefMATHGoogle Scholar
  25. 25.
    Cronshaw MB (1997) Algorithms for finding repeated game equilibria. Comput Econ 10(2):139–168CrossRefMATHGoogle Scholar
  26. 26.
    Curtat L (1996) Markov equilibria of stochastic games with complementarities. Games Econ Behav 17(2):177–199MathSciNetCrossRefMATHGoogle Scholar
  27. 27.
    Doraszelski U, Escobar JF (2012) Restricted feedback in long term relationships. J Econ Theory 147(1):142–161MathSciNetCrossRefMATHGoogle Scholar
  28. 28.
    Duggan J (2012) Noisy stochastic games. Econometrica 80(5):2017–2045MathSciNetCrossRefMATHGoogle Scholar
  29. 29.
    Dutta PK, Sundaram R (1992) Markovian equilibrium in a class of stochastic games: existence theorems for discounted and undiscounted models. Econ Theory 2(2):197–214MathSciNetCrossRefMATHGoogle Scholar
  30. 30.
    Feng Z, Miao J, Peralta-Alva A, Santos MS (2014) Numerical simulation of nonoptimal dynamic equilibrium models. Int Econ Rev 55(1):83–110MathSciNetCrossRefMATHGoogle Scholar
  31. 31.
    Harris C, Laibson D (2001) Dynamic choices of hyperbolic consumers. Econometrica 69(4):935–57MathSciNetCrossRefMATHGoogle Scholar
  32. 32.
    Jaśkiewicz A, Nowak AS (2014) Stationary Markov perfect equilibria in risk sensitive stochastic overlapping generations models. J Econ Theory 151:411–447MathSciNetCrossRefMATHGoogle Scholar
  33. 33.
    Jaśkiewicz A, Nowak A S (2015) Stationary almost Markov perfect equilibria in discounted stochastic games. Math Oper Res. doi:10.1287/moor.2015.0734
  34. 34.
    Judd KL, Yeltekin S, Conklin J (2003) Computing supergame equilibria. Econometrica 71(4):1239–1254MathSciNetCrossRefMATHGoogle Scholar
  35. 35.
    Karp L (2005) Global warming and hyperbolic discounting. J Polit Econ 89(2–3):261–282Google Scholar
  36. 36.
    Krusell P, Kuruşçu B, Smith AAJ (2002) Equilibrium welfare and government policy with quasi-geometric discounting. J Econ Theory 105(1):42–72MathSciNetCrossRefMATHGoogle Scholar
  37. 37.
    Kydland F, Prescott E (1980) Dynamic optimal taxation, rational expectations and optimal control. J Econ DynControl 2(1):79–91Google Scholar
  38. 38.
    Laibson D (1997) Golden eggs and hyperbolic discounting. Q J Econ 112(2):443–477CrossRefMATHGoogle Scholar
  39. 39.
    Levy Y, McLennan A (2015) Corrigendum to: discounted stochastic games with no stationary Nash equilibrium: two examples. Econometrica 83(3):1237–1252MathSciNetCrossRefGoogle Scholar
  40. 40.
    Mertens J-F, Parthasarathy T (1987) Equilibria for discounted stochastic games. C.O.R.E. Discussion Paper 8750Google Scholar
  41. 41.
    Mertens J-F, Parthasarathy T (2003) Equilibria for discounted stochastic games. In: Neyman A, Sorin S (eds) Stochastic games and applications. NATO Advanced Science Institutes series D: behavioural and social sciences. Kluwer, BostonGoogle Scholar
  42. 42.
    Mertens J-F, Sorin S, Zamir S (2015) Repeated games. Cambridge University Press, New YorkCrossRefMATHGoogle Scholar
  43. 43.
    Milgrom P, Roberts J (1994) Comparing equilibria. Am Econ Rev 84(3):441–459Google Scholar
  44. 44.
    Nowak AS (2003) On a new class of nonzero-sum discounted stochastic games having stationary Nash equilibrium points. Int JGame Theory 32:121–132MathSciNetCrossRefMATHGoogle Scholar
  45. 45.
    Nowak AS (2007) On stochastic games in economics. Math Methods Oper Res 66(3):513–530MathSciNetCrossRefMATHGoogle Scholar
  46. 46.
    Nowak AS (2010) On a noncooperative stochastic game played by internally cooperating generations. J Optim Theory Appl 144(1):88–106MathSciNetCrossRefMATHGoogle Scholar
  47. 47.
    Nowak AS, Raghavan T (1992) Existence of stationary correlated equilibria with symmetric information for discounted stochastic games. Math Oper Res 17(3):519–526MathSciNetCrossRefMATHGoogle Scholar
  48. 48.
    Pearce D, Stacchetti E (1997) Time consistent taxation by a government with redistributive goals. J Econ Theory 72(2):282–305MathSciNetCrossRefMATHGoogle Scholar
  49. 49.
    Phelan C, Stacchetti E (2001) Sequential equilibria in a Ramsey tax model. Econometrica 69(6):1491–1518MathSciNetCrossRefMATHGoogle Scholar
  50. 50.
    Shapley L (1953) Stochastic games. Proc Natl Acad Sci USA 39(10):1095–1100MathSciNetCrossRefMATHGoogle Scholar
  51. 51.
    Sleet C, Yeltekin S (2015) On the computation of value correspondences. Dyn Games Appl doi:10.1007/s13235-015-0139-1
  52. 52.
    Stokey NL (1991) Credible public policy. J Econ DynControl 15(4):627–656MATHGoogle Scholar
  53. 53.
    Strotz RH (1955) Myopia and inconsistency in dynamic utility maximization. Rev Econ Stud 23(3):165–180CrossRefGoogle Scholar
  54. 54.
    Topkis DM (1978) Minimizing a submodular function on a lattice. Oper Res 26(2):305–321MathSciNetCrossRefMATHGoogle Scholar
  55. 55.
    Topkis DM (1979) Equilibrium points in nonzero-sum n-person submodular games. SIAM J Control Optim 17(6):773–787MathSciNetCrossRefMATHGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2015

Authors and Affiliations

  1. 1.Faculty of Mathematics, Computer Sciences and EconometricsUniversity of Zielona GóraZielona GoraPoland
  2. 2.Department of Quantitative EconomicsWarsaw School of EconomicsWarsawPoland

Personalised recommendations