International Journal of Game Theory

, Volume 44, Issue 1, pp 83–112 | Cite as

Time consistent Markov policies in dynamic economies with quasi-hyperbolic consumers



We study the question of existence and computation of time-consistent Markov policies of quasi-hyperbolic consumers under a stochastic transition technology in a general class of economies with multidimensional action spaces and uncountable state spaces. Under standard complementarity assumptions on preferences, as well as a mild geometric condition on transition probabilities, we prove existence of time-consistent solutions in Markovian policies, and provide conditions for the existence of continuous and monotone equilibria. We present applications of our methods to habit formation models, environmental policies, and models of consumption under borrowing constraints, and hence show how our methods extend the results obtained by Harris and Laibson (Econometrica 69:935–957, 2001) to a broad class of dynamic economies. We also present a simple successive approximation scheme for computing extremal equilibrium, and provide some results on the existence of monotone equilibrium comparative statics in the model’s deep parameters.


Time consistency Markov equilibria Stochastic games  Constructive methods 



We thank Robert Becker, Madhav Chandrasekher, Manjira Datta, Paweł Dziewulski, Amanda Friedenberg, Ed Green, Seppo Heikkilä, Len Mirman, Peter Streufert, and especially Ed Prescott, as well participants of our SAET 2011 session for helpful conversations on the topics of this paper. We especially thank two anonymous referees and the associate editor for their excellent comments on an earlier draft of this paper. Balbus and Woźny reserach has been supported by NCN Grant No. UMO-2012/07/D/HS4/01393. All the usual caveats apply.


  1. Abreu D, Pearce D, Stacchetti E (1990) Toward a theory of discounted repeated games with imperfect monitoring. Econometrica 58(5):1041–1063CrossRefGoogle Scholar
  2. Acemoglu D, Aghion P, Bursztyn L, Hemous D (2012) The environment and directed technical change. Am Econ Rev 102(1):131–166CrossRefGoogle Scholar
  3. Alj A, Haurie A (1983) Dynamic equilibria in multigenerational stochastic games. IEEE Trans Autom Control 28:193–203CrossRefGoogle Scholar
  4. Amir R (1996) Strategic intergenerational bequests with stochastic convex production. Econ Theory 8: 367–376CrossRefGoogle Scholar
  5. Amir R (2002) Complementarity and diagonal dominance in discounted stochastic games. Ann Oper Res 114:39–56CrossRefGoogle Scholar
  6. Angeletos GM, Laibson D, Repetto A, Tobacman J, Weinberg S (2001) The hyberbolic consumption model: calibration, simulation, and empirical evaluation. J Econ Perspect 15(3):47–68CrossRefGoogle Scholar
  7. Balbus Ł, Nowak AS (2008) Existence of perfect equilibria in a class of multigenerational stochastic games of capital accumulation. Automatica 44:1471–1479CrossRefGoogle Scholar
  8. Balbus Ł, Reffett K, Woźny Ł (2013) A constructive geometrical approach to the uniqueness of Markov perfect equilibrium in stochastic games of intergenerational altruism. J Econ Dyn Control 37(5): 1019–1039Google Scholar
  9. Balbus Ł, Reffett K, Woźny Ł (2014) A constructive study of Markov equilibria in stochastic games with strategic complementarities. J Econ Theory 150:815–840Google Scholar
  10. Becker GS, Murphy KM (1988) A theory of rational addiction. J Polit Econ 96(4):675–700CrossRefGoogle Scholar
  11. Benabou R, Pycia M (2002) Dynamic inconsistency and self-control: a planner–doer interpretation. Econ Lett 77(3):419–424CrossRefGoogle Scholar
  12. Bernheim BD, Ray D (1986) On the existence of Markov-consistent plans under production uncertainty. Rev Econ Stud 53(5):877–882Google Scholar
  13. Bernheim D, Ray D, Yeltekin S (1999) Self-control, savings, and the low-asset trap. Stanford UniversityGoogle Scholar
  14. Billingsley P (1999) Convergence of Probability Measures. Wiley, New YorkCrossRefGoogle Scholar
  15. Brock WA, Taylor MS (2005) Economic growth and the environment: a review of theory and empirics, Chap. 28. In: Durlauf S, Aghion P (eds) Handbook of economic growth. Elsevier, Amsterdam, pp 1749–1821Google Scholar
  16. Caplin A, Leahy J (2006) The recursive approach to time inconsistency. J Econ Theory 131(1):134–156CrossRefGoogle Scholar
  17. Chade H, Prokopovych P, Smith L (2008) Repeated games with present-biased preferences. J Econ Theory 139(1):157–175CrossRefGoogle Scholar
  18. Davis A (1955) A characterization of complete lattices. Pac J Math 5:311–319CrossRefGoogle Scholar
  19. Dekel E, Lipman BL (2012) Costly self-control and random self-indulgence. Econometrica 80(3): 1271–1302CrossRefGoogle Scholar
  20. Dekel E, Lipman BL, Rustichini A (2001) Representing preferences with a unique subjective state space. Econometrica 69(4):891–934CrossRefGoogle Scholar
  21. Dekel E, Lipman BL, Rustichini A (2009) Temptation-driven preferences. Rev Econ Stud 76(3):937–971CrossRefGoogle Scholar
  22. Duffie D, Geanakoplos J, Mas-Colell A, McLennan A (1994) Stationary Markov equilibria. Econometrica 62(4):745–781CrossRefGoogle Scholar
  23. Dugundji J, Granas A (1982) Fixed point theory. Polish Scientific, WarsawGoogle Scholar
  24. Eisenhauer JG, Ventura L (2006) The prevalence of hyperbolic discounting: some European evidence. Appl Econ 38(11):1223–1234Google Scholar
  25. Feinberg E, Shwartz A (1995) Constrained Markov decision models with weighted discounted rewards. Math Oper Res 20:302–320Google Scholar
  26. Fudenberg D, Levine DK (2006) A dual-self model of impulse control. Am Econ Rev 96(5):1449–1476CrossRefGoogle Scholar
  27. Goldman SM (1980) Consistent plans. Rev Econ Stud 47(3):533–537CrossRefGoogle Scholar
  28. Gong L, Smith W (2007) Consumption and risk with hyperbolic discounting. Econ Lett 96(2):153–160CrossRefGoogle Scholar
  29. Gul F, Pesendorfer W (2001) Temptation and self-control. Econometrica 69(6):1403–1435CrossRefGoogle Scholar
  30. Gul F, Pesendorfer W (2004) Self-control and the theory of consumption. Econometrica 72(1):119–158CrossRefGoogle Scholar
  31. Harris C (1985) Existence and characterization of perfect equilibrium in games of perfect information. Econometrica 53(3):613–628CrossRefGoogle Scholar
  32. Harris C, Laibson D (2001) Dynamic choices of hyperbolic consumers. Econometrica 69(4):935–957CrossRefGoogle Scholar
  33. Jones LE, Manuelli RE (1995) A positive model of growth and pollution controls. Technical report, NBER working papers 5205Google Scholar
  34. Jones LE, Manuelli RE (2001) Endogenous policy choice: the case of pollution and growth. Rev Econ Dyn 4(2):369–405CrossRefGoogle Scholar
  35. Judd KL (2004) Existence, uniqueness, and computational theory for time consistent equilibria: a hyperbolic discounting example. Hoover Institution, Stanford, CA.Google Scholar
  36. Kall P (1986) Approximation to optimization problems: an elementary review. Math Oper Res 11(1):9–18CrossRefGoogle Scholar
  37. Karp L, Tsur Y (2011) Time perspective and climate change policy. J Environ Econ Manag 62(1):1–14CrossRefGoogle Scholar
  38. Krusell P, Smith A (2003) Consumption-savings decisions with quasi-geometric discounting. Econometrica 71(1):365–375CrossRefGoogle Scholar
  39. Krusell P, Kuruscu B, Smith AJ (2002a) Equilibrium welfare and government policy with quasi-geometric discounting. J Econ Theory 105(1):42–72CrossRefGoogle Scholar
  40. Krusell P, Kuruscu B, Smith AJ (2002b) Time orientation and asset prices. J Monet Econ 49(1):107–135CrossRefGoogle Scholar
  41. Kydland F, Prescott E (1977) Rules rather than discretion: the inconsistency of optimal plans. J Polit Econ 85(3):473–491CrossRefGoogle Scholar
  42. Kydland F, Prescott E (1980) Dynamic optimal taxation, rational expectations and optimal control. J Econ Dyn Control 2(1):79–91CrossRefGoogle Scholar
  43. Laibson D (1997) Golden eggs and hyperbolic discounting. Q J Econ 112(2):443–477CrossRefGoogle Scholar
  44. Leininger W (1986) The existence of perfect equilibria in model of growth with altruism between generations. Rev Econ Stud 53(3):349–368CrossRefGoogle Scholar
  45. Lemoine D, Traeger C (2012) Tipping points and ambiguity in the economics of climate change. Technical report, NBER working paper 18230.Google Scholar
  46. Marcet A, Marimon R (2011) Recursive contracts. Technical report, Barcelona GSE working paper series working paper no. 552.Google Scholar
  47. Marinacci M, Montrucchio L (2010) Unique solutions for stochastic recursive utilities. J Econ Theory 145(5):1776–1804CrossRefGoogle Scholar
  48. Markowsky G (1976) Chain-complete posets and directed sets with applications. Algebra Univ 6:53–68CrossRefGoogle Scholar
  49. Maskin E, Tirole J (2001) Markov perfect equilibrium: I. Observable actions. J Econ Theory 100(2):191–219CrossRefGoogle Scholar
  50. Matkowski J, Nowak A (2011) On discounted dynamic programming with unbounded returns. Econ Theory 46(3):455–474CrossRefGoogle Scholar
  51. Messner M, Pavoni N (2004) On the recursive saddlepoint methods. Technical report, MSGoogle Scholar
  52. Messner M, Pavoni N, Sleet C (2012) Recursive methods for incentive problems. Rev Econ Stud 15(4): 501–525Google Scholar
  53. Messner M, Pavoni N, Sleet C (2013) The dual approach to recursive optimization: theory and examples. Technical reportGoogle Scholar
  54. Nowak AS (2003) On a new class of nonzero-sum discounted stochastic games having stationary Nash equilibrium points. Int J Game Theory 32:121–132Google Scholar
  55. Nowak AS (2006) On perfect equilibria in stochastic models of growth with intergenerational altruism. Econ Theory 28:73–83CrossRefGoogle Scholar
  56. Nowak AS (2010) Existence of perfect equilibria in a class of multigenerational stochastic games of capital accumulation. J Optim Theory Appl 144:88–106CrossRefGoogle Scholar
  57. O’Donoghue T, Rabin M (1999a) Doing it now or later. Am Econ Rev 89(1):103–124CrossRefGoogle Scholar
  58. O’Donoghue T, Rabin M (1999b) Incentives for procrastinators. Q J Econ 114(3):769–816CrossRefGoogle Scholar
  59. Peleg B, Yaari ME (1973) On the existence of a consistent course of action when tastes are changing. Rev Econ Stud 40(3):391–401CrossRefGoogle Scholar
  60. Phelps E, Pollak R (1968) On second best national savings and game equilibrium growth. Rev Econ Stud 35:195–199CrossRefGoogle Scholar
  61. Martins-da Rocha VF, Vailakis Y (2010) Existence and uniqueness of a fixed point for local contractions. Econometrica 78(3):1127–1141CrossRefGoogle Scholar
  62. Saez-Marti M, Weibull J (2005) Discounting and altruism towards future decision-makers. J Econ Theory 122:254–266CrossRefGoogle Scholar
  63. Sorger G (2004) Consistent planning under quasi-geometric discounting. J Econ Theory 118(1):118–129CrossRefGoogle Scholar
  64. Strotz RH (1956) Myopia and inconsistency in dynamic utility maximization. Rev Econ Stud 23(3):165–180CrossRefGoogle Scholar
  65. Tarski A (1955) A lattice-theoretical fixpoint theorem and its applications. Pac J Math 5:285–309CrossRefGoogle Scholar
  66. Topkis DM (1978) Minimazing a submodular function on a lattice. Oper Res 26(2):305–321CrossRefGoogle Scholar
  67. Topkis DM (1998) Supermodularity and complementarity. Frontiers of economic research. Princeton University Press, PrincetonGoogle Scholar
  68. Vailakis Y, Van Le C (2012) Monotone concave and convex operators: applications to stochastic dynamic programming with unbounded returns.Google Scholar
  69. Veinott (1992) Lattice programming: qualitative optimization and equilibria. MS StandfordGoogle Scholar
  70. Vulikh BZ (1967) Introduction to the theory of partially ordered spaces. Wolters–Noordhoff Press, GroningenGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2014

Authors and Affiliations

  1. 1.Faculty of Mathematics, Computer Sciences and EconometricsUniversity of Zielona GóraZielona GoraPoland
  2. 2.Department of EconomicsArizona State UniversityTempeUSA
  3. 3.Department of Quantitative EconomicsWarsaw School of EconomicsWarsawPoland

Personalised recommendations