# Time consistent Markov policies in dynamic economies with quasi-hyperbolic consumers

- 303 Downloads
- 8 Citations

## Abstract

We study the question of existence and computation of time-consistent Markov policies of quasi-hyperbolic consumers under a stochastic transition technology in a general class of economies with multidimensional action spaces and uncountable state spaces. Under standard complementarity assumptions on preferences, as well as a mild geometric condition on transition probabilities, we prove existence of time-consistent solutions in Markovian policies, and provide conditions for the existence of continuous and monotone equilibria. We present applications of our methods to habit formation models, environmental policies, and models of consumption under borrowing constraints, and hence show how our methods extend the results obtained by Harris and Laibson (Econometrica 69:935–957, 2001) to a broad class of dynamic economies. We also present a simple successive approximation scheme for computing extremal equilibrium, and provide some results on the existence of monotone equilibrium comparative statics in the model’s deep parameters.

## Keywords

Time consistency Markov equilibria Stochastic games Constructive methods## Notes

### Acknowledgments

We thank Robert Becker, Madhav Chandrasekher, Manjira Datta, Paweł Dziewulski, Amanda Friedenberg, Ed Green, Seppo Heikkilä, Len Mirman, Peter Streufert, and especially Ed Prescott, as well participants of our SAET 2011 session for helpful conversations on the topics of this paper. We especially thank two anonymous referees and the associate editor for their excellent comments on an earlier draft of this paper. Balbus and Woźny reserach has been supported by NCN Grant No. UMO-2012/07/D/HS4/01393. All the usual caveats apply.

## References

- Abreu D, Pearce D, Stacchetti E (1990) Toward a theory of discounted repeated games with imperfect monitoring. Econometrica 58(5):1041–1063CrossRefGoogle Scholar
- Acemoglu D, Aghion P, Bursztyn L, Hemous D (2012) The environment and directed technical change. Am Econ Rev 102(1):131–166CrossRefGoogle Scholar
- Alj A, Haurie A (1983) Dynamic equilibria in multigenerational stochastic games. IEEE Trans Autom Control 28:193–203CrossRefGoogle Scholar
- Amir R (1996) Strategic intergenerational bequests with stochastic convex production. Econ Theory 8: 367–376CrossRefGoogle Scholar
- Amir R (2002) Complementarity and diagonal dominance in discounted stochastic games. Ann Oper Res 114:39–56CrossRefGoogle Scholar
- Angeletos GM, Laibson D, Repetto A, Tobacman J, Weinberg S (2001) The hyberbolic consumption model: calibration, simulation, and empirical evaluation. J Econ Perspect 15(3):47–68CrossRefGoogle Scholar
- Balbus Ł, Nowak AS (2008) Existence of perfect equilibria in a class of multigenerational stochastic games of capital accumulation. Automatica 44:1471–1479CrossRefGoogle Scholar
- Balbus Ł, Reffett K, Woźny Ł (2013) A constructive geometrical approach to the uniqueness of Markov perfect equilibrium in stochastic games of intergenerational altruism. J Econ Dyn Control 37(5): 1019–1039Google Scholar
- Balbus Ł, Reffett K, Woźny Ł (2014) A constructive study of Markov equilibria in stochastic games with strategic complementarities. J Econ Theory 150:815–840Google Scholar
- Becker GS, Murphy KM (1988) A theory of rational addiction. J Polit Econ 96(4):675–700CrossRefGoogle Scholar
- Benabou R, Pycia M (2002) Dynamic inconsistency and self-control: a planner–doer interpretation. Econ Lett 77(3):419–424CrossRefGoogle Scholar
- Bernheim BD, Ray D (1986) On the existence of Markov-consistent plans under production uncertainty. Rev Econ Stud 53(5):877–882Google Scholar
- Bernheim D, Ray D, Yeltekin S (1999) Self-control, savings, and the low-asset trap. Stanford UniversityGoogle Scholar
- Billingsley P (1999) Convergence of Probability Measures. Wiley, New YorkCrossRefGoogle Scholar
- Brock WA, Taylor MS (2005) Economic growth and the environment: a review of theory and empirics, Chap. 28. In: Durlauf S, Aghion P (eds) Handbook of economic growth. Elsevier, Amsterdam, pp 1749–1821Google Scholar
- Caplin A, Leahy J (2006) The recursive approach to time inconsistency. J Econ Theory 131(1):134–156CrossRefGoogle Scholar
- Chade H, Prokopovych P, Smith L (2008) Repeated games with present-biased preferences. J Econ Theory 139(1):157–175CrossRefGoogle Scholar
- Davis A (1955) A characterization of complete lattices. Pac J Math 5:311–319CrossRefGoogle Scholar
- Dekel E, Lipman BL (2012) Costly self-control and random self-indulgence. Econometrica 80(3): 1271–1302CrossRefGoogle Scholar
- Dekel E, Lipman BL, Rustichini A (2001) Representing preferences with a unique subjective state space. Econometrica 69(4):891–934CrossRefGoogle Scholar
- Dekel E, Lipman BL, Rustichini A (2009) Temptation-driven preferences. Rev Econ Stud 76(3):937–971CrossRefGoogle Scholar
- Duffie D, Geanakoplos J, Mas-Colell A, McLennan A (1994) Stationary Markov equilibria. Econometrica 62(4):745–781CrossRefGoogle Scholar
- Dugundji J, Granas A (1982) Fixed point theory. Polish Scientific, WarsawGoogle Scholar
- Eisenhauer JG, Ventura L (2006) The prevalence of hyperbolic discounting: some European evidence. Appl Econ 38(11):1223–1234Google Scholar
- Feinberg E, Shwartz A (1995) Constrained Markov decision models with weighted discounted rewards. Math Oper Res 20:302–320Google Scholar
- Fudenberg D, Levine DK (2006) A dual-self model of impulse control. Am Econ Rev 96(5):1449–1476CrossRefGoogle Scholar
- Goldman SM (1980) Consistent plans. Rev Econ Stud 47(3):533–537CrossRefGoogle Scholar
- Gong L, Smith W (2007) Consumption and risk with hyperbolic discounting. Econ Lett 96(2):153–160CrossRefGoogle Scholar
- Gul F, Pesendorfer W (2001) Temptation and self-control. Econometrica 69(6):1403–1435CrossRefGoogle Scholar
- Gul F, Pesendorfer W (2004) Self-control and the theory of consumption. Econometrica 72(1):119–158CrossRefGoogle Scholar
- Harris C (1985) Existence and characterization of perfect equilibrium in games of perfect information. Econometrica 53(3):613–628CrossRefGoogle Scholar
- Harris C, Laibson D (2001) Dynamic choices of hyperbolic consumers. Econometrica 69(4):935–957CrossRefGoogle Scholar
- Jones LE, Manuelli RE (1995) A positive model of growth and pollution controls. Technical report, NBER working papers 5205Google Scholar
- Jones LE, Manuelli RE (2001) Endogenous policy choice: the case of pollution and growth. Rev Econ Dyn 4(2):369–405CrossRefGoogle Scholar
- Judd KL (2004) Existence, uniqueness, and computational theory for time consistent equilibria: a hyperbolic discounting example. Hoover Institution, Stanford, CA.Google Scholar
- Kall P (1986) Approximation to optimization problems: an elementary review. Math Oper Res 11(1):9–18CrossRefGoogle Scholar
- Karp L, Tsur Y (2011) Time perspective and climate change policy. J Environ Econ Manag 62(1):1–14CrossRefGoogle Scholar
- Krusell P, Smith A (2003) Consumption-savings decisions with quasi-geometric discounting. Econometrica 71(1):365–375CrossRefGoogle Scholar
- Krusell P, Kuruscu B, Smith AJ (2002a) Equilibrium welfare and government policy with quasi-geometric discounting. J Econ Theory 105(1):42–72CrossRefGoogle Scholar
- Krusell P, Kuruscu B, Smith AJ (2002b) Time orientation and asset prices. J Monet Econ 49(1):107–135CrossRefGoogle Scholar
- Kydland F, Prescott E (1977) Rules rather than discretion: the inconsistency of optimal plans. J Polit Econ 85(3):473–491CrossRefGoogle Scholar
- Kydland F, Prescott E (1980) Dynamic optimal taxation, rational expectations and optimal control. J Econ Dyn Control 2(1):79–91CrossRefGoogle Scholar
- Laibson D (1997) Golden eggs and hyperbolic discounting. Q J Econ 112(2):443–477CrossRefGoogle Scholar
- Leininger W (1986) The existence of perfect equilibria in model of growth with altruism between generations. Rev Econ Stud 53(3):349–368CrossRefGoogle Scholar
- Lemoine D, Traeger C (2012) Tipping points and ambiguity in the economics of climate change. Technical report, NBER working paper 18230.Google Scholar
- Marcet A, Marimon R (2011) Recursive contracts. Technical report, Barcelona GSE working paper series working paper no. 552.Google Scholar
- Marinacci M, Montrucchio L (2010) Unique solutions for stochastic recursive utilities. J Econ Theory 145(5):1776–1804CrossRefGoogle Scholar
- Markowsky G (1976) Chain-complete posets and directed sets with applications. Algebra Univ 6:53–68CrossRefGoogle Scholar
- Maskin E, Tirole J (2001) Markov perfect equilibrium: I. Observable actions. J Econ Theory 100(2):191–219CrossRefGoogle Scholar
- Matkowski J, Nowak A (2011) On discounted dynamic programming with unbounded returns. Econ Theory 46(3):455–474CrossRefGoogle Scholar
- Messner M, Pavoni N (2004) On the recursive saddlepoint methods. Technical report, MSGoogle Scholar
- Messner M, Pavoni N, Sleet C (2012) Recursive methods for incentive problems. Rev Econ Stud 15(4): 501–525Google Scholar
- Messner M, Pavoni N, Sleet C (2013) The dual approach to recursive optimization: theory and examples. Technical reportGoogle Scholar
- Nowak AS (2003) On a new class of nonzero-sum discounted stochastic games having stationary Nash equilibrium points. Int J Game Theory 32:121–132Google Scholar
- Nowak AS (2006) On perfect equilibria in stochastic models of growth with intergenerational altruism. Econ Theory 28:73–83CrossRefGoogle Scholar
- Nowak AS (2010) Existence of perfect equilibria in a class of multigenerational stochastic games of capital accumulation. J Optim Theory Appl 144:88–106CrossRefGoogle Scholar
- O’Donoghue T, Rabin M (1999a) Doing it now or later. Am Econ Rev 89(1):103–124CrossRefGoogle Scholar
- O’Donoghue T, Rabin M (1999b) Incentives for procrastinators. Q J Econ 114(3):769–816CrossRefGoogle Scholar
- Peleg B, Yaari ME (1973) On the existence of a consistent course of action when tastes are changing. Rev Econ Stud 40(3):391–401CrossRefGoogle Scholar
- Phelps E, Pollak R (1968) On second best national savings and game equilibrium growth. Rev Econ Stud 35:195–199CrossRefGoogle Scholar
- Martins-da Rocha VF, Vailakis Y (2010) Existence and uniqueness of a fixed point for local contractions. Econometrica 78(3):1127–1141CrossRefGoogle Scholar
- Saez-Marti M, Weibull J (2005) Discounting and altruism towards future decision-makers. J Econ Theory 122:254–266CrossRefGoogle Scholar
- Sorger G (2004) Consistent planning under quasi-geometric discounting. J Econ Theory 118(1):118–129CrossRefGoogle Scholar
- Strotz RH (1956) Myopia and inconsistency in dynamic utility maximization. Rev Econ Stud 23(3):165–180CrossRefGoogle Scholar
- Tarski A (1955) A lattice-theoretical fixpoint theorem and its applications. Pac J Math 5:285–309CrossRefGoogle Scholar
- Topkis DM (1978) Minimazing a submodular function on a lattice. Oper Res 26(2):305–321CrossRefGoogle Scholar
- Topkis DM (1998) Supermodularity and complementarity. Frontiers of economic research. Princeton University Press, PrincetonGoogle Scholar
- Vailakis Y, Van Le C (2012) Monotone concave and convex operators: applications to stochastic dynamic programming with unbounded returns.Google Scholar
- Veinott (1992) Lattice programming: qualitative optimization and equilibria. MS StandfordGoogle Scholar
- Vulikh BZ (1967) Introduction to the theory of partially ordered spaces. Wolters–Noordhoff Press, GroningenGoogle Scholar