Abstract
We develop a theory for a general class of discrete-time stochastic control problems that, in various ways, are time-inconsistent in the sense that they do not admit a Bellman optimality principle. We attack these problems by viewing them within a game theoretic framework, and we look for subgame perfect Nash equilibrium points. For a general controlled Markov process and a fairly general objective functional, we derive an extension of the standard Bellman equation, in the form of a system of nonlinear equations, for the determination of the equilibrium strategy as well as the equilibrium value function. Most known examples of time-inconsistent stochastic control problems in the literature are easily seen to be special cases of the present theory. We also prove that for every time-inconsistent problem, there exists an associated time-consistent problem such that the optimal control and the optimal value function for the consistent problem coincide with the equilibrium control and value function, respectively for the time-inconsistent problem. To exemplify the theory, we study some concrete examples, such as hyperbolic discounting and mean–variance control.
Similar content being viewed by others
Notes
The main reason is that in order to get a good recursion, we need to express the right-hand side of the equation above as E[ ⋅ ]-expectations of objects involving \(X_{n+1}^{u}\).
An easy sufficient condition is that φ takes values in [0,1).
References
Barro, R.: Ramsey meets Laibson in the neoclassical growth model. Q. J. Econ. 114, 1125–1152 (1999)
Basak, S., Chabakauri, G.: Dynamic mean–variance asset allocation. Rev. Financ. Stud. 23, 2970–3016 (2010)
Björk, T., Murgoci, A.: A general theory of Markovian time-inconsistent stochastic control problems. Working paper (2009). http://www.hhs.se/se/Search/Person/Pages/Person.aspx?PersonID=37
Björk, T., Murgoci, A.: A theory of Markovian time-inconsistent stochastic control in continuous time. Submitted, Stockholm School of Economics (2010). http://hhs.se/DF/People/Pages/Person.aspx?PersonID=37
Björk, T., Murgoci, A., Zhou, X.-Y.: Mean–variance portfolio optimization with state dependent risk aversion. Math. Finance 24, 1–24 (2014)
Czichowsky, C.: Time-consistent mean–variance portfolio selection in discrete and continuous time. Finance Stoch. 17, 227–271 (2013)
Ekeland, I., Lazrak, A.: Being serious about non-commitment: subgame perfect equilibrium in continuous time. arXiv:math/0604264v1 [math.OC]
Ekeland, I., Privu, T.: Investment and consumption without commitment. Math. Financ. Econ. 2, 57–86 (2008)
Goldman, S.: Consistent plans. Rev. Econ. Stud. 47, 533–537 (1980)
Harris, C., Laibson, D.: Dynamic choices of hyperbolic consumers. Econometrica 69, 935–957 (2001)
Krusell, P., Smith, A.: Consumption and savings decisions with quasi-geometric discounting. Econometrica 71, 366–375 (2003)
Kryger, E., Steffensen, M.: Some solvable portfolio problems with quadratic and collective objectives. Working paper (2010). http://papers.ssrn.com/sol3/papers.cfm?abstract_id=1577265
Luttmer, E., Mariotti, T.: Subjective discounting in an exchange economy. J. Polit. Econ. 111, 959–989 (2003)
Peleg, B., Menahem, E.: On the existence of a consistent course of action when tastes are changing. Rev. Econ. Stud. 40, 391–401 (1973)
Pollak, R.: Consistent planning. Rev. Econ. Stud. 35, 185–199 (1968)
Strotz, R.: Myopia and inconsistency in dynamic utility maximization. Rev. Econ. Stud. 23, 165–180 (1955)
Vieille, N., Weibull, J.: Multiple solutions under quasi-exponential discounting. Econ. Theory 39, 513–526 (2009)
Acknowledgements
The authors are greatly indebted to Ivar Ekeland, Ali Lazrak, Traian Pirvu, Suleyman Basak, Mogens Steffensen, Jörgen Weibull, and Eric Böse-Wolf for very helpful comments. A number of very valuable comments from two anonymous referees have helped to improve the paper considerably.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Björk, T., Murgoci, A. A theory of Markovian time-inconsistent stochastic control in discrete time. Finance Stoch 18, 545–592 (2014). https://doi.org/10.1007/s00780-014-0234-y
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00780-014-0234-y
Keywords
- Time consistency
- Time inconsistency
- Time-inconsistent control
- Dynamic programming
- Stochastic control
- Bellman equation
- Hyperbolic discounting
- Mean–variance