A theory of Markovian time-inconsistent stochastic control in discrete time
- 871 Downloads
We develop a theory for a general class of discrete-time stochastic control problems that, in various ways, are time-inconsistent in the sense that they do not admit a Bellman optimality principle. We attack these problems by viewing them within a game theoretic framework, and we look for subgame perfect Nash equilibrium points. For a general controlled Markov process and a fairly general objective functional, we derive an extension of the standard Bellman equation, in the form of a system of nonlinear equations, for the determination of the equilibrium strategy as well as the equilibrium value function. Most known examples of time-inconsistent stochastic control problems in the literature are easily seen to be special cases of the present theory. We also prove that for every time-inconsistent problem, there exists an associated time-consistent problem such that the optimal control and the optimal value function for the consistent problem coincide with the equilibrium control and value function, respectively for the time-inconsistent problem. To exemplify the theory, we study some concrete examples, such as hyperbolic discounting and mean–variance control.
KeywordsTime consistency Time inconsistency Time-inconsistent control Dynamic programming Stochastic control Bellman equation Hyperbolic discounting Mean–variance
Mathematics Subject Classification (2010)49L20 49L99 60J05 60J20 91A10 91G10 91G80
JEL ClassificationC61 C72 C73 G11
The authors are greatly indebted to Ivar Ekeland, Ali Lazrak, Traian Pirvu, Suleyman Basak, Mogens Steffensen, Jörgen Weibull, and Eric Böse-Wolf for very helpful comments. A number of very valuable comments from two anonymous referees have helped to improve the paper considerably.
- 3.Björk, T., Murgoci, A.: A general theory of Markovian time-inconsistent stochastic control problems. Working paper (2009). http://www.hhs.se/se/Search/Person/Pages/Person.aspx?PersonID=37
- 4.Björk, T., Murgoci, A.: A theory of Markovian time-inconsistent stochastic control in continuous time. Submitted, Stockholm School of Economics (2010). http://hhs.se/DF/People/Pages/Person.aspx?PersonID=37
- 7.Ekeland, I., Lazrak, A.: Being serious about non-commitment: subgame perfect equilibrium in continuous time. arXiv:math/0604264v1 [math.OC]
- 12.Kryger, E., Steffensen, M.: Some solvable portfolio problems with quadratic and collective objectives. Working paper (2010). http://papers.ssrn.com/sol3/papers.cfm?abstract_id=1577265