Finance and Stochastics

, Volume 18, Issue 3, pp 545–592 | Cite as

A theory of Markovian time-inconsistent stochastic control in discrete time

  • Tomas Björk
  • Agatha Murgoci


We develop a theory for a general class of discrete-time stochastic control problems that, in various ways, are time-inconsistent in the sense that they do not admit a Bellman optimality principle. We attack these problems by viewing them within a game theoretic framework, and we look for subgame perfect Nash equilibrium points. For a general controlled Markov process and a fairly general objective functional, we derive an extension of the standard Bellman equation, in the form of a system of nonlinear equations, for the determination of the equilibrium strategy as well as the equilibrium value function. Most known examples of time-inconsistent stochastic control problems in the literature are easily seen to be special cases of the present theory. We also prove that for every time-inconsistent problem, there exists an associated time-consistent problem such that the optimal control and the optimal value function for the consistent problem coincide with the equilibrium control and value function, respectively for the time-inconsistent problem. To exemplify the theory, we study some concrete examples, such as hyperbolic discounting and mean–variance control.


Time consistency Time inconsistency Time-inconsistent control Dynamic programming Stochastic control Bellman equation Hyperbolic discounting Mean–variance 

Mathematics Subject Classification (2010)

49L20 49L99 60J05 60J20 91A10 91G10 91G80 

JEL Classification

C61 C72 C73 G11 



The authors are greatly indebted to Ivar Ekeland, Ali Lazrak, Traian Pirvu, Suleyman Basak, Mogens Steffensen, Jörgen Weibull, and Eric Böse-Wolf for very helpful comments. A number of very valuable comments from two anonymous referees have helped to improve the paper considerably.


  1. 1.
    Barro, R.: Ramsey meets Laibson in the neoclassical growth model. Q. J. Econ. 114, 1125–1152 (1999) CrossRefMATHGoogle Scholar
  2. 2.
    Basak, S., Chabakauri, G.: Dynamic mean–variance asset allocation. Rev. Financ. Stud. 23, 2970–3016 (2010) CrossRefGoogle Scholar
  3. 3.
    Björk, T., Murgoci, A.: A general theory of Markovian time-inconsistent stochastic control problems. Working paper (2009).
  4. 4.
    Björk, T., Murgoci, A.: A theory of Markovian time-inconsistent stochastic control in continuous time. Submitted, Stockholm School of Economics (2010).
  5. 5.
    Björk, T., Murgoci, A., Zhou, X.-Y.: Mean–variance portfolio optimization with state dependent risk aversion. Math. Finance 24, 1–24 (2014) CrossRefMATHMathSciNetGoogle Scholar
  6. 6.
    Czichowsky, C.: Time-consistent mean–variance portfolio selection in discrete and continuous time. Finance Stoch. 17, 227–271 (2013) CrossRefMATHMathSciNetGoogle Scholar
  7. 7.
    Ekeland, I., Lazrak, A.: Being serious about non-commitment: subgame perfect equilibrium in continuous time. arXiv:math/0604264v1 [math.OC]
  8. 8.
    Ekeland, I., Privu, T.: Investment and consumption without commitment. Math. Financ. Econ. 2, 57–86 (2008) CrossRefMATHMathSciNetGoogle Scholar
  9. 9.
    Goldman, S.: Consistent plans. Rev. Econ. Stud. 47, 533–537 (1980) CrossRefMATHGoogle Scholar
  10. 10.
    Harris, C., Laibson, D.: Dynamic choices of hyperbolic consumers. Econometrica 69, 935–957 (2001) CrossRefMATHMathSciNetGoogle Scholar
  11. 11.
    Krusell, P., Smith, A.: Consumption and savings decisions with quasi-geometric discounting. Econometrica 71, 366–375 (2003) CrossRefGoogle Scholar
  12. 12.
    Kryger, E., Steffensen, M.: Some solvable portfolio problems with quadratic and collective objectives. Working paper (2010).
  13. 13.
    Luttmer, E., Mariotti, T.: Subjective discounting in an exchange economy. J. Polit. Econ. 111, 959–989 (2003) CrossRefGoogle Scholar
  14. 14.
    Peleg, B., Menahem, E.: On the existence of a consistent course of action when tastes are changing. Rev. Econ. Stud. 40, 391–401 (1973) CrossRefMATHGoogle Scholar
  15. 15.
    Pollak, R.: Consistent planning. Rev. Econ. Stud. 35, 185–199 (1968) CrossRefGoogle Scholar
  16. 16.
    Strotz, R.: Myopia and inconsistency in dynamic utility maximization. Rev. Econ. Stud. 23, 165–180 (1955) CrossRefGoogle Scholar
  17. 17.
    Vieille, N., Weibull, J.: Multiple solutions under quasi-exponential discounting. Econ. Theory 39, 513–526 (2009) CrossRefMATHMathSciNetGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2014

Authors and Affiliations

  1. 1.Department of FinanceStockholm School of EconomicsStockholmSweden
  2. 2.Finance DepartmentCopenhagen Business SchoolFrederiksbergDenmark

Personalised recommendations