On timeinconsistent stochastic control in continuous time
 2.4k Downloads
 34 Citations
Abstract
In this paper, which is a continuation of the discretetime paper (Björk and Murgoci in Finance Stoch. 18:545–592, 2004), we study a class of continuoustime stochastic control problems which, in various ways, are timeinconsistent in the sense that they do not admit a Bellman optimality principle. We study these problems within a gametheoretic framework, and we look for Nash subgame perfect equilibrium points. For a general controlled continuoustime Markov process and a fairly general objective functional, we derive an extension of the standard Hamilton–Jacobi–Bellman equation, in the form of a system of nonlinear equations, for the determination of the equilibrium strategy as well as the equilibrium value function. The main theoretical result is a verification theorem. As an application of the general theory, we study a timeinconsistent linearquadratic regulator. We also present a study of timeinconsistency within the framework of a general equilibrium production economy of Cox–Ingersoll–Ross type (Cox et al. in Econometrica 53:363–384, 1985).
Keywords
Timeconsistency Timeinconsistency Timeinconsistent control Dynamic programming Stochastic control Bellman equation Hyperbolic discounting Meanvariance EquilibriumMathematics Subject Classification
49L99 49N90 60J70 91A10 91A80 91B02 91B25 91B51 91G80JEL Classification
C61 C72 C73 D5 G11 G121 Introduction
The purpose of this paper is to study a class of stochastic control problems in continuous time which have the property of being timeinconsistent in the sense that they do not allow a Bellman optimality principle. As a consequence, the very concept of optimality becomes problematic, since a strategy which is optimal given a specific starting point in time and space may be nonoptimal when viewed from a later date and a different state. In this paper, we attack a fairly general class of timeinconsistent problems by using a gametheoretic approach; so instead of searching for optimal strategies, we search for subgame perfect Nash equilibrium strategies. The paper presents a continuoustime version of the discretetime theory developed in our previous paper [5]. Since we build heavily on the discretetime paper, the reader is referred to that for motivating examples and more detailed discussions on conceptual issues.
1.1 Previous literature
For a detailed discussion of the gametheoretic approach to timeinconsistency using Nash equilibrium points as above, the reader is referred to [5]. A list of some of the most important papers on the subject is given by [2, 6, 8, 9, 10, 11, 12, 13, 14, 16, 18, 19, 20, 21, 22, 23, 24, 25].
All the papers above deal with particular model choices, and different authors use different methods in order to solve the problems. To our knowledge, the present paper, which is the continuoustime part of the working paper [4], is the first attempt to study a reasonably general (albeit Markovian) class of timeinconsistent control problems in continuous time. We should, however, like to stress that for the present paper, we have been greatly inspired by [2, 9, 11].
1.2 Structure of the paper

In Sect. 2, we present the basic setup, and in Sect. 3, we discuss the concept of equilibrium. This replaces in our setting the optimality concept for a standard stochastic control problem, and in Definition 3.4, we give a precise definition of the equilibrium control and the equilibrium value function.

Since the equilibrium concept in continuous time is quite delicate, we build the continuoustime theory on the discretetime theory previously developed in [5]. In Sect. 4, we start to study the continuoustime problem by going to the limit for a discretized problem, and using the results from [5]. This leads to an extension of the standard HJB equation to a system of equations with an embedded static optimization problem. The limiting procedure described above is done in an informal manner. It is largely heuristic, and it thus remains to clarify precisely how the derived extended HJB system is related to the precisely defined equilibrium problem under consideration.

The needed clarification is in fact delivered in Sect. 5. In Theorem 5.2, which is the main theoretical result of the paper, we give a precise statement and proof of a verification theorem. This theorem says that a solution to the extended HJB system does indeed deliver the equilibrium control and equilibrium value function to our original problem.

In Sect. 6, the results of Sect. 5 are extended to a more general reward functional.

Section 7 treats the infinitehorizon case.

In Sect. 8, we study a timeinconsistent version of the linearquadratic regulator to illustrate how the theory works in a concrete case.

Section 9 is devoted to a rather detailed study of a general equilibrium model for a production economy with timeinconsistent preferences.

In Sect. 10, we review some remaining open problems.
2 The model
We now turn to the formal continuoustime theory. In order to present this, we need some input data.
Definition 2.1
 1.
A drift mapping \({\mu}:{\mathbb{R}_{+} \times\mathbb{R}^{n} \times {{\mathbb{R}^{k}}}}\rightarrow{\mathbb{R}^{n}}\).
 2.
A diffusion mapping \({\sigma}:{\mathbb{R}_{+} \times\mathbb{R}^{n} \times{{\mathbb{R}^{k}}}}\rightarrow{M(n,d)}\), where \(M(n,d)\) denotes the set of all \(n \times d\) matrices.
 3.
A control constraint mapping \({U}:{\mathbb{R}_{+} \times \mathbb{R} ^{n} }\rightarrow{2^{\mathbb{R}^{k}}}\).
 4.
A mapping \({F}:{\mathbb{R}^{n} \times\mathbb {R}^{n}}\rightarrow {\mathbb{R}}\).
 5.
A mapping \({G}:{\mathbb{R}^{n} \times\mathbb {R}^{n}}\rightarrow {\mathbb{R}}\).
Definition 2.2
 1.
For each \((t,x)\in[0,T] \times\mathbb{R}^{n}\), we have \({\mathbf{u}}(t,x) \in U(t,x)\).
 2.For each initial point \((s,y)\in[0,T] \times\mathbb {R}^{n}\), the SDEhas a unique strong solution denoted by \(X^{{\mathbf{u}}}\).$$dX_{t}=\mu\big(t,X_{t},{\mathbf{u}}(t,X_{t})\big)dt + \sigma\big(t,X _{t},{\mathbf{u}}(t,X_{t})\big)dW_{t},\quad X_{s}=y $$
We now go on to define the controlled infinitesimal generator of the SDE above. In the present paper, we use the (somewhat nonstandard) convention that the infinitesimal operator acts on the time variable as well as on the space variable; so it includes the term \(\frac{\partial }{\partial t}\).
Definition 2.3
 For any fixed \(u \in{{\mathbb{R}^{k}}}\), the functions \(\mu^{u}\), \(\sigma^{u}\) and \(C^{u}\) are defined by$$\begin{aligned} \mu^{u} (t,x) =&\mu(t,x,u),\quad\sigma^{u} (t,x)=\sigma(t,x,u), \\ C^{u}(t,x) =&\sigma(t,x,u)\sigma(t,x,u)'. \end{aligned}$$
 For any admissible control law \({\mathbf{u}}\), the functions \(\mu^{\mathbf{u}}\), \(\sigma^{\mathbf{u}}\), \(C^{\mathbf{u}}(t,x)\) are defined by$$\begin{aligned} \mu^{\mathbf{u}} (t,x) =&\mu\big(t,x,{\mathbf{u}}(t,x)\big),\quad \sigma^{\mathbf{u}} (t,x)=\sigma\big(t,x,{\mathbf{u}}(t,x)\big), \\ C^{\mathbf{u}}(t,x) =&\sigma\big(t,x,{\mathbf{u}}(t,x)\big)\sigma \big(t,x,{\mathbf{u}}(t,x)\big)'. \end{aligned}$$
 For any fixed \(u \in{{\mathbb{R}^{k}}}\), the operator \({{\mathbf{A}}} ^{u}\) is defined by$${{\mathbf{A}}}^{u}=\frac{\partial}{\partial t} +\sum_{i=1}^{n}\mu _{i}^{u}(t,x)\frac{\partial}{\partial x_{i}}+ \frac{1}{2}\sum_{i,j=1} ^{n}C_{ij}^{u}(t,x)\frac{{\partial}^{2} }{\partial{x_{i}}\partial {x_{j}}}. $$
 For any admissible control law \({\mathbf{u}}\), the operator \({{\mathbf{A}}}^{\mathbf{u}}\) is defined by$${{\mathbf{A}}}^{\mathbf{u}}=\frac{\partial}{\partial t} + \sum_{i=1} ^{n}\mu_{i}^{\mathbf{u}}(t,x)\frac{\partial}{\partial x_{i}}+ \frac{1}{2}\sum_{i,j=1}^{n}C_{ij}^{\mathbf{u}}(t,x)\frac{{\partial} ^{2} }{\partial{x_{i}}\partial{x_{j}}}. $$
3 Problem formulation
In order to formulate our problem, we need an objective functional. We thus consider the two functions \(F\) and \(G\) from Definition 2.1.
Definition 3.1
Remark 3.2
In Sect. 6, we consider a more general reward functional. The restriction to the functional (3.1) above is done in order to minimize the notational complexity of the derivations below, which otherwise would be somewhat messy.
In order to have a nondegenerate problem, we need a formal integrability assumption.
Assumption 3.3
Our objective is loosely that of maximizing \(J(t, x,{\mathbf{u}})\) for each \((t,x)\), but conceptually this turns out to be far from trivial, so instead of optimal controls we will study equilibrium controls. The equilibrium concept is made precise in Definition 3.4 below, but in order to motivate that definition, we need a brief discussion concerning the reward functional above.

The present state \(x\) appears in the function \(F\).

In the second term, we have (even apart from the appearance of the present state \(x\)) a nonlinear function \(G\) operating on the expected value \(E_{t,x}[ X_{T}^{{\mathbf{u}}} ]\).

Consider a noncooperative game where we have one player for each point in time \(t\). We refer to this player as “Player \(t\)”.

For each fixed \(t\), Player \(t\) can only control the process \(X\) exactly at time \(t\). He/she does that by choosing a control function \({\mathbf{u}}(t, \cdot)\); so the action taken at time \(t\) with state \(X_{t}\) is given by \({\mathbf{u}}(t, X_{t})\).

Gluing together the control functions for all players, we thus have a feedback control law \({{\mathbf{u}}}:{[0,T] \times\mathbb{R}^{n}} \rightarrow{\mathbb{R}^{k}}\).
 Given the feedback law \({\mathbf{u}}\), the reward to Player \(t\) is given by the reward functional$$J(t, x,{\mathbf{u}})=E_{t,x}[ F(x,X_{T}^{{\mathbf{u}}}) ] + G( {x, E _{t,x}[ X_{T}^{{\mathbf{u}}} ]} ). $$

If for each \(s > t\), Player \(s\) chooses the control \(\hat{{\mathbf{u}}}(s, \cdot)\), then it is optimal for Player \(t\) to choose \(\hat{{\mathbf{u}}}(t, \cdot)\).
Definition 3.4
We sometimes refer to this as an intrapersonal equilibrium, since it can be viewed as a game between different future manifestations of your own preferences.
Remark 3.5
This is our continuoustime formalization of the corresponding discretetime equilibrium concept.
4 An informal derivation of the extended HJB equation

We discretize (to some extent) the continuoustime problem. We then use our results from discretetime theory to obtain a discretized recursion for \(\hat{{\mathbf{u}}}\), and we then let the time step tend to zero.

In the limit, we obtain our continuoustime extension of the HJB equation. Not surprisingly, it will in fact be a system of equations.

In the discretizing and limiting procedure, we mainly rely on informal heuristic reasoning. In particular, we do not claim that the derivation is rigorous. The derivation is, from a logical point of view, only of motivational value.

In Sect. 5, we then go on to show that our (informally derived) extended HJB equation is in fact the “correct” one, by proving a rigorous verification theorem.
4.1 Deriving the equation

Choose an arbitrary initial point \((t,x)\). Also choose a “small” time increment \(h >0\) and an arbitrary admissible control \({\mathbf{u}}\).
 Define the control law \({\mathbf{u}}_{h}\) on the time interval \([t,T]\) by$${\mathbf{u}}_{h}(s,y)= \left\{ \textstyle\begin{array}{ccl} {\mathbf{u}}(s,y)&& \mbox{for} \ t \leq s < t+h, y \in\mathbb{R}^{n}, \\ \hat{{\mathbf{u}}}(s,y)&& \mbox{for} \ t+h \leq s \leq T, y \in \mathbb{R}^{n}. \end{array}\displaystyle \right. $$
 If now \(h\) is “small enough”, we expect to haveand in the limit as \(h \rightarrow0\), we should have equality if \({\mathbf{u}}(t,x)=\hat{{\mathbf{u}}}(t,x)\).$$J(t,x,{\mathbf{u}}_{h}) \leq J(t,x,\hat{{\mathbf{u}}}), $$
 For any fixed \(y \in\mathbb{R}^{n}\), the mapping \({f^{y}}:{[0,T] \times\mathbb{R}^{n}}\rightarrow{\mathbb{R}}\) is defined by$$f^{y}(t,x)=E_{t,x}[ F( {y,X_{T}^{\hat{{\mathbf{u}}}}} ) ]. $$
 The function \({f}:{[0,T] \times\mathbb{R}^{n} \times\mathbb{R}^{n}} \rightarrow{\mathbb{R}}\) is defined byWe sometimes also, with a slight abuse of notation, denote the entire family of functions \(\left\{ {f^{y}: y\in\mathbb{R}^{n}} \right\} \) by \(f\).$$f(t,x,y)=f^{y}(t,x). $$
 For any function \(k(t,x)\), the operator \({\mathbf{A}}_{h}^{{\mathbf{u}}}\) is defined by$$( {{\mathbf{A}}_{h}^{{\mathbf{u}}}k} )(t,x)= E_{t,x}[ k(t+h, X_{t+h} ^{\mathbf{u}}) ]k(t,x). $$
 The function \({g}:{[0,T] \times\mathbb{R}^{n}}\rightarrow{\mathbb{R} ^{n}}\) is defined by$$g(t,x)=E_{t,x}[ X_{T}^{\hat{{\mathbf{u}}}} ]. $$
 The function \(G \diamond g\) is defined by$$\left( {G \diamond g} \right) (t,x)=G\big( {x,g(t,x)} \big). $$
 The term \({\mathbf{H}}_{h}^{{\mathbf{u}}}g\) is defined by$$( {{\mathbf{H}}_{h}^{{\mathbf{u}}}g} )(t,x)=G\big(x,E_{t,x}[ g(t+h,X _{t+h}^{{\mathbf{u}}}) ]\big) G\big(x,g(t,x)\big). $$
We now divide the above inequality by \(h\) and let \(h\) tend to zero. Then the term coming from the operator \({\mathbf{A}}_{h}^{{\mathbf{u}}}\) converges to the infinitesimal operator \({\mathbf{A}}^{u}\), where \(u= {\mathbf{u}}(t,x)\), but the limit of \(h^{1}( {{\mathbf{H}}_{h}^{ {\mathbf{u}}}g} )(t,x) \) requires closer investigation.
Definition 4.1
 1.The function \(V\) is determined by$$\begin{aligned} \sup_{u \in U(t,x)} \big( ( {{\mathbf{A}}^{u}V} )(t,x)  ( {{\mathbf{A}} ^{u}f} )(t,x,x)+( &{\mathbf{A}}^{u} f^{x})(t,x) \\  {\mathbf{A}}^{u} ( {{G \diamond g}} )(t,x) + ( {{\mathbf{H}}^{u}g} )(t,x) \big) & = 0,\quad0 \leq t \leq T, \\ V(T,x) & = F(x,x)+G(x,x). \end{aligned}$$(4.1)
 2.For every fixed \(y \in\mathbb{R}^{n}\), the function \((t,x) \mapsto f ^{y}(t,x)\) is defined by$$ \textstyle\begin{array}{rcl} {\mathbf{A}}^{\hat{{\mathbf{u}}}}f^{y}(t,x)&=&0,\quad0 \leq t \leq T, \\ f^{y}(T,x)&=&F(y,x). \end{array} $$(4.2)
 3.The function \(g\) is defined by$$ \textstyle\begin{array}{rcl} {\mathbf{A}}^{\hat{{\mathbf{u}}}}g(t,x)&=&0, \quad0 \leq t \leq T, \\ g(T,x)&=&x. \end{array} $$(4.3)

The first point to notice is that we have a system of equations (4.1)–(4.3) for the simultaneous determination of \(V\), \(f\) and \(g\).

In the expressions above, \(\hat{{\mathbf{u}}}\) always denotes the control law which realizes the supremum in the first equation.

In order to solve the \(V\)equation, we need to know \(f\) and \(g\), but these are determined by the equilibrium control law \(\hat{{\mathbf{u}}}\), which in turn is determined by the suppart of the \(V\)equation.
 We have used the notation$$\begin{aligned} f(t,x,y) =&f^{y}(t,x),\quad\left( {G \diamond g} \right) (t,x)=G \big(x,g(t,x)\big), \\ {\mathbf{H}}^{u}g(t,x) =&G_{y}\big(x,g(t,x)\big) {\mathbf{A}}^{u}g(t,x), \quad G_{y}(x,y)=\frac{\partial G}{\partial y}(x,y). \end{aligned}$$

The operator \({\mathbf{A}}^{u}\) only operates on variables within parentheses. So for instance, the expression \(\left( {{\mathbf{A}} ^{u}f} \right) (t,x,x)\) is interpreted as \(\left( {{\mathbf{A}}^{u}h} \right) (t,x)\) with \(h\) defined by \(h(t,x)=f(t,x,x)\). In the expression \(\left( {{\mathbf{A}}^{u}f^{y}} \right) (t,x)\) the operator does not act on the upper case index \(y\), which is viewed as a fixed parameter. Similarly, in the expression \(\left( {{\mathbf{A}}^{u}f ^{x}} \right) (t,x)\), the operator only acts on the variables \(t,x\) within the parentheses, and does not act on the upper case index \(x\).
 If \(F(x,y)\) does not depend on \(x\) and there is no \(G\)term, the problem trivializes to a standard timeconsistent problem. The terms \(\left( {{\mathbf{A}}^{u}f} \right) (t,x,x)+\left( {{\mathbf{A}}^{u}f ^{x}} \right) (t,x)\) in the \(V\)equation cancel, and the system reduces to the standard Bellman equation$$( {{\mathbf{A}}^{u}V} )(t,x)=0,\quad V(T,x)=F(x). $$

We note that the \(g\) function above appears, in a more restricted framework, already in [2, 9, 11].
4.2 Existence and uniqueness
The task of proving existence and/or uniqueness of solutions to the extended HJB system seems (at least to us) to be technically extremely difficult. We have no idea about how to proceed, so we leave it for future research. It is thus very much an open problem. See Sect. 10 for more open problems.
5 A verification theorem
 1.Assume that there exists an equilibrium law \(\hat{{\mathbf {u}}}\) and that \(V\) is the corresponding value function. Assume furthermore that \(V\) is in \(C^{1,2}\). Define \(f^{y}\) and \(g\) by$$\begin{aligned} f^{y}(t,x) =&E_{t,x}[ F(y,X_{T}^{\hat{{\mathbf{u}}}}) ], \end{aligned}$$(5.1)We then conjecture that \(V\) satisfies the extended HJB system and that \(\hat{{\mathbf{u}}}\) realizes the supremum in the equation.$$\begin{aligned} g(t,x) =&E_{t,x}[ X_{T}^{\hat{{\mathbf{u}}}} ]. \end{aligned}$$(5.2)
 2.
Assume that \(V\), \(f\) and \(g\) solve the extended HJB system and that the supremum in the \(V\)equation is attained for every \((t,x)\). We then conjecture that there exists an equilibrium law \(\hat{{\mathbf{u}}}\), and that it is given by the maximizing \(u\) in the \(V\)equation. Furthermore, we conjecture that \(V\) is the corresponding equilibrium value function, and \(f\) and \(g\) allow the interpretations (5.1) and (5.2).
In this paper, we do not attempt to prove the first conjecture. Even for a standard timeconsistent control problem within an SDE framework, it is well known that this is technically quite complicated, and it typically requires the theory of viscosity solutions. It is thus left as an open problem. We shall, however, prove the second conjecture. This obviously has the form of a verification result, and from standard theory, we should expect that it can be proved with a minimum of technical complexity. We now give the precise formulation and proof of the verification theorem, but first we need to define a function space.
Definition 5.1
We can now state and prove the main result of the present paper.
Theorem 5.2
(Verification theorem)
 1.
\(V\), \(f^{y}\) and \(g\) solve the extended HJB system in Definition 4.1.
 2.
\(V(t,x)\) and \(g(t,x)\) are smooth in the sense that they are in \(C^{1,2}\), and \(f(t,x,y)\) is in \(C^{1,2,2}\).
 3.
The function \(\hat{{\mathbf{u}}}\) realizes the supremum in the \(V\)equation, and \(\hat{{\mathbf{u}}}\) is an admissible control law.
 4.
\(V\), \(f^{y}\), \(g\) and \(G \diamond g\) as well as the function \((t,x) \mapsto f(t,x,x)\) all belong to the space \(L_{T}^{2}(X^{ \hat{{\mathbf{u}}}})\).
Proof

We start by showing that \(f\) and \(g\) have the interpretations (5.1) and (5.2) and that \(V\) is the value function corresponding to \(\hat{{\mathbf{u}}}\), i.e., that \(V(t,x)=J(t,x, \hat{{\mathbf{u}}})\).

In the second step, we then prove that \(\hat{{\mathbf{u}}}\) is indeed an equilibrium control law.
6 The general case
Assumption 6.1
The treatment of this case is very similar to the previous one; so we directly give the final result, which is the relevant extended HJB system.
Definition 6.2
 1.The function \(V\) is determined bywith boundary condition$$\begin{aligned} & \sup_{u \in{{\mathbb{R}^{k}}}} \big( ( {{\mathbf{A}}^{u}V} )(t,x) + H(t,x,t,x,u) ({\mathbf{A}}^{u}f)(t,x,t,x)+ ({\mathbf{A}}^{u}f^{tx})(t,x) \\ & \phantom{ \sup_{u \in{{\mathbb{R}^{k}}}} \big(} {\mathbf{A}}^{u} \left( {{G \diamond g}} \right) (t,x) + ( {{\mathbf{H}}^{u}g} )(t,x) \big) = 0, \end{aligned}$$(6.2)$$ V(T,x)=F(T,x,x)+G(T,x,x). $$(6.3)
 2.For each fixed \(s\) and \(y\), the function \(f^{sy}(t,x)\) is defined by$$\begin{aligned} {\mathbf{A}}^{\hat{{\mathbf{u}}}}f^{sy}(t,x)+H\big(s,y,t,x, \hat{{\mathbf{u}}}_{t}(x)\big) =&0, \quad0 \leq t \leq T , \end{aligned}$$(6.4)$$\begin{aligned} f^{sy}(T,x) =&F(s,y,x). \end{aligned}$$(6.5)
 3.The function \(g(t,x)\) is defined by$$\begin{aligned} {\mathbf{A}}^{\hat{{\mathbf{u}}}}g(t,x) =&0,\quad0 \leq t \leq T , \end{aligned}$$(6.6)$$\begin{aligned} g(T,x) =&x. \end{aligned}$$(6.7)
Also for this case, we have a verification theorem. The proof is almost identical to that of Theorem 5.2, so we omit it.
Theorem 6.3
(Verification theorem)
 1.
\(V\), \(f^{sy}\) and \(g\) are a solution to the extended HJB system in Definition 6.2.
 2.
\(V\), \(f^{sy}\) and \(g\) are smooth in the sense that they are in \(C^{1,2}\).
 3.
The function \(\hat{{\mathbf{u}}}\) realizes the supremum in the \(V\)equation, and \(\hat{{\mathbf{u}}}\) is an admissible control law.
 4.
\(V\), \(f^{sy}\), \(g\) and \(G \diamond g\) as well as the function \((t,x) \mapsto f(t,x,t, x)\) all belong to the space \(L_{T}^{2}(X^{ \hat{{\mathbf{u}}}})\).
7 Infinite horizon
8 Example: the timeinconsistent linearquadratic regulator
To illustrate how the theory works in a simple case, we consider a variation of the classical linearquadratic regulator. Other “quadratic” control problems are considered in [2, 6, 8], which study meanvariance problems within the present gametheoretic framework. In the papers [19] and [20], the authors study the meanvariance criterion where you are continuously rolling over instantaneously updated precommitted strategies.
 The value functional for Player \(t\) is given bywhere \(\gamma\) is a positive constant.$$E_{t,x}\left[ \frac{1}{2}\int_{t}^{T}u_{s}^{2}ds \right] + \frac{ \gamma}{2}E_{t,x}[ \left( {X_{T}x} \right) ^{2} ], $$
 The state process \(X\) is scalar with dynamicswhere \(a\), \(b\) and \(\sigma\) are given constants.$$dX_{t}=(aX_{t} + bu_{t})dt+\sigma dW_{t}, $$

The control \(u\) is scalar with no constraints.
Proposition 8.1
Proof
9 Example: a Cox–Ingersoll–Ross production economy with timeinconsistent preferences
In this section, we apply the previously developed theory to a rather detailed study of a general equilibrium model for a production economy with timeinconsistent preferences. The model under consideration is a timeinconsistent analogue of the classic Cox–Ingersoll–Ross model in [7]. Our main objective is to investigate the structure of the equilibrium short rate, the equilibrium Girsanov kernel, and the equilibrium stochastic discount factor.
There are a few earlier papers on equilibrium with timeinconsistent preferences. In [1] and [17], the authors study continuoustime equilibrium models of a particular type of timeinconsistency, namely nonexponential discounting. While [1] considers a deterministic neoclassical model of economic growth, [17] analyze general equilibrium in a stochastic endowment economy.
Our present study is much inspired by the earlier paper [15] which in very great detail studies equilibrium in a very general setting of an endowment economy with dynamically inconsistent preferences that are not limited to the particular case of nonexponential discounting.
Unlike the papers mentioned above, which all studied endowment models, we study a stochastic production economy of Cox–Ingersoll–Ross type.
9.1 The model
We start with some formal assumptions concerning the production technology.
Assumption 9.1
From a purely formal point of view, investment in the technology \(S\) is equivalent to investing in a risky asset with price process \(S\), with the constraint that short selling is not allowed.
We also need a riskfree asset, and this is provided by the next assumption.
Assumption 9.2
Assumption 9.3
9.2 Equilibrium definitions

intrapersonal equilibrium;

market equilibrium.
9.2.1 Intrapersonal equilibrium
9.2.2 Market equilibrium
By a market equilibrium, we mean a situation where the agent follows an intrapersonal equilibrium strategy and the market clears for the riskfree asset. The formal definition is as follows.
Definition 9.4
 1.
Given the riskfree rate of the functional form \(r(t,x)\), the intrapersonal equilibrium consumption and investment are given by \(\hat{c}\) and \(\hat{u}\) respectively.
 2.The market clears for the riskfree asset, i.e.,$$\hat{u}(t,x)\equiv1. $$
9.3 Main goals of the study
9.4 The extended HJB equation
In order to determine the intrapersonal equilibrium, we use the results from Sect. 6. At this level of generality, we are not able to provide a priori conditions on the model which guarantee that the conditions of the verification theorem are satisfied. For the general theory below (but of course not for the concrete example presented later), we are thus forced to make the following ad hoc assumption.
Assumption 9.5
We assume that the model under study is such that the verification theorem is in force.
This assumption of course has to be checked for every concrete application. In Sect. 9.8.2, we consider the example of power utility and for this case, we can in fact prove that the assumption is satisfied.
9.5 Determining market equilibrium
In order to determine the market equilibrium, we use the equilibrium condition \(\hat{u}=1\). Plugging this into (9.2), we immediately obtain our first result.
Proposition 9.6
 The equilibrium short rate is given by$$ r(t,x)=\alpha+ \sigma^{2} \frac{xf_{xx}(t,x,t,x)}{f_{x}(t,x,t,x)}. $$(9.3)
 The equilibrium Girsanov kernel \({\varphi}\) is given by$$ {\varphi}(t,x)=\sigma\frac{xf_{xx}(t,x,t,x)}{f_{x}(t,x,t,x)}. $$(9.4)
 The extended equilibrium HJB system has the form$$ \textstyle\begin{array}{rcl} \displaystyle U(t,x,t,\hat{c}) +f_{t}+ ( {\alpha x \hat{c}} )f_{x}+\frac{1}{2}x ^{2} \sigma^{2} f_{xx}&=&0, \\ {\mathbf{A}}^{\hat{{\mathbf{c}}}}f^{sy}(t,x)+U\left( {s,y,t,\hat{c}(t,x)} \right) &=&0. \end{array} $$(9.5)
 The equilibrium consumption \(\hat{c}\) is determined by the first order condition$$U_{c}(t,x,t,\hat{c})=f_{x}(t,x,t,x). $$
 The term \({\mathbf{A}}^{\hat{{\mathbf{c}}}}f^{tx}(t,x)\) is given by$${\mathbf{A}}^{\hat{{\mathbf{c}}}}f^{tx}(t,x)=f_{t}+ x\left( {\alphar} \right) f_{x}+ (rx\hat{c})f_{x}+\frac{1}{2}x^{2} \sigma^{2} f_{xx}. $$
 The equilibrium dynamics for \(X\) are given by$$dX_{t}=( {\alpha X_{t} \hat{c}_{t}} )dt + X_{t}\sigma dW_{t}. $$
9.6 Recap of standard results
9.7 The stochastic discount factor
From Proposition 9.6, we know \(r\) and \({\varphi}\), so in principle we have in fact already determined \(M\); but we now want to investigate the relation between \(M\), the direct utility function \(U\), and the indirect utility function \(f\) in the extended HJB equation.
9.7.1 A representation formula for \(M\)
Definition 9.7
Theorem 9.8
Remark 9.9
Note again that the operator \(\partial_{tx}\) in (9.13) only acts on the first occurrence of \(t\) and \(X_{t}\) in \(f_{x} \left( {t,X_{t},t, X_{t}} \right) \), whereas the operator \(d\) acts on the entire process \(t \mapsto f_{x}\left( {t,X_{t},t, X_{t}} \right) \).
Proof
of Theorem 9.8 Formulas (9.12) and (9.13) follow from (9.11) and the first order condition \(U_{c}( {t,X_{t},t, \hat{c}_{t}} ) =f_{x} ( {t,X_{t},t, X_{t}} )\). It thus remains to prove (9.11).
9.8 Production economy with nonexponential discounting
9.8.1 Generalities
Proposition 9.10
 The equilibrium short rate is given by$$r(x)=\alpha+ \sigma^{2} \frac{xV_{xx}(x)}{V_{x}(x)}. $$
 The equilibrium Girsanov kernel \({\varphi}\) is given by$${\varphi}(x)=\sigma\frac{xV_{xx}(x)}{V_{x}(x)}. $$
 The extended equilibrium HJB system has the form$$\begin{aligned} U(\hat{c}) +g_{t}(0,x)+ ( {\alpha x \hat{c}} )g_{x}(0,x)+\frac{1}{2}x ^{2} \sigma^{2} g_{xx}(0,x) =&0, \\ {\mathbf{A}}^{\hat{{\mathbf{c}}}}g(t,x)+\beta(t)U\big( {\hat{c}(x)} \big) =&0. \end{aligned}$$
 The function \(g\) has the representation$$g(t,x)=E_{0,x}\left[ \int_{0}^{\infty}\beta(t+ s)U( {\hat{c}_{s}} )ds \right] . $$
 The equilibrium consumption \(\hat{c}\) is determined by the first order condition$$ U_{c}(\hat{c})=g_{x}(0,x). $$(9.17)
 The term \({\mathbf{A}}^{\hat{{\mathbf{c}}}}g(t,x)\) is given by$$ {\mathbf{A}}^{\hat{{\mathbf{c}}}}g(t,x)=g_{t}(t,x)+ \left( {\alpha x  \hat{c}(x)} \right) g_{x}(t,x)+ \frac{1}{2}x^{2} \sigma^{2} g_{xx}(t,x). $$
 The equilibrium dynamics of \(X\) are given by$$ dX_{t}=( {\alpha X_{t} \hat{c}_{t}} )dt + X_{t}\sigma dW_{t}. $$(9.18)
We see that the short rate \(r\) and the Girsanov kernel \({\varphi}\) have exactly the same structural form as the standard case formulas (9.6) and (9.7). We now move to the stochastic discount factor and after some calculations, we have the following version of Theorem 9.8.
Proposition 9.11
9.8.2 Power utility
9.8.3 Checking the verification theorem conditions for power utility
From this example with nonexponential discounting, we see that the riskfree rate and Girsanov kernel only depend on the production opportunities in the economy. These objects are unaffected by the timeinconsistency stemming from nonexponential discounting. The equilibrium consumption, however, is determined by the discounting function of the representative agent.
10 Conclusion and open problems

A theorem proving convergence of the discretetime theory to the continuoustime limit. For the quadratic case, this is done in [8], but the general problem is open.

An open and difficult problem is to provide conditions on primitives which guarantee that the functions \(V\) and \(f\) are regular enough to satisfy the extended HJB system.

A related (hard) open problem is to prove existence and/or uniqueness for solutions of the extended HJB system.

Another related problem is to give conditions on primitives which guarantee that the assumptions of the verification theorem are satisfied.

The present theory depends critically on the Markovian structure. It would be interesting to see what can be done without this assumption.
Notes
Acknowledgements
The authors are greatly indebted to the Associate Editor, two anonymous referees, Ivar Ekeland, Ali Lazrak, Martin Schweizer, Traian Pirvu, Suleyman Basak, Mogens Steffensen, and Eric BöseWolf for very helpful comments.
References
 1.Barro, R.: Ramsey meets Laibson in the neoclassical growth model. Q. J. Econ. 114, 1125–1152 (1999) CrossRefzbMATHGoogle Scholar
 2.Basak, S., Chabakauri, G.: Dynamic meanvariance asset allocation. Rev. Financ. Stud. 23, 2970–3016 (2010) CrossRefGoogle Scholar
 3.Björk, T., Khapko, M., Murgoci, A.: Time inconsistent stochastic control in continuous time: Theory and examples. Working paper (2016). Available online at http://arxiv.org/abs/1612.03650
 4.Björk, T., Murgoci, A.: A general theory of Markovian time inconsistent stochastic control problems. Working paper (2010). Available online at https://ssrn.com/abstract=1694759
 5.Björk, T., Murgoci, A.: A theory of Markovian timeinconsistent stochastic control in discrete time. Finance Stoch. 18, 545–592 (2014) MathSciNetCrossRefzbMATHGoogle Scholar
 6.Björk, T., Murgoci, A., Zhou, X.Y.: Meanvariance portfolio optimization with state dependent risk aversion. Math. Finance 24, 1–24 (2014) MathSciNetCrossRefzbMATHGoogle Scholar
 7.Cox, J., Ingersoll, J., Ross, S.: An intertemporal general equilibrium model of asset prices. Econometrica 53, 363–384 (1985) MathSciNetCrossRefzbMATHGoogle Scholar
 8.Czichowsky, C.: Timeconsistent meanvariance portfolio selection in discrete and continuous time. Finance Stoch. 17, 227–271 (2013) MathSciNetCrossRefzbMATHGoogle Scholar
 9.Ekeland, I., Lazrak, A.: The golden rule when preferences are time inconsistent. Math. Financ. Econ. 4, 29–55 (2010) MathSciNetCrossRefzbMATHGoogle Scholar
 10.Ekeland, I., Mbodji, O., Pirvu, T.A.: Timeconsistent portfolio management. SIAM J. Financ. Math. 3, 1–32 (2010) MathSciNetCrossRefzbMATHGoogle Scholar
 11.Ekeland, I., Pirvu, T.A.: Investment and consumption without commitment. Math. Financ. Econ. 2, 57–86 (2008) MathSciNetCrossRefzbMATHGoogle Scholar
 12.Goldman, S.: Consistent plans. Rev. Econ. Stud. 47, 533–537 (1980) CrossRefzbMATHGoogle Scholar
 13.Harris, C., Laibson, D.: Dynamic choices of hyperbolic consumers. Econometrica 69, 935–957 (2001) MathSciNetCrossRefzbMATHGoogle Scholar
 14.Harris, C., Laibson, D.: Instantaneous gratification. Q. J. Econ. 128, 205–248 (2013) CrossRefGoogle Scholar
 15.Khapko, M.: Asset pricing with dynamically inconsistent agents. Working paper (2015). Available online at https://ssrn.com/abstract=2854526
 16.Krusell, P., Smith, A.: Consumption and savings decisions with quasigeometric discounting. Econometrica 71, 366–375 (2003) CrossRefzbMATHGoogle Scholar
 17.Luttmer, E., Mariotti, T.: Subjective discounting in an exchange economy. J. Polit. Econ. 111, 959–989 (2003) CrossRefGoogle Scholar
 18.Marín Solano, J., Navas, J.: Consumption and portfolio rules for timeinconsistent investors. Eur. J. Oper. Res. 201, 860–872 (2010) MathSciNetCrossRefzbMATHGoogle Scholar
 19.Pedersen, J.L., Peskir, G.: Optimal meanvariance portfolio selection. Math. Financ. Econ. 11, 137–160 (2017) MathSciNetCrossRefzbMATHGoogle Scholar
 20.Pedersen, J.L., Peskir, G.: Optimal meanvariance selling strategies. Math. Financ. Econ. 10, 203–220 (2016) MathSciNetCrossRefzbMATHGoogle Scholar
 21.Peleg, B., Menahem, E.: On the existence of a consistent course of action when tastes are changing. Rev. Econ. Stud. 40, 391–401 (1973) CrossRefzbMATHGoogle Scholar
 22.Pirvu, T.A., Zhang, H.: Investmentconsumption with regimeswitching discount rates. Math. Soc. Sci. 71, 142–150 (2014) MathSciNetCrossRefzbMATHGoogle Scholar
 23.Pollak, R.: Consistent planning. Rev. Econ. Stud. 35, 185–199 (1968) CrossRefGoogle Scholar
 24.Strotz, R.: Myopia and inconsistency in dynamic utility maximization. Rev. Econ. Stud. 23, 165–180 (1955) CrossRefGoogle Scholar
 25.Vieille, N., Weibull, J.: Multiple solutions under quasiexponential discounting. Econ. Theory 39, 513–526 (2009) MathSciNetCrossRefzbMATHGoogle Scholar
Copyright information
Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.