Abstract
This paper presents three versions of maximum principle for a stochastic optimal control problem of Markov regime-switching forward–backward stochastic differential equations with jumps. First, a general sufficient maximum principle for optimal control for a system, driven by a Markov regime-switching forward–backward jump–diffusion model, is developed. In the regime-switching case, it might happen that the associated Hamiltonian is not concave and hence the classical maximum principle cannot be applied. Hence, an equivalent type maximum principle is introduced and proved. In view of solving an optimal control problem when the Hamiltonian is not concave, we use a third approach based on Malliavin calculus to derive a general stochastic maximum principle. This approach also enables us to derive an explicit solution of a control problem when the concavity assumption is not satisfied. In addition, the framework we propose allows us to apply our results to solve a recursive utility maximization problem.
Similar content being viewed by others
1 Introduction
The optimal control problem for regime-switching models has received a lot of attention recently, see, for example, [1,2,3,4,5]. There are two existing approaches to solving stochastic optimal control problem in the literature: the dynamic programming and the stochastic maximum principle. We refer the reader to [6,7,8] and references therein for more information on the dynamic programming approach. The stochastic maximum principle is a generalization of the Pontryagin maximum principle, in which optimizing a value function corresponds to optimizing a functional called Hamiltonian. The stochastic maximum principle is presented in terms of an adjoint equation, which is a solution to a backward stochastic differential equation (BSDE). There is a vast literature on stochastic maximum principle, and the reader may consult [7, 9,10,11,12,13] for more information. Some applications of stochastic maximum principle in finance include: the mean–variance portfolio selection (see, e.g. [7, 12] and references therein) and the utility maximization (classical and recursive) or risk minimization (see, e.g. [12, 14, 15]);
The stochastic maximum principle for regime-switching models was introduced in [1, 2] for Markov regime-switching diffusion systems and extended in [5] for Markov regime-switching jump–diffusion systems. In both cases, the authors developed a sufficient stochastic maximum principle. However, when solving the sufficient maximum principle, one of the main assumption is the concavity of the Hamiltonian which may be violated in some applications. In [3], the authors prove a weak sufficient and necessary maximum principle (that does not require concavity assumption) for Markov regime-switching diffusion systems. Let us mention that [5, Theorem 3.1] does not include the cases in which the profit Hamiltonian is not concave (see, e.g. Sect. 5.1).
This paper discusses a partial information stochastic maximum principle for optimal control of forward–backward stochastic differential equation (FBSDE) driven by Markov regime-switching jump–diffusion process. We first prove a general sufficient maximum principle for optimal control with partial information (Theorem 3.1). This can be seen as a generalization of [5, Theorem 3.1] to the FBSDE setting and [15, Theorem 2.3] to the regime-switching setting. Second, we prove a stochastic maximum principle that does not require a concavity condition (Theorem 3.2). In fact, we prove the following: a critical point of the performance functional of a partial information FBSDE problem is a conditional critical point for the associated Hamiltonian, and vice versa. The proof of Theorem 3.2 requires the use of some variational equations (compare with [16, Section 4]), and the maximum principle obtained is of a local form. One of the drawbacks of the two preceding maximum principles is, the need of an assumption on existence and uniqueness of the solution to the BSDE characterizing the adjoint processes. These equations are usually hard to solve explicitly in the partial information case and worse, may not have a solution. Therefore, a stochastic maximum principle via Malliavin calculus is proposed to overcome this problem. In this approach, the adjoint processes are given in terms of the coefficients of the system and their Malliavin derivatives and not by a BSDE. The Malliavin calculus approach was introduced in [17] and further developed in [18, 19], where the authors study optimal control in the presence of additional information. Let us mention that the latter works do not consider systems of forward–backward stochastic differential equations nor the presence of an external Markov chain in the coefficients of the systems. Using the aforementioned technique, the results obtained in [3, Example 4.7] can be extended to the jump–diffusion case. Our results also generalize the ones derived in [4].
One of the motivations of this paper is the problem of stochastic differential utility (SDU) maximization of terminal wealth under Markov switching. The notion of recursive utility in discrete time was introduced in [20, 21] in order to separate the risk aversion and intertemporal substitution aversion of a decision maker. This concept was generalized to that of stochastic differential utility (SDU) in [22]. The cost function in the stochastic differential utility depends on an intermediate consumption rate and a future utility and can be expressed as a BSDE. For more information on maximization of SDU, the reader may consult [14, 15, 23, 24] and references therein.
The paper is organized as follows: In Sect. 2, the framework for the partial information control problem is introduced. Section 3 presents a partial information sufficient maximum principle for a forward–backward stochastic differential equation (FBSDE) driven by a Markov switching jump–diffusion process. An equivalent maximum principle is also given, and we end the section by presenting the Malliavin calculus approach. In Sect. 4, we prove the main results. Section 5 uses the results obtained to solve a problem of optimal control for Markov switching jump–diffusion model. A problem of recursive utility maximization with Markov regime switching is also studied.
2 Framework
This section presents the model and formulates the stochastic control problem for a Markov regime-switching forward–backward SDE with jumps. The model in [5] is adopted for the forward Markov regime-switching jump diffusion.
Let \(({\varOmega },{\mathscr {F}},P)\) be a complete probability space, where P is a reference probability measure. On this probability space, we assume that we are given a one-dimensional Brownian motion \(B=\{B(t)\}_{0\le t\le T}\), an irreducible homogeneous continuous-time, finite state space Markov chain \(\alpha :=\{\alpha (t)\}_{0\le t\le T}\) and an independent Poisson random measure \(N(\mathrm {d}\zeta ,\mathrm {d}s)\) on \(({\mathbb {R}}_+\times {\mathbb {R}}_0,\mathscr {B}({\mathbb {R}}_+)\otimes \mathscr {B}_0)\) under P. Here \({\mathbb {R}}_0={\mathbb {R}} \backslash \{0\}\) and \(\mathscr {B}_0\) is the Borel \(\sigma \)-algebra generated by open subset O of \({\mathbb {R}}_0\).
We suppose that the filtration \({\mathbb {F}}=\{{\mathscr {F}}_t\}_{0\le t\le T}\) is the P-augmented natural filtration generated by B, N and \(\alpha \) (see, e.g. [1, Section 2] or [25, Page 369]).
\(\alpha :=\{\alpha (t)\}_{0\le t\le T}\) is an irreducible homogeneous continuous-time Markov chain with a finite state space \({\mathbb {S}}=\{e_1,e_2, \ldots ,e_D\}\subset {\mathbb {R}}^D\), where \(D\in {\mathbb {N}}\), and the jth component of \(e_i\) is the Kronecker delta \(\delta _{ij}\) for each \(i,j=1,\ldots , D\). The Markov chain is characterized by a rate (or intensity) matrix \({\varLambda }:=\{\lambda _{ij}:1\le i,j\le D\}\) under P. Note that, for each \(1\le i,j\le D,\,\,\lambda _{ij}\) is the constant transition intensity of the chain from state \(e_i\) to state \(e_j\) at time t. In addition for \(i\ne j,\,\,\lambda _{ij}\ge 0\) and \(\sum _{j=1}^D \lambda _{ij}=0\), therefore \(\lambda _{ii}\le 0\). It follows from [26] that the dynamics of the semi-martingale \(\alpha \) is given as follows
where \(M:=\{M(t)\}_{t\in [0,T]}\) is a \({\mathbb {R}}^D\)-valued \(({\mathbb {F}},P)\)-martingale and \({\varLambda }^T\) is the transpose of the matrix \({\varLambda }\). Let us now present the set of jump martingales associated with the Markov chain \(\alpha \), see, for example, [5] or [26] for more information. For each \(1\le i,j\le D\), with \(i\ne j\), and \(t\in [0,T]\), let \(J^{ij}(t)\) be the number of jumps from state \(e_i\) to state \(e_j\) up to time t. It follows from [26] that
with \(m_{ij}:=\{m_{ij}(t)\}_{t\in [0,T]}\), where \(m_{ij}(t):=\int _0^t\langle \alpha (s-),e_i\rangle \langle \mathrm {d}M(s),e_j\rangle \) is a \(({\mathbb {F}},P)\)-martingale.
Fix \(j\in \{1,2,\ldots ,D\}\) and let \({\varPhi }_j(t)\) be the number of jumps into state \(e_j\) up to time t. Then
where \(\widetilde{{\varPhi }}_{j}(t)=\sum _{i=1,i\ne j}^D m_{ij}(t)\) and \(\lambda _j(t)=\sum _{i=1,i\ne j}^D \lambda _{ij}\int _0^t\langle \alpha (s-),e_i\rangle \mathrm {d}s \). It is worth mentioning that for each \(j\in \{1,2,\ldots ,D\},\,\,\,\widetilde{{\varPhi }}_{j}:=\{\widetilde{{\varPhi }}_{j}(t)\}_{t\in [0,T]}\) is a \(({\mathbb {F}},P)\)-martingale.
Assume that the compensator of \(N(\mathrm {d}\zeta ,\mathrm {d}s)\) is defined by
where \(\eta (\mathrm {d}s)\) is a \(\sigma \)-finite measure on \({\mathbb {R}}_+\) and
\(\nu (\mathrm {d}\zeta |s):=(\nu _{e_1}(\mathrm {d}\zeta |s),\nu _{e_2}(\mathrm {d}\zeta |s),\ldots ,\nu _{e_D}(\mathrm {d}\zeta |t))\in {\mathbb {R}}^D\) is a function of s. Let us observe that for each \(j=1,\ldots ,D\), \(\nu _{e_j}(\mathrm {d}\zeta |s)=\nu _j(\mathrm {d}\zeta |s) \) is the conditional Lévy density of jump sizes of \(N(\mathrm {d}\zeta ,\mathrm {d}s)\) at time s when \(\alpha (s^-)=e_j\) and satisfies \(\int _{{\mathbb {R}}_{0}}\min (1,\zeta ^{2})\nu _j (\mathrm {d}\zeta |s)< \infty \). In this work, we further suppose that \(\eta (\mathrm {d}s)=\mathrm {d}s\) and \(\nu (\mathrm {d}\zeta |s)\) is a function of \(\zeta \), that is,
Let
be the compensated Markov regime-switching Poisson random measure.
Suppose that the state process \(X(t)=X^{(u)}(t,\omega );\,\,0 \le t \le T,\,\omega \in {\varOmega }\) is a controlled Markov regime-switching jump diffusion of the form
where \(T>0\) is a given constant. \(u(\cdot )\) is the control process.
The functions \(b:[0,T]\times {\mathbb {R}} \times {\mathbb {S}} \times {\mathscr {U}} \times {\varOmega }\rightarrow {\mathbb {R}}\,,\,\, \sigma :[0,T]\times {\mathbb {R}} \times {\mathbb {S}} \times {\mathscr {U}} \times {\varOmega }\rightarrow {\mathbb {R}}\),
\(\gamma :[0,T]\times {\mathbb {R}} \times {\mathbb {S}} \times {\mathscr {U}} \times {\mathbb {R}}_0\times {\varOmega }\rightarrow {\mathbb {R}}\) and \(\eta :[0,T]\times {\mathbb {R}} \times {\mathbb {S}} \times {\mathscr {U}} \times {\varOmega }\rightarrow {\mathbb {R}}\) are given such that for all \(t,\,\,\,b(t,x,e_i,u,\cdot )\), \( \sigma (t,x,e_i,u,\cdot )\), \(\gamma (t,x,e_i,u,z,\cdot )\) and \( \eta (t,x,e_i,u,\cdot )\) are \({\mathbb {F}}\)-progressively measurable for all \(x \in {\mathbb {R}},\,\,\,e_i \in {\mathbb {S}},\,\,u \in {\mathscr {U}}\) and \(z \in {\mathbb {R}}_0\).
We suppose that we are given a subfiltration
representing the information available to the controller at time t. A possible subfiltration \({\mathscr {E}}_{t}\) in (7) is the \( \delta \)-delayed information given by \( {\mathscr {E}}_{t}={\mathscr {F}}_{(t-\delta )^+ };\,\,\,t\ge 0,\) where \(\delta \ge 0\) is a known constant delay.
We consider the associated BSDE’s in the unknowns \(\Big (Y(t), Z(t), K(t,\zeta ), V(t)\Big )\) of the form
where \(g:[0,T]\times {\mathbb {R}} \times {\mathbb {S}} \times {\mathbb {R}} \times {\mathbb {R}} \times \mathscr {R} \times {\mathbb {R}} \times {\mathscr {U}} \times {\varOmega }\rightarrow {\mathbb {R}}\) and \(h:{\mathbb {R}}\times {\mathbb {S}} \rightarrow {\mathbb {R}}\) are such that the BSDE (8) has a unique solution. As for sufficient conditions for existence and uniqueness of Markov regime-switching BSDEs, we refer the reader, for example, to [27,28,29] and references therein.
Let \(f:[0,T]\times {\mathbb {R}} \times {\mathbb {S}} \times {\mathbb {R}} \times {\mathbb {R}}\times \mathscr {R} \times {\mathbb {R}} \times {\mathscr {U}} \times {\varOmega }\rightarrow {\mathbb {R}}, \,\,\,\varphi :{\mathbb {R}} \times {\mathbb {S}}\rightarrow {\mathbb {R}}\) and \(\psi :{\mathbb {R}} \rightarrow {\mathbb {R}}\) be given \(C^1\) functions with respect to their arguments. Assume that the performance functional is as follows
Here, \(f,\,\varphi \) and \(\psi \) represent profit rates, bequest functions and “utility evaluations”, respectively, of the controller.
Let \({\mathscr {A}}_{{\mathscr {E}}}\) denote the family of admissible control u, such that they are contained in the set of \({\mathscr {E}}_t\)-predictable control, and systems (6)–(8) have a unique solution, and
The set \({\mathscr {U}}\subset {\mathbb {R}}\) is a given convex set such that \(u(t)\in {\mathscr {U}}\) for all \(t\in [0,T]\) a.s., for all \(u \in {\mathscr {A}}_{{\mathscr {E}}}\).
Remark 2.1
Systems (6)–(8) are a semi-coupled forward–backward stochastic differential equations (SDEs). Under globally Lipschitz continuity and linear growth condition of the coefficients, there exists a unique strong solution to the SDE (6). Therefore, existence and uniqueness of the solution to systems (6)–(8) will follow from the existence and uniqueness of the BSDE (8).
The problem we consider is the following: find \(u^*\in {\mathscr {A}}_{{\mathscr {E}}} \) such that
3 Maximum Principle for a Markov Regime-Switching Forward–Backward Stochastic Differential Equation with Jumps
In this section, we derive a general sufficient stochastic maximum principle for a forward–backward Markov regime-switching jump–diffusion model. After, we derive an equivalent type maximum principle.
For this purposes, define the Hamiltonian
by
where \(\mathscr {R} \) denotes the set of all functions \(k:[0,T]\times {\mathbb {R}}_0 \rightarrow {\mathbb {R}}\) for which the integral in (11) converges.
We suppose that H is Fréchet differentiable in the variables x, y, z, k, v, u and that \(\nabla _k H(t,\zeta )\) is a random measure, which is absolutely continuous with respect to \(\nu _\alpha \). This happens, for example, when f and g are “quasi-strong generator”, that is,
where \({\varPsi }\) is predictable and satisfies \(C_1\min (1,|\zeta |)\le {\varPsi }(t,\zeta )\le C_2\min (1,|\zeta |)\) P-a.e. In addition, the constants \(C_1\) and \(C_2\) are such that: \(C_2\ge 0\) and \(C_1\in ]-1,0]\). Letting \(\tilde{k}=\int _{{\mathbb {R}}_0}k(\zeta ){\varPsi }(t,\zeta )\nu _i(\mathrm {d}\zeta )\) on the right-hand side, one can show that
Next, define the adjoint processes \(A(t),\,p(t),\,q(t), r(t,\cdot )\) and \(w(t),\,\,\,t\in [0,T]\) associated with these Hamiltonians by the following system of Markov regime-switching FBSDEJs:
-
1.
Forward SDE in A(t):
$$\begin{aligned} \mathrm {d}A (t)= & {} \frac{\partial H}{\partial y} (t) \,\mathrm {d}t + \frac{\partial H}{\partial z} (t) \mathrm {d}B(t)+ \displaystyle \int _{{\mathbb {R}}_0} \frac{\mathrm {d}\nabla _k H}{\mathrm {d}\nu _\alpha (\zeta )} (t,\zeta ) \,\widetilde{N}_\alpha (\mathrm {d}\zeta ,\mathrm {d}t)\nonumber \\&+ \nabla _vH(t)\cdot \mathrm {d}\widetilde{{\varPhi }}(t);\,\,\,t\in [0,T] \nonumber \\ A(0)= & {} \psi ^\prime (Y(0)). \end{aligned}$$(12)Here and in the sequel, we use the notation
$$\begin{aligned} \frac{\partial H}{\partial y} (t)= & {} \frac{\partial H}{\partial y} (t,X(t),\alpha (t),u(t),Y(t), Z(t), K(t,\cdot ),V(t),A(t),p(t),\\&q(t),r(t,\cdot ),w(t)), \end{aligned}$$etc., \(\frac{\mathrm {d}\nabla _k H}{\mathrm {d}\nu _\alpha (\zeta )} (t,\zeta ) \) is the Radon–Nikodym derivative of \( \nabla _k H(t,\zeta )\) with respect to \(\nu _\alpha (\zeta )\) and \(\nabla _vH(t)\cdot \mathrm {d}\widetilde{{\varPhi }}(t)=\sum _{j=1}^D \frac{\partial H}{\partial v^j} (t)\mathrm {d}\widetilde{{\varPhi }}_j(t)\), with \(V^j=V(t,e_j)\).
-
2.
The Markov regime-switching BSDE in \((p(t),q(t),r(t,\cdot ),w(t))\):
$$\begin{aligned} \mathrm {d}p (t)= & {} - \frac{\partial H}{\partial x} (t) \mathrm {d}t + q(t)\,\mathrm {d}B(t)+ \displaystyle \int _{{\mathbb {R}}_0} r (t,\zeta ) \, \widetilde{N}_\alpha (\mathrm {d}\zeta ,\mathrm {d}t)+ w(t)\cdot \mathrm {d}\widetilde{{\varPhi }}(t);\,\,\,t\in [0,T] \nonumber \\ p (T)= & {} \frac{\partial \varphi }{\partial x}(X(T),\alpha (T))\, +A(T) \frac{\partial h}{\partial x} (X(T),\alpha (T)). \end{aligned}$$(13)
Remark 3.1
Let V be an open subset of a Banach space \({\mathscr {X}}\) and let \(F: V \rightarrow {\mathbb {R}}\).
-
We say that F has a directional derivative (or Gâteaux derivative) at \(x\in V\) in the direction \(y\in {\mathscr {X}}\), if
$$\begin{aligned} D_yF(x):=\underset{\varepsilon \rightarrow 0}{\lim } \frac{1}{\varepsilon }(F(x + \varepsilon y)-F(x)) \text { exists.} \end{aligned}$$ -
We say that F is Fréchet differentiable at \(x \in V\), if there exists a linear map
$$\begin{aligned} L:{\mathscr {X}} \rightarrow {\mathbb {R}}, \end{aligned}$$such that
$$\begin{aligned} \underset{\underset{h \in {\mathscr {X}}}{h \rightarrow 0}}{\lim } \frac{1}{\Vert h\Vert }|F(x+h)-F(x)-L(h)|=0. \end{aligned}$$In this case, we call L the Fréchet derivative of F at x, and we write
$$\begin{aligned} L=\nabla _x F. \end{aligned}$$ -
If F is Fréchet differentiable, then F has a directional derivative in all directions \(y \in {\mathscr {X}}\) and
$$\begin{aligned} D_yF(x)= \nabla _x F(y). \end{aligned}$$
3.1 A Sufficient Maximum Principle
In what follows, we give the sufficient maximum principle.
Theorem 3.1
(Sufficient maximum principle) Let \(\widehat{u}\in {\mathscr {A}}_{{\mathscr {E}} }\) with corresponding solutions \(\widehat{X}(t),(\widehat{Y}(t), \widehat{Z}(t), \widehat{K}(t,\zeta ), \widehat{V}(t)), \widehat{A}(t),(\widehat{p}(t),\widehat{q}(t),\widehat{r}(t,\zeta ),\widehat{w}(t))\) of (6), (8), (12) and (13), respectively. Suppose that the following are true:
-
1.
For each \(e_i\in {\mathbb {S}}\), the functions
$$\begin{aligned} x \mapsto h(x, e_i),\,\,x \mapsto \varphi (x, e_i),\,\,y \mapsto \psi (y) \end{aligned}$$(14)are concave.
-
2.
The function
$$\begin{aligned}&\widetilde{H}(x,y,z,k,v)\nonumber \\&\quad ={{\mathrm{ess \text { } sup}}}_{u\in {\mathscr {U}} }E\Big [H( t,x,e_i,y,z,k,v,u,\widehat{a},\widehat{p}(t),\widehat{q}(t),\widehat{r}(t,\cdot ), \widehat{w}(t))| {\mathscr {E}}_t\Big ] \end{aligned}$$(15)is concave for all \((t,e_i) \in [0,T]\times {\mathbb {S}}\) a.s.
-
3.
$$\begin{aligned} \underset{u\in {\mathscr {U}} }{{{\mathrm{ess \text { } sup}}}}&\Big \{E\Big [H (t,\widehat{X}(t),\alpha (t), u,\widehat{Y}(t), \widehat{Z}(t), \widehat{K}(t,\cdot ),\widehat{V}(t), \widehat{A}(t),\widehat{p}(t),\widehat{q}(t),\widehat{r}(t,\cdot ),\widehat{w}(t)) \Big . \Big | {\mathscr {E}}_t\Big ]\Big \} \nonumber \\&\le E\Big [H (t,\widehat{X}(t),\alpha (t), \widehat{u},\widehat{Y}(t), \widehat{Z}(t), \widehat{K}(t,\cdot ),\widehat{V}(t),\widehat{A}(t), \widehat{p}(t),\widehat{q}(t),\widehat{r}(t,\cdot ),\widehat{w}(t)) \Big . \Big | {\mathscr {E}}_t\Big ] \end{aligned}$$(16)
for all \(t\in [0,T]\), a.s.
-
4.
Assume that \(\frac{\mathrm {d}}{\mathrm {d}\nu }\nabla _k\widehat{H}(t,\zeta )>-1\).
-
5.
In addition, assume the following integrability condition:
$$\begin{aligned}&E\left[ \int _0^T\left\{ \widehat{p}^2(t) \left( (\sigma (t)-\widehat{\sigma }(t))^2+ {\int }_{{\mathbb {R}}_0}( \gamma (t,\zeta )-\widehat{\gamma } (t,\zeta ) )^2\,\nu _\alpha (\mathrm {d}\zeta )\right. \right. \right. \nonumber \\ {}&\quad \left. +\sum _{j=1}^D(\eta ^j(t)-\widehat{\eta }^j(t) )^2\lambda _{j}(t) \right) +(X(t)-\widehat{X}(t))^2 \left( \widehat{q}^2(t)+ {\int }_{{\mathbb {R}}_0} \widehat{r}^2 (t,\zeta ) \nu _\alpha (\mathrm {d}\zeta )\Big . \Big .\right. \nonumber \\&\quad \left. +\sum _{j=1}^D(w^j)^2(t) \lambda _{j}(t) \right) +(Y(t)-\widehat{Y}(t))^2 \left( (\frac{\partial \widehat{H}}{\partial z} )^2(t)\right. \nonumber \\&\left. \quad \ + {\int }_{{\mathbb {R}}_0} \Big \Vert \frac{\mathrm {d}\nabla _k H(t,\zeta ) }{\mathrm {d}\nu _\alpha (\zeta )} \Big \Vert ^2 \nu _\alpha (\mathrm {d}\zeta )+ \sum _{j=1}^D (\frac{\partial \widehat{H}}{\partial v^j} )^2(t) \lambda _{j}(t) \right) \nonumber \\&\quad \ \Big . \Big . +\widehat{A}^2(t) \left( (Z(t)-\widehat{Z}(t))^2 + {\int }_{{\mathbb {R}}_0}\left( K (t,\zeta )-\widehat{K} (t,\zeta ) \right) ^2\nu _\alpha (\mathrm {d}\zeta ) + \sum _{j=1}^D(V^j(t)\right. \nonumber \\&\left. \left. \left. \quad \ -\widehat{V}^j(t) )^2\lambda _{j}(t) \right) \right\} \mathrm {d}t \right] <\infty . \end{aligned}$$(17)
Then, \(\widehat{u}\) is an optimal control process and \(\widehat{X}\) is the corresponding controlled state process.
Remark 3.2
In Theorem 3.1 and in the following, we shall use the notations \(X(t)=X^{\widehat{u}}(t)\) and \(Y(t)=Y^{\widehat{u}}(t)\) are the processes associated with the control \(\widehat{u}(t)\). Furthermore, put \( \frac{\partial \widehat{H}}{\partial x}(t):=\frac{\partial H}{\partial x} (t,\widehat{X}(t),\alpha (t), \widehat{u},\widehat{Y}(t), \widehat{Z}(t), \widehat{K}(t,\cdot ),\widehat{V}(t),\widehat{A}(t),\widehat{p}(t),\widehat{q}(t),\widehat{r}(t,\cdot ),\widehat{w}(t)) \) and similarly for \( \frac{\partial \widehat{H}}{\partial y}(t), \frac{\partial \widehat{H}}{\partial z}(t), \nabla _{k} \widehat{H}(t,\zeta ), \frac{\partial \widehat{H}}{\partial v^j}(t)\) and \(\frac{\partial \widehat{H}}{\partial u}(t)\).
Remark 3.3
Let us mention that the above maximum principle requires some concavity assumptions. However, this concavity assumption may not be satisfied in some applications, see, for example, Sect. 5. Therefore, we need a maximum principle which does not require the above assumption. The maximum principle derived in the next section gives a first-order necessary and sufficient condition but not the optimality of the control. In fact, it says that, if it exists, then the equivalent maximum principle enables us to derive the expression for the optimal control.
3.2 An Equivalent Maximum Principle
In this section, we prove a version of the maximum principle that does not require a concavity condition. We call it an equivalent maximum principle. Let us make the following additional assumptions:
Assumption A1
For all \(t_0\in [0,T]\) and all bounded \({\mathscr {E}}_t\)-measurable random variable \(\theta (\omega )\), the control process \(\beta (t)\) defined by
Assumption A2
For all \(u \in {\mathscr {A}}_{{\mathscr {E}}}\) and all bounded \(\beta \in {\mathscr {A}}_{{\mathscr {E}}}\), there exists \(\delta >0\) such that
Assumption A3
For all bounded \(\beta \in {\mathscr {A}}_{{\mathscr {E}}}\), the derivatives processes
exist and belong to \(L^2([0,T]\times {\varOmega })\).
In the following, we write \(\frac{\partial b}{\partial x}(t)\) for \(\frac{\partial b}{\partial x}(t,X(t),\alpha (t),u(t))\), etc. It follows from (6) and (8) that
and
Remark 3.4
For sufficient conditions for the existence and uniqueness of solutions to (20) and (21), we refer the reader to [16, (4.1)]. A set of sufficient conditions under which systems (20)–(21) admits a unique solution is as follows:
-
1.
Assume that the coefficients \(b,\sigma , \gamma , \eta , g, f, \psi \) and \(\phi \) are continuous with respect to their arguments and are continuously differentiable with respect to (x, y, z, k, v, u). (Here, the dependence of g and f on k is trough \(\int _{{\mathbb {R}}_0}k(\zeta )\rho (t,\zeta )\nu (\mathrm {d}\zeta )\), where \(\rho \) is a measurable function satisfying \(0\le \rho (t,\zeta )\le c(1\wedge |\zeta |),\) \( \forall \zeta \in {\mathbb {R}}_0\). Hence the differentiability in this argument is in the Fréchet sense.)
-
2.
The derivatives of \(b,\sigma , \gamma , \eta \) and g are bounded.
-
3.
The derivatives of f are bounded by \(C(1+|x|+|y|+(\int _{{\mathbb {R}}_0} |k(.,\zeta )|^2\nu (\mathrm {d}\zeta ))^{1\backslash 2}+|v|+|u|)\).
-
4.
The derivatives of \(\psi \) and \(\phi \) with respect to x are bounded by \(C(1+|x|).\)
Remark 3.5
Assumption A1 (which includes linear model) is common in the literature and allows to build the control step by step, see, for example, [15, 18, 30]. However, a drawback of this method is that it does not work when the set of controls is not convex.
Theorem 3.2
(Equivalent Maximum Principle) Let \(u\in {\mathscr {A}}_{{\mathscr {E}}}\) with corresponding solutions X(t) of (6), \((Y(t),Z(t),K(t,\zeta ),V(t))\) of (8), A(t) of (12), \((p(t),q(t),r(t,\zeta ),w(t))\) of (13) and corresponding derivative processes \(x_1(t)\) and \((y_1(t),z_1(t),k_1(t,\zeta ),v_1(t))\) given by (20) and (21), respectively. Suppose that Assumptions A1, A2 and A3 hold. Moreover, assume the following growth conditions
and
Then the following are equivalent:
(A) \(\frac{\mathrm {d}}{\mathrm {d}\ell }J^{(u+\ell \beta )}(t)\Big . \Big |_{\ell =0}=0 \text { for all bounded } \beta \in {\mathscr {A}}_{{\mathscr {E}}}.\)
(B) \(E\Big [\frac{\partial H}{\partial u} (t,X(t),\alpha (t),Y(t), Z(t), K(t,\cdot ),V(t),u,A(t),p(t),q(t),r(t,\cdot ),w(t))_{u=u(t)} \Big . \Big | {\mathscr {E}}_t\Big ]=0\) for almost all (a.a.) \( t \in [0,T].\)
Remark 3.6
Let us observe that the two previous maximum principles require the existence and uniqueness of the solution to the BSDE satisfied by the adjoint processes. In the partial information case, the above result is not always true. Hence, in the next section, we propose a stochastic maximum principle via Malliavin calculus. In this approach, the adjoint processes depend on the coefficients of the system, their Malliavin derivatives and a modified Hamiltonian.
3.3 A Malliavin Calculus Approach
In this section, we present a method based on Malliavin calculus. The set up we adopt here is that of a Markov regime-switching forward–backward stochastic differential equations with jumps, as in the previous sections and the notation are the same. For basic concepts of Malliavin calculus, we refer the reader to [31, 32].
In the sequel, let us denote by \(D^B_{t}F\) (respectively \(D^{\widetilde{N}_\alpha }_{t,\zeta } F\) and \(D^{\widetilde{{\varPhi }}}_t F\)) the Malliavin derivative in the direction of the Brownian motion B (respectively pure jump Lévy process \(\widetilde{N}_\alpha \) and the pure jump process \(\widetilde{{\varPhi }}\)) of a given (Malliavin differentiable) random variable \(F=F(\omega );\,\,\,\omega \in {\varOmega }\). We denote by \(\mathbb {D}_{1,2}\) the set of all random variables which are Malliavin differentiable with respect to \(B(\cdot ),\,\widetilde{N}_\alpha (\cdot ,\cdot )\) and \(\widetilde{{\varPhi }}(\cdot )\). A crucial argument in the proof of our general maximum principle rests on duality formulas for the Malliavin derivatives \( D_{t}\) and \(D_{t,\zeta }\) (see, e.g. [31, 32]):
These formulae hold true for all Malliavin differentiable, random variable F and \({\mathscr {F}}_t\)-predictable processes \(\varphi \) and \(\psi \) such that the integrals on the right-hand side converge absolutely.
We also need some basic properties of the Malliavin derivatives. Let \(F \in \mathbb {D}_{1,2}\) be a \({\mathscr {F}}_s\)-measurable random variable, then \( D^B_{t}F=D^{\widetilde{N}_\alpha }_{t,\zeta }F=D^{\widetilde{{\varPhi }}}_{t}F=0 \text { for all } t>s.\) We have the following results known as the fundamental theorems of calculus
under the assumption that all the terms involved are well defined and belong to \(\mathbb {D}_{1,2}\).
In view of the optimization problem (10), we define the following processes: suppose that for all \(u\in \) \({\mathscr {A}}_{{\mathscr {E}}}\) the processes
are all well defined. In (35) and in the sequel, we use the shorthand notation \(H_0(t)=H_{0}\Big (t,X(t),\alpha (t),Y(t),Z(t),K(t,\cdot ),V(t),u, \widetilde{A}(t),\kappa (t) \Big )\). We also assume that the following modified adjoint processes \((\tilde{p}(t),\tilde{q}(t),\tilde{r}(t,\zeta ),\tilde{w}(t))\) and \(\tilde{A}(t)\) given by
and
are well defined. Here, the general Hamiltonian H is given by (11), with p, q, r, w replaced by \(\tilde{p}, \tilde{q}, \tilde{r}, \tilde{w}\).
Remark 3.7
Assume that the coefficients of the control problem satisfy the conditions for existence and uniqueness of solution to systems (6)–(8). Assume moreover that conditions in Remark 3.4 hold. Then, the processes given by (30)–(39) are well defined. The conditions on existence of processes defined in (30)–(39) play an important role in the proof of Theorem 3.3. For example, if the Malliavin differentiability of the process \(\tilde{p}\) with respect to \(B,\widetilde{N}_\alpha , \widetilde{{\varPhi }}\) is not guaranteed, then the theorem cannot be proved. We can now state a general stochastic maximum principle for our control problem (10):
Theorem 3.3
Let \(u\in {\mathscr {A}}_{{\mathscr {E}}}\) with corresponding solutions X(t) of (6), \((Y(t),Z(t),K(t,\zeta ),V(t))\) of (8), \(\tilde{A}(t)\) of (39), \(\tilde{p}(t),\tilde{q}(t),\tilde{r}(t,\zeta ),\tilde{w}^j(t)\) of (35)–(38) and corresponding derivative processes \(x_1(t)\) and \((y_1(t),z_1(t),k_1(t,\zeta ),v_1(t))\) given by (20) and (21), respectively. Suppose that Assumptions A1, A2 and A3 hold. Moreover, suppose that the random variables \(F(T),{\varTheta }(t,s)\) given by (32) and (33), and \(\frac{\partial f}{\partial x}(t)\) are Malliavin differentiable with respect to \(B,\widetilde{N}\) and \(\widetilde{{\varPhi }}\). Furthermore, suppose the following integrability conditions:
Then, the following are equivalent:
(A) \(\frac{\mathrm {d}}{\mathrm {d}\ell }J^{(u+\ell \beta )}(t)\Big . \Big |_{\ell =0}=0 \text { for all bounded } \beta \in {\mathscr {A}}_{{\mathscr {E}}}.\)
(B) \(E\Big [\frac{\partial H}{\partial u} (t,X(t),\alpha (t),Y(t), Z(t), K(t,\cdot ),V(t),u,\tilde{A}(t),\tilde{p}(t),\tilde{q}(t),\tilde{r}(t,\cdot ),\tilde{w}(t))_{u=u(t)}\Big . \Big | {\mathscr {E}}_t\Big ] =0\) for a.a. \((t ,\omega )\in [0,T]\times {\varOmega }\).
Remark 3.8
Assume that conditions in Remark 3.7 hold. Assume moreover that the coefficients are twice continuously differentiable with the second-order derivatives satisfying the conditions in Remark 3.7. Then, \(F(T),{\varTheta }(t,s)\) and \(\frac{\partial f}{\partial x}(t)\) are Malliavin differentiable with respect to \(B,\widetilde{N}_{\alpha }\) and \(\widetilde{{\varPhi }}\).
4 Proof of the Results
In this section, we prove the main results.
Proof
(Proof of Theorem 3.1) We prove that \(J(x,\widehat{u},e_i)\ge J(x,u,e_i) \text { for all } u \in {\mathscr {A}}_{{\mathscr {E}}}.\)
Choose \(u \in {\mathscr {A}}_{{\mathscr {E}}}\) and consider
where
By the definition of H, we get
By the concavity of \(\varphi \) in x, the Itô’s formula (see, e.g. [5, Theorem 4.1]), (6), (13) and (17), we get
By the concavity of \(\psi ,h\), the Itô’s formula, (8) and (12), we get
One can show, using similar arguments as in [33] (see also [5]), that the right-hand side of (48) is non-positive. For sake of completeness, we give the details here. Fix \(t\in [0,T]\). Since \(\widetilde{H}(x,y,z,k,v)\) is concave, it follows by the standard hyperplane argument (see, e.g. [34, Chapter 5, Section 23]) that there exists a subgradient \(d=(d_1,d_2,d_3,d_4(\cdot ),d_5) \in {\mathbb {R}}^3\times \mathscr {R}\times {\mathbb {R}}\) for \(\widetilde{H}(x,y,z,k,v)\) at \(x=\widehat{X}(t),y=\widehat{Y}(t),\,z=\widehat{Z}(t),\) \(k=\widehat{K}(t,\cdot ),\,v=\widehat{V}(t)\) such that, if we define
Then \(i(x,y,z,k,v)\le 0\) for all x, y, z, k, v.
Furthermore, we have \(i(\widehat{X}(t),\widehat{Y}(t),\widehat{Z}(t),\widehat{K}(t,\cdot ),\widehat{V}(t))\). It follows that,
Substituting this into (48), using conditions 2. and 3. in Theorem 3.1, and the concavity of \(\widetilde{H}\), we conclude that \(J(x,\widehat{u},e_i)\ge J(x,u,e_i) \text { for all } u \in {\mathscr {A}}_{{\mathscr {E}}}.\) This completes the proof. \(\square \)
Proof
(Proof of Theorem 3.2) We have that
By (13), the Itô’s formula, (20) and (22), we have
By (12), the Itô’s formula, (21) and (23), we get
Substituting (51) and (52) into (50), we get
By the definition of H, the coefficients of \(x_1(t),y_1(t),z_1(t), k_1(t,\zeta )\) and \(v_1(t)\) are all equal to zero in (53). Hence, if
it follows that
This holds in particular for \(\beta \in {\mathscr {A}}_{{\mathscr {E}}}\) of the form \( \beta (t)=\beta _{t_0}(t,\omega )=\theta (\omega )\xi _{[t_0,T]}(t)\) for a fix \(t_0\in [0,T[\), where \(\theta (\omega )\) is a bounded \({\mathscr {E}}_{t_0}\)-measurable random variable. Hence
Differentiating with respect to \(t_0\), we have
Since the equality is true for all bounded \({\mathscr {E}}_{t_0}\)-measurable random variable, we conclude that
This shows that (A) \(\Rightarrow \) (B).
Conversely, using the fact that every bounded \(\beta \in {\mathscr {A}}_{{\mathscr {E}}}\) can be approximated by a linear combinations of controls \(\beta (t)\) of the form (18), the above argument can be reversed to show that (B) \(\Rightarrow \) (A). \(\square \)
Proof
(Proof of Theorem 3.3) (A) \(\Rightarrow \) (B). We split this proof into two steps: in the first step, we show that the directional derivative of the value function can be written as a sum of the two terms \(J_1\) and \(J_2\), given by (55) and (56). In the second step, we show that the condition (A) implies the condition (B).
Lemma 4.1
Assume that the conditions of Theorem 3.3 hold. Then
where
Proof
It follows from (20) and duality formula that for F(T) defined by (32) we get
Similarly, we have
Changing the notation \(s \leftrightarrow t\), this becomes
Combining (30), (32), (58) and (59), we have
By the Itô’s formula and (39), it follows as in (52) that
But
Hence we have
Substituting (58)–(61) into (57), we get
Equation (62) holds for all \(\beta \in {\mathscr {A}}_{{\mathscr {E}}}\). In particular, choose \( \beta _{\theta }=\beta _{\theta }(s)=\theta (\omega )\chi _{(t,t+h]}(s),\) where \(\theta (\omega )\) is \({\mathscr {E}}_t\)-measure and \(0\le t\le t+h \le T.\) Then, (20) yields \( x_1=x_1^{(\beta _{\theta })}(s)=0 \text { for } 0\le s\le t.\) Hence (62) can be rewritten as
where
This completes the first step. \(\square \)
Next, we conclude the proof of (A) \(\Rightarrow \) (B).
Lemma 4.2
Assume that the conditions of Theorem 3.3 are satisfied. Assume that (A) in Theorem 3.3 holds. Then, (B) in Theorem 3.3 also holds.
Proof
Assume that (A) holds, that is, \(\frac{\mathrm {d}}{\mathrm {d}\ell }J^{(u+\ell \beta )}(t)= 0\). Then, from Lemma 4.1, we have \(J_1(h)+J_2(h)=0\).
Let \(x_1(s)=x_1^{(\beta _{\theta })}(s)\). Assume that \(s\ge t+h\). Then, it follows from the choice of \(\beta _{\theta }\) and (20) that
By the Itô’s formula, it is easy to show that \(x_1(s)=x_1(t+h)G(t+h,s); \,\,\,s\ge t+h,\) where G is defined by (34). Let us observe that G(t, s) does not depend on h. It follows from the definition of \(H_0\) [see (31)] that
Differentiating with respect to h at \(h=0\) gives
Since \(x_1(t)=0\), we get \( \frac{\mathrm {d}}{\mathrm {d}h}E\left[ \displaystyle \int _t^{t+h} \frac{\partial H_0}{\partial x}(s)x_1(s)\mathrm {d}s\right] _{h=0}=0.\) Using the definition of \(x_1(s)\), we have
where \(x_1(t+h)\) is given by
Therefore, by (67) and (68) \( \frac{\mathrm {d}}{\mathrm {d}h} J_1(h)\Big |_{h=0}=J_{1,1}(0)+J_{1,2}(0),\) with
Since \(x_1(t)=0\), we have \(J_{1,2}(0)=0\), from which we get \(\frac{\mathrm {d}}{\mathrm {d}h} J_1(h)\Big |_{h=0}=J_{1,1}(0).\) Using once more the duality formula, we get from (33) that
On the other hand, differentiating (65) with respect to h at \(h=0\), we have
Using (36)–(38) and (11) with A, p, q, r, w replaced by \(\tilde{A},\tilde{p}, \tilde{q}, \tilde{r}, \tilde{w}\), we get
Since this holds for all \({\mathscr {E}}_t\)-measurable random variables \(\theta \), we conclude that
The proof of (A) \(\Rightarrow \) (B) is completed. \(\square \)
Finally, we prove that (B) \(\Rightarrow \) (A). Conversely, assume that there exists \(u\in {\mathscr {A}}_{{\mathscr {E}}}\) such that (74) holds. Then by reversing the previous argument, we obtain that (A) holds for \(\beta _{\theta }(s)=\theta (\omega )\chi _{(t,t+h]}(s) \in {\mathscr {A}}_{{\mathscr {E}}} \), where \(\theta \) is bounded and \({\mathscr {E}}_t\)-measurable. Then (63) holds for all linear combinations of \(\beta _{\theta }\). Since all bounded \(\beta \in {\mathscr {A}}_{{\mathscr {E}}} \) can be approximated pointwise boundedly in \((t,\omega )\) by such linear combination, it follows that (63) is satisfied for all bounded \(\beta \in {\mathscr {A}}_{{\mathscr {E}}} \). Thus, reversing the remaining part of the previous proof, we get \(\frac{\mathrm {d}}{\mathrm {d}\ell }J^{(u+\ell \beta )}(t)\Big . \Big |_{\ell =0}=0\) for all bounded \(\beta \in {\mathscr {A}}_{{\mathscr {E}}}\). \(\square \)
5 Applications
5.1 Application to Optimal Control Problem for Markov Regime Switching with No Concave Value Function
In this section, we apply the results obtained to study an optimal control problem for a Markov regime-switching system, assuming that the value function is not concave. Suppose that the state process \(X(t)=X^{(u)}(t,\omega );\,\,0 \le t \le T,\,\omega \in {\varOmega }\) is a controlled Markov regime-switching jump diffusion of the form
where \(T>0\) is a given constant. \(u(\cdot )\) is the control process. We assume here that \(\widetilde{N}_\alpha =\widetilde{N}\) for any state of the Markov chain. Let us introduce the performance functional
In this case, we have that
with the modified adjoint processes are reduced to
Remark 5.1
The Hamiltonian in this case is not concave and therefore Theorem 3.1 cannot be applied. Using the Malliavin calculus approach (Theorem 3.3), we derive the expression of the optimal control if it exists. Note that, when \({\mathscr {E}}_t={\mathscr {F}}_t \text { for all } t\in [0,T]\), one can also use Theorem 3.2 to derive the optimal control. In fact, in this case, it is possible to guess the form of the adjoint processes and employ techniques from ordinary differential equations to get the solution and hence the optimal control.
Theorem 5.1
Assume that the state process is given by (75) and let the performance functional be given by (76). Moreover, assume that \(\alpha (t)\) is a two-state Markov chain and \({\mathscr {E}}_t={\mathscr {F}}_t \text { for all } t\in [0,T]\). Assume in addition that an optimal control exists. Then, \(u^*\) is an optimal control for (10) iff
where
and \({\varGamma }(t,T,2)\) is obtained in a similar way.
Proof
The condition (2) in Theorem 3.3 for an optimal control \(\hat{u}(t)\) is one of the two
Equation (80) can be seen as a partial information, Markov switching Malliavin-differential type equation in the unknown random variable \(\tilde{p}(t)\). A similar equation was solved in [15] without regime switching, when \({\mathscr {E}}_t={\mathscr {F}}_t\). For simplicity, we assume from now on that \({\mathscr {E}}_t={\mathscr {F}}_t \text { for all } t\in [0,T]\) and that \(\alpha \) is a two-state Markov chain. Using the fundamental theorem of calculus (see, e.g. [31, Theorem 3.1]), we have
Using integration by parts formula (or product rule), we get
Taking conditional expectation with respect to \({\mathscr {F}}_t\), we have
Let \(\alpha (t)=e_1\) and for \(n=1,2,3,4\), let \(C_n(i)\) be the value of the function \(C_n\) at 1. Define \(C_n(2,1)\) for \(n=1,2,3,4\) by \(C_n(2,1):=C_n(2)-C_n(1).\) Then, we have
It follows from the transition probability of a two-state Markov chain that
On the other hand, set \(\alpha (t)=e_1\). Using the integration by parts formula and the fundamental theorem of calculus, we have
Similarly, we get
Then, the result follows for \(\alpha (t)=e_1\). Performing the same computations, one get an expression for \({\varGamma }(t,T,2)\). This completes the proof. \(\square \)
The following corollary is a generalization of [3, Example 4.7].
Corollary 5.1
Assume that conditions of Theorem 5.1 are satisfied. Moreover, assume that \(C_1,C_2,C_3,C_4:I\rightarrow {\mathbb {R}}\) satisfy \(C_1(1)=-1,C_1(2)=0,C_2(1)=0,C_2(2)=-\frac{1}{2}, C_3(1)=0,C_3(2)=1\), \(C_4(1)=\frac{1}{2},C_4(2)=1\).
Then, the optimal control \(u^*\) for (10) satisfies:
where \({\varGamma }(t,T,1)=\frac{1}{2}+\frac{\lambda _{1,2}}{\lambda _{1,2} +\lambda _{2,1}}(T-t)+\frac{\lambda _{1,2}\Big \{\frac{1}{2}(\lambda _{1,2} +\lambda _{2,1})-1\Big \}}{(\lambda _{1,2} +\lambda _{2,1})^2}\Big \{1-e^{(\lambda _{1,2} +\lambda _{2,1})(t-T)}\Big \}.\)
5.2 Application to Recursive Utility Maximization
In this section, we use the results from Sect. 3.3 to study a problem of recursive utility maximization. Consider a financial market with two investments possibilities: a risk-free asset (bond) with the unit price \(S_0(t)\) at time t and a risky asset (stock) with unit price S(t) at time t. Let r(t) be the instantaneous interest rate of the risk-free asset at time t. If \(r_t:=r(t,\alpha (t))=\langle \underline{r}|\alpha (t)\rangle \), where \(\langle \cdot |\cdot \rangle \) is the usual scalar product in \({\mathbb {R}}^D\) and
\(\underline{r}=(r_1,r_2, \ldots ,r_D)\in \mathbb {R_+}^D\), then the price dynamic of \(S_0\) is given by:
The appreciation rate \(\mu (t)\) and the volatility \(\sigma (t)\) of the stock at time t are defined by
where \(\underline{\mu }=(\mu _1,\mu _2, \ldots ,\mu _D)\in {\mathbb {R}}^D\) and \(\underline{\sigma }=(\sigma _1,\sigma _2, \ldots ,\sigma _D)\in \mathbb {R_+}^D\). The stock price process S is described by the following Markov modulated Lévy process:
The general setting considered here can be seen as an extension of the exponential-Lévy model described in [35], where a factor of modulation is introduced. Hence, we can retrieve in a simple way some of the existing models in the literature (e.g. the classical Black–Scholes model and the family of exponential-Lévy models.)
Here \(r(t)\ge 0,\,\,\mu (t),\,\,\sigma (t)\) and \(\gamma (t,\zeta )>-1+\varepsilon \) (for some constant \(\varepsilon >0\)) are given \({\mathscr {E}}_t\)-predictable, integrable processes, with \(\left\{ {\mathscr {E}}_t\right\} _{t\in \left[ 0,T\right] }\) being a given filtration, such that
Suppose that a trader in this market chooses a portfolio u(t), representing the amount she invests in the risky asset at time t. In the partial information case, this portfolio is a \({\mathscr {E}}_t\)-predictable stochastic process. Choosing \(S_0(t)\) as a numeraire, and setting without loss of generality \(r(t)=0\), one can show (see [18] for such a derivation) that the corresponding wealth process \(X(t)=X^{(u)}(t)\) satisfies
The above process is a controlled Itô-Lévy process.
We consider a small agent endowed with an initial wealth x, who can choose her portfolio between time 0 and time T. We suppose that there exists a terminal reward X(T) at time T. In this setting, the utility at time t depends on the utility up to time t and on the future utility. More precisely, the recursive utility at time t is defined by
where g is called the driver. One can show as in [14] (see also [29, 36]) that the above process can be regarded as a solution to the following Markov regime-switching BSDE.
where \(g:[0,T]\times {\mathbb {R}} \times {\mathbb {S}} \times {\mathscr {U}} \times {\varOmega }\rightarrow {\mathbb {R}}\) is such that the BSDE (91) has a unique solution and \((t,\omega ) \rightarrow g(t,x,e_i,\omega )\) is \({\mathscr {F}}_t\)-predictable for each given x and \(e_i\). For more information about recursive utility, the reader may consult [14, 20, 22]. Such unique solution exists if one assumes that \(g(\cdot , y,e_i)\) is uniformly Lipschitz continuous with respect to y, the random variable X(T) is squared integrable and \(g(t,0,e_i)\) is uniformly bounded.”
We want to apply Theorem 3.3 to find the control u (if it exists) that maximizes the recursive utility Y(0) defined by (91). This means that we aim at finding \(u^*\) and \(Y^*\) such that
Note that the performance functional J(u) given by (9) is reduced to:
This means that
We also have
The Hamiltonian is therefore reduced to:
with the modified adjoint processes \(\tilde{A}\) and \((\tilde{p}(t),\tilde{q}(t),\tilde{r}(t,\zeta ),\tilde{w}(t))\) given, respectively by:
and
Equation (93) can be solved explicitly and the solution is given by:
Condition (B) in Theorem 3.3 for an optimal control \(u^*\) becomes
for \(i=1,\ldots ,D\). For each \(i=1,\ldots ,D\), Eq. (99) is called a partial information, Malliavin differentiable type of equation in the unknown variable \(\tilde{A}(T)\), see, for example, [15, 17]. For \({\mathscr {E}}_t={\mathscr {F}}_t\), one can solve this equation explicitly (see [15]) and get
for some \({\mathscr {F}}_t\)-predictable processes \(\beta (t,\alpha )\) and \(\theta (t,\alpha ,\zeta )\) such that
The processes \(\beta \) and \(\theta \) are completely determined by the vector \((\beta _1,\ldots , \beta _D)\) and \((\theta _1,\ldots ,\theta _D)\), solutions to the system of equations
for all \(i=1,\ldots ,D\). Under condition (101), the measure Q defined by
is an equivalent local martingale measure (ELMM) for the process X(t). For more discussion on this, we refer the reader to [15, Section 5].
Assume that \(\alpha (t)\) is a two-state Markov process and that \(g(t,Y(t),\alpha (t),\omega )\) is given by:
Using Theorem 3.3, similar arguments as in [15, Section 5] yield the following:
Theorem 5.2
Suppose that \(g(t,y,\alpha )\) is as in (104) and \(c_1\) is deterministic. Let \(\tilde{A} (T)\) be the solution of the modified forward adjoint equation and suppose that \(\beta \) and \(\theta \) satisfy
Moreover, assume that \( E\Big [\exp \Big (\int _0^Tc(t) \mathrm {d}t\Big )\Big (1+\int _0^T|c_0(t)| \mathrm {d}t\Big )\Big ]<\infty .\) In addition, suppose that an optimal control \(u^*\) exists . Then, the maximal differential utility is given by:
Proof
It follows from Theorem 3.3 and the arguments in [15, Section 5]. \(\square \)
6 Conclusions
In this paper, we presented three versions of the stochastic maximum principle for Markov regime-switching forward–backward stochastic differential equation with jumps. We then applied the results to study both the problem of optimal control when the Hamiltonian is not concave and the problem of recursive utility maximization. In the former case, the Malliavin calculus approach was used. There are many advantages of using Malliavin calculus approach. First, it does not require the study of the existence and uniqueness of the solution of a BSDE usually satisfied by the adjoint equation. Second, it does not assume concavity of the Hamiltonian. Third, it enables us to get an “explicit” solution for the optimal control problem for non-concave Hamiltonian in some cases.
In this work, it is assumed that the sensitivity towards risk of the controller when making decisions is implicitly given in the utility function. It is often the case that the risk-sensitive parameter is explicitly taken into consideration when dealing with the controller preference. Such control problem is known as risk-sensitive control and has been studied in the past years by several authors, see, for example, [37,38,39,40]. It would therefore be interesting to extend the current Malliavin calculus approach to the risk-sensitive case. A risk-sensitive maximum principle for a Markov regime-switching jump–diffusion system is derived in [41] using the classical approach.
Another interesting study would be to address the problem of partial observation maximum principle for Markov regime-switching systems. In fact, in many economic applications the target variables are not always observed and a specific observation process is given (see, e.g. [37]). A way of solving the control problem in this case is to derive the stochastic partial differential equation of the associated filtering and consider an optimal control problem for stochastic partial differential equations.
In this paper, we do not analyse the effect of a change in a parameter (e.g. volatility, initial value) of the state process could have in the obtained optimal control. Such study could also be of interest.
References
Donnelly, C.: Sufficient stochastic maximum principle in a regime-switching diffusion model. Appl. Math. Optim. 64, 155–169 (2011)
Donnelly, C., Heunis, A.J.: Quadratic risk minimization in a regime-switching model with portfolio constraints. SIAM J. Control Optim. 50(4), 2431–2461 (2012)
Li, Y., Zheng, H.: Weak necessary and sufficient stochastic maximum principle for markovian regime-switching diffusion models. Math. Optim, Appl (2014). doi:10.1007/s00245-014-9252-6
Tao, R., Wu, Z.: Maximum principle for optimal control problems of forward-backward regime-switching system and applications. Syst. Control Lett. 61, 911–917 (2012)
Zhang, X., Elliott, R.J., Siu, T.K.: A stochastic maximum principle for a markov regime-switching jump-diffusion model and its application to finance. SIAM J. Control Optim. 50(2), 964–990 (2012)
Fleming, V.H., Soner, H.M.: Controlled Markov Processes and Viscosity Solutions. Springer, New York (2006)
Yong, J., Zhou, X.: Stochastic Controls: Hamiltonian Systems and HJB Equations. Springer, New York (1999)
Eloe, P., Liu, R.H., Yatsuki, M., Yin, G., Zhang, Q.: Optimal selling rules in a regime-switching exponential gaussian diffusion model. SIAM J. Appl. Math. 69, 810–829 (2008)
Bensousssan, A.: Maximum principle and dynamic programming approaches of the optimal control of partially observed diffusions. Stochastics 9(3), 169–222 (1983)
Bismut, J.: An introductory approach to duality in optimal stochastic control. SIAM Rev. 20, 62–78 (1978)
Kushner, H.J.: Necessary conditions for continuous parameter stochastic optimization problems. SIAM J. Control Optim. 10, 550–565 (1972)
Øksendal, B., Sulem, A.: Applied Stochastic Control of Jump Diffusions, 3rd edn. Springer, Berlin (2009)
Peng, S.: A general stochastic maximum principle for optimal control problems. SIAM J. Control Optim. 28, 966–979 (1990)
Karoui, N.E., Peng, S., Quenez, M.C.: A dynamic maximum principle for the optimization of recursive utilities under constraints. Ann. Appl. Probab. 11(3), 664–693 (2001)
Øksendal, B., Sulem, A.: Maximum principles for optimal control of forward-backward stochastic differential equations with jumps. SIAM J. Control Optim. 48(5), 2845–2976 (2009)
Peng, S.: Backward stochastic differential equations and applications to optimal control. Appl. Math. Optim. 27, 125–144 (1993)
Meyer-Brandis, T., Øksendal, B., Zhou, X.: A mean-field stochastic maximum principle via Malliavin calculus. Stochastic 84, 643–666 (2012)
DiNunno, G., Menoukeu-Pamen, O., Øksendal, B., Proske, F.: A General Maximum Principle for Anticipative Stochastic Control and Applications to Insider Trading. Springer, New York (2011)
Menoukeu-Pamen, O., Meyer-Brandis, T., Proske, F., Binti-Salleh, H.: Malliavin calculus applied to optimal control of stochastic partial differential equations with jumps. Int. J. Probab. Stoch. Process. 85(3), 431–463 (2013)
Epstein, L., Zin, S.: Substitution, risk aversion and the temporal behavior of consumption and asset returns: a theoretical framework. Econometrica 57, 937–969 (1989)
Weil, P.: Non-expected utility in macroeconomics. Q. J. Econ. 105, 29–42 (1990)
Duffie, D., Epstein, M.: Stochastic differential utility. Econometrica 60, 353–394 (1992)
Faidi, W., Matoussi, A., Mnif, M.: Maximization of recursive utilities: a dynamic maximum principle approach. SIAM J. Financ. Math. 2(1), 1014–1041 (2011)
Pham, H.: Continuous-time Stochastic Control and Optimization with Financial Applications. Springer, Berlin (2009)
Elliott, R.J., Siu, T.K.: A stochastic differential game for optimal investment of an insurer with regime switching. Quant. Financ. 11, 365–380 (2011)
Elliott, R.J., Aggoun, L., Moore, J.: Hidden Markov Models: Estimation and Control. Springer, New York (1994)
Cohen, S.N., Elliott, R.J.: Comparisons for backward stochastic differential equations on markov chains and related no-arbitrage conditions. Ann. Appl. Probab. 20, 267–311 (2010)
Cohen, S.N., Elliott, R.J.: Existence, uniqueness and comparisons for BSDEs in general spaces. Ann. Probab. 40, 2264–2297 (2012)
Crepey, S.: About the Pricing Equations in Finance. Springer, Berlin (2010)
Baghery, F., Øksendal, B.: A maximum principle for stochastic control with partial information. Stoch. Anal. Appl. 25, 705–717 (2007)
DiNunno, G., Øksendal, B., Proske, F.: Malliavin Calculus for Lévy Processes with Applications to Finance. Springer, Berlin (2009)
Nualart, D.: The Malliavin Calculus and Related Topics, 2nd edn. Springer, Berlin (2006)
Framstad, N., Øksendal, B., Sulem, A.: Stochastic maximum principle for optimal control of jump diffusions and applications to finance. J. Optim. Theory Appl. 121, 77–98 (2004)
Rockafeller, R.T.: Convex Analysis. Princeton University Press, Princeton (1970)
Cont, R., Tankov, P.: In: Dempster, M.A.H., Madan, D.B. (eds.) Financial Modeling With Jump Processes, Chapman & Hall/CRC Financial Mathematics Series (2004)
Situ, R.: On strong solution backward stochastic differential equations with jumps and applications. Stoch. Process. Appl. 66, 209–236 (1997)
Bensousssan, A.: Lectures on stochastic control. In: Mittler S.K., Moro A. (eds.) Nonlinear Filtering and Stochastic Control, Lecture note in Mathematics, vol. 972, chap. 1, pp. 1–62. Springer, Berlin (1982)
Bielecki, T.R., Pliska, S.R.: Risk-sensitive dynamic asset management. Appl. Math. Optim. 39, 337–360 (1999)
Dupuis, P., McEneaney, W.M.: Risk-sensitive and robust escape criteria. SIAM J. Control Optim. 35(6), 2021–2049 (1996)
Whittle, P.: Risk-sensitive Optimal Control. Wiley, New York (1990)
Sun, Z., Kemajou-Brown, I., Menoukeu Pamen, O.: A risk-sensitive maximum principle for Markov Regime-switching jump-diffusion systems and applications. ESAIM Control Optim. Calc. Var. (2017). doi:10.1051/cocv/2017039
Acknowledgements
The author would like to thank Corina Constantinescu, Apostolos Papaioannou, the anonymous referees and Bruno Bouchard for their helpful comments. The project on which this publication is based has been carried out with funding provided by the Alexander von Humboldt foundation, under the programme financed by the German federal ministry of education and research entitled German research chair No 01DG15010 and by the European Union’s Seventh Framework Programme for Research, Technological Development and Demonstration under Grant Agreement No. 318984-RARE.
Author information
Authors and Affiliations
Corresponding author
Additional information
Communicated by Bruno Bouchard.
Rights and permissions
Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.
About this article
Cite this article
Pamen, O.M. Maximum Principles of Markov Regime-Switching Forward–Backward Stochastic Differential Equations with Jumps and Partial Information. J Optim Theory Appl 175, 373–410 (2017). https://doi.org/10.1007/s10957-017-1144-x
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10957-017-1144-x
Keywords
- Forward–backward stochastic differential equations
- Malliavin calculus
- Regime switching
- Recursive utility maximization
- Stochastic maximum principle