Weak Necessary and Sufficient Stochastic Maximum Principle for Markovian Regime-Switching Diffusion Models

Abstract

In this paper we prove a weak necessary and sufficient maximum principle for Markovian regime switching stochastic optimal control problems. Instead of insisting on the maximum condition of the Hamiltonian, we show that $0$ belongs to the sum of Clarke’s generalized gradient of the Hamiltonian and Clarke’s normal cone of the control constraint set at the optimal control. Under a joint concavity condition on the Hamiltonian and a convexity condition on the terminal objective function, the necessary condition becomes sufficient. We give four examples to demonstrate the weak stochastic maximum principle.

Appendix

1.1 Proof of Theorem 5.15

Proof

Consider the function $\Phi$ on $\mathbb {S}^2([0,T])\times L^2(W,[0,T])\times L^2(Q,[0,T])$ mapping $(Y,Z,S)\in \mathbb {S}^2([0,T])\times L^2(W,[0,T])\times L^2(Q,[0,T])$ to $\left( \hat{Y},\hat{Z},\hat{S}\right) =\Phi (Y,Z,S)$ defined by

$$\begin{aligned} \hat{Y}(t)=\xi +\int \limits _t^Tf(s,Y(s),Z(s))ds-\int \limits _t^TZ(s)dW(s)-\int \limits _t^TS(s)\cdot dQ(s). \end{aligned}$$

Consider the square-integrable martingale

$$\begin{aligned} M(t)=E\left[ \xi +\int \limits _0^Tf(s,Y(s),Z(s))ds\bigg |\mathcal {F}_t\right] . \end{aligned}$$

According to Theorem 5.13, there exists unique $\left( \hat{Z},\hat{S} \right) \in L^2(W,[0,T])\times L^2(Q,[0,T])$ such that

$$\begin{aligned} M(t)=M(0)+\int \limits _0^t\hat{Z}(s)dW(s)+\int \limits _0^t\hat{S}(s)\cdot dQ(s). \end{aligned}$$

We then define the process $\hat{Y}(t)$ by

$$\begin{aligned} \hat{Y}(t)&= E\left[ \xi +\int \limits _t^Tf(s,Y(s),Z(s))ds\bigg |\mathcal {F}_t \right] =M(t)-\int \limits _0^tf(s,\alpha (s),Y(s),Z(s))ds\\&= M(0)+\int \limits _0^t\hat{Z}(s)dW(s)+\int \limits _0^t\hat{S}(s)\cdot dQ(s)-\int \limits _0^tf(s,Y(s),Z(s))ds\\&= \xi +\int \limits _t^Tf(s,Y(s),Z(s))ds-\int \limits _t^T\hat{Z}(s)dW(s)-\int \limits _t^T\hat{S}(s)\cdot dQ(s). \end{aligned}$$

By Doob’s $L^2$ inequality, we have

$$\begin{aligned} \begin{aligned}&E\left[ \sup _{0\le t\le T}\bigg |\int \limits _t^T\hat{Z}(s)dW(s)\bigg |\right] \le 4E\left[ \int \limits _0^T|\hat{Z}(s)|^2ds \right] <\infty ,\\&E\left[ \sup _{0\le t\le T}\bigg |\int \limits _t^T\hat{S}(s)\cdot dQ(s)\bigg |\right] \le 4E\left[ \sum _{l=1}^n\sum _{i,j=1}^d\int \limits _0^T|\hat{S}_{ij}^{(l)}(s)|^2d[Q_{ij}](s) \right] <\infty . \end{aligned} \end{aligned}$$

Under the assumptions on $(\xi ,f)$, we conclude that $\hat{Y}\in S^2([0,T])$. Hence $\Phi$ is a well defined function from $S^2([0,T])\times L^2(W,[0,T])\times L^2(Q,[0,T])$ into itself. Next, we show that $(\hat{Y},\hat{Z},\hat{S})$ is a solut ion to the regime switching BSDE (5.3) if and only if it is a fixed point of $\Phi$.

Let $(U,V,\Gamma )$, $(U',V',\Gamma ') \in S^2([0,T])\times L^2(W,[0,T])\times L^2(Q,[0,T])$. Apply function $\Phi$ and obtain $(Y,Z,S)=\Phi (U,V,\Gamma ),\ (Y',Z',S')=\Phi (U',V',\Gamma ')$. Set $(\bar{U},\bar{V},\bar{\Gamma }) =(U-U',V-V',\Gamma -\Gamma ')$, $(\bar{Y},\bar{Z},\bar{S})=(Y-Y',Z-Z',S-S')$ and $\bar{f}(t)=f(t,U(t),V(t))-f(t,U'(t),V'(t))$. Take $\beta >0$ to be chosen later and apply Ito’s formula to $e^{\beta s}\vert \bar{Y} \vert ^2$ on $[0,T]$,

$$\begin{aligned} \begin{aligned} \vert \bar{Y}(0) \vert ^2 =&-\displaystyle \int \limits _0^Te^{\beta t}\left( \beta \vert \bar{Y}(t)\vert ^2-2\bar{Y}(t)^\intercal \bar{f}(t) \right) dt-\int \limits _0^Te^{\beta t}\vert \bar{Z}(t) \vert ^2dt\\&-\displaystyle \int \limits _0^Te^{\beta t}\sum _{l=1}^n\sum _{i,j=1}^d\vert \bar{S}^{(l)}_{ij} \vert ^2d\left[ Q_{ij}\right] (t)-2\int \limits _0^T e^{\beta t}\bar{Y}(t)^\intercal \bar{Z}(t)dW(t)\\&-2\displaystyle \int \limits _0^Te^{\beta t}\sum _{l=1}^n\sum _{i,j=1}^d\bar{Y}^{(l)}(t)\bar{S}_{ij}^{(l)}(t)dQ_{ij}(t). \end{aligned} \end{aligned}$$

(8.1)

Observe that, according to Young’s inequality

$$\begin{aligned}&E\left[ \left( \int \limits _0^T e^{2\beta t}\vert \bar{Y}(t) \vert ^2\vert \bar{Z}(t) \vert ^2dt \right) ^{\frac{1}{2}}\right] \le \dfrac{e^{\beta T}}{2}E\left[ \sup _{0\le t\le T}\vert \bar{Y}(t) \vert ^2+\int \limits _0^T\vert \bar{Z}(t) \vert dt \right] <\infty ,\\&E\left[ \left( \int \limits _0^T e^{2\beta t}\vert \bar{Y}^{(l)}(t) \vert ^2\vert \bar{S}_{ij}^{(l)}(t) \vert ^2 d\left[ Q_{ij}\right] (t) \right) ^{\frac{1}{2}}\right] \\&\quad \le \dfrac{e^{\beta T}}{2}E\left[ \sup _{0\le t\le T}\vert \bar{Y}^{(l)}(t) \vert ^2+\int \limits _0^T\vert \bar{S}^{(l)}_{ij}(t) \vert ^2d\left[ Q_{ij}\right] (t) \right] <\infty . \end{aligned}$$

Hence $\int _0^t e^{\beta s}\bar{Y}(s)^\intercal \bar{Z}(s)dW(s)$ and $\int _0^te^{\beta s}\sum _{l=1}^n\sum _{i,j=1}^d\bar{Y}^{(l)}(s)\bar{S}_{ij}^{(l)}(s)dQ_{ij}(s)$ are true martingales by the Burkholder-Davis-Gundy inequality. Taking expectation in (8.1), we get

(8.2)

Take $\beta =1+4C_f^2$ and substitute into (8.2), we have

$$\begin{aligned} \begin{aligned}&E\bigg [ \int \limits _0^T e^{\beta t}\left( \vert \bar{Y}(t) \vert ^2+\vert \bar{Z}(t) \vert ^2 \right) dt+\int \limits _0^Te^{\beta t}\sum _{l=1}^n\sum _{i,j=1}^d\vert \bar{S}^{(l)}_{ij}(t)\vert ^2d[Q_{ij}](t) \bigg ]\\&\quad \le \dfrac{1}{2}E\bigg [ \int \limits _0^T e^{\beta t}\left( \vert \bar{U}(t) \vert ^2+\vert \bar{V}(t) \vert ^2 \right) dt \bigg ] \le \dfrac{1}{2}E\bigg [ \int \limits _0^T e^{\beta t}\left( \vert \bar{U}(t) \vert ^2+\vert \bar{V}(t) \vert ^2 \right) dt \bigg ]\\&\qquad +\dfrac{1}{2}E\bigg [ \int \limits _0^Te^{\beta t}\sum _{l=1}^n\sum _{i,j=1}^d\vert \bar{\Gamma }^{(l)}_{ij}(t) \vert ^2d[Q_{ij}](t) \bigg ]. \end{aligned} \end{aligned}$$

Notice that $L^2(W,[0,T])$ and $L^2(Q,[0,T])$ are Hilbert spaces and therefore the space $\mathbb {S}^2([0,T])\times L^2(W,[0,T])\times L^2(Q,[0,T])$ endowed with the norm

$$\begin{aligned} \Vert (Y,Z,S) \Vert _{\beta }&= \left\{ E\bigg [ \int \limits _0^T e^{\beta t}\left( \vert \bar{Y}(t) \vert ^2+\vert \bar{Z}(t) \vert ^2 \right) dt\right. \\&\left. +\int \limits _0^Te^{\beta t}\sum _{l=1}^n\sum _{i,j=1}^d\vert \bar{S}^{(l)}_{ij}(t)\vert ^2 d[Q_{ij}](t) \bigg ] \right\} ^{\frac{1}{2}} \end{aligned}$$

is a Banach space. We conclude that $\Phi$ admits a unique fixed point which is the solution to the BSDE (5.3).