Skip to main content
Log in

Weak Necessary and Sufficient Stochastic Maximum Principle for Markovian Regime-Switching Diffusion Models

  • Published:
Applied Mathematics & Optimization Aims and scope Submit manuscript

Abstract

In this paper we prove a weak necessary and sufficient maximum principle for Markovian regime switching stochastic optimal control problems. Instead of insisting on the maximum condition of the Hamiltonian, we show that \( 0 \) belongs to the sum of Clarke’s generalized gradient of the Hamiltonian and Clarke’s normal cone of the control constraint set at the optimal control. Under a joint concavity condition on the Hamiltonian and a convexity condition on the terminal objective function, the necessary condition becomes sufficient. We give four examples to demonstrate the weak stochastic maximum principle.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Similar content being viewed by others

References

  1. Aubin, J.-P., Frankowska, H.: Set-valued analysis. Birkhäuser (1990)

  2. Bensoussan, A.: Lectures on stochastic control. Lect. Notes Math. 972, 1–62 (1981)

    Article  MathSciNet  Google Scholar 

  3. Bismut, J.M.: Conjugate convex functions in optimal stochastic control. J. Math. Anal. Appl. 44, 384–404 (1973)

    Article  MathSciNet  Google Scholar 

  4. Bismut, J.M.: Linear quadratic optimal control with random coefficients. SIAM J. Control Optim. 14, 419–444 (1976)

    Article  MATH  MathSciNet  Google Scholar 

  5. Bismut, J.M.: An introductory approach to duality in optimal stochastic control. SIAM Rev. 20, 62–78 (1978)

    Article  MATH  MathSciNet  Google Scholar 

  6. Cadenillas, A., Karatzas, I.: The stochastic maximum principle for linear convex optimal control with random coefficients. SIAM J. Control Optim. 33, 590–624 (1995)

    Article  MATH  MathSciNet  Google Scholar 

  7. Clarke, F.H.: Generalized gradients and applications. Trans. Am. Math. Soc. 205, 247 (1975)

    Article  MATH  Google Scholar 

  8. Clarke, F.H.: Shadow prices and duality for a class of optimal control problems. SIAM J. Control Optim. 17, 567 (1979)

    Article  MATH  MathSciNet  Google Scholar 

  9. Clarke, F.H.: Generalized gradients of lipschitz functionals. Adv. Math. 40, 52–67 (1981)

    Article  MATH  Google Scholar 

  10. Clarke, F.H.: Optimization and nonsmooth analysis. SIAM (1990)

  11. Cohen, S., Elliott, R.: Solutions of backward stochastic differential equations on markov chains. Commun. Stoch. Anal. 2, 251–262 (2008)

    MathSciNet  Google Scholar 

  12. Cohen, S., Elliott, R.: Comparisons for backward stochastic dierential equations on markov chains and related no-arbitrage conditions. Ann. Appl. Probab. 20, 267–311 (2010)

    Article  MATH  MathSciNet  Google Scholar 

  13. Crepey, S.: About the pricing equations in finance. In: Ekeland, I., Jouini, E., Scheinkman, J., Touzi, N., Carmona, R., Cinlar, E. (eds.) Paris-Princeton Lectures on Mathematical Finance, pp. 63–203. Springer, Berlin (2010)

    Google Scholar 

  14. Donnelly, C: Convex duality in constrained mean-variance portfolio optimization under a regime-switching model. PhD thesis, University of Waterloo (2008)

  15. Donnelly, C.: Sufficient stochastic maximum principle in the regime-switching diffusion model. Appl. Math. Optim. 62(2), 155–169 (2011)

    Article  MathSciNet  Google Scholar 

  16. Donnelly, C., Heunis, A.J.: Quadratic risk minimization in a regime-switching model with portfolio constraints. SIAM J. Control Optim. 50(4), 2431–2461 (2012)

    Article  MATH  MathSciNet  Google Scholar 

  17. Fleming, W.H., Rishel, R.W.: Deterministic and Stochastic Optimal Control. Springer, Berlin (1975)

    Book  MATH  Google Scholar 

  18. Fleming, W.H., Soner, H.M.: Controlled Markov Processes and Viscosity Solutions. Springer, Berlin (2006)

    MATH  Google Scholar 

  19. Framstad, N.C., Oksendal, B., Sulem, A.: Sufficient stochastic maximum principle for the optimal control of jump difussions and applications to finance. J. Optim. Theory Appl. 121(1), 77–98 (2004)

    Article  MathSciNet  Google Scholar 

  20. Haussmann, U.G.: A Stochastic Maximum Principle for Optimal Control of Diffusions. Longman Scientific and Technical (1986)

  21. Kushner, H.J.: On the stochastic maximum principle: fixed time of control. J. Math. Anal. Appl. 11, 78–92 (1965)

    Article  MATH  MathSciNet  Google Scholar 

  22. Kushner, H.J.: Necessary conditons for continuous parameter stochastic optimization problems. SIAM J. Control Optim. 10, 550–565 (1972)

    Article  MATH  MathSciNet  Google Scholar 

  23. Mao, X., Yuan, C.: Stochastic Differential Equations with Markovian Switching. Imperial College Press, London (2006)

    Book  MATH  Google Scholar 

  24. Peng, S.: A general stochastic maximum principle for optimal control problems. SIAM J. Control Optim. 28, 966–979 (1990)

    Article  MATH  MathSciNet  Google Scholar 

  25. Peng, S.: Nonlinear expectations, nonlinear evaluations and risk measures. In: Hipp, C., Peng, S., Schachermayer, W., Back, K., Bielecki, T.R. (eds.) Stochastic Methods in Finance, pp. 165–253. Springer, Berlin (2004)

    Chapter  Google Scholar 

  26. Pham, H.: Continuous-Time Stochastic Control and Optimization with Financial Applications. Springer, Berlin (2009)

    Book  MATH  Google Scholar 

  27. Rockafeller, R.T.: Convex Analysis. Princeton University Press, Princeton (1970)

    Google Scholar 

  28. Rogers, L.C.G.: Diffusions, Markov Processes and Martingales: Volume 2, Itô Calculus. Cambridge University Press, Cambridge (2000)

    Book  Google Scholar 

  29. Tao, R., Wu, Z.: Maximum principle for optimal control problems of forwardcbackward regime-switching system and applications. Syst. Control Lett. 61, 911–917 (2012)

    Article  MATH  MathSciNet  Google Scholar 

  30. Tao, R., Wu, Z., Zhang, Q.: Bsdes with regime switching: weak convergence and applications. J. Math. Anal. Appl. 407, 97–111 (2013)

    Article  MathSciNet  Google Scholar 

  31. Tang, S., Li, X.: Necessary conditions for optimality control of stochastic systems with random jumps. SIAM J. Control Optim. 32, 1447–1475 (1994)

    Article  MATH  MathSciNet  Google Scholar 

  32. Wu, Z.: A general maximum principle for optimal control of forward–backward stochastic systems. Automatica 49, 1473–1480 (2013)

    Article  Google Scholar 

  33. Yong, J., Zhou, X.Y.: Stochastic Controls: Hamiltonian Systems and HJB Equations. Springer, New York (1999)

    Book  MATH  Google Scholar 

  34. Zhang, X., Elliott, R.J., Siu, T.K.: A stochastic maximum principle for a markov regime-switching jump-diffusion model and its application to finance. SIAM J. Control Optim. 50, 964–990 (2012)

    Article  MATH  MathSciNet  Google Scholar 

  35. Zhou, X.Y.: A unified treatment of maximum principle and dynamic programming in stochastic controls. Stoch. Stoch. Rep. 36, 137–161 (1991)

    MATH  Google Scholar 

  36. Zhou, X.Y.: Sufficient conditions of optimality for stochastic systems with controllable diffusions. IEEE Trans. Autom. Control 41, 1176–1179 (1996)

    Article  MATH  Google Scholar 

Download references

Acknowledgments

The authors are grateful to Professor Nicole El Karoui for the useful discussions on the paper, especially on the contents of the measurability of stochastic processes. The authors also thank two anonymous referees for their comments that have helped to improve the previous version.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Harry Zheng.

Appendix

Appendix

1.1 Proof of Theorem 5.15

Proof

Consider the function \( \Phi \) on \( \mathbb {S}^2([0,T])\times L^2(W,[0,T])\times L^2(Q,[0,T]) \) mapping \( (Y,Z,S)\in \mathbb {S}^2([0,T])\times L^2(W,[0,T])\times L^2(Q,[0,T]) \) to \( \left( \hat{Y},\hat{Z},\hat{S}\right) =\Phi (Y,Z,S) \) defined by

$$\begin{aligned} \hat{Y}(t)=\xi +\int \limits _t^Tf(s,Y(s),Z(s))ds-\int \limits _t^TZ(s)dW(s)-\int \limits _t^TS(s)\cdot dQ(s). \end{aligned}$$

Consider the square-integrable martingale

$$\begin{aligned} M(t)=E\left[ \xi +\int \limits _0^Tf(s,Y(s),Z(s))ds\bigg |\mathcal {F}_t\right] . \end{aligned}$$

According to Theorem 5.13, there exists unique \( \left( \hat{Z},\hat{S} \right) \in L^2(W,[0,T])\times L^2(Q,[0,T]) \) such that

$$\begin{aligned} M(t)=M(0)+\int \limits _0^t\hat{Z}(s)dW(s)+\int \limits _0^t\hat{S}(s)\cdot dQ(s). \end{aligned}$$

We then define the process \( \hat{Y}(t) \) by

$$\begin{aligned} \hat{Y}(t)&= E\left[ \xi +\int \limits _t^Tf(s,Y(s),Z(s))ds\bigg |\mathcal {F}_t \right] =M(t)-\int \limits _0^tf(s,\alpha (s),Y(s),Z(s))ds\\&= M(0)+\int \limits _0^t\hat{Z}(s)dW(s)+\int \limits _0^t\hat{S}(s)\cdot dQ(s)-\int \limits _0^tf(s,Y(s),Z(s))ds\\&= \xi +\int \limits _t^Tf(s,Y(s),Z(s))ds-\int \limits _t^T\hat{Z}(s)dW(s)-\int \limits _t^T\hat{S}(s)\cdot dQ(s). \end{aligned}$$

By Doob’s \( L^2 \) inequality, we have

$$\begin{aligned} \begin{aligned}&E\left[ \sup _{0\le t\le T}\bigg |\int \limits _t^T\hat{Z}(s)dW(s)\bigg |\right] \le 4E\left[ \int \limits _0^T|\hat{Z}(s)|^2ds \right] <\infty ,\\&E\left[ \sup _{0\le t\le T}\bigg |\int \limits _t^T\hat{S}(s)\cdot dQ(s)\bigg |\right] \le 4E\left[ \sum _{l=1}^n\sum _{i,j=1}^d\int \limits _0^T|\hat{S}_{ij}^{(l)}(s)|^2d[Q_{ij}](s) \right] <\infty . \end{aligned} \end{aligned}$$

Under the assumptions on \( (\xi ,f) \), we conclude that \( \hat{Y}\in S^2([0,T]) \). Hence \( \Phi \) is a well defined function from \( S^2([0,T])\times L^2(W,[0,T])\times L^2(Q,[0,T]) \) into itself. Next, we show that \( (\hat{Y},\hat{Z},\hat{S}) \) is a solut ion to the regime switching BSDE (5.3) if and only if it is a fixed point of \( \Phi \).

Let \( (U,V,\Gamma ) \), \( (U',V',\Gamma ') \in S^2([0,T])\times L^2(W,[0,T])\times L^2(Q,[0,T])\). Apply function \( \Phi \) and obtain \( (Y,Z,S)=\Phi (U,V,\Gamma ),\ (Y',Z',S')=\Phi (U',V',\Gamma ') \). Set \( (\bar{U},\bar{V},\bar{\Gamma }) =(U-U',V-V',\Gamma -\Gamma ')\), \( (\bar{Y},\bar{Z},\bar{S})=(Y-Y',Z-Z',S-S') \) and \( \bar{f}(t)=f(t,U(t),V(t))-f(t,U'(t),V'(t)) \). Take \( \beta >0 \) to be chosen later and apply Ito’s formula to \( e^{\beta s}\vert \bar{Y} \vert ^2 \) on \( [0,T] \),

$$\begin{aligned} \begin{aligned} \vert \bar{Y}(0) \vert ^2 =&-\displaystyle \int \limits _0^Te^{\beta t}\left( \beta \vert \bar{Y}(t)\vert ^2-2\bar{Y}(t)^\intercal \bar{f}(t) \right) dt-\int \limits _0^Te^{\beta t}\vert \bar{Z}(t) \vert ^2dt\\&-\displaystyle \int \limits _0^Te^{\beta t}\sum _{l=1}^n\sum _{i,j=1}^d\vert \bar{S}^{(l)}_{ij} \vert ^2d\left[ Q_{ij}\right] (t)-2\int \limits _0^T e^{\beta t}\bar{Y}(t)^\intercal \bar{Z}(t)dW(t)\\&-2\displaystyle \int \limits _0^Te^{\beta t}\sum _{l=1}^n\sum _{i,j=1}^d\bar{Y}^{(l)}(t)\bar{S}_{ij}^{(l)}(t)dQ_{ij}(t). \end{aligned} \end{aligned}$$
(8.1)

Observe that, according to Young’s inequality

$$\begin{aligned}&E\left[ \left( \int \limits _0^T e^{2\beta t}\vert \bar{Y}(t) \vert ^2\vert \bar{Z}(t) \vert ^2dt \right) ^{\frac{1}{2}}\right] \le \dfrac{e^{\beta T}}{2}E\left[ \sup _{0\le t\le T}\vert \bar{Y}(t) \vert ^2+\int \limits _0^T\vert \bar{Z}(t) \vert dt \right] <\infty ,\\&E\left[ \left( \int \limits _0^T e^{2\beta t}\vert \bar{Y}^{(l)}(t) \vert ^2\vert \bar{S}_{ij}^{(l)}(t) \vert ^2 d\left[ Q_{ij}\right] (t) \right) ^{\frac{1}{2}}\right] \\&\quad \le \dfrac{e^{\beta T}}{2}E\left[ \sup _{0\le t\le T}\vert \bar{Y}^{(l)}(t) \vert ^2+\int \limits _0^T\vert \bar{S}^{(l)}_{ij}(t) \vert ^2d\left[ Q_{ij}\right] (t) \right] <\infty . \end{aligned}$$

Hence \( \int _0^t e^{\beta s}\bar{Y}(s)^\intercal \bar{Z}(s)dW(s) \) and \( \int _0^te^{\beta s}\sum _{l=1}^n\sum _{i,j=1}^d\bar{Y}^{(l)}(s)\bar{S}_{ij}^{(l)}(s)dQ_{ij}(s) \) are true martingales by the Burkholder-Davis-Gundy inequality. Taking expectation in (8.1), we get

(8.2)

Take \( \beta =1+4C_f^2 \) and substitute into (8.2), we have

$$\begin{aligned} \begin{aligned}&E\bigg [ \int \limits _0^T e^{\beta t}\left( \vert \bar{Y}(t) \vert ^2+\vert \bar{Z}(t) \vert ^2 \right) dt+\int \limits _0^Te^{\beta t}\sum _{l=1}^n\sum _{i,j=1}^d\vert \bar{S}^{(l)}_{ij}(t)\vert ^2d[Q_{ij}](t) \bigg ]\\&\quad \le \dfrac{1}{2}E\bigg [ \int \limits _0^T e^{\beta t}\left( \vert \bar{U}(t) \vert ^2+\vert \bar{V}(t) \vert ^2 \right) dt \bigg ] \le \dfrac{1}{2}E\bigg [ \int \limits _0^T e^{\beta t}\left( \vert \bar{U}(t) \vert ^2+\vert \bar{V}(t) \vert ^2 \right) dt \bigg ]\\&\qquad +\dfrac{1}{2}E\bigg [ \int \limits _0^Te^{\beta t}\sum _{l=1}^n\sum _{i,j=1}^d\vert \bar{\Gamma }^{(l)}_{ij}(t) \vert ^2d[Q_{ij}](t) \bigg ]. \end{aligned} \end{aligned}$$

Notice that \( L^2(W,[0,T]) \) and \( L^2(Q,[0,T]) \) are Hilbert spaces and therefore the space \( \mathbb {S}^2([0,T])\times L^2(W,[0,T])\times L^2(Q,[0,T]) \) endowed with the norm

$$\begin{aligned} \Vert (Y,Z,S) \Vert _{\beta }&= \left\{ E\bigg [ \int \limits _0^T e^{\beta t}\left( \vert \bar{Y}(t) \vert ^2+\vert \bar{Z}(t) \vert ^2 \right) dt\right. \\&\left. +\int \limits _0^Te^{\beta t}\sum _{l=1}^n\sum _{i,j=1}^d\vert \bar{S}^{(l)}_{ij}(t)\vert ^2 d[Q_{ij}](t) \bigg ] \right\} ^{\frac{1}{2}} \end{aligned}$$

is a Banach space. We conclude that \( \Phi \) admits a unique fixed point which is the solution to the BSDE (5.3).

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Li, Y., Zheng, H. Weak Necessary and Sufficient Stochastic Maximum Principle for Markovian Regime-Switching Diffusion Models. Appl Math Optim 71, 39–77 (2015). https://doi.org/10.1007/s00245-014-9252-6

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00245-014-9252-6

Keywords

Mathematics Subject Classification

Navigation