Skip to main content
SpringerLink
Log in

Near-Optimal Control of Stochastic Recursive Systems Via Viscosity Solution

  • Published:
Journal of Optimization Theory and Applications Aims and scope Submit manuscript

Abstract

In this paper, we study the near-optimal control for systems governed by forward–backward stochastic differential equations via dynamic programming principle. Since the nonsmoothness is inherent in this field, the viscosity solution approach is employed to investigate the relationships among the value function, the adjoint equations along near-optimal trajectories. Unlike the classical case, the definition of viscosity solution contains a perturbation factor, through which the illusory differentiability conditions on the value function are dispensed properly. Moreover, we establish new relationships between variational equations and adjoint equations. As an application, a kind of stochastic recursive near-optimal control problem is given to illustrate our theoretical results.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

References

  1. Bismut, J.M.: Théorie Probabiliste du Contrôle des Diffusions. Memoirs of the American Mathematical Society, 176, Providence, Rhode Island (1973)

  2. Pardoux, E., Peng, S.G.: Adapted solution of a backward stochastic differential equation. Syst. Control Lett. 14, 55–61 (1990)

    Article  MathSciNet  MATH  Google Scholar 

  3. Duffie, D., Epstein, L.G.: Stochastic differential utility. Econometrica 60(2), 353–394 (1992)

    Article  MathSciNet  MATH  Google Scholar 

  4. El Karoui, N., Peng, S.G., Quenez, M.C.: A dynamic maximum principle for the optimization of recursive utilities under constraints. Ann. Appl. Probab. 11(3), 664–693 (2001)

    Article  MathSciNet  MATH  Google Scholar 

  5. Peng, S.G., Wu, Z.: Fully coupled forward–backward stochastic differential equations and applications to optimal control. SIAM J. Control Optim. 37(3), 825–843 (1999)

    Article  MathSciNet  MATH  Google Scholar 

  6. Pardoux, E., Tang, S.J.: Forward–backward stochastic differential equations and quasilinear parabolic PDEs. Probab. Theory Relat. Fields 114(2), 123–150 (1999)

    Article  MathSciNet  MATH  Google Scholar 

  7. Ma, J., Yong, J.M.: Forward–Backward Stochastic Differential Equations and their Applications. Springer, New York (1999)

    MATH  Google Scholar 

  8. Zhou, X.Y., Sethi, S.: A sufficient condition for near optimal stochastic controls and its applications to manufacturing systems. Appl. Math. Optim. 29, 67–92 (1994)

    Article  MathSciNet  MATH  Google Scholar 

  9. Chighoub, F., Mezerdi, B.: Near optimality conditions in stochastic control of jump diffusion processes. Syst. Control Lett. 60, 907–916 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  10. Lin, X.Y., Zhang, W.H.: A maximum principle for optimal control of discrete-time stochastic systems with multiplicative noise. IEEE Trans. Autom. Control 60(4), 1121–1126 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  11. Meng, Q.X., Shen, Y.: A revisit to stochastic near-optimal controls: the critical case. Syst. Control Lett. 82, 79–85 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  12. Liu, Y., Yin, G., Zhou, X.Y.: Near-optimal controls of random-switching LQ problems with indefinite control weight costs. Automatica 41, 1063–1070 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  13. Zhou, X.Y.: Deterministic near-optimal controls, part I: necessary and sufficient conditions for near-optimality. J. Optim. Theory Appl. 85, 473–488 (1995)

    Article  MathSciNet  MATH  Google Scholar 

  14. Zhou, X.Y.: Deterministic near-optimal controls, part II: dynamic programming and viscosity solution approach. Math. Oper. Res. 21, 655–674 (1996)

    Article  MathSciNet  MATH  Google Scholar 

  15. Zhou, X.Y.: Stochastic near-optimal controls: necessary and sufficient conditions for near-optimality. SIAM J. Control Optim. 39, 929–947 (1998)

    Article  MathSciNet  MATH  Google Scholar 

  16. Zhang, W.H., Lin, X.Y., Chen, B.: LaSalle-type theorem and its applications to infinite horizon optimal control of discrete-time nonlinear stochastic systems. IEEE Trans. Autom. Control 62, 1 (2016). https://doi.org/10.1109/TAC.2016.2558044

    MathSciNet  Google Scholar 

  17. Bahlali, K., Khelfallah, N., Mezerdi, B.: Necessary and sufficient conditions for near-optimality in stochastic control of FBSDEs. Syst. Control Lett. 58, 857–864 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  18. Hafayed, M., Abbas, S.: On near-optimal mean-field stochastic singular controls: necessary and sufficcient conditions for near-optimality. J. Optim. Theory Appl. 160(3), 778–808 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  19. Hafayed, M., Abbas, A., Abba, S.: On mean-field stochastic maximum principle for near-optimal controls for Poisson jump diffusion with applications. Int. J. Dyn. Control 2, 262–284 (2014)

    Article  Google Scholar 

  20. Hafayed, M., Abbas, S.: Stochastic near-optimal singular controls for jump diffusions: necessary and sufficient conditions. J. Dyn. Control Syst. 19(4), 503–517 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  21. Hafayed, M., Abbas, S., Veverka, P.: On necessary and sufficient conditions for near-optimal singular stochastic controls. Optim. Lett. 7(5), 949–966 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  22. Hafayed, M., Veverka, P., Abbas, S.: On maximum principle of near-optimality for diffusions with Jumps, with Application to consumption–investment problem. Differ. Equ. Dyn. Syst. 20(2), 111–125 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  23. Hafayed, M., Veverka, P., Abbas, A.: On near-optimal necessary and sufficient conditions for forward–backward stochastic systems with jumps, with applications to finance. Appl. Math. 59(4), 407–440 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  24. Huang, J.H., Li, X., Wang, G.C.: Near-optimal control problems for linear forward–backward stochastic systems. Automatica 46, 397–404 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  25. Hui, E., Huang, J.H., Li, X., Wang, G.C.: Near-optimal control for stochasic recursive problems. Syst. Control Lett. 60, 161–168 (2011)

    Article  MATH  Google Scholar 

  26. Zhang, L.Q., Huang, J.H., Li, X.: Necessary condition for near optimal control of linear forward–backward stochastic differential equations. Int. J. Control 88(8), 1594–1608 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  27. Zhang, L.Q.: Sufficient condition for near-optimal control of general controlled linear forward–backward stochastic differential equations. Int. J. Dyn. Control 5(2), 306–313 (2017)

    Article  MathSciNet  Google Scholar 

  28. Tang, M.N.: Stochastic maximum principle of near-optimal control of fully coupled forward–backward stochastic differential equation. In: Abstract and Applied Analysis 2014, 12 (2014). https://doi.org/10.1155/2014/361259

  29. Nie, T.Y., Shi, J.T., Wu Z.: Connection between MP and DPP for stochastic recursive optimal control problems: viscosity solution framework in local case. In: Proceedings of 2016 American Control Conference, pp. 7225–7230, July 6–8, Boston (2016)

  30. Nie, T.Y., Shi, J.T., Wu, Z.: Connection between MP and DPP for stochastic recursive optimal control problems: viscosity solution framework in general case. SIAM J. Control Optim. 55(5), 3258–3294 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  31. Hu, Y., Peng, S.G.: Solution of forward–backward stochastic differential equations. Probab. Theory Relat. Fields 103, 273–283 (1995)

    Article  MathSciNet  MATH  Google Scholar 

  32. Yong, J.M., Zhou, X.Y.: Stochastic Controls: Hamiltonian Systems and HJB Equations. Springer, New York (1999)

    Book  MATH  Google Scholar 

  33. Ekeland, I.: On the variational principle. J. Math. Anal. Appl. 47, 324–353 (1974)

    Article  MathSciNet  MATH  Google Scholar 

  34. Peng, S.G.: A general stochastic maximum principle for optimal control problem. SIAM J. Control Optim. 28, 966–979 (1990)

    Article  MathSciNet  MATH  Google Scholar 

  35. Yong, J.M.: Optimality variational principle for controlled forward–backward stochastic differential equations with mixed initial–terminal conditions. SIAM J. Control Optim. 48, 4119–4156 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  36. Wu, Z.: A general maximum principle for optimal control of forward–backward stochastic systems. Automatica 49, 1473–1480 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  37. Hu, M.S.: Stochastic global maximum principle for optimization with recursive utilities. Probab. Uncertain. Quant. Risk 2, 1 (2017)

    Article  MathSciNet  Google Scholar 

  38. Zhang, L.Q.: Stochastic verification theorem of forward–backward controlled systems for viscosity solutions. Syst. Control Lett. 61, 649–654 (2012)

    Article  MathSciNet  MATH  Google Scholar 

Download references

Acknowledgements

The authors wish to thank the editor and the referees for their valuable comments and constructive suggestions which improved the presentation of this manuscript. We also thank Dr. J. Yang for her careful reading and suggestions. L. Zhang acknowledges the financial support partly by the National Nature Science Foundation of China (Nos. 11701040, 11471051 and 11371362) and Innovation Foundation of BUPT for Youth (No. 500417024). Q. Zhou acknowledges the financial support partly by the National Nature Science Foundation of China (Nos. 11471051 and 11371362).

Author information

Authors and Affiliations

  1. School of Science, Beijing University of Posts and Telecommunications, Beijing, China

    Liangquan Zhang & Qing Zhou

Authors
  1. Liangquan Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Qing Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Liangquan Zhang.

Additional information

Communicated by Boris Vexler.

Appendix: Proof of Lemma A.1

Appendix: Proof of Lemma A.1

Recalling the adjoint equation (6) and variation equations (11) and (13) again, we have

Lemma A.1

Under the assumptions (A1)–(A3), we have

$$\begin{aligned} \begin{array}{lll} \eta ^{z}\left( r\right) &{} = &{} -\left( \frac{p\left( r\right) }{q\left( r\right) }\right) ^{\top }\xi ^{z}\left( r\right) ,\qquad r\in \left[ t,T \right] ,\,P\text {-a.s.,} \\ \lambda ^{z}\left( r\right) &{} = &{} \bar{f}_{z}\left( r,\cdot \right) \left( \frac{p\left( r\right) }{q\left( r\right) }\right) ^{\top }\xi ^{z}\left( r\right) -\left( \frac{k\left( r\right) }{q\left( r\right) }\right) ^{\top }\xi ^{z}\left( r\right) \\ &{} &{} -\left( \frac{p\left( r\right) }{q\left( r\right) }\right) ^{\top }\bar{ \sigma }_{x}\left( r,\cdot \right) \xi ^{z}\left( r\right) ,\qquad r\in \left[ t,T\right] ,\,P\text {-a.s.,} \end{array} \end{aligned}$$

where \(\eta ^{z}\left( r\right) \) and \(\lambda ^{z}\left( r\right) \) are defined in (12).

Proof

We first compute by Itô’s formula

$$\begin{aligned} \mathrm {d}\left( \frac{p\left( r\right) }{q\left( r\right) }\right)= & {} \Bigg (\bar{f}_{x}\left( r,\cdot \right) -\frac{p\left( r\right) }{q\left( r\right) }\bar{b}_{x}\left( r,\cdot \right) -\frac{k\left( r\right) }{ q\left( r\right) }\bar{\sigma }_{x}\left( r,\cdot \right) -\frac{p\left( r\right) }{q\left( r\right) }\bar{f}_{y}\left( r,\cdot \right) \\&+\frac{p\left( r\right) }{q\left( r\right) }\left( \bar{f}_{z}\left( r,\cdot \right) \right) ^{2}-\frac{k\left( r\right) }{q\left( r\right) }\bar{ f}_{z}\left( r,\cdot \right) \Bigg )\mathrm {d}r \\&+\left( \frac{k\left( r\right) }{q\left( r\right) }-\frac{p\left( r\right) }{q\left( r\right) }\bar{f}_{z}\left( r,\cdot \right) \right) \mathrm {d} W\left( r\right) . \end{aligned}$$

Then,

$$\begin{aligned}&-\mathrm {d}\left[ \left( \frac{p\left( r\right) }{q\left( r\right) } \right) ^{\top }\xi ^{z}\left( r\right) \right] \nonumber \\&\quad =\Bigg (-\bar{f}_{x}\left( r,\cdot \right) ^{\top }\xi ^{z}\left( r\right) \nonumber \\&\qquad -\bar{f}_{y}\left( r,\cdot \right) \left( -\frac{p\left( r\right) }{q\left( r\right) }\right) ^{\top }\xi ^{z}\left( r\right) -\left( \frac{p\left( r\right) }{q\left( r\right) }\right) ^{\top }\left( \bar{f}_{z}\left( r,\cdot \right) \right) ^{2}\xi ^{z}\left( r\right) \nonumber \\&\qquad +\left( \frac{k\left( r\right) }{q\left( r\right) }\right) ^{\top }\bar{f} _{z}\left( r,\cdot \right) \xi ^{z}\left( r\right) +\bar{f}_{z}\left( r,\cdot \right) \left( \frac{p\left( r\right) }{q\left( r\right) }\right) ^{\top }\bar{\sigma }_{x}\left( r,\cdot \right) \xi ^{z}\left( r\right) \nonumber \\&\qquad -\left( \frac{p\left( r\right) }{q\left( r\right) }\right) ^{\top }\varepsilon _{z1}\left( r\right) -\left( \frac{k\left( r\right) }{q\left( r\right) }\right) ^{\top }\varepsilon _{z2}\left( r\right) +\bar{f} _{z}\left( r,\cdot \right) \left( \frac{p\left( r\right) }{q\left( r\right) } \right) ^{\top }\varepsilon _{z2}\left( r\right) \Bigg )\mathrm {d}r \nonumber \\&\qquad +\Bigg (\bar{f}_{z}\left( r,\cdot \right) \left( \frac{p\left( r\right) }{ q\left( r\right) }\right) ^{\top }\xi ^{z}\left( r\right) -\left( \frac{ p\left( r\right) }{q\left( r\right) }\right) ^{\top }\bar{\sigma }_{x}\left( r,\cdot \right) \xi ^{z}\left( r\right) -\left( \frac{k\left( r\right) }{ q\left( r\right) }\right) ^{\top }\xi ^{z}\left( r\right) \nonumber \\&\qquad +\left( \frac{p\left( r\right) }{q\left( r\right) }\right) ^{\top }\varepsilon _{z2}\left( r\right) \Bigg )\mathrm {d}W\left( r\right) \end{aligned}$$
(25)

Comparing (25) with (13), we get the desired results. \(\square \)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhang, L., Zhou, Q. Near-Optimal Control of Stochastic Recursive Systems Via Viscosity Solution. J Optim Theory Appl 178, 363–382 (2018). https://doi.org/10.1007/s10957-018-1300-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10957-018-1300-y

Keywords

Mathematics Subject Classification

Search

Navigation