Journal of Optimization Theory and Applications

, Volume 155, Issue 2, pp 572–593 | Cite as

Some Applications of Linear Programming Formulations in Stochastic Control

  • Dan GoreacEmail author
  • Oana-Silvia Serea


We present two applications of the linearization techniques in stochastic optimal control. In the first part, we show how the assumption of stability under concatenation for control processes can be dropped in the study of asymptotic stability domains. Generalizing Zubov’s method, the stability domain is then characterized as some level set of a semicontinuous generalized viscosity solution of the associated Hamilton–Jacobi–Bellman equation. In the second part, we extend our study to unbounded coefficients and apply the method to obtain a linear formulation for control problems whenever the state equation is a stochastic variational inequality.


Stochastic control Linear programming HJB equations Zubov’s method Stochastic variational inequality 



The second author would like to thank the ANR Jeudy: ANR-10-BLAN 0112 (Long term behavior for discrete and continuous time dynamic games), The GDR 2932 Théorie des Jeux and the FP7 European project SADCO—Sensitivity Analysis for Deterministic Controller Design (Marie Curie ITN-264735).


  1. 1.
    Bhatt, A., Borkar, V.: Occupation measures for controlled Markov processes: Characterization and optimality. Ann. Probab. 24, 1531–1562 (1996) MathSciNetzbMATHCrossRefGoogle Scholar
  2. 2.
    Borkar, V., Gaitsgory, V.: Averaging of singularly perturbed controlled stochastic differential equations. Appl. Math. Optim. 56(2), 169–209 (2007) MathSciNetzbMATHCrossRefGoogle Scholar
  3. 3.
    Fleming, W., Vermes, D.: Convex duality approach to the optimal control of diffusions. SIAM J. Control Optim. 36(2), 1136–1155 (1989) MathSciNetCrossRefGoogle Scholar
  4. 4.
    Kurtz, T., Stockbridge, R.: Existence of Markov controls and characterization of optimal Markov control. SIAM J. Control Optim. 36(2), 609–653 (1998) MathSciNetzbMATHCrossRefGoogle Scholar
  5. 5.
    Lasserre, J., Henrion, D., Prieur, C., Trélat, E.: Nonlinear optimal control via occupational measures and LMI-relaxations. SIAM J. Control Optim. 47(4), 1643–1666 (2008) MathSciNetzbMATHCrossRefGoogle Scholar
  6. 6.
    Stockbridge, R.: Time-average control of a martingale problem. Existence of a stationary solution. Ann. Probab. 18, 190–205 (1990) MathSciNetzbMATHCrossRefGoogle Scholar
  7. 7.
    Yin, G., Zhang, Q.: Continuous-Time Markov Chains and Applications. A Singular Perturbation Approach. Springer, New York (1997) Google Scholar
  8. 8.
    Gaitsgory, V., Quincampoix, M.: Linear programming approach to deterministic infinite horizon optimal control problems with discounting. SIAM J. Control Optim. 48(4), 2480–2512 (2009) MathSciNetzbMATHCrossRefGoogle Scholar
  9. 9.
    Buckdahn, R., Goreac, D., Quincampoix, M.: Stochastic optimal control and linear programming approach. Appl. Math. Optim. 63(2), 257–276 (2011) MathSciNetzbMATHCrossRefGoogle Scholar
  10. 10.
    Goreac, D., Serea, O.: Mayer and optimal stopping stochastic control problems with discontinuous cost. J. Math. Anal. Appl. 380(1), 327–342 (2011) MathSciNetzbMATHCrossRefGoogle Scholar
  11. 11.
    Zubov, V.: Methods of A.M. Lyapunov and Their Application. Noordhoff, Groningen (1964). Translation prepared under the auspices of the United States Atomic Energy Commission; edited by Leo F. Boron zbMATHGoogle Scholar
  12. 12.
    Camilli, F., Grüne, L., Wirth, F.: A generalization of Zubov’s method to perturbed systems. SIAM J. Control Optim. 40(2), 496–515 (2001) MathSciNetzbMATHCrossRefGoogle Scholar
  13. 13.
    Malisoff, M.: Further results on Lyapunov functions and domains of attraction for perturbed asymptotically stable systems. Dyn. Contin. Discrete Impuls. Syst. 12(2), 193–225 (2005) MathSciNetzbMATHGoogle Scholar
  14. 14.
    Sontag, E.D.: A Lyapunov-like characterization of asymptotic controllability. SIAM J. Control Optim. 21(3), 462–471 (1983) MathSciNetzbMATHCrossRefGoogle Scholar
  15. 15.
    Grüne, L., Wirth, F.: Computing control Lyapunov functions via a Zubov type algorithm. In: Proceedings of the 39th IEEE Conference on Decision and Control, Sydney, Australia, pp. 2129–2134 (2000) Google Scholar
  16. 16.
    Grüne, L.: Asymptotic Behavior of Dynamical and Control Systems Under Perturbation and Discretization. Lecture Notes in Mathematics, vol. 1783. Springer, Berlin (2002) zbMATHCrossRefGoogle Scholar
  17. 17.
    Camilli, F., Grüne, L., Wirth, F.: Control Lyapunov functions and Zubov’s method. SIAM J. Control Optim. 47(1), 301–326 (2008) MathSciNetzbMATHCrossRefGoogle Scholar
  18. 18.
    Grüne, L., Serea, O.: Differential games and Zubov’s method. Preprint (2011) Google Scholar
  19. 19.
    Camilli, F., Grüne, L.: Characterizing attraction probabilities via the stochastic Zubov equation. Discrete Contin. Dyn. Syst., Ser. B 3(3), 457–468 (2003) MathSciNetzbMATHCrossRefGoogle Scholar
  20. 20.
    Camilli, F., Cesaroni, A., Grüne, L., Wirth, F.: Stabilization of controlled diffusions and Zubov’s method. Stoch. Dyn. 6(3), 373–393 (2006) MathSciNetzbMATHCrossRefGoogle Scholar
  21. 21.
    Grüne, L., Semmler, W., Bernard, L.: Firm value, diversified capital assets, and credit risk: towards a theory of default correlation. J. Credit Risk 3, 81–109 (2007/08) Google Scholar
  22. 22.
    Dupuis, P., Ishii, H.: SDEs with oblique reflection on nonsmooth domains. Ann. Probab. 1(21), 554–580 (1993) MathSciNetCrossRefGoogle Scholar
  23. 23.
    Asiminoaei, I., Rascanu, A.: Approximation and simulation of stochastic variational inequalities-splitting up method. Numer. Funct. Anal. Optim. 18(3–4), 251–282 (1996) MathSciNetGoogle Scholar
  24. 24.
    Zalinescu, A.: Second-order Hamilton–Jacobi–Bellman inequalities. C. R. Math. Acad. Sci. Paris, Ser. I 335, 591–596 (2002) MathSciNetzbMATHCrossRefGoogle Scholar
  25. 25.
    Bouchard, B.: Optimal reflection of diffusions and barrier options pricing under constraints. SIAM J. Control Optim. 47(4), 1785–1813 (2008) MathSciNetzbMATHCrossRefGoogle Scholar
  26. 26.
    Krylov, N.V.: On the rate of convergence of finite-difference approximations for Bellmanís equations with variable coefficients. Probab. Theory Relat. Fields 117(1), 1–16 (2000) zbMATHCrossRefGoogle Scholar
  27. 27.
    Barles, G., Jakobsen, E.R.: On the convergence rate of approximation schemes for Hamilton–Jacobi–Bellman equations. ESAIM, Math. Model. Numer. Anal. 36(1) (2002) Google Scholar
  28. 28.
    Camilli, F., Grüne, L., Wirth, F.: Construction of Lyapunov functions on the domain of asymptotic null controllability: numerics. In: Proceedings of NOLCOS 2004, Stuttgart, Germany, pp. 883–888 (2004) Google Scholar
  29. 29.
    Goebel, R., Rockafellar, R.: Generalized conjugacy in Hamilton–Jacobi theory for fully convex Lagrangians. J. Convex Anal. 9, 2 (2002) MathSciNetGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2012

Authors and Affiliations

  1. 1.LAMA, UMR8050Université Paris-EstMarne-la-ValléeFrance
  2. 2.Laboratoire de Mathématiques et de Physique, EA 4217Université de Perpignan Via DomitiaPerpignan CedexFrance

Personalised recommendations