Journal of Optimization Theory and Applications

, Volume 155, Issue 2, pp 572–593 | Cite as

Some Applications of Linear Programming Formulations in Stochastic Control

Article

Abstract

We present two applications of the linearization techniques in stochastic optimal control. In the first part, we show how the assumption of stability under concatenation for control processes can be dropped in the study of asymptotic stability domains. Generalizing Zubov’s method, the stability domain is then characterized as some level set of a semicontinuous generalized viscosity solution of the associated Hamilton–Jacobi–Bellman equation. In the second part, we extend our study to unbounded coefficients and apply the method to obtain a linear formulation for control problems whenever the state equation is a stochastic variational inequality.

Keywords

Stochastic control Linear programming HJB equations Zubov’s method Stochastic variational inequality 

References

  1. 1.
    Bhatt, A., Borkar, V.: Occupation measures for controlled Markov processes: Characterization and optimality. Ann. Probab. 24, 1531–1562 (1996) MathSciNetMATHCrossRefGoogle Scholar
  2. 2.
    Borkar, V., Gaitsgory, V.: Averaging of singularly perturbed controlled stochastic differential equations. Appl. Math. Optim. 56(2), 169–209 (2007) MathSciNetMATHCrossRefGoogle Scholar
  3. 3.
    Fleming, W., Vermes, D.: Convex duality approach to the optimal control of diffusions. SIAM J. Control Optim. 36(2), 1136–1155 (1989) MathSciNetCrossRefGoogle Scholar
  4. 4.
    Kurtz, T., Stockbridge, R.: Existence of Markov controls and characterization of optimal Markov control. SIAM J. Control Optim. 36(2), 609–653 (1998) MathSciNetMATHCrossRefGoogle Scholar
  5. 5.
    Lasserre, J., Henrion, D., Prieur, C., Trélat, E.: Nonlinear optimal control via occupational measures and LMI-relaxations. SIAM J. Control Optim. 47(4), 1643–1666 (2008) MathSciNetMATHCrossRefGoogle Scholar
  6. 6.
    Stockbridge, R.: Time-average control of a martingale problem. Existence of a stationary solution. Ann. Probab. 18, 190–205 (1990) MathSciNetMATHCrossRefGoogle Scholar
  7. 7.
    Yin, G., Zhang, Q.: Continuous-Time Markov Chains and Applications. A Singular Perturbation Approach. Springer, New York (1997) Google Scholar
  8. 8.
    Gaitsgory, V., Quincampoix, M.: Linear programming approach to deterministic infinite horizon optimal control problems with discounting. SIAM J. Control Optim. 48(4), 2480–2512 (2009) MathSciNetMATHCrossRefGoogle Scholar
  9. 9.
    Buckdahn, R., Goreac, D., Quincampoix, M.: Stochastic optimal control and linear programming approach. Appl. Math. Optim. 63(2), 257–276 (2011) MathSciNetMATHCrossRefGoogle Scholar
  10. 10.
    Goreac, D., Serea, O.: Mayer and optimal stopping stochastic control problems with discontinuous cost. J. Math. Anal. Appl. 380(1), 327–342 (2011) MathSciNetMATHCrossRefGoogle Scholar
  11. 11.
    Zubov, V.: Methods of A.M. Lyapunov and Their Application. Noordhoff, Groningen (1964). Translation prepared under the auspices of the United States Atomic Energy Commission; edited by Leo F. Boron MATHGoogle Scholar
  12. 12.
    Camilli, F., Grüne, L., Wirth, F.: A generalization of Zubov’s method to perturbed systems. SIAM J. Control Optim. 40(2), 496–515 (2001) MathSciNetMATHCrossRefGoogle Scholar
  13. 13.
    Malisoff, M.: Further results on Lyapunov functions and domains of attraction for perturbed asymptotically stable systems. Dyn. Contin. Discrete Impuls. Syst. 12(2), 193–225 (2005) MathSciNetMATHGoogle Scholar
  14. 14.
    Sontag, E.D.: A Lyapunov-like characterization of asymptotic controllability. SIAM J. Control Optim. 21(3), 462–471 (1983) MathSciNetMATHCrossRefGoogle Scholar
  15. 15.
    Grüne, L., Wirth, F.: Computing control Lyapunov functions via a Zubov type algorithm. In: Proceedings of the 39th IEEE Conference on Decision and Control, Sydney, Australia, pp. 2129–2134 (2000) Google Scholar
  16. 16.
    Grüne, L.: Asymptotic Behavior of Dynamical and Control Systems Under Perturbation and Discretization. Lecture Notes in Mathematics, vol. 1783. Springer, Berlin (2002) MATHCrossRefGoogle Scholar
  17. 17.
    Camilli, F., Grüne, L., Wirth, F.: Control Lyapunov functions and Zubov’s method. SIAM J. Control Optim. 47(1), 301–326 (2008) MathSciNetMATHCrossRefGoogle Scholar
  18. 18.
    Grüne, L., Serea, O.: Differential games and Zubov’s method. Preprint (2011) Google Scholar
  19. 19.
    Camilli, F., Grüne, L.: Characterizing attraction probabilities via the stochastic Zubov equation. Discrete Contin. Dyn. Syst., Ser. B 3(3), 457–468 (2003) MathSciNetMATHCrossRefGoogle Scholar
  20. 20.
    Camilli, F., Cesaroni, A., Grüne, L., Wirth, F.: Stabilization of controlled diffusions and Zubov’s method. Stoch. Dyn. 6(3), 373–393 (2006) MathSciNetMATHCrossRefGoogle Scholar
  21. 21.
    Grüne, L., Semmler, W., Bernard, L.: Firm value, diversified capital assets, and credit risk: towards a theory of default correlation. J. Credit Risk 3, 81–109 (2007/08) Google Scholar
  22. 22.
    Dupuis, P., Ishii, H.: SDEs with oblique reflection on nonsmooth domains. Ann. Probab. 1(21), 554–580 (1993) MathSciNetCrossRefGoogle Scholar
  23. 23.
    Asiminoaei, I., Rascanu, A.: Approximation and simulation of stochastic variational inequalities-splitting up method. Numer. Funct. Anal. Optim. 18(3–4), 251–282 (1996) MathSciNetGoogle Scholar
  24. 24.
    Zalinescu, A.: Second-order Hamilton–Jacobi–Bellman inequalities. C. R. Math. Acad. Sci. Paris, Ser. I 335, 591–596 (2002) MathSciNetMATHCrossRefGoogle Scholar
  25. 25.
    Bouchard, B.: Optimal reflection of diffusions and barrier options pricing under constraints. SIAM J. Control Optim. 47(4), 1785–1813 (2008) MathSciNetMATHCrossRefGoogle Scholar
  26. 26.
    Krylov, N.V.: On the rate of convergence of finite-difference approximations for Bellmanís equations with variable coefficients. Probab. Theory Relat. Fields 117(1), 1–16 (2000) MATHCrossRefGoogle Scholar
  27. 27.
    Barles, G., Jakobsen, E.R.: On the convergence rate of approximation schemes for Hamilton–Jacobi–Bellman equations. ESAIM, Math. Model. Numer. Anal. 36(1) (2002) Google Scholar
  28. 28.
    Camilli, F., Grüne, L., Wirth, F.: Construction of Lyapunov functions on the domain of asymptotic null controllability: numerics. In: Proceedings of NOLCOS 2004, Stuttgart, Germany, pp. 883–888 (2004) Google Scholar
  29. 29.
    Goebel, R., Rockafellar, R.: Generalized conjugacy in Hamilton–Jacobi theory for fully convex Lagrangians. J. Convex Anal. 9, 2 (2002) MathSciNetGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2012

Authors and Affiliations

  1. 1.LAMA, UMR8050Université Paris-EstMarne-la-ValléeFrance
  2. 2.Laboratoire de Mathématiques et de Physique, EA 4217Université de Perpignan Via DomitiaPerpignan CedexFrance

Personalised recommendations