Zubov’s method for controlled diffusions with state constraints

  • Lars Grüne
  • Athena PicarelliEmail author


We consider a controlled stochastic system in presence of state-constraints. Under the assumption of exponential stabilizability of the system near a target set, we aim to characterize the set of points which can be asymptotically driven by an admissible control to the target with positive probability. We show that this set can be characterized as a level set of the optimal value function of a suitable unconstrained optimal control problem which in turn is the unique viscosity solution of a second order PDE which can thus be interpreted as a generalized Zubov equation.

Mathematics Subject Classification

Primary 93B05 Secondary 93E20 49L25 


Controllability for diffusion systems Hamilton–Jacobi–Bellman equations Viscosity solutions Stochastic optimal control 


  1. 1.
    Abu Hassan M., Storey C.: Numerical determination of domains of attraction for electrical power systems using the method of Zubov. Int. J. Control 34, 371–381 (1981)zbMATHCrossRefGoogle Scholar
  2. 2.
    Artstein Z.: Stabilization with relaxed controls. Nonlinear Anal. 7, 1163–1173 (1983)zbMATHMathSciNetCrossRefGoogle Scholar
  3. 3.
    Aulbach, B.: Asymptotic stability regions via extensions of Zubov’s method. I and II. Nonlinear Anal. Theory Methods Appl. 7, 1431–1440 and 1441–1454 (1983)Google Scholar
  4. 4.
    Bardi, M., Capuzzo Dolcetta, I.: Optimal Control and Viscosity Solutions of Hamilton-Jacobi-Bellman equations. In: Systems Control Found. Appl. Birkhäuser, Boston (1997)Google Scholar
  5. 5.
    Barles G., Burdeau J.: The Dirichlet problem for semilinear second-order degenerate elliptic equations and applications to stochastic exit time control problems. Commun. Partial Differ. Equ. 20(1-2), 129–178 (1995)zbMATHMathSciNetCrossRefGoogle Scholar
  6. 6.
    Barles G., Daher C., Romano M.: Optimal control on the \({l^\infty}\) norm of a diffusion process. SIAM J. Control Optim. 32(3), 612–634 (1994)zbMATHMathSciNetCrossRefGoogle Scholar
  7. 7.
    Bokanowski, O., Picarelli, A., Zidani, H.: Dynamic Programming and Error Estimates for Stochastic Control Problems with Maximum Cost. Appl. Math. Optim. 71(1), 125–163 (2015)Google Scholar
  8. 8.
    Bouchard B., Touzi N.: Weak dynamic programming principle for viscosity solutions. SIAM J. Control Optim. 49(3), 948–962 (2011)zbMATHMathSciNetCrossRefGoogle Scholar
  9. 9.
    Camilli F., Cesaroni A., Grüne L., Wirth F.: Stabilization of controlled diffusions via Zubov’s method. Stochast. Dyn. 6, 373–394 (2006)zbMATHCrossRefGoogle Scholar
  10. 10.
    Camilli F., Grüne L.: Characterizing attraction probabilities via the stochastic zubov method. Discrete Contin. Dyn. Syst. B 3, 457–468 (2003)zbMATHCrossRefGoogle Scholar
  11. 11.
    Camilli F., Grüne L., Wirth F.: A generalization of the Zubov’s equation to perturbed systems. SIAM J. Control Optim. 40, 496–515 (2002)CrossRefGoogle Scholar
  12. 12.
    Camilli, F., Grüne, L., Wirth, F.: Characterizing controllability probabilities of stochastic control systems via Zubov’s method. In: Proceedings of the 16th International Symposium on Mathematical Theory of Networks and Systems (MTNS2004) (2004)Google Scholar
  13. 13.
    Camilli, F., Grüne, L., Wirth, F.: Control Lyapunov Functions and Zubov’s Method. SIAM J. Control Optim. 47, 301–326 (2008)Google Scholar
  14. 14.
    Camilli F., Loreti P.: A characterization of the domain of attraction for a locally exponentially stable stochastic system. NoDea 13, 205–222 (2006)zbMATHMathSciNetCrossRefGoogle Scholar
  15. 15.
    Cesaroni A.: Lyapunov stabilizability of controlled diffusions via a superoptimality principle for viscosity solutions. Appl. Math. Optim. 53(1), 1–29 (2006)zbMATHMathSciNetCrossRefGoogle Scholar
  16. 16.
    Crandall M.G., Ishii H., Lions P.L.: User’s guide to viscosity solutions of second order partial differential equations. Bull. Am. Math. Soc. 27(1), 1–67 (1992)zbMATHMathSciNetCrossRefGoogle Scholar
  17. 17.
    Dupuis P., Ishii H.: On oblique derivative problems for fully nonlinear second-order elliptic partial differential equations on nonsmooth domains. Nonlinear Anal. Theory Methods Appl. 15(12), 1123–1138 (1990)zbMATHMathSciNetCrossRefGoogle Scholar
  18. 18.
    Genesio R., Tartaglia M., Vicino A.: On the estimation of asymptotic stability regions: state of the art and new proposals. IEEE Trans. Autom. Control 30, 747–755 (1985)zbMATHMathSciNetCrossRefGoogle Scholar
  19. 19.
    Grüne, L., Zidani, H.: Zubov’s equation for state-constrained perturbed nonlinear systems. Math. Control Relat Fields 5(1), 55–71 (2014)Google Scholar
  20. 20.
    Hahn, W.: Stability of Motion (Translated from the German manuscript “Die Grundlehren der mathematischen Wissenschaften” Band 138, by Arne P. Baartz). Springer, New York (1967)Google Scholar
  21. 21.
    Hasminskii, R.Z.: Stochastic Stability of Differential equations. Sjithoff and Noordhoff International Publishers, Leyden (1980)Google Scholar
  22. 22.
    Haussmann U.G., Lepeltier J.P.: On the existence of optimal controls. SIAM J. Control Optim. 28(4), 851–902 (1990)zbMATHMathSciNetCrossRefGoogle Scholar
  23. 23.
    Ikeda, N., Watanabe, S.: Stochastic Differential Equations and Diffusion Processes, 1st edn. North Holland Publishing Company, Kodansha LTD, Amsterdam, Tokyo (1981)Google Scholar
  24. 24.
    Kirin N.E., Nelepin R.A., Bajdaev V.N.: Construction of the attraction region by Zubov’s method. Differ. Equ. 17, 871–880 (1982)zbMATHGoogle Scholar
  25. 25.
    Kurzweil, Y.: On the inversion of the second theorem of Lyapunov on stability of motion. Czechoslovak Math. J. 81(6), 217–259, 455–473 (1956)Google Scholar
  26. 26.
    Kushner H.J.: Converse theorems for stochastic Lyapunov functions. SIAM J. Control Optim. 5, 228–233 (1967)zbMATHMathSciNetCrossRefGoogle Scholar
  27. 27.
    Kushner, H.J.: Stochastic stability and control. Academic Press, New York (1967)Google Scholar
  28. 28.
    Kushner, H.J.: Stochastic stability. In: Stability of stochastic dynamical systems (Proc. In- ternat. Sympos., Univ. Warwick, Coventry, 1972), Lecture Notes in Math., vol. 294. Springer, Berlin (1972)Google Scholar
  29. 29.
    Lyapunov, A.M.: The general problem of the stability of motion (English translation), Int. J Control. 55(3), 531–534 (1992)Google Scholar
  30. 30.
    Massera J.L.: On Lyapunov’s condition of stability. Ann. Math. 50, 705–721 (1949)zbMATHMathSciNetCrossRefGoogle Scholar
  31. 31.
    Sontag E.: A Lyapunov-like characterization of asymptotic controllability. SIAM J. Control Opt. 21, 462–471 (1983)zbMATHMathSciNetCrossRefGoogle Scholar
  32. 32.
    Touzi, N.: Optimal stochastic control, stochastic target problems, and backward SDE. In: Fields Institute Monographs, vol. 49. Springer, Berlin (2012)Google Scholar
  33. 33.
    Zubov, V.I.: Methods of A.M. Lyapunov and their Application. P. Noordhoff, Groningen (1964)Google Scholar

Copyright information

© Springer Basel 2015

Authors and Affiliations

  1. 1.Universität BayreuthBayreuthGermany
  2. 2.Mathematical InstituteUniversity of OxfordOxfordUK

Personalised recommendations