Applied Mathematics & Optimization

, Volume 66, Issue 2, pp 209–238 | Cite as

Linearization Techniques for Controlled Piecewise Deterministic Markov Processes; Application to Zubov’s Method

  • Dan Goreac
  • Oana-Silvia Serea


We aim at characterizing domains of attraction for controlled piecewise deterministic processes using an occupational measure formulation and Zubov’s approach. Firstly, we provide linear programming (primal and dual) formulations of discounted, infinite horizon control problems for PDMPs. These formulations involve an infinite-dimensional set of probability measures and are obtained using viscosity solutions theory. Secondly, these tools allow to construct stabilizing measures and to avoid the assumption of stability under concatenation for controls. The domain of controllability is then characterized as some level set of a convenient solution of the associated Hamilton-Jacobi integral-differential equation. The theoretical results are applied to PDMPs associated to stochastic gene networks. Explicit computations are given for Cook’s model for gene expression.


Stability domain Viscosity solutions PDMP Occupational measure Zubov’s method Gene networks 



The second author was supported in part by the ANR-10-BLAN 0112.


  1. 1.
    Bardi, M., Capuzzo Dolcetta, I.: Optimal Control and Viscosity Solutions of Hamilton-Jacobi-Bellman Equations. Systems and Control: Foundations and Applications. Birkhäuser, Boston (1997) zbMATHCrossRefGoogle Scholar
  2. 2.
    Barles, G., Jakobsen, E.R.: On the convergence rate of approximation schemes for Hamilton-Jacobi-Bellman equations. M2AN Math. Model. Numer. Anal. 36(1), 33–54 (2002) MathSciNetzbMATHCrossRefGoogle Scholar
  3. 3.
    Billingsley, P.: Convergence of Probability Measures, 2nd edn. Wiley Series in Probability and Statistics: Probability and Statistics. Wiley, New York (1999). A Wiley-Interscience Publication zbMATHCrossRefGoogle Scholar
  4. 4.
    Buckdahn, R., Goreac, D., Quincampoix, M.: Stochastic optimal control and linear programming approach. Appl. Math. Optim. 63(2), 257–276 (2011) MathSciNetzbMATHCrossRefGoogle Scholar
  5. 5.
    Camilli, F., Cesaroni, A., Grüne, L., Wirth, F.: Stabilization of controlled diffusions and Zubov’s method. Stoch. Dyn. 6(3), 373–393 (2006) MathSciNetzbMATHCrossRefGoogle Scholar
  6. 6.
    Camilli, F., Grüne, L.: Characterizing attraction probabilities via the stochastic Zubov equation. Discrete Contin. Dyn. Syst., Ser. B 3(3), 457–468 (2003) MathSciNetzbMATHCrossRefGoogle Scholar
  7. 7.
    Camilli, F., Grüne, L., Wirth, F.: A generalization of Zubov’s method to perturbed systems. SIAM J. Control Optim. 40(2), 496–515 (2001) MathSciNetzbMATHCrossRefGoogle Scholar
  8. 8.
    Camilli, F., Grüne, L., Wirth, F.: A regularization of Zubov’s equation for robust domains of attraction. In: Nonlinear Control in the Year 2000, Vol. 1 (Paris). Lecture Notes in Control and Inform. Sci., vol. 258, pp. 277–289. Springer, London (2001) CrossRefGoogle Scholar
  9. 9.
    Camilli, F., Grüne, L., Wirth, F.: Construction of Lyapunov functions on the domain of asymptotic nullcontrollability: numerics. In: Proceedings of NOLCOS 2004, Stuttgart, Germany, pp. 883–888 (2004) Google Scholar
  10. 10.
    Camilli, F., Grüne, L., Wirth, F.: Control Lyapunov functions and Zubov’s method. SIAM J. Control Optim. 47(1), 301–326 (2008) MathSciNetzbMATHCrossRefGoogle Scholar
  11. 11.
    Cook, D.L., Gerber, A.N., Tapscott, S.J.: Modelling stochastic gene expression: implications for haploinsufficiency. Proc. Natl. Acad. Sci. USA 95, 15641–15646 (1998) CrossRefGoogle Scholar
  12. 12.
    Crudu, A., Debussche, A., Radulescu, O.: Hybrid stochastic simplifications for multiscale gene networks. BMC Syst. Biol., 3, 89 (2009) CrossRefGoogle Scholar
  13. 13.
    Davis, M.H.A.: Markov Models and Optimization. Monographs on Statistics and Applied Probability, vol. 49. Chapman & Hall, London (1993) zbMATHGoogle Scholar
  14. 14.
    Delbruck, M.: Statistical fluctuations in autocatalytic reactions. J. Chem. Phys. 8(1), 120–124 (1940) CrossRefGoogle Scholar
  15. 15.
    Dempster, M.A.H.: Optimal control of piecewise deterministic Markov processes. In: Applied Stochastic Analysis, London, 1989. Stochastics Monogr., vol. 5, pp. 303–325. Gordon and Breach, New York (1991) Google Scholar
  16. 16.
    Gaitsgory, V., Quincampoix, M.: Linear programming approach to deterministic infinite horizon optimal control problems with discounting. SIAM J. Control Optim. 48(4), 2480–2512 (2009) MathSciNetzbMATHCrossRefGoogle Scholar
  17. 17.
    Giesl, P.: Construction of Global Lyapunov Functions Using Radial Basis Functions. Lecture Notes in Mathematics, vol. 1904. Springer, Berlin (2007) zbMATHGoogle Scholar
  18. 18.
    Goreac, D.: Viability, invariance and reachability for controlled piecewise deterministic Markov processes associated to gene networks. ESAIM Control Optim. Calc. Var. (2011). doi: 10.1051/cocv/2010103 Google Scholar
  19. 19.
    Goreac, D., Serea, O.S.: Linearization techniques for l -control problems and dynamic programming principles in classical and l -control problems. ESAIM Control Optim. Calc. Var. (2011). doi: 10.1051/cocv/2011183 Google Scholar
  20. 20.
    Goreac, D., Serea, O.S.: Mayer and optimal stopping stochastic control problems with discontinuous cost. J. Math. Anal. Appl. 380(1), 327–342 (2011) MathSciNetzbMATHCrossRefGoogle Scholar
  21. 21.
    Grüne, L.: Asymptotic Behavior of Dynamical and Control Systems Under Perturbation and Discretization. Lecture Notes in Mathematics, vol. 1783. Springer, Berlin (2002) zbMATHCrossRefGoogle Scholar
  22. 22.
    Grüne, L., Semmler, W., Bernard, L.: Firm value, diversified capital assets, and credit risk: towards a theory of default correlation. J. Credit Risk 3, 81–109 (2007/2008) Google Scholar
  23. 23.
    Grüne, L., Serea, O.S.: Differential games and Zubov’s method. Preprint (2011) Google Scholar
  24. 24.
    Grüne, L., Wirth, F.: Computing control Lyapunov functions via a Zubov type algorithm. In: Proceedings of the 39th IEEE Conference on Decision and Control, Sydney, Australia, pp. 2129–2134 (2000) Google Scholar
  25. 25.
    Hasty, J., Pradines, J., Dolnik, M., Collins, J.J.: Noise-based switches and amplifiers for gene expression. Proc. Natl. Acad. Sci. 97(5), 2075–2080 (2000) CrossRefGoogle Scholar
  26. 26.
    Krylov, N.V.: On the rate of convergence of finite-difference approximations for Bellman’s equations with variable coefficients. Probab. Theory Relat. Fields 117(1), 1–16 (2000) zbMATHCrossRefGoogle Scholar
  27. 27.
    Malisoff, M.: Further results on Lyapunov functions and domains of attraction for perturbed asymptotically stable systems. Dyn. Contin. Discrete Impuls. Syst. Ser. A Math. Anal. 12(2), 193–225 (2005) MathSciNetzbMATHGoogle Scholar
  28. 28.
    Soner, H.M.: Optimal control with state-space constraint. II. SIAM J. Control Optim. 24(6), 1110–1122 (1986) MathSciNetzbMATHCrossRefGoogle Scholar
  29. 29.
    Sontag, E.D.: A Lyapunov-like characterization of asymptotic controllability. SIAM J. Control Optim. 21(3), 462–471 (1983) MathSciNetzbMATHCrossRefGoogle Scholar
  30. 30.
    Zubov, V.I.: In: Boron, Leo F. (ed.) Methods of A.M. Lyapunov and Their Application. P. Noordhoff Ltd, Groningen (1964). Translation prepared under the auspices of the United States Atomic Energy Commission Google Scholar

Copyright information

© Springer Science+Business Media, LLC 2012

Authors and Affiliations

  1. 1.LAMA, UMR8050Université Paris-EstMarne-la-ValléeFrance
  2. 2.Laboratoire de Mathéematiques et PhySiqueUniv. Perpignan Via DomitiaPerpignanFrance
  3. 3.Combinatoire et Optimisation, IMJ CNRS UMR 7586Univ. Paris 6ParisFrance

Personalised recommendations