Optimality Issues for a Class of Controlled Singularly Perturbed Stochastic Systems

  • Dan Goreac
  • Oana-Silvia SereaEmail author


The present paper aims at studying stochastic singularly perturbed control systems. We begin by recalling the linear (primal and dual) formulations for classical control problems. In this framework, we give necessary and sufficient support criteria for optimality of the measures intervening in these formulations. Motivated by these remarks, in a first step, we provide linearized formulations associated with the value function in the averaged dynamics setting. Second, these formulations are used to infer criteria allowing to identify the optimal trajectory of the averaged stochastic system.


Optimal stochastic control Singularly perturbed Brownian diffusions Occupation measures Linear programming 

Mathematics Subject Classfication

93E20 49J45 49L25 


  1. 1.
    Naidu, D.: Singular perturbations and time scales in control theory and applications: an overview 2002–2012. Dyn. Contin. Discrete. Impuls. Syst. Ser. B 9(24), 233–278 (2002)MathSciNetzbMATHGoogle Scholar
  2. 2.
    Zhang, Y., Naidu, D., Cai, C., Zou, Y.: Singular perturbations and time scales in control theory and applications: an overview. Int. J. Inf. Syst. Sci. 1, 1–36 (2014)Google Scholar
  3. 3.
    Fleming, W., Vermes, D.: Convex duality approach to the optimal control of diffusions. SIAM J. Control Optim. 36(2), 1136–1155 (1989)MathSciNetCrossRefGoogle Scholar
  4. 4.
    Bhatt, A., Borkar, V.: Occupation measures for controlled Markov processes: characterization and optimality. Ann. Probab. 24, 1531–1562 (1996)MathSciNetCrossRefzbMATHGoogle Scholar
  5. 5.
    Gaitsgory, V., Leizarowitz, A.: Limit occupational measures set for a control system and averaging of singularity perturbed control systems. J. Math. Anal. Appl. 233(2), 461–475 (1999)MathSciNetCrossRefzbMATHGoogle Scholar
  6. 6.
    Gaitsgory, V., Nguyen, M.T.: Multiscale singularly perturbed control systems: limit occupational measures sets and averaging. SIAM J. Control Optim. 41(3), 954–974 (2002)MathSciNetCrossRefzbMATHGoogle Scholar
  7. 7.
    Gaitsgory, V.: On a representation of the limit occupational measures set of a control system with applications to singularly perturbed control systems. SIAM J. Control Optim. 43(1), 325–340 (2004)MathSciNetCrossRefzbMATHGoogle Scholar
  8. 8.
    Gaitsgory, V., Rossomakhine, S.: Linear programming approach to deterministic long run average problems of optimal control. SIAM J. Control Optim. 44(6), 2006–2037 (2006)MathSciNetCrossRefzbMATHGoogle Scholar
  9. 9.
    Borkar, V., Gaitsgory, V.: Averaging of singularly perturbed controlled stochastic differential equations. Appl. Math. Optim. 56(2), 169–209 (2007)MathSciNetCrossRefzbMATHGoogle Scholar
  10. 10.
    Finlay, L., Gaitsgory, V., Lebedev, I.: Linear programming solutions of periodic optimization problems: approximation of the optimal control. J. Ind. Manag. Optim. 3(2), 399–413 (2007)MathSciNetCrossRefzbMATHGoogle Scholar
  11. 11.
    Gaitsgory, V., Quincampoix, M.: Linear programming approach to deterministic infinite horizon optimal control problems with discounting. SIAM J. Control Optim. 48(4), 2480–2512 (2009)Google Scholar
  12. 12.
    Bardi, M., Capuzzo Dolcetta, I.: Optimal control and viscosity solutions of Hamilton–Jacobi–Bellman equations. Systems and control: foundations and applications. Birkhäuser, Boston (1997)CrossRefGoogle Scholar
  13. 13.
    Buckdahn, R., Goreac, D., Quincampoix, M.: Stochastic optimal control and linear programming approach. Appl. Math. Optim. 63(2), 257–276 (2011)MathSciNetCrossRefzbMATHGoogle Scholar
  14. 14.
    Fridman, E.: Exact slow–fast decomposition of a class of non-linear singularly perturbed optimal control problems via invariant manifolds. Int. J. Control 72(17), 1609–1618 (1999)MathSciNetCrossRefzbMATHGoogle Scholar
  15. 15.
    Gaitsgory, V.: Suboptimization of singularly perturbed control systems. SIAM J. Control Optim. 30(5), 1228–1249 (1992)MathSciNetCrossRefzbMATHGoogle Scholar
  16. 16.
    Goreac, D., Serea, O.: Mayer and optimal stopping stochastic control problems with discontinuous cost. J. Math. Anal. Appl. 380(1), 327–342 (2011)MathSciNetCrossRefzbMATHGoogle Scholar
  17. 17.
    Goreac, D., Serea, O.: A note on linearization methods and dynamic programming principles for stochastic discontinuous control problems. Electron. Commun. Probab. 17(12), 1–12 (2012)MathSciNetGoogle Scholar
  18. 18.
    Borkar, V., Gaitsgory, V.: On existence of limit occupational measures set of a controlled stochastic differential equation. SIAM J. Control Optim. 44(4), 1436–1473 (2005). (electronic)MathSciNetCrossRefzbMATHGoogle Scholar
  19. 19.
    Tichonov, A.: Systems of differential equations containing small parameter near derivatives. Mat. Sbornik. 31, 575–586 (1952)MathSciNetGoogle Scholar
  20. 20.
    Veliov, V.: A generalization of the Tikhonov theorem for singularly perturbed differential inclusions. J. Dyn. Control Syst. 3(3), 291–319 (1997)MathSciNetCrossRefzbMATHGoogle Scholar
  21. 21.
    Grammel, G.: Singularly perturbed differential inclusions: an averaging approach. Set-Valued Anal. 4(4), 361–374 (1996)MathSciNetCrossRefzbMATHGoogle Scholar
  22. 22.
    Krylov, N.V.: On the rate of convergence of finite-difference approximations for Bellman’s equations with variable coefficients. Probab. Theory Relat. Fields 117(1), 1–16 (2000)CrossRefzbMATHGoogle Scholar
  23. 23.
    Buckdahn, R., Goreac, D., Quincampoix, M.: Existence of asymptotic values for nonexpansive stochastic control systems. Appl. Math. Optim. 70, 1–28 (2014)MathSciNetCrossRefzbMATHGoogle Scholar
  24. 24.
    Bardi, M., Goatin, P.: Invariant sets for controlled degenerate diffusions: a viscosity solutions approach. In: McEneaney, W., Yin, G., Zhang, Q. (eds.) Stochastic analysis, control, optimization and applications, systems & control: foundations and applications, pp. 191–208. Birkhäuser, Boston (1999)CrossRefGoogle Scholar
  25. 25.
    Buckdahn, R., Peng, S., Quincampoix, M., Rainer, C.: Existence of stochastic control under state constraints. C. R. Acad. Sci. Paris, Sér. I Math. 327, 17–22 (1998)MathSciNetCrossRefzbMATHGoogle Scholar
  26. 26.
    Goreac, D., Serea, O.S.: Linearization techniques for controlled piecewise deterministic Markov processes; application to Zubov’s method. Appl. Math. Optim. 66, 209–238 (2012)MathSciNetCrossRefzbMATHGoogle Scholar
  27. 27.
    Goreac, D.: On linearized formulations for control problems with piecewise deterministic Markov dynamics. In: B. published by Transilvania University Press, P.H. of the Romanian Academy (eds.) Bulletin of the Transilvania University of Brasov * series III: mathematics, informatics, physics. Special Issue: Proceedings of the Seventh Congress of Romanian Mathematicians, vol. 5, pp. 131–144. Brasov, Roumanie (2012)Google Scholar
  28. 28.
    Dufour, F., Stockbridge, R.H.: On the existence of strict optimal controls for constrained, controlled Markov processes in continuous time. Stochastics 84(1), 55–78 (2012)MathSciNetzbMATHGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2015

Authors and Affiliations

  1. 1.Université Paris-Est, LAMA (UMR 8050), UPEMLV, UPEC, CNRSMarne-la-ValléeFrance
  2. 2.Laboratoire de Mathématiques et de Physique, EA 4217Université de Perpignan Via DomitiaPerpignan CedexFrance

Personalised recommendations