On the Policy Iteration Algorithm for Nondegenerate Controlled Diffusions Under the Ergodic Criterion

  • Ari ArapostathisEmail author
Part of the Systems & Control: Foundations & Applications book series (SCFA)


The subject of this chapter is the policy iteration algorithm for nondegenerate controlled diffusions. The results parallel the ones in Meyn (IEEE Trans Automat Control 42:1663–1680, 1997) for discrete-time controlled Markov chains. The model in (Meyn, IEEE Trans Automat Control 42:1663–1680, 1997) uses norm-like running costs, while we opt for the milder assumption of near-monotone costs. Also, instead of employing a blanket Lyapunov stability hypothesis, we provide a characterization of the region of attraction of the optimal control.



This work was supported in part by the Office of Naval Research through the Electric Ship Research and Development Consortium.


  1. 1.
    Arapostathis, A., Borkar, V.S.: Uniform recurrence properties of controlled diffusions and applications to optimal control. SIAM J. Control Optim. 48(7), 152–160 (2010)MathSciNetCrossRefGoogle Scholar
  2. 2.
    Arapostathis, A., Borkar, V.S., Fernández-Gaucherand, E., Ghosh, M.K., Marcus, S.I.: Discrete-time controlled Markov processes with average cost criterion: a survey. SIAM J. Control Optim. 31(2), 282–344 (1993)MathSciNetzbMATHCrossRefGoogle Scholar
  3. 3.
    Arapostathis, A., Borkar, V.S., Ghosh, M.K.: Ergodic control of diffusion processes, Encyclopedia of Mathematics and its Applications, vol. 143. Cambridge University Press, Cambridge (2011)CrossRefGoogle Scholar
  4. 4.
    Bensoussan, A., Borkar, V.: Ergodic control problem for one-dimensional diffusions with near-monotone cost. Systems Control Lett. 5(2), 127–133 (1984)MathSciNetzbMATHCrossRefGoogle Scholar
  5. 5.
    Bogachev, V.I., Krylov, N.V., Röckner, M.: On regularity of transition probabilities and invariant measures of singular diffusions under minimal conditions. Comm. Partial Differential Equations 26(11–12), 2037–2080 (2001)MathSciNetzbMATHCrossRefGoogle Scholar
  6. 6.
    Costa, O.L.V., Dufour, F.: The policy iteration algorithm for average continuous control of piecewise deterministic Markov processes. Appl. Math. Optim. 62(2), 185–204 (2010)MathSciNetzbMATHCrossRefGoogle Scholar
  7. 7.
    Gilbarg, D., Trudinger, N.S.: Elliptic partial differential equations of second order, Grundlehren der Mathematischen Wissenschaften, vol. 224, second edn. Springer-Verlag, Berlin (1983)Google Scholar
  8. 8.
    Gyöngy, I., Krylov, N.: Existence of strong solutions for Itô’s stochastic equations via approximations. Probab. Theory Related Fields 105(2), 143–158 (1996)MathSciNetzbMATHCrossRefGoogle Scholar
  9. 9.
    Hernández-Lerma, O., Lasserre, J.B.: Policy iteration for average cost Markov control processes on Borel spaces. Acta Appl. Math. 47(2), 125–154 (1997)MathSciNetzbMATHCrossRefGoogle Scholar
  10. 10.
    Krylov, N.V.: Controlled diffusion processes, Applications of Mathematics, vol. 14. Springer-Verlag, New York (1980)CrossRefGoogle Scholar
  11. 11.
    Meyn, S.P.: The policy iteration algorithm for average reward Markov decision processes with general state space. IEEE Trans. Automat. Control 42(12), 1663–1680 (1997)MathSciNetzbMATHCrossRefGoogle Scholar
  12. 12.
    Stannat, W.: (Nonsymmetric) Dirichlet operators on L 1: existence, uniqueness and associated Markov processes. Ann. Scuola Norm. Sup. Pisa Cl. Sci. (4) 28(1), 99–140 (1999)Google Scholar

Copyright information

© Springer Science+Business Media, LLC 2012

Authors and Affiliations

  1. 1.Department of Electrical and Computer EngineeringThe University of Texas at AustinAustinUSA

Personalised recommendations