Skip to main content
SpringerLink
Log in

Characterizations of Overtaking Optimality for Controlled Diffusion Processes

  • Published:
Applied Mathematics and Optimization Submit manuscript

Abstract

In this paper we give conditions for (the existence and) several characterizations of overtaking optimal policies for a general class of controlled diffusion processes. Our characterization results are of a lexicographical type; namely, first we identify the class of so-called canonical policies, and then within this class we search for policies with some special feature—for instance, canonical policies that in addition maximize the bias.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Arapostathis, A., Ghosh, M.K., Borkar, V.S.: Ergodic Control of Diffusion Processes (2006, in preparation)

  2. Atsumi, H.: Neoclassical growth and the efficient program of capital accumulation. Rev. Econ. Stud. 32, 127–136 (1965)

    Article  Google Scholar 

  3. Bensoussan, A., Frehse, J.: On Bellman equations of ergodic control in ℝn. J. Reine Angew. Math. 429, 125–160 (1992)

    MathSciNet  MATH  Google Scholar 

  4. Bhattacharya, R.N.: On classical limit theorems for diffusions. Sankhya 44, 47–71 (1982)

    MATH  Google Scholar 

  5. Borkar, V.S., Ghosh, M.K.: Ergodic control of multidimensional diffusions II: Adaptive control. Appl. Math. Optim. 21, 191–220 (1990)

    Article  MathSciNet  MATH  Google Scholar 

  6. Carlson, D.A., Haurie, D.B., Leizarowitz, A.: Infinite Horizon Optimal Control: Deterministic and Stochastic Systems. Springer, Berlin (1991)

    MATH  Google Scholar 

  7. Dana, R.A., Le Van, C.: On the Bellman equation of the overtaking criterion. J. Optim. Theory Appl. 67, 587–600 (1990)

    Article  MathSciNet  MATH  Google Scholar 

  8. Durrett, R.: Stochastic Calculus: A Practical Introduction. CRC Press, Boca Raton (1996)

    MATH  Google Scholar 

  9. Fort, G., Roberts, G.O.: Subgeometric ergodicity of strong Markov processes. Ann. Appl. Probab. 15, 1565–1589 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  10. Friedman, A.: Partial Differential Equations of Parabolic Type. Prentice Hall, Englewood Cliffs (1964)

    MATH  Google Scholar 

  11. Friedman, A.: Stochastic Differential Equations and Applications, vol. 1. Academic Press, New York (1975)

    MATH  Google Scholar 

  12. Ghosh, M.K., Arapostathis, A., Marcus, S.I.: Optimal control of switching diffusions to flexible manufacturing systems. SIAM J. Control Optim. 31, 1183–1204 (1993)

    Article  MathSciNet  MATH  Google Scholar 

  13. Ghosh, M.K., Arapostathis, A., Marcus, S.I.: Ergodic control of switching diffusions. SIAM J. Control Optim. 35, 1952–1988 (1997)

    Article  MathSciNet  MATH  Google Scholar 

  14. Ghosh, M.K., Marcus, S.I.: Infinite horizon controlled diffusion problems with nonstandard criteria. J. Math. Syst. Estim. Control 1, 44–69 (1991)

    MathSciNet  Google Scholar 

  15. Glynn, P.W., Meyn, S.P.: A Lyapunov bound for solutions of the Poisson equation. Ann. Probab. 24, 916–931 (1996)

    Article  MathSciNet  MATH  Google Scholar 

  16. Gordienko, E., Hernández-Lerma, O.: Average cost Markov control processes with weighted norms: existence of canonical policies. Appl. Math. (Warsaw) 23, 199–218 (1995)

    MathSciNet  MATH  Google Scholar 

  17. Guo, X.P., Hernández-Lerma, O.: Continuous-time Markov chains with discounted rewards. Acta Appl. Math. 79, 195–216 (2003)

    Article  MathSciNet  MATH  Google Scholar 

  18. Hashemi, S.N., Heunis, A.J.: On the Poisson equation for singular diffusions. Stochastics 77, 155–189 (2005)

    MathSciNet  Google Scholar 

  19. Has’minskii, R.Z.: Stochastic Stability of Differential Equations. Sijthoff & Noordhoff, Rockville (1980)

    Google Scholar 

  20. Hernández-Lerma, O.: Lectures on Continuous-Time Markov Control Processes. Sociedad Matemática Mexicana, Mexico City (1994)

    MATH  Google Scholar 

  21. Hernández-Lerma, O., Lasserre, J.B.: Discrete-Time Markov Control Processes. Springer, New York (1996)

    Google Scholar 

  22. Hernández-Lerma, O., Lasserre, J.B.: Further Topics on Discrete-Time Markov Control Processes. Springer, New York (1999)

    MATH  Google Scholar 

  23. Himmelberg, C.J., Parthasarathy, T., Van Vleck, F.S.: Optimal plans for dynamic programming problems. Math. Oper. Res. 1, 390–394 (1976)

    Article  MathSciNet  MATH  Google Scholar 

  24. Klokov, S.A., Veretennikov, A.Y.: On subexponential mixing rate for Markov processes. Theory Probab. Appl. 49, 110–122 (2005)

    Article  MathSciNet  Google Scholar 

  25. Kuratowski, K.: Topology II. Academic Press, New York (1968)

    Google Scholar 

  26. Le Van, C., Dana, R.A.: Dynamic Programming in Economics. Kluwer, Dordrecht (2003)

    MATH  Google Scholar 

  27. Leizarowitz, A.: Optimal controls for diffusion in ℝd—min–max max–min formula for the minimal cost growth rate. J. Math. Anal. Appl. 149, 180–209 (1990)

    Article  MathSciNet  MATH  Google Scholar 

  28. Leizarowitz, A.: Controlled diffusion process on infinite horizon with the overtaking criterion. Appl. Math. Optim. 17, 61–78 (1988)

    Article  MathSciNet  MATH  Google Scholar 

  29. Meyn, S.P., Tweedie, R.L.: Markov Chains and Stochastic Stability. Springer, New York (1993)

    MATH  Google Scholar 

  30. Meyn, S.P., Tweedie, R.L.: Stability of Markovian processes III: Foster–Lyapunov criteria for continuous-time precesses. Adv. Appl. Probab. 25, 518–548 (1993)

    Article  MathSciNet  MATH  Google Scholar 

  31. Nowak, A.S., Vega-Amaya, O.: A counterexample on overtaking optimality. Math. Meth. Oper. Res. 49, 435–439 (1998)

    MathSciNet  Google Scholar 

  32. Pardoux, E., Veretennikov, A.Y.: On the Poisson equation and diffusion approximation I. Ann. Probab. 29, 1061–1085 (2001)

    Article  MathSciNet  MATH  Google Scholar 

  33. Prieto-Rumeau, T., Hernandez-Lerma, O.: Bias and overtaking equilibria for zero-sum continuous-time Markov games. Math. Meth. Oper. Res. 61, 437–454 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  34. Prieto-Rumeau, T., Hernández-Lerma, O.: Bias optimality for continuous-time controlled Markov chains. SIAM J. Control Optim. 45, 51–73 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  35. Ramsey, F.P.: A mathematical theory of savings. Econ. J. 38, 543–559 (1928)

    Article  Google Scholar 

  36. Schäl, M.: Conditions for optimality and for the limit of n-stage optimal policies to be optimal. Z. Wahrs. Verw. Gerb. 32, 179–196 (1975)

    Article  MATH  Google Scholar 

  37. Tan, H., Rugh, W.J.: Nonlinear overtaking optimal control: sufficiency, stability, and approximation. IEEE Trans. Autom. Control 43, 1703–1718 (1998)

    Article  MathSciNet  MATH  Google Scholar 

  38. Tan, H., Rugh, W.J.: On overtaking optimal tracking for linear systems. Syst. Control Lett. 33, 63–72 (1998)

    Article  MathSciNet  MATH  Google Scholar 

  39. Tuominen, P., Tweedie, R.L.: The recurrence structure of general Markov processes. Proc. Lond. Math. Soc. 39, 554–576 (1979)

    Article  MathSciNet  MATH  Google Scholar 

  40. von Weizsäcker, C.C.: Existence of optimal programs of accumulation for an infinite horizon. Rev. Econ. Stud. 32, 85–104 (1965)

    Article  Google Scholar 

  41. Zaslavski, A.J.: Turnpike Properties in the Calculus of Variations and Optimal Control. Springer, New York (2006)

    MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Departamento de Matemáticas, CINVESTAV-IPN, A. Postal 14-740, México D.F., 07000, México

    Héctor Jasso-Fuentes & Onésimo Hernández-Lerma

Authors
  1. Héctor Jasso-Fuentes
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Onésimo Hernández-Lerma
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Onésimo Hernández-Lerma.

Additional information

This research was partially supported by CONACyT grant 45693-F. The research of the first author (HJF) was also supported by a CONACyT scholarship.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Jasso-Fuentes, H., Hernández-Lerma, O. Characterizations of Overtaking Optimality for Controlled Diffusion Processes. Appl Math Optim 57, 349–369 (2008). https://doi.org/10.1007/s00245-007-9025-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00245-007-9025-6

Keywords

Search

Navigation