Skip to main content
SpringerLink
Log in

Stationary and Markov policies in countable state dynamic programming

  • Conference paper
  • First Online:
Probability Theory and Mathematical Statistics

Part of the book series: Lecture Notes in Mathematics ((LNM,volume 1021))

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 54.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 69.95
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Bertsekas, D.P. and S.E. Shreve, Existence of optimal stationary policies in deterministic optimal control, J. Math. Anal. Appl., 1979, v. 69, 607–620.

    Article  MATH  MathSciNet  Google Scholar 

  2. Blackwell, D., Discrete dynamic programming, Ann. Math. Statist., 1962, v. 33, 719–726.

    Article  MATH  MathSciNet  Google Scholar 

  3. Blackwell, D., Positive dynamic programming, Proceedings of the 5th Berkeley symposium on Mathematical Statistics and Probability, 1967, v. I, 159–163.

    Google Scholar 

  4. Dynkin, E.B. and A.A. Yushkevich, Controlled Markov Processes, Springer-Verlag, Berlin, 1979.

    Book  Google Scholar 

  5. Fainberg, E.A., Nonrandomized Markov and semi-Markov policies in dynamic programming, Theory Probability Appl., 1982, v. 27, N. 1.

    Google Scholar 

  6. Fainberg, E.A., Markov decision processes with arbitrary real-valued criteria, Theory Probability Appl., 1982, v. 27, N. 3.

    Google Scholar 

  7. Frid, E.B., On a problem of D. Blackwell from the theory of dynamic programming, Theory Probability Appl., 1970, v. 15, 719–722.

    Article  MATH  MathSciNet  Google Scholar 

  8. Hee, K.M. van, Markov strategies in dynamic programming, Math. Oper. Res., 1978, v. 3, 37–41.

    Article  MATH  MathSciNet  Google Scholar 

  9. Hinderer, K., Foundations of Non-stationary Dynamic Programming with Discrete Time Parameter, Springer-Verlag, Berlin, 1970.

    Book  MATH  Google Scholar 

  10. Hordijk, A., Dynamic Programming and Markov Potential Theory, Math. Centre Tracts 51, Mathematish Centrum, Amsterdam, 1974.

    Google Scholar 

  11. Krylov, N.V., Construction of an optimal strategy for a finite controlled chain, Theory Probability Appl., 1965, v. 10, 45–54.

    Article  Google Scholar 

  12. Ornstein, D., On the existence of stationary optimal strategies, Proc. Amer. Math. Soc., 1969, v. 20, 563–569.

    Article  MATH  MathSciNet  Google Scholar 

  13. Schäl, M., Stationary policies in dynamic programming models under compactness assumptions, Universität Bonn, preprint, 1981.

    Google Scholar 

  14. Shiryayev, A.N., Optimal Stopping Rules, Springer-Verlag, New York, 1978.

    MATH  Google Scholar 

  15. Strauch, R., Negative dynamic programming, Ann. Math. Statist., 1966, v. 37, 871–889.

    Article  MATH  MathSciNet  Google Scholar 

  16. Wal, J. van der, Stochastic Dynamic Programming, Math. Centre Tracts 139, Mathematish Centrum, Amsterdam, 1981.

    Google Scholar 

  17. Wal, J. van der, On stationary strategies, Eindhoven Univ. of Technology, Dept. of Math., Memorandum-COSOR 81-14, 1981.

    Google Scholar 

  18. Wal, J. van der, On uniformly nearly-optimal Markov strategies, Eindhoven Univ. of Technology, Dept. of Math., Memorandum-COSOR 81-16, 1981.

    Google Scholar 

  19. Wal, J. van der and J. Wessels, On the use of information in Markov decision processes, Eindhoven Univ. of Technology, Dept. of Math., Memorandum-COSOR 81-20, 1981.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Moscow Institute of Transport Engineers (MIIT), Department of Applied Mathematics, Moscow

    E. A. Fainberg

  2. Central Econ.-Math., Institute, Academy of Science of the USSR, Moscow

    I. M. Sonin

Authors
  1. E. A. Fainberg
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. I. M. Sonin
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Steklov Mathematical Institute of the Academy of Sciences of the USSR, Vavilov str. 42, Moscow, 117966, USSR

    Jurii V. Prokhorov

  2. Department of Mathematics, Gakushuin University, 1-5-1 Mejiro, Toshima- ku, Tokyo, Japan

    Kiyosi Itô

Rights and permissions

Reprints and permissions

Copyright information

© 1983 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Fainberg, E.A., Sonin, I.M. (1983). Stationary and Markov policies in countable state dynamic programming. In: Prokhorov, J.V., Itô, K. (eds) Probability Theory and Mathematical Statistics. Lecture Notes in Mathematics, vol 1021. Springer, Berlin, Heidelberg. https://doi.org/10.1007/BFb0072908

Download citation

  • DOI: https://doi.org/10.1007/BFb0072908

  • Published:

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-12718-5

  • Online ISBN: 978-3-540-38701-5

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics

Search

Navigation