Skip to main content
SpringerLink
Log in

The structure of persistently nearly-optimal strategies in stochastic dynamic programming problems

  • Section I Controlled Stochastic Processes
  • Conference paper
  • First Online:
Stochastic Optimization

Part of the book series: Lecture Notes in Control and Information Sciences ((LNCIS,volume 81))

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. D. Blackwell. Discrete dynamic programming. Annals of Mathematical Statistics, 33(1962) 719–726.

    Google Scholar 

  2. N.V. Krylov. Construction of an optimal strategy for a finite controlled chain. Theory of Probability and its Application, 10(1965) 45–54.

    Google Scholar 

  3. E.B. Dynkin and A.A. Yushkevich. Controlled Markov Processes. Springer-Verlag, Berlin 1979.

    Google Scholar 

  4. J. van der Wal and J. Wessels. On the Use of Information in Markov Decision Processes. Eindhoven University of Technology, Memorandum-COSOR 81-20, 1981.

    Google Scholar 

  5. D. Blackwell. Positive dynamic programming. In Proceedings of the 5th Berkeley Symposium on Mathematical Statistics and Probability, 1967, pp. 415–418.

    Google Scholar 

  6. E.B. Frid. On a problem of D. Blackwell from the theory of dynamic programming. Theory of Probability and its Applications, 15(1970) 719–722.

    Google Scholar 

  7. D. Ornstein. On the existence of stationary optimal strategies. Proceedings of the American Mathematical Society, 20(1969) 563–569.

    Google Scholar 

  8. M. Schäl. Stationary policies in dynamic programming models under compactness assumptions. Mathematics of Operational Research, 8(1983) 366–372.

    Google Scholar 

  9. J. van der Wal. Stochastic Dynamic Programming. Mathematical Centre Tracts 139, Mathematical Centre, Amsterdam, 1981.

    Google Scholar 

  10. K. van Hee, A. Hordijk and J. van der Wal. Successive approximations for convergent dynamic programming. In Markov Decision Theory, Mathematical Centre Tracts 93, Mathematical Centre, Amsterdam, 1977, p. 183–211.

    Google Scholar 

  11. N.L. Lazrieva. On the existence of an ε-optimal stationary strategy for controlled Markov processes with a general additive critrion (in Russian). In Studies in Probability Theory and Mathematical Statistics, Tbilisi, 1982.

    Google Scholar 

  12. J. van Nunen. Contracting Markov Decision Processes. Mathematical Centre Tracts 71, Mathematical Centre, Amsterdam, 1976.

    Google Scholar 

  13. E.A. Fainberg and I.M. Sonin. Stationary and Markov policies in countable state dynamic programming. Lecture Notes in Mathematics, 1021(1983) 111–129.

    Google Scholar 

  14. E.A. Fainberg and I.M. Sonin. Persistently nearly-optimal strategies in stochastic dynamic programming. In R.S. Liptser, N.V. Krylov and A.A. Novikov (Eds.), Proceedings of the Steklov Seminar Statistics and Control of Random Processes. 1984. To appear in Lecture Notes in Control and Information Sciences.

    Google Scholar 

  15. J. van der Wal. On uniformly nearly-optimal Markov strategies. In Operational Research Proceedings, Springer-Verlag, Berlin, 1983, pp. 461–467.

    Google Scholar 

  16. I.M. Sonin. The existence of a uniformly nearly-optimal Markov strategy for a controlled Markov chain with a countable state space (in Russian). In Models and Methods of Stochastic Optimization, Central Economic-Mathematical Institute, Moscow, USSR, 1984, pp. 213–232.

    Google Scholar 

  17. R. van Dawen. Stationäre Politiken in Stochastischen Entscheidungsmodellen. Dissertation, Universität Bonn, 1984.

    Google Scholar 

  18. H. Everett. Recursive games. Annals of Mathematical Statistics, 39(1957) 47–78.

    Google Scholar 

  19. R.Ja. Chitashvily. On the existence of ɛ-optimal stationary policies for a controlled Markov chain (in Russian). Communications of the Academy of Sciences of the Georgian SSR, 83(1976) 549–552.

    Google Scholar 

  20. A.A. Yushkevich and R.Ja. Chitashvili. Controlled random sequences and Markov chains (in Russian). Uspekhi Matematicheskie Nauk, 37(1982) 213–242.

    Google Scholar 

  21. I.M. Sonin and E.A. Fainberg. Sufficient classes of policies in countable state Markov decision chains with a total reward criterion (in Russian). Doklady Academii Nauk USSR, 275(1984) 806–809.

    Google Scholar 

  22. E.A. Fainberg. On sufficient classes of strategies in dynamic programming. Submitted to Theory of Probability and its Application.

    Google Scholar 

  23. R. van Dawen and M. Schäl. On the existence of stationary optimal policies in Markov decision models. Zeitschrift für Angewandte Mathematik und Mechanik, 63(1983) 404–405.

    Google Scholar 

  24. E.A. Fainberg. On some classes of policies in dynamic programming. Submitted to Theory of Probability and its Applications.

    Google Scholar 

  25. J. van der Wal. On uniformly nearly-optimal stationary strategies. Mathematics of Operational Research, 9(1984) 290–300.

    Google Scholar 

  26. E.A. Fainberg. Nonrandomized Markov and semi-Markov policies in dynamic programming. Theory of Probability and its Application, 27(1)(1982).

    Google Scholar 

  27. E.A. Fainberg. Markov decision processes with arbitrary real-valued criteria. Theory of Probability and its Application, 27(3)(1982).

    Google Scholar 

  28. K. van Hee. Markov strategies in dynamic programming. Mathematics of Operational Research, 3(1978) 37–41.

    Google Scholar 

  29. A.N. Shiryaev. Optimal Stopping Rules. Springer-Verlag, New York, 1978.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Applied Mathematics, Moscow Institute of Transport Engineers (MIIT), Moscow, USSR

    E. A. Fainberg

Authors
  1. E. A. Fainberg
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Vadim I. Arkin A. Shiraev R. Wets

Rights and permissions

Reprints and permissions

Copyright information

© 1986 International Institute for Applied Systems Analysis

About this paper

Cite this paper

Fainberg, E.A. (1986). The structure of persistently nearly-optimal strategies in stochastic dynamic programming problems. In: Arkin, V.I., Shiraev, A., Wets, R. (eds) Stochastic Optimization. Lecture Notes in Control and Information Sciences, vol 81. Springer, Berlin, Heidelberg. https://doi.org/10.1007/BFb0007079

Download citation

  • DOI: https://doi.org/10.1007/BFb0007079

  • Published:

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-16659-7

  • Online ISBN: 978-3-540-39841-7

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics

Search

Navigation