On Markov Policies in Continuous Time Discounted Dynamic Programming

Idzik, Adam

doi:10.1007/978-94-010-9910-3_28

On Markov Policies in Continuous Time Discounted Dynamic Programming

Adam Idzik¹

Chapter

309 Accesses
2 Citations

Part of the book series: Transactions of the Seventh Prague Conference on Information Theory, Statistical Decision Functions, Random Processes and of the 1974 European Meeting of Statisticians ((TPCI,volume 7A))

Abstract

We consider the problem of discounted dynamic programming with a continuous time parameter (CDP) when the Markov policies are used. We give an axiomatization of such discounted CDP. We also give necessary and sufficient conditions for the existence of an optimal policy. Analogously to the discrete case we formulate improvement’s theorems and a theorem on the existence of a (p, ε)-optimal policy in a class of semi-Markov policies.

This is a preview of subscription content, log in via an institution.

Chapter: USD 29.95; Price excludes VAT (USA)

eBook: USD 84.99; Price excludes VAT (USA)

Softcover Book: USD 109.99; Price excludes VAT (USA)

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

References

D. Blackwell: On the functional equation of dynamic programming. J. Math. Anal. Appl. 2 (1961), 273–276.
Article MathSciNet MATH Google Scholar
D. Blackwell: Discrete dynamic programming. The Annals of Math. Statist. 33 (1962), 719–726.
Article MathSciNet MATH Google Scholar
D. Blackwell: Discounted dynamic programming. The Annals of Math. Statist. 36 (1965), 226–235.
Article MathSciNet MATH Google Scholar
K. Hinderer: Foundations of Non-stationary Dynamic Programming. Springer-Verlag, Berlin—Heidelberg 1970.
MATH Google Scholar
R. A. Howard: Dynamic Programming and Markov Processes. Wiley, New York 1960.
MATH Google Scholar
P. Kakumanu: Continuously discounted Markov decision model with countable state and action space. The Annals of Math. Statist. 42 (1971), 919–926.
Article MathSciNet MATH Google Scholar
A. Maitra: Dynamic programming for countable state systems. Sankhyá Ser. A 27 (1965), 241–248.
MathSciNet MATH Google Scholar
A. Maitra: Discounted dynamic programming on compact metric spaces. Sankhyá Ser. A 30 (1968), 211–216.
MathSciNet MATH Google Scholar
P. A. Meyer: Probability and Potentials. Blaisdell Publishing Company: Waltham, Massachusetts—Toronto—London 1966.
MATH Google Scholar
B. L. Miller: Finite state continuous time Markov decision processes with a finite planning horizon. SI AM J. Control 6 (1968), 266–280.
Article MATH Google Scholar
R. E. Strauch: Negative dynamic programming. The Annals of Math. Statist. 37 (1966), 871–890.
Article MathSciNet MATH Google Scholar

Download references

Author information

Authors and Affiliations

Computation Centre, Polish Academy of Sciences, Warszawa, Poland
Adam Idzik

Authors

Adam Idzik
View author publications
You can also search for this author in PubMed Google Scholar

Editor information

J. Kožešnik

Rights and permissions

Reprints and permissions

Copyright information

About this chapter

Cite this chapter

Idzik, A. (1977). On Markov Policies in Continuous Time Discounted Dynamic Programming. In: Kožešnik, J. (eds) Transactions of the Seventh Prague Conference on Information Theory, Statistical Decision Functions, Random Processes and of the 1974 European Meeting of Statisticians. Transactions of the Seventh Prague Conference on Information Theory, Statistical Decision Functions, Random Processes and of the 1974 European Meeting of Statisticians, vol 7A. Springer, Dordrecht. https://doi.org/10.1007/978-94-010-9910-3_28

Download citation

DOI: https://doi.org/10.1007/978-94-010-9910-3_28
Publisher Name: Springer, Dordrecht
Print ISBN: 978-94-010-9912-7
Online ISBN: 978-94-010-9910-3
eBook Packages: Springer Book Archive

Publish with us

Policies and ethics