Skip to main content
SpringerLink
Log in

A note on asymptotics of discounted value function and strong 0-discount optimality

  • Published:
Mathematical Methods of Operations Research Aims and scope Submit manuscript

Abstract

We consider a Markov decision process with a Borel state space, bounded rewards, and a bounded transition density satisfying a simultaneous Doeblin-Doob condition. An asymptotics for the discounted value function related to the existence of stationary strong 0-discount optimal policies is extended from the case of finite action sets to the case of compact action sets and continuous in action rewards and transition densities.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Balder EI (1989) On compactness of the space of policies in stochastic dynamic programming. Stoch Proc Appl 32:141–150

    Google Scholar 

  • Cavazos-Cadena R, Lasserre JB (1988) Strong 1-optimal stationary policies in denumerable Markov decision processes. Systems Control Letters 11:65–71

    Google Scholar 

  • Doob JL (1953) Stochastic Processes. Wiley, New York

    Google Scholar 

  • Himmelberg CJ, Parthasarathy T, VanVleck FS (1976) Optimal plans for dynamic programming models. Math Oper Res 1:390–394

    Google Scholar 

  • Schäl M (1975) On dynamic programming: compactness of the space of policies. Stoch Proc Appl 3:345–354

    Google Scholar 

  • Schäl M (1979) On dynamic programming and statistical decision theory. Ann Stat 7:432–445

    Google Scholar 

  • Yushkevich AA (1994) Blackwell optimal policies in a Markov decision process with a Borel state space. ZOR 40:253–288

    Google Scholar 

  • Yushkevich AA (1995) Strong 0-discount optimal policies in a Markov decision process with a Borel state space. ZOR 42:93–108

    Google Scholar 

  • Yushkevich AA (1995a) Compactness of a measure space in dynamic programming revisited: an approach via extension theorem for Carathéodory functions. Technical report 95-2, Dept Math UNC-Charlotte

  • Yushkevich AA (1995b) Blackwell optimality in Borelian continuous in action Markov decision processes. Technical report 95-5, Dept Math UNC-Charlotte

Download references

Author information

Authors and Affiliations

  1. Department of Mathematics, University of North Carolina at Charlotte, 28223, NC, USA

    A. A. Yushkevich

Authors
  1. A. A. Yushkevich
    View author publications

    You can also search for this author in PubMed Google Scholar

Additional information

Supported by NSF grant DMS-9404177

Rights and permissions

Reprints and permissions

About this article

Cite this article

Yushkevich, A.A. A note on asymptotics of discounted value function and strong 0-discount optimality. Mathematical Methods of Operations Research 44, 223–231 (1996). https://doi.org/10.1007/BF01194332

Download citation

  • Received:

  • Revised:

  • Issue Date:

  • DOI: https://doi.org/10.1007/BF01194332

Key words

Search

Navigation