Continuous Time Markov Decision Processes with Expected Discounted Total Rewards

  • Qiying Hu
  • Jianyong Liu
  • Wuyi Yue
Conference paper

DOI: 10.1007/3-540-44862-4_8

Part of the Lecture Notes in Computer Science book series (LNCS, volume 2658)
Cite this paper as:
Hu Q., Liu J., Yue W. (2003) Continuous Time Markov Decision Processes with Expected Discounted Total Rewards. In: Sloot P.M.A., Abramson D., Bogdanov A.V., Gorbachev Y.E., Dongarra J.J., Zomaya A.Y. (eds) Computational Science — ICCS 2003. ICCS 2003. Lecture Notes in Computer Science, vol 2658. Springer, Berlin, Heidelberg

Abstract

This paper discusses continuous time Markov decision processes with criterion of expected discounted total rewards, where the state space is countable, the reward rate function is extended real-valued and the discount rate is a real number. Under necessary conditions that the model is well defined, the state space is partitioned into three subsets, on which the optimal value function is positive infinity, negative infinity, or finite, respectively. Correspondingly, the model is reduced into three submodels, by generalizing policies and eliminating some worst actions. Then for the submodel with finite optimal value, the validity of the optimality equation is shown and some its properties are obtained.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Copyright information

© Springer-Verlag Berlin Heidelberg 2003

Authors and Affiliations

  • Qiying Hu
    • 1
  • Jianyong Liu
    • 2
  • Wuyi Yue
    • 3
  1. 1.College of International Business & ManagementShanghai UniversityShanghaiChina
  2. 2.Institute of Applied MathematicsAcademia SinicaBeijingChina
  3. 3.Dept. of Information Science and Systems EngineeringKonan UniversityKobeJAPAN

Personalised recommendations