Advertisement

Continuous-Time Markov Decision Processes

Theory and Applications

  • Xianping Guo
  • Onésimo Hernández-Lerma

Part of the Stochastic Modelling and Applied Probability book series (SMAP, volume 62)

Table of contents

  1. Front Matter
    Pages I-XVII
  2. Xianping Guo, Onésimo Hernández-Lerma
    Pages 1-8
  3. Xianping Guo, Onésimo Hernández-Lerma
    Pages 9-18
  4. Xianping Guo, Onésimo Hernández-Lerma
    Pages 19-53
  5. Xianping Guo, Onésimo Hernández-Lerma
    Pages 55-70
  6. Xianping Guo, Onésimo Hernández-Lerma
    Pages 71-86
  7. Xianping Guo, Onésimo Hernández-Lerma
    Pages 87-103
  8. Xianping Guo, Onésimo Hernández-Lerma
    Pages 105-125
  9. Xianping Guo, Onésimo Hernández-Lerma
    Pages 127-142
  10. Xianping Guo, Onésimo Hernández-Lerma
    Pages 143-162
  11. Xianping Guo, Onésimo Hernández-Lerma
    Pages 163-173
  12. Xianping Guo, Onésimo Hernández-Lerma
    Pages 175-186
  13. Xianping Guo, Onésimo Hernández-Lerma
    Pages 187-194
  14. Back Matter
    Pages 195-234

About this book

Introduction

Continuous-time Markov decision processes (MDPs), also known as controlled Markov chains, are used for modeling decision-making problems that arise in operations research (for instance, inventory, manufacturing, and queueing systems), computer science, communications engineering, control of populations (such as fisheries and epidemics), and management science, among many other fields. This volume provides a unified, systematic, self-contained presentation of recent developments on the theory and applications of continuous-time MDPs. The MDPs in this volume include most of the cases that arise in applications, because they allow unbounded transition and reward/cost rates. Much of the material appears for the first time in book form.

Keywords

Markov chain Markov decision process Markov decision processes controlled Markov chains operations research stochastic control stochastic dynamic programming

Authors and affiliations

  • Xianping Guo
    • 1
  • Onésimo Hernández-Lerma
    • 2
  1. 1.Dept. MathematicsZhongshan UniversityGuangzhouChina, People's Republic
  2. 2.Depto. MatematicaCINVESTAV-IPNMexicoMexico

Bibliographic information

  • DOI https://doi.org/10.1007/978-3-642-02547-1
  • Copyright Information Springer-Verlag Berlin Heidelberg 2009
  • Publisher Name Springer, Berlin, Heidelberg
  • eBook Packages Mathematics and Statistics
  • Print ISBN 978-3-642-02546-4
  • Online ISBN 978-3-642-02547-1
  • Series Print ISSN 0172-4568
  • Buy this book on publisher's site