A Characterization of Meaningful Schedulers for Continuous-Time Markov Decision Processes
Continuous-time Markov decision process are an important variant of labelled transition systems having nondeterminism through labels and stochasticity through exponential fire-time distributions. Nondeterministic choices are resolved using the notion of a scheduler. In this paper we characterize the class of measurable schedulers, which is the most general one, and show how a measurable scheduler induces a unique probability measure on the sigma-algebra of infinite paths. We then give evidence that for particular reachability properties it is sufficient to consider a subset of measurable schedulers. Having analyzed schedulers and their induced probability measures we finally show that each probability measure on the sigma-algebra of infinite paths is indeed induced by a measurable scheduler which proves that this class is complete.
KeywordsProbability Measure Sojourn Time Markov Decision Process Combine Transition Negative Exponential Distribution
Unable to display preview. Download preview PDF.
- 1.Bertsekas, D.P.: Dynamic Programming and Optimal Control, vol. II. Athena Scientific (1995)Google Scholar
- 4.Sennot, L.: Stochastic Dynamic Programming and the Control of Queueing Systems. John Wiley & Sons, Chichester (1999)Google Scholar
- 7.Qiu, Q., Pedram, M.: Dynamic power managment based on continuous-time Markov decision processes. In: Proceedings DAC, pp. 555–561 (1999)Google Scholar
- 11.Hernndez-Lerma, O., Lassere, J.B.: Discrete-time Markov control processes: Basic optimality criteria. Appl. Math., vol. 30. Springer, Heidelberg (1996)Google Scholar
- 12.Grabiszewski, K.: Type space with disintegrability. Draft (2005)Google Scholar
- 13.Valadier, M.: Désintégration d’une mesure sur un produit. C.R. Acad. Sc. Paris 276, 33–35 (1973) Serie AGoogle Scholar
- 16.Panangaden, P.: Measure and probability for concurrency theorists. TCS: Theoretical Computer Science 253 (2001)Google Scholar
- 17.Panangaden, P.: Stochastic Techniques in Concurrency. Lecture Notes from a course given at BRICS (unpublished, 1997)Google Scholar