Skip to main content
SpringerLink
Log in

Bisimulation and Logical Preservation for Continuous-Time Markov Decision Processes

  • Conference paper
CONCUR 2007 – Concurrency Theory (CONCUR 2007)

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 4703))

Included in the following conference series:

Abstract

This paper introduces strong bisimulation for continuous-time Markov decision processes (CTMDPs), a stochastic model which allows for a nondeterministic choice between exponential distributions, and shows that bisimulation preserves the validity of CSL . To that end, we interpret the semantics of CSL —a stochastic variant of CSL for continuous-time Markov chains—on CTMDPs and show its measure-theoretic soundness. The main challenge faced in this paper is the proof of logical preservation that is substantially based on measure theory.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Abdeddaïm, Y., Asarin, E., Maler, O.: On optimal scheduling under uncertainty. In: Garavel, H., Hatcliff, J. (eds.) ETAPS 2003 and TACAS 2003. LNCS, vol. 2619, pp. 240–253. Springer, Heidelberg (2003)

    Google Scholar 

  2. Ash, R.B., Doléans-Dade, C.A.: Probability & Measure Theory, 2nd edn. Academic Press, London (2000)

    MATH  Google Scholar 

  3. Aziz, A., Sanwal, K., Singhal, V., Brayton, R.K.: Model-checking continous-time Markov chains. ACM Trans. Comput. Log. 1, 162–170 (2000)

    Article  MathSciNet  Google Scholar 

  4. Baier, C.: On Algorithmic Verification Methods for Probabilistic Systems. Habilitation Thesis, University of Mannheim (1998)

    Google Scholar 

  5. Baier, C., Haverkort, B.R., Hermanns, H., Katoen, J.-P.: Model-checking algorithms for continuous-time Markov chains. IEEE TSE 29, 524–541 (2003)

    Google Scholar 

  6. Baier, C., Haverkort, B.R., Hermanns, H., Katoen, J.-P.: Nonuniform CTMDPs. unpublished manuscript (2004)

    Google Scholar 

  7. Baier, C., Hermanns, H., Katoen, J.-P., Haverkort, B.R.: Efficient computation of time-bounded reachability probabilities in uniform continuous-time Markov decision processes. Theor. Comp. Sci. 345, 2–26 (2005)

    Article  MATH  MathSciNet  Google Scholar 

  8. Baier, C., Katoen, J.-P., Hermanns, H., Wolf, V.: Comparative branching-time semantics for Markov chains. Information and Computation 200, 149–214 (2005)

    Article  MATH  MathSciNet  Google Scholar 

  9. Baier, C., Kwiatkowska, M.Z.: Model checking for a probabilistic branching time logic with fairness. Distr. Comp. 11, 125–155 (1998)

    Article  Google Scholar 

  10. Beutler, F.J., Ross, K.W.: Optimal policies for controlled Markov chains with a constraint. Journal of Mathematical Analysis and Appl. 112, 236–252 (1985)

    Article  MATH  MathSciNet  Google Scholar 

  11. Bianco, A., de Alfaro, L.: Model checking of probabilistic and nondeterministic systems. In: Thiagarajan, P.S. (ed.) Foundations of Software Technology and Theoretical Computer Science. LNCS, vol. 1026, pp. 499–513. Springer, Heidelberg (1995)

    Google Scholar 

  12. Boudali, H., Crouzen, P., Stoelinga, M.I.A.: Dynamic fault tree analysis using input/output interactive Markov chains. In: Dependable Systems and Networks, IEEE Computer Society Press, Los Alamitos (2007)

    Google Scholar 

  13. Buchholz, P.: Exact and ordinary lumpability in finite Markov chains. Journal of Applied Probability 31, 59–75 (1994)

    Article  MATH  MathSciNet  Google Scholar 

  14. Chiola, G., Marsan, M.A., Balbo, G., Conte, G.: Generalized stochastic Petri nets: A definition at the net level and its implications. IEEE TSE 19, 89–107 (1993)

    Google Scholar 

  15. de Alfaro, L.: Formal Verification of Probabilistic Systems. PhD thesis, Stanford University (1997)

    Google Scholar 

  16. Desharnais, J., Panangaden, P.: Continuous stochastic logic characterizes bisimulation of continuous-time Markov processes. Journal of Logic and Algebraic Programming 56, 99–115 (2003)

    Article  MATH  MathSciNet  Google Scholar 

  17. Feinberg, E.A.: Continuous time discounted jump Markov decision processes: A discrete-event approach. Mathematics of Operations Research 29, 492–524 (2004)

    Article  MATH  MathSciNet  Google Scholar 

  18. Givan, R., Dean, T., Greig, M.: Equivalence notions and model minimization in Markov decision processes. Artificial Intelligence 147, 163–223 (2003)

    MATH  MathSciNet  Google Scholar 

  19. Hermanns, H.: Interactive Markov Chains. LNCS, vol. 2428. Springer, Heidelberg (2002)

    MATH  Google Scholar 

  20. Howard, R.A.: Dynamic Probabilistic Systems. John Wiley and Sons, West Sussex, England (1971)

    Google Scholar 

  21. Katoen, J.-P., Klink, D., Leucker, M., Wolf, V.: Three-valued abstraction for continuous-time Markov chains. In: CAV. LNCS, Springer, Heidelberg (2007)

    Google Scholar 

  22. Kemeny, J.G., Snell, J.L., Knapp, A.W.: Denumerable Markov Chains, 2nd edn. Springer, Heidelberg (1976)

    MATH  Google Scholar 

  23. Larsen, K.G., Skou, A.: Bisimulation through probabilistic testing. Information and Computation 94, 1–28 (1991)

    Article  MATH  MathSciNet  Google Scholar 

  24. López, G.G.I., Hermanns, H., Katoen, J.-P.: Beyond memoryless distributions: Model checking semi-Markov chains. In: de Luca, L., Gilmore, S.T. (eds.) PROBMIV 2001, PAPM-PROBMIV 2001, and PAPM 2001. LNCS, vol. 2165, pp. 57–70. Springer, Heidelberg (2001)

    Chapter  Google Scholar 

  25. Neuhäußer, M.R.: Bisimulation and logical preservation for continuous-time markov decision processes. Technical Report 10, RWTH Aachen (2007)

    Google Scholar 

  26. Puterman, M.L.: Markov Decision Processes: Discrete Stochastic Dynamic Programming. John Wiley and Sons, West Sussex, England (1994)

    MATH  Google Scholar 

  27. Qiu, Q., Pedram, M.: Dynamic power management based on continuous-time Markov decision processes. In: DAC, pp. 555–561. ACM Press, New York (1999)

    Chapter  Google Scholar 

  28. Sanders, W.H., Meyer, J.F.: Stochastic activity networks: Formal definitions and concepts. In: Brinksma, E., Hermanns, H., Katoen, J.-P. (eds.) Lectures on Formal Methods and Performance Analysis. LNCS, vol. 2090, pp. 315–343. Springer, Heidelberg (2001)

    Chapter  Google Scholar 

  29. Segala, R., Lynch, N.: Probabilistic simulations for probabilistic processes. Nordic Journal of Computing 2, 250–273 (1995)

    MATH  MathSciNet  Google Scholar 

  30. Vardi, M.Y.: Automatic verification of probabilistic concurrent finite-state programs. In: FOCS, pp. 327–338. IEEE Computer Society Press, Los Alamitos (1985)

    Google Scholar 

  31. Wolovick, N., Johr, S.: A characterization of meaningful schedulers for continuous-time Markov decision processes. In: Asarin, E., Bouyer, P. (eds.) FORMATS 2006. LNCS, vol. 4202, pp. 352–367. Springer, Heidelberg (2006)

    Chapter  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Software Modeling and Verification Group, RWTH Aachen University, Germany

    Martin R. Neuhäußer & Joost-Pieter Katoen

  2. Formal Methods and Tools Group, University of Twente, The Netherlands

    Martin R. Neuhäußer & Joost-Pieter Katoen

Authors
  1. Martin R. Neuhäußer
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Joost-Pieter Katoen
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Luís Caires Vasco T. Vasconcelos

Rights and permissions

Reprints and permissions

Copyright information

© 2007 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Neuhäußer, M.R., Katoen, JP. (2007). Bisimulation and Logical Preservation for Continuous-Time Markov Decision Processes. In: Caires, L., Vasconcelos, V.T. (eds) CONCUR 2007 – Concurrency Theory. CONCUR 2007. Lecture Notes in Computer Science, vol 4703. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-74407-8_28

Download citation

  • DOI: https://doi.org/10.1007/978-3-540-74407-8_28

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-74406-1

  • Online ISBN: 978-3-540-74407-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation