Skip to main content
SpringerLink
Log in

Probabilistic Model Checking of Labelled Markov Processes via Finite Approximate Bisimulations

  • Chapter
Horizons of the Mind. A Tribute to Prakash Panangaden

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 8464))

Abstract

This paper concerns labelled Markov processes (LMPs), probabilistic models over uncountable state spaces originally introduced by Prakash Panangaden and colleagues. Motivated by the practical application of the LMP framework, we study its formal semantics and the relationship to similar models formulated in control theory. We consider notions of (exact and approximate) probabilistic bisimulation over LMPs and, drawing on methods from both formal verification and control theory, propose a simple technique to compute an approximate probabilistic bisimulation of a given LMP, where the resulting abstraction is characterised as a finite-state labelled Markov chain (LMC). This construction enables the application of automated quantitative verification and policy synthesis techniques over the obtained abstract model, which can be used to perform approximate analysis of the concrete LMP. We illustrate this process through a case study of a multi-room heating system that employs the probabilistic model checker PRISM.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Abate, A.: A contractivity approach for probabilistic bisimulations of diffusion processes. In: Proc. 48th IEEE Conf. Decision and Control, Shanghai, China, pp. 2230–2235 (December 2009)

    Google Scholar 

  2. Abate, A., D’Innocenzo, A., Di Benedetto, M.D.: Approximate abstractions of stochastic hybrid systems. IEEE Transactions on Automatic Control 56(11), 2688–2694 (2011), doi:10.1109/TAC.2011.2160595

    Article  MathSciNet  Google Scholar 

  3. Abate, A., Katoen, J.-P., Lygeros, J., Prandini, M.: Approximate model checking of stochastic hybrid systems. European Journal of Control 16, 624–641 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  4. Abate, A., Prandini, M.: Approximate abstractions of stochastic systems: A randomized method. In: 2011 50th IEEE Conference on Decision and Control and European Control Conference (CDC-ECC), pp. 4861–4866 (2011)

    Google Scholar 

  5. Abate, A., Prandini, M., Lygeros, J., Sastry, S.: Probabilistic reachability and safety for controlled discrete time stochastic hybrid systems. Automatica 44(11), 2724–2734 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  6. Baier, C., Katoen, J.-P.: Principles of model checking. The MIT Press (2008)

    Google Scholar 

  7. Bertsekas, D.P., Shreve, S.E.: Stochastic optimal control: The discrete time case, vol. 139. Academic Press (1978)

    Google Scholar 

  8. Bianco, A., de Alfaro, L.: Model checking of probabilistic and nondeterministic systems. In: Thiagarajan, P.S. (ed.) FSTTCS 1995. LNCS, vol. 1026, pp. 499–513. Springer, Heidelberg (1995)

    Chapter  Google Scholar 

  9. Bouchard-Cote, A., Ferns, N., Panangaden, P., Precup, D.: An approximation algorithm for labelled Markov processes: towards realistic approximation. In: Proc. 2nd Int. Conf. Quantitative Evaluation of Systems, QEST 2005 (2005)

    Google Scholar 

  10. Cattani, S., Segala, R., Kwiatkowska, M., Norman, G.: Stochastic transition systems for continuous state spaces and non-determinism. In: Sassone, V. (ed.) FOSSACS 2005. LNCS, vol. 3441, pp. 125–139. Springer, Heidelberg (2005)

    Chapter  Google Scholar 

  11. Chaput, P., Danos, V., Panangaden, P., Plotkin, G.: Approximating markov processes by averaging. In: Albers, S., Marchetti-Spaccamela, A., Matias, Y., Nikoletseas, S., Thomas, W. (eds.) ICALP 2009, Part II. LNCS, vol. 5556, pp. 127–138. Springer, Heidelberg (2009)

    Chapter  Google Scholar 

  12. Comanici, G., Panangaden, P., Precup, D.: On-the-fly algorithms for bisimulation metrics. In: Proc. 9th Int. Conf. Quantitative Evaluation of Systems (QEST 2012), pp. 681–692 (2012)

    Google Scholar 

  13. Danos, V., Desharnais, J., Panangaden, P.: Labelled markov processes: Stronger and faster approximations. Electr. Notes Theor. Comput. Sci. 87, 157–203 (2004)

    Article  MATH  Google Scholar 

  14. Desharnais, J., Edalat, A., Panangaden, P.: Bisimulation for labelled Markov processes. Information and Computation 179(2), 163–193 (2002)

    Article  MathSciNet  MATH  Google Scholar 

  15. Desharnais, J., Gupta, V., Jagadeesan, R., Panangaden, P.: Metrics for labelled Markov processes. Theoretical Computer Science 318(3), 323–354 (2004)

    Article  MathSciNet  MATH  Google Scholar 

  16. Desharnais, J., Jagadeesan, R., Gupta, V., Panangaden, P.: The metric analogue of weak bisimulation for probabilistic processes. In: Proc. 17th Annual IEEE Symp. Logic in Computer Science (LICS 2002), pp. 413–422 (2002)

    Google Scholar 

  17. Desharnais, J., Laviolette, F., Tracol, M.: Approximate analysis of probabilistic processes: logic, simulation and games. In: Proc. 5th Int. Conf. Quantitative Evaluation of SysTems (QEST 2008), pp. 264–273 (September 2008)

    Google Scholar 

  18. Desharnais, J., Panangaden, P.: Continuous stochastic logic characterizes bisimulation of continuous-time Markov processes. The Journal of Logic and Algebraic Programming 56(1-2), 99–115 (2003)

    Article  MathSciNet  MATH  Google Scholar 

  19. Desharnais, J., Panangaden, P., Jagadeesan, R., Gupta, V.: Approximating labeled Markov processes. In: Proc. 15th Annual IEEE Symp. Logic in Computer Science (LICS 2000), pp. 95–105 (2000)

    Google Scholar 

  20. D’Innocenzo, A., Abate, A., Katoen, J.-P.: Robust PCTL model checking. In: Proc. 15th ACM Int. Conf. Hybrid Systems: computation and control, Beijing, PRC, pp. 275–285 (April 2012)

    Google Scholar 

  21. Esmaeil Zadeh Soudjani, S., Abate, A.: Adaptive and sequential gridding procedures for the abstraction and verification of stochastic processes. SIAM Journal on Applied Dynamical Systems 12(2), 921–956 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  22. Fehnker, A., Ivančić, F.: Benchmarks for hybrid systems verification. In: Alur, R., Pappas, G.J. (eds.) HSCC 2004. LNCS, vol. 2993, pp. 326–341. Springer, Heidelberg (2004)

    Chapter  Google Scholar 

  23. Ferns, N., Panangaden, P., Precup, D.: Metrics for finite Markov decision processes. In: Proc. 20th Conf. Uncertainty in Artificial Intelligence (UAI 2004), Banff, Canada, pp. 162–169 (2004)

    Google Scholar 

  24. Ferns, N., Panangaden, P., Precup, D.: Metrics for Markov decision processes with infinite state spaces. In: Proc. 21st Conf. Uncertainty in Artificial Intelligence, UAI 2005 (2005)

    Google Scholar 

  25. Ferns, N., Panangaden, P., Precup, D.: Bisimulation metrics for continuous Markov decision processes. SIAM Journal of Computing 60(4), 1662–1724 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  26. Forejt, V., Kwiatkowska, M., Norman, G., Parker, D.: Automated verification techniques for probabilistic systems. In: Bernardo, M., Issarny, V. (eds.) SFM 2011. LNCS, vol. 6659, pp. 53–113. Springer, Heidelberg (2011)

    Chapter  Google Scholar 

  27. Hansson, H., Jonsson, B.: A logic for reasoning about time and reliability. Formal Aspects of Computing 6(5), 512–535 (1994)

    Article  MATH  Google Scholar 

  28. Hernández-Lerma, O., Lasserre, J.B.: Discrete-time Markov control processes, Applications of Mathematics, vol. 30. Springer, New York (1996)

    Book  MATH  Google Scholar 

  29. Huth, M.: On finite-state approximants for probabilistic computation tree logic. Theoretical Computer Science 346(1), 113–134 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  30. Huth, M., Kwiatkowska, M.Z.: Quantitative analysis and model checking. In: 12th Annual IEEE Symp. Logic in Computer Science, pp. 111–122. IEEE Computer Society (1997)

    Google Scholar 

  31. Julius, A.A., Pappas, G.J.: Approximations of stochastic hybrid systems. IEEE Transactions on Automatic Control 54(6), 1193–1203 (2009)

    Article  MathSciNet  Google Scholar 

  32. Kwiatkowska, M., Norman, G., Parker, D.: PRISM 4.0: Verification of probabilistic real-time systems. In: Gopalakrishnan, G., Qadeer, S. (eds.) CAV 2011. LNCS, vol. 6806, pp. 585–591. Springer, Heidelberg (2011)

    Chapter  Google Scholar 

  33. Larsen, K.G., Mardare, R., Panangaden, P.: Taking it to the limit: Approximate reasoning for markov processes. In: Rovan, B., Sassone, V., Widmayer, P. (eds.) MFCS 2012. LNCS, vol. 7464, pp. 681–692. Springer, Heidelberg (2012)

    Chapter  Google Scholar 

  34. Larsen, K.G., Skou, A.: Bisimulation through probabilistic testing. Information and Computation 94, 1–28 (1991)

    Article  MathSciNet  MATH  Google Scholar 

  35. Malhame, R., Chong, C.-Y.: Electric load model synthesis by diffusion approximation of a high-order hybrid-state stochastic system. IEEE Transactions on Automatic Control 30(9), 854–860 (1985)

    Article  MATH  Google Scholar 

  36. Meyn, S.P., Tweedie, R.L.: Markov chains and stochastic stability. Communications and Control Engineering Series. Springer, London (1993)

    Book  MATH  Google Scholar 

  37. Panangaden, P.: Labelled Markov Processes. Imperial College Press (2009)

    Google Scholar 

  38. Puterman, M.: Markov decision processes: discrete stochastic dynamic programming. John Wiley & Sons, Inc. (1994)

    Google Scholar 

  39. Tkachev, I., Abate, A.: On infinite-horizon probabilistic properties and stochastic bisimulation functions. In: 2011 50th IEEE Conf. Decision and Control and European Control Conference (CDC-ECC), pp. 526–531 (2011)

    Google Scholar 

  40. Tkachev, I., Abate, A.: Characterization and computation of infinite horizon specifications over Markov processes. Theoretical Computer Science 515, 1–18 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  41. Tkachev, I., Abate, A.: Formula-free finite abstractions for linear temporal verification of stochastic hybrid systems. In: Proc. 16th ACM Int. Conf. Hybrid Systems: Computation and Control, pp. 283–292 (2013)

    Google Scholar 

  42. Tkachev, I., Abate, A.: On approximation metrics for linear temporal model-checking of stochastic systems. In: Proc. 17th ACM Int. Conf. Hybrid Systems: Computation and Control (2014)

    Google Scholar 

  43. van Breugel, F., Worrell, J.: An algorithm for quantitative verification of probabilistic transition systems. In: Larsen, K.G., Nielsen, M. (eds.) CONCUR 2001. LNCS, vol. 2154, pp. 336–350. Springer, Heidelberg (2001)

    Chapter  Google Scholar 

  44. van Breugel, F., Worrell, J.: Towards quantitative verification of probabilistic transition systems. In: Orejas, F., Spirakis, P.G., van Leeuwen, J. (eds.) ICALP 2001. LNCS, vol. 2076, pp. 421–432. Springer, Heidelberg (2001)

    Chapter  Google Scholar 

  45. Wimmer, R., Becker, B.: Correctness issues of symbolic bisimulation computation for Markov chains. In: Proc. 15th Int. GI/ITG Conf. Measurement, Modelling, and Evaluation of Computing Systems and Dependability and Fault Tolerance (MMB-DFT), pp. 287–301. Springer, Heidelberg (2010)

    Chapter  Google Scholar 

  46. Zamani, M., Esfahani, P.M., Majumdar, R., Abate, A., Lygeros, J.: Symbolic control of stochastic systems via approximately bisimilar finite abstractions. arXiv: 1302.3868 (2013)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science, University of Oxford, UK

    Alessandro Abate & Marta Kwiatkowska

  2. School of Computing Science, University of Glasgow, UK

    Gethin Norman

  3. School of Computer Science, University of Birmingham, UK

    David Parker

Authors
  1. Alessandro Abate
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Marta Kwiatkowska
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Gethin Norman
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. David Parker
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Department of Electrical Engineering and Computer Science, York University, Keele Street, 4700, M3J 1P3, Toronto, ON, Canada

    Franck van Breugel

  2. School of Informatics, Informatics Forum, University of Edinburgh, 10 Crichton Street, EH8 9LE, Edinburgh, UK

    Elham Kashefi

  3. Inria Saclay, Campus de l’École Polytechnique, Bâtiment Alan Turing, 1, rue Honoré d’Estienne d’Orves, 91120, Palaiseau, France

    Catuscia Palamidessi

  4. CWI, P.O. Box 94079, 1090, Amsterdam, GB, The Netherlands

    Jan Rutten

Rights and permissions

Reprints and permissions

Copyright information

© 2014 Springer International Publishing Switzerland

About this chapter

Cite this chapter

Abate, A., Kwiatkowska, M., Norman, G., Parker, D. (2014). Probabilistic Model Checking of Labelled Markov Processes via Finite Approximate Bisimulations. In: van Breugel, F., Kashefi, E., Palamidessi, C., Rutten, J. (eds) Horizons of the Mind. A Tribute to Prakash Panangaden. Lecture Notes in Computer Science, vol 8464. Springer, Cham. https://doi.org/10.1007/978-3-319-06880-0_2

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-06880-0_2

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-06879-4

  • Online ISBN: 978-3-319-06880-0

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation