Skip to main content

Survey of Statistical Verification of Linear Unbounded Properties: Model Checking and Distances

  • Conference paper
  • First Online:

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 9952))

Abstract

We survey statistical verification techniques aiming at linear properties with unbounded or infinite horizon, as opposed to properties of runs of fixed length. We discuss statistical model checking of Markov chains and Markov decision processes against reachability, unbounded-until, LTL and mean-payoff properties. Moreover, the respective strategies can be represented efficiently using statistical techniques. Further, we also discuss when it is possible to statistically estimate linear distances between Markov chains.

This research was partially supported by the Czech Science Foundation under grant agreement P202/12/G061.

This is a preview of subscription content, log in via an institution.

Buying options

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Notes

  1. 1.

    Formally, the measurable space of \(\omega \)-languages is given by the set \(\varSigma ^\omega \) equipped with a \(\sigma \)-algebra \(\mathcal F(\varSigma )\) generated by the set of cones \(\{w\varSigma ^\omega \mid w\in \varSigma ^*\}\). This ensures, for every measurable \(\omega \)-language X, that \(L^{-1}(X)\) is measurable in every MC.

References

  1. Abate, A.: Approximation metrics based on probabilistic bisimulations for general state-space Markov processes: a survey. Electr. Notes Theor. Comput. Sci. 297, 3–25 (2013)

    Article  MATH  Google Scholar 

  2. Aljazzar, H., Leue, S.: Generation of counterexamples for model checking of markov decision processes. In: QEST, pp. 197–206. IEEE Computer Society (2009)

    Google Scholar 

  3. Basu, A., Bensalem, S., Bozga, M., Caillaud, B., Delahaye, B., Legay, A.: Statistical abstraction and model-checking of large heterogeneous systems. In: Hatcliff, J., Zucca, E. (eds.) FMOODS/FORTE-2010. LNCS, vol. 6117, pp. 32–46. Springer, Heidelberg (2010). doi:10.1007/978-3-642-13464-7_4

    Chapter  Google Scholar 

  4. Bacci, G., Bacci, G., Larsen, K.G., Mardare, R.: The BisimDist, library: efficient computation of bisimilarity distances for Markovian models. In: QEST, pp. 278–281 (2013)

    Google Scholar 

  5. Bacci, G., Bacci, G., Larsen, K.G., Mardare, R.: Computing behavioral distances, compositionally. In: Chatterjee, K., Sgall, J. (eds.) MFCS 2013. LNCS, vol. 8087, pp. 74–85. Springer, Heidelberg (2013). doi:10.1007/978-3-642-40313-2_9

    Chapter  Google Scholar 

  6. Bacci, G., Bacci, G., Larsen, K.G., Mardare, R.: On-the-fly exact computation of bisimilarity distances. In: Piterman, N., Smolka, S.A. (eds.) TACAS 2013. LNCS, vol. 7795, pp. 1–15. Springer, Heidelberg (2013). doi:10.1007/978-3-642-36742-7_1

    Chapter  Google Scholar 

  7. Bacci, G., Bacci, G., Larsen, K.G., Mardare, R: Converging from branching to linear metrics on Markov chains. In: ICTAC, pp. 349–367 (2015)

    Google Scholar 

  8. Bacci, G., Bacci, G., Larsen, K.G., Mardare, R.: On the total variation distance of Semi-Markov chains. In: Pitts, A. (ed.) FoSSaCS 2015. LNCS, vol. 9034, pp. 185–199. Springer, Heidelberg (2015). doi:10.1007/978-3-662-46678-0_12

    Chapter  Google Scholar 

  9. Brázdil, T., Chatterjee, K., Chmelík, M., Forejt, V., Křetínský, J., Kwiatkowska, M., Parker, D., Ujma, M.: Verification of Markov decision processes using learning algorithms. In: Cassez, F., Raskin, J.-F. (eds.) ATVA 2014. LNCS, vol. 8837, pp. 98–114. Springer, Heidelberg (2014). doi:10.1007/978-3-319-11936-6_8

    Google Scholar 

  10. Brázdil, T., Chatterjee, K., Chmelík, M., Fellner, A., Křetínský, J.: Counterexample explanation by learning small strategies in markov decision processes. In: Kroening, D., Păsăreanu, C.S. (eds.) CAV 2015. LNCS, vol. 9206, pp. 158–177. Springer, Heidelberg (2015). doi:10.1007/978-3-319-21690-4_10

    Chapter  Google Scholar 

  11. Boyer, B., Corre, K., Legay, A., Sedwards, S.: PLASMA-lab: a flexible, distributable statistical model checking library. In: QEST, pp. 160–164 (2013)

    Google Scholar 

  12. Boutilier, C., Dearden, R.: Approximating value trees in structured dynamic programming. In: Proceedings of the Thirteenth International Conference on Machine Learning, pp. 54–62 (1996)

    Google Scholar 

  13. Boutilier, C., Dearden, R., Goldszmidt, M.: Exploiting structure in policy construction. In: IJCAI-95, pp. 1104–1111 (1995)

    Google Scholar 

  14. Boutilier, C., Dean, T., Hanks, S.: Decision-theoretic planning: structural assumptions and computational leverage. JAIR 11, 1–94 (1999)

    MathSciNet  MATH  Google Scholar 

  15. Bulychev, P.E., David, A., Larsen, K.G., Mikucionis, M., Poulsen, D.B., Legay, A., Wang, Z.: UPPAAL-SMC: statistical model checking for priced timed automata. In: QAPL (2012)

    Google Scholar 

  16. Bogdoll, J., Ferrer Fioriti, L.M., Hartmanns, A., Hermanns, H.: Partial order methods for statistical model checking and simulation. In: Bruni, R., Dingel, J. (eds.) FMOODS/FORTE -2011. LNCS, vol. 6722, pp. 59–74. Springer, Heidelberg (2011). doi:10.1007/978-3-642-21461-5_4

    Chapter  Google Scholar 

  17. Bogdoll, J., Hartmanns, A., Hermanns, H.: Simulation and statistical model checking for modestly nondeterministic models. In: MMB/DFT, pp. 249–252 (2012)

    Google Scholar 

  18. Chapman, D., Kaelbling, L.P.: Input generalization in delayed reinforcement learning: an algorithm and performance comparisons. Morgan Kaufmann (1991)

    Google Scholar 

  19. Chen, T., Kiefer, S., On the total variation distance of labelled Markov chains. In: CSL-LICS, pp. 33:1–33:10 (2014)

    Google Scholar 

  20. Courcoubetis, C., Yannakakis, M.: The complexity of probabilistic verification. J. ACM 42(4), 857–907 (1995)

    Article  MathSciNet  MATH  Google Scholar 

  21. Clarke, E.M., Zuliani, P.: Statistical model checking for cyber-physical systems. In: Bultan, T., Hsiung, P.-A. (eds.) ATVA 2011. LNCS, vol. 6996, pp. 1–12. Springer, Heidelberg (2011). doi:10.1007/978-3-642-24372-1_1

    Chapter  Google Scholar 

  22. Alfaro, L., Faella, M., Stoelinga, M.: Linear and branching metrics for quantitative transition systems. In: Díaz, J., Karhumäki, J., Lepistö, A., Sannella, D. (eds.) ICALP 2004. LNCS, vol. 3142, pp. 97–109. Springer, Heidelberg (2004). doi:10.1007/978-3-540-27836-8_11

    Chapter  Google Scholar 

  23. Alfaro, L., Kwiatkowska, M., Norman, G., Parker, D., Segala, R.: Symbolic model checking of probabilistic processes using MTBDDs and the kronecker representation. In: Graf, S., Schwartzbach, M. (eds.) TACAS 2000. LNCS, vol. 1785, pp. 395–410. Springer, Heidelberg (2000). doi:10.1007/3-540-46419-0_27

    Chapter  Google Scholar 

  24. de Alfaro, L., Majumdar, R., Raman, V., Stoelinga, M.: Game relations and metrics. In: LICS, pp. 99–108 (2007)

    Google Scholar 

  25. David, A., Du, D., Larsen, K.G., Legay, A., Mikucionis, M., Poulsen, D.B., Sedwards, S.: Statistical model checking for stochastic hybrid systems. In: HSB, pp. 122–136 (2012)

    Google Scholar 

  26. David, A., Du, D., Larsen, K.G., Legay, A., Mikucionis, M.: Optimizing control strategy using statistical model checking. In: NASA Formal Methods, pp. 352–367 (2013)

    Google Scholar 

  27. Desharnais, J., Gupta, V., Jagadeesan, R., Panangaden, P.: Metrics for labeled Markov systems. In: Baeten, J.C.M., Mauw, S. (eds.) CONCUR 1999. LNCS, vol. 1664, pp. 258–273. Springer, Heidelberg (1999). doi:10.1007/3-540-48320-9_19

    Chapter  Google Scholar 

  28. Daca, P., Henzinger, T.A., Křetínský, J., Petrov, T.: Faster statistical model checking for unbounded temporal properties. In: Chechik, M., Raskin, J.-F. (eds.) TACAS 2016. LNCS, vol. 9636, pp. 112–129. Springer, Heidelberg (2016). doi:10.1007/978-3-662-49674-9_7

    Chapter  Google Scholar 

  29. Daca, P., Henzinger, T.A., Křetínský, J., Petrov, T.: Linear distances between Markov chains. In: CONCUR (2016)

    Google Scholar 

  30. Doyen, L., Henzinger, T.A., Raskin, J.-F.: Equivalence of labeled Markov chains. Int. J. Found. Comput. Sci. 19(3), 549–563 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  31. Dehnert, C., Jansen, N., Wimmer, R., Ábrahám, E., Katoen, J.-P.: Fast debugging of PRISM models. In: Cassez, F., Raskin, J.-F. (eds.) ATVA 2014. LNCS, vol. 8837, pp. 146–162. Springer, Heidelberg (2014). doi:10.1007/978-3-319-11936-6_11

    Google Scholar 

  32. David, A., Larsen, K.G., Legay, A., Mikučionis, M., Poulsen, D.B., Vliet, J., Wang, Z.: Statistical model checking for networks of priced timed automata. In: Fahrenberg, U., Tripakis, S. (eds.) FORMATS 2011. LNCS, vol. 6919, pp. 80–96. Springer, Heidelberg (2011). doi:10.1007/978-3-642-24310-3_7

    Chapter  Google Scholar 

  33. David, A., Larsen, K.G., Legay, A., Mikučionis, M., Wang, Z.: Time for statistical model checking of real-time systems. In: Gopalakrishnan, G., Qadeer, S. (eds.) CAV 2011. LNCS, vol. 6806, pp. 349–355. Springer, Heidelberg (2011). doi:10.1007/978-3-642-22110-1_27

    Chapter  Google Scholar 

  34. David, A., Larsen, K.G., Legay, A., Mikucionis, M., Poulsen, D.B.: Uppaal SMC tutorial. STTT 17(4), 397–415 (2015)

    Article  Google Scholar 

  35. Ellen, C., Gerwinn, S., Fränzle, M.: Confidence bounds for statistical model checking of probabilistic hybrid systems. In: Jurdziński, M., Ničković, D. (eds.) FORMATS 2012. LNCS, vol. 7595, pp. 123–138. Springer, Heidelberg (2012). doi:10.1007/978-3-642-33365-1_10

    Chapter  Google Scholar 

  36. Fijalkow, N., Kiefer, S., Shirmohammadi, M.: Trace refinement in labelled Markov decision processes. In: FOSSACS, pp. 303–318 (2016)

    Google Scholar 

  37. Ferns, N., Panangaden, P., Precup, D.: Metrics for finite Markov decision processes. In: IAAI, pp. 950–951 (2004)

    Google Scholar 

  38. Girard, A., Pappas, G.J.: Approximate bisimulation: a bridge between computer science and control theory. Eur. J. Control 17(5–6), 568–578 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  39. Hansson, H., Jonsson, B.: A logic for reasoning about time and reliability. Formal Asp. Comput. 6(5), 512–535 (1994)

    Article  MATH  Google Scholar 

  40. He, R., Jennings, P., Basu, S., Ghosh, A.P., Wu, H.: A bounded statistical approach for model checking of unbounded until properties. In: ASE, pp. 225–234 (2010)

    Google Scholar 

  41. Hermanns, H., Kwiatkowska, M., Norman, G., Parker, D., Siegle, M.: On the use of MTBDDs for performability analysis and verification of stochastic systems. J. Logic Algebraic Program. 56(1–2), 23–67 (2003)

    Article  MathSciNet  MATH  Google Scholar 

  42. Hérault, T., Lassaigne, R., Magniette, F., Peyronnet, S.: Approximate probabilistic model checking. In: Steffen, B., Levi, G. (eds.) VMCAI 2004. LNCS, vol. 2937, pp. 73–84. Springer, Heidelberg (2004). doi:10.1007/978-3-540-24622-0_8

    Chapter  Google Scholar 

  43. Henriques, D., Martins, J., Zuliani, P., Platzer, A., Clarke, E.M.: Statistical model checking for Markov decision processes. In: QEST, pp. 84–93 (2012)

    Google Scholar 

  44. Hoey, J., St-aubin, R., Hu, A., Boutilier, C.: Spudd: stochastic planning using decision diagrams. In: Proceedings of the Fifteenth Conference on Uncertainty in Artificial Intelligence, pp. 279–288. Morgan Kaufmann (1999)

    Google Scholar 

  45. Jha, S.K., Clarke, E.M., Langmead, C.J., Legay, A., Platzer, A., Zuliani, P.: A Bayesian approach to model checking biological systems. In: Degano, P., Gorrieri, R. (eds.) CMSB 2009. LNCS, vol. 5688, pp. 218–234. Springer, Heidelberg (2009). doi:10.1007/978-3-642-03845-7_15

    Chapter  Google Scholar 

  46. Jégourel, C., Legay, A., Sedwards, S.: A platform for high performance statistical model checking - PLASMA. In: TACAS, pp. 498–503 (2012)

    Google Scholar 

  47. Jaeger, M., Mao, H., Guldstrand Larsen, K., Mardare, R.: Continuity properties of distances for Markov processes. In: Norman, G., Sanders, W. (eds.) QEST 2014. LNCS, vol. 8657, pp. 297–312. Springer, Heidelberg (2014). doi:10.1007/978-3-319-10696-0_24

    Google Scholar 

  48. Kushmerick, N., Hanks, S., Weld, D.: An algorithm for probabilistic least-commitment planning. In: Proceedings of AAAI-94, pp. 1073–1078 (1994)

    Google Scholar 

  49. Kearns, M., Koller, D.: Efficient reinforcement learning in factored MDPs. In: IJCAI, pp. 740–747. Morgan Kaufmann Publishers Inc., San Francisco (1999)

    Google Scholar 

  50. Koller, D., Parr, R.: Computing factored value functions for policies in structured MDPs. In: Proceedings of the Sixteenth International Joint Conference on Artificial Intelligence, pp. 1332–1339. Morgan Kaufmann (1999)

    Google Scholar 

  51. Kwiatkowska, M., Parker, D.: Automated verification and strategy synthesis for probabilistic systems. In: Hung, D., Ogawa, M. (eds.) ATVA 2013. LNCS, vol. 8172, pp. 5–22. Springer, Heidelberg (2013). doi:10.1007/978-3-319-02444-8_2

    Chapter  Google Scholar 

  52. Kemeny, J., Snell, J., Knapp, A.: Denumerable Markov Chains. Springer, New York (1976)

    Book  MATH  Google Scholar 

  53. Larsen, K.G.: Statistical model checking, refinement checking, optimization, \(..\)for stochastic hybrid systems. In: Jurdziński, M., Ničković, D. (eds.) FORMATS 2012. LNCS, vol. 7595, pp. 7–10. Springer, Heidelberg (2012). doi:10.1007/978-3-642-33365-1_2

    Chapter  Google Scholar 

  54. Guldstrand Larsen, K.: Priced timed automata and statistical model checking. In: Johnsen, E.B., Petre, L. (eds.) IFM 2013. LNCS, vol. 7940, pp. 154–161. Springer, Heidelberg (2013). doi:10.1007/978-3-642-38613-8_11

    Chapter  Google Scholar 

  55. Leitner-Fischer, F., Leue, S.: Probabilistic fault tree synthesis using causality computation. IJCCBS 4(2), 119–143 (2013)

    Article  Google Scholar 

  56. Lassaigne, R., Peyronnet, S.: Probabilistic verification and approximation. Ann. Pure Appl. Logic 152(1–3), 122–131 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  57. Lassaigne, R., Peyronnet, S.: Approximate planning and verification for large Markov decision processes. In: SAC, pp. 1314–1319 (2012)

    Google Scholar 

  58. Lehmann, D., Shelah, S.: Reasoning with time and chance. In: Diaz, J. (ed.) ICALP 1983. LNCS, vol. 154, pp. 445–457. Springer, Heidelberg (1983). doi:10.1007/BFb0036928

    Chapter  Google Scholar 

  59. Larsen, K.G., Skou, A: Bisimulation through probabilistic testing. In: POPL, pp. 344–352 (1989)

    Google Scholar 

  60. Legay, A., Sedwards, S., Traonouez, L.-M.: Scalable verification of Markov decision processes. In: Canal, C., Idani, A. (eds.) SEFM 2014. LNCS, vol. 8938, pp. 350–362. Springer, Heidelberg (2015). doi:10.1007/978-3-319-15201-1_23

    Google Scholar 

  61. Miner, A., Parker, D.: Symbolic representations and analysis of large probabilistic systems. In: Baier, C., Haverkort, B.R., Hermanns, H., Katoen, J.-P., Siegle, M. (eds.) Validation of Stochastic Systems. LNCS, vol. 2925, pp. 296–338. Springer, Heidelberg (2004). doi:10.1007/978-3-540-24611-4_9

    Chapter  Google Scholar 

  62. Palaniappan, S.K., Gyori, B.M., Liu, B., Hsu, D., Thiagarajan, P.S.: Statistical model checking based calibration and analysis of bio-pathway models. In: Gupta, A., Henzinger, T.A. (eds.) CMSB 2013. LNCS, vol. 8130, pp. 120–134. Springer, Heidelberg (2013). doi:10.1007/978-3-642-40708-6_10

    Chapter  Google Scholar 

  63. Pyeatt, L.D.: Reinforcement learning with decision trees. In: The 21st IASTED International Multi-Conference on Applied Informatics (AI 2003), February 10–13, 2003, Innsbruck, Austria, pp. 26–31 (2003)

    Google Scholar 

  64. Rabih, D., Pekergin, N.: Statistical model checking using perfect simulation. In: Liu, Z., Ravn, A.P. (eds.) ATVA 2009. LNCS, vol. 5799, pp. 120–134. Springer, Heidelberg (2009). doi:10.1007/978-3-642-04761-9_11

    Chapter  Google Scholar 

  65. Sutton, R., Barto, A., Learning, R.: An Introduction. MIT Press, Cambridge (1998)

    Book  Google Scholar 

  66. Raghavendra, C.S., Liu, S., Panangadan, A., Talukder, A.: Compact representation of coordinated sampling policies for body sensor networks. In: Proceedings of Workshop on Advances in Communication and Networks (Smart Homes for Tele-Health), pp. 6–10. IEEE (2010)

    Google Scholar 

  67. Strehl, A.L., Li, L., Wiewiora, E., Langford, J., Littman, M.L.: PAC model-free reinforcement learning. In: ICML, pp. 881–888 (2006)

    Google Scholar 

  68. Sen, K., Viswanathan, M., Agha, G.: Statistical model checking of black-box probabilistic systems. In: Alur, R., Peled, D.A. (eds.) CAV 2004. LNCS, vol. 3114, pp. 202–215. Springer, Heidelberg (2004). doi:10.1007/978-3-540-27813-9_16

    Chapter  Google Scholar 

  69. Sen, K., Viswanathan, M., Agha, G.: On statistical model checking of stochastic systems. In: Etessami, K., Rajamani, S.K. (eds.) CAV 2005. LNCS, vol. 3576, pp. 266–280. Springer, Heidelberg (2005). doi:10.1007/11513988_26

    Chapter  Google Scholar 

  70. Vardi, M.Y.: Automatic verification of probabilistic concurrent finite-state programs. In: FOCS, pp. 327–338 (1985)

    Google Scholar 

  71. van Breugel, F., Sharma, B., Worrell, J.: Approximating a behavioural pseudometric without discount for probabilistic systems. In: FOSSACS, pp. 123–137 (2007)

    Google Scholar 

  72. van Breugel, F., Worrell, J.: Approximating and computing behavioural distances in probabilistic transition systems. Theor. Comput. Sci. 360(1–3), 373–385 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  73. Wimmer, R., Braitling, B., Becker, B., Hahn, E.M., Crouzen, P., Hermanns, H., Dhama, A., Theel, O.: Symblicit calculation of long-run averages for concurrent probabilistic systems. In: QEST, pp. 27–36, Washington, DC, USA. IEEE Computer Society (2010)

    Google Scholar 

  74. Wimmer, R., Jansen, N., Vorpahl, A., Ábrahám, E., Katoen, J.-P., Becker, B.: High-level counterexamples for probabilistic automata. In: Joshi, K., Siegle, M., Stoelinga, M., D’Argenio, P.R. (eds.) QEST 2013. LNCS, vol. 8054, pp. 39–54. Springer, Heidelberg (2013). doi:10.1007/978-3-642-40196-1_4

    Chapter  Google Scholar 

  75. Younes, H.L.S., Clarke, E.M., Zuliani, P.: Statistical verification of probabilistic properties with unbounded until. In: Davies, J., Silva, L., Simao, A. (eds.) SBMF 2010. LNCS, vol. 6527, pp. 144–160. Springer, Heidelberg (2011). doi:10.1007/978-3-642-19829-8_10

    Chapter  Google Scholar 

  76. Younes, H.L.S., Kwiatkowska, M.Z., Norman, G., Parker, D.: Numerical vs. statistical probabilistic model checking. STTT 8(3), 216–228 (2006)

    Article  MATH  Google Scholar 

  77. Younes, H.L.S., Simmons, R.G.: Probabilistic verification of discrete event systems using acceptance sampling. In: Brinksma, E., Larsen, K.G. (eds.) CAV 2002. LNCS, vol. 2404, pp. 223–235. Springer, Heidelberg (2002). doi:10.1007/3-540-45657-0_17

    Chapter  Google Scholar 

  78. Zuliani, P., Platzer, A., Clarke, E.M. Bayesian statistical model checking with application to simulink/stateflow verification. In: HSCC, pp. 243–252 (2010)

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Jan Křetínský .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Springer International Publishing AG

About this paper

Cite this paper

Křetínský, J. (2016). Survey of Statistical Verification of Linear Unbounded Properties: Model Checking and Distances. In: Margaria, T., Steffen, B. (eds) Leveraging Applications of Formal Methods, Verification and Validation: Foundational Techniques. ISoLA 2016. Lecture Notes in Computer Science(), vol 9952. Springer, Cham. https://doi.org/10.1007/978-3-319-47166-2_3

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-47166-2_3

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-47165-5

  • Online ISBN: 978-3-319-47166-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics