A Review of Statistical Model Checking Pitfalls on Real-Time Stochastic Models

  • Dimitri Bohlender
  • Harold Bruintjes
  • Sebastian Junges
  • Jens Katelaan
  • Viet Yen Nguyen
  • Thomas Noll
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8803)

Abstract

Statistical model checking (SMC) is a technique inspired by Monte-Carlo simulation for verifying time-bounded temporal logical properties. SMC originally focused on fully stochastic models such as Markov chains, but its scope has recently been extended to cover formalisms that mix functional real-time aspects, concurrency and non-determinism. We show by various examples using the tools UPPAAL SMC and Modes that combining the stochastic interpretation of such models with SMC algorithms is extremely subtle. This may yield significant discrepancies in the analysis results. As these subtleties are not so obvious to the end-user, we present five semantic caveats and give a classification scheme for SMC algorithms. We argue that caution is needed and believe that the caveats and classification scheme in this paper serve as a guiding reference for thoroughly understanding them.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Alur, R., Dill, D.L.: A Theory of Timed Automata. Theoretical Computer Science 126(2), 183–235 (1994)MathSciNetCrossRefMATHGoogle Scholar
  2. 2.
    Alur, R., La Torre, S., Pappas, G.J.: Optimal Paths in Weighted Timed Automata. In: Di Benedetto, M.D., Sangiovanni-Vincentelli, A.L. (eds.) HSCC 2001. LNCS, vol. 2034, pp. 49–62. Springer, Heidelberg (2001)Google Scholar
  3. 3.
    Baier, C., Bertrand, N., Bouyer, P., Brihaye, T., Größer, M.: Probabilistic and Topological Semantics for Timed Automata. In: Arvind, V., Prasad, S. (eds.) FSTTCS 2007. LNCS, vol. 4855, pp. 179–191. Springer, Heidelberg (2007)Google Scholar
  4. 4.
    Baier, C., Haverkort, B., Hermanns, H., Katoen, J.-P.: Model Checking Algorithms for Continuous-Time Markov Chains. IEEE Transactions on Software Engineering 29(6), 524–541 (2003)CrossRefGoogle Scholar
  5. 5.
    Behrmann, G., Fehnker, A., Hune, T., Larsen, K.G., Pettersson, P., Romijn, J., Vaandrager, F.W.: Minimum-Cost Reachability for Priced Timed Automata. In: Di Benedetto, M.D., Sangiovanni-Vincentelli, A.L. (eds.) HSCC 2001. LNCS, vol. 2034, pp. 147–161. Springer, Heidelberg (2001)Google Scholar
  6. 6.
    Bogdoll, J., Ferrer Fioriti, L.M., Hartmanns, A., Hermanns, H.: Partial Order Methods for Statistical Model Checking and Simulation. In: Bruni, R., Dingel, J. (eds.) FMOODS/FORTE 2011. LNCS, vol. 6722, pp. 59–74. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  7. 7.
    Bogdoll, J., Hartmanns, A., Hermanns, H.: Simulation and Statistical Model Checking for Modestly Nondeterministic Models. In: Schmitt, J.B. (ed.) MMB & DFT 2012. LNCS, vol. 7201, pp. 249–252. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  8. 8.
    Bohnenkamp, H., D’Argenio, P.R., Hermanns, H., Katoen, J.-P.: MODEST: A Compositional Modeling Formalism for Hard and Softly Timed Systems. IEEE Transactions on Software Engineering 32(10), 812–830 (2006)CrossRefGoogle Scholar
  9. 9.
    Boyer, B., Corre, K., Legay, A., Sedwards, S.: PLASMA-lab: A Flexible, Distributable Statistical Model Checking Library. In: Joshi, K., Siegle, M., Stoelinga, M., D’Argenio, P.R. (eds.) QEST 2013. LNCS, vol. 8054, pp. 160–164. Springer, Heidelberg (2013)CrossRefGoogle Scholar
  10. 10.
    Bruintjes, H., Nguyen, V.Y.: Test results from experiments, http://www-i2.informatik.rwth-aachen.de/~isola2014/smc/index.html (Online; accessed May 14, 2014)
  11. 11.
    Bulychev, P., David, A., Larsen, K.G., Mikučionis, M., Bøgsted Poulsen, D., Legay, A., Wang, Z.: UPPAAL-SMC: Statistical Model Checking for Priced Timed Automata. In: Wiklicky, H., Massink, M. (eds.) QAPL. Electronic Proceedings in Theoretical Computer Science, vol. 85, pp. 1–16. Open Publishing Association (2012)Google Scholar
  12. 12.
    Fränzle, M., Hahn, E.M., Hermanns, H., Wolovick, N., Zhang, L.: Measurability and Safety Verification for Stochastic Hybrid Systems. In: HSCC 2011, pp. 43–52. ACM (2011)Google Scholar
  13. 13.
    Grosu, R., Smolka, S.A.: Monte Carlo Model Checking. In: Halbwachs, N., Zuck, L.D. (eds.) TACAS 2005. LNCS, vol. 3440, pp. 271–286. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  14. 14.
    Hartmanns, A., Timmer, M.: On-the-Fly Confluence Detection for Statistical Model Checking. In: Brat, G., Rungta, N., Venet, A. (eds.) NFM 2013. LNCS, vol. 7871, pp. 337–351. Springer, Heidelberg (2013)CrossRefGoogle Scholar
  15. 15.
    Hermanns, H. (ed.): Interactive Markov Chains. LNCS, vol. 2428. Springer, Heidelberg (2002)MATHGoogle Scholar
  16. 16.
    Jegourel, C., Legay, A., Sedwards, S.: A Platform for High Performance Statistical Model Checking – PLASMA. In: Flanagan, C., König, B. (eds.) TACAS 2012. LNCS, vol. 7214, pp. 498–503. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  17. 17.
    Legay, A., Delahaye, B., Bensalem, S.: Statistical Model Checking: An Overview. In: Barringer, H., Falcone, Y., Finkbeiner, B., Havelund, K., Lee, I., Pace, G., Roşu, G., Sokolsky, O., Tillmann, N. (eds.) RV 2010. LNCS, vol. 6418, pp. 122–135. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  18. 18.
    Marsan, M.A., Balbo, G., Bobbio, A., Chiola, G., Conte, G., Cumani, A.: The Effect of Execution Policies on the Semantics and Analysis of Stochastic Petri Nets. IEEE Transactions on Software Engineering 15(7), 832–846 (1989)CrossRefGoogle Scholar
  19. 19.
    Sproston, J.: Decidable Model Checking of Probabilistic Hybrid Automata. In: Joseph, M. (ed.) FTRTFT 2000. LNCS, vol. 1926, pp. 31–45. Springer, Heidelberg (2000)CrossRefGoogle Scholar
  20. 20.
    Tripakis, S.: Verifying Progress in Timed Systems. In: Katoen, J.-P. (ed.) ARTS 1999. LNCS, vol. 1601, pp. 299–314. Springer, Heidelberg (1999)CrossRefGoogle Scholar
  21. 21.
    Younes, H.L., Simmons, R.G.: Statistical Probabilistic Model Checking With a Focus on Time-Bounded Properties. Information and Computation 204(9), 1368–1409 (2006)MathSciNetCrossRefMATHGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2014

Authors and Affiliations

  • Dimitri Bohlender
    • 1
  • Harold Bruintjes
    • 1
  • Sebastian Junges
    • 1
  • Jens Katelaan
    • 1
  • Viet Yen Nguyen
    • 1
    • 2
  • Thomas Noll
    • 1
  1. 1.Software Modeling and Verification GroupRWTH Aachen UniversityGermany
  2. 2.Fraunhofer IESEGermany

Personalised recommendations