Advertisement

Sound statistical model checking for MDP using partial order and confluence reduction

  • Arnd Hartmanns
  • Mark Timmer
SMC

Abstract

Statistical model checking (SMC) is an analysis method that circumvents the state space explosion problem in model-based verification by combining probabilistic simulation with statistical methods that provide clear error bounds. As a simulation-based technique, it can in general only provide sound results if the underlying model is a stochastic process. In verification, however, models are very often variations of nondeterministic transition systems. In classical exhaustive model checking, partial order reduction and confluence reduction allow the removal of spurious nondeterministic choices. In this paper, we show that both can be adapted to detect and discard such choices on-the-fly during simulation, thus extending the applicability of SMC to a subclass of Markov decision processes. We prove their correctness in a uniform way and study their effectiveness and efficiency using an implementation in the modes simulator for the Modest modelling language. The examples we use highlight the different strengths and limitations of the two approaches. We find that runtime may be affected by unnecessary recomputations, and thus also investigate the feasibility of caching results to speed up simulation at the cost of increased memory usage.

Keywords

Statistical model checking Simulation Markov decision processes Partial order reduction Confluence reduction 

Notes

Acknowledgments

This work was supported by the Transregional Collaborative Research Centre SFB/TR 14 AVACS, by the NWO-DFG bilateral project ROCKS, by the 7th EU Framework Programme under grant agreements 295261 (MEALS) and 318490 (SENSATION), by NWO under grant 612.063.817 (SYRUP), and by the CAS/SAFEA International Partnership Program for Creative Research Teams.

References

  1. 1.
    Agresti, A., Coull, B.A.: Approximate is better than “exact” for interval estimation of binomial proportions. Am. Stat. 52(2), 119–126 (1998)MathSciNetGoogle Scholar
  2. 2.
    Andel, T.R., Yasinsac, A.: On the credibility of MANET simulations. IEEE Comput. 39(7), 48–54 (2006)CrossRefGoogle Scholar
  3. 3.
    Baier, C., D’Argenio, P.R., Größer, M.: Partial order reduction for probabilistic branching time. Electron. Notes Theor. Comput. Sci. 153(2), 97–116 (2006)CrossRefGoogle Scholar
  4. 4.
    Baier, C., Größer, M., Ciesinski, F.: Partial order reduction for probabilistic systems. In: QEST, pp. 230–239. IEEE Computer Society, New York (2004)Google Scholar
  5. 5.
    Baier, C., Katoen, J.-P.: Principles of Model Checking. MIT Press, Cambridge (2008)zbMATHGoogle Scholar
  6. 6.
    Blom, S.: Partial \(\tau \)-confluence for efficient state space generation. Technical Report SEN-R0123, CWI (2001)Google Scholar
  7. 7.
    Blom, S., van de Pol, J.: State space reduction by proving confluence. In: Brinksma, E., Larsen, K.G. (eds.) CAV. Lecture Notes in Computer Science, vol. 2404, pp. 596–609. Springer, Berlin (2002)Google Scholar
  8. 8.
    Bogdoll, J., Fioriti, L.M.F., Hartmanns, A., Hermanns, H.: Partial order methods for statistical model checking and simulation. In: Bruni, R., Dingel, J. (eds.) FMOODS/FORTE. Lecture Notes in Computer Science, vol. 6722, pp. 59–74. Springer, Berlin (2011)Google Scholar
  9. 9.
    Bogdoll, J., Hartmanns, A., Hermanns, H.: Simulation and statistical model checking for modestly nondeterministic models. In: Schmitt, J.B. (ed.) MMB/DFT. Lecture Notes in Computer Science, vol. 7201, pp. 249–252. Springer, Berlin (2012)Google Scholar
  10. 10.
    Brázdil, T., Chatterjee, K., Chmelik, M., Forejt, V., Kretínský, J., Kwiatkowska, M.Z., Parker, D., Ujma, M.: Verification of Markov decision processes using learning algorithms. CoRR, abs/1402.2967 (2014)Google Scholar
  11. 11.
    Chaum, D.: The dining cryptographers problem: unconditional sender and recipient untraceability. J. Cryptol. 1(1), 65–75 (1988)MathSciNetCrossRefzbMATHGoogle Scholar
  12. 12.
    D’Argenio, P.R., Niebert, P.: Partial order reduction on concurrent probabilistic programs. In: QEST, pp. 240–249. IEEE Computer Society, New York (2004)Google Scholar
  13. 13.
    Evangelista, S., Pajault, C.: Solving the ignoring problem for partial order reduction. STTT 12(2), 155–170 (2010)CrossRefGoogle Scholar
  14. 14.
    Forejt, V., Kwiatkowska, M.Z., Norman, G., Parker, D.: Automated verification techniques for probabilistic systems. In: Bernardo, M., Issarny, V. (eds.) SFM. Lecture Notes in Computer Science, vol. 6659, pp. 53–113. Springer, Berlin (2011)Google Scholar
  15. 15.
    Giro, S., D’Argenio, P.R., Fioriti, L.M.F.: Partial order reduction for probabilistic systems: a revision for distributed schedulers. In: Bravetti, M., Zavattaro, G. (eds.) CONCUR. Lecture Notes in Computer Science, vol. 5710, pp. 338–353. Springer, Berlin (2009)Google Scholar
  16. 16.
    Godefroid, P.: Partial-order methods for the verification of concurrent systems—an approach to the state-explosion problem. Lecture Notes in Computer Science, vol. 1032. Springer, Berlin (1996)Google Scholar
  17. 17.
    Groote, J.F., van de Pol, J.: State space reduction using partial tau-confluence. In: Nielsen, M., Rovan, B. (eds.) MFCS. Lecture Notes in Computer Science, vol. 1893, pp. 383–393. Springer, Berlin (2000)Google Scholar
  18. 18.
    Hansen, H., Kwiatkowska, M.Z., Qu, H.: Partial order reduction for model checking Markov decision processes under unconditional fairness. In: QEST, pp. 203–212. IEEE Computer Society, New York (2011)Google Scholar
  19. 19.
    Hansen, H., Timmer, M.: A comparison of confluence and ample sets in probabilistic and non-probabilistic branching time. Theor. Comput. Sci. 538C, 103–123 (2014)MathSciNetCrossRefGoogle Scholar
  20. 20.
    Hartmanns, A., Hermanns, H.: A modest approach to checking probabilistic timed automata. In: QEST, pp. 187–196. IEEE Computer Society, New York (2009)Google Scholar
  21. 21.
    Hartmanns, A., Hermanns, H.: The modest toolset: an integrated environment for quantitative modelling and verification. In: Ábrahám, E., Havelund, K. (eds.) TACAS. Lecture Notes in Computer Science, vol. 8413, pp. 593–598. Springer, Berlin (2014)Google Scholar
  22. 22.
    Hartmanns, A., Timmer, M.: On-the-fly confluence detection for statistical model checking. In: Brat, G., Rungta, N., Venet, A. (eds.) NASA Formal Methods. Lecture Notes in Computer Science, vol. 7871, pp. 337–351. Springer, Berlin (2013)Google Scholar
  23. 23.
    Henriques, D., Martins, J., Zuliani, P., Platzer, A., Clarke, E.M.: Statistical model checking for Markov decision processes. In: QEST, pp. 84–93. IEEE Computer Society, New York (2012)Google Scholar
  24. 24.
    Hérault, T., Lassaigne, R., Magniette, F., Peyronnet, S.: Approximate probabilistic model checking. In: Steffen, B., Levi, G. (eds.) VMCAI. Lecture Notes in Computer Science, vol. 2937, pp. 73–84. Springer, Berlin (2004)Google Scholar
  25. 25.
    Hoeffding, W.: Probability inequalities for sums of bounded random variables. J. Am. Stat. Assoc. 58(301), 13–30 (1963)MathSciNetCrossRefzbMATHGoogle Scholar
  26. 26.
    Kwiatkowska, M.Z., Norman, G., Parker, D.: PRISM 4.0: Verification of probabilistic real-time systems. In: Gopalakrishnan, G., Qadeer, S. (eds.) CAV. LNCS, vol. 6806, pp. 585–591. Springer, Berlin (2011)Google Scholar
  27. 27.
    Lassaigne, R., Peyronnet, S.: Approximate planning and verification for large Markov decision processes. In: Ossowski, S., Lecca, P. (eds.) SAC, pp. 1314–1319. ACM, New York (2012)Google Scholar
  28. 28.
    Legay, A., Delahaye, B., Bensalem, S.: Statistical model checking: an overview. In: Barringer, H., Falcone, Y., Finkbeiner, B., Havelund, K., Lee, I., Pace, G.J., Rosu, G., Sokolsky, O., Tillmann, N. (eds.) RV. Lecture Notes in Computer Science, vol. 6418, pp. 122–135. Springer, Berlin (2010)Google Scholar
  29. 29.
    Legay, A., Sedwards, S.: Lightweight Monte Carlo algorithm for Markov decision processes. CoRR, abs/1310.3609 (2013)Google Scholar
  30. 30.
    Mateescu, R., Wijs, A.: Sequential and distributed on-the-fly computation of weak tau-confluence. Sci. Comput. Program. 77(10–11), 1075–1094 (2012)CrossRefzbMATHGoogle Scholar
  31. 31.
    Nimal, V.: Statistical approaches for probabilistic model checking. Master’s thesis, Oxford University (2010)Google Scholar
  32. 32.
    Pace, G.J., Lang, F., Mateescu, R.: Calculating-confluence compositionally. In: Hunt Jr., W.A., Somenzi, F. (eds.) CAV. Lecture Notes in Computer Science, vol. 2725, pp. 446–459. Springer, Berlin (2003)Google Scholar
  33. 33.
    Peled, D.: Combining partial order reductions with on-the-fly model-checking. In: Dill, D.L. (ed.) CAV. Lecture Notes in Computer Science, vol. 818, pp. 377–390. Springer, Berlin (1994)Google Scholar
  34. 34.
    Peled, D.: Combining partial order reductions with on-the-fly model-checking. Formal Methods Syst. Des. 8(1), 39–64 (1996)CrossRefGoogle Scholar
  35. 35.
    Ross, S.M.: Simulation, 4th edn. Elsevier Academic Press, Amsterdam (2006)zbMATHGoogle Scholar
  36. 36.
    Timmer, M.: Efficient Modelling, Generation and Analysis of Markov Automata. PhD thesis, University of Twente, The Netherlands (2013)Google Scholar
  37. 37.
    Timmer, M., Stoelinga, M., van de Pol, J.: Confluence reduction for probabilistic systems. In: Abdulla, P.A., Rustan, K., Leino, M. (eds.) TACAS. Lecture Notes in Computer Science, vol. 6605, pp. 311–325. Springer, Berlin (2011)Google Scholar
  38. 38.
    Timmer, M., van de Pol, J., Stoelinga, M.: Confluence reduction for Markov automata. In: Braberman, V.A., Fribourg, L. (eds.) FORMATS. Lecture Notes in Computer Science, vol. 8053, pp. 243–257. Springer, Berlin (2013)Google Scholar
  39. 39.
    Valmari, A.: A stubborn attack on state explosion. In: Clarke, E.M., Kurshan, R.P. (eds.) CAV. Lecture Notes in Computer Science, vol. 531, pp. 156–165. Springer, Berlin (1990)Google Scholar
  40. 40.
    Younes, H.L.S., Kwiatkowska, M.Z., Norman, G., Parker, D.: Numerical vs. statistical probabilistic model checking. STTT 8(3), 216–228 (2006)CrossRefGoogle Scholar
  41. 41.
    Younes, H.L.S., Simmons, R.G.: Probabilistic verification of discrete event systems using acceptance sampling. In: Brinksma, E., Larsen, K.G. (eds.) CAV. Lecture Notes in Computer Science, vol. 2404, pp. 223–235. Springer, Berlin (2002)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2014

Authors and Affiliations

  1. 1.Saarland UniversitySaarbrückenGermany
  2. 2.University of TwenteEnschedeThe Netherlands

Personalised recommendations