Abstract
Several schemes have been provided in Statistical Model Checking (SMC) for the estimation of property occurrence based on predefined confidence and absolute or relative error. Simulations might be however costly if many samples are required and the usual algorithms implemented in statistical model checkers tend to be conservative. Bayesian and rare event techniques can be used to reduce the sample size but they can not be applied without prerequisite or knowledge about the system under scrutiny. Recently, sequential algorithms based on Monte Carlo estimations and Massart bounds have been proposed to reduce the sample size while providing guarantees on error bounds which has been shown to outperform alternative frequentist approachesĀ [15]. In this work, we discuss some features regarding the distribution and the optimisation of these algorithms.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
- 2.
The Okamoto bound is sometimes called the Chernoff bound in the literature.
- 3.
A journal version with the proofs is currently submittedĀ [14]. The proofs are also available here: https://www.researchgate.net/publication/317823195_Sequenti-al_Schemes_for_Frequentist_Estimation_of_Properties_in_Statistical_Model_Checking.
References
Agresti, A., Caffo, B.: Simple and effective confidence intervals for proportions and differences of proportions result from adding two successes and two failures. Am. Stat. 54(4), 280ā288 (2000)
Ballarini, P., Barbot, B., Duflot, M., Haddad, S., Pekergin, N.: HASL: a new approach for performance evaluation and model checking from concepts to experimentation. Perform. Eval. 90, 53ā77 (2015)
Brown, L., Cai, T., DasGupta, A.: Interval estimation for a binomial proportion. Stat. Sci. 16(2), 101ā133 (2001)
Chen, Y., Poskitt, C.M., Sun, J.: Learning from mutants: using code mutation to learn and monitor invariants of a cyber-physical system. In: SP, pp. 648ā660 (2018)
Chernoff, H.: A measure of asymptotic efficiency for tests of a hypothesis based on the sum of observations. Ann. Math. Statist. 23(4), 493ā507 (1952)
Clopper, C.J., Pearson, E.S.: The use of confidence or fiducial limits illustrated in the case of the binomial. Biometrika 26, 404ā413 (1934)
Dagum, P., Karp, R.M., Luby, M., Ross, S.M.: An optimal algorithm for Monte Carlo estimation. SIAM J. Comput. 29(5), 1484ā1496 (2000)
David, A., Larsen, K.G., Legay, A., Mikucionis, M., Poulsen, D.B.: Uppaal SMC tutorial. STTT 17(4), 397ā415 (2015)
Grosu, R., Peled, D., Ramakrishnan, C.R., Smolka, S.A., Stoller, S.D., Yang, J.: Using statistical model checking for measuring systems. In: Margaria, T., Steffen, B. (eds.) ISoLA 2014. LNCS, vol. 8803, pp. 223ā238. Springer, Heidelberg (2014). https://doi.org/10.1007/978-3-662-45231-8_16
HĆ©rault, T., Lassaigne, R., Magniette, F., Peyronnet, S.: Approximate probabilistic model checking. In: Steffen, B., Levi, G. (eds.) VMCAI 2004. LNCS, vol. 2937, pp. 73ā84. Springer, Heidelberg (2004). https://doi.org/10.1007/978-3-540-24622-0_8
Jegourel, C., Legay, A., Sedwards, S.: A platform for high performance statistical model checking ā PLASMA. In: Flanagan, C., Kƶnig, B. (eds.) TACAS 2012. LNCS, vol. 7214, pp. 498ā503. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-28756-5_37
Jegourel, C., Legay, A., Sedwards, S.: Importance splitting for statistical model checking rare properties. In: Sharygina, N., Veith, H. (eds.) CAV 2013. LNCS, vol. 8044, pp. 576ā591. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-39799-8_38
Jegourel, C., Legay, A., Sedwards, S.: Command-based importance sampling for statistical model checking. Theor. Comput. Sci. 649, 1ā24 (2016)
Jegourel, C., Sun, J., Dong, J.S.: Sequential schemes for frequentist estimation of properties in statistical model checking (Journal version). Currently submitted
Jegourel, C., Sun, J., Dong, J.S.: Sequential schemes for frequentist estimation of properties in statistical model checking. In: Bertrand, N., Bortolussi, L. (eds.) QEST 2017. LNCS, vol. 10503, pp. 333ā350. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-66335-7_23
Kwiatkowska, M.Z., Norman, G., Parker, D.: PRISM 2.0: a tool for probabilistic model checking. In: QEST, pp. 322ā323. IEEE (2004)
Massart, P.: The tight constant in the Dvoretzky-Kiefer-Wolfowitz inequality. Ann. Probab. 18, 1269ā1283 (1990)
Metropolis, N., Ulam, S.: The Monte Carlo method. J. Am. Stat. Assoc. 44(247), 335ā341 (1949)
Okamoto, M.: Some inequalities relating to the partial sum of binomial probabilities. Ann. Inst. Stat. Math. 10, 29ā35 (1958)
Samuels, M.L., Witmer, J.W.: Statistics for the Life Sciences, 2nd edn. Prentice Hall, Englewood Cliffs (1999)
Wald, A.: Sequential tests of statistical hypotheses. Ann. Math. Stat. 16(2), 117ā186 (1945)
Younes, H.: Verification and planning for stochastic processes with asynchronous events. Ph.D. thesis, Carnegie Mellon University (2004)
Zuliani, P., Platzer, A., Clarke, E.M.: Bayesian statistical model checking with application to stateflow/simulink verification. FMSD 43(2), 338ā367 (2013)
Acknowledgment
This work was supported in part by the National Research Foundation (NRF), Prime Ministerās Office, Singapore, under its National Cybersecurity R&D Programme (Award No. NRF2014NCR-NCR001-040) and administered by the National Cybersecurity R&D Directorate.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
Ā© 2018 Springer Nature Switzerland AG
About this paper
Cite this paper
Jegourel, C., Sun, J., Dong, J.S. (2018). On the Sequential Massart Algorithm for Statistical Model Checking. In: Margaria, T., Steffen, B. (eds) Leveraging Applications of Formal Methods, Verification and Validation. Verification. ISoLA 2018. Lecture Notes in Computer Science(), vol 11245. Springer, Cham. https://doi.org/10.1007/978-3-030-03421-4_19
Download citation
DOI: https://doi.org/10.1007/978-3-030-03421-4_19
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-03420-7
Online ISBN: 978-3-030-03421-4
eBook Packages: Computer ScienceComputer Science (R0)