Sequential Schemes for Frequentist Estimation of Properties in Statistical Model Checking
Statistical Model Checking (SMC) is an approximate verification method that overcomes the state space explosion problem for probabilistic systems by Monte Carlo simulations. Simulations might be however costly if many samples are required. It is thus necessary to implement efficient algorithms to reduce the sample size while preserving precision and accuracy. In the literature, some sequential schemes have been provided for the estimation of property occurrence based on predefined confidence and absolute or relative error. Nevertheless, these algorithms remain conservative and may result in huge sample sizes if the required precision standards are demanding. In this article, we compare some useful bounds and some sequential methods based on frequentist estimations. We propose outperforming and rigorous alternative schemes, based on Massart bounds and robust confidence intervals. Our theoretical and empirical analysis show that our proposal reduces the sample size while providing guarantees on error bounds.
Cyrille Jegourel and Jun Sun are partially supported by NRF grant GNRF1501 and Jin Song Dong by the project: Reliable Prototyping Framework for Daily Living Assistance of Frail Ageing People (RELIANCE).
- 3.Chen, J.: Properties of a new adaptive sampling method with applications to scalable learning. In: WI, pp. 9–15, Atlanta (2013)Google Scholar
- 12.Hérault, T., Lassaigne, R., Peyronnet, S.: APMC 3.0: approximate verification of discrete and continuous time Markov chains. In: QEST, pp. 129–130 (2006)Google Scholar
- 16.Kwiatkowska, M.Z., Norman, G., Parker, D.: PRISM 2.0: a tool for probabilistic model checking. In: QEST, pp. 322–323. IEEE (2004)Google Scholar
- 22.Younes, H.: Verification and planning for stochastic processes with asynchronous events. Ph.D. thesis, Carnegie Mellon University (2004)Google Scholar