Abstract
Statistical model checking uses Monte Carlo simulation to analyse stochastic formal models. It avoids state space explosion, but requires rare event simulation techniques to efficiently estimate very low probabilities. One such technique is \(\textsc {Restart}\). Villén-Altamirano recently showed—by way of a theoretical study and ad-hoc implementation—that a generalisation of \(\textsc {Restart}\) to prolonged retrials offers improved performance. In this paper, we demonstrate our independent replication of the original experimental results. We implemented \(\textsc {Restart}\) with prolonged retrials in the and modes tools, and apply them to the models used originally. To do so, we had to resolve ambiguities in the original work, and refine our setup multiple times. We ultimately confirm the previous results, but our experience also highlights the need for precise documentation of experiments to enable replicability in computer science.
Authors are listed alphabetically. This work was supported by NWO via project no. 15474 (SEQUOIA) and VENI grant no. 639.021.754.
Chapter PDF
References
ACM: Artifact review and badging (2020), https://www.acm.org/publications/policies/artifact-review-and-badging-current, version 1.1.
Agha, G., Palmskog, K.: A survey of statistical model checking. ACM Trans. Model. Comput. Simul. 28(1), 6:1–6:39 (2018). https://doi.org/10.1145/3158668
Baier, C., de Alfaro, L., Forejt, V., Kwiatkowska, M.: Model checking probabilistic systems. In: Handbook of Model Checking, pp. 963–999. Springer (2018). https://doi.org/10.1007/978-3-319-10575-8_28
Budde, C.E.: FIG: The finite improbability generator. In: TACAS. LNCS, vol. 12078, pp. 483–491. Springer (2020). https://doi.org/10.1007/978-3-030-45190-5_27
Budde, C.E., Biagi, M., Monti, R.E., D’Argenio, P.R., Stoelinga, M.: Rare event simulation for non-Markovian repairable fault trees. In: TACAS. LNCS, vol. 12078, pp. 463–482. Springer (2020). https://doi.org/10.1007/978-3-030-45190-5_26
Budde, C.E., D’Argenio, P.R., Hartmanns, A.: Automated compositional importance splitting. Sci. Comput. Program. 174, 90–108 (2019). https://doi.org/10.1016/j.scico.2019.01.006
Budde, C.E., D’Argenio, P.R., Hartmanns, A., Sedwards, S.: A statistical model checker for nondeterminism and rare events. In: TACAS. LNCS, vol. 10806, pp. 340–358. Springer (2018). https://doi.org/10.1007/978-3-319-89963-3_20
Buijsrogge, A., de Boer, P.T., Scheinhardt, W.R.W.: Importance sampling for non-Markovian tandem queues using subsolutions. Queueing Systems 93, 31–65 (2019). https://doi.org/10.1007/s11134-019-09623-0
Chow, Y.S., Robbins, H.: On the asymptotic theory of fixed-width sequential confidence intervals for the mean. Ann. Math. Statist. 36(2), 457–462 (1965). https://doi.org/10.1214/aoms/1177700156
Dean, T., Dupuis, P.: Splitting for rare event simulation: A large deviation approach to design and analysis. Stochastic Processes and their Applications 119(2), 562–587 (2009). https://doi.org/10.1016/j.spa.2008.02.017
Garvels, M.J.J., van Ommeren, J.K.C.W., Kroese, D.P.: On the importance function in splitting simulation. Eur. Trans. Telecommun. 13(4), 363–371 (2002). https://doi.org/10.1002/ett.4460130408
Hartmanns, A., Hensel, C., Klauck, M., Klein, J., Kretínský, J., Parker, D., Quatmann, T., Ruijters, E., Steinmetz, M.: The 2019 comparison of tools for the analysis of quantitative formal models. In: TACAS. LNCS, vol. 11429. Springer (2019). https://doi.org/10.1007/978-3-030-17502-3_5
Hartmanns, A., Hermanns, H.: The Modest Toolset: An integrated environment for quantitative modelling and verification. In: TACAS. LNCS, vol. 8413, pp. 593–598. Springer (2014). https://doi.org/10.1007/978-3-642-54862-8_51
Hult, H., Nyquist, P.: Large deviations for weighted empirical measures arising in importance sampling. Stochastic Processes and their Applications 126(1), 138–170 (2016). https://doi.org/10.1016/j.spa.2015.08.002
Legay, A., Sedwards, S., Traonouez, L.M.: Plasma Lab: A modular statistical model checking platform. In: ISoLA. LNCS, vol. 9952, pp. 77–93 (2016). https://doi.org/10.1007/978-3-319-47166-2_6
Reijsbergen, D., Boer, P.T.D., Scheinhardt, W., Juneja, S.: Path-ZVA: General, efficient, and automated importance sampling for highly reliable Markovian systems. ACM Trans. Model. Comput. Simul. 28(3) (2018). https://doi.org/10.1145/3161569
Rubino, G., Tuffin, B.: Introduction to rare event simulation. pp. 1–13. Wiley (2009). https://doi.org/10.1002/9780470745403.ch1
Ruijters, E., Reijsbergen, D., de Boer, P.T., Stoelinga, M.: Rare event simulation for dynamic fault trees. Reliability Engineering & System Safety 186, 220–231 (2019). https://doi.org/10.1016/j.ress.2019.02.004
Villén-Altamirano, J.: RESTART vs Splitting: A comparative study. Performance Evaluation 121–122, 38–47 (2018). https://doi.org/10.1016/j.peva.2018.02.002
Villén-Altamirano, J.: Simulation details of the paper “RESTART vs Splitting: a comparative study”. [19]. https://doi.org/10.1016/j.peva.2018.02.002, Appendix A. Supplementary data
Villén-Altamirano, J.: An improved variant of the rare event simulation method RESTART using prolonged retrials. Operations Research Perspectives 6, 100–108 (2019). https://doi.org/10.1016/j.orp.2019.100108
Villén-Altamirano, M., Villén-Altamirano, J.: RESTART: a method for accelerating rare event simulations. In: Queueing, Performance and Control in ATM (ITC-13). pp. 71–76. Elsevier (1991)
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Open Access This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.
The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.
Copyright information
© 2021 The Author(s)
About this paper
Cite this paper
Budde, C.E., Hartmanns, A. (2021). Replicating \(\textsc {Restart}\) with Prolonged Retrials: An Experimental Report. In: Groote, J.F., Larsen, K.G. (eds) Tools and Algorithms for the Construction and Analysis of Systems. TACAS 2021. Lecture Notes in Computer Science(), vol 12652. Springer, Cham. https://doi.org/10.1007/978-3-030-72013-1_21
Download citation
DOI: https://doi.org/10.1007/978-3-030-72013-1_21
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-72012-4
Online ISBN: 978-3-030-72013-1
eBook Packages: Computer ScienceComputer Science (R0)
-
Published in cooperation with
http://www.etaps.org/
