Skip to main content

Replicating \(\textsc {Restart}\) with Prolonged Retrials: An Experimental Report

  • 1625 Accesses

Part of the Lecture Notes in Computer Science book series (LNTCS,volume 12652)


Statistical model checking uses Monte Carlo simulation to analyse stochastic formal models. It avoids state space explosion, but requires rare event simulation techniques to efficiently estimate very low probabilities. One such technique is \(\textsc {Restart}\). Villén-Altamirano recently showed—by way of a theoretical study and ad-hoc implementation—that a generalisation of \(\textsc {Restart}\) to prolonged retrials offers improved performance. In this paper, we demonstrate our independent replication of the original experimental results. We implemented \(\textsc {Restart}\) with prolonged retrials in the and modes tools, and apply them to the models used originally. To do so, we had to resolve ambiguities in the original work, and refine our setup multiple times. We ultimately confirm the previous results, but our experience also highlights the need for precise documentation of experiments to enable replicability in computer science.

Authors are listed alphabetically. This work was supported by NWO via project no. 15474 (SEQUOIA) and VENI grant no. 639.021.754.


  1. ACM: Artifact review and badging (2020),, version 1.1.

  2. Agha, G., Palmskog, K.: A survey of statistical model checking. ACM Trans. Model. Comput. Simul. 28(1), 6:1–6:39 (2018).

  3. Baier, C., de Alfaro, L., Forejt, V., Kwiatkowska, M.: Model checking probabilistic systems. In: Handbook of Model Checking, pp. 963–999. Springer (2018).

  4. Budde, C.E.: FIG: The finite improbability generator. In: TACAS. LNCS, vol. 12078, pp. 483–491. Springer (2020).

  5. Budde, C.E., Biagi, M., Monti, R.E., D’Argenio, P.R., Stoelinga, M.: Rare event simulation for non-Markovian repairable fault trees. In: TACAS. LNCS, vol. 12078, pp. 463–482. Springer (2020).

  6. Budde, C.E., D’Argenio, P.R., Hartmanns, A.: Automated compositional importance splitting. Sci. Comput. Program. 174, 90–108 (2019).

  7. Budde, C.E., D’Argenio, P.R., Hartmanns, A., Sedwards, S.: A statistical model checker for nondeterminism and rare events. In: TACAS. LNCS, vol. 10806, pp. 340–358. Springer (2018).

  8. Buijsrogge, A., de Boer, P.T., Scheinhardt, W.R.W.: Importance sampling for non-Markovian tandem queues using subsolutions. Queueing Systems 93, 31–65 (2019).

  9. Chow, Y.S., Robbins, H.: On the asymptotic theory of fixed-width sequential confidence intervals for the mean. Ann. Math. Statist. 36(2), 457–462 (1965).

  10. Dean, T., Dupuis, P.: Splitting for rare event simulation: A large deviation approach to design and analysis. Stochastic Processes and their Applications 119(2), 562–587 (2009).

  11. Garvels, M.J.J., van Ommeren, J.K.C.W., Kroese, D.P.: On the importance function in splitting simulation. Eur. Trans. Telecommun. 13(4), 363–371 (2002).

  12. Hartmanns, A., Hensel, C., Klauck, M., Klein, J., Kretínský, J., Parker, D., Quatmann, T., Ruijters, E., Steinmetz, M.: The 2019 comparison of tools for the analysis of quantitative formal models. In: TACAS. LNCS, vol. 11429. Springer (2019).

  13. Hartmanns, A., Hermanns, H.: The Modest Toolset: An integrated environment for quantitative modelling and verification. In: TACAS. LNCS, vol. 8413, pp. 593–598. Springer (2014).

  14. Hult, H., Nyquist, P.: Large deviations for weighted empirical measures arising in importance sampling. Stochastic Processes and their Applications 126(1), 138–170 (2016).

  15. Legay, A., Sedwards, S., Traonouez, L.M.: Plasma Lab: A modular statistical model checking platform. In: ISoLA. LNCS, vol. 9952, pp. 77–93 (2016).

  16. Reijsbergen, D., Boer, P.T.D., Scheinhardt, W., Juneja, S.: Path-ZVA: General, efficient, and automated importance sampling for highly reliable Markovian systems. ACM Trans. Model. Comput. Simul. 28(3) (2018).

  17. Rubino, G., Tuffin, B.: Introduction to rare event simulation. pp. 1–13. Wiley (2009).

  18. Ruijters, E., Reijsbergen, D., de Boer, P.T., Stoelinga, M.: Rare event simulation for dynamic fault trees. Reliability Engineering & System Safety 186, 220–231 (2019).

  19. Villén-Altamirano, J.: RESTART vs Splitting: A comparative study. Performance Evaluation 121–122, 38–47 (2018).

  20. Villén-Altamirano, J.: Simulation details of the paper “RESTART vs Splitting: a comparative study”. [19]., Appendix A. Supplementary data

  21. Villén-Altamirano, J.: An improved variant of the rare event simulation method RESTART using prolonged retrials. Operations Research Perspectives 6, 100–108 (2019).

  22. Villén-Altamirano, M., Villén-Altamirano, J.: RESTART: a method for accelerating rare event simulations. In: Queueing, Performance and Control in ATM (ITC-13). pp. 71–76. Elsevier (1991)

    Google Scholar 

Download references

Author information

Authors and Affiliations


Corresponding authors

Correspondence to Carlos E. Budde or Arnd Hartmanns .

Editor information

Editors and Affiliations

Rights and permissions

Open Access This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

Reprints and Permissions

Copyright information

© 2021 The Author(s)

About this paper

Verify currency and authenticity via CrossMark

Cite this paper

Budde, C.E., Hartmanns, A. (2021). Replicating \(\textsc {Restart}\) with Prolonged Retrials: An Experimental Report. In: Groote, J.F., Larsen, K.G. (eds) Tools and Algorithms for the Construction and Analysis of Systems. TACAS 2021. Lecture Notes in Computer Science(), vol 12652. Springer, Cham.

Download citation

  • DOI:

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-72012-4

  • Online ISBN: 978-3-030-72013-1

  • eBook Packages: Computer ScienceComputer Science (R0)