Cross-Entropy Optimisation of Importance Sampling Parameters for Statistical Model Checking

  • Cyrille Jegourel
  • Axel Legay
  • Sean Sedwards
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7358)


Statistical model checking avoids the exponential growth of states associated with probabilistic model checking by estimating probabilities from multiple executions of a system and by giving results within confidence bounds. Rare properties are often important but pose a particular challenge for simulation-based approaches, hence a key objective for statistical model checking (SMC) is to reduce the number and length of simulations necessary to produce a result with a given level of confidence. Importance sampling can achieves this, however to maintain the advantages of SMC it is necessary to find good importance sampling distributions without considering the entire state space.

Here we present a simple algorithm that uses the notion of cross-entropy to find an optimal importance sampling distribution. In contrast to previous work, our algorithm uses a naturally defined low dimensional vector of parameters to specify this distribution and thus avoids the intractable explicit representation of a transition matrix. We show that our parametrisation leads to a unique optimum and can produce many orders of magnitude improvement in simulation efficiency. We demonstrate the efficacy of our methodology by applying it to models from reliability engineering and biochemistry.


Model Check Temporal Logic Importance Sampling Linear Temporal Logic Continuous Time Markov Chain 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Barbot, B., Haddad, S., Picaronny, C.: Coupling and Importance Sampling for Statistical Model Checking. In: Flanagan, C., König, B. (eds.) TACAS 2012. LNCS, vol. 7214, pp. 331–346. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  2. 2.
    Basu, A., Bensalem, S., Bozga, M., Caillaud, B., Delahaye, B., Legay, A.: Statistical Abstraction and Model-Checking of Large Heterogeneous Systems. In: Hatcliff, J., Zucca, E. (eds.) FMOODS 2010. LNCS, vol. 6117, pp. 32–46. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  3. 3.
    Bengtsson, J., Larsen, K., Larsson, F., Pettersson, P., Yi, W.: Uppaal — a Tool Suite for Automatic Verification of Real-Time Systems. In: Alur, R., Sontag, E.D., Henzinger, T.A. (eds.) HS 1995. LNCS, vol. 1066, pp. 232–243. Springer, Heidelberg (1996)CrossRefGoogle Scholar
  4. 4.
    Chernoff, H.: A Measure of Asymptotic Efficiency for Tests of a Hypothesis Based on the sum of Observations. Ann. Math. Statist. 23(4), 493–507 (1952)MathSciNetzbMATHCrossRefGoogle Scholar
  5. 5.
    Clarke, E.M., Zuliani, P.: Statistical Model Checking for Cyber-Physical Systems. In: Bultan, T., Hsiung, P.-A. (eds.) ATVA 2011. LNCS, vol. 6996, pp. 1–12. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  6. 6.
    De Boer, P.-T., Nicola, V.F., Rubinstein, R.Y.: Adaptive importance sampling simulation of queueing networks. In: Winter Simulation Conference, vol. 1, pp. 646–655 (2000)Google Scholar
  7. 7.
    Dijkstra, E.W.: Guarded commands, nondeterminacy and formal derivation of programs. Commun. ACM 18, 453–457 (1975)MathSciNetzbMATHCrossRefGoogle Scholar
  8. 8.
    Gillespie, D.T.: Exact stochastic simulation of coupled chemical reactions. Journal of Physical Chemistry 81, 2340–2361 (1977)CrossRefGoogle Scholar
  9. 9.
    Godefroid, P., Levin, M., Molnar, D.: Automated whitebox fuzz testing. In: NDSS (2008)Google Scholar
  10. 10.
    Heidelberger, P.: Fast simulation of rare events in queueing and reliability models. ACM Trans. Model. Comput. Simul. 5, 43–85 (1995)zbMATHCrossRefGoogle Scholar
  11. 11.
    Hérault, T., Lassaigne, R., Magniette, F., Peyronnet, S.: Approximate Probabilistic Model Checking. In: Steffen, B., Levi, G. (eds.) VMCAI 2004. LNCS, vol. 2937, pp. 73–84. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  12. 12.
    Hoeffding, W.: Probability Inequalities for Sums of Bounded Random Variables. Journal of the American Statistical Association 58(301), 13–30 (1963)MathSciNetzbMATHGoogle Scholar
  13. 13.
    Jegourel, C., Legay, A., Sedwards, S.: A Platform for High Performance Statistical Model Checking – PLASMA. In: Flanagan, C., König, B. (eds.) TACAS 2012. LNCS, vol. 7214, pp. 498–503. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  14. 14.
    Jha, S.K., Clarke, E.M., Langmead, C.J., Legay, A., Platzer, A., Zuliani, P.: A Bayesian Approach to Model Checking Biological Systems. In: Degano, P., Gorrieri, R. (eds.) CMSB 2009. LNCS, vol. 5688, pp. 218–234. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  15. 15.
    Kahn, H.: Stochastic (Monte Carlo) Attenuation Analysis. Technical Report P-88, Rand Corporation (July 1949)Google Scholar
  16. 16.
    Kullback, S.: Information Theory and Statistics. Dover (1968)Google Scholar
  17. 17.
    Kwiatkowska, M., Norman, G., Parker, D.: PRISM: Probabilistic Symbolic Model Checker. In: Field, T., Harrison, P.G., Bradley, J., Harder, U. (eds.) TOOLS 2002. LNCS, vol. 2324, pp. 200–204. Springer, Heidelberg (2002)Google Scholar
  18. 18.
    Metropolis, N., Ulam, S.: The Monte Carlo Method. Journal of the American Statistical Association 44(247), 335–341 (1949)MathSciNetzbMATHGoogle Scholar
  19. 19.
    Ridder, A.: Importance sampling simulations of markovian reliability systems using cross-entropy. Annals of Operations Research 134, 119–136 (2005)MathSciNetzbMATHCrossRefGoogle Scholar
  20. 20.
    Ridder, A.: Asymptotic optimality of the cross-entropy method for markov chain problems. Procedia Computer Science 1(1), 1571–1578 (2010)CrossRefGoogle Scholar
  21. 21.
    Rubino, G., Tuffin, B. (eds.): Rare Event Simulation using Monte Carlo Methods. Wiley (2009)Google Scholar
  22. 22.
    Rubinstein, R.: The Cross-Entropy Method for Combinatorial and Continuous Optimization 1, 127–190 (1999)Google Scholar
  23. 23.
    Sen, K., Viswanathan, M., Agha, G.A.: VESTA: A statistical model-checker and analyzer for probabilistic systems. In: QEST, pp. 251–252. IEEE (September 2005)Google Scholar
  24. 24.
    Shahabuddin, P.: Importance Sampling for the Simulation of Highly Reliable Markovian Systems. Management Science 40(3), 333–352 (1994)zbMATHCrossRefGoogle Scholar
  25. 25.
    Shore, J., Johnson, R.: Axiomatic derivation of the principle of maximum entropy and the principle of minimum cross-entropy. IEEE Transactions on Information Theory 26(1), 26–37 (1980)MathSciNetzbMATHCrossRefGoogle Scholar
  26. 26.
  27. 27.
    Younes, H.L.S., Simmons, R.G.: Probabilistic verification of discrete event systems using acceptance sampling. In: Brinksma, E., Larsen, K.G. (eds.) CAV 2002. LNCS, vol. 2404, pp. 223–235. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  28. 28.
    Younes, H.L.S.: Ymer: A Statistical Model Checker. In: Etessami, K., Rajamani, S.K. (eds.) CAV 2005. LNCS, vol. 3576, pp. 429–433. Springer, Heidelberg (2005)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Cyrille Jegourel
    • 1
  • Axel Legay
    • 1
  • Sean Sedwards
    • 1
  1. 1.INRIA Rennes - Bretagne AtlantiqueFrance

Personalised recommendations