Statistical Model Checking for Safety Critical Hybrid Systems: An Empirical Evaluation

  • Youngjoo Kim
  • Moonzoo Kim
  • Tai-Hyo Kim
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7857)


As more computing systems are utilized in various areas of our society, the reliability of computing systems becomes a significant issue. However, as the complexity of computing systems increases, conventional verification and validation techniques such as testing and model checking have limitations to assess reliability of complex safety critical systems. Such systems often control highly complex continuous dynamics to interact with physical environments. To assure the reliability of safety critical hybrid systems, statistical model checking (SMC) techniques have been proposed. SMC techniques approximately compute probabilities for a target system to satisfy given requirements based on randomly sampled execution traces. In this paper, we empirically evaluated four state-ofthe- art SMC techniques on a fault-tolerant fuel control system in the automobile domain. Through the experiments, we could demonstrate that SMC is practically useful to assure the reliability of a safety critical hybrid system and we compared pros and cons of the four different SMC techniques.


Model Check Sample Path Linear Temporal Logic Precision Parameter Sequential Probability Ratio Test 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Clarke, E., Biere, A., Raimi, R., Zhu, Y.: Bounded model checking using satisfiability solving. Formal Methods System Design (FMSD) 19(1), 7–34 (2001)zbMATHCrossRefGoogle Scholar
  2. 2.
    Clarke, E., Donzé, A., Legay, A.: Statistical model checking of mixed-analog circuits with an application to a third order Δ − Σ modulator. In: Chockler, H., Hu, A.J. (eds.) HVC 2008. LNCS, vol. 5394, pp. 149–163. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  3. 3.
    Halstead, M.H.: Elements of Software Science. Elsevier Science Ltd. (1977)Google Scholar
  4. 4.
    Hérault, T., Lassaigne, R., Magniette, F., Peyronnet, S.: Approximate probabilistic model checking. In: Steffen, B., Levi, G. (eds.) VMCAI 2004. LNCS, vol. 2937, pp. 73–84. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  5. 5.
    IEEE Computer Society. IEEE Std 1633: IEEE Recommend Practice on Software Reliability (2008)Google Scholar
  6. 6.
    International Electrotechnical Commission (IEC). IEC 61508: Functional safety of electrical/electronic/programmable electronic (E/E/PE) safety related systems (2005)Google Scholar
  7. 7.
    International Organization for Standardization (ISO). ISO 26262: Road vehicles – functional safety (2011),
  8. 8.
    Jha, S.K., Clarke, E.M., Langmead, C.J., Legay, A., Platzer, A., Zuliani, P.: A bayesian approach to model checking biological systems. In: Degano, P., Gorrieri, R. (eds.) CMSB 2009. LNCS, vol. 5688, pp. 218–234. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  9. 9.
    Kim, Y., Choi, O., Kim, M., Baik, J., Kim, T.: Validating software reliability through statistical model checking: Safer, cheaper, and faster. IEEE Software (under review)Google Scholar
  10. 10.
    Kim, Y., Kim, M., Kim, T.: Hybrid statistical model checking technique for reliable safety critical systems. In: IEEE International Symposium on Software Reliability Engineering, ISSRE (2012)Google Scholar
  11. 11.
    Kwiatkowska, M., Norman, G., Parker, D.: PRISM 4.0: Verification of probabilistic real-time systems. In: Gopalakrishnan, G., Qadeer, S. (eds.) CAV 2011. LNCS, vol. 6806, pp. 585–591. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  12. 12.
    Lauber, J., Guerra, T.M., Dambrine, M.: Air-fuel ratio control in a gasoline engine. International Journal of Systems Science (IJSySc) 42(2), 277–286 (2011)MathSciNetzbMATHCrossRefGoogle Scholar
  13. 13.
    Radio Technical Commission for Aeronautics (RTCA). Do-178c: Software considerations in airborne systems and equipment certification (2012)Google Scholar
  14. 14.
    Sen, P.K., Singer, J.M.: Large sample methods in statistics: An Introduction with Applications. Chapman & Hall, New York (1993)zbMATHGoogle Scholar
  15. 15.
    Wald, A.: Sequential tests of statistical hypotheses. Annals of Mathematical Statistics 16(2), 117–186 (1945)MathSciNetzbMATHCrossRefGoogle Scholar
  16. 16.
    Yi, S., Heo, J., Cho, Y., Hong, J.: Adaptive mobile checkpointing facility for wireless sensor networks. In: Gavrilova, M.L., Gervasi, O., Kumar, V., Tan, C.J.K., Taniar, D., Laganá, A., Mun, Y., Choo, H. (eds.) ICCSA 2006. LNCS, vol. 3981, pp. 701–709. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  17. 17.
    Younes, H.L.S.: Verification and Planning for Stochastic Processes with Asynchronous Events. PhD thesis, CMU (January 2005)Google Scholar
  18. 18.
    Younes, H.L.S., Kwiatkowska, M., Norman, G., Parker, D.: Numerical vs. statistical probabilistic model checking. Software Tools for Technology Transfer (STTT) 8(3), 216–228 (2006)CrossRefGoogle Scholar
  19. 19.
    Younes, H.L.S., Musliner, D.J.: Probabilistic plan verification through acceptance sampling. In: AIPS Workshop on Planning via Model Checking (2002)Google Scholar
  20. 20.
    Younes, H.L.S., Simmons, R.G.: Statistical probabilistic model checking with a focus on time-bounded properties. Journal Information and Computation (JIC) 204(9), 1368–1409 (2006)MathSciNetzbMATHCrossRefGoogle Scholar
  21. 21.
    Zuliani, P., Platzer, A., Clarke, E.M.: Bayesian statistical model checking with application to stateflow/simulink verification. In: Hybrid Systems: Computation and Control, HSCC (2010)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  • Youngjoo Kim
    • 1
  • Moonzoo Kim
    • 1
  • Tai-Hyo Kim
    • 2
  1. 1.CS Dept.KAISTDaejeonSouth Korea
  2. 2.Formal Works Inc.SeoulSouth Korea

Personalised recommendations