Chapter

Tools and Algorithms for the Construction and Analysis of Systems

Volume 2988 of the series Lecture Notes in Computer Science pp 46-60

Numerical vs. Statistical Probabilistic Model Checking: An Empirical Study

  • Håkan L. S. YounesAffiliated withComputer Science Department, Carnegie Mellon University
  • , Marta KwiatkowskaAffiliated withSchool of Computer Science, University of Birmingham
  • , Gethin NormanAffiliated withSchool of Computer Science, University of Birmingham
  • , David ParkerAffiliated withSchool of Computer Science, University of Birmingham

Abstract

Numerical analysis based on uniformisation and statistical techniques based on sampling and simulation are two distinct approaches for transient analysis of stochastic systems. We compare the two solution techniques when applied to the verification of time-bounded until formulae in the temporal stochastic logic CSL. This study differs from most previous comparisons of numerical and statistical approaches in that CSL model checking is a hypothesis testing problem rather than a parameter estimation problem. We can therefore rely on highly efficient sequential acceptance sampling tests, which enables statistical solution techniques to quickly return a result with some uncertainty. This suggests that statistical techniques can be useful as a first resort during system prototyping, rather than as a last resort as often suggested. We also propose a novel combination of the two solution techniques for verifying CSL queries with nested probabilistic operators.