Data-Efficient Bayesian Verification of Parametric Markov Chains

  • E. PolgreenEmail author
  • V. B. Wijesuriya
  • S. Haesaert
  • A. Abate
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9826)


Obtaining complete and accurate models for the formal verification of systems is often hard or impossible. We present a data-based verification approach, for properties expressed in a probabilistic logic, that addresses incomplete model knowledge. We obtain experimental data from a system that can be modelled as a parametric Markov chain. We propose a novel verification algorithm to quantify the confidence the underlying system satisfies a given property of interest by using this data. Given a parameterised model of the system, the procedure first generates a feasible set of parameters corresponding to model instances satisfying a given probabilistic property. Simultaneously, we use Bayesian inference to obtain a probability distribution over the model parameter set from data sampled from the underlying system. The results of both steps are combined to compute a confidence the underlying system satisfies the property. The amount of data required is minimised by exploiting partial knowledge of the system. Our approach offers a framework to integrate Bayesian inference and formal verification, and in our experiments our new approach requires one order of magnitude less data than standard statistical model checking to achieve the same confidence.


  1. 1.
    Baier, C., Katoen, J.: Principles of Model Checking. MIT Press, Cambridge (2008)zbMATHGoogle Scholar
  2. 2.
    Bartocci, E., Bortolussi, L., Sanguinetti, G.: Learning temporal logical properties discriminating ECG models of cardiac arrhytmias. CoRR abs/1312.7523 (2013)Google Scholar
  3. 3.
    Bernardo, J., Smith, A.: Bayesian Theory. Wiley, Chichester (1994)CrossRefzbMATHGoogle Scholar
  4. 4.
    Bortolussi, L., Sanguinetti, G.: Learning and designing stochastic processes from logical constraints. In: Joshi, K., Siegle, M., Stoelinga, M., D’Argenio, P.R. (eds.) QEST 2013. LNCS, vol. 8054, pp. 89–105. Springer, Heidelberg (2013)CrossRefGoogle Scholar
  5. 5.
    Brim, L., Češka, M., Dražan, S., Šafránek, D.: Exploring parameter space of stochastic biochemical systems using quantitative model checking. In: Sharygina, N., Veith, H. (eds.) CAV 2013. LNCS, vol. 8044, pp. 107–123. Springer, Heidelberg (2013)CrossRefGoogle Scholar
  6. 6.
    Chen, Y., Nielsen, T.D.: Active learning of Markov decision processes for system verification. In: ICMLA, pp. 289–294. IEEE (2012)Google Scholar
  7. 7.
    Daws, C.: Symbolic and parametric model checking of discrete-time Markov chains. In: Liu, Z., Araki, K. (eds.) ICTAC 2004. LNCS, vol. 3407, pp. 280–294. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  8. 8.
    Dehnert, C., Junges, S., Jansen, N., Corzilius, F., Volk, M., Bruintjes, H., Katoen, J.-P., Ábrahám, E.: PROPhESY: a PRObabilistic ParamEter SYnthesis tool. In: Kroening, D., Păsăreanu, C.S. (eds.) CAV 2015. LNCS, vol. 9206, pp. 214–231. Springer, Heidelberg (2015)CrossRefGoogle Scholar
  9. 9.
    Haesaert, S., Van den Hof, P.M.J., Abate, A.: Data-driven property verification of grey-box systems by Bayesian experiment design. In: ACC, pp. 1800–1805. IEEE (2015)Google Scholar
  10. 10.
    Hahn, E.M., Hermanns, H., Wachter, B., Zhang, L.: PARAM: a model checker for parametric markov models. In: Touili, T., Cook, B., Jackson, P. (eds.) CAV 2010. LNCS, vol. 6174, pp. 660–664. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  11. 11.
    Hahn, E.M., Hermanns, H., Zhang, L.: Probabilistic reachability for parametric Markov models. In: Păsăreanu, C.S. (ed.) Model Checking Software. LNCS, vol. 5578, pp. 88–106. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  12. 12.
    Henriques, D., Martins, J., Zuliani, P., Platzer, A., Clarke, E.M.: Statistical model checking for Markov decision processes. In: QEST, pp. 84–93. IEEE (2012)Google Scholar
  13. 13.
    Lanotte, R., Maggiolo-Schettini, A., Troina, A.: Parametric probabilistic transition systems for system design and analysis. Formal Asp. Comput. 19(1), 93–109 (2007)CrossRefzbMATHGoogle Scholar
  14. 14.
    Legay, A., Delahaye, B., Bensalem, S.: Statistical model checking: an overview. In: Barringer, H., Falcone, Y., Finkbeiner, B., Havelund, K., Lee, I., Pace, G., Roşu, G., Sokolsky, O., Tillmann, N. (eds.) RV 2010. LNCS, vol. 6418, pp. 122–135. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  15. 15.
    Mao, H., Jaeger, M.: Learning and model-checking networks of I/O automata. In: ACML. JMLR, vol. 25, pp. 285–300. (2012)Google Scholar
  16. 16.
    Eichelsbacher, P., Ganesh, A.: Bayesian inference for Markov chains. J. Appl. Probab. 39(1), 91–99 (2002)MathSciNetCrossRefzbMATHGoogle Scholar
  17. 17.
    Poupart, P., Vlassis, N.A., Hoey, J., Regan, K.: An analytic solution to discrete Bayesian reinforcement learning. In: ICML. ACM, vol. 148, pp. 697–704. ACM (2006)Google Scholar
  18. 18.
    Su, G., Rosenblum, D.S.: Nested reachability approximation for discrete-time Markov chains with univariate parameters. In: Cassez, F., Raskin, J.-F. (eds.) ATVA 2014. LNCS, vol. 8837, pp. 364–379. Springer, Heidelberg (2014)Google Scholar
  19. 19.
    Younes, H.L.S.: Probabilistic verification for “black-box” systems. In: Etessami, K., Rajamani, S.K. (eds.) CAV 2005. LNCS, vol. 3576, pp. 253–265. Springer, Heidelberg (2005)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  • E. Polgreen
    • 1
    Email author
  • V. B. Wijesuriya
    • 1
  • S. Haesaert
    • 2
  • A. Abate
    • 1
  1. 1.Department of Computer ScienceUniversity of OxfordOxfordUK
  2. 2.Department of Electrical EngineeringEindhoven University of TechnologyEindhovenThe Netherlands

Personalised recommendations