Skip to main content
SpringerLink
Log in

Automated Verification and Strategy Synthesis for Probabilistic Systems

  • Conference paper
Automated Technology for Verification and Analysis

Part of the book series: Lecture Notes in Computer Science ((LNPSE,volume 8172))

Abstract

Probabilistic model checking is an automated technique to verify whether a probabilistic system, e.g., a distributed network protocol which can exhibit failures, satisfies a temporal logic property, for example, “the minimum probability of the network recovering from a fault in a given time period is above 0.98”. Dually, we can also synthesise, from a model and a property specification, a strategy for controlling the system in order to satisfy or optimise the property, but this aspect has received less attention to date. In this paper, we give an overview of methods for automated verification and strategy synthesis for probabilistic systems. Primarily, we focus on the model of Markov decision processes and use property specifications based on probabilistic LTL and expected reward objectives. We also describe how to apply multi-objective model checking to investigate trade-offs between several properties, and extensions to stochastic multi-player games. The paper concludes with a summary of future challenges in this area.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Abdeddaim, Y., Kerbaa, A., Maler, O.: Task graph scheduling using timed automata. In: Proc. IPDPS 2003 (2003)

    Google Scholar 

  2. de Alfaro, L.: Formal Verification of Probabilistic Systems. Ph.D. thesis, Stanford University (1997)

    Google Scholar 

  3. Baier, C., Bertrand, N., Größer, M.: On decision problems for probabilistic büchi automata. In: Amadio, R.M. (ed.) FOSSACS 2008. LNCS, vol. 4962, pp. 287–301. Springer, Heidelberg (2008)

    Chapter  Google Scholar 

  4. Baier, C., Größer, M., Leucker, M., Bollig, B., Ciesinski, F.: Controller synthesis for probabilistic systems. In: Proc. TCS 2006, pp. 493–5062. Kluwer (2004)

    Google Scholar 

  5. Baier, C., Katoen, J.P.: Principles of Model Checking. MIT Press (2008)

    Google Scholar 

  6. Bellman, R.: Dynamic Programming. Princeton University Press (1957)

    Google Scholar 

  7. Benini, L., Bogliolo, A., Paleologo, G., De Micheli, G.: Policy optimization for dynamic power management. IEEE Trans. CADICS 8(3), 299–316 (2000)

    Google Scholar 

  8. Bertsekas, D.: Dynamic Programming and Optimal Control, vol. 1&2. Athena Scientific (1995)

    Google Scholar 

  9. Bozzano, M., Cimatti, A., Katoen, J.-P., Nguyen, V.Y., Noll, T., Roveri, M.: The COMPASS approach: Correctness, modelling and performability of aerospace systems. In: Buth, B., Rabe, G., Seyfarth, T. (eds.) SAFECOMP 2009. LNCS, vol. 5775, pp. 173–186. Springer, Heidelberg (2009)

    Chapter  Google Scholar 

  10. Brázdil, T., Brožek, V., Forejt, V., Kučera, A.: Stochastic games with branching-time winning objectives. In: Proc. LICS 2006, pp. 349–358. IEEE CS Press (2006)

    Google Scholar 

  11. Černý, P., Chatterjee, K., Henzinger, T.A., Radhakrishna, A., Singh, R.: Quantitative synthesis for concurrent programs. In: Gopalakrishnan, G., Qadeer, S. (eds.) CAV 2011. LNCS, vol. 6806, pp. 243–259. Springer, Heidelberg (2011)

    Chapter  Google Scholar 

  12. Chatterjee, K., Henzinger, T.A.: Strategy improvement and randomized subexponential algorithms for stochastic parity games. In: Durand, B., Thomas, W. (eds.) STACS 2006. LNCS, vol. 3884, pp. 512–523. Springer, Heidelberg (2006)

    Chapter  Google Scholar 

  13. Chatterjee, K., Henzinger, T.A.: Strategy improvement for stochastic rabin and streett games. In: Baier, C., Hermanns, H. (eds.) CONCUR 2006. LNCS, vol. 4137, pp. 375–389. Springer, Heidelberg (2006)

    Chapter  Google Scholar 

  14. Chen, T., Forejt, V., Kwiatkowska, M., Parker, D., Simaitis, A.: Automatic verification of competitive stochastic systems. Formal Methods in System Design 43(1), 61–92 (2013)

    Article  Google Scholar 

  15. Chen, T., Forejt, V., Kwiatkowska, M., Parker, D., Simaitis, A.: PRISM-games: A model checker for stochastic multi-player games. In: Piterman, N., Smolka, S.A. (eds.) TACAS 2013. LNCS, vol. 7795, pp. 185–191. Springer, Heidelberg (2013)

    Chapter  Google Scholar 

  16. Condon, A.: The complexity of stochastic games. Information and Computation 96(2), 203–224 (1992)

    Article  MathSciNet  Google Scholar 

  17. Condon, A.: On algorithms for simple stochastic games. DIMACS Series in Discrete Mathematics and Theoretical Computer Science 13, 51–73 (1993)

    Article  MathSciNet  Google Scholar 

  18. Daniele, M., Giunchiglia, F., Vardi, M.Y.: Improved automata generation for linear temporal logic. In: Halbwachs, N., Peled, D.A. (eds.) CAV 1999. LNCS, vol. 1633, pp. 249–260. Springer, Heidelberg (1999)

    Chapter  Google Scholar 

  19. Duflot, M., Kwiatkowska, M., Norman, G., Parker, D.: A formal analysis of Bluetooth device discovery. STTT 8(6), 621–632 (2006)

    Article  Google Scholar 

  20. Etessami, K., Kwiatkowska, M., Vardi, M., Yannakakis, M.: Multi-objective model checking of Markov decision processes. LMCS 4(4), 1–21 (2008)

    MathSciNet  MATH  Google Scholar 

  21. Forejt, V., Kwiatkowska, M., Norman, G., Parker, D.: Automated verification techniques for probabilistic systems. In: Bernardo, M., Issarny, V. (eds.) SFM 2011. LNCS, vol. 6659, pp. 53–113. Springer, Heidelberg (2011)

    Chapter  Google Scholar 

  22. Forejt, V., Kwiatkowska, M., Norman, G., Parker, D., Qu, H.: Quantitative multi-objective verification for probabilistic systems. In: Abdulla, P.A., Leino, K.R.M. (eds.) TACAS 2011. LNCS, vol. 6605, pp. 112–127. Springer, Heidelberg (2011)

    Chapter  Google Scholar 

  23. Forejt, V., Kwiatkowska, M., Parker, D.: Pareto curves for probabilistic model checking. In: Chakraborty, S., Mukund, M. (eds.) ATVA 2012. LNCS, vol. 7561, pp. 317–332. Springer, Heidelberg (2012)

    Chapter  Google Scholar 

  24. Giro, S., Rabe, M.N.: Verification of partial-information probabilistic systems using counterexample-guided refinements. In: Chakraborty, S., Mukund, M. (eds.) ATVA 2012. LNCS, vol. 7561, pp. 333–348. Springer, Heidelberg (2012)

    Chapter  Google Scholar 

  25. Hermanns, H. (ed.): Interactive Markov Chains and the Quest for Quantified Quality. LNCS, vol. 2428. Springer, Heidelberg (2002)

    MATH  Google Scholar 

  26. Howard, R.: Dynamic Programming and Markov Processes. The MIT Press (1960)

    Google Scholar 

  27. Katoen, J.P., Hahn, E., Hermanns, H., Jansen, D., Zapreev, I.: The ins and outs of the probabilistic model checker MRMC. In: Proc. QEST 2009. IEEE CS Press (2009)

    Google Scholar 

  28. Kemeny, J., Snell, J., Knapp, A.: Denumerable Markov Chains, 2nd edn. Springer (1976)

    Chapter  Google Scholar 

  29. Kwiatkowska, M., Norman, G., Parker, D.: PRISM 4.0: Verification of probabilistic real-time systems. In: Gopalakrishnan, G., Qadeer, S. (eds.) CAV 2011. LNCS, vol. 6806, pp. 585–591. Springer, Heidelberg (2011)

    Chapter  Google Scholar 

  30. Kwiatkowska, M., Parker, D.: Automated verification and strategy synthesis for probabilistic systems (extended version) (2013), available from [49]

    Chapter  Google Scholar 

  31. Lahijanian, M., Wasniewski, J., Andersson, S., Belta, C.: Motion planning and control from temporal logic specifications with probabilistic satisfaction guarantees. In: Proc. ICRA 2010, pp. 3227–3232 (2010)

    Google Scholar 

  32. Lakin, M., Parker, D., Cardelli, L., Kwiatkowska, M., Phillips, A.: Design and analysis of DNA strand displacement devices using probabilistic model checking. Journal of the Royal Society Interface 9(72), 1470–1485 (2012)

    Article  Google Scholar 

  33. Larsen, K., Pettersson, P., Yi, W.: UPPAAL in a nutshell. International Journal on Software Tools for Technology Transfer 1(1-2), 134–152 (1997)

    Article  Google Scholar 

  34. Masuam, Kolobov, A.: Planning with Markov Decision Processes: An AI Perspective. Morgan & Claypool (2012)

    Google Scholar 

  35. Norman, G., Parker, D., Sproston, J.: Model checking for probabilistic timed automata. Formal Methods in System Design (2012) (to appear)

    Google Scholar 

  36. Poupart, P.: Exploiting Structure to Efficiently Solve Large Scale Partially Observable Markov Decision Processes. Ph.D. thesis, University of Toronto (2005)

    Google Scholar 

  37. Puggelli, A., Li, W., Sangiovanni-Vincentelli, A.L., Seshia, S.A.: Polynomial-time verification of PCTL properties of MDPs with convex uncertainties. In: Sharygina, N., Veith, H. (eds.) CAV 2013. LNCS, vol. 8044, pp. 527–542. Springer, Heidelberg (2013)

    Chapter  Google Scholar 

  38. Puterman, M.: Markov Decision Processes: Discrete Stochastic Dynamic Programming. John Wiley and Sons (1994)

    Google Scholar 

  39. Satia, J., Lave Jr., R.: Markovian decision processes with uncertain transition probabilities. Oper. Res. 21, 728–740 (1970)

    Article  MathSciNet  Google Scholar 

  40. Sen, K., Viswanathan, M., Agha, G.: Model-checking Markov chains in the presence of uncertainties. In: Hermanns, H., Palsberg, J. (eds.) TACAS 2006. LNCS, vol. 3920, pp. 394–410. Springer, Heidelberg (2006)

    Chapter  Google Scholar 

  41. Steel, G.: Formal analysis of PIN block attacks. Theoretical Computer Science 367(1-2), 257–270 (2006)

    Article  MathSciNet  Google Scholar 

  42. Sutton, R., Barto, A.: Reinforcement Learning: An Introduction. MIT Press (1998)

    Google Scholar 

  43. Timmer, M., Katoen, J.-P., van de Pol, J., Stoelinga, M.I.A.: Efficient modelling and generation of markov automata. In: Koutny, M., Ulidowski, I. (eds.) CONCUR 2012. LNCS, vol. 7454, pp. 364–379. Springer, Heidelberg (2012)

    Chapter  Google Scholar 

  44. Tkachev, I., Abate, A.: Formula-free finite abstractions for linear temporal verification of stochastic hybrid systems. In: Proc. HSCC 2013, pp. 283–292 (2013)

    Google Scholar 

  45. Vardi, M., Wolper, P.: Reasoning about infinite computations. Information and Computation 115(1), 1–37 (1994)

    Article  MathSciNet  Google Scholar 

  46. Wolff, E., Topcu, U., Murray, R.: Robust control of uncertain Markov decision processes with temporal logic specifications. In: Proc. CDC 2012, pp. 3372–3379 (2012)

    Google Scholar 

  47. Wongpiromsarn, T., Topcu, U., Murray, R.: Receding horizon temporal logic planning. IEEE Trans. Automat. Contr. 57(11), 2817–2830 (2012)

    Article  MathSciNet  Google Scholar 

  48. Zhang, L., She, Z., Ratschan, S., Hermanns, H., Hahn, E.M.: Safety verification for probabilistic hybrid systems. Eur. J. Control 18(6), 572–587 (2012)

    Article  MathSciNet  Google Scholar 

  49. http://www.prismmodelchecker.org/files/stratsynth/

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science, University of Oxford, UK

    Marta Kwiatkowska

  2. School of Computer Science, University of Birmingham, UK

    David Parker

Authors
  1. Marta Kwiatkowska
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. David Parker
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. University of Engineering and Technology, Vietnam National University, Xuan Thuy Street, 144, Hanoi, Vietnam

    Dang Van Hung

  2. School of Information Science, Japan Advanced Institute of Science and Technology, 1-1 Asahidai, 923-1292, Nomi, Ishikawa, Japan

    Mizuhito Ogawa

Rights and permissions

Reprints and permissions

Copyright information

© 2013 Springer International Publishing Switzerland

About this paper

Cite this paper

Kwiatkowska, M., Parker, D. (2013). Automated Verification and Strategy Synthesis for Probabilistic Systems. In: Van Hung, D., Ogawa, M. (eds) Automated Technology for Verification and Analysis. Lecture Notes in Computer Science, vol 8172. Springer, Cham. https://doi.org/10.1007/978-3-319-02444-8_2

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-02444-8_2

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-02443-1

  • Online ISBN: 978-3-319-02444-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation