Verification and Control of Partially Observable Probabilistic Real-Time Systems

  • Gethin NormanEmail author
  • David Parker
  • Xueyi Zou
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9268)


We propose automated techniques for the verification and control of probabilistic real-time systems that are only partially observable. To formally model such systems, we define an extension of probabilistic timed automata in which local states are partially visible to an observer or controller. We give a probabilistic temporal logic that can express a range of quantitative properties of these models, relating to the probability of an event’s occurrence or the expected value of a reward measure. We then propose techniques to either verify that such a property holds or to synthesise a controller for the model which makes it true. Our approach is based on an integer discretisation of the model’s dense-time behaviour and a grid-based abstraction of the uncountable belief space induced by partial observability. The latter is necessarily approximate since the underlying problem is undecidable, however we show how both lower and upper bounds on numerical results can be generated. We illustrate the effectiveness of the approach by implementing it in the PRISM model checker and applying it to several case studies, from the domains of computer security and task scheduling.


Temporal Logic Markov Decision Process Strategy Synthesis Covert Channel Expected Reward 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    de Alfaro, L.: The verification of probabilistic systems under memoryless partial-information policies is hard. In: Proc. PROBMIV 1999, pp. 19–32 (1999)Google Scholar
  2. 2.
    Alur, R., Dill, D.: A theory of timed automata. Theoretical Computer Science 126, 183–235 (1994)CrossRefMathSciNetzbMATHGoogle Scholar
  3. 3.
    Baier, C., Bertrand, N., Größer, M.: On decision problems for probabilistic Büchi automata. In: Amadio, R.M. (ed.) FOSSACS 2008. LNCS, vol. 4962, pp. 287–301. Springer, Heidelberg (2008) CrossRefGoogle Scholar
  4. 4.
    Baier, C., Größer, M., Leucker, M., Bollig, B., Ciesinski, F.: Controller synthesis for probabilistic systems (extended abstract). In: Levy, J.-J., Mayr, E.W., Mayr, J.C. (eds.) TCS 2004. IFIP, vol. 155, pp. 493–506. Springer, Heidelberg (2004) CrossRefGoogle Scholar
  5. 5.
    Behrmann, G., Fehnker, A., Hune, T., Larsen, K., Pettersson, P., Romijn, J., Vaandrager, F.: Minimum-cost reachability for priced time automata. In: Di Benedetto, M.D., Sangiovanni-Vincentelli, A. (eds.) Hybrid Systems: Computation and Control. LNCS, vol. 2034, pp. 147–161. Springer, Heidelberg (2001) CrossRefGoogle Scholar
  6. 6.
    Bouyer, P., D’Souza, D., Madhusudan, P., Petit, A.: Timed Control with partial observability. In: Hunt Jr., W.A., Somenzi, F. (eds.) CAV 2003. LNCS, vol. 2725, pp. 180–192. Springer, Heidelberg (2003) CrossRefGoogle Scholar
  7. 7.
    Bouyer, P., Fahrenberg, U., Larsen, K., Markey, N.: Quantitative analysis of real-time systems using priced timed automata. Comm. of the ACM 54(9), 78–87 (2011)CrossRefGoogle Scholar
  8. 8.
    Cassandra, A.: A survey of POMDP applications. Presented at the AAAI Fall Symposium, 1998. (1998)
  9. 9.
    Cassez, F., David, A., Larsen, K.G., Lime, D., Raskin, J.-F.: Timed Control with observation based and stuttering invariant strategies. In: Namjoshi, K.S., Yoneda, T., Higashino, T., Okamura, Y. (eds.) ATVA 2007. LNCS, vol. 4762, pp. 192–206. Springer, Heidelberg (2007) CrossRefGoogle Scholar
  10. 10.
    Černý, P., Chatterjee, K., Henzinger, T.A., Radhakrishna, A., Singh, R.: Quantitative synthesis for concurrent programs. In: Gopalakrishnan, G., Qadeer, S. (eds.) CAV 2011. LNCS, vol. 6806, pp. 243–259. Springer, Heidelberg (2011) CrossRefGoogle Scholar
  11. 11.
    Chatterjee, K., Chmelik, M., Tracol, M.: What is decidable about partially observable Markov decision processes with omega-regular objectives. In: CSL 2013. LIPIcs, vol. 23, pp. 165–180. Schloss Dagstuhl-Leibniz-Zentrum fuer Informatik (2013)Google Scholar
  12. 12.
    Chatterjee, K., Doyen, L.: Partial-observation stochastic games: How to win when belief fails. ACM Transactions on Computational Logic 15(2) (2014)Google Scholar
  13. 13.
    Eaves, B.: A course in triangulations for solving equations with deformations. Springer (1984)Google Scholar
  14. 14.
    Finkbeiner, B., Peter, H.-J.: Template-based controller synthesis for timed systems. In: Flanagan, C., König, B. (eds.) TACAS 2012. LNCS, vol. 7214, pp. 392–406. Springer, Heidelberg (2012) CrossRefGoogle Scholar
  15. 15.
    Giro, S., Rabe, M.N.: Verification of partial-information probabilistic systems using counterexample-guided refinements. In: Chakraborty, S., Mukund, M. (eds.) ATVA 2012. LNCS, vol. 7561, pp. 333–348. Springer, Heidelberg (2012) CrossRefGoogle Scholar
  16. 16.
    Henzinger, T.A., Manna, Z., Pnueli, A.: What good are digital clocks? In: Kuich, W. (ed.) ICALP 1992. LNCS, vol. 623, pp. 545–558. Springer, Heidelberg (1992) CrossRefGoogle Scholar
  17. 17.
    Kang, M., Moore, A., Moskowitz, I.: Design and assurance strategy for the NRL pump. Computer 31(4), 56–64 (1998)CrossRefGoogle Scholar
  18. 18.
    Kemeny, J., Snell, J., Knapp, A.: Denumerable Markov Chains (1976)Google Scholar
  19. 19.
    Kwiatkowska, M., Norman, G., Parker, D.: PRISM 4.0: verification of probabilistic real-time systems. In: Gopalakrishnan, G., Qadeer, S. (eds.) CAV 2011. LNCS, vol. 6806, pp. 585–591. Springer, Heidelberg (2011) CrossRefGoogle Scholar
  20. 20.
    Kwiatkowska, M., Norman, G., Parker, D., Sproston, J.: Performance analysis of probabilistic timed automata using digital clocks. FMSD 29, 33–78 (2006)zbMATHGoogle Scholar
  21. 21.
    Lanotte, R., Maggiolo-Schettini, A., Tini, S., Troina, A., Tronci, E.: Automatic analysis of the NRL pump. In: ENTCS, vol. 99, pp. 245–266 (2004)Google Scholar
  22. 22.
    Lanotte, R., Maggiolo-Schettini, A., Troina, A.: Automatic analysis of a non-repudiation protocol. In: Proc. QAPL 2004. ENTCS, vol. 112, pp. 113–129 (2005)Google Scholar
  23. 23.
    Lovejoy, W.: Computationally feasible bounds for partially observed Markov decision processes. Operations Research 39(1), 162–175 (1991)CrossRefMathSciNetzbMATHGoogle Scholar
  24. 24.
    Madani, O., Hanks, S., Condon, A.: On the undecidability of probabilistic planning and related stochastic optimization problems. Artif. Intell. 147(1–2), 5–34 (2003)CrossRefMathSciNetzbMATHGoogle Scholar
  25. 25.
    Markowitch, O., Roggeman, Y.: Probabilistic non-repudiation without trusted third party. In: Proc. Workshop on Security in Communication Networks (1999)Google Scholar
  26. 26.
    Norman, G., Parker, D., Zou, X.: Verification and control of partially observable probabilistic real-time systems (2015).
  27. 27.
    Norman, G., Parker, D., Sproston, J.: Model checking for probabilistic timed automata. FMSD 43(2), 164–190 (2013)zbMATHGoogle Scholar
  28. 28.
    Shani, G., Pineau, J., Kaplow, R.: A survey of point-based POMDP solvers. Autonomous Agents and Multi-Agent Systems 27(1), 1–51 (2013)CrossRefGoogle Scholar
  29. 29.
    Yu, H.: Approximate Solution Methods for Partially Observable Markov and Semi-Markov Decision Processes. Ph.D. thesis, MIT (2006)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  1. 1.School of Computing ScienceUniversity of GlasgowGlasgowUK
  2. 2.School of Computer ScienceUniversity of BirminghamBirminghamUK
  3. 3.Department of Computer ScienceUniversity of YorkYorkUK

Personalised recommendations