Safety Verification of Continuous-Space Pure Jump Markov Processes

  • Sadegh Esmaeil Zadeh SoudjaniEmail author
  • Rupak Majumdar
  • Alessandro Abate
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9636)


We study the probabilistic safety verification problem for pure jump Markov processes, a class of models that generalizes continuous-time Markov chains over continuous (uncountable) state spaces. Solutions of these processes are piecewise constant, right-continuous functions from time to states. Their jump (or reset) times are realizations of a Poisson process, characterized by a jump rate function that can be both time- and state-dependent. Upon jumping in time, the new state of the solution process is specified according to a (continuous) stochastic conditional kernel. After providing a full characterization of safety properties of these processes, we describe a formal method to abstract the process as a finite-state discrete-time Markov chain; this approach is formal in that it provides a-priori error bounds on the precision of the abstraction, based on the continuity properties of the stochastic kernel of the process and of its jump rate function. We illustrate the approach on a case study of thermostatically controlled loads.


Transition Kernel Jump Time Continuous State Space Gaussian Density Function Demand Response Program 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Abate, A., Katoen, J.-P., Lygeros, J., Prandini, M.: Approximate model checking of stochastic hybrid systems. Eur. J. Control 6, 624–641 (2010)MathSciNetCrossRefzbMATHGoogle Scholar
  2. 2.
    Abate, A., Prandini, M., Lygeros, J., Sastry, S.: Probabilistic reachability and safety for controlled discrete time stochastic hybrid systems. Automatica 44(11), 2724–2734 (2008)MathSciNetCrossRefzbMATHGoogle Scholar
  3. 3.
    Aït-Sahalia, Y., Mykland, P.A.: The effects of random and discrete sampling when estimating continuous-time diffusions. Econometrica 71(2), 483–549 (2003)MathSciNetCrossRefzbMATHGoogle Scholar
  4. 4.
    Allen, L.J.S.: An Introduction to Stochastic Processes with Applications to Biology. Pearson/Prentice Hall, Englewood Cliffs (2003)zbMATHGoogle Scholar
  5. 5.
    Aziz, A., Sanwal, K., Singhal, V., Brayton, R.: Verifying continuous time Markov chains. In: Alur, R., Henzinger, T.A. (eds.) CAV 1996. LNCS, vol. 1102, pp. 269–276. Springer, Heidelberg (1996)CrossRefGoogle Scholar
  6. 6.
    Baier, C., Haverkort, B., Hermanns, H., Katoen, J.-P.: Model checking continuous-time Markov chains by transient analysis. In: Allen Emerson, E., Prasad Sistla, A. (eds.) CAV 2000. LNCS, vol. 1855, pp. 358–372. Springer, Heidelberg (2000)CrossRefGoogle Scholar
  7. 7.
    Baier, C., Haverkort, B., Hermanns, H., Katoen, J.-P.: Model-checking algorithms for continuous-time Markov chains. IEEE Trans. Softw. Eng. 29(6), 524–541 (2003)CrossRefzbMATHGoogle Scholar
  8. 8.
    Baier, C., Katoen, J.-P.: Principles of Model Checking. MIT Press, Cambridge (2008)zbMATHGoogle Scholar
  9. 9.
    Baier, C., Katoen, J.-P., Hermanns, H.: Approximate symbolic model checking of continuous-time Markov chains (extended abstract). In: Baeten, J.C.M., Mauw, S. (eds.) CONCUR 1999. LNCS, vol. 1664, pp. 146–161. Springer, Heidelberg (1999)CrossRefGoogle Scholar
  10. 10.
    Bertsekas, D.P., Shreve, S.E.: Stchastic Optimal Control: the Discrete-Time Case. Athena Scientific, Belmont (1996)Google Scholar
  11. 11.
    Billingsley, P.: Convergence of Probability Measures. Wiley, New York (1999)CrossRefzbMATHGoogle Scholar
  12. 12.
    Callaway, D.S.: Tapping the energy storage potential in electric loads to deliver load following and regulation, with application to wind energy. Ener. Convers. Manag. 50(5), 1389–1400 (2009)CrossRefGoogle Scholar
  13. 13.
    Chen, T., Diciolla, M., Kwiatkowska, M., Mereacre, A.: Time-bounded verification of CTMCs against real-time specifications. In: Fahrenberg, U., Tripakis, S. (eds.) FORMATS 2011. LNCS, vol. 6919, pp. 26–42. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  14. 14.
    Chen, T., Han, T., Katoen, J.-P., Mereacre, A.: Model checking of continuous-time Markov chains against timed automata specifications. Logical Methods Comput. Sci. 7(1), 1–34 (2011)MathSciNetCrossRefzbMATHGoogle Scholar
  15. 15.
    Davis, M.H.A.: Markov Models and Optimization. Chapman & Hall/CRC Press, London (1993)CrossRefzbMATHGoogle Scholar
  16. 16.
    Esmaeil Zadeh Soudjani, S., Abate, A.: Adaptive gridding for abstraction and verification of stochastic hybrid systems. In: QEST, pp. 59–69 (2011)Google Scholar
  17. 17.
    Esmaeil Zadeh Soudjani, S., Abate, A.: Higher-order approximations for verification of stochastic hybrid systems. In: Chakraborty, S., Mukund, M. (eds.) ATVA 2012. LNCS, vol. 7561, pp. 416–434. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  18. 18.
    Esmaeil Zadeh Soudjani, S., Abate, A.: Adaptive and sequential gridding procedures for the abstraction and verification of stochastic processes. SIAM J. Appl. Dyn. Syst. 12(2), 921–956 (2013)MathSciNetCrossRefzbMATHGoogle Scholar
  19. 19.
    Esmaeil Zadeh Soudjani, S., Abate, A.: Precise approximations of the probability distribution of a Markov process in time: an application to probabilistic invariance. In: Ábrahám, E., Havelund, K. (eds.) TACAS 2014 (ETAPS). LNCS, vol. 8413, pp. 547–561. Springer, Heidelberg (2014)CrossRefGoogle Scholar
  20. 20.
    Esmaeil Zadeh Soudjani, S., Abate, A.: Aggregation and control of populations of thermostatically controlled loads by formal abstractions. IEEE Trans. Control Syst. Technol. 23(3), 975–990 (2015)MathSciNetCrossRefzbMATHGoogle Scholar
  21. 21.
    Soudjani, S.E.Z., Abate, A.: Quantitative approximation of the probability distribution of a Markov process by formal abstractions. Logical Methods Comput. Sci 11(3), 1–29 (2015)MathSciNetzbMATHGoogle Scholar
  22. 22.
    Esmaeil Zadeh Soudjani, S., Gevaerts, C., Abate, A.: \({\sf FAUST^2}\): formal abstractions of uncountable-state stochastic processes. In: Baier, C., Tineli, C. (eds.) TACAS 2015. LNCS, vol. 9035, pp. 272–286. Springer, Heidelberg (2015)Google Scholar
  23. 23.
    Gihman, I.I., Skorokhod, A.V.: The Theory of Stochastic Processes: II. Die Grundlehren der mathematischen Wissenschaften, vol. 218. Springer, Heidelberg (1975)CrossRefGoogle Scholar
  24. 24.
    Hinton, A., Kwiatkowska, M., Norman, G., Parker, D.: PRISM: a tool for automatic verification of probabilistic systems. In: Hermanns, H., Palsberg, J. (eds.) TACAS 2006. LNCS, vol. 3920, pp. 441–444. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  25. 25.
    Jacod, J., Shiryaev, A.: Limit Theorems for Stochastic Processes. Grundlehren der mathematischen Wissenschaften. Springer, Heidelberg (2010)zbMATHGoogle Scholar
  26. 26.
    Kallenberg, O.: Foundations of Modern Probability. Probability and its Applications. Springer, New York (2002)CrossRefzbMATHGoogle Scholar
  27. 27.
    Katoen, J.-P., Khattri, M., Zapreev, I.S.: A Markov reward model checker. In: QEST, pp. 243–244. IEEE (2005)Google Scholar
  28. 28.
    Mathieu, J.L., Koch, S., Callaway, D.S.: State estimation and control of electric loads to manage real-time energy imbalance. IEEE Trans. Power Syst. 28(1), 430–440 (2013)CrossRefGoogle Scholar
  29. 29.
    Micheli, M., Jordan, M.: Random sampling of a continuous-time stochastic dynamical system. In: Proceedings of the 15th International Symposium on the Mathematical Theory of Networks and Systems (MTNS), pp. 1–15 (2002)Google Scholar
  30. 30.
    Panangaden, P.: Labelled Markov Processes. Imperial College Press, London (2009)CrossRefzbMATHGoogle Scholar
  31. 31.
    Platen, E., Bruti-Liberati, N.: Numerical Solution of Stochastic Differential Equations with Jumps in Finance. Stochastic Modelling and Applied Probability. Springer, Heidelberg (2010)CrossRefzbMATHGoogle Scholar
  32. 32.
    Tkachev, I., Abate, A.: On infinite-horizon probabilistic properties and stochastic bisimulation functions. In: Proceedings of the 50th IEEE Conference on Decision and Control and European Control Conference, pp. 526–531, Orlando, FL, December 2011Google Scholar
  33. 33.
    Tkachev, I., Abate, A.: Characterization and computation of infinite-horizon specifications over Markov processes. Theor. Comput. Sci. 515, 1–18 (2014)MathSciNetCrossRefzbMATHGoogle Scholar
  34. 34.
    Van Kampen, N.G.: Stochastic Processes in Physics and Chemistry. North-Holland Personal Library. Elsevier Science, Amsterdam (2011)zbMATHGoogle Scholar
  35. 35.
    Ziai, Y.: Statistical models of claim amount distributions in general insurance. Ph.D. thesis, School of Engineering and Maths, City University London (1979)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2016

Authors and Affiliations

  • Sadegh Esmaeil Zadeh Soudjani
    • 1
    Email author
  • Rupak Majumdar
    • 2
  • Alessandro Abate
    • 1
  1. 1.Department of Computer ScienceUniversity of OxfordOxfordUK
  2. 2.Max Planck Institute for Software SystemsKaiserslautern and SaarbrückenGermany

Personalised recommendations