Precise Approximations of the Probability Distribution of a Markov Process in Time: An Application to Probabilistic Invariance

  • Sadegh Esmaeil Zadeh Soudjani
  • Alessandro Abate
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8413)


The goal of this work is to formally abstract a Markov process evolving over a general state space as a finite-state Markov chain, with the objective of precisely approximating the state probability distribution of the Markov process in time. The approach uses a partition of the state space and is based on the computation of the average transition probability between partition sets. In the case of unbounded state spaces, a procedure for precisely truncating the state space within a compact set is provided, together with an error bound that depends on the asymptotic properties of the transition kernel of the Markov process. In the case of compact state spaces, the work provides error bounds that depend on the diameters of the partitions, and as such the errors can be tuned. The method is applied to the problem of computing probabilistic invariance of the model under study, and the result is compared to an alternative approach in the literature.


Markov Chain State Space Markov Process Transition Kernel General State Space 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Abate, A., Katoen, J.-P., Lygeros, J., Prandini, M.: Approximate model checking of stochastic hybrid systems. European Journal of Control 6, 624–641 (2010)CrossRefMathSciNetGoogle Scholar
  2. 2.
    Abate, A., Prandini, M., Lygeros, J., Sastry, S.: Probabilistic reachability and safety for controlled discrete time stochastic hybrid systems. Automatica 44(11), 2724–2734 (2008)CrossRefMATHMathSciNetGoogle Scholar
  3. 3.
    Baier, C., Katoen, J.-P., Hermanns, H.: Approximate symbolic model checking of continuous-time Markov chains (Extended abstract). In: Baeten, J.C.M., Mauw, S. (eds.) CONCUR 1999. LNCS, vol. 1664, pp. 146–162. Springer, Heidelberg (1999)CrossRefGoogle Scholar
  4. 4.
    Esmaeil Zadeh Soudjani, S., Abate, A.: Adaptive gridding for abstraction and verification of stochastic hybrid systems. In: Proceedings of the 8th International Conference on Quantitative Evaluation of Systems, Aachen, DE, pp. 59–69 (September 2011)Google Scholar
  5. 5.
    Esmaeil Zadeh Soudjani, S., Abate, A.: Higher-Order Approximations for Verification of Stochastic Hybrid Systems. In: Chakraborty, S., Mukund, M. (eds.) ATVA 2012. LNCS, vol. 7561, pp. 416–434. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  6. 6.
    Esmaeil Zadeh Soudjani, S., Abate, A.: Probabilistic invariance of mixed deterministic-stochastic dynamical systems. In: ACM Proceedings of the 15th International Conference on Hybrid Systems: Computation and Control, Beijing, PRC, pp. 207–216 (April 2012)Google Scholar
  7. 7.
    Esmaeil Zadeh Soudjani, S., Abate, A.: Adaptive and sequential gridding procedures for the abstraction and verification of stochastic processes. SIAM Journal on Applied Dynamical Systems 12(2), 921–956 (2013)CrossRefMATHMathSciNetGoogle Scholar
  8. 8.
    Koutsoukos, X., Riley, D.: Computational methods for reachability analysis of stochastic hybrid systems. In: Hespanha, J.P., Tiwari, A. (eds.) HSCC 2006. LNCS, vol. 3927, pp. 377–391. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  9. 9.
    Kurshan, R.P.: Computer-Aided Verification of Coordinating Processes: The Automata-Theoretic Approach. Princeton Series in Computer Science. Princeton University Press (1994)Google Scholar
  10. 10.
    Kushner, H.J., Dupuis, P.G.: Numerical Methods for Stochastic Control Problems in Continuous Time. Springer, New York (2001)CrossRefMATHGoogle Scholar
  11. 11.
    Kvasnica, M., Grieder, P., Baotić, M.: Multi-parametric toolbox, MPT (2004)Google Scholar
  12. 12.
    Kwiatkowska, M., Norman, G., Segala, R., Sproston, J.: Verifying quantitative properties of continuous probabilistic timed automata. In: Palamidessi, C. (ed.) CONCUR 2000. LNCS, vol. 1877, pp. 123–137. Springer, Heidelberg (2000)CrossRefGoogle Scholar
  13. 13.
    Mitchell, I.M.: Comparing forward and backward reachability as tools for safety analysis. In: Bemporad, A., Bicchi, A., Buttazzo, G. (eds.) HSCC 2007. LNCS, vol. 4416, pp. 428–443. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  14. 14.
    Papoulis, A.: Probability, Random Variables, and Stochastic Processes, 3rd edn. Mcgraw-hill (1991)Google Scholar
  15. 15.
    Prandini, M., Hu, J.: Stochastic reachability: Theory and numerical approximation. In: Cassandras, C.G., Lygeros, J. (eds.) Stochastic Hybrid Systems. Automation and Control Engineering Series, vol. 24, pp. 107–138. Taylor & Francis Group/CRC Press (2006)Google Scholar
  16. 16.
    Tkachev, I., Abate, A.: On infinite-horizon probabilistic properties and stochastic bisimulation functions. In: Proceedings of the 50th IEEE Conference on Decision and Control and European Control Conference, Orlando, FL, pp. 526–531 (December 2011)Google Scholar
  17. 17.
    Tkachev, I., Abate, A.: Characterization and computation of infinite-horizon specifications over markov processes. Theoretical Computer Science 515, 1–18 (2014)CrossRefMATHMathSciNetGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2014

Authors and Affiliations

  • Sadegh Esmaeil Zadeh Soudjani
    • 1
  • Alessandro Abate
    • 2
    • 1
  1. 1.Delft Center for Systems & ControlTU DelftThe Netherlands
  2. 2.Department of Computer ScienceUniversity of OxfordUK

Personalised recommendations