Computing Continuous-Time Markov Chains as Transformers of Unbounded Observables

  • Vincent Danos
  • Tobias Heindel
  • Ilias Garnier
  • Jakob Grue Simonsen
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10203)


The paper studies continuous-time Markov chains (CTMCs) as transformers of real-valued functions on their state space, considered as generalised predicates and called observables. Markov chains are assumed to take values in a countable state space \(\mathbf {S}\); observables \(f: \mathbf {S} \rightarrow {\mathbb {R}}\) may be unbounded. The interpretation of CTMCs as transformers of observables is via their transition function \( P_{t} \): each observable \(f\) is mapped to the observable \( P_{t} f\) that, in turn, maps each state \(x\) to the mean value of \(f\) at time \(t\) conditioned on being in state \(x\) at time \(0\).

The first result is computability of the time evolution of observables, i.e., maps of the form \((t,f)\,{\mapsto }\, P_{t} f\), under conditions that imply existence of a Banach sequence space of observables on which the transition function \( P_{t} \) of a fixed CTMC induces a family of bounded linear operators that vary continuously in time (w.r.t. the usual topology on bounded operators). The second result is PTIME-computability of the projections \(t\,{\mapsto }\,( P_{t} f)(x)\), for each state \(x\), provided that the rate matrix of the CTMC \(X_t\) is locally algebraic on a subspace containing the observable \(f\).

The results are flexible enough to accommodate unbounded observables; explicit examples feature the token counts in stochastic Petri nets and sub-string occurrences of stochastic string rewriting systems. The results provide a functional analytic alternative to Monte Carlo simulation as test bed for mean-field approximations, moment closure, and similar techniques that are fast, but lack absolute error guarantees.


  1. [ABHN11]
    Arendt, W., Batty, C.J.K., Hieber, M., Neubrander, F.: Vector-Valued Laplace Transforms and Cauchy Problems, vol. 96. Springer, Basel (2011)CrossRefzbMATHGoogle Scholar
  2. [AFR11]
    Ackerman, N.L., Freer, C.E., Roy, D.M.: Noncomputable conditional distributions. In: Proceedings of the 26th Annual IEEE Symposium on Logic in Computer Science, LICS 2011, Ontario, Canada, pp. 107–116, 21–24 June 2011Google Scholar
  3. [Ahr99]
    Ahrendt, T.: Fast computations of the exponential function. In: Meinel, C., Tison, S. (eds.) STACS 1999. LNCS, vol. 1563, pp. 302–312. Springer, Heidelberg (1999). doi: 10.1007/3-540-49116-3_28 Google Scholar
  4. [And91]
    Anderson, W.J.: Continuous-Time Markov Chains. Springer, New York (1991)CrossRefzbMATHGoogle Scholar
  5. [Aus55]
    Austin, D.G.: On the existence of the derivative of Markoff transition probability functions. Proc. Natl. Acad. Sci. USA 41(4), 224–226 (1955)MathSciNetCrossRefzbMATHGoogle Scholar
  6. [BCO+07]
    Bostan, A., Chyzak, F., Ollivier, F., Salvy, B., Schost, É, Sedoglavic, A.: Fast computation of power series solutions of systems of differential equations. In: Proceedings of the Eighteenth Annual ACM-SIAM Symposium on Discrete Algorithms, SODA 2007, Philadelphia, PA, USA, pp. 1012–1021. Society for Industrial and Applied Mathematics (2007)Google Scholar
  7. [BHHK03]
    Baier, C., Haverkort, B., Hermanns, H., Katoen, J.-P.: Model-checking algorithms for continuous-time Markov chains. IEEE Trans. Softw. Eng. 29(6), 524–541 (2003)CrossRefzbMATHGoogle Scholar
  8. [DFF+10]
    Danos, V., Feret, J., Fontana, W., Harmer, R., Krivine, J.: Abstracting the differential semantics of rule-based models: exact and automated model reduction. In: Proceedings of the 25th Annual IEEE Symposium on Logic in Computer Science, LICS 2010, Edinburgh, United Kingdom, pp. 362–381, 11–14 July 2010Google Scholar
  9. [DHHZS14]
    Danos, V., Heindel, T., Honorato-Zimmer, R., Stucki, S.: Approximations for stochastic graph rewriting. In: Merz, S., Pang, J. (eds.) ICFEM 2014. LNCS, vol. 8829, pp. 1–10. Springer, Cham (2014). doi: 10.1007/978-3-319-11737-9_1 Google Scholar
  10. [DHHZS15]
    Danos, V., Heindel, T., Honorato-Zimmer, R., Stucki, S.: Moment semantics for reversible rule-based systems. In: Krivine, J., Stefani, J.-B. (eds.) RC 2015. LNCS, vol. 9138, pp. 3–26. Springer, Cham (2015). doi: 10.1007/978-3-319-20860-2_1 CrossRefGoogle Scholar
  11. [DKH97]
    Drewes, F., Kreowski, H.-J., Habel, A.: Hyperedge: replacement, graph grammars. In: Rozenberg, G. (ed.) Handbook of Graph Grammars and Computing by Graph Transformation, pp. 95–162. World Scientific, Singapore (1997)CrossRefGoogle Scholar
  12. [Ein52]
    Einar, H.: A note on Cauchy’s problem. Annales de la Société Polonaise de Mathématique 25, 56–68 (1952)MathSciNetzbMATHGoogle Scholar
  13. [EN00]
    Engel, K.-J., Nagel, R.: One-Parameter Semigroups for Linear Evolution Equations. Springer, New York (2000)zbMATHGoogle Scholar
  14. [Gle13]
    Gleeson, J.P.: Binary-state dynamics on complex networks: pair approximation and beyond. Phys. Rev. X 3, 021004 (2013)Google Scholar
  15. [GM84]
    Gross, D., Miller, D.R.: The randomization technique as a modeling tool and solution procedure for transient Markov processes. Oper. Res. 32(2), 343–361 (1984)MathSciNetCrossRefzbMATHGoogle Scholar
  16. [HLM06]
    Heckel, R., Lajios, G., Menge, S.: Stochastic graph transformation systems. Fundamenta Informaticae 74(1), 63–84 (2006)MathSciNetzbMATHGoogle Scholar
  17. [Kol51]
    Andrey Nikolaevich Kolmogorov: On the differentiability of the transition probabilities in stationary Markov processes with a denumberable number of states. Moskovskogo Gosudarstvennogo Universiteta Učenye Zapiski Matematika 148, 53–59 (1951)MathSciNetGoogle Scholar
  18. [Koz83]
    Kozen, D.: A probabilistic PDL. In: Proceedings of the Fifteenth Annual ACM Symposium on Theory of Computing, STOC 1983, pp. 291–297. ACM, New York (1983)Google Scholar
  19. [Nor98]
    Norris, J.R.: Markov Chains. Cambridge Series in Statistical and Probabilistic Mathematics. Cambridge University Press, Cambridge (1998)zbMATHGoogle Scholar
  20. [Par12]
    Paraskevopoulos, A.G.: The infinite Gauss-Jordan elimination on row-finite \(\omega \times \omega \) matrices. arXiv preprint math (2012)Google Scholar
  21. [PG16]
    Pouly, A., Graça, D.S.: Computational complexity of solving polynomial differential equations over unbounded domains. Theor. Comput. Sci. 626, 67–82 (2016)MathSciNetCrossRefzbMATHGoogle Scholar
  22. [Reu57]
    Reuter, G.E.H.: Denumerable Markov processes and the associated contraction semigroups on l. Acta Mathematica 97(1), 1–46 (1957)MathSciNetCrossRefzbMATHGoogle Scholar
  23. [RR72]
    Reuter, G.E.H., Riley, P.W.: The Feller property for Markov semigroups on a countable state space. J. Lond. Math. Soc. s2–5(2), 267–275 (1972)MathSciNetCrossRefzbMATHGoogle Scholar
  24. [Spi12]
    Spieksma, F.M.: Kolmogorov forward equation and explosiveness in countable state Markov processes. Ann. Oper. Res. 241, 3–22 (2012)MathSciNetCrossRefzbMATHGoogle Scholar
  25. [Spi15]
    Spieksma, F.M.: Countable state Markov processes: non-explosiveness and moment function. Probab. Eng. Inf. Sci. 29, 623–637 (2015)MathSciNetCrossRefGoogle Scholar
  26. [Spi16]
    Spieksma, F.M.: Personal communication, October 2016Google Scholar
  27. [SSG15]
    David Schnoerr, Guido Sanguinetti, Ramon Grima: Comparison of different moment-closure approximations for stochastic chemical kinetics. J. Chem. Phys. 143(18) (2015)Google Scholar
  28. [TR03]
    Rajae Ben Taher and Mustapha Rachidi: On the matrix powers and exponential by the r-generalized fibonacci sequences methods: the companion matrix case. Linear Algebra Appl. 370, 341–353 (2003)MathSciNetCrossRefzbMATHGoogle Scholar
  29. [VMS93]
    Van Moorsel, A.P., Sanders, W.H.: Adaptive uniformization: technical details. Technical report, Department of Computer Science and Department of Electrical Engineering, University of Twente (1993)Google Scholar
  30. [VMS94]
    Van Moorsel, A.P., Sanders, W.H.: Adaptive uniformization. Commun. Stat. Stoch. Models 10, 619–647 (1994)MathSciNetCrossRefzbMATHGoogle Scholar
  31. [WZ07]
    Weihrauch, K., Zhong, N.: Computable analysis of the abstract Cauchy problem in a Banach space and its applications I. Math. Logic Q. 53(4–5), 511–531 (2007)MathSciNetCrossRefzbMATHGoogle Scholar

Copyright information

© Springer-Verlag GmbH Germany 2017

Authors and Affiliations

  • Vincent Danos
    • 1
    • 3
  • Tobias Heindel
    • 2
  • Ilias Garnier
    • 1
    • 3
  • Jakob Grue Simonsen
    • 2
  1. 1.Département d’InformatiqueÉcole Normale SupérieureParisFrance
  2. 2.Department of Computer ScienceUniversity of CopenhagenCopenhagenDenmark
  3. 3.School of InformaticsUniversity of EdinburghEdinburghUK

Personalised recommendations