Skip to main content

Abstraction-Guided Truncations for Stationary Distributions of Markov Population Models

Part of the Lecture Notes in Computer Science book series (LNTCS,volume 12846)


To understand the long-run behavior of Markov population models, the computation of the stationary distribution is often a crucial part. We propose a truncation-based approximation that employs a state-space lumping scheme, aggregating states in a grid structure. The resulting approximate stationary distribution is used to iteratively refine relevant and truncate irrelevant parts of the state-space. This way, the algorithm learns a well-justified finite-state projection tailored to the stationary behavior. We demonstrate the method’s applicability to a wide range of non-linear problems with complex stationary behaviors.


This is a preview of subscription content, log in via an institution.

Buying options

USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
USD   79.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   99.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions


  1. 1.

    Note that in addition mild regularity assumptions are necessary for the existence of a unique CTMC X, such as non-explosiveness [4]. These assumptions are typically valid for realistic reaction networks.

  2. 2.

    In the sequel, we assume an enumeration of all states in \(\mathcal {S}\). We simply write \(x_i\) for the state with index i and drop this notation for entries of a state x.

  3. 3.

  4. 4.

    We note, that \(\sum _{i=0}^n i / (i + k_7)\) can be solved analytically. However, the approximation presented above is much simpler to compute.


  1. Ale, A., Kirk, P., Stumpf, M.P.: A general moment expansion method for stochastic kinetic models. J. Chem. Phys. 138(17), 174101 (2013)

    Article  Google Scholar 

  2. Anderson, D.F., Craciun, G., Kurtz, T.G.: Product-form stationary distributions for deficiency zero chemical reaction networks. Bull. Math. Biol. 72(8), 1947–1970 (2010)

    Article  MathSciNet  Google Scholar 

  3. Anderson, D.F., Kurtz, T.G.: Continuous time Markov chain models for chemical reaction networks. In: Koeppl, H., Setti, G., di Bernardo, M., Densmore, D. (eds.) Design and Analysis of Biomolecular Circuits, pp. 3–42. Springer, New York (2011)

    Chapter  Google Scholar 

  4. Anderson, W.J.: Continuous-Time Markov Chains: An Applications-Oriented Approach. Springer, New York (2012).

    Book  Google Scholar 

  5. Andreychenko, A., Mikeev, L., Spieler, D., Wolf, V.: Parameter identification for Markov models of biochemical reactions. In: Gopalakrishnan, G., Qadeer, S. (eds.) CAV 2011. LNCS, vol. 6806, pp. 83–98. Springer, Heidelberg (2011).

    Chapter  Google Scholar 

  6. Backenköhler, M., Bortolussi, L., Großmann, G., Wolf, V.: Analysis of Markov jump processes under terminal constraints. arXiv preprint arXiv:2010.10096 (2020)

  7. Backenköhler, M., Bortolussi, L., Wolf, V.: Generalized method of moments for stochastic reaction networks in equilibrium. In: Bartocci, E., Lio, P., Paoletti, N. (eds.) CMSB 2016. LNCS, vol. 9859, pp. 15–29. Springer, Cham (2016).

    Chapter  MATH  Google Scholar 

  8. Barzel, B., Biham, O.: Calculation of switching times in the genetic toggle switch and other bistable systems. Phys. Rev. E 78(4), 041919 (2008)

    Article  Google Scholar 

  9. Breuer, L.: From Markov Jump Processes to Spatial Queues. Springer, New York (2003).

    Book  MATH  Google Scholar 

  10. Cao, W.L., Stewart, W.J.: Iterative aggregation/disaggregation techniques for nearly uncoupled Markov chains. J. ACM (JACM) 32(3), 702–719 (1985)

    Article  MathSciNet  Google Scholar 

  11. Češka, M., Křetínský, J.: Semi-quantitative abstraction and analysis of chemical reaction networks. In: Dillig, I., Tasiran, S. (eds.) CAV 2019. LNCS, vol. 11561, pp. 475–496. Springer, Cham (2019).

    Chapter  Google Scholar 

  12. Dayar, T., Hermanns, H., Spieler, D., Wolf, V.: Bounding the equilibrium distribution of Markov population models. Numer. Linear Algebra Appl. 18(6), 931–946 (2011)

    Article  MathSciNet  Google Scholar 

  13. Dowdy, G.R., Barton, P.I.: Bounds on stochastic chemical kinetic systems at steady state. J. Chem. Phys. 148(8), 084106 (2018)

    Article  Google Scholar 

  14. Geva-Zatorsky, N., et al.: Oscillations and variability in the p53 system. Mol. Syst. Biol. 2(1) (2006). 2006.0033

    Google Scholar 

  15. Ghusinga, K.R., Vargas-Garcia, C.A., Lamperski, A., Singh, A.: Exact lower and upper bounds on stationary moments in stochastic biochemical systems. Phys. Biol. 14(4), 04LT01 (2017)

    Article  Google Scholar 

  16. Gillespie, D.T.: Exact stochastic simulation of coupled chemical reactions. J. Phys. Chem. 81(25), 2340–2361 (1977)

    Article  Google Scholar 

  17. Gupta, A., Briat, C., Khammash, M.: A scalable computational framework for establishing long-term behavior of stochastic reaction networks. PLoS Comput. Biol. 10(6), e1003669 (2014)

    Article  Google Scholar 

  18. Gupta, A., Mikelson, J., Khammash, M.: A finite state projection algorithm for the stationary solution of the chemical master equation. J. Chem. Phys. 147(15), 154101 (2017)

    Article  Google Scholar 

  19. Harris, C.R., et al.: Array programming with NumPy. Nature 585, 357–362 (2020).

    Article  Google Scholar 

  20. Henzinger, T.A., Mateescu, M., Wolf, V.: Sliding window abstraction for infinite Markov chains. In: Bouajjani, A., Maler, O. (eds.) CAV 2009. LNCS, vol. 5643, pp. 337–352. Springer, Heidelberg (2009).

    Chapter  Google Scholar 

  21. Jahnke, T., Huisinga, W.: Solving the chemical master equation for monomolecular reaction systems analytically. J. Math. Biol. 54(1), 1–26 (2007)

    Article  MathSciNet  Google Scholar 

  22. Knuth, D.E.: Johann faulhaber and sums of powers. Math. Comput. 61(203), 277–294 (1993)

    Article  MathSciNet  Google Scholar 

  23. Kuntz, J., Thomas, P., Stan, G.B., Barahona, M.: Rigorous bounds on the stationary distributions of the chemical master equation via mathematical programming. arXiv preprint arXiv:1702.05468 (2017)

  24. Kuntz, J., Thomas, P., Stan, G.B., Barahona, M.: Approximations of countably infinite linear programs over bounded measure spaces. SIAM J. Optim. 31(1), 604–625 (2021)

    Article  MathSciNet  Google Scholar 

  25. Kuntz, J., Thomas, P., Stan, G.B., Barahona, M.: Stationary distributions of continuous-time Markov chains: a review of theory and truncation-based approximations. SIAM Rev. 63(1), 3–64 (2021)

    Article  MathSciNet  Google Scholar 

  26. Kurasov, P., Lück, A., Mugnolo, D., Wolf, V.: Stochastic hybrid models of gene regulatory networks-a PDE approach. Math. Biosci. 305, 170–177 (2018)

    Article  MathSciNet  Google Scholar 

  27. Kwiatkowska, M., Norman, G., Parker, D.: PRISM 4.0: verification of probabilistic real-time systems. In: Gopalakrishnan, G., Qadeer, S. (eds.) CAV 2011. LNCS, vol. 6806, pp. 585–591. Springer, Heidelberg (2011).

    Chapter  Google Scholar 

  28. Lapin, M., Mikeev, L., Wolf, V.: SHAVE: stochastic hybrid analysis of Markov population models. In: Proceedings of the 14th International Conference on Hybrid Systems: Computation and Control, pp. 311–312 (2011)

    Google Scholar 

  29. Mélykúti, B., Hespanha, J.P., Khammash, M.: Equilibrium distributions of simple biochemical reaction systems for time-scale separation in stochastic reaction networks. J. R. Soc. Interface 11(97), 20140054 (2014)

    Article  Google Scholar 

  30. Meyn, S.P., Tweedie, R.L.: Stability of Markovian processes III: Foster-Lyapunov criteria for continuous-time processes. Adv. Appl. Probab. 25, 518–548 (1993)

    Article  MathSciNet  Google Scholar 

  31. Meyn, S.P., Tweedie, R.L.: Markov Chains and Stochastic Stability. Springer, London (2012).

    Book  MATH  Google Scholar 

  32. Meyn, S.P., Tweedie, R.L., et al.: Computable bounds for geometric convergence rates of Markov chains. Ann. Appl. Probab. 4(4), 981–1011 (1994)

    Article  MathSciNet  Google Scholar 

  33. Mikeev, L., Neuhäußer, M.R., Spieler, D., Wolf, V.: On-the-fly verification and optimization of DTA-properties for large Markov chains. Formal Methods Syst. Des. 43(2), 313–337 (2013)

    Article  Google Scholar 

  34. Milias-Argeitis, A., Khammash, M.: Optimization-based Lyapunov function construction for continuous-time Markov chains with affine transition rates. In: 53rd IEEE Conference on Decision and Control, pp. 4617–4622. IEEE (2014)

    Google Scholar 

  35. Mode, C.J., Sleeman, C.K.: Stochastic Processes in Epidemiology: HIV/AIDS, Other Infectious Diseases, and Computers. World Scientific (2000)

    Google Scholar 

  36. Munsky, B., Khammash, M.: The finite state projection algorithm for the solution of the chemical master equation. J. Chem. Phys. 124(4), 044104 (2006)

    Article  Google Scholar 

  37. Pardoux, E.: Markov Processes and Applications: Algorithms, Networks, Genome and Finance, vol. 796. Wiley (2008)

    Google Scholar 

  38. Sakurai, Y., Hori, Y.: A convex approach to steady state moment analysis for stochastic chemical reactions. In: 2017 IEEE 56th Annual Conference on Decision and Control (CDC), pp. 1206–1211. IEEE (2017)

    Google Scholar 

  39. Schweitzer, P.J.: A survey of aggregation-disaggregation in large Markov chains. Numer. Solution Markov Chains 8, 63–88 (1991)

    MathSciNet  MATH  Google Scholar 

  40. Spieler, D.: Numerical analysis of long-run properties for Markov population models. Ph.D. thesis, Saarland University (2014)

    Google Scholar 

  41. Stewart, W.J.: Introduction to the Numerical Solution of Markov Chains. Princeton University Press (1994)

    Google Scholar 

  42. Ullah, M., Wolkenhauer, O.: Stochastic Approaches for Systems Biology. Springer, New York (2011).

    Book  MATH  Google Scholar 

  43. Virtanen, P., et al.: SciPy 1.0: fundamental algorithms for scientific computing in Python. Nat. Methods 17, 261–272 (2020).

    Article  Google Scholar 

  44. Van der Vorst, H.A.: Bi-CGSTAB: a fast and smoothly converging variant of BI-CG for the solution of nonsymmetric linear systems. SIAM J. Sci. Stat. Comput. 13(2), 631–644 (1992)

    Article  MathSciNet  Google Scholar 

  45. Wilkinson, D.J.: Stochastic Modelling for Systems Biology. CRC Press, Boca Raton (2018)

    Google Scholar 

Download references


This work is supported by the DFG project “MULTIMODE”.

Author information

Authors and Affiliations


Corresponding author

Correspondence to Michael Backenköhler .

Editor information

Editors and Affiliations


A Detailed Results

See Tables 2, 3 and Fig. 6.

Table 2. Detailed results for Model 2. The errors are computed wrt. the reference Poissonian product. The total absolute error and the maximum absolute errors are given.
Fig. 6.
figure 6

The error over the truncation wrt. the analytical solution

Table 3. Detailed results for Model 3. Upper bounds on the total absolute error and the maximum absolute error are given. The worst-case errors are computed wrt. the reference Geobound solution with \(\epsilon _{\ell }=1e-2\).

B Lyapunov Analysis of the p53 Oscillator

We now derive Lyapunov-sets for the p53 oscillator case study (Model 4). Let the Lyapunov function

$$\begin{aligned} g(x) = 120 x_{\mathrm {p53}} + 0.2 x_{\mathrm {pMdm2}} + 0.1 x_{\mathrm {Mdm2}}\,. \end{aligned}$$

Then the drift

$$\begin{aligned} d(x) =&- \frac{k_3 x_{\mathrm {Mdm2}} x_{\mathrm {p53}}}{x_{\mathrm {p53}} + k_7} - 0.1 k_6 x_{\mathrm {Mdm2}} + 120 k_1 \nonumber \\&- 120 k_2 x_{\mathrm {p53}} + 0.2 k_4 x_{\mathrm {p53}} - 0.1 k_5 x_{\mathrm {pMdm2}} \nonumber \\ =&- \frac{204 x_{\mathrm {Mdm2}} x_{\mathrm {p53}}}{x_{\mathrm {p53}} + 0.01} - 0.096 x_{\mathrm {Mdm2}} - 0.02 x_{\mathrm {p53}} \nonumber \\&- 0.0093 x_{\mathrm {pMdm2}} + 10800\,. \end{aligned}$$

Clearly, \(c = \sup _{x\in {S}} d(x) = 10800\). In particular, the supremum c is at the origin since all non-constant terms are negative. The slowest rate of decrease for (17) is \(x_{\mathrm {p53}}\) with \(x_{\mathrm {Mdm2}} = x_{\mathrm {pMdm2}} = 0\). We are content with a superset of a Lyapunov set (9) for some threshold \(\epsilon _{\ell }\). Therefore taking (9), we can solve the inequality

$$ \frac{\epsilon _{\ell }}{c}(c - 0.02 x_{\mathrm {p53}}) > \epsilon _{\ell } - 1 $$

for \(x_{\mathrm {p53}}\) and

$$\begin{aligned} \frac{c}{0.02 \epsilon _{\ell }} < x_{\mathrm {p53}}\,. \end{aligned}$$


$$\begin{aligned} \pi _{\infty }\left( \left\{ x\in \mathcal {S} \mid \frac{c}{0.2\epsilon _{\ell }} < \Vert x \Vert \right\} \right) > 1 - \epsilon _{\ell }\,. \end{aligned}$$

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Backenköhler, M., Bortolussi, L., Großmann, G., Wolf, V. (2021). Abstraction-Guided Truncations for Stationary Distributions of Markov Population Models. In: Abate, A., Marin, A. (eds) Quantitative Evaluation of Systems. QEST 2021. Lecture Notes in Computer Science(), vol 12846. Springer, Cham.

Download citation

  • DOI:

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-85171-2

  • Online ISBN: 978-3-030-85172-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics