Adaptive Aggregation of Markov Chains: Quantitative Analysis of Chemical Reaction Networks

  • Alessandro Abate
  • Luboš Brim
  • Milan Češka
  • Marta Kwiatkowska
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9206)


Quantitative analysis of Markov models typically proceeds through numerical methods or simulation-based evaluation. Since the state space of the models can often be large, exact or approximate state aggregation methods (such as lumping or bisimulation reduction) have been proposed to improve the scalability of the numerical schemes. However, none of the existing numerical techniques provides general, explicit bounds on the approximation error, a problem particularly relevant when the level of accuracy affects the soundness of verification results. We propose a novel numerical approach that combines the strengths of aggregation techniques (state-space reduction) with those of simulation-based approaches (automatic updates that adapt to the process dynamics). The key advantage of our scheme is that it provides rigorous precision guarantees under different measures. The new approach, which can be used in conjunction with time uniformisation techniques, is evaluated on two models of chemical reaction networks, a signalling pathway and a prokaryotic gene expression network: it demonstrates marked improvement in accuracy without performance degradation, particularly when compared to known state-space truncation techniques.


Error Bound Chemical Master Equation Empirical Error Probabilistic Model Check Uniformisation Step 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Abate, A., Katoen, J.-P., Lygeros, J., Prandini, M.: Approximate model checking of stochastic hybrid systems. Eur. J. Control 16, 624–641 (2010)MathSciNetCrossRefzbMATHGoogle Scholar
  2. 2.
    Abate, A., Kwiatkowska, M., Norman, G., Parker, D.: Probabilistic model checking of labelled Markov processes via finite approximate bisimulations. In: van Breugel, F., Kashefi, E., Palamidessi, C., Rutten, J. (eds.) Horizons of the Mind. LNCS, vol. 8464, pp. 40–58. Springer, Heidelberg (2014) Google Scholar
  3. 3.
    Angius, A., Horváth, A., Wolf, V.: Quasi Product form approximation for markov models of reaction networks. In: Priami, C., Petre, I., de Vink, E. (eds.) Transactions on Computational Systems Biology XIV. LNCS, vol. 7625, pp. 26–52. Springer, Heidelberg (2012) CrossRefGoogle Scholar
  4. 4.
    Baier, C., Katoen, J.-P.: Principles of Model Checking. The MIT Press, Cambridge (2008)zbMATHGoogle Scholar
  5. 5.
    Bortolussi, L., Hillston, J.: Fluid model checking. In: Koutny, M., Ulidowski, I. (eds.) CONCUR 2012. LNCS, vol. 7454, pp. 333–347. Springer, Heidelberg (2012) CrossRefGoogle Scholar
  6. 6.
    Buchholz, P.: Exact performance equivalence: an equivalence relation for stochastic automata. Theor. Comput. Sci. 215(1–2), 263–287 (1999)MathSciNetCrossRefzbMATHGoogle Scholar
  7. 7.
    Česka, M., Šafránek, D., Dražan, S., Brim, L.: Robustness analysis of stochastic biochemical systems. PloS One 9(4), e94553 (2014)CrossRefGoogle Scholar
  8. 8.
    Chen, T., Kiefer, S.: On the total variation distance of labelled Markov chains. In: Computer Science Logic (CSL) and Logic in Computer Science (LICS) (2014)Google Scholar
  9. 9.
    Dannenberg, F., Hahn, E.M., Kwiatkowska, M.: Computing cumulative rewards using fast adaptive uniformisation. ACM Trans. Model. Comput. Simul. Spec. Issue Comput. Methods Syst. Biol. (CMSB) 25, 9 (2015)MathSciNetGoogle Scholar
  10. 10.
    Desharnais, J., Laviolette, F., Tracol, M.: Approximate analysis of probabilistic processes: logic, simulation and games. In: Quantitative Evaluation of SysTems (QEST), pp. 264–273 (2008)Google Scholar
  11. 11.
    D’Innocenzo, A., Abate, A., Katoen, J.-P.: Robust PCTL model checking. In: Hybrid Systems: Computation and Control (HSCC), pp. 275–285. ACM (2012)Google Scholar
  12. 12.
    Engblom, S.: Computing the moments of high dimensional solutions of the master equation. Appl. Math. Comput. 180(2), 498–515 (2006)MathSciNetCrossRefzbMATHGoogle Scholar
  13. 13.
    Ferm, L., Lötstedt, P.: Adaptive solution of the master equation in low dimensions. Appl. Numer. Math. 59(1), 187–204 (2009)MathSciNetCrossRefGoogle Scholar
  14. 14.
    Fox, B.L., Glynn, P.W.: Computing poisson probabilities. Commun. ACM 31(4), 440–445 (1988)MathSciNetCrossRefGoogle Scholar
  15. 15.
    Gillespie, D.T.: Exact stochastic simulation of coupled chemical reactions. J. Phys. Chem. 81(25), 2340–2381 (1977)CrossRefGoogle Scholar
  16. 16.
    Hasenauer, J., Wolf, V., Kazeroonian, A., Theis, F.: Method of conditional moments (MCM) for the chemical master equation. J. Math. Biol. 69(3), 687–735 (2014)MathSciNetCrossRefzbMATHGoogle Scholar
  17. 17.
    Hegland, M., Burden, C., Santoso, L., MacNamara, S., Booth, H.: A solver for the stochastic master equation applied to gene regulatory networks. J. Comput. Appl. Math. 205(2), 708–724 (2007)MathSciNetCrossRefzbMATHGoogle Scholar
  18. 18.
    Henzinger, T.A., Mateescu, M., Wolf, V.: Sliding window abstraction for infinite Markov chains. In: Bouajjani, A., Maler, O. (eds.) CAV 2009. LNCS, vol. 5643, pp. 337–352. Springer, Heidelberg (2009) CrossRefGoogle Scholar
  19. 19.
    Kierzek, A.M., Zaim, J., Zielenkiewicz, P.: The effect of transcription and translation initiation frequencies on the stochastic fluctuations in prokaryotic gene expression. J. Biol. Chem. 276(11), 8165–8172 (2001)CrossRefGoogle Scholar
  20. 20.
    Kwiatkowska, M., Norman, G., Parker, D.: PRISM 4.0: verification of probabilistic real-time systems. In: Gopalakrishnan, G., Qadeer, S. (eds.) CAV 2011. LNCS, vol. 6806, pp. 585–591. Springer, Heidelberg (2011) CrossRefGoogle Scholar
  21. 21.
    Larsen, K.G., Skou, A.: Bisimulation through probabilistic testing. Inf. Comput. 94(1), 1–28 (1991)MathSciNetCrossRefGoogle Scholar
  22. 22.
    Madsen, C., Myers, C., Roehner, N., Winstead, C., Zhang, Z.: Utilizing stochastic model checking to analyze genetic circuits. In: Computational Intelligence in Bioinformatics and Computational Biology (CIBCB), pp. 379–386. IEEE Computer Society (2012)Google Scholar
  23. 23.
    Mateescu, M., Wolf, V., Didier, F., Henzinger, T.A.: Fast adaptive uniformization of the chemical master equation. IET Syst. Biol. 4(6), 441–452 (2010)CrossRefGoogle Scholar
  24. 24.
    Munsky, B., Khammash, M.: The finite state projection algorithm for the solution of the chemical master equation. J. Chem. Phys. 124, 044104 (2006)CrossRefGoogle Scholar
  25. 25.
    Sidje, R., Stewart, W.: A numerical study of large sparse matrix exponentials arising in Markov chains. Comput. Stat. Data Anal. 29(3), 345–368 (1999)CrossRefzbMATHGoogle Scholar
  26. 26.
    Soudjani, S.E.Z., Abate, A.: Adaptive and sequential gridding procedures for the abstraction and verification of stochastic processes. SIAM J. Appl. Dyn. Syst. 12(2), 921–956 (2013)MathSciNetCrossRefzbMATHGoogle Scholar
  27. 27.
    Esmaeil Zadeh Soudjani, S., Abate, A.: Precise approximations of the probability distribution of a markov process in time: an application to probabilistic invariance. In: Ábrahám, E., Havelund, K. (eds.) TACAS 2014 (ETAPS). LNCS, vol. 8413, pp. 547–561. Springer, Heidelberg (2014) CrossRefGoogle Scholar
  28. 28.
    Steuer, R., Waldherr, S., Sourjik, V., Kollmann, M.: Robust signal processing in living cells. PLoS Comput. Biol. 7(11), e1002218 (2011)MathSciNetCrossRefGoogle Scholar
  29. 29.
    Tkachev, I., Abate, A.: On approximation metrics for linear temporal model-checking of stochastic systems. In: Hybrid Systems: Computation and Control (HSCC), pp. 193–202. ACM (2014)Google Scholar
  30. 30.
    van Moorsel, A.P., Sanders, W.H.: Adaptive uniformization. Stoch. Models 10(3), 619–647 (1994)CrossRefzbMATHGoogle Scholar
  31. 31.
    Zhang, J., Watson, L.T., Cao, Y.: Adaptive aggregation method for the chemical master equation. Int. J. Comput. Biol. Drug Des. 2(2), 134–148 (2009)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  • Alessandro Abate
    • 1
  • Luboš Brim
    • 2
  • Milan Češka
    • 1
    • 2
  • Marta Kwiatkowska
    • 1
  1. 1.Department of Computer ScienceUniversity of OxfordOxfordUK
  2. 2.Faculty of InformaticsMasaryk UniversityBrnoCzech Republic

Personalised recommendations