Skip to main content
Book cover

International Conference on Tools and Algorithms for the Construction and Analysis of Systems

TACAS 2021: Tools and Algorithms for the Construction and Analysis of Systems pp 210–229Cite as

Analysis of Markov Jump Processes under Terminal Constraints

  • 1662 Accesses

Part of the Lecture Notes in Computer Science book series (LNTCS,volume 12651)

Abstract

Many probabilistic inference problems such as stochastic filtering or the computation of rare event probabilities require model analysis under initial and terminal constraints. We propose a solution to this bridging problem for the widely used class of population-structured Markov jump processes. The method is based on a state-space lumping scheme that aggregates states in a grid structure. The resulting approximate bridging distribution is used to iteratively refine relevant and truncate irrelevant parts of the state-space. This way, the algorithm learns a well-justified finite-state projection yielding guaranteed lower bounds for the system behavior under endpoint constraints. We demonstrate the method’s applicability to a wide range of problems such as Bayesian inference and the analysis of rare events.

Keywords

  • Bayesian Inference
  • Bridging problem
  • Smoothing
  • Lumping
  • Rare Events.

Download conference paper PDF

References

  1. Adan, A., Alizada, G., Kiraz, Y., Baran, Y., Nalbant, A.: Flow cytometry: basic principles and applications. Critical reviews in biotechnology 37(2), 163–176 (2017)

    Google Scholar 

  2. Amparore, E.G., Donatelli, S.: Backward solution of Markov chains and Markov regenerative processes: Formalization and applications. Electron. Notes Theor. Comput. Sci. 296, 7–26 (2013)

    Google Scholar 

  3. Andreychenko, A., Crouzen, P., Mikeev, L., Wolf, V.: On-the-fly uniformization of time-inhomogeneous infinite Markov population models. arXiv preprint arXiv:1006.4425 (2010)

  4. Andreychenko, A., Mikeev, L., Spieler, D., Wolf, V.: Parameter identification for Markov models of biochemical reactions. In: International Conference on Computer Aided Verification. pp. 83–98. Springer (2011)

    Google Scholar 

  5. Backenköhler, M., Bortolussi, L., Wolf, V.: Control variates for stochastic simulation of chemical reaction networks. In: Bortolussi, L., Sanguinetti, G. (eds.) Computational Methods in Systems Biology. pp. 42–59. Springer, Cham (2019)

    Google Scholar 

  6. Backenköhler, M., Bortolussi, L., Wolf, V.: Bounding mean first passage times in population continuous-time Markov chains. To appear in Proc. of QEST’20 (2020)

    Google Scholar 

  7. Barzel, B., Biham, O.: Calculation of switching times in the genetic toggle switch and other bistable systems. Physical Review E 78(4), 041919 (2008)

    Google Scholar 

  8. Bortolussi, L., Lanciani, R.: Stochastic approximation of global reachability probabilities of Markov population models. In: Computer Performance Engineering - 11th European Workshop, EPEW 2014, Florence, Italy, September 11–12, 2014. Proceedings. pp. 224–239 (2014)

    Google Scholar 

  9. Bortolussi, L., Lanciani, R., Nenzi, L.: Model checking markov population models by stochastic approximations. Inf. Comput. 262, 189–220 (2018)

    Google Scholar 

  10. Breuer, L.: From Markov jump processes to spatial queues. Springer Science & Business Media (2003)

    Google Scholar 

  11. Broemeling, L.D.: Bayesian Inference for Stochastic Processes. CRC Press (2017)

    Google Scholar 

  12. Buchholz, P.: Exact and ordinary lumpability in finite Markov chains. Journal of applied probability pp. 59–75 (1994)

    Google Scholar 

  13. Byrne, G.D., Hindmarsh, A.C.: A polyalgorithm for the numerical solution of ordinary differential equations. ACM Transactions on Mathematical Software (TOMS) 1(1), 71–96 (1975)

    Google Scholar 

  14. Daigle Jr, B.J., Roh, M.K., Gillespie, D.T., Petzold, L.R.: Automated estimation of rare event probabilities in biochemical systems. The Journal of Chemical Physics 134(4), 01B628 (2011)

    Google Scholar 

  15. Dayar, T., Stewart, W.J.: Quasi lumpability, lower-bounding coupling matrices, and nearly completely decomposable Markov chains. SIAM Journal on Matrix Analysis and Applications 18(2), 482–498 (1997)

    Google Scholar 

  16. Gillespie, D.T.: Exact stochastic simulation of coupled chemical reactions. The journal of physical chemistry 81(25), 2340–2361 (1977)

    Google Scholar 

  17. Golightly, A., Sherlock, C.: Efficient sampling of conditioned Markov jump processes. Statistics and Computing 29(5), 1149–1163 (2019)

    Google Scholar 

  18. Golightly, A., Wilkinson, D.J.: Bayesian inference for stochastic kinetic models using a diffusion approximation. Biometrics 61(3), 781–788 (2005)

    Google Scholar 

  19. Golightly, A., Wilkinson, D.J.: Bayesian parameter inference for stochastic biochemical network models using particle Markov chain monte carlo. Interface focus 1(6), 807–820 (2011)

    Google Scholar 

  20. Grossmann, G., Backenköhler, M., Wolf, V.: Importance of interaction structure and stochasticity for epidemic spreading: A COVID-19 case study. In: Seventeenth international conference on the quantitative evaluation of systems (QEST 2020). IEEE (2020)

    Google Scholar 

  21. Hajnal, M., Nouvian, M., Šafránek, D., Petrov, T.: Data-informed parameter synthesis for population markov chains. In: International Workshop on Hybrid Systems Biology. pp. 147–164. Springer (2019)

    Google Scholar 

  22. Hayden, R.A., Stefanek, A., Bradley, J.T.: Fluid computation of passage-time distributions in large Markov models. Theoretical Computer Science 413(1), 106–141 (2012)

    Google Scholar 

  23. He, S., Peng, Y., Sun, K.: SEIR modeling of the COVID-19 and its dynamics. Nonlinear Dynamics pp. 1–14 (2020)

    Google Scholar 

  24. Henzinger, T.A., Mateescu, M., Wolf, V.: Sliding window abstraction for infinite Markov chains. In: International Conference on Computer Aided Verification. pp. 337–352. Springer (2009)

    Google Scholar 

  25. Huang, L., Pauleve, L., Zechner, C., Unger, M., Hansen, A.S., Koeppl, H.: Reconstructing dynamic molecular states from single-cell time series. Journal of The Royal Society Interface 13(122), 20160533 (2016)

    Google Scholar 

  26. Kuwahara, H., Mura, I.: An efficient and exact stochastic simulation method to analyze rare events in biochemical systems. The Journal of chemical physics 129(16), 10B619 (2008)

    Google Scholar 

  27. Kwiatkowska, M., Norman, G., Parker, D.: Prism 4.0: Verification of probabilistic real-time systems. In: International conference on computer aided verification. pp. 585–591. Springer (2011)

    Google Scholar 

  28. Lapin, M., Mikeev, L., Wolf, V.: SHAVE: stochastic hybrid analysis of Markov population models. In: Proceedings of the 14th international conference on Hybrid systems: computation and control. pp. 311–312 (2011)

    Google Scholar 

  29. Lipshtat, A., Loinger, A., Balaban, N.Q., Biham, O.: Genetic toggle switch without cooperative binding. Physical review letters 96(18), 188101 (2006)

    Google Scholar 

  30. Loinger, A., Lipshtat, A., Balaban, N.Q., Biham, O.: Stochastic simulations of genetic switch systems. Physical Review E 75(2), 021904 (2007)

    Google Scholar 

  31. Mikeev, L., Neuhäußer, M.R., Spieler, D., Wolf, V.: On-the-fly verification and optimization of DTA-properties for large Markov chains. Formal Methods in System Design 43(2), 313–337 (2013)

    Google Scholar 

  32. Mikeev, L., Sandmann, W.: Approximate numerical integration of the chemical master equation for stochastic reaction networks. arXiv preprint arXiv:1907.10245 (2019)

  33. Mikeev, L., Sandmann, W., Wolf, V.: Efficient calculation of rare event probabilities in Markovian queueing networks. In: Proceedings of the 5th International ICST Conference on Performance Evaluation Methodologies and Tools. pp. 186–196 (2011)

    Google Scholar 

  34. Mikeev, L., Sandmann, W., Wolf, V.: Numerical approximation of rare event probabilities in biochemically reacting systems. In: International Conference on Computational Methods in Systems Biology. pp. 5–18. Springer (2013)

    Google Scholar 

  35. Milner, P., Gillespie, C.S., Wilkinson, D.J.: Moment closure based parameter inference of stochastic kinetic models. Statistics and Computing 23(2), 287–295 (2013)

    Google Scholar 

  36. Mode, C.J., Sleeman, C.K.: Stochastic processes in epidemiology: HIV/AIDS, other infectious diseases, and computers. World Scientific (2000)

    Google Scholar 

  37. Munsky, B., Khammash, M.: The finite state projection algorithm for the solution of the chemical master equation. The Journal of chemical physics 124(4), 044104 (2006)

    Google Scholar 

  38. Neupane, T., Myers, C.J., Madsen, C., Zheng, H., Zhang, Z.: Stamina: Stochastic approximate model-checker for infinite-state analysis. In: International Conference on Computer Aided Verification. pp. 540–549. Springer (2019)

    Google Scholar 

  39. Pardoux, E.: Markov processes and applications: algorithms, networks, genome and finance, vol. 796. John Wiley & Sons (2008)

    Google Scholar 

  40. Rabiner, L., Juang, B.: An introduction to hidden Markov models. IEEE ASSP Magazine 3(1), 4–16 (1986)

    Google Scholar 

  41. Särkkä, S.: Bayesian filtering and smoothing, vol. 3. Cambridge University Press (2013)

    Google Scholar 

  42. Schnoerr, D., Cseke, B., Grima, R., Sanguinetti, G.: Efficient low-order approximation of first-passage time distributions. Phys. Rev. Lett. 119, 210601 (Nov 2017). https://doi.org/10.1103/PhysRevLett.119.210601

  43. Schnoerr, D., Sanguinetti, G., Grima, R.: Validity conditions for moment closure approximations in stochastic chemical kinetics. The Journal of chemical physics 141(8), 08B616\_1 (2014)

    Google Scholar 

  44. Siegal-Gaskins, D., Mejia-Guerra, M.K., Smith, G.D., Grotewold, E.: Emergence of switch-like behavior in a large family of simple biochemical networks. PLoS Comput Biol 7(5), e1002039 (2011)

    Google Scholar 

  45. Strasser, M., Theis, F.J., Marr, C.: Stability and multiattractor dynamics of a toggle switch based on a two-stage model of stochastic gene expression. Biophysical journal 102(1), 19–29 (2012)

    Google Scholar 

  46. Ullah, M., Wolkenhauer, O.: Stochastic approaches for systems biology. Springer Science & Business Media (2011)

    Google Scholar 

  47. Virtanen, P., Gommers, R., Oliphant, T.E., Haberland, M., Reddy, T., Cournapeau, D., Burovski, E., Peterson, P., Weckesser, W., Bright, J., van der Walt, S.J., Brett, M., Wilson, J., Jarrod Millman, K., Mayorov, N., Nelson, A.R.J., Jones, E., Kern, R., Larson, E., Carey, C., Polat, İ., Feng, Y., Moore, E.W., Vand erPlas, J., Laxalde, D., Perktold, J., Cimrman, R., Henriksen, I., Quintero, E.A., Harris, C.R., Archibald, A.M., Ribeiro, A.H., Pedregosa, F., van Mulbregt, P., Contributors, S...: SciPy 1.0: Fundamental Algorithms for Scientific Computing in Python. Nature Methods 17, 261–272 (2020). https://doi.org/10.1038/s41592-019-0686-2

  48. Wildner, C., Koeppl, H.: Moment-based variational inference for Markov jump processes. arXiv preprint arXiv:1905.05451 (2019)

  49. Wilkinson, D.J.: Stochastic modelling for systems biology. CRC Press (2018)

    Google Scholar 

  50. Zapreev, I., Katoen, J.P.: Safe on-the-fly steady-state detection for time-bounded reachability. In: Third International Conference on the Quantitative Evaluation of Systems-(QEST’06). pp. 301–310. IEEE (2006)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Saarbrücken Graduate School of Computer Science, Saarland University, Saarland Informatics Campus E1 3, Saarbrücken, Germany

    Michael Backenköhler, Gerrit Großmann & Verena Wolf

  2. Univeristy of Trieste, Trieste, Italy

    Luca Bortolussi

  3. Saarland University, Saarland Informatics Campus E1 3, Saarbrücken, Germany

    Luca Bortolussi & Verena Wolf

Authors
  1. Michael Backenköhler
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Luca Bortolussi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Gerrit Großmann
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Verena Wolf
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Michael Backenköhler .

Editor information

Editors and Affiliations

  1. Eindhoven University of Technology, Eindhoven, The Netherlands

    Prof. Jan Friso Groote

  2. Aalborg University, Aalborg East, Denmark

    Prof. Kim Guldstrand Larsen

Rights and permissions

Open Access This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

Reprints and Permissions

Copyright information

© 2021 The Author(s)

About this paper

Verify currency and authenticity via CrossMark

Cite this paper

Backenköhler, M., Bortolussi, L., Großmann, G., Wolf, V. (2021). Analysis of Markov Jump Processes under Terminal Constraints. In: Groote, J.F., Larsen, K.G. (eds) Tools and Algorithms for the Construction and Analysis of Systems. TACAS 2021. Lecture Notes in Computer Science(), vol 12651. Springer, Cham. https://doi.org/10.1007/978-3-030-72016-2_12

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-72016-2_12

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-72015-5

  • Online ISBN: 978-3-030-72016-2

  • eBook Packages: Computer ScienceComputer Science (R0)