Skip to main content
SpringerLink
Log in

Continuous Time Markov Chain Models for Chemical Reaction Networks

  • Chapter
  • First Online:
Design and Analysis of Biomolecular Circuits

Abstract

A reaction network is a chemical system involving multiple reactions and chemical species. The simplest stochastic models of such networks treat the system as a continuous time Markov chain with the state being the number of molecules of each species and with reactions modeled as possible transitions of the chain. This chapter is devoted to the mathematical study of such stochastic models. We begin by developing much of the mathematical machinery we need to describe the stochastic models we are most interested in. We show how one can represent counting processes of the type we need in terms of Poisson processes. This random time-change representation gives a stochastic equation for continuous-time Markov chain models. We include a discussion on the relationship between this stochastic equation and the corresponding martingale problem and Kolmogorov forward (master) equation. Next, we exploit the representation of the stochastic equation for chemical reaction networks and, under what we will refer to as the classical scaling, show how to derive the deterministic law of mass action from the Markov chain model. We also review the diffusion, or Langevin, approximation, include a discussion of first order reaction networks, and present a large class of networks, those that are weakly reversible and have a deficiency of zero, that induce product-form stationary distributions. Finally, we discuss models in which the numbers of molecules and/or the reaction rate constants of the system vary over several orders of magnitude. We show that one consequence of this wide variation in scales is that different subsystems may evolve on different time scales and this time-scale variation can be exploited to identify reduced models that capture the behavior of parts of the system. We will discuss systematic ways of identifying the different time scales and deriving the reduced models.

Research supported in part by NSF grant DMS 05-53687

Research supported in part by NSF grants DMS 05-53687 and DMS 08-05793

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 129.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 169.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 169.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Anderson DF (2007) A modified next reaction method for simulating chemical systems with time dependent propensities and delays. J Chem Phys 127(21):214107

    Article  Google Scholar 

  2. Anderson DF, Craciun G, Kurtz TG (2010) Product-form stationary distributions for deficiency zero chemical reaction networks. Bull Math Biol 72(8):1947–1970

    Article  MathSciNet  MATH  Google Scholar 

  3. Athreya KB, Ney PE (1972) Branching processes. Springer-Verlag, New York. Die Grundlehren der mathematischen Wissenschaften, Band 196

    Google Scholar 

  4. Ball K, Kurtz TG, Popovic L, Rempala G (2006) Asymptotic analysis of multiscale approximations to reaction networks. Ann Appl Probab 16(4):1925–1961

    Article  MathSciNet  MATH  Google Scholar 

  5. Barrio M, Burrage K, Leier A, Tian T (2006) Oscillatory regulation of Hes1: discrete stochastic delay modelling and simulation. PLoS Comp Biol 2:1017–1030

    Article  Google Scholar 

  6. Bartholomay AF (1958) Stochastic models for chemical reactions. I. Theory of the unimolecular reaction process. Bull Math Biophys 20:175–190

    MathSciNet  Google Scholar 

  7. Bartholomay AF (1959) Stochastic models for chemical reactions. II. The unimolecular rate constant. Bull Math Biophys 21:363–373

    MathSciNet  Google Scholar 

  8. Bratsun D, Volfson D, Tsimring LS, Hasty J (2005) Delay-induced stochastic oscillations in gene regulation. PNAS 102:14593–14598

    Article  Google Scholar 

  9. Darden T (1979) A pseudo-steady state approximation for stochastic chemical kinetics. Rocky Mt J Math 9(1):51–71. Conference on Deterministic Differential Equations and Stochastic Processes Models for Biological Systems, San Cristobal, N.M., 1977

    Google Scholar 

  10. Darden TA (1982) Enzyme kinetics: stochastic vs. deterministic models. In: Reichl LE, Schieve WC (eds) Instabilities, bifurcations, and fluctuations in chemical systems (Austin, Tex., 1980). University of Texas Press, Austin, TX, pp 248–272

    Google Scholar 

  11. Davis MHA (1993) Markov models and optimization. Monographs on statistics and applied probability, vol 49. Chapman & Hall, London

    Google Scholar 

  12. Delbrück M (1940) Statistical fluctuations in autocatalytic reactions. J Chem Phys 8(1): 120–124

    Article  Google Scholar 

  13. Donsker MD (1951) An invariance principle for certain probability limit theorems. Mem Amer Math Soc 1951(6):12

    MathSciNet  Google Scholar 

  14. Ethier SN, Kurtz TG (1986) Markov processes. Wiley series in probability and mathematical statistics: probability and mathematical statistics. John Wiley & Sons Inc, New York. Characterization and convergence

    Google Scholar 

  15. Feinberg M (1987) Chemical reaction network structure and the stability of complex isothermal reactors i. the deficiency zero and deficiency one theorems. Chem Engr Sci 42(10):2229–2268

    Google Scholar 

  16. Feinberg M (1988) Chemical reaction network structure and the stability of complex isothermal reactors ii. multiple steady states for networks of deficiency one. Chem Engr Sci 43(1):1–25

    Google Scholar 

  17. Gadgil C, Lee CH, Othmer HG (2005) A stochastic analysis of first-order reaction networks. Bull Math Biol 67(5):901–946

    Article  MathSciNet  Google Scholar 

  18. Gibson MA, Bruck J (2000) Efficient exact simulation of chemical systems with many species and many channels. J Phys Chem A 104(9):1876–1889

    Article  Google Scholar 

  19. Gillespie DT (1976) A general method for numerically simulating the stochastic time evolution of coupled chemical reactions. J Comput Phys 22(4):403–434

    Article  MathSciNet  Google Scholar 

  20. Gillespie DT (1977) Exact stochastic simulation of coupled chemical reactions. J Phys Chem 81:2340–61

    Article  Google Scholar 

  21. Gillespie DT (1992). A rigorous derivation of the chemical master equation. Physica A 188:404–425

    Article  Google Scholar 

  22. Gillespie DT (2001) Approximate accelerated stochastic simulation of chemically reacting systems. J Chem Phys 115(4):1716–1733

    Article  Google Scholar 

  23. Jacod J (1974/75) Multivariate point processes: predictable projection, Radon-Nikodým derivatives, representation of martingales. Z Wahrscheinlichkeit und Verw Gebiete 31:235–253

    Google Scholar 

  24. Kang HW (2009) The multiple scaling approximation in the heat shock model of e. coli. In Preparation

    Google Scholar 

  25. Kang HW, Kurtz TG (2010) Separation of time-scales and model reduction for stochastic reaction networks. Ann Appl Probab (to appear)

    Google Scholar 

  26. Kang HW, Kurtz TG, Popovic L (2010) Diffusion approximations for multiscale chemical reaction models. In Preparation

    Google Scholar 

  27. Kelly FP (1979) Reversibility and stochastic networks. Wiley series in probability and mathematical statistics. John Wiley & Sons Ltd, Chichester

    MATH  Google Scholar 

  28. Kolmogorov AN (1956) Foundations of the theory of probability. Chelsea Publishing Co, New York. Translation edited by Nathan Morrison, with an added bibliography by A. T. Bharucha-Reid

    Google Scholar 

  29. Komlós J, Major P, Tusnády G (1975) An approximation of partial sums of independent RV’s and the sample DF. I. Z Wahrscheinlichkeit und Verw Gebiete 32:111–131

    Article  MATH  Google Scholar 

  30. Komlós J, Major P, Tusnády G (1976) An approximation of partial sums of independent RV’s, and the sample DF. II. Z Wahrscheinlichkeit und Verw Gebiete 34(1):33–58

    Article  MATH  Google Scholar 

  31. Kurtz TG (1970) Solutions of ordinary differential equations as limits of pure jump Markov processes. J Appl Probab 7:49–58

    Article  MathSciNet  MATH  Google Scholar 

  32. Kurtz TG (1971) Limit theorems for sequences of jump Markov processes approximating ordinary differential processes. J Appl Probab 8:344–356

    Article  MathSciNet  MATH  Google Scholar 

  33. Kurtz TG (1972) The relationship between stochastic and deterministic models for chemical reactions. J Chem Phys 57(7):2976–2978

    Article  Google Scholar 

  34. Kurtz TG (1977/78) Strong approximation theorems for density dependent Markov chains. Stoch Proc Appl 6(3):223–240

    Google Scholar 

  35. Kurtz TG (1980) Representations of Markov processes as multiparameter time changes. Ann Probab 8(4):682–715

    Article  MathSciNet  MATH  Google Scholar 

  36. Kurtz TG (2007) The Yamada-Watanabe-Engelbert theorem for general stochastic equations and inequalities. Electron J Probab 12:951–965

    MathSciNet  MATH  Google Scholar 

  37. Kurtz TG (2010) Equivalence of stochastic equations and martingale problems. In: Dan Crisan (ed) Stochastic analysis 2010. Springer, Heidelberg

    Google Scholar 

  38. E W, Liu D, Vanden-Eijnden E (2005) Nested stochastic simulation algorithm for chemical kinetic systems with disparate rates. J Chem Phys 123(19):194107

    Google Scholar 

  39. McQuarrie DA (1967) Stochastic approach to chemical kinetics. J Appl Probab 4:413–478

    Article  MathSciNet  MATH  Google Scholar 

  40. Meyer PA (1971) Démonstration simplifiée d’un théorème de Knight. In: Dellacherie C, Meyer PA (eds) Séminaire de Probabilités, V (Univ. Strasbourg, année universitaire 1969–1970). Lecture Notes in Math, vol 191. Springer, Berlin, pp 191–195

    Google Scholar 

  41. Ross S (1984) A first course in probability, 2ed edn Macmillan Co, New York

    MATH  Google Scholar 

  42. van Kampen NG (1961) A power series expansion of the master equation. Canad J Phys 39:551–567

    Article  MathSciNet  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Departments of Mathematics, University of Wisconsin-Madison, 480 Lincoln Drive, Madison, WI, 53706-1388, USA

    Thomas G. Kurtz

Authors
  1. David F. Anderson
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Thomas G. Kurtz
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Thomas G. Kurtz .

Editor information

Editors and Affiliations

  1. , Automatic Control Laboratory, ETH Zurich, Physikstrasse 3, Zurich, 8092, Switzerland

    Heinz Koeppl

  2. ARCES, Università di Bologna, Via Toffano 2/2, Bologna, 40125, Italy

    Gianluca Setti

  3. Università di Napoli Federico II, Via Claudio 21, Napoli, 80125, Italy

    Mario di Bernardo

  4. , Department of Electrical and Computer En, Boston University, Saint Mary's St. 8, Boston, 02446, Massachusetts, USA

    Douglas Densmore

Rights and permissions

Reprints and permissions

Copyright information

© 2011 Springer Science+Business Media, LLC

About this chapter

Cite this chapter

Anderson, D.F., Kurtz, T.G. (2011). Continuous Time Markov Chain Models for Chemical Reaction Networks. In: Koeppl, H., Setti, G., di Bernardo, M., Densmore, D. (eds) Design and Analysis of Biomolecular Circuits. Springer, New York, NY. https://doi.org/10.1007/978-1-4419-6766-4_1

Download citation

  • DOI: https://doi.org/10.1007/978-1-4419-6766-4_1

  • Published:

  • Publisher Name: Springer, New York, NY

  • Print ISBN: 978-1-4419-6765-7

  • Online ISBN: 978-1-4419-6766-4

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation