Continuous Time Markov Chain Models for Chemical Reaction Networks

  • David F. Anderson
  • Thomas G. Kurtz


A reaction network is a chemical system involving multiple reactions and chemical species. The simplest stochastic models of such networks treat the system as a continuous time Markov chain with the state being the number of molecules of each species and with reactions modeled as possible transitions of the chain. This chapter is devoted to the mathematical study of such stochastic models. We begin by developing much of the mathematical machinery we need to describe the stochastic models we are most interested in. We show how one can represent counting processes of the type we need in terms of Poisson processes. This random time-change representation gives a stochastic equation for continuous-time Markov chain models. We include a discussion on the relationship between this stochastic equation and the corresponding martingale problem and Kolmogorov forward (master) equation. Next, we exploit the representation of the stochastic equation for chemical reaction networks and, under what we will refer to as the classical scaling, show how to derive the deterministic law of mass action from the Markov chain model. We also review the diffusion, or Langevin, approximation, include a discussion of first order reaction networks, and present a large class of networks, those that are weakly reversible and have a deficiency of zero, that induce product-form stationary distributions. Finally, we discuss models in which the numbers of molecules and/or the reaction rate constants of the system vary over several orders of magnitude. We show that one consequence of this wide variation in scales is that different subsystems may evolve on different time scales and this time-scale variation can be exploited to identify reduced models that capture the behavior of parts of the system. We will discuss systematic ways of identifying the different time scales and deriving the reduced models.


Reaction network Markov chain Law of mass action Law of large numbers Central limit theorem Diffusion approximation Langevin approximation Stochastic equations Multiscale analysis Stationary distributions 


  1. 1.
    Anderson DF (2007) A modified next reaction method for simulating chemical systems with time dependent propensities and delays. J Chem Phys 127(21):214107CrossRefGoogle Scholar
  2. 2.
    Anderson DF, Craciun G, Kurtz TG (2010) Product-form stationary distributions for deficiency zero chemical reaction networks. Bull Math Biol 72(8):1947–1970MathSciNetMATHCrossRefGoogle Scholar
  3. 3.
    Athreya KB, Ney PE (1972) Branching processes. Springer-Verlag, New York. Die Grundlehren der mathematischen Wissenschaften, Band 196Google Scholar
  4. 4.
    Ball K, Kurtz TG, Popovic L, Rempala G (2006) Asymptotic analysis of multiscale approximations to reaction networks. Ann Appl Probab 16(4):1925–1961MathSciNetMATHCrossRefGoogle Scholar
  5. 5.
    Barrio M, Burrage K, Leier A, Tian T (2006) Oscillatory regulation of Hes1: discrete stochastic delay modelling and simulation. PLoS Comp Biol 2:1017–1030CrossRefGoogle Scholar
  6. 6.
    Bartholomay AF (1958) Stochastic models for chemical reactions. I. Theory of the unimolecular reaction process. Bull Math Biophys 20:175–190MathSciNetGoogle Scholar
  7. 7.
    Bartholomay AF (1959) Stochastic models for chemical reactions. II. The unimolecular rate constant. Bull Math Biophys 21:363–373MathSciNetGoogle Scholar
  8. 8.
    Bratsun D, Volfson D, Tsimring LS, Hasty J (2005) Delay-induced stochastic oscillations in gene regulation. PNAS 102:14593–14598CrossRefGoogle Scholar
  9. 9.
    Darden T (1979) A pseudo-steady state approximation for stochastic chemical kinetics. Rocky Mt J Math 9(1):51–71. Conference on Deterministic Differential Equations and Stochastic Processes Models for Biological Systems, San Cristobal, N.M., 1977Google Scholar
  10. 10.
    Darden TA (1982) Enzyme kinetics: stochastic vs. deterministic models. In: Reichl LE, Schieve WC (eds) Instabilities, bifurcations, and fluctuations in chemical systems (Austin, Tex., 1980). University of Texas Press, Austin, TX, pp 248–272Google Scholar
  11. 11.
    Davis MHA (1993) Markov models and optimization. Monographs on statistics and applied probability, vol 49. Chapman & Hall, LondonGoogle Scholar
  12. 12.
    Delbrück M (1940) Statistical fluctuations in autocatalytic reactions. J Chem Phys 8(1): 120–124CrossRefGoogle Scholar
  13. 13.
    Donsker MD (1951) An invariance principle for certain probability limit theorems. Mem Amer Math Soc 1951(6):12MathSciNetGoogle Scholar
  14. 14.
    Ethier SN, Kurtz TG (1986) Markov processes. Wiley series in probability and mathematical statistics: probability and mathematical statistics. John Wiley & Sons Inc, New York. Characterization and convergenceGoogle Scholar
  15. 15.
    Feinberg M (1987) Chemical reaction network structure and the stability of complex isothermal reactors i. the deficiency zero and deficiency one theorems. Chem Engr Sci 42(10):2229–2268Google Scholar
  16. 16.
    Feinberg M (1988) Chemical reaction network structure and the stability of complex isothermal reactors ii. multiple steady states for networks of deficiency one. Chem Engr Sci 43(1):1–25Google Scholar
  17. 17.
    Gadgil C, Lee CH, Othmer HG (2005) A stochastic analysis of first-order reaction networks. Bull Math Biol 67(5):901–946MathSciNetCrossRefGoogle Scholar
  18. 18.
    Gibson MA, Bruck J (2000) Efficient exact simulation of chemical systems with many species and many channels. J Phys Chem A 104(9):1876–1889CrossRefGoogle Scholar
  19. 19.
    Gillespie DT (1976) A general method for numerically simulating the stochastic time evolution of coupled chemical reactions. J Comput Phys 22(4):403–434MathSciNetCrossRefGoogle Scholar
  20. 20.
    Gillespie DT (1977) Exact stochastic simulation of coupled chemical reactions. J Phys Chem 81:2340–61CrossRefGoogle Scholar
  21. 21.
    Gillespie DT (1992). A rigorous derivation of the chemical master equation. Physica A 188:404–425CrossRefGoogle Scholar
  22. 22.
    Gillespie DT (2001) Approximate accelerated stochastic simulation of chemically reacting systems. J Chem Phys 115(4):1716–1733CrossRefGoogle Scholar
  23. 23.
    Jacod J (1974/75) Multivariate point processes: predictable projection, Radon-Nikodým derivatives, representation of martingales. Z Wahrscheinlichkeit und Verw Gebiete 31:235–253Google Scholar
  24. 24.
    Kang HW (2009) The multiple scaling approximation in the heat shock model of e. coli. In PreparationGoogle Scholar
  25. 25.
    Kang HW, Kurtz TG (2010) Separation of time-scales and model reduction for stochastic reaction networks. Ann Appl Probab (to appear)Google Scholar
  26. 26.
    Kang HW, Kurtz TG, Popovic L (2010) Diffusion approximations for multiscale chemical reaction models. In PreparationGoogle Scholar
  27. 27.
    Kelly FP (1979) Reversibility and stochastic networks. Wiley series in probability and mathematical statistics. John Wiley & Sons Ltd, ChichesterMATHGoogle Scholar
  28. 28.
    Kolmogorov AN (1956) Foundations of the theory of probability. Chelsea Publishing Co, New York. Translation edited by Nathan Morrison, with an added bibliography by A. T. Bharucha-ReidGoogle Scholar
  29. 29.
    Komlós J, Major P, Tusnády G (1975) An approximation of partial sums of independent RV’s and the sample DF. I. Z Wahrscheinlichkeit und Verw Gebiete 32:111–131MATHCrossRefGoogle Scholar
  30. 30.
    Komlós J, Major P, Tusnády G (1976) An approximation of partial sums of independent RV’s, and the sample DF. II. Z Wahrscheinlichkeit und Verw Gebiete 34(1):33–58MATHCrossRefGoogle Scholar
  31. 31.
    Kurtz TG (1970) Solutions of ordinary differential equations as limits of pure jump Markov processes. J Appl Probab 7:49–58MathSciNetMATHCrossRefGoogle Scholar
  32. 32.
    Kurtz TG (1971) Limit theorems for sequences of jump Markov processes approximating ordinary differential processes. J Appl Probab 8:344–356MathSciNetMATHCrossRefGoogle Scholar
  33. 33.
    Kurtz TG (1972) The relationship between stochastic and deterministic models for chemical reactions. J Chem Phys 57(7):2976–2978CrossRefGoogle Scholar
  34. 34.
    Kurtz TG (1977/78) Strong approximation theorems for density dependent Markov chains. Stoch Proc Appl 6(3):223–240Google Scholar
  35. 35.
    Kurtz TG (1980) Representations of Markov processes as multiparameter time changes. Ann Probab 8(4):682–715MathSciNetMATHCrossRefGoogle Scholar
  36. 36.
    Kurtz TG (2007) The Yamada-Watanabe-Engelbert theorem for general stochastic equations and inequalities. Electron J Probab 12:951–965MathSciNetMATHGoogle Scholar
  37. 37.
    Kurtz TG (2010) Equivalence of stochastic equations and martingale problems. In: Dan Crisan (ed) Stochastic analysis 2010. Springer, HeidelbergGoogle Scholar
  38. 38.
    E W, Liu D, Vanden-Eijnden E (2005) Nested stochastic simulation algorithm for chemical kinetic systems with disparate rates. J Chem Phys 123(19):194107Google Scholar
  39. 39.
    McQuarrie DA (1967) Stochastic approach to chemical kinetics. J Appl Probab 4:413–478MathSciNetMATHCrossRefGoogle Scholar
  40. 40.
    Meyer PA (1971) Démonstration simplifiée d’un théorème de Knight. In: Dellacherie C, Meyer PA (eds) Séminaire de Probabilités, V (Univ. Strasbourg, année universitaire 1969–1970). Lecture Notes in Math, vol 191. Springer, Berlin, pp 191–195Google Scholar
  41. 41.
    Ross S (1984) A first course in probability, 2ed edn Macmillan Co, New YorkMATHGoogle Scholar
  42. 42.
    van Kampen NG (1961) A power series expansion of the master equation. Canad J Phys 39:551–567MathSciNetMATHCrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2011

Authors and Affiliations

  1. 1.Departments of MathematicsUniversity of Wisconsin-MadisonMadisonUSA

Personalised recommendations