Stochastic Description of Biochemical Networks
Abstract
Conventional deterministic chemical kinetics often breaks down in the small volume of a living cell where cellular species (e.g., genes, mRNAs, etc.) exist in discrete, low copy numbers and react through reaction channels whose timing and order is random. In such an environment, a stochastic chemical kinetics framework that models species abundances as discrete random variables is more suitable. The resulting models consist of continuetime discretestate Markov chains. Here we describe how such models can be formulated and numerically simulated, and we present some of the key analysis techniques for studying such reactions.
Keywords
Stochastic biochemical reactions Stochastic models Gillespie algorithm Chemical master equation Moment dynamicsIntroduction
The time evolution of a spatially homogeneous mixture of chemically reacting molecules is often modeled using a stochastic formulation, which takes into account the inherent randomness of thermal molecular motion. This formulation is important when modeling complex reactions inside living cells, where small populations of key reactants can set the stage for significant stochastic effects. In this entry, we review the basic stochastic model of chemical reactions and discuss the most common techniques used to simulate and analyze this model.
Stochastic Models of Chemical Reactions
Propensity functions for elementary reactions. The constants c, c ^{ ′ }, and c ^{ ′′ } are related to k, k ^{ ′ }, and k ^{ ′′ }, the reaction rate constants from deterministic massaction kinetics. Indeed it can be shown that c = k, \(c^{{\prime}} = k^{{\prime}}/\Omega \), and \(c^{{\prime\prime}} = 2k^{{\prime\prime}}/\Omega \)
Reaction type  Propensity function  

S _{ i } → Products  \(c\boldsymbol{x}_{i}\)  
\(S_{i} + S_{j} \rightarrow \mbox{ Products}\quad (i\neq j)\)  \(c^{{\prime}}\boldsymbol{x}_{i}\boldsymbol{x}_{j}\)  
S _{ i } + S _{ i } → Products  \(c^{{\prime\prime}}\boldsymbol{x}_{i}(\boldsymbol{x}_{i}  1)/2\) 
Limiting to the Deterministic Regime
Hence, over any finite time interval, the stochastic model converges to the deterministic massaction one in the thermodynamic limit. Note that this is only a large volume limit result. In practice, for a fixed volume, a stochastic description may differ considerably from the deterministic description.
Stochastic Simulations
 1.
Initialize the state X(0) and set t = 0.
 2.
Draw a random number τ ∈ (0, ∞) with exponential distribution and mean equal to \(1/\sum _{k}w_{k}(X(t))\).
 3.
Draw a random number \(k \in \{ 1,2,\ldots ,M\}\) such that the probability of \(k = i \in \{ 1,2,\ldots ,M\}\) is proportional to w _{ i }(X(t)).
 4.
Set \(X(t+\tau ) = X(t) +\boldsymbol{ s}_{k}\) and \(t = t+\tau\).
 5.
Repeat from (2) until t reaches the desired simulation time.
By running this algorithm multiple times with independent random draws, one can estimate the distribution and statistical moments of the random process X(t).
The Chemical Master Equation (CME)

The probability that an R _{ k } reaction fires exactly once in the time interval [t, t + h) is given by \(w_{k}(\boldsymbol{x})h\).

The probability that no reactions fire in the time interval [t, t + h) is given by \(1 \sum _{k}w_{k}(\boldsymbol{x})dx\).

The probability that more than one reaction fires in the time interval [t, t + h) is zero.
The left panel in the figure shows the infinite states of a system with two species. The arrows indicate transitions among states caused by allowable chemical reactions. The underlying stochastic process is a continuoustime discretestate Markov process. The right panel shows the projected (finitestate) system for a specific projection region (box). The projection is obtained as follows: transitions within the retained sates are kept, while transitions that emanate from these states and end at states outside the box are channeled to a single new absorbing state. Transitions into the box are deleted. The resulting projected system is a finitestate Markov process. The probability of each of its finite states can be computed exactly. It can be shown that the truncation, as defined here, gives a lower bound for the probability for the original full system. The FSP algorithm provides a way for constructing an approximation of the CME that satisfies any prespecified accuracy requirement.
Moment Dynamics
While the probability distribution \(P(\boldsymbol{x},t)\) provides great detail on the state \(\boldsymbol{x}\) at time t, often statistical moments of the molecule copy numbers already provide important information about their variability, which motivates the construction of mathematical models for the evolution of such models over time.
When all chemical reactions have only one reactant, the term \(B\bar{\mu }\) does not appear in (2), and we say that the exact moment dynamics are closed. However, when at least one chemical reaction has two or more reactants, then the term \(B\bar{\mu }\) appears, and we say that the moment dynamics are open since (2) depends on the moments in \(\bar{\mu }\), which are not part of the state μ. When all chemical reactions are elementary (i.e., with at most two reactants), then all moments in \(\bar{\mu }\) are exactly of order k + 1.
 1.
Matchingbased methods directly attempt to match the solutions to (2) and (3) (e.g., Singh and Hespanha 2011).
 2.
Distributionbased methods construct \(\varphi (\cdot )\) by making reasonable assumptions on the statistical distribution of the molecule counts vector x (e.g., GomezUribe and Verghese 2007).
 3.
Large volume methods construct \(\varphi (\cdot )\) by assuming that reactions take place on a large volume (e.g., Van Kampen 2001).
It is important to emphasize that this classification is about methods to construct moment closure. It turns out that sometimes different methods lead to the same moment closure function \(\varphi (\cdot )\).
Conclusion and Outlook
We have introduced complementary approaches to study the evolution of biochemical networks that exhibit important stochastic effects.
Stochastic simulations permit the construction of sample paths for the molecule counts, which can be averaged to study the ensemble behavior of the system. This type of approach scales well with the number of molecular species, but can be computationally very intensive when the number of reactions is very large. This challenge has led to the development of approximate stochastic simulation algorithms that attempt to simulate multiple reactions in the same simulation step (e.g., Rathinam et al. 2003).
Solving the CME provides the most detailed and accurate approach to characterize the ensemble properties of the molecular counts, but for most biochemical systems such solution cannot be found in closed form, and numerical methods scale exponentially with the number of species. This challenge has led to the development of algorithms that compute approximate solutions to the CME, e.g., by aggregating states with low probability, while keeping track of the error (e.g., Munsky and Khammash 2006).
Moment dynamics is attractive in that the number of kthorder moments only scales polynomially with the number of chemical species, but one only obtains closed dynamics for very simple biochemical networks. This limitation has led to the development of moment closure techniques to approximate the open moment dynamics by a closed system of ordinary differential equations.
Bibliography
 Ethier SN, Kurtz TG (1986) Markov processes: characterization and convergence. Wiley series in probability and mathematical statistics: probability and mathematical statistics. Hoboken, New Jersey. See http://books.google.com/books?id=zvE9RFouKoMC&printsec=frontcover
 Gillespie DT (1976) A general method for numericallysimulating the stochastic time evolution of coupled chemical reactions. J Comput Phys 22:403–434CrossRefMathSciNetGoogle Scholar
 GomezUribe CA, Verghese GC (2007) Mass fluctuation kinetics: capturing stochastic effects in systems of chemical reactions through coupled meanvariance computations. J Chem Phys 126(2):024109–024109–12Google Scholar
 Munsky B, Khammash M (2006) The finite state projection algorithm for the solution of the chemical master equation. J Chem Phys 124:044104CrossRefGoogle Scholar
 Rathinam M, Petzold LR, Cao Y, Gillespie DT (2003) Stiffness in stochastic chemically reacting systems: the implicit tauleaping method. J Chem Phys 119(24):12784–12794CrossRefGoogle Scholar
 Singh A, Hespanha JP (2011) Approximate moment dynamics for chemically reacting systems. IEEE Trans Autom Control 56(2):414–418CrossRefMathSciNetGoogle Scholar
 Van Kampen NG (2001) Stochastic processes in physics and chemistry. Elsevier Science, AmsterdamzbMATHGoogle Scholar