Reference Work Entry

Encyclopedia of Systems Biology

pp 396-399

# Chemical Master Equation

• Hao GeAffiliated withSchool of Mathematical Sciences and Centre for Computational Systems Biology, Fudan University Email author
• , Hong QianAffiliated withDepartment of Applied Mathematics, University of Washington

## Definition

Chemical master equation is the stochastic counterpart of the chemical kinetic equation based on the law of mass action. It describes the kinetics of chemical reactions in a rapidly stirred tank with small volume in terms of stochastic reaction times giving rise to fluctuating copy numbers of reaction species.

Consider a system of fixed volume V at constant temperature T. Let there be well-stirred mixture of N ≥ 1 molecular species {S1, …,SN} and M ≥ 1 reactions {R1, …, RM}. One specifies the dynamical state of this system by X(t) = (X1(t), …, XN(t)), where Xi(t) is the copy number of molecular species Si in the system at time t.

One describes the time evolution of X(t) from some given initial state X(t0) = x0. Both single-molecule experimental measurements and theoretical investigations have shown that X(t) is a stochastic process because the time at which a particular reaction occurs is random.

The chemical master equation kinetics assumes that the system is well stirred such that at any moment each reaction occurs with equal probability at any position in space. Furthermore, it assumes that for each reaction Rj, there is a corresponding rate function rj and a stoichiometry vector vj = (vj1, …, vjN), which are defined as
$$\begin{array}{ll} {{\text{r}}_{\text{j}}}\left( {\text{x}} \right){\text{dt}} = {\text{the probability}}, {\text{ given X}}\left( {\text{t}} \right) = {\text{x}}, \\ {\text{ that one reaction }}{{\text{R}}_{\text{j }}} {\text{will occur some where}} \\{\text{in the next infinitestimal time interval}}\ \left[ {{\text{t}},{\text{ t}} + {\text{dt}}} \right),\\ { }({\text{j}} = {1}, \cdots, {\text{M}}). \\ \end{array}$$
(1)
and
$$\begin{array}{ll} {{\text{v}}_{\text{ji}}} = {\text{the change in the number of }}{{\text{S}}_{\text{i}}} {\text{ molecules }} \\ {\text {caused by one }}{{\text{R}}_{\text{j}}} {\text{ reaction}}, ({\text{j}} = {1}, \cdots, {\text{M}};{\text{ i}} = {1}, \cdots, {\text{ N}}). \\ \end{array}$$
(2)

The stoichiometry vector {vji} can be obtained from the difference between the numbers of a molecular species that are consumed and produced in the reaction.

Exact description of the rate function associates with a reaction that can be either from phenomenological models for stochastic chemical kinetics or from more fundamental molecular physics based on the concept of elementary reactions. In general, the function rj has the mathematical form:
$${{\text{r}}_{\text{j}}}\left( {\text{x}} \right) = {{\text{k}}_{\text{j}}}{{\text{h}}_{\text{j}}}\left( {\text{x}} \right).$$
(3)

Here kj is the specific probability rate constant for reaction Rj, which is defined such that kjdt is the probability that a randomly chosen combination of the reactants of Rj will react accordingly in the next infinitesimal time interval dt. kj is intimately related to the rate constants in the traditional law of mass action kinetics.

The function hj(x) in Eq. 3 measures the number of distinct combinations of Rj reactant molecules available in the state x. It is a combinatorial factor that can be easily obtained from the reaction Rj, i.e., $${{\mathbf {h_j}(\rm x)}} = {\prod\nolimits_{{k = 1}}^N {\frac{{{x_k}!}}{{{m_{{jk}}}!({x_k} - {m_{{jk}}})!{^.}}}}}$$. In the transitional mass-action kinetics, this term is related to the product of the concentrations of all reactants.

In general, for a chemical reaction
\eqalign{ {{\text{R}}_{\text{j}}}:{{\text{m}}_{\text{j1}}}{{\text{S}}_{{1}}} + \cdots + {{\text{m}}_{\text{jN}}}{{\text{S}}_{\text{N}}}{ } \to { }{{\text{n}}_{\text{j1}}}{{\text{S}}_{{1}}} + \cdots + { }{{\text{n}}_{\text{jN}}}{{\text{S}}_{\text{N}}}, }
(4)
one has
$$\begin{array}{ll} \qquad \qquad \quad {{\text{v}}_{\text{ji}}} = {{\text{n}}_{\text{ji}}} -{{\text{m}}_{\text{ji}}}, \\ {{\rm r_j}(\rm x)} = {{\rm k_j}\prod\limits_{{\rm k = 1}}^{\rm N} {\frac{{{\rm x_k}!}}{{{\rm m_{{jk}}}!({\rm x_k} - {\rm m_{{jk}}})!}}}} \\ \end{array}$$
(5)
If for any k and j, have xk >> mjk, then approximately
$${{\text{r}}_{\text{j}}}\left( {\text{x}} \right) = {k_j}\prod\limits_{{k = 1}}^N {\frac{{{x_k}^{{{m_{{jk}}}}}}}{{{m_{{jk}}}!}}}$$
(6)
Then
$$\frac{{{{\rm r_j}(\rm x)}}}{{\bf V}} = {\left( {\frac{{{\rm k_j}{\rm V^{{n - 1}}}}}{{\prod \limits_{{k = 1}}^N {{m_{{jk}}}!} }}} \right)}\,\prod \limits_{{k = 1}}^N {\,{{\left( {\frac{{{x_k}}}{V}} \right)}^{{{m_{{jk}}}}}}},\;n = \sum\limits_{{k = 1}}^N {{m_{{jk}}}}.$$
(7)
is precisely the reaction flux per unit volume in the mass-action kinetics; the (x k /V) is the concentration of species k, and the term in the parenthesis, $$k_j^{\prime} = \frac{{{k_j}{V^{{n - 1}}}}}{{\prod\limits_{{k = 1}}^N {{m_{{jk}}}!} }}$$, is the reaction rate constant in the traditional chemical kinetics. Here, kj is regarded as “stochastic reaction constant,” while $$k_j^{\prime}$$ is the corresponding reaction constant for Law of Mass Action.

The reaction rate constant is deduced only from experiments, or calculated based on Kramers-Marcus theory. It is usually a function of temperature but is independent of the volume of a reaction system. However, the stochastic reaction constant k j depends on the system volume and the temperature.

From the rate function given from above, the state vector X(t) is a Markov jump process on the nonnegative N-dimensional integer lattice.

Any Markov process can be described by two completely different, complementary mathematical methods. One follows its stochastic trajectories and one consider its probability distribution function changing with time. The Gillespie algorithm gives the former, and the chemical master equation is for the latter.

In the chemical master equation perspective, one no longer asks what are the copy number (or concentration) of species i at time t, but rather what is the probability of the system having xi copies of species Xi at time t.

We focus on the probability
$${\text{P}}\left( {{\text{x}},{\text{t}}} \right) = { \Pr }\left\{ {{\text{X}}\left( {\text{t}} \right){ } = {\text{ x}}} \right\},$$
(8)
and now derive its evolutionary equations, i.e., the chemical master equation.
We take a time increment dt and consider the variation between the probability of X(t) = x and of X(t + dt) = x. This variation is
$$\begin{array}{ll} {\text{P}}\left( {{\text{x}},{\text{t }} +{\text{ dt}}} \right) - {\text{P}}\left( {{\text{x}},{\text{ t}}}\right) = {\text{Increasing of the probability in dt}} \\ \qquad \qquad \qquad \qquad \qquad \qquad - {\text{Decreasing of the probability in dt}}\end{array}$$
(9)
We take dt so small such that the probability of having two or more reactions in dt is negligible compared to the probability for only one reaction. Then increasing of the probability in dt occurs when a system with state X(t) = x − vj reacts according to Rj in (t, t + dt), the probability of which is rj(x − vj)dt. Thus,
$${\text{Increasing of the probability in dt }} = \sum\limits_{{j = 1}}^M {{r_j}(x - {v_j})P(x - {v_j},t)} \,dt.$$
(10)
Similarly, when a system with state X(t) = x reacts according to any reaction channel Rj in (t, t + dt), the probability P(x, t) will decrease. Thus,
$${\text{Decreasing of the probability in dt}} = \sum\limits_{{j = 1}}^M {{r_j}(x)P(x,t)} \,dt.$$
(11)
Substituting Eqs. 10 and 11 into Eq. 9, we obtain
$$\begin{array}{ll} {\text{P}}\left( {{\text{x}},{\text{t}} +{\text{dt}}} \right) - {\text{P}}\left( {{\text{x}},{\text{ t}}}\right) =\sum\limits_{{j = 1}}^M {{r_j}(x - {v_j})P(x - {v_j},t)}\,dt \\ \qquad \qquad \qquad \qquad \qquad \quad - \sum\limits_{{j =1}}^M {{r_j}(x)P(x,t)} \,dt,\end{array}$$
(12)
which yields, with the limit dt → 0, the chemical master equation:
$$\begin{array}{ll} \frac{\partial }{{\partial t}}P(x,t) = \sum\limits_{{j = 1}}^M {{r_j}(x - {v_j})P(x - {v_j},t)\,} dt \\ \qquad \qquad \quad - \sum\limits_{{j = 1}}^M {{r_j}(x)P(x,t)} \,dt. \end{array}$$
(13)

## Characteristics

### Consistent with Law of Mass Action

The relationship between deterministic law of mass action and stochastic chemical master equation for chemical reactions was established by T. Kurtz in a general theory in which a stochastic Markov chain model for general chemical reaction is studied alongside its deterministic counterpart. Recall that both Gillespie algorithm and the chemical master equation are two different descriptions of the same stochastic, jump process. Let XV(t) denote this stochastic process, where V is the volume of the system, and the initial condition (in the thermodynamic limit) is
$$\mathop{{\lim }}\limits_{{V \to \infty }} \frac{{{X^V}(0)}}{V} = {x_0}$$
(14)
Then, the solution to the initial value problem of the corresponding chemical kinetics based on the law of mass action model is denoted by c(t,x0). Kurtz has shown that the relationship between these two solutions is
$$\begin{array}{ll} \mathop{{\lim }}\limits_{{V \to \infty }} \Pr \left\{ {\mathop{{\sup }}\limits_{{s \leq T}} \left| {\frac{{{X^V}(t)}}{V} - c(t,{x_0})} \right|> \epsilon } \right\} = 0, \\ {\text{ for every T and }}\epsilon> 0, \end{array}$$
(15)

It is important to note the mathematical subtlety of $$s \leq T$$. The above result is only valid for finite time interval, even though it can be as long as one likes: The ε is a function of T; greater the T, larger the ε.

Compare the solution to the chemical master equation with initial distribution concentrated at x 0 , P(x,t|x 0) and the deterministic kinetics c(t,x 0); if the latter approaches to a steady-state value then the former cumulates probability at the steady state. In general, there is an agreement between the steady states of the deterministic kinetics and the peaks of the stationary distribution of the chemical master equation.

However, in the limit of infinitely large volume, the situation is quite different: The peaks of the stationary distribution of chemical master equation may not be consistent with the stable fixed points of the mass-action kinetics. This is exactly due to the $$s \leq T$$ in Eq. 15. This can be explained by a theory of multiple timescales.

### Numerical Simulation: Gillespie Algorithm

We now introduce stochastic simulation methods that generate the random trajectories of X(t). Once we have enough sample trajectories of a stochastic process, we will be able to calculate the probability distribution function P(x, t) and all other statistical behaviors including the mean trajectory, variances, and correlations.

The stochastic simulation algorithm (SSA) (also known as the Gillespie algorithm) is based on the following next-reaction probability distribution: assume the system is in state x at time t, and let
$$\begin{array}{ll} p(\tau, j|x,t) = {\text{probability that}}, {\text{ given X}}\left( {\text{t}} \right) = {\text{x}},\\ {\text{ the next reaction will occur in the}} \\{\text{infinitesimal time interval }}[{\text{t}} + \tau, {\text{t}} + \tau + {\text{d}}\tau ),\\ {\text{ and will be the }}{{\text{R}}_{\text{j}}}{\text{\ reaction}}. \\ \end{array}$$
(16)
In probability theory, there is an elementary theorem, which states that
\eqalign{ p(\tau, j|x,t) = {r_j}(x)\exp ( - {r_0}(x)\tau ),\;0 \leq \tau < \infty, j = 1,...,M }
(17)
where
$${r_0}(x) = \sum\limits_{{k = 1}}^M {{r_k}(x)}$$
(18)
The key step of this simulation method is to generate the pair of numbers ($$\tau$$, j) in accordance with the probability Eq. 18: First generate two random numbers a1 and a2 from the uniform distribution in the unit interval, and then take
$$\tau = \frac{{ - \ln ({a_1})}}{{{r_0}(x)}},$$
$${\text{j}} = {\text{the smallest integer satisfying }}\sum\limits_{{i = 1}}^j {{r_i}(x)}> {a_2}{r_0}(x)$$
(19)
Below are main steps in the stochastic simulation algorithm.
1. 1.

Initialization: Let the initial state as X = X0, and t = t0.

2. 2.

Simulation: Generate a pair of random numbers ($$\tau$$, j) according to the probability density function (17).

3. 3.

Update: Increase the time by $$\tau$$, and replace the molecule numbers by X + vj.

4. 4.

Iterate: Go back to Step 2 unless the end of the simulation procedure.

The exact stochastic simulation algorithm consists with the chemical master equation and gives an exact sample trajectory of the real system.