Continuous-time Markov-chain models for reaction systems: fast and slow processes

Abstract

It is sometimes of interest to identify the fast and slow processes in a reaction system. We present here an approach to this problem which is based on a simple stochastic model, a continuous-time Markov chain on a small number of states. We show how it is possible to use such a stochastic model to find and plot the time-courses of concentrations, and to find simple short-time and long-time approximations to these time-courses; that is, to separate the fast and the slow processes. The most significant computation involved is the exponentiation of many small matrices, which is easily accomplished in the computing environment R.

Introduction

In the modelling of chemical and other reaction systems, there are essentially two alternatives: deterministic models and stochastic (i.e. probabilistic) ones. The former type enables one to draw conclusions about (e.g.) concentrations from a system of ordinary differential equations, the rate equations, the latter typically seeks to answer similar questions by finding transition probabilities in a continuous-time Markov chain representing the reactions.

To many chemists and other physical scientists, deterministic models and their associated differential equations are probably the more familiar approach, but that is not necessarily the simpler route to a useful conclusion, merely the more familiar of the two. Similarly, those of a statistical background would tend to choose stochastic models. The books of Érdi and Tóth [4] and Érdi and Lente [3] cover (respectively) both deterministic and stochastic models, and stochastic models only.

One sometimes sees authors writing as if the system of rate equations IS the system of chemical reactions, as opposed to one very useful representation thereof; we prefer to distinguish the two systems. For example, there are literature references to Robertson [12] which cite that work as the source of the ‘Robertson reaction’. There are no reactions at all in [12], but there is discussed in that work a system of differential equations (DEs) that could very well arise as the rate equations implied by a system of chemical reactions. Furthermore, it is in our view useful to have at one’s disposal both deterministic and stochastic modelling. There may be insights readily available from one of the techniques that are not as readily available from the other. See, for instance, MacDonald [10, pp. 3,4], in which continuous-time Markov chains are used (instead of the rate equations) as a simple route to clear-cut conclusions about a unimolecular triangle reaction system.

Alvarez-Ramirez et al. [1] use the rate equations, the singular value decomposition (SVD) of a certain matrix arising in those equations, and singular perturbation theory in order to separate the fast and slow processes in a reaction system. We present here an alternative formulation of their worked examples which demonstrates how continuous-time Markov chains (CTMCs) can instead be used to study the time-courses of the concentrations of the species involved, and thereby draw conclusions about the fast and slow processes in the system. It is to some extent possible to derive implied fast or slow manifolds, but examination of these time-courses is also informative, and apparently easier to carry out.

Our approach is essentially that described by MacDonald [10]; states are defined, the generator matrix of the (time-homogeneous) CTMC representing the system is identified, and the generator is then exponentiated in order to find the transition probabilities, which can deduce the vector of concentrations at a given time from the initial concentrations. The time-courses of the concentrations can thereby be computed and plotted.

Kinetics of three-lump hydrocracking

The reaction system

The first example of Alvarez-Ramirez et al. [1] relates to the kinetics of three-lump hydrocracking (HDC). The system, consisting of three monomolecular reactions, is as follows:

$$\begin{aligned} \begin{array}{lcr} \mathrm R&{} \overset{k_1}{\longrightarrow }\ &{} \mathrm L\\ \mathrm L&{} \overset{k_3}{\longrightarrow }\ &{} \mathrm G\\ \mathrm R&{} \overset{k_2}{\longrightarrow }\ &{} \mathrm G, \end{array} \end{aligned}$$

and can be depicted as in Fig. 1.

Fig. 1

Three-lump hydrocracking reaction system

The symbol R denotes heavy oil residue, and L and G are the liquid and the gas products. (In passing: the use of $k_3$, not $k_2$, in the second reaction above is intentional.)

As noted by Alvarez-Ramirez et al., the system of ordinary differential equations (ODEs) representing the reaction system is:

$$\begin{aligned} \frac{\textrm{d}C_R(t)}{\textrm{d}t}= & {} -(k_1+k_2) C_R(t) \nonumber \\ \frac{\textrm{d}C_L(t)}{\textrm{d}t}= & {} k_1 C_R(t) - k_3 C_L(t) \nonumber \\ \frac{\textrm{d}C_G(t)}{\textrm{d}t}= & {} k_2 C_R(t) + k_3 C_L(t). \end{aligned}$$

(1)

The linearity and triangular structure of the Eq. 1 make it easy to solve them explicitly,¹ but we take a slightly different route.

In view of the explicit solution, the approximations based on the SVD and singular perturbation may seem redundant, but the implications of such approximations can if necessary be compared to those of the exact solution. In the example presented by Alvarez-Ramirez et al., the values of the rate constants (per hour) are: $k_1= 0.2787$, $k_2=0.0361$, $k_3 = 0.0101$; these values arise in the work of Soto-Azuara et al. [13].

The Markov-chain model

With the states being R, L and G, the generator of the associated CTMC is

$$\begin{aligned} \varvec{Q} = \begin{pmatrix} -(k_1+k_2) &{} k_1&{} k_2\\ 0 &{}-k_3 &{} k_3 \\ 0 &{} 0 &{} 0 \end{pmatrix}. \end{aligned}$$

(We use the convention that the row sums of such a matrix are zero.) The last row reflects the fact that G is an absorbing state. Since $\varvec{Q}$ is upper-triangular, its eigenvalues are $-(k_1+k_2)$, $-k_3$ and 0. Apart from certain obvious special cases (which we exclude here), these eigenvalues are distinct and the generator therefore diagonalizable. A matrix of right (i.e. column) eigenvectors, with the same ordering, is

$$\begin{aligned} \varvec{U} = \begin{pmatrix} 1 &{} k_1 &{} 1\\ 0 &{} (k_1+k_2-k_3) &{} 1\\ 0 &{} 0 &{} 1 \end{pmatrix}, \end{aligned}$$

the inverse of which can be found explicitly:

$$\begin{aligned} \varvec{U}^{-1} = \frac{1}{k_1+k_2-k_3} \begin{pmatrix} (k_1+k_2-k_3)\ \ &{} -k_1\ \ &{} -(k_2-k_3)\\ 0 &{} 1 &{} -1\\ 0 &{} 0 &{} (k_1+k_2-k_3) \end{pmatrix}. \end{aligned}$$

One can therefore find the required transition probability matrix (TPM) $\varvec{P}(t) = \exp (t\varvec{Q})$ explicitly as

$$\begin{aligned} \varvec{P}(t) = \textit{\textbf{U D}}_t \varvec{U}^{-1}, \end{aligned}$$

where $\varvec{D}_t$ is the diagonal matrix with diagonal $(e^{-(k_1+k_2)t}, e^{-k_3 t}, 1 )$. In any case, it follows that each transition probability $p_{ij}(t)$ is of the form

$$\begin{aligned} a e^{-(k_1+k_2)t} + b e^{-k_3 t} +c, \end{aligned}$$

where a, b and c are independent of t. For instance,

$$\begin{aligned} p_{11}(t)=\, & {} e^{-(k_1+k_2)t}, \nonumber \\ p_{12}(t)=\, & {} k_1 (e^{-k_3t} - e^{-(k_1+k_2)t})/(k_1+k_2-k_3), \nonumber \\ p_{13}(t)=\, & {} 1 - p_{11}(t) - p_{12}(t), \end{aligned}$$

(2)

the second of which peaks at time $t= \frac{1}{k_1+k_2-k_3} \log \left( \frac{k_1+k_2}{k_3} \right)$.

The (relative) concentrations at time t, adding to 1 and corresponding to the three states, are supplied by the row vector

$$\begin{aligned} \varvec{u}(t) = \varvec{u}(0)\varvec{P}(t), \end{aligned}$$

where $\varvec{P}(t)$ is given either by the explicit expression derived above, or numerically by $\varvec{P}(t)=\exp (t\varvec{Q}),$ the matrix exponential being evaluated by (e.g.) the excellent R routine expm [5]. (If a matrix is diagonalizable, it is easily exponentiated; expm does not assume diagonalizability.) For ease of exposition we shall assume here that $\varvec{u} (0) = (1,0,0)$, but that assumption is inessential. The concentrations at time t are then

$$\begin{aligned} (u_1(t), u_2(t), u_3(t))= (1,0,0) \varvec{P}(t)= (p_{11}(t), p_{12}(t), p_{13}(t)), \end{aligned}$$

with the transition probabilities $p_{1j}(t)$ as given by (2). Differences or sums of concentrations might well be of interest, and are of course easily deduced.

Fast and slow processes

Since we have simple explicit expressions for the concentrations, we can directly investigate their approximate behaviour for small t and for large t, and thereby possibly find fast and slow ‘manifolds’. Firstly, note that, if $(k_1+k_2)t$ and $k_3 t$ are sufficiently small,

$$\begin{aligned} 1-u_1(t) \approx (k_1+k_2)t \end{aligned}$$

and

$$\begin{aligned} u_2(t) \approx \frac{k_1}{k_1+k_2-k_3} \left( -k_3t + (k_1+k_2)t \right) = k_1 t. \end{aligned}$$

Hence

$$\begin{aligned} u_2(t) / (1-u_1(t)) \approx \frac{k_1}{k_1+k_2} = 0.8853. \end{aligned}$$

This identifies a fast manifold which is almost exactly in agreement with that given by Eq. 21 of Alvarez-Ramirez et al.:

$$\begin{aligned} 0 = 0.8859(C_{R,0} -C_R) -C_L. \end{aligned}$$

It is obviously less in agreement with their Eq. 33:

$$\begin{aligned} 0 = 0.9147(C_{R,0} -C_R) -C_L. \end{aligned}$$

If one makes the assumption (only) that $k_3=0$, then it follows that (exactly)

$$\begin{aligned} u_2(t) / (1-u_1(t)) = \frac{k_1}{k_1+k_2} = 0.8853; \end{aligned}$$

so the fast manifold discussed above is an exact consequence of the assumption that $k_3=0$.

Secondly, note that, if $(k_1+k_2)t$ is sufficiently large, $u_1(t) \approx 0$ and so $u_2(t) + u_3(t) \approx 1$, which identifies a slow manifold. But the search for fast and slow manifolds does not seem to add much information if it is easily possible to plot $\varvec{u}(t)$ for a given generator $\varvec{Q}$ and initial distribution $\varvec{u}(0)$.

If (as in this example) $k_3$ is less than $k_2$ and very much less than $k_1$, it is clear that conversion to gas is slow, as seen in Fig. 2, which can be compared with Fig. 1 of Alvarez-Ramirez et al.

Fig. 2

Three-lump hydrocracking example of Alvarez-Ramirez et al.: time-courses of concentrations of R (continuous black line), L (dashed red line) and G (dotted blue line) if the initial concentrations are (1,0,0). The vertical line is at the (log of the) reciprocal of the larger singular value of the top-left $2\times 2$ block of the generator, and is close to the time (2.42) at which the concentrations of R and L are equal. (Compare with Fig. 1 of Alvarez-Ramirez et al.) This and other plots were produced by using the R routine expm

Generator lacking distinct eigenvalues

It has been suggested that we should not confine ourselves here to the case of the generator $\varvec{Q}$ having distinct eigenvalues. As pointed out by Hirsch et al. [6, p. 100], ‘most’ matrices have distinct eigenvalues. Hence ‘most’ (square) matrices are diagonalizable. More precisely, the set of real $n\times n$ matrices that have n distinct eigenvalues is (open and) dense in the set of all real $n\times n$ matrices [6, p. 101]. The case of $\varvec{Q}$ being deficient in that respect is therefore of theoretical rather than practical significance. To proceed with a theoretical analysis, one would need to write the generator in Jordan form or Schur triangular form; for the latter, see Meyer [11, p. 524]. But there would seem to be little to be gained by doing so. If such a deficient generator $\varvec{Q}$ were to turn out to be of interest, it could of course be nondiagonalizable, but that would not prevent one from exponentiating $t\varvec{Q}$ with appropriate software, e.g. expm in R.

Kinetics of four-lump hydrocracking

In addition to three-lump hydrocracking, Alvarez-Ramirez et al. also discuss more briefly the four-lump HDC model of Soto-Azuara et al [13]. Here the four states are (in order) R, HLP (heavy liquid product), LLP (light liquid product) and G, and the generator of our CTMC model is

$$\begin{aligned} \varvec{Q} = \begin{pmatrix} - (k_1+k_2+k_3) &{} k_1&{} k_2 &{} k_3\\ 0 &{}-(k_4+k_5) &{} k_4 &{} k_5\\ 0 &{} 0 &{} -k_6 &{} k_6 \\ 0 &{} 0 &{} 0 &{} 0 \end{pmatrix}. \end{aligned}$$

Its eigenvalues are $-(k_1+k_2+k_3)$, $-(k_4+k_5)$, $-k_6$ and 0. Again, apart from certain special cases the eigenvalues are distinct and the generator therefore diagonalizable. But in the example considered, one of those special cases does indeed arise, since $k_4=k_5=0$. Zero is then a repeated eigenvalue, and states 2 (HLP) and 4 (G) are both absorbing states. Alvarez-Ramirez et al. describe the resulting model as ‘physically meaningless’, but it seems worthwhile to explore the consequences, for the CTMC, of the assumption that $k_4=k_5=0$.

The full set of rate constants considered by Soto-Azuara et al. and Alvarez-Ramirez et al. is: $\varvec{k} =(0.2323, 0.0487, 0.0336, 0, 0, 0.1244)$. The two absorbing states give rise to an infinity of equilibrium distributions (all convex linear combinations of the two row vectors (0, 1, 0, 0) and (0, 0, 0, 1)). But from the structure of the resulting network it seems plausible that, for a given initial distribution $\varvec{u}(0)$, there is a unique limiting distribution.

That conjecture can be proved as follows. In spite of the repeated eigenvalue (zero) the generator $\varvec{Q}$ can be diagonalized: two linearly independent right eigenvectors, both corresponding to the eigenvalue zero, are $(k_1/(k_1+k_2+k_3), 1,0,0)^{\textrm{T}}$ and $(1,1,1,1)^{\textrm{T}}$. More simply, perhaps: two linearly independent left eigenvectors, corresponding to the eigenvalue zero, are (0, 1, 0, 0) and (0, 0, 0, 1). Hence, for some nonsingular matrix $\varvec{U}$,

$$\begin{aligned} \varvec{P}(t)= & {} \varvec{U} \text{ diag }(e^{-(k_1+k_2+k_3)t}, 1, e^{-k_6 t}, 1) \varvec{U}^{-1}\\\overset{t \rightarrow \infty }{\longrightarrow } & {} \varvec{U} \text{ diag }(0, 1, 0, 1) \varvec{U}^{-1}, \end{aligned}$$

provided that $k_1+k_2+k_3$ and $k_6$ are strictly positive. It then follows that

$$\begin{aligned} \varvec{u}(t) = \varvec{u}(0) \varvec{P}(t) \overset{t \rightarrow \infty }{\longrightarrow }\ \varvec{u}(0) \varvec{U} \text{ diag }(0, 1, 0, 1) \varvec{U}^{-1}. \end{aligned}$$

The model may indeed be physically meaningless, in that it precludes conversion of HLP to LLP or G, but its properties are not difficult to establish. Fig. 3 displays the time-courses of the concentrations of the four species corresponding to $\varvec{k}$ as given above and initial concentrations $\varvec{u}(0) = (1,0,0,0)$.

Fig. 3

Four-lump hydrocracking example of Alvarez-Ramirez et al.: time-courses of concentrations of R (continuous black line), HLP (dashed, red), LLP (dotted, blue) and G (dashed and dotted, green) if $\varvec{k} =(0.2323, 0.0487, 0.0336, 0, 0, 0.1244)$ and the initial concentrations are (1,0,0,0). Note the nonzero limiting concentrations for both HLP and G, adding to 1

A catalytic reaction system

The reaction system

The other example of Alvarez-Ramirez et al. is the following system:

$$\begin{aligned} \begin{array}{lcl} \textrm{A}_1 + \textrm{A}_2 &{}\overset{k_1}{\longrightarrow }\ &{} \textrm{A}_3 \\ \textrm{A}_3 &{}\overset{k_2}{\longrightarrow }\ {} &{} \textrm{A}_1 + \textrm{A}_2\\ \textrm{A}_3 &{}\overset{k_3}{\longrightarrow }\ &{} \textrm{A}_2 + \textrm{A}_4, \end{array} \end{aligned}$$

which can also be displayed as follows:

$$\begin{aligned} \textrm{A}_1 + \textrm{A}_2 \overset{k_1}{\underset{k_2}{\rightleftharpoons }} \textrm{A}_3 \overset{k_3}{\longrightarrow }\ \textrm{A}_2 + \textrm{A}_4. \end{aligned}$$

Alvarez-Ramirez et al. give the four ODEs for the four concentrations $C_i= [\textrm{A}_i]$. In the second one (their Eq. 44b) there appears to be a typographical error; the term $-k_2C_3$ should apparently be $+k_2C_3$. Note in particular that three of these four rate equations (Eqs. 44a–44c) involve a nonlinear term; the system of rate equations is nonlinear.

As Alvarez-Ramirez et al. note, the traditional Michaelis–Menten enzymatic reaction scheme is an example of such a system, with the species $\textrm{A}_1$–$\textrm{A}_4$ being (respectively) the substrate, enzyme, intermediate complex (‘adduct’) and product. (But advances in single-molecule experimental technology suggest that the Michaelis–Menten scheme may, at least in some cases, be an oversimplification of such reactions: see Kou et al. [8] and Kou [7]. Kou [7] describes a very detailed CTMC model for the case of the enzyme $\beta$-galactosidase.)

The Markov-chain model

In the case of this system it is less obvious than in the preceding examples how one should define the states of a CTMC. We explore here a simple three-state CTMC involving the four species, and investigate what useful conclusions can be drawn from it, including conclusions relating to fast and slow processes. We define the following three states:

• state 1, (1,1,0,0), that is, one molecule of $\textrm{A}_1$ and one of $\textrm{A}_2$;

• state 2, (0,0,1,0), one molecule of $\textrm{A}_3$;

• state 3, (0,1,0,1), one molecule each of $\textrm{A}_2$ and $\textrm{A}_4$.

Unless one is considering only first-order reactions, one cannot assume that the transition rates of the CTMC are in the same units as are the rate constants of the deterministic model, but for notational convenience we shall also use the symbols $k_i$ for the three positive transition rates in the CTMC. The generator is then

$$\begin{aligned} \varvec{Q} = \begin{pmatrix} -k_1 &{} k_1&{} 0\\ k_2 &{}-(k_2+k_3) &{} k_3 \\ 0 &{} 0 &{} 0 \end{pmatrix}. \end{aligned}$$

(3)

We assume (unless it is otherwise indicated) that all the $k_i$s are strictly positive.

The model is a random process that is in exactly one of the three states at any given time, although once it reaches state 3 it remains there. It is perhaps worth stressing that the three probabilities $u_j(t)$ add to 1, unlike the four concentrations $C_i(t)$ of Alvarez-Ramirez et al.: note (e.g.) the illustrative values $C_{1,0}=\, 3$ and $C_{2,0}=\, 2$ that they use. In our model, the relation between the three probabilities $u_j(t)$ and the four concentrations $[\textrm{A}_i]$ is as follows:

$$\begin{aligned} u_1(t)=\, & {} [\textrm{A}_1] \\ u_2(t)=\, & {} [\textrm{A}_3] \\ u_3(t)=\, & {} [\textrm{A}_4] \\ 1-u_2(t) =\, u_1(t)+u_3(t)=\, & {} [\textrm{A}_2]. \end{aligned}$$

The generator used here is not triangular; but it is tridiagonal, which implies detailed balance. Its nonzero eigenvalues are

$$\begin{aligned} \frac{1}{2}\left( -\sum _{i=1}^3 k_i \pm \sqrt{\left( \sum _{i=1}^3 k_i\right) ^2 - 4 k_1 k_3}\right) , \end{aligned}$$

which are real (and negative). Denote these eigenvalues by $\lambda _1$ and $\lambda _2$, with $\lambda _1 < \lambda _2$. The following is a matrix of right eigenvectors corresponding to (respectively) $\lambda _1$, $\lambda _2$ and zero:

$$\begin{aligned} \varvec{U} = \begin{pmatrix} 1 &{} 1 &{}1 \\ 1+ \lambda _1/k_1 \ &{} 1+ \lambda _2/k_1 \ &{} 1 \\ 0 &{} 0&{}1 \end{pmatrix}; \end{aligned}$$

its inverse is:

$$\begin{aligned} \varvec{U}^{-1} = \frac{1}{\lambda _2 - \lambda _1} \begin{pmatrix} k_1+\lambda _2 &{} -k_1 &{} -\lambda _2 \\ -(k_1+\lambda _1 ) &{} k_1 &{} \lambda _1 \\ 0 &{} 0&{} \lambda _2-\lambda _1 \end{pmatrix}. \end{aligned}$$

(The last row is the equilibrium distribution (0,0,1), a left eigenvector corresponding to eigenvalue zero.) The TPM $\varvec{P}(t)$ can then be found explicitly by diagonalization. For instance:

$$\begin{aligned} p_{11}(t)= & {} \frac{1}{\lambda _2-\lambda _1} \left( (k_1+ \lambda _2) e^{\lambda _1 t}- (k_1+\lambda _1) e^{\lambda _2 t } \right) , \nonumber \\ p_{12}(t)= & {} \frac{1}{\lambda _2-\lambda _1} \left( -k_1 e^{\lambda _1 t} +k_1 e^{\lambda _2 t}\right) , \nonumber \\ p_{13}(t)= & {} 1 - p_{11}(t) - p_{12}(t) = \frac{1}{\lambda _2-\lambda _1} \left( - \lambda _2 e^{\lambda _1 t} + \lambda _1 e^{\lambda _2 t } + \lambda _2-\lambda _1 \right) . \end{aligned}$$

(4)

(Notice that $p_{11}(t) + p_{12}(t) = (\lambda _2 e^{\lambda _1 t}- \lambda _1 e^{\lambda _2 t })/(\lambda _2-\lambda _1)$.) If we assume that $\varvec{u}(0)= (1,0,0)$, we therefore have explicit expressions for $u_j(t) = p_{1j}(t)$ (j=1, 2, 3). By one means or another, therefore, it is possible to compute the probabilities $u_j(t)$ corresponding to states 1, 2 and 3, for a given time t and given initial distribution $\varvec{u}(0)$.

The illustrative sets of values of rate constants considered by Alvarez-Ramirez et al. are: firstly, $\varvec{k} = (1.0, 0.2,0.2)$ (‘case I’), and secondly $\varvec{k} = (0.1, 0.2, 3.0)$ (‘case II’). We also use those values for illustrative purposes. For case I ($k_1$ dominant), Fig. 4 displays the time-courses of the concentrations $[\textrm{A}_1]$, $[\textrm{A}_3]$ and $[\textrm{A}_4]$.

Fig. 4

Catalytic reaction: time-courses of the concentrations of $\textrm{A}_1$, $\textrm{A}_3$ and $\textrm{A}_4$ in the CTMC with the generator (3) in case I, $\varvec{k} = (1.0, 0.2,0.2 )$. (In the Michaelis–Menten scheme, these three species are respectively the substrate, complex and product.)

Fig. 5

Catalytic reaction: time-courses of the concentrations of $\textrm{A}_1$, $\textrm{A}_3$ and $\textrm{A}_4$ in the CTMC with the generator (3) in case II, $\varvec{k} = (0.1, 0.2, 3.0)$

For case II ($k_3$ dominant), Fig. 5 displays the corresponding plot. There is a marked difference between the two plots as regards $[\textrm{A}_3]$, the concentration of the intermediate complex (the dashed red line). In case I, the concentration of $\hbox {A}_3$ peaks at height about 0.6 before descending to its limiting value, zero. In case II, $\hbox {A}_3$ is (as one would expect) very quickly converted to $\textrm{A}_2+\textrm{A}_4$, and its concentration never rises above very low values.

By plotting $u_2(t)$ (the concentration of $\textrm{A}_3$, the intermediate complex) against $u_1(t)$ (the concentration of $\hbox {A}_1$, the substrate) it is also possible to produce plots similar to the phase plots of Alvarez-Ramirez et al., their Figs. 3 and 4. The initial conditions differ, however; our initial condition is $\varvec{u}(0) = (1,0,0)$, i.e. (with probability 1) one molecule each of $\hbox {A}_1$ (substrate) and of $\hbox {A}_2$ (enzyme). Our Figs. 6 and 7 (respectively) are comparable to their Figs. 3 and 4, and are qualitatively similar in both cases.

Fig. 6

Catalytic reaction, case I ($k_1$ dominant). Concentration of $\textrm{A}_3$ (complex) plotted against that of $\textrm{A}_1$ (substrate). The fast- and slow-process approximations (the straight lines) are also shown. Compare with Fig. 3 of Alvarez-Ramirez et al.

Fig. 7

Catalytic reaction, case II ($k_3$ dominant). Concentration of $\textrm{A}_3$ (complex) plotted against that of $\textrm{A}_1$ (substrate). The fast- and slow-process approximations (the straight lines) are also shown. Compare with Fig. 4 of Alvarez-Ramirez et al.

Fast and slow processes

As regards identifying fast and slow processes, we explore first the consequences of the simple assumption that $k_3 = 0$ (in order to identify the fast process). If $k_3 = 0$ is assumed, it follows that $\lambda _1 = -(k_1+k_2)$, $\lambda _2=0$ and, rather obviously, that $p_{11}(t) + p_{12}(t) =1$. With the initial condition $\varvec{u}(0) = (1,0,0)$, this produces the relation $u_1(t) + u_2(t) =1$. If $k_3$ is ‘small’ (as in case I), this assumption should produce a reasonable approximation for small t, i.e. the fast dynamics. (See our Figs. 6 and 7, in which this approximation appears as a dashed black line.) But the small-t approximations of Alvarez-Ramirez et al. appear better, if more laborious, in both Case I and Case II; see their Figs. Figs. 3 and 4.

An alternative route to the same conclusion proceeds as follows. Equation 4 implies that

$$\begin{aligned} p_{11}(t) + p_{12}(t)= & {} \frac{\lambda _2 e^{\lambda _1 t} - \lambda _1 e^{\lambda _2 t} }{\lambda _2 - \lambda _1}\\= & {} 1 + O(t^2), \end{aligned}$$

hence that $p_{11}(t) + p_{12}(t) \approx 1$ for small t. This approximation could be improved by not ignoring the (rather simple) term of order $t^2$.

In order to approximate the slow dynamics, consider the ratio $p_{11}(t)/p_{12}(t)$, and the limit thereof as t tends to infinity. From Eq. 4 we have

$$\begin{aligned} \frac{p_{11}(t)}{p_{12}(t)}= & {} \frac{(k_1+ \lambda _2) e^{\lambda _1 t}- (k_1+\lambda _1) e^{\lambda _2 t }}{-k_1 e^{\lambda _1 t} +k_1 e^{\lambda _2 t}}\\= & {} \frac{1}{k_1} \frac{(k_1+ \lambda _2) e^{-(\lambda _2 - \lambda _1) t}- (k_1+\lambda _1) }{- e^{-(\lambda _2 -\lambda _1) t} + 1} \\\overset{t \rightarrow \infty }{\longrightarrow } & {} \frac{- (k_1+\lambda _1) }{k_1} \,{ (\hbox {since } \lambda _2 - \lambda _1 > 0)}. \end{aligned}$$

(Recall that $\lambda _1< \lambda _2 <0$. It can also be shown that $\lambda _1< -k_1$, so this limit is indeed positive, as expected.) With the usual assumption that $\varvec{u}(0) = (1,0,0)$, this implies that $u_2(t)/u_1(t)$ tends to $\frac{k_1}{- (k_1+\lambda _1) }$, hence the approximation, for large t, that $u_2(t)/u_1(t) \approx \frac{k_1}{- (k_1+\lambda _1) }$. This approximation was used in Figs. 6 and 7, and appears there as a dashed and dotted red line.

A variation

A variation that may be of interest is to assume that the concentration of $\hbox {A}_1$ (in the Michaelis–Menten scheme, the substrate) is constant. That is the context considered by Kou [7]. The system under discussion here then becomes:

$$\begin{aligned} \begin{array}{lcl} \textrm{A}_2 &{}\overset{k_1[\textrm{A}_1] }{\longrightarrow } &{} \textrm{A}_3 \\ \textrm{A}_3 &{}\overset{k_2}{\longrightarrow }\ {} &{} \textrm{A}_2 \\ \textrm{A}_3 &{}\overset{k_3}{\longrightarrow }\ &{} \textrm{A}_2 + \textrm{A}_4, \end{array} \end{aligned}$$

and the resulting three ODEs can be written down. The corresponding generator is

$$\begin{aligned} \varvec{Q} = \begin{pmatrix} -k_1' &{} k_1'&{} 0\\ k_2 &{}-(k_2+k_3) &{} k_3 \\ 0 &{} 0 &{} 0 \end{pmatrix}, \end{aligned}$$

with $k_1'$ defined to be $k_1 [\textrm{A}_1 ]$. The TPM $\varvec{P}(t)$, and anything based on it, e.g. probabilities of the kind given by Eq. 4, then follow.

Nonlinearity

It might seem that the use of CTMCs as models merely provides a different route, in different notation and terminology, to conclusions that are also available from the theory of linear dynamical systems. In our opinion that would not be correct. In fact, one of the examples already considered (the catalytic reaction system) involves significant nonlinearity. In that case, three of the four rate equations include a nonlinear term. The formulation of the system as a CTMC enabled us to proceed with an explicit analysis in spite of this nonlinearity.

We provide here two further examples of cases in which the rate equations are nonlinear, yet the technique of using CTMCs applies without any essential alteration.

Consider the very simple reaction $\textrm{A}_1 + \textrm{A}_2 \longrightarrow \textrm{A}_3$, with rate proportional to the product of the concentrations $[\textrm{A}_1]$ and $[\textrm{A}_2]$. That is, the reaction is of first order in both $\textrm{A}_1$ and $\textrm{A}_2$. The resulting rate equations are all nonlinear. With $C_i$ denoting $[\textrm{A}_i]$, and the prime symbol denoting differentiation with respect to time, these are, for some $k>0$: $C_1'=C_2'= -kC_1C_2$, $C_3'= +kC_1C_2$. A CTMC model for the reaction, on the two states (1,1,0) and (0,0,1), would have generator $\varvec{Q} = \begin{pmatrix} -\ell &{}\ell \\ 0&{}0 \end{pmatrix}$ for some $\ell >0$. The transition rate $\ell$ would be per unit time, the rate constant k per unit time per unit concentration. For this model, as for any (time-homogeneous) CTMC, the row vector of unconditional probabilities $\varvec{u}(t)$ is $\varvec{u}(0) \exp (t\varvec{Q})$.

A rather more complex example would be the ‘Robertson reaction’:

$$\begin{aligned} \textrm{A}_1&\overset{}{\longrightarrow }&\textrm{A}_2 \\ \textrm{A}_2 + \textrm{A}_3&\overset{}{\longrightarrow }&\textrm{A}_1 + \textrm{A}_3\\ 2 \textrm{A}_2&\overset{}{\longrightarrow }&\textrm{A}_2 + \textrm{A}_3. \end{aligned}$$

(This system of reactions is consistent with the system of three nonlinear ODEs discussed by Robertson [12], and does appear in some of the literature, e.g. Amat et al. [2].) If one adds the nonreacting species $\textrm{A}_3$ to the first reaction, the system can be written as

$$\begin{aligned} \textrm{A}_1 + \textrm{A}_3\overset{}{\underset{}{\rightleftharpoons }} \textrm{A}_2 +\textrm{A}_3 \overset{}{\longleftarrow } 2 \textrm{A}_2. \end{aligned}$$

Now take $\textrm{A}_1 + \textrm{A}_3$, $\textrm{A}_2 + \textrm{A}_3$ and $2 \textrm{A}_2$ (in that order) to be the states of a CTMC with generator of the form

$$\begin{aligned} \varvec{Q} = \begin{pmatrix} -k_1&{} k_1 &{} 0 \\ k_2 &{} -k_2 &{} 0 \\ 0 &{} k_3 &{} -k_3 \end{pmatrix}. \end{aligned}$$

That is, state 1 is one molecule of $\textrm{A}_1$ plus one of $\textrm{A}_3$, state 2 is one each of $\textrm{A}_2$ and $\textrm{A}_3$, and state 3 is two of $\textrm{A}_2$. The generator is tridiagonal, which implies detailed balance.

The probability distribution over the three states at time t is given, as usual, by the row vector $\varvec{u} (t) = \varvec{u} (0) \exp (t\varvec{Q})$. The concentrations $C_j \equiv [\textrm{A}_j]$ are given in terms of $\varvec{u}(t)$ by the following (invertible) relationship:

$$\begin{aligned} C_1(t)= u_1(t) \\ C_2(t)= u_2(t) + 2 u_3(t) \\ C_3(t)= u_1(t) + u_2(t). \end{aligned}$$

Discussion and conclusion

Fig. 8

General three-species monomolecular reaction system

Both the three-lump HDC model and (as formulated here) the catalytic reaction are special cases of the general three-species monomolecular reaction system in which some of the rates $q_{ij}$ are zero; see Fig. 8. But regardless of whether some rates are zero, the same technique applies: if concentrations $\varvec{u}(t)$ at time t are of interest, exponentiate $t\varvec{Q}$ to get the transition probabilities $\varvec{P}(t)$ and then use $\varvec{u}(t)= \varvec{u} (0) \varvec{P}(t)$.

A phenomenon that can cause difficulty, in both stochastic and deterministic models for reaction systems, is the coexistence of very fast and very slow reactions. In the deterministic case, the system of differential equations is then ‘stiff’, and special-purpose numerical software designed for stiff systems can solve the problem. In the stochastic case, one can use the technique known as uniformization (see e.g. van Dijk et al. [14]). The basis of that technique is that the transition probability matrix $\varvec{P}(t) = \exp (t\varvec{Q})$ (surprisingly?) is exactly

$$\begin{aligned} \sum _{i=0}^{\infty } \left( e^{-\mu t} (\mu t)^i /i!\right) \varvec{R}^i, \end{aligned}$$

if $\mu \ge \max _{i} |q_{ii}|$ and the (discrete-time) transition probability matrix $\varvec{R}$ is defined by $\varvec{R} = \varvec{I} + \varvec{Q}/\mu$. For this result see Lange [9, p. 207]. The TPM $\varvec{P}(t)$ is then approximated by taking sufficient terms in the above infinite series.

It is interesting to compare the structures of the two models for the catalytic reaction. One consists of four ODEs (three being nonlinear) in the four concentrations, the other is a (time-homogeneous) CTMC on three states. Both seek to represent the evolution of the process over time, and in this endeavour they use the same information: the structure of the reaction system, and the three quantities $k_i$, although these will not all be in the same units for the two models. The CTMC is more restricted in the initial conditions that it can represent.

Some of what we have described here can certainly be done by using the standard ‘dynamical system’ approach to the rate equations, although (unfortunately) there are differences in commonly used notation which lead to the conclusions of one approach being a transposed version of those of the other. The use of CTMCs as models for systems of reactions is of course not new, and it may be a matter of taste and background which approach one prefers. But it would seem that the two approaches can answer similar questions, and the use of a CTMC seems simple and direct, at least as regards the derivation of fast- and slow-process approximations; singular perturbation theory is not required. There would be no great difficulty in using a CTMC in a case that required more states than the three or four needed in the examples discussed here. Furthermore, the CTMC technique can avoid the complications which result from nonlinearity in the rate equations.