Metastable behavior in Markov processes with internal states

Abstract

A perturbation framework is developed to analyze metastable behavior in stochastic processes with random internal and external states. The process is assumed to be under weak noise conditions, and the case where the deterministic limit is bistable is considered. A general analytical approximation is derived for the stationary probability density and the mean switching time between metastable states, which includes the pre exponential factor. The results are illustrated with a model of gene expression that displays bistable switching. In this model, the external state represents the number of protein molecules produced by a hypothetical gene. Once produced, a protein is eventually degraded. The internal state represents the activated or unactivated state of the gene; in the activated state the gene produces protein more rapidly than the unactivated state. The gene is activated by a dimer of the protein it produces so that the activation rate depends on the current protein level. This is a well studied model, and several model reductions and diffusion approximation methods are available to analyze its behavior. However, it is unclear if these methods accurately approximate long-time metastable behavior (i.e., mean switching time between metastable states of the bistable system). Diffusion approximations are generally known to fail in this regard.

Appendices

Appendix A: Curvature prefactor

The purpose of this section is to show that the part of the eigenvalue estimate that contains information about the curvature of the stability well at the stable and unstable fixed point is unaffected by the QSS diffusion approximation. This is a reflection of the fact that diffusion approximations, in general, are accurate in a neighborhood of a deterministic fixed point. The eigenvalue approximation (3.61) contains a prefactor term of the form $\sqrt{\left| \Phi ''(x_{*}) \right| \Phi ''(x_{\pm })}$. We would like to show that, when evaluated at a fixed point, $x_{c}$, the second derivative of $\Phi$ for the discrete, semi-continuous, and QSS processes are all identical. We can express the second derivative in terms of $\mathcal{H }$, defined by (3.19), as follows.

Differentiating $\mathcal{H }(x,\Phi '(x)) = 0$ with respect to $x$ yields

$$\begin{aligned} \frac{d}{dx}\mathcal{H }(x,\Phi '(x)) = \mathcal{H }_{x}(x,\Phi '(x)) + \Phi ''(x)\mathcal{H }_{p}(x,\Phi '(x)) = 0, \end{aligned}$$

(6.1)

and it follows that

$$\begin{aligned} \Phi ''(x) = - \frac{\mathcal{H }_{x}(x,\Phi '(x))}{\mathcal{H }_{p} (x,\Phi '(x))}. \end{aligned}$$

(6.2)

However, we have that

$$\begin{aligned} \mathcal{H }_{p}(x_{c},0) = \mathcal{H }_{x}(x_{c},0) = \mathcal{H }_{xx}(x_{c},0) = 0. \end{aligned}$$

(6.3)

A formula valid at fixed points can be obtained as follows. Differentiating $\mathcal{H }(x, \Phi '(x)) = 0$ twice with respect to $x$ yields

$$\begin{aligned} \frac{d^{2}}{dx^{2}}\mathcal{H }(x,\Phi '(x)) = \mathcal{H }_{xx}+ \Phi ''\mathcal{H }_{x p} + \Phi ''(\mathcal{H }_{px} + \Phi ''\mathcal{H }_{pp}) + \Phi '''\mathcal{H }_{p} = 0,\quad \end{aligned}$$

(6.4)

and it follows from (6.3) that

$$\begin{aligned} \Phi ''(x_{c}) = \frac{-2\frac{\partial ^{2}}{\partial p\partial x}\mathcal{H }(x_{c},0)}{\frac{\partial ^{2} }{\partial p ^{2}}\mathcal{H }(x_{c},0)}. \end{aligned}$$

(6.5)

At a fixed point, we have that $p=0$. Expand $\mathcal{H }(x, p)$ in a Taylors series around $p=0$. To second order in $p$, the expansion is consistent with a diffusion approximation, which always corresponds to a Hamiltonian that is quadratic in $p$ with

$$\begin{aligned} \mathcal{H }_{\mathrm{diff}}(x, p) = a(x)p + g(x)p^{2}, \end{aligned}$$

(6.6)

where $a(x)$ is the drift and $g(x)$ is the scaled diffusivity. For a QSS diffusion approximation of the processes described in Sect. 2, one can show that

$$\begin{aligned} a(x) = \varvec{\rho }(x)^{T}\mathbf{v }(x),\quad g(x) = \varvec{\rho }(x)^{T}\mathbf{b }(x) - \varvec{\rho }(x)^{T}\left( \Sigma _{\mathbf{v }(x)}-a(x)I\right) [A^{\dag }(x)]^{T}\mathbf{v }(x).\nonumber \\ \end{aligned}$$

(6.7)

It follows that at a fixed point $a(x_{c}) = 0$ and $g(x_{c}) = \varvec{\rho }(x_{c})^{T}(\mathbf{b }(x) - \Sigma _{\mathbf{v }(x_{c})}\varvec{\zeta })$. Substituting (6.6) into (6.5) yields $\Phi ''(x_{c}) = -\frac{a'(x_{c})}{g(x_{c})}$.

Appendix B: Adiabatic limit of the discrete process

Consider the Master equation for the probability distribution function $\mathrm{p }_{j}(\mathbf{n },t) \equiv \mathrm{p }(j,\mathbf{n },t| j_{0},\mathbf{n }_{0},t_{0})$. In matrix/operator form, the CK equation is

$$\begin{aligned} \frac{d\mathbf{p }}{dt} = L_{1}p + \frac{1}{\epsilon }L_{2}p, \end{aligned}$$

(7.1)

where $\mathbf{p }(\mathbf{n },t)=(\mathrm{p }_{1}(\mathbf{n },t),\, \mathrm{p }_{2}(\mathbf{n },t),\ldots ,\,\mathrm{p }_{M}(\mathbf{n },t))^{T}$; $L_{1}=\Sigma _{\mathcal{D }_{j}}$ is a diagonal matrix of linear operators acting on $\mathbf{n }$, each of which has a $\mathbb{W }$-matrix representation; $L_{2}$ is an $M\times M\;\mathbb{W }$-matrix governing the transitions between internal states, with transition rates that may depend on $\mathbf{n }$. Define the projection operator $\mathcal{P } \equiv \varvec{\rho }\mathbf{1 }^{T}$, where $L_{2}\varvec{\rho }(\mathbf{n })=0$, with $\varvec{\rho }(\mathbf{n })>0$ and $\sum _{j=1}^{M}\rho _{j}(\mathbf{n }) = 1$; and $\mathbf{1 }\equiv (1,\, 1,\ldots ,\,1)^{T}$. We assume the solution has the following form

$$\begin{aligned} \mathbf{p }(\mathbf{n },t) = \mathcal{P }\mathbf{p }(\mathbf{n },t) + (I-\mathcal{P })\mathbf{p }(\mathbf{n },t) =u(\mathbf{n },t)\varvec{\rho }(\mathbf{n }) + \epsilon \mathbf{w }(\mathbf{n },t), \end{aligned}$$

(7.2)

where

$$\begin{aligned} u(\mathbf{n },t) \equiv \mathbf{1 }^{T}\mathbf{p }(\mathbf{n },t),\quad \mathbf{1 }^{T}\mathbf{w }(\mathbf{n },t) = 0. \end{aligned}$$

(7.3)

Applying the projection operator to both sides of (7.1) yields

$$\begin{aligned} \frac{d u}{dt} \varvec{\rho }= \mathcal{P }L_{1}(u\varvec{\rho }+ \epsilon \mathbf{w }). \end{aligned}$$

(7.4)

On the other hand, applying the orthogonal projection yields

$$\begin{aligned} \epsilon \frac{d\mathbf{w }}{dt} - \epsilon (I-\mathcal{P })L_{1}\mathbf{w }= (I-\mathcal{P })L_{1}(u\varvec{\rho }) + L_{2}\mathbf{w }. \end{aligned}$$

(7.5)

After setting $\epsilon =0$ in the above equation we get

$$\begin{aligned} \mathbf{w }(\mathbf{n },t) \sim -L_{2}^{-1}(I-\mathcal{P })L_{1}(u(\mathbf{n },t)\varvec{\rho }(\mathbf{n })). \end{aligned}$$

(7.6)

Substituting (7.6) into (7.4) yields the scalar-valued operator equation for $u(\mathbf{n },t)$

$$\begin{aligned} \frac{du}{dt} = \mathbf{1 }^{T}L_{1}(u\varvec{\rho }) - \epsilon \mathbf{1 }^{T}L_{1}L_{2}^{-1}(I-\mathcal{P })L_{1}(u\varvec{\rho }). \end{aligned}$$

(7.7)

One can rewrite (7.7) in matrix form to obtain a linear system of ODEs for the vector $\mathbf{u }(t)$ with elements $u_{\mathbf{n }}(t)\equiv u(\mathbf{n },t)$

$$\begin{aligned} \frac{d\mathbf{u }}{dt} = W \mathbf{u }, \end{aligned}$$

(7.8)

where $W \equiv \sum _{j=1}^{M}\mathcal{D }_{j}\rho _{j}$. In general, the reduced equation represents a Markov process only at leading order.

Appendix C: WKB/KM expansion

Consider the action of the operator $\mathbb{e }^{\partial x}$ on $g(x)e^{-\alpha _{\mathrm{e}}\tilde{\Phi }(x)}$ where $g(x)$ is scalar function and $\tilde{\Phi }(x) = \varphi \Phi (x)$. We have that

$$\begin{aligned}&\mathbb{e }^{\pm \partial x}\left( g(x)e^{-\alpha _{\mathrm{e}}\tilde{\Phi }(x)}\right) \nonumber \\&\quad = \sum _{n=0}^{\infty }\frac{(\pm 1)^{n}}{n!\alpha _{\mathrm{e}}^{n}} \frac{d^{n}}{dx^{n}}\left[ g(x)e^{-\alpha _{\mathrm{e}}\tilde{\Phi }(x)} \right] \nonumber \\&\quad = \sum _{n=0}^{\infty }\frac{(\pm 1)^{n}}{n!\alpha _{\mathrm{e}}^{n}} \sum _{k=0}^{n}\left( {\begin{array}{c}n\\ k\end{array}}\right) g^{(n-k)}(x)\frac{d^{k}}{dx^{k}} e^{-\alpha _{\mathrm{e}}\tilde{\Phi }(x)}\nonumber \\&\quad = \sum _{n=0}^{\infty }\frac{(\pm 1)^{n}}{n!\alpha _{\mathrm{e}}^{n}} \sum _{k=0}^{n}\left( {\begin{array}{c}n\\ k\end{array}}\right) g^{(n-k)}(x)(1+\mathrm{B }_{k} (-\alpha _{\mathrm{e}}\tilde{\Phi }(x)))e^{-\alpha _{\mathrm{e}}\tilde{\Phi }(x)}, \end{aligned}$$

(8.1)

where $\mathrm{B }_{k}$ is the $k$th complete Bell polynomial,

$$\begin{aligned} \mathrm{B }_{n}(f(x)) \equiv \det \begin{bmatrix} f'&\quad \left( {\begin{array}{c}n-1\\ 1\end{array}}\right) f''&\quad n \left( {\begin{array}{c}n-1\\ 2\end{array}}\right) f^{(3)}&\quad n \cdots&\quad n f^{(n)}\\ -1&\quad n f'&\left( {\begin{array}{c}n-2\\ 1\end{array}}\right) f''&\quad n \cdots&\quad n f^{(n-1)} \\ 0&\quad n -1&f'&\quad n \cdots&\quad n f^{(n-2)} \\ \vdots&\quad n \vdots&\quad n \ddots&\ddots&\vdots \\ 0&0&\cdots&-1&f' \\ \end{bmatrix}, \end{aligned}$$

(8.2)

and $B_{0}=0$. One can show that

$$\begin{aligned} \mathrm{B }_{k}(-\alpha _{\mathrm{e}}\tilde{\Phi }(x)) = \alpha _{\mathrm{e}}^{k}(- \tilde{\Phi }' (x))^{k} -\alpha _{\mathrm{e}}^{k-1}\frac{k}{2}(k-1)\tilde{\Phi }''(x)(-\tilde{\Phi } '(x))^{k-2} + O(\alpha _{\mathrm{e}}^{k-2}).\nonumber \\ \end{aligned}$$

(8.3)

Expanding (8.1) in terms of $1/\alpha _{\mathrm{e}}$ yields

$$\begin{aligned}&\mathbb{e }^{\pm \partial x}\left( g(x)e^{-\alpha _{\mathrm{e}}\tilde{\Phi }(x)}\right) \nonumber \\&\quad = e^{\mp \tilde{\Phi }'(x)}\left[ g(x) - \frac{1}{\alpha _{\mathrm{e}}} \left( g'(x) \mp \frac{1}{2}g(x)\tilde{\Phi }''(x)\right) + O(\alpha _{\mathrm{e}}^{-2})\right] e^{-\alpha _{\mathrm{e}}\tilde{\Phi }(x)}.\qquad \qquad \end{aligned}$$

(8.4)

Appendix D: Evaluating $\lim _{x\rightarrow x_{c}}\Psi '(x)$ for $x_{c}=x_{\pm },x_{*}$

Using L’Hôpital’s rule, we find that

$$\begin{aligned} \Psi '(x_{c})&= \Bigg [ {H}_{px x} +\frac{1}{2}\Phi ''(x_{c}) (3{H}_{ppx} +\Phi ''(x_{c}){H}_{ppp}) + \frac{1}{2}\Phi '''(x_{c}){H}_{pp}\nonumber \\&\quad +\varvec{l}'(x_{c})^{T}\mathbf{H }_{px}(x_{c}, 0) + \frac{1}{2}\Phi ''(x_{c})\varvec{l}'(x_{c})^{T}\mathbf{H }_{pp}(x_{c}, 0) \Bigg ]\nonumber \\&\quad \Big / \Big [{\varvec{l}'(x_{c})^{T}\mathbf{H }_{p} (x_{c}, 0) + {H}_{px}+ \Phi ''(x_{c}){H}_{pp}}\Big ], \end{aligned}$$

(9.1)

where $\mathbf{H }(x, p)$ is defined by (3.22) and partial derivatives of $H(x, p) \equiv \mathbf{1 }^{T}\mathbf{H }(x, p)$ are evaluated at $x=x_{c}$ and $p=0$, as for example,

$$\begin{aligned} H_{xp} \equiv \mathbf{1 }^{T}\frac{\partial ^{2}}{\partial x \partial p}\mathbf{H }(x_{c}, 0). \end{aligned}$$

(9.2)

We also have that $\Phi ''(x_{c})$ is given by (6.5), and

$$\begin{aligned} \Phi '''(x_{c}) = -2\frac{\mathcal{H }_{px x}(x_{c}, 0) + \frac{1}{3}\Phi ''(x_{c})\mathcal{H }_{ppp}(x_{c}, 0)}{\mathcal{H }_{pp}(x_{c}, 0)}. \end{aligned}$$

(9.3)

Note that $H(x, p)\ne \mathcal{H }(x, p)$, where $\mathcal{H }(x, p)$ is the Hamiltonian (3.19).

Appendix E: $x\rightarrow 0$ limit of the quasi-stationary density

The WKB approximation (3.25) of the discrete process breaks down in the limit $x\rightarrow 0$, due to small copy number effects (i.e., fluctuations are on the same order). This fact is not relevant if one is interested only in approximating the mean exit time. However, we also approximate the effective potential. Although $\Phi (x)$ is bounded in the limit $x\rightarrow 0, \Psi (x)$ has a logarithmic singularity. To correct this, we use the discrete master equation (4.4) to calculate $\varvec{\phi }_{}(0)$, with $\varvec{\phi }_{}(0) = - (\alpha _{\mathrm{i}}A(0)-\alpha _{\mathrm{e}}\Sigma _{v(0)})^{-1}\varvec{\phi }_{}(\frac{1}{\alpha _{\mathrm{e}}})$.