Fokker–Planck Equations for Time-Delayed Systems via Markovian Embedding

Abstract

For stochastic systems with discrete time delay, the Fokker–Planck equation (FPE) of the one-time probability density function (PDF) does not provide a complete, self-contained probabilistic description. It explicitly involves the two-time PDF, and represents, in fact, only the first member of an infinite hierarchy. We here introduce a new approach to obtain a Fokker–Planck description by using a Markovian embedding technique and a subsequent limiting procedure. On this way, we find a closed, complete FPE in an infinite-dimensional space, from which one can derive a hierarchy of FPEs. While the first member is the well-known FPE for the one-time PDF, we obtain, as second member, a new representation of the equation for the two-time PDF. From a conceptual point of view, our approach is simpler than earlier derivations and it yields interesting insight into both, the physical meaning, and the mathematical structure of delayed processes. We further propose an approximation for the two-time PDF, which is a central quantity in the description of these non-Markovian systems as it directly gives the correlation between the present and the delayed state. Application to a prototypical bistable system reveals that this approximation captures the non-trivial effects induced by the delay remarkably well, despite its surprisingly simple form. Moreover, it outperforms earlier approaches for the one-time PDF in the regime of large delays.

Appendices

Appendix A: The Discrete Delayed Case as Limit of the Coarse-Grained Dynamics

In this appendix, we show how the dynamical equation of the delayed system (1) can be recovered by taking the limit $n\rightarrow \infty$ of the corresponding equation for the embedded system (7). To this end, we project onto the the dynamics of the variable $X_0$. Until completion of the projection, we will keep n finite. Formally integrating Eq. (7b) (by first solving the homogeneous problem and then finding a specific solution by variation of constants) yields

$$\begin{aligned} X_j(t)= X_j(-\tau ) e^{-n (t+\tau ) /\tau }+\frac{n}{\tau }\int _{-\tau }^{t} e^{-n(t-s)/\tau } X_{j-1}(s) \mathrm {d}s, ~~~\forall j \in \{1,2,\ldots ,n \}\, . \end{aligned}$$

(23)

In principle, this solution can be simplified using an appropriate initial condition for the auxiliary variables, for example $X_j(0) = 0, ~\forall j>0$. However, we here proceed with the general case, i. e., without specifying the initial condition. As a next step (assuming $n>1$), we iteratively plug Eq. (23) into the corresponding solution for $j+1$ [given by Eq. (23) with $j+1$ instead of j]. For $1\le j < n$, we find

$$\begin{aligned} X_{j+1}(t)&= \left[ X_{j+1}(-\tau ) +\frac{n}{\tau } X_{j}(-\tau ) \,t\right] e^{-n (t+\tau ) /\tau }\nonumber \\&\quad +\frac{n^2}{\tau ^2} \int _{-\tau }^{t} \left\{ \int _{-\tau }^{s} e^{-n (t-s')/\tau } X_{j-1}(s') \mathrm {d}s' \right\} \mathrm {d}s. \end{aligned}$$

(24a)

Using the Heaviside function defined by $\varTheta (z)=0 ,\forall \, z < 0$ and $\varTheta (z)=1,\forall \, 0\le z$, we can simplify the double integral,

$$\begin{aligned} X_{j+1}(t)&= \left[ X_{j+1}(-\tau ) + \frac{n}{\tau } X_{j}(-\tau ) \,t \right] e^{-n (t+\tau ) /\tau } \nonumber \\&\quad + \frac{n^2}{\tau ^2} \int _{-\tau }^{s} \left\{ \int _{-\tau }^{t} \varTheta (s-s') \mathrm {d}s \right\} e^{-n (t-s')/\tau } X_{j-1}(s') \mathrm {d}s' , \end{aligned}$$

(24b)

$$\begin{aligned}&= \left[ X_{j+1}(-\tau ) + \frac{n}{\tau } X_{j}(-\tau ) \,t \right] e^{-n (t+\tau ) /\tau } \nonumber \\&\quad + \int _{-\tau }^{t} \frac{n^2}{\tau ^2} [t-s'] e^{-n (t-s')/\tau } X_{j-1}(s') \mathrm {d}s' . \end{aligned}$$

(24c)

Repeating this iterative procedure, we finally obtain for $j=n$

$$\begin{aligned} X_{n}(t)= & {} \left[ X_{n}(-\tau )+\frac{n}{\tau }X_{n-1}(-\tau )t\right. \nonumber \\&\left. +\frac{n^2}{\tau ^2}X_{n-2}(-\tau )t^2+ \cdots \right] \, e^{-n (t+\tau ) /\tau } + \int _{-\tau }^{t} K_n(t-s) X_{0}(s) \mathrm {d}s , \end{aligned}$$

(24d)

with the Gamma-distributed memory kernel defined in Eq. (8). The first term in (24d) vanishes trivially, if the auxiliary variables satisfy the initial condition $X_{j\in \{1,2,\ldots ,n\}}(-\tau )=0$. Interestingly, this terms also vanishes if the asymptotic behavior at $t\rightarrow \infty$ is considered, due to the exponential damping with time. Plugging (24d) into (7a) [neglecting the initial condition term] then yields the projected Eq. (8).

Now we consider the limit $n\rightarrow \infty$ of the memory kernel (8). First, we find that the mean value of the kernel and the integral over it are, independently of the value of n, given by $\mu =\int _{0}^{\infty } K_n(T)T \mathrm {d}T =\tau$ and $\int _{0}^{\infty } K_n(T) \mathrm {d}T =1$, respectively. Furthermore, for large n, the value of the kernel at $\tau$ can be estimated by using the Stirling formula $n!\approx \sqrt{2\pi n}(n/e)^n$, yielding $K_n(\tau )= \frac{{n}^n \,e^{-n}}{(n-1)! \tau } \approx \frac{n!}{(n-1)! \sqrt{2\pi n}\tau }=\sqrt{\frac{n }{ 2\pi }}\frac{1}{\tau }\rightarrow \infty .$ On the other hand, the variance of the kernel vanishes for $n\rightarrow \infty$, since $\int _{0}^{\infty } K_n(T)T^2 \mathrm {d}T -\mu ^2=\tau ^2/{n}\rightarrow 0$. This implies

$$\begin{aligned} \lim _{n \rightarrow \infty } K_n(T)= \delta (T-\tau ). \end{aligned}$$

(25)

Thus, the projected equation (8) is indeed equal to Eq. (1) in the limit $n\rightarrow \infty$. We note that even the transient dynamics (i. e., finite t) is equivalent for the initial conditions $X_{j\in \{1,2,\ldots ,n\}}(-\tau )\equiv 0$, as mentioned above.

Appendix B: Connection to Fokker–Planck Hierarchy from Novikov’s Theorem

While the FPE for $\rho _1$ obtained by our approach is equivalent to the first member of the FP hierarchy from Novikov’s theorem (2), the second members differ. However, the apparent disagreement can be resolved. Indeed, as we show below, our result (17) can be transformed to the representation obtained from Novikov’s theorem [given in (3)]. In the following, we will consider the term which differs, i. e., the drift term $\partial _{x_\tau } [\rho _2(x,t;x_\tau ,t-\tau )$$\langle {\dot{X}} ( t - \tau ) |x=X(t) , x_\tau =X({t - \tau })\rangle ]$. To this end, we first use the definition of $\rho _2$ via delta-distributions, plug in the LE (1), which yields

$$\begin{aligned}&\partial _{x_\tau } \left[ \langle {\dot{X}} ( t - \tau ) |x=X(t) ,x_\tau =X({t - \tau })\rangle \rho _2(x,t;x_\tau ,t-\tau ) \right] \nonumber \\&\quad = \partial _{x_\tau } \left\langle {\dot{X}}(t-\tau ) \,\delta [X(t)-x]\delta [X(t-\tau )-x_\tau ] \right\rangle \nonumber \\&\quad {\mathop {=}\limits ^{\mathrm {LE}}} \partial _{x_\tau } \left\langle F[X(t-\tau ),X(t-2\tau )]\,\delta [X(t)-x]\delta [X(t-\tau )-x_\tau ] \right\rangle \nonumber \\&\qquad \quad + \partial _{x_\tau } \left\langle \sqrt{2D_0}\xi (t-\tau ) \,\delta [X(t)-x]\delta [X(t-\tau )-x_\tau ] \right\rangle . \end{aligned}$$

(26a)

At this point, we have two correlations to deal with. The first one (involving $F_{}$) can easily be simplified by expressing the ensemble average with the help of the PDF $\rho _3$, i. e.,

$$\begin{aligned}&\partial _{x_\tau } \Big \langle F\left[ X(t-\tau ),X(t-2\tau )\right] \,\delta [X(t)-x]\delta [X(t-\tau )-x_\tau ] \Big \rangle \nonumber \\&\quad = \partial _{x_\tau } \iiint _{\varOmega }^{ }F_{}(x_\tau ,x_{2\tau }) \,\delta [X(t)-x]\delta [X(t-\tau )-x_\tau ]\times \nonumber \\&\qquad \quad \rho _3(x,t;x_\tau ,t-\tau ;x_{2\tau },t-2\tau ) \mathrm {d}x \mathrm {d}x_\tau \mathrm {d}x_{2\tau } \nonumber \\&\quad = \partial _{x_\tau } \int _{\varOmega }^{ } F_{}(x_\tau ,x_{2\tau }) \rho _3(x,t;x_\tau ,t-\tau ;x_{2\tau },t-2\tau ) \mathrm {d}x_{2\tau }. \end{aligned}$$

(26b)

The remaining term on the right side of (26a) (involving $\xi$) is simplified with the help of Novikov’s theorem (28) (see Appendix 1). Specifically, we define $\varLambda [\xi ]:= \delta [x-X(t)]\, \delta [x_\tau -X(t\!-\!\tau )]$, which yields

$$\begin{aligned}&\partial _{x_\tau } \left\langle \sqrt{2D_0}\xi (t-\tau ) \,\delta [X(t)-x]\delta [X(t-\tau )-x_\tau ] \right\rangle \nonumber \\&\quad =\sqrt{2D_0}\partial _{x_\tau } \left\langle \xi (t-\tau ) \varLambda [\xi ] \right\rangle , \nonumber \\&\quad = \sqrt{2D_0} \partial _{x_\tau } \left\langle \frac{\delta \varLambda [\xi ] }{\delta \xi (t-\tau )} \right\rangle \nonumber \\&\quad = \sqrt{2D_0}\partial _{x_\tau } \left[ \left\langle \frac{\delta \varLambda [\xi ] }{\delta X(t)}\frac{\delta X(t) }{\delta \xi (t-\tau )} \right\rangle +\left\langle \frac{\delta \varLambda [\xi ] }{\delta X(t-\tau )} \frac{\delta X(t-\tau ) }{\delta \xi (t-\tau )}\right\rangle \right] \nonumber \\&\quad = -\sqrt{2D_0}\partial _{x_\tau } \left\langle \partial _{x }\delta [x-X(t)]\, \delta [x_\tau -X(t\!-\!\tau )] {\frac{\delta X(t) }{\delta \xi (t-\tau )}} \right\rangle \nonumber \\&\qquad -\sqrt{2D_0}\partial _{x_\tau }\left\langle \partial _{x_\tau } \delta [x-X(t)]\, \delta [x_\tau -X(t\!-\!\tau )]\underbrace{\frac{\delta X(t-\tau ) }{\delta \xi (t-\tau )}}_{\psi }\right\rangle . \end{aligned}$$

(26c)

In the last step we have plugged in the definition of $\varLambda$ and used the chain rule. Now, the term denoted $\psi$ is calculated by replacing $X(t-\tau )$ with the integral form of the LE (1), i. e., $X(t')= \int _{0}^{t'}\left[ F[X(s),X(s-\tau )]+\sqrt{2D_0}\xi (s)\right] \mathrm {d}s$, $\forall \,t'>0$, yielding

$$\begin{aligned} \psi =\frac{\delta X(t-\tau )}{\delta \xi (t-\tau )}&= \frac{\delta }{\delta \xi (t-\tau )} \int _{0}^{t-\tau } \left[ F[X(s),X(s-\tau )]+\sqrt{2D_0}\xi (s)\right] \mathrm {d}s\nonumber \\&=\sqrt{2D_0}\,\int _{0}^{t-\tau } \delta [s-(t-\tau )]\mathrm {d}s=\sqrt{\frac{D_0}{2}}, \end{aligned}$$

(26d)

independently of the specific form of $F_{}$. We note that the other functional derivative in (26c) cannot be treated analogously, since evaluation of the functional derivative of the trajectory X w. r. t. the earlier noise $\xi$ requires the formal solution of (1), which is only known for linear systems. Indeed, for a LE with $F= -c_1 X(t) -c_2 X_{}(t-\tau )$, one finds $\frac{\delta X(t) }{\delta \xi (t-\tau )}=\sqrt{2D_0} e^{-c_1 \tau }$ by using the method of steps, see [31]. For general nonlinear forces F, this term must be treated differently, e. g. by approximation methods, but a discussion of this goes beyond the scope of this paper. We therefore keep this functional derivative and only evaluate the delta-distributions. Based on these considerations, we can rewrite (26c) as

$$\begin{aligned} \sqrt{2D_0}\,\partial _{x_\tau } \left\langle \xi (t-\tau ) \varLambda [\xi ] \right\rangle&=-\sqrt{2D_0}\,\partial _{x_\tau }\partial _{x } \left\langle \frac{\delta X(t) }{\delta \xi (t-\tau )}\Big |_{\begin{array}{c} X(t)=x \\ X(t - \tau )=x_\tau \end{array}} \right\rangle \nonumber \\&\qquad \rho _2(x,t;x_{\tau },t-\tau ) -D_0\,\partial _{x_\tau }^2\rho _2(x,t;x_{\tau },t-\tau ). \end{aligned}$$

(26e)

In combination with (26a,26b), we obtain the identity

$$\begin{aligned}&\partial _{x_\tau } \left\langle {\dot{X}}(t-\tau ) \,\delta [X(t)-x]\delta [X(t-\tau )-x_\tau ] \right\rangle \nonumber \\&\quad = -\sqrt{2D_0}\partial _{x_\tau }\partial _{x} \left\langle \frac{\delta X(t) }{\delta \xi (t-\tau )}\Big |_{\begin{array}{c} X(t)=x \\ X(t - \tau )=x_\tau \end{array}} \right\rangle \rho _2(x,t;x_{\tau },t-\tau ) \nonumber \\&\qquad -D_0 \partial _{x_\tau }^2 \rho _2(x,t;x_{\tau },t-\tau )+\partial _{x_\tau } \int _{\varOmega }^{ } F_{}(x_\tau ,x_{2\tau }) \rho _3(x,t;x_\tau ,t-\tau ;x_{2\tau },t-2\tau ) \mathrm {d}x_{2\tau } \nonumber \\&\quad = \partial _{x_\tau } \left[ \langle {\dot{X}} ( t - \tau ) |x=X(t) , x_\tau =X({t - \tau })\rangle \rho _2(x,t;x_\tau ,t-\tau ) \right] , \end{aligned}$$

(27)

proving that the second member of the here presented FP hierarchy, Eq. (17), is identical to the corresponding one (3) obtained from Novikov’s theorem.

Appendix C: Novikov’s Theorem

In this section, we establish the relation [40]

$$\begin{aligned} \big \langle \,\varLambda [\xi ] \xi (t) \,\big \rangle = \Bigg \langle \frac{\delta \varLambda [\xi ]}{\delta \xi (t)}\Bigg \rangle , \end{aligned}$$

(28)

for a functional $\varLambda$ of a Gaussian white noise$\xi$. It links the functional derivative w. r. t. the noise to the cross-correlation between functional and noise. First, we consider the case where the ensemble is w. r. t. a fixed initial condition $\phi =X(-\tau \le t\le t)$, yielding $\varLambda (0)={\varLambda _0}$. In a second step we will generalize towards initial conditions drawn from an arbitrary distribution.

We express the ensemble average $\langle \cdots \rangle _{\varLambda _0}$ of the left hand side of (28) via the path integral over all possible paths $\xi$ between time 0 and t, accounting for all possible realizations of the random process $\xi$ at each instant in time

$$\begin{aligned} \left\langle \varLambda [\xi ]\xi (t) \right\rangle _{\varLambda _0} = \int _{\xi _0}^{\xi _t}\varLambda \left[ \xi |{\varLambda _0}\right] \,\xi \, {\mathcal {P}}[\xi ]\, {\mathcal {D}}[\xi ], \end{aligned}$$

(29a)

with arbitrary but fixed $\xi _0:=\xi (t_0)$ and $\xi _t:=\xi (t)$. Please note that specifying the noise process at the boundaries does not impose a restriction on the generality as we deal with white noise. The weight ${\mathcal {P}}[\xi ]$ of each white noise realization is given by the Gaussian path probability (for arbitrary $t>0$)

$$\begin{aligned} {\mathcal {P}}[\xi ]={\mathcal {J}} e^{-\frac{1}{2}\int _{0}^{t}\xi ({t'})^2 \mathrm d t'}, \end{aligned}$$

(29b)

with Jacobian ${\mathcal {J}}$. Now, we rewrite the integrand in (29a) using that the functional derivative of ${\mathcal {P}}[\xi ]$ w. r. t. $\xi$ is simply $(-\xi ) {\mathcal {P}}[\xi ]$, yielding

$$\begin{aligned} \left\langle \varLambda [\xi ]\xi (t) \right\rangle _{\varLambda _0} =&- \int _{\xi _0}^{\xi _t}\varLambda \left[ \xi |{\varLambda _0}\right] \frac{\delta }{\delta \xi }\!\left\{ e^{-\frac{1}{2}\int _{0}^{t}\xi ({t'})^2 \mathrm d t'}\right\} \,{\mathcal {D}}[\xi ] \nonumber \\ =&- \int _{\xi _0}^{\xi _t} \frac{\delta \left\{ \varLambda [\xi ] {\mathcal {P}}[\xi ] \right\} }{\delta \xi } \,{\mathcal {D}}[\xi ] + \int _{\xi _0}^{\xi _t}\frac{\delta \varLambda [\xi ]}{\delta \xi } {\mathcal {P}}[\xi ] \,{\mathcal {D}}[\xi ]. \end{aligned}$$

(29c)

In the last step we have used the product rule (and omitted the initial condition $\varLambda _0$ for sake of a shorter notation). The functional derivative in the first path integral yields

$$\begin{aligned} \int _{\xi _0}^{\xi _t}\frac{\delta \left\{ \varLambda [\xi ] {\mathcal {P}}[\xi ] \right\} }{\delta \xi } {\mathcal {D}}[\xi ]= \int _{\xi _0}^{\xi _t} \varLambda [\xi +\delta \xi ] {\mathcal {P}}[\xi +\delta \xi ] {\mathcal {D}}[\xi ]-\int _{\xi _0}^{\xi _t}\varLambda [\xi ] {\mathcal {P}}[\xi ] {\mathcal {D}}[\xi ]. \end{aligned}$$

(29d)

Now we use that the variations of the paths $\xi +\delta \xi$ are already contained in the integral over all paths, and find

$$\begin{aligned} \int _{\xi _0}^{\xi _t}\frac{\delta \left\{ \varLambda [\xi ] {\mathcal {P}}[\xi ] \right\} }{\delta \xi } {\mathcal {D}}[\xi ]= \int _{\xi _0}^{\xi _t}\varLambda [\xi ] {\mathcal {P}}[\xi ] {\mathcal {D}}[\xi ]-\int _{\xi _0}^{\xi _t}\varLambda [\xi ] {\mathcal {P}}[\xi ] {\mathcal {D}}[\xi ]=0, \end{aligned}$$

(29e)

(as long as $\int \varLambda [\xi ] {\mathcal {P}}[\xi ] {\mathcal {D}}[\xi ] <\infty$), see also [19] (on p. 273f). Hence, we obtain from (29c),

$$\begin{aligned} \left\langle \varLambda [\xi ]\xi (t) \right\rangle _{\varLambda _0} =&\int _{\xi _0}^{\xi _t}\frac{\delta \varLambda [\xi ]}{\delta \xi } {\mathcal {P}}[\xi ] \,{\mathcal {D}}[\xi ] = \left\langle \frac{\delta \varLambda [\xi ]}{\delta \xi } \right\rangle _{\varLambda _0}. \end{aligned}$$

(29f)

This result can readily be generalized to ensembles where the initial conditions are instead drawn from an arbitrary, normalized distribution $P(\varLambda _0)$. In particular, Eq. (29f) implies

$$\begin{aligned} \left\langle \varLambda [\xi ]\xi (t) \right\rangle = \int \left\langle \varLambda [\xi ]\xi (t) \right\rangle _{\varLambda _0} P(\varLambda _0) \mathrm {d}\varLambda _0 = \int \left\langle \frac{\delta \varLambda [\xi ]}{\delta \xi } \right\rangle _{\varLambda _0} P(\varLambda _0) \mathrm {d}\varLambda _0 =\left\langle \frac{\delta \varLambda [\xi ]}{\delta \xi } \right\rangle . \end{aligned}$$

(30)

This is the relation (28), which is often referred to as Novikov’s theorem.

Appendix D: The Approximation in the Regime of Short Delay Times

The approximate two-time PDF (22) is, in general, not reliable, if the delay time is very short, as the underlying assumption is violated. For the moderate $|k|/V_0$ and small $\tau$ values considered in Fig. 6a, the corresponding one-time PDF (obtained by marginalization) still gives a reasonable approximation. Here also the other approaches from the literature are justified. For the large (and negative) $|k|/V_0$ value in Fig. 6b, only the small $\tau$ expansion yields a reasonable approximation.

Fig. 6

Comparison of different approximations for the one-time PDF in the doublewell potential with linear delay force $k X(t-\tau )$ at $V_0=3.5$, for two exemplary values of k. In contrast to Fig. 5 where $\tau =10$, here a shorter delay time of $\tau =0.1$ is considered