Fokker–Planck Equations for Time-Delayed Systems via Markovian Embedding

We’re sorry, something doesn't seem to be working properly.

Please try refreshing the page. If that doesn't work, please contact support so we can address the problem.


For stochastic systems with discrete time delay, the Fokker–Planck equation (FPE) of the one-time probability density function (PDF) does not provide a complete, self-contained probabilistic description. It explicitly involves the two-time PDF, and represents, in fact, only the first member of an infinite hierarchy. We here introduce a new approach to obtain a Fokker–Planck description by using a Markovian embedding technique and a subsequent limiting procedure. On this way, we find a closed, complete FPE in an infinite-dimensional space, from which one can derive a hierarchy of FPEs. While the first member is the well-known FPE for the one-time PDF, we obtain, as second member, a new representation of the equation for the two-time PDF. From a conceptual point of view, our approach is simpler than earlier derivations and it yields interesting insight into both, the physical meaning, and the mathematical structure of delayed processes. We further propose an approximation for the two-time PDF, which is a central quantity in the description of these non-Markovian systems as it directly gives the correlation between the present and the delayed state. Application to a prototypical bistable system reveals that this approximation captures the non-trivial effects induced by the delay remarkably well, despite its surprisingly simple form. Moreover, it outperforms earlier approaches for the one-time PDF in the regime of large delays.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5


  1. 1.

    As an illustrative example, consider the following. Let \(\rho ^\mathrm {nM}_1(x)\) be the non-Markovian one-time PDF of an arbitrary nonequilibrium steady state. This PDF can in general not be distinguished from a PDF \(\rho ^\mathrm {M}_1(x)\) of a Markovian (equilibrium) process in a (fictive) potential defined by \(U_\mathrm {ficitve}(x)\equiv k_\mathrm {B}{\mathcal {T}} \ln {\rho ^\mathrm {nM}_1(x)}\), which would equally yield a distribution \(\rho ^\mathrm {M}_1 =Z^{-1}e^{-U_\mathrm {fictive}/k_\mathrm {B}{\mathcal {T}}}=\rho ^\mathrm {nM}_1\).

  2. 2.

    The force is scaled with the friction coefficient \(\gamma \), i. e., \(F \rightarrow F/\gamma \), which is related with the bath’s temperature \({\mathcal {T}}\) via \(\gamma D_0 = k_\mathrm {B}{\mathcal {T}}\), where \(k_\mathrm {B}\) is the Boltzmann constant.

  3. 3.

    We use the general relation between joint, \(\rho _2\), and marginal distributions \(\rho _{1}(z)=\int _{a} \rho _{2}(y,z) \mathrm {d}y\), with the normalization \(\int _{b}\rho _{1}(z)\mathrm {d}z=\int _{b}\int _{a} \rho _{2}(y,z) \mathrm {d}y\mathrm {d}z=1\), with \(y\in a\), \(z\in b\).

  4. 4.

    The dimensionless LE is obtained by rescaling position \(x/\sigma \rightarrow x\) and time \((D_0/\sigma ^2)t \rightarrow t\), \((D_0/\sigma ^2)\tau \rightarrow \tau \), with the the minima \(x\pm \sigma \) of the doublewell potential \(V=V_0 [(x/\sigma )^4-2(x/\sigma )^2]\), and by rescaling the parameters \(V_0/(\gamma D_0) \rightarrow V_0 \) and \(k/(\gamma D_0) \rightarrow k\) with the bath’s thermal energy \(\gamma D_0\).

  5. 5.

    Due to the symmetry of the external potentials, we further always expect that \(\rho _2(x,x_\tau )\) is symmetric w. r. t.  \(\{x,x_\tau \} \rightarrow -\{x,x_\tau \} \).


  1. 1.

    Bao, J.D., Hänggi, P., Zhuo, Y.Z.: Non-Markovian Brownian dynamics and nonergodicity. Phys. Rev. E 72(6), 061107 (2005)

    ADS  Article  Google Scholar 

  2. 2.

    Bruot, N., Damet, L., Kotar, J., Cicuta, P., Lagomarsino, M.C.: Noise and synchronization of a single active colloid. Phys. Rev. Lett. 107(9), 094101 (2011)

    ADS  Article  Google Scholar 

  3. 3.

    Cabral, J.R., Luckhoo, H., Woolrich, M., Joensson, M., Mohseni, H., Baker, A., Kringelbach, M.L., Deco, G.: Exploring mechanisms of spontaneous functional connectivity in MEG: how delayed network interactions lead to structured amplitude envelopes of band-pass filtered oscillations. Neuroimage 90, 423–435 (2014)

    Article  Google Scholar 

  4. 4.

    Callen, J.L., Khan, M., Lu, H.: Accounting quality, stock price delay, and future stock returns. Contemp. Acc. Res. 30(1), 269–295 (2013)

    Article  Google Scholar 

  5. 5.

    Carmele, A., Kabuss, J., Schulze, F., Reitzenstein, S., Knorr, A.: Single photon delayed feedback: a way to stabilize intrinsic quantum cavity electrodynamics. Phys. Rev. Lett. 110(1), 013601 (2013)

    ADS  Article  Google Scholar 

  6. 6.

    Crisanti, A., Puglisi, A., Villamaina, D.: Nonequilibrium and information: the role of cross correlations. Phys. Rev. E 85(6), 061127 (2012)

    ADS  Article  Google Scholar 

  7. 7.

    De Vries, B., Principe, J.C.: A theory of neural networks with time delays. In: Lippmann, R.P., Moody., J.E., Touretzky, D.S. (eds.) Advances in Neural Information Processing Systems, pp. 162–168. Morgan Kaufmann, San Mateo, CA (1991)

  8. 8.

    Durve, M., Saha, A., Sayeed, A.: Active particle condensation by non-reciprocal and time-delayed interactions. Eur. Phys. J. E 41(4), 49 (2018)

    Article  Google Scholar 

  9. 9.

    Frank, T.D.: Analytical results for fundamental time-delayed feedback systems subjected to multiplicative noise. Phys. Rev. E 69, 061104 (2004)

    ADS  MathSciNet  Article  Google Scholar 

  10. 10.

    Frank, T.D.: Delay Fokker–Planck equations, Novikov’s theorem, and Boltzmann distributions as small delay approximations. Phys. Rev. E 72(1), 011112 (2005)

    ADS  MathSciNet  Article  Google Scholar 

  11. 11.

    Frank, T.D.: Delay Fokker–Planck equations, perturbation theory, and data analysis for nonlinear stochastic systems with time delays. Phys. Rev. E 71(3), 031106 (2005)

    ADS  Article  Google Scholar 

  12. 12.

    Frank, T.D., Beek, P.J.: Stationary solutions of linear stochastic delay differential equations: applications to biological systems. Phys. Rev. E 64(2), 021917 (2001)

    ADS  Article  Google Scholar 

  13. 13.

    Frank, T.D., Beek, P.J., Friedrich, R.: Fokker–Planck perspective on stochastic delay systems: exact solutions and data analysis of biological systems. Phys. Rev. E 68(2), 021912 (2003)

    ADS  Article  Google Scholar 

  14. 14.

    Gardiner, C.W.: Handbook of Stochastic Methods, 2nd edn. Springer, Berlin (2002)

    Google Scholar 

  15. 15.

    Gernert, R., Loos, S.A.M., Lichtner, K., Klapp, S.H.L.: Feedback control of colloidal transport. In: Schöll, E., Klapp, S., Hövel, P. (eds.) Control of Self-Organizing Nonlinear Systems, pp. 375–392. Springer, Berlin (2016)

  16. 16.

    Giuggioli, L., McKetterick, T.J., Kenkre, V.M., Chase, M.: Fokker–Planck description for a linear delayed Langevin equation with additive Gaussian noise. J. Phys. A 49(38), 384002 (2016)

    ADS  MathSciNet  MATH  Article  Google Scholar 

  17. 17.

    Guillouzic, S., L’Heureux, I., Longtin, A.: Small delay approximation of stochastic delay differential equations. Phys. Rev. E 59(4), 3970 (1999)

    ADS  MATH  Article  Google Scholar 

  18. 18.

    Gupta, V., Kadambari, K., et al.: Neuronal model with distributed delay: analysis and simulation study for gamma distribution memory kernel. Biol. Cybern. 104(6), 369–383 (2011)

    MathSciNet  MATH  Article  Google Scholar 

  19. 19.

    Hall, M.J.W., Rignatto, M.: Ensembles on configuration space. In: Fundamental Theories of Physics. Springer Nature, Switzerland (2016)

  20. 20.

    Kane, D.M., Shore, K.A. (eds.): Unlocking Dynamical Diversity: Optical Feedback Effects on Semiconductor Lasers. Wiley, New York (2005)

    Google Scholar 

  21. 21.

    Kawaguchi, K., Nakayama, Y.: Fluctuation theorem for hidden entropy production. Phys. Rev. E 88(2), 022147 (2013)

    ADS  Article  Google Scholar 

  22. 22.

    Khadka, U., Holubec, V., Yang, H., Cichos, F.: Active particles bound by information flows. Nat. Commun. 9, 3864 (2018)

    ADS  Article  Google Scholar 

  23. 23.

    Kotar, J., Leoni, M., Bassetti, B., Lagomarsino, M.C., Cicuta, P.: Hydrodynamic synchronization of colloidal oscillators. Proc. Natl. Acad. Sci. USA 107(17), 7669–7673 (2010)

    ADS  Article  Google Scholar 

  24. 24.

    Krüeger, M., Maes, C.: The modified Langevin description for probes in a nonlinear medium. J. Phys.: Condens. Matter 29(6), 064004 (2016)

    ADS  Google Scholar 

  25. 25.

    Kubo, R.: The fluctuation-dissipation theorem. Rep. Prog. Phys. 29(1), 255 (1966)

    ADS  MATH  Article  Google Scholar 

  26. 26.

    Küchler, U., Mensch, B.: Langevins stochastic differential equation extended by a time-delayed term. Stoch. Stoch. Rep. 40(1–2), 23–42 (1992)

    MathSciNet  MATH  Article  Google Scholar 

  27. 27.

    Liu, Y., Chen, H., Liu, J., Davis, P., Aida, T.: Synchronization of optical-feedback-induced chaos in semiconductor lasers by optical injection. Phys. Rev. A 63(3), 031802 (2001)

    ADS  Article  Google Scholar 

  28. 28.

    Longtin, A.: Complex Time-Delay Systems: Theory and Applications. Springer, Berlin (2010)

    Google Scholar 

  29. 29.

    Longtin, A., Milton, J.G., Bos, J.E., Mackey, M.C.: Noise and critical behavior of the pupil light reflex at oscillation onset. Phys. Rev. A 41(12), 6992 (1990)

    ADS  Article  Google Scholar 

  30. 30.

    Loos, S.A.M., Gernert, R., Klapp, S.H.L.: Delay-induced transport in a rocking ratchet under feedback control. Phys. Rev. E 89(5), 052136 (2014)

    ADS  Article  Google Scholar 

  31. 31.

    Loos, S.A.M., Klapp, S.H.L.: Force-linearization closure for non-Markovian Langevin systems with time delay. Phys. Rev. E 96(13), 012106 (2017)

    ADS  Article  Google Scholar 

  32. 32.

    Loos, S.A.M., Klapp, S.H.L.: Heat flow due to time-delayed feedback. Sci. Rep. 9, 2491 (2019)

    ADS  Article  Google Scholar 

  33. 33.

    Maes, C.: On the second fluctuation-dissipation theorem for nonequilibrium baths. J. Stat. Phys. 154(3), 705–722 (2014)

    ADS  MathSciNet  MATH  Article  Google Scholar 

  34. 34.

    Maes, C., Thomas, S.R.: From Langevin to generalized Langevin equations for the nonequilibrium rouse model. Phys. Rev. E 87(2), 022145 (2013)

    ADS  Article  Google Scholar 

  35. 35.

    Masoller, C.: Noise-induced resonance in delayed feedback systems. Phys. Rev. Lett. 88, 034102 (2002)

    ADS  Article  Google Scholar 

  36. 36.

    Mehl, J., Lander, B., Bechinger, C., Blickle, V., Seifert, U.: Role of hidden slow degrees of freedom in the fluctuation theorem. Phys. Rev. Lett. 108(22), 220601 (2012)

    ADS  Article  Google Scholar 

  37. 37.

    Mijalkov, M., McDaniel, A., Wehr, J., Volpe, G.: Engineering sensorial delay to control phototaxis and emergent collective behaviors. Phys. Rev. X 6(1), 011008 (2016)

    Google Scholar 

  38. 38.

    Német, N., Parkins, S.: Enhanced optical squeezing from a degenerate parametric amplifier via time-delayed coherent feedback. Phys. Rev. A 94(2), 023809 (2016)

    ADS  Article  Google Scholar 

  39. 39.

    Niculescu, S.I., Gu, K.: Advances in Time-Delay Systems, vol. 38. Springer, Berlin (2012)

    Google Scholar 

  40. 40.

    Novikov, E.A.: Functionals and the random-force method in turbulence theory. Sov. Phys. JETP 20(5), 1290–1294 (1965)

    ADS  MathSciNet  Google Scholar 

  41. 41.

    Puglisi, A., Villamaina, D.: Irreversible effects of memory. EPL 88(3), 30004 (2009)

    ADS  Article  Google Scholar 

  42. 42.

    Rateitschak, K., Wolkenhauer, O.: Intracellular delay limits cyclic changes in gene expression. Math. Biosci. 205(2), 163–179 (2007)

    MathSciNet  MATH  Article  Google Scholar 

  43. 43.

    Reimann, P.: Brownian motors noisy transport far from equilibrium. Phys. Rep. 361, 57–265 (2002)

    ADS  MathSciNet  MATH  Article  Google Scholar 

  44. 44.

    René, A., Longtin, A.: Mean, covariance, and effective dimension of stochastic distributed delay dynamics. Chaos 27(11), 114322 (2017)

    ADS  MathSciNet  MATH  Article  Google Scholar 

  45. 45.

    Rosinberg, M.L., Tarjus, G., Munakata, T.: Influence of time delay on information exchanges between coupled linear stochastic systems. Phys. Rev. E 98(3), 032130 (2018)

    ADS  Article  Google Scholar 

  46. 46.

    Rosinberg, M.L., Munakata, T., Tarjus, G.: Stochastic thermodynamics of Langevin systems under time-delayed feedback control: second-law-like inequalities. Phys. Rev. E 91, 042114 (2015)

    ADS  MathSciNet  Article  Google Scholar 

  47. 47.

    Schneider, I.: Delayed feedback control of three diffusively coupled Stuart–Landau oscillators: a case study in equivariant Hopf bifurcation. Philos. Trans. R. Soc. A 371(1999), 20120472 (2013)

    ADS  MathSciNet  MATH  Article  Google Scholar 

  48. 48.

    Schneider, I., Bosewitz, M.: Eliminating restrictions of time-delayed feedback control using equivariance. Discret. Contin. Dyn. Syst. A 36(1), 451–467 (2016)

    MathSciNet  MATH  Article  Google Scholar 

  49. 49.

    Schöll, E., Klapp, S.H.L., Hövel, P. (eds.): Control of Self-organizing Nonlinear Systems. Springer, Berlin (2016)

    Google Scholar 

  50. 50.

    Schöll, E., Schuster, H.G. (eds.): Handbook of Chaos Control. Wiley, New York (2008)

    Google Scholar 

  51. 51.

    Seifert, U.: Stochastic thermodynamics, fluctuation theorems and molecular machines. Rep. Prog. Phys. 75(12), 126001 (2012)

    ADS  Article  Google Scholar 

  52. 52.

    Sekimoto, K.: Stochastic Energetics, vol. 799. Springer, Berlin (2010)

    Google Scholar 

  53. 53.

    Siegle, P., Goychuk, I., Hänggi, P.: Markovian embedding of fractional superdiffusion. EPL 93(2), 20002 (2011)

    ADS  Article  Google Scholar 

  54. 54.

    Siegle, P., Goychuk, I., Talkner, P., Hänggi, P.: Markovian embedding of non-Markovian superdiffusion. Phys. Rev. E 81(1), 011136 (2010)

    ADS  Article  Google Scholar 

  55. 55.

    Smith, H.L.: An Introduction to Delay Differential Equations with Applications to the Life Sciences, vol. 57. Springer, New York (2011)

    Google Scholar 

  56. 56.

    Tambue, A., Brown, E.K., Mohammed, S.: A stochastic delay model for pricing debt and equity: numerical techniques and applications. Commun. Nonlinear Sci. Numer. Simul. 20(1), 281–297 (2015)

    ADS  MathSciNet  MATH  Article  Google Scholar 

  57. 57.

    Tsimring, L.S., Pikovsky, A.: Noise-induced dynamics in bistable systems with delay. Phys. Rev. Lett. 87(25), 250602 (2001)

    ADS  Article  Google Scholar 

  58. 58.

    Villamaina, D., Baldassarri, A., Puglisi, A., Vulpiani, A.: The fluctuation-dissipation relation: how does one compare correlation functions and responses? J. Stat. Mech. Theory Exp. 2009(07), P07024 (2009)

    Article  Google Scholar 

  59. 59.

    Waibel, A., Hanazawa, T., Hinton, G., Shikano, K., Lang, K.J.: Backpropagation: Theory, Architectures and Applications. Lawrence Erlbaum Associates, Mahwah, NJ (1995)

    Google Scholar 

  60. 60.

    Zakharova, A., Loos, S.A.M., Siebert, J., Gjurchinovski, A., Claussen, J.C., Schöll, E.: Controlling chimera patterns in networks: interplay of structure, noise, delay. In: Schöll, E., Klapp, S., Hövel, P. (eds.) Control of Self-Organizing Nonlinear Systems, pp. 3–23. Springer, Berlin (2016)

  61. 61.

    Zheng, Y., Sun, X.: Governing equations for Probability densities of stochastic differential equations with discrete time delays. Discret. Contin. Dyn. Syst. Ser. B 22(9) (2017)

  62. 62.

    Zwanzig, R.: Nonlinear generalized Langevin equations. J. Stat. Phys. 9(3), 215–220 (1973)

    ADS  Article  Google Scholar 

Download references


This work was funded by the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) - Projektnummer 163436311 - SFB 910. We further thank Christian Kuehn for fruitful discussions.

Author information



Corresponding author

Correspondence to Sarah A. M. Loos.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Funded by the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation)—Projektnummer 163436311 - SFB 910.

Communicated by Abishek Dhar.


Appendix A: The Discrete Delayed Case as Limit of the Coarse-Grained Dynamics

In this appendix, we show how the dynamical equation of the delayed system (1) can be recovered by taking the limit \(n\rightarrow \infty \) of the corresponding equation for the embedded system (7). To this end, we project onto the the dynamics of the variable \(X_0\). Until completion of the projection, we will keep n finite. Formally integrating Eq. (7b) (by first solving the homogeneous problem and then finding a specific solution by variation of constants) yields

$$\begin{aligned} X_j(t)= X_j(-\tau ) e^{-n (t+\tau ) /\tau }+\frac{n}{\tau }\int _{-\tau }^{t} e^{-n(t-s)/\tau } X_{j-1}(s) \mathrm {d}s, ~~~\forall j \in \{1,2,\ldots ,n \}\, . \end{aligned}$$

In principle, this solution can be simplified using an appropriate initial condition for the auxiliary variables, for example \(X_j(0) = 0, ~\forall j>0\). However, we here proceed with the general case, i. e., without specifying the initial condition. As a next step (assuming \(n>1\)), we iteratively plug Eq. (23) into the corresponding solution for \(j+1\) [given by Eq. (23) with \(j+1\) instead of j]. For \(1\le j < n\), we find

$$\begin{aligned} X_{j+1}(t)&= \left[ X_{j+1}(-\tau ) +\frac{n}{\tau } X_{j}(-\tau ) \,t\right] e^{-n (t+\tau ) /\tau }\nonumber \\&\quad +\frac{n^2}{\tau ^2} \int _{-\tau }^{t} \left\{ \int _{-\tau }^{s} e^{-n (t-s')/\tau } X_{j-1}(s') \mathrm {d}s' \right\} \mathrm {d}s. \end{aligned}$$

Using the Heaviside function defined by \(\varTheta (z)=0 ,\forall \, z < 0\) and \(\varTheta (z)=1,\forall \, 0\le z\), we can simplify the double integral,

$$\begin{aligned} X_{j+1}(t)&= \left[ X_{j+1}(-\tau ) + \frac{n}{\tau } X_{j}(-\tau ) \,t \right] e^{-n (t+\tau ) /\tau } \nonumber \\&\quad + \frac{n^2}{\tau ^2} \int _{-\tau }^{s} \left\{ \int _{-\tau }^{t} \varTheta (s-s') \mathrm {d}s \right\} e^{-n (t-s')/\tau } X_{j-1}(s') \mathrm {d}s' , \end{aligned}$$
$$\begin{aligned}&= \left[ X_{j+1}(-\tau ) + \frac{n}{\tau } X_{j}(-\tau ) \,t \right] e^{-n (t+\tau ) /\tau } \nonumber \\&\quad + \int _{-\tau }^{t} \frac{n^2}{\tau ^2} [t-s'] e^{-n (t-s')/\tau } X_{j-1}(s') \mathrm {d}s' . \end{aligned}$$

Repeating this iterative procedure, we finally obtain for \(j=n\)

$$\begin{aligned} X_{n}(t)= & {} \left[ X_{n}(-\tau )+\frac{n}{\tau }X_{n-1}(-\tau )t\right. \nonumber \\&\left. +\frac{n^2}{\tau ^2}X_{n-2}(-\tau )t^2+ \cdots \right] \, e^{-n (t+\tau ) /\tau } + \int _{-\tau }^{t} K_n(t-s) X_{0}(s) \mathrm {d}s , \end{aligned}$$

with the Gamma-distributed memory kernel defined in Eq. (8). The first term in (24d) vanishes trivially, if the auxiliary variables satisfy the initial condition \(X_{j\in \{1,2,\ldots ,n\}}(-\tau )=0\). Interestingly, this terms also vanishes if the asymptotic behavior at \(t\rightarrow \infty \) is considered, due to the exponential damping with time. Plugging (24d) into (7a) [neglecting the initial condition term] then yields the projected Eq. (8).

Now we consider the limit \(n\rightarrow \infty \) of the memory kernel (8). First, we find that the mean value of the kernel and the integral over it are, independently of the value of n, given by \(\mu =\int _{0}^{\infty } K_n(T)T \mathrm {d}T =\tau \) and \(\int _{0}^{\infty } K_n(T) \mathrm {d}T =1\), respectively. Furthermore, for large n, the value of the kernel at \(\tau \) can be estimated by using the Stirling formula \(n!\approx \sqrt{2\pi n}(n/e)^n\), yielding \(K_n(\tau )= \frac{{n}^n \,e^{-n}}{(n-1)! \tau } \approx \frac{n!}{(n-1)! \sqrt{2\pi n}\tau }=\sqrt{\frac{n }{ 2\pi }}\frac{1}{\tau }\rightarrow \infty .\) On the other hand, the variance of the kernel vanishes for \(n\rightarrow \infty \), since \(\int _{0}^{\infty } K_n(T)T^2 \mathrm {d}T -\mu ^2=\tau ^2/{n}\rightarrow 0\). This implies

$$\begin{aligned} \lim _{n \rightarrow \infty } K_n(T)= \delta (T-\tau ). \end{aligned}$$

Thus, the projected equation (8) is indeed equal to Eq. (1) in the limit \(n\rightarrow \infty \). We note that even the transient dynamics (i. e., finite t) is equivalent for the initial conditions \(X_{j\in \{1,2,\ldots ,n\}}(-\tau )\equiv 0\), as mentioned above.

Appendix B: Connection to Fokker–Planck Hierarchy from Novikov’s Theorem

While the FPE for \(\rho _1\) obtained by our approach is equivalent to the first member of the FP hierarchy from Novikov’s theorem (2), the second members differ. However, the apparent disagreement can be resolved. Indeed, as we show below, our result (17) can be transformed to the representation obtained from Novikov’s theorem [given in (3)]. In the following, we will consider the term which differs, i. e., the drift term \(\partial _{x_\tau } [\rho _2(x,t;x_\tau ,t-\tau )\)\( \langle {\dot{X}} ( t - \tau ) |x=X(t) , x_\tau =X({t - \tau })\rangle ]\). To this end, we first use the definition of \(\rho _2\) via delta-distributions, plug in the LE (1), which yields

$$\begin{aligned}&\partial _{x_\tau } \left[ \langle {\dot{X}} ( t - \tau ) |x=X(t) ,x_\tau =X({t - \tau })\rangle \rho _2(x,t;x_\tau ,t-\tau ) \right] \nonumber \\&\quad = \partial _{x_\tau } \left\langle {\dot{X}}(t-\tau ) \,\delta [X(t)-x]\delta [X(t-\tau )-x_\tau ] \right\rangle \nonumber \\&\quad {\mathop {=}\limits ^{\mathrm {LE}}} \partial _{x_\tau } \left\langle F[X(t-\tau ),X(t-2\tau )]\,\delta [X(t)-x]\delta [X(t-\tau )-x_\tau ] \right\rangle \nonumber \\&\qquad \quad + \partial _{x_\tau } \left\langle \sqrt{2D_0}\xi (t-\tau ) \,\delta [X(t)-x]\delta [X(t-\tau )-x_\tau ] \right\rangle . \end{aligned}$$

At this point, we have two correlations to deal with. The first one (involving \(F_{}\)) can easily be simplified by expressing the ensemble average with the help of the PDF \(\rho _3\), i. e.,

$$\begin{aligned}&\partial _{x_\tau } \Big \langle F\left[ X(t-\tau ),X(t-2\tau )\right] \,\delta [X(t)-x]\delta [X(t-\tau )-x_\tau ] \Big \rangle \nonumber \\&\quad = \partial _{x_\tau } \iiint _{\varOmega }^{ }F_{}(x_\tau ,x_{2\tau }) \,\delta [X(t)-x]\delta [X(t-\tau )-x_\tau ]\times \nonumber \\&\qquad \quad \rho _3(x,t;x_\tau ,t-\tau ;x_{2\tau },t-2\tau ) \mathrm {d}x \mathrm {d}x_\tau \mathrm {d}x_{2\tau } \nonumber \\&\quad = \partial _{x_\tau } \int _{\varOmega }^{ } F_{}(x_\tau ,x_{2\tau }) \rho _3(x,t;x_\tau ,t-\tau ;x_{2\tau },t-2\tau ) \mathrm {d}x_{2\tau }. \end{aligned}$$

The remaining term on the right side of (26a) (involving \(\xi \)) is simplified with the help of Novikov’s theorem (28) (see Appendix 1). Specifically, we define \(\varLambda [\xi ]:= \delta [x-X(t)]\, \delta [x_\tau -X(t\!-\!\tau )] \), which yields

$$\begin{aligned}&\partial _{x_\tau } \left\langle \sqrt{2D_0}\xi (t-\tau ) \,\delta [X(t)-x]\delta [X(t-\tau )-x_\tau ] \right\rangle \nonumber \\&\quad =\sqrt{2D_0}\partial _{x_\tau } \left\langle \xi (t-\tau ) \varLambda [\xi ] \right\rangle , \nonumber \\&\quad = \sqrt{2D_0} \partial _{x_\tau } \left\langle \frac{\delta \varLambda [\xi ] }{\delta \xi (t-\tau )} \right\rangle \nonumber \\&\quad = \sqrt{2D_0}\partial _{x_\tau } \left[ \left\langle \frac{\delta \varLambda [\xi ] }{\delta X(t)}\frac{\delta X(t) }{\delta \xi (t-\tau )} \right\rangle +\left\langle \frac{\delta \varLambda [\xi ] }{\delta X(t-\tau )} \frac{\delta X(t-\tau ) }{\delta \xi (t-\tau )}\right\rangle \right] \nonumber \\&\quad = -\sqrt{2D_0}\partial _{x_\tau } \left\langle \partial _{x }\delta [x-X(t)]\, \delta [x_\tau -X(t\!-\!\tau )] {\frac{\delta X(t) }{\delta \xi (t-\tau )}} \right\rangle \nonumber \\&\qquad -\sqrt{2D_0}\partial _{x_\tau }\left\langle \partial _{x_\tau } \delta [x-X(t)]\, \delta [x_\tau -X(t\!-\!\tau )]\underbrace{\frac{\delta X(t-\tau ) }{\delta \xi (t-\tau )}}_{\psi }\right\rangle . \end{aligned}$$

In the last step we have plugged in the definition of \(\varLambda \) and used the chain rule. Now, the term denoted \(\psi \) is calculated by replacing \(X(t-\tau )\) with the integral form of the LE (1), i. e., \(X(t')= \int _{0}^{t'}\left[ F[X(s),X(s-\tau )]+\sqrt{2D_0}\xi (s)\right] \mathrm {d}s\), \(\forall \,t'>0\), yielding

$$\begin{aligned} \psi =\frac{\delta X(t-\tau )}{\delta \xi (t-\tau )}&= \frac{\delta }{\delta \xi (t-\tau )} \int _{0}^{t-\tau } \left[ F[X(s),X(s-\tau )]+\sqrt{2D_0}\xi (s)\right] \mathrm {d}s\nonumber \\&=\sqrt{2D_0}\,\int _{0}^{t-\tau } \delta [s-(t-\tau )]\mathrm {d}s=\sqrt{\frac{D_0}{2}}, \end{aligned}$$

independently of the specific form of \(F_{}\). We note that the other functional derivative in (26c) cannot be treated analogously, since evaluation of the functional derivative of the trajectory X w. r. t. the earlier noise \(\xi \) requires the formal solution of (1), which is only known for linear systems. Indeed, for a LE with \( F= -c_1 X(t) -c_2 X_{}(t-\tau )\), one finds \(\frac{\delta X(t) }{\delta \xi (t-\tau )}=\sqrt{2D_0} e^{-c_1 \tau }\) by using the method of steps, see [31]. For general nonlinear forces F, this term must be treated differently, e. g. by approximation methods, but a discussion of this goes beyond the scope of this paper. We therefore keep this functional derivative and only evaluate the delta-distributions. Based on these considerations, we can rewrite (26c) as

$$\begin{aligned} \sqrt{2D_0}\,\partial _{x_\tau } \left\langle \xi (t-\tau ) \varLambda [\xi ] \right\rangle&=-\sqrt{2D_0}\,\partial _{x_\tau }\partial _{x } \left\langle \frac{\delta X(t) }{\delta \xi (t-\tau )}\Big |_{\begin{array}{c} X(t)=x \\ X(t - \tau )=x_\tau \end{array}} \right\rangle \nonumber \\&\qquad \rho _2(x,t;x_{\tau },t-\tau ) -D_0\,\partial _{x_\tau }^2\rho _2(x,t;x_{\tau },t-\tau ). \end{aligned}$$

In combination with (26a,26b), we obtain the identity

$$\begin{aligned}&\partial _{x_\tau } \left\langle {\dot{X}}(t-\tau ) \,\delta [X(t)-x]\delta [X(t-\tau )-x_\tau ] \right\rangle \nonumber \\&\quad = -\sqrt{2D_0}\partial _{x_\tau }\partial _{x} \left\langle \frac{\delta X(t) }{\delta \xi (t-\tau )}\Big |_{\begin{array}{c} X(t)=x \\ X(t - \tau )=x_\tau \end{array}} \right\rangle \rho _2(x,t;x_{\tau },t-\tau ) \nonumber \\&\qquad -D_0 \partial _{x_\tau }^2 \rho _2(x,t;x_{\tau },t-\tau )+\partial _{x_\tau } \int _{\varOmega }^{ } F_{}(x_\tau ,x_{2\tau }) \rho _3(x,t;x_\tau ,t-\tau ;x_{2\tau },t-2\tau ) \mathrm {d}x_{2\tau } \nonumber \\&\quad = \partial _{x_\tau } \left[ \langle {\dot{X}} ( t - \tau ) |x=X(t) , x_\tau =X({t - \tau })\rangle \rho _2(x,t;x_\tau ,t-\tau ) \right] , \end{aligned}$$

proving that the second member of the here presented FP hierarchy, Eq. (17), is identical to the corresponding one (3) obtained from Novikov’s theorem.

Appendix C: Novikov’s Theorem

In this section, we establish the relation [40]

$$\begin{aligned} \big \langle \,\varLambda [\xi ] \xi (t) \,\big \rangle = \Bigg \langle \frac{\delta \varLambda [\xi ]}{\delta \xi (t)}\Bigg \rangle , \end{aligned}$$

for a functional \(\varLambda \) of a Gaussian white noise\(\xi \). It links the functional derivative w. r. t. the noise to the cross-correlation between functional and noise. First, we consider the case where the ensemble is w. r. t. a fixed initial condition \(\phi =X(-\tau \le t\le t)\), yielding \(\varLambda (0)={\varLambda _0}\). In a second step we will generalize towards initial conditions drawn from an arbitrary distribution.

We express the ensemble average \(\langle \cdots \rangle _{\varLambda _0}\) of the left hand side of (28) via the path integral over all possible paths \(\xi \) between time 0 and t, accounting for all possible realizations of the random process \(\xi \) at each instant in time

$$\begin{aligned} \left\langle \varLambda [\xi ]\xi (t) \right\rangle _{\varLambda _0} = \int _{\xi _0}^{\xi _t}\varLambda \left[ \xi |{\varLambda _0}\right] \,\xi \, {\mathcal {P}}[\xi ]\, {\mathcal {D}}[\xi ], \end{aligned}$$

with arbitrary but fixed \(\xi _0:=\xi (t_0)\) and \(\xi _t:=\xi (t)\). Please note that specifying the noise process at the boundaries does not impose a restriction on the generality as we deal with white noise. The weight \({\mathcal {P}}[\xi ]\) of each white noise realization is given by the Gaussian path probability (for arbitrary \(t>0\))

$$\begin{aligned} {\mathcal {P}}[\xi ]={\mathcal {J}} e^{-\frac{1}{2}\int _{0}^{t}\xi ({t'})^2 \mathrm d t'}, \end{aligned}$$

with Jacobian \({\mathcal {J}}\). Now, we rewrite the integrand in (29a) using that the functional derivative of \({\mathcal {P}}[\xi ]\) w. r. t. \( \xi \) is simply \((-\xi ) {\mathcal {P}}[\xi ]\), yielding

$$\begin{aligned} \left\langle \varLambda [\xi ]\xi (t) \right\rangle _{\varLambda _0} =&- \int _{\xi _0}^{\xi _t}\varLambda \left[ \xi |{\varLambda _0}\right] \frac{\delta }{\delta \xi }\!\left\{ e^{-\frac{1}{2}\int _{0}^{t}\xi ({t'})^2 \mathrm d t'}\right\} \,{\mathcal {D}}[\xi ] \nonumber \\ =&- \int _{\xi _0}^{\xi _t} \frac{\delta \left\{ \varLambda [\xi ] {\mathcal {P}}[\xi ] \right\} }{\delta \xi } \,{\mathcal {D}}[\xi ] + \int _{\xi _0}^{\xi _t}\frac{\delta \varLambda [\xi ]}{\delta \xi } {\mathcal {P}}[\xi ] \,{\mathcal {D}}[\xi ]. \end{aligned}$$

In the last step we have used the product rule (and omitted the initial condition \(\varLambda _0\) for sake of a shorter notation). The functional derivative in the first path integral yields

$$\begin{aligned} \int _{\xi _0}^{\xi _t}\frac{\delta \left\{ \varLambda [\xi ] {\mathcal {P}}[\xi ] \right\} }{\delta \xi } {\mathcal {D}}[\xi ]= \int _{\xi _0}^{\xi _t} \varLambda [\xi +\delta \xi ] {\mathcal {P}}[\xi +\delta \xi ] {\mathcal {D}}[\xi ]-\int _{\xi _0}^{\xi _t}\varLambda [\xi ] {\mathcal {P}}[\xi ] {\mathcal {D}}[\xi ]. \end{aligned}$$

Now we use that the variations of the paths \(\xi +\delta \xi \) are already contained in the integral over all paths, and find

$$\begin{aligned} \int _{\xi _0}^{\xi _t}\frac{\delta \left\{ \varLambda [\xi ] {\mathcal {P}}[\xi ] \right\} }{\delta \xi } {\mathcal {D}}[\xi ]= \int _{\xi _0}^{\xi _t}\varLambda [\xi ] {\mathcal {P}}[\xi ] {\mathcal {D}}[\xi ]-\int _{\xi _0}^{\xi _t}\varLambda [\xi ] {\mathcal {P}}[\xi ] {\mathcal {D}}[\xi ]=0, \end{aligned}$$

(as long as \(\int \varLambda [\xi ] {\mathcal {P}}[\xi ] {\mathcal {D}}[\xi ] <\infty \)), see also [19] (on p. 273f). Hence, we obtain from (29c),

$$\begin{aligned} \left\langle \varLambda [\xi ]\xi (t) \right\rangle _{\varLambda _0} =&\int _{\xi _0}^{\xi _t}\frac{\delta \varLambda [\xi ]}{\delta \xi } {\mathcal {P}}[\xi ] \,{\mathcal {D}}[\xi ] = \left\langle \frac{\delta \varLambda [\xi ]}{\delta \xi } \right\rangle _{\varLambda _0}. \end{aligned}$$

This result can readily be generalized to ensembles where the initial conditions are instead drawn from an arbitrary, normalized distribution \(P(\varLambda _0)\). In particular, Eq. (29f) implies

$$\begin{aligned} \left\langle \varLambda [\xi ]\xi (t) \right\rangle = \int \left\langle \varLambda [\xi ]\xi (t) \right\rangle _{\varLambda _0} P(\varLambda _0) \mathrm {d}\varLambda _0 = \int \left\langle \frac{\delta \varLambda [\xi ]}{\delta \xi } \right\rangle _{\varLambda _0} P(\varLambda _0) \mathrm {d}\varLambda _0 =\left\langle \frac{\delta \varLambda [\xi ]}{\delta \xi } \right\rangle . \end{aligned}$$

This is the relation (28), which is often referred to as Novikov’s theorem.

Appendix D: The Approximation in the Regime of Short Delay Times

The approximate two-time PDF (22) is, in general, not reliable, if the delay time is very short, as the underlying assumption is violated. For the moderate \(|k|/V_0\) and small \(\tau \) values considered in Fig. 6a, the corresponding one-time PDF (obtained by marginalization) still gives a reasonable approximation. Here also the other approaches from the literature are justified. For the large (and negative) \(|k|/V_0\) value in Fig. 6b, only the small \(\tau \) expansion yields a reasonable approximation.

Fig. 6

Comparison of different approximations for the one-time PDF in the doublewell potential with linear delay force \(k X(t-\tau )\) at \(V_0=3.5\), for two exemplary values of k. In contrast to Fig. 5 where \(\tau =10\), here a shorter delay time of \(\tau =0.1\) is considered

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Loos, S.A.M., Klapp, S.H.L. Fokker–Planck Equations for Time-Delayed Systems via Markovian Embedding. J Stat Phys 177, 95–118 (2019).

Download citation


  • Time delay
  • Stochastic delay differential equations
  • Fokker–Planck equations
  • Non-Markovian processes
  • Novikov’s theorem