Large deviations conditioned on large deviations I: Markov chain and Langevin equation

Abstract

We present a systematic analysis of stochastic processes conditioned on an empirical observable $Q_T$ defined in a time interval [0, T], for large T. We build our analysis starting with a discrete time Markov chain. Results for a continuous time Markov process and Langevin dynamics are derived as limiting cases. In the large T limit, we show how conditioning on a value of $Q_T$ modifies the dynamics. For a Langevin dynamics with weak noise and conditioned on $Q_T$, we introduce large deviation functions and calculate them using either a WKB method or a variational formulation. This allows us, in particular, to calculate the typical trajectory and the fluctuations around this trajectory when conditioned on a certain value of $Q_T$, for large T.

Appendices

Ensemble equivalence

In this appendix we show that, for large T, the equivalence of ensembles holds for an arbitrary time t. For an earlier derivation of the ensemble equivalence see [47].

As the reasoning is very similar in the five regions of Fig. 1, we will limit our discussion to the case of region II, i.e. for $0\le t \ll T$. Let $P_t(C_T,C,Q \vert C_0)$ be the joint probability of configuration $C_T$ at time T, configuration C at time t, and of the observable $Q_T$ to take value Q given its initial configuration $C_0$ at time 0.

To establish the equivalence of ensembles in (17), we need to show that the conditioned probability in the microcanonical ensemble

$$\begin{aligned} {\mathcal {P}}_t(C \vert Q=qT)=\frac{\sum _{C_T}\sum _{C_0}P_t(C_T,C,Q=qT\vert C_0)R_{0}(C_0)}{\sum _{C'}\left[ \sum _{C_T}\sum _{C_0}P_t(C_T,C',Q=qT\vert C_0)R_{0}(C_0)\right] } \end{aligned}$$

(75)

and the canonical measure

$$\begin{aligned} P_t^{(\lambda )}(C)=\frac{\sum _{C_T}\sum _{C_0}\int dQ\, e^{\lambda Q}P_t(C_T,C,Q\vert C_0)R_{0}(C_0)}{\sum _{C'}\left[ \sum _{C_T}\sum _{C_0}\int dQ\, e^{\lambda Q}P_t(C_T,C',Q\vert C_0)R_{0}(C_0)\right] } \end{aligned}$$

(76)

converge to the same distribution for large T when $\lambda$ and q are related by (12).

For this, we write, in terms of the probability (5),

$$\begin{aligned} P_t(C_T,C,Q=qT\vert C_0)=\int dQ_t\;P_{T-t}(C_T,qT-Q_t \vert C)\;P_t(C,Q_t \vert C_0) \end{aligned}$$

(77)

and use the large T asymptotics (13), which gives

$$\begin{aligned} P_{T-t}(C_T,qT-Q_t \vert C)\simeq e^{-(T-t)\phi (q)-(tq-Q_t)\phi '(q)}\sqrt{\frac{\phi ''(q)}{2\pi T}}R_{\phi '(q)}(C_T)L_{\phi '(q)}(C) \end{aligned}$$

Substituting in (75) and simplifying the expression for large T we get the microcanonical probability

$$\begin{aligned} P_t(C \vert Q=qT)\simeq \frac{L_{\phi '(q)}(C)\sum _{C_0}G_t^{(\phi '(q))}(C\vert C_0)\;R_{0}(C_0)}{\sum _{C'} L_{\phi '(q)}(C')\sum _{C_0}G_t^{(\phi '(q))}(C'\vert C_0)\;R_{0}(C_0)} \end{aligned}$$

(78)

where $G_t^{(\lambda )}(C \vert C_0)$ is defined in (6). On the other hand, using (9) for large T we get the canonical probability

$$\begin{aligned} P_t^{(\lambda )}(C)\simeq \frac{L_{\lambda }(C)\sum _{C_0}G_t^{(\lambda )}(C\vert C_0)\;R_{0}(C_0)}{\sum _{C'} L_{\lambda }(C')\sum _{C_0}G_t^{(\lambda )}(C'\vert C_0)\;R_{0}(C_0)} \end{aligned}$$

(79)

Clearly the two probabilities in the two ensembles coincide for $\lambda =\phi '(q)$. Replacing $G_t^{(\lambda )}(C\vert C_0)$ by $M_{\lambda }^t(C,C_0)$ in (79) leads to the canonical measure (18b).

The same reasoning can be easily adapted in the other regions of Fig. 1.

Continuous time Markov process

In this Appendix, we describe a continuous time limit of the Markov process, illustrated in Fig. 7. In this, the empirical observable analogous to (4) is the $dt\rightarrow 0$ limit of

$$\begin{aligned} Q_T=dt\sum _{i=0}^{\frac{T}{dt}-1}f(C_i)+\sum _{n}g(C_n^+,C_n^-) \end{aligned}$$

(80)

where $t=i\, dt$, and $(C_n^-, C_n^+)$ are the configurations before and after the nth jump during the time interval [0, T].

Fig. 7

A schematic of a time evolution in a Markov process with discrete time steps dt. The continuous time limit is obtained by taking $dt\rightarrow 0$ limit

From (7) we get

$$\begin{aligned} G_{T}^{(\lambda )}(C' \vert C_0)=\sum _{C}M_{\lambda }(C',C)\,G_{T-dt}^{(\lambda )}(C\vert C_0) \end{aligned}$$

where

$$\begin{aligned} M_\lambda (C',C)={\left\{ \begin{array}{ll} M_0(C',C)e^{\lambda [dt\, f(C)+g(C',C)]} &{} \text {for }C'\ne C,\\ M_0(C,C)e^{\lambda \, dt \, f(C)} &{} \text {for }C'= C.\end{array}\right. } \end{aligned}$$

Using the construction (27) for $M_0(C',C)$ we take the continuous time limit $dt\rightarrow 0$ and get

$$\begin{aligned} \frac{d}{dT} G_T^{(\lambda )}(C'\vert C_0)=\sum _{C}\mathcal {M}_{\lambda }(C',C)G_T^{(\lambda )}(C\vert C_0) \end{aligned}$$

(81)

where we recover an earlier result [6, 47] for the tilted matrix $\mathcal {M}_{\lambda }$ for the continuous time process, given by

$$\begin{aligned} \mathcal {M}_{\lambda }(C',C)={\left\{ \begin{array}{ll} e^{\lambda g(C',C)}\mathcal {M}_0(C',C) &{} \text {for } C'\ne C,\\ \lambda f(C)-\sum _{C^{''}\ne C}\mathcal {M}_0(C^{''},C) &{} \text {for } C'= C. \end{array}\right. } \end{aligned}$$

(82)

This shows that the generating function is the $(C,C_0)$th element of $e^{T\mathcal {M}_{\lambda }}$, i.e.

$$\begin{aligned} G_T^{(\lambda )}(C\vert C_0)=e^{T\mathcal {M}_{\lambda }}(C,C_0) \end{aligned}$$

(83)

Although (81) resembles a Master equation, the tilted matrix $\mathcal {M}_{\lambda }$ is not a Markov matrix as $\sum _{C'} \mathcal {M}_{\lambda }(C',C)$ does not necessarily vanish.

For large T, one would get $G_{T}^{(\lambda )}(C \vert C_0)\simeq e^{T \mu (\lambda )}R_{\lambda }(C)L_{\lambda }(C_0)$ where the cumulant generating function $\mu (\lambda )$ is the largest eigenvalue of $\mathcal {M}_\lambda$ with $L_{\lambda }(C)$ and $R_\lambda (C)$ being the left and right eigenvectors, respectively. (Note the difference with the discrete time case (9), where $\mu (\lambda )$ is the logarithm of the largest eigenvalue of the tilted matrix $M_\lambda$ in (8).)

In a similar construction, one could get the continuous time limit of the canonical measure (18a–18d) and its time evolution (20a–20d). The analysis is straightforward and we present only the final result.

The time evolution of the canonical measure $P_t^{(\lambda )}(C)$ for a continuous time Markov process is also a Markov process

$$\begin{aligned} \frac{d}{dt} P_t^{(\lambda )}(C')=\sum _C\mathcal {W}_t^{(\lambda )}(C',C)P_t^{(\lambda )}(C) \end{aligned}$$

(84)

where $\mathcal {W}_t^{(\lambda )}(C',C)$ is the transition rate from C to $C'$ at time t in the canonical ensemble. The canonical measure and transition rate have different expressions in the five regions indicated in Fig. 1. Their expression is given below, where we use a matrix product notation, e.g. $[L_\lambda \mathcal {M}_\lambda ](C)\equiv \sum _{C'}L_\lambda (C') \mathcal {M}_\lambda (C',C)$.

• 1. Region I.

  $$\begin{aligned} P_t^{(\lambda )}(C)=&\frac{[L_{\lambda }e^{-t\mathcal {M}_0}](C)R_{0}(C)}{\sum _{C'}L_{\lambda }(C')R_{0}(C')} \end{aligned}$$

  (85a)
  $$\begin{aligned} \mathcal {W}_t^{(\lambda )} (C',C)=&{[L_\lambda e^{-t\mathcal {M}_0}](C') \over [L_\lambda e^{-t\mathcal {M}_0}](C) }\mathcal {M}_0(C',C) -{\left[ L_\lambda e^{-t\mathcal {M}_0}\mathcal {M}_0\right] (C) \over [L_\lambda e^{-t\mathcal {M}_0}](C) }\;\delta _{C',C} \end{aligned}$$

  (85b)

• 2. Region II.

  $$\begin{aligned} P_t^{(\lambda )}(C)&=\frac{L_{\lambda }(C)[e^{t\mathcal {M}_\lambda }R_0](C)}{e^{t\mu (\lambda )} \sum _{C'}L_{\lambda }(C')R_{0}(C')} \end{aligned}$$

  (86a)
  $$\begin{aligned} \mathcal {W}_t^{(\lambda )}(C',C)&= { L_\lambda (C') \over L_\lambda (C) }\mathcal {M}_\lambda (C',C)-\mu (\lambda )\delta _{C',C} \end{aligned}$$

  (86b)

• 3. Region III.

  $$\begin{aligned} P_t^{(\lambda )}(C)&=L_{\lambda }(C)R_\lambda (C) \end{aligned}$$

  (87a)
  $$\begin{aligned} \mathcal {W}_t^{(\lambda )}(C',C)&= { L_\lambda (C') \over L_\lambda (C) }\mathcal {M}_\lambda (C',C) -\mu (\lambda )\delta _{C',C} \end{aligned}$$

  (87b)

• 4. Region IV.

  $$\begin{aligned} P_t^{(\lambda )}(C)=&\frac{[L_0\, e^{(T-t)\mathcal {M}_\lambda }](C)R_{\lambda }(C)}{e^{(T-t)\mu (\lambda )} \sum _{C'}R_{\lambda }(C')} \end{aligned}$$

  (88a)
  $$\begin{aligned} \mathcal {W}_t^{(\lambda )}(C',C)=&{[L_0 \, e^{(T-t)\mathcal {M}_\lambda }](C') \over [L_0\, e^{(T-t)\mathcal {M}_\lambda }](C) }\mathcal {M}_\lambda (C',C) - { \left[ L_0\, e^{(T-t)\mathcal {M}_\lambda } \mathcal {M}_\lambda \right] (C)\over \left[ L_0\, e^{(T-t)\mathcal {M}_\lambda }\right] (C) } \delta _{C',C} \end{aligned}$$

  (88b)

  where the left eigenvector $L_0$ for the original (unconditioned) evolution is a unit vector such that $[L_0\, \mathcal {M}_\lambda ](C)\equiv \sum _{C'}\mathcal {M}_\lambda (C',C)$.

• 5. Region V.

  $$\begin{aligned} P_t^{(\lambda )}(C)=&\,\frac{[e^{(t-T)\mathcal {M}_0}R_{\lambda }](C)}{\sum _{C'}R_{\lambda }(C')} \end{aligned}$$

  (89a)
  $$\begin{aligned} \mathcal {W}_t^{(\lambda )}(C',C)=&\,\mathcal {M}_0(C',C) \end{aligned}$$

  (89b)

These expressions of $P_{t}^{(\lambda )}$ and $\mathcal {W}_t^{(\lambda )}$, particularly (86a) and (87), have been derived earlier in [47]. The results for $\mathcal {W}_t^{(\lambda )}$ can be viewed as a generalization of the Doob’s h-transformation [42, 47].

One can verify the property $\sum _{C'}\mathcal {W}_t^{(\lambda )}(C',C)=0$ in all five regions. Moreover, setting $\lambda =0$, and $L_0(C)=1$, gives $\mathcal {W}_t^{(0)}(C',C)=\mathcal {M}_0(C',C)$, as one would expect.

Langevin dynamics as a limit of a Markov process

In this appendix, we show how the case of Langevin dynamics in Sect. 3 can be obtained as a continuous limit of the discrete time Markov process in Sect. 2.

Let us consider a jump process on a one-dimensional lattice where a configuration C is given by the site index i as indicated in Fig. 8. Only nearest neighbor jumps are allowed with transition rates that we take of the form

$$\begin{aligned} M_0(i\pm 1,i)=\frac{\epsilon }{2}\pm \frac{a}{2}F(a\,i) \end{aligned}$$

(90)

with $M_0(i,i)=1-\epsilon$, where a is the unit lattice spacing, ${\epsilon <1}$ is a fixed parameter, and F(x) is an arbitrary function defined on the lattice.

Fig. 8

A jump process on a one-dimensional chain where a particle jumps to its nearest neighbour site with rates indicated in the figure

The probability $P_{t,i}$ of the jump process to be in site i at time t satisfies the Master equation

$$\begin{aligned} P_{t+1,i}=M_0(i,i+1)P_{t,i+1}+M_0(i,i-1)P_{t,i-1}+M_0(i,i)P_{t,i} \end{aligned}$$

(91)

Taking the limit $a\rightarrow 0$, keeping $\epsilon$ arbitrary, one can easily see that $P_{a^2 t}(a\, i)\equiv P_{t,i}$ follows the Fokker–Planck equation (29). This shows that the continuous limit of the jump process is indeed identical to the Langevin dynamics (28).

One can similarly obtain the tilted Langevin dynamics from the continuous limit of the jump process when weighted by $e^{\lambda Q}$ with the observable Q in (24). For this we define

$$\begin{aligned} f_t(i)=a^2 f(a\, i, a^2 t)\qquad \text {and}\qquad {g_t(j,i)=(j-i)\, a\, \left\{ \alpha \, h(a\, j, a^ 2 t)+(1-\alpha ) \,h(a\, i, a^ 2 t)\right\} } \end{aligned}$$

where $\alpha$ is the parameter, which specifies the prescription (Îto or Stratonovich) as in (31). Then, the continuous limit of (24) corresponds to an observable Q of the Langevin dynamics

$$\begin{aligned} Q=\int dt\, f(X_t,t)+{\int dX_t\, h(X_t,t)} \end{aligned}$$

(92)

In the expression (25a) for the canonical measure if we define $H^{(\lambda )}_{a^2 t}(a\, i)\equiv Z_t^{(\lambda )}(i)$ and ${\mathbb {H}}^{(\lambda )}_{a^2 t}(a\, i)\equiv {\mathbb {Z}}_t^{(\lambda )}(i)$, then in the continuous limit $a\rightarrow 0$ we get the canonical measure for the Langevin dynamics weighted by $e^{\lambda Q}$ with Q in (92):

$$\begin{aligned} P_t^{(\lambda )} (x)=\frac{ H^{(\lambda )}_t(x)\, {\mathbb {H}}^{(\lambda )}_t(x)}{\int dy\, H^{(\lambda )}_t(y) \, {\mathbb {H}}^{(\lambda )}_t(y)} \end{aligned}$$

(93)

The time evolution of $H^{(\lambda )}_t(x)$ and ${\mathbb {H}}^{(\lambda )}_t(x)$ are obtained from (25b–25c) for the jump process by taking the $a\rightarrow 0$ limit, keeping $\epsilon$ fixed. We get

$$\begin{aligned} \frac{d}{dt}H^{(\lambda )}_t (x)&= \lambda f(x,t)H^{(\lambda )}_t(x)-\left( \frac{d}{dx}-\lambda \,h(x,t) \right) F(x)H^{(\lambda )}_t(x) +\frac{\epsilon }{2} \bigg (\frac{d^2}{dx^2}H^{(\lambda )}_t(x)\\&\quad {-}2\lambda \,h(x,t)\frac{d}{dx}H^{(\lambda )}_t(x){-}2(1{-}\alpha )\lambda \, \partial _xh(x,t)H^{(\lambda )}_t(x)+\lambda ^2 h(x,t)^2H^{(\lambda )}_t(x)\bigg ) \end{aligned}$$

(94a)

$$\begin{aligned} -\frac{d}{dt} {\mathbb {H}}^{(\lambda )}_t (x)&= \lambda f(x,t) {\mathbb {H}}^{(\lambda )}_t(x)+F(x)\left( \frac{d}{dx}+\lambda \,h(x,t)\right) {\mathbb {H}}^{(\lambda )}_t(x) + \frac{\epsilon }{2}\bigg (\frac{d^2}{dx^2}{\mathbb {H}}^{(\lambda )}_t(x)\\&\quad +2\lambda \,h(x,t)\frac{d}{dx}{\mathbb {H}}^{(\lambda )}_t(x)+2\alpha \lambda \, \partial _xh(x,t){\mathbb {H}}^{(\lambda )}_t(x) +\lambda ^2 h(x,t)^2{\mathbb {H}}_t(x)\bigg ) \end{aligned}$$

(94b)

Similarly, the continuous limit of (21, 26) gives the Fokker–Planck equation

$$\begin{aligned} \frac{d}{dt}P_t^{(\lambda )}(x)=-\frac{d}{dx}\left[ F_t^{(\lambda )}(x)P_t^{(\lambda )}(x) \right] +\frac{\epsilon }{2}\frac{d^2}{dx^2}P_t^{(\lambda )}(x) \end{aligned}$$

(95a)

where the modified force

$$\begin{aligned} F_t^{(\lambda )}(x)=F(x)+\epsilon \left( \lambda \,h(x,t)+\frac{d}{dx}\log {\mathbb {H}}_t^{(\lambda )}(x)\right) \end{aligned}$$

(95b)

This gives the time evolution of the Langevin dynamics when it is weighted by the observable (92).

Remarks

• 1. In the derivation of (95a) we have used that the denominator in (93) is time independent, which can be checked using (94a, 94b).

• 2. The Fokker–Planck equation (95a) shows that the effect of biasing a Langevin dynamics by $e^{\lambda Q}$ with an arbitrary time dependent observable (92) is described by another Langevin dynamics with a modified force (95b), but the noise strength $\epsilon$ remains unchanged. This works even without a large parameter T (see [38, 49] for earlier examples of conditioned dynamics).

Our results in Sect. 3 belongs to a particular case, where the observable (92) is defined in a large time interval [0, T]. This corresponds to [see (30)]

$$\begin{aligned} f(x,t)={\left\{ \begin{array}{ll} f(x) \text { for } t\in [0,T],\\ 0 \text { otherwise,}\end{array}\right. } \quad \text {and}\qquad h(x,t)={\left\{ \begin{array}{ll} h(x) \text { for }t\in [0,T],\\ 0 \text { otherwise.}\end{array}\right. } \end{aligned}$$

In this case, (94a, 94b) gives

$$\begin{aligned} \frac{d}{dt}H^{(\lambda )}_t (x)={\mathcal {L}}(t)\cdot H^{(\lambda )}_t (x),\qquad \frac{d}{dt}{\mathbb {H}}^{(\lambda )}_t (x)=-{\mathcal {L}}^\dagger (t)\cdot {\mathbb {H}}^{(\lambda )}_t (x) \end{aligned}$$

(96)

where ${\mathcal {L}}(t)={\mathcal {L}}_\lambda$ for $t\in [0,T]$ and ${\mathcal {L}}(t)={\mathcal {L}}_0$ outside this time window, with the operators defined in (29) and (33); similar for the conjugate operator ${\mathcal {L}}^\dagger (t)$.

This gives, for example, for $t\le 0$, $H^{(\lambda )}_t (x)=r_0(x)$ (defined in (35)), whereas ${\mathbb {H}}^{(\lambda )}_t (x)\sim e^{-t{\mathcal {L}}_0^\dagger }\cdot e^{T{\mathcal {L}}_\lambda ^\dagger }\cdot \ell _0(x)$, (upto a constant pre-factor) which in the large T limit, gives ${\mathbb {H}}^{(\lambda )}_t (x)\sim e^{T\mu (\lambda )}\left[ e^{-t{\mathcal {L}}_0^\dagger }\cdot \ell _\lambda \right] (x)$. Substituting these in (93) and (95b) we get the expression for the canonical measure (38a) and effective force (39a), respectively, in region I of Fig. 1. Results for rest of the regions in Sect. 3.2 can be obtained similarly.

Lastly, from (25b) one could see that for the observable (4), the generating function $G_T^{(\lambda )}(C\vert C_0)$ in (6) is identical to $Z_T^{(\lambda )}(C)$ if one sets $Z_0^{(\lambda )}(C)=\delta _{C,C_0}$. Then from the above calculation it is straightforward to show that in the continuous limit one would get (32).

Path integral formulation

The path integral formulation of a Fokker–Planck equation is standard [83]. The Fokker–Planck equation (29) can be written as

$$\begin{aligned} \frac{dP_t(x)}{dt}=-\frac{d}{dx}\left[ F(x)P_t(x)\right] +\frac{\epsilon }{2}\frac{d^2}{dx^2}P_t(x)\equiv -\mathcal {H}\left( x,-i \frac{d}{dx} \right) P_t(x) \end{aligned}$$

such that $H(x,p)=F'(x)+iF(x)p+\frac{\epsilon }{2}p^2$. Considering a small increment dt in time, we get

$$\begin{aligned} P_{t+dt}(x)\simeq&\int dx'\left[ 1-dt\,\mathcal {H}\left( x,-i \frac{d}{dx} \right) \right] \delta (x-x') P_t(x') \\\simeq&\int \frac{dp\, dx'}{2\pi }\left[ 1-dt\,\mathcal {H}\left( x,p \right) \right] e^{i\,p(x-x')} P_t(x') \end{aligned}$$

where we used the Fourier transform of the Dirac delta function $\delta (x-x')$. Iterating the evolution and taking $dt\rightarrow 0$ limit we get a path integral representation

$$\begin{aligned} P_T(x)=\int _{z(0)=y}^{z(T)=x}\mathcal {D}[z,p]e^{\int _0^Tdt\, [ip{\dot{z}}-H(z,p)]} \end{aligned}$$

with an initial condition $P_0(z)=\delta (z-y)$. The H(z, p) is quadratic in p, and the corresponding path integral can be evaluated exactly, giving

$$\begin{aligned} P_T(x)=\int _{z(0)=y}^{z(T)=x}\mathcal {D}[z]e^{-\frac{1}{2\epsilon }\int _0^Tdt\, ({\dot{z}}-F(z))^2-\int _0^Tdt\,F'(z)} \end{aligned}$$

This is the path integral representation of the Fokker–Planck equation (29).

It is straightforward to generalize the above analysis for the generating function (32) and we get

$$\begin{aligned} G_T^{(\lambda )}(x\vert y)=\int _{z(0)=y}^{z(T)=x}\mathcal {D}[z]e^{{\mathbb {S}}_T^{(\lambda )}[z(t)]} \end{aligned}$$

(97a)

where the Action

$$\begin{aligned} {\mathbb {S}}_T^{(\lambda )}[z]=\int _0^T dt\left[ \lambda f(z)+\lambda {\dot{z}} h(z)-\frac{({\dot{z}}-F(z))^2}{2\epsilon }-F'(z)-\epsilon \lambda \left( \alpha -\frac{1}{2}\right) h'(z)\right] \nonumber \\ \end{aligned}$$

(97b)

Taking small $\epsilon$ limit, we get ${\mathbb {S}}_T^{(\frac{\kappa }{\epsilon })}[z]\simeq \frac{1}{\epsilon }S_T^{(\kappa )}[z]$ with the latter given in (56) where we used $h(x)=0$.