Extending Transition Path Theory: Periodically Driven and Finite-Time Dynamics

Abstract

Given two distinct subsets A, B in the state space of some dynamical system, transition path theory (TPT) was successfully used to describe the statistical behavior of transitions from A to B in the ergodic limit of the stationary system. We derive generalizations of TPT that remove the requirements of stationarity and of the ergodic limit and provide this powerful tool for the analysis of other dynamical scenarios: periodically forced dynamics and time-dependent finite-time systems. This is partially motivated by studying applications such as climate, ocean, and social dynamics. On simple model examples, we show how the new tools are able to deliver quantitative understanding about the statistical behavior of such systems. We also point out explicit cases where the more general dynamical regimes show different behaviors to their stationary counterparts, linking these tools directly to bifurcations in non-deterministic systems.

1 Introduction

The understanding of when and how dynamical transitions, such as tipping processes, happen is important for many systems from physics, biology (Noé et al. 2009), ecology (Scheffer et al. 2001; Hastings et al. 2018), the climate (Lenton et al. 2008; Lenton 2013), and the social sciences (Nyborg et al. 2016; Otto et al. 2020).

If the system dynamics can be modeled by a stationary Markov process running for infinite time, transition path theory (TPT) provides a rigorous approach for studying the transitions from one subset A to another subset B of the state space. The main tool of transition path theory (Weinan and Vanden-Eijnden 2006; Metzner et al. 2009) is the forward and backward committor probabilities telling us the probability of the Markov process to next commit to (i.e., hit) B relative to A, either forward or backward in time. Given these committor probabilities, one can derive important statistics of the ensemble of reactive trajectories (i.e., of the collection of all possible paths of the Markov process that start in A and end in B), such as

• the density of reactive trajectories telling us about the bottlenecks during transitions,

• the current of reactive trajectories indicating the most likely transition channels,

• the rate of reactive trajectories leaving A or entering B, and

• the mean duration of reactive trajectories.

Other approaches that characterize the ensemble of transition paths, on the one hand, place the focus elsewhere: For instance, in transition path sampling (Bolhuis et al. 2002) one is interested in directly sampling trajectories of the reactive ensemble. On the other hand, these approaches consider different objects, such as the steepest descent path (Ulitsky and Elber 1990; Czerminski and Elber 1990; Olender and Elber 1997), the most probable path (Olender and Elber 1996; Elber and Shalloway 2000; Pinski and Stuart 2010; Faccioli et al. 2010; Beccara et al. 2012) [also in temporal networks (Ser-Giacomi et al. 2015)], or the first passage path ensemble (von Kleist et al. 2018) (see also Remark 5.6).

For many physical, especially molecular, systems that are equilibrated and where transitions happen on a smaller time scale than the observation window, the assumption of a stationary, infinite-time Markov process is reasonable and common practice (Noé et al. 2009). For illustration, we consider the overdamped Langevin dynamics

$$\begin{aligned} \mathrm{d}X_t = - \nabla V(X_t) \mathrm{d}t + \sigma \mathrm{d}W_t \end{aligned}$$

in the triple well landscape V(x, y) (as in Fig. 1), and we are interested in the transitions between the deep well A and the other deep well B. If the noise intensity $\sigma$ is sufficiently small, then the system tends to spend long times near local minima of deep wells (this behavior is called metastability) and transitions predominantly happen across regions of possibly low values of the potential V (e.g., saddle points). If the system from Fig. 1 is stationary and has infinite time for transitioning, the transition channel via the metastable well centered at (0, 1.5) is preferred since the barriers are lower and it does not matter that transitions take a very long time due to being stuck in the metastable set.

Fig. 1

a Triple well potential landscape with two possible reactive trajectories, i.e., transitions from A to B, highlighted in black and red. b The effective current of reactive trajectories from A to B in infinite time; the channel via the well at (0, 1.5) is dominantly taken. c The effective current of reactive trajectories restricted to a small time window; the transition channel via the direct barrier between A and B is more likely. More details on the numerical example can be found in Sect. 6, but here we chose a smaller noise intensity $\sigma = 0.26$ (Color figure online)

However, in order to study transitions and tipping paths in different dynamical contexts (e.g., social systems or climate models) which are often characterized by time-dependent (e.g., seasonal) dynamics as well as transitions of interest within a finite-time window, the current theory of transition paths has to be extended.

In these applications, one might, for instance, ask: What are the possible transition channels from the current state to a desirable and sustainable state of our social or climate system within the next 30 years (Steffen et al. 2018; Otto et al. 2020)? By requiring the transitions to depart from A and arrive in B within a finite-time interval, the affinity of the system taking the different transition channels is altered. This is visualized in the triple well dynamics, as shown in Fig. 1c, where now only the lower transition channel passing the high barrier is possible. Whenever the trajectory takes the channel through the upper metastable set (cf. black trajectory in subfigure (a)), it is stuck there for a long time and will not reach B anymore within the finite-time horizon.

Moreover, systems containing human agents are usually time-inhomogeneous and not equilibrated, while climate systems are often affected by seasonal forcing, raising questions such as: What are the likely spreading paths of a contagion in a time-evolving network (Pan and Saramäki 2011; Brockmann and Helbing 2013; Valdano et al. 2018)? What are bottlenecks in the transient dynamics toward the equilibrium state (Schonmann 1992; Hollander et al. 2000)? How does tipping occur under the joint effect of noise and parameter changes (Ashwin et al. 2012; Giorgini et al. 2019) or periodic forcing (Herrmann et al. 2005)?

In this paper, we generalize TPT to a broader class of dynamical scenarios. In particular, we focus on two generalizations which we consider as natural but not exclusive building blocks for these more general cases: (a) periodically forced infinite-time system and (b) arbitrary time-inhomogeneous finite-time system.

We start in Sect. 2 by formulating the general setting of TPT for Markov chains $(X_n)_{n\in {\mathbb {T}}}$ on a finite state space¹ by introducing time-dependent forward committor functions $q^+_i(n)$, giving the probability within the time horizon ${\mathbb {T}}$ to next commit to B and not A conditional on being in $X_n=i$, as well as time-dependent backward committor functions $q^-_i(n)$, both of which are needed for computing the desired statistics of the transitions from A to B.

We then in more detail work out the following main cases.

• (i) Under the assumption of stationary, infinite-time dynamics, it is known Weinan and Vanden-Eijnden (2006), Metzner et al. (2009) that the committor functions are time-independent $q^+(n) = q^+$ by stationarity and solve the following linear system

  $$\begin{aligned} \left\{ \begin{array}{rcll} q_i^+ &{}=&{} \sum \limits _{j\in {\mathbb {S}}} \, P_{ij} \, q_j^+ &{}i \in (A\cup B)^c \\ q_i^+ &{}=&{} 0&{} i \in A \\ q_i^+ &{}=&{} 1&{} i \in B \\ \end{array}\right. \end{aligned}$$

  (1)

  where $P = (P_{ij})_{i,j\in {\mathbb {S}}}$ is the transition matrix. Similarly, all the transition statistics are time-independent and by ergodicity, the statistics can also be found by averaging along one infinitely long equilibrium trajectory (Weinan and Vanden-Eijnden 2006; Metzner et al. 2009). In Sect. 3, we recall the theory (Weinan and Vanden-Eijnden 2006; Metzner et al. 2009) from a different point of view. Instead of defining all quantities in terms of trajectory-wise time averages, we prove by using the Markov property that they can be written in terms of the committors and the stationary distribution. Further, we extend the theory by Lemma 3.5 decomposing the committors into path probabilities.

• (ii) In Sect. 4, we derive the committors and transition statistics for periodically varying dynamics with a period of length M that are equilibrated (i.e., the law of the chain is also periodic). It follows that the committors are periodically varying $q^+(n) = q^+_m$ whenever $n=m$ modulo M, and we show that the committors solve the following linear system with periodic boundary conditions in time $q^+_0 = q^+_M$

  $$\begin{aligned} \left\{ \begin{array}{rcll} q_{m,i}^+ &{}=&{} \sum \limits _{j\in {\mathbb {S}}} \, P_{m,ij} \, q_{m+1,j}^+ &{} i \in (A\cup B)^c \\ q_{m,i}^+ &{}=&{} 0&{} i \in A \\ q_{m,i}^+ &{}=&{} 1&{} i \in B \\ \end{array}\right. \end{aligned}$$

  (2)

  where $P_m = (P_{m,ij})_{i,j\in {\mathbb {S}}}$ is the transition matrix at time $n=m$ modulo M. This is consistent with the previous case when choosing a period of length $M=1$.

• (iii) In Sect. 5, we derive the committor equations and transition statistics for general time-inhomogeneous Markov chains $(X_n)_{n\in {\mathbb {T}}}$ on a finite-time interval ${\mathbb {T}}=\{0,\dots ,N-1\}$ defined by the transition probabilities $P(n) = (P_{ij}(n))_{i,j\in {\mathbb {S}}}$ and an initial density $\lambda _{}(0)$. The forward committor $q^+(n)$ for a finite-time Markov chain satisfies the following iterative system of equations:

  $$\begin{aligned} \left\{ \begin{array}{rcll} q_i^+(n) &{}=&{} \sum \limits _{j \in {\mathbb {S}}} \, P_{ij}(n) \, q_j^+(n+1) &{} i \in (A\cup B)^c \\ q_i^+(n) &{}=&{} 0 &{} i \in A \\ q_i^+(n) &{}=&{} 1 &{} i \in B \end{array}\right. \end{aligned}$$

  (3)

  with final condition $q^+_i(N-1) = \mathbb {1}_B(i)$. The transition statistics depend on the current time point in the time interval and can be related to an average over the ensemble of reactive trajectories. We also show consistency, i.e., that given a stationary process on a finite-time interval, the committors and statistics converge to their classical counterparts (i) in the infinite time limit.

We note that in the stationary regime, committor functions have been used for finding basins of attraction of stochastic dynamics (Koltai 2011b; Koltai and Volf 2014; Lindner and Hellmann 2019), as reaction coordinates (Lu and Vanden-Eijnden 2014), as basis functions in the core-set approach (Schütte et al. 2011; Sarich 2011; Schütte and Sarich 2013) and for studying modules and flows in networks (Djurdjevac et al. 2011; Cameron and Vanden-Eijnden 2014). We expect our results to enable similar uses for the time-dependent regime.

The theoretical results from this paper are accompanied by numerical studies on two toy examples, a network of 5 nodes with time-varying transition probabilities, and a discrete model of the overdamped Langevin dynamics in a triple well potential with time-dependent forcing (as in Fig. 1). In these examples, we will particularly show to what extent time-dependent dynamics or finite-time restrictions affect the transition statistics in contrast to those in stationary, infinite time dynamics. The TPT-related objects derived here allow for a quantitative assessment of the dominant statistical behavior in complicated dynamical regimes:

• (i) By adding a periodic forcing to the stationary dynamics, the reaction channels are perturbed and new transition paths, that were not possible before, can appear (see Example 2 and 7).

• (ii) By restricting the stationary dynamics with matrix P to a finite-time window (cf. Example 3 and 8), only transitions within this window are allowed and the average rate of transitions is much lower than in the infinite-time situation. By additionally applying a forcing (cf. Example 4), the system is not equilibrated anymore and we can get a higher average rate of transitions than without forcing, although we set the time-dependent transition matrix P(n) such that its time average equals the transition matrix P of the stationary case. A similar approach is used in rare events simulation where the system is pushed by an optimal non-equilibrium forcing under which the rare events become more likely (Hartmann and Schütte 2012; Hartmann et al. 2014).

• (iii) The finite-time case can also be employed for studying qualitative changes in the transition dynamics when parameters are perturbed. In Example 9, we exemplarily show that by increasing the finite-time interval length N, the transition dynamics change qualitatively, even though the dynamics are stationary. Thus, TPT can be used as a quantitative tool describing bifurcations in non-stationary non-deterministic systems.

The code used for the examples is available on Github at www.github.com/LuzieH/pytpt.

2 Preliminaries and General Setup

The objective of transition path theory (TPT) is to understand the mechanisms by which noise-induced transitions of a Markov chain $(X_n)_{n\in {\mathbb {T}}}$ from one subset of the state space $A \subset {\mathbb {S}}$ to another subset $B \subset {\mathbb {S}}$ take place.²A and B are chosen as two non-empty, disjoint subsets of the finite state space ${\mathbb {S}}$, such that the transition region $C:= (A \cup B)^c$ is also non-empty. Since historically in TPT one thinks of A as the reactant states of a system, B as the product states and the transitions from A to B as reaction events, we call the pieces of a trajectory that connect A to B by the name reactive trajectories. Each reactive trajectory contains the successive states in C visited during a transition that starts in A and ends in B, see Fig. 2, and we are interested in gathering dynamical information about the ensemble of reactive trajectories.

Fig. 2

Reactive trajectories, i.e., excursions of trajectories that start in A and end in B

In this paper, we are interested not only in Markov chains living on the infinite-time frame ${\mathbb {T}}= {\mathbb {Z}}$ but also those on finite-time intervals ${\mathbb {T}}=\{0,1,\dots , N-1\}$, that is where the transitions from A to B have to take place during a finite-time frame ${\mathbb {T}}$. Moreover, we also consider non-stationary dynamics. This either means

• that the system is in the transient phase toward equilibrium but otherwise has a time-independent transition matrix, or

• that the dynamical rules (the transition matrices) are varying in time, as well as

• that we are dealing with time-varying sets A and B (see the comment in Sect. 4.2 for the case of periodic dynamics and Remark 5.4 for finite-time dynamics).

In this section, we will define the committor functions and show how they can be used to derive important statistics of the ensemble of reactive trajectories, and this entails, e.g., the frequency of transitions, the most important transition channels and where the process on the way to B gets stuck or spends most of its time. In doing so, we will keep everything general enough for time-dependent and finite-time dynamics and for the moment only need to assume the Markovianity of the chain $(X_n)_{n\in {\mathbb {T}}}$ and that the distribution ${\mathbb {P}}(X_n=i)$ and time-dependent transition rule $P(n) = ({\mathbb {P}}(X_{n+1}=j|X_n=i))_{i,j\in {\mathbb {S}}}$ is given for all $n\in {\mathbb {T}}$.

Later, the results from this section will be applied to the special cases of (i) infinite-time, stationary dynamics (Sect. 3), (ii) infinite-time, periodic dynamics (Sect. 4), and (iii) finite-time, time-dependent systems (Sect. 5), where we will also prove the existence of committors and give the linear system of equations they are solving. These three are just some selected special cases, and other interesting cases of systems are, e.g., stochastic regime-switching (see Remark A.1).

2.1 Committor Equations

All of the transition statistics and characteristics can be computed from the committor probabilities; therefore, we will start by defining them. The forward committor is the probability that the Markov chain, currently in some state i, will next³ go to set B and not to A. The backward committor is the same for the time-reversed Markov chain, i.e., the probability that the time-reversed process will next hit A and not B, or equivalently, the probability that the chain last came from A and not B.

More precisely, the forward committor $q^+(n)=(q^+_i(n))_{i\in {\mathbb {S}}}$ gives the probability that the process starting in $i\in {\mathbb {S}}$ at time $n\in {\mathbb {T}}$ reaches at next within ${\mathbb {T}}$ first B and not A. We write

$$\begin{aligned} \begin{aligned} q_i^+(n) := {\mathbb {P}}(\tau _B^+(n) < \tau _A^+(n) \,|\, X_n=i), \end{aligned} \end{aligned}$$

(4)

where the first entrance time of a set $S\subset {\mathbb {S}}$ after or at time $n\in {\mathbb {T}}$ is given by

$$\begin{aligned} \tau _S^+(n) := \inf \{k\in {\mathbb {T}}\text { s.t. } k \ge n, X_k \in S \}, \ \inf \emptyset := \infty . \end{aligned}$$

The backward committor $q^-(n)=(q^-_i(n))_{i\in {\mathbb {S}}}$ gives the probability that the trajectory arriving at time $n\in {\mathbb {T}}$ in state $i\in {\mathbb {S}}$ last within ${\mathbb {T}}$ came from A not B,

$$\begin{aligned} \begin{aligned} q_i^-(n) := {\mathbb {P}}( \tau _A^-(n) >\tau _B^-(n) \,|\, X_n=i), \end{aligned} \end{aligned}$$

(5)

where the last exit time of a set S before time or at time n is given by

$$\begin{aligned} \tau _S^- (n) := \sup \{k \in {\mathbb {T}}\text { s.t. }k \le n, X_k \in S \},\ \sup \emptyset := - \infty . \end{aligned}$$

Note that the first entrance and last exit times are stopping times with respect to the forward and time-reversed process (Ribera Borrell 2019, Lemma 3.1.1). The time-reversed process will be introduced in the following sections for infinite-time and finite-time Markov chains.

The forward committor, as it is defined, only considers trajectory pieces arriving at B within the time horizon ${\mathbb {T}}$; similarly, the backward committor only considers excursions that left A within ${\mathbb {T}}$.

2.2 Transition Statistics

We will now define statistical objects that characterize the ensemble of reactive trajectories from subset A to B and see that they can be computed using the committor probabilities and the Markovianity assumption.

The first two objects, the distribution of reactive trajectories $\mu ^{AB}(n)$ and its normalized version ${\hat{\mu }}^{AB}(n)$, tell us where the reactive trajectories are most likely to be found, i.e., where the transitioning trajectories spend most of their time.

Definition 2.1

The distribution of reactive trajectories $\mu ^{AB}(n) =(\mu ^{AB}_i(n))_{i\in {\mathbb {S}}}$ for $n\in {\mathbb {T}}$ gives the joint probability that the Markov chain is in a state i at time n while transitioning from A to B:

$$\begin{aligned} \mu ^{AB}_i(n) := {\mathbb {P}}(X_n=i, \tau _A^-(n) >\tau _B^-(n) , \tau _B^+(n) < \tau _A^+(n)). \end{aligned}$$

Note that $\mu ^{AB}_i(n)=0$ for $i \notin C$, i.e., we only get information about the density of transitions passing through C. Direct transitions from A to B are neglected, and in general assumed not to exist.

Theorem 2.2

For a general Markov chain $(X_n)_{n\in {\mathbb {T}}}$ with committors $q^+(n)$, $q^-(n)$, the distribution of reactive trajectories can be expressed as

$$\begin{aligned} \begin{aligned} \mu ^{AB}_i(n)&=q_i^-(n) \, {\mathbb {P}}(X_n=i) \, q_i^+ (n). \end{aligned} \end{aligned}$$

Proof

We can compute

$$\begin{aligned} \begin{aligned} \mu ^{AB}_i(n)&={\mathbb {P}}(\tau _A^-(n) >\tau _B^-(n) , \tau _B^+(n) < \tau _A^+(n)\,|\, X_n=i) \, {\mathbb {P}}(X_n=i) \\&=q_i^-(n)\, q_i^+ (n) \, {\mathbb {P}}(X_n=i) \end{aligned} \end{aligned}$$

by conditioning on $\{X_n=i\}$, and by using independence of the two events $\{\tau _A^-(n) >\tau _B^-(n) \}$, $\{\tau _B^+(n) < \tau _A^+(n)\}$ given $\{X_n=i\}$, which follows from the Markov property.⁴$\square$

The distribution $\mu ^{AB}(n)$ is not normalized but can easily be normalized by dividing $\mu ^{AB}(n)$ by the probability to be on a transition at time n,

$$\begin{aligned} Z^{AB}(n) := \sum _{j \in C} \mu ^{AB}_j(n) ={\mathbb {P}}(\tau _A^-(n) >\tau _B^-(n) , \tau _B^+(n) < \tau _A^+(n)), \end{aligned}$$

to give a probability distribution on ${\mathbb {S}}$:

Definition 2.3

Whenever ${\mathbb {P}}(\tau _A^-(n)>\tau _B^-(n) , \tau _B^+(n) < \tau _A^+(n))>0$ for $n\in {\mathbb {T}}$, we can define the normalized distribution of reactive trajectories at time $n\in {\mathbb {T}}$ by

$$\begin{aligned} {\hat{\mu }}^{AB}_i(n) := {\mathbb {P}}(X_n=i|\tau _A^-(n) >\tau _B^-(n) , \tau _B^+(n) < \tau _A^+(n)) \end{aligned}$$

giving the density of states in which trajectories transitioning from A to B spend their time.

The next object tells us about the average number of jumps from i to j during one time step (i.e., the probability flux), while the trajectory is on its way from A to B:

Definition 2.4

The current of reactive trajectories $f^{AB}(n) = (f^{AB}_{ij}(n))_{i,j\in {\mathbb {S}}}$ at time $n\in {\mathbb {T}}$ gives the average flux of trajectories going through i at time $n\in {\mathbb {T}}$ and j at time $n+1\in {\mathbb {T}}$ consecutively while on their way from A to B:

$$\begin{aligned} f^{AB}_{ij}(n):= {\mathbb {P}}( X_n=i, X_{n+1}=j, \tau _A^-(n) >\tau _B^-(n) , \tau _B^+(n+1) < \tau _A^+(n+1)) \end{aligned}$$

Theorem 2.5

The current of reactive trajectories for a Markov chain $(X_n)_{n\in {\mathbb {T}}}$ with transition probabilities $P(n)$ and committors $q^+(n)$, $q^-(n)$, is given by

$$\begin{aligned} \begin{aligned} f^{AB}_{ij} (n) = q_i^-(n) \, {\mathbb {P}}(X_n=i) \, P_{ij}(n) \, q^+_j(n+1) \end{aligned} \end{aligned}$$

Proof

The reactive current can be computed as

$$\begin{aligned} \begin{aligned} f^{AB}_{ij} (n)&={\mathbb {P}}( X_{n+1}=j, \tau _A^-(n)>\tau _B^-(n) , \tau _B^+(n+1)< \tau _A^+(n+1)| X_n=i) \, \\&\quad \qquad {\mathbb {P}}(X_n=i) \\&={\mathbb {P}}( X_{n+1}=j,\tau _B^+(n+1)< \tau _A^+(n+1)| X_n=i) \\&\quad \qquad {\mathbb {P}}(\tau _A^-(n) >\tau _B^-(n)| X_n=i) {\mathbb {P}}(X_n=i) \\&={\mathbb {P}}( X_{n+1}=j| X_n=i) \, {\mathbb {P}}( \tau _B^+(n+1) < \tau _A^+(n+1)| X_{n+1}=j, X_n=i)\\&\quad \qquad q_i^-(n)\, {\mathbb {P}}(X_n=i) \\&= q_i^-(n)\, {\mathbb {P}}(X_n=i)\, P_{ij}(n) \, q^+_j (n+1), \end{aligned} \end{aligned}$$

by first conditioning on $\{X_n=i\}$, then by independence of $\{X_{n+1}=j, \tau _B^+(n+1) < \tau _A^+(n+1)\}$ and $\{ \tau _A^-(n) >\tau _B^-(n)\}$ given $\{X_n=i\}$, by conditioning on $\{X_{n+1}=j\}$, and last by the Markov property. $\square$

Let us note that the reactive current also counts the transitions going directly from $i\in A$ to $j\in B$, these are not accounted for in the reactive distribution which only accounts for transitions passing the region $(A\cup B)^c$.

In order to eliminate information about detours of reactive trajectories, we define:

Definition 2.6

The effective current of reactive trajectories $f^+(n) = (f^+_{ij}(n))_{i,j\in {\mathbb {S}}}$ at time $n\in {\mathbb {T}}$ gives the net amount of reactive current going through i at time $n\in {\mathbb {T}}$ and j at time $n+1\in {\mathbb {T}}$ consecutively,

$$\begin{aligned} f^+_{ij}(n):= \max \{f^{AB}_{ij}(n)-f^{AB}_{ji}(n),0\}. \end{aligned}$$

Ultimately, the effective current of reactive trajectories can be used to find the dominant transition channels in state space between A and B (see, e.g., Metzner et al. 2009).

The current of reactive trajectories only goes out of A, not into A; moreover, the current of reactive trajectories only points into B, not out of B. Therefore, A can be thought of as a source of reactive trajectories, whereas B acts like their sink. This leads us to our next characteristic of reactive trajectories: By summing the current of reactive trajectories over A, we get the discrete rate of reactive trajectories flowing out of A, and by summing the current over B, we obtain the rate of inflow into B:

Definition 2.7

For $n, n+1 \in {\mathbb {T}}$, the discrete rate of transitions leaving A at time n is given by

$$\begin{aligned} k^{A\rightarrow }(n):= {\mathbb {P}}(X_n\in A, \tau _B^+(n+1) < \tau _A^+(n+1)), \end{aligned}$$

i.e., the probability of a reactive trajectory leaving A at time n. When $n-1, n \in {\mathbb {T}}$, the discrete rate of transitions entering B at time n is given by

$$\begin{aligned} k^{\rightarrow B}(n):= {\mathbb {P}}(X_n\in B,\tau _A^-(n-1) >\tau _B^-(n-1)), \end{aligned}$$

i.e., the probability of a reactive trajectory entering B at time n.

Theorem 2.8

For a Markov chain $(X_n)_{n\in {\mathbb {T}}}$ with current of reactive trajectories $f^{AB}(n)$, we find the discrete rates to be

$$\begin{aligned} \begin{aligned} k^{A\rightarrow }(n)&= \sum _{i\in A, j\in {\mathbb {S}}} f^{AB}_{ij}(n) \\ k^{\rightarrow B}(n)&= \sum _{i\in {\mathbb {S}}, j\in B} f^{AB}_{ij}(n-1). \\ \end{aligned} \end{aligned}$$

(6)

Proof

We can compute by using the law of total probability

$$\begin{aligned} \begin{aligned}&\sum _{i\in A, j\in {\mathbb {S}}} f^{AB}_{ij}(n) \\&= \sum _{i\in A, j\in {\mathbb {S}}} {\mathbb {P}}( X_n=i, X_{n+1}=j, \tau _A^-(n)>\tau _B^-(n) , \tau _B^+(n+1)< \tau _A^+(n+1))\\&= k^{A\rightarrow }(n) \\&\sum _{i\in {\mathbb {S}}, j\in B} f^{AB}_{ij}(n-1)\\&=\sum _{i\in {\mathbb {S}}, j\in B} {\mathbb {P}}( X_{n-1}=i, X_{n}=j, \tau _A^-(n-1) >\tau _B^-(n-1) ,\tau _B^+(n) < \tau _A^+(n))\\&= k^{\rightarrow B}(n). \end{aligned} \end{aligned}$$

(7)

$\square$

3 TPT for Stationary, Infinite-Time Markov Chains

TPT was originally designed for stationary, infinite-time Markov processes (Weinan and Vanden-Eijnden 2006; Metzner et al. 2009) that are often used as models for molecular systems (Noé et al. 2009). Here, we will recall that theory by using the results from the previous section and equip it with some new results (e.g., Lemma 3.5) that will be needed later.

3.1 Setting

We begin with describing the processes of interest in this section.

Assumption 3.1

We consider a Markov chain $(X_n)_{n\in {\mathbb {Z}}}$ taking values in a discrete and finite state space ${\mathbb {S}}$, and the time-discrete jumps between states $i\in {\mathbb {S}}$ and $j\in {\mathbb {S}}$ occur with probability

$$\begin{aligned} P_{ij} := {\mathbb {P}}(X_{n+1}=j|X_n=i) \end{aligned}$$

stored in the row-stochastic transition matrix $P=(P_{ij})_{i,j \in {\mathbb {S}}}$. We assume that the process is irreducible, and ergodic with respect to the unique, strictly positive invariant distribution $\pi = (\pi _i)_{i \in {\mathbb {S}}}$ (also called stationary distribution interchangeably) solving $\pi ^\top = \pi ^\top P$.

The time-reversed process $(X^-_n)_{n \in {\mathbb {Z}}}$, $X^-_n := X_{-n}$ traverses the chain backwards in time. It is also a Markov chain (Ribera Borrell 2019, Thm 2.1.19) and stationary with respect to the same invariant distribution. The transition probabilities of the time-reversed process $P^-= (P^-_{ij})_{i,j \in {\mathbb {S}}}$ with entries

$$\begin{aligned} P^-_{ij} := {\mathbb {P}}(X^-_{-n+1} =j |X^-_{-n}=i) = {\mathbb {P}}(X_{n-1}=j |X_{n}=i) \end{aligned}$$

can be found from expressing the flux in two ways,

$$\begin{aligned} {\mathbb {P}}(X_n=i, X_{n+1}=j)= P_{ij} \pi _i = P^-_{ji} \pi _j. \end{aligned}$$

3.2 Committor Probabilities

Due to stationarity of the chain (Assumption 3.1), the law of the chain is the same for all times, and we simply have that the committors (4) and (5) are time-independent $q_i^+(n)= q_i^+$, similarly $q_i^-(n)=q_i^-$ for all n.

The forward and backward committors can be found by solving a linear matrix equation of size |C| with appropriate boundary conditions (Norris 1998, Chapter 1.3).

Theorem 3.2

The forward committor for a Markov chain according to Assumption 3.1 with transition probabilities $P=(P_{ij})_{i,j\in {\mathbb {S}}}$ satisfies the following linear system

$$\begin{aligned} \left\{ \begin{array}{rcll} q_i^+ &{}=&{} \sum \limits _{j\in {\mathbb {S}}} \, P_{ij} \, q_j^+ &{}i \in C \\ q_i^+ &{}=&{} 0&{} i \in A \\ q_i^+ &{}=&{} 1&{} i \in B. \\ \end{array}\right. \end{aligned}$$

(8)

Analogously for the backward committor, we have to solve the following linear system

$$\begin{aligned} \left\{ \begin{array}{rcll} q_i^- &{}=&{} \sum \limits _{j\in {\mathbb {S}}} \, P^-_{ij} \, q_j^- &{}i \in C \\ q_i^- &{}=&{} 0&{} i \in B \\ q_i^- &{}=&{} 1&{} i \in A. \\ \end{array}\right. \end{aligned}$$

(9)

Proof

From the definition of the committors (4), it immediately follows that we have $q_i^+ =0$ for $i \in A$ since we always have $\tau _A^+(n)=n$, while $\tau _B^+(n)>n$. Analogously, we have $q_i^+ = 1$ for $i \in B$ since in that case, $\tau _A^+(n)>n$ and $\tau _B^+(n)=n$. For the committor at node i in the transition region C, we can sum the forward committor at all the other states j weighted with the transition probability to transition from i to j. This follows from

$$\begin{aligned} \begin{aligned} q_i^+&={\mathbb {P}}(\tau _B^+(n)< \tau _A^+(n) | X_n=i) = \sum _{j\in {\mathbb {S}}} {\mathbb {P}}(X_{n+1}=j, \tau _{B}^+(n)<\tau _{A}^+(n) |X_n=i) \\&= \sum _{j\in {\mathbb {S}}} {\mathbb {P}}(X_{n+1}=j|X_n=i) {\mathbb {P}}(\tau _{B}^+(n)<\tau _{A}^+(n) |X_{n+1}=j,X_n=i) \\&= \sum _{j\in {\mathbb {S}}} {\mathbb {P}}(X_{n+1}=j|X_n=i) {\mathbb {P}}(\tau _{B}^+(n+1) <\tau _{A}^+(n+1) |X_{n+1}=j,X_n=i)\\&= \sum _{j\in {\mathbb {S}}}P_{ij} q_j^+ \end{aligned} \end{aligned}$$

first using the law of total probability, then conditioning on $\{X_{n+1}=j\}$, using that at time n the chain is in $i\in C$ and thus $\tau _A^+(n),\tau _B^+(n)\ge n+1$, and last using the Markov property.

For the backward committor equations, we can proceed in a similar way, by additionally using the time-reversed process. $\square$

Remark 3.3

If the Markov chain in addition is reversible, i.e., if $P_{ij} \pi _i = P_{ji} \pi _j$ (equivalently, $P^-_{ij}=P_{ij}$) holds, then it follows from Theorem 3.2 that the forward and backward committors are related by $q_i^+=1-q_i^-$.

The following lemma provides us with the necessary condition such that existence and uniqueness of the committors is guaranteed. For a proof, see Ribera Borrell (2019, Lemma 3.2.4) or Norris (1998, Chapter 4.2).

Lemma 3.4

If $P$ is irreducible, then the two problems (8) and (9) each have a unique solution.

There is a second characterization of the committors in the transition region using path probabilities, as summarized in the following lemma. The proof can be found in Appendix A.2.1.

Lemma 3.5

For any $i \in C$, the forward committor can also be specified as the probability of all possible paths starting from node i that reach the set B before A

$$\begin{aligned} \begin{aligned} q_i^+&= \sum _{\tau \in {\mathbb {Z}}^+} \sum _{\begin{array}{c} i_1 \dots i_{\tau -1} \in C \\ i_\tau \in B \end{array}} P_{ii_1} \cdots P_{i_{\tau -1} i_{\tau }}. \end{aligned} \end{aligned}$$

(10)

Similarly for any $i \in C$, the backward committor can also be understood as the sum of path probabilities of all possible paths arriving at node i that last came from A and not B

$$\begin{aligned} \begin{aligned} q_i^-&= \sum _{\tau \in {\mathbb {Z}}^-} \sum _{\begin{array}{c} i_\tau \in A\\ i_{\tau +1} \dots i_{-1} \in C \end{array}} P^-_{i_{\tau +1}i_{\tau }} \cdots P^-_{i i_{-1}}. \end{aligned} \end{aligned}$$

(11)

3.3 Transition Statistics

The committor, the distribution, and the transition probabilities are time-independent; thus, the statistics from Sect. 2.2 are time-independent. We write the distribution of reactive trajectories (Theorem 2.2) as $\mu ^{AB}=(\mu ^{AB}_i)_{i\in {\mathbb {S}}}$, where

$$\begin{aligned} \mu ^{AB}_i =q_i^- q_i^+ \ \pi _i, \end{aligned}$$

the normalized distribution as ${\hat{\mu }}^{AB}=({\hat{\mu }}^{AB}_i)_{i\in {\mathbb {S}}}$. The current of reactive trajectories (Theorem 2.5) $f^{AB}=(f^{AB}_{ij})_{i,j\in {\mathbb {S}}}$ is given by

$$\begin{aligned} f^{AB}_{ij} = q_i^- \pi _i P_{ij} q^+_j, \end{aligned}$$

and the effective reactive current is denoted $f^+= (f^+_{ij})_{i,j\in {\mathbb {S}}}$.

Theorem 3.6

For a stationary Markov chain $(X_n)_{n\in {\mathbb {Z}}}$, the reactive current out of a node $i\in C$ equals the current flowing into the node $i\in C$, i.e.,

$$\begin{aligned} \sum _{j\in {\mathbb {S}}} f^{AB}_{ij} = \sum _{j\in {\mathbb {S}}} f^{AB}_{ji}. \end{aligned}$$

(12)

Further, the the reactive current flowing out of A into ${\mathbb {S}}$ (equivalently into $C\cup B$) equals the flow of reactive trajectories from ${\mathbb {S}}$ (equivalently from $C \cup A$) into B

$$\begin{aligned} \sum _{i\in A, j\in {\mathbb {S}}} f^{AB}_{ij} = \sum _{ i\in {\mathbb {S}}, j\in B} f^{AB}_{ij}. \end{aligned}$$

(13)

For the proof, see Appendix A.2.2.

Remark 3.7

Due to these conservation laws, there is a close relation of the committors and the effective current (when the chain is reversible) to the voltage and the electric current in an electric resistor network (Doyle 1984; Norris 1998) with a voltage applied between A and B, see Metzner (2008), Metzner et al. (2009) for work in this direction. Also, in Metzner (2008), Metzner et al. (2009), a decomposition algorithm of the effective current into the paths from A to B carrying a substantial portion of the transition rate is proposed, yielding the dominant transition channels between A and B. The effective current of reactive trajectories is loop-erased; therefore, in Banisch et al. (2015) a decomposition of the current of reactive trajectories into cyclic structures and non-cyclic parts is suggested.

Further, since $\smash { \sum _{i\in A, j\in {\mathbb {S}}} f^{AB}_{ij} = \sum _{ i\in {\mathbb {S}}, j\in B} f^{AB}_{ij} }$ by Theorem 3.6, the discrete rate of leaving A equals the discrete rate of entering into B; thus, we denote $k^{AB}:= k^{A\rightarrow }= k^{\rightarrow B}$. This tells us the probability of a realized transition per time step, i.e., either the probability to leave A and be on the way to B next, or the probability to reach B when coming from A. That $k^{AB}$ indeed has the physical interpretation of a rate becomes clear from the characterization in Theorem 3.8.

3.4 Interpretation of the Statistics as Time Averages Along Trajectories

The statistics from the previous section give us dynamical information about the ensemble of reactive trajectories. Due to the ergodicity, the Markov chain will visit all states infinitely many times and the ensemble space average of a quantity equals the time average this quantity along a single infinitely long trajectory (Birkhoff’s ergodic theorem). Therefore, the reactive trajectory statistics from the previous section can also be found by considering the reactive pieces along a single infinitely long trajectory and by averaging over them.

Theorem 3.8

For a Markov chain $(X_n)_{n\in {\mathbb {Z}}}$ satisfying Assumption 3.1, we have the following ${\mathbb {P}}-$almost sure convergence results:

$$\begin{aligned} \begin{aligned} \mu ^{AB}_i&= \lim _{N \rightarrow \infty } \frac{1}{2N+1} \sum _{n=-N}^N \mathbb {1}_{\{i\}} (X_n) \mathbb {1}_A \left( X_{\tau _{A\cup B}^-(n)}\right) \mathbb {1}_B \left( X_{\tau _{A\cup B}^+(n)}\right) \\ f^{AB}_{ij}&=\lim _{N \rightarrow \infty } \frac{1}{2N+1} \sum _{n=-N}^N \mathbb {1}_{\{i\}} (X_n) \mathbb {1}_A \left( X_{\tau _{A\cup B}^-(n)}\right) \mathbb {1}_{\{j\}} (X_{n+1}) \mathbb {1}_B \left( X_{\tau _{A\cup B}^+(n+1)}\right) \\ k^{AB}&= \lim _{N \rightarrow \infty } \frac{1}{2N+1} \sum _{n=-N}^N \mathbb {1}_{A} (X_n) \mathbb {1}_B\left( X_{\tau _{A\cup B}^+(n+1)}\right) \\&= \lim _{N \rightarrow \infty } \frac{1}{2N+1} \sum _{n=-N}^N \mathbb {1}_A\left( X_{\tau _{A\cup B}^-(n-1)}\right) \mathbb {1}_{B} (X_n) \end{aligned} \end{aligned}$$

(14)

where $i,j \in {\mathbb {S}}$ and $\mathbb {1}_A(x)$ is the indicator function on the set A.

The proof of Theorem 3.8 can be found in Ribera Borrell (2019, Thm 3.3.2, Thm 3.3.7, Thm 3.3.11) and relies on Birkhoff’s ergodic theorem (Walters 2000) for the canonical representation of the process as a Markov shift.

The theorem not only offers a data-driven approach to approximate the transition statistics by averaging along a given sufficiently long trajectory sample but also gives interpretability to the statistics. While $\mu ^{AB}$ as the (not normalized) trajectory-wise distribution of reactive trajectories and $f^{AB}$ as the trajectory-wise flux of reactive trajectories are still straightforward to understand, we can also give meaning to the rate $k^{AB}$ and to $Z^{AB}=\sum _{i \in C} \mu ^{AB}_i$. We can think of $k^{AB}$ as the total number of reactive transitions taking place within the time interval $\{-N,\dots ,N\}$ divided by the number of time steps $2N+1$ in the limit of $N\rightarrow \infty$. Similarly, we can give meaning to $Z^{AB}$ as the total time spent transitioning during $\{-N,\dots ,N\}$ divided by $2N+1$ in the limit of $N\rightarrow \infty$. Last, we note that the ratio between $Z^{AB}$ and $k^{AB}$ provides us with a further characteristic transition quantity (Vanden-Eijnden 2006),

$$\begin{aligned} t^{AB}:= \frac{Z^{AB}}{k^{AB}} = \lim _{N \rightarrow \infty } \frac{ \text {total time spent transitioning during }\{-N,\dots ,N\}}{ \#\{\text {reactive transitions during }\{-N,\dots ,N\}\} }, \end{aligned}$$

telling us the total time spent transitioning divided by the total number of transitions, i.e., the expected length of a transition from A to B.

3.5 Numerical Example 1: Infinite-Time, Stationary Dynamics on a 5-State Network

We consider a Markov chain on a network of five states ${\mathbb {S}}= \{0,1,2,3,4\}$ of which 0, 2, 4 have a high probability to remain in the same state. The transition matrix is given by the row-stochastic matrix $P$, see Fig. 3. We are interested in transitions from the subset $A = \{0\}$ to the subset $B=\{4\}$. What is the most likely route that the transitions take?

The numerically computed committors and transition statistics are shown in Fig. 3, and the discrete rate of transitions is $k^{AB}= 0.018$, i.e., on average every 56 time steps a transition from A to B is completed. We see that the effective current is the strongest along the path $A \rightarrow 1 \rightarrow B$, i.e., most transitions effectively happen via this path, but a share of 33.3% of transitions happens via the route from $A \rightarrow 3 \rightarrow B$. Also, we can note that the density of reactive trajectories has a very high value in the state 2, indicating that many reactive trajectories pass this state or stay there for a long time. Thus, reactive trajectories on the effective path $A \rightarrow 1 \rightarrow B$ will likely do a detour to 2, which is not visible from the effective current since it does not tell us about detours.

Fig. 3

The dynamics of the infinite-time system in Example 1 with a transition matrix $P$ and statistics of the transitions from A to B are shown. The node color of b $q^+$, c $q^-$, and d ${\hat{\mu }}^{AB}$ indicates the values of the respective statistics, ranging from yellow (low values) via orange to purple (high value). e The strength of the effective current from one state i to state j is shown by the thickness and grayscale of the arrow pointing from i to j. The node color of A and B indicates the sum of current flowing out, respectively, into the node; here, this is just $k^{AB}$ (Color figure online)

4 TPT for Periodically Driven, Infinite-Time Markov Chains

Many real-world systems showcase periodicity, for example, any system subject to seasonal driving, or physical systems with periodic external stimuli.

For studying transitions in these systems, we extend TPT to Markov chains with periodically varying transition probabilities that are equilibrated to the forcing and cycle through the same distributions each period. If the period is only one time step long, this case reduces to the previous case of stationary, infinite-time dynamics.

We start by laying out the exact setting of the process that we consider, before turning to the computation of committors and transition statistics for periodically forced dynamics. As we will see, by writing the committor equations on a time-augmented state space, we can also find committors for systems with stochastic switching between different dynamical regimes.

4.1 Setting

Assumption 4.1

Consider a Markov chain $(X_n)_{n\in {\mathbb {Z}}}$ on a finite and discrete state space ${\mathbb {S}}$ with transition probabilities $P_{ij} (n) = {\mathbb {P}}(X_{n+1}=j \, | \, X_n=i)$ that are periodically varying in time with period length $M\in {\mathbb {N}}$, i.e., the transition matrices fulfill

$$\begin{aligned} P(n) = P(n+M)\ \forall n\in {\mathbb {Z}}. \end{aligned}$$

Therefore, the transition matrices at the times within one period ${\mathbb {M}}:= \{0,\dots ,M-1\}$ are sufficient to describe all the dynamics and we denote them by $P_m := P(n)$ for time $n\in {\mathbb {Z}}$ congruent to $m\in {\mathbb {M}}$ modulo M.

The product of transition matrices over one $M-$period starting at a time equivalent to $m\in {\mathbb {M}}$ (modulo M) is denoted by ${\bar{P}}_m := P_m P_{m+1} \cdots P_{m+M-1}$, which is again a transition matrix pushing the Markov chain M time instances in time forward starting from time $\equiv m \ (\mathrm {mod}\ M)$, see Fig. 4. The chain described by ${\bar{P}}_m$ is not time-dependent anymore, and it resolves the state of the original system only every M time instance with a time-independent transition matrix ${\bar{P}}_m$.

Proposition 4.2

If ${\bar{P}}_0$ is irreducible and assuming the setting 4.1 as described above, then for all $m\in {\mathbb {M}}$, there exists an invariant distribution $\pi _m$ such that $\pi _m^\top = \pi _m^\top {\bar{P}}_m$ of the transition matrix ${\bar{P}}_m$. Further, $\pi _0$ is unique and $\pi _{0,i}>0$ for all $i\in {\mathbb {S}}$. If we in addition require $\pi _{m+1}^\top = \pi _m^\top P_m$ for all m, then the entire family $(\pi _m)_{m=0,\ldots ,M-1}$ is unique.⁵

Proof

Since ${\bar{P}}_0$ is irreducible and the state space is finite, the Markov chain induced by ${\bar{P}}_0$ has a unique and positive invariant density $\pi _0=(\pi _{0,i})_{i\in {\mathbb {S}}}$ such that $\pi _0^\top {\bar{P}}_0 = \pi _0^\top$. It follows that also ${\bar{P}}_1$ has an invariant density, namely $\pi _1^\top :=\pi _0^\top P_0$, since it fulfills

$$\begin{aligned} \pi _1^\top {\bar{P}}_1 = \pi _0^\top P_0 {\bar{P}}_1 = \pi _0^\top P_0 P_1 \cdots P_{M-1} P_{0}= \pi _0^\top {\bar{P}}_0 P_{0} = \pi _0^\top P_{0} = \pi _1^\top . \end{aligned}$$

In the same way, ${{\bar{P}}_m, m=2,\dots , M-1,}$ have an invariant distribution $\pi _m^\top :=\pi _{m-1}^\top P_{m-1}$, such that ${\pi _m^\top {\bar{P}}_m = \pi _m^\top }$. $\square$

Thus, the densities $\pi _1,\dots \pi _{M-1}$ are not necessarily unique, unless we require irreducibility. Doing so, there is a unique periodic family of distributions that the chain can admit, and we call such a chain $M-$stationary; see Fig. 4. Having the long-time behavior of chains in mind in this section, we will assume this property, relying on ergodicity.

Fig. 4

The evolution of densities for periodic dynamics assuming $M-$stationarity. The density evolution described by $P_m$, $m\in {\mathbb {M}}$ varies periodically with time n. Whereas the dynamics described by ${\bar{P}}_m$ for a fixed m are time-independent, they resolve the system only every M time instances at times n equivalent to m modulo M

Assumption 4.3

We assume that ${\bar{P}}_0$ is irreducible and that the chain is M-stationary, i.e., ${\mathbb {P}}(X_n=i)=\pi _{m,i}$ whenever n is equivalent to m modulo M.

Next, we introduce the time-reversed chain $(X^-_n)_{n\in {\mathbb {Z}}}$ with $X^-_n = X_{-n}$. Due to $M-$stationarity, the transition probabilities of the time-reversed chain are also $M-$periodic, and it is enough to give the transition probabilities backward in time $P^-_{m}$ for each time point during the period $m\in {\mathbb {M}}$

$$\begin{aligned} \begin{aligned} P^-_{m,ij} :&={\mathbb {P}}(X^-_{M-m+1}=j|X^-_{M-m}=i) = {\mathbb {P}}(X_{m-1}=j|X_m=i) \\&= {\mathbb {P}}(X_m=i|X_{m-1}=j) \frac{{\mathbb {P}}(X_{m-1}=j)}{{\mathbb {P}}(X_{m}=i)} = P_{m-1,ji} \frac{\pi _{m-1,j}}{\pi _{m,i}} \end{aligned} \end{aligned}$$

whenever $\pi _{m,i}>0$, else for $\pi _{m,i}=0$ we set $P^-_{m,ij} :=0$.

4.2 Committor Probabilities

We will first look at the forward and backward committors and the system of equations that can be solved to acquire them. The forward $q^+(n)$, respectively, backward committor $q^-(n)$ is defined as before in (4), respectively, (5), but now since the law of the Markov chain is the same every M time instances, the committors also vary periodically and are identical every M time steps, and we therefore denote

$$\begin{aligned} \begin{aligned} q^+_{m}&:= q^+(n) \\ q^-_{m}&:= q^-(n) \end{aligned} \end{aligned}$$

(15)

whenever n is equivalent to $m\in {\mathbb {M}}$.

Again, we consider non-empty and disjoint subsets A, B of the state space. It is straightforward to extend the theory to periodically varying sets $A = (A_m)_{m\in {\mathbb {M}}}$, $B=(B_m)_{m\in {\mathbb {M}}}$ defined on ${\mathbb {S}}^M$.

Theorem 4.4

Suppose a Markov chain meeting Assumptions 4.1 and 4.3. Then, the $M-$periodic forward committor $q^+_{m}=(q^+_{m,i})_{i\in {\mathbb {S}}}$ fulfills the following iterative system with periodic conditions $q_{M}^+=q_{0}^+$

$$\begin{aligned} \left\{ \begin{array}{rcll} q_{m,i}^+ &{}=&{} \sum \limits _{j\in {\mathbb {S}}} \, P_{m,ij} \, q_{m+1,j}^+ &{} i \in C \\ q_{m,i}^+ &{}=&{} 0&{} i \in A \\ q_{m,i}^+ &{}=&{} 1&{} i \in B, \\ \end{array}\right. \end{aligned}$$

(16)

whereas the $M-$periodic backward committor $q^-_{m}=(q^-_{m,i})_{i\in {\mathbb {S}}}$ satisfies

$$\begin{aligned} \left\{ \begin{array}{rcll} q_{m,i}^- &{}=&{} \sum \limits _{j\in {\mathbb {S}}} \, P^-_{m,ij} \, q_{m-1,j}^- &{} i\in C \\ q_{m,i}^- &{}=&{} 0&{} i \in B \\ q_{m,i}^- &{}=&{} 1&{} i \in A \\ \end{array}\right. \end{aligned}$$

(17)

where $q_{M}^-=q_{0}^-$.

The proof follows the lines of the proof above for the stationary, infinite-time case and can be found in Appendix A.2.3.

Before proving that the two systems are uniquely solvable, we characterize the forward and backward committors in terms of the path probabilities over one period ${\mathbb {M}}$:

Lemma 4.5

For any time $n=m$ modulo M and $i \in C$, the committor functions (15) satisfy the following equalities

$$\begin{aligned} \begin{aligned} q_{m,i}^+&= \sum _{i_1 \dots i_{M} \in C} P_{m,ii_1} P_{m+1,i_1i_2} \cdots P_{m+M-1,i_{M-1} i_M} q_{m,i_M}^+ \\&\quad + \sum _{\tau =1}^{M} \sum _{\begin{array}{c} i_1\dots i_{\tau -1} \in C \\ i_{\tau }\in B \end{array}} P_{m,i i_1} \cdots P_{m+\tau -1,i_{\tau -1}i_\tau } \end{aligned} \end{aligned}$$

(18)

$$\begin{aligned} \begin{aligned} q_{m,i}^-&= \sum _{i_{1} \dots i_{M} \in C} P^-_{m,ii_{1}} P^-_{m-1,i_{1}i_{2}} \cdots P^-_{m-M+1,i_{M-1} i_{M}} q_{m,i_{M}}^-\\&\quad + \sum _{\tau =1}^{M} \sum _{\begin{array}{c} i_1\dots i_{\tau -1} \in C \\ i_{\tau }\in A \end{array}} P^-_{m,i i_1} \cdots P^-_{m-\tau +1,i_{\tau -1}i_\tau } \end{aligned} \end{aligned}$$

(19)

Proof

First, it follows from (16) for $i\in C$ that

$$\begin{aligned} q_{m,i}^+ = \sum _{i_1\in C} P_{m,ii_1} q_{m+1,i_1}^+ + \sum _{i_1\in B} P_{m,i i_1} \end{aligned}$$

(20)

since $q^+_{m+1,i_1} = 1$ if $i_1\in B$, $q^+_{m+1,i_1} = 0$ if $i_1\in A$. By inserting the committor equations at the following times iteratively and by using that $q^+_0=q^+_M$, we get (18). We can proceed analogously for the backward committor, starting from (17) and re-inserting committor equations. $\square$

Equation (18) with $m=0$ only contains one unknown and can be solved, e.g., numerically for all $i\in C$, whereas for A, respectively, B, the committor is simply 0, respectively, 1. The committor for the remaining times $m=1,2,\ldots$ can then be computed thereof by using (16), analogously for (19).

The time resolution of the Markov chain during the period is important for the committors since we can resolve hitting events of B within the period. The committors one would compute for a more coarsely resolved chain without state information during the period, i.e., for the chain described by ${\bar{P}}_0$ (time-homogeneous, but mapping one period in time forward), will not notice that the chain has hit B at times other than $m=0$. To see that, compare Lemma 4.5 with Lemma 3.5 using ${\bar{P}}_0$.

Lemma 4.6

By the irreducibility of ${\bar{P}}_0$, the solutions to (16) and (17) exist and are unique.

The proof can be found in Appendix A.2.4.

Remark 4.7

The committor equations can also be written on a time-augmented state space using a period-augmented transition matrix that pushes the dynamics deterministically forward in time

$$\begin{aligned} P_\text {Aug} = \begin{pmatrix} 0 &{} P_0 &{} &{} 0 \\ &{} \ddots &{} \ddots &{} \\ &{} &{} 0&{}P_{M-2} \\ P_{M-1} &{} &{} &{} 0 \end{pmatrix}. \end{aligned}$$

Extending this approach, one can also consider committor equations for systems that switch stochastically with probabilities ${\hat{P}}\in {\mathbb {R}}^{M\times M}$ between M different regimes, and each regime is described by a transition matrix $P_m$. This is essentially the Markov chain analogue of a random dynamical system (Arnold 1995). The regime-augmented transition matrix is given by

$$\begin{aligned} P^{\text {switch}}_\text {Aug} = \begin{pmatrix} {\hat{P}}_{11} P_1 &{} \dots &{} {\hat{P}}_{1M} P_M \\ \vdots &{} \ddots &{} \vdots \\ {\hat{P}}_{M1} P_1&{} \dots &{} {\hat{P}}_{MM} P_M \end{pmatrix}. \end{aligned}$$

We refer to Appendix A.1 for more details on both ansatzes and the computation of committor probabilities on the augmented space. The augmented approach also offers a numerical way of solving the committor equations with periodic boundary conditions.

4.3 Transition Statistics

We have seen that the forward and backward committors in the case of periodically driven dynamics are also $M-$periodic and can be computed from the iterative equations (16), (17) with periodic conditions in time. Since committors, densities, and transition matrices are $M-$periodic, all statistics computed thereof are so too by the theory in Sect. 2.2, and we equip them with a subscript m, e.g., ${\hat{\mu }}^{AB}(n) = {\hat{\mu }}^{AB}_m$, $f^{AB}(n) = f^{AB}_{m}$, whenever $n\equiv m$ modulo M.

Compared to the previous case of stationary, infinite-time, the discrete rate of reactive trajectories leaving A at time m, $\smash { k^{A\rightarrow }_m = \sum _{i\in A, j\in {\mathbb {S}}} f^{AB}_{ij}(m) }$, does not anymore equal the discrete rate of reactive trajectories arriving in B at time m, $\smash { k^{\rightarrow B}_m = \sum _{i\in {\mathbb {S}}, j\in B} f^{AB}_{ij}(m-1) }$.

The next theorem provides us with the reactive current conservation laws in the case of periodic dynamics and will allow us to find the relation between $k^{A\rightarrow }_m$ and $k^{\rightarrow B}_m$.

Theorem 4.8

Consider a Markov chain satisfying Assumptions 4.1 and 4.3. Then, for each node $i\in C$ and time $m\in {\mathbb {M}}$ we have the following current conservation law

$$\begin{aligned} \sum _{j\in {\mathbb {S}}}&f_{m,ij}^{AB} = \sum _{j\in {\mathbb {S}}} f_{m-1,ji}^{AB}, \end{aligned}$$

(21)

i.e., all the reactive trajectories that flow out of i at time (congruent to) m have flown into i at time equivalent to $m-1$.

Further, over one period the amount of reactive flux leaving A is the same as the amount of flux entering B, i.e.,

$$\begin{aligned} \sum _{m\in {\mathbb {M}}} \sum \limits _{\begin{array}{c} i \in A\\ j\in {\mathbb {S}} \end{array}} f_{m,ij}^{AB} = \sum _{m\in {\mathbb {M}}} \sum \limits _{\begin{array}{c} i\in {\mathbb {S}}\\ j\in B \end{array}} f_{m,ij}^{AB}. \end{aligned}$$

(22)

The proof can be found in Appendix A.2.5 and follows from straightforward computations using the committor equations to rewrite the reactive current.

As a result of (22), the discrete out-rate averaged over one period equals the average discrete in-rate, which we define to be ${\bar{k}}^{AB}_{M}$, i.e.,

$$\begin{aligned} {\bar{k}}^{AB}_{M}:= \frac{1}{M} \sum _{m\in {\mathbb {M}}} k^{A\rightarrow }_m = \frac{1}{M} \sum _{m\in {\mathbb {M}}} k^{\rightarrow B}_m. \end{aligned}$$

This period-averaged discrete rate tells us the average probability per time step of a reactive trajectory to depart in A or in other words, the expected number of reactive trajectories leaving A per time step.

4.4 Numerical Example 2: Periodically Varying Dynamics on a 5-State Network

We consider the same $5-$state network as before in Example 1, but this time the Markov chain is in equilibrium to the periodically varying transition probabilities

$$\begin{aligned} P_m = T + \cos \left( \frac{2m \pi }{M}\right) \, L,\ m\in \{0,\dots ,M-1\}, \end{aligned}$$

with period length $M=6$. The transition matrices are chosen such that for time $m=0$, the dynamics are the same as in the stationary Example 1, $P_0=T+L=P$, whereas at time $m=3$, the dynamics are reversed $P_3 = T-L$. Matrix T is a fixed row-stochastic matrix which has transition probabilities that are symmetric along the axis through A and B and L is a 0-rowsum matrix that is not symmetric along the axis through A and B. The probabilities of the transition matrices are shown in Fig. 5.

With the numerically computed transition statistics as shown in Fig. 6, we try to answer whether the added perturbation results in alternative effective paths compared to Example 1. The effective current indicates that the most likely effective transition path from A to B either goes via $A\rightarrow 1\rightarrow B$ (with a likely detour to 2) during the first half of the period, or via $A\rightarrow 3\rightarrow B$ (with a detour to 2) toward the second half of the period. But interestingly, we have additional transition paths that go via $A \rightarrow 1\rightarrow 2\rightarrow 3\rightarrow B$ and $A \rightarrow 3\rightarrow 2\rightarrow 1\rightarrow B$. Neither in the stationary system described by $P_0=T+L$ nor by $P_3 = T-L$, this path would be possible. Additionally, the period-averaged rate ${\bar{k}}^{AB}_{6} = 0.034$ is higher than the rate in the stationary case (Example 1).

Fig. 5

The dynamics of the periodic system of Example 2 are given by the transition matrix $P_m = T + \cos \left( \frac{m\pi }{3}\right) L$ for the times $m = n \, (\text {mod 6})$; shown here are the transition matrices a $T+L$, b T and c $T-L$

Fig. 6

We show statistics of Example 2, namely a the normalized distribution which is shown by the color of the nodes, ranging from yellow (low values) via orange to purple (high value), as well as b the effective current from one state i to state j that is indicated by the thickness and grayscale of the arrow pointing from i to j. The node color of the effective current plot indicates the outflow of A and the inflow into B of current at the corresponding time point m, which can be related to the $k^{A\rightarrow }_m$ and $k^{\rightarrow B}_{m+1}$ (Color figure online)

5 TPT for Markov Chains on a Finite-Time Interval

We now develop TPT for Markov chains with the transitions of interest taking place during a finite-time interval. The transition rules can be time-dependent, and the dynamics can be non-stationary, i.e., out of equilibrium.

The resulting committor equations (Sect. 5.2) and statistics (Sect. 5.3) in case of finite-time dynamics are similar as in the periodic case, yet there are some distinctions. The committor equations are now equipped with final, respectively, initial conditions, and the statistics show some boundary effects at the time interval limits.

In Sect. 5.4, we also provide a consistency result between the stationary, infinite-time case and the finite-time case for a stationary Markov chain, by considering the limit of the time interval going to infinity.

5.1 Setting

Let us start by describing the systems of interest in this section.

Assumption 5.1

We consider a Markov chain on a finite-time interval $(X_n)_{0 \le n \le N-1}$, $N \in {\mathbb {N}}$, taking values in a discrete and finite space ${\mathbb {S}}$. The probability of a jump from the state $i \in {\mathbb {S}}$ to the state $j \in {\mathbb {S}}$ at time $n \in \{0, \dots , N-2\}$ is given by the $(i,j)-$entry of the row-stochastic transition matrix $P(n) = (P_{ij}(n))_{i,j \in {\mathbb {S}}}$:

$$\begin{aligned} P_{ij}(n) := {\mathbb {P}}(X_{n+1} = j \,|\, X_n = i). \end{aligned}$$

Setting the initial density $\lambda (0) = (\lambda _i(0))_{i \in {\mathbb {S}}}$, the densities at later times $n\in \{1,\dots ,N-1\}$ are given by $\lambda (n+1)^\top = \lambda (n)^\top P(n)$.

By these assumptions, the chain can have time-inhomogeneous transition probabilities, or even if $P(n)=P$ for all n, the densities $\lambda (n)$ can be changing in time. Also, we are not requiring the chain to be irreducible anymore.

The time-reversed process $(X^-_n)_{0 \le n \le N-1}$ defined by ${X^-_n := X_{N-1-n}}$ is also a Markov chain (Ribera Borrell 2019, Thm 2.1.18). Its transition probabilities are given for any $n \in \{1, \dots , N-1\}$ by

$$\begin{aligned} \begin{aligned} P^-_{ij}(n)&:= {\mathbb {P}}(X^-_{N-1-n+1} = j \,|\, X^-_{N-1-n} = i) \\&= {\mathbb {P}}(X_{n-1} = j \,|\, X_{n} = i) = \frac{\lambda _{j}(n-1)}{\lambda _{i}(n)} \, P_{ji}(n-1) \end{aligned} \end{aligned}$$

(23)

whenever $\lambda _{n}(i) >0$. From the backward transition probabilities (23), we note that even the time-reversed process of a finite-time, time-homogeneous Markov chain (i.e., $P(n)=P$ for all n) is in general a finite-time, time-inhomogeneous Markov chain, unless also the distribution $\lambda _{}(n)$ is time-independent.

5.2 Committor Probabilities

The forward (4) and backward committors (5) keep their dependence on the time of the chain $n \in \{ 0, \dots , N-1\}$. The following theorem provides us with two iterative equations for the forward and backward committors. Because one can solve (24) and (25) iteratively, the solutions exist and are unique.

Theorem 5.2

The forward committor for a finite-time Markov chain of form 5.1 satisfies the following iterative system of equations for $n \in \{0, \dots , N-2\}$:

$$\begin{aligned} \left\{ \begin{array}{rcll} q_i^+(n) &{}=&{} \sum \limits _{j \in {\mathbb {S}}} \, P_{ij}(n) \, q_j^+(n+1) &{} i \in C \\ q_i^+(n) &{}=&{} 0 &{} i \in A \\ q_i^+(n) &{}=&{} 1 &{} i \in B \end{array}\right. \end{aligned}$$

(24)

with final condition $q^+_i(N-1) = \mathbb {1}_B(i)$. Analogously, the backward committor for a finite-time Markov chain satisfies for ${n \in \{1, \dots , N-1\}}$

$$\begin{aligned} \left\{ \begin{array}{rcll} q_i^-(n) &{}=&{} \sum \limits _{j \in {\mathbb {S}}} \, P^-_{ij}(n) \, q_j^-(n-1) &{} i \in C\\ q_i^-(n) &{}=&{} 1 &{} i \in A \\ q_i^-(n) &{}=&{} 0 &{} i \in B \end{array}\right. \end{aligned}$$

(25)

with initial condition $q_i^-(0) = \mathbb {1}_A(i)$.

The proof uses some of the arguments of the proofs above for the stationary, infinite-time and periodic infinite-time cases and can be found in Appendix A.2.6.

The following theorem provides us with an analogue result to Theorem 3.5 for the forward and the backward committors of a finite-time Markov chain written in terms of path probabilities of the paths that start in A and end in B within the restricted time frame ${\mathbb {T}}$.

Lemma 5.3

The forward committor at time $n \in \{0, \dots , N-2\}$ and the backward committor at time ${n \in \{1, \dots , N-1\}}$, respectively, satisfy for $i\in C$ the following equalities:

$$\begin{aligned} q_i^+(n)&= \sum \limits _{\tau =n+1}^{N-1} \sum _{\begin{array}{c} i_\tau \in B \\ i_{n +1}, \dots , i_{\tau -1} \in C \end{array}} P_{i i_{n+1}}(n) \cdots P_{i_{\tau -1} i_\tau }(\tau -1) \end{aligned}$$

(26)

$$\begin{aligned} q_i^-(n)&= \sum \limits _{\tau =0}^{n-1} \sum _{\begin{array}{c} i_{\tau } \in A \\ i_{\tau + 1}, \dots , i_{n-1} \in C \end{array}} P^-_{i i_{n-1}}(n) \cdots P^-_{i_{\tau +1} i_{\tau }}(\tau +1) . \end{aligned}$$

(27)

The proof follows from rewriting $q_i^+(n)$ for any time $n \in \{0, \dots , N-2\}$ into a decomposition of the probabilities of all possible paths that reach B within the time interval $\{n+1,\dots ,N-1\}$ and rewriting $q_i^-(n)$ for any time ${n \in \{1, \dots , N-1\}}$ into a decomposition of the probabilities of all possible paths that came from A within $\{1, \dots , n-1\}$.

Remark 5.4

Similar as in the periodic case, it is possible to extend the approach to time-dependent sets (i.e., space-time sets) A(n) and B(n). For instance, in order to study transitions that leave a set at a certain time (e.g., at time $n=0)$ and arrive at a certain time (e.g., at $n=N$) we can choose $A(n) = \mathbb {1}_{\{0\}}(n) A$ and $B(n) = \mathbb {1}_{\{N\}}(n) B$.

5.3 Transition Statistics and Their Interpretation

We have seen that the forward and backward committors for a finite-time Markov chain can be computed from the iterative equations (24) and (25) with final, respectively, initial conditions. Based on these, we will next introduce the corresponding transition statistics.

The distribution of reactive trajectories (Definition 2.1) is defined for any time ${n \in \{0, \dots , N-1 \}}$, and by Theorem 2.2, it is given by

$$\begin{aligned} \mu ^{AB}_i(n) = q_i^-(n) \ \lambda _{n}(i) \ q_i^+ (n). \end{aligned}$$

Observe that $\mu ^{AB}_i(0) = \mu ^{AB}_i(N-1) = 0$ because there are no reactive trajectories at these times. Thus, the distribution of reactive trajectories cannot be normalized at times 0 and $N-1$. As a consequence, the normalized distribution of reactive trajectories ${\hat{\mu }}^{AB}(n)$ is just defined for times $n \in \{1, \dots , N-2\}$.

The current of reactive trajectories is defined for any time $n \in \{0, \dots , N-2\}$, and it is given by (Theorem 2.5)

$$\begin{aligned} f^{AB}_{ij}(n) = q_i^-(n) \ \lambda _{n}(i) \ P_{ij}(n) \ q^+_j(n+1) . \end{aligned}$$

Similarly, the effective current of reactive trajectories

$$\begin{aligned} f^+_{ij}(n) :=\max \{ f^{AB}_{ij}(n) - f^{AB}_{ji}(n),0\} \end{aligned}$$

is defined only for times $n \in \{0, \dots , N-2\}$.

Also in this case, the current satisfies certain conservation principles:

Theorem 5.5

For a finite-time Markov chain $(X_n)_{0 \le n \le N-1}$ satisfying Assumption 5.1, the reactive current flowing out of the node $i \in C$ at time n equals the current flowing into a node $i \in C$ at time $n-1$, i.e.,

$$\begin{aligned} \sum _{j\in {\mathbb {S}}} f^{AB}_{ij}(n) = \sum _{j\in {\mathbb {S}}} f^{AB}_{ji}(n-1) \end{aligned}$$

(28)

for $n \in \{1, \dots , N-2\}$. Further, the reactive current flowing out of A into ${\mathbb {S}}$ over the whole time period $\{0,\dots ,N-2\}$ equals the flow of reactive trajectories from ${\mathbb {S}}$ into B over the period

$$\begin{aligned} \sum \limits _{n=0}^{N-2} \sum _{i\in A, j\in {\mathbb {S}}} f^{AB}_{ij}(n) = \sum \limits _{n=0}^{N-2} \sum _{ i\in {\mathbb {S}}, j\in B} f^{AB}_{ij}(n). \end{aligned}$$

(29)

The proof can be found in Appendix A.2.7.

The discrete rate of transitions leaving A is defined for times $n \in \{0, \dots , N-2\}$

$$\begin{aligned} k^{A\rightarrow }(n) = \sum _{i\in A, j\in {\mathbb {S}}} f^{AB}_{ij}(n), \end{aligned}$$

whereas the discrete rate of transitions entering B is defined for times $n \in \{1, \dots , N-1\}$

$$\begin{aligned} k^{\rightarrow B}(n) = \sum _{i\in {\mathbb {S}}, j\in B} f^{AB}_{ij}(n-1) . \end{aligned}$$

By plugging the definitions of the rates into result (29) of Theorem 5.5 and by re-indexing the times, we note that the discrete departure rate averaged over the times ${n \in \{0, \dots , N-2\}}$ equals the time-averaged discrete arrival rate over the times ${n \in \{1, \dots , N-1\}}$, which we denote by ${\bar{k}}^{AB}_{N}$, i.e.,

$$\begin{aligned} {\bar{k}}^{AB}_{N} := \frac{1}{N}\sum _{n=0}^{N-2} k^{A\rightarrow }(n) = \frac{1}{N}\sum _{n=1}^{N-1} k^{\rightarrow B}(n) . \end{aligned}$$

In the infinite-time, stationary case, Theorem 3.8 tells us that $k^{AB}$ equals the time average of the number of reactive pieces departing per time step along a single infinitely long trajectory. Here, we cannot apply the ergodic theorem to turn ${\bar{k}}^{AB}_{N}$ into an average along a single trajectory. Instead, we can write the time-averaged rate in terms of an ensemble of trajectories to get a better understanding. For this, we take $K\in {\mathbb {N}}$ i.i.d. realizations of the finite-time chain, i.e., each sample $\smash { ({\hat{X}}_n^i)_{n\in \{0,\dots ,N-1\}} }$ is distributed according to the law of the finite-time dynamics with ${\hat{X}}_0^i \sim \lambda (0)$. Then, we have by the law of large numbers:

$$\begin{aligned} \begin{aligned} {\bar{k}}^{AB}_{N}&=\frac{1}{N} \sum \limits _{n=0}^{N-2} {\mathbb {P}}(X_n\in A, \tau _B^+(n+1) < \tau _A^+(n+1)) \\&=\frac{1}{N} \sum \limits _{n=0}^{N-2} {\mathbb {E}}\left[ \mathbb {1}_A(X_n) \mathbb {1}_B\left( X_{\tau ^+_{A\cup B}(n+1)}\right) \right] \\&= \lim _{K\rightarrow \infty } \frac{1}{N} \sum \limits _{n=0}^{N-2}\frac{1}{K} \sum \limits _{i=1}^{K} \mathbb {1}_A({\hat{X}}_n^i) \mathbb {1}_B\left( {\hat{X}}^i_{\tau ^+_{A\cup B}(n+1)}\right) . \end{aligned} \end{aligned}$$

(30)

Further, we can rewrite the second line of (30) as

$$\begin{aligned} \begin{aligned} {\bar{k}}^{AB}_{N}&=\frac{1}{N} \sum \limits _{n=0}^{N-2} {\mathbb {E}}\left[ \mathbb {1}_A(X_n) \mathbb {1}_B\left( X_{\tau ^+_{A\cup B}(n+1)}\right) \right] \\&= \frac{1}{N} {\mathbb {E}}\left[ \sum \limits _{n=0}^{N-2} \mathbb {1}_A(X_n) \mathbb {1}_B\left( X_{\tau ^+_{A\cup B}(n+1)}\right) \right] \end{aligned} \end{aligned}$$

(31)

and thus giving the average rate ${\bar{k}}^{AB}_{N}$ the interpretation of the total expected amount of reactive trajectories within $\{0,\dots ,N-1\}$ divided by the number of time steps. Analogously, we can apply the same argument for the time-averaged probability of being on a transition

$$\begin{aligned} {\bar{Z}}_N^{AB} := \frac{1}{N} \sum _{n=0}^{N-1} Z^{AB}(n) = \frac{1}{N}{\mathbb {E}}\left[ \sum \limits _{n=0}^{N-1} \mathbb {1}_A\left( X_{\tau ^-_{A\cup B}(n)}\right) \mathbb {1}_B\left( X_{\tau ^+_{A\cup B}(n)}\right) \right] , \end{aligned}$$

which can be understood as the expected number of time steps the Markov chain is on a transition during $\{0, \dots , N-1\}$ divided by N. Last, we define the ratio

$$\begin{aligned} {\bar{t}}_N^{AB}:= \frac{{\bar{Z}}_N^{AB}}{{\bar{k}}_N^{AB}} \end{aligned}$$

and observe that it provides us with the average expected duration of a reactive trajectory over ${n \in \{0, \dots , N-1\}}$.

Remark 5.6

In von Kleist et al. (2018), transitions from A to B are studied in non-ergodic and non-stationary processes. But the focus there is put on the first pieces of the trajectories starting in $B^c$ which arrive in B after having been to A (called the first passage paths), whereas we consider the ensemble of all the transitions from A to B within the time interval ${\mathbb {T}}$. Further, the first passage paths are divided into non-reactive and reactive segments and statistics for both, the reactive and the non-reactive ensemble, are computed. Last, their approach does not have any restrictions on the length of the first passage path and therefore is not straightforwardly extendable to finite-time processes.

5.4 Convergence Results of TPT Objects from Finite-Time Systems to Infinite-Time

The aim of this section is to show the consistency of the committors and TPT objects between the finite-time case and infinite-time case. We will show that when assuming a time-homogeneous chain on a finite-time interval that is stationary, the TPT objects converge to the infinite-time, stationary case when letting the time interval go to infinity.

To this end, let us consider the Markov chain $(X_n)_{-N \le n \le N}$ on the time interval $\{-N, \dots , N\}$ with a time-homogeneous and irreducible transition matrix $P=(P_{ij})_{i,j \in {\mathbb {S}}}$. When choosing the unique, strictly positive invariant density ${\pi = (\pi _i)_{i \in {\mathbb {S}}}}$ of P as an initial density, the density for all ${n \in \{-N, \dots , N\}}$ is given by

$$\begin{aligned} {\mathbb {P}}(X_n = i) = \pi _i . \end{aligned}$$

(32)

The time-reversed process $(X^-_n)_{-N \le n \le N}$ is also time-homogeneous and stationary, since its transition probabilities are given by

$$\begin{aligned} P^-_{ij} = \frac{\pi _j}{\pi _i} \, P_{ji}. \end{aligned}$$

(33)

The forward committor and the backward committor for a finite-time Markov chain on the time interval $\{-N, \dots , N\}$ satisfy (24) and (25) with a slight adjustment of the time interval.

The next theorem provides us the desired result:

Theorem 5.7

The committors and transition statistics defined for an irreducible Markov chain in stationarity on a finite-time interval correspond in the limit that the interval $\{-N, \dots , N\}$ tends to ${\mathbb {Z}}$ to the objects defined for a stationary, infinite-time Markov chain. For any $i, j \in {\mathbb {S}}$, it holds that

$$\begin{aligned}&\lim _{\begin{array}{c} n \in \{-N, \dots , N-1\} \\ N \rightarrow \infty \end{array}} q_i^+(n) = q_i^+ , \\&\lim _{\begin{array}{c} n \in \{-N+1, \dots , N\} \\ N \rightarrow \infty \end{array}} q_i^-(n) = q_i^- , \\&\lim _{\begin{array}{c} n \in \{-N+1, \dots , N-1\} \\ N \rightarrow \infty \end{array}} \mu ^{AB}_i(n) = \mu ^{AB}_i , \\&\lim _{\begin{array}{c} n \in \{-N+1, \dots , N-2\} \\ N \rightarrow \infty \end{array}} f^{AB}_{ij}(n) = f^{AB}_{ij} , \\&\lim _{\begin{array}{c} n \in \{-N+1, \dots , N-2\} \\ N \rightarrow \infty \end{array}} k^{A\rightarrow }(n) = k^{A\rightarrow }, \\&\lim _{\begin{array}{c} n \in \{-N+2, \dots , N-1\} \\ N \rightarrow \infty \end{array}} k^{\rightarrow B}(n) = k^{\rightarrow B}. \end{aligned}$$

Proof

First, we see that the forward committor at time ${n \in \{-N, \dots , N-1\}}$ and the backward committor at time ${n \in \{-N+1, \dots , N\}}$ for finite-time Markov chains correspond in the limit that $N \rightarrow \infty$ to the forward and the backward committors defined for stationary, infinite-time Markov chains. For any ${n \in \{-N, \dots , N-1\}}$, we can see that

$$\begin{aligned} \begin{aligned} \lim _{\begin{array}{c} n \in \{-N, \dots , N-1\} \\ N \rightarrow \infty \end{array}} q_i^+(n) \overset{(1)}{=} \sum \limits _{\tau \in {\mathbb {Z}}^+} \sum _{\begin{array}{c} i_1, \dots , i_{\tau -1} \in C \\ i_\tau \in B \end{array}} P_{i i_1} \dots P_{i_{\tau -1} i_\tau }&\overset{(2)}{=} q_i^+ , \end{aligned} \end{aligned}$$

(34)

where (1) follows from applying first Lemma 5.3 and then re-indexing the coefficients of the transition matrix P, and (2) follows directly from Lemma 3.5. Analogously, we obtain for any $n \in \{-N +1, \dots , N\}$ that

$$\begin{aligned} \begin{aligned} \lim \limits _{\begin{array}{c} n \in \{-N+1, \dots , N\} \\ N \rightarrow \infty \end{array}} q_i^-(n) = \sum \limits _{\tau \in {\mathbb {Z}}^-} \sum _{\begin{array}{c} i_\tau \in A \\ i_{\tau +1}, \dots , i_{-1} \in C \end{array}} P^-_{i_{\tau +1} i_\tau } \dots P^-_{i i_{-1}} = q_i^- . \end{aligned} \end{aligned}$$

(35)

Hence, by putting together (34), (35) and (32) we show that for any $i, j \in {\mathbb {S}}$

$$\begin{aligned} \lim _{\begin{array}{c} n \in \{-N+1, \dots , N-1\} \\ N \rightarrow \infty \end{array}} \mu ^{AB}_i(n)&= q_i^-(n) \, {\mathbb {P}}(X_n = i)\, q_i^+(n) = q_i^- \, \pi _i \, q_i^+ = \mu ^{AB}_i , \end{aligned}$$

(36)

$$\begin{aligned} \lim _{\begin{array}{c} n \in \{-N+1, \dots , N-2\} \\ N \rightarrow \infty \end{array}} f^{AB}_{ij}(n)&= q_i^-(n) \, {\mathbb {P}}(X_n = i) \, P_{ij} \, q_j^+(n+1) = q_i^- \, \pi _i \, P_{ij} \, q_j^+ = f^{AB}_{ij} , \end{aligned}$$

(37)

$$\begin{aligned} \lim _{\begin{array}{c} n \in \{-N+1, \dots , N-2\} \\ N \rightarrow \infty \end{array}} k^{A\rightarrow }(n)&= \sum _{\begin{array}{c} i \in A \\ j \in {\mathbb {S}} \end{array}} f^{AB}_{ij}(n) = \sum _{\begin{array}{c} i \in A \\ j \in {\mathbb {S}} \end{array}} f^{AB}_{ij} = k^{A\rightarrow }, \end{aligned}$$

(38)

$$\begin{aligned} \lim _{\begin{array}{c} n \in \{-N+2, \dots , N-1\} \\ N \rightarrow \infty \end{array}} k^{\rightarrow B}(n)&= \sum _{\begin{array}{c} i \in {\mathbb {S}}\\ j \in B \end{array}} f^{AB}_{ij}(n-1) = \sum _{\begin{array}{c} i \in {\mathbb {S}}\\ j \in B \end{array}} f^{AB}_{ij} = k^{\rightarrow B}. \end{aligned}$$

(39)

$\square$

5.5 Numerical Example 3: Time-Homogeneous, Finite-Time Dynamics on a 5-State Network

Fig. 7

The transition statistics of the finite-time system of Example 3 are shown for times $n\in \{0,\dots , 4\}$. In a, the node color indicates the value of the normalized distribution, ranging from yellow (low values) via orange to purple (high value). In b, the strength of the effective current from one state i to state j is shown by the thickness and grayscale of the arrow pointing from i to j, and rates $k^{A\rightarrow }_n$, respectively, $k^{\rightarrow B}_{n+1}$ are shown by the color of the nodes A and B of the effective current plot (Color figure online)

We consider a time-homogeneous Markov chain over ${\mathbb {T}}=\{0,1,\dots ,N-1\}$, $N=5$ with the transition rules given by $P(n) = P$ for all $n \in \{0,1,\dots ,N-2\}$. P is the transition matrix used in Example 1, and the initial density is given by the stationary density $\pi$ of P. Thus, we are in the same setting as before in Example 1, but we are now looking at transitions between A and B that are restricted to take place within the finite-time window.

The transition statistics computed for this example are shown in Fig. 7. Since the time interval was chosen rather small, transitions via 1, that usually do a detour to the metastable state 2, are very unlikely. The transition path via 3 is more likely, we can see that the effective current through 3 is much stronger, and also, the normalized reactive distribution indicates that it is the most likely to be in node 3 when reactive. We also note that the rate ${\bar{k}}_5^{AB}=0.005$ is much smaller than in the stationary case, and only few transitions from A to B are completed (on average) in the short time frame $\{0,1,\ldots ,N-1\}$.

5.6 Numerical Example 4: Time-Inhomogeneous, Finite-Time Dynamics on a 5-State Network

Fig. 8

The dynamics of the finite-time, time-inhomogeneous system (Example 4) are given by a $P(n) = P + K$ at even times, and b $P(n) = P - K$ at odd times

Fig. 9

Here we show a the normalized distribution and b the current of transitions from A to B for the finite-time, time-inhomogeneous system (Example 4)

As a next example, we consider a time-inhomogeneous chain over the finite-time window ${\mathbb {T}}= \{0, \dots , N-1\}$, $N=5$. We again set the stationary density of P as an initial density and let the transition matrices depend on the time but in such a way that $\frac{1}{N-1}\sum _n P(n) = P$. For any $n \in \{0, \dots , N-2\}$, let

$$\begin{aligned} P(n) = \left\{ \begin{array}{ll} P + K &{} \text {if} \quad n \; (\text {mod} \; 2) = 0 \\ P - K &{} \text {if} \quad n \; (\text {mod} \; 2) = 1 \end{array}\right. \end{aligned}$$

where K is the 0-rowsum matrix as given in Fig. 8. At times $n = 0, 2$, the transition matrices become $P + K$ and the subsets A and B are less metastable. At times $n=1, 3$ the transition matrices become $P - K$ and A and B are more metastable.

The computed transition statistics are shown in Fig. 9. The effective current plot shows that the majority of reactive trajectories are leaving A at times $n=0, 2$ and go to 3 and at times $n=1, 3$, they move from 3 to B. The time-averaged rate for the finite-time, time-homogeneous case from Example 3 is ${\bar{k}}_{5}^{AB} = 0.005$, while the time-averaged rate for this time-inhomogeneous case is ${\bar{k}}_{5}^{AB} =0.012$, more than two times larger. We thus demonstrated that by adding a forcing to the time-homogeneous dynamics that changes the metastability of A and B, we can increase the rate of transitions, even though the forcing vanishes on average. We note that utilizing perturbations that tip a system out of equilibrium is used in statistical mechanics to accelerate convergence of statistics (Hamelberg et al. 2004; Lelièvre et al. 2013).

5.7 Numerical Example 5: Growing Finite-Time Window for the 5-State Network

Let us consider the stationary Markov chain introduced in Example 3 but on the time interval ${\mathbb {T}}= \{-N, \dots , N\}$. We want to see numerically that the forward, respectively, backward committor at time n converges under the $l^2$-norm to the forward, respectively, backward committor of the infinite-time, stationary system by extending the time interval ${{\mathbb {T}}= \{-N, \dots , N\}}$ for N big enough such that $N \pm n \gg 1$, i.e.,

$$\begin{aligned} \lim _{\begin{array}{c} n \in \{-N, \dots , N-1\} \\ N \pm n\gg 1 \end{array}} ||q^+ - q^+(n)||_2 \approx 0, \&\lim _{\begin{array}{c} n \in \{-N +1, \dots , N\} \\ N \pm n \gg 1 \end{array}} ||q^- - q^-(n)||_2 \approx 0. \end{aligned}$$

In Fig. 10, we show this convergence numerically; note that the statistics will not necessarily converge near the boundary of the interval.

Fig. 10

The committors at time ${n=0}$ of the finite-time chain with stationary dynamics (Example 5) converge under the $l^2$-norm to the committors of the infinite-time chain (Example 1). The $l_2$-errors decrease at an exponential rate

6 Numerical Examples in a Triple Well Landscape

In this section, we exemplarily study the transition behavior of a particle diffusing in a triple well energy landscape for several scenarios: for infinite-time, stationary dynamics (Sect. 6.1), for infinite-time dynamics with an added periodic forcing (Sect. 6.2), and for stationary, finite-time dynamics (Sects. 6.3, 6.4).

6.1 Numerical Example 6: Infinite-Time, Stationary Dynamics

We consider the diffusive motion of a particle $(X_t)_{t\in {\mathbb {R}}}$ in ${\mathbb {R}}^2$ according to the overdamped Langevin equation

$$\begin{aligned} \mathrm{d}X_t = - \nabla V(X_t) \mathrm{d}t + \sigma \mathrm{d}W_t \end{aligned}$$

(40)

where $\sigma >0$ is the diffusion constant, $W_t$ is a $2-$dimensional standard Brownian motion (Wiener process), and $V(x,y): {\mathbb {R}}^2 \rightarrow {\mathbb {R}}$ is the triple well potential given in Metzner et al. (2009), Schütte and Sarich (2013),

$$\begin{aligned} \begin{aligned} V(x,y) =&\frac{3}{4} \exp \left( -x^2-\left( y-\frac{1}{3}\right) ^2\right) - \frac{3}{4} \exp \left( -x^2-\left( y-\frac{5}{3}\right) ^2\right) \\&-\frac{5}{4} \exp \left( -\left( x-1\right) ^2 - y^2\right) - \frac{5}{4} \exp \left( -\left( x+1\right) ^2-y^2\right) + \frac{1}{20} x^4 \\&+ \frac{1}{20} \left( y-\frac{1}{3}\right) ^4 \end{aligned} \end{aligned}$$

(41)

and shown in Fig. 11. The particle is pushed by the force $-\nabla V(x,y)$ “downhill” in the energy landscape, while simultaneously being subject to a random forcing due to the Brownian motion term. The stationary density of the process is given by the Boltzmann distribution

$$\begin{aligned} \pi (x,y) = Z^{-1} \exp (-2\sigma ^{-2} V(x,y)) \end{aligned}$$

with normalization factor Z.

Fig. 11

a Triple well landscape V(x, y) and the resulting force when unperturbed and when periodically perturbed. b The negative gradient $-\nabla V(x,y)$ is the force that each particle in Example 6 without perturbation feels, while in Example 7, a periodically varying circulative force is added resulting at times $m=0$ and $m=3$ in the shown forces

Fig. 12

TPT objects for stationary, infinite-time dynamics in the triple well landscape as explained in Example 6. The last plot shows the accumulated $f^+$ for each state i (see the footnote)

Before applying TPT on this example, we have to discretize the process into a Markov chain. We want to estimate a transition matrix that gives us the probability to jump between discrete cells of the state space $[-2,2]\times [-1,2]$, here we choose regular square grid cells $\{A_i,i=1,\ldots \}$ of size $0.2 \times 0.2$. By means of Ulams method [Ulam 1960, see also (Koltai 2011a, Chapter 2.3) for a summary of the method] one can project the dynamics of (40) onto the space spanned by indicator functions on the grid cells and then further approximate the projected dynamics by a Monte Carlo sum of sampled trajectory snippets

$$\begin{aligned} \begin{aligned} P_{ij}&\approx \frac{1}{C} \sum _{c=1}^C \mathbb {1}_{A_j}({\hat{Y}}^c), \end{aligned} \end{aligned}$$

(42)

where we sample $\hat{X}^c$, $c=1,\dots ,C$, uniformly from the cell $A_i$ and we get ${\hat{Y}}^c$ by evolving the sample forward in time according to (40) with time step $\tau$ (e.g., by using the Euler–Maruyama discretization of the stochastic differential equation (40)). The resulting process defined by the transition matrix $P$ is a discrete-time, discrete-space Markov chain with time steps $\tau$.⁶

We are now interested in the transition behavior between the deep wells of V when the dynamics are stationary. Therefore, we choose sets A and B as centered at $(-1,0)$ and (1, 0), respectively, and we ask: Which transition path from A to B is more likely; via the third metastable shallow well at (0, 1.5), or via the direct barrier between A and B?

The computed committor functions and statistics of the reactive trajectories are shown in Fig. 12, and we chose $\sigma = 1$, $\tau = 0.3$ and sampled $C=10,000$ for estimating the transition matrix. Since the dynamics are reversible, the backward committor is just $1-q^+$. Also, we can see that the committors are close to constant inside the metastable sets (the wells) due to the fast mixing inside the wells, but vary across the barriers. The computed effective current⁷ indicates that most transitions occur via the direct barrier between A and B. The effective current through the shallow well at (0, 1.5) is only very small, but due to the metastability of the well, reactive trajectories taking that path are stuck there for long and thus contributing a lot to the density $\mu ^{AB}$. The rate of transitions is $k^{AB}= 0.0142$, and the mean transition time is $t^{AB}= 10.01$.

6.2 Numerical Example 7: Periodic, Infinite-Time Dynamics

Fig. 13

a The density and b effective current of reactive trajectories for the triple well dynamics with periodic circulative forcing that is sometimes clockwise, sometimes anticlockwise, see Example 7. c The rates $k^{A\rightarrow }_m$ and $k^{\rightarrow B}_m$ indicate the probability of a reactive trajectory to leave A, respectively, enter B per time step

Next, we are interested in studying transitions in the triple well landscape when a periodically varying forcing is applied. We add the force $F(x_1,x_2, t) = 1.4 \, \cos \left( \frac{2 \pi t}{ 1.8}\right) (-x_2,x_1)$ that alternatingly due to the cosine modulation exhibits an anticlockwise circulation and a clockwise circulation, to the dynamics (40), resulting in the diffusion process with 1.8-periodic forcing,

$$\begin{aligned} \mathrm{d}X_t =( - \nabla V(X_t) + F(X_t, t) ) \mathrm{d}t + \sigma \mathrm{d}W_t. \end{aligned}$$

(43)

We again discretize the dynamics and estimate transition matrices $P_0, P_1, \dots ,P_{M-1}$ ($M=6$) for $\tau$-spaced time points during the period, and each transition matrix is mapping $\tau =0.3$ into the future. In Fig. 11, the force from the potential plus circulation is shown for time points $m=0$ and $m=3$.

Considering the same A and B as before, we are now interested in the transition channels when the dynamics are equilibrated to the periodic forcing. The computed results are shown in Fig. 13, the reactive trajectories take the lower channel via the direct barrier at the beginning of the period due to the additional push from the forcing, and the upper channel via the shallow well toward the end of the period when the applied circulation is clockwise. It is interesting to note that the rate of reactive trajectories leaving A and entering B is highest toward the end of the period, when the added forcing is clockwise and the preferred channel is the upper channel. This is contrary to the stationary example before, where reactive trajectories dominantly passed the lower channel.

Fig. 14

a The density and b effective current of reactive trajectories for the stationary triple well dynamics over a finite-time window $\{0,\dots , 5\}$, see Example 8. Note that some quantities are not defined for all $n\in \{0,\dots ,N-1\}$

6.3 Numerical Example 8: Finite-Time, Time-Homogeneous Dynamics

To demonstrate the effect of the finite-time restriction on the transition behavior between A and B, we now study the time-homogeneous triple well dynamics restricted to the time window ${\mathbb {T}}= \{0,\dots , N-1\}$, $N=6$, and initiated in the stationary density.

Even though we study the same underlying dynamics as in the stationary, infinite-time case (Example 6), the possible transition paths between A and B are limited to the pathway that is fast to traverse, i.e., the lower channel via the direct barrier, see Fig. 14. Since only a small portion of the reactive trajectories from the infinite-time example 6 has a short enough transition time to be considered in this case, the average rate of transitions ${\bar{k}}^{AB}_6= 0.0017$ is much lower than the corresponding rate $k^{AB}= 0.0142$ in the infinite-time case (Example 6), and the average time a transition takes ${\bar{t}}^{AB}_6= 2.055$ is much shorter than in the infinite-time case.

6.4 Numerical Example 9: Bifurcation Studies Using Finite-Time TPT

Fig. 15

Qualitative changes in the transition behavior for finite-time, stationary TPT and increasing time-interval lengths, see Example 9. The plots show a the density of reactive trajectories (normalized to being reactive) and b the current at a time point in the middle of the finite-time interval

Last, we want to highlight the usage of finite-time TPT to study large qualitative changes in the transition behavior of a system. We consider the stationary triple well example over a finite interval ${\mathbb {T}}=\{0,\dots ,N-1\}$, but this time with a smaller noise strength of $\sigma =0.26$ compared to the previous examples.

As we increase the interval length N from $N=20$ to $N=500$, we allow reactive trajectories to be longer and longer, and thus, the average reactive trajectory changes. Whereas for $N=20$, most of the density and current are around the lower transition channel, see Fig. 15, for $N=500$, most of the density and current are around the upper transition channel. The transition behavior restricted to the time interval of size $N=500$ is already close to the infinite-time transition dynamics, this can be seen by comparing the case of $N=500$ with Fig. 1b, where the effective current of the same dynamics but in infinite time is depicted.

7 Conclusion

In this paper, we generalized transition path theory such that it is applicable not only to infinite-time, stationary Markov processes but also to periodically varying and to time-dependent Markov processes on finite-time intervals. We restricted our results to Markov processes on discrete state spaces and in discrete time, but generalizations should be straightforward (e.g., following Weinan and Vanden-Eijnden 2006; Metzner et al. 2009).

The theory is intended to generalize TPT toward applicability in, e.g., non-equilibrium molecular, climate, fluid, or social dynamics (agent-based models). In most of these applications, the problem of computing the TPT objects arises, as the state space can be high- and also infinite dimensional. This is a non-trivial task even in the traditional TPT context stemming from molecular dynamics—where these tools have nevertheless been successfully applied. Resolving the time-dependence poses an additional computational challenge. All this goes beyond the scope of the present work and will be addressed elsewhere.

First results toward the application of stationary TPT in high-dimensional state spaces have already been proposed. In Thiede et al. (2019), a workaround was given for solving the committor equations in the case of infinite-time, stationary dynamics by a Galerkin projection which works well as long as the dynamics of the Markov chain can be described by low-dimensional representations. Another interesting first work (Khoo et al. 2019) goes into the direction of using neural networks for solving the committor equations in high dimensions.

Further, the interpretability and visualization of the transition statistics in high dimensions or for large time intervals (large N or M) is a point of future research. In Metzner et al. (2009), an algorithm for computing transition channels from the effective current of reactive trajectories was proposed, and a generalization for time-dependent dynamics is a ongoing work.

An implementation of the tools developed in this paper can be found at www.github.com/LuzieH/pytpt.

Appendix

1.1 Committors for Periodic, Infinite-Time Dynamics: Augmenting the State Space

As an alternative to the approach in Sect. 4.2, we can consider the dynamics and the committor equations on the augmented state space ${\mathbb {S}}^M$. The augmented transition matrix $P_\text {Aug}$ of size $M|{\mathbb {S}}| \times M|{\mathbb {S}}|$ contains all transition matrices $P_0,\dots , P_{M-1}$ and applies them consecutively in a row

$$\begin{aligned} P_\text {Aug} = \begin{pmatrix} 0 &{} P_0 &{} &{} 0 \\ &{} \ddots &{} \ddots &{} \\ &{} &{} 0&{}P_{M-2} \\ P_{M-1} &{} &{} &{} 0, \end{pmatrix}, \end{aligned}$$

e.g. applying $P_\text {Aug}$ to a space-time distribution with mass only at the $0-$th time point shifts all the mass to the next time point. Since the transition matrices are row-stochastic, so is the augmented transition matrix.

With that set, we can take a slightly different approach for writing the forward committor equations as one augmented matrix equation. Again, we need to solve the committor only on C; therefore, we define the augmented $|C|M \times |C|M$ matrix ${\tilde{P}}_\text {Aug}$, which encodes only the transitions inside C

$$\begin{aligned} {\tilde{P}}_\text {Aug} := \begin{pmatrix} 0 &{} \left. P_0\right| _{C\rightarrow C}&{} &{} \\ &{} \ddots &{} \ddots &{} \\ &{} &{} 0 &{} \left. P_{M-2}\right| _{C\rightarrow C} \\ \left. P_{M-1}\right| _{C\rightarrow C} &{} &{} &{} 0 \end{pmatrix} \end{aligned}$$

and the |C|M-dimensional vectors

$$\begin{aligned} {\tilde{q}}^+ := \begin{pmatrix} (q^+_{0,i})_{i\in C}\\ (q^+_{1,i})_{i\in C} \\ \vdots \\ (q^+_{M-1,i})_{i\in C} \end{pmatrix} \text { and } {\tilde{b}} := \begin{pmatrix} (\sum _{j\in B} P_{0,ij})_{i\in C}\\ (\sum _{j\in B} P_{1,ij})_{i\in C} \\ \vdots \\ (\sum _{j\in B} P_{M-1,ij})_{i\in C} \end{pmatrix}. \end{aligned}$$

Then from (16), we arrive at the following equation for ${\tilde{q}}^+$

$$\begin{aligned} {\tilde{q}}^+ = {\tilde{P}}_\text {Aug} \, {\tilde{q}}^++ {\tilde{b}} \Leftrightarrow (I - {\tilde{P}}_\text {Aug}) \, {\tilde{q}}^+= {\tilde{b}}. \end{aligned}$$

(44)

Note that $(I - {\tilde{P}}_\text {Aug})$ is large but sparse compared to using the stacked equations (18).

For the backward committor equations, we proceed similarly and arrive at the equations

$$\begin{aligned} {\tilde{q}}^- = {\tilde{P}}^-_\text {Aug} {\tilde{q}}^- + {\tilde{a}} \Leftrightarrow (I - {\tilde{P}}^-_\text {Aug}) {\tilde{q}}^-= {\tilde{a}} \end{aligned}$$

(45)

where we defined

$$\begin{aligned}&{\tilde{P}}^-_\text {Aug} := \begin{pmatrix} 0 &{}&{} &{} \left. P^-_0\right| _{C\rightarrow C} \\ \left. P^-_{1}\right| _{C\rightarrow C} &{} \ddots &{} &{} \\ &{} \ddots &{} 0 &{} \\ &{} &{} \left. P^-_{M-1}\right| _{C\rightarrow C} &{} 0 \end{pmatrix}, \ {\tilde{q}}^- := \begin{pmatrix} (q^-_{0,i})_{i\in C}\\ (q^-_{1,i})_{i\in C} \\ \vdots \\ (q^-_{M-1,i})_{i\in C} \end{pmatrix} \text { and }\\&{\tilde{a}} := \begin{pmatrix} (\sum _{j\in A} P^-_{0,ij})_{i\in C}\\ (\sum _{j\in A} P^-_{1,ij})_{i\in C} \\ \vdots \\ (\sum _{j\in A} P^-_{M-1,ij})_{i\in C} \end{pmatrix}. \end{aligned}$$

Remark A.1

This approach can also be used for studying committors and transitions in systems with stochastic switching between different regimes. Consider the dynamics of a Markov chain that can be in M different regimes each described by a transition matrix $P_m$, $m\in \{1,\dots ,M\}$. The probability to switch between regimes is given by ${\hat{P}}\in {\mathbb {R}}^{M \times M}$. The committor probabilities would give the probability to next hit B and not A, given the chain is in a certain state and regime and assuming the dynamics are equilibrated. Note that the periodic dynamics described in this paper are just a special case of stochastic switching, where the switching is deterministic from time m to time $m+1$, i.e., ${\hat{P}}_{mm'} = \delta _{mm'-1}$ modulo M. The regime-augmented matrix can be written as:

$$\begin{aligned} P^{\text {switch}}_\text {Aug} = \begin{pmatrix} {\hat{P}}_{11} P_1 &{} \dots &{} {\hat{P}}_{1M} P_M \\ \vdots &{} \ddots &{} \vdots \\ {\hat{P}}_{M1} P_1&{} \dots &{} {\hat{P}}_{MM} P_M \end{pmatrix} \end{aligned}$$

which is still a row-stochastic matrix, since $\sum _{m'=1}^M {\hat{P}}_{mm'} =1$ for all $m=1,\dots ,M$. Exactly as above, $P^{\text {switch}}_\text {Aug}$ can be used for finding committor probabilities.

1.2 Proofs

1.2.1 Lemma 3.5

Proof

Let us consider the joint probability that the process starting in $i \in {\mathbb {S}}$ at time $n \in {\mathbb {Z}}$ reaches first B before A at time $\tau \in {\mathbb {Z}}_n^+ \cup \{ \infty \}$:

$$\begin{aligned} J_i^+(n, \tau ) := {\mathbb {P}}(\tau _B^+(n) < \tau _A^+(n), \tau _{A\cup B}^+(n) = \tau | X_n = i) . \end{aligned}$$

The law of total probability lets us write the forward committor in terms of a countable sum of the above-mentioned joint probability:

$$\begin{aligned} q_i^+(n) = \sum \limits _{\tau \in {\mathbb {Z}}_n^+ \cup \{\infty \}} J_i^+(n, \tau ) = \sum \limits _{\tau \in {\mathbb {Z}}_n^+} J_i^+(n, \tau ), \end{aligned}$$

(46)

where we have used that ${\mathbb {P}}(\tau _{A \cup B}^+(n) = \infty ) = 0$ due to the ergodicity of the process, which ensures that the process will arrive at some time to the subset $A \cup B$.

Next, by applying the same arguments of the proof of Theorem 3.2 we see that the joint probability $J_i^+(n, \tau )$ satisfies the following iterative system of equations for $\tau \in {\mathbb {Z}}_{n+1}^+$:

$$\begin{aligned} \left\{ \begin{array}{rcll} J_i^+(n, \tau ) &{}=&{} \sum \limits _{j \in {\mathbb {S}}} \, P_{ij} \, J_j^+(n+1, \tau ) &{} i \in C \\ J_i^+(n, \tau ) &{}=&{} 0 &{} i \in A \cup B \end{array}\right. \end{aligned}$$

(47)

with initial condition $J_i^+(n, n) = \mathbb {1}_B(i)$.

Last, by using (47) recursively we can compute for any $i \in C$ and $\tau \in {\mathbb {Z}}_n^+$

$$\begin{aligned} J_i^+(n, \tau )= & {} \sum \limits _{i_{n+1}, \dots , i_{\tau -1} \in C} \, P_{ii_{n+1}} \cdots P_{i_{\tau -2}i_{\tau -1}} \, J_i^+(\tau -1, \tau ) \nonumber \\= & {} \sum _{\begin{array}{c} i_{n+1} \dots i_{\tau -1} \in C \\ i_\tau \in B \end{array}} \, P_{ii_{n +1}} \cdots P_{i_{\tau -1} i_{\tau }}, \end{aligned}$$

(48)

where we recursively used $J_i^+(k, \tau ) = 0$ for any $i \in A \cup B$ and $n+1\le k<\tau$, and in the last iteration the fact that $J_i^+(\tau , \tau ) = \mathbb {1}_B(i)$.

Putting the results (46) and (48) together completes the proof for the forward committor. The proof for the backward committor follows by using the same arguments. $\square$

1.2.2 Theorem 3.6

Proof

First, for any $i\in C$, we have

$$\begin{aligned} \begin{aligned} \sum _{j\in {\mathbb {S}}} (f^{AB}_{ij}-f^{AB}_{ji})&= q_i^- \pi _i \sum _{j\in {\mathbb {S}}} P_{ij} q^+_j - q^+_i \pi _i \sum _{j\in {\mathbb {S}}} q_j^-P^-_{ij}\\&= q_i^- \pi _i q^+_i - q^+_i \pi _i q_i^- =0 \end{aligned} \end{aligned}$$

(49)

using the definition of the time-reversed transition probabilities and the committor equations (4), (5) for $i\in C$.

Second, using that $f^{AB}_{ij}=0$ if $i\in B, j\in {\mathbb {S}}$ and also if $i\in {\mathbb {S}}, j\in A$, we can compute

$$\begin{aligned} \begin{aligned} \sum _{\begin{array}{c} i\in {\mathbb {S}}\\ j \in {\mathbb {S}} \end{array}} f^{AB}_{ij} = \sum _{\begin{array}{c} i\in A \\ j \in {\mathbb {S}} \end{array}} f^{AB}_{ij} +\sum _{\begin{array}{c} i\in C \\ j \in {\mathbb {S}} \end{array}} f^{AB}_{ij} +\sum _{\begin{array}{c} i\in B\\ j \in {\mathbb {S}} \end{array}} \underbrace{f^{AB}_{ij}}_{ = 0} = \sum _{\begin{array}{c} i\in {\mathbb {S}}\\ j \in A \end{array}} \underbrace{f^{AB}_{ij}}_{ = 0} + \sum _{\begin{array}{c} i\in {\mathbb {S}}\\ j \in C \end{array}} f^{AB}_{ij} + \sum _{\begin{array}{c} i\in {\mathbb {S}}\\ j \in B \end{array}} f^{AB}_{ij} \end{aligned} \end{aligned}$$

and by the current conservation (49) for $i\in C$, $\sum _{\begin{array}{c} i\in C \\ j \in {\mathbb {S}} \end{array}} f^{AB}_{ij} = \sum _{\begin{array}{c} i\in C \\ j \in {\mathbb {S}} \end{array}} f^{AB}_{ji}=\sum _{\begin{array}{c} j\in C \\ i \in {\mathbb {S}} \end{array}} f^{AB}_{ij}$, we arrive at

$$\begin{aligned} \begin{aligned} \sum _{\begin{array}{c} i\in A \\ j \in {\mathbb {S}} \end{array}} f^{AB}_{ij} +\sum _{\begin{array}{c} i \in {\mathbb {S}}\\ j\in C \end{array}} f^{AB}_{ij} = \sum _{\begin{array}{c} i\in {\mathbb {S}}\\ j \in C \end{array}} f^{AB}_{ij} +\sum _{\begin{array}{c} i\in {\mathbb {S}}\\ j \in B \end{array}} f^{AB}_{ij} \end{aligned} \end{aligned}$$

implying that $\sum _{\begin{array}{c} i\in A \\ j \in {\mathbb {S}} \end{array}} f^{AB}_{ij} = \sum _{\begin{array}{c} i\in {\mathbb {S}}\\ j \in B \end{array}} f^{AB}_{ij}.$ $\square$

1.2.3 Theorem 4.4

Proof

For $i\in C$, the forward committor at time $n=m$ modulo M reads

$$\begin{aligned} q^+_{i}(n)= & {} {\mathbb {P}}(\tau ^+_B(n)<\tau ^+_A(n)| X_n=i) \overset{(1)}{=} {\mathbb {P}}(\tau ^+_B(n+1)<\tau ^+_A(n+1)| X_n=i) \nonumber \\&\overset{(2)}{=} \sum _{j\in {\mathbb {S}}} {\mathbb {P}}(\tau ^+_B(n+1)<\tau ^+_A(n+1), X_{n+1}=j| X_n=i) \nonumber \\&\overset{(3)}{=} \sum _{j\in {\mathbb {S}}} {\mathbb {P}}(\tau ^+_B(n+1)<\tau ^+_A(n+1) |X_{n+1}=j, X_n=i) {\mathbb {P}}( X_{n+1}=j| X_n=i) \nonumber \\&\overset{(4)}{=} \sum _{j\in {\mathbb {S}}} {\mathbb {P}}(\tau ^+_B(n+1)<\tau ^+_A(n+1) |X_{n+1}=j) P_{n,ji} = \sum _{j\in {\mathbb {S}}} q^+_{j}(n+1) P_{n,ij} \end{aligned}$$

(50)

using (1) that $\tau _B^+(n), \tau _A^+(n)\ge n+1$ for $i\in C$, (2) the law of total probability, (3) the definition of conditional probabilities, (4) the strong Markov property.

And for $i\in A$ at time n, it follows from the definition that $\tau ^+_A(n)=n$, whereas $\tau ^+_B(n)>n$, thus $q^+_{i}(n)=0$, in a similar way for $i\in B$, $\tau ^+_A(n)>n$, $\tau ^+_B(n)=n$ thus $q^+_{ i}(n)=1$.

The proof for the backward committor equations follows the same lines. $\square$

1.2.4 Lemma 4.6

Proof

We start with the case of the forward committor (16).

We can rewrite (18) with $m=0$, with $\left. P_0\right| _{I\rightarrow J}$ denoting the restriction of the matrix $P_0$ to entries from $i\in I$ to $j\in J$, as the matrix equation

$$\begin{aligned} \begin{aligned} (q^+_{0,i_0})_{i_0\in C}&=\underbrace{\left. P_0\right| _{C\rightarrow C} \cdots \left. P_{M-1}\right| _{C\rightarrow C} }_{=: D} (q^+_{0,i_M})_{i_M\in C} + \sum _{\tau =1}^{M} \left. P_0\right| _{C \rightarrow C} \cdots \left. P_{\tau -1}\right| _{C \rightarrow B} (1)_{B} \end{aligned} \end{aligned}$$

equivalently

$$\begin{aligned} \begin{aligned} (I-D)(q^+_{0,i_0})_{i_0\in C}&= \sum _{\tau =1}^{M} \left. P_0\right| _{C \rightarrow C} \cdots \left. P_{\tau -1}\right| _{C \rightarrow B} (1)_{B} \end{aligned} \end{aligned}$$

We note that the equation is uniquely solvable as long as $(I-D)$ is invertible, and $(I-D)$ is invertible if $\rho (D) <1$.

By assuming that ${\bar{P}}_0$ is irreducible, we will show that for all $v \in {\mathbb {R}}^{ |C|}$, $\Vert vD\Vert _1 < \Vert v \Vert _1$. Since this holds in particular for the eigenvectors, it follows that all eigenvalues $|\lambda |<1$ and thus $\rho (D) <1$.

We know that $D$ is a substochastic matrix (row sum is $\le 1$) since it is a product of substochastic matrices, and that all entries are nonnegative, i.e., $\sum _j |D_{ij}| \le 1$ for all $i\in C$.

Moreover, there exists at least one row with row sum less than 1 since by irreducibility of ${\bar{P}}_0$, there must be at least one state i in C with a positive probability to go to A or B. We call the first one of this kind by $i^*$ with $\sum _{j} |D_{i^*j}| < 1$. Thus, we can compute

$$\begin{aligned} \begin{aligned} \Vert v D\Vert _1&= \Vert \sum _i v_i D_{ij} \Vert _1 = \sum _j \sum _i |v_i| |D_{ij}| = \sum _i |v_i| \left( \sum _j |D_{ij}| \right) \\&= \left( \sum _{i\ne i^*} |v_i| \sum _{j} |D_{ij}| \right) + |v_{i^*}| \sum _{j} |D_{i^*j}| \\&\le \sum _{i\ne i^*} |v_i| + |v_{i^*}| \sum _{j} |D_{i^*j}| < \sum _{i } |v_i| = \Vert v \Vert _1. \end{aligned} \end{aligned}$$

We have shown that a unique solution which we will call $q^+_0$ exists. The forward committors for $m=1,\dots , M-1$ can uniquely be computed thereof by using (16).

For the case of the backward committor, we can proceed analogously by using the time-reversed transition probabilities mapping M instances back

$$\begin{aligned} \bar{P^-_m}:= P^-_m \cdot P^-_{m-1} \cdots P^-_{m-M+1}, \end{aligned}$$

and by noting that if ${\bar{P}}_0$ is irreducible, also $\bar{P^-_0}$ is irreducible, which follows from the definition of irreducibility of ${\bar{P}}_0$ and using (15). $\square$

1.2.5 Theorem 4.8

Proof

To show that the flux conservation in node $i\in C$ holds, we compute

$$\begin{aligned} \sum _{j\in {\mathbb {S}}} \left( f_{m,ij}^{AB}-f_{m-1,ji}^{AB}\right) \overset{(1)}{=} \pi _{m,i} q^-_{m,i} \sum _{j\in {\mathbb {S}}} P_{m,ij} q^+_{m+1,j} - \pi _{m,i} q^+_{m,i} \sum _{j\in {\mathbb {S}}} q^-_{m-1,j} P^-_{m,ij} \overset{(2)}{=} 0 \end{aligned}$$

using (1) $P^-_{m,ij}\pi _{m,i} = P_{m-1,ji}\pi _{m-1,j}$ and (2) the backward and forward committor equations for $i\in C$.

Next, we want to show that the current of reactive trajectories leaving A during one period equals the current entering B during one period. We calculate

$$\begin{aligned} \begin{aligned} \sum _{m\in {\mathbb {M}}} \left( \sum _{\begin{array}{c} i\in {\mathbb {S}}\\ j \in {\mathbb {S}} \end{array}} f^{AB}_{m,ij} \right)= & {} \sum _{m\in {\mathbb {M}}} \left( \sum _{\begin{array}{c} i\in A \\ j \in {\mathbb {S}} \end{array}} f^{AB}_{m,ij} +\sum _{\begin{array}{c} i\in C \\ j \in {\mathbb {S}} \end{array}} f^{AB}_{m,ij} +\sum _{\begin{array}{c} i\in B\\ j \in {\mathbb {S}} \end{array}} \underbrace{f^{AB}_{m,ij}}_{ = 0} \right) \\= & {} \sum _{m\in {\mathbb {M}}} \left( \sum _{\begin{array}{c} i\in {\mathbb {S}}\\ j \in A \end{array}} \underbrace{f^{AB}_{m,ij}}_{ = 0} + \sum _{\begin{array}{c} i\in {\mathbb {S}}\\ j \in C \end{array}} f^{AB}_{m,ij} + \sum _{\begin{array}{c} i\in {\mathbb {S}}\\ j \in B \end{array}} f^{AB}_{m,ij} \right) \end{aligned} \end{aligned}$$

using that $f^{AB}_{m,ij}=0$ if $i\in B, j\in {\mathbb {S}}$ and if $i\in {\mathbb {S}}, j\in A$. And by the current conservation for $i\in C$, $m\in {\mathbb {M}}$ and by relabeling i, j, m,

$$\begin{aligned} \sum _{m\in {\mathbb {M}}} \sum _{\begin{array}{c} i\in C \\ j \in {\mathbb {S}} \end{array}} f^{AB}_{m,ij} = \sum _{m\in {\mathbb {M}}} \sum _{\begin{array}{c} i\in C \\ j \in {\mathbb {S}} \end{array}} f^{AB}_{m-1, ji}=\sum _{m\in {\mathbb {M}}} \sum _{\begin{array}{c} j\in C \\ i \in {\mathbb {S}} \end{array}} f^{AB}_{m,ij} \end{aligned}$$

we arrive at

$$\begin{aligned} \begin{aligned} \sum _{m\in {\mathbb {M}}} \left( \sum _{\begin{array}{c} i\in A \\ j \in {\mathbb {S}} \end{array}} f^{AB}_{m,ij} +\sum _{\begin{array}{c} i \in {\mathbb {S}}\\ j\in C \end{array}} f^{AB}_{m,ij} \right) = \sum _{m\in {\mathbb {M}}} \left( \sum _{\begin{array}{c} i\in {\mathbb {S}}\\ j \in C \end{array}} f^{AB}_{m,ij} +\sum _{\begin{array}{c} i\in {\mathbb {S}}\\ j \in B \end{array}} f^{AB}_{m,ij} \right) \end{aligned} \end{aligned}$$

implying that $\sum _{m\in {\mathbb {M}}} \sum _{\begin{array}{c} i\in A \\ j \in {\mathbb {S}} \end{array}} f^{AB}_{m,ij} = \sum _{m\in {\mathbb {M}}} \sum _{\begin{array}{c} i\in {\mathbb {S}}\\ j \in B \end{array}} f^{AB}_{m,ij}.$ $\square$

1.2.6 Theorem 5.2

Proof

First, we recall what we have already seen in the proof of Theorem 4.4. If $i \in A$ for any $n \in \{0, \dots , N-1\}$ we have that $q_i^+(n) = 0$, $q_i^-(n) = 1$ and if $i \in B$ for any $n \in \{0, \dots , N-1\}$ we have that $q_i^+(n) = 1$, $q_i^-(n) = 0$.

Second, we find a final condition for the forward committor on $i \in C$

$$\begin{aligned} q_i^+(N-1) = {\mathbb {P}}(\tau _B^+(N) < \tau _A^+(N) | X_N = i) = {\mathbb {P}}(X_N \in B | X_N = i) \overset{(1)}{=} 0 \end{aligned}$$

and an initial condition for the backward committor on $i \in C$

$$\begin{aligned} q_i^-(0) = {\mathbb {P}}(\tau _A^-(0) > \tau _B^-(0) | X_0 = i) = {\mathbb {P}}(X_0 \in A | X_0 = i) \overset{(2)}{=} 0 , \end{aligned}$$

where we have used in (1) and (2) that $i \in C$.

Third, by following the same arguments used to prove (50) (in the proof of Theorem 4.4) we get for any $n \in \{0, \dots , N-2\}$ that

$$\begin{aligned} q_i^+(n) = \sum \limits _{j \in {\mathbb {S}}} \, P_{ij}(n) \, q_j^+(n+1) . \end{aligned}$$

Analogously, for any $n \in \{1, \dots , N-1\}$ we get that

$$\begin{aligned} q_i^-(n) = \sum \limits _{j \in {\mathbb {S}}} \, P^-_{ij}(n) \, q_j^-(n-1). \end{aligned}$$

$\square$

1.2.7 Theorem 5.5

Proof

First, for any $i\in C$ and $n \in \{1, \dots , N-2\}$ we have on one hand that

$$\begin{aligned} \begin{aligned} \sum _{j\in {\mathbb {S}}} f^{AB}_{ij}(n)&= q_i^-(n) \lambda _{n}(i) \Big ( \sum _{j\in {\mathbb {S}}} P_{ij}(n) q^+_j(n+1) \Big ) \\&\overset{(1)}{=} q_i^-(n) \lambda _{n}(i) q^+_i(n) \end{aligned} \end{aligned}$$

(51)

and on the other hand that

$$\begin{aligned} \begin{aligned} \sum _{j\in {\mathbb {S}}} f^{AB}_{ji}(n-1)&= \sum _{j\in {\mathbb {S}}} q_j^-(n-1) \lambda _{n-1}(j) P_{ji}(n-1) q^+_i(n) \\&\overset{(2)}{=} q^+_i(n) \lambda _{n}(i) \Big ( \sum _{j\in {\mathbb {S}}} P^-_{ij}(n) q_j^-(n-1) \Big ) \\&\overset{(3)}{=} q^+_i(n) \lambda _{n}(i) q_i^-(n) , \end{aligned} \end{aligned}$$

(52)

where (1) and (3) follow by (24) and (25) and (2) follows by (23).

Second, by using that $f^{AB}_{ij}(n)=0$ for any $n \in \{0, \dots , N-2\}$ if $i\in B, j\in {\mathbb {S}}$ and if $i\in {\mathbb {S}}, j\in A$ we arrive at the following equality

$$\begin{aligned} \begin{aligned} \sum \limits _{n=0}^{N-2} \left( \sum _{\begin{array}{c} i\in {\mathbb {S}}\\ j \in {\mathbb {S}} \end{array}} f^{AB}_{ij}(n) \right) = \sum \limits _{n=0}^{N-2} \left( \sum _{\begin{array}{c} i\in A \\ j \in {\mathbb {S}} \end{array}} f^{AB}_{ij}(n) +\sum _{\begin{array}{c} i\in C \\ j \in {\mathbb {S}} \end{array}} f^{AB}_{ij}(n) \right) \\ = \sum \limits _{n=0}^{N-2} \left( \sum _{\begin{array}{c} i\in {\mathbb {S}}\\ j \in C \end{array}} f^{AB}_{ij}(n) + \sum _{\begin{array}{c} i\in {\mathbb {S}}\\ j \in B \end{array}} f^{AB}_{ij}(n) \right) . \end{aligned} \end{aligned}$$

Then, we show that

$$\begin{aligned} \begin{aligned} \sum \limits _{n=0}^{N-2} \sum _{\begin{array}{c} i\in C \\ j \in {\mathbb {S}} \end{array}} f^{AB}_{ij}(n)&= \sum _{\begin{array}{c} i\in C \\ j \in {\mathbb {S}} \end{array}} f^{AB}_{ij}(0) + \sum \limits _{n=1}^{N-2} \sum _{\begin{array}{c} i\in C \\ j \in {\mathbb {S}} \end{array}} f^{AB}_{ij}(n) \overset{(4)}{=} \sum \limits _{n=1}^{N-2} \sum _{\begin{array}{c} i\in C \\ j \in {\mathbb {S}} \end{array}} f^{AB}_{ji}(n-1) \\&\overset{(5)}{=} \sum \limits _{n=0}^{N-3} \sum _{\begin{array}{c} j\in C \\ i \in {\mathbb {S}} \end{array}} f^{AB}_{ij}(n) + \sum _{\begin{array}{c} j\in C \\ i \in {\mathbb {S}} \end{array}} f^{AB}_{ij}(N-2) = \sum \limits _{n=0}^{N-2} \sum _{\begin{array}{c} j\in C \\ i \in {\mathbb {S}} \end{array}} f^{AB}_{ij}(n) , \end{aligned} \end{aligned}$$

(53)

where in (4) we have applied the time-dependent current conservation for $i \in C$, ${n \in \{1, \dots , N-2\}}$ and we have used that ${f^{AB}_{ij}(0)=0}$, and in (5) we have relabeled i, j and used that ${f^{AB}_{ij}(N-2)=0}$. As a consequence,

$$\begin{aligned} \sum \limits _{n=0}^{N-2} \sum _{\begin{array}{c} i\in A \\ j \in {\mathbb {S}} \end{array}} f^{AB}_{ij}(n) = \sum \limits _{n=0}^{N-2} \sum _{\begin{array}{c} i\in {\mathbb {S}}\\ j \in B \end{array}} f^{AB}_{ij}(n) . \end{aligned}$$

$\square$