Extending Transition Path Theory: Periodically-Driven and Finite-Time Dynamics

Given two distinct subsets $A,B$ in the state space of some dynamical system, Transition Path Theory (TPT) was successfully used to describe the statistical behavior of transitions from $A$ to $B$ in the ergodic limit of the stationary system. We derive generalizations of TPT that remove the requirements of stationarity and of the ergodic limit, and provide this powerful tool for the analysis of other dynamical scenarios: periodically forced dynamics and time-dependent finite-time systems. This is partially motivated by studying applications such as climate, ocean, and social dynamics. On simple model examples we show how the new tools are able to deliver quantitative understanding about the statistical behavior of such systems. We also point out explicit cases where the more general dynamical regimes show different behaviors to their stationary counterparts, linking these tools directly to bifurcations in non-deterministic systems.

If the system dynamics can be modelled by a stationary Markov process running for infinite time, Transition Path Theory (TPT) provides a rigorous approach for studying the transitions from one subset A to another subset B of the state space. The main tool of Transition Path Theory [WVE06, MSVE09] are the forward and backward committor probabilities telling us the probability of the Markov process to next commit to (i.e., hit) B relative to A, either forward or backward in time. Given these committor probabilities, one can derive important statistics of the ensemble of reactive trajectories (i.e., of the collection of all possible paths of the Markov process that start in A and end in B), such as -the density of reactive trajectories telling us about the bottlenecks during transitions, -the current of reactive trajectories indicating the most likely transition channels, -the rate of reactive trajectories leaving A or entering B, and -the mean duration of reactive trajectories.
Other approaches that characterize the ensemble of transition paths, on the one hand, place the focus elsewhere: For instance, in Transition Path Sampling [BCDG02] one is interested in directly sampling trajectories of the reactive ensemble. On the other hand, these approaches consider different objects, such as the steepest descent path [UE90,CE90,OE97], the most probable path [OE96,ES00,PS10,FLO10,aBvCF12] (also in temporal networks [SGVHGL15]), or the first passage path ensemble [vKSZ18] (see also Remark 5.6).
For many physical, especially molecular, systems that are equilibrated and where transitions happen on a smaller time scale than the observation window, the assumption of a stationary, infinite-time Markov process is reasonable and common practice [NSVE + 09]. For illustration we consider the overdamped Langevin dynamics dX t = −∇V (X t )dt + σdW t in the triple well landscape V (x, y) (as in Figure 1), and we are interested in the transitions between the deep well A and the other deep well B. If the noise intensity σ is sufficiently small, then the system tends to spend long times near local minima of deep wells (this behavior is called metastability) and transitions predominantly happen across regions of possibly low values of the potential V (e.g., saddle points). If the system from Figure 1 is stationary and has infinite time for transitioning, the transition channel via the metastable well centered at (0, 1.5) is preferred since the barriers are lower and it does not matter that transitions take a very long time due to being stuck in the metastable set. The effective current of reactive trajectories restricted to a small time window; the transition channel via the direct barrier between A and B is more likely. More details on the numerical example can be found in Section 6, but here we chose a smaller noise intensity σ = 0.26.
However, in order to study transitions and tipping paths in different dynamical contexts (e.g., social systems or climate models) which are often characterized by time-dependent (e.g., seasonal) dynamics as well as transitions of interest within a finite time window, the current theory of transition paths has to be extended.
In these applications one might, for instance, ask: What are the possible transition channels from the current state to a desirable and sustainable state of our social or climate system within the next 30 years [SRR + 18, ODC + 20]? By requiring the transitions to depart from A and arrive in B within a finite time interval, the affinity of the system taking the different transition channels is altered. This is visualized in the triple well dynamics, Figure 1(c), where now only the lower transition channel passing the high barrier is possible. Whenever the trajectory takes the channel through the upper metastable set (cf. white trajectory in Subfigure (a)), it is stuck there for a long time, and will not reach B anymore within the finite time horizon.
Moreover, systems containing human agents are usually time-inhomogeneous and not equilibrated while climate systems are often affected by seasonal forcing, raising questions such as: What are the likely spreading paths of a contagion in a time-evolving network [PS11,BH13,VFPC18]? What are bottlenecks in the transient dynamics towards the equilibrium state [Sch92,HOS00]? How does tipping occur under the joint effect of noise and parameter changes [AWVC12,GLMW19] or periodic forcing [HI + 05]?
In this paper we generalize TPT to a broader class of dynamical scenarios. In particular, we focus on two generalizations which we consider as natural but not exclusive building blocks for these more general cases: (a) periodically forced infinite-time system and (b) arbitrary timeinhomogeneous finite-time system.
We start in Section 2 by formulating the general setting of TPT for Markov chains (X n ) n∈T on a finite state space 1 by introducing time-dependent forward committor functions q + i (n), giving the probability within the time horizon T to next commit to B and not A conditional on being in X n = i, as well as time-dependent backward committor functions q − i (n), both of which are needed for computing the desired statistics of the transitions from A to B.
We then in more detail work out the following main cases.
(i) Under the assumption of stationary, infinite-time dynamics, it is known [WVE06, MSVE09] that the committor functions are time-independent q + (n) = q + by stationarity and solve the following linear system (1) where P = (P ij ) i,j∈S is the transition matrix. Similarly all the transition statistics are timeindependent and by ergodicity the statistics can also be found by averaging along one infinitely long equilibrium trajectory [WVE06, MSVE09]. In Section 3 we recall the theory [WVE06, MSVE09] from a different point of view. Instead of defining all quantities in terms of trajectory-wise time-averages, we prove by using the Markov property that they can be written in terms of the committors and the stationary distribution. Further, we extend the theory by Lemma 3.5 decomposing the committors into path probabilities.
(ii) In Section 4 we derive the committors and transition statistics for periodically varying dynamics with a period of length M that are equilibrated (i.e., the law of the chain is also periodic). It follows that the committors are periodically varying q + (n) = q + m whenever n = m modulo M , and we show that the committors solve the following linear system with periodic boundary conditions in time where P m = (P m,ij ) i,j∈S is the transition matrix at time n = m modulo M . This is consistent with the previous case when choosing a period of length M = 1.
(iii) In Section 5 we derive the committor equations and transition statistics for general timeinhomogeneous Markov chains (X n ) n∈T on a finite-time interval T = {0, . . . , N − 1} defined by the transition probabilities P (n) = (P ij (n)) i,j∈S and an initial density λ(0). The forward committor q + (n) for a finite-time Markov chain satisfies the following iterative system of equations: with final condition q + i (N − 1) = 1 B (i). The transition statistics depend on the current time point in the time interval and can be related to an average over the ensemble of reactive trajectories. We also show consistency, i.e., that given a stationary process on a finite time interval, the committors and statistics converge to their classical counterparts (i) in the infinite time limit.
We note that in the stationary regime, committor functions have been used for finding basins of attraction of stochastic dynamics [Kol11b,KV14,LH19], as reaction coordinates [LVE14], as basis functions in the core-set approach [SNL + 11, Sar11, SS13] and for studying modules and flows in networks [DBCS11,CVE14]. We expect our results to enable similar uses for the time-dependent regime.
The theoretical results from this paper are accompanied by numerical studies on two toy examples, a network of 5 nodes with time-varying transition probabilities, and a discrete model of the overdamped Langevin dynamics in a triple well potential with time-dependent forcing (as in Figure 1). In these examples we will particularly show to what extent time-dependent dynamics or finite-time restrictions affect the transition statistics in contrast to those in stationary, infinite time dynamics. The TPT-related objects derived here allow for a quantitative assessment of the dominant statistical behavior in complicated dynamical regimes: (i) By adding a periodic forcing to the stationary dynamics, the reaction channels are perturbed and new transition paths, that were not possible before, can appear (see Example 2 and 7).
(ii) By restricting the stationary dynamics with matrix P to a finite-time window (cf. Example 3 and 8), only transitions within this window are allowed and the average rate of transitions is much lower than in the infinite-time situation. By additionally applying a forcing (cf. Example 4), the system is not equilibrated anymore and we can get a higher average rate of transitions than without forcing, although we set the time-dependent transition matrix P (n) such that its time-average equals the transition matrix P of the stationary case. A similar approach is used in rare events simulation where the system is pushed by an optimal non-equilibrium forcing under which the rare events become more likely [HS12, HBS + 14].
(iii) The finite-time case can also be employed for studying qualitative changes in the transition dynamics when parameters are perturbed. In Example 9, we exemplarily show that by increasing the finite-time interval length N , the transition dynamics change qualitatively, even though the dynamics are stationary. Thus, TPT can be used as a quantitative tool describing bifurcations in non-stationary non-deterministic systems.
The code used for the examples is available on Github at www.github.com/LuzieH/transitions.

Preliminaries and General Setup
The objective of Transition Path Theory (TPT) is to understand the mechanisms by which noiseinduced transitions of a Markov chain (X n ) n∈T from one subset of the state space A ⊂ S to another subset B ⊂ S take place 2 . A and B are chosen as two non-empty, disjoint subsets of the finite state space S, such that the transition region C ∶= (A ∪ B) c is also non-empty. Since historically in TPT one thinks of A as the reactant states of a system, B as the product states and the transitions from A to B as reaction events, we call the pieces of a trajectory that connect A to B by the name reactive trajectories. Each reactive trajectory contains the successive states in C visited during a transition that starts in A and ends in B, see Figure 2, and we are interested in gathering dynamical information about the ensemble of reactive trajectories.
2 TPT can be generalized to consider transitions between N subsets A1, A2, . . . , A N by looking at the transitions between A = Ai and B = ⋃ N j≠i Aj for each i = 1, . . . , N (e.g., used in the core set approach [SNL + 11, Sar11, SS13]). In this paper, we are interested not only in Markov chains living on the infinite time frame T = Z but also those on finite time intervals T = {0, 1, . . . , N − 1}, that is where the transitions from A to B have to take place during a finite time frame T. Moreover, we also consider non-stationary dynamics. This either means -that the system is in the transient phase towards equilibrium but otherwise has a timeindependent transition matrix, or -that the dynamical rules (the transition matrices) are varying in time, as well as -that we are dealing with time-varying sets A and B (see the comment in Section 4.2 for the case of periodic dynamics and Remark 5.4 for finite-time dynamics). In this section, we will define the committor functions and show how they can be used to derive important statistics of the ensemble of reactive trajectories, this entails, e.g., the frequency of transitions, the most important transition channels and where the process on the way to B gets stuck or spends most of its time. Thereby we will keep everything general enough for timedependent and finite-time dynamics and for the moment only need to assume the Markovianity of the chain (X n ) n∈T and that the distribution P(X n = i) and time-dependent transition rule P (n) = (P(X n+1 = j X n = i)) i,j∈S is given for all n ∈ T.
Later, the results from this section will be applied to the special cases of (i) infinite-time, stationary dynamics (Section 3), (ii) infinite-time, periodic dynamics (Section 4) and (iii) finitetime, time-dependent systems (Section 5), where we will also prove the existence of committors and give the linear system of equations they are solving. These three are just some selected special cases, other interesting cases of systems are, e.g., stochastic regime-switching (see the Remark A.1).

Committor Equations
All of the transition statistics and characteristics can be computed from the committor probabilities, therefore we will start by defining them. The forward committor is the probability that the Markov chain, currently in some state i, will next 3 go to set B and not to A. The backward committor is the same for the time-reversed Markov chain, i.e., the probability that the time-reversed process will next hit A and not B, or equivalently, the probability that the chain last came from A and not B.
More precisely, the forward committor q + (n) = (q + i (n)) i∈S gives the probability that the process starting in i ∈ S at time n ∈ T reaches at next within T first B and not A. We write where the first entrance time of a set S ⊂ S after or at time n ∈ T is given by The backward committor q − (n) = (q − i (n)) i∈S gives the probability that the trajectory arriving at time n ∈ T in state i ∈ S last within T came from A not B, 3 where with next we also include the current time point, i.e., that the system is already in B where the last exit time of a set S before time or at time n is given by Note that the first entrance and last exit times are stopping times with respect to the forward and time-reversed process [Rib19, Lemma 3.1.1]. The time-reversed process will be introduced in the following sections for infinite-time and finite-time Markov chains. The forward committor, as it is defined, only considers trajectory pieces arriving at B within the time horizon T, similarly, the backward committor only considers excursions that left A within T.

Transition Statistics
We will now define statistical objects that characterize the ensemble of reactive trajectories from subset A to B and see that they can be computed using the committor probabilities and the Markovianity assumption.
The first two objects, the distribution of reactive trajectories µ AB (n) and its normalized versionμ AB (n), tell us where the reactive trajectories are most likely to be found, i.e., where the transitioning trajectories spend most of their time.
Definition 2.1. The distribution of reactive trajectories µ AB (n) = (µ AB i (n)) i∈S for n ∈ T gives the joint probability that the Markov chain is in a state i at time n while transitioning from A to B: Note that µ AB i (n) = 0 for i ∉ C, i.e., we only get information about the density of transitions passing through C. Direct transitions from A to B are neglected, and in general assumed not to exist.
Theorem 2.2. For a general Markov chain (X n ) n∈T with committors q + (n), q − (n), the distribution of reactive trajectories can be expressed as Proof. We can compute by conditioning on {X n = i}, and by using independence of the two events {τ − A (n) > τ − B (n)}, {τ + B (n) < τ + A (n)} given {X n = i}, which follows from the Markov property. 4 The distribution µ AB (n) is not normalized but can easily be normalized by dividing µ AB (n) by the probability to be on a transition at time n, to give a probability distribution on S: > 0 for n ∈ T, we can define the normalized distribution of reactive trajectories at time n ∈ T bŷ giving the density of states in which trajectories transitioning from A to B spend their time. The next object tells us about the average number of jumps from i to j during one time step (i.e., the probability flux) while the trajectory is on its way from A to B: Definition 2.4. The current of reactive trajectories f AB (n) = (f AB ij (n)) i,j∈S at time n ∈ T gives the average flux of trajectories going through i at time n ∈ T and j at time n + 1 ∈ T consecutively while on their way from A to B: Theorem 2.5. The current of reactive trajectories for a Markov chain (X n ) n∈T with transition probabilities P (n) and committors q + (n), q − (n), is given by Proof. The reactive current can be computed as Let us note that the reactive current also counts the transitions going directly from i ∈ A to j ∈ B, these are not accounted for in the reactive distribution which only accounts for transitions passing the region (A ∪ B) c .
In order to eliminate information about detours of reactive trajectories, we define: Definition 2.6. The effective current of reactive trajectories f + (n) = (f + ij (n)) i,j∈S at time n ∈ T gives the net amount of reactive current going through i at time n ∈ T and j at time n + 1 ∈ T consecutively, Ultimately, the effective current of reactive trajectories can be used to find the dominant transition channels in state space between A and B (see, e.g., [MSVE09]).
The current of reactive trajectories only goes out of A, not into A, moreover the current of reactive trajectories only points into B, not out of B. Therefore, A can be thought of as a source of reactive trajectories, whereas B acts like their sink. This leads us to our next characteristic of reactive trajectories: by summing the current of reactive trajectories over A we get the the discrete rate of reactive trajectories flowing out of A, and by summing the current over B, we obtain the rate of inflow into B: Definition 2.7. For n, n + 1 ∈ T, the discrete rate of transitions leaving A at time n is given by i.e., the probability of a reactive trajectory leaving A at time n. When n − 1, n ∈ T, the discrete rate of transitions entering B at time n is given by i.e., the probability of a reactive trajectory entering B at time n.
Theorem 2.8. For a Markov chain (X n ) n∈T with current of reactive trajectories f AB (n), we find the discrete rates to be Proof. We can compute by using the law of total probability i∈A,j∈S

TPT for Stationary, Infinite-time Markov Chains
TPT was originally designed for stationary, infinite-time Markov processes [WVE06, MSVE09], that are often used as models for molecular systems [NSVE + 09]. Here we will recall that theory by using the results from the previous section and equip it with some new results (e.g. Lemma 3.5) that will be needed later.

Setting
We begin with describing the processes of interest in this section.
Assumption 3.1. We consider a Markov chain (X n ) n∈Z taking values in a discrete and finite state space S, the time-discrete jumps between states i ∈ S and j ∈ S occur with probability stored in the row-stochastic transition matrix P = (P ij ) i,j∈S . We assume that the process is irreducible, and ergodic with respect to the unique, strictly positive invariant distribution π = (π i ) i∈S (also called stationary distribution interchangeably) solving π ⊺ = π ⊺ P . The time-reversed process (X − n ) n∈Z , X − n ∶= X −n traverses the chain backwards in time. It is also a Markov chain [Rib19, Thm 2.1.19] and stationary with respect to the same invariant distribution. The transition probabilities of the time-reversed process P − = (P − ij ) i,j∈S with entries can be found from expressing the flux in two ways,

Committor Probabilities
Due to stationarity of the chain (Assumption 3.1), the law of the chain is the same for all times, and we simply have that the committors (4) and (5) are time-independent q + i (n) = q + i , similarly q − i (n) = q − i for all n. The forward and backward committors can be found by solving a linear matrix equation of size C with appropriate boundary conditions [Nor98, Chapter 1.3].
Theorem 3.2. The forward committor for a Markov chain according to Assumption 3.1 with transition probabilities P = (P ij ) i,j∈S satisfies the following linear system (8) Analogously for the backward committor, we have to solve the following linear system Proof. From the definition of the committors (4), it immediately follows that we have For the committor at node i in the transition region C, we can sum the forward committor at all the other states j weighted with the transition probability to transition from i to j. This follows from first using the law of total probability, then conditioning on {X n+1 = j}, using that at time n the chain is in i ∈ C and thus τ + A (n), τ + B (n) ≥ n + 1, and last using the Markov property. For the backward committor equations we can proceed in a similar way, by additionally using the time-reversed process.
Remark 3.3. If the Markov chain in addition is reversible, i.e., if P ij π i = P ji π j (equivalently, P − ij = P ij ) holds, then it follows from Theorem 3.2 that the forward and backward committor are related by The following lemma provides us with the necessary condition such that existence and uniqueness of the committors is guaranteed. For a proof see [ Lemma 3.4. If P is irreducible, then the two problems (8) and (9) each have a unique solution.
There is a second characterization of the committors in the transition region using path probabilities, as summarized in the following lemma. The proof can be found in the Appendix A.2.
Lemma 3.5. For any i ∈ C, the forward committor can also be specified as the probability of all possible paths starting from node i that reach the set B before A Similarly for any i ∈ C, the backward committor can also be understood as the sum of path probabilities of all possible paths arriving at node i that last came from A and not B

Transition Statistics
The committor, the distribution, and the transition probabilities are time-independent, thus the statistics from Section 2.2 are time-independent. We write the distribution of reactive trajectories (Theorem 2.2) as µ AB = (µ AB i ) i∈S , where Theorem 3.6. For a stationary Markov chain (X n ) n∈Z the reactive current out of a node i ∈ C equals the current flowing into the node i ∈ C, i.e., Further, the the reactive current flowing out of A into S (equivalently into C ∪ B) equals the flow of reactive trajectories from S (equivalently from C ∪ A) into B i∈A,j∈S For the proof see the Appendix A.2.
Remark 3.7. Due to these conservation laws, there is a close relation of the committors and the effective current (when the chain is reversible) to the voltage and the electric current in an electric resistor network [Doy84,Nor98] with a voltage applied between A and B, see [Met08,MSVE09] for work in this direction. Also in [Met08,MSVE09], a decomposition algorithm of the effective current into the paths from A to B carrying a substantial portion of the transition rate is proposed, yielding the dominant transition channels between A and B. The effective current of reactive trajectories is loop-erased, therefore in [BCS15] a decomposition of the current of reactive trajectories into cyclic structures and non-cyclic parts is suggested. Further, since ∑ i∈A,j∈S f AB ij = ∑ i∈S,j∈B f AB ij by Theorem 3.6, the discrete rate of leaving A equals the discrete rate of entering into B, thus we denote k AB ∶= k A→ = k →B . This tells us the probability of a realized transition per time step, i.e., either the probability to leave A and be on the way to B next, or the probability to reach B when coming from A. That k AB indeed has the physical interpretation of a rate becomes clear from the characterization in Theorem 3.8 below.

Interpretation of the Statistics as Time-averages along Trajectories
The statistics from the previous section give us dynamical information about the ensemble of reactive trajectories. Due to the ergodicity, the Markov chain will visit all states infinitely many times and the ensemble space average of a quantity equals the time average this quantity along a single infinitely long trajectory (Birkhoff's ergodic theorem). Therefore the reactive trajectory statistics from the previous section can also be found by considering the reactive pieces along a single infinitely long trajectory and by averaging over them.
Theorem 3.8. For a Markov chain (X n ) n∈Z satisfying Assumption 3.1, we have the following P−almost sure convergence results: where i, j ∈ S and 1 A (x) is the indicator function on the set A.
The proof of Theorem 3.8 can be found in [Rib19, Thm 3.3.2, Thm 3.3.7, Thm 3.3.11] and relies on Birkhoff's ergodic theorem [Wal00] for the canonical representation of the process as a Markov shift.
The theorem not only offers a data-driven approach to approximate the transition statistics by averaging along a given sufficiently long trajectory sample but also gives interpretability to the statistics. While µ AB as the (not normalized) trajectory-wise distribution of reactive trajectories and f AB as the trajectory-wise flux of reactive trajectories are still straightforward to understand, we can also give meaning to the rate k AB and to Z AB = ∑ i∈C µ AB i . We can think of k AB as the total number of reactive transitions taking place within the time interval {−N, . . . , N } divided by the number of time stpng 2N + 1 in the limit of N → ∞. Similarly we can give meaning to Z AB as the total time spent transitioning during {−N, . . . , N } divided by 2N + 1 in the limit of N → ∞. Last, we note that the ratio between Z AB and k AB provides us with a further characteristic transition quantity [VE06],

Numerical Example 1: Infinite-time, Stationary Dynamics on a 5-state Network
We consider a Markov chain on a network of five states S = {0, 1, 2, 3, 4} of which 0, 2, 4 have a high probability to remain in the same state. The transition matrix is given by the row-stochastic matrix P , see Figure 3. We are interested in transitions from the subset A = {0} to the subset B = {4}. What is the most likely route that the transitions take?
The numerically computed committors and transition statistics are shown in Figure 3, the discrete rate of transitions is k AB = 0.018, i.e., on average every 56 time stpng a transition from A to B is completed. We see that the effective current is the strongest along the path A → 1 → B, i.e., most transitions effectively happen via this path, but a share of 33, 3% of transitions happen via the route from A → 3 → B. Also, we can note that the density of reactive trajectories has a very high value in the state 2, indicating that many reactive trajectories pass this state or stay there for a long time. Thus reactive trajectories on the effective path A → 1 → B will likely do a detour to 2, which is not visible from the effective current since it does not tell us about detours.

TPT for Periodically Driven, Infinite-time Markov Chains
Many real-world systems showcase periodicity, for example any system subject to seasonal driving, or physical systems with periodic external stimuli. For studying transitions in these systems, we extend TPT to Markov chains with periodically varying transition probabilities that are equilibrated to the forcing and cycle through the same distributions each period. If the period is only one time step long, this case reduces to the previous case of stationary, infinite-time dynamics.
We start by laying out the exact setting of the process that we consider, before turning to the computation of committors and transition statistics for periodically forced dynamics. As we will see, by writing the committor equations on a time-augmented state space, we can also find committors for systems with stochastic switching between different dynamical regimes.

Setting
Assumption 4.1. Consider a Markov chain (X n ) n∈Z on a finite and discrete state space S with transition probabilities P ij (n) = P(X n+1 = j X n = i) that are periodically varying in time with period length M ∈ N, i.e., the transition matrices fulfill P (n) = P (n + M ) ∀n ∈ Z.
Therefore the transition matrices at the times within one period M ∶= {0, 1, . . . , M −1} are sufficient to describe all the dynamics and we denote them by P m ∶= P (n) for time n ∈ Z congruent to m ∈ M modulo M .
The product of transition matrices over one M −period starting at a time equivalent to m ∈ M (modulo M ) is denoted byP m ∶= P m P m+1 ⋯P m+M −1 , which is again a transition matrix pushing the Markov chain M time instances in time forward starting from time ≡ m (mod M ), see Figure  4. The chain described byP m is not time-dependent anymore, it resolves the state of the original system only every M time instance with a time-independent transition matrixP m .
Proposition 4.2. IfP 0 is irreducible and assuming the setting 4.1 as described above, then for all m ∈ M there exists an invariant distribution π m such that π ⊺ m = π ⊺ mP m of the transition matrix P m . Further, π 0 is unique and π 0,i > 0 for all i ∈ S. If we in addition require π ⊺ m+1 = π ⊺ m P m for all m, then the entire family (π m ) m=0,...,M −1 is unique. 5 Proof. SinceP 0 is irreducible and the state space is finite, the Markov chain induced byP 0 has a unique and positive invariant density π 0 = (π 0,i ) i∈S such that π ⊺ 0P 0 = π ⊺ 0 . It follows that alsoP 1 has an invariant density, namely π ⊺ 1 ∶= π ⊺ 0 P 0 , since it fulfills 5 Sometimes such a family of invariant densities is called equivariant.
Thus, the densities π 1 , . . . π M −1 are not necessarily unique, unless we require irreducibility. Doing so, there is a unique periodic family of distributions that the chain can admit, and we call such a chain M −stationary; see Figure 4. Having the long-time behavior of chains in mind in this section, we will assume this property, relying on ergodicity. Figure 4: The evolution of densities for periodic dynamics assuming M −stationarity. The density evolution described by P m , m ∈ M varies periodically with time n. Whereas the dynamics described byP m for a fixed m are timeindependent, they resolve the system only every M time instances at times n equivalent to m modulo M .
Assumption 4.3. We assume thatP 0 is irreducible and that the chain is M -stationary, i.e., P(X n = i) = π m,i whenever n is equivalent to m modulo M .
Next, we introduce the time-reversed chain (X − n ) n∈Z with X − n = X −n . Due to M −stationarity the transition probabilities of the time-reversed chain are also M −periodic, and it is enough to give the transition probabilities backward in time P − m for each time point during the period m ∈ M whenever π m,i > 0, else for π m,i = 0 we set P − m,ij ∶= 0.

Committor Probabilities
We will first look at the forward and backward committors and the system of equations that can be solved to acquire them. The forward q + (n) respectively backward committor q − (n) is defined as before in (4) respectively (5), but now since the law of the Markov chain is the same every M time instances, the committors also vary periodically and are identical every M time stpng, we therefore denote Whereas the M −periodic backward committor q − m = (q − m,i ) i∈S satisfies The proof follows the lines of the proof above for the stationary, infinite-time case and can be found in the Appendix A.2.
Before proving that the two systems are uniquely solvable, we characterize the forward and backward committors in terms of the path probabilities over one period M: Lemma 4.5. For any time n = m modulo M and i ∈ C the committor functions (15) satisfy the following equalities Proof. First it follows from (16) for i ∈ C that By inserting the committor equations at the following times iteratively and by using that q + 0 = q + M , we get (18). We can proceed analogously for the backward committor, starting from (17) and re-inserting committor equations.
The equation (18) with m = 0 only contains one unknown and can be solved, e.g., numerically for all i ∈ C, whereas for A respectively B the committor is simply 0 respectively 1. The committor for the remaining times m = 1, 2, .. can then be computed thereof by using (16). Analogously for (19).
The time-resolution of the Markov chain during the period is important for the committors since we can resolve hitting events of B within the period. The committors one would compute for a more coarsely resolved chain without state information during the period, i.e., for the chain described byP 0 (time-homogeneous, but mapping one period in time forward), will not notice that the chain has hit B at times other than m = 0. To see that, compare Lemma 4.5 with Lemma 3.5 usingP 0 .
Lemma 4.6. By the irreducibility ofP 0 , the solutions to (16) and (17) exist and are unique.
The proof can be found in the Appendix A.2.
Remark 4.7. The committor equations can also be written on a time-augmented state space using a period-augmented transition matrix that pushes the dynamics deterministically forward in time Extending this approach, one can also consider committor equations for systems that switch stochastically with probabilitiesP ∈ R M ×M between M different regimes, each regime is described by a transition matrix P m . This is essentially the Markov-chain analogue of a random dynamical system [Arn95]. The regime-augmented transition matrix is given by We refer to the Appendix A.1 for more details on both ansatzes and the computation of committor probabilities on the augmented space. The augmented approach also offers a numerical way of solving the committor equations with periodic boundary conditions.

Transition Statistics
We have seen that the forward and backward committor in the case of periodically driven dynamics are also M −periodic and can be computed from the iterative equations (16), (17) with periodic conditions in time. Since committors, densities and transition matrices are M −periodic, all statistics computed thereof are so too by the theory in Section 2.2, and we equip them with a subscript m, e.g.,μ AB (n) =μ AB m , f AB (n) = f AB m , whenever n ≡ m modulo M . Compared to the previous case of stationary, infinite-time, the discrete rate of reactive trajectories leaving A at time m, k A→ m = ∑ i∈A,j∈S f AB ij (m), does not anymore equal the discrete rate of reactive trajectories arriving in B at time m, k →B m = ∑ i∈S,j∈B f AB ij (m − 1). The next theorem provides us with the reactive current conservation laws in the case of periodic dynamics and will allow us to find the relation between k A→ m and k →B m .
The proof can be found in the Appendix A.2 and follows from straightforward computations using the committor equations to rewrite the reactive current.
As a result of (22), the discrete out-rate averaged over one period equals the average discrete in-rate, which we define to bek AB M , i.e., This period-averaged discrete rate tells us the average probability per time step of a reactive trajectory to depart in A or in other words, the expected number of reactive trajectories leaving A per time step.   Figure 6: We show statistics of Example 2, namely, (a) the normalized distribution which is shown by the color of the nodes, ranging from yellow (low values) via orange to purple (high value), as well as (b) the effective current from one state i to state j that is indicated by the thickness and greyscale of the arrow pointing from i to j. The node color of the effective current plot indicates the outflow of A and the inflow into B of current at the corresponding time point m, which can be related to the k A→ m and k →B m+1 .

Numerical Example 2: Periodically Varying Dynamics on a 5-state Network
We consider the same 5−state network as before in Example 1, but this time the Markov chain is in equilibrium to the periodically varying transition probabilities with period length M = 6. The transition matrices are chosen such that for time m = 0 the dynamics are the same as in the stationary Example 1, P 0 = T + L = P , whereas at time m = 3 the dynamics are reversed P 3 = T − L. Matrix T is a fixed row-stochastic matrix which has transition probabilities that are symmetric along the axis through A and B and L is a 0-rowsum matrix that is not symmetric along the axis through A and B. The probabilities of the transition matrices are shown in Figure 5.
With the numerically computed transition statistics as shown in Figure 6 we try to answer whether the added perturbation results in alternative effective paths compared to Example 1. The effective current indicates that the most likely effective transition path from A to B either goes via A − 1 − B (with a likely detour to 2) during the first half of the period, or via A − 3 − B (with a detour to 2) towards the second half of the period. But interestingly, we have additional transition paths that go via A − 1 − 2 − 3 − B and A − 3 − 2 − 1 − B. Neither in the stationary system described by P 0 = T + L nor by P 3 = T − L this path would be possible. Additionally, the period-averaged ratek AB 6 = 0.034 is higher than the rate in the stationary case (Example 1).

TPT for Markov Chains on a Finite Time Interval
We now develop TPT for Markov chains with the transitions of interest taking place during a finite time interval. The transition rules can be time-dependent and the dynamics can be non-stationary, i.e., out of equilibrium.
The resulting committor equations (Section 5.2) and statistics (Section 5.3) in case of finitetime dynamics are similiar as in the periodic case, yet there are some distinctions. The committor equations are now equipped with final respectively initial conditions and the statistics show some boundary effects at the time interval limits.
In Section 5.4 we also provide a consistency result between the stationary, infinite-time case and the finite-time case for a stationary Markov chain, by considering the limit of the time-interval going to infinity.

Setting
Let us start by describing the systems of interest in this section.
Assumption 5.1. We consider a Markov chain on a finite time interval (X n ) 0≤n≤N −1 , N ∈ N, taking values in a discrete and finite space S. The probability of a jump from the state i ∈ S to the state j ∈ S at time n ∈ {0, . . . , N − 2} is given by the (i, j)−entry of the row-stochastic transition matrix P (n) = (P ij (n)) i,j∈S : P ij (n) ∶= P(X n+1 = j X n = i).
Setting the initial density λ(0) = (λ i (0)) i∈S , the densities at later times n ∈ {1, . . . , N − 1} are given by λ(n + 1) ⊺ = λ(n) ⊺ P (n). By these assumptions, the chain can have time-inhomogeneous transition probabilities, or even if P (n) = P for all n, the densities λ(n) can be changing in time. Also, we are not requiring the chain to be irreducible anymore.
The time-reversed process (X − n ) 0≤n≤N −1 defined by X − n ∶= X N −1−n is also a Markov chain [Rib19, Thm 2.1.18]. Its transition probabilities are given for any n ∈ {1, . . . , N − 1} by whenever λ n (i) > 0. From the backward transition probabilities (23), we note that even the timereversed process of a finite-time, time-homogeneous Markov chain (i.e., P (n) = P for all n) is in general a finite-time, time-inhomogeneous Markov chain, unless also the distribution λ(n) is time-independent.

Committor Probabilities
The forward (4) and backward committors (5) Analogously, the backward committor for a finite-time Markov chain satisfies for n ∈ {1, . . . , N − 1} The proof uses some of the arguments of the proofs above for the stationary, infinite-time and periodic infinite-time cases and can be found in the Appendix A.2.
The following theorem provides us with an analogue result to Theorem 3.5 for the forward and the backward committors of a finite-time Markov chain written in terms of path probabilities of the paths that start in A and end in B within the restricted time frame T.
Lemma 5.3. The forward committor at time n ∈ {0, . . . , N − 2} and the backward committor at time n ∈ {1, . . . , N − 1}, respectively, satisfy for i ∈ C the following equalities: The proof follows from rewriting q Remark 5.4. Similar as in the periodic case, it is possible to extend the approach to timedependent sets (i.e., space-time sets) A(n) and B(n). For instance, in order to study transitions that leave a set at a certain time (e.g., at time n = 0) and arrive at a certain time (e.g., at n = N ) we can choose A(n) = 1 {0} (n)A and B(n) = 1 {N } (n)B.

Transition Statistics and their Interpretation
We have seen that the forward and backward committors for a finite-time Markov chain can be computed from the iterative equations (24) and (25) with final respectively initial conditions. Based on these, we will next introduce the corresponding transition statistics.
The distribution of reactive trajectories (Definition 2.1) is defined for any time n ∈ {0, . . . , N − 1}, and by Theorem 2.2 it is given by Observe that µ AB i (0) = µ AB i (N − 1) = 0 because there are no reactive trajectories at these times. Thus the distribution of reactive trajectories cannot be normalized at times 0 and N − 1. As a consequence, the normalized distribution of reactive trajectoriesμ AB (n) is just defined for times n ∈ {1, . . . , N − 2}.
Also in this case, the current satisfies certain conservation principles: Theorem 5.5. For a finite-time Markov chain (X n ) 0≤n≤N −1 satisfying Assumption 5.1, the reactive current flowing out of the node i ∈ C at time n equals the current flowing into a node i ∈ C at time n − 1, i.e., The proof can be found in the Appendix A.2. The discrete rate of transitions leaving A is defined for times n ∈ {0, . . . , N − 2} whereas the discrete rate of transitions entering B is defined for times n ∈ {1, . . . , N − 1} By plugging the definitions of the rates into result (29) of Theorem 5.5 and by re-indexing the times, we note that the discrete departure rate averaged over the times n ∈ {0, . . . , N − 2} equals the time-averaged discrete arrival rate over the times n ∈ {1, . . . , N − 1}, which we denote byk AB N , i.e.,k In the infinite-time, stationary case, Theorem 3.8 tells us that k AB equals the time average of the number of reactive pieces departing per time step along a single infinitely long trajectory. Here, we cannot apply the ergodic theorem to turnk AB N into an average along a single trajectory. Instead, we can write the time-averaged rate in terms of an ensemble of trajectories to get a better understanding. For this, we take K ∈ N i.i.d. realizations of the finite-time chain, i.e., each sample (X i n ) n∈{0,...,N −1} is distributed according to the law of the finite-time dynamics withX i 0 ∼ λ(0). Then we have by the law of large numbers: Further, we can rewrite the second line of (30) as and thus giving the average ratek AB N the interpretation of the total expected amount of reactive trajectories within {0, . . . , N − 1} divided by the number of time stpng. Analogously, we can apply the same argument for the time-averaged probability of being on a transition Remark 5.6. In [vKSZ18], transitions from A to B are studied in non-ergodic and non-stationary processes. But the focus there is put on the first pieces of the trajectories starting in B c which arrive in B after having been to A (called the first passage paths), whereas we consider the ensemble of all the transitions from A to B within the time interval T. Further, the first passage paths are divided into nonreactive and reactive segments and statistics for both, the reactive and the non-reactive ensemble, are computed. Last, their approach does not have any restrictions on the length of the first passage path and therefore is not straightforwardly extendable to finite-time processes.

Convergence Results of TPT Objects from Finite-time Systems to Infinitetime
The aim of this section is to show the consistency of the committors and TPT objects between the finite-time case and infinite-time case. We will show that when assuming a time-homogeneous chain on a finite-time interval that is stationary, the TPT objects converge to the infinite-time, stationary case when letting the time interval go to infinity.
To this end, let us consider the Markov chain (X n ) −N ≤n≤N on the time interval {−N, . . . , N } with a time-homogeneous and irreducible transition matrix P = (P ij ) i,j∈S . When choosing the unique, strictly positive invariant density π = (π i ) i∈S of P as an initial density, the density for all n ∈ {−N, . . . , N } is given by The time-reversed process (X − n ) −N ≤n≤N is also time-homogeneous and stationary, since its transition probabilities are given by The Proof.
where (1) follows from applying first Lemma 5.3 and then re-indexing the coefficients of the transition matrix P , and (2) follows directly from Lemma 3.5. Analogously we obtain for any n ∈ {−N + 1, . . . , N } that Hence, by putting together (34), (35) and (32) we show that for any i, j ∈ S

Numerical Example 3: Time-homogeneous, Finite-time Dynamics on a 5state Network
We consider a time-homogeneous Markov chain over T = {0, 1, . . . , N − 1}, N = 5 with the transition rules given by P (n) = P for all n ∈ {0, 1, . . . , N − 2}. P is the transition matrix used in Example 1 and the initial density is given by the stationary density π of P . Thus we are in the same setting as before in Example 1, but we are now looking at transitions between A and B that are restricted to take place within the finite time window. The transition statistics computed for this example are shown in Figure 7. Since the time interval was chosen rather small, transitions via 1, that usually do a detour to the metastable state 2, are very unlikely. The transition path via 3 is more likely, we can see that the effective current through 3 is much stronger, also the normalized reactive distribution indicates that it is the most likely to be in node 3 when reactive. We also note that the ratek AB 5 = 0.005 is much smaller than in the stationary case, only few transitions from A to B are completed (on average) in the short time frame {0, 1, .., N − 1}.

Numerical Example 4: Time-inhomogeneous, finite-time dynamics on a 5state network
As a next example, we consider a time-inhomogeneous chain over the finite time window T = {0, . . . , N − 1}, N = 5. We again set the stationary density of P as an initial density and let the transition matrices depend on the time but in such a way that 1 N −1 ∑ n P (n) = P . For any n ∈ {0, . . . , N − 2} let where K is the 0-rowsum matrix as given in Figure 8. At times n = 0, 2 the transition matrices become P +K and the subsets A and B are less metastable. At times n = 1, 3 the transition matrices become P − K and A and B are more metastable.
The computed transition statistics are shown in Figure 9. The effective current plot shows that the majority of reactive trajectories are leaving A at times n = 0, 2 and go to 3 and at times n = 1, 3 they move from 3 to B. The time-averaged rate for the finite-time, time-homogeneous case from Example 3 isk AB 5 = 0.005 while the time-averaged rate for this time-inhomogeneous case isk AB 5 = 0.012, more than two times larger. We thus demonstrated that by adding a forcing to the time-homogeneous dynamics that changes the metastability of A and B, we can increase the rate of transitions, even though the forcing vanishes on average. We note that utilizing perturbations that tip a system out of equilibrium is used in statistical mechanics to accelerate convergence of statistics [HMM04,LNP13].
In Figure 10 we show this convergence numerically, note that the statistics will not necessary converge near the boundary of the interval.

Numerical Examples in a Triplewell Landscape
In this section, we exemplarily study the transition behaviour of a particle diffusing in a triple well energy landscape for several scenarios: for infinite-time, stationary dynamics (Section 6.1), for infinite-time dynamics with an added periodic forcing (Section 6.2), and for stationary, finite-time dynamics (Sections 6.3, 6.4).

Numerical Example 6: Infinite-time, stationary dynamics
We consider the diffusive motion of a particle (X t ) t∈R in R 2 according to the overdamped Langevin equation where σ > 0 is the diffusion constant, W t is a 2−dimensional standard Brownian motion (Wiener process) and V (x, y) ∶ R 2 → R is the triple well potential given in [MSVE09,SS13], and shown in Figure 11. The particle is pushed by the force −∇V (x, y) "downhill" in the energy landscape, while simultaneously being subject to a random forcing due to the Brownian motion term. The stationary density of the process is given by the Boltzmann distribution π(x, y) = Z −1 exp(−2σ −2 V (x, y)) with normalization factor Z.
Before applying TPT on this example, we have to discretize the process into a Markov chain. We want to estimate a transition matrix that gives us the probability to jump between discrete cells of the state space [−2, 2] × [−1, 2], here we choose regular square grid cells {A i , i = 1, ...} of size 0.2 × 0.2. By means of Ulams method ( [Ula60], see also [Kol11a,Chapter 2.3] for a summary of the method) one can project the dynamics of (40) onto the space spanned by indicator functions on the grid cells, and then further approximate the projected dynamics by a Monte-Carlo sum of sampled trajectory snippets whereX c , c = 1, . . . , C, are sampled uniformly from the cell A i and we getŶ c by evolving the sample forward in time according to (40) with time step τ (e.g., by using the Euler-Maruyama discretization of the stochastic differential equation (40)). The resulting process defined by the transition matrix P is a discrete-time, discrete-space Markov chain with time stpng τ . 6 We are now interested in the transition behaviour between the deep wells of V when the dynamics are stationary. Therefore, we choose sets A and B as centered at (−1, 0) and (1, 0) respectively and we ask: which transition path from A to B is more likely; via the third metastable shallow well at (0, 1.5), or via the direct barrier between A and B?
The computed committor functions and statistics of the reactive trajectories are shown in Figure 12, we chose σ = 1, τ = 0.3 and sampled C = 10, 000 for estimating the transition matrix. Since the dynamics are reversible, the backward committor is just 1 − q + . Also, we can see that the committors are close to constant inside the metastable sets (the wells) due to the fast mixing inside the wells, but vary across the barriers. The computed effective current 7 indicates that most transitions occur via the direct barrier between A and B. The effective current through the shallow well at (0, 1.5) is only very small, but due to the metastability of the well, reactive trajectories taking that path are stuck there for long and thus contributing a lot to the density µ AB . The rate of transitions is k AB = 0.0142 and the mean transition time is t AB = 10.01.

Numerical Example 7: Periodic, Infinite-time Dynamics
Next, we are interested in studying transitions in the triplewell landscape when a periodically varying forcing is applied. We add the force F (x 1 , x 2 , t) = 1.4 cos( 2πt 1.8 )(−x 2 , x 1 ), that alternatingly due to the cosine modulation exhibits an anti-clockwise circulation and a clockwise circulation, to the dynamics (40), resulting in the diffusion process with 1.8-periodic forcing, We again discretize the dynamics and estimate transition matrices P 0 , P 1 , . . . , P M −1 (M = 6) for τ -spaced time points during the period, each transition matrix is mapping τ = 0.3 into the future. In Figure 11 the force from the potential plus circulation is shown for time points m = 0 and m = 3. Considering the same A and B as before, we are now interested in the transition channels when the dynamics are equilibrated to the periodic forcing. The computed results are shown in Figure 13, the reactive trajectories take the lower channel via the direct barrier at the beginning of the period due to the additional push from the forcing, and the upper channel via the shallow well towards the end of the period when the applied circulation is clockwise. It is interesting to note, that the rate of reactive trajectories leaving A and entering B is highest towards the end of the period, when the added forcing is clockwise and the preferred channel is the upper channel. This is contrary to the stationary example before, where reactive trajectories dominantly passed the lower channel.

Numerical Example 8: Finite-time, Time-homogeneous Dynamics
To demonstrate the effect of the finite-time restriction on the transition behaviour between A and B, we now study the time-homogeneous triplewell dynamics restricted to the time window T = {0, . . . , N − 1}, N = 6, and initiated in the stationary density. Even though we study the same underlying dynamics as in the stationary, infinite-time case (Example 6), the possible transition paths between A and B are limited to the pathway that is fast to traverse, i.e., the lower channel via the direct barrier, see Figure 14. Since only a small portion of the reactive trajectories from the infinite-time example 6 has a short enough transition time to be considered in this case, the average rate of transitionsk AB 6 = 0.0017 is much lower than the corresponding rate k AB = 0.0142 in the infinite time case (Example 6), and the average time a transition takest AB 6 = 2.055 is much shorter than in the infinite-time case.
6.4 Numerical Example 9: Bifurcation Studies using Finite-time TPT Last, we want to highlight the usage of finite-time TPT to study large qualitative changes in the transition behavior of a system. We consider the stationary triple well example over a finite interval T = {0, . . . , N − 1}, but this time with a smaller noise strength of σ = 0.26 compared to the previous examples.
As we increase the interval length N from N = 20 to N = 500, we allow reactive trajectories to be longer and longer, and thus the average reactive trajectory changes. Whereas for N = 20 most of the density and current is around the lower transition channel, see Figure 15, for N = 500 most of the density and current is around the upper transition channel. The transition behaviour restricted to the time interval of size N = 500 is already close to the infinite-time transition dynamics, this can be seen by comparing the case of N = 500 with Figure 1(b), where the effective current of the same dynamics but in infinite time is depicted.

Conclusion
In this paper, we generalized Transition Path Theory such that it is applicable not only to infinitetime, stationary Markov processes but also to periodically varying and to time-dependent Markov processes on finite time intervals. We restricted our results to Markov processes on discrete state spaces and in discrete time, but generalizations should be straightforward (e.g., following [WVE06, MSVE09]).
The theory is intended to generalize TPT towards applicability in, e.g., non-equilibrium molecular, climate, fluid, or social dynamics (agent-based models). In most of these applications, the problem of computing the TPT objects arises, as the state space can be high-and also infinite dimensional. This is a non-trivial task even in the traditional TPT context stemming from molecular dynamics-where these tools have nevertheless been successfully applied. Resolving the timedependence poses an additional computational challenge. All this goes beyond the scope of the present work and will be addressed elsewhere.
First results towards the application of stationary TPT in high-dimensional state spaces have already been proposed. In [TGDW19] a workaround was given for solving the committor equations in the case of infinite-time, stationary dynamics by a Galerkin projection which works well as long as the dynamics of the Markov chain can be described by low-dimensional representations. Another interesting first work [KLY19] goes into the direction of using neural networks for solving the committor equations in high dimensions.
Further, the interpretability and visualization of the transition statistics in high dimensions or for large time intervals (large N or M ) is a point of future research. In [MSVE09] an algorithm for computing transition channels from the effective current of reactive trajectories was proposed, a generalization for time-dependent dynamics is ongoing work.
An implementation of the tools developed in this paper can be found at www.github.com/ LuzieH/transitions. As an alternative to the approach in Section 4.2, we can consider the dynamics and the committor equations on the augmented state space S M . The augmented transition matrix P Aug of size M S × M S contains all transition matrices P 0 , . . . , P M −1 and applies them consecutively in a row e.g., applying P Aug to a space-time distribution with mass only at the 0−th time point shifts all the mass to the next time point. Since the transition matrices are row-stochastic, so is the augmented transition matrix.
With that set, we can take a slightly different approach for writing the forward committor equations as one augmented matrix equation. Again, we need to solve the committor only on C, therefore we define the augmented C M × C M matrixP Aug , which encodes only the transitions inside CP Then from (16), we arrive at the following equation forq + Note that (I −P Aug ) is large but sparse compared to using the stacked equations (18). For the backward committor equations, we proceed similarly and arrive at the equations where we defined Remark A.1. This approach can also be used for studying committors and transitions in systems with stochastic switching between different regimes. Consider the dynamics of a Markov chain that can be in M different regimes each described by a transition matrix P m , m ∈ {1, . . . , M }. The probability to switch between regimes is given byP ∈ R M ×M . The committor probabilities would give the probability to next hit B and not A, given the chain is in a certain state and regime and assuming the dynamics are equilibrated. Note that the periodic dynamics described in this paper are just a special case of stochastic switching, where the switching is deterministic from time m to time m + 1, i.e.,P mm ′ = δ mm ′ −1 modulo M . The regime-augmented matrix can be written as:
The law of total probability lets us write the forward committor in terms of a countable sum of the above mentioned joint probability: where we have used that P(τ + A∪B (n) = ∞) = 0 due to the ergodicity of the process, which ensures that the process will arrive at some time to the subset A ∪ B.
Next, by applying the same arguments of the proof of Theorem 3.2 we see that the joint probability J + i (n, τ ) satisfies the following iterative system of equations for τ ∈ Z + n+1 : with initial condition J + i (n, n) = 1 B (i).
Putting the results (46) and (48) together completes the proof for the forward committor. The proof for the backward committor follows by using the same arguments.
A.2.2 Theorem 3.6 Proof. First, for any i ∈ C, we have using the definition of the time-reversed transition probabilities and the committor equations (4), (5) for i ∈ C.
And for i ∈ A at time n it follows from the definition that τ + A (n) = n, whereas τ + B (n) > n, thus q + i (n) = 0, in a similar way for i ∈ B, τ + A (n) > n, τ + B (n) = n thus q + i (n) = 1. The proof for the backward committor equations follows the same lines.

A.2.4 Lemma 4.6
Proof. We start with the case of the forward committor (16). We can rewrite (18) with m = 0, with P 0 I→J denoting the restriction of the matrix P to entries from i ∈ I to j ∈ J, as the matrix equation We note, that the equation is uniquely solvable as long as (I − D) is invertible, (I − D) is invertible if ρ(D) < 1. By assuming thatP 0 is irreducible, we will show that for all v ∈ R C , vD 1 < v 1 . Since this holds in particular for the eigenvectors, it follows that all eigenvalues λ < 1 and thus ρ(D) < 1.
We know that D is a substochastic matrix (row sum is ≤ 1) since it is a product of substochastic matrices, and that all entries are non-negative, i.e., ∑ j D ij ≤ 1 for all i ∈ C.
Moreover, there exists at least one row with row sum less than 1 since by irreducibility ofP 0 , there must be at least one state i in C with a positive probability to go to A or B. We call the first one of this kind by i * with ∑ j D i * j < 1. Thus we can compute We have shown that a unique solution which we will call q + 0 exists. The forward committors for m = 1, . . . , M − 1 can uniquely be computed therof by using (16).
For the case of the backward committor, we can proceed analogously by using the time-reversed transition probabilities mapping M instances backP − m ∶= P − m ⋅ P − m−1 ⋯P − m−M +1 , and by noting that ifP 0 is irreducible, alsoP − 0 is irreducible, which follows from the definition of irreducibility ofP 0 and using (15).
where in (4) we have applied the time-dependent current conservation for i ∈ C, n ∈ {1, . . . , N − 2} and we have used that f AB ij (0) = 0, and in (5) we have relabeled i, j and used that f AB ij (N − 2) = 0. As a consequence N −2 n=0 i∈A j∈S f AB ij (n) = N −2 n=0 i∈S j∈B f AB ij (n).