Abstract
Large continuoustime Markov chains with exponentially small transition rates arise in modeling complex systems in physics, chemistry, and biology. We propose a constructive graphalgorithmic approach to determine the sequence of critical timescales at which the qualitative behavior of a given Markov chain changes, and give an effective description of the dynamics on each of them. This approach is valid for both timereversible and timeirreversible Markov processes, with or without symmetry. Central to this approach are two graph algorithms, Algorithm 1 and Algorithm 2, for obtaining the sequences of the critical timescales and the hierarchies of Typical Transition Graphs or Tgraphs indicating the most likely transitions in the system without and with symmetry, respectively. The sequence of critical timescales includes the subsequence of the reciprocals of the real parts of eigenvalues. Under a certain assumption, we prove sharp asymptotic estimates for eigenvalues (including prefactors) and show how one can extract them from the output of Algorithm 1. We discuss the relationship between Algorithms 1 and 2 and explain how one needs to interpret the output of Algorithm 1 if it is applied in the case with symmetry instead of Algorithm 2. Finally, we analyze an example motivated by R. D. Astumian’s model of the dynamics of kinesin, a molecular motor, by means of Algorithm 2.
Similar content being viewed by others
References
Astumian, R.D.: Biasing the random walk of a molecular motor. J. Phys.: Condens. Matter 17, S3753 (2005)
Arkus, N., Manoharan, V., Brenner, M.P.: Minimal energy clusters of hard spheres with short ranged attractions. Phys. Rev. Lett. 103, 118303 (2009)
Arkus, N., Manoharan, V., Brenner, M.P.: Deriving finite sphere packings. SIAM J. Discret. Math. 25, 1860–1901 (2011)
Berglund, N., Dutercq, S.: The Eyring—Kramers Law for Markovian Jump Processes with Symmetries. J. Theory Prob. First online: 21 May 2015, doi:10.1007/s1095901506179
Bouchet, F., Reygner, J.: Generalization of the EyringKramers transition rate formula to irreversible diffusion processes, Annales Henri Poincare, First online: 11 June 2016, pp. 1–34 arXiv:1507.02104v1
Bovier, A., Eckhoff, M., Gayrard, V., Klein, M.: Metastability and low lying spectra in reversible Markov chains. Comm. Math. Phys. 228, 219–255 (2002)
Bovier, A., Eckhoff, M., Gayrard, V., Klein, M.: Metastability in reversible diffusion processes 1. Sharp estimates for capacities and exit times. J. Eur. Math. Soc. 6, 399–424 (2004)
Bovier, A., Gayrard, V., Klein, M.: Metastability in reversible diffusion processes 2. Precise estimates for small eigenvalues. J. Eur. Math. Soc. 7, 69–99 (2005)
Bovier, A., den Hollander, F.: Metastability: A PotentialTheoretic Approach. Springer, Berlin (2016)
Cameron, M.K.: Computing Freidlin’s cycles for the overdamped Langevin dynamics. J. Stat. Phys. 152(3), 493–518 (2013)
Cameron, M.K.: Computing the asymptotic spectrum for networks representing energy landscapes using the minimal spanning tree, M. Cameron. Netw. Heterog. Media 9, 3 (2014)
Cameron, M.K., Gan, T.: Spectral analysis and clustering of large stochastic networks. Application to the LennardJones75 cluster. Mol. Simul. 42(16), 1410–1428 (2016)
Chu, Y.J., Liu, T.H.: On the shortest arborescence of a directed graph. Sci. Sin. 14, 1396–1400 (1965)
Edmonds, J.: Optimum branchings. J. Res. Natl. Bur. Stand.: B Math. Math. Phys. 71 B, 4 (1967)
Freidlin, M.I.: Sublimiting distributions and stabilization of solutions of parabolic equations with small parameter. Sov. Math. Dokl. 18(4), 1114–1118 (1977)
Freidlin, M.I.: Quasideterministic approximation, metastability and stochastic resonance. Phys. D 137, 333–352 (2000)
Freidlin, M.: On stochastic perturbations of dynamical systems with a “rough” symmetry. Hierarchy of Markov chains. J. Stat. Phys. 157(6), 1031–1045 (2014)
Freidlin, M.I., Wentzell, A.D.: Random Perturbations of Dynamical Systems, 3rd edn. Springer, Berlin (2012)
Gaveau, B., Schulman, L.S.: Theory of nonequilibrium firstorder phase transitions for stochastic dynamics. J. Phys. 33, 4837–4850 (2000)
Gerschgorin, S.: Uber die abgrenzung der eigenwerte einer matrix. Izv. Akad. Nauk. USSR Otd. Fiz.Mat. Nauk 7, 749–754 (1931)
HolmesCerfon, M., Gortler, S.J., Brenner, M.P.: A geometrical approach to computing freeenergy landscapes from shortranged potentials. Proc. Natl. Acad. Sci. 110(1), E5–E14 (2013)
Meng, G., Arkus, N., Brenner, M.P., Manoharan, V.: The free energy landscape of hard sphere clusters. Science 327, 560 (2010)
Metzner, P., Schuette, Ch., VandenEijnden, E.: Transition path theory for Markov jump processes. SIAM Multiscale Model Simul. 7, 1192–1219 (2009)
Norris, J.P.: Markov Chains. Cambridge University Press, Cambridge (1997)
Wales, D.J.: Discrete path sampling. Mol. Phys. 100, 3285–3306 (2002)
Wales, D.J.: Energy Landscapes: Applications to Clusters, Biomolecules and Glasses. Cambridge University Press, Cambridge (2003)
Wales, D.J.: Energy landscapes: calculating pathways and rates. Int. Rev. Chem. Phys. 25(1–2), 237–282 (2006)
Wentzell, A.D.: On the asymptotics of eigenvalues of matrices with elements of order \(exp \{ V_{ij} / 2 (\varepsilon ^2) \} \), (in Russian) Dokl. Akad. Nauk SSSR 202, 263–265 (1972)
Acknowledgements
We would like to thank Mr. Weilin Li for valuable discussions in the early stage of this work. We are grateful to Prof. C. Jarzynski and Prof. M. Fisher for suggesting us to consider molecular motors as an example of a natural timeirreversible system with transition rates in the exponential form. We also thank the anonymous reviewers for their valuable feedback and comments. This work was partially supported by NSF Grants 1217118 and 1554907.
Author information
Authors and Affiliations
Corresponding author
Additional information
Communicated by Charles R. Doering.
Appendices
Appendix 1: Proof of Theorem 2.4
The following notations will be used throughout the proof.

\({\mathcal {G}}(nl)\) is the set of all Wgraphs with \(nl\) sinks and l arcs for the graph \(G({\mathcal {S}},{\mathcal {A}},{\mathcal {U}})\). The set of subgraphs of G with exactly l arcs emanated from distinct vertices will be denoted by \({\mathcal {H}}(l)\). Note that graphs in \({\mathcal {H}}(l)\) might contain directed cycles. Therefore, \({\mathcal {G}}(nl) \subsetneq {\mathcal {H}}(l)\).

\({\mathcal {S}}^l \) is the set of all ordered selections of l vertices out of n (in the combinatorial sense):
$$\begin{aligned} {\mathcal {S}}^l: = \{ i_1 i_2 \cdots i_l ~~ i_m \in \{ 1, 2 , \cdots , n1\}, 1 \le m \le l , {\text { distinct}} \}. \end{aligned}$$Note that \({\mathcal {S}}^l = n(n1)\ldots (nl+1)\).

\({\mathcal {O}}^l \) is the set of combinations of l vertices out of n (in the combinatorial sense). Each combination in \({\mathcal {O}}^l\) is ordered so that \(i_1<i_2<\ldots <i_l\), i.e.,
$$\begin{aligned} {\mathcal {O}}^l := \{ i_1 i_2 \cdots i_l \in {\mathcal {S}}^l ~~ i_1< i_2< \cdots < i_l \}. \end{aligned}$$Note that \({\mathcal {O}}^l = \left( \begin{array}{c}n\\ l\end{array}\right) \).

Every sequence \(j_1,\ldots j_l\) in \({\mathcal {S}}^l\) can be permuted to an ordered sequence \(\{i_1,\ldots , i_l\}\) in \({\mathcal {O}}^l\). This defines the permutation map \(\sigma \): \(\sigma (j_1,\ldots ,j_l) = (i_1,\ldots ,i_l)\). Note that the map \(\sigma \) is onto but not one to one.

We will call sequences in \({\mathcal {S}}^l\) equivalent if and only if they are mapped to the same sequence in \({\mathcal {O}}^l\). Therefore, \({\mathcal {O}}^l={\mathcal {S}}^l/\sim \), meaning that \({\mathcal {O}}^l\) is the set of equivalence classes in \({\mathcal {S}}^l\).

For any \(i_1 \cdots i_l \in {\mathcal {O}}^l\), we will denote by \(L_{ i_1 \cdots i_l }\) the \(l\times l\) submatrix of L consisting of the intersection of \( i_1, \cdots , i_l \) rows and columns of L.
Proof
The proof of Theorem 2.4 consists of the following three steps.
 Step 1: :

The characteristic polynomial of the generator matrix L is
$$\begin{aligned} P_L(t) := {\text {det}} ( t I  L) = t^n + \sum _{l = 1}^{n1} C_l t^{nl} \end{aligned}$$whose coefficients \(C_l\) are given by
$$\begin{aligned} C_l&= \sum _{i_1 \cdots i_l \in {\mathcal {O}}^l} {\text {det}} (  L_{i_1 \cdots i_l} )\nonumber \\&= (1)^l \sum _{i_1 \cdots i_l \in {\mathcal {O}}^l} ~\sum _{ \begin{array}{c} j_1 \cdots j_l \in {\mathcal {S}}^l\\ j_1 \cdots j_l\sim i_1 \cdots i_l \end{array}} (1)^{ \sigma (j_1, \cdots , j_l) } L_{i_1 j_1} \cdots L_{i_l j_l}, \end{aligned}$$(23)where \( \sigma (j_1, \cdots , j_l) \) is the number of inversions in \(\sigma (j_1, \cdots , j_l) \).
 Step 2: :

Using the zero row sum property of the generator matrix L, Eq. (23) can be further simplified resulting at the following more compact expression:
$$\begin{aligned} C_l = \sum _{g \in {\mathcal {G}}(nl)} \Pi (g), \quad \mathrm{where}\quad \Pi (g) = \prod _{(i\rightarrow j)\in g}L_{ij}. \end{aligned}$$(24)The key idea of the derivation of Eq. (24) is to show that
$$\begin{aligned} C_l = (1)^l \sum _{g \in {\mathcal {H}}(l) } a_g \Pi (g), \quad \mathrm{where}\quad a_g = {\left\{ \begin{array}{ll} (1)^l, &{} {\text { if }} g \in {\mathcal {G}}(nl),\\ 0, &{} {\text { if }} g \in {\mathcal {H}}(l) {\setminus } {\mathcal {G}}(nl). \end{array}\right. } \end{aligned}$$(25)  Step 3: :

Comparing the coefficient of the characteristic polynomial
$$\begin{aligned} P_L(t) = {\text {det}} ( t I  L ) = t^n + \sum _{l = 1}^{n1} C_l t^{nl} = t (t + \lambda _1) \cdots (t + \lambda _{n1}), \end{aligned}$$we obtain the following estimates for eigenvalues:
$$\begin{aligned} \lambda _m = \frac{ \Pi (g^{*}_{m}) }{ \Pi (g^{*}_{m+1}) } (1 + o(1) ). \end{aligned}$$(26)
Now we elaborate each step.
Step 1:
Consider the following polynomial in n variables \(t_1, t_2, \cdots , t_n\):
where \(\{ i_{l+1}, \cdots , i_n \} = \{ 1, \cdots , n\} {\setminus } \{ i_1, \cdots , i_l \}\). Replacing all \(t_i\) by t, we recover the characteristic polynomial \(P_L(t)\) where
The term \(C_{i_1 \cdots i_l} t_{i_{l+1}} \cdots t_{i_n}\) in Eq. (27) is obtained by picking \(t_{i_{l+1}}\), ..., \(t_{i_n}\) from the diagonal entries in rows of the matrix \({\text {diag}} \{ t_1, t_2, \cdots , t_n\}  L\) and multiplying them by the determinant of \({L_{i_1\ldots i_l}}\). Hence,
Combining Eqs. (28) and (29) and applying the Leibniz formula for determinants, we obtain Eq. (23).
Step 2:
Consider the product terms \(L_{i_1 j_1} \cdots L_{i_l j_l}\), \( i_1 \cdots i_l \in {\mathcal {O}}^l, i_1 \cdots i_l \sim j_1 \cdots j_l\) in Eq. (23). Suppose the sequences \(i_1 \cdots i_l\) and \(j_1 \cdots j_l\) agree on exactly s entries and differ at \(ls\) entries, i.e.,
Note that s can be any number between 0 and l except for \(l1\). Using the zero sum property of L, we obtain
It is helpful to consider the collection of graphs g with n vertices and sets of arcs
corresponding to the products
in Eq. (32). Each of the vertices \(i_1,\ldots ,i_l\) of g has exactly one outgoing arc, while the other vertices have no outgoing arcs. Hence, the graphs g belong to the set of graphs \({\mathcal {H}}(l)\). If \(s \le l2\), each of the vertices \(\{ i_{m_{s+1}}, \cdots , i_{m_l} \}\) of the graph g has exactly one outgoing arc heading to \(\{ i_{m_{s+1}}, \cdots , i_{m_l} \}\), and exactly one incoming arc from \(\{ i_{m_{s+1}}, \cdots , i_{m_l} \}\). Therefore, the arcs \(\{(i_{m_{s+1} } \rightarrow j_{m_{s+1}}), \cdots , ( i_{m_l} \rightarrow j_{m_l} ) \}\) necessarily form cycles in g. The arcs \(\{ (i_{m_1} \rightarrow d_1) , \cdots , (i_{m_s} \rightarrow d_s)\}\) do not necessarily form cycles. Therefore, Eq. (23) can be rewritten using Eq. (32) and the graphs \({\mathcal {H}}(l)\) as
where the factors \(a_g\) will be determined below.
First we assume that the graph g with the set of arcs \(\{ (i_1 \rightarrow j_1), \cdots , (i_l \rightarrow j_l) \}\) contains no cycles, i.e., \( g \in {\mathcal {G}} (nl)\subset {\mathcal {H}}(l)\). This can happen only if \(s=l\) in Eq. (33), i.e., \(i_1=j_1\), ...,\(i_l=j_l\). Therefore, \((1)^{ \sigma (j_1, \cdots , j_l) } =(1)^0 = 1\), and
Hence, the product \( L_{i_1 d_1} \cdots L_{i_l d_l}\) corresponding to the graph g enters Eq. (33) only once. Therefore, \(a_{g} = (1)^l\) for all \(g \in {\mathcal {G}} (nl)\).
Now we assume that the set of arcs \(\{ (i_1 \rightarrow x_1), \cdots , (i_l \rightarrow x_l) \}\) of g contains \(N\ge 1\) cycles, i.e., \(g \in {\mathcal {H}}(l) {\setminus } {\mathcal {G}}(nl)\). In this case, the product \( L_{i_1 x_1} \cdots L_{i_l x_l}\), \(i_p\ne x_p\), \(1\le p\le l\), enters Eq. (33) \(2^N\) times either with the plus or minus sign. The number \(2^N\) comes from the fact that each cycle in g can be formed in two ways: (i) by arcs corresponding to offdiagonal factors \(L_{i_pj_p}\), \(i_p\ne j_p\), in Eq. (33), or (ii) by arcs originating from the replacement of diagonal factors \(L_{i_pi_p}\) in Eq. (33) with \(\sum _{d_p\ne i_p}L_{i_pd_p}\). We will prove that \( L_{i_1 x_1} \cdots L_{i_l x_l}\), \(i_p\ne x_p\), \(1\le p\le l\), enters Eq. (33) with sign plus the same number of times as it does with sign minus. This will imply that \(a_g=0\). To do so, we show that for each entry of \( L_{i_1 x_1} \cdots L_{i_l x_l}\), one can uniquely define another entry with an opposite sign. Let c be a cycle in g. Consider two terms in Eq. (33) containing the product \(L_{i_1x_1}\ldots L_{i_lx_l}\) that correspond to possibilities (i) and (ii) for the origin of a selected cycle c in g, while all other factors corresponding to the arcs not in c originate in the same way. Let \(\sigma _1(j^1_1,\ldots j^1_l)\) and \(\sigma _2(j^2_1,\ldots ,j^2_l)\) be the permutations in Eq. (33) corresponding to (i) and (ii), respectively, and \(s_1\) and \(s_2\) be the corresponding numbers of fixed entries in \(\sigma _1\) and \(\sigma _2\), respectively. If the cycle c has length c, then
Here we have used the known combinatorial fact that the parity of a permutation consisting of cycles \(c_1\), ..., \(c_k\) is \((1)^{\sum _{j=1}^kc_j1}\). Therefore, the signs \((1)^l(1)^{\sigma _1}(1)^{s_1}\) and \((1)^l(1)^{\sigma _2}(1)^{s_2}\) preceding the corresponding products in Eq. (33) are opposite. This implies that all products corresponding to any graph \(g\in {\mathcal {H}}(l) {\setminus } {\mathcal {G}}(nl)\) cancel out, i.e., \(a_g=0\).
Therefore, \(C_l = \sum _{g\in {\mathcal {G}}(nl)}\Pi (g)\), i.e., Eq. (24) holds. An example illustrating this cancellation is given at the end of “Appendix 1.”
Step 3:
Let us write the characteristic polynomial \(P_L(t)\) in the form
where \(\lambda _1<\lambda _2<\ldots <\lambda _{n1}\). A simple calculation gives
Comparing Eqs. (24) and (36), we obtain
Since \(L_{ij}\) are of the form \(\kappa _{ij}\exp (U_{ij}/\varepsilon )\), the sums in the enumerators and the denominators are dominated by their largest summands in Eq. (37). According to Assumption 3, all optimal Wgraphs are unique. Therefore,
and Eq. (26) immediately follows.
Example 9.1
Let us illustrate the cancellation of terms in Eq. (33). Let \(n=4\), \(l=3\), and \(\{i_1,i_2,i_3\}=\{1,2,3\}\). Then the inner sum in Eq. (33) becomes
For each product term of the form \(L_{ix_1}L_{2x_2}L_{3x_3}\), \(x_q\in \{1,2,3,4\}\), one can draw a graph with the set of vertices \({\mathcal {S}} = \{1,2,3,4\}\) and the set of arcs \({\mathcal {A}} = \{1\rightarrow x_1,2\rightarrow x_2,3\rightarrow x_3,4\rightarrow x_4\}\). One can check that all terms corresponding to graphs with no cycles are encountered just once and only in the terms originating from \((j_1,j_2,j_3) = (1,2,3)\). All of them are preceded by the sign “.” This corresponds to \(a_g = (1)^l = (1)^3\). On the contrary, each term corresponding to a graph with cycles (in this example, there can be at most \(N=1\) cycle) is encountered exactly twice (\(2^1 = 2\)): once it comes from the product corresponding to \((j_1,j_2,j_3) = (1,2,3)\) with sign “,” and once it comes from some nonidentical permutation with sign “+.” Hence, all of such term cancel out.
Appendix 2: Proof of Theorem 6.1
Prior to start proving Theorem 6.1, we prove some auxiliary facts and introduce some use useful definitions. The proof of Theorem 6.1 will exploit the following lemmas.
Lemma 10.1
Suppose the function FindTgraphs is run on a graph \(G({\mathcal {S}},{\mathcal {A}},{\mathcal {U}})\) satisfying Assumptions 1 and 2: FindTgraphs \((r,k,G,\Gamma ,{\mathcal {B}}')\). Let c be a cycle detected in the graph \(\Gamma \) at step k. Suppose the weights of all outgoing arcs with tails in c and heads not in c are modified according to the update rule
Then \(U_{ij}^{new} \ge \gamma _k\) for all \(i\in c\), \(j \notin c\), and \(U_{ij}^{new} = \gamma _k\) if and only if \(U_{ij}=U_{\mu (i)}\).
Proof
The fact that \(U_{ij}^{new} \ge \gamma _k\) for all \(i\in c\) and \(j \notin c\) follows from the fact that \(U_{ij}\ge U_{\mu (i)}\). The equality takes place if and only if \(U_{\mu (i)} = U_{ij}\), i.e., if \(i\rightarrow j\) is another minarc from i.
Corollary 10.1
Suppose the function FindSymTgraphs is run on a graph \(G({\mathcal {S}},{\mathcal {A}},{\mathcal {U}})\) satisfying Assumptions 1 and 2: FindSymTgraphs \((r,p,G,T,{\mathcal {B}})\). Let C be a closed communicating class detected in the graph T at step p. Suppose the weights of all outgoing arcs with tails in C and heads not in C are modified according to the update rule
Then \(U_{ij}^{new} > \theta _p\) for all \(i\in C\), \(j \notin C\).
We will denote by \({\mathcal {S}}(v_{c})\) the subset of vertices of the original graph \(G({\mathcal {S}},{\mathcal {A}},{\mathcal {U}})\) contracted into the supervertex \(v_{c}\).
Corollary 10.2
Suppose the function FindTgraphs is run on a graph \(G({\mathcal {S}},{\mathcal {A}},{\mathcal {U}})\) satisfying Assumptions 1 and 2: FindTgraphs \((r,k,G,\Gamma ,{\mathcal {B}}')\). Let \(c_1\), ..., \(c_N\) be a sequence of cycles created after the addition of arcs of weights \(\gamma _1 \le \gamma _2 \le \ldots \le \gamma _N\), respectively, such that \({\mathcal {S}}(v_{c_N})\supset {\mathcal {S}}(v_{c_l})\) for all \(l < N\). Let i and j be vertices of G such that \((i\rightarrow j)\in {\mathcal {A}}\) and \(i\in {\mathcal {S}}(v_{c_l})\), \(l < N\), and \(j\notin {\mathcal {S}}(v_{c_N})\). Let \(U^{(l)}\) the (possibly modified) weight of the arc \(i\rightarrow j\) after the creation of the cycle \(c_l\). Then \(U^{(N)}_{ij} \ge \gamma _N\) and \(U^{(N)}_{ij} = \gamma _N\) if and only if \((i\rightarrow j)\) is a minarc from i.
Corollary 10.3
Suppose the function FindTgraphs is run on a graph \(G({\mathcal {S}},{\mathcal {A}},{\mathcal {U}})\) satisfying Assumptions 1 and 2: FindTgraphs \((r,k,G,\Gamma ,{\mathcal {B}}')\). Let \(c_1\), ..., \(c_N\) be a sequence of nested cycles created after the addition of arcs of weights \(\gamma _1 \le \gamma _2 \le \ldots \le \gamma _N\), respectively, i.e., \({\mathcal {S}}(v_{c_1})\subset {\mathcal {S}}(v_{c_2})\subset \ldots \subset {\mathcal {S}}(v_{c_N})\). Suppose each set of vertices \({\mathcal {S}}(v_{c_l})\) contains a vertex x with an outgoing arc \(x\rightarrow y\) such that \(U_{xy} = U_{\min }(x)\), but \(y\notin {\mathcal {S}}(v_{c_l})\), \(l = 1,\ldots ,N1\). Suppose there is an arc \((i\rightarrow j)\in {\mathcal {A}}\) such that \(i\in {\mathcal {S}}(v_{c_1})\), \(j\notin {\mathcal {S}}(v_{c_N})\). Then \(U^{(N)}_{ij} = U_{ij}U_{\min }(i) + \gamma _{N}\).
Proof
For all \(1\le l\le N1\), we have \(U_{\min }(v_{c_l}) = \gamma _l\). Therefore,
Lemma 10.2
Let \({\mathcal {S}}'\subset {\mathcal {S}} \) be a subset of vertices of a directed graph \(G({\mathcal {S}},{\mathcal {A}})\). Suppose every vertex in \({\mathcal {S}}'\) has at least one outgoing arc. Then

(i)
if all arcs with tails in \({\mathcal {S}}'\) have heads also in \({\mathcal {S}}'\), then there is at least one directed cycle formed by the arcs with tails in \({\mathcal {S}}'\);

(ii)
if the arcs with tails in \({\mathcal {S}}'\) form no directed cycles, then at least one of them must head in \({\mathcal {S}}\backslash {\mathcal {S}}'\).
Proof
Let us select one outgoing arc for each vertex in \({\mathcal {S}}'\) and denote the set of the selected arcs by \({\mathcal {A}}'\). If all arcs in \({\mathcal {A}}'\) head in \({\mathcal {S}}'\), then \({\mathcal {A}}' = {\mathcal {S}}'\) in the graph \(G':=G'({\mathcal {S}}',{\mathcal {A}}')\). Hence, \(G'\) cannot be a directed forest. Hence, it contains at least one cycle which proves Statement (i). Statement (ii) is the negation of (i).
Proof
(Theorem 6.1.) We start with Statement 4 because its proof is the shortest. A vertex i is an absorbing state of \(T_p\) if and only if the weight of minarcs from i in the original graph G is greater than \(\theta _p\). In turn, this happens if and only if i has no outgoing arc in \(\Gamma _{K_p}\), i.e., i is absorbing in \(\Gamma _{K_p}\).
Auxiliary Statement: At the end of steps \(K_p\) and p of Algorithms 1 and 2, respectively, for all \(p\ge 0\) we have: \({\mathcal {B}}'\subseteq {\mathcal {B}}\) and the sets of distinct values in \({\mathcal {B}}'\) and \({\mathcal {B}}\) coincide.
Statements 1, 2, and 3 and the auxiliary statement will be proven by induction in the recursion level r of Algorithm 2.
Basis. The initial graphs \(T_0 = T_0({\mathcal {S}},\emptyset ,\emptyset )\) and \(\Gamma _0=\Gamma _0({\mathcal {S}},\emptyset ,\emptyset )\) in Algorithms 1 and 2 coincide. Furthermore, as the initializations in Algorithms 1 and 2 are complete, \({\mathcal {B}}'\subseteq {\mathcal {B}}\), and the sets of distinct arc weights in the buckets \({\mathcal {B}}'\) and \({\mathcal {B}}\) are the same. This gives us the induction basis.
Induction Assumption. Assume that at step \(p_r\) of Algorithm 2 and the corresponding step \(K_{p_r}\) of Algorithm 1, we have:

\({\mathcal {B}}'\subseteq {\mathcal {B}}\);

the set of distinct arc weights in \({\mathcal {B}}\) and \({\mathcal {B}}'\) are the same;

all closed communicating classes are contracted into single supervertices in \(T^{(r)}_{p_r}\) and \(\Gamma ^{(r')}_{K_{p_r}}\), where \(r'\) is the recursion level in Algorithm 1 at the end of step \(K_{p_r}\); furthermore, the set of vertices of \(T^{(r)}_{p_r}\) and \(\Gamma ^{(r')}_{K_{p_r}}\) coincide, and each supervertex v of \(T^{(r)}_{p_r}\) has the corresponding supervertex \(v'\) of \(\Gamma ^{(r')}_{K_{p_r}}\) such that \({\mathcal {S}}(v) = {\mathcal {S}}(v')\);

\(\Gamma _{K_{p_r}}\) is a subgraph of \(T_{p_r}\);

the sets of distinct values in \(\{\theta _p\}_{p=1}^{p_r}\) and \(\{\gamma _{k}\}_{k=1}^{ K_{p_r}}\) coincide.
To prove the induction step, we need to show that Statements 1, 2, 3, and the auxiliary statement hold up to \(p=p_{r+1}\).
Induction Step.

1.
The induction assumptions imply that all graphs \(\Gamma _{k}\), \(K_{p1}<k\le K_p\), are subgraphs of \(T_p\) for all p such that the recursion levels remain \(r'\) and r in Algorithms 1 and 2, respectively. For such p, the graphs \(T_p^{(r)}\) and \(\Gamma _{K_p}^{(r')}\) are built by adding arcs from buckets \({\mathcal {B}}\) and \({\mathcal {B}}'\), respectively, while no new arcs are added to these buckets and no arc weights are modified. Hence, Statements 1, 2, 3, and the auxiliary statement hold for all such p. Therefore, if no cycles are encountered by Algorithm 1 at steps \(K_{p_r}<k\le K_{p_{r+1}1}\), Statements 1, 2, 3, and the auxiliary statement hold for \(K_{p_r}<k\le K_{p_{r+1}1}\).

2.
Now we show that Statements 1, 2, 3, and the auxiliary statement hold independent of whether or not cycles were encountered in Algorithm 1 at some \(K_{p_r}<k\le K_{p_{r+1}1}\). Note that a cycle can be formed in Algorithm 1 at some step \(K_p<k\le K_{p+1}\) where \(p_r<p<p_{r+1}\) only if an open communicating class is formed at step p of Algorithm 2.
Since no arcs are added to the bucket \({\mathcal {B}}\) by Algorithm 2 at steps \(p_r<p<p_{r+1}\), the graphs \(T_p^{(r)}\) consist of all minarcs of the graph \(G^{(r)}\) of weights \(\le \theta _{p_{r+1}1}\). Now we consider Algorithm 1 for steps \(K_{p_r}<k\le K_{p_{r+1}1}\) and prove Statements 1, 2, 3, and the auxiliary statement by induction in the number of cycles.
Let \(c_1\) be the first cycle created in \(\Gamma ^{(r')}\) at step \(k_1\) after the addition of an arc of weight \(\theta _p\). Since \(\Gamma _{k_1}^{(r')}\) is a subgraph of \(T_{p}^{(r)}\), the cycle \(c_1\) must be a subclass of an open communicating class C created in \(T^{(r)}\) after the addition of a set of arcs of weight \(\theta _p\). Therefore, \(c_1\) is an open communicating class in \(T_p^{(r)}\). This means the set of minarcs with tails in \(c_1\) and heads not in \(c_1\) in not empty. All these minarcs are in \(T_p^{(r)}\), and none of them is in \(\Gamma _{k_1}^{(r')}\). By Lemma 10.1, the weights of these minarcs become \(\theta _p\) after the modification. One of them is picked and added to the bucket \({\mathcal {B}}'\). Hence, this arc will be added to \(\Gamma ^{(r'+1)}\) at some \(k_1<k\le K_p\). This allows us to conclude that at least one more arc of weight \(\theta _p\) will be removed from \({\mathcal {B}}'\) after the cycle \(c_1\) is formed. Hence, \(c_1\) is not a closed communicating class in \(\Gamma _{K_p}^{(r')}\). Therefore, Statements 1, 2, 3 and the auxiliary statement hold for \(K_{p_r}<k\le \min \{k_2,K_{p_{r+1}1}\}\).
Assume that Statements 1, 2, 3, and the auxiliary statement hold up to step \(k_{N} < K_{p_{r+1} 1}\) of Algorithm 1. Let cycle \(c_N\) be encountered at step \(k_N\) in Algorithm 1 after the addition of an arc of weight \(\theta _p\). The set of vertices \({\mathcal {S}}^{(r')}(v_{c_N})\) is a subclass of an open communicating class C of the graph \(T_p\) by the induction assumption. Hence, the set of the minarcs with tails in \({\mathcal {S}}^{(r')}(v_{c_N})\) and heads not in \({\mathcal {S}}^{(r')}(v_{c_N})\) is not empty. All of these arcs are in \(T_p^{(r)}\) but not in \(\Gamma _{k_N}^{(r')}\). By Corollary 10.2, their weights will become \(\theta _p\) during step \(k_N\) of Algorithm 1. One of these minarcs will be added to the bucket \({\mathcal {B}}'\) and then removed from it at some step \(k_N<k\le K_{p}\). Hence, the cycle \(c_N\) cannot be a closed communicating class in \(T_p^{(r)}\). This proves Statements 1, 2, 3, and the auxiliary statement for all \(K_{p_r}<k\le K_{p_{r+1}1}\).

3.
Now we show that Statements 1, 2, 3, and the auxiliary statement hold for \(K_{p_{r+1}1}+1\le k\le K_{p_{r+1}}\). Let C be the closed communicating class formed in the graph \(T_{p_{r+1}}^{(r)}\) after the addition of the set of minarcs of weight \(\theta _{p_{r+1}}\). Therefore, all minarcs from the vertices in C head in C. After contracting C into a single supervertex \(v_C\), the weight of minarcs from it will be
$$\begin{aligned} \min _{i\in C,~j\notin C}[U_{ij}U_{\min }(i) +\theta _{p_{r+1}}] >\theta _{p_{r+1}}. \end{aligned}$$(39)
(Here \(U_{\min }(i)\) is the weight of minarcs from i in the graph \(G^{(r)}\).) Let the recursion level at the end of step \(K_{p_{r+1}1}\) in Algorithm 1 be \(r''\), and \(C'\) be the set of vertices in \(\Gamma ^{(r'')}\) corresponding to C. If no cycles were formed in Algorithm 1 with vertices in C then \(C'=C\). Otherwise, some of the vertices of C are contacted into supervertices in \(\Gamma ^{(r'')}\). In this case, the subset of vertices of C contracted into supervertices is an open communicating subclass of C. Let us show that \(C'\) is a closed communicating class of \(\Gamma _{K_{p_{r+1}}}^{(r'')}\).
Lemma 10.2 implies that the minarcs from \(C'\) all heading in \(C'\) form at least one cycle c. Consider two cases.
Case 1: cycle c includes whole \(C'\). Then by Corollary 10.3, the weight of minarcs from the vertex \(v_c\) in Algorithm 1 will be given by Eq. (39), i.e., the same as it is in Algorithm 2, and one of those minarcs will be added to \({\mathcal {B}}'\). Hence, Statements 1, 2, 3, and the auxiliary statement hold at \(p=p_{r+1}\) and \(K_{p_{r+1}1}<k\le K_{p_{r+1}}\).
Case 2: the cycle c does not include all vertices from \(C'\). Since C does not contain closed communicating subclasses, the set of minarcs in \(G^{(r)}\) with tails in c and heads not in c is not empty. By Corollary 10.3, these minarcs will be the minarcs from \(v_c\), and their weights will be \(\theta _p\). One of these minarcs will be added to \({\mathcal {B}}'\) and then removed. Hence, the minarcs from \((C\backslash {\mathcal {S}}^{(r)}(v_c))\cup \{v_c\}\) head to \((C\backslash {\mathcal {S}}^{(r)}(v_c))\cup \{v_c\}\). By Lemma 10.2, they form at least one cycle c. Again, there are two options: either c includes all vertices of C or not. In the former case, using the argument from Case 1, we prove the induction step. In the latter case, we use the argument from Case 2. Repeating this argument at most \(C1\) number of times (as each new cycle includes at least one more vertex of \(C'\) in comparison with the previous one), we obtain a cycle including all vertices of C.
Repeating this argument for all closed communicating classes formed in \(T^{(r)}\) at step \(p_{r+1}\), we conclude that all closed communicating classes encountered in Algorithm 2 at steps \(p_{r+1}\) will be contracted into single supervertices by both Algorithms 1 and 2, and arcs of the same weight will be added to \({\mathcal {B}}\) and \({\mathcal {B}}'\). Hence, the induction step is proven. This completes the proof of Theorem 6.1.
Rights and permissions
About this article
Cite this article
Gan, T., Cameron, M. A GraphAlgorithmic Approach for the Study of Metastability in Markov Chains. J Nonlinear Sci 27, 927–972 (2017). https://doi.org/10.1007/s0033201693550
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s0033201693550