Skip to main content
Log in

A Graph-Algorithmic Approach for the Study of Metastability in Markov Chains

  • Published:
Journal of Nonlinear Science Aims and scope Submit manuscript

Abstract

Large continuous-time Markov chains with exponentially small transition rates arise in modeling complex systems in physics, chemistry, and biology. We propose a constructive graph-algorithmic approach to determine the sequence of critical timescales at which the qualitative behavior of a given Markov chain changes, and give an effective description of the dynamics on each of them. This approach is valid for both time-reversible and time-irreversible Markov processes, with or without symmetry. Central to this approach are two graph algorithms, Algorithm 1 and Algorithm 2, for obtaining the sequences of the critical timescales and the hierarchies of Typical Transition Graphs or T-graphs indicating the most likely transitions in the system without and with symmetry, respectively. The sequence of critical timescales includes the subsequence of the reciprocals of the real parts of eigenvalues. Under a certain assumption, we prove sharp asymptotic estimates for eigenvalues (including pre-factors) and show how one can extract them from the output of Algorithm 1. We discuss the relationship between Algorithms 1 and 2 and explain how one needs to interpret the output of Algorithm 1 if it is applied in the case with symmetry instead of Algorithm 2. Finally, we analyze an example motivated by R. D. Astumian’s model of the dynamics of kinesin, a molecular motor, by means of Algorithm 2.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16

Similar content being viewed by others

Notes

  1. No proof of Theorem 2.3 was provided in Wentzell (1972). We are not aware of any published proof of this result.

References

  • Astumian, R.D.: Biasing the random walk of a molecular motor. J. Phys.: Condens. Matter 17, S3753 (2005)

    Google Scholar 

  • Arkus, N., Manoharan, V., Brenner, M.P.: Minimal energy clusters of hard spheres with short ranged attractions. Phys. Rev. Lett. 103, 118303 (2009)

    Article  Google Scholar 

  • Arkus, N., Manoharan, V., Brenner, M.P.: Deriving finite sphere packings. SIAM J. Discret. Math. 25, 1860–1901 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  • Berglund, N., Dutercq, S.: The Eyring—Kramers Law for Markovian Jump Processes with Symmetries. J. Theory Prob. First online: 21 May 2015, doi:10.1007/s10959-015-0617-9

  • Bouchet, F., Reygner, J.: Generalization of the Eyring-Kramers transition rate formula to irreversible diffusion processes, Annales Henri Poincare, First online: 11 June 2016, pp. 1–34 arXiv:1507.02104v1

  • Bovier, A., Eckhoff, M., Gayrard, V., Klein, M.: Metastability and low lying spectra in reversible Markov chains. Comm. Math. Phys. 228, 219–255 (2002)

    Article  MathSciNet  MATH  Google Scholar 

  • Bovier, A., Eckhoff, M., Gayrard, V., Klein, M.: Metastability in reversible diffusion processes 1. Sharp estimates for capacities and exit times. J. Eur. Math. Soc. 6, 399–424 (2004)

    Article  MATH  Google Scholar 

  • Bovier, A., Gayrard, V., Klein, M.: Metastability in reversible diffusion processes 2. Precise estimates for small eigenvalues. J. Eur. Math. Soc. 7, 69–99 (2005)

    Article  MATH  Google Scholar 

  • Bovier, A., den Hollander, F.: Metastability: A Potential-Theoretic Approach. Springer, Berlin (2016)

    MATH  Google Scholar 

  • Cameron, M.K.: Computing Freidlin’s cycles for the overdamped Langevin dynamics. J. Stat. Phys. 152(3), 493–518 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  • Cameron, M.K.: Computing the asymptotic spectrum for networks representing energy landscapes using the minimal spanning tree, M. Cameron. Netw. Heterog. Media 9, 3 (2014)

    Article  MathSciNet  Google Scholar 

  • Cameron, M.K., Gan, T.: Spectral analysis and clustering of large stochastic networks. Application to the Lennard-Jones-75 cluster. Mol. Simul. 42(16), 1410–1428 (2016)

    Article  Google Scholar 

  • Chu, Y.J., Liu, T.H.: On the shortest arborescence of a directed graph. Sci. Sin. 14, 1396–1400 (1965)

    MathSciNet  MATH  Google Scholar 

  • Edmonds, J.: Optimum branchings. J. Res. Natl. Bur. Stand.: B Math. Math. Phys. 71 B, 4 (1967)

    MathSciNet  MATH  Google Scholar 

  • Freidlin, M.I.: Sublimiting distributions and stabilization of solutions of parabolic equations with small parameter. Sov. Math. Dokl. 18(4), 1114–1118 (1977)

    MATH  Google Scholar 

  • Freidlin, M.I.: Quasi-deterministic approximation, metastability and stochastic resonance. Phys. D 137, 333–352 (2000)

    Article  MathSciNet  MATH  Google Scholar 

  • Freidlin, M.: On stochastic perturbations of dynamical systems with a “rough” symmetry. Hierarchy of Markov chains. J. Stat. Phys. 157(6), 1031–1045 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  • Freidlin, M.I., Wentzell, A.D.: Random Perturbations of Dynamical Systems, 3rd edn. Springer, Berlin (2012)

    Book  MATH  Google Scholar 

  • Gaveau, B., Schulman, L.S.: Theory of nonequilibrium first-order phase transitions for stochastic dynamics. J. Phys. 33, 4837–4850 (2000)

    MathSciNet  Google Scholar 

  • Gerschgorin, S.: Uber die abgrenzung der eigenwerte einer matrix. Izv. Akad. Nauk. USSR Otd. Fiz.-Mat. Nauk 7, 749–754 (1931)

    MATH  Google Scholar 

  • Holmes-Cerfon, M., Gortler, S.J., Brenner, M.P.: A geometrical approach to computing free-energy landscapes from short-ranged potentials. Proc. Natl. Acad. Sci. 110(1), E5–E14 (2013)

    Article  Google Scholar 

  • Meng, G., Arkus, N., Brenner, M.P., Manoharan, V.: The free energy landscape of hard sphere clusters. Science 327, 560 (2010)

    Article  Google Scholar 

  • Metzner, P., Schuette, Ch., Vanden-Eijnden, E.: Transition path theory for Markov jump processes. SIAM Multiscale Model Simul. 7, 1192–1219 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  • Norris, J.P.: Markov Chains. Cambridge University Press, Cambridge (1997)

    Book  MATH  Google Scholar 

  • Wales, D.J.: Discrete path sampling. Mol. Phys. 100, 3285–3306 (2002)

    Article  Google Scholar 

  • Wales, D.J.: Energy Landscapes: Applications to Clusters, Biomolecules and Glasses. Cambridge University Press, Cambridge (2003)

    Google Scholar 

  • Wales, D.J.: Energy landscapes: calculating pathways and rates. Int. Rev. Chem. Phys. 25(1–2), 237–282 (2006)

    Article  Google Scholar 

  • Wentzell, A.D.: On the asymptotics of eigenvalues of matrices with elements of order \(exp \{ -V_{ij} / 2 (\varepsilon ^2) \} \), (in Russian) Dokl. Akad. Nauk SSSR 202, 263–265 (1972)

    Google Scholar 

Download references

Acknowledgements

We would like to thank Mr. Weilin Li for valuable discussions in the early stage of this work. We are grateful to Prof. C. Jarzynski and Prof. M. Fisher for suggesting us to consider molecular motors as an example of a natural time-irreversible system with transition rates in the exponential form. We also thank the anonymous reviewers for their valuable feedback and comments. This work was partially supported by NSF Grants 1217118 and 1554907.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Maria Cameron.

Additional information

Communicated by Charles R. Doering.

Appendices

Appendix 1: Proof of Theorem 2.4

The following notations will be used throughout the proof.

  • \({\mathcal {G}}(n-l)\) is the set of all W-graphs with \(n-l\) sinks and l arcs for the graph \(G({\mathcal {S}},{\mathcal {A}},{\mathcal {U}})\). The set of subgraphs of G with exactly l arcs emanated from distinct vertices will be denoted by \({\mathcal {H}}(l)\). Note that graphs in \({\mathcal {H}}(l)\) might contain directed cycles. Therefore, \({\mathcal {G}}(n-l) \subsetneq {\mathcal {H}}(l)\).

  • \({\mathcal {S}}^l \) is the set of all ordered selections of l vertices out of n (in the combinatorial sense):

    $$\begin{aligned} {\mathcal {S}}^l: = \{ i_1 i_2 \cdots i_l ~|~ i_m \in \{ 1, 2 , \cdots , n-1\}, 1 \le m \le l , {\text { distinct}} \}. \end{aligned}$$

    Note that \(|{\mathcal {S}}^l| = n(n-1)\ldots (n-l+1)\).

  • \({\mathcal {O}}^l \) is the set of combinations of l vertices out of n (in the combinatorial sense). Each combination in \({\mathcal {O}}^l\) is ordered so that \(i_1<i_2<\ldots <i_l\), i.e.,

    $$\begin{aligned} {\mathcal {O}}^l := \{ i_1 i_2 \cdots i_l \in {\mathcal {S}}^l ~|~ i_1< i_2< \cdots < i_l \}. \end{aligned}$$

    Note that \(|{\mathcal {O}}^l| = \left( \begin{array}{c}n\\ l\end{array}\right) \).

  • Every sequence \(j_1,\ldots j_l\) in \({\mathcal {S}}^l\) can be permuted to an ordered sequence \(\{i_1,\ldots , i_l\}\) in \({\mathcal {O}}^l\). This defines the permutation map \(\sigma \): \(\sigma (j_1,\ldots ,j_l) = (i_1,\ldots ,i_l)\). Note that the map \(\sigma \) is onto but not one to one.

  • We will call sequences in \({\mathcal {S}}^l\) equivalent if and only if they are mapped to the same sequence in \({\mathcal {O}}^l\). Therefore, \({\mathcal {O}}^l={\mathcal {S}}^l/\sim \), meaning that \({\mathcal {O}}^l\) is the set of equivalence classes in \({\mathcal {S}}^l\).

  • For any \(i_1 \cdots i_l \in {\mathcal {O}}^l\), we will denote by \(L_{ i_1 \cdots i_l }\) the \(l\times l\) submatrix of L consisting of the intersection of \( i_1, \cdots , i_l \) rows and columns of L.

Proof

The proof of Theorem 2.4 consists of the following three steps.

Step 1: :

The characteristic polynomial of the generator matrix L is

$$\begin{aligned} P_L(t) := {\text {det}} ( t I - L) = t^n + \sum _{l = 1}^{n-1} C_l t^{n-l} \end{aligned}$$

whose coefficients \(C_l\) are given by

$$\begin{aligned} C_l&= \sum _{i_1 \cdots i_l \in {\mathcal {O}}^l} {\text {det}} ( - L_{i_1 \cdots i_l} )\nonumber \\&= (-1)^l \sum _{i_1 \cdots i_l \in {\mathcal {O}}^l} ~\sum _{ \begin{array}{c} j_1 \cdots j_l \in {\mathcal {S}}^l\\ j_1 \cdots j_l\sim i_1 \cdots i_l \end{array}} (-1)^{| \sigma (j_1, \cdots , j_l) |} L_{i_1 j_1} \cdots L_{i_l j_l}, \end{aligned}$$
(23)

where \(| \sigma (j_1, \cdots , j_l) |\) is the number of inversions in \(\sigma (j_1, \cdots , j_l) \).

Step 2: :

Using the zero row sum property of the generator matrix L, Eq. (23) can be further simplified resulting at the following more compact expression:

$$\begin{aligned} C_l = \sum _{g \in {\mathcal {G}}(n-l)} \Pi (g), \quad \mathrm{where}\quad \Pi (g) = \prod _{(i\rightarrow j)\in g}L_{ij}. \end{aligned}$$
(24)

The key idea of the derivation of Eq. (24) is to show that

$$\begin{aligned} C_l = (-1)^l \sum _{g \in {\mathcal {H}}(l) } a_g \Pi (g), \quad \mathrm{where}\quad a_g = {\left\{ \begin{array}{ll} (-1)^l, &{} {\text { if }} g \in {\mathcal {G}}(n-l),\\ 0, &{} {\text { if }} g \in {\mathcal {H}}(l) {\setminus } {\mathcal {G}}(n-l). \end{array}\right. } \end{aligned}$$
(25)
Step 3: :

Comparing the coefficient of the characteristic polynomial

$$\begin{aligned} P_L(t) = {\text {det}} ( t I - L ) = t^n + \sum _{l = 1}^{n-1} C_l t^{n-l} = t (t + \lambda _1) \cdots (t + \lambda _{n-1}), \end{aligned}$$

we obtain the following estimates for eigenvalues:

$$\begin{aligned} \lambda _m = \frac{ \Pi (g^{*}_{m}) }{ \Pi (g^{*}_{m+1}) } (1 + o(1) ). \end{aligned}$$
(26)

Eq. (26) is equivalent to Eq. (7).

Now we elaborate each step.

Step 1:

Consider the following polynomial in n variables \(t_1, t_2, \cdots , t_n\):

$$\begin{aligned} P_L( t_1, t_2, \cdots , t_n)&:= {\text {det}} ( {\text {diag}} \{ t_1, t_2, \cdots , t_n\} - L)\nonumber \\&= t_1 \cdots t_n + \sum _{l = 1}^{n - 1} ~\sum _{i_1 \cdots i_l \in {\mathcal {O}}^l} C_{i_1 \cdots i_l} t_{i_{l+1}} \cdots t_{i_n} \end{aligned}$$
(27)

where \(\{ i_{l+1}, \cdots , i_n \} = \{ 1, \cdots , n\} {\setminus } \{ i_1, \cdots , i_l \}\). Replacing all \(t_i\) by t, we recover the characteristic polynomial \(P_L(t)\) where

$$\begin{aligned} C_l = \sum _{i_1 \cdots i_l \in {\mathcal {O}}^l} C_{ i_1 \cdots i_{l} }. \end{aligned}$$
(28)

The term \(C_{i_1 \cdots i_l} t_{i_{l+1}} \cdots t_{i_n}\) in Eq. (27) is obtained by picking \(t_{i_{l+1}}\), ..., \(t_{i_n}\) from the diagonal entries in rows of the matrix \({\text {diag}} \{ t_1, t_2, \cdots , t_n\} - L\) and multiplying them by the determinant of \(-{L_{i_1\ldots i_l}}\). Hence,

$$\begin{aligned} C_{ i_1 \cdots i_l } = {\text {det}} \left( -L_{ i_1 \cdots i_l } \right) . \end{aligned}$$
(29)

Combining Eqs. (28) and (29) and applying the Leibniz formula for determinants, we obtain Eq. (23).

Step 2:

Consider the product terms \(L_{i_1 j_1} \cdots L_{i_l j_l}\), \( i_1 \cdots i_l \in {\mathcal {O}}^l, i_1 \cdots i_l \sim j_1 \cdots j_l\) in Eq. (23). Suppose the sequences \(i_1 \cdots i_l\) and \(j_1 \cdots j_l\) agree on exactly s entries and differ at \(l-s\) entries, i.e.,

$$\begin{aligned} i_{m_1} =&j_{m_1}, \cdots , i_{m_s} = j_{m_s}, \end{aligned}$$
(30)
$$\begin{aligned} i_{m_{s+1}} \ne&j_{m_{s+1}}, \cdots , i_{m_l} \ne j_{m_l}, {\text { where }} i_{m_{s+1}} \cdots i_{m_l} \sim j_{m_{s+1}} \cdots j_{m_l} \in {\mathcal {S}}^{l-s}. \end{aligned}$$
(31)

Note that s can be any number between 0 and l except for \(l-1\). Using the zero sum property of L, we obtain

$$\begin{aligned}&L_{i_1 j_1} \cdots L_{i_l j_l} = L_{ i_{m_1} i_{m_1} } \cdots L_{ i_{m_s} i_{m_s} } L_{ i_{m_{s+1}} j_{m_{s+1}} } \cdots L_{ i_{m_l} j_{m_l} }\nonumber \\&= \left( - \sum _{d_1 \ne i_{m_1} } L_{i_{m_1} d_1} \right) \cdots \left( - \sum _{d_s \ne i_{m_s} } L_{i_{m_s} d_s} \right) L_{ i_{m_{s+1}} j_{m_{s+1}} } \cdots L_{ i_{m_l} j_{m_l} } \nonumber \\&= (-1)^s \sum _{d_1 \ne i_{m_1}, \cdots , d_s \ne i_{m_s} } L_{ i_{m_1} d_1 } \cdots L_{ i_{m_s} d_s } L_{ i_{m_{s+1}} j_{m_{s+1}} } \cdots L_{ i_{m_l} j_{m_l} }. \end{aligned}$$
(32)

It is helpful to consider the collection of graphs g with n vertices and sets of arcs

$$\begin{aligned} \{ (i_{m_1} \rightarrow d_1) , \cdots , (i_{m_s} \rightarrow d_s), (i_{m_{s+1} } \rightarrow j_{m_{s+1}}), \cdots , ( i_{m_l} \rightarrow j_{m_l} ) \} \end{aligned}$$

corresponding to the products

$$\begin{aligned} L_{ i_{m_1} d_1 } \cdots L_{ i_{m_s} d_s } L_{ i_{m_{s+1}} j_{m_{s+1}} } \cdots L_{ i_{m_l} j_{m_l} } \end{aligned}$$

in Eq. (32). Each of the vertices \(i_1,\ldots ,i_l\) of g has exactly one outgoing arc, while the other vertices have no outgoing arcs. Hence, the graphs g belong to the set of graphs \({\mathcal {H}}(l)\). If \(s \le l-2\), each of the vertices \(\{ i_{m_{s+1}}, \cdots , i_{m_l} \}\) of the graph g has exactly one outgoing arc heading to \(\{ i_{m_{s+1}}, \cdots , i_{m_l} \}\), and exactly one incoming arc from \(\{ i_{m_{s+1}}, \cdots , i_{m_l} \}\). Therefore, the arcs \(\{(i_{m_{s+1} } \rightarrow j_{m_{s+1}}), \cdots , ( i_{m_l} \rightarrow j_{m_l} ) \}\) necessarily form cycles in g. The arcs \(\{ (i_{m_1} \rightarrow d_1) , \cdots , (i_{m_s} \rightarrow d_s)\}\) do not necessarily form cycles. Therefore, Eq. (23) can be rewritten using Eq. (32) and the graphs \({\mathcal {H}}(l)\) as

$$\begin{aligned} C_l&= \sum _{i_1 \cdots i_l \in {\mathcal {O}}^l} {\text {det}} ( - L_{i_1 \cdots i_l} ) = (-1)^l \sum _{i_1 \cdots i_l \in {\mathcal {O}}^l} ~\sum _{ \begin{array}{c} j_1 \cdots j_l \in {\mathcal {S}}^l\\ j_1 \cdots j_l\sim i_1 \cdots i_l \end{array}} (-1)^{| \sigma (j_1, \cdots , j_l) |} L_{i_1 j_1} \cdots L_{i_l j_l} \nonumber \\&= (-1)^l \sum _{i_1 \cdots i_l \in {\mathcal {O}}^l} ~\sum _{ \begin{array}{c} j_1 \cdots j_l \in {\mathcal {S}}^l\\ j_1 \cdots j_l\sim i_1 \cdots i_l \end{array}} (-1)^{| \sigma (j_1, \cdots , j_l) |} (-1)^s L_{ i_{m_{s+1}} j_{m_{s+1}} } \nonumber \\&\qquad \cdots L_{ i_{m_l} j_{m_l} } \sum _{ \begin{array}{c} d_1\ne i_{m_1}\\ \cdots \\ d_s \ne i_{m_s} \end{array}} L_{ i_{m_1} d_1 } \cdots L_{ i_{m_s} d_s } \nonumber \\&=(-1)^l \sum _{g\in {\mathcal {H}}(l)}a_g\Pi (g), \end{aligned}$$
(33)

where the factors \(a_g\) will be determined below.

First we assume that the graph g with the set of arcs \(\{ (i_1 \rightarrow j_1), \cdots , (i_l \rightarrow j_l) \}\) contains no cycles, i.e., \( g \in {\mathcal {G}} (n-l)\subset {\mathcal {H}}(l)\). This can happen only if \(s=l\) in Eq. (33), i.e., \(i_1=j_1\), ...,\(i_l=j_l\). Therefore, \((-1)^{| \sigma (j_1, \cdots , j_l) |} =(-1)^0 = 1\), and

$$\begin{aligned} L_{i_1 i_1} \cdots L_{i_l i_l}= & {} \left( - \sum _{d_1 \ne i_1} L_{i_1 d_1} \right) \cdots \left( - \sum _{d_l \ne i_l} L_{i_l d_l} \right) \\= & {} (-1)^l \sum _{d_1 \ne i_1, \cdots , d_l \ne i_l} L_{i_1 d_1} \cdots L_{i_l d_l}. \end{aligned}$$

Hence, the product \( L_{i_1 d_1} \cdots L_{i_l d_l}\) corresponding to the graph g enters Eq. (33) only once. Therefore, \(a_{g} = (-1)^l\) for all \(g \in {\mathcal {G}} (n-l)\).

Now we assume that the set of arcs \(\{ (i_1 \rightarrow x_1), \cdots , (i_l \rightarrow x_l) \}\) of g contains \(N\ge 1\) cycles, i.e., \(g \in {\mathcal {H}}(l) {\setminus } {\mathcal {G}}(n-l)\). In this case, the product \( L_{i_1 x_1} \cdots L_{i_l x_l}\), \(i_p\ne x_p\), \(1\le p\le l\), enters Eq. (33) \(2^N\) times either with the plus or minus sign. The number \(2^N\) comes from the fact that each cycle in g can be formed in two ways: (i) by arcs corresponding to off-diagonal factors \(L_{i_pj_p}\), \(i_p\ne j_p\), in Eq. (33), or (ii) by arcs originating from the replacement of diagonal factors \(L_{i_pi_p}\) in Eq. (33) with \(-\sum _{d_p\ne i_p}L_{i_pd_p}\). We will prove that \( L_{i_1 x_1} \cdots L_{i_l x_l}\), \(i_p\ne x_p\), \(1\le p\le l\), enters Eq. (33) with sign plus the same number of times as it does with sign minus. This will imply that \(a_g=0\). To do so, we show that for each entry of \( L_{i_1 x_1} \cdots L_{i_l x_l}\), one can uniquely define another entry with an opposite sign. Let c be a cycle in g. Consider two terms in Eq. (33) containing the product \(L_{i_1x_1}\ldots L_{i_lx_l}\) that correspond to possibilities (i) and (ii) for the origin of a selected cycle c in g, while all other factors corresponding to the arcs not in c originate in the same way. Let \(\sigma _1(j^1_1,\ldots j^1_l)\) and \(\sigma _2(j^2_1,\ldots ,j^2_l)\) be the permutations in Eq. (33) corresponding to (i) and (ii), respectively, and \(s_1\) and \(s_2\) be the corresponding numbers of fixed entries in \(\sigma _1\) and \(\sigma _2\), respectively. If the cycle c has length |c|, then

$$\begin{aligned} (-1)^{|\sigma _1|} = (-1)^{|\sigma _2|}(-1)^{|c|-1},\quad (-1)^{s_1} = (-1)^{s_2+|c|}. \end{aligned}$$
(34)

Here we have used the known combinatorial fact that the parity of a permutation consisting of cycles \(c_1\), ..., \(c_k\) is \((-1)^{\sum _{j=1}^k|c_j|-1}\). Therefore, the signs \((-1)^l(-1)^{|\sigma _1|}(-1)^{s_1}\) and \((-1)^l(-1)^{|\sigma _2|}(-1)^{s_2}\) preceding the corresponding products in Eq. (33) are opposite. This implies that all products corresponding to any graph \(g\in {\mathcal {H}}(l) {\setminus } {\mathcal {G}}(n-l)\) cancel out, i.e., \(a_g=0\).

Therefore, \(C_l = \sum _{g\in {\mathcal {G}}(n-l)}\Pi (g)\), i.e., Eq. (24) holds. An example illustrating this cancellation is given at the end of “Appendix 1.”

Step 3:

Let us write the characteristic polynomial \(P_L(t)\) in the form

$$\begin{aligned} P_L(t) = {\text {det}} ( tI -L ) = t ( t + \lambda _1 ) \cdots ( t + \lambda _{n-1} ), \end{aligned}$$
(35)

where \(|\lambda _1|<|\lambda _2|<\ldots <|\lambda _{n-1}|\). A simple calculation gives

$$\begin{aligned} C_l = \sum _{i_1 \cdots i_r \in {\mathcal {O}}^l } \lambda _{i_1} \cdots \lambda _{i_l}. \end{aligned}$$
(36)

Comparing Eqs. (24) and (36), we obtain

$$\begin{aligned} \frac{C_l}{ C_{l-1}} =\frac{\sum _{i_1 \cdots i_l \in {\mathcal {O}}^l } \lambda _{i_1} \ldots \lambda _{i_l}}{\sum _{i_1 \cdots i_{l-1} \in {\mathcal {O}}^{l-1} } \lambda _{i_1} \ldots \lambda _{i_{l-1}}} =\frac{\sum _{g\in {\mathcal {G}}(n-l)}\Pi (g)}{\sum _{g\in {\mathcal {G}}(n-l+1)}\Pi (g)},\quad l\le l\le n-1. \end{aligned}$$
(37)

Since \(L_{ij}\) are of the form \(\kappa _{ij}\exp (-U_{ij}/\varepsilon )\), the sums in the enumerators and the denominators are dominated by their largest summands in Eq. (37). According to Assumption 3, all optimal W-graphs are unique. Therefore,

$$\begin{aligned} \frac{C_l}{ C_{l-1}} = \lambda _{n-l}(1+o(1)) = \frac{\Pi (g^{*}_{n-l}) }{\Pi (g^{*}_{n-l+1}) } (1+o(1)), \end{aligned}$$
(38)

and Eq. (26) immediately follows.

Example 9.1

Let us illustrate the cancellation of terms in Eq. (33). Let \(n=4\), \(l=3\), and \(\{i_1,i_2,i_3\}=\{1,2,3\}\). Then the inner sum in Eq. (33) becomes

$$\begin{aligned} \begin{array}{lll} &{} &{}\sum \limits _{(j_1,j_2,j_3)\sim (1,2,3)}(-1)^{|\sigma (j_1,j_2,j_3)|}L_{1j_1}L_{2j_2}L_{3j_3} = \\ &{}(j_1,j_2,j_3) = (1,2,3):~\sigma = 0,~s = 3:\qquad &{}-(L_{12}+L_{13}+L_{14})(L_{21}+L_{23}+L_{24})\\ &{} &{} \qquad (L_{31}+L_{32}+L_{34})\\ &{}(j_1,j_2,j_3) = (1,3,2):~\sigma =1,~s = 1:\qquad &{}+(L_{12}+L_{13}+L_{14})L_{23}L_{32} \\ &{}(j_1,j_2,j_3) = (2,1,3):~\sigma =1,~s = 1:\qquad &{}+L_{12}L_{21}(L_{31}+L_{32}+L_{34}) \\ &{}(j_1,j_2,j_3) = (2,3,1):~\sigma =2,~s = 0:\qquad &{}+L_{12}L_{23}L_{31} \\ &{}(j_1,j_2,j_3) = (3,2,1):~\sigma =1,~s = 1:\qquad &{}+L_{13}(L_{21}+L_{23}+L_{24})L_{31} \\ &{}(j_1,j_2,j_3) = (3,1,2):~\sigma =2,~s = 0:\qquad &{}+L_{13}L_{21}L_{32}. \end{array} \end{aligned}$$

For each product term of the form \(L_{ix_1}L_{2x_2}L_{3x_3}\), \(x_q\in \{1,2,3,4\}\), one can draw a graph with the set of vertices \({\mathcal {S}} = \{1,2,3,4\}\) and the set of arcs \({\mathcal {A}} = \{1\rightarrow x_1,2\rightarrow x_2,3\rightarrow x_3,4\rightarrow x_4\}\). One can check that all terms corresponding to graphs with no cycles are encountered just once and only in the terms originating from \((j_1,j_2,j_3) = (1,2,3)\). All of them are preceded by the sign “-.” This corresponds to \(a_g = (-1)^l = (-1)^3\). On the contrary, each term corresponding to a graph with cycles (in this example, there can be at most \(N=1\) cycle) is encountered exactly twice (\(2^1 = 2\)): once it comes from the product corresponding to \((j_1,j_2,j_3) = (1,2,3)\) with sign “-,” and once it comes from some nonidentical permutation with sign “+.” Hence, all of such term cancel out.

Appendix 2: Proof of Theorem 6.1

Prior to start proving Theorem 6.1, we prove some auxiliary facts and introduce some use useful definitions. The proof of Theorem 6.1 will exploit the following lemmas.

Lemma 10.1

Suppose the function FindTgraphs is run on a graph \(G({\mathcal {S}},{\mathcal {A}},{\mathcal {U}})\) satisfying Assumptions 1 and 2: FindTgraphs \((r,k,G,\Gamma ,{\mathcal {B}}')\). Let c be a cycle detected in the graph \(\Gamma \) at step k. Suppose the weights of all outgoing arcs with tails in c and heads not in c are modified according to the update rule

$$\begin{aligned} U^{new}_{ij} = U_{ij} -U_{\mu (i)} +\gamma _k. \end{aligned}$$

Then \(U_{ij}^{new} \ge \gamma _k\) for all \(i\in c\), \(j \notin c\), and \(U_{ij}^{new} = \gamma _k\) if and only if \(U_{ij}=U_{\mu (i)}\).

Proof

The fact that \(U_{ij}^{new} \ge \gamma _k\) for all \(i\in c\) and \(j \notin c\) follows from the fact that \(U_{ij}\ge U_{\mu (i)}\). The equality takes place if and only if \(U_{\mu (i)} = U_{ij}\), i.e., if \(i\rightarrow j\) is another min-arc from i.

Corollary 10.1

Suppose the function FindSymTgraphs is run on a graph \(G({\mathcal {S}},{\mathcal {A}},{\mathcal {U}})\) satisfying Assumptions 1 and 2: FindSymTgraphs \((r,p,G,T,{\mathcal {B}})\). Let C be a closed communicating class detected in the graph T at step p. Suppose the weights of all outgoing arcs with tails in C and heads not in C are modified according to the update rule

$$\begin{aligned} U^{new}_{ij} = U_{ij} - U_{\min }(i) +\theta _p. \end{aligned}$$

Then \(U_{ij}^{new} > \theta _p\) for all \(i\in C\), \(j \notin C\).

We will denote by \({\mathcal {S}}(v_{c})\) the subset of vertices of the original graph \(G({\mathcal {S}},{\mathcal {A}},{\mathcal {U}})\) contracted into the super-vertex \(v_{c}\).

Corollary 10.2

Suppose the function FindTgraphs is run on a graph \(G({\mathcal {S}},{\mathcal {A}},{\mathcal {U}})\) satisfying Assumptions 1 and 2: FindTgraphs \((r,k,G,\Gamma ,{\mathcal {B}}')\). Let \(c_1\), ..., \(c_N\) be a sequence of cycles created after the addition of arcs of weights \(\gamma _1 \le \gamma _2 \le \ldots \le \gamma _N\), respectively, such that \({\mathcal {S}}(v_{c_N})\supset {\mathcal {S}}(v_{c_l})\) for all \(l < N\). Let i and j be vertices of G such that \((i\rightarrow j)\in {\mathcal {A}}\) and \(i\in {\mathcal {S}}(v_{c_l})\), \(l < N\), and \(j\notin {\mathcal {S}}(v_{c_N})\). Let \(U^{(l)}\) the (possibly modified) weight of the arc \(i\rightarrow j\) after the creation of the cycle \(c_l\). Then \(U^{(N)}_{ij} \ge \gamma _N\) and \(U^{(N)}_{ij} = \gamma _N\) if and only if \((i\rightarrow j)\) is a min-arc from i.

Corollary 10.3

Suppose the function FindTgraphs is run on a graph \(G({\mathcal {S}},{\mathcal {A}},{\mathcal {U}})\) satisfying Assumptions 1 and 2: FindTgraphs \((r,k,G,\Gamma ,{\mathcal {B}}')\). Let \(c_1\), ..., \(c_N\) be a sequence of nested cycles created after the addition of arcs of weights \(\gamma _1 \le \gamma _2 \le \ldots \le \gamma _N\), respectively, i.e., \({\mathcal {S}}(v_{c_1})\subset {\mathcal {S}}(v_{c_2})\subset \ldots \subset {\mathcal {S}}(v_{c_N})\). Suppose each set of vertices \({\mathcal {S}}(v_{c_l})\) contains a vertex x with an outgoing arc \(x\rightarrow y\) such that \(U_{xy} = U_{\min }(x)\), but \(y\notin {\mathcal {S}}(v_{c_l})\), \(l = 1,\ldots ,N-1\). Suppose there is an arc \((i\rightarrow j)\in {\mathcal {A}}\) such that \(i\in {\mathcal {S}}(v_{c_1})\), \(j\notin {\mathcal {S}}(v_{c_N})\). Then \(U^{(N)}_{ij} = U_{ij}-U_{\min }(i) + \gamma _{N}\).

Proof

For all \(1\le l\le N-1\), we have \(U_{\min }(v_{c_l}) = \gamma _l\). Therefore,

$$\begin{aligned} U^{(1)}_{ij}&= U_{ij}-U_{\min }(i) + \gamma _1,\\ U^{(2)}_{ij}&= U_{ij}^{(1)} - U_{\min }(v_{c_1}) + \gamma _2 = U_{ij}-U_{\min }(i) + \gamma _2,\\&\ldots \\ U^{(N)}_{ij}&= U_{ij}^{(N-1)} - U_{\min }(v_{c_{N-1}}) + \gamma _N = U_{ij}-U_{\min }(i) + \gamma _N. \end{aligned}$$

Lemma 10.2

Let \({\mathcal {S}}'\subset {\mathcal {S}} \) be a subset of vertices of a directed graph \(G({\mathcal {S}},{\mathcal {A}})\). Suppose every vertex in \({\mathcal {S}}'\) has at least one outgoing arc. Then

  1. (i)

    if all arcs with tails in \({\mathcal {S}}'\) have heads also in \({\mathcal {S}}'\), then there is at least one directed cycle formed by the arcs with tails in \({\mathcal {S}}'\);

  2. (ii)

    if the arcs with tails in \({\mathcal {S}}'\) form no directed cycles, then at least one of them must head in \({\mathcal {S}}\backslash {\mathcal {S}}'\).

Proof

Let us select one outgoing arc for each vertex in \({\mathcal {S}}'\) and denote the set of the selected arcs by \({\mathcal {A}}'\). If all arcs in \({\mathcal {A}}'\) head in \({\mathcal {S}}'\), then \(|{\mathcal {A}}'| = |{\mathcal {S}}'|\) in the graph \(G':=G'({\mathcal {S}}',{\mathcal {A}}')\). Hence, \(G'\) cannot be a directed forest. Hence, it contains at least one cycle which proves Statement (i). Statement (ii) is the negation of (i).

Proof

(Theorem 6.1.) We start with Statement 4 because its proof is the shortest. A vertex i is an absorbing state of \(T_p\) if and only if the weight of min-arcs from i in the original graph G is greater than \(\theta _p\). In turn, this happens if and only if i has no outgoing arc in \(\Gamma _{K_p}\), i.e., i is absorbing in \(\Gamma _{K_p}\).

Auxiliary Statement: At the end of steps \(K_p\) and p of Algorithms 1 and 2, respectively, for all \(p\ge 0\) we have: \({\mathcal {B}}'\subseteq {\mathcal {B}}\) and the sets of distinct values in \({\mathcal {B}}'\) and \({\mathcal {B}}\) coincide.

Statements 1, 2, and 3 and the auxiliary statement will be proven by induction in the recursion level r of Algorithm 2.

Basis. The initial graphs \(T_0 = T_0({\mathcal {S}},\emptyset ,\emptyset )\) and \(\Gamma _0=\Gamma _0({\mathcal {S}},\emptyset ,\emptyset )\) in Algorithms 1 and 2 coincide. Furthermore, as the initializations in Algorithms 1 and 2 are complete, \({\mathcal {B}}'\subseteq {\mathcal {B}}\), and the sets of distinct arc weights in the buckets \({\mathcal {B}}'\) and \({\mathcal {B}}\) are the same. This gives us the induction basis.

Induction Assumption. Assume that at step \(p_r\) of Algorithm 2 and the corresponding step \(K_{p_r}\) of Algorithm 1, we have:

  • \({\mathcal {B}}'\subseteq {\mathcal {B}}\);

  • the set of distinct arc weights in \({\mathcal {B}}\) and \({\mathcal {B}}'\) are the same;

  • all closed communicating classes are contracted into single super-vertices in \(T^{(r)}_{p_r}\) and \(\Gamma ^{(r')}_{K_{p_r}}\), where \(r'\) is the recursion level in Algorithm 1 at the end of step \(K_{p_r}\); furthermore, the set of vertices of \(T^{(r)}_{p_r}\) and \(\Gamma ^{(r')}_{K_{p_r}}\) coincide, and each super-vertex v of \(T^{(r)}_{p_r}\) has the corresponding super-vertex \(v'\) of \(\Gamma ^{(r')}_{K_{p_r}}\) such that \({\mathcal {S}}(v) = {\mathcal {S}}(v')\);

  • \(\Gamma _{K_{p_r}}\) is a subgraph of \(T_{p_r}\);

  • the sets of distinct values in \(\{\theta _p\}_{p=1}^{p_r}\) and \(\{\gamma _{k}\}_{k=1}^{ K_{p_r}}\) coincide.

To prove the induction step, we need to show that Statements 1, 2, 3, and the auxiliary statement hold up to \(p=p_{r+1}\).

Induction Step.

  1. 1.

    The induction assumptions imply that all graphs \(\Gamma _{k}\), \(K_{p-1}<k\le K_p\), are subgraphs of \(T_p\) for all p such that the recursion levels remain \(r'\) and r in Algorithms 1 and 2, respectively. For such p, the graphs \(T_p^{(r)}\) and \(\Gamma _{K_p}^{(r')}\) are built by adding arcs from buckets \({\mathcal {B}}\) and \({\mathcal {B}}'\), respectively, while no new arcs are added to these buckets and no arc weights are modified. Hence, Statements 1, 2, 3, and the auxiliary statement hold for all such p. Therefore, if no cycles are encountered by Algorithm 1 at steps \(K_{p_r}<k\le K_{p_{r+1}-1}\), Statements 1, 2, 3, and the auxiliary statement hold for \(K_{p_r}<k\le K_{p_{r+1}-1}\).

  2. 2.

    Now we show that Statements 1, 2, 3, and the auxiliary statement hold independent of whether or not cycles were encountered in Algorithm 1 at some \(K_{p_r}<k\le K_{p_{r+1}-1}\). Note that a cycle can be formed in Algorithm 1 at some step \(K_p<k\le K_{p+1}\) where \(p_r<p<p_{r+1}\) only if an open communicating class is formed at step p of Algorithm 2.

    Since no arcs are added to the bucket \({\mathcal {B}}\) by Algorithm 2 at steps \(p_r<p<p_{r+1}\), the graphs \(T_p^{(r)}\) consist of all min-arcs of the graph \(G^{(r)}\) of weights \(\le \theta _{p_{r+1}-1}\). Now we consider Algorithm 1 for steps \(K_{p_r}<k\le K_{p_{r+1}-1}\) and prove Statements 1, 2, 3, and the auxiliary statement by induction in the number of cycles.

    Let \(c_1\) be the first cycle created in \(\Gamma ^{(r')}\) at step \(k_1\) after the addition of an arc of weight \(\theta _p\). Since \(\Gamma _{k_1}^{(r')}\) is a subgraph of \(T_{p}^{(r)}\), the cycle \(c_1\) must be a subclass of an open communicating class C created in \(T^{(r)}\) after the addition of a set of arcs of weight \(\theta _p\). Therefore, \(c_1\) is an open communicating class in \(T_p^{(r)}\). This means the set of min-arcs with tails in \(c_1\) and heads not in \(c_1\) in not empty. All these min-arcs are in \(T_p^{(r)}\), and none of them is in \(\Gamma _{k_1}^{(r')}\). By Lemma 10.1, the weights of these min-arcs become \(\theta _p\) after the modification. One of them is picked and added to the bucket \({\mathcal {B}}'\). Hence, this arc will be added to \(\Gamma ^{(r'+1)}\) at some \(k_1<k\le K_p\). This allows us to conclude that at least one more arc of weight \(\theta _p\) will be removed from \({\mathcal {B}}'\) after the cycle \(c_1\) is formed. Hence, \(c_1\) is not a closed communicating class in \(\Gamma _{K_p}^{(r')}\). Therefore, Statements 1, 2, 3 and the auxiliary statement hold for \(K_{p_r}<k\le \min \{k_2,K_{p_{r+1}-1}\}\).

    Assume that Statements 1, 2, 3, and the auxiliary statement hold up to step \(k_{N} < K_{p_{r+1} -1}\) of Algorithm 1. Let cycle \(c_N\) be encountered at step \(k_N\) in Algorithm 1 after the addition of an arc of weight \(\theta _p\). The set of vertices \({\mathcal {S}}^{(r')}(v_{c_N})\) is a subclass of an open communicating class C of the graph \(T_p\) by the induction assumption. Hence, the set of the min-arcs with tails in \({\mathcal {S}}^{(r')}(v_{c_N})\) and heads not in \({\mathcal {S}}^{(r')}(v_{c_N})\) is not empty. All of these arcs are in \(T_p^{(r)}\) but not in \(\Gamma _{k_N}^{(r')}\). By Corollary 10.2, their weights will become \(\theta _p\) during step \(k_N\) of Algorithm 1. One of these min-arcs will be added to the bucket \({\mathcal {B}}'\) and then removed from it at some step \(k_N<k\le K_{p}\). Hence, the cycle \(c_N\) cannot be a closed communicating class in \(T_p^{(r)}\). This proves Statements 1, 2, 3, and the auxiliary statement for all \(K_{p_r}<k\le K_{p_{r+1}-1}\).

  3. 3.

    Now we show that Statements 1, 2, 3, and the auxiliary statement hold for \(K_{p_{r+1}-1}+1\le k\le K_{p_{r+1}}\). Let C be the closed communicating class formed in the graph \(T_{p_{r+1}}^{(r)}\) after the addition of the set of min-arcs of weight \(\theta _{p_{r+1}}\). Therefore, all min-arcs from the vertices in C head in C. After contracting C into a single super-vertex \(v_C\), the weight of min-arcs from it will be

    $$\begin{aligned} \min _{i\in C,~j\notin C}[U_{ij}-U_{\min }(i) +\theta _{p_{r+1}}] >\theta _{p_{r+1}}. \end{aligned}$$
    (39)

(Here \(U_{\min }(i)\) is the weight of min-arcs from i in the graph \(G^{(r)}\).) Let the recursion level at the end of step \(K_{p_{r+1}-1}\) in Algorithm 1 be \(r''\), and \(C'\) be the set of vertices in \(\Gamma ^{(r'')}\) corresponding to C. If no cycles were formed in Algorithm 1 with vertices in C then \(C'=C\). Otherwise, some of the vertices of C are contacted into super-vertices in \(\Gamma ^{(r'')}\). In this case, the subset of vertices of C contracted into super-vertices is an open communicating subclass of C. Let us show that \(C'\) is a closed communicating class of \(\Gamma _{K_{p_{r+1}}}^{(r'')}\).

Lemma 10.2 implies that the min-arcs from \(C'\) all heading in \(C'\) form at least one cycle c. Consider two cases.

Case 1: cycle c includes whole \(C'\). Then by Corollary 10.3, the weight of min-arcs from the vertex \(v_c\) in Algorithm 1 will be given by Eq. (39), i.e., the same as it is in Algorithm 2, and one of those min-arcs will be added to \({\mathcal {B}}'\). Hence, Statements 1, 2, 3, and the auxiliary statement hold at \(p=p_{r+1}\) and \(K_{p_{r+1}-1}<k\le K_{p_{r+1}}\).

Case 2: the cycle c does not include all vertices from \(C'\). Since C does not contain closed communicating subclasses, the set of min-arcs in \(G^{(r)}\) with tails in c and heads not in c is not empty. By Corollary 10.3, these min-arcs will be the min-arcs from \(v_c\), and their weights will be \(\theta _p\). One of these min-arcs will be added to \({\mathcal {B}}'\) and then removed. Hence, the min-arcs from \((C\backslash {\mathcal {S}}^{(r)}(v_c))\cup \{v_c\}\) head to \((C\backslash {\mathcal {S}}^{(r)}(v_c))\cup \{v_c\}\). By Lemma 10.2, they form at least one cycle c. Again, there are two options: either c includes all vertices of C or not. In the former case, using the argument from Case 1, we prove the induction step. In the latter case, we use the argument from Case 2. Repeating this argument at most \(|C|-1\) number of times (as each new cycle includes at least one more vertex of \(C'\) in comparison with the previous one), we obtain a cycle including all vertices of C.

Repeating this argument for all closed communicating classes formed in \(T^{(r)}\) at step \(p_{r+1}\), we conclude that all closed communicating classes encountered in Algorithm 2 at steps \(p_{r+1}\) will be contracted into single super-vertices by both Algorithms 1 and 2, and arcs of the same weight will be added to \({\mathcal {B}}\) and \({\mathcal {B}}'\). Hence, the induction step is proven. This completes the proof of Theorem 6.1.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Gan, T., Cameron, M. A Graph-Algorithmic Approach for the Study of Metastability in Markov Chains. J Nonlinear Sci 27, 927–972 (2017). https://doi.org/10.1007/s00332-016-9355-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00332-016-9355-0

Keywords

Navigation