Hitting Times and Interlacing Eigenvalues: A Stochastic Approach Using Intertwinings

Abstract

We develop a systematic matrix-analytic approach, based on intertwinings of Markov semigroups, for proving theorems about hitting-time distributions for finite-state Markov chains—an approach that (sometimes) deepens understanding of the theorems by providing corresponding sample-path-by-sample-path stochastic constructions. We employ our approach to give new proofs and constructions for two theorems due to Mark Brown, theorems giving two quite different representations of hitting-time distributions for finite-state Markov chains started in stationarity. The proof, and corresponding construction, for one of the two theorems elucidates an intriguing connection between hitting-time distributions and the interlacing eigenvalues theorem for bordered symmetric matrices.

This is a preview of subscription content, access via your institution.

References

  1. 1.

    Aldous, D.J., Fill, J.A.: Reversible Markov Chains and Random Walks on Graphs. Chapter drafts available from http://www.stat.berkeley.edu/users/aldous/RWG/book.html

  2. 2.

    Aldous, D., Diaconis, P.: Strong uniform times and finite random walks. Adv. Appl. Math. 8(1), 69–97 (1987)

    Article  MATH  MathSciNet  Google Scholar 

  3. 3.

    Aldous, D.J., Brown, M.: Inequalities for rare events in time-reversible Markov chains. I. In: Stochastic Inequalities (Seattle, WA, 1991), vol. 22 of IMS Lecture Notes Monograph Series, pp. 1–16. Institute for Mathematical Statististics, Hayward (1992).

  4. 4.

    Biane, P.: Intertwining of Markov semi-groups, some examples. In: Séminaire de Probabilités, XXIX, vol. 1613 of Lecture Notes in Mathematics, pp. 30–36. Springer, Berlin (1995).

  5. 5.

    Brown, M.: The first passage time distribution for a parallel exponential system with repair. In: Reliability and Fault Tree Analysis (Conference, Universuty of California, Berkeley, CA, 1974), pp. 365–396. Society for Industry and Applied Mathematics, Philadelphia (1975).

  6. 6.

    Carmona, P., Petit, F., Yor, M.: Beta–gamma random variables and intertwining relations between certain Markov processes. Rev. Mat. Iberoamericana 14(2), 311–367 (1998)

    Article  MATH  MathSciNet  Google Scholar 

  7. 7.

    Diaconis, P., Fill, J.A.: Strong stationary times via a new form of duality. Ann. Probab. 18(4), 1483–1522 (1990)

    Article  MATH  MathSciNet  Google Scholar 

  8. 8.

    Diaconis, P., Miclo, L.: On times to quasi-stationarity for birth and death processes. J. Theor. Probab. 22(3), 558–586 (2009)

    Article  MATH  MathSciNet  Google Scholar 

  9. 9.

    Durrett, R.: Probability Models for DNA Sequence Evolution. Probability and Its Applications, 2nd edn. Springer, New York (2008)

    Book  Google Scholar 

  10. 10.

    Etheridge, A.: Some Mathematical Models from Population Genetics, vol. 2012 of Lecture Notes in Mathematics. Springer, Heidelberg, 2011. Lectures from the 39th Probability Summer School held in Saint-Flour (2009).

  11. 11.

    Fill, J.A.: Strong stationary duality for continuous-time Markov chains I. Theory. J. Theor. Probab. 5(1), 45–70 (1992)

    Article  MATH  MathSciNet  Google Scholar 

  12. 12.

    Fill, J.A.: An interruptible algorithm for perfect sampling via Markov chains. Ann. Appl. Probab. 8(1), 131–162 (1998)

    Article  MATH  MathSciNet  Google Scholar 

  13. 13.

    Fill, J.A.: On hitting times and fastest strong stationary times for skip-free and more general chains. J. Theor. Probab. 22(3), 587–600 (2009)

    Article  MATH  MathSciNet  Google Scholar 

  14. 14.

    Fill, J.A.: The passage time distribution for a birth-and-death chain: strong stationary duality gives a first stochastic proof. J. Theor. Probab. 22(3), 543–557 (2009)

    Article  MATH  MathSciNet  Google Scholar 

  15. 15.

    Fill, J.A., Machida, M., Murdoch, D.J., Rosenthal, J.S.: Extension of Fill’s perfect rejection sampling algorithm to general chains. Random Structures Algorithms, 17(3–4): 290–316, 2000. Special issue: Proceedings of the Ninth International Conference “Random Structures and Algorithms” (Poznan, 1999).

  16. 16.

    Horn, R.A., Johnson, C.R.: Matrix Analysis. Cambridge University Press, Cambridge (1985)

    Book  MATH  Google Scholar 

  17. 17.

    Karlin, S., McGregor, J.: Coincidence properties of birth and death processes. Pacific J. Math. 9, 1109–1140 (1959)

    Article  MATH  MathSciNet  Google Scholar 

  18. 18.

    Kingman, J.F.C.: Random partitions in population genetics. Proc. R. Soc. Lond. Ser. A 361(1704), 1–20 (1978)

    Article  MATH  MathSciNet  Google Scholar 

  19. 19.

    Kingman J. F. C.: The representation of partition structures. J. Lond. Math. Soc. 18(2), 374–380 (1978).

    Google Scholar 

  20. 20.

    Kingman, J.F.C.: The coalescent. Stoch. Proc. Appl. 13(3), 235–248 (1982).

  21. 21.

    Mark, B.: Interlacing eigenvalues in time reversible Markov chains. Math. Oper. Res. 24(4), 847–864 (1999)

    Article  MATH  MathSciNet  Google Scholar 

  22. 22.

    Mark, B., Shao, Y.S.: Identifying coefficients in the spectral representation for first passage time distributions. Probab. Eng. Inform. Sci. 1, 69–74 (1987)

    Article  MATH  Google Scholar 

  23. 23.

    Mena, R., Murray, W.: Markov chains for collaboration. Math. Mag. 84(1), 3–15 (2011)

    Article  MATH  MathSciNet  Google Scholar 

  24. 24.

    Miclo, L.: On absorption times and Dirichlet eigenvalues. ESAIM Probab. Stat. 14, 117–150 (2010)

    Article  MATH  MathSciNet  Google Scholar 

  25. 25.

    Morris, B., Peres, Y.: Evolving sets, mixing and heat kernel bounds. Probab. Theory Related Fields 133(2), 245–266 (2005)

    Article  MATH  MathSciNet  Google Scholar 

  26. 26.

    Rogers, L.C.G., Pitman, J.W.: Markov functions. Ann. Probab. 9(4), 573–582 (1981)

    Article  MATH  MathSciNet  Google Scholar 

Download references

Acknowledgments

Research was supported by the Acheson J. Duncan Fund for the Advancement of Research in Statistics, and by U.S. Department of Education GAANN Grant P200A090128. The authors thank Mark Brown for stimulating discussions and the anonymous referee for helpful comments.

Author information

Affiliations

Authors

Corresponding author

Correspondence to James Allen Fill.

Appendix: \(P^*\) When \(P\) is a Star Chain

Appendix: \(P^*\) When \(P\) is a Star Chain

In Remark 3.6, it is claimed that if the given chain \(P\) is a star chain, then the star chain of Lemma 3.4 is simply obtained by collapsing all leaves with the same one-step transition probability to state \(0\) into a single leaf. More precisely, we establish the following:

Proposition 5.1

Let \(P\) be the transition matrix of an ergodic star chain with hub at \(0\). If for each \(\gamma _i\) in the reduced set of eigenvalues of \(P_0\), we define

$$\begin{aligned} m(i):=\{j\in [n]:\eta _j=\gamma _i\}, \end{aligned}$$

then \(P^*(0,i) = \sum _{j\in m(i)} P(0,j)\).

Proof

Define \(H:=\text{ diag}(\eta _1,\ldots ,\eta _n)\) and

$$\begin{aligned} x&:= (P(0,1),\ldots ,P(0,n)), \\ y&:= (1-\eta _1,\ldots ,1-\eta _n), \end{aligned}$$

so that

$$\begin{aligned} P = \left( \begin{array}{c|c} P(0,0)&x \\ \hline y^T&H \end{array} \right). \end{aligned}$$

By the standard formula for the determinant of a partitioned matrix (e.g., [16, Sect. 0.8.5]), if \(t\) is not in the spectrum \(\{\eta _1, \dots , \eta _n\}\) of \(H\), then we find

$$\begin{aligned} \det (tI-P) = [t-P(0,0)-x(tI-H)^{-1}y^T] \det (tI-H) \end{aligned}$$
(5.1)

for the characteristic polynomial of \(P\). Analogously, define \(\Gamma :=\text{ diag}(\gamma _1,\ldots ,\gamma _r)\) and

$$\begin{aligned} x^*&:= (P^*(0,1),\ldots ,P^*(0,r)), \\ y^*&:= (1-\gamma _1, \ldots ,1-\gamma _r); \end{aligned}$$

if \(t\) is not in the spectrum \(\{\gamma _1, \dots , \gamma _r\}\) of \(\Gamma \), then we find

$$\begin{aligned} \det (tI-P^*)=[t-P^*(0,0)-x^*(tI-\Gamma )^{-1}{y^{*T}}] \det (tI-\Gamma ) \end{aligned}$$
(5.2)

for the characteristic polynomial of \(P^*\).

Note that

$$\begin{aligned} P(0,0)&= \text{ tr}P -\text{ tr}H = \sum _{i = 0}^n \theta _i - \sum _{i = 1}^n \eta _i \\ \nonumber&= \sum _{i = 0}^r \lambda _i - \sum _{i = 1}^r \gamma _i = \text{ tr}P^*-\text{ tr}\Gamma = P^*(0,0), \end{aligned}$$
(5.3)

where the third equality is a result of the eigenvalue reduction procedure discussed in Sect. 3.1 and the fourth equality is from Lemma 2.6 in Ref.  [21]. Similarly, for all \(t\notin \{\eta _1,\ldots ,\eta _n\}\), we have

$$\begin{aligned} \frac{\det (tI-P)}{\det (tI-H)} = \frac{\det (tI-P^*)}{\det (tI-\Gamma )}. \end{aligned}$$
(5.4)

Therefore, for all \(t\notin \{\eta _1,\ldots ,\eta _n\}\), we have

$$\begin{aligned} \sum _{i=1}^n P(0,i) \frac{1-\eta _i}{t-\eta _i} = \sum _{i=1}^r P^*(0,i) \frac{1-\gamma _i}{t-\gamma _i}, \end{aligned}$$
(5.5)

because using definitions of \(H, x, y, \Gamma , x^*, y^*\) and equations (5.1)–(5.4) we find

$$\begin{aligned}&\sum _{i=1}^n P(0,i) \frac{1-\eta _i}{t-\eta _i} =x(tI-H)^{-1}y^T = t - P(0, 0) - \frac{\det (tI-P)}{\det (tI-H)} \\&\quad = t - P^*(0, 0) - \frac{\det (tI-P^*)}{\det (tI-\Gamma )} = x^*(tI-\Gamma )^{-1}{y^*}^T= \sum _{i=1}^r P^*(0,i) \frac{1-\gamma _i}{t-\gamma _i}. \end{aligned}$$

Rewrite (5.5) as

$$\begin{aligned} \sum _{i=1}^r P^*(0,i) \frac{1-\gamma _i}{t-\gamma _i}=\sum _{i=1}^r\left(\sum _{j\in m(i)} P(0,j)\right) \frac{1-\gamma _i}{t-\gamma _i}. \end{aligned}$$

Since \(\gamma _1, \dots , \gamma _r\) are distinct, it follow easily that \(P^*(0,i)=\sum _{j\in m(i)} P(0,j)\) for \(i = 1, \dots , r\), as desired. \(\square \)

Let \(\pi \) be the stationary distribution for \(P\). Using the formula for \(P^*(0, i)\) provided by Proposition 5.1, it is a simple matter to check that the probability mass function \(\pi ^*\) defined by \(\pi ^*(0) := \pi (0)\) and \(\pi ^*(i)=\sum _{j\in m(i)} \pi (j)\) for \(i \ne 0\) satisfies the detailed balance condition and is therefore the stationary distribution for \(P^*\); indeed, using the reversibility of \(P\) with respect to \(\pi \), we have

$$\begin{aligned} \pi ^*(0) P^*(0, i)&= \pi (0) \sum _{j \in m(i)} P(0, j) = \sum _{j \in m(i)} \pi (j) P(j, 0) \\&= \sum _{j \in m(i)} \pi (j) (1 - \gamma _i) = \pi ^*(i) P^*(i, 0). \end{aligned}$$

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Fill, J.A., Lyzinski, V. Hitting Times and Interlacing Eigenvalues: A Stochastic Approach Using Intertwinings. J Theor Probab 27, 954–981 (2014). https://doi.org/10.1007/s10959-012-0457-9

Download citation

Keywords

  • Markov chains
  • Hitting times
  • Interlacing eigenvalues
  • Intertwinings

Mathematics Subject Classification (2010)

  • Primary 60J10
  • Secondary 60J27
  • 15A18