Abstract
In a recent paper in this journal, Gong, Mao and Zhang, using the theory of Dirichlet forms, extended Karlin and McGregor’s classical results on first-hitting times of a birth–death process on the nonnegative integers by establishing a representation for the Laplace transform \({\mathbb {E}}[e^{sT_{ij}}]\) of the first-hitting time \(T_{ij}\) for any pair of states i and j, as well as asymptotics for \({\mathbb {E}}[e^{sT_{ij}}]\) when either i or j tends to infinity. It will be shown here that these results may also be obtained by employing tools from the orthogonal-polynomial toolbox used by Karlin and McGregor, in particular associated polynomials and Markov’s theorem.
Similar content being viewed by others
Avoid common mistakes on your manuscript.
1 Introduction
A birth–death process is a continuous-time Markov chain \(\mathcal {X} := \{X(t),~t \ge 0\}\) taking values in \(S := \{0,1,2,\ldots \}\) with q-matrix \(Q := (q_{ij},~i,j \in S)\) given by
where \(\lambda _i > 0\) for \(i \ge 0,~ \mu _i > 0\) for \(i \ge 1\) and \(\mu _0 \ge 0\). Positivity of \(\mu _0\) entails that the process may evanesce by escaping from S, via state 0, to an absorbing state \(-1\). Throughout this paper, we will assume that the transition probabilities
satisfy both the backward and forward Kolmogorov equations, and mostly also that they are uniquely determined by the birth rates \(\lambda _i\) and death rates \(\mu _i\). Karlin and McGregor [14] have shown that the latter is equivalent to assuming
where the \(\pi _n\) are constants given by
We note that condition (1) does not exclude the possibility of explosion, escape from S, via all states larger than the initial state, to an absorbing state \(\infty \).
We denote by \(T_{ij}\) the (possibly defective) first-hitting time of state j, starting in state \(i\ne j\). Then, writing
and
we have the well-known result
(see, for example, [15, Equation (1.3)]). Karlin and McGregor give in [14, Equation (3.21)] a representation for \(\hat{P}_{ij}(s)\), which upon substitution in (2) yields
where \(Q_n,\,n=0,1,\ldots ,\) are the birth–death polynomials associated with the process \(\mathcal {X}\), that is, the \(Q_n\) satisfy the recurrence relation
The representation (3) was observed explicitly for the first time by Karlin and McGregor themselves in [16, p 378]. Since then several authors have rediscovered the result or provided alternative proofs (see Diaconis and Miclo [4] for some references).
In a recent paper in this journal, Gong et al. [12], using the theory of Dirichlet forms, extended Karlin and McGregor’s result by establishing a representation for the Laplace transform of the first-hitting time \(T_{ij}\) for any pair of states \(i\ne j\), as well as asymptotics when either i or j tends to infinity. It will be shown here that these results may also be obtained by exploiting Karlin and McGregor’s toolbox, which is the theory of orthogonal polynomials.
Our findings, which are actually somewhat more general than those of Gong et al. are presented in Sect. 3 and proven in Sect. 4. In the next section, we introduce some further notation, terminology and preliminary results. Since a path between two states in a birth–death process has to hit all intermediate states, we obviously have
So for notational simplicity—and without loss of generality—we will restrict ourselves to an analysis of \(T_{0n}\) and \(T_{n0}\) for \(n>0\).
2 Preliminaries
We will use the shorthand notation
and, following Anderson [1, Chapter 8],
We have \(K_\infty +L_\infty =\infty \) by our assumption (1), while, obviously,
Also, \(C+D = K_\infty L_\infty \), so (1) is actually equivalent to \(C+D = \infty \). Whether the quantities C and D are infinite or not determines the type of the boundary at infinity (see, for example, Anderson [1, Section 8.1]), but also, as we shall see, the asymptotic behavior of the polynomials \(Q_n\) of (4).
Since the birth–death polynomials \(Q_n\) satisfy the three-terms recurrence relation (4), they are orthogonal with respect to a positive Borel measure on the nonnegative real axis, and have positive and simple zeros. The orthogonalizing measure for the polynomials \(Q_n\) (normalized to be a probability measure) is not necessarily uniquely determined by the birth and death rates, but there exists, in any case, a unique natural measure \(\psi \), characterized by the fact that the minimum of its support is maximal. We refer to Chihara’s book [3] for properties of orthogonal polynomials in general, and to Karlin and McGregor’s papers [14, 15] for results on birth–death polynomials in particular (see also [10, Section 3.1] for a concise overview). For our purposes, the following properties of birth–death polynomials are furthermore relevant.
With \(x_{n1}<x_{n2}<\cdots <x_{nn}\) denoting the n zeros of \(Q_n(x),\) there is the classical separation result
so that the limits
exist. We further let
(possibly infinity). The numbers \(\xi _i\) may be defined alternatively as
where supp stands for support. So knowledge of the (natural) orthogonalizing measure for the polynomials \(Q_n\) implies knowledge of the numbers \(\xi _i\). It is clear from the definition of \(\xi _i\) that
Moreover, we have, for all \(i \ge 1\),
as is evident from the alternative definition of \(\xi _i\). By suitably interpreting [6, Equations (2.6) and (2.11)], it follows that
where the left-hand side should be interpreted as infinity if \(\xi _1 = 0\). In particular,
Also, by [6, Theorem 2],
and
Given a sequence of birth–death polynomials \(\{Q_n\}\), we obtain the sequence \(\{Q_n^{(l)}\}\) of associated polynomials of order \(l\ge 0\) by replacing \(Q_n\) by \(Q_n^{(l)}\), \(\lambda _n\) by \(\lambda _{n+l}\) and \(\mu _n\) by \(\mu _{n+l}\) in the recurrence relation (4). Evidently, the polynomials \(Q_n^{(l)}\) are birth–death polynomials again, so \(Q_n^{(l)}(x)\) has simple, positive zeros \(x_{n1}^{(l)}<x_{n2}^{(l)}<\dots <x_{nn}^{(l)}\) and we can write
while it follows by induction that
where \(L_{-1}:=0\). Note that \(Q_n^{(0)}(0)=Q_n(0)=1\) for all n if \(\mu _0=0\).
Defining the quantities \(\xi _i^{(l)}\) and \(\sigma ^{(l)}\) in analogy to (7) and (8), we have, by [3, Theorem III.4.2],
so that
Moreover, [5, Theorem 1] tells us that
Since the polynomials \(Q_n^{(l)}\) are birth–death polynomials, they are orthogonal with respect to a unique natural (probability) measure \(\psi ^{(l)}\) on the nonnegative real axis. A key ingredient in our analysis is Markov’s theorem, which relates the Stieltjes transform of the measure \(\psi ^{(l)}\) to the polynomials \(Q_n^{(l)}\) and \(Q_n^{(l+1)}\), namely
We note that \(\psi ^{(l)}\) is not necessarily the only orthogonalizing measure for the polynomials \(Q_n^{(l)}\), a setting usually not covered in statements of Markov’s theorem in the literature (see, for example, [3, p 89]). However, an extension of the original theorem that serves our needs can be found in Berg [2] (see in particular [2, Section 3], where the measure \(\mu ^{(0)}\) corresponds to our \(\psi ^{(l)}\)).
We will also have use for a classical result in the theory of continued fractions relating the Stieltjes transforms of the measures \(\psi ^{(l)}\) and \(\psi ^{(l+1)}\), namely
Again we refer to Berg [2, Section 4] for statements of this result in the generality required in our setting.
Our final preliminary results concern asymptotics for the polynomials \(Q_n^{(l)}\) as \(n\rightarrow \infty \), which may be obtained by suitably interpreting the results of [17] (which extend those of [6]). We state the results in three propositions and give more details about their derivations in Sect. 4. Recall that \(\xi ^{(l)}_0=-\infty \) and \(Q_n(0)=1\) if \(\mu _0=0\).
Proposition 1
Let \(K_\infty = L_\infty = \infty \). Then \(C = D = \infty \), \(\sigma = 0\) and, for \(l\ge 0\),
Proposition 2
Let \(K_\infty = \infty \) and \(L_\infty < \infty \). Then \(D = \infty \) and,
-
(i)
for \(l\ge 0\),
$$\begin{aligned} \lim _{n\rightarrow \infty } Q_n^{(l)}(0) = 1 + \mu _l\pi _l(L_\infty - L_{l-1}) < \infty ; \end{aligned}$$ -
(ii)
if \(C=\infty \), for \(l\ge 0\),
$$\begin{aligned} \lim _{n\rightarrow \infty } Q_n^{(l)}(x) = \left\{ \begin{array}{ll} \infty &{}\quad \text {if}~~x < 0\\ 0 &{}\quad \text {if}~~0 < x \le \xi _k^{(l)} \text {~for~some~}k\ge 1; \end{array} \right. \end{aligned}$$ -
(iii)
if \(C<\infty \), for \(l \ge 0\),
$$\begin{aligned} \lim _{n\rightarrow \infty } \frac{Q_n^{(l)}(x)}{Q_n^{(l)}(0)} = \prod _{i=1}^\infty \left( 1-\frac{x}{\xi _i^{(l)}}\right) , \quad x \in \mathbb {R}, \end{aligned}$$an entire function with simple, positive zeros \(\xi _i^{(l)},~i\ge 1\).
Proposition 3
Let \(K_\infty < \infty \) and \(L_\infty = \infty \). Then \(C = \infty \) and,
-
(i)
for \(l=0\) and \(\mu _0>0\), or \(l \ge 1\),
$$\begin{aligned} \lim _{n\rightarrow \infty } Q_n^{(l)}{(0)} = \infty ; \end{aligned}$$ -
(ii)
if \(D=\infty \), for \(l \ge 0\),
$$\begin{aligned} \lim _{n\rightarrow \infty } Q_n^{(l)}(x) = \left\{ \begin{array}{ll} \infty &{}\quad \text {if}~~\xi ^{(l)}_{2k} < x \le \xi ^{(l)}_{2k+1} \text {~for~some~}k\ge 0\\ -\infty &{}\quad \text {if}~~\xi ^{(l)}_{2k+1} < x \le \xi ^{(l)}_{2k+2} \text {~for~some~}k\ge 0; \end{array} \right. \end{aligned}$$ -
(iii)
if \(D<\infty \) and \(\mu _0=0\),
$$\begin{aligned} \lim _{n\rightarrow \infty } \frac{Q_n(x)}{L_{n-1}} = -x K_\infty \prod _{i=1}^\infty \left( 1-\frac{x}{\xi _{i+1}}\right) , \quad x \in \mathbb {R}, \end{aligned}$$an entire function with simple zeros \(\xi _1=0\) and \(\xi _{i+1}>0,~i\ge 1\);
-
(iv)
if \(D<\infty \), for \(l=0\) and \(\mu _0>0\), or \(l \ge 1\),
$$\begin{aligned} \lim _{n\rightarrow \infty } \frac{Q_n^{(l)}(x)}{Q_n^{(l)}(0)} = \prod _{i=1}^\infty \left( 1-\frac{x}{\xi _i^{(l)}}\right) , \quad x \in \mathbb {R}, \end{aligned}$$an entire function with simple, positive zeros \(\xi _i^{(l)},~i\ge 1\).
3 Results
Representations for \({\mathbb {E}}[e^{sT_{0n}}{\mathbb {I}}_{\{T_{0n}<\infty \}}]\) and \({\mathbb {E}}[e^{sT_{n0}}{\mathbb {I}}_{\{T_{n0}<\infty \}}]\) in terms of the polynomials \(Q_n^{(l)}\) are collected in the first theorem.
Theorem 1
We have, for \(\mu _0 \ge 0\) and \(n\ge 1\),
and, if \(C+D=\infty \),
Note that for \(s<0\), we have \({\mathbb {E}}[e^{sT_{0n}}{\mathbb {I}}_{\{T_{0n}<\infty \}}] ={\mathbb {E}}[e^{sT_{0n}}]\), so the representation (16) reduces to Karlin and McGregor’s result (3). The explicit representation (17) is new, but may be obtained by a limiting procedure from Gong et al. [12, Corollary 3.6], where a finite state space is assumed.
By choosing \(s=0\) in (16) and (17) and using (11), we obtain expressions for the probabilities \(\mathbb {P}(T_{0n}<\infty )\) and \(\mathbb {P}(T_{n0}<\infty )\) that are in accordance with [15, an unnumbered formula on page 387 and Theorem 10]. For convenience, we state the results as a corollary of Theorem 1, but remark that a proof of (19) on the basis of (17) would require additional motivation in the case \(\xi _1^{(1)} = 0\).
Corollary 1
([15]) We have, for \(\mu _0 \ge 0\) and \(n\ge 1\),
and, if \(C+D=\infty \),
After a little algebra, (17) and (11) lead to
Subsequently applying Propositions 1, 2 (iii) and 3 (iv), we obtain the second corollary of Theorem 1.
Corollary 2
If \(C+D=\infty \), but \(C<\infty \) or \(D<\infty \), then, for \(n\ge 1\),
where the infinite products are entire functions with simple, positive zeros \(\xi ^{(n+1)}_i\) and \(\xi ^{(1)}_i,~i\ge 1\).
Assuming a denumerable state space, but under the condition \(C=\infty \) and \(D<\infty \), Gong et al. give in [12, Theorem 5.5 (a)] a representation for \({\mathbb {E}}[e^{sT_{n0}}],~s<0\), which is encompassed by Corollary 2. Indeed, in this case we have \(L_\infty = \infty \), and hence, by (19), \(\mathbb {P}(T_{n0}<\infty ) = 1\).
Asymptotic results for \({\mathbb {E}}[e^{sT_{0n}}{\mathbb {I}}_{\{T_{0n}<\infty \}}]\) and \({\mathbb {E}}[e^{sT_{n0}}{\mathbb {I}}_{\{T_{n0}<\infty \}}]\) as \(n\rightarrow \infty \) are summarized in the second theorem.
Theorem 2
We have, for \(\mu _0 \ge 0\) and \(s< 0\),
and
The infinite products in (21) and (22) are reciprocals of entire functions with simple, positive zeros \(\xi _i\) and \(\xi ^{(1)}_i,~i\ge 1\), respectively.
By (18), we have
so (21) implies
which generalizes [12, Theorem 4.6] where \(\mu _0=0\) is assumed. (At the end of [12, Section 4], the authors remark that the case \(\mu _0>0\) may be treated in a way analogous to the case \(\mu _0=0\), but no explicit result is given.) If \(C=\infty \) and \(D<\infty \), we must have \(L_\infty =\infty \), and hence, by (19), \(\mathbb {P}(T_{n0}<\infty ) = 1\). So (22) implies
which is [12, Theorem 5.5 (b)].
4 Proofs
4.1 Proofs of Propositions 1–3
The conclusions regarding C and D in the Propositions 1, 2 and 3 are given already in (6), while the statements (i) in Propositions 2 and 3 are implied by (11). The other statements follow from results in [17], where two cases—corresponding in the setting at hand to \(\mu _0=0\) and \(\mu _0>0\)—are considered simultaneously by means of a duality relation involving polynomials \(R_n\) and \(R_n^*\). The asymptotic results for \(R_n\) may be translated into asymptotics for \(Q_n\) if \(\mu _0=0\), while the results for \(R_n^*\), suitably interpreted, give asymptotics for \(Q_n\) if \(\mu _0>0\), and for \(Q_n^{(l)}\) with \(l\ge 1\). Concretely, the statements in Proposition 1, Proposition 2 (ii) and Proposition 3 (ii) regarding the case \(x<0\) follow from [17, Lemma 2.4 and Theorems 3.1 and 3.3], while the results for \(x>0\) are implied by [17, Theorems 2.2, 3.6 and 3.8]. Proposition 2 (iii) follows from [17, Theorem 3.1] for \(l=0\) and \(\mu _0=0\), and from [17, Corollary 3.2] for \(l=0\) and \(\mu _0>0\), and for \(l\ge 1\). Proposition 3 (iii) is implied by [17, Theorems 2.2, 3.3 and 3.4 (ii)], while Proposition 3 (iv) is a consequence of [17, Corollary 3.2]. Finally, the fact that \(\sigma =0\) in the setting of Proposition 1 is stated, for example, in [17, Theorem 2.2 (iv)].
4.2 Proof of Theorem 1
As observed already, substitution in (2) of Karlin and McGregor’s formula for \(\hat{P}_{ij}(s)\) given on [14, Equation (3.21)] leads to (3) and hence, by analytic continuation, to (16).
To obtain (17) we note that [15, Equation (3.21)] also yields
which upon substitution in (2) leads to
Moreover, by Karlin and McGregor’s representation formula for the transition probabilities \(P_{ij}(t)\) (see [14, Section III.6]) we have \(P_{00}(t) = \int _0^\infty e^{-xt}\psi ({\hbox {d}}x)\), where \(\psi \) is a (probability) measure with respect to which the polynomials \(Q_n\) are orthogonal. Since the condition \(C+D=\infty \) is equivalent to (1), it ensures that the transition probabilities are uniquely determined by the birth and death rates, whence \(\psi \) must be the natural measure (see [14]). So we have
Subsequently applying (15) with \(l=0\), it follows that
whence, more generally,
and, by analytic continuation,
Since \(T_{n0} = T_{n,n-1}+\cdots +T_{10}\), while \(T_{n,n-1},\dots ,T_{10}\) are independent random variables, Markov’s theorem (14) implies that we can write
Recalling (12), we conclude that this expression holds for \(s<\xi _1^{(1)}\).
4.3 Proof of Theorem 2
Letting \(n\rightarrow \infty \) in (16) and applying the results of Propositions 1, 2 and 3 readily yields the first statement of Theorem 2.
To prove the second statement, we employ Corollary 2. First note that, for \(a>0\) and \(s\le 0\), we have \(1 \le 1-\frac{s}{a}\le e^{-s/a}\), so that, for \(l \ge 0\),
provided \(\xi _1^{(l)}>0\). Defining \(C^{(l)}\) and \(D^{(l)}\) in analogy to (5) it is easily seen that
So, assuming \(C<\infty \) or \(D<\infty \), we have, by (10),
Hence \(\sigma ^{(l)} = \sigma =\infty \), so that, by (13), \(\xi _i^{(l)} \rightarrow \infty \) as \(l \rightarrow \infty \). As a consequence
and the result follows since \(L_\infty <\infty \) if \(C<\infty \), whereas \(L_\infty = \infty \) if \(C=\infty \) and \(D<\infty \).
5 Concluding Remarks
First we note that the result (24)—or rather a generalization of (24)—may be derived directly from the Kolmogorov differential equations and Karlin and McGregor’s representation formula for the transition probabilities \(P_{ij}(t)\). The argument is given on [8, p 508] (and essentially already on [15, p 385]) and yields
so that
Note that as a consequence of (17) and (26), we have, for all \(m \ge 0\),
which implies a partial extension of Markov’s theorem (14) to the effect that, for \(m \ge 0\) and \(l \ge 1\) (and \(l=0\) if \(\mu _0>0\)),
Using (14), (15), and the recurrence relation for the polynomials \(Q^{(l)}_n\), it may be shown by induction that (27) is actually valid for all \(l \ge 0\), \(m \ge 0\) and Re\((s) < \xi ^{(l)}_1\). Substitution \(s=0\) in (27) and (11) leads in particular to
which is consistent with [15, Equations (9.9) and (9.14)] (also when \(\xi _1=0\)).
If we do not impose the condition \(C+D=\infty \), the birth and death rates do not necessarily determine a birth–death process uniquely. However, as observed in [12], several results remain valid if \(C+D<\infty \), provided they are interpreted as properties of the minimal process, which is the process with an absorbing boundary at infinity (and which is always associated with the natural measure for the polynomials \(Q_n\), see [7]). Concretely, if \(C+D<\infty \) the arguments leading to Theorem 1, and hence Theorem 1 itself and Corollary 1, remain valid. Moreover, the results in [17] imply that, for \(l\ge 0\),
an entire function with simple, positive zeros \(\xi _i^{(l)},~i\ge 1\). (Note that this complements Propositions 1—3.) Hence, also (20) remains valid. Finally, letting \(n\rightarrow \infty \) in (16) and (20), we readily conclude that the results in Theorem 2 for \(C<\infty ,~D=\infty \) are actually valid for \(C+D<\infty \) as well.
In the setting \(C+D<\infty \), Gong et al. [12] pay attention also to the maximal process, the process that is characterized by a reflecting barrier at infinity. In this case, the measure featuring in the representation for \(P_{00}(t)\), and hence in (23), is not the natural measure. Although, applying the results of [7], the relevant measure can be identified and expressed in terms of a natural measure corresponding to a dual birth–death process, application of Markov’s theorem does not seem feasible in this case.
Our final remark is the following. Choosing \(l=0\), letting \(s \uparrow \xi _1\) in (15), and using the recurrence relation (4), we readily get
In fact, using (15) again, it is not difficult to generalize this result to
which upon substitution in (25) leads to
(Since, by [9, Theorem 3.1], the condition above is equivalent to \(\xi _1\)-recurrence of the process, this result may also be obtained by applying [13, Lemma 3.3.3 (iii)] to the setting at hand, see [11, Lemma 3.2].) It now follows from (22) that
If \(\mu _0=0\) then \(\xi _1=0\), so the result does not take us by surprise, but for \(\mu _0>0\) we regain an interesting extension of Proposition 3 (iv)—recently obtained by Gao and Mao [11, Lemma 3.4]—since it has consequences for the existence of quasi-stationary distributions.
References
Anderson, W.J.: Continuous-Time Markov Chains. Springer, New York (1991)
Berg, C.: Markov’s theorem revisited. J. Approx. Theory 78, 260–275 (1994)
Chihara, T.S.: An Introduction to Orthogonal Polynomials. Gordon & Breach, New York (1978)
Diaconis, P., Miclo, L.: On times to quasi-stationarity for birth and death processes. J. Theor. Probab. 22, 558–586 (2009)
van Doorn, E.A.: On oscillation properties and the interval of orthogonality of orthogonal polynomials. SIAM J. Math. Anal. 15, 1031–1042 (1984)
van Doorn, E.A.: On orthogonal polynomials with positive zeros and the associated kernel polynomials. J. Math. Anal. Appl. 86, 441–450 (1986)
van Doorn, E.A.: The indeterminate rate problem for birth–death processes. Pac. J. Math. 130, 379–393 (1987)
van Doorn, E.A.: On associated polynomials and decay rates for birth–death processes. J. Math. Anal. Appl. 278, 500–511 (2003)
van Doorn, E.A.: On the \(\alpha \)-classification of birth–death and quasi-birth–death processes. Stoch. Models 22, 411–421 (2006)
van Doorn, E.A.: Representations for the decay parameter of a birth–death process based on the Courant–Fischer theorem. J. Appl. Probab. 52, 278–289 (2015)
Gao, W.-J., Mao, Y.-H.: Quasi-stationary distribution for the birth–death process with exit boundary. J. Math. Anal. Appl. 427, 114–125 (2015)
Gong, Y., Mao, Y.-H., Zhang, C.: Hitting time distributions for denumerable birth and death processes. J. Theor. Probab. 25, 950–980 (2012)
Jacka, S.D., Roberts, G.O.: Weak convergence of conditioned processes on a countable state space. J. Appl. Probab. 32, 902–916 (1995)
Karlin, S., McGregor, J.L.: The differential equations of birth-and-death processes, and the Stieltjes moment problem. Trans. Am. Math. Soc. 86, 489–546 (1957)
Karlin, S., McGregor, J.L.: The classification of birth and death processes. Trans. Am. Math. Soc. 86, 366–400 (1957)
Karlin, S., McGregor, J.L.: A characterization of birth and death processes. Proc. Nat. Acad. Sci. 45, 375–379 (1959)
Kijima, M., van Doorn, E.A.: Weighted sums of orthogonal polynomials with positive zeros. J. Comput. Appl. Math. 65, 195–206 (1995)
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.
About this article
Cite this article
van Doorn, E.A. An Orthogonal-Polynomial Approach to First-Hitting Times of Birth–Death Processes. J Theor Probab 30, 594–607 (2017). https://doi.org/10.1007/s10959-015-0659-z
Received:
Revised:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10959-015-0659-z