The Limit Behaviour of Imprecise Continuous-Time Markov Chains

Abstract

We study the limit behaviour of a nonlinear differential equation whose solution is a superadditive generalisation of a stochastic matrix, prove convergence, and provide necessary and sufficient conditions for ergodicity. In the linear case, the solution of our differential equation is equal to the matrix exponential of an intensity matrix and can then be interpreted as the transition operator of a homogeneous continuous-time Markov chain. Similarly, in the generalised nonlinear case that we consider, the solution can be interpreted as the lower transition operator of a specific set of non-homogeneous continuous-time Markov chains, called an imprecise continuous-time Markov chain. In this context, our convergence result shows that for a fixed initial state, an imprecise continuous-time Markov chain always converges to a limiting distribution, and our ergodicity result provides a necessary and sufficient condition for this limiting distribution to be independent of the initial state.

Appendix: Proofs

1.1 Proofs of Results in Sect. 2

Let A and B be two non-negatively homogeneous operators from $\mathcal {L}(\mathcal {X})$ to $\mathcal {L}(\mathcal {X})$ and consider any $f,g\in \mathcal {L}(\mathcal {X})$ and $\lambda \in \mathbb {R}$.

It is well known that the maximum norm on $\mathcal {L}(\mathcal {X})$ satisfies the defining properties of a norm: it is absolutely homogeneous ($\left\| \lambda f \right\| =\left| \lambda \right| \left\| f \right\|$), it is subadditive ($\left\| f+g \right\| \le \left\| f \right\| +\left\| g \right\|$), and it separates points ($\left\| f \right\| =0\Rightarrow f=0$). The induced operator norm also satisfies these properties. Firstly, it is absolutely homogeneous because the maximum norm is:

$$\begin{aligned} \left\| \lambda A \right\|&=\sup \{\left\| \lambda Af \right\| :f\in \mathcal {L}(\mathcal {X}),\left\| f \right\| =1\}\\&=\sup \{\left| \lambda \right| \left\| Af \right\| :f\in \mathcal {L}(\mathcal {X}),\left\| f \right\| =1\}\\&=\left| \lambda \right| \sup \{\left\| Af \right\| :f\in \mathcal {L}(\mathcal {X}),\left\| f \right\| =1\} =\left| \lambda \right| \left\| A \right\| . \end{aligned}$$

Secondly, it is subadditive because the maximum norm is:

$$\begin{aligned} \left\| A+B \right\|&=\sup \{\left\| (A+B)f \right\| :f\in \mathcal {L}(\mathcal {X}),\left\| f \right\| =1\}\\&=\sup \{\left\| Af+Bf \right\| :f\in \mathcal {L}(\mathcal {X}),\left\| f \right\| =1\}\\&\le \sup \{\left\| Af \right\| +\left\| Bf \right\| :f\in \mathcal {L}(\mathcal {X}),\left\| f \right\| =1\}\\&\le \sup \{\left\| A \right\| +\left\| B \right\| :f\in \mathcal {L}(\mathcal {X}),\left\| f \right\| =1\} =\left\| A \right\| +\left\| B \right\| . \end{aligned}$$

Thirdly, it separates points because the maximum norm does: if $\left\| A \right\| =0$, then $A=0$ because, for all $f\in \mathcal {L}(\mathcal {X})$, it follows from N1—which we will prove next—that

$$\begin{aligned} 0\le \left\| Af \right\| \le \left\| A \right\| \left\| f \right\| =0 \end{aligned}$$

and therefore, since the maximum norm separates points, that $Af=0$.

In order to prove N1, we consider two cases: $f=0$ and $f\ne 0$. If $f\ne 0$, or equivalently, if $\left\| f \right\| \ne 0$, we let . If $f=0$, or equivalently, if $\left\| f \right\| =0$, we let $g\,{:}{=}\,1$. In both cases, this guarantees that $f=\left\| f \right\| g$ and $\left\| g \right\| =1$ and therefore, we find that

$$\begin{aligned} \left\| Af \right\| =\left\| A(\left\| f \right\| g) \right\| =\left\| \left\| f \right\| Ag \right\| =\left\| f \right\| \left\| Ag \right\| \le \left\| f \right\| \left\| A \right\| , \end{aligned}$$

where the inequality holds because $\left\| f \right\| \ge 0$ and $\left\| Ag \right\| \le \left\| A \right\|$.

Finally, N2 follows rather easily from N1:

$$\begin{aligned} \left\| AB \right\|&=\sup \{\left\| ABf \right\| :f\in \mathcal {L}(\mathcal {X}),\left\| f \right\| =1\}\\&\le \sup \{\left\| A \right\| \left\| Bf \right\| :f\in \mathcal {L}(\mathcal {X}),\left\| f \right\| =1\}\\&=\left\| A \right\| \sup \{\left\| Bf \right\| :f\in \mathcal {L}(\mathcal {X}),\left\| f \right\| =1\}=\left\| A \right\| \left\| B \right\| . \end{aligned}$$

1.2 Proofs of Results in Sect. 3

Proof of L9, L10 and L11

L9 follows from Eq. (4) because we know from L4 that $\left\| \underline{T}f \right\| \le \left\| f \right\|$ for all $f\in \mathcal {L}(\mathcal {X})$. L10 follows from L7 and L4 (in that order). L11 follows from Eq. (4) and L10. $\square$

Proof of Proposition 1

For any $f,g\in \mathcal {L}(\mathcal {X})$, we find that

$$\begin{aligned} \underline{T}(f+g)=\lim _{n\rightarrow +\infty }\underline{T}_{n}(f+g) \ge \lim _{n\rightarrow +\infty }\underline{T}_{n}(f) + \lim _{n\rightarrow +\infty }\underline{T}_{n}(g) =\underline{T}(f) +\underline{T}(g), \end{aligned}$$

where the inequality holds because L2 implies that $\underline{T}_{n}(f+g)\ge \underline{T}_{n}(f)+\underline{T}_{n}(g)$ for all $n\in \mathbb {N}$. Since this is true for any $f,g\in \mathcal {L}(\mathcal {X})$, $\underline{T}_\infty$ satisfies L2. The proof for L1 and L3 is completely analogous. Hence, $\underline{T}$ is a lower transition operator. $\square$

Proof of Proposition 2

The direct implication follows trivially from N1. For the converse implication, we provide a proof by contradiction. Assume that $\lim _{n\rightarrow +\infty }\underline{T}_nf=\underline{T}f$ for all $f\in \mathcal {L}(\mathcal {X})$. Assume ex absurdo that the sequence $\{\underline{T}_n\}_{n\in \mathbb {N}}$ does not converge to $\underline{T}$, or equivalently, that $\limsup _{n\rightarrow +\infty }\left\| \underline{T}_n-\underline{T} \right\| >0$. Then clearly, there is some $\epsilon >0$ and an increasing sequence $n_k$, $k\in \mathbb {N}$, of natural numbers such that $\left\| \underline{T}_{n_k}-\underline{T} \right\| >\epsilon$ for all $k\in \mathbb {N}$. Furthermore, for all $k\in \mathbb {N}$, it follows from $\left\| \underline{T}_{n_k}-\underline{T} \right\| >\epsilon$ and Eq. (4) that there is some $f_k\in \mathcal {L}(\mathcal {X})$ such that $\left\| f_k \right\| =1$ and $\left\| \underline{T}_{n_k}f_k-\underline{T}f_k \right\| >\epsilon$. Since the sequence $f_k$, $k\in \mathbb {N}$, is clearly bounded—because $\left\| f_k \right\| =1$—it follows from the Bolzano–Weierstrass theorem that it has a convergent subsequence, which implies that there is some $f\in \mathcal {L}(\mathcal {X})$ and an increasing sequence $k_i$, $i\in \mathbb {N}$, of natural numbers such that $\lim _{i\rightarrow +\infty }\left\| f_{k_i}-f \right\| =0$. Furthermore, since we have assumed that $\lim _{n\rightarrow +\infty }\underline{T}_nf=\underline{T}f$, it follows that

$$\begin{aligned} \lim _{i\rightarrow +\infty }\left\| \underline{T}_{n_{k_i}}f-\underline{T}f \right\| =\lim _{n\rightarrow +\infty }\left\| \underline{T}_{n}f-\underline{T}f \right\| =0. \end{aligned}$$

Hence, since it follows from L10 that

$$\begin{aligned} \left\| \underline{T}_{n_{k_i}}f_{k_i}-\underline{T}f_{k_i} \right\|&= \left\| \big (\underline{T}_{n_{k_i}}f_{k_i}-\underline{T}_{n_{k_i}}f\big ) + \big (\underline{T}_{n_{k_i}}f-\underline{T}f\big ) + \big (\underline{T}f-\underline{T}f_{k_i}\big ) \right\| \\&\le \left\| \underline{T}_{n_{k_i}}f_{k_i}-\underline{T}_{n_{k_i}}f \right\| + \left\| \underline{T}_{n_{k_i}}f-\underline{T}f \right\| + \left\| \underline{T}f-\underline{T}f_{k_i} \right\| \\&\le \left\| f_{k_i}-f \right\| + \left\| \underline{T}_{n_{k_i}}f-\underline{T}f \right\| + \left\| f_{k_i}-f \right\| , \end{aligned}$$

we find that

$$\begin{aligned} \lim _{i\rightarrow +\infty }\left\| \underline{T}_{n_{k_i}}f_{k_i}-\underline{T}f_{k_i} \right\| =0. \end{aligned}$$

Since $\left\| \underline{T}_{n_{k_i}}f_{k_i}-\underline{T}f_{k_i} \right\|>\epsilon >0$ for all $i\in \mathbb {N}$, this is a contradiction. $\square$

Proof of Proposition 3

Since $\underline{T}$ is non-expansive with respect to the maximum norm (it satisfies L10) and has at least one recurrent point (for $f=1$, L4 implies that $\underline{T}^kf=f$ for all $k\in \mathbb {N}$), this result is an immediate consequence of Theorem 20, with $D=\mathcal {L}(\mathcal {X})$ and $A=\underline{T}$. $\square$

Theorem 20

For any finite-dimensional space D that is equipped with the maximum norm, there is a natural number $p\in \mathbb {N}$ such that, for any non-expansive map A from D to D that has at least one recurrent point, and any $f\in D$, the sequence $\{A^{pn}f\}_{n\in \mathbb {N}}$ converges.

Proof

This result corresponds to the second part of Sine (1990, Theorem 1). $\square$

1.3 Proofs of Results in Sect. 4

Lemma 21

A lower transition operator $\underline{T}$ is regularly absorbing if and only if

$$\begin{aligned} \mathcal {X}'_{\mathrm {RA}}\,{:}{=}\,\left\{ x\in \mathcal {X}:(\exists n\in \mathbb {N})(\forall k\ge n)~ \min \overline{T}^k\mathbb {I}_{x}>0 \right\} \ne \emptyset \end{aligned}$$

and

$$\begin{aligned} \left( \forall x\in \mathcal {X}\!\setminus \!\mathcal {X}'_{\mathrm {RA}}\right) (\exists n\in \mathbb {N})~ \overline{T}^n\mathbb {I}_{\mathcal {X}\!\setminus \!\mathcal {X}'_{\mathrm {RA}}}(x)<1. \end{aligned}$$

Furthermore, the set $\mathcal {X}'_{\mathrm {RA}}$ is equal to the set $\mathcal {X}_{\mathrm {RA}}$ that was used in Definition 3.

Proof of Lemma 21

Consider any $x\in \mathcal {X}_{\mathrm {RA}}$. Definition 3 then implies that there is some $n\in \mathbb {N}$ such that $\min \overline{T}^n\mathbb {I}_{x}>0$, and therefore, because of L4, we know that $\overline{T}^{n+1}\mathbb {I}_{x}=\overline{T}(\overline{T}^n\mathbb {I}_{x})\ge \min \overline{T}^n\mathbb {I}_{x}>0$, which implies that $\min \overline{T}^{n+1}\mathbb {I}_{x}>0$. In the same way, we also find that $\min \overline{T}^{n+2}\mathbb {I}_{x}>0$ and, by continuing in this way, that $\min \overline{T}^k\mathbb {I}_{x}>0$ for all $k\ge n$. Since this holds for all $x\in \mathcal {X}_{\mathrm {RA}}$, it follows that $\mathcal {X}_{\mathrm {RA}}\subseteq \mathcal {X}'_{\mathrm {RA}}$. Since $\mathcal {X}'_{\mathrm {RA}}$ is clearly a subset of $\mathcal {X}_{\mathrm {RA}}$, this implies that $\mathcal {X}'_{\mathrm {RA}}=\mathcal {X}_{\mathrm {RA}}$. Hence, trivially, $\mathcal {X}_{\mathrm {RA}}\ne \emptyset$ if and only if $\mathcal {X}'_{\mathrm {RA}}\ne \emptyset$. The result now follows because it holds for all $x\in \mathcal {X}\!\setminus \!\mathcal {X}'_{\mathrm {RA}}=\mathcal {X}\!\setminus \!\mathcal {X}_{\mathrm {RA}}$ and all $n\in \mathbb {N}$ that

$$\begin{aligned} \overline{T}^n\mathbb {I}_{\mathcal {X}\!\setminus \!\mathcal {X}'_{\mathrm {RA}}}(x) =\overline{T}^n(1-\mathbb {I}_{\mathcal {X}'_{\mathrm {RA}}})(x) =1-\underline{T}^n(\mathbb {I}_{\mathcal {X}'_{\mathrm {RA}}})(x) =1-\underline{T}^n(\mathbb {I}_{\mathcal {X}_{\mathrm {RA}}})(x), \end{aligned}$$

where the second equality follows from L5 and Eq. (5). $\square$

Proof of Proposition 4

Since we know from Lemma 21 that our definition of a regularly absorbing lower transition operator is equivalent to the definition in Hermans and Cooman (2012), this result is identical to Hermans and Cooman (2012, Proposition 3). $\square$

1.4 Proofs of Results in Sect. 5

Proof of R5, R6, R7, R8, and R9

R5 holds because it follows from Eq. (7), R2 and R1 that

$$\begin{aligned} \underline{Q}(f)-\overline{Q}(f)=\underline{Q}(f)+\underline{Q}(-f)\le \underline{Q}(f-f)=\underline{Q}(0)=0. \end{aligned}$$

R6 holds because it follows from R1 and R2 that

$$\begin{aligned} \underline{Q}(f) =\underline{Q}(f)+\underline{Q}(\mu ) \le \underline{Q}(f+\mu ) =\underline{Q}(f+\mu )+\underline{Q}(-\mu ) \le \underline{Q}(f). \end{aligned}$$

R7 holds because it follows from Eq. (7), R6, R2, and R4—in that order—that

$$\begin{aligned} \overline{Q}(\mathbb {I}_{x})(x) =-\underline{Q}(-\mathbb {I}_{x})(x)&=-\underline{Q}(1-\mathbb {I}_{x})(x)\\&=-\underline{Q}(\textstyle \sum _{y\in \mathcal {X}\!\setminus \!\{x\}}\mathbb {I}_{y})(x) \le -\textstyle \sum _{y\in \mathcal {X}\!\setminus \!\{x\}}\underline{Q}(\mathbb {I}_{y})(x) \le 0. \end{aligned}$$

R8 holds because it follows from R6, R2, R3, R4, R7, and R5—in that order—that

$$\begin{aligned} \underline{Q}(f)(x) =\underline{Q}(f-\min f)(x)&\ge \textstyle \sum _{y\in \mathcal {X}}\underline{Q}\big ((f(y)-\min f)\mathbb {I}_{y}\big )(x)\\&=\textstyle \sum _{y\in \mathcal {X}}(f(y)-\min f)\underline{Q}(\mathbb {I}_{y})(x)\\&\ge (f(x)-\min f)\underline{Q}(\mathbb {I}_{x})(x)\\&\ge (\max f-\min f)\underline{Q}(\mathbb {I}_{x})(x) \ge 2\left\| f \right\| \underline{Q}(\mathbb {I}_{x})(x). \end{aligned}$$

We end by proving R9. Consider any $g\in \mathcal {L}(\mathcal {X})$ such that $\left\| g \right\| =1$. It then follows from R8 and R7 that

$$\begin{aligned} \underline{Q}(g)\ge 2\left\| g \right\| \min _{x\in \mathcal {X}}\underline{Q}(\mathbb {I}_{x})(x) \ge -2\max _{x\in \mathcal {X}}\left| \underline{Q}(\mathbb {I}_{x})(x) \right| . \end{aligned}$$

Similarly, since $\left\| -g \right\| \!=\!\left\| g \right\| =1$, we also find that $\underline{Q}(-g)\ge -2\max _{x\in \mathcal {X}}\left| \underline{Q}(\mathbb {I}_{x})(x) \right| .$ By combining these two inequalities with R5 and Eq. (7), it follows that

$$\begin{aligned} -2\max _{x\in \mathcal {X}}\left| \underline{Q}(\mathbb {I}_{x})(x) \right| \le \underline{Q}(g)\le \overline{Q}(g) =-\underline{Q}(-g)\le 2\max _{x\in \mathcal {X}}\left| \underline{Q}(\mathbb {I}_{x})(x) \right| , \end{aligned}$$

which implies that $\left\| \underline{Q}(g) \right\| \le 2\max _{x\in \mathcal {X}}\left| \underline{Q}(\mathbb {I}_{x})(x) \right|$. Since this is true for all $g\in \mathcal {L}(\mathcal {X})$ such that $\left\| g \right\| =1$, R9 now follows from Eq. (4). $\square$

Proof of Proposition 5

L2 and L3 follow trivially from R2 and R3. We only prove L1. Consider any $f\in \mathcal {L}(\mathcal {X})$. Then,

$$\begin{aligned} (I+\Delta \underline{Q})f&=f+\Delta \underline{Q}f =f+\Delta \sum _{x\in \mathcal {X}}\mathbb {I}_{x}\underline{Q}(f)(x)\\&\ge f+\Delta \sum _{x\in \mathcal {X}}(f(x)-\min f)\mathbb {I}_{x}\underline{Q}(\mathbb {I}_{x})(x)\\&\ge f-\Delta \sum _{x\in \mathcal {X}}(f(x)-\min f)\mathbb {I}_{x}\left\| \underline{Q}(\mathbb {I}_{x}) \right\| \\&\ge f-\Delta \sum _{x\in \mathcal {X}}(f(x)-\min f)\mathbb {I}_{x}\left\| \underline{Q} \right\| \\&=f-\Delta \left\| \underline{Q} \right\| (f-\min f)\\&=(f-\min f)\left( 1-\Delta \left\| \underline{Q} \right\| \right) +\min f\ge \min f, \end{aligned}$$

where the first inequality follows from R8 and the third inequality follows from Eq. (4) and the fact that $\left\| \mathbb {I}_{x} \right\| =1$. $\square$

Proof of Proposition 6

Simply check each of the defining properties: R1 holds because L4 implies that $\underline{T}(\mu )=\mu$ for all $\mu \in \mathbb {R}$, R2 follows from L2, R3 follows from L3, and R4 follows from L1. $\square$

Proof of R10, R11, and R12

R10, R11, and R12 are trivial if $\underline{Q}=0$. Therefore, we may assume that $\underline{Q}\ne 0$, which implies that $\left\| \underline{Q} \right\| >0$. Now, let . It then follows from Proposition 5 that $\underline{T}$ is a lower transition operator. We first prove R10. If $\lim _{n\rightarrow +\infty }f_n=f$, then $\lim _{n\rightarrow +\infty }\underline{T}f_n=\underline{T}f$ because of L8. Since $\underline{Q}=\left\| \underline{Q} \right\| (\underline{T}-I)$, this implies that $\lim _{n\rightarrow +\infty }\underline{Q}f_n=\underline{Q}$. R11 holds because

$$\begin{aligned} \left\| \underline{Q}f-\underline{Q}g \right\|&=\Big ||\left\| \underline{Q} \right\| (\underline{T}f-f)-\left\| \underline{Q} \right\| (\underline{T}g-g)\Big ||\\&\le \left\| \underline{Q} \right\| \left\| \underline{T}f-\underline{T}g \right\| +\left\| \underline{Q} \right\| \left\| f-g \right\| \le 2\left\| \underline{Q} \right\| \left\| f-g \right\| , \end{aligned}$$

where the last inequality follows from L10. Similarly, R12 holds because

$$\begin{aligned} \left\| \underline{Q}A-\underline{Q}B \right\|&=\Big ||\left\| \underline{Q} \right\| (\underline{T}A-A)-\left\| \underline{Q} \right\| (\underline{T}B-B)\Big ||\\&\le \left\| \underline{Q} \right\| \left\| \underline{T}A-\underline{T}B \right\| +\left\| \underline{Q} \right\| \left\| A-B \right\| \le 2\left\| \underline{Q} \right\| \left\| A-B \right\| , \end{aligned}$$

where the last inequality follows from L11. $\square$

Proof of Proposition 7

The direct implication follows trivially from N1. We only prove the converse implication. Assume that $\lim _{n\rightarrow +\infty }\underline{Q}_nf=\underline{Q}f$ for all $f\in \mathcal {L}(\mathcal {X})$. For all $x\in \mathcal {X}$, this implies that $\lim _{n\rightarrow +\infty }\underline{Q}_n(\mathbb {I}_{x})(x)=\underline{Q}(\mathbb {I}_{x})(x)$, which in turn implies that there is some $c_x>0$ such that $\vert \underline{Q}(\mathbb {I}_{x})(x)\vert <c_x$ and $\vert \underline{Q}_n(\mathbb {I}_{x})(x)\vert <c_x$ for all $n\in \mathbb {N}$. Let $c\,{:}{=}\,\max _{x\in \mathcal {X}}c_x$. It then follows from R9 that $\vert \vert \underline{Q}\vert \vert \le 2c$ and $\vert \vert \underline{Q}_n\vert \vert \le 2c$ for all $n\in \mathbb {N}$. Choose any . It then follows from Proposition 5 that $\underline{T}\,{:}{=}\,I+\Delta \underline{Q}$ and $\underline{T}_n\,{:}{=}\,I+\Delta \underline{Q}_n$, $n\in \mathbb {N}$, are lower transition operators. Furthermore, since $\lim _{n\rightarrow +\infty }\underline{Q}_nf=\underline{Q}f$ for all $f\in \mathcal {L}(\mathcal {X})$, it follows that $\lim _{n\rightarrow +\infty }\underline{T}_nf=\underline{T}f$ for all $f\in \mathcal {L}(\mathcal {X})$. By applying Proposition 2, we now find that $\lim _{n\rightarrow +\infty }\underline{T}_n=\underline{T}$, which implies that $\lim _{n\rightarrow +\infty }\underline{Q}_n=\underline{Q}$ because

$$\begin{aligned} \left\| \underline{Q}_n-\underline{Q} \right\| =\frac{1}{\Delta }\left\| \Delta \underline{Q}_n-\Delta \underline{Q} \right\| =\frac{1}{\Delta }\left\| (I+\Delta \underline{Q}_n)-(I+\Delta \underline{Q}) \right\| =\frac{1}{\Delta }\left\| \underline{T}_n-\underline{T} \right\| . \end{aligned}$$

$\square$

1.5 Proofs of Results in Sect. 6

Lemma 22

Let $\underline{Q}$ be a lower transition rate operator. Then, for all $f\in \mathcal {L}(\mathcal {X})$, $\underline{T}_sf$ is continuously differentiable on $[0,\infty )$.

Proof

It follows from Eq. (8) that $\underline{T}_sf$ is continuous on $[0,\infty )$. Therefore, since $\underline{Q}$ is a continuous operator [R10], $\underline{Q}\underline{T}_sf$ is also continuous on $[0,\infty )$. Because of Eq. (8), this implies that $\underline{T}_sf$ is continuously differentiable on $[0,\infty )$. $\square$

Lemma 23

Let $\underline{Q}$ be a lower transition rate operator, and let $\Gamma (s)$ be a continuously differentiable map from [0, t] to $\mathcal {L}(\mathcal {X})$ for which ${\frac{\mathrm{d}}{\mathrm{d}s}}\Gamma (s)\ge \underline{Q}\Gamma (s)$ for all $s\in [0,t]$. Then, $\min \Gamma (t)\ge \min \Gamma (0)$.

Proof

Since $\Gamma (s)$ is continuously differentiable on [0, t], it follows that for every $x\in \mathcal {X}$, $\Gamma (s)(x)$ is also continuously differentiable on [0, t], which implies that it is absolutely continuous on [0, t]. Hence, since a minimum of a finite number of absolutely continuous functions is again absolutely continuous, we find that $\min \Gamma (s)$ is absolutely continuous on [0, t], which implies—see Royden and Fitzpatrick (2010, Theorem 10, Section 6.5)—that $\min \Gamma (s)$ has a derivative $\frac{\mathrm{d}}{\mathrm{d}s}\min \Gamma (s)$ almost everywhere on (0, t), that this derivative is Lebesgue integrable over [0, t], and that

$$\begin{aligned} \min \Gamma (t)=\min \Gamma (0)+\int _0^t\Big (\frac{\mathrm{d}}{\mathrm{d}s}\min \Gamma (s)\Big )\mathrm{d}s. \end{aligned}$$

(13)

Consider now any $t^*\in (0,t)$ for which $\min \Gamma (s)$ has a derivative and consider any $x\in \mathcal {X}$ for which $\Gamma (t^*)(x)=\min \Gamma (t^*)$ [clearly, there is at least one such x]. Since $\Gamma (s)(x)$ is differentiable, ${\frac{\mathrm{d}}{\mathrm{d}s}}\Gamma (s)(x)$ exists in $t^*$. Assume ex absurdo that ${\frac{\mathrm{d}}{\mathrm{d}s}\Gamma (s)(x)\vert _{s=t^*}}$ is not equal to $\frac{\mathrm{d}}{\mathrm{d}s}\min \Gamma (s)\vert _{s=t^*}$ or, equivalently, that $\frac{\mathrm{d}}{\mathrm{d}s}(\Gamma (s)(x)-\min \Gamma (s))\vert _{s=t^*}\ne 0$. Then, because $\Gamma (s)(x)-\min \Gamma (s)$ is continuous [since $\Gamma (s)(x)$ and $\min \Gamma (s)$ are both (absolutely) continuous] and because $t^*\in (0,t)$ and $\Gamma (t^*)(x)-\min \Gamma (t^*)=0$, it follows that there is some $t'\in (0,t)$ such that $\Gamma (t')(x)-\min \Gamma (t')<0$ or, equivalently, such that $\Gamma (t')(x)<\min \Gamma (t')$. Since this is clearly a contradiction, it follows that

$$\begin{aligned} \frac{\mathrm{d}}{\mathrm{d}s}\Gamma (s)(x)\Big \vert _{s=t^*}=\frac{\mathrm{d}}{\mathrm{d}s}\min \Gamma (s)\Big \vert _{s=t^*}. \end{aligned}$$

(14)

We also have that

$$\begin{aligned} \frac{\mathrm{d}}{\mathrm{d}s}\Gamma (s)(x)\Big \vert _{s=t^*} \ge \underline{Q}\big (\Gamma (t^*)\big )(x) \ge \big (\Gamma (t^*)(x)-\min \Gamma (t^*)\big )\underline{Q}(\mathbb {I}_{x})(x)=0, \end{aligned}$$

where the second inequality follows from R8 and the last equality follows because $\Gamma (t^*)(x)=\min \Gamma (t^*)$. By combining this result with Eq. (14), we find that, for all $t^*\in (0,t)$ for which $\min \Gamma (s)$ has a derivative, $\frac{\mathrm{d}}{\mathrm{d}s}\min \Gamma (s)\big \vert _{s=t^*}\ge 0$. It therefore follows from Eq. (13) that $\min \Gamma (t)\ge \min \Gamma (0)$. $\square$

Proof of Proposition 8

We first prove L1. Consider any $f\in \mathcal {L}(\mathcal {X})$. It then follows from Lemma 22 that $\underline{T}_sf$ is continuously differentiable on [0, t]. Therefore, and because of Eq. (8), we infer from Lemma 23 that $\min \underline{T}_tf\ge \min \underline{T}_0f$. Since $\underline{T}_0f=f$, this implies that $\min \underline{T}_tf\ge \min f$, which in turn implies that $\underline{T}_tf\ge \min f$.

Let us now prove L2. Consider any $f,g\in \mathcal {L}(\mathcal {X})$. It follows from Lemma 22 that $\underline{T}_sf$, $\underline{T}_sg$, and $\underline{T}_s(f+g)$ are continuously differentiable on [0, t], which implies that $\Gamma (s)\,{:}{=}\,\underline{T}_s(f+g)-\underline{T}_sf-\underline{T}_sg$ is continuously differentiable on [0, t]. Furthermore, for all $s\in [0,t]$, it follows from Eq. (8) and R2 that

$$\begin{aligned} \frac{\mathrm{d}}{\mathrm{d}s}\Gamma (s)&=\frac{\mathrm{d}}{\mathrm{d}s}\underline{T}_s(f+g)-\frac{\mathrm{d}}{\mathrm{d}s}\underline{T}_sf-\frac{\mathrm{d}}{\mathrm{d}s}\underline{T}_sg\\&=\underline{Q}\underline{T}_s(f+g)-\underline{Q}\underline{T}_sf-\underline{Q}\underline{T}_sg\\&=\underline{Q}\big (\Gamma (s)+\underline{T}_sf+\underline{T}_sg\big )-\underline{Q}\underline{T}_sf-\underline{Q}\underline{T}_sg \ge \underline{Q}\Gamma (s). \end{aligned}$$

Therefore, we infer from Lemma 23 that $\min \Gamma (t)\ge \min \Gamma (0)$. Since $\Gamma (0)=\underline{T}_0(f+g)-\underline{T}_0f-\underline{T}_0g=0$, this implies that $\min \Gamma (t)\ge 0$, which in turn implies that $\Gamma (t)\ge 0$ or, equivalently, that $\underline{T}_t(f+g)\ge \underline{T}_tf+\underline{T}_tg$.

We end by proving L3. Consider any $f\in \mathcal {L}(\mathcal {X})$ and $\lambda \ge 0$. It then follows from Eq. (8) and R3 that

$$\begin{aligned} \frac{\mathrm{d}}{\mathrm{d}s}(\lambda \underline{T}_sf) =\lambda \frac{\mathrm{d}}{\mathrm{d}s}\underline{T}_sf =\lambda \underline{Q}\underline{T}_sf =\underline{Q}(\lambda \underline{T}_sf) \quad \text {for all } s\ge 0. \end{aligned}$$

Since we also have that $\lambda \underline{T}_0f=\lambda f=\underline{T}_0(\lambda f)$, it follows that $\lambda \underline{T}_sf$ satisfies the same differential equation and boundary condition as $\underline{T}_s(\lambda f)$. Since we know that this differential equation and boundary condition lead to a unique solution on $[0,\infty )$, it follows that $\underline{T}_t(\lambda f)=\lambda \underline{T}_t(f)$. $\square$

Lemma 24

Let $\underline{Q}$ be a lower transition rate operator. Then,

Proof

For any $f\in \mathcal {L}(\mathcal {X})$, it follows from Eq. (8) that $\underline{T}_tf$ is continuous on $[0,\infty )$, which implies that $\lim _{\Delta \rightarrow 0}\underline{T}_\Delta f=\underline{T}_0f=f$. Therefore, we infer from Proposition 2 that $\lim _{\Delta \rightarrow 0}\underline{T}_\Delta =I$, which proves the first part of this lemma. We end by proving the second part. For any $f\in \mathcal {L}(\mathcal {X})$, it follows from Eq. (8) that

Therefore, and since, for all $\Delta >0$, is a lower transition rate operator because of Proposition 6, it follows from Proposition 7 that . $\square$

Proof of Proposition 9

Since $\underline{T}_0f\,{:}{=}\,f$ for all $f\in \mathcal {L}(\mathcal {X})$, it follows trivially that $\underline{T}_0=I$. Consider now any $t\ge 0$. In order to prove that $\frac{\mathrm{d}}{\mathrm{d}t}\underline{T}_t=\underline{Q}\underline{T}_t$, it suffices to show that for all $\epsilon >0$, there is some $\delta >0$ such that

$$\begin{aligned} \left\| \frac{\underline{T}_s-\underline{T}_t}{s-t}-\underline{Q}\underline{T}_t \right\|<\epsilon \quad \text {for all } s\ge 0\text { such that }0<\left| t-s \right| <\delta . \end{aligned}$$

(15)

So consider any $\epsilon >0$. If $\underline{Q}=0$, Eq. (15) is trivially true because, since I clearly satisfies Eq. (8), it follows from the unicity of the solution of Eq. (8) that $\underline{T}_t=\underline{T}_s=I$. Therefore, in the remainder of this proof, we may assume that $\underline{Q}\ne 0$, which implies that $\Vert \underline{Q}\Vert \ne 0$. It then follows from Lemma 24 that there are $\delta _1>0$ and $\delta _2>0$ such that for all $0<q<\delta _1$ and for all $0<\Delta <\delta _2$. Now define $\delta \,{:}{=}\,\min \{\delta _1,\delta _2\}$ and consider any $s\ge 0$ such that $0<\left| t-s \right| <\delta$. Let $u\,{:}{=}\,\min \{s,t\}$, $\Delta \,{:}{=}\,\left| t-s \right|$ and $q\,{:}{=}\,t-u$, which implies that $0\le q\le \Delta <\delta \le \delta _1$ and $0<\Delta <\delta \le \delta _2$. If $q=0$, then $\underline{T}_q=\underline{T}_0=I$ and therefore $\Vert \underline{Q}\underline{T}_q-\underline{Q}\Vert =\Vert \underline{Q}-\underline{Q}\Vert =0$. If $q>0$, it follows from R12 and Proposition 8 that . Hence, in all cases, we find that . The result now holds because

$$\begin{aligned} \left\| \frac{\underline{T}_s-\underline{T}_t}{s-t}-\underline{Q}\underline{T}_t \right\|&=\left\| \frac{\underline{T}_{\Delta +u}-\underline{T}_u}{\Delta }-\underline{Q}\underline{T}_{q+u} \right\| =\left\| \frac{\underline{T}_\Delta \underline{T}_u-\underline{T}_u}{\Delta }-\underline{Q}\underline{T}_q\underline{T}_u \right\| \\&\le \left\| \frac{\underline{T}_{\Delta }-I}{\Delta }-\underline{Q}\underline{T}_{q} \right\| \left\| \underline{T}_u \right\| \le \left\| \frac{\underline{T}_{\Delta }-I}{\Delta }-\underline{Q}\underline{T}_{q} \right\| \\&\le \left\| \frac{\underline{T}_{\Delta }-I}{\Delta }-\underline{Q} \right\| +\left\| \underline{Q}\underline{T}_q-\underline{Q} \right\| <\frac{\epsilon }{2}+\frac{\epsilon }{2}=\epsilon , \end{aligned}$$

where the second equality follows from Eq. (10), the first inequality follows from N2 and the second inequality follows from Proposition 8 and L9. $\square$

Proof of Proposition 10

The result is trivial if $t=0$. In the remainder of this proof, we assume that $t>0$. The result for $\underline{Q}=0$ is also trivial because, since I then clearly satisfies Eq. (8), it follows from the unicity of the solution of Eq. (8) that $\underline{T}_t=I$. Therefore, in the remainder of this proof, we assume that $\underline{Q}\ne 0$, which implies that $\left\| \underline{Q} \right\| \ne 0$. We will now prove that for every $\epsilon >0$, there is some $n\in \mathbb {N}$ such that

$$\begin{aligned} \left\| \underline{T}_t-\left( I+\frac{t}{k}\underline{Q}\right) ^k \right\| <\epsilon \quad \text {for all }k\ge n. \end{aligned}$$

So consider any $\epsilon >0$. It then follows from Lemma 24 that there is some $\delta >0$ such that for all $0<\Delta <\delta$. Now choose $n\in \mathbb {N}$ such that and consider any $k\ge n$. Let , which implies that $0<\Delta <\delta$ and $\Delta \left\| \underline{Q} \right\| <1$. Then

$$\begin{aligned}&\left\| (\underline{T}_\Delta )^k-(I+\Delta \underline{Q})^k \right\| \\&\quad =\left\| (\underline{T}_\Delta )^k-(\underline{T}_\Delta )^{k-1}(I+\Delta \underline{Q}) +(\underline{T}_\Delta )^{k-1}(I+\Delta \underline{Q})-(I+\Delta \underline{Q})^k \right\| \\&\quad \le \left\| (\underline{T}_\Delta )^k-(\underline{T}_\Delta )^{k-1}(I+\Delta \underline{Q}) \right\| +\left\| (\underline{T}_\Delta )^{k-1}(I+\Delta \underline{Q})-(I+\Delta \underline{Q})^k \right\| \\&\quad \le \left\| \underline{T}_\Delta -(I+\Delta \underline{Q}) \right\| +\left\| (\underline{T}_\Delta )^{k-1} -(I+\Delta \underline{Q})^{k-1} \right\| \left\| I+\Delta \underline{Q} \right\| \\&\quad \le \left\| \underline{T}_\Delta -(I+\Delta \underline{Q}) \right\| +\left\| (\underline{T}_\Delta )^{k-1} -(I+\Delta \underline{Q})^{k-1} \right\| , \end{aligned}$$

where the second inequality follows from Proposition 8 and L11 [by applying them repeatedly] and N2, and the third inequality follows from L9 and Proposition 5. By continuing in this way, we find that

$$\begin{aligned} \left\| (\underline{T}_\Delta )^k-(I+\Delta \underline{Q})^k \right\| \le k\left\| \underline{T}_\Delta -(I+\Delta \underline{Q}) \right\| . \end{aligned}$$

Therefore, since

$$\begin{aligned} \left\| \underline{T}_\Delta -(I+\Delta \underline{Q}) \right\| =\Delta \left\| \frac{\underline{T}_\Delta -I}{\Delta }-\underline{Q} \right\| <\Delta \frac{\epsilon }{t}=\frac{t}{k}\frac{\epsilon }{t}=\frac{\epsilon }{k}, \end{aligned}$$

and because it follows from Eq. (10) that , we find that

$$\begin{aligned} \left\| \underline{T}_t-\left( I+\frac{t}{k}\underline{Q}\right) ^k \right\| =\left\| (\underline{T}_{\Delta })^k-(I+\Delta \underline{Q})^k \right\| \le k\left\| \underline{T}_\Delta -(I+\Delta \underline{Q}) \right\| <k\frac{\epsilon }{k}=\epsilon . \end{aligned}$$

$\square$

1.6 Proofs of Results in Sect. 7

Lemma 25

Let $\underline{Q}$ be a lower transition rate operator. Then, $\lim _{\Delta \rightarrow 0+}\underline{T}_\Delta =I$.

Proof

Fix any $\epsilon >0$ and consider any $\delta _1,\alpha >0$ such that $\delta _1\left( \alpha +\left\| \underline{Q} \right\| \right) <\epsilon$. Since Proposition 9 implies that $\frac{\mathrm{d}}{\mathrm{d}t}\underline{T}_t\big \vert _{t=0}=\underline{Q}\underline{T}_0=\underline{Q}I=\underline{Q}$, there is some $\delta _2>0$ such that for all $0<\Delta <\delta _2$. Hence, if we let $\delta _\epsilon \,{:}{=}\,\min \{\delta _1,\delta _2\}$, then

$$\begin{aligned} \left\| \underline{T}_\Delta -I \right\|= & {} \Delta \left\| \frac{\underline{T}_\Delta -I}{\Delta } \right\| \le \Delta \left\| \frac{\underline{T}_\Delta -I}{\Delta }-\underline{Q} \right\| \\&+\,\Delta \left\| \underline{Q} \right\| \le \Delta (\alpha +\left\| Q \right\| ) \le \delta _1(\alpha +\left\| Q \right\| )<\epsilon \end{aligned}$$

for all $0\le \Delta <\delta _\epsilon$. Since $\epsilon >0$ is arbitrary, this implies that $\lim _{\Delta \rightarrow 0+}\underline{T}_\Delta =I$. $\square$

Proof of Proposition 11

Let p be the natural number whose existence is guaranteed by Proposition 3, and let . Fix any $f\in \mathcal {L}(\mathcal {X})$. Since $\underline{T}_t$ is a lower transition operator, it follows from Proposition 3 that $\underline{T}_t^{pn}f$ converges to a limit in $\mathcal {L}(\mathcal {X})$ as n goes to infinity. Hence, since we know from Eq. (10) that $\underline{T}_t^{pn}=\underline{T}_{tpn}=\underline{T}_{\Delta n}$, it follows that $\underline{T}_{\Delta n}f$ converges to a limit in $\mathcal {L}(\mathcal {X})$ as n goes to infinity. We will denote this limit by $\underline{T}f$. By doing this for every $f\in \mathcal {L}(\mathcal {X})$, we obtain an operator $\underline{T}$. Since we know from Proposition 8 that the operators $\underline{T}_{\Delta n}$, $n\in \mathbb {N}$, are lower transition operators, it now follows from Proposition 1 that $\underline{T}$ is a lower transition operator. $\square$

Proof of Theorem 12

Consider any $\Delta >0$, and let $\underline{T}_{\infty }\,{:}{=}\,\lim _{n\rightarrow +\infty }\underline{T}_{\Delta n}$ be the lower transition operator whose existence is guaranteed by Proposition 11. We will now prove that $\underline{T}_{\infty }=\lim _{t\rightarrow +\infty }\underline{T}_t$, or equivalently, that

$$\begin{aligned} (\forall \epsilon >0)\,(\exists t_\epsilon \ge 0)\,(\forall t\ge t_\epsilon )~\left\| \underline{T}_{t}-\underline{T}_\infty \right\| <\epsilon . \end{aligned}$$

To this end, fix any $\epsilon >0$. It then follows from Lemma 25 that there is some $\delta _1>0$ such that for all $0<\delta <\delta _1$. Let k be any natural number such that $\Delta <\delta _1k$, and let . It then follows from Proposition 11 that the sequence $\{\underline{T}_{\Delta ^*i}\}_{i\in \mathbb {N}}$ converges to a lower transition operator that, since $\{\underline{T}_{\Delta n}\}_{n\in \mathbb {N}}$ is clearly a subsequence of $\{\underline{T}_{\Delta ^* i}\}_{i\in \mathbb {N}}$, is equal to $\underline{T}_\infty$. Hence, there is some $i_\epsilon \in \mathbb {N}$ such that for all $i\ge i_\epsilon$. Now, let $t_\epsilon \,{:}{=}\,i_\epsilon \Delta ^*$ and consider any $t\ge t_\epsilon$. We will prove that $\left\| \underline{T}_t-\underline{T}_\infty \right\| <\epsilon$. In order to do that, we let i be the unique natural number such that $\Delta ^*i< t\le \Delta ^*(i+1)$ and let $\delta \,{:}{=}\,t-\Delta ^*i$. Since we then clearly have that $i\ge i_\epsilon$ and $0<\delta \le \Delta ^*<\delta _1$, we find that indeed, as required,

where the first equality follows from Eq. (10) and the fact that $\underline{T}_0=I$—see Proposition 9—and where the second inequality follows from L11. $\square$

1.7 Proofs of Results in Sect. 8

Proof of Proposition 13

First assume that $\underline{Q}$ is ergodic. For all $f\in \mathcal {L}(\mathcal {X})$, it then follows from that $\lim _{s\rightarrow +\infty }\underline{T}_s f$ exists and is a constant function. Therefore, for all $f\in \mathcal {L}(\mathcal {X})$, it follows from Eq. (10) that

$$\begin{aligned} \lim _{n\rightarrow +\infty }\underline{T}_t^n f=\lim _{n\rightarrow +\infty }\underline{T}_{nt} f=\lim _{s\rightarrow +\infty }\underline{T}_s f \end{aligned}$$

exists and is a constant function, which implies that $\underline{T}_t$ is ergodic.

Next, assume that $\underline{T}_t$ is ergodic. This means that, for all $f\in \mathcal {L}(\mathcal {X})$, there is some $c_f\in \mathbb {R}$ such that

$$\begin{aligned} (\forall \epsilon >0)(\exists n\in \mathbb {N})(\forall k\ge n) \left\| \underline{T}_t^k f-c_f \right\| <\epsilon . \end{aligned}$$

(16)

Consider now any $f\in \mathcal {L}(\mathcal {X})$ and any $\epsilon >0$. It then follows from Eq. (16) that there is some $n_\epsilon \in \mathbb {N}$ such that $\left\| \underline{T}_t^{n_\epsilon } f-c_f \right\| <\epsilon$, which, because of Proposition 8 and L5, implies that $\left\| \underline{T}_t^{n_\epsilon }(f-c_f) \right\| <\epsilon$. Now, let $s_\epsilon \,{:}{=}\,n_\epsilon t$. Then, for all $s\ge s_\epsilon$, we have that

$$\begin{aligned} \left\| \underline{T}_s f - c_f \right\|= & {} \left\| \underline{T}_s(f - c_f) \right\| =\left\| \underline{T}_{s-s_\epsilon }\underline{T}_{t}^{n_\epsilon }(f - c_f) \right\| \\\le & {} \left\| \underline{T}_{s-s_\epsilon } \right\| \left\| \underline{T}_{t}^{n_\epsilon }(f - c_f) \right\| <\epsilon , \end{aligned}$$

where the first equality follows from Proposition 8 and L5, the second equality follows from Eq. (10), the first inequality follows from N1, and the last inequality follows from Proposition 8, L9 and the fact that $\left\| \underline{T}_t^{n_\epsilon } f-c_f \right\| <\epsilon$. Hence, we have found that for all $\epsilon >0$, there is some $s_\epsilon >0$ such that $\left\| \underline{T}_sf-c_f \right\| <\epsilon$ for all $s\ge s_\epsilon$. In other words: $\lim _{s\rightarrow +\infty }\underline{T}_s f=c_f$. Since this is true for all $f\in \mathcal {L}(\mathcal {X})$, it follows from Definition 6 that $\underline{Q}$ is ergodic. $\square$

Proof of Corollary 14

Immediate consequence of Propositions 4, 8 and 13. $\square$

Lemma 26

Let $\underline{Q}$ be a lower transition rate operator. Consider any $f\in \mathcal {L}(\mathcal {X})$ and $x\in \mathcal {X}$ such that $f(x)>\min f$. Then, for all $t\ge 0$: $\underline{T}_t f(x)>\min f$.

Proof

Since we know from Lemma 22 that $\underline{T}_tf$ is continuously differentiable on $[0,\infty )$, we know that $r_t\,{:}{=}\,\underline{T}_t f-\min f$ and therefore also $r_t(x)$ is continuously differentiable on $[0,\infty )$. Furthermore, for all $t\ge 0$, it follows from Proposition 8 and L1 that $r_t\ge 0$, which in turn implies that

$$\begin{aligned} \frac{\mathrm{d}}{\mathrm{d}t}r_t(x)= & {} \frac{\mathrm{d}}{\mathrm{d}t}\underline{T}_tf(x) =\underline{Q}(\underline{T}_tf)(x) =\underline{Q}(r_t)(x) \\\ge & {} \sum _{y\in \mathcal {X}}r_t(y)\underline{Q}(\mathbb {I}_{y})(x) \ge r_t(x)\underline{Q}(\mathbb {I}_{x})(x), \end{aligned}$$

where the second equality follows from Eq. (8), the third equality follows from R6, the first inequality follows from R2 and R3, and the last inequality follows from R4. Hence, for all $t\ge 0$, we find that $r_t(x)\ge r_0(x)e^{[\underline{Q}(\mathbb {I}_{x})(x)]t}$. Since we also know that $r_0(x)=\underline{T}_0f(x)-\min f=f(x)-\min f>0$, this implies that for all $t\ge 0$:

$$\begin{aligned} \underline{T}_tf(x)-\min f=r_t(x)\ge r_0(x)e^{[\underline{Q}(\mathbb {I}_{x})(x)]t}>0. \end{aligned}$$

$\square$

Lemma 27

Let $\underline{Q}$ be a lower transition rate operator. Consider any $f\in \mathcal {L}(\mathcal {X})$, $x\in \mathcal {X}$ and $s\ge 0$ such that $\underline{T}_sf(x)>\min f$. Then, for all $t\ge s$: $\underline{T}_tf(x)>\min f$.

Proof

Because of Eq. (10), it suffices to prove that $\underline{T}_{t-s}\underline{T}_s f(x)>\min f$. We consider two cases: $\min \underline{T}_s f>\min f$ and $\min \underline{T}_s f=\min f$; $\min \underline{T}_s f<\min f$ is not possible because of Proposition 8 and L1. If $\min \underline{T}_s f>\min f$, it follows from Proposition 8 and L1 that $\underline{T}_{t-s}\underline{T}_s f(x)\ge \min \underline{T}_s f>\min f$. If $\min \underline{T}_s f=\min f$, then $\underline{T}_sf(x)>\min \underline{T}_s f$ and therefore, because of Lemma 26, $\underline{T}_{t-s}\underline{T}_s f(x)>\min \underline{T}_s f=\min f$. $\square$

Lemma 28

Let $\underline{Q}$ be a lower transition rate operator. Consider any $f\in \mathcal {L}(\mathcal {X})$, $x\in \mathcal {X}$ and $t,s>0$. Then,

$$\begin{aligned} \underline{T}_tf(x)>\min f\Leftrightarrow \underline{T}_sf(x)>\min f. \end{aligned}$$

Proof

For any $\tau \ge 0$, let

$$\begin{aligned} \mathcal {X}_\tau \,{:}{=}\,\{y\in \mathcal {X}:\underline{T}_\tau f(y)>\min f\}. \end{aligned}$$

(17)

It then follows from Lemma 27 that $\mathcal {X}_\tau$ is an increasing function of $\tau$:

$$\begin{aligned} \tau \le \tau '~\Rightarrow ~\mathcal {X}_{\tau }\subseteq \mathcal {X}_{\tau '}. \end{aligned}$$

(18)

Assume ex absurdo that

$$\begin{aligned} (\forall \tau '>0)~(\forall \mathcal {X}'\subseteq \mathcal {X})~(\exists \tau \in (0,\tau ']) ~\mathcal {X}_\tau \ne \mathcal {X}'. \end{aligned}$$

(19)

Choose any $\tau _1>0$. Then clearly, $\mathcal {X}_{\tau _1}\subseteq \mathcal {X}$. Therefore, due to Eq. (19), we know that there is some $0<\tau _2<\tau _1$ such that $\mathcal {X}_{\tau _2}\ne \mathcal {X}_{\tau _1}$, which, because of Eq. (18), implies that $\mathcal {X}_{\tau _2}\subset \mathcal {X}_{\tau _1}$. Similarly, we infer that there is some $0<\tau _3<\tau _2$ such that $\mathcal {X}_{\tau _3}\subset \mathcal {X}_{\tau _2}$. By continuing in this way, we obtain an infinite sequence of time points $\tau _1>\tau _2>\tau _3>\cdots>\tau _i>\cdots >0$ such that $\mathcal {X}\supseteq \mathcal {X}_{\tau _1}\supset \mathcal {X}_{\tau _2}\supset \mathcal {X}_{\tau _3}\supset \cdots \supset \mathcal {X}_{\tau _i}\supset \cdots$. Since $\mathcal {X}$ is a finite set, this is a contradiction, leading us to conclude that Eq. (19) is false. This implies that there is some $\tau ^*>0$ and $\mathcal {X}^*\subseteq \mathcal {X}$ such that

$$\begin{aligned} (\forall \tau \in (0,\tau ^*])~\mathcal {X}_\tau =\mathcal {X}^*. \end{aligned}$$

(20)

Fix any $\tau >\tau ^*$ and choose $n\in \mathbb {N}$ high enough such that . It then follows from Eq. (20) that . Furthermore, because of Proposition 8, L1 and Eq. (17), we know that for all $y\in \mathcal {X}\!\setminus \!\mathcal {X}^*$. Therefore, we infer from Eq. (17) that there is some $\lambda >0$ such that

which, because of Eq. (10), Proposition 8 and L5 implies that

Hence, it follows from Proposition 8, L5, Eq. (10), L6 and L3 that

(21)

Consider now any $y\in \mathcal {X}\!\setminus \!\mathcal {X}^*$. Since , it follows from Eq. (21) that $\underline{T}_{\tau }f(y)\le \min f$, which in turn implies that $y\notin \mathcal {X}_\tau$. Since this holds for all $y\in \mathcal {X}\!\setminus \!\mathcal {X}^*$, we find that . Furthermore, since , it follows from Eq. (18) that . Hence, we find that $\mathcal {X}_\tau =\mathcal {X}^*$. Since this is true for all $\tau >\tau ^*$, it follows from Eq. (20) that

$$\begin{aligned} \mathcal {X}_\tau =\mathcal {X}^* \quad \text {for all }\tau >0. \end{aligned}$$

Therefore, due to Eq. (17), we find that

$$\begin{aligned} \underline{T}_tf(x)>\min f \Leftrightarrow x\in \mathcal {X}_t \Leftrightarrow x\in \mathcal {X}_s \Leftrightarrow \underline{T}_sf(x)>\min f. \end{aligned}$$

$\square$

Lemma 29

Let $\underline{Q}$ be a lower transition rate operator. Consider any $f\in \mathcal {L}(\mathcal {X})$, $x\in \mathcal {X}$ and $s\ge 0$ such that $\overline{T}_sf(x)>\min f$. Then, for all $t\ge s$: $\overline{T}_tf(x)>\min f$.

Proof

Because of Eqs. (5) and (10), it suffices to prove that $\overline{T}_{t-s}\overline{T}_s f(x)>\min f$. We consider two cases: $\min \overline{T}_s f>\min f$ and $\min \overline{T}_s f=\min f$; $\min \overline{T}_s f<\min f$ is not possible because of Proposition 8 and L4. If $\min \overline{T}_s f>\min f$, it follows from Proposition 8 and L4 that $\overline{T}_{t-s}\overline{T}_s f(x)\ge \min \overline{T}_s f>\min f$. If $\min \overline{T}_s f=\min f$, then $\overline{T}_sf(x)>\min \overline{T}_s f$ and therefore, it follows from Proposition 8, L4 and Lemma 26 that $\overline{T}_{t-s}\overline{T}_s f(x)\ge \underline{T}_{t-s}\overline{T}_s f(x)>\min \overline{T}_s f=\min f$. $\square$

Lemma 30

Let $\underline{Q}$ be a lower transition rate operator. Consider any $f\in \mathcal {L}(\mathcal {X})$, $x\in \mathcal {X}$ and $t,s>0$. Then,

$$\begin{aligned} \overline{T}_tf(x)>\min f\Leftrightarrow \overline{T}_sf(x)>\min f. \end{aligned}$$

Proof

For any $\tau \ge 0$, let

$$\begin{aligned} \mathcal {X}_\tau \,{:}{=}\,\{y\in \mathcal {X}:\overline{T}_\tau f(y)>\min f\}. \end{aligned}$$

(22)

It then follows from Lemma 29 that $\mathcal {X}_\tau$ is an increasing function of $\tau$:

$$\begin{aligned} \tau \le \tau '~\Rightarrow ~\mathcal {X}_{\tau }\subseteq \mathcal {X}_{\tau '}. \end{aligned}$$

(23)

Using an argument that is identical to that in Lemma 28, we find that this implies that there is some $\tau ^*>0$ and $\mathcal {X}^*\subseteq \mathcal {X}$ such that

$$\begin{aligned} (\forall \tau \in (0,\tau ^*])~\mathcal {X}_\tau =\mathcal {X}^*. \end{aligned}$$

(24)

Fix any $\tau >\tau ^*$ and choose $n\in \mathbb {N}$ high enough such that . It then follows from Eq. (24) that . Furthermore, because of Proposition 8, L4 and Eq. (22), we know that for all $y\in \mathcal {X}\!\setminus \!\mathcal {X}^*$. Therefore, we infer from Eq. (22) that there is some $\lambda >0$ such that

which, because of Eqs. (5) and (10), Proposition 8 and L5 implies that

Hence, it follows from Proposition 8, L5, Eqs. (5) and (10), L6 and L3 that

(25)

Consider now any $y\in \mathcal {X}\!\setminus \!\mathcal {X}^*$. Since , it follows from Eq. (25) that $\overline{T}_{\tau }f(y)\le \min f$, which in turn implies that $y\notin \mathcal {X}_\tau$. Since this holds for all $y\in \mathcal {X}\!\setminus \!\mathcal {X}^*$, we find that . Furthermore, since , it follows from Eq. (23) that . Hence, we find that $\mathcal {X}_\tau =\mathcal {X}^*$. Since this is true for all $\tau >\tau ^*$, it follows from Eq. (24) that

$$\begin{aligned} \mathcal {X}_\tau =\mathcal {X}^* \quad \text {for all }\tau >0. \end{aligned}$$

Therefore, due to Eq. (22), we find that

$$\begin{aligned} \overline{T}_tf(x)>\min f \Leftrightarrow x\in \mathcal {X}_t \Leftrightarrow x\in \mathcal {X}_s \Leftrightarrow \overline{T}_sf(x)>\min f. \end{aligned}$$

$\square$

Proposition 31

Let $\underline{Q}$ be a lower transition rate operator. Then, for all $f\in \mathcal {L}(\mathcal {X})$, $x\in \mathcal {X}$ and $t,s>0$:

$$\begin{aligned} f(x)>\min f~&\Rightarrow ~\underline{T}_tf(x)>\min f~\Leftrightarrow ~\underline{T}_sf(x)>\min f;\\ f(x)<\max f~&\Rightarrow ~\underline{T}_tf(x)<\max f~\Leftrightarrow ~\underline{T}_sf(x)<\max f;\\ f(x)>\min f~&\Rightarrow ~\overline{T}_tf(x)>\min f~\Leftrightarrow ~\overline{T}_sf(x)>\min f;\\ f(x)<\max f~&\Rightarrow ~\overline{T}_tf(x)<\max f~\Leftrightarrow ~\overline{T}_sf(x)<\max f; \end{aligned}$$

Proof

The first implication [$f(x)>\min f\Rightarrow \underline{T}_tf(x)>\min f$] follows from Lemma 26, and the first equivalence [$\underline{T}_tf(x)>\min f\Leftrightarrow \underline{T}_sf(x)>\min f$] follows from Lemma 28. Since $\overline{T}_0f=f$, the third implication [$f(x)>\min f\Rightarrow \overline{T}_tf(x)$] follows from Lemma 29. The third equivalence [$\overline{T}_tf(x)>\min f\Leftrightarrow \overline{T}_sf(x)>\min f$] follows from Lemma 30. The rest of the result now follows directly because we know from Eq. (5) that $\overline{T}_tf(x)=-\underline{T}_t(-f)(x)$, $\overline{T}_sf(x)=-\underline{T}_s(-f)(x)$ and $\max f=-\min (-f)$. $\square$

Corollary 32

Let $\underline{Q}$ be a lower transition rate operator. Then, for all $A\subseteq \mathcal {X}$, all $x\in \mathcal {X}$ and all $t, s>0$:

$$\begin{aligned} x\in A~&\Rightarrow ~\underline{T}_t\mathbb {I}_{A}(x)>0~\Leftrightarrow ~\underline{T}_s\mathbb {I}_{A}(x)>0;\\ x\notin A~&\Rightarrow ~\underline{T}_t\mathbb {I}_{A}(x)<1~\Leftrightarrow ~\underline{T}_s\mathbb {I}_{A}(x)<1;\\ x\in A~&\Rightarrow ~\overline{T}_t\mathbb {I}_{A}(x)>0~\Leftrightarrow ~\overline{T}_s\mathbb {I}_{A}(x)>0;\\ x\notin A~&\Rightarrow ~\overline{T}_t\mathbb {I}_{A}(x)<1~\Leftrightarrow ~\overline{T}_s\mathbb {I}_{A}(x)<1. \end{aligned}$$

Proof

If $A=\emptyset$ or $A=\mathcal {X}$, the result follows trivially from Proposition 8 and L4. In all other cases, the result follows directly from Proposition 31, with $f=\mathbb {I}_{A}$. $\square$

Proof of Proposition 15

If $t=0$, we know from Proposition 9 that $\underline{T}_t=I$, which implies that $\underline{T}_t^n=I=\underline{T}_t$ and $\overline{T}_t^n=I=\overline{T}_t$ for all $n\in \mathbb {N}$. In that case, Definitions 3 and 4 are trivially equal. If $t>0$, then since we know from Eqs. (10) and (5) that $\underline{T}_t^n=\underline{T}_{nt}$ and $\overline{T}_t^n=\overline{T}_{nt}$ for all $n\in \mathbb {N}$, the equivalence of Definitions 3 and 4 follows directly from Corollary 32. $\square$

Proof of Corollary 16

This result is a trivial consequence of Corollary 14 and Proposition 15. $\square$

Proof of Proposition 17

First assume that $\overline{T}_t\mathbb {I}_{x}(y)>0$. It then follows from Proposition 10 and Eqs. (5) and (7) that there is some $n\in \mathbb {N}$ such that $n\ge t\left\| \underline{Q} \right\|$ and

$$\begin{aligned} \left( \left( I+\frac{t}{n}\overline{Q}\right) ^n\mathbb {I}_{x}\right) (y) >0. \end{aligned}$$

(26)

Let and define $\underline{T}_*\,{:}{=}\,I+\Delta \underline{Q}$. Since $n\ge t\left\| \underline{Q} \right\|$ implies that $\Delta \left\| \underline{Q} \right\| \le 1$, it then follows from Proposition 5 that $\underline{T}_*$ is a lower transition operator. Therefore, for all $z\in \mathcal {X}$ and $w\in \mathcal {X}$, it follows from L4 that $c(w,z)\,{:}{=}\,\big (\overline{T}_*\mathbb {I}_{z}\big )(w)\ge 0$. For all $z\in \mathcal {X}$, we now have that

$$\begin{aligned} \overline{T}_*\mathbb {I}_{z} =\sum _{w\in \mathcal {X}}\mathbb {I}_{w}\cdot \big (\overline{T}_*\mathbb {I}_{z}\big )(w) =\sum _{w\in \mathcal {X}}c(w,z)\mathbb {I}_{w}. \end{aligned}$$

Hence, for all $x_n\in \mathcal {X}$, it follows from Eq. (5), L2 and L6 that

$$\begin{aligned} \overline{T}_*^n\mathbb {I}_{x_n}&=\overline{T}_*^{n-1}\overline{T}_*\mathbb {I}_{x_n} = \overline{T}_*^{n-1}\sum _{x_{n-1}\in \mathcal {X}}c(x_{n-1},x_n)\mathbb {I}_{x_{n-1}}\\&\le \sum _{x_{n-1}\in \mathcal {X}}c(x_{n-1},x_n)\overline{T}_*^{n-1}\mathbb {I}_{x_{n-1}} \end{aligned}$$

and, by continuing in this way, that

$$\begin{aligned} \overline{T}_*^n\mathbb {I}_{x_n} \le \sum _{x_{n-1}\in \mathcal {X}}c(x_{n-1},x_n) \sum _{x_{n-2}\in \mathcal {X}}c(x_{n-2},x_{n-1}) \cdots \sum _{x_1\in \mathcal {X}}c(x_1,x_2)\overline{T}_*\mathbb {I}_{x_1}. \end{aligned}$$

Therefore, for all $x_n\in \mathcal {X}$ and $x_0\in \mathcal {X}$, we find that

$$\begin{aligned} \big (\overline{T}_*^n\mathbb {I}_{x_n}\big )(x_0)&\le \sum _{x_{n-1}\in \mathcal {X}}c(x_{n-1},x_n) \sum _{x_{n-2}\in \mathcal {X}}c(x_{n-2},x_{n-1}) \cdots \sum _{x_1\in \mathcal {X}}c(x_1,x_2)c(x_0,x_1). \end{aligned}$$

Hence, if we let $x_0\,{:}{=}\,y$ and $x_n\,{:}{=}\,x$, it follows from Eq. (26) that

$$\begin{aligned} \sum _{x_{n-1}\in \mathcal {X}}c(x_{n-1},x_n) \sum _{x_{n-2}\in \mathcal {X}}c(x_{n-2},x_{n-1}) \cdots \sum _{x_1\in \mathcal {X}}c(x_1,x_2)c(x_0,x_1) >0. \end{aligned}$$

This implies that there is some sequence $y=x_0,x_1,\ldots ,x_n=x$ such that

$$\begin{aligned} c(x_{n-1},x_n) c(x_{n-2},x_{n-1}) \cdots c(x_1,x_2)c(x_0,x_1) >0. \end{aligned}$$

Since each of the factors in this product is non-negative, it follows that $c(x_{k-1},x_{k})>0$ for all $k\in \{1,\ldots ,n\}$. Therefore, for any $k\in \{1,\ldots ,n\}$ such that $x_k\ne x_{k-1}$, it follows that

$$\begin{aligned} \overline{Q}(\mathbb {I}_{x_k})(x_{k-1})&=\frac{1}{\Delta }\Big (\mathbb {I}_{x_k}(x_{k-1})+\Delta \overline{Q}(\mathbb {I}_{x_k})(x_{k-1})\Big )\\&=\frac{1}{\Delta }\Big ((\mathbb {I}_{x_k}+\Delta \overline{Q}(\mathbb {I}_{x_k}))(x_{k-1})\Big )\\&=\frac{1}{\Delta }\Big (\big ((I+\Delta \overline{Q})\mathbb {I}_{x_k}\big )(x_{k-1})\Big )\\&=\frac{1}{\Delta }\Big (\big (\overline{T}_*\mathbb {I}_{x_k}\big )(x_{k-1})\Big ) =\frac{1}{\Delta }c(x_{k-1},x_k)>0. \end{aligned}$$

If $x_k\ne x_{k-1}$ for all $k\in \{1,\ldots ,n\}$, this implies that x is upper reachable from y. Otherwise, let $x'_0,\ldots ,x'_m$ be a new sequence, obtained by removing from $x_0,\ldots ,x_n$ those elements $x_k$ for which $x_k=x_{k-1}$; $n-m$ is the number of elements that is removed. Then, $x'_0=y$, $x'_m=x$ and, for all $k\in \{1,\ldots ,m\}$, we have that $x'_k\ne x'_{k-1}$ and $\overline{Q}(\mathbb {I}_{x'_k})(x'_{k-1})>0$. Therefore, x is upper reachable from y.

Conversely, assume that x is upper reachable from y, meaning that there is some sequence $y=x_0,x_1,\ldots ,x_n=x$ such that, for all $k\in \{1,\ldots ,n\}$, $x_k\ne x_{k-1}$ and $\overline{Q}\mathbb {I}_{x_k}(x_{k-1})>0$. If $n=0$, then $x=y$, and therefore, it follows from Corollary 32 that $\overline{T}_t\mathbb {I}_{x}(y)>0$. Hence, for the remainder of this proof, we may assume that $n\ge 1$. Fix any $k\in \{1,\ldots ,n\}$. We then have that $\overline{T}_0 \mathbb {I}_{x_k}(x_{k-1})=\mathbb {I}_{x_k}(x_{k-1})=0$ and

$$\begin{aligned} \frac{\mathrm{d}}{\mathrm{d}s}\overline{T}_s \mathbb {I}_{x_k}(x_{k-1})\Big \vert _{s=0}&=-\frac{\mathrm{d}}{\mathrm{d}s}\underline{T}_s(-\mathbb {I}_{x_k})(x_{k-1})\Big \vert _{s=0}\\&=-\underline{Q}(\underline{T}_0(-\mathbb {I}_{x_k}))(x_{k-1})\\&=-\underline{Q}(-\mathbb {I}_{x_k})(x_{k-1}) =\overline{Q}(\mathbb {I}_{x_k})(x_{k-1})>0, \end{aligned}$$

where the first equality follows from Eq. (5), the second equality follows from Eq. (8), and the last equality follows from Eq. (7). Therefore, there is some $\epsilon _k>0$ such that $\overline{T}_{\epsilon _k}\mathbb {I}_{x_k}(x_{k-1})>0$. Consequently, if we let $c_k\,{:}{=}\,\overline{T}_{\epsilon _k}\mathbb {I}_{x_k}(x_{k-1})>0$, then because it follows from Proposition 8 and L4 that $\overline{T}_{\epsilon _k}\mathbb {I}_{x_k}\ge 0$, we have that $\overline{T}_{\epsilon _k}\mathbb {I}_{x_k}\ge c_k\mathbb {I}_{x_{k-1}}$. Let $\epsilon \,{:}{=}\,\sum _{k=1}^{n}\epsilon _k>0$. Then,

$$\begin{aligned} \overline{T}_\epsilon \mathbb {I}_{x_n}=\overline{T}_{\epsilon _1}\cdots \overline{T}_{\epsilon _{n-1}} \overline{T}_{\epsilon _n}\mathbb {I}_{x_n} \ge c_n\overline{T}_{\epsilon _1}\cdots \overline{T}_{\epsilon _{n-1}}\mathbb {I}_{x_{n-1}} \ge \cdots \ge \left( \prod _{k=1}^n c_k\right) \mathbb {I}_{x_0}, \end{aligned}$$

where the equality follows from Eqs. (5) and (10) and where the inequalities follow from Proposition 8, L6, L3, and Eq. (5). Therefore, we find that

$$\begin{aligned} \overline{T}_\epsilon \mathbb {I}_{x}(y)=\overline{T}_\epsilon \mathbb {I}_{x_n}(x_0) \ge \prod _{k=1}^n c_k \mathbb {I}_{x_0}(x_0) =\prod _{k=1}^n c_k>0, \end{aligned}$$

which implies that $\overline{T}_t\mathbb {I}_{x}(y)>0$ because of Corollary 32. $\square$

Proof of Proposition 18

Let $\{A_k\}_{k\in \mathbb {N}_{0}}$ and n be defined as in Definition 8. We need to prove that $x\in A_n$ if and only if $\underline{T}_t\mathbb {I}_{A}(x)>0$.

First assume that $\underline{T}_t\mathbb {I}_{A}(x)>0$. It then follows from Proposition 10 that there is some $m\in \mathbb {N}$ such that $m\ge n$, $m\ge t\left\| \underline{Q} \right\|$ and

$$\begin{aligned} \left( \left( I+\frac{t}{m}\underline{Q}\right) ^m\mathbb {I}_{A}\right) (x) >0. \end{aligned}$$

(27)

Let and define $\underline{T}_*\,{:}{=}\,I+\Delta \underline{Q}$. Since $m\ge t\left\| \underline{Q} \right\|$ implies that $\Delta \left\| \underline{Q} \right\| \le 1$, it then follows from Proposition 5 that $\underline{T}_*$ is a lower transition operator. Consider any $k\in \mathbb {N}_{0}$ such that $k\le m$ and any $y\in \mathcal {X}\!\setminus \! A_{k+1}$. Since $A_k\subseteq A_{k+1}$, this implies that $y\notin A_k$. Assume ex absurdo that $\underline{Q}(\mathbb {I}_{A_k})(y)>0$. It then follows from Eq. (12) that $y\in A_{k+1}$, a contradiction. Hence, we find that $\underline{Q}(\mathbb {I}_{A_k})(y)\le 0$. Since $y\notin A_k$, it follows from R2 and R4 that $\underline{Q}(\mathbb {I}_{A_k})(y)\ge \sum _{z\in A_k}\underline{Q}(\mathbb {I}_{z})(y)\ge 0$. Hence, we infer that $\underline{Q}(\mathbb {I}_{A_k})(y)=0$. Furthermore, since $y\notin A_k$, we also have that $\mathbb {I}_{A_k}(y)=0$. Hence, we find that $(\underline{T}_*\mathbb {I}_{A_k})(y)=((I+\Delta \underline{Q})\mathbb {I}_{A_k})(y)=0$. Since this holds for all $y\in \mathcal {X}\!\setminus \! A_{k+1}$, there is some $c_k>0$ such that $\underline{T}_*\mathbb {I}_{A_k}\le c_k\mathbb {I}_{A_{k+1}}$. Due to L3 and L6, this implies that $\underline{T}_*^{m-k}\mathbb {I}_{A_k}\le c_k\underline{T}_*^{m-k-1}\mathbb {I}_{A_{k+1}}$. Since this holds for all $k\in \mathbb {N}_{0}$ such that $k\le m$, we find that

$$\begin{aligned} \underline{T}_*^m\mathbb {I}_{A_0} \le c_0\underline{T}_*^{m-1}\mathbb {I}_{A_1} \le c_0c_1\underline{T}_*^{m-2}\mathbb {I}_{A_2} \le \cdots \le c_0c_1\cdots c_{m-1}\mathbb {I}_{A_m}. \end{aligned}$$

Therefore, since $A_0=A$, it follows from Eq. (27) that $x\in A_m$. Since $A_n=A_{n+1}$, it follows from Eq. (12) that $A_r=A_n$ for all $r\ge n$ and therefore, in particular, that $A_m=A_n$. Since $x\in A_m$, this implies that $x\in A_n$.

Conversely, assume that $x\in A_n$. If $n=0$, then $A_n=A_0=A$ and therefore $x\in A$, which, due to Corollary 32, implies that $\underline{T}_t\mathbb {I}_{A}(x)>0$. Therefore, for the remainder of this proof, we may assume that $n\ge 1$. Fix any $k\in \{0,\ldots ,n-1\}$. Consider any $y\in A_{k+1}\!\setminus \! A_k$. Then, $\underline{T}_0\mathbb {I}_{A_k}(y)=\mathbb {I}_{A_k}(y)=0$ and

$$\begin{aligned} \frac{\mathrm{d}}{\mathrm{d}s}\underline{T}_s \mathbb {I}_{A_k}(y)\Big \vert _{s=0} =\underline{Q}(\underline{T}_0(\mathbb {I}_{A_k}))(y) =\underline{Q}(\mathbb {I}_{A_k})(y)>0, \end{aligned}$$

where the first equality follows from Eq. (8) and the inequality follows from Eq. (12). Therefore, there is some $\epsilon _{k,y}>0$ such that $\underline{T}_s\mathbb {I}_{A_k}(y)>0$ for all $s\in (0,\epsilon _{k,y}]$. Hence, if we let $\epsilon _k\,{:}{=}\,\min _{y\in A_{k+1}\!\setminus \! A_k}\epsilon _{k,y}$, then $\underline{T}_{\epsilon _k}\mathbb {I}_{A_k}(y)>0$ for all $y\in A_{k+1}\!\setminus \! A_k$. For all $y\in A_k$, it follows from Corollary 32 that $\underline{T}_{\epsilon _k}\mathbb {I}_{A_k}(y)>0$. Hence, in summary, we have that $\underline{T}_{\epsilon _k}\mathbb {I}_{A_k}(y)>0$ for all $y\in A_{k+1}$. Since we know from Proposition 8 and L1 that $\underline{T}_{\epsilon _k}\mathbb {I}_{A_k}\ge 0$, this implies that there is some $c_k>0$ such that $\underline{T}_{\epsilon _k}\mathbb {I}_{A_k}\ge c_k\mathbb {I}_{A_{k+1}}$. Let $\epsilon \,{:}{=}\,\sum _{k=0}^{n-1}\epsilon _k$. Then,

$$\begin{aligned} \underline{T}_\epsilon \mathbb {I}_{A_0}=\underline{T}_{\epsilon _{n-1}}\cdots \underline{T}_{\epsilon _{1}}\underline{T}_{\epsilon _0}\mathbb {I}_{A_0} \ge c_0\underline{T}_{\epsilon _{n-1}}\cdots \underline{T}_{\epsilon _{1}}\mathbb {I}_{A_1} \ge \cdots \ge \left( \prod _{k=0}^{n-1} c_k\right) \mathbb {I}_{A_{n}}, \end{aligned}$$

where the equality follows from Eq. (10) and the inequalities follow from Proposition 8, L3 and L6. Therefore, we find that

$$\begin{aligned} \underline{T}_\epsilon \mathbb {I}_{A}(x)=\underline{T}_\epsilon \mathbb {I}_{A_0}(x) \ge \left( \prod _{k=0}^{n-1} c_k\right) \mathbb {I}_{A_n}(x) =\prod _{k=0}^{n-1}c_k>0, \end{aligned}$$

which implies that $\underline{T}_t\mathbb {I}_{A}(x)>0$ because of Corollary 32. $\square$

Proof of Theorem 19

Fix any $t>0$. It then follows from Corollary 16 that $\underline{Q}$ is ergodic if and only if $\underline{T}_t$ is 1-step absorbing or, equivalently, because of Definition 4, if

$$\begin{aligned} \mathcal {X}_{\mathrm {1A}}\,{:}{=}\,\{ x\in \mathcal {X}:\min \overline{T}_t\mathbb {I}_{x}>0 \}\ne \emptyset \text { and } (\forall x\in \mathcal {X}\!\setminus \!\mathcal {X}_{\mathrm {1A}})~ \underline{T}_t\mathbb {I}_{\mathcal {X}_{\mathrm {1A}}}(x)>0. \end{aligned}$$

The result now follows immediately because we know from Proposition 17 that, for all $x\in \mathcal {X}$,

and because we know from Proposition 18 that, for all $x\in \mathcal {X}\!\setminus \!\mathcal {X}_{\mathrm {1A}}$, $\underline{T}_t\mathbb {I}_{\mathcal {X}_{\mathrm {1A}}}(x)>0$ if and only if . $\square$