Poly-symmetry in processor-sharing systems

Abstract

We consider a system of processor-sharing queues with state-dependent service rates. These are allocated according to balanced fairness within a polymatroid capacity set. Balanced fairness is known to be both insensitive and Pareto-efficient in such systems, which ensures that the performance metrics, when computable, will provide robust insights into the real performance of the system considered. We first show that these performance metrics can be evaluated with a complexity that is polynomial in the system size if the system is partitioned into a finite number of parts, so that queues are exchangeable within each part and asymmetric across different parts. This in turn allows us to derive stochastic bounds for a larger class of systems which satisfy less restrictive symmetry assumptions. These results are applied to practical examples of tree data networks, such as backhaul networks of Internet service providers, and computer clusters.

Appendices

Appendix 1: Proof of Theorem 3

1.1 Recursion (6)

Let $a \in {\mathscr {N}} {\setminus } \{0\}$. By (4), we have

$$\begin{aligned} \pi (a) = \sum _{\begin{array}{c} A \subset I \\ |A|_\varSigma = a \end{array}} \pi (A) = \sum _{\begin{array}{c} A \subset I \\ |A|_\varSigma = a \end{array}} \frac{\sum _{i \in A} \rho _i \pi (A {\setminus } \{i\})}{\mu (A) - \sum _{i \in A} \rho _i}. \end{aligned}$$

The regularity assumptions ensure that $\mu (A) - \sum _{i \in A} \rho _i = h(a) - \sum _{k=1}^K a_k \varrho _k$ for any $A \subset I$ with $|A|_\varSigma = a$. Thus, we obtain

$$\begin{aligned} \left( h(a) - \sum _{k=1}^K a_k \varrho _k \right) \pi (a) = \sum _{\begin{array}{c} A \subset I \\ |A|_\varSigma = a \end{array}} \sum _{i \in A} \rho _i \pi (A {\setminus } \{i\}) = \sum _{k=1}^K \varrho _k \sum _{i \in I_k} \sum _{\begin{array}{c} A \subset I, i \in A \\ |A|_\varSigma = a \end{array}} \pi (A {\setminus } \{i\}). \end{aligned}$$

For any $k = 1,\ldots ,K$ and any $i \in I_k$, we make the substitution

$$\begin{aligned} \sum _{\begin{array}{c} A \subset I, i \in A \\ |A|_\varSigma = a \end{array}} \pi (A {\setminus } \{i\}) = \sum _{\begin{array}{c} B \subset I {\setminus } \{i\} \\ |B|_\varSigma = a-e_k \end{array}} \pi (B), \end{aligned}$$

and thus we obtain, for any $k = 1,\ldots ,K$,

$$\begin{aligned} \sum _{i \in I_k} \sum _{\begin{array}{c} A \subset I, i \in A \\ |A|_\varSigma = a \end{array}} \pi (A {\setminus } \{i\}) = \sum _{\begin{array}{c} B \subset I \\ |B|_\varSigma = a-e_k \end{array}} \sum _{i \in I_k {\setminus } (B \cap I_k)} \pi (B) = (n_k - a_k + 1) \pi ({a-e_k}). \end{aligned}$$

(9)

This proves (6).

1.2 Recursion (7)

Let $k = 1,\ldots ,K$ and $a \in {\mathscr {N}} {\setminus } \{0\}$. By the definition of $L_k(a)$, we have

$$\begin{aligned} L_k(a) = \mathbb {E}\left[ \sum _{i \in I_k} \mathbf{X}_i \bigg | |I(\mathbf{X})|_\varSigma = a \right] = \sum _{i \in I_k} \mathbb {E}\left[ \mathbf{X}_i | |I(\mathbf{X})|_\varSigma = a \right] . \end{aligned}$$

It follows that

$$\begin{aligned} \pi (a) L_k(a)&= \sum _{i \in I_k} \sum _{\begin{array}{c} A \subset I, i \in A \\ |A|_\varSigma = a \end{array}} \pi (A) L_i(A), \end{aligned}$$

(10)

and by (5) we obtain

$$\begin{aligned} \pi (a) L_k(a)&= \sum _{i \in I_k} \sum _{\begin{array}{c} A \subset I, i \in A \\ |A|_\varSigma = a \end{array}} \frac{\rho _i \pi (A {\setminus } \{i\}) + \rho _i \pi (A) + \sum _{j \in A {\setminus } \{i\}} \rho _j \pi (A {\setminus } \{j\}) L_i(A {\setminus } \{j\})}{\mu (A) - \sum _{j \in A} \rho _j}. \end{aligned}$$

Using the regularity assumptions, this can be rewritten as

$$\begin{aligned} \left( h(a) - \sum _{l=1}^K a_l \varrho _l \right) \pi (a) L_k(a) {}={}&\varrho _k \sum _{i \in I_k} \sum _{\begin{array}{c} A \subset I, i \in A \\ |A|_\varSigma = a \end{array}} \pi (A {\setminus } \{i\}) + \varrho _k \sum _{i \in I_k} \sum _{\begin{array}{c} A \subset I, i \in A \\ |A|_\varSigma = a \end{array}} \pi (A) \\&{}+{} \sum _{i \in I_k} \sum _{\begin{array}{c} A \subset I, i \in A \\ |A|_\varSigma = a \end{array}} \sum _{\begin{array}{c} j \in A \\ j \ne i \end{array}} \rho _j \pi (A {\setminus } \{j\}) L_i(A {\setminus } \{j\}). \end{aligned}$$

The first term is given by (9). The second term is simply

$$\begin{aligned} \varrho _k \sum _{i \in I_k} \sum _{\begin{array}{c} A \subset I, i \in A \\ |A|_\varSigma = a \end{array}} \pi (A) = \varrho _k \sum _{\begin{array}{c} A \subset I \\ |A|_\varSigma = a \end{array}} \sum _{i \in A \cap I_k} \pi (A) = a_k \varrho _k \pi (a). \end{aligned}$$

Finally, for any $i \in I_k$, we have

$$\begin{aligned} \sum _{\begin{array}{c} A \subset I, i \in A \\ |A|_\varSigma = a \end{array}} \sum _{\begin{array}{c} j \in A \\ j \ne i \end{array}} \rho _j \pi (A {\setminus } \{j\}) L_i(A {\setminus } \{j\}) = \sum _{l=1}^K \varrho _l \sum _{\begin{array}{c} j \in I_l \\ j \ne i \end{array}} \sum _{\begin{array}{c} A \subset I, i,j \in A \\ |A|_\varSigma = a \end{array}} \pi (A {\setminus } \{j\}) L_i(A {\setminus } \{j\}). \end{aligned}$$

Making the same substitution as in (9), we have, for any $l = 1,\ldots ,K$,

$$\begin{aligned} \sum _{\begin{array}{c} j \in I_l \\ j \ne i \end{array}} \sum _{\begin{array}{c} A \subset I, i,j \in A \\ |A|_\varSigma = a \end{array}} \pi (A {\setminus } \{j\}) L_i(A {\setminus } \{j\})&= \sum _{\begin{array}{c} B \subset I, i \in B \\ |B|_\varSigma = a-e_l \end{array}} \sum _{j \in I_l {\setminus } (B \cap I_l)} \pi (B) L_i(B) \\&= (n_l - a_l + 1) \sum _{\begin{array}{c} B \subset I, i \in B \\ |B|_\varSigma = a-e_l \end{array}} \pi (B) L_i(B). \end{aligned}$$

Hence the third term of the sum is equal to

$$\begin{aligned}&\sum _{i \in I_k} \sum _{l=1}^K (n_l - a_l + 1) \varrho _l \sum _{\begin{array}{c} B \subset I, i \in B \\ |B|_\varSigma = a-e_l \end{array}} \pi (B) L_i(B) \\&\quad = \sum _{l=1}^K (n_l - a_l + 1) \varrho _l \sum _{i \in I_k} \sum _{\begin{array}{c} B \subset I, i \in B \\ |B|_\varSigma = a-e_l \end{array}} \pi (B) L_i(B) \\&\quad = \sum _{l=1}^K (n_l - a_l + 1) \varrho _l \pi ({a-e_l}) L_k(a-e_l), \end{aligned}$$

where the second equality holds by (10). When we substitute the three terms by their expressions, we obtain (7).

Appendix 2: Proof of Theorem 5

We give the proof only for the case $K = 2$ for ease of notation; the other cases follow analogously. For now, we assume that for all $n \ge 1$, all servers have the same capacity $\mu _s^{(n)} = \xi ^{(n)}$ for any $s \in S^{(n)}$.

Let $0< \varepsilon < 1$. We will show that there exists a sequence $(g_n : n \ge 1)$ such that $g_n = \omega (\log n)$ and, for any $n \ge 1$,

$$\begin{aligned} \mathbb {P} \left\{ \exists A \subset I^{(n)} \text { s.t. } \mathbf{M}^{(n)}(A) \le (1 - \varepsilon ) \mu ^{(n)}(A) \right\} \le \mathrm{e}^{-g_n}. \end{aligned}$$

Let us first give the main ideas of the proof. As in [18], it is divided into three steps. We first provide a bound for $\mathbb {P} \left\{ \mathbf{M}^{(n)}(A) \le (1 - \varepsilon ) \mu ^{(n)}(A) \right\}$ for each $A \subset I^{(n)}$ for n large enough. Then, for each $a \in {\mathscr {N}}^{(n)} = \{0,1,\ldots ,n\}^2$, we use the union bound to obtain a uniform bound over all sets $A \subset I$ with $|A|_{\varSigma ^{(n)}} = a$. Finally, another use of the union bound over all $a \in {\mathscr {N}}^{(n)}$ gives us the result.

1.1 Partial bound

Let $n \ge 1$, $a \in {\mathscr {N}}^{(n)}$ and $A \subset I^{(n)}$ such that $|A|_{\varSigma ^{(n)}} = a$. Recall that $\mu ^{(n)}(A) = \mathbb {E}[\mathbf{M}^{(n)}(A)]$ with

$$\begin{aligned} \mathbf{M}^{(n)}(A) = \xi ^{(n)} \sum _{s \in S^{(n)}} 1_{s \in \bigcup _{i \in A} \mathbf{S}_i^{(n)}}. \end{aligned}$$

The variables $1_{s \in \bigcup _{i \in A} \mathbf{S}_i^{(n)}}$ for $s \in S^{(n)}$ are Bernoulli distributed with parameter

$$\begin{aligned} p_a^{(n)} = 1 - \prod _{k=1}^K \left( 1 - \frac{d_k^{(n)}}{m^{(n)}} \right) ^{a_k}. \end{aligned}$$

Dubbashi et al. proved in Theorem 10 of [9] that these random variables are negatively associated in the sense of Definition 3 of [9]. Their Theorem 14 then showed that the Chernoff–Hoeffding bounds (see for instance [15, 19]), which hold for independent random variables, can also be applied to these random variables. Hence we have

$$\begin{aligned} \mathbb {P} \left\{ \mathbf{M}^{(n)}(A) \le (1 - \varepsilon ) \mu ^{(n)}(A) \right\}&\le \mathrm{e}^{- \frac{\varepsilon ^2}{2} m^{(n)} p_a^{(n)}}, \end{aligned}$$

(11)

$$\begin{aligned} \mathbb {P} \left\{ \mathbf{M}^{(n)}(A) \le (1 - \varepsilon ) \mu ^{(n)}(A) \right\}&\le \mathrm{e}^{- m^{(n)} H\left[ (1 - \varepsilon ) p_a^{(n)} || p_a^{(n)} \right] }, \end{aligned}$$

(12)

where, for any $p,q \in (0,1)$, H[p||q] is the KL divergence between two Bernoulli random variables with parameters p and q, respectively, given by

$$\begin{aligned} H[p||q] = p \log \left( \frac{p}{q} \right) + (1-p) \log \left( \frac{1-p}{1-q} \right) . \end{aligned}$$

We also use the following lemmas, which will be proved later in Appendix 3.

Lemma 3

Let $0< \delta < \frac{1}{2}$. Consider a sequence $(g_n : n \ge 1)$ such that $g_n = o\left( d_1^{(n)} \right)$ and $g_n = o\left( d_2^{(n)} \right)$. For large enough n, we have

$$\begin{aligned} p_a^{(n)} \ge \delta \frac{(a_1 + a_2) g_n}{n}, \quad \forall a = (a_1, a_2) \in \left\{ 0,1,\ldots ,\left\lfloor \frac{n}{g_n} \right\rfloor \right\} ^2. \end{aligned}$$

Lemma 4

There exists a positive constant $\delta$ such that

$$\begin{aligned} H\left[ (1-\varepsilon ) p_a^{(n)} || p_a^{(n)} \right] \ge -\delta + \varepsilon \frac{a_1 d_1^{(n)} + a_2 d_2^{(n)}}{m^{(n)}}, \quad \forall n \ge 1, \quad \forall a \in {\mathscr {N}}^{(n)}. \end{aligned}$$

Consider the sequence $(g_n : n \ge 1)$ given by

$$\begin{aligned} g_n = \left( \min \left( d_1^{(n)}, d_2^{(n)} \right) \log n \right) ^{1/2}, \quad \forall n \ge 1. \end{aligned}$$

Observe that $g_n = \omega (\log n)$, $g_n = o\left( d_1^{(n)} \right)$ and $g_n = o\left( d_2^{(n)} \right)$. Now let $n \ge 1$ and $a \in {\mathscr {N}}^{(n)}$. We distinguish two cases depending on the value of a.

1.2 Case 1: $0 \le a_1 \le \frac{n}{g_n}$ and $0 \le a_2 \le \frac{n}{g_n}$

By Lemma 3, there is a positive constant $\delta _1$ such that, for large enough n,

$$\begin{aligned} p_a^{(n)} \ge \delta _1 \frac{(a_1 + a_2) g_n}{n}. \end{aligned}$$

Using (11), we deduce that

$$\begin{aligned} \mathbb {P} \left\{ \mathbf{M}^{(n)}(A) \le (1 - \varepsilon ) \mu ^{(n)}(A) \right\} \le \mathrm{e}^{- \frac{\varepsilon ^2}{2} \delta _1 b (a_1 + a_2) g_n} \end{aligned}$$

for any $A \subset I^{(n)}$ such that $|A|_{\varSigma ^{(n)}} = a$. The union bound yields

$$\begin{aligned}&\mathbb {P} \left\{ \exists A \subset I^{(n)} \text { s.t. } |A|_{\varSigma ^{(n)}} = a \text { and } \mathbf{M}^{(n)}(A) \le (1 - \varepsilon ) \mu ^{(n)}(A) \right\} \\&\quad \le \mathrm{e}^{- \frac{\varepsilon ^2}{2} \delta _1 b (a_1 + a_2) g_n} \left( {\begin{array}{c}n\\ a_1\end{array}}\right) \left( {\begin{array}{c}n\\ a_2\end{array}}\right) , \\&\quad \le \mathrm{e}^{- \frac{\varepsilon ^2}{2} \delta _1 b (a_1 + a_2) g_n} n^{a_1} n^{a_2}, \\&\quad \le \mathrm{e}^{a_1 g_n \left( - \frac{\varepsilon ^2}{2} \delta _1 b + \frac{\log n}{g_n}\right) } \mathrm{e}^{a_2 g_n \left( - \frac{\varepsilon ^2}{2} \delta _1 b + \frac{\log n}{g_n}\right) }. \end{aligned}$$

Since $g_n = \omega (\log n)$, we obtain for large enough n

$$\begin{aligned} \mathbb {P} \left\{ \exists A \subset I \text { s.t. } |A|_{\varSigma ^{(n)}} = a \text { and } \mathbf{M}^{(n)}(A) \le (1 - \varepsilon ) \mu ^{(n)}(A) \right\} \le \mathrm{e}^{- \delta _2 (a_1 + a_2) g_n} \end{aligned}$$

with $\delta _2 = \frac{\varepsilon ^2}{4} \delta _1 b > 0$.

1.3 Case 2: $a_1 > \frac{n}{g_n}$ or $a_2 > \frac{n}{g_n}$

Combining Lemma 4 with (12), we deduce that there is a positive constant $\delta _3$ such that

$$\begin{aligned} \mathbb {P} \left\{ \mathbf{M}^{(n)}(A) \le (1 - \varepsilon ) \mu ^{(n)}(A) \right\} \le \mathrm{e}^{\delta _3 m^{(n)} - \varepsilon \left( a_1 d_1^{(n)} + a_2 d_2^{(n)} \right) } \end{aligned}$$

for any $A \subset I^{(n)}$ such that $|A|_{\varSigma ^{(n)}} = a$. Since $m^{(n)} = \lceil b n \rceil$ and $g_n = o\left( d_1^{(n)} \right)$, we have $\delta _3 m^{(n)} \le \frac{\varepsilon }{2} \frac{n d_1^{(n)}}{g_n}$ when n is large enough. If $a_1 > \frac{n}{g_n}$, we also have that $\frac{\varepsilon }{2} \frac{n d_1^{(n)}}{g_n} \le \frac{\varepsilon }{2} a_1 d_1^{(n)}$ so that

$$\begin{aligned} \delta _3 m^{(n)} - \varepsilon \left( a_1 d_1^{(n)} + a_2 d_2^{(n)} \right) \le - \frac{\varepsilon }{2} a_1 d_1^{(n)} - \varepsilon a_2 d_2^{(n)} \le - \frac{\varepsilon }{2} \left( a_1 d_1^{(n)} + a_2 d_2^{(n)} \right) \end{aligned}$$

for large enough n. The same argument holds by inverting $a_1$ and $a_2$ when $a_2 > \frac{n}{g_n}$, so we conclude that there is a positive constant $\delta _4$ such that, for large enough n, we have

$$\begin{aligned} \mathbb {P} \left\{ M^{(n)}(A) \le (1 - \varepsilon ) \mu ^{(n)}(A) \right\} \le \mathrm{e}^{- \delta _4 \left( a_1 d_1^{(n)} + a_2 d_2^{(n)} \right) } \end{aligned}$$

for any $A \subset I^{(n)}$ such that $|A|_{\varSigma ^{(n)}} = a$. The union bound yields

$$\begin{aligned}&\mathbb {P} \left\{ \exists A \subset I^{(n)} \text { s.t. } |A|_{\varSigma ^{(n)}} = a \text { and } \mathbf{M}^{(n)}(A) \le (1 - \varepsilon ) \mu ^{(n)}(A) \right\} \\&\quad \le \mathrm{e}^{- \delta _4 \left( a_1 d_1^{(n)} + a_2 d_2^{(n)} \right) } \left( {\begin{array}{c}n\\ a_1\end{array}}\right) \left( {\begin{array}{c}n\\ a_2\end{array}}\right) , \\&\quad \le \mathrm{e}^{- \delta _4 \left( a_1 d_1^{(n)} + a_2 d_2^{(n)} \right) } n^{a_1} n^{a_2}, \\&\quad \le \mathrm{e}^{a_1 d_1^{(n)} \left( - \delta _4 + \log n / d_1^{(n)} \right) } \mathrm{e}^{a_2 d_2^{(n)} \left( - \delta _4 + \log n / d_2^{(n)} \right) }. \end{aligned}$$

Since $d_1^{(n)} = \omega (\log n)$ and $d_2^{(n)} = \omega (\log n)$, for large enough n, we have

$$\begin{aligned}&\mathbb {P} \left\{ \exists A \subset I^{(n)} \text { s.t. } |A|_{\varSigma ^{(n)}} = a \text { and } \mathbf{M}^{(n)}(A) \le (1 - \varepsilon ) \mu ^{(n)}(A) \right\} \\&\quad \le \mathrm{e}^{- \delta _5 a_1 d_1^{(n)}} \mathrm{e}^{- \delta _5 a_2 d_2^{(n)}} = \mathrm{e}^{- \delta _5 \left( a_1 d_1^{(n)} + a_2 d_2^{(n)} \right) } \end{aligned}$$

for some positive constant $\delta _5 < \delta _4$.

1.4 Conclusion

Combining cases 1 and 2, we deduce that there exists a positive constant $\delta _6$ such that

$$\begin{aligned} \mathbb {P} \left\{ \exists A \subset I^{(n)} \text { s.t. } |A|_{\varSigma ^{(n)}} = a \text { and } \mathbf{M}^{(n)}(A) \le (1 - \varepsilon ) \mu ^{(n)}(A) \right\} \le \mathrm{e}^{- \delta _6 g_n}, \quad \forall a \in {\mathscr {N}}^{(n)} \end{aligned}$$

when n is large enough. Using the union bound again, we obtain

$$\begin{aligned}&\mathbb {P} \left\{ \exists A \subset I^{(n)} \text { s.t. } \mathbf{M}^{(n)}(A) \le (1 - \varepsilon ) \mu ^{(n)}(A) \right\} \\&\quad \le n^2 \mathrm{e}^{- \delta _6 g_n} = \mathrm{e}^{2 \log n} \mathrm{e}^{- \delta _6 g_n} = \mathrm{e}^{- g_n \left( \delta _6 - \frac{2 \log n}{g_n} \right) }. \end{aligned}$$

Since $g_n = \omega (\log n)$, we conclude that for a constant $\delta _7 < \delta _6$, we have for large enough n

$$\begin{aligned} \mathbb {P} \left\{ \exists A \subset I^{(n)} \text { s.t. } \mathbf{M}^{(n)}(A) \le (1 - \varepsilon ) \mu ^{(n)}(A) \right\} \le \mathrm{e}^{- \delta _7 g_n}. \end{aligned}$$

Finally, when servers are in groups as in Assumption 1, we can break down $\mathbf{M}^{(n)}$ into a sum of random rank functions, one for each group. The result follows by showing the concentration in each group as above, and then using the union bound again.

Appendix 3: Proofs of the lemmas for Theorem 5

Lemma 3

Let $0< \delta < \frac{1}{2}$. Consider a sequence $(g_n : n \ge 1)$ such that $g_n = o\left( d_1^{(n)} \right)$ and $g_n = o\left( d_2^{(n)} \right)$. For large enough n, we have

$$\begin{aligned} p_a^{(n)} \ge \delta \frac{(a_1 + a_2) g_n}{n}, \quad \forall a = (a_1, a_2) \in \left\{ 0,1,\ldots ,\left\lfloor \frac{n}{g_n} \right\rfloor \right\} ^2. \end{aligned}$$

Proof

Consider the sequence $(f_n : n \ge 1)$ of functions defined on $\mathbb {R}_+^2$ by

$$\begin{aligned} f_n(t_1,t_2) = 1 - \left( 1 - \frac{d_1^{(n)}}{bn} \right) ^{t_1} \left( 1 - \frac{d_2^{(n)}}{bn} \right) ^{t_2}, \quad \forall (t_1,t_2) \in \mathbb {R}_+^2. \end{aligned}$$

We have

$$\begin{aligned}&f_n\left( \frac{2n}{g_n}, 0 \right) = 1 - \left[ \left( 1 - \frac{d_1^{(n)}}{bn} \right) ^\frac{n}{g_n} \right] ^2 \xrightarrow [n \rightarrow \infty ]{} 1, \\ \quad \text {and} \quad&f_n\left( 0, \frac{2n}{g_n} \right) = 1 - \left[ \left( 1 - \frac{d_2^{(n)}}{bn} \right) ^\frac{n}{g_n} \right] ^2 \xrightarrow [n \rightarrow \infty ]{} 1. \end{aligned}$$

Thus, there is $n_\delta \ge 1$ such that $f_n\left( \frac{2n}{g_n}, 0 \right) \ge 2 \delta$ and $f_n\left( 0, \frac{2n}{g_n} \right) \ge 2 \delta$ for all $n \ge n_\delta$.

Then, for any $n \ge n_\delta$ and any $t_1, t_2 \le \frac{n}{g_n}$, we have

$$\begin{aligned} f_n(t_1, t_2)&= f_n \left( \frac{t_1}{t_1+t_2} (t_1 + t_2, 0) + \left( 1 - \frac{t_1}{t_1+t_2} \right) (0, t_1 + t_2) \right) \\&\ge \frac{t_1}{t_1+t_2} f_n(t_1+t_2,0) + \frac{t_2}{t_1+t_2} f_n(0,t_1+t_2) \\&\ge \frac{t_1}{t_1+t_2} \frac{t_1+t_2}{\frac{2n}{g_n}} f_n\left( \frac{2n}{g_n}, 0 \right) + \frac{t_2}{t_1+t_2} \frac{t_1+t_2}{\frac{2n}{g_n}} f_n\left( 0, \frac{2n}{g_n} \right) \\&\ge 2 \delta \frac{t_1 g_n}{2n} + 2 \delta \frac{t_2 g_n}{2n} \\&= \delta \frac{(t_1 + t_2) g_n}{n}, \end{aligned}$$

where the first two inequalities hold by the concavity of $f_n$. $\square$

Lemma 4

There exists a positive constant $\delta$ such that

$$\begin{aligned} H\left[ (1-\varepsilon ) p_a^{(n)} || p_a^{(n)} \right] \ge -\delta + \varepsilon \frac{a_1 d_1^{(n)} + a_2 d_2^{(n)}}{m^{(n)}}, \quad \forall n \ge 1, \quad \forall a \in {\mathscr {N}}^{(n)}. \end{aligned}$$

Proof

By the definition of H,

$$\begin{aligned} H\left[ (1 - \varepsilon ) p_a^{(n)} || p_a^{(n)} \right] {}={}&(1 - \varepsilon ) p_a^{(n)} \log (1 - \varepsilon ) \\&{}+{} \left( 1 - (1 - \varepsilon ) p_a^{(n)} \right) \log \left( 1 - (1 - \varepsilon ) p_a^{(n)} \right) \\&{}-{} \left( 1 - (1 - \varepsilon ) p_a^{(n)} \right) \log \left( 1 - p_a^{(n)} \right) . \end{aligned}$$

The first and the second terms are greater than $(1 - \varepsilon ) \log (1 - \varepsilon )$ and $\log (\varepsilon )$, respectively. With $\delta = (1 - \varepsilon ) \log \left( \frac{1}{1-\varepsilon } \right) + \log \left( \frac{1}{\varepsilon }\right) > 0$, we obtain

$$\begin{aligned} H\left[ (1 - \varepsilon ) p_a^{(n)} || p_a^{(n)} \right]&\ge - \delta - \left( 1 - (1 - \varepsilon ) p_a^{(n)} \right) \log \left( 1 - p_a^{(n)} \right) \\&\ge - \delta - \varepsilon \log \left( 1 - p_a^{(n)} \right) . \end{aligned}$$

Finally, observe that

$$\begin{aligned} \log \left( 1 - p_a^{(n)} \right) = a_1 \log \left( 1 - \frac{d_1^{(n)}}{m^{(n)}} \right) + a_2 \log \left( 1 - \frac{d_2^{(n)}}{m^{(n)}} \right) \le - \frac{a_1 d_1^{(n)} + a_2 d_2^{(n)}}{m^{(n)}}, \end{aligned}$$

where in the inequality we used the fact that $\log \left( 1 - \frac{d_k^{(n)}}{m^{(n)}} \right) \le - \frac{d_k^{(n)}}{m^{(n)}}$ for $k = 1,2$. Hence, we obtain the expected result. $\square$