Concentration Inequalities for Randomly Permuted Sums

Abstract

Initially motivated by the study of the non-asymptotic properties of non-parametric tests based on permutation methods, concentration inequalities for uniformly permuted sums have been largely studied in the literature. Recently, Delyon et al. proved a new Bernstein-type concentration inequality based on martingale theory. This work presents a new proof of this inequality based on the fundamental inequalities for random permutations of Talagrand. The idea is to first obtain a rough inequality for the square root of the permuted sum, and then, iterate the previous analysis and plug this first inequality to obtain a general concentration of permuted sums around their median. Then, concentration inequalities around the mean are deduced. This method allows us to obtain the Bernstein-type inequality up to constants, and, in particular, to recovers the Gaussian behavior of such permuted sums under classical conditions encountered in the literature. Then, an application to the study of the second kind error rate of permutation tests of independence is presented.

Appendix: A Non-asymptotic Control of the Second Kind Error Rates

Consider the notation from Sect. 17.3. Since this section focuses on the study of the second kind error rate of the test, in all the sequel, the observation is assumed to satisfy the alternative $({\mathcal {H}}_1)$. Let thus P be an alternative, that is P ≠ (P ¹ ⊗ P ²), n ≥ 4 and ${\mathbb {X}}_n=(X_i,\dots ,X_n)$ be an i.i.d. sample from distribution P. Fix α and β be two fixed values in (0, 1). Consider T the test statistic introduced in (17.16), the (random) critical value $q_{1-\alpha }({\mathbb {X}}_n)$ defined in (17.18), and the corresponding permutation test defined in (17.19) by

which precisely rejects independence when $T({\mathbb {X}}_n) > q_{1-\alpha }({\mathbb {X}}_n)$. Notice that this test is exactly the upper-tailed test by permutation introduced in [2].

The aim of this section is to provide different conditions on the alternative P ensuring a control of the second kind error rate by a fixed value β > 0, that is ${\mathbb {P}\left (\Delta _\alpha ({\mathbb {X}}_n)=0\right )} \leq \beta .$ The following steps constitute the first steps of a general study of the separation rates for the previous independence test, and is worked through in the specific case of continuous real-valued random variables in [1, Chapter 4].

Recall the notation introduced in (17.17) for a better readability. For all real-valued measurable function g on ${\mathcal {X}}^2$, denote respectively

$$\displaystyle \begin{aligned} {\mathbb{E}_{P}\left[ g \right]} = {\mathbb{E}\left[ g(X_1^1,X_1^2) \right]}\quad \mbox{and}\quad {\mathbb{E}_{{\perp\perp}}\left[ g \right]} = {\mathbb{E}\left[ g(X_1^1,X_2^2) \right]}, \end{aligned}$$

the expectations of g(X) under the alternative P, that is if X ∼ P, and under the null hypothesis $({\mathcal {H}}_0)$, that is if X ∼ (P ¹ ⊗ P ²).

Assume the following moment assumption holds, that is

so that all variance and second-order moments exist. Then, the following statements hold.

• 1. By Chebychev’s inequality, one has ${\mathbb {P}\left (\Delta _\alpha ({\mathbb {X}}_n)=0\right )} \leq \beta$ as soon as Condition (17.20) is satisfied, that is

  $$\displaystyle \begin{aligned}{\mathbb{E}\left[ T({\mathbb{X}}_n) \right]} \geq q^\alpha_{1-\beta/2} + \sqrt{\frac{2}{\beta} {\operatorname{Var}\left( T({\mathbb{X}}_n) \right)}}.\end{aligned}$$

• 2. On the one hand,

  $$\displaystyle \begin{aligned} {\operatorname{Var}\left( T({\mathbb{X}}_n) \right)} \leq \frac{8}{n} {\left( {\mathbb{E}_{P}\left[ \varphi^2 \right]} + {\mathbb{E}_{{\perp\perp}}\left[ \varphi^2 \right]} \right)}, \end{aligned} $$

  (17.48)

• 3. On the other hand, in order to control the quantile $q^\alpha _{1-\beta /2}$, let us first upper bound the conditional quantile, following Hoeffding’s approach based on the Cauchy-Schwarz inequality, by

  $$\displaystyle \begin{aligned} q_{1-\alpha}({\mathbb{X}}_n) \leq \sqrt{\frac{1-\alpha}{\alpha} {\operatorname{Var}\left( T{\left( {\mathbb{X}}_n^{{{\Pi}}} \right)}\middle| {\mathbb{X}}_n \right)}}. \end{aligned} $$

  (17.49)

• 4. Markov’s inequality allows us to deduce the following bound for the quantile:

  $$\displaystyle \begin{aligned} q^\alpha_{1-\beta/2} \leq 2\sqrt{\frac{1-\alpha}{\alpha}}\sqrt{\frac{2}{\beta}\frac{{\left( {\mathbb{E}_{{\perp\perp}}\left[ \varphi^2 \right]} + {\mathbb{E}_{P}\left[ \varphi^2 \right]} \right)}}{n}}. \end{aligned} $$

  (17.50)

• 5. Finally, combining (17.20), (17.48) and (17.50) ensures that ${\mathbb {P}\left (\Delta _\alpha ({\mathbb {X}}_n)=0\right )} \leq \beta$ as soon as Condition (17.21) is satisfied, that is

  $$\displaystyle \begin{aligned}{\mathbb{E}\left[ T({\mathbb{X}}_n) \right]} \geq \frac{4}{\sqrt{\alpha}}\sqrt{\frac{2}{\beta}\frac{{\mathbb{E}_{P}\left[ \varphi^2 \right]} + {\mathbb{E}_{{\perp\perp}}\left[ \varphi^2 \right]}}{n}}.\end{aligned}$$

This section is divided in five subsections, each one of them respectively proving a point stated above. The first one proves the sufficiency of Condition (17.20) in order to control the second kind error rate. The second, third and fourth ones provide respectively upper-bounds of the variance term, the critical value and the quantile $q^\alpha _{1-\beta /2}$. Finally, the fifth one provides the sufficiency of Condition (17.21).

17.1.1 A First Condition Ensuing from Chebychev’s Inequality

In this section, we prove the sufficiency of a first simple condition, derived from Chebychev’s inequality in order to control the second error rate. Assume that (17.20) is satisfied, that is

$$\displaystyle \begin{aligned}{\mathbb{E}\left[ T({\mathbb{X}}_n) \right]} \geq q^\alpha_{1-\beta/2} + \sqrt{\frac{2}{\beta} {\operatorname{Var}\left( T({\mathbb{X}}_n) \right)}}.\end{aligned} $$

Then,

$$\displaystyle \begin{aligned} \begin{array}{rcl} {\mathbb{P}\left(\Delta_\alpha({\mathbb{X}}_n)=0\right)} &\displaystyle =&\displaystyle {\mathbb{P}\left(T({\mathbb{X}}_n) \leq q_{1-\alpha}({\mathbb{X}}_n)\right)} \end{array} \end{aligned} $$

(17.51)

$$\displaystyle \begin{aligned} \begin{array}{rcl} &\displaystyle =&\displaystyle {\mathbb{P}\left({\left\{ T({\mathbb{X}}_n) \leq q_{1-\alpha}({\mathbb{X}}_n) \right\}} \cap {\left\{ q_{1-\alpha}({\mathbb{X}}_n) \leq q^\alpha_{1-\beta/2} \right\}}\right)} \\ &\displaystyle &\displaystyle + \ {\mathbb{P}\left({\left\{ T({\mathbb{X}}_n) \leq q_{1-\alpha}({\mathbb{X}}_n) \right\}} \cap {\left\{ q_{1-\alpha}({\mathbb{X}}_n) > q^\alpha_{1-\beta/2} \right\}}\right)} \\ &\displaystyle \leq &\displaystyle {\mathbb{P}\left(T({\mathbb{X}}_n) \leq q^\alpha_{1-\beta/2}\right)} + {\mathbb{P}\left(q_{1-\alpha}({\mathbb{X}}_n) > q^\alpha_{1-\beta/2}\right)} \\ &\displaystyle \leq &\displaystyle {\mathbb{P}\left(T({\mathbb{X}}_n) \leq q^\alpha_{1-\beta/2}\right)} + \frac{\beta}{2}, {} \end{array} \end{aligned} $$

(17.52)

by definition of the quantile $q^\alpha _{1-\beta /2}$. Yet, from (17.20) one obtains from Chebychev’s inequality that

$$\displaystyle \begin{aligned} \begin{array}{rcl} {\mathbb{P}\left(T({\mathbb{X}}_n) \leq q^\alpha_{1-\beta/2}\right)} &\displaystyle \leq &\displaystyle {\mathbb{P}\left(T({\mathbb{X}}_n) \leq {\mathbb{E}\left[ T({\mathbb{X}}_n) \right]} - \sqrt{\frac{2}{\beta} {\operatorname{Var}\left( T({\mathbb{X}}_n) \right)}}\right)} \\ &\displaystyle \leq &\displaystyle {\mathbb{P}\left({\left| T({\mathbb{X}}_n) - {\mathbb{E}\left[ T({\mathbb{X}}_n) \right]} \right|} \geq \sqrt{\frac{2}{\beta} {\operatorname{Var}\left( T({\mathbb{X}}_n) \right)}}\right)} \\ &\displaystyle \leq &\displaystyle \frac{\beta}{2}. {} \end{array} \end{aligned} $$

(17.53)

Finally, both (17.52) and (17.53) lead to the desired control ${\mathbb {P}\left (\Delta _\alpha ({\mathbb {X}}_n)=0\right )}\leq \beta$ which ends the proof.

17.1.2 Control of the Variance in the General Case

To upper bound the variance term, we apply Lemma 17.3.1 which directly implies that

$$\displaystyle \begin{aligned}{\operatorname{Var}\left( T({\mathbb{X}}_n) \right)} \leq \frac{2}{n} {\left( {\mathbb{E}_{P}\left[ \varphi^2 \right]} + 4 {\mathbb{E}_{{\perp\perp}}\left[ \varphi^2 \right]} \right)},\end{aligned} $$

which directly leads to (17.48).

17.1.3 Control of the Critical Value Based on Hoeffding’s Approach

This section is devoted to the proof the inequality (17.49), namely

The proof of this upper-bound follows Hoeffding’s approach in [16], and relies on a normalizing trick, and the Cauchy-Schwarz inequality. From now on, for a better readability, denote respectively ${\mathbb {E}^*\left [ \cdot \right ]}$ and ${\operatorname {Var}^*\left ( \cdot \right )}$ the conditional expectation and variance given the sample ${\mathbb {X}}_n$.

As in Hoeffding [16], the first step is to center and normalize the permuted test statistic. Yet, by construction the permuted test statistic is automatically centered, that is ${\mathbb {E}^*\left [ T{\left ( {\mathbb {X}}_n^{{{\Pi }}} \right )} \right ]} = 0$, as one can notice that

$$\displaystyle \begin{aligned}T{\left( {\mathbb{X}}_n^{{{\Pi}}} \right)}=\frac{1}{n-1}{\left( \sum_{i=1}^n \varphi{\left( X_i^1,X_{{{\Pi}}(i)}^2 \right)} - {\mathbb{E}^*\left[ \sum_{i=1}^n \varphi{\left( X_i^1,X_{{{\Pi}}(i)}^2 \right)} \right]} \right)}.\end{aligned} $$

Therefore, just consider the normalizing term

$$\displaystyle \begin{aligned}\nu(\mathbb{X}_n) = {\operatorname{Var}^*\left( T{\left( {\mathbb{X}}_n^{{{\Pi}}} \right)} \right)} = {\mathbb{E}^*\left[ T{\left( {\mathbb{X}}_n^{{{\Pi}}} \right)}^2 \right]} = \frac{1}{n!}\sum_{{{\tau}}\in{\mathfrak{S}_{n}}} {\left( T{\left( {\mathbb{X}}_n^{{{\tau}}} \right)} \right)}^2.\end{aligned} $$

Two cases appear: either $\nu (\mathbb {X}_n)=0$ or not.

In the first case, the nullity of the conditional variance implies that all the permutations of the test statistic are equal. Hence, for all permutation τ of ${\left \{ 1,\dots ,n \right \}}$, one has $T({\mathbb {X}}_n^{{{\tau }}}) = T({\mathbb {X}}_n)$. Since the centering term ${\mathbb {E}^*\left [ \sum _{i=1}^n \varphi {\left ( X_i^1,X_{{{\Pi }}(i)}^2 \right )} \right ]}=n^{-1}\sum _{i,j=1}^n\varphi (X_i^1,X_j^2)$ is permutation invariant, one obtains the equality of the permuted sums, that is

$$\displaystyle \begin{aligned}\sum_{i=1}^n \varphi{\left( X_i^1,X_{{{\tau}}(i)}^2 \right)} = \sum_{i=1}^n \varphi{\left( X_i^1,X_i^2 \right)},\end{aligned} $$

and this for all permutation τ. In particular, the centering term is also equal to $\sum _{i=1}^n \varphi {\left ( X_i^1,X_i^2 \right )}$. Indeed, by invariance of the sum (applied in the third equality below),

Therefore, $T({\mathbb {X}}_n)$ is equal to zero, and thus, so is $q_{1-\alpha }({\mathbb {X}}_n)$. Finally, inequality (17.55) is satisfied since

$$\displaystyle \begin{aligned}q_{1-\alpha}({\mathbb{X}}_n) = 0 \leq 0 = \sqrt{\frac{1-\alpha}{\alpha} {\operatorname{Var}\left( T{\left( {\mathbb{X}}_n^{{{\Pi}}} \right)}\middle| {\mathbb{X}}_n \right)}}.\end{aligned}$$

Consider now the second case, and assume $\nu (\mathbb {X}_n)>0$. Let us introduce the (centered and) normalized statistic

$$\displaystyle \begin{aligned}T'{\left( \mathbb{X}_n \right)} = \frac{1}{\sqrt{\nu(\mathbb{X}_n)}}{\left( T({\mathbb{X}}_n) \right)}.\end{aligned}$$

In particular, the new statistic $T'{\left ( \mathbb {X}_n \right )}$ satisfies

$$\displaystyle \begin{aligned}{\mathbb{E}^*\left[ T'{\left( {\mathbb{X}}_n^{{{\Pi}}} \right)} \right]} = 0 \quad \mbox{and}\quad {\operatorname{Var}^*\left( T'{\left( {\mathbb{X}}_n^{{{\Pi}}} \right)} \right)}\leq 1.\end{aligned}$$

One may moreover notice that the normalizing term $\nu ({\mathbb {X}}_n)$ is permutation invariant, that is, for all permutations τ and τ′ in ${\mathfrak {S}_{n}}$,

$$\displaystyle \begin{aligned}\nu{\left( \mathbb{X}_n^{{{\tau}}} \right)}=\nu{\left( \mathbb{X}_n \right)}=\nu{\left( \mathbb{X}_n^{{{\tau}}'} \right)}.\end{aligned}$$

In particular, since $\nu (\mathbb {X}_n) >0$,

$$\displaystyle \begin{aligned}T{\left( {\mathbb{X}}_n^{{{\tau}}} \right)}\leq T{\left( {\mathbb{X}}_n^{{{\tau}}'} \right)}\quad \Leftrightarrow \quad T'{\left( {\mathbb{X}}_n^{{{\tau}}} \right)}\leq T'{\left( {\mathbb{X}}_n^{{{\tau}}'} \right)}.\end{aligned}$$

Therefore, as the test Δ_α depends only on the comparison of the ${\left \{ T{\left ( {\mathbb {X}}_n^{{{\tau }}} \right )} \right \}}_{{{\tau }}\in {\mathfrak {S}_{n}}}$, the test statistic T can be replaced by T′, and the new critical value becomes

$$\displaystyle \begin{aligned} q^{\prime}_{1-\alpha}({\mathbb{X}}_n) = T^{\prime (n!-\lfloor n!\alpha\rfloor)}{\left( \mathbb{X}_n \right)} = \frac{T^{(n!-\lfloor n!\alpha\rfloor)}{\left( \mathbb{X}_n \right)}}{\nu({\mathbb{X}}_n)}=\frac{q_{1-\alpha}({\mathbb{X}}_n)}{\nu({\mathbb{X}}_n)}. \end{aligned} $$

(17.54)

Moreover, following the proof of Theorem 2.1. of Hoeffding [16], one can show (as below) that

$$\displaystyle \begin{aligned} q^{\prime}_{1-\alpha}({\mathbb{X}}_n) \leq \sqrt{\frac{1-\alpha}{\alpha}}. \end{aligned} $$

(17.55)

Hence, combining (17.55) with (17.54) leads straightforwardly to (17.49).

Finally, remains the proof of (17.55). There are two cases:

First Case

If $q^{\prime }_{1-\alpha }({\mathbb {X}}_n)\leq 0$, then (17.55) is satisfied.

Second Case

If $q^{\prime }_{1-\alpha }({\mathbb {X}}_n) > 0$, then introduce $Y=q^{\prime }_{1-\alpha }({\mathbb {X}}_n) - T'{\left ( {\mathbb {X}}_n^{{{\Pi }}} \right )}.$

First, since by construction, ${\mathbb {E}^*\left [ T'{\left ( {\mathbb {X}}_n^{{{\Pi }}} \right )} \right ]}=0$, one directly obtains ${\mathbb {E}^*\left [ Y \right ]} = q^{\prime }_{1-\alpha }({\mathbb {X}}_n)$. Hence,

and by the Cauchy-Schwarz inequality,

Yet, on one hand,

$$\displaystyle \begin{aligned} \begin{array}{rcl} {\mathbb{E}^*\left[ Y^2 \right]} &\displaystyle =&\displaystyle {\mathbb{E}^*\left[ {\left( q^{\prime}_{1-\alpha}({\mathbb{X}}_n) - T'{\left( {\mathbb{X}}_n^{{{\Pi}}} \right)} \right)}^2 \right]} \\ &\displaystyle =&\displaystyle {\left( q^{\prime}_{1-\alpha}({\mathbb{X}}_n) \right)}^2 + {\mathbb{E}^*\left[ {\left( T'{\left( {\mathbb{X}}_n^{{{\Pi}}} \right)} \right)}^2 \right]} -2 q^{\prime}_{1-\alpha}({\mathbb{X}}_n){\mathbb{E}^*\left[ T'{\left( {\mathbb{X}}_n^{{{\Pi}}} \right)} \right]} \\ &\displaystyle =&\displaystyle {\left( q^{\prime}_{1-\alpha}({\mathbb{X}}_n) \right)}^2 + {\operatorname{Var}^*\left( T'{\left( {\mathbb{X}}_n^{{{\Pi}}} \right)} \right)} \\ &\displaystyle \leq &\displaystyle {\left( q^{\prime}_{1-\alpha}({\mathbb{X}}_n) \right)}^2 + 1, \end{array} \end{aligned} $$

since by the normalizing initial step, ${\operatorname {Var}^*\left ( T'{\left ( {\mathbb {X}}_n^{{{\Pi }}} \right )} \right )}\leq 1$.

And, on the other hand,

So finally,

$$\displaystyle \begin{aligned}{\left( q^{\prime}_{1-\alpha}({\mathbb{X}}_n) \right)}^2 \leq {\left( 1 - \alpha \right)}{\left( {\left( q^{\prime}_{1-\alpha}({\mathbb{X}}_n) \right)}^2 + 1 \right)},\end{aligned}$$

which is equivalent to ${\left ( q^{\prime }_{1-\alpha }({\mathbb {X}}_n) \right )}^2 \leq (1-\alpha )/\alpha ,$ and thus ends the proof of (17.55).

17.1.4 Control of the Quantile of the Critical Value

The control of the conditional quantile allows us to upper bound its own quantile $q^\alpha _{1-\beta /2}$ as stated in (17.50), that is

$$\displaystyle \begin{aligned}q^\alpha_{1-\beta/2} \leq 2\sqrt{\frac{1-\alpha}{\alpha}}\sqrt{\frac{2}{\beta}\frac{{\left( {\mathbb{E}_{{\perp\perp}}\left[ \varphi^2 \right]} + {\mathbb{E}_{P}\left[ \varphi^2 \right]} \right)}}{n}}.\end{aligned} $$

Indeed, (17.49) ensures that

$$\displaystyle \begin{aligned}q_{1-\alpha}({\mathbb{X}}_n) \leq \sqrt{\frac{1-\alpha}{\alpha}} \sqrt{{\mathbb{E}\left[ T{\left( {\mathbb{X}}_n^{{{\Pi}}} \right)}^2\middle| {\mathbb{X}}_n \right]}},\end{aligned} $$

and in particular, the (1 − β∕2)-quantile of $q_{1-\alpha }({\mathbb {X}}_n)$ satisfies

$$\displaystyle \begin{aligned} q^\alpha_{1-\beta/2} \leq \sqrt{\frac{1-\alpha}{\alpha}}\sqrt{\zeta_{1-\beta/2}},\end{aligned} $$

(17.56)

where ζ _1−β∕2 is the (1 − β∕2)-quantile of ${\mathbb {E}\left [ T{\left ( {\mathbb {X}}_n^{{{\Pi }}} \right )}^2\middle | {\mathbb {X}}_n \right ]}$. Yet, from Markov’s inequality, for all positive x,

$$\displaystyle \begin{aligned}{\mathbb{P}\left({\mathbb{E}\left[ T{\left( {\mathbb{X}}_n^{{{\Pi}}} \right)}^2\middle| {\mathbb{X}}_n \right]} \geq x\right)} \leq \frac{{\mathbb{E}\left[ T{\left( {\mathbb{X}}_n^{{{\Pi}}} \right)}^2 \right]}}{x}.\end{aligned} $$

In particular, the choice of $x=2{\mathbb {E}\left [ T{\left ( {\mathbb {X}}_n^{{{\Pi }}} \right )}^2 \right ]}/\beta$ leads to the control of the quantile

$$\displaystyle \begin{aligned} \zeta_{1-\beta/2}\leq \frac{2{\mathbb{E}\left[ T{\left( {\mathbb{X}}_n^{{{\Pi}}} \right)}^2 \right]}}{\beta}.\end{aligned} $$

(17.57)

Moreover, noticing that one can write

the second-order moment in (17.57) can be rewritten

by independence between Π and ${\mathbb {X}}_n$, where

On the one hand, for all 1 ≤ i, j, k, l ≤ n, the Cauchy-Schwarz inequality always ensures

$$\displaystyle \begin{aligned} {\mathbb{E}\left[ \varphi(X_i^1,X_j^2) \varphi(X_k^1,X_l^2) \right]} \leq \sqrt{{\mathbb{E}\left[ \varphi^2(X_i^1,X_j^2) \right]} {\mathbb{E}\left[ \varphi^2(X_k^1,X_l^2) \right]}} \leq {\mathbb{E}_{{\perp\perp}}\left[ \varphi^2 \right]} + {\mathbb{E}_{P}\left[ \varphi^2 \right]}, \end{aligned} $$

(17.58)

since for all 1 ≤ i, j ≤ n, ${\mathbb {E}\left [ \varphi ^2(X_i^1,X_j^2) \right ]}\leq {\mathbb {E}_{{\perp \perp }}\left [ \varphi ^2 \right ]} + {\mathbb {E}_{P}\left [ \varphi ^2 \right ]}$.

On the other hand, remains to control the sum $(n-1)^{-2} \sum _{i,j=1}^n \sum _{k,l=1}^n E_{i,j,k,l}$. Three cases appear.

First Case

If i ≠ k and j ≠ l (occurring [n(n − 1)]² times), then

$$\displaystyle \begin{aligned}E_{i,j,k,l} = \frac{1}{n(n-1)} - \frac{1}{n^2} = \frac{1}{n^2(n-1)}.\end{aligned}$$

Second Case

If [i ≠ k and j = l] or [i = k and j ≠ l], then

$$\displaystyle \begin{aligned} E_{i,j,k,l} = 0-1/n^2 \leq 0. \end{aligned}$$

Third Case

If i = k and j = l (occurring n(n − 1) times), then

$$\displaystyle \begin{aligned}E_{i,j,k,l} = \frac{1}{n} - \frac{1}{n^2} = \frac{n-1}{n^2}\leq \frac{1}{n}.\end{aligned}$$

Therefore,

$$\displaystyle \begin{aligned} \begin{array}{rcl} \frac{1}{(n-1)^2} \sum_{i,j=1}^n \sum_{k,l=1}^n E_{i,j,k,l} &\displaystyle \leq &\displaystyle \frac{1}{(n-1)^2}{\left( [n(n-1)]^2 \times\frac{1}{n^2(n-1)} + n(n-1) \times \frac{1}{n} \right)} \\ &\displaystyle \leq &\displaystyle \frac{2}{n-1} \\ &\displaystyle \leq &\displaystyle \frac{4}{n}. {} \end{array} \end{aligned} $$

(17.59)

Finally, both (17.58) and (17.59) imply that

$$\displaystyle \begin{aligned} {\mathbb{E}\left[ T{\left( {\mathbb{X}}_n^{{{\Pi}}} \right)}^2 \right]} \leq \frac{4}{n}{\left( {\mathbb{E}_{{\perp\perp}}\left[ \varphi^2 \right]} + {\mathbb{E}_{P}\left[ \varphi^2 \right]} \right)}, \end{aligned} $$

(17.60)

Therefore, combining (17.56), (17.57) and (17.60) ends the proof of (17.50).

17.1.5 A First Condition Ensuing from Hoeffding’s Approach

Back to the condition (17.20) derived from Chebychev’s inequality, both (17.48) and (17.50) imply that

$$\displaystyle \begin{aligned}q^\alpha_{1-\beta/2} + \sqrt{\frac{2}{\beta} {\operatorname{Var}\left( T({\mathbb{X}}_n) \right)}} \leq \sqrt{\frac{2}{\beta}\frac{{\left( {\mathbb{E}_{P}\left[ \varphi^2 \right]} + {\mathbb{E}_{{\perp\perp}}\left[ \varphi^2 \right]} \right)}}{n}} {\left( 2\sqrt{\frac{1-\alpha}{\alpha}}+ \sqrt{8} \right)},\end{aligned}$$

with $2\sqrt {(1-\alpha )/\alpha }+ \sqrt {8}\leq 4/\sqrt {\alpha }$, since $\sqrt {1-\alpha }+\sqrt {\alpha }\leq \sqrt {2}$. Finally, the right-hand side of condition (17.20) being upper bounded by

$$\displaystyle \begin{aligned}\frac{4}{\sqrt{\alpha}}\sqrt{\frac{2}{\beta}\frac{{\left( {\mathbb{E}_{P}\left[ \varphi^2 \right]} + {\mathbb{E}_{{\perp\perp}}\left[ \varphi^2 \right]} \right)}}{n}},\end{aligned}$$

which is exactly the right-hand side of (17.21), this ensures the sufficiency of condition 17.21 to control the second kind error rate by β.