On data complexity of distinguishing attacks versus message recovery attacks on stream ciphers

Abstract

We revisit the different approaches used in the literature to estimate the data complexity of distinguishing attacks on stream ciphers and analyze their inter-relationships. In the process, we formally argue which approach is applicable (or not applicable) in what scenario. To our knowledge, this is the first kind of such an exposition. We also perform a rigorous statistical analysis of the message recovery attack that exploits a distinguisher and show that in practice there is a significant gap between the data complexities of a message recovery attack and the underlying distinguishing attack. This gap is not necessarily determined by a constant factor as a function of the false positive and negative rate, as one would expect. Rather this gap is also a function of the number of samples of the distinguishing attack. We perform a case study on RC4 stream cipher to demonstrate that the typical complexities for message recovery attack inferred in the literature are but under-estimates and the actual estimates are quite larger.

Appendices

Appendix A: Proof of Lemma 2

Proof

Suppose ${\mathcal {S}}$ is the sample space and $\phi : {\mathcal {S}} \longrightarrow [0,1]$ be the test function [9] for the concerned test with false positive rate ($\alpha$) and false negative rate ($\beta$), i.e., we reject $H_0$ with probability $\phi (\varvec{x})$, when $\varvec{X}=\varvec{x}$ is observed. Then we have by definition

$$\begin{aligned} E_{H_0}[\phi (\varvec{X})]= & {} \sum _{\varvec{x} \in {\mathcal {S}}} \phi (\varvec{x})P_n(\varvec{x})= \alpha ,\\ E_{H_1}[(1-\phi )(\varvec{X})]= & {} \sum _{\varvec{x} \in {\mathcal {S}}} (1-\phi (\varvec{x}))Q_n(\varvec{x})= \beta . \end{aligned}$$

Note that,

$$\begin{aligned} D_{KL}(Q_n\Vert P_n)= & {} \sum _{\varvec{x} \in {\mathcal {S}}} Q_n(\varvec{x}) \log _2 \frac{Q_n(\varvec{x})}{P_n(\varvec{x})} = \sum _{\varvec{x} \in {\mathcal {S}}} P_n(\varvec{x})\frac{Q_n(\varvec{x})}{P_n(\varvec{x})} \log _2 \frac{Q_n(\varvec{x})}{P_n(\varvec{x})} \\= & {} \sum _{\varvec{x} \in {\mathcal {S}}} P_n(\varvec{x}) f\left( \frac{Q_n(\varvec{x})}{P_n(\varvec{x})}\right) \end{aligned}$$

where $f:{\mathbb {R}}^{+} \rightarrow {\mathbb {R}}$ defined as $f(z) = z \log _2 (z),\,\forall \, z >0.$ Then, $\dfrac{d^2f(z)}{dz^2} = {(z\ln 2)}^{-1}> 0,\, \forall \,z >0$; which implies f is convex and continuous also. Hence, using Jensen’s Inequality, we have

$$\begin{aligned}&\sum _{\varvec{x} \in {\mathcal {S}}} \frac{\phi (\varvec{x})P_n (\varvec{x})}{\sum _{\varvec{x} \in {\mathcal {S}}} \phi (\varvec{x}) P_n(\varvec{x})} f\left( \frac{Q_n(\varvec{x})}{P_n(\varvec{x})}\right) \\ \\&\quad \ge f\left( \sum _{\varvec{x} \in {\mathcal {S}}} \frac{ \phi (\varvec{x}) P_n(\varvec{x})}{\sum _{\varvec{x} \in {\mathcal {S}}} \phi (\varvec{x}) P_n(\varvec{x})} \frac{Q_n(\varvec{x})}{P_n(\varvec{x})}\right) \\ \\&\quad = f\left( \frac{\sum _{\varvec{x} \in {\mathcal {S}}} \phi (\varvec{x}) Q_n(\varvec{x})}{ \sum _{\varvec{x} \in {\mathcal {S}}} \phi (\varvec{x})P_n(\varvec{x})}\right) \\ \\&\quad = f\left( \frac{1-\beta }{\alpha }\right) . \end{aligned}$$

Hence,

$$\begin{aligned} \sum _{\varvec{x} \in {\mathcal {S}}} \phi (\varvec{x}) P_n(\varvec{x}) f\left( \frac{Q_n(\varvec{x})}{P_n(\varvec{x})}\right)\ge & {} \sum _{\varvec{x} \in {\mathcal {S}}} \phi (\varvec{x}) P_n(\varvec{x}) f\left( \frac{1-\beta }{\alpha }\right) \\ \\= & {} \alpha f\left( \frac{1-\beta }{\alpha }\right) \\ \\= & {} (1-\beta )\log _2\left( \frac{1-\beta }{\alpha }\right) . \end{aligned}$$

Replacing $\phi$ by $1-\phi$ and taking similar sums we get,

$$\begin{aligned} \sum _{\varvec{x} \in {\mathcal {S}}} (1-\phi (\varvec{x})) P_n(\varvec{x}) f\left( \frac{Q_n(\varvec{x})}{P_n(\varvec{x})}\right) \ge \beta \log _2\frac{\beta }{1-\alpha }. \end{aligned}$$

Summing the above two inequalities we get

$$\begin{aligned}&\sum _{\varvec{x} \in {\mathcal {S}}} P_n(\varvec{x}) f\left( \frac{Q_n(\varvec{x})}{P_n(\varvec{x})}\right) \\&\quad \ge \beta \log _2\frac{\beta }{1-\alpha } +(1- \beta )\log _2\frac{1-\beta }{\alpha }, \end{aligned}$$

and hence the desired result. $\square$

Appendix B: Proof of Lemma 4

Proof

This proof of Chernoff–Stein Lemma occurs in [10]. First note that,

$$\begin{aligned} \log _2 \left[ \frac{P_n(X_1,\ldots ,X_n)}{Q(X_1,\ldots ,X_n)} \right] = \sum _{k=1}^n \log _2 \left[ \frac{P(X_k)}{Q(X_k)} \right] , \end{aligned}$$

and by Law of Large numbers

$$\begin{aligned} \frac{1}{n} \sum _{k=1}^n \log _2 \left[ \frac{P(X_k)}{Q(X_k)} \right]&\mathop {\longrightarrow }\limits ^{p}&E_P \left[ \log _2 \left( \frac{P(X_1)}{Q(X_1)} \right) \right] \\= & {} D_{KL} (P \Vert Q), \end{aligned}$$

under the null. Hence,

$$\begin{aligned} \frac{1}{n} \log _2 \left[ \frac{P_n(X_1,\ldots ,X_n)}{Q(X_1,\ldots ,X_n)} \right] \mathop {\longrightarrow }\limits ^{p} D_{KL} (P \Vert Q), \end{aligned}$$

which by definition gives that $\forall \epsilon , \alpha > 0$, $\exists$ $N_{\epsilon ,\alpha } \in {\mathbb {N}}$ such that, $\forall \, n \ge N_{\epsilon ,\alpha }$, we have

$$\begin{aligned} P_n \left[ \Bigg |\frac{1}{n} \log _2 \left[ \frac{P_n(\varvec{X})}{Q_n(\varvec{X})} \right] - D_{KL} (P\Vert Q)\Bigg | < \epsilon \right] \ge 1- \alpha , \end{aligned}$$

(26)

where $D=D_{KL}(P\Vert Q)$ and $\varvec{X}=(X_1,\ldots ,X_n)$. Now, define $A_n^{\epsilon }$ be the subset of ${\mathcal {\chi }}^n$ consisting of all $\varvec{x}=(x_1,\ldots ,x_n)$ such that

$$\begin{aligned} P_n(\varvec{x})2^{-n(D+\epsilon )}< Q_n(\varvec{x}) < P_n(\varvec{x}) 2^{-n(D-\epsilon )}, \end{aligned}$$

i.e.,

$$\begin{aligned} \left| \frac{1}{n} \log _2 \left[ \frac{P_n(\varvec{x})}{Q_n(\varvec{x})} \right] - D\right| < \epsilon . \end{aligned}$$

Then, Eq. (26) gives,

$$\begin{aligned} P_n (A_n^{\epsilon }) \ge 1- \alpha , \end{aligned}$$

$\forall \, n \ge N_{\epsilon ,\alpha }$. Also note that

$$\begin{aligned} Q_n(A_n^{\epsilon })= & {} \sum _{\varvec{x} \in A_n^{\epsilon }} Q_n (\varvec{x}) \end{aligned}$$

(27)

$$\begin{aligned}< & {} \sum _{\varvec{x} \in A_n^{\epsilon }} P_n (\varvec{x}) 2^{-n(D-\epsilon )} < 2^{-n(D-\epsilon )}, \end{aligned}$$

(28)

and

$$\begin{aligned} Q_n(A_n^{\epsilon })= & {} \sum _{\varvec{x} \in A_n^{\epsilon }} Q_n (\varvec{x}) \end{aligned}$$

(29)

$$\begin{aligned}> & {} \sum _{\varvec{x} \in A_n^{\epsilon }} P_n (\varvec{x}) 2^{-n(D+\epsilon )} \end{aligned}$$

(30)

$$\begin{aligned}= & {} 2^{-n(D+\epsilon )} P_n(A_n^{\epsilon }) \ge (1 - \alpha )2^{-n(D+\epsilon )}, \end{aligned}$$

(31)

$\forall \, n \ge N_{\epsilon ,\delta }$. Now consider the test which rejects the null if and only if $\varvec{x} \notin A_n^{\epsilon }$. Then, by Eq. (27) $\forall \; n \ge N_{\epsilon ,\alpha }$,

$$\begin{aligned} 1-P_n(A_n^{\epsilon })< \alpha \quad \text {and} \quad Q_n(A_n^{\epsilon })< 2^{-n(D-\epsilon )}, \end{aligned}$$

which says that the non-randomized test with acceptance region $A_n^{\epsilon }$ has size less than $\alpha$ and has false negative error less than $2^{-n(D-\epsilon )}$. So, by definition of $\beta _{n,\alpha }$, which is the least attainable false negative error for level $\alpha$ non-randomized tests, we have

$$\begin{aligned} \beta _{n,\alpha } < 2^{-n(D-\epsilon )}. \end{aligned}$$

Thus we have, $\forall \, n \ge N_{\epsilon ,\alpha },\epsilon > 0$,

$$\begin{aligned} \frac{\log _2 \beta _{n,\alpha }}{n} < -D + \epsilon \Longrightarrow \limsup _{n \rightarrow \infty } \frac{\log _2 \beta _{n,\alpha }}{n} \le - D. \end{aligned}$$

(32)

On the other hand, consider any other test with rejection region ${\mathcal {R}}$, such that $P_n({\mathcal {R}})< \alpha$. Then we have, $\forall \, n \ge N_{\epsilon ,\alpha }$,

$$\begin{aligned} Q_n({\mathcal {R}}^c)\ge & {} Q_n({\mathcal {R}}^c \cap A_n^{\epsilon }) \\= & {} \sum _{\varvec{x} \in {\mathcal {R}}^c \cap A_n^{\epsilon }}Q_n(\varvec{x}) \\> & {} \sum _{\varvec{x} \in {\mathcal {R}}^c \cap A_n^{\epsilon }} 2^{-n(D+\epsilon )} P_n(\varvec{x}) \\= & {} 2^{-n(D+\epsilon )} P_n({\mathcal {R}}^c \cap A_n^{\epsilon }) \\\ge & {} 2^{-n(D+\epsilon )} (P_n({A_n^{\epsilon }})-P_n({\mathcal {R}})) \\\ge & {} 2^{-n(D+\epsilon )} (1-2\alpha ) \end{aligned}$$

Hence, $\forall \, n \ge N_{\epsilon ,\alpha }$,

$$\begin{aligned} \beta _{n,\alpha } = \min _{{\mathcal {R}},P_n({\mathcal {R}})<\alpha }Q_n({\mathcal {R}}^c) > 2^{-n(D+\epsilon )} (1-2\alpha ), \end{aligned}$$

which in turn gives,

$$\begin{aligned} \liminf _{n \rightarrow \infty } \frac{\log _2 \beta _{n,\alpha }}{n} \ge -D - \epsilon , \;\forall \, \epsilon >0. \end{aligned}$$

Therefore,

$$\begin{aligned} \liminf _{n \rightarrow \infty } \frac{\log _2 \beta _{n,\alpha }}{n} \ge -D. \end{aligned}$$

(33)

Combining Eqs. (32) and (33) we get the desired result. $\square$

Appendix C: Proof of Lemma 6

Proof

Without loss of generality assume that $s=0$. Note that

$$\begin{aligned} \sum _{r=1}^{N-1} \Pr (W_0< W_r)\ge & {} \Pr ( \exists \; 1 \le k \le (N-1) \; s.t. \; W_0 < W_k ) \\= & {} \Pr \Bigg (\max _{0 \le i \le N-1} W_i \ne W_0\Bigg ) \\\ge & {} \Pr ( W_0 \le W_l), \quad \forall \; l=1,\ldots ,N-1. \end{aligned}$$

$\forall \, k=1,\ldots ,N-1,$ we have, $\Pr (W_0< W_k) = \Pr (W_0-W_k < 0)$ and

$$\begin{aligned} W_0-W_k \sim {\mathcal {N}}(\mu _0-\mu _k, \frac{1}{n}(\sigma _{00} +\sigma _{kk}-2\sigma _{0k})), \end{aligned}$$

i.e.,

$$\begin{aligned} R_k := \dfrac{\sqrt{n}}{\sqrt{\sigma _{00}+\sigma _{kk}-2 \sigma _{0k}}}(W_0-W_k) \sim {\mathcal {N}}(\sqrt{n}\delta _k,1). \end{aligned}$$

Hence,

$$\begin{aligned} \Pr (W_0-W_k< 0) = \Pr (R_k < 0)= \varPhi (-\sqrt{n} \delta _k). \end{aligned}$$

(34)

Thus,

$$\begin{aligned} \sum _{r=1}^{N-1} \varPhi (-\sqrt{n} \delta _r)\ge & {} \Pr ( \exists \; 1 \le k \le (N-1) \; s.t. \; W_0 < W_k ) \\\ge & {} \varPhi (-\sqrt{n}\delta _l), \quad \forall \; l=1,\ldots ,N-1. \end{aligned}$$

which gives

$$\begin{aligned} \sum _{r=1}^{N-1} \varPhi (-\sqrt{n} \delta _r)\ge & {} \Pr ( \exists \; 1 \le k \le (N-1) \; s.t. \; W_0 < W_k ) \\\ge & {} \max _{1 \le l \le N-1} \varPhi (-\sqrt{n}\delta _l) = \varPhi (-\sqrt{n}\delta ) \end{aligned}$$

Let, Now, we shall show that the ratio of the two extremes in the inequality stated above goes to 1 as n goes to infinity, i.e., for large n they are quite close and then we can approximate the middle term by the right-hand extreme. The limit we get by using L’Hospital’s Rule is as follows,

$$\begin{aligned} \lim _{n \rightarrow \infty } \frac{ \varPhi (-\sqrt{n}\delta )}{\sum _{r=1}^{N-1} \varPhi (-\sqrt{n} \delta _r)}= & {} \lim _{n \rightarrow \infty } \frac{\delta e^{-\frac{n \delta ^2}{2}}}{\sum _{r=1}^{N-1} \delta _r e^{-\frac{n \delta _r^2}{2}}} \\= & {} 1 \end{aligned}$$

as $\delta < \delta _k, \quad \forall k \ne 0,j.$ So,

$$\begin{aligned} \Pr \Bigg (\max _{0 \le i \le N-1} W_i \ne W_0\Bigg ) = \Pr (\exists \; 1 \le k \le (N-1) \; s.t. \; W_0 < W_k ) \approx \varPhi (-\sqrt{n}\delta ). \end{aligned}$$

$\square$