Revisiting Mutual Information Analysis: Multidimensionality, Neural Estimation and Optimality Proofs

Abstract

Recent works showed how Mutual Information Neural Estimation (MINE) could be applied to side-channel analysis in order to evaluate the amount of leakage of an electronic device. One of the main advantages of MINE over classical estimation techniques is to enable the computation between high dimensional traces and a secret, which is relevant for leakage assessment. However, optimally exploiting this information in an attack context in order to retrieve a secret remains a non-trivial task especially when a profiling phase of the target is not allowed. Within this context, the purpose of this paper is to address this problem based on a simple idea: there are multiple leakage sources in side-channel traces and optimal attacks should necessarily exploit most/all of them. To this aim, a new mathematical framework, designed to bridge classical Mutual Information Analysis (MIA) and the multidimensional aspect of neural-based estimators, is proposed. One of the goals is to provide rigorous proofs consolidating the mathematical basis behind MIA, thus alleviating inconsistencies found in the state of the art. This framework allows to derive a new attack called Neural Estimated Mutual Information Analysis (NEMIA). To the best of our knowledge, it is the first unsupervised attack able to benefit from both the power of deep learning techniques and the valuable theoretical properties of MI. From simulations and experiments conducted in this paper, it seems that NEMIA performs better than classical and more recent deep learning based unsupervised side-channel attacks, especially in low-information contexts.

Appendices

A Proof of Lemma 1

Lemma 1

Let f: ${\mathcal {Z}} \rightarrow {\mathbb {R}}^n$ be any function. For any leakage model $\varphi$: ${\mathcal {Z}} \rightarrow {\mathbb {R}}^n$ there exists a decomposition of f into $f = f_2 \circ f_1$, with $f_1: {\mathcal {Z}} \rightarrow {\mathbb {N}}$, $f_2: {\mathbb {N}} \rightarrow {\mathbb {R}}^n$, satisfying the two following properties:

• 1) $\exists \,f_3: {{\,\textrm{Im}\,}}f_1 \rightarrow {\mathbb {R}}^n \text { such that } f_3 \circ f_1 = \varphi$

• 2) $\forall z \in {\mathcal {Z}}, f_2\vert _{f_1 \big (\varphi ^{-1}(\{\varphi (z)\})\big )}$ is bijective of reciprocal $f_2^{-1}\vert _{f_2\circ f_1\big (\varphi ^{-1}(\{\varphi (z)\}) \big )}$

Proof

Let us create a partition of ${\mathcal {Z}} = \sqcup _{i=1}^n P_i$ where two elements $z_1, z_2 \in {\mathcal {Z}}$ are in the same $P_i$ if and only if:

• $\varphi (z_1) = \varphi (z_2)$

• $f(z_1) = f(z_2)$

Then, one may define $f_1$ as $f_1(z) = i, \forall z \in P_i$. Since $f_1$ only collides for z that already collides through $\varphi$, there exists $f_3$ such that $f_3 \circ f_1 = \varphi$. As f is constant on $P_i$, let us denote by $v_i$ its output on elements of $P_i$. Then $f_2$ can be defined as $f_2(i) = v_i$ so that $f_2 \circ f_1 = f$. Now let us prove 2). Let $z \in {\mathcal {Z}}$ and $a,b \in f_1(\varphi ^{-1}(\{\varphi (z)\}))$ such that $f_2(a) = f_2(b)$. There exists $z_a$ and $z_b$ such that $a = f_1(z_a)$ and $b=f_1(z_b)$ with $\varphi (z_a) = \varphi (z_b) = \varphi (z)$. So:

• $\varphi (z_a) = \varphi (z_b)$

• $f_2(f_1(z_a)) = f_2(f_1(z_b)) \iff f(z_a) = f(z_b)$

which means that $z_a$ and $z_b$ are in the same $P_i$ and thus collides through $f_1$. So $a = b$ which proves that $f_2\vert _{f_1(\varphi ^{-1}(\{\varphi (z)\}))}$ is injective. Then, considering its set of destination being its image, one can say that this function is bijective with reciprocal function: $f_2^{-1}\vert _{f_2\circ f_1(\varphi ^{-1}(\{\varphi (z)\}))}$. $\square$

B Proof of Corollary 1

Definition 1

A function f is said wider- than g if there exists another function h such that: $h \circ f =g$.

Corollary 1

Let L be defined as in (47). Then, for any function ${\bar{h}}$ wider than $\text {HW}$, ${\mathcal {S}}_{\text {HW}} \ge {\mathcal {S}}_{{\bar{h}}}$.

Proof

There exists h such that $h \circ {\bar{h}} = \text {HW}$. So:

$$\begin{aligned} \begin{aligned} {\mathcal {S}}_{\text {HW}}&= {\mathcal {I}} \big (\text {HW}(Z_{k^*}), L \big ) - \max _{k\ne k^*} \big [ {\mathcal {I}} \big (\text {HW}(Z_k), L \big )\big ] \\&= {\mathcal {I}} \big (h \circ {\bar{h}}(Z_{k^*}), L \big ) - \max _{k\ne k^*} \big [ {\mathcal {I}} \big (h \circ {\bar{h}}(Z_k), L \big )\big ] \end{aligned} \end{aligned}$$

(81)

Since removing h in the second term can only increase the information:

$$\begin{aligned} {\mathcal {S}}_{\text {HW}} \ge {\mathcal {I}} \big (h \circ {\bar{h}}(Z_{k^*}), L \big ) - \max _{k\ne k^*} \big [ {\mathcal {I}} \big ({\bar{h}}(Z_k), L \big )\big ] \end{aligned}$$

(82)

By Th.2, $\text {HW}$ maximizes over g the quantity: ${\mathcal {I}} \big (g(Z_{k^*}), L \big )$, so removing h in the first term cannot increase the information:

$$\begin{aligned} \begin{aligned} {\mathcal {S}}_{\text {HW}}&\ge {\mathcal {I}} \big ({\bar{h}}(Z_{k^*}), L \big ) - \max _{k\ne k^*} \big [ {\mathcal {I}} \big ({\bar{h}}(Z_k), L \big )\big ] \\ {\mathcal {S}}_{\text {HW}}&\ge {\mathcal {S}}_{{\bar{h}}} \end{aligned} \end{aligned}$$

(83)

$\square$

C Complementary Material on the Entropy

Lemma 2

Let A and B be a two discrete random variables. Let f: ${\mathcal {A}} \rightarrow {\mathbb {R}}^n$ be any function. Then:

$$\begin{aligned} {\mathcal {H}}\big (f(A) \mid B\big ) \le {\mathcal {H}}(A \mid B) \end{aligned}$$

(84)

Proof

The data processing inequality [5] ensures that applying f to any variables can not increase its mutual information with another variable so:

$$\begin{aligned} \begin{aligned} {\mathcal {I}}\big (f(A),f(A) \mid B\big )&\le {\mathcal {I}}(A,A \mid B) \\ {\mathcal {H}}\big (f(A) \mid B\big )&\le {\mathcal {H}}(A \mid B) \end{aligned} \end{aligned}$$

(85)

$\square$

Lemma 3

Let A and B be a two discrete random variables. Let f: ${\mathcal {A}} \rightarrow {\mathbb {R}}^n$ be any function. Then:

$$\begin{aligned} {\mathcal {H}}\big (A \mid f(B)\big ) \ge {\mathcal {H}}(A \mid B) \end{aligned}$$

(86)

Proof

Again, using the data processing inequality [5]:

$$\begin{aligned} \begin{aligned} {\mathcal {I}}\big (A, f(B)\big )&\le {\mathcal {I}}(A, B) \\ {\mathcal {H}}(A) - {\mathcal {H}}\big (A \mid f(B)\big )&\le {\mathcal {H}}(A) - {\mathcal {H}}(A \mid B)\\ {\mathcal {H}}\big (A \mid f(B)\big )&\ge {\mathcal {H}}(A \mid B) \end{aligned} \end{aligned}$$

(87)

$\square$

Fig. 6

Network architecture for MINE

Fig. 7

Network architecture for the classifiers (Supervised and DDLA)

D Network Architectures

Figures 6 and 7 show the network architectures used for the experiments performed respectfully with MINE and classifiers (supervised and DDLA). For fairness, we tried to keep the two architectures as close as possible. The optimizer used in both cases is Adam [15] with default parameters. The loss function used for the classifiers is the categorical cross-entropy. Note that when using convolutional layers with MINE, the convolutional layers should only be applied to the trace variable and not to $f(Z_k)$ which would not make sense.