Improving the Algorithm 2 in Multidimensional Linear Cryptanalysis

Abstract

In FSE’09 Hermelin et al. introduced the Algorithm 2 of multidimensional linear cryptanalysis. If this algorithm is m-dimensional and reveals l bits of the last round key with N plaintext-ciphertext pairs, then its time complexity is $\mathcal{O}(mN2^l)$. In this paper, we show that by applying the Fast Fourier Transform and Fast Walsh Hadamard Transform to the Algorithm 2 of multidimensional linear cryptanalysis, we can reduce the time complexity of the attack to $\mathcal{O}(N + \lambda2^{m+l})$, where λ is 3(m + l) or 4m + 3l . The resulting attacks are the best known key recovery attacks on 11-round and 12-round Serpent. The data, time, and memory complexity of the previously best known attack on 12-round Serpent are reduced by factor of 2^7.5, 2^11.7, and 2^7.5, respectively. This paper also simulates the experiments of the improved Algorithm 2 in multidimensional linear cryptanalysis on 5-round Serpent.