Skip to main content

Cryptanalysis of SHA-0 and Reduced SHA-1

Abstract

We present new techniques for the cryptanalysis of hash functions. Our contributions are two-fold: both on the search level of the compression function and on the meta-structure. The former led to the neutral bits technique, while the latter led to the multi-block technique. The usefulness of these techniques is demonstrated on SHA-0 and SHA-1, but they are applicable to other hash functions as well. We use these techniques to find a collision of the full SHA-0 which is the first published collision of this function, and very efficient collision attacks on reduced versions of SHA-1.

Introduction

The cryptographic hash function SHA (called SHA-0 in this paper) was designed by the National Security Agency (NSA) and issued by NIST in 1993 as a Federal Information Processing Standard (FIPS-180)  [28]. A revised version, called SHA-1, which introduces an additional rotate operation in the message expansion, was later issued in 1995 as FIPS-180-1 [29]. The revised version is intended to be a more secure replacement, that improves the security provided by the hash function. No details of the weaknesses found in SHA-0 were provided.

SHA-0 and SHA-1, as well as most hash functions currently in use, are based on Merkle–Damgård construction [12, 25]. This construction is proved to be collision resistant if the underlying compression function is collision resistant. Our techniques use differential cryptanalysis [6] in a way that exploits the iterative nature of the Merkle–Damgård construction as well as weaknesses of the compression functions. The differences we use are in both inputs of the compression function, i.e., the initial values and the message blocks, whereas previous techniques were limited to differences in the message blocks. This extension allows us on one hand to use characteristics with much higher probabilities, but on the other hand they do not lead to collisions, but to near-collisions. In order to utilize the high probability characteristics that predict near-collisions, we concatenate them to a longer characteristic that predicts a collision in a technique we call multi-block technique. It should be noted that Wang et al. [36, 37] independently used two-block messages to find the collision of MD5, using a first block that creates a near-collision, and a second block that restarts from this near-collision and ends with a collision.

The compression functions of SHA-0 and SHA-1 (as well as RIPMD-128 [17], RIPEMD-160 [18], HAVAL [42]) are based on the principles of MD4 [31] and MD5 [32]. These compression functions take a 512-bit message block, and output a pseudo-random string (128 bits in MD4 and MD5, and 160 bits in SHA-0 and SHA-1). The basic components of these functions are a message expansion, and an iterative round function that manipulates one of the expanded message words with some intermediate values. The round functions use the IF, XOR, MAJORITY, and addition modulo \(2^{32}\) operations to manipulate the data.

Previous Attacks on MD4/SHA Family of Hash Functions

Shortly after Rivest introduced MD4 [31] in 1990, Merkle [26] showed in an unpublished work that it is possible to find a collision of MD4 reduced to the first 32 (out of the 48) rounds of MD4. Den Boer and Bosselaers [8] followed Merkle’s work and presented an attack on the last 32 rounds of MD4 (rounds 16–47). Few years later, Dobbertin [15] presented a full collision of MD4 with a complexity of \(2^{22}\). His attack is differential, and the measure of difference is subtraction. The two colliding messages that he found differ in a single word by three consecutive bits, where the subtraction difference is 1. The attack is split into two parts: In the first part, a set of equations that describes the evolution of the differences is solved to achieve a predefined difference at some Round \(i\). In the second part, a differential attack that starts with the difference of the first part and leads to a collision is performed. In the first part of the attack an attacker has a full control on the intermediate data, thus he reaches the predefined difference with a negligible complexity. The second part has a probability of about \(2^{-22}\), thus the overall complexity is about \(2^{22}\).

At CRYPTO’91 Rivest introduced MD5 [32] as a strengthened version of MD4. Two years later at EUROCRYPT’93 [9], den Boer and Bosselaers presented an attack that easily finds collisions of the compression function (also known as pseudo-collisions). Their attack finds two colliding inputs, each consists of an initial value and a message block, where the two initial values are different, but the two message blocks are identical. Though their attack showed a substantial weakness of the function, MD5 became the de facto standard of the industry in the following years. In the rump session of EUROCRYPT’96 [16] Dobbertin presented an attack on the compression function of MD5 that finds collisions of two different message blocks with a chosen (non-standard) initial value.

At CRYPTO’98, Chabaud and Joux [10] proposed a theoretical differential attack on the full SHA-0 with a complexity of \(2^{61}\), using a weakness of the expansion algorithm. Their attack is faster than the generic birthday attack, and partially explains the withdrawal of SHA-0 by NIST. This attack uses the XOR operation as a measure of difference, and the characteristic is determined by approximating the non-linear operations by XOR. Similarly to the attacks on MD4, the attack on SHA-0 is split into two parts. In the first 18 rounds, an attacker has almost full control on the conditions that a message should satisfy. From Round 19, the attack is probabilistic, and the chosen characteristic determines the complexity of the attack. Since our attack is based on this attack, we give a detailed description of it in Sect. 3.

For completeness we describe attacks that were published in parallel to or after our contribution in Table 1 and Sect. 9.

Table 1 Up-to-date results on SHA-0 and SHA-1

Our Contribution

This paper presents two cryptanalysis tools: the neutral bits technique (first described in [3]), and the multi–block technique (first described in [4, 7]). The neutral bits technique is used to attack the compression function by using a poor avalanche of the round function, and the multi-block technique uses the iterative mode-of-operation of Merkle–Damgård to enable efficient attacks. The relevance of these techniques to attack SHA-0 and SHA-1 was presented in the rump session of CRYPTO 2004 in the sessions “New results on SHA-0 and SHA-1” by Biham and Chen [4] and “Collisions in SHA-0” by Joux [20].

We define the notion of neutral bits to describe many bits of a pair of messages that do not affect the differences and conditions that a pair should satisfy for a collision to occur. These neutral bits allow an attacker to start the attack from Round 22 or laterFootnote 1 by eliminating the probabilistic behavior of prior rounds.

We analyze the complexity of attacking reduced and extended versions of SHA-0, and show that their complexities are not monotonous in the number of rounds. We then observe that characteristics that predict collisions of reduced and extended versions of SHA-0 may also be used to find small differences in the chaining values of the full 80-round SHA-0. Following this observation, we discuss the usefulness of characteristics that start with a zero or small difference in the chaining value and predict a collision or a small difference of the chaining value. We show that the complexity of finding a pair of message blocks that creates a small difference in the chaining values is much lower than a pair that creates a collision. The reason is that for the former we use characteristics with any differences at the message block, input chaining value, and output chaining value, while for the latter the differences are limited to the message block, and the differences of the input and output chaining values are zero. We then introduce the multi-block technique that links the different types of characteristics to produce a collision of a multi-block pair with much lower complexity than a collision of a single-block pair.

In the following subsection we present the results we achieved using these techniques. We note that although all of our examples are on the SHA family of hash functions, the techniques presented in this paper are general and may be used to cryptanalyze other hash functions. We also note that the neutral bits technique was found applicable for the cryptanalysis of stream ciphers as well [1, 19, 24].

Results

The applicability of the neutral bits and multi-block techniques is demonstrated on SHA-0 and SHA-1. In Table 1 we summarize the main results on attacking these functions in a chronological order. We start from the attack of Chabaud and Joux in 1998 then our results in 2004–2005 and other substantial results until 2013 (a short description of these later results is given in Sect. 9). The first column specifies the attacked hash function, and the next two columns specify the number of rounds of the attacked function and the number of message blocks used in the attack. The attack complexity is given in number of message pairs under the Pair column, and in number of SHA calls under the Time column. These two measures are given since some previous papers use the number of tested pairs as a measure of attack complexity, while others use number of SHA calls. In the remainder of this paper, we use number of SHA calls as a measure of attack complexity. In the Found column a “\(+\)” indicates that a colliding pair is found. The Cite column cites the publication of the result with our results in boldface. The last column specifies the year in which the result was published.

The full collision of SHA-0 is the first published collision of SHA-0, and it uses the neutral bits and multi-block techniques along with additional improvements. This result is an improvement by a factor of \(2^{10}\) to the best previously known attack on SHA-0 [10].

Our attacks and collisions on reduced SHA-1 are the first published attacks on this function [4]. Each attack shows different weaknesses of the algorithm: SHA-1 reduced to 34 rounds is the easiest to attack, thus we were able to find colliding messages with ASCII letters and even meaningful words. The attack on SHA-1 reduced to 36 rounds shows a workaround for a limitation that was identified in [10] as “the consecutive disturbance problem in the IF rounds”. It also demonstrates how the first block in the multi-block technique can be used to replace the standard initial values, in case they are incompatible with the characteristic. The attacks on SHA-1 reduced to 53 and 58 rounds were the highest reduced versions. we could attack with a complexity less than the generic birthday attack. In these attacks we use the technique mentioned above to resolve the consecutive disturbances problem. In parallel to our results, Rijmen and Oswald independently studied reduced versions of SHA-1, and found a characteristic of SHA-1 reduced to 53 rounds [30].

Paper Organization

This paper is organized as follows: Sect. 2 describes SHA-0 and SHA-1, and Sect. 3 reviews the original attack of [10] on SHA-0. Section 4 defines neutral bits, describes how to find and use them, and gives an example of such bits in SHA-0. Section 5 presents analysis of attacks on variants of SHA-0 with different number of rounds. In Sect. 6 we describe the multi-block technique, define near-collisions, pseudo-collisions and near-pseudo collisions, and show how they are used to construct an attack. In Sect. 7 we give a four-block collision of the full SHA-0, along with a further refinement of the prior techniques that make them applicable. Section 8 describes the extension to variants of SHA-1, and various attacks and results on reduced versions of SHA-1. Section 9 describes remarkable advances in the last few years. Finally, Sect. 10 summarizes the paper.

Description of SHA-0, SHA-1, and Notations

Throughout this paper, big-endian is used to convert words into bit strings, i.e., the first bit position is the most significant bit. A 32-bit word \(\{b_{31},\ldots ,b_0\}\) is converted to an integer by \(\sum _{i=0}^{i=31}b_i\cdot 2^i\). A right shift of a 32-bit word by \(r\) positions, where \(r\) zeroes are appended to the leftmost \(32-r\) bits of the shifted word, is denoted by \(\gg r\). Similarly, a left shift is denoted by \(\ll r\). \(X^{\lll r}\) and \(X^{\ggg r}\) denote a left and a right rotation of \(X\) by \(r\) positions, respectively.

SHA-0 and SHA-1 Algorithms

SHA-0 and SHA-1 are Merkle–Damgård iterative hash functions using specially designed compression functions. The Merkle–Damgård construction is outlined in Fig. 1. In this figure the sizes (in number of bits) of each message block \(M_i\) and chaining variable \(h_i\) are denoted by \(b\) and \(m_c\), respectively. The construction and the padding are as follows:

  1. 1.

    A single “1” bit is appended to the message \(M\), followed by a variable number of “0” bits, followed by a 64-bit representation of the message length in bits. The number of zeroes is in the range \(\{0,\ldots , 511\}\) such that the total length of the padded message is a multiple of 512 bits. The padded message is divided into blocks of 512 bits each: \(M_1,\ldots , M_n\).

  2. 2.

    A chaining variable \(h_0\) of five 32-bit words is initialized to

    $$\begin{aligned} h_0= (\mathtt{67452301}, \mathtt{EFCDAB89}, \mathtt{98BADCFE}, \mathtt{10325476}, \mathtt{C3D2E1F0}). \end{aligned}$$
  3. 3.

    For \(k=1\) to \(n\), call the compression function with the current block \(M_k\) (512 bits) and the current chaining variable \(h_{k-1}\) (160 bits). In each iteration the output is a 160-bit chaining variable \(h_k\):

    $$\begin{aligned} h_k =\hbox {C}(M_k,h_{k-1}). \end{aligned}$$
  4. 4.

    \(h_n\) is the output of the hash function.

Fig. 1
figure1

Merkle–Damgård construction

We give here a non-standard (but equivalent) description of the compression function of SHA-0 and SHA-1, which we found more convenient for the purpose of cryptanalysis. A traditional description of SHA-0 and SHA-1 is given in [29]. In the traditional description, five registers \(A_i, B_i, C_i, D_i\), and \(E_i\) are used along with a word \(W_i\) from the expanded message \(\bar{W}\) to compute the values of \(A_{i+1}, B_{i+1}, C_{i+1}, D_{i+1}\), and \(E_{i+1}\), respectively. In our description we use five entries \(A_{i-4}, A_{i-3}, A_{i-2}, A_{i-1}\), and \(A_i\) of a reduced state vector \(A\) (that stores the values of \(A_i, B_i, C_i, D_i\), and \(E_i\) up to a rotation), along with \(W_i\) to compute the value of the next entry \(A_{i+1}\). We define a state of the compression function \(s_i, i\in \{0,\ldots ,80\}\), and a transformation to the five registers representation by:

$$\begin{aligned} s_i=(A_i,A_{i-1},A_{i-2}^{\lll 30},A_{i-3}^{\lll 30},A_{i-4}^{\lll 30})=(A_i,B_i,C_i,D_i,E_i). \end{aligned}$$

An illustration of this description is given in Fig. 2. The definition of the compression function of SHA-0 and SHA-1 and the computations of the entries of \(\bar{W}\) and \(\bar{A}\) are as follows:

  1. 1.

    Divide the message block \(M_k\) into 16 words of 32 bits: \(M_k=W_0,\ldots , W_{15}\).

  2. 2.

    Expand the 16 words to 80 words by the recurrence equation:

    $$\begin{aligned} W_i = (W_{i-3}\oplus W_{i-8}\oplus W_{i-14}\oplus W_{i-16})^{\lll r}, \quad i =16, \ldots , 79, \end{aligned}$$
    (1)

    where \(r\) is 0 for SHA-0 and 1 for SHA-1. This rotate operation is the only difference between SHA-0 and SHA-1. We note that Eq. (1) represents a 16-word linear feedback shift register (LFSR). The LFSR is loaded with the initial value \(W_{0},\ldots , W_{15}\), and it is clocked 64 times to generate \(W_{16},\ldots , W_{79}\).

  3. 3.

    Divide \(h_{k-1}\) to the five first entries of the reduced state vector by

    $$\begin{aligned} h_{k-1}=(A_0,B_0,C_0,D_0,E_0), \end{aligned}$$

    and

    $$\begin{aligned} (A_0,A_{-1},A_{-2},A_{-3},A_{-4}) = (A_0,B_0,C_0^{\ggg 30},D_0^{\ggg 30},E_0^{\ggg 30}). \end{aligned}$$
    (2)
  4. 4.

    For \(i\) = 0–79 compute the reduced state \(A_{i+1}\) by the following round function (Rounds \(0,\ldots ,79\)):

    $$\begin{aligned} A_{i+1} = A_{i}^{\lll 5} + {\hbox {f}}_{i}(A_{i-1},A_{i-2}^{\lll 30},A_{i-3}^{\lll 30}) +A_{i-4}^{\lll 30} + W_{i}+K_{i}, \end{aligned}$$
    (3)

    where the functions \({\hbox {f}}_i\) and the constants \(K_i\) are given in Table 2.

  5. 5.

    The output of the compression function is the sum of the final state \(s_{80}\) and the last chaining variable:

    $$\begin{aligned} h_k = h_{k-1}+s_{80}, \end{aligned}$$
    (4)

    where the addition is word-wise modulo \(2^{32}\).

Fig. 2
figure2

The compression function of SHA-0 and SHA-1

Table 2 Functions and constants

Notations

Unless it is explicitly written, an index in subscript denotes an index of a word in a vector or a round number. An index in superscript denotes a bit index, e.g., \(A_i^j\) is bit \(j\) of the reduced state vector word \(A_i\). The function \({\hbox {f}}_i\) denotes the 32-bit result of \({\hbox {f}}_i(A_{i-1},{A_{i-2}^{\lll 30}},{A_{i-3}^{\lll 30}})\) in Round \(i\). In SHA-0 and SHA-1 \({\hbox {f}}_i\) is a bit-wise function that processes each bit location independently, hence, we may use the notation \({\hbox {f}}_i^j(A_{i-1}^j,A_{i-2}^{j-30},A_{i-3}^{j-30})\) to describe the \(j\)’th bit of its output. We sometimes use the explicit name of the function instead of \({\hbox {f}}_i\), i.e., instead of \({\hbox {f}}_i\) we may write \(\hbox {IF}_i, \hbox {XOR}_i\), or \(\hbox {MAJ}_i\).

We use the notation \(\hbox {CARRY}_i^j\) to describe the value of the (single) carry bit from bit \(j-1\) (and prior bits) to bit \(j\) in the computation of \(A_i\) by Eq. (3), i.e., the XOR difference of the output and all inputs of the addition operations in a SHA-round, which is

$$\begin{aligned} \hbox {CARRY}_i=A_i\oplus A_{i-1}^{\lll 5}\oplus {\hbox {f}}_{i-1}(A_{i-2}, A_{i-3}^{\lll 30},A_{i-4}^{\lll 30})\oplus A_{i-5}^{\lll 30}\oplus W_{i-1}\oplus K_{i-1}. \end{aligned}$$
(5)

We use the standard notations of differential cryptanalysis [5] to specify the values and differences of two messages and their parameters, e.g., \(M, M^*\) and \(M'\) describe the values and the difference of two messages, respectively. In addition, a bit is marked in boldface, e.g., \(\mathbf{W}_i^1\), to indicate that the value of this bit is different in both messages, e.g., that \(W_i^{\prime 1}=W_i^1\oplus W_i^{*1}=1\). We call such a bit an active bit.

The Basic Attack on SHA-0

In [10], Chabaud and Joux present a differential attack on SHA-0 that uses the XOR operation as a measure of difference. Their attack is aimed at finding a collision of a pair of single-block messages that are hashed with the standard initial value.

The basic idea of the attack is to generate a pair of messages with specific patterns of XOR differences in the first 16 words. Each pattern of differences that starts at Round \(i\) is constructed such that with a non-negligible probability it creates a difference of a single bit between the reduced states \(A_{i+1}\) and \(A_{i+1}^*\). After five rounds, at Round \(i+6\), the reduced states \(A_{i+1}\) and \(A_{i+1}^*\) do not affect the computations of succeeding reduced states, thus the states of the two messages are not affected by the difference that the pattern created. These patterns of differences are duplicated by the LFSR of Eq. (1) to the expanded messages. The result is a pair of expanded messages with patterns of differences such that each pattern creates with some probability a difference of a single bit between the reduced states. If each pattern in the expanded messages succeeds in creating a difference of a single bit, then the differences in the reduced states follow the patterns of differences in the expanded messages. Hence, if the difference that the last pattern creates is in \(A_{75}^{\prime }\) (or before), then the last five reduced states are equal and a collision occurs.

The basic pattern of differences between the two messages that creates such equal states consists of six XOR differences. The first difference creates a minimal difference of one bit in the reduced state vector, and the other five differences avoid the propagation of this difference to the next words of the reduced state vector. We call such a pattern of differences, a local collision sequence. The next subsection describes a local collision sequence in detail.

A Local Collision Sequence

In the attacks on SHA-0 we concentrate only on local collision sequences that start at bit 1 of any of the 32-bit message words. Such a pattern that creates a local collision starts with a single-bit difference at \(\mathbf{W}_i^1\), which we call a disturbance (or perturbation). With a probability \(1/2\) this disturbance creates a difference in \(\mathbf{A}_{i+1}^1\) while leaving the carry to \(A_{i+1}^2\) unchanged. In order to avoid the propagation of the difference from \({\mathbf{A}_{\mathbf{i+1}}^\mathbf{1}}\) to the next rounds, five additional differences, called corrections, are inserted in the next five rounds. In Table 3 we demonstrate a disturbance and five corrections which form a local collision sequence. This table gives bit-wise computations of the round function for the bits that are affected by the active bits of the local collision sequence. The local collision sequence we show is: \(({\mathbf{W}_\mathbf{i}^\mathbf{1}},{\mathbf{W}_{\mathbf{i+1}}^\mathbf{6}},{\mathbf{W}_{\mathbf{i+2}}^\mathbf{1}},{\mathbf{W}_{\mathbf{i+3}}^{\mathbf{31}}},{\mathbf{W}_{\mathbf{i+4}}^{\mathbf{31}}}, {\mathbf{W}_{\mathbf{i+5}}^{\mathbf{31}}})\), and the desired resulting difference in the reduced state vector is \({\mathbf{A}_{\mathbf{i+1}}^\mathbf{1}}\) only, i.e., \({\mathbf{W}_\mathbf{i}^\mathbf{1}}\) activates \({\mathbf{A}_{\mathbf{i+1}}^\mathbf{1}}\), and \({\mathbf{W}_{\mathbf{i+1}}^\mathbf{6}}, {\mathbf{W}_{\mathbf{i+2}}^\mathbf{1}}, {\mathbf{W}_{\mathbf{i+3}}^{\mathbf{31}}}, {\mathbf{W}_{\mathbf{i+4}}^{\mathbf{31}}}, {\mathbf{W}_{\mathbf{i+5}}^{\mathbf{31}}}\) result in \(A_{i+2}^6, A_{i+3}^1, A_{i+4}^{31}, A_{i+5}^{31}\), and \(A_{i+6}^{31}\) inactive. In terms of intermediate states differences, the state differences \(s_{i+1}^{\prime },\ldots , s_{i+5}^{\prime }\) show differences due to \({\mathbf{A}_{\mathbf{i+1}}^\mathbf{1}}\), while \(s_{i+6}^{\prime }\) is not affected, i.e.,

$$\begin{aligned} s_{i+1}^{\prime }&= \mathtt{(00000002,0,0,0,0)}\\ s_{i+2}^{\prime }&= \mathtt{(0,00000002,0,0,0)}\\ s_{i+3}^{\prime }&= \mathtt{(0,0,80000000,0,0)}\\ s_{i+4}^{\prime }&= \mathtt{(0,0,0,80000000,0)}\\ s_{i+5}^{\prime }&= \mathtt{(0,0,0,0,80000000)}\\ s_{i+6}^{\prime }&= \mathtt{(0,0,0,0,0)}. \end{aligned}$$

During a desired computation of a pair of messages that has a local collision sequence difference (as in Table 3), only the bits marked in boldface may be active. Thus, we require that the carries in the computations of \(A_{i+1}, A_{i+2}\), and \(A_{i+3}\), (at rounds \(i, i+1\), and \(i+2\)) remain unchanged (inactive). We note that since the addition is modulo \(2^{32}\), there is no carry to bits next to bit 31. Thus, even though a disturbance may be located at any bit index, we prefer bit 1, since the probabilistic behavior of the carry after the rotation by 30 bits is eliminated at rounds \(i+3, i+4\), and \(i+5\).

Table 3 A description of a pattern of differences that creates a local collision

An illustration of a local collision sequence is given in Fig. 3. In this figure the disturbance \({\mathbf{W}_\mathbf{i}^\mathbf{1}}\) and the difference of the reduced state \({\mathbf{A}_{\mathbf{i+1}}^\mathbf{1}}\) are marked in dark gray. The entries in light gray are the corrections, and entries with no background have no difference. The middle part of this figure shows the computation of the reduced state difference \(A_{i+1}^{\prime }\) with the inputs of the disturbance \({\mathbf{W}_\mathbf{i}^{\varvec{\prime }}}=\mathtt{00000002}\) and the state differences \(s_i^{\prime }=\mathtt{(0,0,0,0,0)}\). The desired result is \(A_{i+1}^{\prime }=\mathtt{00000002}\), i.e., \(s_{i+1}^{\prime }=\mathtt{(00000002,0,0,0,0)}\). In Round \(i+1\) the middle part of the figure is advanced by one entry such that its input differences are: \({\mathbf{W}_{\mathbf{i+1}}^{\varvec{\prime }}}=\mathtt{40}\) and \(s_{i+1}^{\prime }=\mathtt{(00000002,0,0,0,0)}\). The desired output difference of this round is \(A_{i+2}^{\prime }=0\). In each consequent round the middle part in the figure is advanced by one entry, and the reduced state difference is computed. After five such computations, the state difference becomes \(s_{i+6}^{\prime }=\mathtt{(0,0,0,0,0)}\), which forms a local collision.

Fig. 3
figure3

A local collision sequence

A local collision sequence is a probabilistic process that depends on the function \({\hbox {f}}_i\) and the carry. We now analyze each row of Table 3, and show how the desired differences are achieved, and what the probabilities are at each round.

A disturbance \({\mathbf{W}_\mathbf{i}^\mathbf{1}}\) turns \({\mathbf{A}_{\mathbf{i+1}}^\mathbf{1}}\) to be an active bit with probability 1. The carry from bit 1 remains unchanged if \({\mathbf{W}_\mathbf{i}^\mathbf{1}}={\mathbf{A}_{\mathbf{i+1}}^\mathbf{1}}\), thus we assume a probability \(1/2\) for this round. In the next five rounds we apply corrections that compensate the active bit \({\mathbf{A}_{\mathbf{i+1}}}\), so that \(A_{i+2},\ldots ,A_{i+6}\) remain inactive.

In the first correction at Round \(i+1\), the active bits \({\mathbf{W}_{\mathbf{i+1}}^\mathbf{6}}\) and \({\mathbf{A}_{\mathbf{i+1}}^\mathbf{1}}\) cancel each other if \({\mathbf{W}_{\mathbf{i+1}}^\mathbf{6}}={\overline{\mathbf{A}_{\mathbf{i+1}}^\mathbf{1}}}=\overline{{\mathbf{W}_\mathbf{i}^\mathbf{1}}}\). Thus, by setting \({\mathbf{W}_{\mathbf{i}}^\mathbf{1}=\overline{{\mathbf{W}_{\mathbf{i+1}}^\mathbf{6}}}}\), both \(A_{i+2}^6\) and the carry remain inactive. In the next three rounds the active bit \({{\mathbf{A}_{\mathbf{i+1}}^\mathbf{1}}}\) is an input to \({\hbox {f}}_i\). For simplicity and clarity we first consider \({\hbox {f}}_i\) being XOR. We will later discuss the differences for the IF and MAJORITY operations.

At Round \(i+2\) we need that \(A_{i+3}^1\) and the carry from this bit remain inactive. The result of \(\mathbf{XOR}_{\mathbf{i+2}}^\mathbf{1}({{\mathbf{A}_{\mathbf{i+1}}^\mathbf{1}}},A_i^3,A_{i-1}^3)\) is always active when \({{\mathbf{A}_{\mathbf{i+1}}^\mathbf{1}}}\) is active in the input of the XOR function, thus the correction \({\mathbf{W}_{\mathbf{i+2}}^{\mathbf{1}}}\) maintain \(A_{i+3}^1\) inactive with probability 1. The carry from bit 1 remains inactive if \(\mathbf{W}_{\mathbf{i+2}}^\mathbf{1}=\overline{{\mathbf{XOR}_{\mathbf{i+2}}^\mathbf{1}}}\). By writing the explicit terms of \({\mathbf{XOR}_{\mathbf{i+2}}^\mathbf{1}}\), and substituting \({\overline{\mathbf{A}_{\mathbf{i+1}}^\mathbf{1}}}={\mathbf{W}_{\mathbf{i+1}}^\mathbf{6}}\), the carry remains inactive if \({\mathbf{W}_{\mathbf{i+2}}^\mathbf{1}}={\mathbf{W}_{\mathbf{i+1}}^\mathbf{6}}\oplus A_i^3 \oplus A_{i-1}^3\), which occurs with probability \(1/2\). We note that the condition of Round \(i+2\) can be tested at round \(i-1\) (when \(A_i^3\) and \(A_{i-1}^3\) are known). Thus, an attacker knows if he achieves the desired result three rounds in advance.

In the computations of \(A_{i+4}^{31}, A_{i+5}^{31}\), and \(A_{i+6}^{31}\) we should not worry about changes of the carry as it is ignored anyway. Furthermore, \({\mathbf{XOR}_{\mathbf{i+3}}^{\mathbf{31}}}(A_{i+2}^{31}, {{\mathbf{A}_{\mathbf{i+1}}^\mathbf{1}}}, A_i^1\)) and \({\mathbf{XOR}_{\mathbf{i+4}}^{\mathbf{31}}}(A_{i+3}^{31}, A_{i+2}^1, {{\mathbf{A}_{\mathbf{i+1}}^\mathbf{1}}}\)) are always active when only \({{\mathbf{A}_{\mathbf{i+1}}^\mathbf{1}}}\) is active in the input. Thus, in each of these three rounds the active bits \(\mathbf{W}_{i+3}^{31}, \mathbf{W}_{i+4}^{31}, \mathbf{W}_{i+5}^{31}\) compensate the active bit \({\mathbf{A}_{\mathbf{i+1}}^\mathbf{1}}\) with probability 1.

The differences between the analysis of the XOR, IF, and MAJORITY operations are only in rounds \(i+2, i+3\), and \(i+4\) where \({\mathbf{A}_{\mathbf{i+1}}^\mathbf{1}}\) is one of the inputs to the function. In these rounds we require \({\mathbf{f}_{\mathbf{i+2}}^\mathbf{1}}({\mathbf{A}_{\mathbf{i+1}}^\mathbf{1}}, A_i^3, A_{i-1}^3), {\mathbf{f}_{\mathbf{i+3}}^{\mathbf{31}}(A_{i+2}^{31}, {\mathbf{A}_{\mathbf{i+1}}^\mathbf{1}}, \mathbf{A}_\mathbf{i}^\mathbf{1})}\), and \({\mathbf{f}_{\mathbf{i+4}}^{\mathbf{31}}}(A_{i+3}^{31},A_{i+2}^{1},{\mathbf{A}_{\mathbf{i+1}}^\mathbf{1}})\) to be active, such that the corrections \({\mathbf{W}_{\mathbf{i+2}}^\mathbf{1}}, {\mathbf{W}_{\mathbf{i+3}}^{\mathbf{31}}}\), and \({\mathbf{W}_{\mathbf{i+4}}^{\mathbf{31}}}\) cancel each of these active bits, respectively. Unlike the XOR operation whose output is always active, IF and MAJORITY create an active output with probability \(1/2\) for a random selection of their three inputs. In addition, at Round \(i+2\) we require

$$\begin{aligned} {\mathbf{f}_{\mathbf{i+2}}^\mathbf{1}}({{\mathbf{A}_{\mathbf{i+1}}^\mathbf{1}}},A_i^3,A_{i-1}^3)= \overline{{\mathbf{W}_{\mathbf{i+2}}^\mathbf{1}}} \end{aligned}$$
(6)

in order for the carry to remain unchanged, i.e., the output of the function is active, and it is opposite to the correction.Footnote 2 When the function is the IF operation and its output is active, Eq. (6) is satisfied with probability \(1/2\). On the other hand, when \({\hbox {f}}_i\) is MAJORITY, and it is active, \({\mathbf{MAJ}_{\mathbf{i+2}}^\mathbf{1}}({{\mathbf{A}_{\mathbf{i+1}}^\mathbf{1}}},A_i^3,A_{i-1}^3) = {\mathbf{A}_{\mathbf{i+1}}^\mathbf{1}}\), and by setting \({\mathbf{W}_{\mathbf{i+2}}^\mathbf{1}}=\overline{{\mathbf{W}_\mathbf{i}^\mathbf{1}}}\), the probability is \(1\). Thus, by adding a condition on the message bits which we control, we gain a factor of \(1/2\) in the probability.

A summary of conditions of a local collision sequence and their probabilities with the XOR operation are given in Table 4. The conditions and probabilities for the IF and MAJORITY are given in Tables 5 and 6. In these tables, the first column shows the round number, where the application of the disturbance is at round \(i\). The second column shows the conditions on the expanded messages and reduced states bits. The third column is the probability that the conditions hold. We note that according to the data in these tables an attacker may know 2–3 rounds in advance if the required behavior of a local collision sequence is achieved, e.g., in Round \(i\) he can already test whether \(A_i^3=\overline{A_{i-1}^3}\) to know if he gains the desired behavior of Round \(i+2\).

Table 4 Local collision conditions for rounds \(i,\ldots , i+5\) with the XOR operation at rounds \(i+2, i+3, i+4\)
Table 5 Conditions for rounds \(i+2, i+3, i+4\) with the IF operation
Table 6 Conditions for rounds \(i+2, i+3, i+4\) with the MAJORITY operation

Two Consecutive Disturbances

The computations of local collision sequences with two consecutive disturbances at rounds \(i\) and \(i+1\) are similar to the computations in Table 3, except for Round \(i+4\) where \(A_{i+5}\) is given by:

$$\begin{aligned} A_{i+5}^{31}= W_{i+4}^{31}\oplus A_{i+4}^{26}\oplus {{\hbox {f}}_{i+4}^{31}}(A_{i+3}^{31},\mathbf{A_{i+2}^1},\mathbf{A_{i+1}^1})\oplus A_i^1\oplus K_{i+4}^{31}\oplus \hbox {CARRY}_{i+5}^{31} \end{aligned}$$
(7)

(see Fig. 4). At round \(i+4\), the two corrections for \({\mathbf{A}_{\mathbf{i+1}}^\mathbf{1}}\) and \({\mathbf{A}_{\mathbf{i+2}}^\mathbf{1}}\) cancel each other, thus \(W_{i+4}^{31}\) is inactive. On the other hand, when \({\hbox {f}}_i\) is the IF operation,

Fig. 4
figure4

Two consecutive disturbances in the IF rounds

\({\mathbf{IF}}_{i+4}^{31}(A_{i+3}^{31}, {\mathbf{A}_{\mathbf{i+2}}^\mathbf{1}}, {\mathbf{A}_{\mathbf{i+1}}^\mathbf{1}})\) is always active with \({\mathbf{A}_{\mathbf{i+2}}^\mathbf{1}}\) and \({\mathbf{A}_{\mathbf{i+1}}^\mathbf{1}}\) active in the input. Thus, Eq. (7) is never satisfied when the IF operation is used. Hence, two consecutive disturbances are not allowed in the range \(i=0,\ldots ,i=16\). This situation is known as the consecutive disturbances problem in the IF rounds.Footnote 3

A similar analysis when \({\hbox {f}}_i\) is the MAJORITY operation shows that:

$$\begin{aligned} {\hbox {MAJ}_{i+4}^{31}}(A_{i+3}^{31}, A_{i+2}^1, A_{i+1}^1)={\hbox {MAJ}_{i+4}^{31}}(A_{i+3}^{31},\overline{A_{i+2}^1}, \overline{A_{i+1}^1})~ \hbox {if}~A_{i+1}^1=\overline{A_{i+2}^1}. \end{aligned}$$

Thus, by keeping the condition

$$\begin{aligned} W_i^1=\overline{W_{i+1}^1}, \end{aligned}$$

when two consecutive disturbances are in the range \(i=37,\ldots ,i=56\), the probability of the correction at round \(i+4\) is 1.

A Characteristic and a Disturbance Vector

The attack on SHA-0 is differential, and the differences we use have the form of local collision sequences. Each local collision sequence starts with a disturbance and leads to a local collision with some probability. It is clear that a pair of message blocks that has such differences collides if all the local collisions occur as expected. We now show how to construct a pair of message blocks that has such differences.

A pair of expanded message blocks is generated by the LFSR of Eq. (1), thus their XOR difference is also generated by this LFSR. We use this property to construct the XOR difference of the pair such that it has the form of local collision sequences, in two steps. Firstly, we construct the differences that represent the disturbances, and describe these differences by a disturbance vector \(\mathcal{D}\). Secondly, we manipulate the disturbance vector \(\mathcal{D}\) by the linear XOR and shift operations to construct the difference of the pair (in the form of local collision sequences). We describe this difference by a disturbances and corrections vector \(\Delta \). We note that since the operations we use for constructing these two vectors (the disturbance vector \(\mathcal{D}\), and the disturbance and corrections vector \(\Delta \)) are linear, both of them can be generated by the LFSR of Eq. (1).

Definition 1

A disturbance vector \(\mathcal{D}\) is a vector of 85 words \(\mathcal{D}_{-5},\ldots ,\mathcal{D}_{79}\), that indicates the locations of the disturbances. A bit in this vector is set to one if there is a disturbance in this location, and is set to zero otherwise.

In the following list we describe the constraints on the disturbance vector that are required for finding collisions of SHA-0.

  1. 1.

    Disturbances are applied only to bit 1 (see the discussion in Sect. 3.1).

  2. 2.

    The pair of messages starts with the standard initial value, i.e., the differences of the initial state are \(s'_{0}\!=\!\mathtt{(0,0,0,0,0)}\), and the corresponding words of the disturbance vector are \(\mathcal{D}_{-5}=0,\ldots ,\mathcal{D}_{-1}\!=\!0\).

  3. 3.

    The differences of the final state are zero, i.e., \(s_{80}^{\prime }=\mathtt{(0,0,0,0,0)}\), and the corresponding words of the disturbance vector are \(\mathcal{D}_{75}=0,\ldots , \mathcal{D}_{79}=0\).

  4. 4.

    No consecutive disturbances are allowed in \(\mathcal{D}_{0},\ldots , \mathcal{D}_{16}\) (as mentioned earlier, this limitation was resolved by Wang et al. [38]).

Under these constraints we construct the disturbance vector using the LFSR of Eq. (1).

Given a disturbance vector, we define the difference of the analyzed expanded messages by a disturbances and corrections vector \(\Delta \). A disturbances and corrections vector \(\Delta \) is generated from a disturbance vector \(\mathcal{D}\) in the form of local collision sequences as follows: Let \(\hbox {SR}^l(\mathcal{D})\) be the vector of 85 words obtained by prepending \(l\) zero words to the first \(85-l\) words of \(\mathcal{D}\) (i.e., a non-cyclic shift operation of the words). Then, for all the disturbances in \(\mathcal{D}\), corrections are made by \(\hbox {SR}^1(\mathcal{D})^{\lll 5}\), \(\hbox {SR}^2(\mathcal{D})\), \(\mathrm{SR}^3(\mathcal{D})^{\lll 30}\), \(\hbox {SR}^4(\mathcal{D})^{\lll 30}\), and \(\hbox {SR}^5(\mathcal{D})^{\lll 30}\), where \(^{\lll }\) denotes a cyclic rotate of each word of the vector separately. The linear combination of local collision sequences that start in each disturbance of \(\mathcal D\) is:

$$\begin{aligned} \mathcal{D} \oplus \left( \hbox {SR}^1(\mathcal{D})^{\lll 5}\right) \oplus \hbox {SR}^2(\mathcal{D}) \oplus \left( \left( \hbox {SR}^3(\mathcal{D}) \oplus \hbox {SR}^4(\mathcal{D}) \oplus \hbox {SR}^5(\mathcal{D})\right) ^{\lll 30}\right) . \end{aligned}$$
(8)

\({\Delta }\) is the 80 entries \(0,\ldots ,79\), of the vector constructed by (8), and the difference \(W'\) of the two expanded message blocks is:

$$\begin{aligned} W^{\prime } = (W_0^{\prime },\ldots ,W_{79}^{\prime })=({\Delta }_0,\ldots ,{\Delta }_{79})=\Delta . \end{aligned}$$

A Construction of Characteristics from Disturbance Vectors

We are now ready to define a characteristic for the attack, and then we show how disturbance vectors are used to construct characteristics. The following definition is for a characteristic which is suitable to attack a pair of single-block messages (see Fig. 5). It is extended later to attack longer messages.

Fig. 5
figure5

A characteristic of a single-block attack

Definition 2

A characteristic of SHA-0 is a tuple \(\Omega =({\Omega }_P,{\Omega }_M,{\Omega }_W,{\Omega }_S,{\Omega }_T)\). \(\Omega _P\) is a 160-bit string that represents the differences of the initial values, i.e., \({\Omega }_P=h_{k-1}\oplus h_{k-1}^*\). \(\Omega _T\) is a 160-bit string that represents the differences of the hash results, i.e., \({\Omega }_T=h_{k}\oplus h_{k}^*\). \({\Omega }_P\) and \({\Omega }_T\) are also called chaining differences. In general we refer to chaining differences as tuples of five words of 32 bits. \(\Omega _M=M\oplus M^*\) is a 512-bit string of the messages difference, and \(\Omega _{W} = (W_0^{\prime },\ldots ,W_{79}^{\prime })\) is the differences of the words of the expanded messages. Since the 512 bits of \({\Omega }_M\) are the first 16 words of \(W'\), we generally refer to them by the 16 words: \({\Omega }_M=(W_0',\ldots ,W_{15}'\)). The differences of the state vector \({\Omega }_S = (s_0^{\prime },\ldots ,s_{80}^{\prime })\) start from a given difference of the initial values (\(s_0^{\prime }={\Omega }_P\)), and continue with the state differences from the first computed state (\(s_{1}^{\prime }\)) to the final state (\(s_{80}^{\prime }\)). We note that \({\Omega }_S\) is derived from the differences of the reduced state vector \(\Omega _A=(A_{-4}^{\prime },\ldots ,A_{80}^{\prime }) \) by:

$$\begin{aligned} s_i^{\prime }=(A_i^{\prime },A_{i-1}^{\prime },A_{i-2}^{\prime \lll 30},A_{i-3}^{\prime \lll 30},A_{i-4}^{\prime \lll 30}),~ i\in \{0,\ldots ,80\}. \end{aligned}$$

We are now ready to describe how we use a disturbance vector to construct a characteristic. A characteristic that predicts a collision with a pair of single-block messages starts with the standard initial values and predicts equal hash results, thus \({\Omega }_P = \mathtt{(0,0,0,0,0)}\) and \({\Omega }_T = \mathtt{(0,0,0,0,0)}\). Using a disturbance vector \(\mathcal{D}\) we generate the disturbances and corrections vector \({\Delta }\) and define the differences of the messages and expanded messages by \({\Omega }_M =({\Delta }_0,\ldots ,{\Delta }_{15})\) and \({\Omega }_W=({\Delta }_0,\ldots ,{\Delta }_{79})\). The predicted differences of the reduced state vector are then: \({\Omega }_A=(A_{-4}^{\prime },\ldots ,A_{80}^{\prime })=(\mathcal{D}_{-5},\ldots ,\mathcal{D}_{79})=\hbox {SR}^1(\mathcal{D})\), and the predicted differences of the state vector \({\Omega }_S=(s_0^{\prime },\ldots ,s_{80}^{\prime })\) are: \(s_i^{\prime }=(\mathcal{D}_{i-1},\mathcal{D}_{i-2},\mathcal{D}_{i-3}^{\lll 30},\mathcal{D}_{i-4}^{\lll 30},\mathcal{D}_{i-5}^{\lll 30}), i\in \{0,\ldots ,80\}\). The probability of the characteristic depends on the number of disturbances, as discussed in Sect. 3.3.

Given a characteristic we measure how close a pair of messages is to a collision by the definition of conformance.

Definition 3

Given a characteristic \(\Omega \), a pair of messages \(M\) and \(M^*\) conforms to \(R\) rounds if

$$\begin{aligned} s_0^{\prime }={\Omega }_P~\hbox {and}~\forall _{i=1,\ldots ,R}~A_{i}^{\prime }=\mathcal{D}_{i-1}. \end{aligned}$$

The pair \(M\) and \(M^*\) is a right pair if it conforms to all rounds (\(R=80\) in SHA-0 and SHA-1).

We note that if a pair of messages conforms to the characteristic up to the last round of the compression function then a collision occurs. In our discussion on SHA-1 we relax the requirements for conformance and give a weaker definition.

Compact Representations of a Disturbance Vector \(\mathcal{D}\), Chaining Differences \({\Omega }_P\), and \({\Omega }_T\), and State Difference \(s_i^{\prime }\)

In SHA-0 disturbances are applied only to bit 1, thus a disturbance vector may be represented by the values of bit 1 in each word from \(-5\) to \(79\). Furthermore, to identify a disturbance vector it suffices to specify the first 16 bits from which the LFSR may generate the complete vector. Thus, for SHA-0 we represent a disturbance vector, which is a vector of 85 words, by a single 16-bit word. In this representation each bit from the LSB to the MSB corresponds to a disturbance in \(\mathcal{D}_{-5},\ldots ,\mathcal{D}_{10}\), respectively.

Similarly, we use a 5-bit word to represent the prediction of the chaining differences \({\Omega }_P\), and \({\Omega }_T\), and state difference \(s_i^{\prime }\). A difference in a word is represented by “1” and no difference by “0”, e.g., \({\Omega }_T={ {{\varvec{10010}}}}\) corresponds to differences in bit 1 of \(A_{80}^{\prime }\) and \(A_{77}^{\prime }\), while \(A_{79}^{\prime }, A_{78}^{\prime }\), and \(A_{76}^{\prime }\) have no differences.

Selecting a Disturbance Vector for the Attack

A conformance of a pair of messages to 80 rounds for a given characteristic is a sufficient condition for a collision. If we assume that the probability of each local collision sequence is equal, then the fewer disturbances, the higher the probability to find a collision.

We now show that the requirement for the smallest number of disturbances can be relaxed and optimized for the attack such that disturbances are not counted from the first round. Note that up to Round 15 an attacker has full control over the results of the round function, thus we assume that the corrections of each condition up to Round 15 are made with negligible complexity. Moreover, following the analysis in Sect. 3.1, we see that an attacker has the ability to look ahead about three rounds. Thus, the conformance conditions up to Round 18 can be tested along with the computation of Round 15, and can be corrected with negligible complexity. We conclude that the disturbance vector with the least number of disturbances between Round 19 and the last round, has the highest probability to generate a collision.

We can now use the following procedure for the selection of the disturbance vector with the highest probability. A 16-word register is initialized such that the first five words are set to zero, and in the other 11 words bit 1 is set either to zero or one. We expand it to 85 words by the LFSR of Eq. (1). From the \(2^{11}\) possible outcomes, \(2^6\) satisfy \(\mathcal{D}_{75}=0,\ldots , \mathcal{D}_{78}=0\) and \(\mathcal{D}_{79}=0\), and have no consecutive disturbances in rounds \(0,\ldots ,16\). From these \(2^6\) outcomes we select an outcome (an 85-word vector) which has the least Hamming weight from round 19 to 79.

Using the compact representation of a disturbance vector, the best disturbance vector for the attack is represented by the 16-bit string \(\mathtt{0880}\). The 85-bit string that describes the disturbance vector with the five leading zeros is shown in Table 7. In this table the entries of the disturbance vector from which the initial and final state differences are derived are marked with a boldface, and the first 16 bits that represent the disturbance vector are in italic.

Table 7 The disturbance vector of SHA-0 attack

The Chaining Differences Transition Graph

A graphical description of the selection of a characteristic for an attack is given in Fig. 6. In this figure we describe characteristics of an attack as edges in a directed graph, in which the vertices are the chaining differences. Each characteristic is derived from a disturbance vector \(\mathcal{D}_i\), and has a probability \(p_i\) to achieve the differences given by its output edge. The vertex 00000 which is the initial and final chaining difference is drawn twice for clarity. The five zeros in it denote that there is no difference at that location. Among the different disturbance vectors that predict this chaining difference transition, the one that has the highest probability is selected for the attack. Following these notations, this simple graph shows the path from no difference in the initial values \(\Omega _{P}^1={{\varvec{00000}}}\) to no difference in the final hash result \(\Omega _{T}^1={{\varvec{00000}}}\), using a pair of single-block messages with \(\Omega _{M}^1\) difference. We call this graph a chaining differences transition graph. The edge that we choose for the attack is called a chaining difference path. This edge is described by the 16-bit compact representation of the disturbance vector. In the case of a single-block attack the chaining difference transition path with the highest probability is represented by \(\mathcal{D}=\mathtt{0880}\).

Fig. 6
figure6

The chaining differences transition graph of a single-block attack on SHA-0

A Characteristic of a Single-Block Attack of SHA-0

Given the disturbance vector \(\mathcal{D}=\mathtt{0880}\) we construct the characteristic of the attack in accordance with the procedure in Sect. 3.2.1. The characteristic we obtain is given in Table 8. At the top of this table we give the initial chaining difference \({\Omega }_P\), and at the bottom the final chaining difference \({\Omega }_T\). The first column (R.) is the round index, and in each row \(i\) we list the disturbance vector \(\mathcal{D}_i\), the expanded message difference \(W_i^{\prime }\), the expected difference \({A_i^{\prime }}\) of the reduced state vector, and the probability to obtain \({A_i^{\prime }}\). We note that the column \({\Omega }_A\) equals the disturbance vector shifted by one, i.e., \({\Omega }_A=\hbox {SR}^1(\mathcal{D})\), the initial difference is given by \({\Omega }_P=(A_{0}^{\prime },A_{-1}^{\prime },A_{-2}^{\prime \lll 30},A_{-3}^{\prime \lll 30}, A_{-4}^{\prime \lll 30})={{{\varvec{00000}}}}\), and the final difference is given by \({\Omega }_T=(A_{80}^{\prime },A_{79}^{\prime },A_{78}^{\prime \lll 30},A_{77}^{\prime \lll 30}, A_{76}^{\prime \lll 30})={{{\varvec{00000}}}}\).

Table 8 A characteristic of a single-block attack on SHA-0

Constructing a Pair for the Attack

The conditions that a pair and its associated state vectors should satisfy are described in Tables 45, and 6. These conditions are sorted into two types: message-bit conditions and state-bit conditions. Message-bit conditions depend only on the expanded message bits, while state-bit conditions depend also on the reduced state vector bits. A pair \(M_1,M_1^*\) is selected such that the message-bit conditions are satisfied for all of the 80 rounds. The message-bit conditions are summarized as follows:

  1. 1.

    For any disturbance in round \(i\in \{0,\ldots ,74\}\)

    $$\begin{aligned} \mathbf{W}_\mathbf{i}^\mathbf{1}=\overline{\mathbf{W}_{\mathbf{i+1}}^\mathbf{6}}. \end{aligned}$$
  2. 2.

    For any disturbance in rounds \(i\in \{38,\ldots ,57\}\)

    $$\begin{aligned} \mathbf{W}_\mathbf{i}^\mathbf{1}=\overline{\mathbf{W}_{\mathbf{i+2}}^\mathbf{1}}. \end{aligned}$$
  3. 3.

    For any two consecutive disturbances in rounds \(i\) and \(i+1\) where \({i\in \{36,\ldots ,55\}}\)

    $$\begin{aligned} \mathbf{W}_\mathbf{i}^\mathbf{1}=\overline{\mathbf{W}_{\mathbf{i+1}}^\mathbf{1}}. \end{aligned}$$
  4. 4.

    For any two disturbances distant by two rounds, in rounds \(i\) and \(i+2\) where \(i\in \{36,\ldots ,55\}\)

    $$\begin{aligned} \mathbf{W}_\mathbf{i}^\mathbf{1}=\overline{\mathbf{W}_{\mathbf{i+4}}^\mathbf{1}}. \end{aligned}$$

We note that all the conditions are on bit 1 and 6, and that there might be some additional conditions for specific disturbance vectors.

The conditions are written as equations of the variables \({\mathbf{W}_\mathbf{i}^\mathbf{1}}\)’s, and \({\mathbf{W}_\mathbf{i}^\mathbf{6}}\)’s, \(i\in \{0,\ldots ,79\}\). The expansion equation (1) is then used to write the \({\mathbf{W}_\mathbf{i}^\mathbf{1}}\)’s and \({\mathbf{W}_\mathbf{i}^\mathbf{6}}\)’s \(i\in \{16,\ldots ,79\}\) as functions of \({\mathbf{W}_\mathbf{i}^\mathbf{1}}\)’s and \({\mathbf{W}_\mathbf{i}^\mathbf{6}}\)’s \(i\in \{0,\ldots ,15\}\). The obtained set of equations are then solved by a Gauss elimination (or any other technique) for all the \({\mathbf{W}_\mathbf{i}^\mathbf{1}}\)’s and \({\mathbf{W}_\mathbf{i}^\mathbf{6}}\)’s, and then the attacker selects a solution from the solution space. Each solution determines all the values of bit 1 and 6 of the expanded pair.

Once the message-bit conditions are solved and bit 1 and 6 are fixed, the attacker chooses the other message bits \(W_i^j, i\in \{0,\ldots ,15\}, j\in \{0,\ldots ,31\}\backslash \{1,6\}\) such that the state-bit conditions of Tables 45, and 6 are satisfied. Even though these conditions depend in a non-linear way on the bits of the message, the state-bit conditions up to Round 15 can easily be satisfied by an appropriate selection of message bits. Once they are solved, the first 15 words are fixed, and many values of \(W_{15}\) are tested. When all (or a selected part) of the possible values of \(W_{15}\) are tested, the pair is replaced, and the above procedure is repeated. We note that conformance up to Round 18 depends on state bits of Round 15 and earlier rounds. Thus, most of the state-bit conditions are corrected up to Round 18 with a negligible complexity.

Complexity Evaluation

As we have already indicated in Sect. 3.6, an attacker may choose message bits such that a pair conforms to the characteristic up to Round 18 with a negligible complexity. Conformance to higher rounds is considered uncontrollable. Thus, in the evaluation of the probability that a chosen pair conforms to 80 rounds, we count the conditions of Tables 45, and 6 from Round 18 up to Round 79, and assume that each condition contributes a factor \(\mathtt{1/2}\) to the probability.

The complexity is measured by the number of SHA-0 calls. Hence, using an early termination, when a non-conformance is detected, a factor 4 is saved, i.e., for each chosen pair the execution of the compression function is terminated after 20 rounds on average.

Under these assumptions, the complexity of the attack of [10] (that uses the characteristic described in Table 8) is \(2^{58}\) SHA-0 calls. We note that in [10] the announced complexity is \(2^{61}\) pairs of messages that an attacker needs to try. However, in an efficient implementation, an attacker selects a message and modify it such that it conforms to the first 15 rounds. He then modifies \(W_{15}\) many times (\(\approx 2^{13}\) times), and reuses the values of \(A_{1},\ldots , A_{15}\). Hence, each computation starts at Round 15, and it is terminated when a non-conformance is detected (on average at Round 20). Thus, the complexity of testing a pair is equivalent to \(1/8\) SHA-0 call, and the overall complexity is about \(2^{58}\) SHA-0 calls.

The Neutral Bits Technique

The neutral bits technique improves the complexity of the attack of Sect. 3. Using the neutral bits technique, an attacker gets the first 22 (or more) rounds for free (instead of 18 as in Sect. 3.6). Thus, the complexity of the attack that uses this technique depends only on the conditions of rounds 22–79.

Let \(\delta _{b_i}\) be a string of 512 bits in which the bit with index \(b_i\) set to “1” and all others are set to “0”. The basic idea of the neutral bits technique is to generate a pair \(M_k, M_k\oplus \Omega _M\) that conforms to a characteristic \(\Omega \) up to some threshold round \(R\) (e.g., \(R=22\)), and that has a set \(B=\{b_1,b_2,\ldots \}\) of bits (i.e., bit indices) with the following property: For each \(b_i\in B\) a simultaneous complementation this bit \(b_i\) in both messages \(M_k\) and \(M_k\oplus \Omega _M\), the obtained pair \(M_k\oplus \delta _{b_i}, M_k\oplus \Omega _M\oplus \delta _{b_i}\) conforms to at least the same number of rounds. We call the bits that have this property neutral bits, and use these bits to generate many pairs that conform to at least \(R\) rounds with a negligible complexity. Thus, effectively we start the attack from Round \(R\).

In the remainder of this section we give a formal definition of neutral bits, and a description of how to find and use them.

Definition 4

Let \(\Omega \) be a characteristic, \(R\) a threshold round number, and \(M_k, M_k\oplus \Omega _M\) a pair of message blocks that conforms to \(R\) rounds. A message bit \(W_i^j\) is a (single) neutral bit with respect to \(M_k, \Omega \), and \(R\), if the pair obtained by a complementation of \(W_i^j\) in \(M_k\) as well as in \(M_k\oplus \Omega _M\) also conforms to (at least) \(R\) rounds.

The above definition is concerned with a single bit difference. We now generalize the definition of neutral bits to any larger set of bits.

Definition 5

Let \(\Omega \) be a characteristic, \(R\) a threshold round number, and \(M_k, M_k\oplus \Omega _M\) a pair that conforms to \(R\) rounds. We say that a subset of bits \(b_i\subseteq \{0,\ldots ,511\}\) is a composite neutral bit Footnote 4 with respect to \(M_k, \Omega \), and \(R\), if the pair obtained by a simultaneous complementation of all the bits in \(b_i\) in \(M_k\) and \(M_k\oplus \Omega _M\) also conforms to (at least) \(R\) rounds.

We note that a composite neutral bit may consist of subsets of message bits that are composite neutral bits by themselves. E.g., let \(\{b_i,b_j,b_k\}\) be a composite neutral bit, then \(\{b_i,b_j\}\) may also be a composite neutral bit and \(b_k\) may or may not be neutral. In the remainder of this paper we use the term neutral bits for composite neutral bit as well as for single neutral bits.

Once we have a set \(B=\{b_1,\ldots ,b_{\left| B\right| }\}\) of neutral bits with respect to \(M_k, \Omega \), and \(R\), we are able to construct with each \(b_i\) a conforming pair, i.e., we can generate \(\left| B\right| \) conforming pairs. We are now interested in constructing a much larger number of neutral bits with a set \(B\) that has a compact representation in memory, i.e., the number of pairs that can be generated from \(M_k\), and \(B\) is not linear with the size of \(B\). To describe how we enlarge the set, and how it is represented in memory, we first define a neutral pair and 2-neutral set with respect to \(M_k, \Omega \) and \(R\).

Definition 6

A pair of neutral bits \(b_i\) and \(b_j\) is a neutral pair with respect to \(M_k, \Omega \), and \(R\), if \(M_k\), and each of the three message blocks obtained by complementing the bits \(b_i, b_j\), or both \(b_i\) and \(b_j\) conform to (at least) \(R\) rounds.

Definition 7

A set \(B=\{b_1,\ldots ,b_{\left| B\right| }\}\) of neutral bits is a 2-neutral-set with respect to \(M_k, \Omega \), and \(R\), if every pair of neutral bits in \(B\) is a neutral pair with respect to \(M_k, \Omega \), and \(R\), and the \(b_i\)’s are disjoint, i.e., composite neutral bits in \(B\) are disjoint.

The following experimental observation was made with data gathered in collision attacks of SHA-0. It shows how we extend our original set of neutral bits, and how we represent them in memory.

Observation 1

Given a 2-neutral-set \(B=\{b_1,\ldots ,b_{\left| B\right| }\}\) with respect to a message block \(M_k, \Omega \), and \(R\), a large fraction of the \(2^{\left| B\right| }\) different pairs which are obtained by complementing the bits of each possible combination of the neutral bits of \(B\), conforms to (at least) \(R\) rounds.

A conclusion from Observation 1 is that conforming pairs may be represented in memory by a logarithmic factor of the number of pairs, e.g., \(M_k, \Omega \), and \(B=\{b_1,\ldots ,b_{\left| B\right| }\}\) may represent about \(2^{\left| B\right| -3}\) pairs that conform to \(R\) rounds.

In subsequent sections we refer to versions of SHA-0 with extended and reduced number of rounds. The version of 82 rounds is particularly convenient, and we use it in the following example.

Example 1

A disturbance vector for 82-round extended SHA-0 is given in Table 9. For this disturbance vector the pair \(M_1, M^*_1\) given in Table 10 has many neutral bits. A 2-neutral set of this pair with respect to 22 rounds is given in Table 11 including single neutral bits (\(\left| b_i\right| =1\)), pairs (\(\left| b_i\right| =2\)), triplets (\(\left| b_i\right| =3\)), quadruplets (\(\left| b_i\right| =4\)), and quintuplets (\(\left| b_i\right| =5\)),each pair of neutral bits in this table is a neutral pair, and the neutral bits are disjoint subsets of the 512 message bits. The size of the 2-neutral set is 40, thus there are \(2^{40}\) possible compositions of neutral bits, from which about \(2^{37}\) (a fraction of \(\mathtt{1/8}\)) are neutral. Moreover,

  1. 1.

    A composition of 10 single neutral bits leads to a conformance of 49 rounds.

  2. 2.

    A composition of four single neutral bits and the first quadruplet leads to a conformance of 54 rounds. Thus, in less than \(2^{18}\) complexity we can find conformance to 54 rounds.

  3. 3.

    One of the \(2^{23}\) compositions of singles and pairs from Table 11 leads to conformance of 58 rounds.

Table 9 A disturbance vector to attack SHA-0 extended to 82 rounds
Table 10 The pair \(M_1, M_1^*\) of Example 1
Table 11 The 2-neutral-set of Example 1

A Collision Attack Using a 2-Neutral Set

Given a 2-neutral set with respect to \(M_k, \Omega \), and \(R\), we generate a new pair \(M_k\oplus \delta , M_k\oplus {\Omega }_M\oplus \delta \), where \(\delta \) is a composition of neutral bits from the set. For each different composition of the neutral bits, we obtain a different pair which with high probability conforms to (at least) \(R\) rounds. If the set is of size \(n\), then \(2^n\) different pairs can be generated by the set, of which a fraction of about \(\mathtt{1/8}\) conforms to \(R\) or more rounds. These pairs are used for the attack. The probability of an attack that uses these pairs is then:

$$\begin{aligned} p(s_{80}^{\prime }=\Omega _{S_{80}}|s_R^{\prime }=\Omega _{S_{R}}) =\prod _{t=R}^{79}p_t, \end{aligned}$$

where \(p_t\) is the probability of Round \(t\). Hence, the probabilistic analysis starts from round \(R\). Using this technique in the attack on SHA-0, the attack complexity is reduced from \(2^{60}\) to \(2^{54}\) SHA-0 calls, and a more careful analysis shows that the complexity is reduced to \(2^{48}\) SHA-0 calls. We note that many neutral bits are in \(W_{15}\), thus, for most tested messages generated in the attack \(A_{1},\ldots ,A_{15}\) are equal. Therefore, an efficient implementation computes these values once and reuses them many times afterward. The computation of each analyzed pair starts at Round 15 (instead of Round 0), and it is terminated when a non-conformance is detected. On average, a non-conformance is detected at Round 22. Hence, the complexity of testing each pair equals to a computation of 8 rounds, i.e., from Round 15 to Round 22. Further speed-up is made by modifying and testing Round 22, and only if this test passes a re-computation and testing of Rounds \(15,\ldots ,22\) is made. In terms of SHA-0 calls, the complexity of such technique is about \(1/8\) SHA-0 call for each pair.

A collision search algorithm that uses neutral bits consists of two phases. It starts with the generation of a 2-neutral set \(B\), and continues with an exhaustive search for all the candidate pairs derived from the set. If the search ends and a collision is not found, then a new message is randomly selected, and the process is repeated.

The size of the set decreases with the round threshold \(R\) for which the set is generated. We use the notation \(k(R)\) for the size of \(B\) with threshold \(R\), i.e., \(B=\{b_1,\ldots ,b_{k_{(R)}}\}\). The round number \(R\) for which we generate the set is selected to be the maximal round such that the complexity of finding a 2-neutral set of size \(k(R)\) is less than or equal to the complexity of exhaustively testing the \(2^{k(R)}\) messages for collision. In SHA-0, typical values for \(R\) are in the range 22–24.

The following observation shows that when the attack is performed using neutral bits, the probabilistic analysis of rounds greater than \(R\), is better than expected:

Observation 2

Let \(R'\) and \(R\) (\(R'>R\)) be some rounds, and \(k(R)\) the size of a 2-neutral set with respect to \(M_k, {\Omega }\), and \(R\), such that \(p(R\rightarrow R')=\prod _{t=R}^{R'-1}p_t \approx 2^{-k(R)}\). By generating the \(2^{k(R)}\) pairs, we obtain few pairs that conform to \(R'+l\), which we would expect to get with a larger set of neutral bits of a size \({k(R)+\alpha }\), where \(2 \le l \le 4\) and \(3 \le \alpha \le 8\).

Finding a 2-Neutral Set

In this section we describe an algorithm that finds a 2-neutral set. This algorithm chooses a random message \(M_k\) and modifies it such that the pair \(M_k, M_k\oplus {\Omega }_M\) conforms to \(R\) rounds, and it has a 2-neutral set of size \(k(R)\). The pair is optimized such that the size \(k(R)\) of the set is as large as possible, and the set is neutral with respect to the highest possible round \(R\). We note that since the time complexity of this algorithm is amortized over \(2^{k(R)}\) pairs, it does not have to be negligible. We also do not claim that this algorithm is optimal.

Finding Neutral Bits and Optimizing a Pair

Given a randomly chosen pair \(M_k, M_k\oplus {\Omega }_M\), we start to modify it one round at a time, to conform to as many rounds of the characteristic as possible. The modifications up to Round 15 are performed by direct complementation of the message bits as mentioned in Sect. 3. Once the pair conforms to some intermediate round \(r\), where \(16\le r< R\), we find and count the neutral bits with respect to \(r\). We then modify the pair such that its conformance to the first \(r\) rounds is not affected, and with the aim of increasing the number of neutral bits. When we cannot increase this number any further, we proceed to the next round. The process ends when we get to round \(R\), which is typically between \(22\) to \(24\).

A search for a single neutral bit for a given pair is performed simply by complementing a bit in both messages of the pair and test whether conformance is not affected. Once it is found, it is added to the set \(B\). This search covers the 512 bits of the message, excluding bit locations 1 and 6 which are set in advance to satisfy the message-bit conditions. The complexity of this search is estimated by the number of tests for neutrality we have to perform. In the case of a search for single neutral bits it is 480 tests (\(512-2*16\)). When a search for single neutral bits ends, a search for composite neutral bits of pairs is conducted. This search is performed similarly to the search for single neutral bits except that every pair of bits is complemented and tested for neutrality, and the search covers the 512 bits of the message to exclude bit 1 and 6, and the bits in \(B\). On average, the size of \(B\) when this search starts is about \(70\), thus the number of pairs we test for neutrality is about \(\left( \begin{array}{l}410\\ 2\end{array}\right) \) = 83,845. Similarly, we find larger composite neutral bits until we identify as many neutral bits as possible.

The modifications of the pair to extend the set are performed by trying to complement two or more bits of a local collision sequence \(W_i^j, W_{i+1}^{j+5}, W_{i+2}^j, W_{i+3}^{j+30}, W_{i+4}^{j+30}, W_{i+5}^{j+30}\), in both messages of a pair. These modifications are repeated where the index \(j\) is advanced by one each time, until all \(j\)’s are covered. Each time we modify the pair, we count the number of neutral bits. If the number is increased by the modification, then we replace the original pair with the new one and start the procedure of modifying and counting all over. We end up with a pair that conforms to \(R\) rounds and has the largest number of neutral bits that we can find.

Finding Neutral Pairs and 2-Neutral Sets

Given a pair with a set of neutral bits, we are interested in finding the largest 2-neutral set. We first identify neutral pairs by a simultaneous complementation of each pair of neutral bits in the set, and by testing whether the conformance to \(R\) rounds is not affected. We then represent each neutral bit as a vertex in a graph, and add an edge for each pair of vertices that corresponds to a neutral pair. The largest 2-neutral set corresponds to the largest clique in the graph, i.e., the maximal subset of vertices for which any vertex in the subset is connected to any other vertex in the subset by an edge.

Although the general problem of finding a maximal clique is an NP-complete problem, in our case finding a large enough clique is not difficult, as many vertices are connected to all other vertices by edges. Therefore, we use simple heuristics that produce a very good approximation to the maximal clique: We initialize the 2-neutral set to be an empty set. We then generate a sorted list of vertices which represent single neutral bits, by the number of edges that are connected to each vertex. The first vertex in the sorted list is added to the 2-neutral set, and the vertices that are not connected to the vertex we have just added are removed from the graph. The procedure is repeated with the remaining graph until the graph is empty. Once the graph is empty, the 2-neutral set contains a clique of single neutral bits. Our next step is to add larger composite neutral bits. We start by searching for neutral sets of size two (pairs) among the bits which are not in the 2-neutral set \(B\) we have just constructed. From the neutral sets we find, we remove those which are not neutral pairs with each neutral bit in \(B\). With the remaining neutral sets we build a new graph and search the maximal clique as before. The procedure is then repeated with neutral sets of size three (triplet), size four (quadruplet), and as many other sets we can find.Footnote 5 The result of such an algorithm is shown in Example 1 and Tables 10 and 11.

The Complexity is not Monotonous with the Number of Rounds

The attacks of Sections 3 and 4 are also applicable to SHA-0 with fewer or more than 80 rounds, i.e., for the attack of SHA with \(R\) rounds, we use the method of Sect. 3.2 to find a disturbance vector with the least number of disturbances from Round 22 up to Round \(R\) and we use it to construct a characteristic. Consulting Tables 7 and 9 we observed that the disturbance vector of the full 80-round SHA-0 has more disturbances than that of the 82-round (22 vs. 19 respectfully). That is to say, we unexpectedly found that the complexity of attacking a version with more rounds is lower than that with fewer rounds.

Table 12 summarizes the complexities of attacking SHA with different number of rounds. From this table we see that there are few reduced and extended versions with no disturbance vector that predicts collisions: in these cases this attack cannot be used. We also see that the complexity of collision attacks on the different versions is not monotonous with the number of rounds, e.g., extended versions with 81 and 82 rounds are much easier to attack than 80-round SHA-0.

Table 12 Complexities of attacks on reduced and extended versions of SHA-0 (further results are given in Table 1)

The phenomenon of non-monotonic complexities with the number of rounds is a result of the expansion of the disturbance vector by the LFSR. The number of disturbances that the LFSR produces depends on its initialization, and it does not necessarily grow with \(R\). Hence, the probability associated with each disturbance vector does not necessarily decrease with the number of rounds.

A prior case of attack on an extended version of a hash function that has a smaller complexity than the attack on the full version appeared in the differential cryptanalysis of N-Hash [5, 27]. N-Hash has eight rounds, while the attack can find collisions for versions with 3, 6, 9, and 12 rounds. In particular, a collision of a 9-round extended version can be found faster than a collision of the full 8-round version.

This non-monotonicity occurs because collision attacks require that the characteristic prediction of the output difference will be zero. For comparison, in block ciphers, a reduction of the characteristic to a smaller number of rounds necessarily results in another characteristic with a higher probability. In hash functions, the reduced characteristic has higher probability as well, but it does not have the zero difference as required for a collision. Thus, unlike in the case of block ciphers, the probabilities of the best characteristics are not monotonically decreasing with the number of rounds.

The Multi-Block Tool

Most hash functions currently in use are based on the Merkle–Damgård construction which is proved to be collision resistant if the underlying compression function is collision resistant. In this section we investigate the strength of the construction in cases the compression function is not collision resistant.

In a single-block attack an attacker uses the standard initial value and tries to find two message blocks that hash to the same value, i.e., \(\Omega _P=\Omega _T=0\), and \(\Omega _M\ne 0\). Though the domain of the input of the compression function consists of the message \(\{0,1\}^{b}\) and chaining value \(\{0,1\}^{m_c}\) domains, only the message domain is analyzed. As we show in Sect. 3, this analysis suffices to break SHA-0. However, there are variants of SHA-0 (see Table 12) in which this type of attack is not applicable. Hence, an extension of the analysis to the message and chaining value domains \(\{0,1\}^{b+m_c}\) is natural.

Given characteristics with \(\Omega _P\ne 0, \Omega _M\ne 0\), and \({\Omega }_T=0\), an attack consists of two steps. Firstly, characteristics that lead to the required \(\Omega _P\ne 0\) are used. Secondly, the characteristic \(\Omega _P\ne 0, \Omega _M\ne 0\), and \({\Omega }_T=0\) is used to find a collision. The iterative structure of the Merkle–Damgård construction enables such concatenations of characteristics.

The multi-block technique allows for differences in both inputs of the compression function and also in the output of the compression function. This extension allows us to use characteristics with higher probabilities than previously used. Each characteristic by itself (except the last one) does not lead to a collision but to a near-collision, and the last characteristic leads to a collision. The technique shows how characteristics that predict near-collisions are found and concatenated to a longer characteristic that predicts a collision. In the next subsections we present the technique with examples that emphasize different aspects and advantages. We note that although our examples are all on the SHA family of hash functions, the multi-block technique is applicable to any differential attack on hash functions that follow the Merkle–Damgård (or any iterative) construction.

As a first and simple example we show a case where several blocks are used for an attack, which is useful when the initial state of the compression function is incompatible with the characteristic. The general and more interesting cases are described afterward.

Solving Initial State Incompatibility by an Additional Block

In some cases the initial state is incompatible with the characteristic. As an example consider:

$$\begin{aligned} A_1^0=W_0^0\oplus A_0^{27}\oplus \hbox {IF}(A_{-1}^0,A_{-2}^2,A_{-3}^2)\oplus A_{-4}^2\oplus K_0^0. \end{aligned}$$

With \(h_0\) the standard initial values, \(A_0^{27}=0, A_{-1}^0=1, A_{-2}^2=0, A_{-3}^2=0, A_{-4}^2=0\), and \(K_0^0=1\), thus:

$$\begin{aligned} A_1^0=W_0^0\oplus 0 \oplus \hbox {IF}(1,0,0)\oplus 0\oplus 1=\overline{W_0^0}. \end{aligned}$$

In this case a disturbance in the first round in bit location 0 always leads to a difference in the carry from bit 0 to bit 1, which contradicts the conditions of the characteristic.

Another incompatibility may occur by the \({\hbox {f}}_i\) function. Consider a disturbance in bit \(j\) of the first message word, \({\mathbf{W}_\mathbf{0}^\mathbf{j}}\), where \(A_0^{j-30}=A_{-1}^{j-30}\). In this case, \(\hbox {IF}_2^j({\mathbf{A}_\mathbf{1}^\mathbf{j}},A_0^{j-30},A_{-1}^{j-30})\) is not active, so the correction \({\mathbf{W}_\mathbf{2}^\mathbf{j}}\) always fails. In case the initial state equals the standard initial value, this incompatibility occurs in 24 out of the 32 bit locations (except for locations 1, 5, 9, 13, 17, 21, 25, and 29).

A straightforward solution for this incompatibility is to replace the initial state with a compatible one. In the case of a first block where the initial state equals the standard initial values, a first arbitrary message block (with zero difference) is added in both runs to obtain a compatible initial state. The actual attack starts with the second block using the compatible \(h_1\) instead of the incompatible \(h_0\). The result is a two-block attack with no difference in the first block but with a difference in the second block.

Two-block Collisions

In this section we describe a technique to find two-block collisions. In a two-block attack, the first pair starts with the same initial value \(h_0\) (which is usually the standard initial value but may be different for reasons mentioned in the previous section). The pair \(M_1, M_1^*\) is found such that \(h_1=C(M_1,IV), h_1^*=C(M_1^*,IV)\), and \(h_1 \approx h_1^* \), where the operator \(\approx \) denotes a small Hamming distance between these two values. The pair \(M_1,M_1^*\) and chaining variables \(h_0=IV, h_1\), and \(h_1^*\) form a near-collision. A second pair of messages \(M_2, M_2^*\) is now found such that \(h_2=C(M_2,h_1)=C(M_2^*,h_1^*)=h_2^*\). This second pair starts from the chaining variables with the small difference of the previous pair, and ends with a collision. The second pair \(M_2, M_2^*\) with the chaining variables \(h_1,h_1^*\), and \(h_2\) form a pseudo-collision. An illustration of near-collision, pseudo-collision, and a two-block collision is given in Fig. 7. Such an attack is useful since in many cases finding near-collisions and pseudo-collisions is much easier than finding a single-block collision. It is also useful when there is no disturbance vector that predicts a single-block collision.

Fig. 7
figure7

Using intermediate near-collisions to find collisions with two blocks

In order to apply such an attack an attacker searches for a pair of disturbance vectors from which he constructs the characteristics \(\Omega ^1, \Omega ^2\). The disturbance vectors are selected such that the differences of the characteristics are: \(\Omega ^1_P=0, \Omega ^2_T=0\), and \(\Omega ^1_T=\Omega ^2_P\), and the least probability of both characteristics (\(\min (p_{\Omega }^1,p_{\Omega }^2)\)) is maximal. After the construction of the characteristics the attacker searches for right pairs. The search starts with the aim of finding a right pair for \(\Omega ^1\) with the initial value \(h_0\). Once it is found, \(h_1\) and \(h_1^*\) of the first right pair are used in the search of a right pair for \(\Omega ^2\). The result forms a two-block collision.

Characteristics For a Two-Block Attack

Given the disturbance vectors \(\mathcal{D}_1\) and \(\mathcal{D}_2\), we proceed with the construction of the characteristic of each block. Constructing the characteristics of a two-block attack is somewhat different than for a single-block attack, since the requirements imposed by the concatenation of the first and second pairs should be considered. In a single-block attack \({\Omega }_T={{{\varvec{00000}}}}\) if and only if

$$\begin{aligned} ({\Omega }_{A_{R^{\prime }}},{\Omega }_{A_{R^{\prime }-1}}, {\Omega }_{A_{R^{\prime }-2}},{\Omega }_{A_{R^{\prime }-3}}, {\Omega }_{A_{R^{\prime }-4}})={{{\varvec{00000}}}}. \end{aligned}$$
(9)

In a two-block attack we set conditions to obtain \({\Omega }_T\). We denote the final chaining value of block \(k\) by \(h_k\), and set conditions such that the difference \(h_k\oplus h_k^*\), equals the difference defined by \({\Omega }_T^k\) where

$$\begin{aligned} h_{k,i}&= A_{R^{\prime }\!-\!i}\!+\!A_{-\!i}\!=\!A_{R^{\prime }\!-\!1\!-i}^{\lll 5}\!+\!{\hbox {f}}_{R^{\prime }\!-\!1\!-\!i}(A_{R^{\prime }\!-\!2\!-\!i},\! A_{R^{\prime }-3-i}^{\lll 30},\!A_{R^{\prime }\!-\!4\!-\!i}^{\lll 30})+\\&A_{R^{\prime }-5-i}^{\lll 30} + W_{R^{\prime }-1-i}+K_{R^{\prime }-1-i}+A_{-i},\quad i=0,\ldots ,4,\quad k=1,2.\nonumber \end{aligned}$$
(10)

In addition we should consider the compatibility of the initial state with the characteristic. In the first block this requirement may be satisfied using the technique of Sect. 6.1, which turns the attack into a three-block attack with the same first block in both runs. Incompatibility of initial values with characteristics other than the first characteristic cannot be resolved by the technique of adding an identical block to both runs. Hence, conditions on the final chaining values \(h_1, h_1^*\) (or \(h_2, h_2^*\) in case a first block is added) may be added in order to ensure that the chaining values are compatible with the characteristic of the next block. These additional conditions contribute to the probability of the previous characteristic, thus increasing the complexity of finding the near-collision in that block.

Complexity Evaluation

The complexity of the attack is measured by the number of SHA-0 calls. It is computed for each block by multiplying the complexity of generating a pair that conforms to at least \(R\) rounds, by the number of pairs we need to generate in order to find a right pair for each characteristic.

As shown in Sect. 4.1, the complexity of generating a pair that conforms to at least \(R=22\) rounds is about \(0.25\) SHA-0 call. The number of pairs we need to generate in the attack depends on the probabilities of obtaining a pair that satisfies the requirements we have just described. These probabilities are computed by counting the number of conditions that such a pair should satisfy. It is assumed that the conditions are independent, and that each condition is satisfied with probability \(1/2\). These probabilities are listed as follows:

  1. 1.

    The probability

    $$\begin{aligned} p_{R\rightarrow R^{\prime }-5}=p(s_{R^{\prime }-5}^{\prime }=\Omega _{S_{R^{\prime }-5}}|s_R^{\prime }=\Omega _{S_{R}}, s_0^{\prime }={\Omega }_P, M^{\prime }={\Omega }_M) \end{aligned}$$

    that a pair that conforms to at least \(R\) rounds also conforms to \(R^{\prime }-5\) rounds (where \(R^{\prime }\) is the number of rounds we attack). In the computation of this probability we count the conditions for conformance from Round \(R\) up to Round \(R^{\prime }-5\).

  2. 2.

    The probability

    $$\begin{aligned} p_{R^{\prime }-5\rightarrow {\Omega }_T}=p(h_i^{\prime }={\Omega }_T|s_{R^{\prime }-5}=\Omega _{S_{R^{\prime }-5}^{\prime }}, s_0^{\prime }={\Omega }_P, M^{\prime }={\Omega }_M) \end{aligned}$$

    that a pair that conforms to \(R^{\prime }-5\) rounds has the required final chaining difference \({\Omega }_T\), where \(i\) is the block number we attack.

  3. 3.

    Let \(p_{connect}\) be the probability that a pair that conforms to the characteristic of the current block, also has chaining values which are compatible with the characteristic of the next block. \(p_{connect}\) is computed by counting the conditions on the initial values of the next block, i.e., that depend on \(A_i^j\)’s where \(i\le 0\).

Notice that in case the technique of Sect. 6.1 is used to generate an initial value compatible with \({\Omega }_P\) of the first block, the complexity of generating the first block equals \(1/p_{connect}\).

Collisions with More than Two Blocks

The two-block attack is generalized to multi-block attack with longer paths of differences. In such an attack a first pair of message blocks \(M_1, M_1^*\) is found such that it leads to a near-collision \(h_1\approx h_1^*\). A second pair \(M_2, M_2^*\) is then found such that \(M_2\) compressed with \(h_1\), and \(M_2^*\) compressed with \(h_1^*\) create a second near-collision \(h_2\approx h_2^*\). We call such near-collision to near-collision blocks, near-pseudo-collision. After one or more near-pseudo-collision blocks, a final pseudo-collision is applied to form a multi-block-collision. An illustration of a multi-block collision is given in Fig. 8. The transition graph contains 32 vertices, as shown in Fig. 9, and each pair of vertices is connected with about \(2^6\) edges.

Fig. 8
figure8

The multi-block technique—using intermediate near-collisions to find collisions

Fig. 9
figure9

The modified chaining transition graph

We note that in some cases it might be desirable to have the same difference twice, i.e., \(h_i'=h_j', i\ne j\), although it looks like the rest of the stream could be directly computed from the first occurrence of that difference. Such cases may appear when the next block cannot start with the currently found pair of intermediate values, due to compatibility requirements with the next block. Of course in such cases it is possible to search again for another previous block, and get another near-collision with the same difference, which will hopefully solve the problem. But in some cases it may be faster to find a block with equal input and output differences (or a series of blocks that lead to the same difference after several blocks). Thus, the attacker may prefer not to discard the near-collision he found, but to use it as an easy starting point for a more usable one with the same difference.

Revisiting Characteristics and Disturbance Vectors

In the attacks we described so far, we assumed a one-to-one correspondence between disturbance vectors and characteristics (see Sect. 3.2). This assumption is a result of our approximation of the non-linear functions: IF, MAJORITY, and addition modulo \(2^{32}\) by the XOR operation.

In particular, we assumed that a transition between a pair of vertices that represents the chaining differences \({\Omega }_P\) and \({\Omega }_T\) is made by an edge (disturbance vector) \(\mathcal D\) that satisfies certain relations: \(\mathcal D\) leaves a vertex which represents a difference \({\Omega }_P\) if

$$\begin{aligned} {\Omega }_P =(\mathcal{D}_{-1},\mathcal{D}_{-2},\mathcal{D}_{-3}^{\lll 30},\mathcal{D}_{-4}^{\lll 30},\mathcal{D}_{-5}^{\lll 30}). \end{aligned}$$
(11)

Similarly, it enters a vertex that represents a difference \({\Omega }_T\) if

$$\begin{aligned} {\Omega }_T&= ((\mathcal{D}_{-1}\oplus \mathcal{D}_{79}), (\mathcal{D}_{-2}\oplus \mathcal{D}_{78}), \\&(\mathcal{D}_{-3} \oplus \mathcal{D}_{77})^{\lll 30}, (\mathcal{D}_{-4} \oplus \mathcal{D}_{76})^{\lll 30}, (\mathcal{D}_{-5} \oplus \mathcal{D}_{75})^{\lll 30}).\nonumber \end{aligned}$$
(12)

Using these relations we constructed the transition graph of Fig. 9 with about \(2^6\) edges (disturbance vectors) that connect each pair of vertices.

We now show how characteristics (edges) that do not follow the XOR approximation in the first few rounds are added to the transition graph. In these characteristics \({\Omega }_P\ne (\mathcal{D}_{-5},\ldots ,\mathcal{D}_{-1})\) but the remainder of the differences of the reduced state vector are still approximated by \({\Omega }_A=\hbox {SR}^1(\mathcal{D})\). The updated transition graph includes edges \(\mathcal D\) that leave vertex \({\Omega }_P\), enter vertex \({\Omega }_T\) and satisfy:

$$\begin{aligned} {\Omega }_T&= {\Omega }_P\oplus (\mathcal{D}_{79}, \mathcal{D}_{78}, \mathcal{D}_{77}^{\lll 30}, \mathcal{D}_{76}^{\lll 30}, \mathcal{D}_{75}^{\lll 30})={\Omega }_P\oplus {\Omega }_{s_{80}}, \end{aligned}$$
(13)

where \({\Omega }_P\) is not necessarily the first five entries of \(\mathcal D\). Note that if these characteristics are used, then a few additional conditions should be added to the first rounds. However, since the overall probability is measured from higher rounds (e.g., Round 22) they are not a factor in the complexity of the attack.

In the following example we demonstrate how a disturbance vector that at a first look seems incompatible is used to connect a pair of vertices in a transition graph.

Example 2

Consider the transition graph in Fig. 10. In this graph, transitions are made from the vertex 11000 to \({\Omega }_T\) by compatible disturbance vectors (i.e., that satisfy Eq. (11)), but in addition disturbance vectors that start with 01000 (which obviously does not equal the chaining difference of the vertex) are used. The disturbance vector that is used to construct the differences \({\Omega }_W\) between the runs, starts with 01000 while the differences between the state vectors start with \({\Omega }_P={{{\varvec{11000}}}}\). With these incompatible \({\Omega }_W\) and \({\Omega }_A\) we show that the difference \({\mathbf{A}_{\mathbf{-4}}^\mathbf{1}}\) that has no corrections in \({\Omega }_W\), does not propagate, i.e.,

$$\begin{aligned} (A_{-3}^{\prime }, A_{-2}^{\prime },A_{-1}^{\prime },A_{0}^{\prime }, A_1^{\prime },A_2^{\prime })=\mathtt{(2,0,0,0,0,0)}. \end{aligned}$$

\({\mathbf{A}_{\mathbf{-4}}^\mathbf{1}}\) affects the computations only at Round 0; thus if it is compensated at Round 0, then it does not affect successive computations. By setting \(A_{-1}^{31}=1\) (instead of \(A_{-1}^{31}=0\) per Table 5), \({\hbox {IF}_{0}^{31}(A_{-1}^{31},A_{-2}^1,{\mathbf{A}_{\mathbf{-3}}^\mathbf{1}})= A_{-2}^1}\) is inactive, and its correction \({\mathbf{W}_\mathbf{0}^{\mathbf{31}}}\) is used to compensate \({\mathbf{A}_{\mathbf{-4}}^\mathbf{1}}\) as required.

Fig. 10
figure10

Matching seemingly incompatible disturbance vectors to a chaining difference

Similar techniques to the one shown in Example 2 are used to find more initial chaining differences (\({\Omega }_P\)’s), and compatible disturbance vectors. A summary of these compatibilities is given in Table 13. In this table the first column is the compact representation of \({\Omega }_P\). The next three columns show the corrections applied to bits 1, 6, and 31 of \(W_4^{\prime },\ldots ,W_0^{\prime }\). In this representation, no correction is denoted by “0”, and a correction by “1”. The next three columns specify the required differences to correct the disturbances. A “0” denotes an inactive bit, “1” an active bit, and “*” denotes a bit that may be active or inactive. Compatibility is determined by considering each “*” in the required differences \(W_4^{\prime },\ldots ,W_0^{\prime }\) once as a “0” and then as “1”. The result is compared with the corrections of the local collision sequence, and compatibility is determined accordingly. The last column gives all the combinations of \(\mathcal{D}_{-1}^1,\ldots ,\mathcal{D}_{-5}^1\) which are compatible with \({\Omega }_P\).

Table 13 Chaining differences and their compatibility with disturbance vectors

Table 13 is now used (instead of Eq. (11) to draw the edges that leave each vertex, and Eq. (13) is used to compute the vertex to which each edge is entered. In the obtained graph we search for the path with the least complexity as before.

A Four-Block Collision of SHA-0

The application of the technique of Sect. 6.6 to SHA-0 results in the 4-block path outlined in Fig. 11. A complete description of the four disturbance vectors of this path is given in Table 14.

Fig. 11
figure11

The four-block chaining differences path of SHA-0

Table 14 The disturbance vectors of the four-block collision

In this path, the first pair of blocks starts with the standard initial value (i.e., \(s_0=s_0^*=IV\) and \({\Omega }_{P_1}={{{\varvec{00000}}}}\)) and ends with a final chaining difference \({\Omega }_{T_1}={\Omega }_{P_2}={{{\varvec{00011}}}}\). The characteristic we use for this transition is constructed using the disturbance vector \(\mathcal{D}_1=\mathtt{A100}\). Consulting Table 14, \(\mathcal{D}_1\) predicts a final state difference \(s_{80}^{\prime }={{\varvec{00011}}}\), hence using Eq. (13), \(\mathcal{D}_1\) is suitable for the first transition. The transition of the second pair of blocks is from a chaining difference \({\Omega }_{P_2}={{{\varvec{00011}}}}\), to a final difference \({\Omega }_{T_2}={{{\varvec{11001}}}}\) using \(\mathcal{D}_2=\mathtt{4022}\). Consulting Table 13, the initial chaining difference \({\Omega }_{P_2}={{{\varvec{00011}}}}\) is compatible with 00010, hence \(\mathcal{D}_2=\mathtt{4022}\) is suitable to leave vertex \({\Omega }_{P_2}\). The predicted final state of \(\mathcal{D}_2\) is \(s^{\prime }_{80}={{{\varvec{11010}}}}\), thus by Eq. (13) the predicted chaining difference of the second block is \({\Omega }_{T_2}={\Omega }_{P_3}={{{\varvec{11010}}}}\), and \(\mathcal{D}_2\) is suitable for the second transition. The transition of the third block is from \({\Omega }_{P_3}={{{\varvec{11001}}}}\) to \({\Omega }_{T_3}={{{\varvec{01000}}}}\) using \(\mathcal{D}_3=\mathtt{1491}\). \({\Omega }_{P_3}={{{\varvec{11001}}}}\) is compatible with 10001, thus \(\mathcal{D}_3=\mathtt{1491}\) is suitable to leave vertex \({\Omega }_{P_3}\). \(\mathcal{D}_3\) predicts a final difference \(s^{\prime }_{80}={{\varvec{10001}}}\), hence the predicted chaining difference of the third block is \({\Omega }_{T_3}={\Omega }_{P_4}={{\varvec{01000}}}\), and \(\mathcal{D}_3\) is suitable for the third transition. The last transition in this path is made from \({\Omega }_{P_4}={{\varvec{01000}}}\) to \({\Omega }_{T_4}={{\varvec{00000}}}\) using \(\mathcal{D}_4=\mathtt{008B}\). \({\Omega }_{P_4}={{\varvec{01000}}}\) is compatible with 01011, thus \(\mathcal{D}_4=\mathtt{008b}\) is suitable to leave \({\Omega }_{P_4}\). \(\mathcal{D}_4\) predicts a final difference \(s^{\prime }_{80}={{\varvec{01000}}}\), hence \({\Omega }_{T_4}={{\varvec{00000}}}\), and a collision is predicted after the fourth block.

Using these disturbance vectors we constructed the characteristics of each block, which are given in Tables 151617, and 18. In these tables, the locations of inconsistency between a disturbance vector and a reduced state vector are marked with a gray background. The conditions at these locations are set as explained in Sect. 6.6. In the last five rounds we set conditions such that the desired \({\Omega }_T\) is obtained.

Table 15 A characteristic of a four-block attack on SHA-0 (first block)
Table 16 A characteristic of a four-block attack on SHA-0 (second block)
Table 17 A characteristic of a four-block attack on SHA-0 (third block)
Table 18 A characteristic of a four-block attack on SHA-0 (fourth block)

We implemented this attack on a highly parallel computer using this path of four characteristics, and a four-block collision was found. The resulting collision is given in Table 19. The chaining variables and differences of this four-block collision of SHA-0 are given in Table 20.

Table 19 A four-block collision of SHA-0
Table 20 The intermediate chaining variables and differences of the four-block collision of SHA-0

Complexity Evaluation

The complexity evaluation of the four-block attack on SHA-0 is similar to the evaluation of the two-block attack. The contributions of each of the factors \(p_{22\rightarrow 75}, p_{75\rightarrow {\Omega }_T}\), and \(p_{connect}\) to the overall probability and to the complexity of the attack are listed in Table 21. In this table we present the probabilities and complexity of each block in two rows. In the first row we present the probabilities to obtain the required differences of the pair of message blocks. In the second row we write the probability of connecting the current block with the next block, and the total probability and complexity of finding a right pair. The overall complexity of the attack under these assumptions is approximately \(2^{46}\) SHA-0 calls.

Table 21 Probabilities and complexities of the four-block attack on SHA-0

When measured in terms of the number of pairs to test, about \(2^{51}\) pairs are tested until a collision is found. This number is equivalent to about \(2^{49}\) SHA-0 calls. The actual number (in comparison with the data in Table 21) is due to an analysis that considers conformance to \({\Omega }_A\) in all of the 80 rounds, and then checks whether the correct \({\Omega }_T\) is obtained. In addition, an implementation error of the neutral bits technique contributed a factor to the excessive pairs that were tested.

Attacks on Reduced Versions of SHA-1

The only difference between SHA-1 and SHA-0 is an additional rotation operation in the expansion process that mixes the bits in the expanded message in a more efficient way than SHA-0 does. However, the expansion remains linear, and therefore the construction of local collision sequences with the technique described in the previous sections is still applicable.

The bit rotation increases the Hamming weight of disturbance vectors. However, it is not sufficient to make this weight as heavy as in random codes of the same size. Indeed, surprisingly low weight vectors still exist in SHA-1.

Even though the additional rotation mixes the message bits in a more efficient way, it does not affect the efficiency of the neutral bits technique. Therefore, the neutral bits technique can be used to attack SHA-1 with no modifications. We note that the existence of neutral bits in SHA-1 is not a surprise, as it is still possible to create small differences in the state vector with the local collision sequences technique.

The multi-block technique is applicable to any differential attack on iterative hash functions, including SHA-1. Actually this technique is more important in the case of SHA-1 since it is difficult to find a disturbance vector that predicts a single-block collision.

Selecting a Disturbance Vector for the Attack

The attacks on SHA-0 use only bit 1 as the location of disturbances. This selection is possible since the LFSR that expands the message does not mix bits in different bit indices. Furthermore, disturbances in bit 1 have the highest probability since the probabilistic behavior of the carry when corrections are made to bit 31 is eliminated. In SHA-1 the rotation by one bit in the expansion process moves the disturbances to different bit indices. Therefore, bits in several locations are used for disturbances. Notice that since disturbances may be located in different bit indices, the short representations we used in SHA-0 for disturbance vectors, state differences, and initial and final chaining differences are not applicable for SHA-1.

Since disturbances can be made at any bit location, the number of candidate disturbance vectors is raised from \(2^{16}\) in SHA-0 to \(2^{512}\) in SHA-1. From these disturbance vectors those that have the minimal Hamming weight should be selected. In order to reduce the search domain, we heuristically chose disturbances in adjacent bits, e.g., bits 0 and 1. With this selection, disturbances sometimes cancel each other in the expansion process due to the rotation by one bit. By searching in a domain of vectors with adjacent disturbances we found some that lead to attacks faster than the birthday attack on various reduced versions of SHA-1.

Once a disturbance vector with a low Hamming weight is found we may cyclically rotate all the words of the vector simultaneously by one bit and obtain a different disturbance vector with the same low Hamming weight. Similarly, different disturbance vectors with the same low Hamming weight are obtained by a rotation of two bits, three bits, and so forth, up to 31 bits. Each of those 32 disturbance vectors with the same Hamming weight may have a different probability, mainly due to the different number of computations with differences at bit location 31 where probabilistic behavior of the carry is eliminated. Therefore, once a low Hamming weight disturbance vector is found, we compute the probabilities of each of the 32 different disturbance vectors that are obtained by rotations, and select the one with the highest probability for the attack.

Constructing a Two-Edge Path with the Same Disturbance Vector

As we have already seen in Example 10 and in the attack on full SHA-0 (Sect. 7), a disturbance vector may be compatible with many initial chaining differences. In particular, a disturbance vector which is compatible with a zero chaining difference and with the chaining difference that it predicts might be used for our construction: Firstly, since it is compatible with a zero chaining difference then it can be used to construct the edge that leaves the zero initial value toward a near-collision. Secondly, it is compatible with the differences it predicts, thus it can also be used to construct an edge that leaves this near-collision. Finally, by Eq. (13) (which is applicable to SHA-0 as well as to SHA-1) it predicts a collision, thus it meets the requirements for the construction of our path.

The first advantage of such a construction is its simplicity: Instead of constructing a very complex transition graph and then search for the optimal path, we are concentrated only on a very specific type of paths in a graph. The second advantage is that using the efficient algorithm that finds disturbance vectors with low Hamming weight (see Sect. 8.1), the paths that are generated by the construction have high probability. Moreover, if the disturbance vector that is used to construct the path has the highest probability then this path is optimal. We note that by using carries as an additional resource for disturbances and corrections the number of compatible chaining differences substantially increases. Thus, if a low Hamming weight disturbance vector is found then it is most likely suitable for the construction.

A Collision of SHA-1 Reduced to 40 Rounds

This section demonstrates the idea of Sect. 8.2 by a typical type of a two-block collision. The two blocks use the same basic characteristic with some changes in the first few rounds that allow the concatenation of the two blocks. This type of attack is generally more efficient than finding two different characteristics when using the general construction of the multi-block technique. It enables high probabilities in the two blocks even when only one characteristic with high probability is known (compared with the other case in which a lower probability characteristic is also used).

The construction of a two-edge path with the same disturbance vector is demonstrated on SHA-1 reduced to 40 rounds. The disturbance vector we are using is the same vector used in the 34-round attack [7], rotated by 28 bits to the left and expanded to 40 rounds. The characteristic of the first and second blocks are given in Table 22.

Table 22 A two-block characteristic of 40-round reduced SHA-1

Using these characteristics we easily found a two-block collision of 40-round SHA-1. The messages and chaining variables of the 40-round collision are given in Table 23. The probability and complexity of the attack is given in Table 24.

Table 23 Two-block collision of 40-round SHA-1
Table 24 Probabilities and complexities of the 40-round SHA-1 attack

Strength of Reduced Versions of SHA-1 with More Rounds

SHA-1 with more than 40 rounds is also vulnerable to the attacks described in this paper. Though all the disturbance vectors that we found have consecutive disturbances in the first 17 rounds, many of them are correctable. We therefore list here two sets of results: the first set of results for SHA-1 reduced to fewer rounds, where consecutive disturbances are correctable by the techniques we have already described. The second set of results, denoted later by NO-IF, have consecutive disturbances which are not correctable by these techniques. These disturbance vectors might be used if the reduced version of SHA-1 starts at a different location (such as from Round 20 with the XOR function at the first 20 rounds). They also may be used if these consecutive disturbances would be corrected with more creative usage of carries as a source of disturbances and corrections.

Table 25 lists the Hamming weights of the best disturbance vectors results we found to attack various reduced versions of SHA-1. For each reduced version, and each set of results (SHA-1 or NO-IF) the table lists the Hamming weight of the disturbance vector from round 20 for three cases: the first, marked by HW, is the Hamming weight of the best disturbance vector predicting a single-block collision. The second, marked by 2B, is the best disturbance vector predicting a two-block collision; and the last, marked by NC, is the best disturbance vector predicting a near-collision. The numbers in boldface are the number of disturbances in the disturbance vectors we actually used to attack the reduced version. We note that the disturbance vectors that are listed under the 2B column are used in a two-block attack with the technique described in Sect. 8.1, i.e., the same disturbance vector is used to construct the characteristic of the first and second block.

Table 25 The hamming weights of the best disturbance vectors that we found (counted from round 20)

The complexities of the attacks can be approximated by the number of disturbances in rounds where the IF, XOR, and MAJ functions are used. For each disturbance where the IF, XOR, and MAJ functions are in use we approximate probabilities of \(2^{-5}, 2^{-2}\), and \(2^{-4}\), respectively. The overall complexity is then \(2^{5\cdot \mathrm{HW}_{\mathrm{IF}}+2\cdot \mathrm{HW}_\mathrm{XOR}+4\cdot \mathrm{HW}_\mathrm{MAJ}}\), where HW is the Hamming weight of the disturbance vector from round 20 and on (i.e., the HW value in Table 25). A less accurate approximation of \(2^{3\cdot \mathrm{HW}}\) might be used.

Using this approximation, we can see that disturbance vectors with Hamming weight of up to 26 predict collisions with complexity (slightly) faster than the generic birthday attack (as \(2^{3\cdot 26} = 2^{78} < 2^{80}\)). We marked the location of this threshold by underlines. Hamming weights much smaller than 26, predict much more practical complexities, and as can be seen from the table, Hamming weights up to about 10 require only a short computation on a personal computer (all the found collisions marked in boldface were found within a few seconds of computation).

According to Table 25, SHA-1 reduced to 53 rounds can be attacked with a complexity less than a generic attack. It is a single block attack and the disturbance vector from which the characteristic is derived is given in Table 26. This disturbance vector has consecutive disturbances in two locations: the first is bit 29 of \(\mathcal{D}_{10}\) and \(\mathcal{D}_{11}\), and the second is bit 31 of \(\mathcal{D}_{15}\) and \(\mathcal{D}_{16}\). From Round 20 to Round 52, this disturbance vector has 16 disturbances, thus the complexity of a characteristic which is derived from it is approximated by \(2^{60}\).

Table 26 The disturbance vector of 53-round SHA-1

The reduced version of SHA-1 with the largest number of rounds we could attack with a complexity less than a generic attack is SHA-1 reduced to 58 rounds. This attack is a two-block attack, and its characteristic is derived from the disturbance vector which is given in Table 27. We note that the disturbance vectors of 53-round and 58-round SHA-1 are derived from the same stream of the LFSR (the latter starts at the consequent three rounds of the former). This disturbance vector has consecutive disturbances in three locations: bit 29 of \(\mathcal{D}_7\) and \(\mathcal{D}_8\), bit 31 of \(\mathcal{D}_{12}\) and \(\mathcal{D}_{13}\) and bit 31 of \(\mathcal{D}_{13}\) and \(\mathcal{D}_{14}\). From Round 20 it has 17 disturbances and the approximated complexity of a two-block attack that uses characteristics which are derived from this disturbance vector, is \(2^{78}\).

Table 27 The disturbance vector of 58-round SHA-1

Advances in Cryptanalysis of Hash Functions

At the rump session of CRYPTO 2004 two novel techniques for the cryptanalysis of hash functions were presented by Wang et al. The first technique applies differential cryptanalysis by a simultaneous usage of subtraction and XOR differences. This technique called precise characteristic is used to construct a characteristic that possibly has a low probability in the first rounds, and relatively high probability in later rounds. Given such a characteristic, the second technique shows how to use it to construct an efficient attack. This technique called message modification technique constructs pairs of messages that conform to the characteristic up to at least Round \(R\) with a low complexity (even though the probability of the characteristic in these rounds may be very low). Hence, the complexity of the attack is affected only by the relatively high probability in rounds beyond \(R\).

These techniques are applicable to any hash function of the MD4 and SHA families. Wang demonstrated their usefulness on MD4 (in [35]), MD5 (in [37]), RIPEMD-128 (in [35]), SHA-0 (in [38]), SHA-1 (in [39] and [40]), and HAVAL (in [41]).

In the context of SHA-0 and SHA-1 the construction of a characteristic starts with a selection of a disturbance vector. Some of the candidate disturbance vectors have low Hamming weights in rounds beyond Round 20, thus a high probability characteristic may be derived. However, they also have disturbances at \(\mathcal{D}_{-5},\ldots ,\mathcal{D}_{-1}\) (that represent differences in the initial values), or consecutive disturbances at the first 17 rounds. In [7] we showed techniques that locally use the non-linear behavior of the addition \(\mathop {\hbox {mod}}\nolimits 2^{32}\) and IF operations to make some of these disturbance vectors usable. Wang et al. show how the analysis by subtraction and XOR differences facilitates the usage of the non-linear addition \(\mathop {\hbox {mod}}\nolimits 2^{32}\) and IF operations such that in practice most disturbance vectors may be selected. In particular, they show how to use disturbance vectors which were considered unusable by previous techniques.

Given a disturbance vector with disturbances that represent differences in the initial values or consecutive disturbances, it is shown how to derive a characteristic. In the first rounds (usually Rounds 0,...,9) the differences of the messages cannot generate differences in the reduced state vector that have the form of local collision sequences. Therefore, the non-linear behavior of the addition \(\mathop {\hbox {mod}}\nolimits 2^{32}\) and IF operations are used to control the differences of the state vector. In these rounds the subtraction and XOR differences of the characteristic represent these controlled differences. The usage of the non-linear behavior of the functions imposes many conditions on the values of certain bits, and substantially reduce the probability of the characteristic. Therefore, the characteristic is designed to minimize the number of rounds where the differences of the reduced state vector are controlled by the non-linear behavior of the functions.

Typically, starting around Round \(10\) (and up to the last round) the differences of the reduced state follow the disturbance vector, and the differences of the characteristic are defined accordingly. The analysis of subtraction and XOR differences in these rounds facilitate the selection of differences that give an overall high probability, and in particular higher than the probabilities of characteristics that use only XOR differences. The complete characteristic is expressed in [39] and [40] as conditions on values of message bits, state bits, and their differences.

Given a characteristic, a collision search starts by satisfying the conditions on the message bits. The remaining bits of the 16 words (that were not involved in the message-bit conditions) are randomly selected, and the conditions on the state bits are gradually satisfied from Round 0 to Round 15 by the message modification technique. The message modification technique complements a message bit for each unsatisfied condition. Advanced message modification technique is applied in Round \(16\) and up to Round 26. In this technique a control path is defined to correct each unsatisfied condition. A control path consists of one or more message and state bits, whose complementation corrects some yet unsatisfied condition, and does not affect any already satisfied conditions. The claimed complexity of these controlled corrections of all the conditions up to Round 25, is two SHA-1 calls [40]. From round 26 to the last round the remaining conditions are satisfied at random (and their success is not controlled by the attacker).

The complexity of an attack that uses these techniques is determined by two factors: The complexity of generating a pair of messages that follows the characteristic up to Round 26, and the number of conditions beyond Round 26. The attack is a two-block attack. The characteristics of the first and second blocks are constructed from the same disturbance vector. The claimed total complexity of [40] is \(2^{63}\).

In [13] De Cannière and Rechberger present an algorithm to find high probability characteristics of SHA-1. Their idea is based on an algorithm that estimates the complexity of a given characteristic, which is then used as a starting point to find more complex characteristics that minimize the complexity. They used their ideas and found a collision of SHA-1 reduced to 64 rounds with a complexity \(2^{32}\). In [14] De Cannière, Mendel and Rechberger described a better characteristic, and used it to find a collision of SHA-1 reduced to 70 rounds.

In 2007 the neutral bits technique was enhanced by Joux and Peyrin in [23]. In their paper they show how the amplified boomerang attack [34] of block ciphers is adapted to cryptanalyze hash functions using ideas from the neutral bits technique. The enhancement in the context of SHA-1 is done by selecting a pair of messages that conforms to \(R\) rounds, and modify the pair by complementing certain groups of bits which are called auxiliary differentials. These auxiliary differentials are used to correct unsatisfied conditions at rounds beyond Round \(R\). The claimed improvement to the attack complexity on SHA-1 is by a factor of \(2^{-5}\).

The techniques of De Cannière et al. [13, 14] were improved in 2010 by Grechnikov [21]. His improvements include speed optimization and an improvement in the search for characteristics. With these improvements he was able to find 2-block collisions of SHA-1 reduced to 72 and 73 rounds. In 2011 Grechnikov and Adinetz [22] optimized the search technique of [21] to run on a GPU cluster, and they found a collision of SHA-1 reduced to 75 rounds. We note that both [21] and [22] report that their results were received earlier than expected. We also note that we had a similar experience in our attacks.

In [11] Chen introduces a characteristic and a collision search called second-order differential that leads to a collision of SHA-1 reduced to 72 rounds with a complexity of \(2^{50}\) SHA-1 calls. It is a 2-block attack in which the first block equals in both runs, and the differences are in the second message block. This attack can be extended to a 3-block attack on the full SHA-1 in which the same disturbance vector (of the 72-round attack) is used to construct the characteristics of the second and third blocks. The selection of this disturbance vector was made such that it takes into account the dependencies of the local collision sequences and it is one of a few optimal disturbance vectors. The complexity of this attack is estimated to be \(2^{58}\) calls of SHA-1.

In [33] Stevens analyzes the dependency of local collision sequences (which were assumed independent in most previous attacks). He introduces techniques that enable to determine the theoretical maximum success probability for a given set of local collisions, and the smallest set of message conditions that attains this probability. Using these techniques a near-collision attack of SHA-1 was found with a claimed complexity equivalent to \(2^{57.5}\) SHA calls. This near-collision may be used to find a collision of SHA-1 with a complexity \(2^{61}\).

Summary

This paper presents various techniques for the cryptanalysis of hash functions. The usefulness of these techniques is demonstrated on SHA-0 and SHA-1, but they are applicable to other hash functions as well.

The neutral bits technique shows that a poor avalanche effect of a round function leads to the elimination of the probabilistic behavior of many rounds.

The multi-block technique is applicable to any iterative hash function. In the context of Merkle–Damgård construction it shows that the analysis of both inputs of the compression function (the message block and chaining variable) may result in a much more efficient attack. We also show that a two-block attack in which the same difference is used in the first and second block, is usually the most efficient.

In the case of SHA-0 we show that the collision resistance of a compression function is not monotonous with the number of rounds. We conclude that adding rounds to a compression function might result in a weaker function.

The discussion on consecutive disturbances reveals some weaknesses of the non-linear IF, MAJORITY, and addition modulo \(2^{32}\) functions. Since these functions are used in many other algorithms, these techniques might be found useful to attack other algorithms as well.

Some of the techniques for the cryptanalysis of hash functions we discussed in this paper are also useful for the cryptanalysis of stream ciphers. We expect that with some adaptations they will be useful for the cryptanalysis of block ciphers as well.

Notes

  1. 1.

    SHA-0 and SHA-1 have 80 rounds in their compression functions.

  2. 2.

    This requirement is not applicable to rounds \(i+3\) and \(i+4\) since at that rounds bit 31 is active, and the carry is not considered.

  3. 3.

    Note that the resolution of this issue by Wang et al. [38] enables the usage of disturbance vectors with much higher probabilities, and as a result it reduced the complexity of the attack by a factor of about \(2^{20}\).

  4. 4.

    In [3] we used the term simultaneous neutral set. The term is replaced to convey both the ideas that it is one neutral bit and that it is made of several bit positions.

  5. 5.

    The search is terminated when the effort of finding a larger set exceeds the effort of testing all the candidates of the set.

References

  1. 1.

    J.-P. Aumasson, S. Fischer, S. Khazaei, W. Meier, C. Rechberger, New features of Latin dances, in FSE 2008. LNCS, vol. 5086 (Springer, Berlin, 2008), pp. 470–488

  2. 2.

    E. Biham, New results on SHA-0 and SHA-1, Stafford Tavares invited lecture in SAC 2004. http://www.cs.technion.ac.il/~biham/Reports/Slides/invited-talk-sac-2004.ps.gz

  3. 3.

    E. Biham, R. Chen, Near-collisions of SHA-0, in Advances in Cryptology, Proceedings of CRYPTO 2004. LNCS, vol. 3152 (Springer, Berlin, 2004), pp. 290–305

  4. 4.

    E. Biham, R. Chen, New results on SHA-0 and SHA-1, in CRYPTO 2004 Rump Session

  5. 5.

    E. Biham, A. Shamir, Differential cryptanalysis of Snefru, Khafre, REDOC-II, LOKI and Lucifer, in Advances in Cryptology, Proceedings of CRYPTO 1991. LNCS, vol. 576 (Springer, Berlin, 1992), pp. 156–171

  6. 6.

    E. Biham, A. Shamir, Differential Cryptanalysis of the Data Encryption Standard (Springer, Berlin, 1993)

  7. 7.

    E. Biham, R. Chen, A. Joux, P. Carribault, C. Lemuet, W. Jalby, Collisions of SHA-0 and reduced SHA-1, Advances in Cryptology, Proceedings of EUROCRYPT 2005. LNCS, vol. 3494 (Springer, Berlin, 2005), pp. 36–57

  8. 8.

    B. den Boer, A. Bosselaers, An attack on the last two rounds of MD4, in Advances in Cryptology, Proceedings of CRYPTO 1991. LNCS, vol. 576 (Springer, Berlin, 1992), pp. 194–203

  9. 9.

    B. den Boer, A. Bosselaers, Collision of the compression function of MD5, in Advances in Cryptology, Proceedings of EUROCRYPT 1993. LNCS, vol. 765 (Springer, Berlin, 1994), pp. 293–304

  10. 10.

    F. Chabaud, A. Joux, Differential collisions in SHA-0, in Advances in Cryptology, Proceedings of CRYPTO ’98. LNCS, vol. 1462 (Springer, Berlin, 1999), pp. 56–71

  11. 11.

    R. Chen, New Techniques for Cryptanalysis of Cryptographic Hash Functions, Ph.D. thesis, Technion, 2011. http://www.cs.technion.ac.il/users/wwwb/cgi-bin/tr-get.cgi/2011/PHD/PHD-2011-08.pdf and https://www.iacr.org/phds/index.php?p=detail&entry=651

  12. 12.

    Ivan B. Damgård, A design principle for Hash functions, Advances in Cryptology, Proceedings of CRYPTO 1989. LNCS, vol. 435 (Springer, Berlin, 1990), pp. 416–427

  13. 13.

    C. De Cannière, C. Rechberger, Finding SHA-1 characteristics: general results and applications, in Advances in Cryptology, Proceedings of ASIACRYPT 2006. LNCS, vol. 4284 (Springer, Berlin, 2006), pp. 1–20

  14. 14.

    C. De Cannière, F. Mendel, C. Rechberger, Collisions for 70-Step SHA-1: on the full cost of collision search, in Advances in Cryptology, Proceedings of SAC 2007. LNCS, vol. 4876 (Springer, Berlin, 2007), pp. 56–73

  15. 15.

    H. Dobbertin, Cryptanalysis of MD4. J. Cryptol. 11, 253–271 (1998)

  16. 16.

    H. Dobbertin, Cryptanalysis of MD5 compress, in EUROCRYPT 1996 Rump Session

  17. 17.

    RIPE, Integrity primitives for secure information systems, in Final Report of RACE Integrity Primitives Evaluation (RIPE Race 1040). LNCS, vol. 1040 (Springer, Berlin, 1995)

  18. 18.

    H. Dobbertin, A. Bosselaers, B. Preneel, RIPEMD-160: a strengthened version of RIPEMD, in Proceedings of Fast Software Encryption. LNCS, vol. 1039 (Springer, Berlin, 1996), pp. 71–82

  19. 19.

    S. Fischer, S. Khazaei, W. Meier, Chosen IV statistical analysis for key recovery attacks on stream ciphers, in AFRICACRYPT 2008. LNCS, vol. 5023 (Springer, Berlin, 2008), pp. 236–245

  20. 20.

    A. Joux, Collisions in SHA-0, in CRYPTO 2004 Rump Session

  21. 21.

    E.A. Grechnikov, Collisions for 72-step and 73-step SHA-1: improvements in the method of characteristics. Cryptology, ePrint Archive 2010/413

  22. 22.

    E.A. Grechnikov, A.V. Adinetz, Collision for 75-step SHA-1: intensive parallelization with GPU. Cryptology, ePrint Archive 2011/641

  23. 23.

    A. Joux, T. Peyrin, Hash functions and the (amplified) Boomerang attack, in Advances in Cryptology, Proceedings of CRYPTO 2007. LNCS, vol. 4622 (Springer, Heidelberg, 2007), pp. 244–263

  24. 24.

    S. Khazaei, W. Meier, New directions in cryptanalysis of self-synchronizing stream ciphers, in INDOCRYPT 2008. LNCS, vol. 5365. (Springer, Berlin, 2008), pp 15–26

  25. 25.

    R. Merkle, One-way Hash function and DES, in Advances in Cryptology, Proceedings of CRYPTO 1989. LNCS, vol. 435 (Springer, Berlin, 1990), pp. 428–446

  26. 26.

    R. Merkle, A fast software one-way Hash function. J. Cryptol. 3(1), 43–58 (1990)

  27. 27.

    S. Miyaguchi, K. Ohta, M. Iwata, 128-bit hash function (N-Hash), in Proceedings of SECURICOM’90, March 1990, pp. 123–137

  28. 28.

    National Institute of Standards and Technologies, Secure Hash standard, in Federal Information Processing Standards, FIPS-180 (U.S. Department of Commerce, Washington, 1993)

  29. 29.

    National Institute of Standards and Technologies, Secure Hash standard, in Federal Information Processing Standards, FIPS-180-1 (U.S. Department of Commerce, Washington, 1995)

  30. 30.

    V. Rijmen, E. Oswald, Update on SHA-1, in RSA Crypto Track 2005. LNCS, vol. 3376 (Springer, Heidelberg, 2005), pp. 58–71

  31. 31.

    R. Rivest, The MD4 message-digest algorithm, in Advances in Cryptology, Proceedings of CRYPTO 1990. LNCS, vol. 537 (Springer, Berlin, 1990), pp. 303–311

  32. 32.

    R. Rivest, The MD5 message-digest algorithm, in Network Working Group Request for Comments: 1321, April 1992

  33. 33.

    M. Stevens, New collision attacks on SHA-1 based on optimal joint local-collision analysis, in Proceedings of EUROCRYPT 2013. LNCS, vol. 7881 (Springer, Berlin, 2013), pp. 245–261

  34. 34.

    D. Wagner, The Boomerang attack, in Advances in Cryptology, Proceedings of FSE 1999. LNCS, vol. 1636 (Springer, Berlin, 1999), pp. 156–170

  35. 35.

    X. Wang, X. Lai, H. Chen, X. Yu, Cryptanalysis of the Hash functions MD4 and RIPEMD, in Advances in Cryptology, Proceedings of EUROCRYPT 2005. LNCS, vol. 3494 (Springer, Berlin, 2005), pp. 1–18

  36. 36.

    X. Wang, D. Feng, X. Lai, H. Yu, Collisions for Hash functions MD4, MD5, in HAVAL-128 and RIPEMD. http://eprint.iacr.org/2004/199

  37. 37.

    X. Wang, H. Yu, How to break MD5 and other Hash functions, in Advances in Cryptology, Proceedings of EUROCRYPT 2005. LNCS, vol. 3494 (Springer, Berlin, 2005), pp. 19–35

  38. 38.

    X. Wang, H. Yu, Y.L. Yin, Efficient collision search attacks on SHA-0, in Advances in Cryptology, Proceedings of CRYPTO 2005. LNCS, vol. 3621 (Springer, Berlin, 2005), pp. 1–16

  39. 39.

    X. Wang, H. Yu, Y.L. Yin, Finding collisions in the full SHA-1, in Advances in Cryptology, Proceedings of CRYPTO 2005. LNCS, vol. 3621 (Springer, Berlin, 2005), pp. 17–36

  40. 40.

    X. Wang, A.C. Yao, F. Yao, Cryptanalysis on SHA-1, Presented by Adi Shamir at CRYPTO 2005 rump session. http://csrc.nist.gov/groups/ST/hash/documents/Wang_SHA1-New-Result.pdf

  41. 41.

    H. Yu, X. Wang, A. Yun, S. Park, Cryptanalysis of the full HAVAL with 4 and 5 passes, in Advances in Cryptology, Proceedings of FSE 2006. LNCS, vol. 4047 (Springer, Berlin, 2006), pp. 89–110

  42. 42.

    Y. Zheng, J. Pieprzyk, J. Sebbery, HAVAL—a one-way algorithm with variable length of output, in Asiacrypt 1992. LNCS, vol. 718 (Springer, Berlin, 1993), pp. 83–104

Download references

Acknowledgments

This research was supported in part by the Israel MOD research and Technology unit.

Author information

Affiliations

Authors

Corresponding author

Correspondence to Rafi Chen.

Additional information

This work was supported in part by the Israel MOD Research and Technology Unit.

Communicated by Preneel

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Biham, E., Chen, R. & Joux, A. Cryptanalysis of SHA-0 and Reduced SHA-1. J Cryptol 28, 110–160 (2015). https://doi.org/10.1007/s00145-014-9179-8

Download citation

Keywords

  • Differential cryptanalysis
  • SHA-1
  • Hash functions