Interactive Oracle Proofs

Abstract

We initiate the study of a proof system model that naturally combines interactive proofs (IPs) and probabilistically-checkable proofs (PCPs), and generalizes interactive PCPs (which consist of a PCP followed by an IP). We define an interactive oracle proof (IOP) to be an interactive proof in which the verifier is not required to read the prover’s messages in their entirety; rather, the verifier has oracle access to the prover’s messages, and may probabilistically query them. IOPs retain the expressiveness of PCPs, capturing NEXP rather than only PSPACE, and also the flexibility of IPs, allowing multiple rounds of communication with the prover. IOPs have already found several applications, including unconditional zero knowledge [BCGV16], constant-rate constant-query probabilistic checking [BCG+16], and doubly-efficient constant-round IPs for polynomial-time bounded-space computations [RRR16].

We offer two main technical contributions. First, we give a compiler that maps any public-coin IOP into a non-interactive proof in the random oracle model. We prove that the soundness of the resulting proof is tightly characterized by the soundness of the IOP against state restoration attacks, a class of rewinding attacks on the IOP verifier that is reminiscent of, but incomparable to, resetting attacks.

Second, we study the notion of state-restoration soundness of an IOP: we prove tight upper and lower bounds in terms of the IOP’s (standard) soundness and round complexity; and describe a simple adversarial strategy that is optimal, in expectation, across all state restoration attacks.

Our compiler can be viewed as a generalization of the Fiat–Shamir paradigm for public-coin IPs (CRYPTO ’86), and of the “CS proof” constructions of Micali (FOCS ’94) and Valiant (TCC ’08) for PCPs. Our analysis of the compiler gives, in particular, a unified understanding of these constructions, and also motivates the study of state restoration attacks, not only for IOPs, but also for IPs and PCPs.

When applied to known IOP constructions, our compiler implies, e.g., blackbox unconditional ZK proofs in the random oracle model with quasilinear prover and polylogarithmic verifier, improving on a result of [IMSX15].

Parts of this paper appear in the third author’s master’s thesis (April 2015) in the Department of Computer Science at ETH Zurich, supervised by Alessandro Chiesa and Thomas Holenstein. Independent of our work, [RRR16] introduce the notion of Probabilistically Checkable Interactive Proofs, which is the same as our notion of Interactive Oracle Proofs.

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

1 Introduction

The notion of proof is central to modern cryptography and complexity theory. The class $\mathsf {NP}$, for example, is the set of languages whose membership can be decided by a deterministic polynomial-time verifier by reading proof strings of polynomial length; this class captures the traditional notion of a mathematical proof. Over the last three decades, researchers have introduced and studied proof systems that generalize the above traditional notion, and investigations from these points of view have led to breakthroughs in cryptography, hardness of approximation, and other areas. In this work we introduce and study a new model of proof system.

1.1 Models of Proof Systems

We give some context by recalling three of the most well-known among alternative models of proof systems.

Interactive Proofs (IPs). Interactive proofs were introduced by Goldwasser, Micali, and Rackoff [GMR89]: in a $k$-round interactive proof, a probabilistic polynomial-time verifier exchanges $k$ messages with an all-powerful prover, and then accepts or rejects; $\mathsf {IP}[k]$ is the class of languages with a $k$-round interactive proof. Independently, Babai [Bab85] introduced Arthur–Merlin games: a $k$-round Arthur–Merlin game is a $k$-round public-coin interactive proof (i.e., the verifier messages are uniformly and independently random); $\mathsf {AM}[k]$ is the class of languages with a $k$-round Arthur–Merlin game. Goldwasser and Sipser [GS86] showed that the two models are equally powerful: for polynomial $k$, $\mathsf {IP}[k] \subseteq \mathsf {AM}[k+2]$. Shamir [Sha92], building on the “sum-check” interactive proof of Lund, Fortnow, Karloff, and Nisan [LFKN92], proved that interactive proofs correspond to languages decidable in polynomial space: . (Also see [Bab90].)

Multi-prover Interactive Proofs (MIPs). Multi-prover interactive proofs were introduced by Ben-Or, Goldwasser, Kilian, and Wigderson [BGKW88]: in a $k$-round $p$-prover interactive proof, a probabilistic polynomial-time verifier interacts $k$ times with $p$ non-communicating all-powerful provers, and then accepts or rejects; $\mathsf {MIP}[p,k]$ is the class of languages that have a $k$-round $p$-prover interactive proof. In [BGKW88], the authors show that two provers always suffice (i.e., $\mathsf {MIP}[p,k]=\mathsf {MIP}[2,k]$), and that all languages in $\mathsf {NP}$ have perfect zero knowledge proofs in this model. Fortnow, Rompel, and Sipser [FRS88] show that interaction with two provers is equivalent to interaction with one prover plus oracle access to a proof string, and from there obtain that ; Babai, Fortnow and Lund [BFL90] show that $\mathsf {NEXP}$ has 1-round 2-prover interactive proofs, thus showing that $\mathsf {MIP}[2,1] = \mathsf {NEXP}$.

Probabilistically Checkable Proofs (PCPs). Probabilistically checkable proofs were introduced by [FRS88, BFLS91, AS98, ALM+98]: in a probabilistically-checkable proof, a probabilistic polynomial-time verifier has oracle access to a proof string; $\mathsf {PCP}[r, q]$ is the class of languages for which the verifier uses at most $r$ bits of randomness, and queries at most $q$ locations of the proof (note that the proof length is at most $2^{r}$). The above results on MIPs imply that . Later works “scaled down” this result to $\mathsf {NP}$: Babai, Fortnow, Levin and Szegedy [BFLS91] show that ; Arora and Safra [AS98] show that ; and Arora, Lund, Motwani, Sudan, and Szegedy [ALM+92] show that . This last is known as the PCP Theorem.

Researchers have studied other models of proof systems, and here we name only a few: linear IPs [BCI+13], no-signaling MIPs [IKM09, Ito10, KRR13, KRR14], linear PCPs [IKO07, Gro10, Lip12, BCI+13, GGPR13, PGHR13, BCI+13, SBW11, SMBW12, SVP+12, SBV+13], interactive PCPs [KR08, KR09, GIMS10].

We introduce interactive oracle proofs (IOPs), a model of proof system that combines aspects of IPs and PCPs, and also generalizes interactive PCPs (which consist of a $\mathrm{PCP}$ followed by an $\mathrm{IP}$). Our work focuses on cryptographic applications of this proof system, as we discuss next.

1.2 Compiling Proof Systems into Argument Systems

The proof systems mentioned so far share a common feature: they make no assumptions on the computational resources of a (malicious) prover trying to convince the verifier. Instead, many proof systems make “structural” assumptions on the prover: MIPs assume that the prover is a collection of non-communicating strategies (each representing a “sub-prover”); PCPs assume that the prover is non-adaptive (the answer to a message does not depend on previous messages); linear IPs assume that the prover is a linear function; and so on.

In contrast, in cryptography, one often considers argument systems [BC86, BCC88, Kil92, Mic00]: these are proof systems where soundness holds only against provers that have a bound on computational resources (e.g., provers that run in probabilistic polynomial time). The relaxation from statistical soundness to computational soundness allows circumventing various limitations of IPs [BHZ87, GH98, GVW02, PSSV07], while also avoiding “structural” assumptions on the prover, which can be hard to enforce in applications.

Constructing Argument Systems. A common methodology to construct argument systems with desirable properties (e.g., sublinear communication complexity) follows these two steps: (1) give a proof system that achieves these properties in a model with structural restrictions on (all-powerful) provers; (2) use cryptographic tools to compile that proof system into an argument system, i.e., one where the only restriction on the prover is that it is an efficient algorithm. Thus, the compilation trades any structural assumptions for computational ones. This methodology has been highly productive.

Proofs in the Random Oracle Model. An idealized model for studying computationally-bounded provers is the random oracle model [FS86, BR93], where every party has access to the same random function. A protocol proved secure in this model can potentially be instantiated in practice by replacing the random function with a concrete “random-looking” efficient function. While this intuition fails in the general case [CGH04, BBP04, GK03, BDG+13], the random oracle model is nonetheless a useful testbed for cryptographic primitives. In this paper we focus on proof systems in this model for which the proof consists of a single message from the prover to the verifier. A non-interactive random-oracle argument ($\mathrm{NIROA}$) for a relation $\mathscr {R}$ is a pair of probabilistic polynomial-time algorithms, the prover $\mathbb {P}$ and verifier $\mathbb {V}$, that satisfy the following. (1) Completeness: for every instance-witness pair in the relation $\mathscr {R}$, , where the probability is taken over the random oracle ${\rho }$ as well as any randomness of $\mathbb {P}$ and $\mathbb {V}$. (2) Soundness: for every instance not in the language of $\mathscr {R}$ and every malicious prover $\tilde{\mathbb {P}}$ that asks at most a polynomial number of queries to the random oracle, it holds that is negligible in the security parameter.

Prior NIROAs and Our Focus. Prior work uses the above 2-step methodology to obtain $\mathrm{NIROA}$s with desirable properties. For example, the Fiat–Shamir paradigm maps 3-message public-coin IPs to corresponding $\mathrm{NIROA}$s [FS86, PS96]; when invoked on suitable $\mathrm{IP}$ constructions, this yields efficient zero knowledge non-interactive proofs. As another example, Micali’s “CS proof” construction, building on [Kil92], transforms PCPs to corresponding $\mathrm{NIROA}$s; Valiant [Val08] revisits Micali’s construction and proves that it is a proof of knowledge; when invoked on suitable PCPs, these yield non-interactive arguments of knowledge that are short and easy to verify. In this work we study the question of how to compile IOPs (which generalize IPs and PCPs) into $\mathrm{NIROA}$s;¹ our work ultimately leads to formulating and studying a game-theoretic property of IOPs, which in turn motivates similar questions for IPs and PCPs. We now discuss our results.

1.3 Results

We present three main contributions: one is definitional and the other two are technical in nature.

Interactive Oracle Proofs A New Proof System Model. We introduce a new proof system model: interactive oracle proofs (IOPs).² This model naturally combines aspects of IPs and PCPs, and also generalizes IPCPs (see comparison in Remark 1 below); namely, an $\mathrm{IOP}$ is a “multi-round $\mathrm{PCP}$” that generalizes an interactive proof as follows: the verifier has oracle access to the prover’s messages, and may probabilistically query them (rather than having to read them in full). In more detail, a $k$-round $\mathrm{IOP}$ comprises $k$ rounds of interaction. In the i-th round of interaction: the verifier sends a message $m_{i}$ to the prover, which he reads in full; then the prover replies with a message $f_{i}$ to the verifier, which he can query, as an oracle proof string, in this and all later rounds. After the $k$ rounds of interaction, the verifier either accepts or rejects.

Like the $\mathrm{PCP}$ model, two fundamental measures of efficiency in the $\mathrm{IOP}$ model are the proof length $p$, which is the total number of bits in all of the prover’s messages, and the query complexity $q$, which is the total number of locations queried by the verifier across all of the prover’s messages. Unlike the $\mathrm{PCP}$ model, another fundamental measure of efficiency is the round complexity $k$; the $\mathrm{PCP}$ model can then be viewed as a special case where $k=1$ (and the first verifier message is empty).

We show that IOPs characterize $\mathsf {NEXP}$ (like PCPs); both sequential and parallel repetition of IOPs yield (perfect) exponential soundness error reduction (like IPs); and any $\mathrm{IOP}$ can be converted into a public-coin one (like IPs). These basic complexity-theoretic properties confirm that our definition of $\mathrm{IOP}$ is a natural way to combine aspects of PCPs and IPs, and to generalize IPCPs.

Motivation: Efficiency. IOPs extend IPs, by treating the prover’s messages as oracle strings, and PCPs, by allowing for more than 1 round. These additional degrees of freedom enable IOPs to retain the expressive power of $\mathrm{PCP}$ while also allowing for additional efficiency, as already demonstrated in several works.

For example, [BCGV16] obtain unconditional zero knowledge via a 2-round $\mathrm{IOP}$ with quasilinear proof length; such a result is not known for PCPs (or even IPCPs [KR08]). Moreover, when combined with our compiler (see next contribution) we obtain blackbox unconditional zero-knowledge with quasilinear prover and polylogarithmic verifier in the random-oracle model, improving prover runtime of [IMSX15, Sect. 2.3];

As another example, [BCG+16] obtain 3-round IOPs for circuit satisfiability with linear proof length and constant query complexity, while for PCPs prior work only achieves sublinear query complexity [BKK+13]. To do so, [BCG+16] show that sumcheck [LFKN92, Sha92] and proof composition [AS98] (used in many $\mathrm{PCP}$ constructions such as [ALM+98, HS00, BGH+04]) have more efficient “$\mathrm{IOP}$ analogues”, which in turn imply a number of probabilistic checking results that are more efficient than corresponding ones that only rely on PCPs. We briefly sketch the intuition for why interactive proof composition, via IOPs, is more efficient. In a composed proof, the prover first writes a part $\pi _{0}$ of the proof (e.g., in [ALM+98] $\pi _{0}$ is an evaluation of a low-degree multivariate polynomial, and in [BS08] it is an evaluation of a low-degree univariate polynomial). Then, to demonstrate that $\pi _{0}$ has certain good properties (e.g., it is low degree), the prover also appends a (long) sequence of sub-proofs, where each sub-proof allegedly demonstrates to the verifier that a subset of entries of $\pi _{0}$ is “good”. Afterwards, in another invocation of the recursion, the prover appends to each sub-proof a sequence of sub-sub-proofs, and so on. A crucial observation is that the verifier typically queries locations of only a small number of such sub-proofs; moreover, once the initial proof $\pi _{0}$ is fixed, soundness is not harmed if the verifier randomly selects the set of sub-proofs he wants to see and tells this to the prover. In sum, in many $\mathrm{PCP}$ constructions (including the aforementioned ones), the proof length can be greatly reduced via interaction between the prover and verifier, via an $\mathrm{IOP}$.

As yet another example, [RRR16] use IOPs to obtain doubly-efficient constant-round IPs for polynomial-time bounded-space computations. The result relies on an “amortization theorem” for IOPs that states that, for a so-called unambiguous IOPs, batch verification of multiple statements can be more efficient than simply running an independent $\mathrm{IOP}$ for each statement.

Remark 1

(comparison with IPCP). Kalai and Raz [KR08] introduce and study interactive PCPs (IPCPs), a model of proof system that also combines aspects of IPs and PCPs, but in a different way: an $\mathrm{IPCP}$ is a $\mathrm{PCP}$ followed by an $\mathrm{IP}$, i.e., the prover sends to the verifier a $\mathrm{PCP}$ and then the prover and verifier engage in an interactive proof. An $\mathrm{IPCP}$ can be viewed as a special case of an $\mathrm{IOP}$, i.e., it is an $\mathrm{IOP}$ in which the verifier has oracle access to the first prover message, but must read in full subsequent prover messages. The works of [KR08, GKR08] show that boolean formulas with n variables, size m, and depth d have IPCPs where the $\mathrm{PCP}$’s size is polynomial in d and n and the communication complexity of the subsequent $\mathrm{IP}$ is polynomial in d and $\log m$. This shows that even IPCPs give efficiency advantages over both IPs and PCPs given separately.

From Interactive Oracle Proofs to Non-interactive Random-Oracle Arguments. We give a polynomial-time transformation that maps any public-coin interactive oracle proof ($\mathrm{IOP}$) to a corresponding non-interactive random-oracle argument ($\mathrm{NIROA}$). We prove that the soundness of the output proof is tightly characterized by the soundness of the $\mathrm{IOP}$ verifier against state restoration attacks, a class of rewinding attacks on the verifier that we now describe.

At a high level, a state restoration attack against an $\mathrm{IOP}$ verifier works as follows: the malicious prover and the verifier start interacting, as they normally would in an $\mathrm{IOP}$; at any moment, however, the prover can choose to set the verifier to any state at which the verifier has previously been, and the verifier then continues onwards from that point with fresh randomness. Of course, if the prover could restore the verifier’s state an unbounded number of times, the prover would eventually succeed in making the verifier accept. We thus only consider malicious provers that interact with the verifier for at most a certain number of rounds: for $b\in \mathbb {N}$, we say a prover is $b$ -round if it plays at most $b$ rounds during any interaction with any verifier. Then, we say that an $\mathrm{IOP}$ has state restoration soundness if every $b$-round state-restoring prover cannot make the $\mathrm{IOP}$ verifier accept an instance (not in the language) with probability greater than . This notion is reminiscent of, but incomparable to, the notion of resettable soundness [BGGL01]; see Remark 2 below.

Informally, our result about transforming IOPs into $\mathrm{NIROA}$s can be stated as follows.

Theorem 1

(IOP $\rightarrow$ NIROA). There exists a polynomial-time transformation $T$ such that, for every relation $\mathscr {R}$, if ${(P ,V)}$ is a public-coin interactive oracle proof system for $\mathscr {R}$ with state restoration soundness , then $(\mathbb {P} ,\mathbb {V}) := T(P ,V)$ is a non-interactive random-oracle argument system for $\mathscr {R}$ with soundness

where $m$ is an upper bound on the number of queries to the random oracle that a malicious prover can make, and $\lambda$ is a security parameter. The aforementioned soundness is tight up to small factors. (Good state restoration soundness can be obtained, e.g., via parallel repetition as in Remark 4.)

Moreover, we prove that the transformation $T$ is benign in the sense that it preserves natural properties of the $\mathrm{IOP}$. Namely, (1) the runtimes of the $\mathrm{NIROA}$ prover and verifier are linear in those of the $\mathrm{IOP}$ prover and verifier (up to a polynomial factor in $\lambda$); (2) the $\mathrm{NIROA}$ is a proof of knowledge if the $\mathrm{IOP}$ is a proof of knowledge (and the extractor strategy straight-line, which has desirable properties [BW15]); and (3) the $\mathrm{NIROA}$ is (malicious-verifier) statistical zero knowledge if the $\mathrm{IOP}$ is honest-verifier statistical zero knowledge.³ See Theorem 3 for the formal statement; the statement employs the notion of restricted state restoration soundness as it allows for a tighter lower bound on soundness.

An immediate application is obtained by plugging the work of [BCGV16] into our compiler, thereby achieving a variant of the black-box ZK results of [IMSX15, Sect. 2.3] where the prover runs in quasilinear (rather than merely polynomial) time.

Corollary 1

(informal). There is a blackbox non-interactive argument system for $\mathsf {NP}$, in the random-oracle model, with unconditional zero knowledge, quasilinear-time prover, and polylogarithmic-time verifier.

Our compiler can be viewed as a generalization of the Fiat–Shamir paradigm for public-coin IPs [FS86, PS96], and of the “CS proof” constructions of Micali [Mic00] and Valiant [Val08] for PCPs. Our analysis of the compiler gives, in particular, a unified understanding of these constructions, and motivates the study of state restoration attacks, not only for IOPs, but also for IPs and PCPs. (Indeed, we are not aware of works that study the security of the Fiat–Shamir paradigm, in the random oracle model, applied to a public-coin $\mathrm{IP}$ with arbitrary number of rounds; the analyses that we are aware of focus on the case of 2 rounds.)

Our next contribution is a first set of results about such kinds of attacks, as described in the next section.

Remark 2

(resetting, backtracking). We compare state restoration soundness with other soundness notions:

• State restoration attacks are reminiscent of, but incomparable to, resetting attacks [BGGL01]. In the latter, the prover invokes multiple verifier incarnations with independent randomness, and may interact multiple times with each incarnation; also, this notion does not assume that the verifier is public-coin. Instead, in a state restoration attack, the verifier must be public-coin and its randomness is not fixed at the start but, instead, a new fresh random message is sampled each time the prover restores to a previously-seen state.

• State restoration is closely related to backtracking [BD16] (independent work). The two notions differ in that: (1) backtracking “charges” more for restoring verifier states that are further in the past, and (2) backtracking also allows the verifier to restore states of the prover (as part of the completeness property of the protocol); backtracking soundness is thus polynomially related to state restoration soundness. Bishop and Dodis [BD16] give a compiler from a public-coin $\mathrm{IP}$ to an error-resilient $\mathrm{IP}$, whose soundness is related to the backtracking soundness of the original $\mathrm{IP}$; essentially, they use hashing techniques to limit a malicious prover impersonating an adversarial channel to choosing when to backtrack the protocol. Their setting is a completely different example in which backtracking, and thus state restoration, plays a role.

Remark 3

(programmability). As in most prior works, soundness and proof of knowledge do not rely on programming the random oracle. As for zero knowledge, the situation is more complicated: there are several notions of zero knowledge in the random oracle model, depending on “how programmable” the random oracle is (see [Wee09]). The notion that we use is zero knowledge in the explicitly-programmable random oracle (EPRO) model; the stronger notion in the non-programmable random oracle model is not achievable for $\mathrm{NIROA}$s. Such a limitation can sometimes be avoided by also using a common random string [Lin15, CPSV16], and extending such techniques to the setting of IOPs is an interesting problem.

State Restoration Attacks on Interactive Oracle Proofs. The analysis of our transformation from public-coin IOPs to $\mathrm{NIROA}$s highlights state restoration soundness as a notion that merits further study. We provide two results in this direction. First, we prove tight upper and lower bounds on state restoration soundness in terms of the $\mathrm{IOP}$’s (standard) soundness and round complexity.

Theorem 2

For any relation $\mathscr {R}$, public-coin $k$-round $\mathrm{IOP}$ for $\mathscr {R}$, and instance not in the language of $\mathscr {R}$,

⁴where is the state restoration soundness of $\mathrm{IOP}$ and its (standard) soundness for the instance . Also, the bounds are tight: there are IOPs that meet the lower bound and IOPs that meet the upper bound.

Remark 4

(good state restoration soundness). A trivial way to obtain state restoration soundness $2^{-\lambda }$ in the general case is to apply $r$-fold parallel repetition to the $\mathrm{IOP}$ with ; note that $r$ is polynomially bounded for natural choices of $k,b,\lambda$. This choice of $r$ is pessimistic, because for IOPs that do not meet the upper bound (i.e., are “robust” against such attacks) a smaller $r$ suffices. This use of parallel repetition is analogous to its use in achieving the incomparable notion of resettable soundness [PTW09, COPV13].

Second, we study the structure of optimal state restoration attacks: we prove that, for any public-coin $\mathrm{IOP}$, there is a simple state restoration attack that has optimal expected cost, where cost is the number of rounds until the prover wins. This result relies on a correspondence that we establish between $\mathrm{IOP}$ verifiers and certain games, which we call tree exploration games, pitting one player against Nature. We go in more detail about this result in later sections (see Sect. 1.4 and full version [BCS16].). A better understanding of state restoration soundness may enable us to avoid trivial soundness amplification (see Remark 4) for IOPs of interest.

1.4 Techniques

We summarize the techniques that we use to prove our technical contributions.

The Transformation. Our transformation maps any public-coin $\mathrm{IOP}$ to a corresponding $\mathrm{NIROA}$, and it generalizes two transformations that we now recall.

The first transformation is the Fiat–Shamir paradigm [FS86, PS96], which maps any public-coin $\mathrm{IP}$ to a corresponding $\mathrm{NIROA}$, and it works as follows. The $\mathrm{NIROA}$ prover runs the interaction between the $\mathrm{IP}$ prover and the $\mathrm{IP}$ verifier “in his head”, by setting the $\mathrm{IP}$ verifier’s next message to be the output of the random oracle on the query that equals the transcript of previously exchanged messages. The $\mathrm{NIROA}$ prover sends a non-interactive proof that contains the final transcript of interaction; the $\mathrm{NIROA}$ verifier checks the proof’s validity by checking that all the $\mathrm{IP}$ verifier’s messages are computed correctly via the random oracle.

The second transformation is the “CS proof” construction of Micali [Mic00] and Valiant [Val08], which maps any $\mathrm{PCP}$ to a corresponding $\mathrm{NIROA}$, and it works as follows. The $\mathrm{NIROA}$ prover first commits to the $\mathrm{PCP}$ via a Merkle tree [Mer89a] based on the random oracle, then queries the random oracle with the root of this tree to obtain randomness for the $\mathrm{PCP}$ verifier, and finally sends a non-interactive proof that contains the root as well as authentication paths for each query by the $\mathrm{PCP}$ verifier to the $\mathrm{PCP}$; the $\mathrm{NIROA}$ verifier checks the proof’s validity by checking that the $\mathrm{PCP}$ verifier’s randomness is computed correctly through the random oracle, and that all authentication paths are valid. (The transformation can be viewed as a non-interactive variant of Kilian’s protocol [Kil92, BG08] that uses ideas from the aforementioned Fiat–Shamir paradigm.)

Our transformation takes as input IOPs, for which both IPs and PCPs are special cases, and hence must support both (i) multiple rounds of interaction between the $\mathrm{IOP}$ prover and $\mathrm{IOP}$ verifier, as well as (ii) oracle access by the $\mathrm{IOP}$ verifier to the $\mathrm{IOP}$ prover messages. Given an instance , the $\mathrm{NIROA}$ prover thus uses the random oracle ${\rho }$ to run the interaction between the $\mathrm{IOP}$ prover and the $\mathrm{IOP}$ verifier “in his head” in a way that combines the aforementioned two approaches, as follows. First, the $\mathrm{NIROA}$ prover computes an initial value . Then, for $i=1,2,\dots$, it simulates the i-th round by deriving the $\mathrm{IOP}$ verifier’s i-th message $m_{i}$ as , compressing the $\mathrm{IOP}$ prover’s i-th message $f_{i}$ via a Merkle tree to obtain the root $\mathsf {rt}_{i}$, and computing the new value $\sigma _{i} := {\rho }(\mathsf {rt}_{i} \Vert \sigma _{i-1})$. The values $\sigma _{0},\sigma _{1},\dots$ are related by the Merkle–Damgård transform [Dam89, Mer89b] that, intuitively, enforces ordering between rounds. If there are rounds of interaction, then is used as randomness for the queries to . The $\mathrm{NIROA}$ prover provides in the non-interactive proof all the roots $\mathsf {rt}_{i}$, the final value , the answers to the queries, and an authentication path for each query. This sketch omits several details; see Sect. 5.

Soundness Analysis of the Transformation. We prove that the soundness of the $\mathrm{NIROA}$ produced by the above transformation is tightly characterized by the state restoration soundness of the underlying $\mathrm{IOP}$. This characterization comprises two arguments: an upper bound and a lower bound on the $\mathrm{NIROA}$’s soundness. We only discuss the upper bound here: proving that the soundness (error) of the $\mathrm{NIROA}$ is at most the soundness (error) of the $\mathrm{IOP}$ against state restoration attacks, up to small additive factors.

The upper bound essentially implies that all that a malicious prover $\tilde{\mathbb {P}}$ can do to attack the $\mathrm{NIROA}$ verifier is to conduct a state restoration attack against the underlying $\mathrm{IOP}$ verifier “in his own head”: roughly, $\tilde{\mathbb {P}}$ can provide multiple inputs to the random oracle in order to induce multiple fresh samples of verifier messages for a given round so to find a lucky one, or instead go back to previous rounds and do the same there.

In more detail, the proof itself relies on a reduction: given a malicious prover $\tilde{\mathbb {P}}$ against the $\mathrm{NIROA}$ verifier, we show how to construct a corresponding malicious prover $\tilde{P}$ that conducts a state restoration attack against the underlying $\mathrm{IOP}$ verifier. We prove that the winning probability of $\tilde{P}$ is essentially the same as that of $\tilde{\mathbb {P}}$; moreover, we also prove that the reduction preserves the resources needed for the attack in the sense that if $\tilde{\mathbb {P}}$ asks at most $m$ queries to the random oracle, then $\tilde{P}$ plays at most $m$ rounds during the attack.

Intuitively, the construction of $\tilde{P}$ in terms of $\tilde{\mathbb {P}}$ must use some form of extraction: $\tilde{\mathbb {P}}$ outputs a non-interactive proof that contains only (i) the roots that (allegedly) are commitments to underlying $\mathrm{IOP}$ prover’s messages, and (ii) answers to the $\mathrm{IOP}$ verifier’s queries and corresponding authentication paths; in contrast, $\tilde{P}$ needs to actually output these $\mathrm{IOP}$ prover’s messages. In principle, the malicious prover $\tilde{\mathbb {P}}$ may not have “in mind” any underlying $\mathrm{IOP}$ prover, and we must prove that, nevertheless, there is a way for $\tilde{P}$ to extract some $\mathrm{IOP}$ prover message for each round that convince the verifier with the claimed probability.

Our starting point is the extractor algorithm of Valiant [Val08] for the “CS proof” construction of Micali [Mic00]: Valiant proves that Micali’s $\mathrm{NIROA}$ construction is a proof of knowledge by exhibiting an algorithm, let us call it Valiant’s extractor, that recovers the underlying $\mathrm{PCP}$ whenever the $\mathrm{NIROA}$ prover convinces the $\mathrm{NIROA}$ verifier with sufficient probability. (In particular, our proof is not based on a “forking lemma” [PS96].) Our setting differs from Valiant’s in that the $\mathrm{IOP}$ prover $\tilde{P}$ obtained from the $\mathrm{NIROA}$ prover $\tilde{\mathbb {P}}$ needs to be able to extract multiple times, “on the fly”, while interacting with the $\mathrm{IOP}$ verifier; this more complex setting can potentially cause difficulties in terms of extractor size (e.g., if relying on rewinding the $\mathrm{NIROA}$ prover) or correlations (e.g., when extracting multiple times from the same $\mathrm{NIROA}$ prover). We tackle the more complex setting in two steps.

First, we prove an extractability property of Valiant’s extractor and state it as a property of Merkle trees in the random oracle model (see Sect. A.1). Informally, we prove that, except with negligible probability, whenever an algorithm with access to a random oracle outputs multiple Merkle tree roots each accompanied with some number of (valid) authentication paths, it holds that Valiant’s extractor run separately on each of these roots outputs a decommitment that is consistent with each of the values revealed in authentication paths relative to that root. We believe that distilling and proving this extractability property of Valiant’s extractor is of independent interest.

Second, we show how the $\mathrm{IOP}$ prover $\tilde{P}$ can interact with an $\mathrm{IOP}$ verifier, by successively extracting messages to send, throughout the interaction, by invoking Valiant’s extractor multiple times on $\tilde{\mathbb {P}}$ relative to different roots. The $\mathrm{IOP}$ prover $\tilde{P}$ does not rely on rewinding $\tilde{\mathbb {P}}$, and its complexity is essentially that of a single run of $\tilde{\mathbb {P}}$ plus a small amount of work.

Preserving Proof of Knowledge. We prove that the above soundness analysis can be adapted so that, if the underlying $\mathrm{IOP}$ is a proof of knowledge, then we can construct an extractor to show that the resulting $\mathrm{NIROA}$ is also a proof of knowledge. Moreover, the extractor algorithm only needs to inspect the queries and answers of one execution of $\tilde{\mathbb {P}}$ if the underlying IOP extractor does not use rewinding (known $\mathrm{IOP}$ constructions are of this type [BCGV16, BCG+16]); such extractors are known as straight line [Pas03] or online [Fis05], and have very desirable properties [BW15].

Preserving Zero Knowledge. We prove that, if the underlying $\mathrm{IOP}$ is honest-verifier statistical zero knowledge, then the resulting $\mathrm{NIROA}$ is statistical zero knowledge (i.e., is a non-interactive statistical zero knowledge proof in the explicitly-programmable random oracle model). This is because the transformation uses a Merkle tree with suitable privacy guarantees (see Sect. A.2) to construct the $\mathrm{NIROA}$. Indeed, the authentication path for a leaf in the Merkle tree reveals the sibling leaf, so one must ensure that the sibling leaf does not leak information about other values; this follows by letting leaves be commitments to the underlying values. A Merkle tree with privacy is similarly used by [IMS12, IMSX15], along with honest-verifier PCPs, to achieve zero knowledge in modifications of Kilian’s [Kil92, BG08] and Micali’s [Mic00] constructions. (Note that the considerations [HT98, DNRS03, GOSV14] seem to only apply to compilation of malicious-verifier IOPs, which neither [IMS12, IMSX15] nor we require.)

Understanding State Restoration Attacks. We prove tight upper and lower bounds to state restoration soundness in terms of the $\mathrm{IOP}$’s (standard) soundness and round complexity $k$. The upper bound takes the form of a reduction: given a $b$-round state-restoring malicious prover ${\tilde{P}}_{\mathrm {sr}}$ that makes the $\mathrm{IOP}$ verifier accept with probability ${s}_{\mathrm {sr}}$, we construct a (non state-restoring) malicious prover $\tilde{P}$ that makes the $\mathrm{IOP}$ verifier accept with probability at least $\left( {\begin{array}{c}b\\ k+1\end{array}}\right) ^{-1}{s}_{\mathrm {sr}}$. Informally, $\tilde{P}$ internally simulates ${\tilde{P}}_{\mathrm {sr}}$, while interacting with the “real” $\mathrm{IOP}$ verifier, as follows: $\tilde{P}$ first selects a random subset $S$ of $\{1, \dots , b\}$ with cardinality $k+1$, and lets $S[i]$ be the i-th smallest value in $S$; then, $\tilde{P}$ runs ${\tilde{P}}_{\mathrm {sr}}$ and simulates its state restoration attack on a “virtual” $\mathrm{IOP}$ verifier, executing round j (a) by interacting with the real verifier if $j = S[i]$ for some i; (b) by sampling fresh randomness otherwise. While this reduction appears wasteful (since it relies on $S$ being a good guess), we show that there are IOPs for which the upper bound is tight. In other words, the sharp degradation as a function of round complexity (for large $b$, $\left( {\begin{array}{c}b\\ k+1\end{array}}\right) \approx b^{k+1}/(k+1)!$) is inherent for some choices of IOPs; this also gives a concrete answer to the intuition that compiling IOPs with large round complexity to $\mathrm{NIROA}$s is “harder” (i.e., incurs in a greater soundness loss) than for IOPs with small round complexity. As for the lower bound on state restoration soundness, it takes the form of a universal state restoration attack that always achieves the lower bound; this bound is also tight.

While state restoration soundness may be far, in the worst case, from (standard) soundness for IOPs with large round complexity, it need not always be far. We thus investigate state restoration soundness for any particular $\mathrm{IOP}$, and derive a simple attack strategy (which depends on the $\mathrm{IOP}$) that we prove has optimal expected cost, where cost is the number of rounds until the prover wins. To do so, we “abstract away” various details of the proof system to obtain a simple game-theoretic notion, which we call tree exploration games, that pits a single player against Nature in reaching a node of a tree with label 1. Informally, such a game is specified by a rooted tree $T$ and a predicate function $\phi$ that maps $T$’s vertices to $\{0,1\}$. The game proceeds in rounds: in the i-th round, a subtree $S_{i-1} \subseteq T$ is accessible to the player; the player picks a node $v\in S_{i-1}$, and Nature randomly samples a child $u$ of $v$; the next accessible subtree is $S_{i} := S_{i-1} \cup \{u\}$. The initial $S_{0}$ is the set consisting of $T$’s root vertex. The player wins in round $r$ if there is $v\in S_{r}$ with $\phi (v) = 1$.

We establish a correspondence between state restoration attacks and strategies for tree exploration games, and then show a simple greedy strategy for such games with optimal expected cost. Via the correspondence, a strategy’s cost determines whether the underlying $\mathrm{IOP}$ is strong or weak against sate restoration attacks.

2 Preliminaries

2.1 Basic Notations

We denote the security parameter by $\lambda$. For $f :\{0,1\}^{*}\rightarrow \mathbb {R}$, we define $\hat{f} :\mathbb {N}\rightarrow \mathbb {R}$ as $\hat{f}(n) := \max _{x \in \{0,1\}^{n}} f(x)$.

Languages and Relations. We denote by $\mathscr {R}$ a relation consisting of pairs , where is the instance and is the witness, and by the restriction of $\mathscr {R}$ to instances of size $n$. We denote by $\mathscr {L}(\mathscr {R})$ the language corresponding to $\mathscr {R}$. For notational convenience, we define .

Random Oracles. We denote by $\mathcal {U}(\lambda )$ the uniform distribution over all functions ${\rho }:\{0,1\}^{*} \rightarrow \{0,1\}^{\lambda }$ (implicitly defined by the probabilistic algorithm that assigns, uniformly and independently at random, a $\lambda$-bit string to each new input). If ${\rho }$ is sampled from $\mathcal {U}(\lambda )$, then we write ${\rho }\leftarrow \mathcal {U}(\lambda )$ and say that ${\rho }$ is a random oracle. Given an oracle algorithm $A$, $\mathsf {NumQueries}(A,{\rho })$ is the number of oracle queries that $A^{{\rho }}$ makes. We say that $A$ is $m$ -query if $\mathsf {NumQueries}(A,{\rho }) \le m$ for any ${\rho }\in \mathcal {U}(\lambda )$ (i.e., for any ${\rho }$ in $\mathcal {U}(\lambda )$’s support).

Statistical Distance. The statistical distance between two discrete random variables X and Y with support V is ${{\mathrm{\varDelta }}}(X;Y) := \frac{1}{2} \sum _{v \in V} |\Pr [X = v] - \Pr [Y = v]|$. We say that X and Y are $\delta$ -close if ${{\mathrm{\varDelta }}}(X;Y) \le \delta$.

Remark 5

An oracle ${\rho }\in \mathcal {U}(\lambda )$ outputs $\lambda$ bits. Occasionally we need ${\rho }$ to output more than $\lambda$ bits; in such cases (we point out where), we implicitly extend ${\rho }$’s output via a simple strategy, e.g., we set $y := y_{1} \Vert y_{2} \Vert \cdots$ where $y_{i} := {\rho }(i \Vert x)$ and prefix 0 to all inputs that do not require an output extension.

2.2 Merkle Trees

We use Merkle trees [Mer89a] based on random oracles as succinct commitments to long lists of values for which one can cheaply decommit to particular values in the list. Concretely, a Merkle-tree scheme is a tuple $\mathsf {MERKLE}= (\mathsf {MERKLE}\mathsf {.GetRoot} ,\mathsf {MERKLE}\mathsf {.GetPath},\mathsf {MERKLE}\mathsf {.CheckPath})$ that uses a random oracle ${\rho }$ sampled from $\mathcal {U}(\lambda )$ and works as follows.

• $\mathsf {MERKLE}\mathsf {.GetRoot}^{{\rho }}(\mathbf {v}) \rightarrow \mathsf {rt}$. Given input list $\mathbf {v}=(v_{i})_{i=1}^{n}$, the root generator $\mathsf {MERKLE}\mathsf {.GetRoot}$ computes, in time $O_{\lambda }(n)$, a root $\mathsf {rt}$ of the Merkle tree over $\mathbf {v}$.

• $\mathsf {MERKLE}\mathsf {.GetPath}^{{\rho }}(\mathbf {v}, i) \rightarrow \mathsf {ap}$. Given input list $\mathbf {v}$ and index $i$, the authentication path generator $\mathsf {MERKLE}\mathsf {.GetPath}$ computes the authentication path $\mathsf {ap}$ for the $i$-th value in $\mathbf {v}$.

• $\mathsf {MERKLE}\mathsf {.CheckPath}^{{\rho }}(\mathsf {rt},i,v,\mathsf {ap}) \rightarrow b$. Given root $\mathsf {rt}$, index $i$, input value $v$, and authentication path $\mathsf {ap}$, the path checker $\mathsf {MERKLE}\mathsf {.CheckPath}$ outputs $b=1$ if $\mathsf {ap}$ is a valid path for $v$ as the $i$-th value in a Merkle tree with root $\mathsf {rt}$; the check can be carried out in time $O_{\lambda }(\log _{2} n)$.

We assume that an authentication path $\mathsf {ap}$ contains the root $\mathsf {rt}$, position $i$, and value $v$; accordingly, we define $\mathsf {Root}(\mathsf {ap}) := \mathsf {rt}$, $\mathsf {Position}(\mathsf {ap}) := i$, and $\mathsf {Value}(\mathsf {ap}) :=~v$.

Merkle trees are well known, so we do not review their construction. Less known, however, are the hiding and extractability properties of Merkle trees that we rely on in this work; we describe these in Appendix A.

2.3 Non-interactive Random-Oracle Arguments

A non-interactive random-oracle argument system for a relation $\mathscr {R}$ with soundness $s:\{0,1\}^{*}\rightarrow {[0,1]}$ is a tuple $(\mathbb {P} ,\mathbb {V})$, where $\mathbb {P},\mathbb {V}$ are (oracle) probabilistic algorithms, that satisfies the following properties.

• 1. Completeness. For every and $\lambda \in \mathbb {N}$,

  

• 2. Soundness. For every , $m$-query $\tilde{\mathbb {P}}$, and $\lambda \in \mathbb {N}$,

  

Complexity Measures. Beyond soundness, we consider other complexity measures. Given $p:\{0,1\}^{*}\rightarrow \mathbb {N}$, we say that $(\mathbb {P} ,\mathbb {V})$ has proof length $p$ if $\pi$ has length . Given $t_{\mathrm {prv}},t_{\mathrm {ver}}:\{0,1\}^{*}\rightarrow \mathbb {N}$, we say that $(\mathbb {P} ,\mathbb {V})$ has prover time complexity $t_{\mathrm {prv}}$ and verifier time complexity $t_{\mathrm {ver}}$ if runs in time and runs in time . In sum, we say that $(\mathbb {P} ,\mathbb {V})$ has complexity $(s ,p,t_{\mathrm {prv}},t_{\mathrm {ver}})$ if $(\mathbb {P} ,\mathbb {V})$ has soundness $s$, proof length $p$, prover time complexity $t_{\mathrm {prv}}$, and verifier time complexity $t_{\mathrm {ver}}$.

Proof of Knowledge. Given $e:\{0,1\}^{*}\rightarrow {[0,1]}$, we say that $(\mathbb {P} ,\mathbb {V})$ has proof of knowledge $e$ if there exists a probabilistic polynomial-time algorithm $\mathbb {E}$ (the extractor) such that, for every , $m$-query $\tilde{\mathbb {P}}$, and $\lambda \in \mathbb {N}$,

The notation means that $\mathbb {E}$ receives as input and may obtain an output of $\tilde{\mathbb {P}}^{{\rho }}$ for choices of oracles ${\rho }$, as we now describe. At any time, $\mathbb {E}$ may send a $\lambda$-bit string z to $\tilde{\mathbb {P}}$; then $\tilde{\mathbb {P}}$ interprets z as the answer to its last query to ${\rho }$ (if any) and then continues computing until it reaches either its next query $\theta$ or its output $\pi$; then this query or output is sent to $\mathbb {E}$ (distinguishing the two cases in some way); in the latter case, $\tilde{\mathbb {P}}$ goes back to the start of its computation (with the same randomness and any auxiliary inputs). Throughout, the code, randomness, and any auxiliary inputs of $\tilde{\mathbb {P}}$ are not available to $\mathbb {E}$.

Zero Knowledge. Given $z:\{0,1\}^{*}\rightarrow {[0,1]}$, we say that $(\mathbb {P} ,\mathbb {V})$ has $z$-statistical zero knowledge (in the explicitly-programmable random oracle model) if there exists a probabilistic polynomial-time algorithm $\mathbb {S}$ (the simulator) such that, for every and unbounded distinguisher $D$, the following two probabilities are -close:

Above, ${\rho }{}[\mu ]$ is the function such that, given an input x, equals $\mu (x)$ if $\mu$ is defined on x, or ${\rho }(x)$ otherwise.

3 Interactive Oracle Proofs

We first define interactive oracle protocols and then interactive oracle proof systems.

3.1 Interactive Oracle Protocols

A $k$-round interactive oracle protocol between two parties, call them Alice and Bob, comprises $k$ rounds of interaction. In the i-th round of interaction: Alice sends a message $m_{i}$ to Bob, which he reads in full; then Bob replies with a message $f_{i}$ to Alice, which she can query (via random access) in this and all later rounds. After the $k$ rounds of interaction, Alice either accepts or rejects.

More precisely, let $k$ be in $\mathbb {N}$ and $A,B$ be two interactive probabilistic algorithms. A $k$-round interactive oracle protocol between $A$ and $B$, denoted $\langle B,A \rangle$, works as follows. Let $r_{A},r_{B}$ denote the randomness for $A,B$ and, for notational convenience, set $f_{0}:=\bot$ and $\mathsf {state}_{0}:=\bot$. For $i=1,\dots ,k$, in the i-th round: (i) Alice sends a message $m_{i} \in \{0,1\}^{u_{i}}$, where $(m_{i},\mathsf {state}_{i}) := A^{f_{0},\dots ,f_{i-1}}(\mathsf {state}_{i-1};r_{A})$ and $u_{i} \in \mathbb {N}$; (ii) Bob sends a message $f_{i} \in \{0,1\}^{\ell _{i}}$, where $f_{i} := B(m_{1},\dots ,m_{i};r_{B})$ and $\ell _{i} \in \mathbb {N}$. The output of the protocol is $m_{\mathsf {fin}} := A^{f_{0},\dots ,f_{k}}(\mathsf {state}_{k};r_{A})$, and belongs to $\{0,1\}$.

The accepting probability of $\langle B,A \rangle$ is the probability that $m_{\mathsf {fin}} =1$ for a random choice of $r_{A},r_{B}$; this probability is denoted $\Pr [\langle B,A \rangle =1]$ (leaving $r_{A},r_{B}$ implicit). The query complexity of $\langle B,A \rangle$ is the number of queries asked by $A$ to any of the oracles during the $k$ rounds. The proof complexity of $\langle B,A \rangle$ is the number of bits communicated by Bob to Alice (i.e., $\sum _{i=1}^{k} \ell _{i}$). The view of $A$ in $\langle B,A \rangle$, denoted $\mathrm {View}_{\langle B,A \rangle }(A)$, is the random variable $(a_{1},\dots ,a_{q},r_{A})$ where $a_{j}$ denotes the answer to the j-th query.

Public Coins. An interactive oracle protocol is public-coin if Alice’s messages are uniformly and independently random and Alice postpones any query to after the $k$-th round (i.e., all queries are asked when running $A^{f_{0},\dots ,f_{k}}(\mathsf {state}_{k};r_{A})$). We can thus take the randomness $r_{A}$ to be of the form $(m_{1},\dots ,m_{k},r)$, where $r$ is additional randomness that $A$ may use of to compute $m_{\mathsf {fin}}$ after the last round.

3.2 Interactive Oracle Proof Systems

An interactive oracle proof system for a relation $\mathscr {R}$ with round complexity $k:\{0,1\}^{*}\rightarrow \mathbb {N}$ and soundness $s:\{0,1\}^{*}\rightarrow {[0,1]}$ is a tuple $(P ,V)$, where $P,V$ are probabilistic algorithms, that satisfies the following properties.

• 1. Completeness. For every , is a -round interactive oracle protocol with accepting probability 1.

• 2. Soundness. For every and $\tilde{P}$, is a -round interactive oracle protocol with accepting probability at most .

Message Lengths. We assume the existence of polynomial-time functions that determine the message lengths. Namely, for any instance and malicious prover $\tilde{P}$, when considering the interactive oracle protocol , the i-th messages $m_{i}$ (from ) and $f_{i}$ (to ) lie in and respectively.

Complexity Measures. Beyond round complexity and soundness, we consider other complexity measures. Given $p,q:\{0,1\}^{*}\rightarrow \mathbb {N}$, we say that $(P ,V)$ has proof length $p$ and query complexity $q$ if the proof length and query complexity of are and respectively. (Note that and .) Given $t_{\mathrm {prv}},t_{\mathrm {ver}}:\{0,1\}^{*}\rightarrow \mathbb {N}$, we say that $(P ,V)$ has prover time complexity $t_{\mathrm {prv}}$ and verifier time complexity $t_{\mathrm {ver}}$ if runs in time and runs in time . In sum, we say that $(P ,V)$ has complexity $(k ,s,p,q,t_{\mathrm {prv}},t_{\mathrm {ver}})$ if $(P ,V)$ has round complexity $k$, soundness $s$, proof length $p$, query complexity $q$, prover time complexity $t_{\mathrm {prv}}$, and verifier time complexity $t_{\mathrm {ver}}$.

Proof of Knowledge. Given $e:\{0,1\}^{*}\rightarrow {[0,1]}$, we say that $(P ,V)$ has proof of knowledge $e$ if there exists a probabilistic polynomial-time oracle algorithm $E$ (the extractor) such that, for every and $\tilde{P}$, .⁵ The notation means that $E$ receives as input and may interact with $\tilde{P}$ via rewinding, as we now describe. At any time, $E$ may send a partial prover-verifier transcript to $\tilde{P}$ and then receive $\tilde{P}$’s next message (which is empty for invalid transcripts) in the subsequent computation step; the code, randomness, and any auxiliary inputs of $\tilde{P}$ are not available to  $E$.

Honest-Verifier Zero Knowledge. Given $z:\{0,1\}^{*}\rightarrow {[0,1]}$, we say that $(P ,V)$ has $z$-statistical honest-verifier zero knowledge if there exists a probabilistic polynomial-time algorithm $S$ (the simulator) such that, for every , is -close to .

Public Coins. We say that $(P ,V)$ is public-coin if the underlying interactive oracle protocol is public-coin.

4 State Restoration Attacks on Interactive Oracle Proofs

We introduce state restoration attacks on interactive oracle proofs.

In an interactive oracle proof, a malicious prover $\tilde{P}$ works as follows: for each round i, $\tilde{P}$ receives the i-th verifier message $m_{i}$ and then sends to the verifier a message $f_{i}$ computed as a function of his own randomness and all the verifier messages received so far, i.e., $m_{1},\dots ,m_{i}$.

For the case of public-coin interactive oracle proof systems, we also consider a larger class of malicious provers, called state-restoring provers. Informally, a state-restoring prover receives in each round a verifier message as well as a complete verifier state, and then sends to the verifier a message and a previously-seen complete verifier state, which sets the verifier to that state; this forms a state restoration attack on the verifier.

More precisely, let $(P ,V)$ be a $k$-round public-coin interactive proof system (see Sect. 3.2) and an instance. A complete verifier state $\mathsf {cvs}$ of takes one of three forms: (1) the symbol $\mathsf {null}$, which denotes the “empty” complete verifier state; (2) a tuple of the form , with , where each $m_{j}$ is in and each $f_{j}$ is in ; (3) a tuple of the form where each $m_{j}$ and $f_{j}$ is as in the previous case and $r$ is the additional randomness of the verifier .

The interaction between a state-restoring prover $\tilde{P}$ and the verifier is mediated through a game:

Note that there are two distinct notions of a round. Verifier rounds are the rounds played by the verifier within a single execution, as tracked by a complete verifier state $\mathsf {cvs}$; the number of such rounds lies in the set (the extra -th round represents the verifier $V$ sampling $r$ after receiving the last prover message). Prover rounds are all verifier rounds played by the prover across different verifier executions; the number of such rounds is the number of states in $\mathsf {SeenStates}$ above. Accordingly, for $b\in \mathbb {N}$, we say a prover is $b$ -round if it plays at most $b$ prover rounds during any interaction with any verifier.

Also note that the prover is not able to set the verifier to arbitrary states but only to previously-seen ones (starting with the empty state $\mathsf {null}$); naturally, setting the verifier multiple times to the same state may yield distinct new states, because the verifier samples his message afresh each time. After being set to a state $\mathsf {cvs}$, the verifier does one of three things: (i) if the number of verifier rounds in $\mathsf {cvs}$ is less than (see Step 2c and Step 2d), the verifier samples a fresh next message; (ii) if the number of verifier rounds in $\mathsf {cvs}$ is (see Step 2e), the verifier samples his additional randomness $r$; (iii) if $\mathsf {cvs}$ contains a full protocol execution (see Step 2f), the verifier outputs the decision corresponding to this execution. The second case means that the prover can set the verifier even after the conclusion of the execution (after $r$ is sampled and known to the prover). The game halts only in the third case.

The above game between a state-restoring prover and a verifier yields corresponding notions of soundness and proof of knowledge. Below, we denote by the probability that the state-restoring prover $\tilde{P}$ makes $V$ accept in this game.

Definition 1

Given ${s}_{\mathrm {sr}},{e}_{\mathrm {sr}} :\{0,1\}^{*}\rightarrow {[0,1]}$, a public-coin interactive oracle proof system $(P ,V)$ has

• State restoration soundness ${s}_{\mathrm {sr}}$ if, for every and $b$-round state-restoring prover $\tilde{P}$, .

• State restoration proof of knowledge ${e}_{\mathrm {sr}}$ if there exists a probabilistic polynomial-time algorithm ${E}_{\mathrm {sr}}$ (the extractor) such that, for every and $b$-round state-restoring prover $\tilde{P}$, .

Due to space limitations, our bounds on state restoration and our results on the corresponding tree exploration games are in the full version [BCS16].

5 From IOPs to Non-interactive Random-Oracle Arguments

We describe a transformation $T$ such that if $(P ,V)$ is a public-coin interactive oracle proof system for a relation $\mathscr {R}$ then $(\mathbb {P} ,\mathbb {V}) := T(P ,V)$ is a non-interactive random-oracle argument system for $\mathscr {R}$. The transformation $T$ runs in polynomial time: given as input code for $P$ and $V$, it runs in time polynomial in the size of this code and then outputs code for $\mathbb {P}$ and $\mathbb {V}$.

Notation. For convenience, we split the random oracle ${\rho }$ into two random oracles, denoted ${\rho _{1}}$ and ${\rho _{2}}$, as follows: ${\rho _{1}}(x) := {\rho }(1 \Vert x)$ and ${\rho _{2}}(x) := {\rho }(2 \Vert x)$. At a high level, we use ${\rho _{1}}$ for the verifier’s randomness, and ${\rho _{2}}$ for Merkle trees and other hashing purposes. When counting queries, we count queries to both ${\rho _{1}}$ and ${\rho _{2}}$.

6 Analysis of the Transformation $T$

The theorem below specifies guarantees of the transformation $T$, described in Sect. 5.

Theorem 3

(IOP $\rightarrow$ NIROA). For every relation $\mathscr {R}$, if $(P ,V)$ is a public-coin interactive oracle proof system for $\mathscr {R}$ with

then $(\mathbb {P} ,\mathbb {V}) := T(P ,V)$ is a non-interactive random-oracle argument system for $\mathscr {R}$ with

By construction, if has accepting probability $\delta$, then the probability that accepts is $\delta$. The complexities $p',t_{\mathrm {prv}}',t_{\mathrm {ver}}'$ above also directly follow from the construction. Therefore, we are left to discuss soundness. Due to space limitations, the discussion of the soundness lower bound, as well as proof of knowledge and zero knowledge, are left to the full version [BCS16].

Let and let $\tilde{\mathbb {P}}$ be an $m$-query prover for the non-interactive random-oracle argument system $(\mathbb {P} ,\mathbb {V})$. We construct a prover $\tilde{P}$ (depending on and $\tilde{\mathbb {P}}$) for the interactive oracle proof system $(P ,V)$, and show that $\tilde{P}$’s ability to cheat in a (restricted) state restoration attack is closely related to $\tilde{\mathbb {P}}$’s ability to cheat.

Construction of ${\varvec{\tilde{P}}}$ . Given no inputs or oracles, the prover $\tilde{P}$ works as follows.

• 1. Let ${\rho _{1}},{\rho _{2}}$ be tables mapping $\{0,1\}^{*}$ to $\{0,1\}^{\lambda }$, and let $\alpha$ be a table mapping $\lambda$-bit strings to verifier states. The tables are initially empty and are later populated with suitable values, during the simulation of $\tilde{\mathbb {P}}$. Intuitively, ${\rho _{1}},{\rho _{2}}$ are used to simulate $\tilde{\mathbb {P}}$’s access to a random oracle, while $\alpha$ is used to keep track of which verifier states $\tilde{\mathbb {P}}$ has “seen in his mind”.

• 2. Draw $\sigma _{0} \in \{0,1\}^{\lambda }$ at random, and define (i.e., the oracle ${\rho _{2}}$ replies the query with the answer $\sigma _{0}$). After receiving $V$’s first message $m_{1}$, also define and $\alpha (\sigma _{0}) := (m_{1})$.

• 3. Begin simulating $\tilde{\mathbb {P}}^{{\rho }}$ and, for $i=1,\dots ,m$:

  • (a) Let $\theta _{i}$ be the i-th query made by $\tilde{\mathbb {P}}^{{\rho }}$.

  • (b) If $\theta _{i}$ is a query to a location of ${\rho _{1}}$ that is defined, respond with ${\rho _{1}}(\theta _{i})$. Otherwise (if $\theta _{i}$ to an undefined location of ${\rho _{1}}$), draw a string in $\{0,1\}^{\lambda }$ at random and respond with it. Then go to the next iteration of Step 3.

  • (c) If $\theta _{i}$ is a query to a location of ${\rho _{2}}$ that is defined, respond with ${\rho _{2}}(\theta _{i})$; then go to the next iteration of Step 3. Otherwise (if $\theta _{i}$ is to an undefined location of ${\rho _{2}}$), draw a string $\sigma ' \in \{0,1\}^{\lambda }$ at random and respond with it; then continue as follows.

  • (d) Let $\mathsf {rt}$ be the first $\lambda$ bits of $\theta _{i}$, and $\sigma$ be the second $\lambda$ bits. (If the length of $\theta _{i}$ is not $2\lambda$ bits, go to the next iteration of Step 3.) If $\alpha (\sigma )$ is defined, let $\mathsf {cvs}:= \alpha (\sigma )$ and let j be the number of verifier rounds in the state $\mathsf {cvs}$. If $\alpha (\sigma )$ is not defined, go to the next iteration of Step 3 .

  • (e) Find the query $\theta _{{i^{\star }}}$ whose result is $\mathsf {rt}$. If this query is not unique, or there is no such query, then answer the verifier $V$ with some dummy message (e.g., an all zero message of the correct length) and skip to Step 3g. Otherwise, note the index ${i^{\star }}$ and continue.

  • (f) Compute ; if $\mathsf {VE}$ aborts, set . Recall that is the length of the prover message in the j-th verifier round, and $\mathsf {VE}$ is Valiant’s extractor (see Sect. A.1). Also note that $\mathsf {VE}$ does not query ${\rho _{2}}$ on any value outside the table, because we have already simulated the first i queries of $\tilde{\mathbb {P}}$ (see Remark 6).

  • (g) Send the message $f$ to the verifier and tell the game to set the verifier to the state $\mathsf {cvs}$. (Whether $\mathsf {cvs}$ lies in the set $\mathsf {SeenStates}$ is a matter of analysis further below.) If the game is not over, the verifier replies with a new message $m'$. (If , for the purposes of the proof, we interpret $m'$ as the additional randomness $r$.) The game adds $\mathsf {cvs}':=\mathsf {cvs}\Vert f\Vert m'$ to $\mathsf {SeenStates}$. The prover defines and $\alpha (\sigma ') := \mathsf {cvs}'$.

Analysis of ${\varvec{\tilde{P}}}$ . We now analyze $\tilde{P}$. We first prove a simple lemma, and then discuss $\tilde{P}$’s ability to cheat.

Lemma 1

Let $A$ be an $m$-query algorithm. Define:

• 1. $E_{1}$ to be the event that $A^{{\rho _{2}}}$ outputs , , and that satisfy the recurrence and $\sigma _{i} = {\rho _{2}}(\mathsf {rt}_{i} \Vert \sigma _{i-1})$ for all ;

• 2. $E_{2}$ to be the event that $A^{{\rho _{2}}}$ queries ${\rho _{2}}$ at (in order) and, if any $\mathsf {rt}_{i}$ is the result of a query, this query first occurs before $\mathsf {rt}_{i} \Vert \sigma _{i-1}$.

Then

$$\begin{aligned} \Pr \left[ (\lnot E_{1}) \vee E_{2} \mid {\rho _{2}} \leftarrow \mathcal {U}(\lambda ) \right] \ge 1 - (m^{2} + 1)2^{-\lambda } . \end{aligned}$$

Proof

Let $\mathsf {rt}_{0}$ be and $\sigma _{-1}$ be the empty string. Suppose, by contradiction, that $E_{1}$ occurs and $E_{2}$ does not. Then there exists for which at least one of the following holds: (i) $A^{{\rho _{2}}}$ does not query $\mathsf {rt}_{i} \Vert \sigma _{i-1}$; (ii) $A^{{\rho _{2}}}$ queries $\mathsf {rt}_{i+1} \Vert \sigma _{i}$ before it queries $\mathsf {rt}_{i} \Vert \sigma _{i-1}$; (iii) $\mathsf {rt}_{i}$ is the result of a query but this query first occurs after $\mathsf {rt}_{i} \Vert \sigma _{i-1}$. Consider the largest index i for which one of the above holds.

In case (i), the behavior of $A^{{\rho _{2}}}$ is independent of ${\rho _{2}}(\mathsf {rt}_{i} \Vert \tilde{\sigma }_{i-1})$. If , then the output of $A^{{\rho _{2}}}$ equals with probability $2^{-\lambda }$. If , then there is a sequence of queries for which $\tilde{\sigma }_{i} = {\rho _{2}}(\mathsf {rt}_{i} \Vert \tilde{\sigma }_{i-1})$ for and . If this sequence is not unique, then $A^{{\rho _{2}}}$ has found a collision. Otherwise, the unique sequence has $\tilde{\sigma }_{i} = \sigma _{i}$ for each i, which occurs with probability at most $2^{-\lambda }$.

In cases (ii) and (iii), $A^{{\rho _{2}}}$ has found a collision, since $\sigma _{i} = {\rho _{2}}(\mathsf {rt}_{i} \Vert \sigma _{i-1})$. The fraction of oracles ${\rho _{2}}$ for which $A^{{\rho _{2}}}$ finds a collision is at most $m^{2} 2^{-\lambda }$. Overall, the probability that $E_{2}$ does not occur and $E_{1}$ does is, by the union bound, at most $(m^{2} + 1)2^{-\lambda }$.

We now state and prove the lemma about the soundness $s'$ as stated in Theorem 3.

Lemma 2

Define . Then there exists $b\in \mathbb {N}$ with $b\le m$ such that $\tilde{P}$ is a $b$-round state-restoring prover that makes $V$ accept with probability at least ${\epsilon } - 3(m^{2} + 1) 2^{-\lambda }$.

Proof

We first note that $\tilde{P}$ described plays no more than $m$ rounds, because $\tilde{P}$ sends a message to the verifier $V$ only in response to $\tilde{\mathbb {P}}$ making a query. Next, we define some useful notions, and use them to prove three claims which together imply the lemma.

Definition 4

We say ${\rho }\in \mathcal {U}(\lambda )$ is good if

• 1. The verifier accepts relative to where $\pi \leftarrow \tilde{\mathbb {P}}^{{\rho }}$.

• 2. Parsing $\pi$ as and setting , for each , where ${\sigma _{i} := {\rho _{2}}(\tilde{\mathsf {rt}}_{i} \Vert \sigma _{i-1})}$, there exist indices $1 \le j_{1}< \dots < j_{k} \le m$ such that:

  • (a) $\tilde{\mathbb {P}}^{{\rho }}$’s $j_{i}$-th query is to ${\rho _{2}}$ at $\tilde{\mathsf {rt}}_{i} \Vert \sigma _{i-1}$;

  • (b) if $\mathsf {rt}_{i}$ is the result of a query, this query first occurs before $j_{i}$;

  • (c) if $\tilde{\mathbb {P}}^{{\rho }}$ queries ${\rho _{1}}$ at , then this query occurs after query $j_{i}$;

  • (d) if there exists l such that $\mathsf {Root}(\tilde{\mathsf {ap}}_l) = \tilde{\mathsf {rt}}_{i}$, there is a unique (up to duplicate queries) $a_{i}\in \{0,\dots ,j_{i}\}$ such that ${\rho _{2}}(\theta _{a_{i}}) = \tilde{\mathsf {rt}}_{i}$ and, for every ${i_{\mathsf {max}}}\in \{a_{i},\dots ,j_{i}\}$, $\mathbf {v}:= \mathsf {VE}^{{\rho _{2}}}(A, \ell _{i}, a_{i}, {i_{\mathsf {max}}})$ is such that, for all l with $\mathsf {Root}(\tilde{\mathsf {ap}}_l) = \tilde{\mathsf {rt}}_{i}$, $\mathsf {Value}(\tilde{\mathsf {ap}}_l)$ equals the $\mathsf {Position}(\tilde{\mathsf {ap}}_l)$-th value in $\mathbf {v}$; we say $\mathbf {v}$ is extracted at i if this holds.

• 3. .

Definition 5

We say that $\tilde{\mathbb {P}}$ chooses ${\rho }\in \mathcal {U}(\lambda )$ if for every query $\theta$ made by $\tilde{\mathbb {P}}^{{\rho }}$ to its oracle, $\tilde{P}$ supplies it with ${\rho }(\theta )$ (ignoring whether this response comes from $\tilde{P}$ itself or the messages sent by $V$; this choice is fixed for a given ${\rho }$).

Claim 6

$(\tilde{P} ,V)$ chooses ${\rho }\in \mathcal {U}(\lambda )$ uniformly at random.

Whenever the simulation of $\tilde{\mathbb {P}}$ makes a query, $\tilde{P}$ responds consistently, either with a uniformly randomly drawn string of its own, or the uniform randomness provided by $V$. This is equivalent in distribution to drawing ${\rho }$ uniformly at random at the beginning of the protocol. ${\scriptstyle \blacksquare }$

Claim 7

For any choice of randomness such that $\tilde{P}$ chooses a good ${\rho }$, $\tilde{P}$ makes accept with a state restoration attack.

We begin by defining a property of the map $\alpha$.

Definition 8

For $i = 0, \dots , k$, we say that $\alpha$ is correct at i if, immediately before $\tilde{\mathbb {P}}$’s $j_{i+1}$-th query is simulated (for $i = k$, at the end of the simulation), it holds that where for each $l \in \{1, \dots , i\}$, $f_{l}$ is extracted at l (see Condition 2d above), and $\alpha (\sigma _{i}) \in \mathsf {SeenStates}$.

We show by induction that $\alpha$ is correct at i for every $i \in \{0,\dots ,k\}$. First, $\alpha$ is correct at 0 since by construction. Suppose that $\alpha$ is correct at $i-1$. When $\tilde{\mathbb {P}}^{{\rho }}$ queries $\tilde{\mathsf {rt}}_{i} \Vert \sigma _{i-1}$ (i.e., query $\theta _{j_{i}}$), $\tilde{P}$ restores $\alpha (\sigma _{i-1}) \in \mathsf {SeenStates}$. By Condition 2d, $f_{i}$ is extracted at i. In Step 3g, is set to the message (or, similarly, internal randomness) sent by $V$ in this round, which is possible by Condition 2c. The newly stored state is then . This state is stored before query $j_{i+1}$ by Condition 2a, and so $\alpha$ is correct at i.

Hence $\tilde{\mathbb {P}}$ sends a state . Since $\mathbb {V}$’s simulation of $V$ accepts with this state, so does the real $V$ when interacting with $\tilde{\mathbb {P}}$. ${\scriptstyle \blacksquare }$

Claim 9

The probability that ${\rho }\in \mathcal {U}(\lambda )$ is good is at least $\epsilon - 3(m^{2} + 1) 2^{-\lambda }$.

By assumption, the density of oracles satisfying Condition 1 is $\epsilon$. Lemma 1 implies that the density of oracles satisfying Condition 1 but not satisfying Condition 2a, Condition 2b, and Condition 3 is at most $(m^2 + 1)2^{-\lambda }$.⁶ The density of oracles failing to satisfy Condition 2c is at most $m^{2} 2^{-\lambda }$, since this implies a ‘collision’ (in the sense of Lemma 3) between ${\rho _{1}}$ and ${\rho _{2}}$. Finally, the density of oracles satisfying Condition 1, Condition 2a, and Condition 2b, but not Condition 2d is at most $(m^2 + 1)2^{-\lambda }$, by Lemma 3 and Condition 2b (where Condition 2b allows us to restrict the possible values for $a_{i}$ to $0 \le a_{i} < j_{i}$).

By the union bound, the density of good oracles ${\rho }$ is at least $\epsilon - 3(m^{2} + 1) 2^{-\lambda }$. ${\scriptstyle \blacksquare }$

Combining the claims, we deduce that $\tilde{P}$ makes $V$ accept with probability at least $\epsilon - 3(m^{2} + 1) 2^{-\lambda }$ with a state restoration attack. Finally, note that this state restoration attack is restricted because $\tilde{P}$ never requests to set $V$ to the empty verifier state $\mathsf {null}$.

A Extractability and Privacy of Merkle Trees

We describe the specific extractability and privacy properties of Merkle trees that we rely on in this work.

1.1 A.1 Extractability

We rely on a certain extractability property of Merkle trees: there is an efficient procedure for extracting the committed list in a Merkle-tree scheme. We call the procedure Valiant’s extractor, and denote it by $\mathsf {VE}$, because it is described in [Val08]. Our presentation of the extractor and its guarantee differs from [Val08] because our use of it in this work requires “distilling” a more general property; see Lemma 3 below.

The Extractor. For any oracle algorithm $A$, integers $\ell ,{i^{\star }},{i_{\mathsf {max}}}>0$ with ${i^{\star }}\in \{1,\dots ,{i_{\mathsf {max}}}\}$, and ${\rho }$ sampled from $\mathcal {U}(\lambda )$, the procedure $\mathsf {VE}$, given input $(A,\ell ,{i^{\star }},{i_{\mathsf {max}}})$ and with oracle access to ${\rho }$, works as follows.

• 1. Run $A^{{\rho }}$ until it has asked ${i_{\mathsf {max}}}$ unique queries to ${\rho }$ (and abort if $A^{{\rho }}$ asks fewer than ${i_{\mathsf {max}}}$). Along the way, record the queries $\theta _{1},\dots ,\theta _{{i_{\mathsf {max}}}}$ and answers ${\rho }(\theta _{1}),\dots ,{\rho }(\theta _{{i_{\mathsf {max}}}})$, in order and omitting duplicates.

• 2. Parse each query $\theta _{i}$ as $\theta _{i}^{0}\Vert \theta _{i}^{1}$ where $\theta _{i}^{0}$ are the first $\lambda$ bits of $\theta _{i}$ and $\theta _{i}^{1}$ the second $\lambda$ bits. For brevity, we write $z \in \theta _{i}$ if $z = \theta _{i}^{0}$ or $z = \theta _{i}^{1}$. (If a query has length not equal to $2\lambda$, then $z \notin \theta _{i}$ for all z.)

• 3. If there exist indices i, j such that $i \ne j$ and ${\rho }(\theta _{i}) = {\rho }(\theta _{j})$, abort.

• 4. If there exist indices i, j such that $i \le j$ and ${\rho }(\theta _{j}) \in \theta _{i}$, abort.

• 5. Construct a directed graph $G$ with nodes $V= \{\theta _{1}, \dots , \theta _{{i_{\mathsf {max}}}}\}$ and edges $E= \{(\theta _{i}, \theta _{j}) : {\rho }(\theta _{j}) \in \theta _{i}\}$. Note that $G$ is acyclic, every node has out-degree $\le 2$, and $\theta _{1},\dots ,\theta _{{i_{\mathsf {max}}}}$ is a (reverse) topological ordering.

• 6. Output $\mathbf {v}$, the string obtained by traversing in order the first $\ell$ leaf nodes of the depth-$\lceil \log _{2} \ell \rceil$ binary tree rooted at $\theta _{{i^{\star }}}$ and recording the first bit of each node. If any such node does not exist, set this entry to 0.

A sample execution of the extractor is depicted in Fig. 1.

Remark 6

The queries to ${\rho }$ asked by $\mathsf {VE}^{{\rho }}(A,\ell , {i^{\star }},{i_{\mathsf {max}}})$ equals the first ${i_{\mathsf {max}}}$ queries to ${\rho }$ asked by $A^{{\rho }}$ (provided that $A$ does not ask fewer than ${i_{\mathsf {max}}}$ queries). Later on we use this fact.

The Extractor’s Guarantee. We interpret $A$’s output as containing a (possibly empty) list of tuples of the form $(\mathsf {rt}, i, v, \mathsf {ap})$, where $\mathsf {rt}$ is a root, i an index, $v$ a value, and $\mathsf {ap}$ an authentication path.⁷ We define the following events:

• (i) $E_{1}$ is the event that, for each tuple $(\mathsf {rt}, i, v, \mathsf {ap})$ output by $A^{{\rho }}$, $\mathsf {MERKLE}\mathsf {.CheckPath}(\mathsf {rt}, i, v, \mathsf {ap}) = 1$;

• (ii) $E_{2}$ is the event that, for each $\mathsf {rt}\in \{0,1\}^{\lambda }$, there exists $\ell _{\mathsf {rt}} \in \mathbb {N}$ such that if $A^{{\rho }}$ outputs a tuple of the form $(\mathsf {rt}, \cdot , \cdot , \mathsf {ap})$ then $\mathsf {ap}$ is an authentication path having the correct length for a $\ell _{\mathsf {rt}}$-leaf Merkle tree;

• (iii) $E_{3}$ is the event that, for every $\mathsf {rt}\in \{0,1\}^{\lambda }$ such that $A^{{\rho }}$ outputs some tuple of the form $(\mathsf {rt}, \cdot , \cdot , \cdot )$, there is a unique $j_{\mathsf {rt}}\in \{0,\dots ,\mathsf {NumQueries}(A, {\rho })\}$ such that ${\rho }(\theta _{j_{\mathsf {rt}}}) = \mathsf {rt}$ and, for every ${i_{\mathsf {max}}}\in \{j_{\mathsf {rt}},\dots , \mathsf {NumQueries}(A,{\rho })\}$, $\mathbf {v}:= \mathsf {VE}^{{\rho }}(A, \ell _{\mathsf {rt}}, j_{\mathsf {rt}}, {i_{\mathsf {max}}})$ is such that $\mathbf {v}$’s $i$-th entry equals $v_{i}$ for any tuple of the form $(\mathsf {rt}, i, v, \mathsf {ap})$ output by $A^{{\rho }}$.

The extractability property that we rely on is the following.

Lemma 3

Let $A^{{\rho }}$ be a $m$-query algorithm. Then

Proof

Observe the following.

• By the union bound, the probability that there exist indices i, j such that $(i \ne j) \wedge ({\rho }(\theta _{i}) = {\rho }(\theta _{j}))$ or $(i \le j) \wedge ({\rho }(\theta _{j}) \in \theta _{i})$ is at most $m^{2} 2^{-\lambda }$. If this occurs, we say that $A^{{\rho }}$ has found a collision.

• The probability that, for a tuple $(\mathsf {rt}, i, v, \mathsf {ap})$ output by $A^{{\rho }}$ such that $\mathsf {MERKLE}\mathsf {.CheckPath}(\mathsf {rt}, i, v, \mathsf {ap}) = 1$, the authentication path $\mathsf {ap}$ contains a node with no corresponding query is at most $2^{-\lambda }$, since this would mean that $A^{{\rho }}$ has ‘guessed’ the answer to the query. In other words, no matter what strategy $A$ uses to generate the result, if it does not query the oracle on this input then it can perform no better than chance.

Now suppose that $E_{1} \wedge E_{2}$ occurs with probability $\delta$. Then, with probability at least $\delta - (m^{2} + 1)2^{-\lambda }$: (a) for each root $\mathsf {rt}$ output by $A^{{\rho }}$ there is a unique query $\theta _{{i^{\star }}}$ such that ${\rho }(\theta _{{i^{\star }}}) = \mathsf {rt}$; (b) for each root $\mathsf {rt}$ output by $A^{{\rho }}$, if an authentication path $\mathsf {ap}$ claims to have root $\mathsf {rt}$ then $\mathsf {ap}$ appears in the tree rooted at $\theta _{{i^{\star }}}$ in $G$; and (c) the condition in the $\mathsf {VE}$’s Step 3 or Step 4 does not hold. In such a case we may take $j_{\mathsf {rt}} := {i^{\star }}$, and then $\mathsf {VE}^{{\rho }}(A, \ell _{\mathsf {rt}}, j_{\mathsf {rt}}, {i_{\mathsf {max}}})$ outputs a list $\mathbf {v}$ with the desired property. Hence, $\Pr [E_{1} \wedge E_{2} \wedge E_{3}] \ge \delta - (m^{2} + 1)2^{-\lambda }$. The predicate is also satisfied if $\lnot (E_{1} \wedge E_{2})$ occurs, which is the case with probability $1 - \delta$ and is disjoint from $E_{1} \wedge E_{2} \wedge E_{3}$. The lemma follows.

Fig. 1.

A diagram of an execution of Valiant’s extractor $\mathsf {VE}$, with input parameters $\ell = 2$, ${i^{\star }}= 4$, and ${i_{\mathsf {max}}}= 6$.

Fig. 2.

A diagram of the data structure of a Merkle tree with privacy. An authentication path for $v_{2}$ is shaded; the corresponding truncated authentication path is the same minus $r_{2}$ and $v_{2}$.

1.2 A.2 Privacy

We rely not only on the fact that the root $\mathsf {rt}$ of a Merkle tree is hiding, but also on the fact that an authentication path $\mathsf {ap}$ reveals no information about values other than the decommitted one. The latter property can be ensured via a slight tweak of the standard construction of Merkle trees: when committing to a list $\mathbf {v}=(v_{i})_{i=1}^{n}$, the $i$-th leaf is not $v_{i}$ but, instead, is a hiding commitment to $v_{i}$. In our case, we will store the value ${\rho }(v_{i} \Vert r_{i})$ in the $i$-th leaf, where $r_{i} \in \{0,1\}^{2\lambda }$ is drawn uniformly at random; see Fig. 2. (An authentication path for $v_{i}$ then additionally includes $r_{i}$, and path verification is modified accordingly.) In what follows, we regard ${\rho }(v_{i} \Vert r_{i})$ as a leaf, rather than $v_{i}$; moreover, a truncated authentication path $\mathsf {ap}_{i}^{\prime }$ is identical to $\mathsf {ap}_{i}$ except that it does not contain $r_{i}$ or $v_{i}$, and the truncated Merkle tree for $\mathbf {v}$ is $T_{\mathbf {v}}^{\prime } := (\mathsf {ap}_{i}^{\prime })_{1 \le i\le n}$. Note that the same randomness $\mathbf {r} \in \{0,1\}^{2\lambda n}$ is used by $\mathsf {MERKLE}\mathsf {.GetRoot}$ and $\mathsf {MERKLE}\mathsf {.GetPath}$ (to be “in sync”).

We summarize the privacy property of Merkle trees as above via the following definition and lemma.

Definition 2

A Merkle-tree scheme has $z(n, \lambda )$-statistical privacy if there exists a probabilistic polynomial-time simulator $S$ such that, for every list $\mathbf {v}=(v_{i})_{i=1}^{n}$ and unbounded distinguisher $D$, the following two probabilities are $z(n,\lambda )$-close:

and

We make no assumption on the power of the distinguisher $D$ in the definition above. In particular, $D$ may query the random oracle ${\rho }$ at every input, and use the information to attempt to learn $v_{i}$ for some $i\notin I$. For example, for some ${\rho }$, it is the case that $\Pr _{r}[v= 1 \mid {\rho }(v\Vert r) = x] \gg \Pr _{r}[v= 0 \mid {\rho }(v\Vert r) = x]$ for $x = {\rho }(v_{2} \Vert r_{2})$, in which case $D$ can determine $v_{2}$ from $\mathsf {ap}_{1}$ with good accuracy. The next (easy to prove) lemma shows that the probability that $D$ gains a significant statistical advantage in this way (or otherwise) is negligible in $\lambda$.

Lemma 4

There exists a Merkle-tree scheme having $z(n, \lambda )$-statistical privacy with $z(n, \lambda ) := n 2^{-\lambda /4+2}$.