Toward practical encrypted email that supports private, regular-expression searches

Abstract

In this paper, we develop a protocol to enable private regular-expression searches on encrypted data stored at a $\mathsf {server}$. A novelty of the protocol lies in allowing a user to securely delegate an encrypted search query to a $\mathsf {proxy}$, which interacts with the $\mathsf {server}$ where the user’s data are stored encrypted to produce the search result for the user. The privacy of the query and the data are both provably protected against an arbitrarily malicious $\mathsf {server}$ and an honest-but-curious $\mathsf {proxy}$ under rigorous security definitions. We then detail a series of optimizations to our initial design that achieve an order-of-magnitude performance improvement over the original protocol. We demonstrate the practicality of the resulting protocol through measurements of private regular-expression searches on a real-world email dataset.

1 Introduction

The tension between encryption and the ability to perform searches on the underlying plaintext has been a topic of considerable attention in the research community. Numerous designs of so-called searchable encryption schemes have been proposed to address this tension (see Sect. 2 for a detailed discussion), but few have made it to practical use. We believe that this state of affairs is in part due to inflexibility, in the sense that such schemes typically require the document creator to tag it by the keywords on which searches will be supported in the future. Even though one can imagine tagging and encrypting all the words in an document to allow for all searches on any word, anything from a typo in the document to different forms of word stemming will render the search results unsatisfactory. For example, a document tagged with the keyword “meetings” would not be returned in the search results for the queries “meet” or “meeting”. A recent study of user email query patterns [1] showed that many queries that users create are only partial words, and so, substring searching capability is important to provide an adequate user experience. Furthermore, very few searchable encryption schemes offer the capabilities of performing substring, conjunctive, disjunctive, and range queries, and we are aware of none that offers all them at the same time.

In this paper, we develop a protocol to enable private regular-expression searches on encrypted data and a system that demonstrates this capability on encrypted email. Regular-expression searches are a widely adopted search primitive in many languages and programming frameworks¹ (e.g., see [3]) and, for example, suffice to support the search options (including Boolean combinations) offered by the Thunderbird email client for the text and numeric email fields. Our system is thus able to support range queries on the date field and various types of substring queries on the source, destination and subject fields of emails.

Our protocol for regular-expression searching gains computational efficiency by using interaction, in fact requiring data transfer between the searching client and the $\mathsf {server}$ holding the ciphertext of a volume larger than the searchable ciphertext itself. This obviously begs the question of whether a more suitable solution would be to download each email to the client and decrypt it there, to be searched locally. With the widespread use of volume-priced networking (i.e., over cellular data plans), however, neither design is particularly appealing. So, we instead explore a different design in which the $\mathsf {user}$ (e.g., via her mobile device) submits her encrypted regular expression (or suitable representation thereof) to a $\mathsf {proxy}$, which then interacts with the $\mathsf {server}$ hosting the encrypted data using our protocol. After this interaction, the $\mathsf {proxy}$ reports information back to the $\mathsf {user}$ that permits her to determine which files matched, so she can retrieve these files from the $\mathsf {server}$. We stress that the volume of interaction between the $\mathsf {user}$ and the $\mathsf {proxy}$ is independent of the lengths of the ciphertexts stored at the $\mathsf {server}$. Moreover, the $\mathsf {proxy}$ is untrusted for the privacy of the search or the file contents (provided that it does not collaborate maliciously with the $\mathsf {server}$), provably so when the $\mathsf {proxy}$ is honest-but-curious. So, for example, the $\mathsf {proxy}$ could be a machine that is well defended and closely monitored to ensure its integrity, but nevertheless untrusted with the details of the $\mathsf {user}$ ’s searches or the emails they match—as might be the case with a $\mathsf {user}$ ’s professionally maintained work computer, through which she also searches her personal email.

The task of constructing such a protocol to be efficient is, as we found, very challenging. Starting from a protocol that implements the above functionality, we detail a series of optimizations that resulted in an optimized protocol with more than an order-of-magnitude improvement in the performance, while enjoying similar security properties. At a high level, the optimizations involve careful redesign of the protocol in order to take advantage of well-known algebraic optimization techniques (e.g., preprocessing to optimize pairing operations) and a few novel algebraic techniques to reduce the online computational costs. After detailing these protocol optimizations, we then explore additional optimizations that leverage specifics of the email setting. These optimizations pertain to the specific regular-expression alphabet that should be utilized for each type of searchable field (i.e., source email address, sender name, date, and subject).

We detail an implementation of our protocol and its performance when searching emails from a real-world email dataset. We show, for example, that our implementation incurs average latencies of 0.89 s per email for performing a nine-character substring search on the sender email address field and 0.17 s per email for performing a range query spanning about six months on the email date field (These numbers were obtained from a $\mathsf {proxy}$ and $\mathsf {server}$ each having eight 2.67 GHz cores with simultaneous multithreading enabled). We also evaluate options for exploiting parallelism with our protocol, ranging from very coarse (i.e., one $\mathsf {server}$ thread and one $\mathsf {proxy}$ thread per $\mathsf {server}$–$\mathsf {proxy}$ protocol instance, but running many protocols instances in parallel) to very fine (i.e., many $\mathsf {server}$ threads and $\mathsf {proxy}$ threads in one protocol instance). While our protocol admittedly does not offer sufficient user responsiveness to search the many thousands of emails in our own email folders while we wait, it is easily efficient enough to search the small minority of those messages that are encrypted or, we believe, that would merit encryption. Moreover, we anticipate other usage modes for which our protocol’s search performance should be more than sufficient: e.g., a $\mathsf {user}$ can register a long-standing “subscription” query at the $\mathsf {proxy}$, which can evaluate the query on each email message as it arrives at the $\mathsf {server}$ and inform the $\mathsf {user}$ of the result.

To summarize, our contributions are as follows. First, we provide a design for an interactive protocol by which (i) a $\mathsf {user}$ provides an encrypted representation of a regular expression to a $\mathsf {proxy}$, (ii) the $\mathsf {proxy}$ interacts with a $\mathsf {server}$ holding an encrypted file to produce an output for the $\mathsf {proxy}$, and (iii) upon receiving that output from the $\mathsf {proxy}$, the $\mathsf {user}$ can determine whether the regular expression matched that file. However, neither the $\mathsf {proxy}$ nor the $\mathsf {server}$ learns anything about the search result, the regular expression (except its size), or the file plaintext (except its size). Second, we develop an implementation of this protocol together with several optimizations to make it perform well and then evaluate the performance of this implementation on a real-world email dataset.

2 Related work

The problem we study falls into the general area of secure “computing on encrypted data” [4] or two-party secure computation [5, 6] and could be implemented using general techniques. The former achieves computations noninteractively using fully homomorphic encryption, for which existing implementations (e.g., [7–10]) are still far from practical. The latter utilizes “garbled circuits” of size linear in the circuit representation of the function to be computed. Despite progress on practical implementations of this technique [11–14], this limitation renders it much more computation and communication intensive for the problem we consider. Perhaps, more importantly, since our protocol aims to enable a resource-constrained $\mathsf {user}$ to outsource the search query and workload to a $\mathsf {proxy}$, the interaction between the $\mathsf {user}$ and $\mathsf {proxy}$ should be minimized. Using our protocol, the communication cost in the direction from the user to the $\mathsf {proxy}$ is only dependent on the size of the search query and is independent of the number and size of the file ciphertexts. We are unaware of how to achieve this property using garbled circuits, however. Since the garbled circuit and its inputs are “unreusable” across different runs of the protocol, the user would need to provide a number of inputs (in this case, encrypted queries) to the $\mathsf {proxy}$ that equals to the number of files to be searched. Furthermore, the fact that in our construction, the user-generated encrypted query can be used an unlimited number of times enables a “subscription” service in which the $\mathsf {proxy}$ holds the encrypted query and periodically informs the user of the arrival of matched emails, without any further communication from the user to the $\mathsf {proxy}$. Again, we are unaware of how to implement this functionality using generic garbled circuit techniques.

The protocol we design here directly builds on a previous protocol [15] that enables a client to evaluate a deterministic finite automaton (DFA) on the plaintext of an encrypted file hosted by a server provided that the client has been given permission to do so by the data owner. Our protocol differs by permitting the client (in our case, the $\mathsf {proxy}$) to perform this evaluation using an encrypted DFA provided by the $\mathsf {user}$, so that the $\mathsf {proxy}$ need not be trusted with the privacy of the search query. We also contribute over that prior work by developing substantial optimizations to the performance of the resulting protocol; many of these optimizations should be applicable to the previous protocol, as well.

The structure of our protocol, involving a $\mathsf {user}$ who splits the DFA evaluation between a $\mathsf {proxy}$ and $\mathsf {server}$, is reminiscent of a protocol due to Blanton and Aliasgari [16]. Their protocol secret-shares the DFA and (plaintext) file, respectively, between two hosts, which then interactively evaluate the DFA on the file without reconstructing either one. Their protocol, however, does not support asymmetric encryption of the file, which in the email context that our protocol is designed to support, is the predominant method for preparing a private email for its intended recipient. With our protocol, as with today’s encrypted email options, the email sender needs only a public key for the intended recipient.

There has been a significant work on searchable symmetric encryption (SSE) (e.g., [17–24]), the key for which an email sender could encrypt using the email recipient’s public key after using it to encrypt the searchable email fields. This would make the search functionality provided by the SSE scheme available to the $\mathsf {user}$, but only after the $\mathsf {user}$ downloads the encrypted symmetric key for each email, so that she can create search queries for each one. In contrast, the protocol we present here does not require the $\mathsf {user}$ to obtain per email information before creating a search query for all emails. As mentioned above, the $\mathsf {proxy}$ could even hold and apply a “subscription” query periodically to inform the $\mathsf {user}$ of newly arrived emails that match it. Using searchable asymmetric encryption (e.g., [19, 22, 23, 25–27]) to encrypt each email could provide this feature, though we are unaware of any such scheme that supports regular-expression queries.

Because our protocol between the $\mathsf {proxy}$ and $\mathsf {server}$ is interactive, other interactive protocols are also related to our work. (Searchable encryption schemes are typically noninteractive.) In particular, interactive protocols for two-party private DFA evaluation, in which a $\mathsf {server}$ has a file and a $\mathsf {user}$ has a DFA to evaluate on that file, have been a topic of recent focus (e.g., [28–31]). Our work differs from these in that in our protocols, the file is available to the parties only in ciphertext form. We overcome this obstacle by the $\mathsf {user}$ two-party sharing the email-decryption key between the $\mathsf {proxy}$ and the $\mathsf {server}$, so that they can interact to evaluate the $\mathsf {user}$ ’s DFA on encrypted email fields. A protocol due to Choi et al. [32] takes a similar approach to enable general two-party computations on data that resides only in encrypted form at the parties. However, because this protocol is based on garbled circuits, it inherits the drawbacks we mentioned earlier.

Returning to the noninteractive case, Waters [33] described a functional encryption scheme that allows a secret key tied to a specific DFA to decrypt a ciphertext if the DFA accepts a fixed string associated with the ciphertext. However, his security requirements are quite different from ours, in that the string to be matched is considered public in his context. In our case, this string corresponds to the encrypted file and so must remain private.

3 Protocol design

We describe our protocols for regular-expression evaluation by assuming the regular expression has first been translated to an equivalent deterministic finite automaton (DFA) [2]. A DFA $M_{}$ is a tuple $\langle Q$, $\varSigma$, $\delta$, $q_{\mathsf {init}}$, $\varDelta \rangle$ where $Q$ is a set of $|Q | = n$ states; $\varSigma$ is a set (alphabet) of $|\varSigma | = m$ symbols; $\delta : Q \times \varSigma \rightarrow Q$ is a transition function; $q_{\mathsf {init}}$ is the initial state; and $\varDelta : Q \rightarrow \{0,1\}$ is a function for which $\varDelta (q) = 1$ indicates that $q$ is an accepting state.

Fig. 1

Protocol $\varPi _1({\mathcal {E}})$

3.1 Our starting point

We first review the protocol that serves as our starting point [15]. The protocol enables a $\mathsf {client}$ having a DFA $M_{}$ to interact with a $\mathsf {server}$ storing the ciphertext of a file to obtain the result as though $M_{}$ was evaluated on the file plaintext. More precisely, the $\mathsf {client}$ should output a bit indicating whether the final state to which the file plaintext drives the DFA is accepting or not; i.e., if the plaintext of the file is a sequence $\langle \sigma _{{}{k}} \rangle _{k \in [{\ell }]}$ where $[{\ell }]$ denotes the set $\{0, 1, \ldots , \ell -1\}$ and where each $\sigma _{{}{k}} \in \varSigma$, then the $\mathsf {client}$ should output $\varDelta (\delta (\ldots \delta (\delta (q_{\mathsf {init}}$, $\sigma _{{}{0}}$), $\sigma _{{}{1}}), \ldots , \sigma _{{}{\ell -1}}))$. The $\mathsf {client}$ can learn the file length $\ell$, and the $\mathsf {server}$ can learn both $\ell$ and the number of states $n$ in the client’s DFA. The $\mathsf {client}$ should learn nothing else about the file; however, the $\mathsf {server}$ should learn nothing else about the file or the $\mathsf {client}$ ’s DFA.

The protocol is shown in Fig. 1. The protocol is built using an additively homomorphic encryption scheme with plaintext space $\mathbb {R}$ where $\langle \mathbb {R}, +_{{}_\mathbb {R}}, \cdot _{{}_\mathbb {R}} \rangle$ denotes a commutative ring. Specifically, an encryption scheme $\mathcal {E}$ includes algorithms $\mathsf {Gen}$, $\mathsf {Enc}_{}$, and $\mathsf {Dec}_{}$ where: $\mathsf {Gen}$ is a randomized algorithm that on input $1^\kappa$ outputs a public key/private key pair $( pk , sk ) \leftarrow \mathsf {Gen} (1^{\kappa })$; $\mathsf {Enc}_{}$ is a randomized algorithm that on input public key $pk$ and plaintext $m_{} \in \mathbb {R}$ (where $\mathbb {R}$ can be determined as a function of $pk$) produces a ciphertext $c_{{}{}} \leftarrow \mathsf {Enc}_{ pk } (m_{})$, where $c_{{}{}} \in C_{ pk }$; $C_{ pk }$ is the ciphertext space determined by $pk$; and $\mathsf {Dec}_{}$ is a deterministic algorithm that on input a private key $sk$ and ciphertext $c_{{}{}} \in C_{ pk }$ produces a plaintext $m_{} \leftarrow \mathsf {Dec}_{ sk } (c_{{}{}})$ where $m_{} \in \mathbb {R}$. In addition, $\mathcal {E}$ supports an operation $+_{ pk }$ on ciphertexts such that for any public key/private key pair $( pk , sk )$, $\mathsf {Dec}_{ sk } (\mathsf {Enc}_{ pk } (m_{1}) +_{ pk } \mathsf {Enc}_{ pk } (m_{2})) = m_{1} +_{{}_\mathbb {R}} m_{2}$. Using $+_{ pk }$, it is possible to implement $\cdot _{ pk }$ for which $\mathsf {Dec}_{ sk } (m_{2} \cdot _{ pk } \mathsf {Enc}_{ pk } (m_{1})) = m_{1} \cdot _{{}_\mathbb {R}} m_{2}$. We use to denote summation using $+_{ pk }$; to denote summation using $+_{{}_\mathbb {R}}$; and to denote the product using $\cdot _{{}_\mathbb {R}}$ of a sequence.

The protocol also requires $\mathcal {E}$ to support two-party decryption. Specifically, there exists an efficient randomized algorithm $\mathsf {Share}$ that on input a private key $sk$ outputs shares $( sk _{1}, sk _{2}) \leftarrow \mathsf {Share} ( sk )$, and there are efficient deterministic algorithms $\mathsf {Dec}^{1}_{}$ and $\mathsf {Dec}^{2}_{}$ such that $\mathsf {Dec}_{ sk } (c_{{}{}})$ $=$ $\mathsf {Dec}^{2}_{ sk _{2}} (c_{{}{}}$, $\mathsf {Dec}^{1}_{ sk _{1}} (c_{{}{}}))$. An example of an encryption scheme $\mathcal {E}$ that meets the above requirements is due to Paillier [34] with modifications by Damgård and Jurik [35].

The main ingredient of the protocol is a method to encode a DFA $\langle Q$, $\varSigma$, $\delta$, $q_{\mathsf {init}}$, $\varDelta \rangle$, and specifically the transition function $\delta$, as a bivariate polynomial $f_{} (x, y)$ over $\mathbb {R}$ where $x$ is the variable representing a DFA state and $y$ is the variable representing an input symbol. That is, if we treat each state $q \in Q$ and each $\sigma \in \varSigma$ as distinct elements of $\mathbb {R}$, then we would like $f_{} (q, \sigma ) = \delta (q, \sigma )$. This can be achieved by choosing $f_{}$ to be the interpolation polynomial

(1)

where $f_{\sigma } (q) = \delta (q, \sigma )$ for each $q \in Q$ and where

(2)

is a Lagrange basis polynomial. Note that $\varLambda _{\sigma } (\sigma ) = 1$ and $\varLambda _{\sigma } (\sigma ') = 0$ for any $\sigma ' \in \varSigma \setminus \{\sigma \}$.

Calculating (2) requires taking multiplicative inverses in $\mathbb {R}$. While not every element of a ring has a multiplicative inverse in the ring, fortunately, the ring used in Paillier encryption, for example, has negligibly few elements with no inverses, and so, there is little risk of encountering an element with no inverse. Using Eq. 2, we can calculate coefficients $\langle \lambda _{{\sigma }{j}} \rangle _{j \in [{m}]}$, so that

This calculation is encapsulated in the procedure $\langle \lambda _{{\sigma }{j}} \rangle _{\sigma \in \varSigma ,j \in [{m}]}$ $\leftarrow$ $\mathsf {Lagrange} (\varSigma )$ in Fig. 1.

Each $f_{\sigma }$ needed to compute Eq. 1 can again be determined as an interpolation polynomial in the Lagrange form and then expressed as . In Fig. 1, this calculation is represented by the operation $\langle a_{{}{\sigma }{i}} \rangle _{\sigma \in \varSigma ,i \in [{n}]}$ $\leftarrow$ $\mathsf {ToPoly} (Q, \varSigma , \delta )$.

At the beginning of the protocol, the $\mathsf {client}$ receives as input a public key $pk$ under which the file stored at the $\mathsf {server}$ is encrypted. In addition, he receives a share $sk _{1}$ of the private key $sk$ corresponding to $pk$, and the DFA $\langle Q$, $\varSigma$, $\delta$, $q_{\mathsf {init}}$, $\varDelta \rangle$. The server receives as input the public key $pk$, another share $sk _{2}$ of the private key $sk$, the DFA alphabet $\varSigma$, and the file ciphertexts $\langle c_{{k}{j}} \rangle _{k \in [{\ell }], j \in [{m}]}$ where $c_{{k}{j}} \leftarrow \mathsf {Enc}_{ pk } ((\sigma _{{}{k}})^{j})$.

The protocol proceeds in rounds, processing one character in each round. The $\mathsf {client}$ begins the $k$ th loop iteration with an encryption $\alpha$ of the current DFA state after being blinded by a random injection $\pi _{1}: Q \rightarrow \mathbb {R}$ it chose in the $(k-1)$th loop at line c108 (or, if $k = 0$, then in line c103), where $\mathsf {Injs}({Q} \rightarrow {\mathbb {R}})$ denotes the set of injections from $Q$ to $\mathbb {R}$ ($\mathbf {I}$ denotes the identity function in c102). The $\mathsf {client}$ uses its share $sk _{1}$ to partially decrypt $\alpha$ (c106) and sends the result to the $\mathsf {server}$. The $\mathsf {server}$ uses his share $sk _{2}$ to fully decrypt $\alpha$ and obtains the blinded state $\gamma _{}$ (s104). The $\mathsf {server}$ then computes, for each $\sigma \in \varSigma$ (s105), a value $\varPsi _{\sigma }$ such that $\varLambda _{\sigma } (\sigma _{{}{k}}) = \mathsf {Dec}_{ sk } (\varPsi _{\sigma })$ (s106) by utilizing coefficients $\langle \lambda _{{\sigma }{j}} \rangle _{\sigma \in \varSigma , j \in [{m}]}$ output from $\mathsf {Lagrange}$ (s102). The $\mathsf {server}$ then returns (in s104) values $\langle \mu _{{\sigma }{i}} \rangle _{\sigma \in \varSigma , i \in [{n}]}$ created, so that $\mathsf {Dec}_{ sk } (\mu _{{\sigma }{i}}) = \gamma _{} ^{i} \cdot _{{}_\mathbb {R}} \varLambda _{\sigma } (\sigma _{{}{k}})$ (s108).

Meanwhile, the $\mathsf {client}$ selects a new random injection $\pi _{1} \;\mathop {\leftarrow }\limits ^{\$}\;\mathsf {Injs}({Q} \rightarrow {\mathbb {R}})$ (c108) and constructs a new DFA transition function $\delta '$ reflecting the injection it chose in the last round (now denoted $\pi _{0}$, see line c107) and the new injection $\pi _{1}$ it chose for this round. Specifically, it creates a new DFA state transition function $\delta '$ defined as $\delta ' (q,\sigma ) = \pi _{1} (\delta (\pi ^{-1}_{0} (q), \sigma ))$ for all $\sigma \in \varSigma$ and $q \in \pi _{0} (Q)$ where $\pi _{0} (Q) = \{\pi _{0} (q)\}_{q \in Q}$. This step is denoted as $\delta ' \leftarrow \mathsf {Blind} (\delta , \pi _{0}, \pi _{1})$ in line c109. That is, $\delta '$ “undoes” the previous injection $\pi _{0}$, applies $\delta$, and then applies the new injection $\pi _{1}$. The $\mathsf {client}$ then interpolates a new bivariate polynomial $f_{} (x, y)$ such that $f_{} (q, \sigma ) = \delta ' (q, \sigma )$ in line c110, using the algorithm described previously. The $\mathsf {client}$ uses these coefficients and $\langle \mu _{{\sigma }{i}} \rangle _{\sigma \in \varSigma , i \in [{n}]}$ sent from the server (m103) to “evaluate” the polynomial and obtains a ciphertext $\alpha$ of the new DFA state under the injection $\pi _{1}$ (c111). After $\ell$ loop iterations, the $\mathsf {client}$ obtains the encrypted final state in c111.

The $\mathsf {client}$ then engages in another round of interaction with the $\mathsf {server}$ in order to obtain a binary answer of whether the final state is an accepting state or not. For that purpose, the $\mathsf {client}$ creates another polynomial such that $F (q)=1$ if and only if $\varDelta (q)=1$ and $F (q)=0$ otherwise (c117). That is, $F (x)$ “converges” all accepting states to 1 and all nonaccepting states to 0. Since the $\mathsf {client}$ needs help from the $\mathsf {server}$ to decrypt the final result (s116), it applies another random injection $\pi _{1} \;\mathop {\leftarrow }\limits ^{\$}\;\mathsf {Injs}({\{0,1\}} \rightarrow {\{0,1\}})$ on the output of the function $\varDelta$ to hide the results from the $\mathsf {server}$. In the protocol, we use $\varDelta ' \leftarrow \mathsf {Blind} (\varDelta , \pi _{0},\pi _{1})$ (c116) to denote the step to generate the blinded function that maps the accepting state to a random element of $\{0,1\}$. The $\mathsf {client}$ then uses the polynomial interpolation procedure to obtain the coefficients of $F (x)$ in c117. After “evaluating” $F (x)$ in c118 and obtaining the encrypted binary output $\varTheta _{}$, the $\mathsf {client}$ interacts with the $\mathsf {server}$ once more to decrypt it and returns the result in c120.

3.2 Our initial construction

Starting from the protocol of the previous section, in this section, we develop the first contribution of this paper, namely a protocol that replaces the $\mathsf {client}$ with two parties: a $\mathsf {user}$ that holds the DFA $\langle Q$, $\varSigma$, $\delta$, $q_{\mathsf {init}}$, $\varDelta \rangle$ and a $\mathsf {proxy}$ that the $\mathsf {user}$ invokes to conduct a protocol to evaluate this DFA on a file stored at the $\mathsf {server}$. Notably, the protocol we develop here protects the secrecy of the DFA $\langle Q$, $\varSigma$, $\delta$, $q_{\mathsf {init}}$, $\varDelta \rangle$ and the evaluation result from the $\mathsf {proxy}$, and so, this modification enables the $\mathsf {proxy}$ to execute the protocol on behalf of others who do not trust it with knowledge of the DFA. One scenario in which this protection is desirable is if the $\mathsf {user}$ does not have the bandwidth or processing available for performing the evaluation herself.

The protocol, denoted $\varPi _2({\mathcal {E}})$, protects the DFA privacy by giving to the $\mathsf {proxy}$ the encryptions of the coefficients of the DFA polynomial $f_{}$, denoted $\langle \hat{a}_{{\sigma }{i}} \rangle _{\sigma \in \varSigma ,i \in [{n}]}$ where $\hat{a}_{{\sigma }{i}} \leftarrow \mathsf {Enc}_{ pk } (a_{{}{\sigma }{i}})$ and $\langle a_{{}{\sigma }{i}} \rangle _{\sigma \in \varSigma , i \in [{n}]}$ $\leftarrow$ $\mathsf {ToPoly} (Q, \varSigma , \delta )$, and the encryptions of the coefficients of the converging polynomial $F$, i.e., $\langle \hat{z_{i}} \rangle _{i \in [{n}]}$ where $\hat{z_{i}} \leftarrow \mathsf {Enc}_{ pk } (z_{i})$ and $\langle z_{i} \rangle _{i \in [{n}]}$ $\leftarrow$ $\mathsf {ToPoly} (Q, \varSigma , \varDelta )$. (The values $\langle z_{i} \rangle _{i \in [{n}]}$ do not depend on $\varSigma$, but we continue to provide $\varSigma$ to $\mathsf {ToPoly}$ for consistency.) The implications of this change to the protocol are far reaching, due to the operations that the $\mathsf {proxy}$ needs to perform using these now-encrypted coefficients.

In the original protocol, in order to hide the current state transition from the $\mathsf {server}$, the $\mathsf {client}$ blinds the current transition state by choosing a random injection $\pi _{1}$ of the state encodings in each round, so that the $\mathsf {server}$ obtains a random ring element $\gamma _{}$ in s104 every time. A new DFA polynomial is then interpolated to accommodate the injections chosen in the last and current round (c107–c110) to continue state transitions consistently. When the coefficients are encrypted, however, the $\mathsf {proxy}$ will not be able to interpolate new polynomials because it does not have access to $\delta$. We thus need another strategy to achieve these “blinding” and “unblinding” effects. Rather than blinding with a random injection, the new protocol does so by adding in a random ring element $r$ to the ciphertext representing the current state (c203–c204). The consequence of this additive blinding operation is that the $\mathsf {proxy}$ needs a way to “shift” $f_{}$ (and its encrypted coefficients) to produce a polynomial $f_{}' (x, y)$ satisfying $f_{}' (q +_{{}_\mathbb {R}} r, \sigma ) = \delta (q,\sigma )$ for each $q \in Q$ and $\sigma \in \varSigma$, for a specified $r \in \mathbb {R}$. If we set

where , then it suffices if $f_{\sigma }' (x +_{{}_\mathbb {R}} r) = f_{\sigma } (x)$ for all $\sigma \in \varSigma$. Note that

(3)

where Eq. 3 follows from the binomial theorem. As such, setting

(4)

ensures $f_{\sigma }' (x +_{{}_\mathbb {R}} r) = f_{\sigma } (x)$ and, therefore, $f_{} (x +_{{}_\mathbb {R}} r, \sigma ) = f_{}' (x, \sigma )$. When the $\mathsf {proxy}$ has access to only the encrypted coefficients, represented by $\langle \hat{a}_{{\sigma }{i}} \rangle _{\sigma \in \varSigma ,i \in [{n}]}$, the operation in Eq. 4 needs to be changed to

(5)

In our pseudocode, we encapsulate calculations of Eq. 5 in the invocation $\langle \hat{a}_{{\sigma }{i}}' \rangle _{\sigma \in \varSigma ,i \in [{n}]}\leftarrow \mathsf {Shift} (r, \langle \hat{a}_{{\sigma }{i}} \rangle _{\sigma \in \varSigma ,i \in [{n}]})$.

Now that the coefficients are encrypted, the operation by the $\mathsf {client}$ to combine coefficients with ciphertexts as was done in line c111 in Fig. 1 no longer works for the $\mathsf {proxy}$. For this reason, we need to expand the properties we require of the encryption system we use, to include the ability to homomorphically “multiply” ciphertexts once. We emphasize that we do not require fully homomorphic encryption. Our construction can be instantiated with any additively homomorphic encryption scheme that allows a single homomorphic multiplication of two ciphertexts (e.g., [36, 37]), provided that it also supports two-party decryption. Here, we build from the more well-studied scheme of Boneh et al. [36], which we denote by $\mathsf {BGN}$.²

Encryption scheme $\mathsf {BGN}$ uses an algorithm $\mathsf {BGNInit}$ that, on input $1^\kappa$, outputs $(p$, $p'$, $\mathbb {G}$, $\mathbb {G}'$, $e)$ where $p$, $p'$ are random $\kappa /2$-bit primes, $\mathbb {G}$ and $\mathbb {G}'$ are cyclic groups of order $N = p p'$, and $e: \mathbb {G} \times \mathbb {G} \rightarrow \mathbb {G}'$ is a bilinear map. In this encryption scheme, the ring $\mathbb {R}$ is $\mathbb {Z}_{N}$, the ciphertext space $C_{\langle N, \mathbb {G}, \mathbb {G}', e, g, h, \hat{g} \rangle }$ is $\mathbb {G} \cup \mathbb {G}'$, and the relevant algorithms are defined as follows. Note that we assume that elements of $\mathbb {G}$ and $\mathbb {G}'$ are encoded distinctly.

${\mathsf {Gen} (1^\kappa )}$: Generate $(p, p', \mathbb {G}, \mathbb {G}', e) \leftarrow \mathsf {BGNInit} (1^\kappa )$; select random generators $g, u \;\mathop {\leftarrow }\limits ^{\$}\;\mathbb {G}$; set $N \leftarrow p p'$, $h \leftarrow u ^{p'}$, and $\hat{g} \leftarrow e (g,g)^{p}$; and return public key $\langle N, \mathbb {G}, \mathbb {G}', e, g, h, \hat{g} \rangle$ and private key $\langle N$, $\mathbb {G}$, $\mathbb {G}'$, $e$, $g$, $\hat{g}$, $p \rangle$.

${\mathsf {Enc}_{\langle N, \mathbb {G}, \mathbb {G}', e, g, h, \hat{g} \rangle } (m_{})}$: Select $x \;\mathop {\leftarrow }\limits ^{\$}\;\mathbb {Z}_{N}$ and return $g ^{m_{}}h ^{x}$.

${\mathsf {Dec}_{\langle N, \mathbb {G}, \mathbb {G}', e, g, \hat{g}, p \rangle } (c_{{}{}})}$: If $c_{{}{}} \in \mathbb {G}$, then return the discrete logarithm of $e (c_{{}{}},g)^{p}$ with respect to base $\hat{g}$. If $c_{{}{}} \in \mathbb {G}'$, then return the discrete logarithm of $c_{{}{}} ^{p}$ with respect to base $\hat{g}$.

${c_{{1}{}} +_{\langle N, \mathbb {G}, \mathbb {G}', e, g, h, \hat{g} \rangle } c_{{2}{}}}$: If $c_{{1}{}}$ and $c_{{2}{}}$ are in the same group (i.e., both are in $\mathbb {G}$ or both are in $\mathbb {G}'$), then return $c_{{1}{}} c_{{2}{}}$. Otherwise, if $c_{{1}{}} \in \mathbb {G}$ and $c_{{2}{}} \in \mathbb {G}'$, then return $e (c_{{1}{}}, g)c_{{2}{}}$.

${m_{} \cdot _{\langle N, \mathbb {G}, \mathbb {G}', e, g, h, \hat{g} \rangle } c_{{}{}}}$: Return $c_{{}{}} ^{m_{}}$.

${c_{{1}{}} \odot _{\langle N, \mathbb {G}, \mathbb {G}', e, g, h, \hat{g} \rangle } c_{{2}{}}}$: If $c_{{1}{}}, c_{{2}{}} \in \mathbb {G}$, then return $e (c_{{1}{}},c_{{2}{}})$. Otherwise, return $\bot$.

${\mathsf {Share} (\langle N, \mathbb {G}, \mathbb {G}', e, g, \hat{g}, p \rangle )}$: Return $sk _{1} = \langle \mathbb {G}, \mathbb {G}', d_{1} \rangle$ and $sk _{2} = \langle \mathbb {G}, \mathbb {G}', e, g, \hat{g}, d_{2} \rangle$ where $d_{1} \;\mathop {\leftarrow }\limits ^{\$}\;\mathbb {Z}_{N}$ and $d_{2} \leftarrow p- d_{1} \hbox { mod }N$.

${\mathsf {Dec}^{1}_{\langle \mathbb {G}, \mathbb {G}', d_{1} \rangle } (c_{{}{}})}$: Return $c_{{}{}} ^{d_{1}}$.

${\mathsf {Dec}^{2}_{\langle \mathbb {G}, \mathbb {G}', e, g, \hat{g}, d_{2} \rangle } (c_{{1}{}}, c_{{2}{}})}$: If $c_{{1}{}}, c_{{2}{}} \in \mathbb {G}$, then return the discrete logarithm of $e (c_{{2}{}} c_{{1}{}} ^{d_{2}}, g)$ with respect to base $\hat{g}$. If $c_{{1}{}}, c_{{2}{}} \in \mathbb {G}'$, then return the discrete logarithm of $c_{{2}{}} c_{{1}{}} ^{d_{2}}$ with respect to base $\hat{g}$.

Note the new operator $\odot _{ pk }$ that homomorphically multiplies two ciphertexts in $\mathbb {G}$. Since the result is in $\mathbb {G}'$, it is not possible to use the result as an argument to $\odot _{ pk }$. This is the sense in which this scheme permits homomorphic multiplication “once”. Also note that though the basic scheme of Boneh et al. did not include $\hat{g} = e (g,g)^{p}$ in the public key, Boneh et al. proposed an extension supporting multiparty threshold decryption [36, Section 5] that did so;³ it is this extension that we adopt here.

A complication of using $\mathsf {BGN}$ is the need to compute a discrete logarithm to decrypt in both $\mathsf {Dec}_{\langle N, \mathbb {G}, \mathbb {G}', e, g, \hat{g}, p \rangle }$ and $\mathsf {Dec}^{2}_{\langle \mathbb {G}, \mathbb {G}', e, g, \hat{g}, d_{2} \rangle }$. We thus need to design our protocol, so that any ciphertext that a party attempts to decrypt should hold a plaintext from a small range $0 \ldots L$. Then, Pollard’s lambda method [39, p. 128] enables recovery of the plaintext in $O(\sqrt{L})$ time. Alternatively, a precomputed table that maps $\hat{g} ^{m_{}}$ to the plaintext $m_{} \in \{0\ldots L \}$ enables decryption to be performed by table lookup.

Fig. 2

Protocol $\varPi _2({\mathcal {E}})$, described in Sect. 3.2

Protocol steps Protocol $\varPi _2({\mathcal {E}})$ is shown in Fig. 2. It has a similar structure to $\varPi _1({\mathcal {E}})$, but differs in many respects.

• Rather than taking $\langle Q$, $\varSigma$, $\delta$, $q_{\mathsf {init}}$, $\varDelta \rangle$ as input, the $\mathsf {proxy}$ takes the encrypted initial state $\alpha _{\mathsf {init}}$ and encrypted coefficients $\langle \hat{a}_{{\sigma }{i}} \rangle _{\sigma \in \varSigma ,i \in [{n}]}$ and $\langle \hat{z_{i}} \rangle _{i \in [{n}]}$ as input. To simplify discussion later, we presume that $\alpha _{\mathsf {init}}$ is created as a ciphertext of $q_{\mathsf {init}}$ in $\mathbb {G}'$, e.g., $\alpha _{\mathsf {init}} \leftarrow \mathsf {Enc}_{ pk } (q_{\mathsf {init}}) \odot _{ pk } \mathsf {Enc}_{ pk } (1)$. Moreover, Fig. 2 presumes that the coefficients are created by performing $\langle a_{{}{\sigma }{i}} \rangle _{\sigma \in \varSigma ,i \in [{n}]}$ $\leftarrow$ $\mathsf {ToPoly} (Q$, $\varSigma$, $\delta )$ and $\langle z_{i} \rangle _{i \in [{n}]}$ $\leftarrow$ $\mathsf {ToPoly} (Q$, $\varSigma$, $\varDelta )$ and then encrypting each coefficient using $pk$, i.e., $\hat{a}_{{\sigma }{i}} \leftarrow \mathsf {Enc}_{ pk } (a_{{}{\sigma }{i}})$ for each $\sigma \in \varSigma$ and $i \in [{n}]$, and $\hat{z_{i}} \leftarrow \mathsf {Enc}_{ pk } (z_{i})$ for each $i \in [{n}]$.

• Because $\mathsf {server}$ decrypts the (blinded) DFA state in line s204, the plaintext should be adequately small so that decryption—which as discussed above, involves computing (or looking up) a discrete logarithm if $\mathsf {BGN}$ encryption is in use—is not too costly. For this reason, and assuming $\mathbb {R} = \mathbb {Z}_{N}$ (as it is in $\mathsf {BGN}$) and $Q = [{n}]$, the blinding term $r$ is drawn from $\{0,1\}^{\kappa '}$ instead of $\mathbb {R}$, where $\kappa ' \ll \kappa$ is another security parameter. Then, the statistical distance between the distribution of $\gamma _{}$ seen by $\mathsf {server}$ in line s204 when the blinded state is $q$ (i.e., when $\gamma _{} = q {} +_{{}_\mathbb {R}} r$) and uniformly random choices from $\{0,1\}^{\kappa '}$ is

  $$\begin{aligned}&\sum _{x} \left| \begin{array}{r}\mathbb {P}({q + r = x}\;| {\;}{r \;\mathop {\leftarrow }\limits ^{\$}\;\{0,1\}^{\kappa '}}) \\ - \mathbb {P}({r = x}\;| {\;}{r \;\mathop {\leftarrow }\limits ^{\$}\;\{0,1\}^{\kappa '}}) \end{array}\right| \\&\quad = \sum _{0 \le x < q} \frac{1}{2^{\kappa '}} + \sum _{2^{\kappa '} \le x < q + 2^{\kappa '}} \frac{1}{2^{\kappa '}} = \frac{q}{2^{\kappa '-1}} \end{aligned}$$

  Since $q \in [{n}]$, we anticipate setting $\kappa ' = \lceil \log _2 n \rceil + 15$ to achieve a reasonable balance between decryption cost and security for moderately sized $n$. In particular, no algorithm can then distinguish the values $\gamma _{}$ from random elements of $\{0,1\}^{\kappa '}$ in a run of the protocol on a file of length $\ell$ with combined type-I and type-II error better than $(1-\frac{1}{2^{15}})^{\ell +1}$ (see [40, Corollary 4]). $\kappa '$ will thus need to grow with $n$, though only logarithmically so. We stress, however, that the need to utilize a $\kappa '$ at all is a side effect of using $\mathsf {BGN}$ encryption in our protocol and is not fundamental to its design.

• The fact that each $\hat{a}_{{\sigma }{i}}$ is a ciphertext necessitates using the “one-time multiplication” operator $\odot _{ pk }$ in line c207 to produce the ciphertext of the new state, versus $\cdot _{ pk }$ as in line c111. The same is true in c213 because each $\hat{z_{i}}$ is a ciphertext.

• The protocol returns an encrypted evaluation result $\varTheta _{}$ to the $\mathsf {proxy}$ (c214), and so, the original round to decrypt the result (m107–m108) is omitted.

Protocol securityWe are able to prove that $\varPi _2({\mathcal {E}})$ protects DFA privacy and file content privacy against arbitrarily malicious $\mathsf {server}$ adversaries, and DFA privacy and file privacy against honest-but-curious $\mathsf {proxy}$ adversaries. We do not present the proofs here but refer readers to an accompanying technical report [41] for the proofs. Moreover, since the $\mathsf {proxy}$ receives only ciphertexts in the protocol that it cannot decrypt (and the file length $\ell$), we believe it heuristically protects DFA and file privacy against even an arbitrarily malicious $\mathsf {proxy}$. In the next section, we develop an optimized protocol that has better efficiency and achieves similar security properties. We formally define the security notions and evaluate that protocol’s security in Sect. 5.

4 Optimizations

In this section, we detail a series of optimizations that we developed for our protocol that, in our implementation, collectively achieved an order-of-magnitude improvement in performance over $\varPi _2({\mathcal {E}})$.

4.1 File representation

We first observe that, in protocol $\varPi _2({\mathcal {E}})$, the computation done by the $\mathsf {server}$ in s206 using the Lagrange coefficients it computed in s202 is effectively evaluating the ciphertext of $\varLambda _{\sigma } (\sigma _{{}{k}})$ for each $\sigma \in \varSigma$, using the values $\langle c_{{k}{j}} \rangle _{j \in [{m}]}$ provided as input to the $\mathsf {server}$ where $c_{{k}{j}} \leftarrow \mathsf {Enc}_{ pk } ((\sigma _{{}{k}})^{j})$. Recall that only for $\sigma = \sigma _{{}{k}}$ does $\varLambda _{\sigma } (\sigma _{{}{k}})=1$; otherwise, $\varLambda _{\sigma } (\sigma _{{}{k}})=0$. Since this calculation only depends on the ciphertexts of the current file character, the result of it, i.e., $\langle \mathsf {Enc}_{ pk } (\varLambda _{\sigma } (\sigma _{{}{k}})) \rangle _{\sigma \in \varSigma }$, could have been provided by the data owner as the ciphertext of file character $\sigma _{{}{k}}$ so that the $\mathsf {server}$ would not need to compute it itself.

With this observation, our first optimization is to eliminate the use of the Lagrange polynomial $\varLambda _{\sigma } (y)$ completely and decompose the original bivariate polynomial $f_{} (x,y)$ to $m$ univariate polynomials, i.e., $f_{\sigma } (x)$ for each $\sigma \in \varSigma$. The encryption of a file character $\sigma _{{}{k}}$ now becomes a vector of encryptions of 0’s and one 1. Specifically, $\sigma _{{}{k}}$ is provided to the $\mathsf {server}$ as ciphertexts $\langle c_{{k}{\sigma }} \rangle _{\sigma \in \varSigma }$ where $c_{{k}{\sigma }} \leftarrow \mathsf {Enc}_{ pk } (1)$ if $\sigma = \sigma _{{}{k}}$ and $c_{{k}{\sigma }} \leftarrow \mathsf {Enc}_{ pk } (0)$ otherwise. This representation has the same storage costs per file character as the original protocol, i.e., $m$ ciphertexts per encrypted file character.

4.2 Pairing operations

During the implementation of our protocol, we noticed that the pairing operations performed by the $\mathsf {proxy}$ in c207 are very costly and became the bottleneck for the overall performance. Accelerating pairing operations is a research area of substantial interest, and any progress made would be beneficial to protocols such as ours that utilize pairing. Our focus here, however, is twofold. One is to adapt the protocol to reduce the number of pairing operations. In particular, $m n$ pairing operations are needed in c207 in each round. In this section, we redesign the protocol to reduce the number of pairing operation down to $m$. The other focus is to make the protocol design amenable to pairing preprocessing [42].

Given a bilinear map $e: \mathbb {G} \times \mathbb {G} \rightarrow \mathbb {G}'$, if it is known in advance that a particular value $c_{{}{}} \in \mathbb {G}$ will be paired with other elements multiple times, then preprocessing on $c_{{}{}}$ can be performed in advance to achieve a significant reduction in pairing time. For example, for the class of machines used in our experiments in Sect. 6, a pairing operation for a 1,024-bit $\mathsf {BGN}$ scheme costs around 35 ms without preprocessing but only 10 ms after preprocessing. In c207, the pairing operation performed is $e (\hat{a}_{{\sigma }{i}}', \mu _{{\sigma }{i}})$. Unfortunately, both $\hat{a}_{{\sigma }{i}}'$ and $\mu _{{\sigma }{i}}$ change in each round, which prohibits preprocessing. This suggests that performing pairing operations on the $\mathsf {proxy}$ side may not be the best choice in terms of the potential for optimization.

We therefore redesigned the protocol with the goals of reducing the number of pairing operations and making pairing preprocessing possible. Fortunately, we were able to achieve both goals by shifting the pairing operations to the $\mathsf {server}$ side. The resulting protocol $\varPi _3({\mathcal {E}})$ is shown in Fig. 3. The new protocol essentially switches the roles of the $\mathsf {proxy}$ and $\mathsf {server}$ (though not entirely, since each still receives the same inputs). Note that the directions of the messages m303 and m304 are reversed from those in Fig. 2. The values $\alpha$ and $\beta$, which used to be produced by the $\mathsf {proxy}$ in c204 and c205, are now produced by the $\mathsf {server}$ in s304 and s305. This role reversal imposes some significant changes in the computations done by the $\mathsf {proxy}$ and $\mathsf {server}$.

Fig. 3

Optimized protocol $\varPi _3({\mathcal {E}})$, described in Sect. 4

We now describe the changes made in the protocol. We ask the readers to ignore the operations c303, c307, and c306, s315 for the time being; these will be discussed in Sect. 4.3. The $\mathsf {proxy}$ now obtains $\gamma _{}$ in c302, which is equal to $q +_{{}_\mathbb {R}} r$, where $q$ is the current DFA state and $r$ was chosen by the $\mathsf {server}$ in s303. It now uses $\gamma _{}$ as input (as opposed to $r$ in $\varPi _2({\mathcal {E}})$) into the $\mathsf {Shift}$ procedure first described in Sect. 3.2, i.e., computing new coefficients as:

(6)

As a consequence of this “shift”, the plaintexts of the coefficients $\langle \hat{a}_{{\sigma }{i}}' \rangle _{\sigma \in \varSigma ,i \in [{n}]}$ define a new polynomial $f_{\sigma }' (x)$ such that $f_{\sigma }' (x)= f_{\sigma } (x +_{{}_\mathbb {R}} (q +_{{}_\mathbb {R}} r))$. The $\mathsf {proxy}$ then sends $\langle \hat{a}_{{\sigma }{i}}' \rangle _{\sigma \in \varSigma ,i \in [{n}]}$ to the $\mathsf {server}$ in m304. The $\mathsf {server}$, knowing $r$, blindly “evaluates” the polynomial $f_{\sigma }' (x)$ on value ($-_{{}_\mathbb {R}}$ $r$) for each $\sigma \in \varSigma$, in lines s307–s309. Specifically, it computes a ciphertext of $f_{\sigma }' (-_{{}_\mathbb {R}} r) = f_{\sigma } (-_{{}_\mathbb {R}} r +_{{}_\mathbb {R}} (q +_{{}_\mathbb {R}} r)) = f_{\sigma } (q)$ as:

(7)

A naive way to compute Eq. 7 requires $O(n ^{2})$ exponentiations, but by leveraging Horner’s rule, it can be reduced to $O(n)$ exponentiations. Once the $\mathsf {server}$ obtains $\{\omega _{\sigma } \}_{\sigma \in \varSigma }$, it calculates a ciphertext $\alpha$ of the correct next DFA state in line s310, i.e., by homomorphically summing $\omega _{\sigma } \odot _{ pk } c_{{k}{\sigma }}$ over all $\sigma \in \varSigma$ (Recall from Sect. 4.1 that, for fixed $k$, exactly one of $\langle c_{{k}{\sigma }} \rangle _{\sigma \in \varSigma }$ is a ciphertext of 1 and the rest are ciphertexts of 0). The key point to notice here is that, by rearranging the protocol messages and letting the $\mathsf {proxy}$ send over the shifted coefficients, the number of pairing operations is chopped down to only $m$ from $n m$, a major improvement.

We have already alluded to the potential benefit of pairing preprocessing to reduce the online cost of pairing operations. The only question left is how to adapt the protocol, so that it is amenable to using this technique. Fortunately, the changes we have just made to the protocol also makes pairing preprocessing possible. The pairing operation that the $\mathsf {server}$ needs to perform in s310 is $e (\omega _{\sigma }, c_{{k}{\sigma }})$, for $\sigma \in \varSigma$ and $k \in [{\ell }]$. The ciphertext $c_{{k}{\sigma }}$ is fixed and known even before the protocol starts. This allows the $\mathsf {server}$ to perform pairing preprocessing using these ciphertexts offline and store them to stable storage for future use. During the protocol run, the preprocessing information can be retrieved and used to greatly reduce the online costs of the pairing operations.

4.3 Shifting

After the above optimizations, a remaining computation in the protocol that is especially expensive is the $\mathsf {Shift}$ procedure, i.e., Eq. 6, which is performed as part of c304 and c308. Computing each $\hat{a}_{{\sigma }{i}}'$ requires $O(n)$ exponentiations with exponents being powers of $\gamma _{}$. Since $\gamma _{}$ is $\kappa '$ bits, this exponentiation is increasingly expensive as $\kappa '$ grows and is one of the performance bottlenecks of our implementation for the $\kappa '$ values we employ (As discussed in Sect. 3.2, we take $\kappa ' \approx \lceil \log _2 n \rceil + 15$ in our present implementation, though this setting is an artifact of using $\mathsf {BGN}$ encryption and could be larger with another encryption scheme). Our next target is thus to find ways to optimize this operation.

One possibility is to use a smaller $\kappa '$ to speed up the exponentiations. However, $\kappa '$ cannot be arbitrarily reduced without reducing the security of the protocol. Instead, here we propose letting the $\mathsf {proxy}$ reduce $\gamma _{}$ modulo $n$ before feeding it into the $\mathsf {Shift}$ procedure, i.e., to compute:

(8)

in $\mathsf {Shift}$, instead (See c303 and c307). By reducing $\gamma _{}$, the exponents that used to be $O(n \kappa ')$ bits long are now reduced to $O(n \log n)$ bits, after taking into account the exponentiations on $\gamma _{}$ itself. However, this change does have implications for the correctness of the computation. Referring to the derivation in Eq. 7, now that the $\mathsf {proxy}$ shifted the polynomial by $\gamma _{} \hbox { mod }n$, the $\mathsf {server}$ needs to adapt to this change accordingly. Intuitively, it should evaluate the new polynomial on $(-r \hbox { mod }n)$ as opposed to on $-_{{}_\mathbb {R}} r$ in Eq. 7, in which case it computes a ciphertext $\omega _{\sigma }$ of

$$\begin{aligned} f_{\sigma }' (-r \hbox { mod }n)&= f_{\sigma } ((-r \hbox { mod }n) +_{{}_\mathbb {R}} ((q +_{{}_\mathbb {R}} r) \hbox { mod }n)) \\&= \left\{ \begin{array}{ll} f_{\sigma } (q) &{} \text{ if } n \mid r \text{ or } q \!+\! (r \hbox { mod }n) \!\ge \! n \\ f_{\sigma } (q + n) &{} \quad \text{ otherwise } \end{array}\right. \end{aligned}$$

assuming that $\kappa ' + 1 < \kappa$ (See lines s306 and s315). However, as indicated, there are two possible outcomes from this calculation. One is exactly what we want, i.e., a ciphertext of $f_{\sigma } (q)$. The other possibility is a ciphertext of $f_{\sigma } (q +n)$, which is problematic because $f_{\sigma } (q +n)$ is arbitrary. The $\mathsf {server}$ unfortunately cannot tell which case happened because everything it operates on is encrypted. Our solution to this problem is to add constraints when constructing $f_{\sigma } (x)$ so that $f_{\sigma } (q +n) = f_{\sigma } (q)$ for all $q \in Q$ and $\sigma \in \varSigma$. These additional constraints guarantee the correct state transition regardless of which case happens. However, the price we pay is that the degree of $f_{\sigma } (x)$ increases to $2n-1$ since additional $n$ constraints need to be added to define the polynomial. But the performance gains we achieve outweigh this loss.

Another key insight to draw from this technique is that $\gamma _{} \hbox { mod }n$ can take on only $n$ different values, and so, there can be at most $n$ different sets of coefficients $\langle \hat{a}_{{\sigma }{i}}' \rangle _{\sigma \in \varSigma ,i \in [{2n}]}$ from the calculation in Eq. 8. This allows the $\mathsf {proxy}$ to precompute all possible sets of $\langle \hat{a}_{{\sigma }{i}}' \rangle _{\sigma \in \varSigma ,i \in [{2n}]}$ for each $\gamma _{} \hbox { mod }n \in [{n}]$ and store them in a table before the start of the protocol. It can then simply perform table lookups depending on which value of $\gamma _{} \hbox { mod }n$ it obtains in c303 (or c307). This way, the $\mathsf {proxy}$ does not need to perform the computations in Eq. 8 during the protocol except for randomizing the ciphertexts before sending them back to the $\mathsf {server}$. This offers tremendous performance gains for the protocol: Without applying this optimization, the cost for the $\mathsf {proxy}$ to calculate Eq. 8 for all $\langle \hat{a}_{{\sigma }{i}}' \rangle _{\sigma \in \varSigma ,i \in [{2n}]}$ involves $O(m n ^{2})$ exponentiations in each round. After applying this optimization, it is reduced to only $O(m n)$ exponentiations, due to the need for ciphertext randomizations.

4.4 Packing the result ciphertexts and boolean combinations

When using $\varPi _3({\mathcal {E}})$ to evaluate a DFA on $k$ files, $k$ encrypted evaluation results $\varTheta _{0}, \ldots , \varTheta _{k-1}$—each the ciphertext of a 0 or 1—need to be communicated back to the $\mathsf {user}$. Sending these $k$ ciphertexts individually to the $\mathsf {user}$ introduces an undesirably high communication cost between the $\mathsf {proxy}$ and the $\mathsf {user}$. A better approach is for the $\mathsf {proxy}$ to aggregate multiple such results into a single ciphertext before sending these results back to the $\mathsf {user}$. Specifically, the $\mathsf {proxy}$ can aggregate these $k$ ciphertexts into ciphertexts ${\mathbf {\Theta }}_{0}, \ldots , {\mathbf {\Theta }}_{\lceil k/z \rceil -1}$ where

for each $i \in [{\lceil k/z \rceil }]$. This aggregation is omitted from Fig. 3. The $\mathsf {user}$ can then decrypt each ${\mathbf {\Theta }}_{i}$ to recover all the evaluation results. The value of $z$ is upper bounded by the bit length of the plaintext space of the cryptosystem $\mathcal {E}$ in general, and in the case of $\mathsf {BGN}$, $z$ must be restricted to a small value such as $\kappa '$ to enable efficient decryption by the $\mathsf {user}$.

This packing technique generalizes nicely to support evaluating conjunctions or disjunctions of $d$ DFAs on $k$ files. That is, after the $\mathsf {proxy}$ interacts with the $\mathsf {server}$ to evaluate DFAs $M_{0}$, $\ldots$, $M_{d}$ on each of $k$ files, yielding encrypted results $\varTheta _{0,0}$, $\ldots$, $\varTheta _{k-1,d-1}$, the $\mathsf {proxy}$ can aggregate these $k d$ ciphertexts into ciphertexts ${\mathbf {\Theta }}_{0}, \ldots , {\mathbf {\Theta }}_{\lceil k/z ' \rceil -1}$ where $z ' = \lfloor z/\lceil \log _2 (d +1)\rceil \rfloor$ and

for each $i \in [{\lceil k/z ' \rceil }]$. Upon decrypting each such aggregate ciphertext, each $\lceil \log _2 (d +1)\rceil$-length sequence of bits represents the number of DFAs $M_{0}$, $\ldots$ $M_{d-1}$ that the corresponding file matched. That is, the file satisfies the disjunction of these DFAs if that count is nonzero, and it satisfies the conjunction of these DFAs if that count is $d$.

To evaluate other Boolean combinations of $d$ DFAs on files, it suffices for the $\mathsf {proxy}$ and $\mathsf {server}$ to evaluate each DFA individually on each file and communicate the results per DFA to the $\mathsf {user}$, and the $\mathsf {user}$ can herself determine which files match the Boolean combination she is interested in. While less communication efficient than the above approach for conjunctions and disjunctions, this approach is more computationally efficient for the $\mathsf {proxy}$ and $\mathsf {server}$ than combining all $d$ DFAs into a single large DFA that represents the Boolean combination of interest.

5 Protocol security

Fig. 4

Experiments for proving security of $\varPi _3({\mathcal {E}})$ against $\mathsf {server}$ adversaries. a Experiment $\mathbf {Expt}^{\mathtt {s}\text{- }\mathtt {dfa}}_{\varPi _3({\mathcal {E}})}$. b Experiment $\mathbf {Expt}^{\mathtt {s}\text{- }\mathtt {file}}_{\varPi _3({\mathcal {E}})}$

In this section, we analyze the security of $\varPi _3({\mathcal {E}})$ as shown in Fig. 3 (For simplicity, here, we elide consideration of the extension in Sect. 4.4, not shown in Fig. 3, though it has no impact on the security of the protocol). We show that the protocol $\varPi _3({\mathcal {E}})$ provably protects the privacy of both the DFA and file contents from either honest-but-curious $\mathsf {proxy}$ adversaries or arbitrarily malicious $\mathsf {server}$ adversaries.

5.1 Security against server adversaries

In this section, we bound the advantage that an arbitrarily malicious $\mathsf {server}$ gains by executing this protocol, in determining either the DFA that the $\mathsf {proxy}$ is evaluating or the plaintext of the file in its possession. That is, we prove the privacy of the file and DFA inputs against $\mathsf {server}$ adversaries.

Following previous security definitions [15], we formalize our security claims against $\mathsf {server}$ compromise by defining two separate $\mathsf {server}$ adversaries. The first $\mathsf {server}$ adversary $S_{} = (S_{1}, S_{2})$ attacks the encrypted DFA $M_{} = \langle Q$, $\varSigma$, $\delta$, $q_{\mathsf {init}}$, $\varDelta \rangle$, i.e., $\langle \hat{a}_{{\sigma }{i}} \rangle _{\sigma \in \varSigma ,i \in [{n}]}$ held by the $\mathsf {proxy}$, as described in experiment $\mathbf {Expt}^{\mathtt {s}\text{- }\mathtt {dfa}}_{\varPi _3({\mathcal {E}})}$ in Fig. 4a. $S_{1}$ first generates a file $\langle \sigma _{{}{k}} \rangle _{k \in [{\ell }]}$ and two DFAs $M_{0}$, $M_{1}$. (Note that we use, e.g., “$M_{0}$.$Q$ ” and “$M_{1}$.$Q$ ” to disambiguate their state sets.) $S_{2}$ is then invoked with the ciphertexts $\langle c_{{k}{\sigma }} \rangle _{k \in [{\ell }], \sigma \in \varSigma }$ of its file and information $\phi$ created for it by $S_{1}$ and is given oracle access to $\mathsf {proxyOr}$. $\mathsf {proxyOr}$ is given input arguments pertaining to one of the two DFAs output by $S_{1}$, selected at random (as indicated by $b$).

$\mathsf {proxyOr}$ responds to queries from $S_{2}$ as follows, ignoring malformed queries. The first query (say, consisting of simply “start”) causes $\mathsf {proxyOr}$ to begin the protocol; $\mathsf {proxyOr}$ responds with a message of the form $n$, $\alpha _{\mathsf {init}}$ (i.e., of the form of message m301). The second invocation by $S_{2}$ must include a single integer $\ell$ (i.e., of the form of message m302). The next $\ell$ queries by $S_{2}$ must be of the form $\alpha$, $\beta$, i.e., two values as in message m303, to which $\mathsf {proxyOr}$ responds by sending $2n m$ elements of $C_{ pk }$, i.e., $\langle \hat{a}_{{\sigma }{i}}' \rangle _{\sigma \in \varSigma ,i \in [{2n}]}$ as in m304. $S_{2}$ ’s next query to $\mathsf {proxyOr}$ again must contain two values of the form $\alpha$, $\beta$ (as in m305), to which $\mathsf {proxyOr}$ responds with $2n$ ciphertexts $\langle \hat{z_{i}} ' \rangle _{i \in [{2n}]}$ as in m306. The next (and last) query by $S_{2}$ can consist one element of $C_{ pk }$ as in m307. Eventually, $S_{2}$ outputs a bit $b '$ , and $\mathbf {Expt}^{\mathtt {s}\text{- }\mathtt {dfa}}_{\varPi _3({\mathcal {E}})} (S_{}) = 1$ only if $b ' = b$. We say the advantage of an arbitrarily malicious $S_{}$ is

and define $\mathbf {Adv}^{\mathtt {s}\text{- }\mathtt {dfa}}_{\varPi _3({\mathcal {E}})} (t_{}, \ell , n, m) = \max _{S_{}} \mathbf {Adv}^{\mathtt {s}\text{- }\mathtt {dfa}}_{\varPi _3({\mathcal {E}})} (S_{})$ where the maximum is taken over all adversaries $S_{}$ taking time $t_{}$ and selecting a file of length $\ell$ and DFAs containing $n$ states and an alphabet of $m$ symbols.

Fig. 5

$\mathbf {Expt}^{\mathtt {ind}\text{- }\mathtt {cpa}}_{\mathcal {E}}$ ($U_{}$)

We reduce DFA privacy against server attacks to the IND-CPA [43] security of the encryption scheme. IND-CPA security is defined using the experiment in Fig. 5, in which an adversary $U_{}$ is provided a public key $\hat{ pk }$ and access to an oracle $\mathsf {Enc}_{\hat{ pk }} ^{\hat{b}} (\cdot , \cdot )$ that consistently encrypts either the first of its two inputs (if $\hat{b} = 0$) or the second of those inputs (if $\hat{b} = 1$). Eventually, $U_{}$ outputs a guess $\hat{b} '$ at $\hat{b}$, and $\mathbf {Expt}^{\mathtt {ind}\text{- }\mathtt {cpa}}_{\mathcal {E}} (U_{}) = 1$ only if $\hat{b} ' = \hat{b}$. The IND-CPA advantage of $U_{}$ is defined as

$$\begin{aligned} \mathbf {Adv}^{\mathtt {ind}\text{- }\mathtt {cpa}}_{\mathcal {E}} (U_{}) = 2\cdot \mathbb {P}\left( {\mathbf {Expt}^{\mathtt {ind}\text{- }\mathtt {cpa}}_{\mathcal {E}} (U_{}) = 1}\right) - 1 \end{aligned}$$

and then $\mathbf {Adv}^{\mathtt {ind}\text{- }\mathtt {cpa}}_{\mathcal {E}} (t_{}, w) = \max _{U_{}} \mathbf {Adv}^{\mathtt {ind}\text{- }\mathtt {cpa}}_{\mathcal {E}} (U_{})$ where the maximum is taken over all adversaries $U_{}$ executing in time $t_{}$ and making $w$ queries to $\mathsf {Enc}_{\hat{ pk }} ^{\hat{b}} (\cdot ,\cdot )$.

In our theorem statements, we omit terms that are negligible as a function of the security parameters $\kappa$ and $\kappa '$. For any $\mathcal {E}$ operation $\mathsf {op}$, we use $t_{\mathsf {op}}$ to denote the time required to perform $\mathsf {op}$; e.g., $t_{\mathsf {Dec}_{}}$ is the time to perform a $\mathsf {Dec}_{}$ operation.

Theorem 1

For $t_{}' =t_{} +t_{\mathsf {Share}} +(\ell m + 2n m +2n +1) \cdot t_{\mathsf {Enc}_{}}$,

$$\begin{aligned} \mathbf Adv ^{\texttt {s}\text{- }\texttt {dfa}}_{{\varPi _{3}}\textsf {(BGN)}} (t, \ell , n, m) \!\le \! n^{\ell +1} \mathbf Adv ^{\texttt {ind}\text{- }\texttt {cpa}}_\textsf {BGN} (t^{\prime }, 2nm\!+\!2n\!+\!1) \end{aligned}$$

Proof

Given an adversary $S_{} =(S_{1}, S_{2})$ for $\varPi _3({\mathsf {BGN}})$ that runs in time $t_{}$, produces a file of length $\ell$, and produces DFAs of $n$ states over an alphabet of $m$ symbols, we construct an IND-CPA attacker $U_{}$ for $\mathsf {BGN}$ to demonstrate the theorem as follows. On input a $\mathsf {BGN}$ public key $\hat{ pk } = \langle N$, $\mathbb {G}$, $\mathbb {G}'$, $e$, $g$, $h$, $\hat{g} \rangle$, $U_{}$ sets $d_{2} \;\mathop {\leftarrow }\limits ^{\$}\;\mathbb {Z}_{N}$, and invokes $S_{1} (\hat{ pk }, sk _{2})$ where $sk _{2} = \langle \mathbb {G}, \mathbb {G}', d_{2} \rangle$ to obtain $(\ell$, $\langle \sigma _{{}{k}} \rangle _{k \in [{\ell }]}$, $M_{0}$, $M_{1}$, $\phi )$. Note that $d_{2}$ is chosen from a distribution that is perfectly indistinguishable from that from which $d_{2}$ is chosen in the real system. If $M_{0}.Q \ne M_{1}.Q$ or $M_{0}.\varSigma \ne M_{1}.\varSigma$, then $U_{}$ aborts the simulation. Otherwise, letting $\varSigma = M_{0}.\varSigma$, $m = |\varSigma |$ and $n = |M_{0}.Q |$, $U_{}$ computes $\langle a_{{0}{\sigma }{i}} \rangle _{\sigma \in \varSigma , i \in [{2n}]}$ $\leftarrow$ $\mathsf {ToPoly} (M_{0}.Q$, $\varSigma$, $M_{0}.\delta )$ and $\langle a_{{1}{\sigma }{i}} \rangle _{\sigma \in \varSigma , i \in [{2n}]}$ $\leftarrow$ $\mathsf {ToPoly} (M_{1}.Q$, $\varSigma$, $M_{1}.\delta )$, and it sets $\hat{a}_{{\sigma }{i}}$ $\leftarrow$ $\mathsf {Enc}_{\hat{ pk }} ^{\hat{b}} (a_{{0}{\sigma }{i}}$, $a_{{1}{\sigma }{i}})$ for $\sigma \in \varSigma$ and $i \in [{n}]$. It then computes $\langle z_{0i} \rangle _{i \in [{2n}]}$ $\leftarrow$ $\mathsf {ToPoly} (M_{0}.Q$, $\varSigma$, $M_{0}.\varDelta )$ and $\langle z_{1i} \rangle _{i \in [{2n}]}$ $\leftarrow$ $\mathsf {ToPoly} (M_{1}.Q$, $\varSigma$, $M_{1}.\varDelta )$, and it sets $\hat{z_{i}} \leftarrow$ $\mathsf {Enc}_{\hat{ pk }} ^{\hat{b}} (z_{0i}$, $z_{1i})$ for $i \in [{n}]$. $U_{}$ finally sets $\alpha _{\mathsf {init}}$ $\leftarrow$ $\mathsf {Enc}_{\hat{ pk }} ^{\hat{b}} (M_{0}.q_{\mathsf {init}}$, $M_{1}.q_{\mathsf {init}})$ $\odot _{\hat{ pk }}$ $\mathsf {Enc}_{\hat{ pk }} (1)$, and then for all $k \in [{\ell }], \sigma \in \varSigma$, it sets $c_{{k}{\sigma }} \leftarrow \mathsf {Enc}_{ pk } (1)$ if $\sigma = \sigma _{{}{k}}$ and $c_{{k}{\sigma }} \leftarrow \mathsf {Enc}_{ pk } (0)$ otherwise.

$U_{}$ then invokes $S_{2} (\phi , \langle c_{{k}{\sigma }} \rangle _{k \in [{\ell }],\sigma \in \varSigma })$ and simulates responses to $S_{2}$ ’s queries to $\mathsf {proxyOr}$ as follows (ignoring malformed invocations). Upon initializing $S_{2}$, $U_{}$ sends $n$, $\alpha _{\mathsf {init}}$ to $S_{2}$ and gets $\ell$ in return. For the $k$ th query of the form $\alpha , \beta$ ($0 \le k < \ell$), $U_{}$ selects $\gamma _{} \;\mathop {\leftarrow }\limits ^{\$}\;[{n}]$, as opposed to decrypting it as in a real execution (see c302 and c303). It computes $\langle \hat{a}_{{\sigma }{i}}' \rangle _{\sigma \in \varSigma ,i \in [{2n}]} \leftarrow \mathsf {Shift} (\gamma _{},\langle \hat{a}_{{\sigma }{i}} \rangle _{\sigma \in \varSigma ,i \in [{2n}]})$ as in c304 and returns $\langle \hat{a}_{{\sigma }{i}}' \rangle _{\sigma \in \varSigma ,i \in [{2n}]}$ to $S_{2}$. For the $(\ell +1)$th query of the form $\alpha , \beta$, $U_{}$ again randomly sets $\gamma _{} \;\mathop {\leftarrow }\limits ^{\$}\;[{n}]$ and $\langle \hat{z_{i}} ' \rangle _{i \in [{2n}]} \leftarrow \mathsf {Shift} (\gamma _{},\langle \hat{z_{i}} \rangle _{i \in [{2n}]} )$ and then sends $\langle \hat{z_{i}} ' \rangle _{i \in [{2n}]}$ to $S_{2}$. Finally, when $S_{2}$ outputs $b '$, $U_{}$ outputs $b '$, as well.

$U_{}$ ’s simulation is perfectly indistinguishable from the real system to a malicious server adversary $S_{}$ if and only if $U_{}$ made the correct guesses on $\gamma _{}$ in each round. When that happens, the advantage of $U_{}$ winning his game is the same with that of $S_{}$. So, $\textbf {Adv}^{\mathtt {ind}\text{- }\mathtt {cpa}}_{\mathsf {BGN}} (U_{})$ $\ge$ $(\frac{1}{n})^{\ell +1} \textbf {Adv}^{\mathtt {s}\text{- }\mathtt {dfa}}_{\varPi _3({\mathsf {BGN}})} (S_{})$. $U_{}$ runs in time $t_{}' =t_{} +t_{\mathsf {Share}} +\ell m \cdot t_{\mathsf {Enc}_{}} + (2n m +2n +1) \cdot t_{\mathsf {Enc}_{}}$ due to the need to generate a secret key share for $S_{}$, to generate $\langle c_{{k}{j}} \rangle _{k \in [{\ell }],j \in [{m}]}$, and to make $2n m +2n +1$ encryption oracle queries to create $\langle \hat{a}_{{\sigma }{i}} \rangle _{\sigma \in \varSigma ,i \in [{2n}]}$, $\langle \hat{z_{i}} \rangle _{i \in [{2n}]}$ and $\alpha _{\mathsf {init}}$. $\square$

The multiplicative factor of $n ^{\ell +1}$ that appears in Theorem  and in Theorem  below, while independent of the security parameters $\kappa$ and $\kappa '$, nevertheless, renders these theorems of limited practical use. That said, we have no reason to believe that the actual security of $\varPi _3({\mathsf {BGN}})$ against $\mathsf {server}$ adversaries decays so dramatically as a function of $\ell$. Rather, this factor is simply an artifact of our proof method, and since the $\mathsf {server}$ in $\varPi _3({\mathsf {BGN}})$ receives (aside from the value $n$) only ciphertexts from the $\mathsf {proxy}$ created using a public key for which it does not hold the private key, we believe these theorems to be overwhelmingly conservative.

In addition, the $n ^{\ell +1}$ factor can be eliminated in both theorems if $S_{}$ is honest-but-curious and so follows the protocol as prescribed, provided that we modify the protocol with the following trick: the data owner initializes the $\mathsf {server}$ with another, distinct public key—for which the corresponding private key is given to nobody—that the $\mathsf {server}$ uses to encrypt each value of $r$ it selects in the protocol (s303 and s312), sending this ciphertext along with $\alpha$, $\beta$ to the $\mathsf {proxy}$ in messages m303 and m305. This ciphertext is useless to the $\mathsf {proxy}$ since it does not hold the key to decrypt it. However, in the proof of security against an honest-but-curious $\mathsf {server}$, who faithfully includes $r$ in this ciphertext, the IND-CPA adversary $U_{}$ can create this new public key for the server adversary $S_{}$, while keeping the private key for decrypting the ciphertexts containing the $r$ values. This enables completion of a security proof for an honest-but-curious $S_{}$ that does not incur a $n ^{\ell +1}$ loss factor. Indeed, a similar trick was used in our proof of security for protocol $\varPi _1({\mathcal {E}})$  [15].

The second $\mathsf {server}$ adversary $S_{} = (S_{1}, S_{2})$ that we consider attacks the file for which it holds the per-symbol ciphertexts $\langle c_{{k}{\sigma }} \rangle _{k \in [{\ell }], \sigma \in \varSigma }$ as in experiment $\mathbf {Expt}^{\mathtt {s}\text{- }\mathtt {file}}_{\varPi _3({\mathcal {E}})}$ shown in Fig. 4b. Here, $S_{1}$ produces two separate, equal-length plaintext files $\langle \sigma _{{0}{k}} \rangle _{k \in [{\ell }]}$, $\langle \sigma _{{1}{k}} \rangle _{k \in [{\ell }]}$ and a DFA $M_{}$. $S_{2}$ then receives ciphertexts $\langle c_{{k}{\sigma }} \rangle _{k \in [{\ell }], \sigma \in \varSigma }$ for file $\langle \sigma _{{b}{k}} \rangle _{k \in [{\ell }]}$ where $b$ is chosen randomly. $S_{2}$ is also given oracle access to $\mathsf {proxyOr} ( pk$, $sk _{1}$, $\varSigma$, $n$, $\alpha _{\mathsf {init}}$, $\langle \hat{a}_{{\sigma }{i}} \rangle _{\sigma \in \varSigma ,i \in [{2n}]}$, $\langle \hat{z_{i}} \rangle _{i \in [{2n}]})$. The interaction between $S_{2}$ and $\mathsf {proxyOr}$ is similar to what was described for the server DFA adversary. Eventually, $S_{2}$ outputs a bit $b '$, and $\mathbf {Expt}^{\mathtt {s}\text{- }\mathtt {file}}_{\varPi _3({\mathcal {E}})} (S_{}) = 1$ iff $b ' = b$. The advantage of $S_{}$ is

$$\begin{aligned} \mathbf {Adv}^{\mathtt {s}\text{- }\mathtt {file}}_{\varPi _3({\mathcal {E}})} (S_{}) = 2 \cdot \mathbb {P}\left( {\mathbf {Expt}^{\mathtt {s}\text{- }\mathtt {file}}_{\varPi _3({\mathcal {E}})} (S_{}) = 1}\right) - 1 \end{aligned}$$

and then $\mathbf {Adv}^{\mathtt {s}\text{- }\mathtt {file}}_{\varPi _3({\mathcal {E}})} (t_{}, \ell , n, m) = \max _{S_{}} \mathbf {Adv}^{\mathtt {s}\text{- }\mathtt {file}}_{\varPi _3({\mathcal {E}})} (S_{})$ where the maximum is taken over all adversaries $S_{} = (S_{1}, S_{2})$ taking time $t_{}$ and producing (from $S_{1}$) files of $\ell$ symbols and a DFA of $n$ states and alphabet of size $m$.

Theorem 2

For $t_{}' =t_{} +t_{\mathsf {Share}} +(\ell m + 2n m +2n +1) \cdot t_{\mathsf {Enc}_{}}$,

$$\begin{aligned} \mathbf Adv ^{\texttt {s}\text{- }\texttt {file}}_{\varPi _{3}\textsf {(BGN)}} (t, \ell , n, m) \le n^{\ell +1} \mathbf Adv ^{\texttt {ind}\text{- }\texttt {cpa}}_\textsf {BGN} (t^{\prime }, \ell m) \end{aligned}$$

Proof

Given an adversary $S_{} = (S_{1}, S_{2})$ running in time $t_{}$ and selecting files of length $\ell$ symbols and a DFA of $n$ states over an alphabet of $m$ symbols, we construct an IND-CPA adversary $U_{}$. On input a $\mathsf {BGN}$ public key $\hat{ pk } = \langle N$, $\mathbb {G}$, $\mathbb {G}'$, $e$, $g$, $h$, $\hat{g} \rangle$, $U_{}$ sets $d_{2} \;\mathop {\leftarrow }\limits ^{\$}\;\mathbb {Z}_{N}$, and invokes $S_{1} (\hat{ pk }, sk _{2})$ where $sk _{2} = \langle \mathbb {G}, \mathbb {G}', d_{2} \rangle$ to obtain $(\ell$, $\langle \sigma _{{0}{k}} \rangle _{k \in [{\ell }]}$, $\langle \sigma _{{1}{k}} \rangle _{k \in [{\ell }]}$, $M_{}$, $\phi )$, where $M_{} = \langle Q$, $\varSigma$, $q_{\mathsf {init}}$, $\delta$, $\varDelta \rangle$ is a DFA. Note that $d_{2}$ is chosen from a distribution that is perfectly indistinguishable from that from which $d_{2}$ is chosen in the real system. For $k \in [{\ell }]$ and $\sigma \in \varSigma$, $U_{}$ sets $c_{{k}{j}} \leftarrow \mathsf {Enc}_{\hat{ pk }} ^{\hat{b}} (I_{0},I_{1})$ where

$$\begin{aligned} I_{0} \leftarrow \left\{ \begin{array}{ll} 1 &{} \quad \text{ if } \sigma = \sigma _{{0}{k}} \\ 0 &{}\quad \text{ otherwise }\\ \end{array}\right. \qquad \qquad I_{1} \leftarrow \left\{ \begin{array}{ll} 1 &{} \quad \text{ if } \sigma = \sigma _{{1}{k}} \\ 0 &{} \quad \text{ otherwise }\\ \end{array}\right. \end{aligned}$$

$U_{}$ also sets $\alpha _{\mathsf {init}}$ $\leftarrow$ $\mathsf {Enc}_{\hat{ pk }} (q_{\mathsf {init}}) \odot _{\hat{ pk }} \mathsf {Enc}_{\hat{ pk }} (1)$, $\langle a_{{}{\sigma }{i}} \rangle _{\sigma \in \varSigma , i \in [{2n}]}$ $\leftarrow$ $\mathsf {ToPoly} (Q, \varSigma ,\delta )$, and $\langle z_{i} \rangle _{i \in [{2n}]}$ $\leftarrow$ $\mathsf {ToPoly} (Q,$ $\varSigma , \varDelta )$. $U_{}$ then computes $\hat{a}_{{\sigma }{i}} \leftarrow \mathsf {Enc}_{\hat{ pk }} (a_{{}{\sigma }{i}})$ and $\hat{z_{i}} \leftarrow \mathsf {Enc}_{\hat{ pk }} (z_{i})$ for all $\sigma \in \varSigma$ and $i \in [{2n}]$.

Fig. 6

Experiments for proving security of $\varPi _3({\mathcal {E}})$ against $\mathsf {proxy}$ adversaries. a Experiment $\mathbf {Expt}^{\mathtt {p}\text{- }\mathtt {dfa}}_{\varPi _3({\mathcal {E}})}$. b Experiment $\mathbf {Expt}^{\mathtt {p}\text{- }\mathtt {file}}_{\varPi _3({\mathcal {E}})}$

$U_{}$ then invokes $S_{2} (\phi , \langle c_{{k}{\sigma }} \rangle _{k \in [{\ell }],\sigma \in \varSigma })$ and simulates responses to $S_{2}$ ’s queries to $\mathsf {proxyOr}$ as follows (ignoring malformed invocations). Upon initializing $S_{2}$, $U_{}$ sends $n$, $\alpha _{\mathsf {init}}$ to $S_{2}$ and gets $\ell$ in return. For the $k$ th query of the form $\alpha , \beta$ ($0 \le k < \ell$), $U_{}$ selects $\gamma _{} \;\mathop {\leftarrow }\limits ^{\$}\;[{n}]$, as opposed to decrypting it as in a real execution c302 and c303. $U_{}$ then sets $\langle \hat{a}_{{\sigma }{i}}' \rangle _{\sigma \in \varSigma ,i \in [{2n}]} \leftarrow \mathsf {Shift} (\gamma _{},\langle \hat{a}_{{\sigma }{i}} \rangle _{\sigma \in \varSigma ,i \in [{2n}]})$ as done in c304 and returns $\langle \hat{a}_{{\sigma }{i}}' \rangle _{\sigma \in \varSigma ,i \in [{2n}]}$ to $S_{2}$. For the $(\ell +1)$th query of the form $\alpha , \beta$, $U_{}$ again randomly sets $\gamma _{} \;\mathop {\leftarrow }\limits ^{\$}\;[{n}]$ and then computes $\langle \hat{z_{i}} ' \rangle _{i \in [{2n}]} \leftarrow \mathsf {Shift} (\gamma _{},\langle \hat{z_{i}} \rangle _{i \in [{2n}]} )$ and sends $\langle \hat{z_{i}} ' \rangle _{i \in [{2n}]}$ to $S_{2}$. Finally, when $S_{2}$ outputs $b '$, $U_{}$ outputs $b '$.

This simulation is perfectly indistinguishable from the real system provided that $U_{}$ made correct guesses for $\gamma _{}$ on each round of the simulation. When that happens, $U_{}$ wins his game if and only if $S_{}$ wins his. So, we have $\textbf {Adv}^{\mathtt {ind}\text{- }\mathtt {cpa}}_{\mathsf {BGN}} (U_{}) \ge (\frac{1}{n})^{\ell +1} \textbf {Adv}^{\mathtt {s}\text{- }\mathtt {file}}_{\varPi _3({\mathsf {BGN}})} (S_{})$. $U_{}$ runs in time $t_{}' =t_{} +t_{\mathsf {Share}} + (2n m +2n +1) \cdot t_{\mathsf {Enc}_{}} +\ell m \cdot t_{\mathsf {Enc}_{}}$ due to the need to generate a secret key share for $S_{}$, to generate $\alpha _{\mathsf {init}}$, $\langle \hat{a}_{{\sigma }{i}} \rangle _{\sigma \in \varSigma ,i \in [{2n}]}$ and $\langle \hat{z_{i}} \rangle _{i \in [{2n}]}$, and to make $\ell m$ queries to its encryption oracle to create $\langle c_{{k}{\sigma }} \rangle _{k \in [{\ell }],\sigma \in \varSigma }$. $\square$

5.2 Security against proxy adversaries

In this section, we analyze the privacy of the DFA and file from $\mathsf {proxy}$ adversaries, specifically honest-but-curious ones. Our protocol’s security is limited to honest-but-curious proxies as an artifact of using $\mathsf {BGN}$ encryption, specifically because this forces us to employ $\kappa ' \ll \kappa$. Advances in additively homomorphic encryption that also supports “one-time” homomorphic multiplication and that also permits us to employ $\kappa ' \approx \kappa$, would permit us to prove security against malicious $\mathsf {proxy}$ adversaries, as well.⁴

The case of $\mathsf {proxy}$ adversaries in $\varPi _3({\mathcal {E}})$ differs more substantially from that in prior work [15]. For one, we need to formalize and prove a result about the degree to which the DFA is protected from the $\mathsf {proxy}$. Such an experiment for defining this type of security is shown in Fig. 6a. In this experiment, $P_{2}$ is invoked with encrypted coefficients $\langle \hat{a}_{{\sigma }{i}} \rangle _{\sigma \in \varSigma ,i \in [{2n}]}$, $\langle \hat{z_{i}} \rangle _{i \in [{2n}]}$ and the encrypted initial state $\alpha _{\mathsf {init}}$ for one of two DFAs output by $P_{1}$ (determined by random selection of $b$). $P_{2}$ can invoke $\mathsf {serverOr}$ first with an integer $n$ and a ciphertext (as in m301), in response to which $\mathsf {serverOr}$ returns $\ell$ (as in m302). In the next $\ell$ rounds, each time $\mathsf {serverOr}$ sends ciphertext $\alpha$ and partial decryption $\beta$ (as in m303) to $P_{2}$. $P_{2}$ then sends ciphertexts $\langle \hat{a}_{{\sigma }{i}}' \rangle _{\sigma \in \varSigma ,i \in [{2n}]}$ in response (as in m304). The next round, $\mathsf {serverOr}$ again sends $\alpha$ and $\beta$ (as in m305) to $S_{2}$, who responds with ciphertexts $\langle \hat{z_{i}} ' \rangle _{i \in [{2n}]}$ as in m306. $\mathsf {serverOr}$ sends one last message consisting one element in $C_{ pk }$ to $S_{2}$ as in m307. Finally, $P_{2}$ outputs a bit $b '$, and $\mathbf {Expt}^{\mathtt {p}\text{- }\mathtt {dfa}}_{\varPi _3({\mathcal {E}})} (P_{}) = 1$ only if $b ' = b$.

We prove DFA privacy against honest-but-curious $\mathsf {proxy}$ adversaries. A $\mathsf {proxy}$ adversary $(P_{1}, P_{2})$ is honest-but-curious if $P_{2}$ invokes $\mathsf {serverOr}$ exactly as $\varPi _3({\mathcal {E}})$ prescribes. The honest-but-curious advantage $\mathbf {hbcAdv}^{\mathtt {p}\text{- }\mathtt {dfa}}_{\varPi _3({\mathcal {E}})} (P_{})$ of adversary $P_{} = (P_{1}, P_{2})$ is

$$\begin{aligned} \mathbf {hbcAdv}^{\mathtt {p}\text{- }\mathtt {dfa}}_{\varPi _3({\mathcal {E}})} (P_{}) = 2 \cdot \mathbb {P}\left( {\mathbf {Expt}^{\mathtt {p}\text{- }\mathtt {file}}_{\varPi _3({\mathcal {E}})} (P_{}) = 1}\right) - 1 \end{aligned}$$

and $\mathbf {hbcAdv}^{\mathtt {p}\text{- }\mathtt {dfa}}_{\varPi _3({\mathcal {E}})} (t_{}, \ell , n, m) = \max _{P_{}} \mathbf {Adv}^{\mathtt {p}\text{- }\mathtt {dfa}}_{\varPi _3({\mathcal {E}})} (P_{})$ where the maximum is taken over all honest-but-curious client adversaries $P_{}$ running in total time $t_{}$ and producing files of length $\ell$ and a DFA of $n$ over an alphabet of $m$ symbols.

We now discuss the DFA privacy against an honest-but-curious $\mathsf {proxy}$ adversary.

Theorem 3

For $t_{}' =t_{} + t_{\mathsf {Share}} +(\ell m + 2n m +2n +2)\cdot t_{\mathsf {Enc}_{}}$,

$$\begin{aligned} \mathbf hbcAdv ^{\texttt {p}\text{- }\texttt {dfa}}_{\varPi _{3}\textsf {(BGN)}} (t, \ell , n, m) \le \mathbf Adv ^{\texttt {ind}\text{- }\texttt {cpa}}_\textsf {BGN} (t^{\prime }, 2(nm+n+1)) \end{aligned}$$

Proof

Given an adversary $P_{} =(P_{1}, P_{2})$ running in time $t_{}$ and selecting files of length $\ell$ and a DFA of $n$ states over an alphabet of $m$ symbols, we construct an IND-CPA adversary $U_{}$. On input a $\mathsf {BGN}$ public key $\hat{ pk } = \langle N$, $\mathbb {G}$, $\mathbb {G}'$, $e$, $g$, $h$, $\hat{g} \rangle$, $U_{}$ sets $d_{1} \;\mathop {\leftarrow }\limits ^{\$}\;\mathbb {Z}_{N}$, and invokes $P_{1} (\hat{ pk }, sk _{1})$ where $sk _{1} = \langle \mathbb {G}, \mathbb {G}', d_{1} \rangle$ to obtain $(\ell$, $\langle \sigma _{{}{k}} \rangle _{k \in [{\ell }]}$, $M_{0}$, $M_{1}$, $\phi )$. Note that $d_{1}$ is chosen from a distribution that is perfectly indistinguishable from that from which $d_{1}$ is chosen in the real system. If $M_{0}.Q \ne M_{1}.Q$ or $M_{0}.\varSigma \ne M_{1}.\varSigma$, then $U_{}$ aborts the simulation. Letting $\varSigma =M_{0}.\varSigma$, $m = |\varSigma |$ and $n = |M_{0}.Q |$, $U_{}$ computes $\langle a_{{0}{\sigma }{i}} \rangle _{\sigma \in \varSigma , i \in [{2n}]}$ $\leftarrow$ $\mathsf {ToPoly} (M_{0}.Q$, $\varSigma$, $M_{0}.\delta )$ and $\langle a_{{1}{\sigma }{i}} \rangle _{\sigma \in \varSigma , i \in [{2n}]}$ $\leftarrow$ $\mathsf {ToPoly} (M_{1}.Q, \varSigma ,M_{1}.\delta )$, and then sets $\hat{a}_{{\sigma }{i}}$ $\leftarrow$ $\mathsf {Enc}_{\hat{ pk }} ^{\hat{b}} (a_{{0}{\sigma }{i}}$, $a_{{1}{\sigma }{i}})$ for $\sigma \in \varSigma$ and $i \in [{2n}]$. It also computes $\langle z_{0i} \rangle _{i \in [{2n}]}$ $\leftarrow$ $\mathsf {ToPoly} (M_{0}.Q,\varSigma , M_{0}.\varDelta )$ and $\langle z_{1i} \rangle _{i \in [{2n}]}$ $\leftarrow$ $\mathsf {ToPoly} (M_{1}.Q, \varSigma , M_{1}.\varDelta )$, and then sets $\hat{z_{i}} \leftarrow$ $\mathsf {Enc}_{\hat{ pk }} ^{\hat{b}} (z_{0i}$, $z_{1i})$ for $i \in [{2n}]$. $U_{}$ then sets $\alpha _{\mathsf {init}}$ $\leftarrow$ $\mathsf {Enc}_{\hat{ pk }} ^{\hat{b}} (M_{0}.q_{\mathsf {init}}$, $M_{1}.q_{\mathsf {init}}) \odot _{\hat{ pk }} \mathsf {Enc}_{\hat{ pk }} (1)$. For all $k \in [{\ell }], \sigma \in \varSigma$, $U_{}$ also sets $c_{{k}{\sigma }}$ $\leftarrow$ $\mathsf {Enc}_{\hat{ pk }} (1)$ if $\sigma = \sigma _{{}{k}}$ and $c_{{k}{\sigma }}$ $\leftarrow$ $\mathsf {Enc}_{\hat{ pk }} (0)$ otherwise.

$U_{}$ invokes $P_{2} (\phi , \alpha _{\mathsf {init}}, \langle \hat{a}_{{\sigma }{i}} \rangle _{\sigma \in \varSigma ,i \in [{2n}]}, \langle \hat{z_{i}} \rangle _{i \in [{2n}]} )$ and simulates responses to $P_{2}$ ’s queries to $\mathsf {serverOr}$ as follows. Upon receiving the first message from $P_{2}$, $U_{}$ sends back $\ell$. In each round, $U_{}$ sets $r \;\mathop {\leftarrow }\limits ^{\$}\;\{0,1\}^{\kappa '}$, $\alpha \leftarrow \mathsf {Enc}_{\hat{ pk }} (r) \odot _{\hat{ pk }} \mathsf {Enc}_{\hat{ pk }} (1)$ and $\beta$ $\leftarrow$ $\hat{g} ^{r}\alpha ^{-d_{1}}$ so that $\alpha ^{d_{1}}\beta = \hat{g} ^{r}$. It then sends $\alpha$ and $\beta$ to $P_{2}$. In the last round, upon receiving $\langle \hat{z_{i}} \rangle _{i \in [{n}]}$ as in m306, $U_{}$ sets $\varTheta _{}$ $\leftarrow$ $\mathsf {Enc}_{\hat{ pk }} ^{\hat{b}} (M_{0} (\langle \sigma _{{}{k}} \rangle _{k \in [{\ell }]})$, $M_{1} (\langle \sigma _{{}{k}} \rangle _{k \in [{\ell }]}))$ where $M_{0} (\langle \sigma _{{}{k}} \rangle _{k \in [{\ell }]})$ denotes the evaluation result of $M_{0}$ on the file and similarly for $M_{1} (\langle \sigma _{{}{k}} \rangle _{k \in [{\ell }]})$. $U_{}$ then sends $\varTheta _{}$ to $P_{2}$. Finally, when $P_{2}$ outputs $b '$, $U_{}$ outputs $b '$ as well.

$U_{}$ ’s simulation is statistically indistinguishable (as a function of $\kappa '$) from a real protocol execution as long as $P_{2}$ is honest-but-curious. So $\textbf {Adv}^{\mathtt {ind}\text{- }\mathtt {cpa}}_{\mathsf {BGN}} (U_{})$ $\ge$ $\textbf {hbcAdv}^{\mathtt {p}\text{- }\mathtt {dfa}}_{\varPi _3({\mathsf {BGN}})} (P_{})$. $U_{}$ runs in time $t_{}' =t_{} + t_{\mathsf {Share}} +\ell m \cdot t_{\mathsf {Enc}_{}} + (2n m +2n +2)\cdot t_{\mathsf {Enc}_{}}$ due to the need to generate a secret key share for $P_{}$, to create $\langle c_{{k}{\sigma }} \rangle _{k \in [{\ell }],\sigma \in \varSigma }$, and to make $2n m +2n +2$ queries to its encryption oracle in order to generate $\langle \hat{a}_{{\sigma }{i}} \rangle _{\sigma \in \varSigma ,i \in [{2n}]}$, $\langle \hat{z_{i}} \rangle _{i \in [{2n}]}$, $\alpha _{\mathsf {init}}$ and $\varTheta _{}$ in the final round. $\square$

Next, we consider security against attacks on the encrypted files from a $\mathsf {proxy}$ adversary. Since the $\mathsf {proxy}$ no longer learns the final state of the DFA evaluation, we do not require the $\mathsf {proxy}$ adversary to choose two files that produce the same final result for the $\mathsf {user}$ ’s DFA, compared to previous definitions [15]. The experiment that we use to define file security against $\mathsf {proxy}$ adversaries $\mathbf {Expt}^{\mathtt {p}\text{- }\mathtt {file}}_{\varPi _3({\mathcal {E}})}$ is shown in Fig. 6b. There, $P_{1}$ produces two separate, equal-length plaintext files $\langle \sigma _{{0}{k}} \rangle _{k \in [{\ell }]}$, $\langle \sigma _{{1}{k}} \rangle _{k \in [{\ell }]}$ and a DFA $M_{}$. $P_{2}$ then receives the ciphertexts $\langle c_{{k}{\sigma }} \rangle _{k \in [{\ell }], \sigma \in \varSigma }$ for file $\langle \sigma _{{b}{k}} \rangle _{k \in [{\ell }]}$ where $b$ is chosen randomly. $P_{2}$ is also given oracle access to $\mathsf {serverOr}$ and finally $P_{2}$ outputs a bit $b '$, and $\mathbf {Expt}^{\mathtt {p}\text{- }\mathtt {file}}_{\varPi _3({\mathcal {E}})} (P_{}) = 1$ iff $b ' = b$. The advantage of an honest-but-curious adversary $P_{} = (P_{1}, P_{2})$ is defined as:

$$\begin{aligned} \mathbf {hbcAdv}^{\mathtt {p}\text{- }\mathtt {file}}_{\varPi _3({\mathcal {E}})} (P_{}) = 2 \cdot \mathbb {P}\left( {\mathbf {Expt}^{\mathtt {p}\text{- }\mathtt {file}}_{\varPi _3({\mathcal {E}})} (P_{}) = 1}\right) - 1 \end{aligned}$$

and $\mathbf {hbcAdv}^{\mathtt {p}\text{- }\mathtt {file}}_{\varPi _3({\mathcal {E}})} (t_{}, \ell , n, m) = \max _{P_{}} \mathbf {hbcAdv}^{\mathtt {p}\text{- }\mathtt {file}}_{\varPi _3({\mathcal {E}})} (P_{})$ where the maximum is taken over all honest-but-curious $\mathsf {proxy}$ adversaries $P_{}$ running in total time $t_{}$ and producing files of length $\ell$ and a DFA of $n$ over an alphabet of $m$ symbols. We now have:

Theorem 4

For $t_{}' =t_{} +t_{\mathsf {Share}} + (2n m +2n +\ell m +2)\cdot t_{\mathsf {Enc}_{}}$,

$$\begin{aligned} \mathbf hbcAdv ^{\texttt {p}\text{- }\texttt {file}}_{\varPi _{3}\textsf {(BGN)}} (t, \ell , n, m) \le \mathbf Adv ^{\texttt {ind}\text{- }\texttt {cpa}}_\textsf {BGN} (t^{\prime }, \ell m+1) \end{aligned}$$

Proof

Given an adversary $P_{} =(P_{1}, P_{2})$ running in time $t_{}$ and selecting files of length $\ell$ and a DFA of $n$ states over an alphabet of $m$ symbols, we construct an IND-CPA adversary $U_{}$. On input a $\mathsf {BGN}$ public key $\hat{ pk } = \langle N$, $\mathbb {G}$, $\mathbb {G}'$, $e$, $g$, $h$, $\hat{g} \rangle$, $U_{}$ sets $d_{1} \;\mathop {\leftarrow }\limits ^{\$}\;\mathbb {Z}_{N}$, and invokes $P_{1} (\hat{ pk }, sk _{1})$ where $sk _{1} = \langle \mathbb {G}, \mathbb {G}', d_{1} \rangle$ to obtain $(\ell , \langle \sigma _{{0}{k}} \rangle _{k \in [{\ell }]}, \langle \sigma _{{1}{k}} \rangle _{k \in [{\ell }]}, M_{}, \phi )$. Note that $d_{1}$ is chosen from a distribution that is perfectly indistinguishable from that from which $d_{1}$ is chosen in the real system. Let $\varSigma =M_{}.\varSigma$, $Q =M_{}.Q$, $\varDelta =M_{}.\varDelta$, $m = |\varSigma |$ and $n =|Q |$. For $k \in [{\ell }]$ and $\sigma \in \varSigma$, $U_{}$ sets $c_{{k}{j}} \leftarrow \mathsf {Enc}_{\hat{ pk }} ^{\hat{b}} (I_{0},I_{1})$ where

$$\begin{aligned} I_{0} \leftarrow \left\{ \begin{array}{ll} 1 &{}\quad \text{ if } \sigma = \sigma _{{0}{k}} \\ 0 &{}\quad \text{ otherwise }\\ \end{array}\right. \qquad \qquad I_{1} \leftarrow \left\{ \begin{array}{ll} 1 &{}\quad \text{ if } \sigma = \sigma _{{1}{k}} \\ 0 &{}\quad \text{ otherwise }\\ \end{array}\right. \end{aligned}$$

$U_{}$ also sets $\alpha _{\mathsf {init}}$ $\leftarrow$ $\mathsf {Enc}_{\hat{ pk }} (q_{\mathsf {init}}) \odot _{\hat{ pk }} \mathsf {Enc}_{\hat{ pk }} (1)$, $\langle a_{{}{\sigma }{i}} \rangle _{\sigma \in \varSigma , i \in [{2n}]}$ $\leftarrow$ $\mathsf {ToPoly} (Q, \varSigma ,\delta )$, and $\langle z_{i} \rangle _{i \in [{2n}]}$ $\leftarrow$ $\mathsf {ToPoly} (Q,$ $\varSigma , \varDelta )$. $U_{}$ then computes $\hat{a}_{{\sigma }{i}} \leftarrow \mathsf {Enc}_{\hat{ pk }} (a_{{}{\sigma }{i}})$ and $\hat{z_{i}} \leftarrow \mathsf {Enc}_{\hat{ pk }} (z_{i})$ for all $\sigma \in \varSigma$ and $i \in [{2n}]$.

$U_{}$ invokes $P_{2} (\phi , \alpha _{\mathsf {init}}, \langle \hat{a}_{{\sigma }{i}} \rangle _{\sigma \in \varSigma ,i \in [{2n}]}, \langle \hat{z_{i}} \rangle _{i \in [{2n}]} )$ and simulates responses to $P_{2}$ ’s queries to $\mathsf {serverOr}$ as follows. Upon receiving the first message from $P_{2}$, $U_{}$ sends back $\ell$. In each round, $U_{}$ sets $r \;\mathop {\leftarrow }\limits ^{\$}\;\{0,1\}^{\kappa '}$, $\alpha \leftarrow \mathsf {Enc}_{\hat{ pk }} (r) \odot _{\hat{ pk }} \mathsf {Enc}_{\hat{ pk }} (1)$ and $\beta$ $\leftarrow$ $\hat{g} ^{r}\alpha ^{-d_{1}}$ so that $\alpha ^{d_{1}}\beta = \hat{g} ^{r}$. It then sends $\alpha$ and $\beta$ to $P_{2}$. In the last round, upon receiving $\langle \hat{z_{i}} \rangle _{i \in [{n}]}$ as in m306, $U_{}$ sets $\varTheta _{}$ $\leftarrow$ $\mathsf {Enc}_{\hat{ pk }} ^{\hat{b}} (M_{} (\langle \sigma _{{0}{k}} \rangle _{k \in [{\ell }]})$, $M_{} (\langle \sigma _{{1}{k}} \rangle _{k \in [{\ell }]}))$. $U_{}$ then sends $\varTheta _{}$ to $P_{2}$. Finally, when $P_{2}$ outputs $b '$, $U_{}$ outputs $b '$.

Fig. 7

Time spent per file character in milliseconds, with pairing preprocessing disabled. a 1 thread per worker. b 4 threads per worker. c 16 threads per worker

$U_{}$ ’s simulation is statistically indistinguishable (as a function of $\kappa '$) from a real protocol execution as long as $P_{2}$ is honest-but-curious. So $\textbf {Adv}^{\mathtt {ind}\text{- }\mathtt {cpa}}_{\mathsf {BGN}} (U_{})$ $\ge$ $\textbf {hbcAdv}^{\mathtt {p}\text{- }\mathtt {file}}_{\varPi _3({\mathsf {BGN}})} (P_{})$. $U_{}$ runs in time $t_{}' =t_{} +t_{\mathsf {Share}} + (2n m +2n +1)\cdot t_{\mathsf {Enc}_{}} +(\ell m +1)\cdot t_{\mathsf {Enc}_{}}$ due to the need to generate a secret key share for $P_{}$, to generate $\langle \hat{a}_{{\sigma }{i}} \rangle _{\sigma \in \varSigma ,i \in [{2n}]}$ , $\langle \hat{z_{i}} \rangle _{i \in [{2n}]}$ and $\alpha _{\mathsf {init}}$, and to make $\ell m +1$ queries to its encryption oracle in order to create $\langle c_{{k}{\sigma }} \rangle _{k \in [{\ell }],\sigma \in \varSigma }$ and $\varTheta _{}$ in the last round. $\square$

6 Performance evaluation

6.1 Implementation

We implemented our optimized protocol $\varPi _3({\mathcal {E}})$ in Java using an open source Java pairing-based cryptography library jPBC [44], which is built on the original C pairing library PBC [42]. Our regular-expression-to-DFA conversion engine is built around the Java dk.brics.automaton library [45]. The complete implementation contains about 5,000 physical source lines of code.

For our evaluation, we chose a $\mathsf {BGN}$ public key of size $\kappa =1{,}024$ and a secondary security parameter of $\kappa ' =22$. To further improve performance, we utilized a fixed-base windowing exponentiation technique [39] to accelerate exponentiation operations on the $\mathsf {proxy}$ side. We also take advantage of pairing preprocessing at the $\mathsf {server}$ (see Sect. 4.2) and compare the performance with and without this optimization. In particular, pairing preprocessing produces approximately 600 KB of information per $\mathsf {BGN}$ ciphertext and so increases the required storage dramatically. As such, it may not be appropriate for use in some environments.

Fig. 8

Time spent per file character in milliseconds, with pairing preprocessing enabled. a 1 thread per worker. b 4 threads per worker. c 16 threads per worker

To exploit parallelism available in the protocol computation and the physical hardware, we implemented two levels of parallelization for the $\mathsf {server}$ and $\mathsf {proxy}$ programs. The first level is a thread pool of workers, each running a single $\mathsf {server}$ or $\mathsf {proxy}$ instance. Each $\mathsf {server}$ worker grabs an encrypted email from a shared queue of all the encrypted emails being searched and runs a protocol instance with its paired $\mathsf {proxy}$ worker independently. Each $\mathsf {server}$ or $\mathsf {proxy}$ worker can further spawn extra threads to assist in its computation. This level of parallelization is designed to take advantage of the computational independence found in many calculations in the protocol. For example, for $\mathsf {server}$ workers, the calculation in line s308 can be split among multiple threads before combining the results to obtain $\alpha$. Similarly for $\mathsf {client}$ workers, the $\mathsf {Shift}$ procedure in line c304 can also be partitioned across multiple threads.

6.2 Microbenchmarks

We first report microbenchmarks for our implementation. The experiments reported below were conducted using two machines connected via a 1 Gb/s network, each equipped with two quad-core Intel Xeon 2.67 GHz CPUs with simultaneous multithreading enabled. All $\mathsf {proxy}$ workers ran on one of these machines, and all $\mathsf {server}$ workers ran on the other.

To understand the performance cost of the protocol and the impact of its two parameters, i.e., the number of DFA states $n$ and the alphabet size $m$, we conducted experiments measuring the average time spent by the $\mathsf {server}$ and $\mathsf {proxy}$ for processing one character (or one round of protocol execution) for various combinations of $n$ and $m$. For this purpose, we generated encrypted files each consisting of 20 characters for $m =1$ to $m =50$. We then created random DFAs with number of states $n$ ranging from 1 to 50 and ran them against the files. We computed the average time spent per character by dividing the total time spent processing a file by 20. The results, with pairing preprocessing disabled, are presented in contour graphs in Fig. 7 where the times are binned into ranges, each shown as a band representing the range indicated in the sidebar legend.

To show that the computation of the protocol is highly parallelizable, in this experiment, we launched a single worker on both $\mathsf {server}$ and $\mathsf {proxy}$ machines and tested its performance when 1, 4, and 16 threads are spawned by each worker to assist in their computations, shown in Fig. 7a–c, respectively. It is clear from all three graphs that the protocol performance scales much better with the increase of $n$ than with $m$; the number of expensive pairing operations performed by the $\mathsf {server}$ in each round is equal to $m$, and the cost resulted from the increase of $m$ significantly outweighs that resulting from the increase of $n$. These results also show that the protocol is highly amenable to parallelization, with dramatically decreased processing time as the number of threads increases.

Since Fig. 7 clearly shows the impact of the pairing operations on the overall performance of the protocol, we went on to evaluate how much improvement pairing preprocessing can provide. In these experiments, we applied preprocessing on the file ciphertexts before conducting the same experiments as described above. The results are shown in Fig. 8. As expected, the overall protocol performance improves significantly in each of the multithreading cases, with darker bands reduced dramatically in size. More importantly, the protocol performance now scales much better with the increase of $m$ because of the significantly reduced cost of pairing operations on the $\mathsf {server}$ side.

To better understand the relative computational burden imposed on the $\mathsf {server}$ and $\mathsf {proxy}$ by the protocol, we also measured the average CPU time spent processing one character for the $\mathsf {server}$ and $\mathsf {proxy}$ processes for each combination of $n$ and $m$. To perform these tests, we instantiated one $\mathsf {server}$ worker and one $\mathsf {proxy}$ worker, each with a single thread. The CPU time takes into account the amount of time spent in both user and kernel modes. The results for the $\mathsf {server}$ and $\mathsf {proxy}$ are plotted in Fig. 9a, c, respectively. For the $\mathsf {server}$ side, it generally takes below or around 100 ms for one round of computation when $m$ is less than 10. The $\mathsf {proxy}$ side enjoys a slightly lighter computational cost and spends around or below 100 ms even for $m$ as high as 50 with $n$ less than 15. The results reveal that the $\mathsf {server}$ side takes more hit when $m$ increases due to the need to perform the pairing operations, while the $\mathsf {proxy}$ achieves a more balanced degradation with the increasing of $n$ or $m$.

Fig. 9

CPU time and network bandwidth measurements. a $\mathsf {server}$ CPU time per file character, with pairing preprocessing disabled. b $\mathsf {server}$ CPU time per file character, with pairing preprocessing enabled. c $\mathsf {proxy}$ CPU time per file character. d Network bandwidth per file character

We also conducted the same experiments when pairing preprocessing is used on the file ciphertexts in advance. Since it does not affect the $\mathsf {proxy}$ side processing time, we only show the $\mathsf {server}$ side CPU time in Fig. 9b. Compared with Fig. 9a, the CPU time spent is reduced significantly and the performance scales better as $m$ increases.

Since the protocol is interactive, we also measured the aggregate network bandwidth consumption between the $\mathsf {server}$ and $\mathsf {proxy}$ in one round of protocol execution. As shown in Fig. 9d, the bandwidth usage ranges from about 15 KB per round (i.e., per file character) for moderate $n$ and $m$ to as much as 640 KB per round when $n$ and $m$ are 50.

6.3 Case study: regular-expression search on encrypted emails

To further provide insight into the expected performance when using our protocol in real-world applications, we conducted a case study for performing regular-expression search on public key encrypted emails. We envision an email system in which the sender encrypts the email body using a traditional hybrid encryption scheme, in which the email body is encrypted using a symmetric encryption key that itself is encrypted by the receiver’s public key. To enable search operations, however, the sender also attaches an encrypted searchable “header” to the encrypted email body that consists of all the information from the email that allows searching. We now detail the design of this header.

6.3.1 Header information

Our current design allows searching on selected header fields of the email that are most commonly searched: (1) date; (2) sender email address; (3) sender name; and (4) subject line. The character-by-character encryption of the four headers is attached to the encrypted email body to enable searches. Since characters in each header are usually drawn from different distributions, we define the dictionary of a header as the set containing all possible characters and field-specific words that can be used in that header. Each header-field text is encoded using the dictionary before encryption, including sanitizing any characters not present in the dictionary (e.g., converting uppercase letters to lowercase, if only lowercase are included in the dictionary). We stress that this sanitization is only applied to the encrypted header that facilitates the search operations. The original field value in the email body is left intact.

The benefit of defining different dictionaries for different header fields is that adding field-specific words provides an opportunity for compressing the header fields, which is reminiscent of dictionary-based compression schemes. In addition, we envision dictionaries to be receiver-specific, e.g., distributed within the public key certificate for the receiver. Below, we describe how each dictionary is defined for each header in our evaluation.

Date:   Date is converted into YYMMDD, where year, month and day each consists of two digits of numerical values. For years, we expect to store emails dated from 1990 to 2050. So, we included “$\mathtt {90}$ ” to “$\mathtt {50}$ ”, as words, in the dictionary to encode the year. Similarly for months and days, we also added “$\mathtt {01}$ ” to “$\mathtt {31}$ ” into the dictionary. So, the dictionary is defined as $\{\mathtt {00}$, $\mathtt {01}$, $\mathtt {02}$, $\ldots$, $\mathtt {49}$, $\mathtt {50} \}$ $\cup$ $\{\mathtt {90}$, $\mathtt {91}$, $\ldots$, $\mathtt {99} \}$.

Sender address:   The sender email address is represented in the usual format, e.g., “alice@abc.com”, where the dictionary consists of “$\mathtt {a}$ ” through “$\mathtt {z}$ ”, “$\mathtt {0}$ ” through “$\mathtt {9}$ ”, “$\mathtt {.}$ ”, “$\mathtt {@}$ ”, “_”, and “$\mathtt {-}$ ”. We also added into the dictionary several common email service names like “$\mathtt {gmail}$ ”, domain names like “$\mathtt {com}$ ”, and “$\mathtt {enron}$ ” because the email dataset used in our evaluation (see Sect. 6.3.3) was from Enron. (Again, dictionaries can be receiver-specific). In total, the dictionary for this field is $\{\mathtt {a}$, $\ldots$, $\mathtt {z} \}$ $\cup$ $\{\mathtt {0}$, $\ldots$, $\mathtt {9} \}$ $\cup$ $\{\mathtt {@}$, $\mathtt {.}$, _, $\mathtt {-} \}$ $\cup$ $\{\mathtt {gmail}$, $\mathtt {yahoo}$, $\mathtt {aol}$, $\mathtt {hotmail}$, $\mathtt {enron}$, $\mathtt {com}$, $\mathtt {edu}$, $\mathtt {net}$, $\mathtt {org} \}$.

Sender name:   The sender name field represents the sender’s name with first name followed by the last name, separated by a space. The name dictionary consists of “$\mathtt {a}$ ” through “$\mathtt {z}$ ” and the space character.

Subject line:   The subject line is allowed to include arbitrary characters that can be typed from a keyboard. However, in practice, users rarely create search queries including special characters [1]. So, in our design, we restrict the dictionary to include “$\mathtt {a}$ ” through “$\mathtt {z}$ ”, “$\mathtt {0}$ ” through “$\mathtt {9}$ ”, and selected special characters including “$\mathtt {@}$ ”, “$\mathtt {!}$ ”, “$\mathtt {\%}$ ”, “$\mathtt {.}$ ”, and the space character.

6.3.2 Encoding

To enable DFA evaluations, we need to define the input alphabet $\varSigma$ that drives the DFA state transitions. The simplest way is to define it as the union of all the dictionaries defined for all header fields, which would result in an $m$ well above 50. However, the experiment results in Sect. 6.2 suggested that the protocol performance is very sensitive to a large $m$. So, we make the DFA alphabet $\varSigma$ and its size $m$ a user-defined parameter and designed a method to encode each word in the dictionary into a representation using the input symbols in $\varSigma$. Since the exact representation of the input symbols in $\varSigma$ is not important, for simplicity, we use numerical values to represent each symbol. For example, a size $m$ alphabet will consists of $\varSigma =\{0, 1, \ldots , m-1 \}$. Then, each word in a dictionary is represented using a distinct sequence of symbols from $\varSigma$. Each of the header fields is first encoded using this method and then encrypted symbol by symbol. The regular-expression query is encoded in the same way before converting it into a DFA.

6.3.3 Evaluation

In order to shed light on the expected performance when using our protocol to perform search operations in real-world email systems, we implemented a prototype search system and evaluated its performance based on the Enron email dataset [46], which is a publicly available real-world email corpus that contains roughly 0.6 million messages from about 150 then-employees of Enron. We randomly sampled 1,000 emails from the inboxes of all the users in the dataset and performed evaluations using selected representative search queries. In the experiments, we fixed a DFA alphabet of size $m =4$.

Motivated by the email search features found in ThunderBird [47], we selected four different queries to evaluate the protocol efficiency. For the date field, we selected a range query to search for all emails with date stamps between 2001/09/10 and 2002/04/20. The corresponding regular expression is

which results in a DFA of $n =23$ states using our conversion engine. For the sender address field, we selected a query to search for emails with sender address ending with the string “enron.com”. The resulting regular expression is $\mathtt {*enron.com}$ where $\mathtt {*}$ denotes zero or more occurrences of dictionary words, which converts into a DFA with $n =9$ states. For the name field, we selected the query to search for sender name containing the word “John”, which translates into the regular expression $\mathtt {*John*}$ ,with a corresponding DFA containing $n =17$ states. Lastly for the subject line field, we chose to search for emails with subject lines containing the word “meet” followed by “Jan” followed by a space and two arbitrary characters. This translates into a regular expression of $\mathtt {*meet~Jan~??*}$ where $\mathtt {?}$ denotes exactly one occurrence of a dictionary word, which results in a DFA of $n =36$ states.

We encrypted the bodies of 1,000 randomly selected emails using GnuPG [48], which results in an average size of 1.5 KB per email. We wrote our own tool to generate the encrypted searchable headers, which take up about 185 KB per email. To understand the performance impact when using the two parallelization techniques described in Sect. 6.1, we report performance numbers for various combinations of the number of workers and the number of threads that each worker spawns. The average time spent processing each email is shown in Table 1, which was calculated by dividing the total time to finish processing all 1,000 emails by 1,000. In order to demonstrate the performance improvement when pairing preprocessing is applied on email ciphertexts, we also performed pairing preprocessing for the email ciphertexts and stored the results on disk. The resulting protocol performance measurements (after preprocessing) are shown in the same table inside parentheses. The performance gain is very compelling, as it offers an approximately 30 % improvement over the version without preprocessing. However, the downside is that it needs significantly more storage space to store the pairing preprocessing information.

Table 1 Average time per email in seconds; numbers in parenthesis are when pairing preprocessing was applied on email ciphertexts in advance

The experiment results also demonstrate the benefit of concurrently processing multiple emails by instantiating multiple workers. In most cases, doubling the number of workers results in a decrease of the timing results by a factor of two. Meanwhile, spawning multiple threads for each worker has similar effect, although to a lesser extent. This can be seen by reading the entries horizontally, where the timing results are typically reduced by about 40 % as the number of threads per worker doubles. The date query (Table 1b) finishes fastest, averaging (in the best worker/thread configuration) only a quarter of a second (s) to process one email and 0.17 s when pairing preprocessing is used. This is due to the fact that the date field is very short for all emails. The sender name query (Table 1c) averaged 1.06 s per email and 0.79 s with pairing preprocessing. This is followed by the sender address query (Table 1a), which averaged 1.25 s per email and 0.89 s with pairing preprocessing. The subject line query (Table 1d) was the slowest, mainly due to the fact that subject lines in the email corpus are usually much longer than the other fields.

7 Conclusion

In this paper, we developed a novel protocol to perform private regular-expression evaluations on encrypted data stored at a $\mathsf {server}$. The protocol allows a $\mathsf {user}$ to delegate a regular expression search request to a $\mathsf {proxy}$, which interacts with the $\mathsf {server}$ and returns the search result to the $\mathsf {user}$. The privacy of the search query is provably protected against both an honest-but-curious $\mathsf {proxy}$ and an arbitrarily malicious $\mathsf {server}$ (provided that they do not maliciously collude), and so, is the privacy of the data itself. In Sect. 4, we described a number of optimization steps for the protocol that resulted in a substantial performance improvement for it, while retaining the protocol’s security properties. Many of these techniques should apply to the previous protocol [15] (summarized in Sect. 3.1) that provided our starting point for this work. We demonstrated the performance of our protocol using a working implementation, in both microbenchmarks and when applied to a real-world dataset (Sect. 6).