Programmable Bootstrapping Enables Efficient Homomorphic Inference of Deep Neural Networks

Abstract

In many cases, machine learning and privacy are perceived to be at odds. Privacy concerns are especially relevant when the involved data are sensitive. This paper deals with the privacy-preserving inference of deep neural networks.

We report on first experiments with a new library implementing a variant of the TFHE fully homomorphic encryption scheme. The underlying key technology is the programmable bootstrapping. It enables the homomorphic evaluation of any function of a ciphertext, with a controlled level of noise. Our results indicate for the first time that deep neural networks are now within the reach of fully homomorphic encryption. Importantly, in contrast to prior works, our framework does not necessitate re-training the model.

Keywords

Appendices

A Complexity Assumptions Over the Real Torus

In 2005, Regev [25] introduced the learning with errors (LWE) problem. Generalizations and extensions to ring structures were subsequently proposed in [22, 27]. The security of TFHE relies on the hardness of torus-based problems [6, 9]: the LWE assumption and the GLWE assumption [5, 20] over the torus.

Definition 1 (LWE problem over the torus)

Let $n \in \mathbb {N}$ and let $\boldsymbol{s} = (s_1, \dots , s_n) {\mathop {\leftarrow }\limits ^{\scriptscriptstyle \$}}\mathbb {B}^n$. Let also $\chi$ be an error distribution over $\mathbb {R}$. The learning with errors (LWE) over the torus problem is to distinguish the following distributions:

• ;

• .

Definition 2 (GLWE problem over the torus)

Let $N, k \in \mathbb {N}$ with N a power of 2 and let . Let also $\chi$ be an error distribution over $\mathbb {R}_N[X]$. The general learning with errors (GLWE) over the torus problem is to distinguish the following distributions:

• ;

• .

The decisional LWE assumption (resp. the decisional GLWE assumption) asserts that solving the LWE problem (resp. GLWE problem) is infeasible for some security parameter $\lambda$, where $n :=n(\lambda )$ and $\chi :=\chi (\lambda )$ (resp. $N :=N(\lambda )$, $k = k(\lambda )$, and $\chi :=\chi (\lambda )$).

B Algorithms

We use the notations of Sect. 4. The input of the (programmable) bootstrapping is an ciphertext that encrypts a plaintext $\overline{\mu }\in \mathbb {Z}/q\mathbb {Z}$ under the secret key $\boldsymbol{s} = (s_1, \dots , s_n) \in \mathbb {B}^n$.

1.1 B.1 Blind Rotation

The secret key bits $s_j$ used to encrypt the input ciphertext cannot be revealed. They are instead provided as bootstrapping keys; i.e., encrypted under some encryption key : for all $j=1, \dots , n$.

We then have:

At the end of the loop, $\mathrm {ACC}$ contains a encryption of under key .

1.2 B.2 Sample Extraction

The sample extraction algorithm extracts the constant coefficient $\overline{\mu }$ of polynomial in ciphertext as a ciphertext of $\overline{\mu }$. In more detail, let with for $1 \le j\le k$. Parsing as with for $1\le j\le k$ and , it can be verified that $\boldsymbol{\overline{c}'} :=(\overline{a}'_{1,0}, -\overline{a}'_{1,N-1}, \dots , -\overline{a}'_{1,1}, \dots ,\overline{a}'_{k,0},-\overline{a}'_{k,N-1},\dots ,-\overline{a}'_{k,1}, \overline{b}'_0) \in (\mathbb {Z}/q\mathbb {Z})^{kN+1}$ is a encryption of $\overline{\mu }$ under the key $\boldsymbol{s'} = (s'_1, \dots , s'_{kN}) \in \mathbb {B}^{kN}$ where $s'_{l+1+(j-1)N} :=s'_{j,l}$ for $1 \le j \le k$ and $0 \le l \le N-1$.

1.3 B.3 Key Switching

The key switching technique can be used to switch encryption keys in different parameter sets [7, § 1.2]. Its implementation requires key-switching keys, i.e., encryptions of the key bits of $\boldsymbol{s}'$ with respect to the original key $\boldsymbol{s}$. Assume we are given the key-switching keys for all $1 \le i \le kN$ and , for some parameters and defining a gadget decomposition (see Sect. 3.3). Adapting [10, § 4.1] teaches that, on input ciphertext under the key $\boldsymbol{s'} = (s_1, \dots , s_{kN}) \in \mathbb {B}^{kN}$,

where is an encryption of $\overline{\mu }$ under key $\boldsymbol{s}$, provided that the resulting noise keeps small.