Abstract
Homomorphic encryption (HE) method can be used to realize arithmetic operations on encrypted data. This method, however, is limited owing to its low efficiency in performing certain functions, especially those involving several multiplications. As a solution, this paper proposes a new HEbased secure computation scheme, termed as the HE for stochastic computing (HESC); this scheme can homomorphically evaluate both the stochastic addition and multiplication operations, without any bootstrapping. This HESC scheme is constructed based on additive/multiplicative HE, which only supports homomorphic addition/multiplication, and realizes the homomorphic evaluation of stochastic multiplication. The HESC employs the features of stochastic computing (SC) for homomorphic stochastic operations, where stochastic additions and multiplications are performed using random multiplexing and bitparallel logic operations, respectively. This paper first presents a basic HESC scheme based on additive/multiplicative HE. It then presents an efficient HESC scheme that utilizes the parallelism of latticebased cryptography (i.e., plaintext packing and vectorized homomorphic evaluation). A new stochastic addition operation is also introduced in this study, which can be used for the HESC instantiated by latticebased cryptography. This new stochastic addition operation significantly improves the accuracy of the HESC, albeit with the tradeoff of increased ciphertext size. Accordingly, this paper also proposes a technique that can reduce the size of ciphertexts, while maintaining the accuracy of the scheme. The basic performance of the HESC implemented with various HEs is demonstrated, along with its applications in polynomial functions and an oblivious inference with a neural network. Lastly, the results thus obtained indicate that the proposed scheme is more advantageous than the conventional schemes. This paper is concluded with some implications/research directions for HESC from perspectives of cryptography and HE implementations.
Introduction
Over recent years, the demand for secure computation has garnered widespread attention for the exploitation of big data and outsourced computation, while preserving user privacy. Homomorphic encryption (HE) is a major secure computation scheme [1]; it is a public key cryptosystem that can perform additions and/or multiplications over ciphertexts via homomorphic evaluations. Since Gentry’s breakthrough work in 2009 [2], HE has received widespread attention. For instance, HE has been applied in privacypreserving statistical processing [3] and machine learning [4,5,6,7,8] applications involving the data provided by various parties. Furthermore, HEbased secure computation has been gaining increasing importance for the realization of machine learning applications with preserved privacy, owing to the remarkable developments in machine learning techniques over recent years.
The implementation cost of HE is largely dependent on the supported secret operations. Fully HE (FHE) supports both addition and multiplication, and it can perform the homomorphic evaluation of any polynomial function. However, it incurs considerably large implementation costs owing to the key/ciphertext size and computational complexity. Particularly, FHE requires a computationally expensive procedure called bootstrapping after evaluating multiplication(s), which is considered as a major bottleneck for FHE. By contrast, somewhat HE (SHE) can be implemented with a significantly smaller cost than FHE. However, it only supports the homomorphic evaluation of polynomial functions with lower degrees (i.e., multiplicative depth), which, in turn, limits its practical applicability. Thus, reducing the costs of FHE/SHE is necessary for broader applications.
A high computational accuracy is not a critical requirement in many realworld applications. Approximate computations with an acceptable accuracy degradation are commonly deployed in such applications. Such computations include rounding off in floatingpoint arithmetic [commonly deployed in central processing units (CPUs)] and probabilistic algorithms. A study conducted in 2017 proposed an HE called homomorphic encryption for arithmetic involving approximate numbers (HEAAN, or CKKS scheme [9]); this approach can homomorphically evaluate the rounding of plaintext over ciphertext. The CKKS has been widely employed in many applications, such as privacypreserving machine learning and oblivious inference [10], owing to its high efficiency. Nevertheless, to the best of our knowledge, an HE scheme that can perform probabilistic arithmetic over ciphertext has not been reported thus far, and probabilistic algorithms are expected to improve the HE efficiency, similar to that when using the CKKS.
This paper proposes an HE for stochastic computing (HESC), which supports both probabilistic addition and multiplication, based on stochastic computing (SC) [11]. SC is a probabilistic arithmetic system, where numbers are represented as probabilities, and additions and multiplications are performed using random numbers. SC has been utilized and investigated in the domain of lowpower digital circuit design, and its advantage has been shown in some practical applications such as digital filter [12, 13]. Recently, SC has been also employed for hardware implementation of neural network (NN) inferences, performance of which is sometimes shown to be superior to binary/floating point representations (e.g., [14,15,16,17]). The basic HESC integrates the additive/multiplicative HE (or SHE) with SC and can homomorphically evaluate both stochastic addition and multiplication, without any bootstrapping. This implies that the HESC can be implemented with a low computational cost, equivalent or comparable to that of the combined additive/multiplicative HE or SHE, while exhibiting better arithmetic flexibility. One major drawback of the HESC, however, is that any plaintext obtained through homomorphic evaluations includes noise, owing to the SC. Essentially, the HESC is useful for certain applications where noise is tolerable.
In this paper, the basic construction of the HESC and its homomorphic evaluation is first presented. Subsequently, HESC construction based on latticebased cryptography and a new stochastic addition method are described. The cost of HESCs is evaluated through prototype implementations, using some typical HEs, including the CKKS. The HESC is further applied for the evaluation of certain polynomial functions and an oblivious inference using neural networks (NNs) to classify the Iris flower dataset. The results indicate that the HESC can achieve sufficiently high accuracy with lower computational costs, as compared with an equivalent CKKSbased NN.
Remark 1
HESC involves noisy decrypted plaintext: The result determined by decrypting a ciphertext obtained via homomorphic evaluation is only approximately equal to the result of the corresponding function evaluation. In this regard, HESC does not satisfy the correctness property of the standard HE. This paper argues, however, that HESC is a useful primitive that can nevertheless be efficiently instantiated and that offers several advantages over conventional techniques, especially in contexts where the inputs and outputs are approximate in nature anyway. In the future, HESC can be applied for the development of privacy enhancing techniques and applied cryptography.
Background
Homomorphic encryption
HE belongs to a class of cryptographic schemes that employ a polynomialtime algorithm for the homomorphic evaluation of addition and/or multiplication operations over ciphertexts. In general, the scheme is a tuple of the algorithm defined as [18]

\(\textsf {HE}.\textsf {KeyGen}(1^\lambda )\rightarrow (\textsf {pk}, \textsf {sk})\): For a given security parameter, \(\lambda \), and a public key, \(\textsf {pk}\), a private key, \(\textsf {sk}\), is generated.

\(\textsf {HE}.\textsf {Enc}_\textsf {pk}(m)\rightarrow c\): For a given, \(\textsf {pk}\), and a plaintext, m, a ciphertext, c, is generated as output.

\(\textsf {HE}.\textsf {Eval}_\textsf {pk}(f,c,c')\rightarrow \textsf {HE}.\textsf {Enc}(f(c,c'))\): For \(\textsf {pk}\), two ciphertexts, c and \(c'\), and a function, f, a ciphertext of the evaluation result of \(f(c,c^{\prime })\), are generated as output.

\(\textsf {HE}.\textsf {Dec}_\textsf {sk}(c)\rightarrow m\): For a given ciphertext, c, and the private key, \(\textsf {sk}\), corresponding to \(\textsf {pk}\), the plaintext, m, is output.
Given that \(\textsf {HE}.\textsf {KeyGen}\), \(\textsf {HE}.\textsf {Enc}\), and \(\textsf {HE}.\textsf {Dec}\) are common algorithms in a public key cryptosystem, \(\mathsf HE.Eval\) homomorphically executes the evaluation function, f, over the ciphertext. The existing HE schemes are roughly classified into three categories based on their executable homomorphic operations [18].

(i)
Additive/multiplicative HE only supports homomorphic addition or multiplication. Typical examples of additive HEs include the Goldwasser–Micali (GM) encryption [19] and lifted(EC)ElGamal encryption [20,21,22,23]. The RSA [24] and Elgamal [20] encryptions are representative multiplicative HEs.

(ii)
The SHE can homomorphically evaluate addition and a limited number of multiplications. This is because the SHE utilizes noise for its encryption, which increases after the evaluation of each multiplication. Thus, the implementation cost of the SHE increases considerably if a large number of multiplication operations are required.

(iii)
The FHE can perform both addition and multiplication over ciphertext and homomorphically evaluate any polynomial function. The most common FHE schemes can be categorized into FHEW [25, 26]type, BGV/BFV [27, 28]type, and CKKS [9]type [29] schemes. The basic concept involves converting a ciphertext with noise into another one with less noise. This conversion is termed as bootstrapping, and it is used to overcome the constraint on the number of multiplications in the SHE.^{Footnote 1} However, this approach is computationally expensive and limits the applicability of the FHE.
Although several SHE and FHE applications can realize finitefield or fixedpoint arithmetic over ciphertexts, the CKKS can efficiently support real and complexnumber arithmetic [9]. The key feature of the CKKS is that it can homomorphically evaluate the rounding of plaintext, which significantly improves the implementation efficiency of the SHE/FHE, as compared with other conventional schemes where the plaintext size exponentially increases with the number of multiplications. Essentially, the CKKS achieves high implementation efficiency based on approximate computing (i.e., rounding). However, the CKKS still requires bootstrapping to perform a large number of multiplications [30].
Stochastic computing
SC is a probabilistic arithmetic system that was developed in the domain of digital circuit design [11]. It employs a specific rational number representation, called a stochastic number, which is represented by the occurrence probability of “1” in an Lbit sequence. There are two typical types of stochastic number representations: unipolar (UP) and inverted bipolar (IBP). An Lbit UP stochastic number \(X_\text {UP}\) represents a rational number \(\text {HW}(X)/L\), where \(\text {HW}(X)\) denotes the Hamming weight of X. The UP stochastic numbers represent rational numbers in the range of [0, 1], with a resolution of 1/L. An Lbit IBP stochastic number \(X_\text {IBP}\) represents a real number \(12\mathrm{HW}(X)/L\) in the range of \([1,1]\), with a resolution of 2/L. All the bit sequences for the stochastic number with the same Hamming weight represent an identical rational number. For example, a 4bit stochastic number, 0001, 0010, 0100, and 1000, represents an identical rational number of 1/4 and 1/2 for UP and IBP, respectively. This transformation from binary to stochastic numbers is performed randomly.
One major advantage of SC is that multiplication and addition can be performed using only Lbit logic gates and multiplexers, respectively. Let A and B be the Lbit stochastic numbers, where \(a_i\) and \(b_i\) are the ith bits, respectively. The stochastic multiplication, \(G = AB\), is given by the bitparallel AND and XOR of A and B (i.e., \(g_i = a_ib_i\) and \(g_i = a_i\oplus b_i\)) for UP and IBP, respectively, where \(g_i\) is the ith bit of G. The stochastic addition, \(D = A+B\), is implemented with a multiplexer that randomly selects \(a_i\) or \(b_i\) for the ith bit of D (denoted by \(d_i\)). If \(a_i\) (or \(b_i\)) is selected with a probability of 1/2, the stochastic addition becomes a normalized addition \((A+B)/2\). These SC features make it possible that very lightweight circuit design, as an SC adder/multiplier has only one logic depth, independently of the bitlength of SC number. Extremely, we can implement an SC adder/multiplier with only one logic gate if we perform the logic operation serially, or, we can implement them with a very low latency if we perform the logic operation in parallel. In fact, this lightweight feature of SC arithmetic is exploited in the domain of digital circuit design for applications such as digital filters [12, 13] and NN inferences (e.g., [14,15,16,17]). The SC computation result is validated only as an expected value owing to its probabilistic nature, which indicates that SC results always contain noise. To ensure that this noise remains within an acceptable range, the stochastic number length must be determined appropriately based on the application.
Proposed scheme
Basic concept and construction
The basic concept behind the HESC is that stochastic addition can be realized by multiplexing inputs without any arithmetic operations, whereas the homomorphic evaluation of stochastic multiplication is realized by either homomorphic addition or multiplication of the underlying HE.
The HESC involves the binarystochastic number transformation (B2S) of plaintext, encryption, and decryption via additive/multiplicative HE (or SHE) used for homomorphic stochastic multiplication, and stochasticbinary number transformation (S2B) of the decrypted plaintext.
Herein, the ciphertext of HESC is represented by a sequence of blocks, each of which is the encrypted result of a bit of a stochastic number representing the plaintext. The key length of the HESC is equivalent to the underlying HE; the ciphertext length and computational cost are proportional to the stochastic number length. The HESC does not require any bootstrapping because either homomorphic addition or multiplication of the underlying HE can help realize the homomorphic evaluation of both the stochastic addition and multiplication of the HESC.
Encryption
The encryption Algorithm 1 \(\textsf {HESC}.\textsf {Enc}\) uses a public key \(\textsf {pk}\), where \(\textsf {HE}.\textsf {Enc}_\textsf {pk}\) is the encryption with the underlying HE with \(\textsf {pk}\). First, the plaintext, M (\(M \in [0, 1]\) for UP and \(M \in [1, 1]\) for IBP), is converted to a stochastic number by B2S. A stochastic number can be easily produced from a binary number by a random number generator. Each bit of the stochastic number is then separately encrypted, i.e., \(\textsf {HE}.\textsf {Enc}_\textsf {pk}\) is performed L times to encrypt all the bits.
Decryption
In the decryption Algorithm 2, \(\textsf {HE}.\textsf {Dec}_\textsf {sk}\) denotes the decryption of the underlying HE with the private key, \(\textsf {sk}\). The HESC decryption follows the inverse procedure of the HESC encryption. Each ciphertext block in the HESC ciphertext is decrypted by \(\textsf {HE}.\textsf {Dec}_\textsf {sk}\) to acquire a stochastic number of the plaintext. Lastly, \(\textsf {S2B}\) provides the result of the operation.
Homomorphic evaluation
In the homomorphic evaluation Algorithms 3 and 4 for stochastic addition and multiplication, respectively, \(\textsf {HE}.\textsf {Eval}\) denotes the homomorphic evaluation of the underlying HE (i.e., homomorphic addition and multiplication for IBP and UP, respectively). The homomorphism of the underlying HE is employed for homomorphic stochastic multiplication (i.e., \(\textsf {HE}.\textsf {Eval}\)). One of the two input blocks is randomly selected for the homomorphic stochastic addition. In Algorithms 3, a stochastic number, S, with a Hamming weight corresponding to a selection signal, A, is generated by an external random number generator. The ith block of the addition result is then selected based on \(s_i\) (i.e., the ith bit of S), as described in Sect. 2.2. In Algorithms 4, \(\textsf {HE}.\textsf {Eval}\) is performed L times to obtain the resulting HESC ciphertext R as \(r_i = \textsf {HE}.\textsf {Eval}(c_i, c'_i)\), where \(c_i\) and \(c'_i\) denote the ith blocks of two inputs (i.e., HESC ciphertexts), C and \(C'\), respectively, and \(r_i\) denotes the ith block of R.
HESC with latticebased cryptography
Basic concept
The underlying concept is that certain HEs, which are based on latticebased cryptography (with a plaintext packing scheme [31]) such as BFV [27] and CKKS [9], can encode a vector (or polynomial) into one ciphertext block. They can process its homomorphic evaluation at once, as shown in CryptoNets [4]; this process is called single instruction multiple data processing.
During the HESC encryption, the \(\textsf {B2S}\) result is given by an Lbit stochastic number (i.e., Ldimensional vector). If the HE encrypts an ndimensional vector in one block for a stochastic number length of L, the HESC encryption is completed within \(\textsf {HE}.\textsf {Enc}\) L/n times. Consequently, the number of HE ciphertext blocks to L/n is reduced. Therefore, the latticebased cryptography reduces the computational cost and ciphertext size for the HESC. Such vectorized homomorphic evaluation is also beneficial for homomorphic stochastic multiplication.
Stochastic addition compatible with latticebased HEs
The classic homomorphic stochastic addition is no longer a step in the encoding process explained earlier because the homomorphic evaluation of random multiplexing in a bitwise manner is not applicable to such packed ciphertexts. More precisely, the random selection of the ciphertext blocks does not correspond to the conventional stochastic addition given as a random selection of bits. However, we can still evaluate the stochastic addition using a plaintextciphertext multiplication for some latticebased scheme including BFV. Let \(A^1\) and \(A^2\) be two stochastic numbers to be added. We generate a random bit string S used for stochastic addition, and let \({\bar{S}}\) be its complement. The stochastic addition is evaluated as \(A = SA^1 + {\bar{S}}A^2\), where \(SA^1\) and \({\bar{S}}A^2\) are computed using a plaintextciphertext multiplication. As such a multiplication is not expensive, we can evaluate stochastic additions even for latticebased HESC.
In addition, a new addition method for stochastic numbers is presented for improving the precision at a cost of ciphertext length increase, wherein two stochastic numbers are concatenated. Let \(A = (a_1, a_2, \dots , a_{L})\) and \(B = (b_1, b_2, \dots , b_{L})\) be the input stochastic numbers. In the new method, the normalized sum of A and B (i.e., \(D = (A+B)/2\)) is given as \(D = A \parallel B = (a_1, a_2, \dots , a_{L}, b_1, b_2, \dots , b_{L})\). The resulting D represents a rational number within the range of [0, 1] for UP or \([1, 1]\) for IBP with a resolution of 1/2L at the expense of the stochastic number length (i.e., 2L). The addition explained earlier is feasible even for the packed ciphertext because it can be realized by concatenating ciphertexts. Although the ciphertext length increases with each addition, the new method exhibits the following features: (i) applicability to the packed ciphertext and (ii) no noise/error during addition.^{Footnote 2}
The concatenated stochastic addition is then formally validated as Proposition 1.
Proposition 1
Let \(A^1, A^2, \dots , A^f\) be f stochastic numbers in IBP with a length of L. Their concatenation \(A = A^1 \parallel A^2 \parallel \dots \parallel A^f\) is a valid stochastic sum of \(A^1, A^2, \dots , A^f\) with a standard deviation of \(\frac{4}{L}\sum _{j=1}^fp^j(1p)^j\).
Proof
Consider the normalized sum of f stochastic numbers \(A^1, \dots ,A^j, \dots ,A^f\) in IBP. Let \(a_i^j\) be a random variable representing the ith bit of the stochastic number, \(A^j\) \((1\le i\le L)\), and \(A^j\) is can be considered as a random variable given as follows:
Here, the sum of f stochastic numbers is expressed as follows:
Using the expected values, these sums are given as follows:
where \(p^j\) is the expected value represented by \(A^j\).
Based on the variance, the error is given as follows:
The standard deviation of the error is inversely proportional to the root of the stochastic number length. The error after the addition decreases as the stochastic number length increases, which validates the concatenationbased sum as a SC addition. \(\square \)
Based on Eq. (2), the resulting sum is accurately normalized from the scaling coefficient. Therefore, the error after the addition is given by the sum of the errors of the input stochastic numbers; no error is added for the concatenated stochastic addition. This indicates that the HESC with the concatenated stochastic addition is advantageous over that with the conventional addition, if the stochastic number length of the resulting ciphertext is acceptable.
The resulting stochastic number has a bias derived from the concatenated bit position because each input stochastic number has a unique bias. Therefore, the subsequent operations after the concatenated addition must be carefully performed. Additionally, when using the HESC with CKKS, the result after the operation includes CKKSderived errors along with the SCderived errors. Consequently, the decrypted value in the HESC with CKKS is not necessarily an integer. Hence, the value must be rounded to the closest integer to accurately realize S2B after decryption.
Experimental evaluation of SC additions
A polynomial function, \(f(x)=\sum _{i=0}^n\), is evaluated by SC, and the resulting errors are analyzed to validate the effectiveness of the new addition method. The input stochastic number length is set to \(L=2048\) bits. Figure 1 shows the outline plots of f(x) evaluated using the conventional and new addition (i.e., concatenated stochastic addition) methods. Furthermore, Figs. 2 and 3 compare the mean and maximum errors of the two methods for \(n \le 10\). The input value is set to \([1, 1]\) in increments of 0.01 (i.e., 200 computations are plotted) to obtain the outline plots. The average and maximum errors are calculated from the difference between the outputs and the true values.
Figures 2 and 3 show that the that the error decreases as the number of averages increases. For example, for the averaging of 10 times, the mean errors are reduced by approximately 70% for both the methods, relative to the corresponding errors without averaging. The error of the stochastic operation results is explained by a binomial distribution, and therefore, the increase in the averaging times suppresses the variance of the binomial distribution (i.e., error). The values in Figs. 2 and 3 are experimental, and the probabilistic calculations may be different in each trial owing to the SC probabilistic feature. Theoretically, the error is inversely proportional to the square root of the number of averages.
The results also indicate that the errors in the new method are considerably smaller than those obtained by the conventional methods. For example, the mean error of the new method is 58.7% smaller than that of the conventional method for the averaging of 10 times. This is because, the resolution/accuracy of the new method is uncompromised after each stochastic addition, as shown in Eq. (4). The value computed via stochastic addition must be multiplied by a constant corresponding to the number of additions to obtain the correct value; however, in the conventional method, this produces a loss of resolution and accuracy. The concatenationbased method resolves this issue by extending the stochastic number length after addition. In addition, the growth of error by an increase in degree, n, is significantly suppressed by the proposed method compared to the conventional one. This feature would be useful for some practical applications with a nontrivial degree function as demonstrated in Sect. 4. Thus, the effectiveness of the new addition method on such a polynomial can be confirmed.
Note that this experiment only shows the comparison of conventional and proposed method; in practice, we can combine them to exploit tradeoff between the accuracy and ciphertext size by adaptively choosing the conventional and concatenationbased SC additions, if we need to perform the stochastic addition many times. The methodology to design circuit based on two stochastic addition with exploiting the tradeoff would be a future work.
Improvement by reduction in ciphertext size
HESC schemes with lattice cryptography use concatenated stochastic addition, which increases the number of ciphertexts after additive evaluation. Further, the schemes perform stochastic multiplication by adding ciphertexts. The resulting plaintexts can be nonnegative integers as in ordinary stochastic operations. In particular, the number of ciphertexts increases significantly when a number of addition and multiplication operations are performed. As the number of ciphertexts increases, the resulting decryption computation costs increase.
To address the above limitation, we introduce an improvement technique for reducing the number of ciphertexts while maintaining the above stochastic operations. Figure 4 shows an overview of the technique, which entails fusion and separation parts before and after decryption, respectively. The fusion part packs several ciphertexts by weighted addition while the separation part separates the ciphertext package with the weights used. Even with the additional two parts, the reduction in ciphertext size can reduce the total computation time.
In the following, let \(\boxed {m}\) be a ciphertext whose plaintext has a maximum value of m. We first assume that ciphertexts fused are not only \(\boxed {1}\). The basic idea is to fuse ciphertexts as a Wdecimal number, where W is the integer larger than the maximum value of m. Let \(a_1\) and \(a_2\) be nonnegative integers (i.e., ciphertexts) less than W. If \(A=a_1\cdot W+a_2\), then \(a_1\) and \(a_2\) can be separated as follows:
Here, one constant multiplication and one addition are performed for generating the fused ciphertext A, which is easily computed in HE. Applying the above operations recursively, we can fuse multiple ciphertexts into a single fused ciphertext. That is, we can fuse N ciphertexts \(\boxed {m_1},\boxed {m_2}\dots \boxed {m_N}\) into \(\mu \) as follows:
where \(W_1,W_2,\dots ,W_i,\dots ,W_N\) are constants that are larger than the maximum values of the corresponding plaintexts \(m_1,m_2,\dots ,m_i,\dots ,m_N\), respectively. If \(W=W_i\) for any i, the ciphertext \(\mu \) after the fusion is given as follows:
This fusion part has a restriction on the maximum number of fused ciphertexts owing to the increase in the plaintext space and noise after the operation. In Eq. (8) if \(m=m_i\) for any i and \(W=m+1\), then the value A after the fusion is given as follows:
This makes it possible to estimate the maximum number of fused ciphertexts under the condition of the plaintext space. It does not consider that the noise increases slightly with each addition. We can separate the fused value by computing the remainder divided by \(W_i\) recursively after decryption.
We then assume that the ciphertexts fused are only \(\boxed {1}\) (i.e., 0 or 1). To recover the binary number from the computed stochastic number, solely the number of “1”s in the decoded stochastic number sequence is required. When the decoded sequence is \(\{0, 1\}^N\), this is equivalent to obtaining the sum of elements. This means that the weight W should be 1 in the fusion. In this case, the fusion part is given solely by the addition of ciphertexts, and the increase in noise is extremely small. Therefore, compared with the case wherein ciphertexts fused are not only \(\boxed {1}\), we can merge more ciphertexts.
Performance evaluation
In this section, the implementation performance of the HESC is evaluated by using several applications. The homomorphic evaluations of a polynomial function are first conducted for a typical application.
Such polynomial functions have been used as activation functions in oblivious inference protocols such as CryptNets [4] because major nonlinear functions in a standard model (e.g., ReLU and Sigmoid) cannot be homomorphically evaluated over ciphertexts. Therefore, the evaluation of the performance of homomorphic evaluation of such polynomial functions presents an important benchmark. The HESC is then applied to the Iris classification for a more practical evaluation. This is a simpler dataset than the MNIST [32] and other datasets used by studies on oblivious inference using HE (CryptNets [4], LoLa [6], Falcon [5], etc.), but it is very effective as a baseline.
In the following sections, all stochastic numbers are expressed in IBP.
Basic implementation and comparison
Firstly, the fundamental performance of the HESC is evaluated by implementing it with the typical HEs. To this end, the HESC schemes with three additive HEs: GM encryption [19], liftedElGamal [20, 21], and liftedECElGamal [22, 23, 36] are implemented. Additionally, BFV [27] and CKKS [9] are employed in the prototype HESCs with latticebased cryptography, as described in Sect. 3.2, and their performance is then evaluated. The execution time is measured using an Intel Core i78665U (2.10 GHz) system with 16 GB of memory.
Table 1 lists the experimental conditions for the HE implementation, where the parameters are set to meet an equivalent security level (128 bits [37]) for each scheme. Table 2 compares the execution times of encryption, homomorphic evaluation, and decryption for the basic HESCs with the three additive HEs at the top. Among these, the HESC with the GM encryption is the fastest. This is because the plaintext space of the GM encryption is \(\mathbb {F}_2\), whereas those of the other two schemes are \(\mathbb {F}_p\) (where p is an odd prime). Thus, the GM encryption can be efficiently implemented as a basic HESC. Additionally, Table 2 compares the respective execution times for the HESCs with latticebased schemes (i.e., BFV and CKKS) at the bottom. The HESCs with latticebased cryptography require packing and unpacking operations that pack multiple plaintexts into a single ciphertext (i.e., packing [31], also known as encoding) at the beginning and unpack it at the end, respectively. However, the total execution time per bit is much smaller than that of the basic HESCs.
For example, if a 2048bit stochastic number can be packed into a single ciphertext, only one \(\textsf {HE}.\textsf {Enc}\) is called to complete the encryption; however, 2048 \(\textsf {HE}.\textsf {Enc}\) calls are required for the basic HESCs without packing techniques. This advantage of reducing the number of function calls is also reflected in the subsequent homomorphic evaluation. A comparison between the BFV and CKKS shows that CKKS is advantageous in terms of the evaluation time required for realizing stochastic addition and multiplication, although the packing/unpacking (i.e., encoding/decoding) of CKKS takes a little longer due to its unique features such as the usage of floatingpoint representation [38].
Based on the above comparison results, the HESCs with BFV and CKKS are considered for the performance evaluation in the following section.
Polynomial functions
Figures 5 and 6 show the evaluation results of the functions \(f(x)=x^2+x+1\) and \(g(x)=x^3+ x^2+x+1\) obtained using “HESC with BFV” and “HESC with CKKS.” Here, the target functions are chosen because, given a polynomial degree, evaluation of such all one polynomials is most severe as its evaluation would require the greatest number of twoinput additions among functions for the degree. Note that addition is critcal for HESC rather than multiplication; therefore, the choice of coefficients has not very big impact on the result. These figures also present the evaluation results of CKKS for comparison. Note that we do NOT perform any averaging for the plots. The figures show that the HESCs approximately compute the polynomial functions with some errors due to the nature of SC.
Figure 7 shows the mean and maximum errors of the three schemes, where the input value is given from \([1,1]\) with a resolution of 0.01. This means that the results of 201 values were evaluated to plot a single error value. The maximum error and average error were calculated using the absolute values of the difference between the calculated and true values, respectively. The horizontal axis represents the number of averaging times. For example, the results of \(201 \times 10\) values were evaluated in total for a case in which the averaging time was 10. Figure 7 shows that the error decreases with the increase in the number of averaging times in the HESCs. Although the error can be reduced by averaging the number of evaluations, it cannot be completely removed.
Particularly, such averaging cannot reduce the expected value of error to the extent of the inverse square root of the number of samples. Therefore, the evaluation result of the HESC is not as accurate as that of the CKKS, which produces almost no error. This error must be considered during the application of the HESC, as evaluated in the Iris classification below.
The error of g(x) in all the schemes is larger than that of f(x). This is because the degree of g(x) is higher than that of f(x), which increases the number of operations. Both the CKKSderived and stochasticderived errors increase with the increase in the depth of operations. “HESC with CKKS” contains both errors, but the effect of the CKKSderived error is trivial due to the rounding process to the stochastic numbers during decoding. Therefore, no significant difference is observed in the calculation accuracy between “HESC with CKKS” and “HESC with BFV.”
Table 3 shows the evaluation times for f(x) and g(x). For comparison, the result of BFV is shown in addition to the three schemes described above. The HESC is faster than the CKKS for both functions, which makes it one of the fastest conventional schemes. This is because it does not require homomorphic multiplication over ciphertexts, which is a major timeconsuming procedure for HEs. Additionally, the CKKS requires larger parameters to realize the multiplication for tolerating noise, which also degrades the computational efficiency. HESC with CKKS is usually advantageous in terms of the evaluation time when compared to HESC with BFV. Since the HESC does not require multiplication, the depth of operations for loworder polynomial functions does not significantly affect the evaluation time.
Figure 8 shows the total computation times of the above four schemes for up to 10degree polynomial functions given as \(\sum ^n_{i=0}x^i \ (2\le n\le 10)\). As the degree of functions becomes higher, the multiplication operations become deeper, thus requiring a larger computation time when evaluating with HE alone. In contrast, the computation times of HESC did not increase much because it can perform stochastic multiplication without using the multiplication of HE. For example, BFV and CKKS evaluated the 10degree function with 162.1 [ms] and 60.9 [ms], respectively, while HESC with CKKS evaluated the same function only with 6.5 [ms].^{Footnote 3}
NN oblivious inference
This section describes the effectiveness of the HESC through its application to an NN oblivious inference. The experimental setup is described as follows:

Iris flower dataset this dataset consists of the petal and sepal lengths of three different types of irises, as feature quantities. The training set contains 120 data elements, while the test set has 30 data elements.

NN model it consists of two fully connected layers of fourdimensional inputs and threedimensional outputs. There are four nodes in the middle layer, and the activation function is \(f(x)=x^2\).
In this experiment, each parameter is normalized and clipped at \([1,1]\) to use the SC. To create the model, we introduced Center Loss [39], which is one of the Deep Metric Learning methods. With this model, we can evaluate the similarity between input data and reduce the influence of SC errors because it can amplify the difference between the output values of the correct label and other labels.
Figure 9 shows the resulting inference accuracy, where the horizontal axis indicates the input stochastic number length used for the HESC. For reference, the model accuracy calculated by the floatingpoint arithmetic is shown as “float” in the figure, which is independent of the stochastic number length. The model is trained using common floatingpoint arithmetic, and the inference is performed on various SCs.
The legends “Conventional SC” and “Concatenated SC” correspond to the accuracies of the conventional SC and proposed SC with concatenationbased stochastic addition, respectively.
As mentioned in Sect. 3, the positiondependent biases must be considered after the concatenationbased addition. In order to address this issue, the activation function is calculated in an expanded form in “Concatenated SC,” such that the multipleinput stochastic addition is performed only once at the end of the computation. In this case, the length of the stochastic number (the number of ciphertexts) is already five times larger than that of the “conventional SC” at the input to the activation function. Consequently, the length of the stochastic number is 25 times longer after the activation function.
Figure 9 shows that both “Concatenated SC” and “Conventional SC” achieve successful inferences at the tradeoff of the stochastic number length, and Concatenated SC offers a higher accuracy. Using the input word length of 1024 bits, the accuracy can be increased to a level that is almost identical to “float.” Furthermore, the stochastic number length (i.e., the number of ciphertexts) after the inference of “Concatenated SC” is 93.75 times larger than the input length, 4. However, the increase in the resulting stochastic number length depends on the model size and the method used for the computation. For example, in the second fully connected layer, its bias is only added to the end (i.e., it is simply attached to the back in the HESC); therefore, it is unnecessary to perform B2S when encrypting the bias. By applying this concept, the resulting length is 75.75 times the length of the input, 4.
In this case, the parameter that renders the NN inference accuracy of floatingpoint arithmetic and SC comparable is employed (i.e., L = 1024). An opensource cryptographic library SEALPython [35] is used. The HESC is determined to be the smallest value considering the parameters in the library such that it can pack a 1024bit vector in a ciphertext block, and the CKKS is determined to be the smallest value such that it can accurately decrypt the inference result without any bootstrapping.
Table 4 shows the execution times per inference with “BFV,” “CKKS,” “HESC with BFV,” and “HESC with CKKS.”^{Footnote 4}
In a typical machine learning as a service (MLaaS) scenario, a network model is deployed on a server to provide inference services to client users. Here, it is assumed that each parameter of the model is already encrypted, and the client user encrypts the input data before sending them to the server for private inference. Note that “Enc” includes Packing and Encryption, and “Dec” includes Decryption and Unpacking. Additionally, the CKKS adopts the cuttingedge method to accelerate the homomorphic matrix multiplication [40].
The comparison result indicates that the HESC can perform the inference faster than the CKKS in terms of the encryption and layer computations because of the lower cost and smaller parameters. This is primarily because the HESC does not require homomorphic multiplication over ciphertexts. This reduces the size of the parameters and significantly contributes to the reduction in the computation time. For example, the firstlayer computation is approximately 306 times faster in “HESC with BFV” and approximately 642 times faster in “HESC with CKKS” when compared with that of CKKS. As the number of ciphertext blocks increases with the addition of the HESC, the secondlayer computation and decryption require a greater amount of time. The secondlayer computation is only approximately 16 times faster in “HESC with BFV” and approximately 20 times faster in “HESC with CKKS” when compared to that of the CKKS; the HESC decryption requires a longer time (approximately 55 times slower). Consequently, the advantage of the HESCs in the total time can be confirmed.
HESC with CKKS takes a slightly longer time to perform the data conversion from the integer (i.e., stochastic number) to complex numbers in B2S and the rounding process in \(\textsf {S2B}\), when compared to that with BFV. However, HESC with CKKS is slightly faster in terms of the total time because of the faster homomorphic SC evaluations. Therefore, it can be confirmed that the HESC is superior in terms of the computation time, while there are some factors to be considered, such as the increase in the ciphertext length and the decrease in the operational accuracy. This result indicates that the HESC is suitable for solving classification problems with an acceptable degree of error. Particularly, HESC with CKKS produces CKKSderived errors in addition to the SCderived errors, but these errors do not significantly affect the inference results, as demonstrated in the successful inference for the iris dataset.
Effect of fusion and separation
In this section, we evaluate the effect of the improvement technique described in Sect. 3.3. The target operation is an oblivious inference for the same Iris dataset as in the previous section, and the HESC is constructed by applying the BFV (i.e., HESC w/BFV). We consider that the value of W is set to 6 because the maximum value of the final output (i.e., plaintext) is at most 5 in the target. The parameters of BFV, called the plaintext space and the noise budget (i.e., stochastic number length or order), limit the number of ciphertexts that can be merged in the fusion. In the previous section, the parameters of BFV were 12289 and 1024 for the plaintext space and order, respectively. In the experimental setup, the maximum value of N is set to 5, that is, the maximum number of fused ciphertexts is 6, according to Eq. 9. For obtaining a sufficient noise budget, the evaluations were performed in the order of 2048 in the experiment.
Table 5 shows the evaluation results of HESC w/ and w/o the improvement method. With the improvement method, the total number of ciphertexts is reduced, and the number of decryption is decreased, which results in a significant reduction in computation times. In particular, we confirmed in the order of 2048 that the decryption time w/ the improvement method was approximately 5.6 times smaller, and the overall computation time is approximately 2.2 times smaller, compared with that w/o the method. This is because the number of ciphertexts was reduced from 101 to 18 by the fusion in the experiment, which showed a good agreement with the result. We can also see that the overall computation time w/ the improvement method is smaller than that w/o the improvement method in the order of 1024 (that is, HESC w/BFV or CKKS in Table 4). Figure 9 shows that the accuracy can be improved by increasing the order. Therefore, the results show that the improvement method can achieve a higher accuracy while reducing the computational cost.
Semantic security
An HESC ciphertext consists of a collection of ciphertexts for the underlying homomorphic encryption scheme (either with or without packing). As a result, it immediately follows by a standard hybrid argument that the HESC scheme is INDCPA secure as soon as the underlying homomorphic encryption scheme is INDCPA secure, which is the case for all the schemes considered in this work: Goldwasser–Micali, both variants of ElGamal and BFV. Although CKKSbased HESC achieved a high performance in our experiment, the original CKKS does not provide a semantic security against semihonest model [41]. Note that the purpose of our experiment is evaluating and comparing the fundamental and potential performance of HEs in the HESC context; in practice, some mitigation would be required for a secure use of HESCbased CKKS.
As a side note, the length of HESC ciphertexts can vary depending on the number of times concatenated stochastic addition is carried out, and the distribution of the noise also depends on the successive stochastic operations. In that sense, the HESC scheme is not a “function private” homomorphic encryption scheme (but the same is true of almost all practical SHE or FHE schemes including BFV and CKKS). Since the homomorphic operations are carried out identically to all ciphertexts, these points have no bearing on semantic security or the confidentiality of plaintexts.
Conclusion
This paper presented the HESC, a new HEbased secure computation scheme based on SC. This HESC can perform both homomorphic stochastic additions and multiplications without bootstrapping, based on an underlying singleoperation homomorphic encryption scheme (either additive or multiplicative). This is achieved at the cost of some noise being included in the decrypted plaintext. This paper also presented the constructions of basic and efficient HESCs, with the latter featuring latticebased cryptography, which improved the implementation efficiency using plaintext packing, a new stochastic addition method, and ciphertext integration. Some of the HESC schemes were validated with typical HEs, and their application for lowdegree polynomial functions was demonstrated, along with an oblivious inference using a neural network.
The validation results indicated that the HESC scheme could potentially mitigate a large part of the computational costs associated with conventional FHEs/SHEs. This HESC scheme can further be improved in terms of its usage in cryptography and related applications; detailed analyses of these improvements will be performed in future research. The development of hardware accelerators dedicated to HESCs is also a potential area of research interest. In addition, we are planning to develop a methodology for optimal circuit design where there are two stochastic additions to exploit the tradeoff between accuracy and ciphertext size.
Notes
The noise in stochastic addition is caused by the information discarded during multiplexing. The concatenated stochastic addition can avoid such accuracy/resolution loss observed during the conventional addition.
The parameters in each scheme are as follows:
BFV: degree \(=\) 8192, coeff size \(=\) 218,
plain space \(=\) 1032193
CKKS: degree \(=\) 8192, coeff size \(=\) 217
HESC w/ BFV: degree \(=\) 2048, coeff size \(=\) 54, plain space \(=\) 12289
HESC w/ CKKS: degree \(=\) 2048, coeff size \(=\) 54.
The parameters in each scheme are as follows:
BFV: degree \(=\) 8192, coeff size \(=\) 218, plain space \(=\) 16760833
CKKS: degree \(=\) 8192, coeff size \(=\) 210
HESC w/ BFV: degree \(=\) 1024, coeff size \(=\) 27, plain space \(=\) 12289
HESC w/ CKKS: degree \(=\) 1024, coeff size \(=\) 18.
References
Rivest, R.L., Adleman, L., Dertouzos, M.L.: On data banks and privacy homomorphisms. Found. Secure Comput. 4(11), 169–180 (1978)
Gentry, C.: Fully homomorphic encryption using ideal lattices. In: Proceedings of the FortyFirst Annual ACM Symposium on Theory of Computing, pp. 169–178 (2009)
Aono, Y., Hayashi, T., Wang, L., Moriai, S., et al.: Privacypreserving deep learning via additively homomorphic encryption. IEEE Trans. Inf. Forensics Secur. 13(5), 1333–1345 (2017)
GiladBachrach, R., Dowlin, N., Laine, K., Lauter, K., Naehrig, M., Wernsing, J.: Cryptonets: Applying neural networks to encrypted data with high throughput and accuracy. In: International Conference on Machine Learning, pp. 201–210. PMLR (2016)
Phong, L.T., Aono, Y., Hayashi, T., Wang, L., Moriai, S.: Fast spectral inference on encrypted data. Adv. Neural Inf. Process. Syst. 33, 2364–2374 (2020)
Brutzkus, A., GiladBachrach, R., Elisha, O.: Low latency privacy preserving inference. In: International Conference on Machine Learning, pp. 812–821. PMLR (2019)
Dathathri, R., Saarikivi, O., Chen, H., Laine, K., Lauter, K., Maleki, S., Musuvathi, M., Mytkowicz, T.: Chet: an optimizing compiler for fullyhomomorphic neuralnetwork inferencing. In: Proceedings of the 40th ACM SIGPLAN Conference on Programming Language Design and Implementation, pp. 142–156 (2019)
Jiang, X., Kim, M., Lauter, K., Song, Y.: Secure outsourced matrix computation and application to neural networks. In: Proceedings of the 2018 ACM SIGSAC Conference on Computer and Communications Security, pp. 1209–1222 (2018)
Cheon, J.H., Kim, A., Kim, M., Song, Y.: Homomorphic encryption for arithmetic of approximate numbers. In: International Conference on the Theory and Application of Cryptology and Information Security, pp. 409–437. Springer (2017)
Boura, C., Gama, N., Georgieva, M.: Chimera: a unified framework for B/FV, TFHE and HEAAN fully homomorphic encryption and predictions for deep learning. IACR Cryptol. ePrint Arch. 2018:758 (2018)
Gaines, B.R.: Stochastic computing systems. In: Advances in Information Systems Science, pp. 37–172. Springer (1969)
Chang, Y.N., Parhi, K.K.: Architectures for digital filters using stochastic computing. In: 2013 IEEE International Conference on Acoustics, Speech and Signal Processing, pp. 2697–2701. IEEE (2013)
Koshita, S., Onizawa, N., Abe, M., Hanyu, T., Kawamata, M.: Highaccuracy and areaefficient stochastic FIR digital filters based on hybrid computation. IEICE Trans. Inf. Syst. 100(8), 1592–1602 (2017)
Ardakani, A., LeducPrimeau, F., Onizawa, N., Hanyu, T., Gross, W.J.: VLSI implementation of deep neural network using integral stochastic computing. IEEE Trans. Very Large Scale Integr. Syst. 25(10), 2688–2699 (2017)
Liu, Y., Liu, S., Wang, Y., Lombardi, F., Han, J.: A stochastic computational multilayer perceptron with backward propagation. IEEE Trans. Comput. 67(9), 1273–1286 (2018)
Li, J., Ren, A., Li, Z., Ding, C., Yuan, B., Qiu, Q., Wang, Y.: Towards acceleration of deep convolutional neural networks using stochastic computing. In: 2017 22nd Asia and South Pacific Design Automation Conference (ASPDAC), pp. 115–120. IEEE (2017)
Ren, A., Li, Z., Ding, C., Qiu, Q., Wang, Y., Li, J., Qian, X., Yuan, B.: SCDCNN: Highlyscalable deep convolutional neural network using stochastic computing. ACM SIGPLAN Notices 52(4), 405–418 (2017)
Martins, P., Sousa, L., Mariano, A.: A survey on fully homomorphic encryption: an engineering perspective. ACM Comput. Surv. 50(6), 1–33 (2017)
Goldwasser, S., Micali, S.: Probabilistic encryption & how to play mental poker keeping secret all partial information. In: Providing Sound Foundations for Cryptography, pp. 173–201. ACM (2019)
ElGamal, T.: A public key cryptosystem and a signature scheme based on discrete logarithms. IEEE Trans. Inf. Theory 31(4), 469–472 (1985)
Liu, J., Asokan, N., Pinkas, B.: Secure deduplication of encrypted data without additional independent servers. In: Proceedings of the 22nd ACM SIGSAC Conference on Computer and Communications Security, pp. 874–885 (2015)
Agnew, G.B., Mullin, R.C., Vanstone, S.A.: An implementation of elliptic curve cryptosystems over F/sub 2/155. IEEE J. Sel. Areas Commun. 11(5), 804–813 (1993)
Virat, M.: A cryptosystem” à la” ELGamal on an elliptic curve over Fp [\(\epsilon \)]. In: WEWoRC 2005—Western European Workshop on Research in Cryptology. Gesellschaft für Informatik eV (2005)
Rivest, R.L., Shamir, A., Adleman, L.: A method for obtaining digital signatures and publickey cryptosystems. Commun. ACM 21(2), 120–126 (1978)
Ducas, L., Micciancio, D.: FHEW: bootstrapping homomorphic encryption in less than a second. In: Annual International Conference on the Theory and Applications of Cryptographic Techniques, pp. 617–640. Springer (2015)
Chillotti, I., Gama, N., Georgieva, M., Izabachène, M.: TFHE: fast fully homomorphic encryption over the torus. J. Cryptol. 33(1), 34–91 (2020)
Fan, J., Vercauteren, F.: Somewhat practical fully homomorphic encryption. IACR Cryptol. ePrint Arch. 2012:144 (2012)
Brakerski, Z., Gentry, C., Vaikuntanathan, V.: (Leveled) fully homomorphic encryption without bootstrapping. ACM Trans. Comput. Theory 6(3), 1–36 (2014)
Kim, A., Deryabin, M., Eom, J., Choi, R., Lee, Y., Ghang, W., Yoo, D.: General bootstrapping approach for RLWEbased homomorphic encryption. Cryptology ePrint Archive, Report 2021/691 (2021). https://ia.cr/2021/691
Cheon, J.H., Han, K., Kim, A., Kim, M., Song, Y.: Bootstrapping for approximate homomorphic encryption. In: Annual International Conference on the Theory and Applications of Cryptographic Techniques, pp. 360–384. Springer (2018)
Smart, N.P., Vercauteren, F.: Fully homomorphic SIMD operations. Des. Codes Cryptogr. 71(1), 57–81 (2014)
LeCun, Y., Cortes, C.: MNIST handwritten digit database (2010)
Laine, K., Player, R.: Simple encrypted arithmetic libraryseal (v2. 0). Technical Report (2016)
Microsoft SEAL (release 3.4). https://github.com/Microsoft/SEAL, October 2019. Microsoft Research, Redmond
Huelse. SEALPython. https://github.com/Huelse/SEALPython
Certicom Research. Recommended elliptic curve domain parameters. Standards for Efficient Cryptography (SEC) 2 (2000)
Barker, E., Barker, E., Burr, W., Polk, W., Smid, M., et al.: Recommendation for key management: Part 1: General. National Institute of Standards and Technology, Technology Administration (2006)
Babenko, M.G.E., Golimblevskaia, E.I., Shiriaev, E.M.: Comparative analysis of homomorphic encryption algorithms based on learning with errors. Proc. Inst. Syst. Program. RAS 32(2), 37–51 (2020)
Wen, Y., Zhang, K., Li, Z., Qiao, Y.: A discriminative feature learning approach for deep face recognition. In: ECCV, pp. 499–515. Springer (2016)
Halevi, S., Shoup, V.: Algorithms in helib. In: Annual Cryptology Conference, pp. 554–571. Springer (2014)
Li, B., Micciancio, D.: On the security of homomorphic encryption on approximate numbers. In: Annual International Conference on the Theory and Applications of Cryptographic Techniques. Lecture Notes in Computer Science, vol. 12696, pp. 648–677 (2021)
Author information
Authors and Affiliations
Corresponding authors
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Koseki, R., Ito, A., Ueno, R. et al. Homomorphic encryption for stochastic computing. J Cryptogr Eng (2022). https://doi.org/10.1007/s13389022002996
Received:
Accepted:
Published:
DOI: https://doi.org/10.1007/s13389022002996
Keywords
 Secure computation
 Homomorphic encryption
 Stochastic computing
 Oblivious inference