Privacy-Preserving Classification on Deep Learning with Exponential Mechanism

Abstract

How to protect the privacy of training data in deep learning has been the subject of increasing amounts of related research in recent years. Private Aggregation of Teacher Ensembles (PATE) uses transfer learning and differential privacy methods to provide a broadly applicable data privacy framework in deep learning. PATE combines the Laplacian mechanism and the voting method to achieve deep learning privacy classification. However, the Laplacian mechanism may greatly distort the histogram vote counts of each class. This paper proposes a novel exponential mechanism with PATE to ensure the privacy protection. This proposed method improves the protection effect and accuracy through the screening algorithm and uses the differential privacy combination theorems to reduce the total privacy budget. The data-dependent analysis demonstrates that the exponential mechanism outperforms the original Laplace mechanism. Experimental results show that the proposed method can train models with improved accuracy while requiring a smaller privacy budget when compared to the original Pate framework.

1 Introduction

Machine learning theory and application technology are evolving iteratively in tandem with the availability of massive datasets. Deep learning models trained on big datasets have shown a few promising findings. In related research, deep learning has become a key technology for solving difficult problems in a series of studies, such as image classification [1, 2], target detection [3, 4], object recognition [5, 6], etc. In addition, research and technologies related to deep learning are also continuing to develop and expand. Deep learning models such as CNN (convolutional neural networks), RNN (recurrent neural networks), and GAN (generative adversarial networks) have made amazing progress in fields such as data mining, machine learning, machine translation, natural language processing, and multimedia learning [7,8,9,10].

Deep learning networks or algorithms trained on large-scale datasets, on the other hand, frequently contain sensitive individual private information, such as personal images, biological information, health history, daily behaviour routines, and action trajectories [11, 12]. These private data will be directly accessible if these sensitive data-trained models are openly disseminated. Potentially hostile individuals will be able to use methods such as repeated query, analysis of query result inference, extraction attack, and model reverse assault to access sensitive private records illegally [13, 14]. The security issue of data privacy in the application scenarios of deep learning models trained on vast volumes of data is a serious concern that requires immediate attention.

Differential privacy (DP) is a potent privacy protection method and algorithm that has undergone years of development [15, 16]. The privacy protection of the deep learning model's training dataset has been a problem that several researchers have attempted to overcome using DP [17,18,19]. Abadi et al. presented a DP-SGD differential private stochastic gradient descent algorithm, which is extensively used for deep neural network training sensitive data to achieve DP. They added Gaussian noise to the gradient of each iteration during the SGD optimization phase. The more intricate the model or network is, the less effective this approach will be. Input disturbance (adding noise to disturb the input data), parameter disturbance (disturbance gradient and other deep learning intermediate parameters), output disturbance (the result of disturbance output), and target disturbance (perturbation target function) are the four typical ways that differential privacy is applied to deep learning. To the best of our knowledge, a very promising technical framework for addressing the issue of protecting deep learning data privacy is PATE (Private Aggregation of Teacher Ensembles) technology, which is based on the addition of noise disturbance to the output. PATE can still ensure the effectiveness of learning models or deep networks under the presumption of obtaining strong data privacy protection.

PATE combines multiple models trained by different sub-datasets. Since these subsets contain sensitive data, direct publication of these models is prohibited. The only purpose of the trained models is to serve as "teachers" for voting. The "student" models learned the ability to predict an outcome without actually gaining access to any sensitive data by introducing noise to all teacher votes. Because the predicted outputs are not only dependent on one teacher's vote or one particular subset of data, even a comprehensive analysis of the model cannot expose sensitive information. Even if an attacker were to look at the "student" models and outputs, the PATE approach could still maintain differential privacy. The PATE technique is based on knowledge transfer from "teacher" to "student," which is then aggregated. Because it only sets very modest restrictions on teachers' training procedures, it can be applied to any deep learning model.

This paper introduces a novel method for screening samples based on an exponential mechanism PATE that increases the precision of identification and classification with the goal of better safeguarding sensitive data. An exponential mechanism scoring function based on the PATE framework is also proposed to generate the best probability model. This paper combines and analyses the cost of data-dependent privacy protection under the exponential mechanism, protects data-dependent parameters, and prevents sensitive privacy data from leaking.

This paper is organized as follows: Sect. 2 reviews similar studies. Backgrounds are described in Sect. 3. The exponential mechanism with PATE is proposed in Sect. 4. Experiments are implemented and evaluated in Sect. 5. Section 6 concludes this paper.

2 Related Studies

In 2002, Latanya Sweeney et al. introduce k-anonymity, an anonymizing data approach [20]. Given an original dataset, an anonymized dataset can be constructed by obfuscating the properties of the data, assuring that the individual identity in it will not be recovered under the condition of ensuring the data's experimental availability. However, k-anonymity does not contain any randomization properties, so the attacker can still infer the private information related to the individual from the dataset satisfying the k-anonymity property. In addition, k-anonymity is also vulnerable to consistency attack and background knowledge attack. Consistency attack basically indicates that even if the criterion is met, these k sensitive characteristics may also be inferred if there are k sensitive values of the same attribute in the dataset. By determining the relationship between one or more quasi-identity information characteristics and sensitive attributes, the attacker can reduce the range of possible guesses for sensitive attributes via the background knowledge attack [21]. Due to the above defects of k-anonymity, some improved schemes are proposed, such as l-diversity by Machanavajjhala et al. [22] and t-closeness by Li Ninghui et al. [23]. However, they all have a common shortcoming: these approaches all rely on the background knowledge of the attacker and fail to make reasonable assumptions about the attack model.

In 2006, Dwork et al. introduce the differential privacy technique to provide a quantitative model for the degree of privacy leakage [8]. In differential privacy, random noise is added to ensure that the public output will not change significantly due to whether an individual is in the dataset. The attacker cannot guess which result corresponds to which dataset based on the published results. The differential privacy model does not need to rely on how much background knowledge the attacker has and provides a higher level of semantic security for private information, so it is widely used as a new type of privacy protection model. Differential privacy techniques do not ensure the privacy of the integrity of the dataset, but to protect the privacy of each individual in the dataset. Its concept requires that every single element in the dataset has a limited influence on the output. Therefore, after observing the query results, the attacker cannot infer which individual's influence in the dataset makes the query return such results, and therefore, it is impossible to infer information about individual privacy from the query results. In other words, the attacker is unable to determine whether a certain person is included in such a dataset. Mathematical details about DP can be found in “Backgrounds”.

Papernot et al. suggest private aggregation of teacher ensembles (PATE) to give strong privacy guarantee broadly applicable for machine learning models for the privacy protection of sensitive data in training (shown in Fig. 1) [13]. PATE splits into multiple teachers to train models on disjoint sensitive datasets, while individual teachers vote on the label of each record in the unlabelled public dataset. Then, using a DP-compliant nosiy-max technique, all of the teachers' votes are combined by adding DP-compliant noise to each label’s vote as shown in Fig. 2. The knowledge from the teacher training set can be transferred to the student model by label voting, and the student model is then semi-supervisedly trained and published on the public dataset. In this way, the protection of the training data of the teacher model is guaranteed, and stricter privacy data guarantee and higher recognition accuracy are realized. PATE can be used to analyse datasets with a variety of properties and can be integrated with different deep learning techniques or models. The accuracy of the machine learning algorithm or model will suffer slightly as a result of the noise disturbance that occurs when the teacher passes the training results to the student model. This loss in accuracy is related to the level of privacy protection that PATE can offer.

Fig. 1

Overview of PATE frame

Fig. 2

Add noise to teachers’ votes

Jordon J et al. introduce a method to ensure the generator of the generative adversarial nets (GAN) framework to satisfy differential privacy [24]. It can be applied on generating synthetic data to be validated and trained, meanwhile the privacy of the original data is not compromised during competitions. They change the framework of PATE so that the GANs can be used.

In 2020, Dominik Fay et al. investigate how PATE can be adapted for semantic segmentation [12]. They use dimensionality reduction to answer high-dimensional queries in the context of PATE. Low-dimensional representations of segmentation masks are built so that the student can obtain through low-sensitivity queries to the private aggregator. They found that their variant of PATE can achieve higher segmentation quality than that of noisy Federated Averaging with institute level privacy.

G-PATE is proposed by Long Y et al. in 2021 to train a scalable differentially provate data generator with high generated data utility [14]. G-PATE combines private aggregations of different discriminators to satisfy strong privacy guarantees. G-PATE uses a private gradient aggregation that an ensemble of teacher discriminators is used to train the student data generator, so that all information flowing from teacher discriminators to the student generator satisfy differential privacy.

3 Backgrounds

In this section, definitions and theorems of differential privacy are recalled. The theoretical backgrounds and mathematical preliminaries of PATE are also introduced.

3.1 Differential Privacy

Definition 1

Algorithm ${\mathcal{M}}:{\mathcal{D}}\to {\mathcal{R}}$ satisfies $(\varepsilon ,\delta )$-differential privacy when any two datasets ($\mathcalligra{d},{\mathcalligra{d}}^{\prime}\in {\mathcal{D}}$, $card(d)-card({\mathcalligra{d}}^{\prime})\le 1$) and any output subset $S\subseteq {\mathcal{R}}$ hold:

$$\Pr [{{\mathcal{M}}}({\mathcalligra{d}}) \in S] = {e^\varepsilon }\Pr [{{\mathcal{M}}}({\mathcalligra{d}}^{\prime} )\in S^\prime ] + \delta ,$$

(1)

where the possibility ${\delta}$ is between 0 and $\frac{1}{|\mathcalligra{d}|}$ [25].

Differential privacy can be very useful in group privacy because its robustness to composability information. If any of the components satisfies differential privacy, then their composition also is an algorithm that satisfies the differentially privacy. In 2010, McSherry et al. [26] proposed two important properties of differential privacy: sequential composition and parallel composition.

Sequential composition property: For any k algorithms, respectively satisfy ${\varepsilon }_{1}$-differential privacy, ${\varepsilon }_{2}$-differential privacy, …, ${\varepsilon }_{k}$-differential privacy. When they are applied to the same dataset, ($\sum_{i=1}^{k}{\varepsilon }_{i})$-differential privacy is satisfied.

The second property—parallel composition is that: divide a dataset D into k sets, respectively, D₁, D₂,…,D_k, and let M₁, M₂,…,M_k be k differential privacy algorithms satisfying ${\varepsilon }_{1},{\varepsilon }_{2,}{\dots ,\varepsilon }_{k}$-differential privacy, respectively; then the result of sequence: M₁(D₁), M₂(D₂), …,M_k(D_k) satisfies $\mathop {{\text{max}}}\nolimits_{i\in [\mathrm{1,2},\dots ,k]} {\varepsilon }_{i}$-differential privacy.

Daniel Kifer et al. [27] suggested and demonstrated two more properties of differential privacy in 2010. The third property is that post-processing the output of a privacy mechanism should not decrease privacy. Given that any algorithm or mechanism M₁ satisfies ε-differential privacy, for any mechanism M₂ (M₂ can be any algorithm even the one that does not satisfy differential privacy), then M₃ = M₂ (M₁) satisfies ε-differential privacy.

The fourth property states: Given two algorithms/mechanism M₁ and M₂ satisfying $\varepsilon$-differential privacy. For any probability p $\in$ [0,1], let M_p be a randomized algorithm that on input i outputs M₁(i) with probability p, and the M₂(i) algorithm with the probability of 1-p, then the M_p mechanism satisfies ε-differential privacy.

Dwork et al. [28] proposed the concept of global sensitivity and Laplacian mechanism in 2006. Through global sensitivity, if the level of the synthetic noise can be adjusted, then a privacy protection mechanism that satisfies differential privacy is built. By controlling the size of the synthetic noise through global sensitivity, a privacy protection mechanism that meets the requirements of differential privacy can be realized.

Definition 2

For a query function $f:D\to {\mathbb{R}}$, where D is a dataset, and ${\mathbb{R}}$ is the return result of the query function. On a pair of arbitrary adjacent datasets D and D′, the global sensitivity is defined as follows:

$$S\left(f\right)=\underset{D, D\mathrm{^{\prime}}}{{\text{max}}}\parallel f\left(D\right)-f(D{^{\prime}}){\parallel }_{1,}$$

(2)

where $\parallel f\left(D\right)-f(D{^{\prime}}){\parallel }_{1}$ is the Manhattan distance between $f\left(D\right)$ and $f(D\mathrm{^{\prime}})$.

The Laplacian mechanism is a widely used privacy protection mechanism for numerical queries. For numerical query results, the Laplacian mechanism satisfies differential privacy protection by adding a Laplace noise. The Laplace noise meets $Lap(0,\frac{\Delta f}{\epsilon })$ distribution to query results, where $\Delta f$ is the global sensitivity of f. That is, $R\left(D\right)=f\left(D\right)+x$, where f is the query function, and x is the Laplace noise. In addition, the mean value of the added Laplace noise is required to be zero, so that the output R(D) is the unbiased estimate of f(D).

Theorem 1

Post-processing [29] random algorithm $M:{\mathbb{N}}^{\left|\mathcal{X}\right|}\to R$ Satisfy $\left(\varepsilon ,\delta \right)$- Differential privacy, arbitrary random mapping $f:R\to {R}^{\prime}$. Then $f\circ M:{\mathbb{N}}^{\left|\mathcal{X}\right|}\to {R}^{\prime}$ is also $\left(\varepsilon ,\delta \right)$-differentially private.

Theorem 2

Advanced composition [7]. For$\forall \varepsilon ,\delta ,{\delta }^{\prime}\ge 0$, if a random algorithm ${{\mathcal{M}}}_{i}$ $\left(1\le i\le k\right)$ satisfies $\left(\varepsilon ,\delta \right)$-differential privacy, then the combined algorithm $\left({{\mathcal{M}}}_{1},{{\mathcal{M}}}_{2},\dots ,{{\mathcal{M}}}_{k}\right)$ satisfies $\left({\varepsilon }^{\prime},k\delta +{\delta }^{\prime}\right)-{\text{dp}}$, where:

$${\varepsilon }^{\prime}=\sqrt{2k{ln}\left(\frac{1}{{\delta }^{\prime}}\right)}\varepsilon +k\varepsilon \left({e}^{\varepsilon }-1\right).$$

(3)

3.2 PATE

Divide the sensitive dataset into $m$ disjoint sub-datasets: ${D}_{1},\dots ,{D}_{m}$. Each of these $m$ classifiers ${T}_{1},\dots ,{T}_{m}$ (referred to as teachers) is independently trained on a sub-dataset, and then PATE voting is performed on the samples x. Given $m$ teachers, $n$ classes and the input feature vector x, then:

$${n}_{j}\left({\varvec{x}}\right)=\left|\left\{{T}_{i}:{T}_{i}\left({\varvec{x}}\right)=j\right\}\right|\left(j=1,...,n\right).$$

(4)

${n}_{j}\left({\varvec{x}}\right)$ is the number of teachers who voted for the output class for the feature vector. For an input x, the output of the $PAT{E}_{\lambda }$ mechanism is defined as:

$$PAT{E}_{\lambda }\left(x\right)=arg\underset{j\in \left[n\right]}{max}\left[{n}_{j}\left({\varvec{x}}\right)+{Y}_{j}\right],$$

(5)

where ${Y}_{1},\dots ,{Y}_{m}$ is a Laplace random variable with variance λ and mean 0.

Perform PATE voting on a public dataset to train a student model and use the privacy-preserved student model for classification tasks [13]. Due to the lack of interpretability of neural network-based machine learning models, this paper uses the output probability model of PATE to quantitatively analyse the noise adding algorithm. The number of teachers is m, with T_i representing the th teacher i ($1\le i\le m$), and the ${T}_{i}$ classification accuracy is ${Q}_{{T}_{i}}$. The overall expectation of accuracy rate is ${E}_{Q}=\frac{{\sum }_{i=1}^{m}{Q}_{{T}_{i}}}{m}$, and the accuracy gap for ${T}_{i}$ is ${|Q}_{{T}_{i}}-{E}_{Q}|\le {H}$| (H is a small positive real number), then the number of correctly classified votes in the histogram is roughly ${E}_{Q}*m$.

4 Proposed Exponential Mechanism with PATE

4.1 Exponential Mechanism

Definition 3

When under the action of the random algorithm $\left.{M}(x,u,R\right)$, adjacent datasets $d,{d}^{\prime}\in D$ output a certain class $r(r\in R)$ with a probability proportional to ${exp}\left(\frac{\left.\varepsilon u(x,r\right)}{2\Delta \mathcal{U}}\right)$, then M is an exponential mechanism algorithm that satisfies $\varepsilon ,0$-differential privacy.${u:{\mathbb{N}}}^{|\mathcal{x}|}\to {\mathbb{R}}^{k}$ is the scoring function for scoring class r, and the global sensitivity of the related scoring function is defined as:

$$\Delta u={\text{ma}}{\text{x}}_{r\in R}{\text{ma}}{\text{x}}_{x,y:\parallel x-y{\parallel }_{1}\le 1}\left|u\left(x,r\right)-u\left(y,r\right)\right|.$$

(6)

The exponential mechanism is suitable for the protection of non-numeric values. In this paper, the exponential mechanism is used to protect the histogram of classification.

4.2 Moments Accountant

PATE relies on sample voting, and the privacy budget caused by different samples are various. The sample-dependent analysis can get a tighter privacy upper bound, and the Moments Accountant is used for sample-dependent analysis [31]. This paper uses the following theorem of Moments Accountant:

Theorem 3

Algorithms ${\mathcal{M}}$ include a series of adaptive algorithms ${{\mathcal{M}}}_{1},\dots ,{{\mathcal{M}}}_{k}$, where ${{\mathcal{M}}}_{i}:{\prod }_{j=1}^{i-1}{{\mathcal{R}}}_{j}\times {\mathcal{D}}\to {{\mathcal{R}}}_{i}$. Then for ${o}_{1},\dots ,{o}_{k-1}$ and any $\lambda$:

$${\alpha }_{{\mathcal{M}}}\left(\lambda ;d,{d}^{\prime}\right)=\sum \limits_{i=1}^{k}{\alpha }_{{{\mathcal{M}}}_{i}}\left(\lambda ;{o}_{1},\dots ,{o}_{i-1},d,{d}^{\prime}\right).$$

(7)

Theorem 4

For any $\varepsilon >0$, the ${\mathcal{M}}$ Moments Accountant of the algorithm can be transformed into $\left(\varepsilon ,\delta \right)$- differential privacy by the following method:

$$\delta =\underset{\lambda }{\text{min}}{\text{exp}}\left({\alpha }_{{\mathcal{M}}}\left(\lambda \right)-\lambda \varepsilon \right).$$

(8)

Theorem 5

Moments Accountant upper bound satisfies $\left(\varepsilon ,0\right)$- differential privacy algorithm ${\mathcal{M}}:D\to {S}$; for any $l$, aux, $d$ and ${d}^{\prime}$, Moments Accountant has the following upper bound:

$$\alpha \left(l;\, aux,\, d,{d}^{\prime}\right)\le \frac{{\varepsilon }^{2}l\left(l+1\right)}{2}.$$

(9)

4.3 Exponential Mechanism Processing of Histogram

The random algorithm $\left.{{\mathcal{M}}}(x,u,{\mathcal{R}}\right)$ scores histogram output class $\left(r\in {\mathcal{R}}\right)$ of the dataset $x$ through the scoring function $\left.u(x,r\right)$, and outputs $r$ with the probability of ${\text{exp}}\left\{\frac{\varepsilon u\left(x,r\right)}{2\Delta U}\right\}/{\sum }_{r1\in R}{\text{exp}}\left\{\frac{\varepsilon u\left(x,r1\right)}{2\Delta U}\right\}$, the global sensitivity $\Delta u=\underset{d\left(x,y\right)=1}{max}\parallel f\left(x\right)-f\left(y\right)\parallel$. Since $\frac{{Pr}\left[{{\mathcal{M}}}\left(x,u,{\mathcal{R}}\right)=r\right]}{{Pr}\left[{{\mathcal{M}}}\left(y,u,{\mathcal{R}}\right)=r\right]}\le {\text{exp}}\left(\varepsilon \right)$, this process satisfies ($\varepsilon ,0$)-differential privacy. In this case, the scoring function of the exponential mechanism is: $u\left(x,r\right)={n}_{r}$ ($\Delta \mathcal{U}=1$).

Proof: From the output probability model of PATE, the principle of the scoring function is to try to maximize the output probability of the $\underset{i}{\text{arg max}}{ (n}_{i}$) th class (${n}_{i}$ is the number of votes of the ith class, and ${n}_{i}={E}_{Q}*m$). Combining with the nature of the scoring function of the exponential mechanism [30], the scoring function can be:

$$\left.u(r\right)=\left\{\begin{array}{c}a*\left|r\right|(r\ge m{q}_{u})\\ b*\left|r\right|\left(r\le m{q}_{L}\right)\\ c*\left|r\right|(m{q}_{L}<r<m{q}_{u})\end{array},\right.$$

(10)

where $m$ is the number of teachers, ${q}_{u}$ is the upper limit of the accuracy rate ($0<{q}_{u}<1$), ${q}_{L}$ is the lower limit of the accuracy rate ($0<{q}_{L}<1$), a is the high threshold weight, b is the low threshold weight, and c is the transition weight to make $\Delta \mathcal{U}$ as small as possible. a = K₁b (K₁ is a larger positive real number) and $\left|\left|a-c\right|-\left|b-c\right|\right|<{K}_{2}$ (${K}_{2}$ is a smaller positive real number).

$$\frac{u(x,r{)}}{\Delta \mathcal{U}}=\frac{a\left|{n}_{r}\right|}{am-b\left(m-1\right)}=\frac{a\left|{n}_{r}\right|}{a+\left(a-b\right)\left(m-1\right)}<\left|{n}_{r}\right|,$$

(11)

when$a=b=c, u(x,r)/\Delta {\mathcal{U}}=\left|{n}_{r}\right|$, and because $\Delta \mathcal{U}=\underset{ }{max}\left\{am-b\left(m-1\right),b{m}_{2}-c\left({m}_{2}-1\right)\right\}$, then $u\left(x,r\right)={n}_{r}$.

4.4 Exponential Mechanism Sample Screening

When the teachers vote for a certain class more consistently and agree less for other classes, this sample generally has a smaller privacy budget and can achieve a better accuracy, so the following mechanism is designed to screen out those “good” samples.

Algorithm 1:

Sample Screening Mechanism

$T$ is the output threshold. Use the Laplacian mechanism to add numerical noise and compare it with the threshold T. Only samples greater than the threshold T will be selected for processing, and then the selected samples will be output $r$ with a probability of $exp\left(\frac{\left.\frac{1}{{\sigma }_{2}}*u(x,r\right)}{2\Delta \mathcal{U}}\right)/{\sum }_{r1\in {\mathcal{R}}}exp\left(\frac{\left.\frac{1}{{\sigma }_{2}}*u(x,r1\right)}{2\Delta \mathcal{U}}\right)$ (${\sigma }_{2}$ is significantly less than${\sigma }_{1}$). Since this process is a post-processing procedure, it satisfies differential privacy and the upper bound of the privacy ${\varepsilon }$ budget is $\frac{2}{{\sigma }_{1}}$. The algorithm output space is {$\dashv$: discard samples,$\dashv$: process samples}. Sensitivity$\Delta f=\left|{n}_{{r}_{1},{d}^{\prime}}-{n}_{{r}_{1},{d}}\right|+\left|{n}_{{r}_{2},{d}^{\prime}}-{n}_{{r}_{2},{d}}\right|=2$, where ${r}_{1} ,{r}_{2}$ are the different classes in the adjacent dataset, defined by the algorithm, the tighter upper bound of privacy budget ${\varepsilon }$ is:

$$\varepsilon \le \underset{{r}_{1},{r}_{2}}{{\text{max}}}\frac{1-{P}_{{r}_{1},T}*{P}_{{r}_{2},T}*{\prod }_{i\ne {r}_{1},{r}_{2}}{P}_{i,T}}{1-{P}_{{r}_{1},T-1}*{P}_{{r}_{2},T+1}*{\prod }_{i\ne {r}_{1},{r}_{2}}{P}_{i,T}},$$

(12)

where ${P}_{i,T}=P\left[{n}_{i}+Lap\left({\sigma }_{1}\right)<T\right]$.

The screening mechanism has the risk of privacy leakage. Using ${\sigma }_{1}$ significantly greater than ${\sigma }_{2}$ to increase the size of random noise and selecting $T$ as small as possible can effectively prevent privacy leakage in this process. If $T$ is too high, the size of the obtained samples will be too small, and the trained student model will lack comprehensiveness.

4.5 Data-Dependent Privacy Budget Combination

Simply combining the privacy budget of each sample will lead to an excessively large privacy budget. Here, Moments Accountant is used to combine the privacy budget. When ${n}_{{j}^{*}}\ge {n}_{j}$ ($1\le j\le m$), the probability value of $q=Pr\left[{\mathcal{M}}\left(d\right)\ne {j}^{*}\right]$ depends on the voting histogram ${\bar{n}}=\left({n}_{1},{n}_{2},\dots ,{n}_{m}\right)$. In the case of satisfying the exponential mechanism $\left(\varepsilon ,0)\right.$-differential privacy:

$$q={\text{Pr}}\left[{\mathcal{M}}\left(d\right)\ne {j}^{*}\right]=1-exp\left(\frac{\varepsilon u(x,{j}^{*}\underset{ }{)}}{2\Delta \mathcal{U}}\right)/{\sum }_{j\in {\mathcal{R}}}exp\left(\frac{\left.\varepsilon u(x,j\right)}{2\Delta \mathcal{U}}\right)={\sum }_{j\in {\mathcal{R}},j\ne {j}^{*}}exp\left(\frac{\left.\varepsilon u(x,j\right)}{2\Delta \mathcal{U}}\right) /{\sum }_{j\in {\mathcal{R}}}{exp}\left(\frac{\left.\varepsilon u(x,j\right)}{2\Delta \mathcal{U}}\right).$$

(13)

When ${\mathcal{M}}$ satisfies the exponential mechanism ($\varepsilon ,0)-$ differential privacy and ($q={Pr}\left[{\mathcal{M}}\left(d\right)\ne {j}^{*}\right]$ (${j}^{*}$ is a certain output, $l,\varepsilon \ge 0$, $q\le \frac{1}{{e}^{\varepsilon }+1}$), for any additional dataset aux and $\parallel d-{d}^{\prime}{\parallel }_{1}\le 1$, then M satisfies:

$${exp}\left(\alpha \left(l;aux,d,{d}^{\prime}\right)\right)=Pr\left[{\mathcal{M}}\left(d\right)={j}^{*}\right]{\left(\frac{{Pr}\left[{\mathcal{M}}\left(d\right)={j}^{*}\right]}{{Pr}\left[{\mathcal{M}}\left({d}^{\prime}\right)={j}^{*}\right]}\right)}^{l}$$

$$+ {\sum }_{j\ne {j}^{*}}Pr\left[{\mathcal{M}}\left(d\right)=j\right]{\left(\frac{Pr\left[{\mathcal{M}}\left(d\right)=j\right]}{{Pr}\left[{\mathcal{M}}\left({d}{{^{\prime}}}\right)=j\right]}\right)}^{l}\le \left(1-q\right){\left(\frac{1-q}{1-{e}^{\varepsilon }q}\right)}^{l}+q{e}^{\varepsilon l}.$$

(14)

Since $\alpha \left(l;\,aux,\,d,{d}^{\prime}\right)$ is the mathematical expectation of the function of the Privacy Loss variable, it has an upper bound:

$$\left.\alpha \left(l;\,aux,\,d,{d}^{\prime}\right)\le {\varepsilon }^{2}l(l+1\right)/2.$$

(15)

Then $exp\left(\alpha \left(l;aux,d,{d}^{\prime}\right)\right)$ is:

$$\underset{ }{exp\left(\alpha \left(l;aux,d,{d}^{\prime}\right)\right)=min}\left(\left(1-{q}\right){\left(\frac{1-{q}}{1-{e}^{\varepsilon }{q}}\right)}^{l}+{q}{e}^{\varepsilon l},{e}^{\frac{{\varepsilon }^{2}l\left(l+1\right)}{2}}\right).$$

(16)

In a simple case, when the teachers only vote for a certain class, then ${q}_{e}=\frac{n-1}{n-1+{e}^{\varepsilon m/2}}$. The upper bound of ${q}_{e}$ can be obtained as $O\left({e}^{-\varepsilon m/2}\right)$. Due to the Laplace mechanism ${q}_{l}\le {\sum }_{j\ne {j}^{*}}\frac{2+\frac{\varepsilon }{2}\left({n}_{{j}^{*}}-{n}_{j}\right)}{4exp\left(\frac{\varepsilon }{2}\left({n}_{{j}^{*}}-{n}_{j}\right)\right)}$ [4], it can be obtained that ${q}_{l}=\frac{\left(4+\varepsilon m\right)\left(n-1\right)}{8{e}^{\varepsilon m/2}}$ in the same way. The upper bound ${q}_{l}$ is $O\left(m{e}^{-\varepsilon m/2}\right)$. When $q\in \left[0,\frac{1}{{e}^{\varepsilon }+1}\right]$, the larger the $q,$ the greater is the privacy budget under the Moments Accountant. The upper bound of ${q}_{e}$ and ${q}_{l}$ shows that the exponential mechanism is better than the Laplace mechanism in this situation, and this situation can be extended to any situation.

4.6 Data-Dependent Parameter Publishing

Since q still can leak the underlying information, it is necessary to publish the noise-added q to prevent privacy leakage. Since n data in the underlying sensitive dataset can affect the votes of n teachers at most, the distance between the histograms (${\bar{n}}=\left({n}_{1},\dots ,{n}_{m}\right)\in {\mathbb{N}}^{m}$) is:

$$d\left({\bar{n}},{{\bar{n}}}^{\prime}\right)={\text{max}}\left\{\mathop{\sum }_{i:{n}_{i}>{n}_{i}^{\prime}}\left({n}_{i}-{n}_{i}^{\prime}\right),\mathop{\sum }\nolimits_{i:{n}_{i}<{n}_{i}^{\prime}}\left({n}_{i}^{\prime}-{n}_{i}\right)\right\}.$$

(17)

The local sensitivity of the query function $q({\bar{n}}):{\mathbb{N}}^{m}\to \left[\mathrm{0,1}\right]$ is:

$$\Delta q={\text{max}}\left\{q\left({{{\bar{n}}}^{\prime}}_{max}\right)-q\left({\bar{n}}\right),q{\left({\bar{n}}\right)}-q\left({{{\bar{n}}}^{\prime}}_{min}\right)\right\},$$

(18)

where ${{{\bar{n}}}^{\prime}}_{max}=\left({n}_{1},\dots ,{n}^{\left(1\right)}+1,..,{n}^{\left(2\right)}-1,\dots ,{n}_{m}\right)$, ${{{\bar{n}}}^{\prime}}_{min}=\left({n}_{1},\dots ,{n}^{\left(1\right)}-1,..,{n}^{\left(2\right)}+1,\dots ,{n}_{m}\right)$, ${n}^{\left(k\right)}$ indicates the kth largest quantity, ${n}^{\left(1\right)}\ne m,{n}^{\left(1\right)}-{n}^{\left(2\right)}>1$. If ${n}^{\left(1\right)}=m$, then ${{\bar{n}}}_{max}={{\bar{n}}}^{\prime}$. If ${n}^{\left(1\right)}-{n}^{\left(2\right)}\le 1$, then ${{{\bar{n}}}^{\prime}}_{min}=\left({n}_{1},\dots ,{n}^{\left(1\right)}-1,..,{n}^{\left(l\right)}+1,\dots ,{n}_{m}\right)$, $l = {\min\nolimits_l}({n^{\left( 2 \right)}} - {n^{\left( l \right)}} \ge 1)$.

When $d\left({\bar{n}},{{\bar{n}}}^{\prime}\right)=1$, ${\bar{n}} \,and\, {{\bar{n}}}^{\prime}$ is the nearest adjacent dataset, we use the Laplacian mechanism to add numerical noise ${\text{Lap}}\left(\frac{\Delta q}{\varepsilon }\right)$ to q and add noise q to Moments Accountant. Advanced Composition is used to compute an upper bound on the total privacy budget of this process q. When the teachers only vote for a certain class, the upper bounds of the local sensitivity ∆q_e and ∆q_l also can be obtained as $O\left({e}^{-\varepsilon m/2}\right)$ and $O\left(m{e}^{-\varepsilon m/2}\right),$ respectively, according to the definition of ∆q. The variance of the added noise distribution is smaller in the exponential mechanism. Then it can be obtained that the privacy budget of the exponential mechanism PATE is less under the same privacy guarantee for q, and this situation can also be extended to any situation.

5 Experiments

The experimental environment is python/pytorch (Intel i5 cpu/Windows 8.1). MNIST (Modified National Institute of Standards and Technology database) and CIFAR-10 (Canadian Institute For Advanced Research) are used in the experiment.

The MNIST database is a sizable collection of handwritten digits. The MNIST dataset is frequently used for training and assessing deep learning models in image classification applications, such as CNNs, SVMs, and other machine learning techniques. The dataset’s straightforward and well-structured format makes it an essential resource for machine learning and computer vision researchers and practitioners.

The MNIST database contains 60,000 training images and 10,000 testing images. It was produced by "re-mixing" the original NIST dataset samples. The original dataset was a set of 128 × 128 binary images, processed into grayscale images by anti-aliasing and normalization to fit inside a 28 × 28 pixel bounding box [32].

The CIFAR-10 dataset is an image collection that is extensively used to train machine learning and computer vision algorithms. It is one of the most common datasets used in machine learning research. The CIFAR-10 dataset contains colour images divided into ten categories. Aeroplanes, vehicles, birds, cats, deer, dogs, frogs, horses, ships, and trucks are represented by the ten various classes. Each class contains 6,000 images. Computer algorithms that recognize things in photographs frequently learn by example. The dataset is divided into five training batches and one test batch, each of which contains 10,000 photos. The test batch contains exactly 1000 images from each class, chosen at random. The remaining images are distributed in random order in the training batches; however, certain training batches may contain more images from one class than another. The training batches each contain exactly 5000 images from each class. Because the images in CIFAR-10 are low resolution (32 × 32), researchers can quickly test various approaches. Published in 2009, CIFAR-10 is a labelled subset of the 80 Million Tiny Images dataset from 2008. Various kinds of convolutional neural networks tend to be the best at recognizing the images in CIFAR-10. CIFAR-10 is also used as a performance benchmark for teams competing to run neural networks faster and cheaper [33].

Both of these datasets are very suitable to evaluate the performance of deep learning models, and therefore are capable of comparing approaches to protect the privacy on deep learning models. The details of testing datasets can be found in Table 1.

Table 1 Dataset details

From the previous analysis, the upper bound of q is obtained as $O({e}^{-\varepsilon m/2})$. The trend can be easily seen in Fig. 3. The greater the number of teachers, the greater is the output probability. However, the more the number of teachers, the smaller is the dataset for each teacher. If the dataset is too small, the accuracy will be severely degraded (can be seen from Fig. 4). In this paper, the number of teachers is selected as 80 for subsequent experiments.

Fig. 3

Relationship between the number of teachers and output probability (MNIST)

Fig. 4

The relationship between the number of teachers and the accuracy of a teacher (MNIST)

The blue histogram in Fig. 5 shows the relationship between the approval rate of the largest category of teachers (i.e. the ratio of the number of votes to the total number of votes) and the number of samples without the sample screening mechanism. The red histogram is screened by the screening mechanism. It can be seen from Fig. 5 that the screening mechanism screened out most samples with low agreement rates and screened out very few samples with high agreement rates.

Fig. 5

Consistency rate and sample distribution

From Figs. 6 and 7, the total privacy budgets of the screening mechanism on MNIST and CIFAR-10 dataset are less than that of the exponential mechanism and the Laplace mechanism. The screening mechanism in this paper is carried out under the parameter design of threshold T = 0.6, σ1 = 25, σ2 = 1/0.3 ≈ 3.33.

Fig. 6

Privacy budget analysis of screening mechanism (MNIST dataset)

Fig. 7

Privacy budget analysis of screening mechanism (CIFAR-10 dataset)

Figure 8 shows the accuracy of the exponential screening mechanism on MNIST datasets is always higher than that of the original PATE Laplace mechanism, and the same situation can also be observed on the CIFAR-10 dataset which is shown in Fig. 9. Both in Figs. 8 and 9, the accuracy changes significantly when the number of samples changes from about 100 to 200, which is due to the excessive number of classes and the single sample being more complex.

Fig. 8

Experimental accuracy under different mechanisms (MNIST dataset)

Fig. 9

Experimental accuracy under different mechanisms (CIFAR-10 dataset)

It can be seen from the following two figures that the privacy budget of the exponential screening mechanism is always lower than the Laplace mechanism of the original PATE. It also can be observed from experiments on both MNIST and CIFAR-10 datasets that when the accuracy of the model is not high, the single sample is more complex and the number of classes is higher, the privacy budget of PATE will be greater as shown in Figs. 10 and 11.

Fig. 10

Privacy budget under different mechanisms (MNIST dataset)

Fig. 11

Privacy budget under different mechanisms (CIFAR-10 dataset)

Figure 12 shows the average sensitivity to the publication of data-dependent parameters of different mechanisms by using MNIST dataset. The exponential screening mechanism always has the lowest sensitivity, and the privacy budget of PATE of the exponential screening mechanism is always the least under the same privacy guarantee for q. It can be seen from Figs. 12 and 13 that the higher the accuracy of the model, the lower is the average sensitivity. It is because the higher the accuracy, the higher is the identity of the teacher for a certain class. The lower the number of classes, the higher is the teacher identity.

Fig. 12

Average sensitivity under different mechanisms (MNIST dataset)

Fig. 13

Average sensitivity under different mechanisms (CIFAR-10 dataset)

The goal of PATE is to train a privacy-preserving model with as high accuracy as possible while keeping the privacy budget as small as possible. In Table 2, sn is the number of samples, ε is the privacy budget, and δ is the differential privacy relaxation factor. Table 2 shows that, when compared to the Laplacian mechanism, the exponential screening mechanism's accuracy on the MNIST dataset increases by around 3.12%, while its privacy budget decreases by about 0.62%. On the CIFAR-10 dataset, the exponential screening mechanism maintains almost or even higher accuracy while reducing the privacy budget by about 12.54%.

Table 2 Privacy budget and accuracy

6 Conclusions

This paper introduces a novel exponential mechanism to replace the Laplacian mechanism to improve the original PATE. The PATE output probability model is introduced and the optimal exponential mechanism scoring function is derived from it. To reduce privacy budget and enhance accuracy, a sample screening process based on an exponential mechanism is also applied. The data-dependent privacy budget combination of the exponential mechanism is analysed in this paper. It has been demonstrated that the exponential mechanism outperforms the Laplace mechanism in both the data-dependent privacy budget combination analysis and data-dependent parameter release process. This paper also proposes an exponential mechanism sample screening mechanism to increase precision while still reducing privacy budget. The experiments demonstrates that the exponential screening mechanism can train a model with higher accuracy than the Laplacian mechanism with a smaller privacy budget.