Utility-Enhancing Flexible Mechanisms for Differential Privacy

Abstract

Differential privacy is a mathematical technique that provides strong theoretical privacy guarantees by ensuring the statistical indistinguishability of individuals in a dataset. It has become the de facto framework for providing privacy-preserving data analysis over statistical datasets. Differential privacy has garnered significant attention from researchers and privacy experts due to its strong privacy guarantees. However, the accuracy loss caused by the noise added has been an issue. First, we propose a new noise adding mechanism that preserves $(\epsilon ,\delta )$-differential privacy. The distribution pertaining to this mechanism can be observed as a generalized truncated Laplacian distribution. We show that the proposed mechanism adds optimal noise in a global context, conditional upon technical lemmas. We also show that the generalized truncated Laplacian mechanism performs better than the optimal Gaussian mechanism. In addition, we also propose an $(\epsilon )$-differentially private mechanism to improve the utility of differential privacy by fusing multiple Laplace distributions. We also derive the closed-form expressions for absolute expectation and variance of noise for the proposed mechanisms. Finally, we empirically evaluate the performance of the proposed mechanisms and show an increase in all utility measures considered, while preserving privacy.

Appendices

A Proof for Lemma 1 and Lemma 2

Here, we present the proof for Lemma 1 and Lemma 2 used in Sect. 4.2.

Proof

(Proof for Lemma 1). Since the probability density function f(x) is monotonically increasing when $x\ge 0$ and is monotonically decreasing when $x < 0$,

$$\begin{aligned}&\max \Big (\int _A^{A+\varDelta }f(x)dx, \int _{B-\varDelta }^B f(x)dx\Big ) =&{\left\{ \begin{array}{ll} \int _{B-\varDelta }^B f(x)dx\text { when } |A|\ge |B|\\ \int _A^{A+\varDelta }f(x)dx\text { when } |A| < |B|. \end{array}\right. } \end{aligned}$$

We will first discuss the case when $|A|\ge |B|$,

$$\begin{aligned}&\max \Big (\int _A^{A+\varDelta }f(x)dx, \int _{B-\varDelta }^B f(x)dx\Big )&= \int _{B-\varDelta }^B f(x)dx&= \int _{B-\varDelta }^B Me^{-\frac{x}{\lambda }} dx\\&= M\lambda \Big (e ^{-\frac{B-\varDelta }{\lambda }} -e ^{-\frac{B}{\lambda }})\Big )\\ \end{aligned}$$

Plugging in $M=\frac{1}{\lambda (2-e^{\frac{A}{\lambda }}-e^{-\frac{B}{\lambda }})}$ as in our definition for the generalized truncated Laplacian distribution, $A = \lambda \ln \left[ 2+(\frac{1 - \delta }{\delta })e^{-\frac{B}{\lambda }} - (\frac{1}{\delta })e^{-\frac{B-\varDelta }{\lambda }}\right]$ as specified in Theorem 1, we have

$$\begin{aligned}&\int _{B-\varDelta }^B f(x)dx = \frac{e ^{-\frac{B-\varDelta }{\lambda }} -e ^{-\frac{B}{\lambda }}}{2-e^{\frac{A}{\lambda }}-e^{-\frac{B}{\lambda }}} =&\frac{e ^{-\frac{B-\varDelta }{\lambda }} -e ^{-\frac{B}{\lambda }}}{(\frac{1}{\delta })\left( e^{-\frac{B-\varDelta }{\lambda }} - e^{-\frac{B}{\lambda }} \right) } = \delta \end{aligned}$$

We omit showing the computation for the case when $|A|\le |B|$ as the derivation is very similar to that of the above mentioned case.

Now, we will proceed to prove Lemma 2.

Proof

(Proof for Lemma 2). Given two neighboring datasets $\mathcal {D}_1\sim \mathcal {D}_2$, we know that $|q(\mathcal {D}_1)-q(\mathcal {D}_2)| \le \varDelta$, thus the condition $\mathcal {P}(\mathcal {S})-e^\epsilon \mathcal {P}(\mathcal {S}+d)\le \delta$ for any $|d|\le \varDelta$ is equivalent to

$$\mathcal {P}(\mathcal {S})-e^\epsilon \mathcal {P}(\mathcal {S}+q(\mathcal {D}_1)-q(\mathcal {D}_2))\le \delta \Leftrightarrow \mathcal {P}(\mathcal {S}-q(\mathcal {D}_1))\le e^\epsilon \mathcal {P}(\mathcal {S}-q(\mathcal {D}_2))+\delta $$

Hence, for any $t\in \mathcal {S}$, the condition is equivalent to

$$\text {Pr} (X = t - q(\mathcal {D}_1))\le e^\epsilon \text {Pr} (X = t -q(\mathcal {D}_2)) +\delta $$

$$\Leftrightarrow \text {Pr}(q(\mathcal {D}_1) + X = t) \le e^\epsilon \text {Pr}(q(\mathcal {D}_2) + X = t) +\delta $$

$$\Leftrightarrow \text {Pr [}\mathcal {A}(\mathcal {D}_1)\in \mathcal {T}\text {]}\le e^\epsilon \text {Pr [}\mathcal {A}(\mathcal {D}_2)\in \mathcal {T}\text {]} + \delta ,$$

which is the necessary condition for mechanism $\mathcal {A}$ to preserve $(\epsilon ,\delta )$-differential privacy.

B Generalized Truncated Laplacian - Evaluation

We empirically show that the ratio of the noise amplitude $L_1^*$ and noise power $L_2^*$ of generalized truncated Laplacian mechanism and the optimal Gaussian mechanism is always less than 1 for appropriate values of $\delta$, A and B as described in Sect. 4. Compared to the optimal Gaussian mechanism, the generalized truncated Laplacian mechanism reduces the noise power and noise amplitude across all privacy regimes. The implementation can be found in https://github.com/vaikkunth/DPMechanisms.

$\epsilon$	$\delta$	A	$L_1^*$	$L_2^*$
0.7	2.5e−06	$-17.46$	0.25	0.13
0.4	4.0e−06	$-27.57$	0.27	0.15
0.4	2.5e−06	$-28.74$	0.27	0.14
0.4	9.5e−06	$-25.4$	0.29	0.17
0.4	3.0e−06	$-28.29$	0.27	0.14
0.7	6.0e−06	$-16.21$	0.27	0.14
0.4	8.5e−06	$-25.68$	0.29	0.16
0.7	3.5e−06	$-16.98$	0.26	0.13
0.7	4.0e−06	$-16.79$	0.26	0.14
0.4	2.0e−06	$-29.3$	0.26	0.14
0.1	4.5e−06	$-93.66$	0.31	0.19
0.7	3.0e−06	$-17.2$	0.26	0.13
0.4	5.5e−06	$-26.77$	0.28	0.15
0.7	9.5e−06	$-15.55$	0.28	0.15
0.4	6.0e−06	$-26.55$	0.28	0.16
0.7	1.0e−06	$-18.77$	0.24	0.12
0.4	6.5e−06	$-26.35$	0.28	0.16
0.4	7.0e−06	$-26.17$	0.28	0.16
0.4	4.5e−06	$-27.27$	0.27	0.15
0.4	9.0e−06	$-25.54$	0.29	0.17
0.4	3.5e−06	$-27.9$	0.27	0.15
0.1	9.5e−06	$-86.19$	0.32	0.21
0.1	5.5e−06	$-91.66$	0.31	0.19
0.1	5.0e−06	$-92.61$	0.31	0.19
0.4	1.5e−06	$-30.02$	0.26	0.13
0.4	8.0e−06	$-25.83$	0.29	0.16
0.7	7.0e−06	$-15.99$	0.27	0.15
0.7	7.5e−06	$-15.89$	0.27	0.15
0.4	1.0e−06	$-31.03$	0.25	0.13
0.1	7.0e−06	$-89.24$	0.32	0.2
0.4	5.0e−06	$-27.01$	0.28	0.15

C Merging Laplacian Distributions - Evaluation

We evaluate the $l^1$ and $l^2$ cost for the Laplacian, Merged Laplacian with 1 break point and Merged Laplacian with 2 break points. We show that the cost for the Merged Laplacian with 2 break points is lower than that of the Laplacian mechanism and hence we achieve better utility for the same privacy loss. The implementation can be found in https://github.com/vaikkunth/DPMechanisms.

($\epsilon _1$, $\epsilon _2$, $\epsilon _3$)	($c_1$, $c_2$)	$L_1^*$	$L_2^*$
(0.2, 0.25, 0.33)	(1, 3)	(3.0, 3.03, 1.12)	(18.0, 18.22, 3.94)
(0.17, 0.25, 0.33)	(1, 3)	(3.0, 3.03, 1.12)	(18.0, 18.22, 3.96)
(0.17, 0.2, 0.33)	(1, 3)	(3.0, 3.05, 1.13)	(18.0, 18.35, 4.07)
(0.14, 0.25, 0.33)	(1, 3)	(3.0, 3.03, 1.12)	(18.0, 18.22, 3.97)
(0.14, 0.2, 0.33)	(1, 3)	(3.0, 3.05, 1.13)	(18.0, 18.35, 4.08)
(0.14, 0.17, 0.33)	(1, 3)	(3.0, 3.06, 1.13)	(18.0, 18.43, 4.15)
(0.14, 0.17, 0.2)	(1, 3)	(5.0, 5.01, 1.96)	(50.0, 50.15, 16.26)
(0.12, 0.25, 0.33)	(1, 3)	(3.0, 3.03, 1.13)	(18.0, 18.22, 3.98)
(0.12, 0.2, 0.33)	(1, 3)	(3.0, 3.05, 1.13)	(18.0, 18.35, 4.08)
(0.12, 0.17, 0.33)	(1, 3)	(3.0, 3.06, 1.13)	(18.0, 18.43, 4.16)
(0.12, 0.17, 0.2)	(1, 3)	(5.0, 5.01, 1.96)	(50.0, 50.15, 16.28)
(0.12, 0.14, 0.33)	(1, 3)	(3.0, 3.07, 1.14)	(18.0, 18.49, 4.21)
(0.12, 0.14, 0.2)	(1, 3)	(5.0, 5.02, 1.98)	(50.0, 50.26, 16.5)
(0.11, 0.25, 0.33)	(1, 3)	(3.0, 3.03, 1.13)	(18.0, 18.22, 3.98)
(0.11, 0.2, 0.33)	(1, 3)	(3.0, 3.05, 1.13)	(18.0, 18.35, 4.09)
(0.11, 0.17, 0.33)	(1, 3)	(3.0, 3.06, 1.14)	(18.0, 18.43, 4.16)
(0.11, 0.17, 0.2)	(1, 3)	(5.0, 5.01, 1.96)	(50.0, 50.15, 16.3)
(0.11, 0.14, 0.33)	(1, 3)	(3.0, 3.07, 1.14)	(18.0, 18.49, 4.21)
(0.11, 0.14, 0.2)	(1, 3)	(5.0, 5.02, 1.98)	(50.0, 50.26, 16.52)
(0.11, 0.12, 0.33)	(1, 3)	(3.0, 3.08, 1.14)	(18.0, 18.53, 4.25)
(0.11, 0.12, 0.2)	(1, 3)	(5.0, 5.03, 1.99)	(50.0, 50.34, 16.68)
(0.11, 0.12, 0.14)	(1, 3)	(7.0, 7.01, 3.28)	(98.0, 98.12, 42.57)
(0.2, 0.25, 0.33)	(1, 5)	(3.0, 3.03, 1.61)	(18.0, 18.22, 5.17)
(0.17, 0.25, 0.33)	(1, 5)	(3.0, 3.03, 1.61)	(18.0, 18.22, 5.19)
(0.17, 0.2, 0.33)	(1, 5)	(3.0, 3.05, 1.63)	(18.0, 18.35, 5.4)
(0.14, 0.25, 0.33)	(1, 5)	(3.0, 3.03, 1.61)	(18.0, 18.22, 5.2)
(0.14, 0.2, 0.33)	(1, 5)	(3.0, 3.05, 1.63)	(18.0, 18.35, 5.41)
(0.14, 0.17, 0.33)	(1, 5)	(3.0, 3.06, 1.64)	(18.0, 18.43, 5.55)
(0.14, 0.17, 0.2)	(1, 5)	(5.0, 5.01, 1.85)	(50.0, 50.15, 10.69)
(0.12, 0.25, 0.33)	(1, 5)	(3.0, 3.03, 1.62)	(18.0, 18.22, 5.21)
(0.12, 0.2, 0.33)	(1, 5)	(3.0, 3.05, 1.63)	(18.0, 18.35, 5.42)
(0.12, 0.17, 0.33)	(1, 5)	(3.0, 3.06, 1.64)	(18.0, 18.43, 5.56)
(0.12, 0.17, 0.2)	(1, 5)	(5.0, 5.01, 1.85)	(50.0, 50.15, 10.7)
(0.12, 0.14, 0.33)	(1, 5)	(3.0, 3.07, 1.65)	(18.0, 18.49, 5.65)
(0.12, 0.14, 0.2)	(1, 5)	(5.0, 5.02, 1.86)	(50.0, 50.26, 10.98)