PrivBUD-Wise: Differentially Private Frequent Itemsets Mining in High-Dimensional Databases

Abstract

In this paper, we study the problem of mining frequent itemsets in high-dimensional databases with differential privacy, and propose a novel algorithm, PrivBUD-Wise, which achieves high result utility as well as a high privacy level. Instead of limiting the cardinality of transactions by truncating or splitting approaches, which causes extra information loss and result in unsatisfactory performance in utility, PrivBUD-Wise doesn’t make any preprocessing on original database and guarantees high result utility by reducing extra $privacy\ budget$ consumption on irrelevant itemsets as much as possible. To achieve that, we first propose a Report Noisy mechanism with optional number of reported itemsets: SRNM, and what is more important is that we give a strict proof for SRNM in the appendix. Moreover, PrivBUD-Wise first proposes a biased $privacy\ budget$ allocation strategy and no assumption or estimation on the maximal cardinality needs to be made. The good performance in utility and efficiency of PrivBUD-Wise is shown by experiments on three real-world datasets.

Appendices

Appendix A

Lemma 4

For $\forall \ \delta >0$, and x is a draw from Lap(b), then:

$$\begin{aligned} \mathrm {P}[x\ge \delta +1]=e^{-\frac{1}{b}}\mathrm {P}[x\ge \delta ] \end{aligned}$$

where $\mathrm {P}$ denotes the probability.

Proof

$$\begin{aligned} \frac{\mathrm {P}[x\ge \delta +1]}{\mathrm {P}[x\ge \delta ]}=\frac{\frac{1}{2b}\int _{\delta +1}^{\infty }e^{-\frac{x}{b}} dx}{\frac{1}{2b}\int _{\delta }^{\infty }e^{-\frac{x}{b}} dx}=\frac{e^{-\frac{\delta +1}{b}}}{e^{-\frac{\delta }{b}}}=e^{-\frac{1}{b}} \end{aligned}$$

Hence, this lemma follows.

Appendix B

Proof of Theorem 2: Fix $D=D'\cup \{t\}$, where t is a transaction. Let v, respectively $v'$, denote the vector of query counts of SRNM when the dataset is D, respectively $D'$. We use m to denote the number of queries(equal to the number of candidate itemsets). Then we have:

• (1) $v_{i}\ge v^{'}_{i}$ for $\forall i\in [m]$;

• (2) $1+v^{'}_{i}\ge v_{i}$ for $\forall i\in [m]$;

Given an integer z, for every $z'\in [z]$, fix any set $j=(j_{1},j_{2}, \ldots ,j_{z'})\in [m]^{z'}$, to prove differential privacy, we want to bound the ratio(from above and below) of the probabilities that $(j_{1},j_{2}, \ldots ,j_{z'})$ is selected with D and with D.

Fix $r_{-j}$, which is a draw from $[Lap(z/\epsilon )]^{m-z'}$ and is used for all noisy query counts except $z'$ counts corresponding to $j=(j_{1},j_{2}, \ldots ,j_{z'})$. We use $\mathrm {P}[j|\theta ]$ to denote the probability that the outputs of SRNM is j under condition $\theta$.

Firstly, we prove that $\mathrm {P}[j|D,r_{-j}]\le e^{\epsilon }\mathrm {P}[j|D',r_{-j}]$: For every $k\in j$, define

$$\begin{aligned} r_{k}^{*}=\mathrm {min}_{r_{k}}:v_{k}+r_{k} > v_{i}+r_{i}, \forall i \in [m]\backslash j \end{aligned}$$

Then j is the output with D iff for $\forall k\in j$: $r_{k}\ge r_{k}^{*}$.

For all $i\in [m]\backslash j, k\in j$:

$$ \begin{array}{c} v_{k}+r_{k}^{*}>v_{i}+r_{i}\\ \Rightarrow (1+v_{k}^{'})+r_{k}^{*}\ge v_{k}+r_{k}^{*}>v_{i}+r_{i}\ge v_{i}^{'}+r_{i}\\ \Rightarrow v_{k}^{'}+(r_{k}^{*}+1)>v_{i}^{'}+r_{i} \end{array} $$

So, if for $\forall k\in j$: $r_{k}\ge r_{k}^{*}+1$, then the output with $D'$ will be j and the added noise will be $(r_{j},r_{-j})$. So we have:

$$\begin{aligned} \begin{aligned}&\mathrm {P}[j|D',r_{-j}]\ge \mathrm {P}[r_{k}\ge r_{k}^{*}+1|k\in j]= \prod \limits _{k\in j}\mathrm {P}[r_{k}\ge r_{k}^{*}+1]\\&\quad = \prod \limits _{k\in j}e^{-\frac{\epsilon }{z}}\mathrm {P}[r_{k}\ge r_{k}^{*}]=e^{-\frac{z'\epsilon }{z}}\mathrm {P}[j|D,r_{-j}]\ge e^{-\epsilon }\mathrm {P}[j|D,r_{-j}] \end{aligned} \end{aligned}$$

(3)

The second equality is due to Lemma 4. multiply by $e^{\epsilon }$: $\mathrm {P}[j|D,r_{-j}]\le e^{\epsilon }\mathrm {P}[j|D',r_{-j}]$

We now prove that $\mathrm {P}[j|D',r_{-j}]\le e^{\epsilon }\mathrm {P}[j|D',r_{-j}]$. For every $k\in j$, define: $r_{k}^{*}=\mathrm {min}_{r_{k}}:v_{k}^{'}+r_{k} > v_{i}^{'}+r_{i}, \forall i \in [m]\backslash j$, then j is the output when the dataset is $D'$ iff for $\forall k\in j$: $r_{k}\ge r_{k}^{*}$.

For all $i\in [m]\backslash j, k\in j$:

$$ \begin{array}{c} v_{k}^{'}+r_{k}^{*}>v_{i}^{'}+r_{i}\\ \Rightarrow 1+v_{k}+r_{k}^{*}\ge 1+v_{k}^{'}+r_{k}^{*}>1+v_{i}^{'}+r_{i}\ge v_{i}+r_{i} \end{array} $$

So, if for $\forall k\in j$: $r_{k}\ge r_{k}^{*}+1$, then the output with D will be j and the added noise will be $(r_{j},r_{-j})$. So we have:

$$\begin{aligned} \begin{aligned}&\mathrm {P}[j|D,r_{-j}]\ge \mathrm {P}[r_{k}\ge r_{k}^{*}+1|k\in j]= \prod \limits _{k\in j}\mathrm {P}[r_{k}\ge r_{k}^{*}+1]\\&\quad = \prod \limits _{k\in j}e^{-\frac{\epsilon }{z}}\mathrm {P}[r_{k}\ge r_{k}^{*}]=e^{-\frac{z'\epsilon }{z}}\mathrm {P}[j|D',r_{-j}]\ge e^{-\epsilon }\mathrm {P}[j|D',r_{-j}] \end{aligned} \end{aligned}$$

(4)

multiply by $e^{\epsilon }$: $\mathrm {P}[j|D',r_{-j}]\le e^{\epsilon }\mathrm {P}[j|D,r_{-j}]$. Hence this theorem follows.