A Framework for Evaluating Privacy Preserving Data Mining Algorithms^*

Abstract

Recently, a new class of data mining methods, known as privacy preserving data mining (PPDM) algorithms, has been developed by the research community working on security and knowledge discovery. The aim of these algorithms is the extraction of relevant knowledge from large amount of data, while protecting at the same time sensitive information. Several data mining techniques, incorporating privacy protection mechanisms, have been developed that allow one to hide sensitive itemsets or patterns, before the data mining process is executed. Privacy preserving classification methods, instead, prevent a miner from building a classifier which is able to predict sensitive data. Additionally, privacy preserving clustering techniques have been recently proposed, which distort sensitive numerical attributes, while preserving general features for clustering analysis. A crucial issue is to determine which ones among these privacy-preserving techniques better protect sensitive information. However, this is not the only criteria with respect to which these algorithms can be evaluated. It is also important to assess the quality of the data resulting from the modifications applied by each algorithm, as well as the performance of the algorithms. There is thus the need of identifying a comprehensive set of criteria with respect to which to assess the existing PPDM algorithms and determine which algorithm meets specific requirements.

In this paper, we present a first evaluation framework for estimating and comparing different kinds of PPDM algorithms. Then, we apply our criteria to a specific set of algorithms and discuss the evaluation results we obtain. Finally, some considerations about future work and promising directions in the context of privacy preservation in data mining are discussed.

Appendix A: Proofs

Proof-of Theorem 4.1:

First, Algorithm GIH sorts the itemset L _h wrt the items it contains and the particular items it supports. In general, sorting N number has a complexity in O(Nlog(N)). However, in our case the length of transactions and thus of the considered itemsets has an upper bound that is very small compared to the size of the database. Then, Algorithm GIH takes O(|L _h |) to sort the set L _h of large itemsets to hide according to a descending order based on their size and minimum support. For each itemset Z in L _h , it takes O(|D|) in order to generate the set T _Z of the transactions in D that support Z, assuming that the transaction length is bounded by a constant. In addition to that, the algorithm sorts T _Z in ascending order of transaction size. Since the transaction length is bound, the sorting algorithm is of order O(|T _Z |). Then, the item i ∈ Z with highest minimum support is selected and a question mark is placed for that item in the transaction with minimum size, and this is repeated until the minimum support of the itemset Z goes below the MST by SM. After k iterations, the minimum support of Z will be ${\it minsup}(Z)^{(k)}=\frac{|T_Z|- k}{|D|}$; thus the number of steps required to hide Z is |T _Z | − (MST − SM) * |D|. Since |T _Z | ≤ |D| and [(MST − SM) * |D|] ≤ |D| we can state that Algorithm GIH requires O(|L _h | * |D|) time to hide all the itemsets belonging to L _h and consequently all the sensitive rules, whose generating itemsets are stored in L _h .

Proof-of Theorem 4.2:

For each sensitive rule r to hide, Algorithm CR performs the following tasks: it generates the set T _r of transactions of D supporting r, taking O(|D|* ATL), then for each transaction t in T _r it counts the number of items in t, that is of order O(|T _r |* ATL). To sort the transactions in T _r in ascending order of the number of items supported the Algorithm CR takes $O(|T'_{l_r}|)$ for the reason explained above. To hide the selected rule r, the algorithm executes the inner loop until the minimum support becomes lower than MST by SM or minimum confidence becomes lower than MCT by SM. The minsup(r) and minconf(r) are initially equal to $\frac{|T_r|}{|D|}$ and $\frac{|T_r|}{|T_{l_r}|}$ respectively, and after k iterations the fractions become $\frac{|T_r|-k}{|D|}$ for the minsup(r) and $\frac{|T_r|-k}{|T_{l_r}|}$ for the minconf(r). This implies that the inner loop executes until $\frac{|T_r|-k}{|D|}<MST-SM$ or $\frac{|T_r|-k}{|T_{l_r}|}<MCT-SM$, that is, $k=min(|D|\ast(minsup(r)-(MST-SM))), |D|\ast(minsup(r)-(MCT-SM)\ast minsup(l_r)))$, which can be assessed in both cases as being O(|D|). The cost of each iteration is given by the choose_item operation, which takes O(1) time. Therefore, Algorithm CR takes O(|R _h |* A _D ) to hide all the sensitive rules in R _h .

Proof-of Theorem 4.3

For each rule r, Algorithm CR2 performs the following tasks: it generates the set $T'_{l_r}$ of transactions of D that partially support l _r and do not fully support r _r , taking O(|D|* ATL), then for each transaction t in $T'_{l_r}$ it counts the number of items of l _r in t, that is of order $O(|T'_{l_r}|\ast ATL)$. To sort the transactions in $T'_{l_r}$ in descending order of the calculated counts, the algorithm takes $O(|T'_{l_r}|)$ in this particular case. To hide the selected rule r, the algorithm executes the inner loop until the minimum confidence becomes lower than MCT by SM. The minconf(r) is initially equal to $\frac{minsup(r)}{maxsup(l_r)}=\frac{|T_r|}{|T'_{l_r}|}$, and after k iterations the fraction becomes $\frac{|T_r|}{|T'_{l_r}|+k}$. This implies that the inner loop executes $\lceil |D|(\frac{minsup(r)}{MCT-SM}-minsup(l_r))\rceil$ steps. The cost of each iteration takes O(1) time, thus Algorithm CR2 performs in $O(|R_h|*A_D)$.