Practical Secure Decision Tree Learning in a Teletreatment Application
In this paper we develop a range of practical cryptographic protocols for secure decision tree learning, a primary problem in privacy preserving data mining. We focus on particular variants of the well-known ID3 algorithm allowing a high level of security and performance at the same time. Our approach is basically to design special-purpose secure multiparty computations, hence privacy will be guaranteed as long as the honest parties form a sufficiently large quorum.
Our main ID3 protocol will ensure that the entire database of transactions remains secret except for the information leaked from the decision tree output by the protocol. We instantiate the underlying ID3 algorithm such that the performance of the protocol is enhanced considerably, while at the same time limiting the information leakage from the decision tree. Concretely, we apply a threshold for the number of transactions below which the decision tree will consist of a single leaf—limiting information leakage. We base the choice of the “best” predicting attribute for the root of a decision tree on the Gini index rather than the well-known information gain based on Shannon entropy, and we develop a particularly efficient protocol for securely finding the attribute of highest Gini index. Moreover, we present advanced secure ID3 protocols, which generate the decision tree as a secret output, and which allow secure lookup of predictions (even hiding the transaction for which the prediction is made). In all cases, the resulting decision trees are of the same quality as commonly obtained for the ID3 algorithm.
We have implemented our protocols in Python using VIFF, where the underlying protocols are based on Shamir secret sharing. Due to a judicious use of secret indexing and masking techniques, we are able to code the protocols in a recursive manner without any loss of efficiency. To demonstrate practical feasibility we apply the secure ID3 protocols to an automated health care system of a real-life rehabilitation organization.
KeywordsDecision Tree Gini Index Class Attribute Recursive Call Homomorphic Encryption
This work was supported by the Dutch national program COMMIT.
- [AJH10]op den Akker, H., Jones, V.M., Hermens, H.J.: Predicting feedback compliance in a teletreatment application. In: Proceedings of ISABEL 2010: The 3rd International Symposium on Applied Sciences in Biomedical and Communication Technologies, Rome, Italy (2010)Google Scholar
- [AS00]Agrawal, R., Srikant, R.: Privacy-preserving data mining. In: Proceedings of the 2000 ACM SIGMOD International Conference on Management of Data, SIGMOD 2000, pp. 439–450. ACM, New York (2000)Google Scholar
- [DZ02]Du, W., Zhan, Z.: Building decision tree classifier on private data. In: Proceedings of the IEEE International Conference on Privacy, Security and Data Mining, vol. 14, pp. 1–8. Australian Computer Society Inc. (2002)Google Scholar
- [FA10]Frank, A., Asuncion, A.: UCI machine learning repository (2010)Google Scholar
- [Gei10]Geisler, M.: Cryptographic protocols: theory and implementation. Ph.D. thesis, Aarhus University, Denmark, February 2010Google Scholar
- [Kel10]Keller, M.: VIFF boost extension (2010). http://lists.viff.dk/pipermail/viff-devel-viff.dk/2010-August/000847.html
- [MGA12]Bashir Malik, M., Asger Ghazi, M., Ali, R.: Privacy preserving data mining techniques: current scenario and future prospects. In: Proceedings of the 2012 Third International Conference on Computer and Communication Technology, ICCCT ’12, pp. 26–32. IEEE Computer Society, Washington, DC (2012)Google Scholar
- [Qui86]Quinlan, J.R.: Induction of decision trees. Mach. Learn. 1(1), 81–106 (1986)Google Scholar
- [RM05]Rokach, L., Maimon, O.: Decision trees. In: The Data Mining and Knowledge Discovery Handbook, pp. 165–192. Springer, US (2005)Google Scholar
- [SM08]Samet, S., Miri, A.: Privacy preserving ID3 using Gini index over horizontally partitioned data. In: IEEE/ACS International Conference on Computer Systems and Applications, AICCSA 2008, pp. 645–651. IEEE (2008)Google Scholar
- [VCKP08]Vaidya, J., Clifton, C., Kantarcıoğlu, M., Scott Patterson, A.: Privacy-preserving decision trees over vertically partitioned data. ACM Trans. Knowl. Discov. Data 2(3), 14:1–14:27 (2008)Google Scholar
- [WXSY06]Wang, K., Xu, Y., She, R., Yu, P.S.: Classification spanning private databases. In: Proceedings of the National Conference on Artificial Intelligence, vol. 21, p. 293. AAAI Press, MIT Press, Cambridge, London (1999, 2006)Google Scholar
- [XHLS05]Xiao, M.-J., Huang, L.-S., Luo, Y.-L., Shen, H.: Privacy preserving ID3 algorithm over horizontally partitioned data. In: Sixth International Conference on Parallel and Distributed Computing, Applications and Technologies, PDCAT 2005, pp. 239–243. IEEE (2005)Google Scholar
- [Yao86]Yao, A.: How to generate and exchange secrets. In: Proceedings of the 27th IEEE Symposium on Foundations of Computer Science (FOCS ’86), pp. 162–167. IEEE Computer Society (1986)Google Scholar