SecureComm 2012: Security and Privacy in Communication Networks pp 292-309 | Cite as
Privacy Preserving Back-Propagation Learning Made Practical with Cloud Computing
Abstract
Back-propagation is an effective method for neural network learning. To improve the accuracy of the learning result, in practice multiple parties may want to collaborate by jointly executing the back-propagation algorithm on the union of their respective data sets. During this process no party wants to disclose her/his private data to others for privacy concerns. Existing schemes supporting this kind of collaborative learning just partially solve the problem by limiting the way of data partition or considering only two parties. There still lacks a solution for more general and practical settings wherein two or more parties, each with an arbitrarily partitioned data set, collaboratively conduct learning.
In this paper, by utilizing the power of cloud computing, we solve this open problem with our proposed privacy preserving back-propagation algorithm, which is tailored for the setting of multiparty and arbitrarily partitioned data. In our proposed scheme, each party encrypts his/her private data locally and uploads the ciphertexts into the cloud. The cloud then executes most of the operations pertaining to the learning algorithms with ciphertexts but learns nothing about the original private data. By securely offloading the expensive operations to the cloud, we keep the local computation and communication costs on each party minimal and independent to the number of participants. To support flexible operations over ciphertexts, we adopt and tailor the BGN ‘doubly homomorphic’ encryption algorithm for the multiparty setting. Thorough analysis shows that our proposed scheme is secure, efficient and scalable.
Keywords
privacy reserving learning neural network back-propagation cloud computing computation outsourcePreview
Unable to display preview. Download preview PDF.
References
- 1.The health insurance portability and accountability act of privacy and security rules, http://www.hhs.gov/ocr/privacy
- 2.National standards to protect the privacy of personal health information, http://www.hhs.gov/ocr/hipaa/finalreg.html
- 3.Abramowitz, M., Stegun, I.A.: Handbook of Mathematical Functions: with Formulas, Graphs, and Mathematical Tables. Dover Books on Mathematics. Dover, New York (1964)MATHGoogle Scholar
- 4.Bansal, A., Chen, T., Zhong, S.: Privacy preserving back-propagation neural network learning over arbitrarily partitioned data. Neural Comput. Appl. 20(1), 143–150 (2011)CrossRefGoogle Scholar
- 5.Boneh, D., Goh, E.-J., Nissim, K.: Evaluating 2-DNF Formulas on Ciphertexts. In: Kilian, J. (ed.) TCC 2005. LNCS, vol. 3378, pp. 325–341. Springer, Heidelberg (2005)CrossRefGoogle Scholar
- 6.Chen, T., Zhong, S.: Privacy-preserving backpropagation neural network learning. Trans. Neur. Netw. 20(10), 1554–1564 (2009)CrossRefGoogle Scholar
- 7.Cun, L., Boser, B., Denker, J.S., Henderson, D., Howard, R.E., Hubbard, W., Jackel, L.D.: Handwritten digit recognition with a back-propagation network. In: Advances in Neural Information Processing Systems, pp. 396–404. Morgan Kaufmann (1990)Google Scholar
- 8.di Vimercati, S.D.C., Foresti, S., Jajodia, S., Paraboschi, S., Samarati, P.: Over-encryption: management of access control evolution on outsourced data. In: Proceedings of the 33rd International Conference on Very Large Data Bases, VLDB 2007, pp. 123–134. VLDB Endowment (2007)Google Scholar
- 9.El Gamal, T.: A Public Key Cryptosystem and a Signature Scheme Based on Discrete Logarithms. In: Blakely, G.R., Chaum, D. (eds.) CRYPTO 1984. LNCS, vol. 196, pp. 10–18. Springer, Heidelberg (1985)CrossRefGoogle Scholar
- 10.Fahlman, S.E.: Faster-learning variations on Back-propagation: An empirical study, pp. 38–51. Morgan Kaufmann (1988)Google Scholar
- 11.Flouri, K., Beferull-lozano, B., Tsakalides, P.: Training a svm-based classifier in distributed sensor networks. In: Proceedings of 14nd European Signal Processing Conference, pp. 1–5 (2006)Google Scholar
- 12.Grossman, R., Gu, Y.: Data mining using high performance data clouds: experimental studies using sector and sphere. In: Proceedings of the 14th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD 2008, New York, USA, pp. 920–927 (2008)Google Scholar
- 13.Grossman, R.L.: The case for cloud computing. IT Professional 11(2), 23–27 (2009)CrossRefGoogle Scholar
- 14.Law, R.: Back-propagation learning in improving the accuracy of neural network-based tourism demand forecasting. Tourism Management 21(4), 331–340 (2000)CrossRefGoogle Scholar
- 15.Menezes, A.J., Oorschot, P.C.V., Vanstone, S.A., Rivest, R.L.: Handbook of applied cryptography (1997)Google Scholar
- 16.Myers, D., Hutchinson, R.: Efficient implementation of piecewise linear activation function for digital vlsi neural networks. Electronics Letters 25(24), 1662–1663 (1989)CrossRefGoogle Scholar
- 17.Rumelhart, D.E., Hinton, G.E., Williams, R.J.: Learning internal representations by error propagation. In: Parallel Distributed Processing: Explorations in the Microstructure of Cognition, vol. 1, pp. 318–362. MIT Press, Cambridge (1986)Google Scholar
- 18.Schlitter, N.: A protocol for privacy preserving neural network learning on horizontal partitioned data. In: Proceedings of the Privacy Statistics in Databases (PSD) (September 2008)Google Scholar
- 19.Yang, B., Wang, Y.-D., Su, X.-H.: Research and Design of Distributed Neural Networks with Chip Training Algorithm. In: Wang, L., Chen, K., S. Ong, Y. (eds.) ICNC 2005, Part I. LNCS, vol. 3610, pp. 213–216. Springer, Heidelberg (2005)CrossRefGoogle Scholar
- 20.Yao, A.C.: Protocols for secure computations. In: Proceedings of the 23rd Annual Symposium on Foundations of Computer Science, SFCS 1982, Washington, DC, USA, pp. 160–164 (1982)Google Scholar
- 21.Zang, S., Zhong, S.: A privacy-preserving algorithm for distributed training of neural network ensembles. To appear in Neural Computing and ApplicationsGoogle Scholar