Advertisement

Secure Multiparty Learning from Aggregation of Locally Trained Models

  • Xu MaEmail author
  • Cunmei Ji
  • Xiaoyu Zhang
  • Jianfeng Wang
  • Jin Li
  • Kuan-Ching Li
Conference paper
  • 663 Downloads
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11806)

Abstract

In this paper, we propose a new protocol for secure multiparty learning (SML) from the aggregation of locally trained models, by using homomorphic proxy re-encryption and aggregate signature techniques. In our scheme, we utilize the method of secure verifiable computation delegation to privately generate labels for auxiliary unlabeled public data. Based on the labeled dataset, a central entity can learn a global learning model without direct access to the local private datasets. The generalization performance of SML is excellent and almost equals to the accuracy of the model learned from the union of all the parties’ datasets. We implement SML on MNIST, and extensive analysis shows that our method is effective, efficient and secure.

Keywords

Aggregate signature Proxy re-encryption Multiparty learning Computation delegation 

References

  1. 1.
    Alipanahi, B., Delong, A., Weirauch, M.T., Frey, B.J.: Predicting the sequence specificities of DNA-and RNA-binding proteins by deep learning. Nat. Biotechnol. 33(8), 831 (2015)CrossRefGoogle Scholar
  2. 2.
    Aono, Y., Hayashi, T., Wang, L., Moriai, S.: Privacy-preserving deep learning via additively homomorphic encryption. IEEE Trans. Inf. Forensics Secur. 13(5), 1333–1345 (2018)CrossRefGoogle Scholar
  3. 3.
    Barni, M., Orlandi, C., Piva, A.: A privacy-preserving protocol for neural-network-based computation. In: Proceedings of the 8th Workshop on Multimedia and Security, pp. 146–151. ACM (2006)Google Scholar
  4. 4.
    Blaze, M., Bleumer, G., Strauss, M.: Divertible protocols and atomic proxy cryptography. In: Nyberg, K. (ed.) EUROCRYPT 1998. LNCS, vol. 1403, pp. 127–144. Springer, Heidelberg (1998).  https://doi.org/10.1007/BFb0054122CrossRefGoogle Scholar
  5. 5.
    Boneh, D., Gentry, C., Lynn, B., Shacham, H.: Aggregate and verifiably encrypted signatures from bilinear maps. In: Biham, E. (ed.) EUROCRYPT 2003. LNCS, vol. 2656, pp. 416–432. Springer, Heidelberg (2003).  https://doi.org/10.1007/3-540-39200-9_26CrossRefGoogle Scholar
  6. 6.
    Chaudhuri, K., Monteleoni, C., Sarwate, A.D.: Differentially private empirical risk minimization. J. Mach. Learn. Res. 12(Mar), 1069–1109 (2011)MathSciNetzbMATHGoogle Scholar
  7. 7.
    Chen, X., Li, J., Ma, J., Tang, Q., Lou, W.: New algorithms for secure outsourcing of modular exponentiations. IEEE Trans. Parallel Distrib. Syst. 25(9), 2386–2396 (2014)CrossRefGoogle Scholar
  8. 8.
    Chen, X., Li, J., Weng, J., Ma, J., Lou, W.: Verifiable computation over large database with incremental updates. IEEE Trans. Comput. 65(10), 3184–3195 (2016)MathSciNetCrossRefGoogle Scholar
  9. 9.
    Du, W., Han, Y.S., Chen, S.: Privacy-preserving multivariate statistical analysis: linear regression and classification. In: Proceedings of the Fourth SIAM International Conference on Data Mining, pp. 222–233 (2004)Google Scholar
  10. 10.
    Fredrikson, M., Jha, S., Ristenpart, T.: Model inversion attacks that exploit confidence information and basic countermeasures. In: Proceedings of the 22nd ACM SIGSAC Conference on Computer and Communications Security, pp. 1322–1333. ACM (2015)Google Scholar
  11. 11.
    Graves, A., Mohamed, A.R., Hinton, G.E.: Speech recognition with deep recurrent neural networks. In: IEEE International Conference on Acoustics, Speech and Signal Processing, pp. 6645–6649 (2013)Google Scholar
  12. 12.
    Hamm, J., Cao, Y., Belkin, M.: Learning privately from multiparty data. In: Proceedings of the 33nd International Conference on Machine Learning, pp. 555–563 (2016)Google Scholar
  13. 13.
    Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. Commun. ACM 60(6), 84–90 (2017)CrossRefGoogle Scholar
  14. 14.
    Lindell, Y., Pinkas, B.: Privacy preserving data mining. J. Cryptol. 15(3), 177–206 (2002)MathSciNetCrossRefGoogle Scholar
  15. 15.
    Ma, X., Chen, X., Zhang, X.: Non-interactive privacy-preserving neural network prediction. Inf. Sci. 481, 507–519 (2019)CrossRefGoogle Scholar
  16. 16.
    Ma, X., Zhang, F., Chen, X., Shen, J.: Privacy preserving multi-party computation delegation for deep learning in cloud computing. Inf. Sci. 459, 103–116 (2018)CrossRefGoogle Scholar
  17. 17.
    Mohassel, P., Zhang, Y.: SecureML: a system for scalable privacy-preserving machine learning. In: Proceedings of the 2017 38th IEEE Symposium on Security and Privacy (SP), pp. 19–38. IEEE (2017)Google Scholar
  18. 18.
    Papernot, N., Abadi, M., Erlingsson, U., Goodfellow, I., Talwar, K.: Semi-supervised knowledge transfer for deep learning from private training data. arXiv preprint arXiv:1610.05755 (2016)
  19. 19.
    Ren, S., He, K., Girshick, R., Sun, J.: Faster R-CNN: towards real-time object detection with region proposal networks. In: Proceedings of the Advances in Neural Information Processing Systems, pp. 91–99 (2015)Google Scholar
  20. 20.
    Shokri, R., Shmatikov, V.: Privacy-preserving deep learning. In: Proceedings of the 22nd ACM SIGSAC Conference on Computer and Communications Security, pp. 1310–1321. ACM (2015)Google Scholar
  21. 21.
    Slavkovic, A.B., Nardi, Y., Tibbits, M.M.: Secure logistic regression of horizontally and vertically partitioned distributed databases. In: Workshops Proceedings of the 7th IEEE International Conference on Data Mining, pp. 723–728 (2007)Google Scholar
  22. 22.
    Zhang, X., Chen, X., Wang, J., Zhan, Z., Li, J.: Verifiable privacy-preserving single-layer perceptron training scheme in cloud computing. Soft. Comput. 22(23), 7719–7732 (2018)CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Xu Ma
    • 1
    • 2
    Email author
  • Cunmei Ji
    • 2
  • Xiaoyu Zhang
    • 1
  • Jianfeng Wang
    • 1
  • Jin Li
    • 3
  • Kuan-Ching Li
    • 4
  1. 1.State Key Laboratory of Integrated Service Networks (ISN)Xidian UniversityXi’anChina
  2. 2.School of SoftwareQufu Normal UniversityQufuChina
  3. 3.School of Computer Science and Educational SoftwareGuangzhou UniversityGuangzhouChina
  4. 4.School of Computer Science and Information EngineeringProvidence UniversityTaichungTaiwan

Personalised recommendations