Skip to main content
SpringerLink
Log in

Towards Practical Secure Privacy-Preserving Machine (Deep) Learning with Distributed Data

  • Conference paper
  • First Online:
Database and Expert Systems Applications - DEXA 2022 Workshops (DEXA 2022)

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 1633))

Included in the following conference series:

Abstract

A methodology for practical secure privacy-preserving distributed machine (deep) learning is proposed via addressing the core issues of fully homomorphic encryption, differential privacy, and scalable fast machine learning. Considering that private data is distributed and the training data may contain directly or indirectly an information about private data, an architecture and a methodology are suggested for

  1. 1.

    mitigating the impracticality issue of fully homomorphic encryption (arising from large computational overhead) via very fast gate-by-gate bootstrapping and introducing a learning scheme that requires homomorphic computation of only efficient-to-evaluate functions;

  2. 2.

    addressing the privacy-accuracy tradeoff issue of differential privacy via optimizing the noise adding mechanism;

  3. 3.

    defining an information theoretic measure of privacy-leakage for the design and analysis of privacy-preserving schemes; and

  4. 4.

    addressing the optimal model size determination issue and computationally fast training issue of scalable and fast machine (deep) learning with an alternative approach based on variational learning.

A biomedical application example is provided to demonstrate the application potential of the proposed methodology.

Supported by the Austrian Research Promotion Agency (FFG) Project PRIMAL (Privacy Preserving Machine Learning for Industrial Applications); FFG Project SMiLe (Secure Machine Learning Applications with Homomorphically Encrypted Data); FFG COMET-Modul S3AI (Security and Safety for Shared Artificial Intelligence); FFG Sub-Project PETAI (Privacy Secured Explainable and Transferable AI for Healthcare Systems); EU Horizon 2020 Project SERUMS (Securing Medical Data in Smart Patient-Centric Healthcare Systems); the BMK; the BMDW; and the Province of Upper Austria in the frame of the COMET Programme managed by FFG.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 109.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 139.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Abadi, M., et al.: Deep learning with differential privacy. In: Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security, pp. 308–318. Association for Computing Machinery, New York (2016)

    Google Scholar 

  2. Balle, B., Wang, Y.: Improving the gaussian mechanism for differential privacy: analytical calibration and optimal denoising. CoRR abs/1805.06530 (2018)

    Google Scholar 

  3. Basciftci, Y.O., Wang, Y., Ishwar, P.: On privacy-utility tradeoffs for constrained data release mechanisms. In: 2016 Information Theory and Applications Workshop (ITA), pp. 1–6 (2016)

    Google Scholar 

  4. Brakerski, Z.: Fully homomorphic encryption without modulus switching from classical GapSVP. In: Safavi-Naini, R., Canetti, R. (eds.) CRYPTO 2012. LNCS, vol. 7417, pp. 868–886. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-32009-5_50

    Chapter  Google Scholar 

  5. Brakerski, Z., Gentry, C., Vaikuntanathan, V.: (leveled) fully homomorphic encryption without bootstrapping. In: Proceedings of the 3rd Innovations in Theoretical Computer Science Conference, ITCS 2012, pp. 309–325. Association for Computing Machinery, New York (2012)

    Google Scholar 

  6. Calmon, F.D.P., Fawaz, N.: Privacy against statistical inference. In: Proceedings of the 50th Annual Allerton Conference on Communication, Control, and Computing, Allerton 2012 (2012)

    Google Scholar 

  7. Chen, X., Duan, Y., Houthooft, R., Schulman, J., Sutskever, I., Abbeel, P.: Infogan: interpretable representation learning by information maximizing generative adversarial nets. In: Lee, D.D., Sugiyama, M., Luxburg, U.V., Guyon, I., Garnett, R. (eds.) Advances in Neural Information Processing Systems, vol. 29, pp. 2172–2180. Curran Associates, Inc. (2016)

    Google Scholar 

  8. Cheon, J.H., Coron, J.-S., Kim, J., Lee, M.S., Lepoint, T., Tibouchi, M., Yun, A.: Batch fully homomorphic encryption over the integers. In: Johansson, T., Nguyen, P.Q. (eds.) EUROCRYPT 2013. LNCS, vol. 7881, pp. 315–335. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-38348-9_20

    Chapter  Google Scholar 

  9. Chillotti, I., Gama, N., Georgieva, M., Izabachène, M.: Faster fully homomorphic encryption: bootstrapping in less than 0.1 seconds. In: Cheon, J.H., Takagi, T. (eds.) ASIACRYPT 2016. LNCS, vol. 10031, pp. 3–33. Springer, Heidelberg (2016). https://doi.org/10.1007/978-3-662-53887-6_1

    Chapter  MATH  Google Scholar 

  10. Chillotti, I., Gama, N., Georgieva, M., Izabachène, M.: Faster packed homomorphic operations and efficient circuit bootstrapping for TFHE. In: Takagi, T., Peyrin, T. (eds.) ASIACRYPT 2017. LNCS, vol. 10624, pp. 377–408. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-70694-8_14

    Chapter  Google Scholar 

  11. Chillotti, I., Gama, N., Georgieva, M., Izabachène, M.: TFHE: fast fully homomorphic encryption library (2016). https://tfhe.github.io/tfhe/

  12. Coron, J.-S., Mandal, A., Naccache, D., Tibouchi, M.: Fully homomorphic encryption over the integers with shorter public keys. In: Rogaway, P. (ed.) CRYPTO 2011. LNCS, vol. 6841, pp. 487–504. Springer, Heidelberg (2011). https://doi.org/10.1007/978-3-642-22792-9_28

    Chapter  Google Scholar 

  13. van Dijk, M., Gentry, C., Halevi, S., Vaikuntanathan, V.: Fully homomorphic encryption over the integers. In: Gilbert, H. (ed.) EUROCRYPT 2010. LNCS, vol. 6110, pp. 24–43. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-13190-5_2

    Chapter  Google Scholar 

  14. Ducas, L., Micciancio, D.: FHEW: bootstrapping homomorphic encryption in less than a second. In: Oswald, E., Fischlin, M. (eds.) EUROCRYPT 2015. LNCS, vol. 9056, pp. 617–640. Springer, Heidelberg (2015). https://doi.org/10.1007/978-3-662-46800-5_24

    Chapter  MATH  Google Scholar 

  15. Dwork, C., Kenthapadi, K., McSherry, F., Mironov, I., Naor, M.: Our data, ourselves: privacy via distributed noise generation. In: Vaudenay, S. (ed.) EUROCRYPT 2006. LNCS, vol. 4004, pp. 486–503. Springer, Heidelberg (2006). https://doi.org/10.1007/11761679_29

    Chapter  Google Scholar 

  16. Dwork, C., Roth, A.: The algorithmic foundations of differential privacy. Found. Trends Theor. Comput. Sci. 9(3–4), 211–407 (2014)

    MathSciNet  MATH  Google Scholar 

  17. Fan, J., Vercauteren, F.: Somewhat practical fully homomorphic encryption. IACR Cryptol. ePrint Arch. 2012, 144 (2012). http://eprint.iacr.org/2012/144

  18. Fredrikson, M., Jha, S., Ristenpart, T.: Model inversion attacks that exploit confidence information and basic countermeasures. In: Proceedings of the 22Nd ACM SIGSAC Conference on Computer and Communications Security, CCS 2015, pp. 1322–1333. ACM, New York (2015)

    Google Scholar 

  19. Geng, Q., Kairouz, P., Oh, S., Viswanath, P.: The staircase mechanism in differential privacy. IEEE J. Sel. Topics Signal Process. 9(7), 1176–1184 (2015)

    Article  Google Scholar 

  20. Geng, Q., Viswanath, P.: The optimal noise-adding mechanism in differential privacy. IEEE Trans. Inf. Theory 62(2), 925–951 (2016)

    Article  MathSciNet  Google Scholar 

  21. Geng, Q., Viswanath, P.: Optimal noise adding mechanisms for approximate differential privacy. IEEE Trans. Inf.Theory 62(2), 952–969 (2016)

    Article  MathSciNet  Google Scholar 

  22. Geng, Q., Ding, W., Guo, R., Kumar, S.: Optimal noise-adding mechanism in additive differential privacy. CoRR abs/1809.10224 (2018)

    Google Scholar 

  23. Gentry, C.: Fully homomorphic encryption using ideal lattices. In: STOC 2009, pp. 169–178. Association for Computing Machinery, New York (2009)

    Google Scholar 

  24. Gentry, C., Sahai, A., Waters, B.: Homomorphic encryption from learning with errors: conceptually-simpler, asymptotically-faster, attribute-based. In: Canetti, R., Garay, J.A. (eds.) CRYPTO 2013. LNCS, vol. 8042, pp. 75–92. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-40041-4_5

    Chapter  Google Scholar 

  25. Ghosh, A., Roughgarden, T., Sundararajan, M.: Universally utility-maximizing privacy mechanisms. SIAM J. Comput. 41(6), 1673–1693 (2012)

    Article  MathSciNet  Google Scholar 

  26. Gupte, M., Sundararajan, M.: Universally optimal privacy mechanisms for minimax agents. In: Proceedings of the Twenty-ninth ACM SIGMOD-SIGACT-SIGART Symposium on Principles of Database Systems, PODS 2010, pp. 135–146. ACM, New York (2010)

    Google Scholar 

  27. Huang, C., Kairouz, P., Chen, X., Sankar, L., Rajagopal, R.: Context-aware generative adversarial privacy. Entropy 19(12), 656 (2017)

    Article  MathSciNet  Google Scholar 

  28. Kifer, D., Machanavajjhala, A.: No free lunch in data privacy. In: Proceedings of the 2011 ACM SIGMOD International Conference on Management of Data, SIGMOD 2011, pp. 193–204. Association for Computing Machinery, New York (2011)

    Google Scholar 

  29. Kumar, M., Freudenthaler, B.: Fuzzy membership functional analysis for nonparametric deep models of image features. IEEE Trans. Fuzzy Syst. 28(12), 3345–3359 (2020)

    Article  Google Scholar 

  30. Kumar, M., Rossbory, M., Moser, B.A., Freudenthaler, B.: Deriving an optimal noise adding mechanism for privacy-preserving machine learning. In: Anderst-Kotsis, G., et al. (eds.) DEXA 2019. CCIS, vol. 1062, pp. 108–118. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-27684-3_15

    Chapter  Google Scholar 

  31. Kumar, M., Rossbory, M., Moser, B.A., Freudenthaler, B.: An optimal \((\epsilon ,\delta )-\)differentially private learning of distributed deep fuzzy models. Inf. Sci. 546, 87–120 (2021)

    Article  MathSciNet  Google Scholar 

  32. Kumar, M.: Differentially private transferrable deep learning with membership-mappings. CoRR abs/2105.04615 (2021). https://arxiv.org/abs/2105.04615v6

  33. Kumar, M., Brunner, D., Moser, B.A., Freudenthaler, B.: Variational optimization of informational privacy. In: Kotsis, G., et al. (eds.) DEXA 2020. CCIS, vol. 1285, pp. 32–47. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-59028-4_4

    Chapter  Google Scholar 

  34. Kumar, M., Moser, B., Fischer, L., Freudenthaler, B.: Membership-mappings for data representation learning: a bregman divergence based conditionally deep autoencoder. In: Kotsis, G., et al. (eds.) DEXA 2021. CCIS, vol. 1479, pp. 138–147. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-87101-7_14

    Chapter  Google Scholar 

  35. Kumar, M., Moser, B., Fischer, L., Freudenthaler, B.: Membership-mappings for data representation learning: measure theoretic conceptualization. In: Kotsis, G., et al. (eds.) DEXA 2021. CCIS, vol. 1479, pp. 127–137. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-87101-7_13

    Chapter  Google Scholar 

  36. Kumar, M., Moser, B.A., Fischer, L., Freudenthaler, B.: Information theoretic evaluation of privacy-leakage, interpretability, and transferability for trustworthy AI. CoRR abs/2106.06046 (2021). https://arxiv.org/abs/2106.06046v5

  37. Kumar, M., Rossbory, M., Moser, B.A., Freudenthaler, B.: Differentially private learning of distributed deep models. In: Adjunct Publication of the 28th ACM Conference on User Modeling, Adaptation and Personalization, UMAP 2020 Adjunct, pp. 193–200. Association for Computing Machinery, New York (2020)

    Google Scholar 

  38. Kumar, M., Singh, S., Freudenthaler, B.: Gaussian fuzzy theoretic analysis for variational learning of nested compositions. Int. J. Approx. Reas. 131, 1–29 (2021)

    Article  MathSciNet  Google Scholar 

  39. Kumar, M., Zhang, W., Weippert, M., Freudenthaler, B.: An explainable fuzzy theoretic nonparametric deep model for stress assessment using heartbeat intervals analysis. IEEE Trans. Fuzzy Syst. 29(12), 3873–3886 (2021)

    Article  Google Scholar 

  40. Liu, C., Chakraborty, S., Mittal, P.: Dependence makes you vulnberable: Differential privacy under dependent tuples. In: 23rd Annual Network and Distributed System Security Symposium, NDSS 2016, San Diego, California, USA, 21–24 February 2016. The Internet Society (2016)

    Google Scholar 

  41. Nuida, K., Kurosawa, K.: (Batch) fully homomorphic encryption over integers for non-binary message spaces. In: Oswald, E., Fischlin, M. (eds.) EUROCRYPT 2015. LNCS, vol. 9056, pp. 537–555. Springer, Heidelberg (2015). https://doi.org/10.1007/978-3-662-46800-5_21

    Chapter  Google Scholar 

  42. Phan, N., Wang, Y., Wu, X., Dou, D.: Differential privacy preservation for deep auto-encoders: An application of human behavior prediction. In: Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence, AAAI 2016, pp. 1309–1316. AAAI Press (2016)

    Google Scholar 

  43. Rebollo-Monedero, D., Forné, J., Domingo-Ferrer, J.: From t-closeness-like privacy to postrandomization via information theory. IEEE Trans. Knowl. Data Eng. 22(11), 1623–1636 (2010)

    Article  Google Scholar 

  44. Regev, O.: On lattices, learning with errors, random linear codes, and cryptography. In: Proceedings of the Thirty-Seventh Annual ACM Symposium on Theory of Computing, STOC 2005, pp. 84–93. Association for Computing Machinery, New York (2005)

    Google Scholar 

  45. Sankar, L., Rajagopalan, S.R., Poor, H.V.: Utility-privacy tradeoffs in databases: an information-theoretic approach. IEEE Trans. Inf. Forensics Secur. 8(6), 838–852 (2013)

    Article  Google Scholar 

  46. Tripathy, A., Wang, Y., Ishwar, P.: Privacy-preserving adversarial networks. In: 2019 57th Annual Allerton Conference on Communication, Control, and Computing (Allerton), pp. 495–505 (2019)

    Google Scholar 

  47. Wang, Y., Basciftci, Y.O., Ishwar, P.: Privacy-utility tradeoffs under constrained data release mechanisms. CoRR abs/1710.09295 (2017). http://arxiv.org/abs/1710.09295

Download references

Author information

Authors and Affiliations

  1. Software Competence Center Hagenberg GmbH, 4232, Hagenberg, Austria

    Mohit Kumar, Bernhard Moser, Lukas Fischer & Bernhard Freudenthaler

  2. Institute of Automation, Faculty of Computer Science and Electrical Engineering, University of Rostock, Rostock, Germany

    Mohit Kumar

Authors
  1. Mohit Kumar
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Bernhard Moser
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Lukas Fischer
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Bernhard Freudenthaler
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mohit Kumar .

Editor information

Editors and Affiliations

  1. Johannes Kepler University of Linz, Linz, Oberösterreich, Austria

    Gabriele Kotsis

  2. Technical University of Vienna, Vienna, Austria

    A Min Tjoa

  3. Johannes Kepler University of Linz, Linz, Austria

    Ismail Khalil

  4. Software Competence Center Hagenberg, Hagenberg, Austria

    Bernhard Moser

  5. (WU) Vienna University of Economics and Business, Vienna, Austria

    Alfred Taudes

  6. Johannes Kepler University of Linz, Linz, Austria

    Atif Mashkoor

  7. Johannes Kepler University of Linz, Linz, Austria

    Johannes Sametinger

  8. Software Competence Center Hagenberg, Hagenberg, Austria

    Jorge Martinez-Gil

  9. Software Competence Center Hagenberg, Hagenberg, Austria

    Florian Sobieczky

  10. Software Competence Center Hagenberg, Hagenberg, Austria

    Lukas Fischer

  11. Software Competence Center Hagenberg, Hagenberg, Austria

    Rudolf Ramler

  12. Pak-Austria Fachhochschule - Institute of Applied Sciences and Technology (PAF-IAST), Haripur, Pakistan

    Maqbool Khan

  13. Software Competence Center Hagenberg, Hagenberg, Austria

    Gerald Czech

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Kumar, M., Moser, B., Fischer, L., Freudenthaler, B. (2022). Towards Practical Secure Privacy-Preserving Machine (Deep) Learning with Distributed Data. In: Kotsis, G., et al. Database and Expert Systems Applications - DEXA 2022 Workshops. DEXA 2022. Communications in Computer and Information Science, vol 1633. Springer, Cham. https://doi.org/10.1007/978-3-031-14343-4_6

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-14343-4_6

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-14342-7

  • Online ISBN: 978-3-031-14343-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation