Advertisement

SoK: Cryptography for Neural Networks

  • Monir Azraoui
  • Muhammad Bahram
  • Beyza Bozdemir
  • Sébastien Canard
  • Eleonora Ciceri
  • Orhan ErmisEmail author
  • Ramy Masalha
  • Marco Mosconi
  • Melek Önen
  • Marie Paindavoine
  • Boris Rozenberg
  • Bastien Vialla
  • Sauro Vicini
Chapter
  • 51 Downloads
Part of the IFIP Advances in Information and Communication Technology book series (IFIPAICT, volume 576)

Abstract

With the advent of big data technologies which bring better scalability and performance results, machine learning (ML) algorithms become affordable in several different applications and areas. The use of large volumes of data to obtain accurate predictions unfortunately come with a high cost in terms of privacy exposures. The underlying data are often personal or confidential and, therefore, need to be appropriately safeguarded. Given the cost of machine learning algorithms, these would need to be outsourced to third-party servers, and hence protection of the data becomes mandatory. While traditional data encryption solutions would not allow accessing the content of the data, these would, nevertheless, prevent third-party servers from executing the ML algorithms properly. The goal is, therefore, to come up with customized ML algorithms that would, by design, preserve the privacy of the processed data. Advanced cryptographic techniques such as fully homomorphic encryption or secure multi-party computation enable the execution of some operations over protected data and, therefore, can be considered as potential candidates for these algorithms. However, these techniques incur high computational and/or communication costs for some operations. In this paper, we propose a Systematization of Knowledge (SoK) whereby we analyze the tension between a particular ML technique, namely, neural networks (NN), and the characteristics of relevant cryptographic techniques.

Keywords

Privacy Neural networks Homomorphic encryption Secure multi-party computation 

Notes

Acknowledgement

This work was partly supported by the PAPAYA project funded by the European Union’s Horizon 2020 Research and Innovation Programme, under Grant Agreement no. 786767.

References

  1. 1.
    HElib: An Implementation of homomorphic encryption (2013). https://github.com/shaih/HElib
  2. 2.
    GDPR. Official Journal of the European Union (2016)Google Scholar
  3. 3.
    Ball, M., Carmer, B., Malkin, T., Rosulek, M., Schimanski, N.: Garbled neural networks are practical. Cryptology ePrint Archive, Report 2019/338 (2019)Google Scholar
  4. 4.
    Beaver, D.: Efficient multiparty protocols using circuit randomization. In: Feigenbaum, J. (ed.) CRYPTO 1991. LNCS, vol. 576, pp. 420–432. Springer, Heidelberg (1992).  https://doi.org/10.1007/3-540-46766-1_34CrossRefGoogle Scholar
  5. 5.
    Ben-David, A., Nisan, N., Pinkas, B.: FairplayMP: a system for secure multi-party computation. In: CCS (2008)Google Scholar
  6. 6.
    Boneh, D., Goh, E.-J., Nissim, K.: Evaluating 2-DNF formulas on ciphertexts. In: Kilian, J. (ed.) TCC 2005. LNCS, vol. 3378, pp. 325–341. Springer, Heidelberg (2005).  https://doi.org/10.1007/978-3-540-30576-7_18CrossRefGoogle Scholar
  7. 7.
    Bourse, F., Minelli, M., Minihold, M., Paillier, P.: Fast homomorphic evaluation of deep discretized neural networks. In: Shacham, H., Boldyreva, A. (eds.) CRYPTO 2018. LNCS, vol. 10993, pp. 483–512. Springer, Cham (2018).  https://doi.org/10.1007/978-3-319-96878-0_17CrossRefGoogle Scholar
  8. 8.
    Brakerski, Z., Gentry, C., Vaikuntanathan, V.: Fully homomorphic encryption without bootstrapping. Cryptology ePrint Archive, Report 2011/277 (2011)Google Scholar
  9. 9.
    Brakerski, Z., Gentry, C., Vaikuntanathan, V.: (Leveled) fully homomorphic encryption without bootstrapping. In: ITCS (2012)Google Scholar
  10. 10.
    Camgöz, N.C., Kındıroğlu, A.A., Akarun, L.: Sign language recognition for assisting the deaf in hospitals. In: Chetouani, M., Cohn, J., Salah, A.A. (eds.) HBU 2016. LNCS, vol. 9997, pp. 89–101. Springer, Cham (2016).  https://doi.org/10.1007/978-3-319-46843-3_6CrossRefGoogle Scholar
  11. 11.
    Canard, S., Carpov, S., Nokam, D., Sirdey, R.: Running compression algorithms in the encrypted domain: a case-study on the homomorphic execution of RLE (2017)Google Scholar
  12. 12.
    Chabanne, H., de Wargny, A., Milgram, J., Morel, C., Prouff, E.: Privacy-preserving classification on deep neural network (2017)Google Scholar
  13. 13.
    Chandran, N., Gupta, D., Rastogi, A., Sharma, R., Tripathi, S.: EzPC: programmable, efficient, and scalable secure two-party computation for machine learning. Euro S&P (2019)Google Scholar
  14. 14.
    Chen, L.F., Liao, H.Y.M., Ko, M.T., Lin, J.C., Yu, G.J.: A new LDA-based face recognition system which can solve the small sample size problem. Pattern Recogn. 33, 1713–1726 (2000)CrossRefGoogle Scholar
  15. 15.
    Cheon, J.H., Kim, A., Kim, M., Song, Y.: Homomorphic encryption for arithmetic of approximate numbers. In: Takagi, T., Peyrin, T. (eds.) ASIACRYPT 2017. LNCS, vol. 10624, pp. 409–437. Springer, Cham (2017).  https://doi.org/10.1007/978-3-319-70694-8_15CrossRefGoogle Scholar
  16. 16.
    Cheon, J.H., Stehlé, D.: Fully homomophic encryption over the integers revisited. In: Oswald, E., Fischlin, M. (eds.) EUROCRYPT 2015. LNCS, vol. 9056, pp. 513–536. Springer, Heidelberg (2015).  https://doi.org/10.1007/978-3-662-46800-5_20CrossRefGoogle Scholar
  17. 17.
    Chillotti, I., Gama, N., Georgieva, M., Izabachène, M.: Faster fully homomorphic encryption: bootstrapping in less than 0.1 seconds. In: Cheon, J.H., Takagi, T. (eds.) ASIACRYPT 2016. LNCS, vol. 10031, pp. 3–33. Springer, Heidelberg (2016).  https://doi.org/10.1007/978-3-662-53887-6_1CrossRefzbMATHGoogle Scholar
  18. 18.
    Chillotti, I., Gama, N., Georgieva, M., Izabachène, M.: Faster packed homomorphic operations and efficient circuit bootstrapping for TFHE. In: Takagi, T., Peyrin, T. (eds.) ASIACRYPT 2017. LNCS, vol. 10624, pp. 377–408. Springer, Cham (2017).  https://doi.org/10.1007/978-3-319-70694-8_14CrossRefGoogle Scholar
  19. 19.
    Demmler, D., Schneider, T., Zohner, M.: ABY - a framework for efficient mixed-protocol secure two-party computation. In: NDSS (2015)Google Scholar
  20. 20.
    Ejgenberg, Y., Farbstein, M., Levy, M., Lindell, Y.: SCAPI: the secure computation application programming interface. Cryptology ePrint Archive, Report 2012/629 (2012)Google Scholar
  21. 21.
    ElGamal, T.: A public key cryptosystem and a signature scheme based on discrete logarithms. In: Blakley, G.R., Chaum, D. (eds.) CRYPTO 1984. LNCS, vol. 196, pp. 10–18. Springer, Heidelberg (1985).  https://doi.org/10.1007/3-540-39568-7_2CrossRefGoogle Scholar
  22. 22.
    Fan, J., Vercauteren, F.: Somewhat practical fully homomorphic encryption. Cryptology ePrint Archive, Report 2012/144 (2012)Google Scholar
  23. 23.
    Fan, J., Vercauteren, F.: Somewhat practical fully homomorphic encryption. IACR Cryptology ePrint Archive (2012)Google Scholar
  24. 24.
    Gentry, C.: Fully homomorphic encryption using ideal lattices. In: Mitzenmacher, M. (ed.) STOC (2009)Google Scholar
  25. 25.
    Gentry, C., Sahai, A., Waters, B.: Homomorphic encryption from learning with errors: conceptually-simpler, asymptotically-faster, attribute-based. In: Canetti, R., Garay, J.A. (eds.) CRYPTO 2013. LNCS, vol. 8042, pp. 75–92. Springer, Heidelberg (2013).  https://doi.org/10.1007/978-3-642-40041-4_5CrossRefGoogle Scholar
  26. 26.
    Gilad-Bachrach, R., Dowlin, N., Laine, K., Lauter, K.E., Naehrig, M., Wernsing, J.: CryptoNets: applying neural networks to encrypted data with high throughput and accuracy. In: ICML (2016)Google Scholar
  27. 27.
    Goldreich, O., Micali, S., Wigderson, A.: How to play any mental game or A completeness theorem for protocols with honest majority. In: STOC (1987)Google Scholar
  28. 28.
    Goldreich, O., Micali, S., Wigderson, A.: How to play any mental game or A completeness theorem for protocols with honest majority. In: ACM Symposium on Theory of Computing (1987)Google Scholar
  29. 29.
    Halevi, S., Shoup, V.: Algorithms in HElib. In: Garay, J.A., Gennaro, R. (eds.) CRYPTO 2014. LNCS, vol. 8616, pp. 554–571. Springer, Heidelberg (2014).  https://doi.org/10.1007/978-3-662-44371-2_31CrossRefzbMATHGoogle Scholar
  30. 30.
    Halevi, S., Shoup, V.: Bootstrapping for HElib. In: Oswald, E., Fischlin, M. (eds.) EUROCRYPT 2015. LNCS, vol. 9056, pp. 641–670. Springer, Heidelberg (2015).  https://doi.org/10.1007/978-3-662-46800-5_25CrossRefGoogle Scholar
  31. 31.
    Hannun, A.Y., et al.: Cardiologist-level arrhythmia detection and classification in ambulatory electrocardiograms using a deep neural network. Nat. Med. 25(1), 65 (2019)CrossRefGoogle Scholar
  32. 32.
    Haralick, R.M., Shanmugam, K., Dinstein, I.: Textural features for image classification. IEEE Trans. Syst. Man Cybern. 6, 610–621 (1973)CrossRefGoogle Scholar
  33. 33.
    Henecka, W., Kögl, S., Sadeghi, A., Schneider, T., Wehrenberg, I.: TASTY: tool for automating secure two-party computations. In: ACM CCS (2010)Google Scholar
  34. 34.
    Hesamifard, E., Takabi, H., Ghasemi, M., Wright, R.N.: Privacy-preserving Machine Learning as a Service. PETS 2018, 123–142 (2018)Google Scholar
  35. 35.
    Ibarrondo, A., Önen, M.: FHE-compatible batch normalization for privacy preserving deep learning. In: Garcia-Alfaro, J., Herrera-Joancomartí, J., Livraga, G., Rios, R. (eds.) DPM/CBT 2018. LNCS, vol. 11025, pp. 389–404. Springer, Cham (2018).  https://doi.org/10.1007/978-3-030-00305-0_27CrossRefGoogle Scholar
  36. 36.
    Juvekar, C., Vaikuntanathan, V., Chandrakasan, A.: Gazelle: a low latency framework for secure neural network inference. arXiv preprint (2018)Google Scholar
  37. 37.
    Lindell, Y., Pinkas, B.: Privacy preserving data mining. In: Bellare, M. (ed.) CRYPTO 2000. LNCS, vol. 1880, pp. 36–54. Springer, Heidelberg (2000).  https://doi.org/10.1007/3-540-44598-6_3CrossRefGoogle Scholar
  38. 38.
    Liu, J., Juuti, M., Lu, Y., Asokan, N.: Oblivious neural network predictions via MiniONN transformations. In: ACM CCS (2017)Google Scholar
  39. 39.
    Liu, J., Juuti, M., Lu, Y., Asokan, N.: Oblivious neural network predictions via MiniONN transformations. Cryptology ePrint Archive, Report 2017/452 (2017)Google Scholar
  40. 40.
    López-Alt, A., Tromer, E., Vaikuntanathan, V.: On-the-fly multiparty computation on the cloud via multikey fully homomorphic encryption. In: STOC (2012)Google Scholar
  41. 41.
    Malkhi, D., Nisan, N., Pinkas, B., Sella, Y.: Fairplay—a secure two-party computation system. In: USENIX (2004)Google Scholar
  42. 42.
    Mansouri, M., Bozdemir, B., Önen, M., Ermis, O.: PAC: privacy-preserving arrhythmia classification with neural networks. In: FPS (2019)Google Scholar
  43. 43.
    Aguilar Melchor, C., Kilijian, M.-O., Lefebvre, C., Ricosset, T.: A comparison of the homomorphic encryption libraries HElib, SEAL and FV-NFLlib. In: Lanet, J.-L., Toma, C. (eds.) SECITC 2018. LNCS, vol. 11359, pp. 425–442. Springer, Cham (2019).  https://doi.org/10.1007/978-3-030-12942-2_32CrossRefGoogle Scholar
  44. 44.
    Mohassel, P., Rindal, P.: ABY\(^{3}\): a mixed protocol framework for machine learning. In: ACM CCS (2018)Google Scholar
  45. 45.
    Mohassel, P., Zhang, Y.: SecureML: a system for scalable privacy-preserving machine learning. In: S&P (2017)Google Scholar
  46. 46.
    Ohrimenko, O., et al.: Oblivious multi-party machine learning on trusted processors. In: USENIX (2016)Google Scholar
  47. 47.
    Orlandi, C., Piva, A., Barni, M.: Oblivious neural network computing via homomorphic encryption. EURASIP (2007)Google Scholar
  48. 48.
    Paillier, P.: Public-key cryptosystems based on composite degree residuosity classes. In: Stern, J. (ed.) EUROCRYPT 1999. LNCS, vol. 1592, pp. 223–238. Springer, Heidelberg (1999).  https://doi.org/10.1007/3-540-48910-X_16CrossRefGoogle Scholar
  49. 49.
    Rabin, M.O.: How to exchange secrets with oblivious transfer. Cryptology ePrint Archive, Report 2005/187 (2005)Google Scholar
  50. 50.
    Rivest, R.L., Shamir, A., Adleman, L.M.: A method for obtaining digital signatures and public-key cryptosystems. Commun. ACM 21, 120–126 (1978)MathSciNetCrossRefGoogle Scholar
  51. 51.
    Rouhani, B.D., Riazi, M.S., Koushanfar, F.: DeepSecure: scalable provably-secure deep learning. In: DAC (2018)Google Scholar
  52. 52.
    Sadegh Riazi, M., Weinert, C., Tkachenko, O., Songhori, E.M., Schneider, T., Koushanfar, F.: Chameleon: a hybrid secure computation framework for machine learning applications. arXiv e-prints (2018)Google Scholar
  53. 53.
    Singh, K., Sirdey, R., Artiguenave, F., Cohen, D., Carpov, S.: Towards confidentiality-strengthened personalized genomic medicine embedding homomorphic cryptography. In: ICISSP (2017)Google Scholar
  54. 54.
    Srinivasan, S., Latchman, H., Shea, J., Wong, T., McNair, J.: Airborne traffic surveillance systems: video surveillance of highway traffic. In: International Workshop on Video Surveillance & Sensor Networks (2004)Google Scholar
  55. 55.
    Wagh, S., Gupta, D., Chandran, N.: SecureNN: efficient and private neural network training. In: PETS (2019)Google Scholar
  56. 56.
    Wahab, A., Chin, S., Tan, E.: Novel approach to automated fingerprint recognition. IEE Proceedings - Vision, Image and Signal Processing (1998)CrossRefGoogle Scholar
  57. 57.
    Wang, X., Malozemoff, A.J., Katz, J.: Faster secure two-party computation in the single-execution setting. Cryptology ePrint Archive, Report 2016/762 (2016)Google Scholar
  58. 58.
    Yao, A.C.C.: Protocols for secure computations (extended abstract). In: FOCS (1982)Google Scholar
  59. 59.
    Yao, A.C.C.: How to generate and exchange secrets (extended abstract). In: FOCS (1986)Google Scholar

Copyright information

© IFIP International Federation for Information Processing 2020

Authors and Affiliations

  • Monir Azraoui
    • 1
  • Muhammad Bahram
    • 2
  • Beyza Bozdemir
    • 3
  • Sébastien Canard
    • 1
  • Eleonora Ciceri
    • 4
  • Orhan Ermis
    • 3
    Email author
  • Ramy Masalha
    • 2
  • Marco Mosconi
    • 4
  • Melek Önen
    • 3
  • Marie Paindavoine
    • 5
  • Boris Rozenberg
    • 2
  • Bastien Vialla
    • 1
  • Sauro Vicini
    • 4
  1. 1.Applied Crypto Group, Orange LabsCaenFrance
  2. 2.IBM HaifaHaifaIsrael
  3. 3.EURECOMSophia AntipolisFrance
  4. 4.MediaClinicsLissoneItaly
  5. 5.Cybersecurity Research, RenaultParisFrance

Personalised recommendations