Recognition Over Encrypted Faces

  • Hervé ChabanneEmail author
  • Roch Lescuyer
  • Jonathan Milgram
  • Constance Morel
  • Emmanuel Prouff
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11005)


Neural Networks (NN) are today increasingly used in Machine Learning where they have become deeper and deeper to accurately model or classify high-level abstractions of data. Their development however also gives rise to important data privacy risks. This observation motives Microsoft researchers to propose a framework, called Cryptonets. The core idea is to combine simplifications of the NN with Fully Homomorphic Encryptions (FHE) techniques to get both confidentiality of the manipulated data and efficiency of the processing. While efficiency and accuracy are demonstrated when the number of non-linear layers is small (e.g. 2), Cryptonets unfortunately becomes ineffective for deeper NNs which let the privacy preserving problem open in these contexts. This work successfully addresses this problem by combining several new ideas including the use of the batch normalization principle and the splitting of the learning phase in several iterations. We experimentally validate the soundness of our approach with a neural network with 6 non-linear layers. When applied to the MNIST database, it competes with the accuracy of the best non-secure versions, thus significantly improving Cryptonets. Additionally, we applied our approach to secure a neural network used for face recognition. This problem is usually considered much harder than the MNIST hand-written digits recognition and can definitely not be addressed with a simple network like Cryptonets. By combining our new ideas with an iterative (learning) approach we experimentally show that we can build an FHE-friendly network achieving good accuracy for face recognition.



This work was partly supported by the TREDISEC project (G.A. no 644412), funded by the European Union (EU) under the Information and Communication Technologies (ICT) theme of the Horizon 2020 (H2020) research and innovation programme. This work has also been supported in part by the CRYPTOCOMP french FUI17 project.


  1. 1.
    Barni, M., Orlandi, C., Piva, A.: A privacy-preserving protocol for neural-network-based computation. In: Proceedings of the 8th Workshop on Multimedia & Security, MM&Sec 2006, pp. 146–151 (2006)Google Scholar
  2. 2.
    Berg, T., Belhumeur, P.N.: Tom-vs-Pete classifiers and identity-preserving alignment for face verification. In: Bowden, R., Collomosse, J.P., Mikolajczyk, K., (eds.) British Machine Vision Conference. BMVC 2012, 3–7 September 2012, pp. 1–11. BMVA Press, Surrey (2012)Google Scholar
  3. 3.
    Bos, J.W., Lauter, K.E., Naehrig, M.: Private predictive analysis on encrypted medical data. J. Biomed. Inform. 50, 234–243 (2014)CrossRefGoogle Scholar
  4. 4.
    Bost, R., Popa, R.A., Tu, S., Goldwasser, S.: Machine learning classification over encrypted data. IACR Cryptology ePrint Archive, vol. 2014, p. 331 (2014)Google Scholar
  5. 5.
    Chen, H., Han, K., Huang, Z., Jalali, A., Laine, K.: Simple encrypted arithmetic library v2.3.0 (2017).
  6. 6.
    Damgård, I., Jurik, M.: A generalisation, a simplification and some applications of Paillier’s probabilistic public-key system. In: 4th International Workshop on Practice and Theory in Public Key Cryptography Public Key Cryptography. PKC 2001, pp. 119–136 (2001)CrossRefGoogle Scholar
  7. 7.
    Dowlin, N., Gilad-Bachrach, R., Laine, K., Lauter, K.E., Naehrig, M., Wernsing, J.: Manual for using homomorphic encryption for bioinformatics. Proc. IEEE 105(3), 552–567 (2017)Google Scholar
  8. 8.
    Fan, J., Vercauteren, F.: Somewhat practical fully homomorphic encryption. IACR Cryptology ePrint Archive, p. 144 (2012)Google Scholar
  9. 9.
    Gentry, C.: A fully homomorphic encryption scheme. Ph.D. thesis, Stanford University (2009).
  10. 10.
    Gilad-Bachrach, R., Dowlin, N., Laine, K., Lauter, K.E., Naehrig, M., Wernsing, J.: Cryptonets: applying neural networks to encrypted data with high throughput and accuracy. In: Proceedings of the 33nd International Conference on Machine Learning. ICML 2016, pp. 201–210 (2016)Google Scholar
  11. 11.
    Graepel, T., Lauter, K., Naehrig, M.: ML confidential: machine learning on encrypted data. In: Kwon, T., Lee, M.-K., Kwon, D. (eds.) ICISC 2012. LNCS, vol. 7839, pp. 1–21. Springer, Heidelberg (2013). Scholar
  12. 12.
    Graves, A., Mohamed, A., Hinton, G.E.: Speech recognition with deep recurrent neural networks. In: IEEE International Conference on Acoustics, Speech and Signal Processing. ICASSP 2013, pp. 6645–6649 (2013)Google Scholar
  13. 13.
    Hastie, T., Tibshirani, R., Friedman, J.: The Elements of Statistical Learning. Springer Series in Statistics. Springer, New York (2001). Scholar
  14. 14.
    Huang, G.B., Ramesh, M., Berg, T., Learned-Miller, E.: Labeled faces in the wild: a database for studying face recognition in unconstrained environments. Technical report 07-49, University of Massachusetts, Amherst, October 2007Google Scholar
  15. 15.
    Ioffe, S., Szegedy, C.: Batch normalization: accelerating deep network training by reducing internal covariate shift. In: Proceedings of the 32nd International Conference on Machine Learning. ICML 2015, pp. 448–456 (2015)Google Scholar
  16. 16.
    Jia, Y., et al.: Caffe: convolutional architecture for fast feature embedding. arXiv preprint arXiv:1408.5093 (2014)
  17. 17.
    Juvekar, C., Vaikuntanathan, V., Chandrakasan, A.: Gazelle: a low latency framework for secure neural network inference. Cryptology ePrint Archive, Report 2018/073 (2018).
  18. 18.
    Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems 25: 26th Annual Conference on Neural Information Processing Systems 2012. Proceedings of a Meeting Held 3–6 December 2012, Lake Tahoe, Nevada, United States, pp. 1106–1114 (2012)Google Scholar
  19. 19.
    Learned-Miller, E., Huang, G.B., RoyChowdhury, A., Li, H., Hua, G.: Labeled faces in the wild: a survey. In: Kawulok, M., Celebi, M.E., Smolka, B. (eds.) Advances in Face Detection and Facial Image Analysis, pp. 189–248. Springer, Cham (2016). Scholar
  20. 20.
    LeCun, Y., Haffner, P., Bottou, L., Bengio, Y.: Object recognition with gradient-based learning. Shape, Contour and Grouping in Computer Vision. LNCS, vol. 1681, pp. 319–345. Springer, Heidelberg (1999). Scholar
  21. 21.
    LeCun, Y., Cortes, C.: MNIST handwritten digit database (2010).
  22. 22.
    Liu, J., Juuti, M., Lu, Y., Asokan, N.: Oblivious neural network predictions via MiniONN transformations. In: Thuraisingham, B.M., Evans, D., Malkin, T., Xu, D. (eds.) Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security. CCS 2017, pp. 619–631. ACM, New York (2017)Google Scholar
  23. 23.
    Mohassel, P., Zhang, Y.: SecureML: a system for scalable privacy-preserving machine learning. In: 2017 IEEE Symposium on Security and Privacy. SP 2017, pp. 19–38. IEEE Computer Society (2017)Google Scholar
  24. 24.
    Naor, M., Pinkas, B.: Oblivious transfer and polynomial evaluation. In: Proceedings of the Thirty-First Annual ACM Symposium on Theory of Computing, 1–4 May 1999, Atlanta, Georgia, USA, pp. 245–254 (1999)Google Scholar
  25. 25.
    Nielsen, M.A.: Neural Networks and Deep Learning. Determination Press (2015)Google Scholar
  26. 26.
    Nikolaenko, V., Weinsberg, U., Ioannidis, S., Joye, M., Boneh, D., Taft, N.: Privacy-preserving ridge regression on hundreds of millions of records. In: 2013 IEEE Symposium on Security and Privacy. SP 2013, 19–22 May 2013, Berkeley, CA, USA, pp. 334–348 (2013)Google Scholar
  27. 27.
    Orlandi, C., Piva, A., Barni, M.: Oblivious neural network computing via homomorphic encryption. EURASIP J. Inf. Secur. 2007, 037343 (2007)CrossRefGoogle Scholar
  28. 28.
    Riazi, M.S., Weinert, C., Tkachenko, O., Songhori, E.M., Schneider, T., Koushanfar, F.: Chameleon: a hybrid secure computation framework for machine learning applications. IACR Cryptology ePrint Archive, vol. 2017, p. 1164 (2017)Google Scholar
  29. 29.
    Schroff, F., Kalenichenko, D., Philbin, J.: FaceNet: a unified embedding for face recognition and clustering. In: IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2015, pp. 815–823 (2015)Google Scholar
  30. 30.
    Wu, D., Haven, J.: Using homomorphic encryption for large scale statistical analysis. Technical report, Stanford University (2012). Report.pdf
  31. 31.
    Xie, P., Bilenko, M., Finley, T., Gilad-Bachrach, R., Lauter, K.E., Naehrig, M.: Crypto-Nets: neural networks over encrypted data. CoRR, abs/1412.6181 (2014)Google Scholar
  32. 32.
    Yao, A.C.: Protocols for secure computations (extended abstract). In: 23rd Annual Symposium on Foundations of Computer Science, 3–5 November 1982, Chicago, Illinois, USA, pp. 160–164 (1982)Google Scholar
  33. 33.
    Yuan, J., Yu, S.: Privacy preserving back-propagation neural network learning made practical with cloud computing. IEEE Trans. Parallel Distrib. Syst. 25(1), 212–221 (2014)CrossRefGoogle Scholar
  34. 34.
    Zhang, Q., Yang, L.T., Chen, Z.: Privacy preserving deep computation model on cloud for big data feature learning. IEEE Trans. Comput. 65(5), 1351–1362 (2016)MathSciNetCrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Hervé Chabanne
    • 1
    • 2
    Email author
  • Roch Lescuyer
    • 1
  • Jonathan Milgram
    • 1
  • Constance Morel
    • 1
  • Emmanuel Prouff
    • 3
  1. 1.IdemiaParisFrance
  2. 2.Télécom ParistechParisFrance
  3. 3.ANSSIParisFrance

Personalised recommendations