Detection of Face Morphing Attacks by Deep Learning

  • Clemens SeiboldEmail author
  • Wojciech Samek
  • Anna Hilsmann
  • Peter Eisert
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10431)


Identification by biometric features has become more popular in the last decade. High quality video and fingerprint sensors have become less expensive and are nowadays standard components in many mobile devices. Thus, many devices can be unlocked via fingerprint or face verification. The state of the art accuracy of biometric facial recognition systems prompted even systems that need high security standards like border control at airports to rely on biometric systems. While most biometric facial recognition systems perform quite accurate under a controlled environment, they can easily be tricked by morphing attacks. The concept of a morphing attack is to create a synthetic face image that contains characteristics of two different individuals and to use this image on a document or as reference image in a database. Using this image for authentication, a biometric facial recognition system accepts both individuals. In this paper, we propose a morphing attack detection approach based on convolutional neural networks. We present an automatic morphing pipeline to generate morphing attacks, train neural networks based on this data and analyze their accuracy. The accuracy of different well-known network architectures are compared and the advantage of using pretrained networks compared to networks learned from scratch is studied.


Automatic face morphing Face image forgery detection Convolutional neural networks Morphing attack 



The work in this paper has been funded in part by the German Federal Ministry of Education and Research (BMBF) through the Research Program ANANAS under Contract No. FKZ: 16KIS0511.


  1. 1.
    Spreeuwers, L.J., Hendrikse, A.J., Gerritsen, K.J.: Evaluation of automatic face recognition for automatic border control on actual data recorded of travellers at schiphol airport. In: Proceedings of the International Conference of Biometrics Special Interest Group (BIOSIG), 1–6 September 2012Google Scholar
  2. 2.
    Ferrara, M., Franco, A., Maltoni, D.: The magic passport. In: IEEE International Joint Conference on Biometrics, 1–7 September 2014Google Scholar
  3. 3.
    Popescu, A.C., Farid, H.: Exposing digital forgeries by detecting traces of resampling. IEEE Trans. Sig. Process. 53(2), 758–767 (2005)MathSciNetCrossRefGoogle Scholar
  4. 4.
    Kirchner, M., Gloe, T.: On resampling detection in re-compressed images. In: 2009 First IEEE International Workshop on Information Forensics and Security (WIFS), 21–25 December 2009Google Scholar
  5. 5.
    Kirchner, M., Fridrich, J.: On detection of median filtering in digital images. Proc. SPIE 7541, 754110–754110–12 (2010)Google Scholar
  6. 6.
    Lukáš, J., Fridrich, J.: Estimation of primary quantization matrix in double compressed JPEG images. In: Proceedings of DFRWS (2003)Google Scholar
  7. 7.
    Farid, H.: Exposing digital forgeries from JPEG ghosts. IEEE Trans. Inf. Forensics Secur. 4(1), 154–160 (2009)MathSciNetCrossRefGoogle Scholar
  8. 8.
    Johnson, M.K., Farid, H.: Exposing digital forgeries through specular highlights on the eye. In: Furon, T., Cayre, F., Doërr, G., Bas, P. (eds.) IH 2007. LNCS, vol. 4567, pp. 311–325. Springer, Heidelberg (2007). doi: 10.1007/978-3-540-77370-2_21 CrossRefGoogle Scholar
  9. 9.
    Kee, E., O’brien, J.F., Farid, H.: Exposing photo manipulation from shading and shadows. ACM Trans. Graph. 33(5), 165:1–165:21 (2014)CrossRefGoogle Scholar
  10. 10.
    Makrushin, A., Neubert, T., Dittmann, J.: Automatic generation and detection of visually faultless facial morphs. In: Proceedings of the 12th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2017), pp. 39–50 (2017)Google Scholar
  11. 11.
    Raghavendra, R., Raja, K.B., Busch, C.: Detecting morphed face images. In: 8th IEEE International Conference on Biometrics Theory, Applications and Systems, BTAS 2016, Niagara Falls, NY, USA, pp. 1–7, 6–9 September 2016Google Scholar
  12. 12.
    Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems 25. Curran Associates, Inc., pp. 1097–1105 (2012)Google Scholar
  13. 13.
    Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. CoRR abs/1409.1556 (2014)Google Scholar
  14. 14.
    Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., Rabinovich, A.: Going deeper with convolutions. In: Computer Vision and Pattern Recognition (CVPR) (2015)Google Scholar
  15. 15.
    Bayar, B., Stamm, M.C.: A deep learning approach to universal image manipulation detection using a new convolutional layer. In: Proceedings of the 4th ACM Workshop on Information Hiding and Multimedia Security, New York, NY, USA, pp. 5–10. ACM (2016)Google Scholar
  16. 16.
    King, D.E.: Dlib-ml: a machine learning toolkit. J. Mach. Learn. Res. 10, 1755–1758 (2009)Google Scholar
  17. 17.
    Kazemi, V., Sullivan, J.: One millisecond face alignment with an ensemble of regression trees. In: Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2014, Computer Society, pp. 1867–1874. IEEE, Washington, DC (2014)Google Scholar
  18. 18.
    Beier, T., Neely, S.: Feature-based image metamorphosis. In: Proceedings of the 19th Annual Conference on Computer Graphics and Interactive Techniques, SIGGRAPH 1992, pp. 35–42. ACM, New York (1992)Google Scholar
  19. 19.
    Delaunay, B.: Sur la sphere vide. Izv. Akad. Nauk SSSR, Otdelenie Matematicheskii i Estestvennyka Nauk 7, 793–800 (1934)Google Scholar
  20. 20.
    Pérez, P., Gangnet, M., Blake, A.: Poisson image editing. In: ACM SIGGRAPH 2003 Papers, SIGGRAPH 2003, pp. 313–318. ACM, New York (2003)Google Scholar
  21. 21.
    Prestele, B., Schneider, D.C., Eisert, P.: System for the automated segmentation of heads from arbitrary background. In: ICIP, pp. 3257–3260. IEEE (2011)Google Scholar
  22. 22.
    Paier, W., Kettern, M., Hilsmann, A., Eisert, P.: Video-based facial re-animation. In: Proceedings of the 12th European Conference on Visual Media Production, London, United Kingdom, pp. 4:1–4:10, 24–25 November 2015Google Scholar
  23. 23.
    Neurotechnology Inc.: Verilook 9.0/megamatcher 9.0 faces identification technology (2017).
  24. 24.
    FRONTEX - Research and Development Unit: Best practice technical guidelines for automated border control (abc) systems - v2.0 (2012)Google Scholar
  25. 25.
    Jia, Y., Shelhamer, E., Donahue, J., Karayev, S., Long, J., Girshick, R., Guadarrama, S., Darrell, T.: Caffe: convolutional architecture for fast feature embedding. arXiv preprint (2014). arXiv:1408.5093
  26. 26.
    Glorot, X., Bengio, Y.: Understanding the difficulty of training deep feedforward neural networks. In: Proceedings of the International Conference on Artificial Intelligence and Statistics (AISTATS10). Society for Artificial Intelligence and Statistics (2010)Google Scholar
  27. 27.
    Stamm, M.C., Liu, K.J.R.: Anti-forensics of digital image compression. IEEE Trans. Inf. Forensics Secur. 6(3), 1050–1065 (2011)CrossRefGoogle Scholar
  28. 28.
    Yu, J., Zhan, Y., Yang, J., Kang, X.: A multi-purpose image counter-anti-forensic method using convolutional neural networks. In: Shi, Y.Q., Kim, H.J., Perez-Gonzalez, F., Liu, F. (eds.) IWDW 2016. LNCS, vol. 10082, pp. 3–15. Springer, Cham (2017). doi: 10.1007/978-3-319-53465-7_1 CrossRefGoogle Scholar
  29. 29.
    Kirchner, M., Bohme, R.: Hiding traces of resampling in digital images. IEEE Trans. Inf. Forensics Secur. 3(4), 582–592 (2008)CrossRefGoogle Scholar
  30. 30.
    Lapuschkin, S., Binder, A., Montavon, G., Müller, K.R., Samek, W.: Analyzing classifiers: fisher vectors and deep neural networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2912–2920 (2016)Google Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Clemens Seibold
    • 1
    Email author
  • Wojciech Samek
    • 1
  • Anna Hilsmann
    • 1
  • Peter Eisert
    • 1
    • 2
  1. 1.Fraunhofer HHIBerlinGermany
  2. 2.Humboldt University BerlinBerlinGermany

Personalised recommendations