Advertisement

Face Anti-Spoofing with Human Material Perception

Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 12352)

Abstract

Face anti-spoofing (FAS) plays a vital role in securing the face recognition systems from presentation attacks. Most existing FAS methods capture various cues (e.g., texture, depth and reflection) to distinguish the live faces from the spoofing faces. All these cues are based on the discrepancy among physical materials (e.g., skin, glass, paper and silicone). In this paper we rephrase face anti-spoofing as a material recognition problem and combine it with classical human material perception, intending to extract discriminative and robust features for FAS. To this end, we propose the Bilateral Convolutional Networks (BCN), which is able to capture intrinsic material-based patterns via aggregating multi-level bilateral macro- and micro- information. Furthermore, Multi-level Feature Refinement Module (MFRM) and multi-head supervision are utilized to learn more robust features. Comprehensive experiments are performed on six benchmark datasets, and the proposed method achieves superior performance on both intra- and cross-dataset testings. One highlight is that we achieve overall 11.3 ± 9.5% EER for cross-type testing in SiW-M dataset, which significantly outperforms previous results. We hope this work will facilitate future cooperation between FAS and material communities.

Keywords

Face anti-spoofing Material perception Bilateral filtering 

Notes

Acknowledgment

This work was supported by the Academy of Finland for project MiGA (grant 316765), ICT 2023 project (grant 328115), and Infotech Oulu. We also acknowledge CSC-IT Center for Science, Finland, for computational resources.

Supplementary material

504444_1_En_33_MOESM1_ESM.pdf (450 kb)
Supplementary material 1 (pdf 450 KB)

References

  1. 1.
    Guo, J., Zhu, X., Zhao, C., Cao, D., Lei, Z., Li, S.Z.: Learning meta face recognition in unseen domains. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 6163–6172 (2020)Google Scholar
  2. 2.
    Guo, J., Zhu, X., Lei, Z., Li, S.Z.: Face synthesis for eyeglass-robust face recognition. In: Zhou, J., et al. (eds.) CCBR 2018. LNCS, vol. 10996, pp. 275–284. Springer, Cham (2018).  https://doi.org/10.1007/978-3-319-97909-0_30CrossRefGoogle Scholar
  3. 3.
    Cao, D., Zhu, X., Huang, X., Guo, J., Lei, Z.: Domain balancing: face recognition on long-tailed domains. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5671–5679 (2020)Google Scholar
  4. 4.
    Boulkenafet, Z., Komulainen, J., Hadid, A.: Face anti-spoofing based on color texture analysis. In: IEEE International Conference on Image Processing (ICIP), pp. 2636–2640 (2015)Google Scholar
  5. 5.
    Boulkenafet, Z., Komulainen, J., Hadid, A.: Face spoofing detection using colour texture analysis. IEEE Trans. Inf. Forensics Secur. 11(8), 1818–1830 (2016)CrossRefGoogle Scholar
  6. 6.
    de Freitas Pereira, T., Anjos, A., De Martino, J.M., Marcel, S.: LBP-top based countermeasure against face spoofing attacks. In: Asian Conference on Computer Vision, pp. 121–132 (2012)Google Scholar
  7. 7.
    Komulainen, J., Hadid, A., Pietikainen, M.: Context based face anti-spoofing. In: 2013 IEEE Sixth International Conference on Biometrics: Theory, Applications and Systems (BTAS), pp. 1–8 (2013)Google Scholar
  8. 8.
    Peixoto, B., Michelassi, C., Rocha, A.: Face liveness detection under bad illumination conditions. In: ICIP, pp. 3557–3560. IEEE (2011)Google Scholar
  9. 9.
    Patel, K., Han, H., Jain, A.K.: Secure face unlock: spoof detection on smartphones. IEEE Trans. Inf. Forensics Secur. 11(10), 2268–2283 (2016)CrossRefGoogle Scholar
  10. 10.
    Qin, Y., et al.: Learning meta model for zero-and few-shot face anti-spoofing. In: The Thirty-Fourth AAAI Conference on Artificial Intelligence (AAAI) (2020)Google Scholar
  11. 11.
    Wang, Z., et al.: Deep spatial gradient and temporal depth learning for face anti-spoofing. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5042–5051 (2020)Google Scholar
  12. 12.
    Yu, Z., et al.: Searching central difference convolutional networks for face anti-spoofing. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5295–5305 (2020)Google Scholar
  13. 13.
    Jourabloo, A., Liu, Y., Liu, X.: Face de-spoofing: anti-spoofing via noise modeling. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 290–306 (2018)Google Scholar
  14. 14.
    Lin, B., Li, X., Yu, Z., Zhao, G.: Face liveness detection by RPPG features and contextual patch-based CNN. In: Proceedings of the 2019 3rd International Conference on Biometric Engineering and Applications, pp. 61–68. ACM (2019)Google Scholar
  15. 15.
    Li, X., Komulainen, J., Zhao, G., Yuen, P.C., Pietikäinen, M.: Generalized face anti-spoofing by detecting pulse from face videos. In: 2016 23rd International Conference on Pattern Recognition (ICPR), pp. 4244–4249. IEEE (2016)Google Scholar
  16. 16.
    Liu, Y., Jourabloo, A., Liu, X.: Learning deep models for face anti-spoofing: binary or auxiliary supervision. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 389–398 (2018)Google Scholar
  17. 17.
    Liu, S.Q., Lan, X., Yuen, P.C.: Remote photoplethysmography correspondence feature for 3D mask face presentation attack detection. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 558–573 (2018)Google Scholar
  18. 18.
    Atoum, Y., Liu, Y., Jourabloo, A., Liu, X.: Face anti-spoofing using patch and depth-based CNNs. In: 2017 IEEE International Joint Conference on Biometrics (IJCB), pp. 319–328 (2017)Google Scholar
  19. 19.
    Guo, J., Zhu, X., Xiao, J., Lei, Z., Wan, G., Li, S.Z.: Improving face anti-spoofing by 3D virtual synthesis. In: 2019 International Conference on Biometrics (ICB), pp. 1–8. IEEE (2019)Google Scholar
  20. 20.
    Tan, X., Li, Y., Liu, J., Jiang, L.: Face liveness detection from a single image with sparse low rank bilinear discriminative model. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010. LNCS, vol. 6316, pp. 504–517. Springer, Heidelberg (2010).  https://doi.org/10.1007/978-3-642-15567-3_37CrossRefGoogle Scholar
  21. 21.
    Li, L., Xia, Z., Jiang, X., Ma, Y., Roli, F., Feng, X.: 3D face mask presentation attack detection based on intrinsic image analysis. arXiv preprint arXiv:1903.11303 (2019)
  22. 22.
    Sharan, L., Liu, C., Rosenholtz, R., Adelson, E.H.: Recognizing materials using perceptually inspired features. Int. J. Comput. Vision 103(3), 348–371 (2013)MathSciNetCrossRefGoogle Scholar
  23. 23.
    Boulkenafet, Z., Komulainen, J., Hadid, A.: Face antispoofing using speeded-up robust features and fisher vector encoding. IEEE Signal Process. Lett. 24(2), 141–145 (2017)Google Scholar
  24. 24.
    Komulainen, J., Hadid, A., Pietikäinen, M.: Face spoofing detection using dynamic texture. In: Park, J.-I., Kim, J. (eds.) ACCV 2012. LNCS, vol. 7728, pp. 146–157. Springer, Heidelberg (2013).  https://doi.org/10.1007/978-3-642-37410-4_13CrossRefGoogle Scholar
  25. 25.
    Siddiqui, T.A., et al.: Face anti-spoofing with multifeature videolet aggregation. In: 2016 23rd International Conference on Pattern Recognition (ICPR), pp. 1035–1040. IEEE (2016)Google Scholar
  26. 26.
    Pan, G., Sun, L., Wu, Z., Lao, S.: Eyeblink-based anti-spoofing in face recognition from a generic webcamera. In: IEEE International Conference on Computer Vision, pp. 1–8 (2007)Google Scholar
  27. 27.
    Yu, Z., et al.: Auto-fas: searching lightweight networks for face anti-spoofing. In: ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 996–1000. IEEE (2020)Google Scholar
  28. 28.
    Li, L., Feng, X., Boulkenafet, Z., Xia, Z., Li, M., Hadid, A.: An original face anti-spoofing approach using partial convolutional neural network. In: IPTA, pp. 1–6 (2016)Google Scholar
  29. 29.
    Patel, K., Han, H., Jain, A.K.: Cross-database face antispoofing with robust feature representation. In: Chinese Conference on Biometric Recognition, pp. 611–619 (2016)Google Scholar
  30. 30.
    George, A., Marcel, S.: Deep pixel-wise binary supervision for face presentation attack detection. In: International Conference on Biometrics. Number CONF (2019)Google Scholar
  31. 31.
    Yu, Z., et al.: Multi-modal face anti-spoofing based on central difference networks. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 650–651 (2020)Google Scholar
  32. 32.
    Kim, T., Kim, Y., Kim, I., Kim, D.: BASN: enriching feature representation using bipartite auxiliary supervisions for face anti-spoofing. In: Proceedings of the IEEE International Conference on Computer Vision Workshops (2019)Google Scholar
  33. 33.
    Liu, Y., Stehouwer, J., Jourabloo, A., Liu, X.: Deep tree learning for zero-shot face anti-spoofing. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4680–4689 (2019)Google Scholar
  34. 34.
    Jia, Y., Zhang, J., Shan, S., Chen, X.: Single-side domain generalization for face anti-spoofing. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8484–8493 (2020)Google Scholar
  35. 35.
    Wang, G., Han, H., Shan, S., Chen, X.: Cross-domain face presentation attack detection via multi-domain disentangled representation learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 6678–6687 (2020)Google Scholar
  36. 36.
    Shao, R., Lan, X., Li, J., Yuen, P.C.: Multi-adversarial discriminative deep domain generalization for face presentation attack detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 10023–10031 (2019)Google Scholar
  37. 37.
    Wang, Z., Zhao, C., Qin, Y., Zhou, Q., Lei, Z.: Exploiting temporal and depth information for multi-frame face anti-spoofing. arXiv preprint arXiv:1811.05118 (2018)
  38. 38.
    Yang, X., et al.: Face anti-spoofing: Model matters, so does data. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2019)Google Scholar
  39. 39.
    Lin, C., Liao, Z., Zhou, P., Hu, J., Ni, B.: Live face verification with multiple instantialized local homographic parameterization. In: IJCAI, pp. 814–820 (2018)Google Scholar
  40. 40.
    Yu, Z., Li, X., Zhao, G.: Remote photoplethysmograph signal measurement from facial videos using spatio-temporal networks. arXiv preprint arXiv:1905.02419 (2019)
  41. 41.
    Yu, Z., Li, X., Niu, X., Shi, J., Zhao, G.: Autohr: a strong end-to-end baseline for remote heart rate measurement with neural searching. arXiv preprint arXiv:2004.12292 (2020)
  42. 42.
    Yu, Z., Peng, W., Li, X., Hong, X., Zhao, G.: Remote heart rate measurement from highly compressed facial videos: an end-to-end deep learning solution with video enhancement. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 151–160 (2019)Google Scholar
  43. 43.
    Maloney, L.T., Brainard, D.H.: Color and material perception: achievements and challenges. J. Vis. 10(9), 19 (2010)CrossRefGoogle Scholar
  44. 44.
    Fleming, R.W.: Visual perception of materials and their properties. Vis. Res. 94, 62–75 (2014)CrossRefGoogle Scholar
  45. 45.
    Nishida, S.: Image statistics for material perception. Curr. Opinion Behav. Sci. 30, 94–99 (2019)CrossRefGoogle Scholar
  46. 46.
    Adelson, E.H.: On seeing stuff: the perception of materials by humans and machines. In: Human vision and electronic imaging VI, vol. 4299, pp. 1–12. International Society for Optics and Photonics (2001)Google Scholar
  47. 47.
    Varma, M., Zisserman, A.: A statistical approach to material classification using image patch exemplars. IEEE Trans. Pattern Anal. Mach. Intell. 31(11), 2032–2047 (2008)CrossRefGoogle Scholar
  48. 48.
    Jiang, X., Du, J., Sun, B., Feng, X.: Deep dilated convolutional network for material recognition. In: 2018 Eighth International Conference on Image Processing Theory, Tools and Applications (IPTA), pp. 1–6. IEEE (2018)Google Scholar
  49. 49.
    Ling, S., Callet, P.L., Yu, Z.: The role of structure and textural information in image utility and quality assessment tasks. Electron. Imaging 2018(14), 1–13 (2018)CrossRefGoogle Scholar
  50. 50.
    Deng, B.W., Yu, Z.T., Ling, B.W., Yang, Z.: Video quality assessment based on features for semantic task and human material perception. In: 2016 IEEE International Conference on Consumer Electronics-China (ICCE-China), pp. 1–4. IEEE (2016)Google Scholar
  51. 51.
    Li, L., Xia, Z., Jiang, X., Ma, Y., Roli, F., Feng, X.: 3D face mask presentation attack detection based on intrinsic image analysis. IET Biometrics 9(3), 100–108 (2020)CrossRefGoogle Scholar
  52. 52.
    Tomasi, C., Manduchi, R.: Bilateral filtering for gray and color images. In: ICCV, vol. 98, p. 2 (1998)Google Scholar
  53. 53.
    Durand, F., Dorsey, J.: Fast bilateral filtering for the display of high-dynamic-range images. In: ACM Transactions on Graphics (TOG), vol. 21, pp. 257–266. ACM (2002)Google Scholar
  54. 54.
    Paris, S., Durand, F.: A fast approximation of the bilateral filter using a signal processing approach. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006. LNCS, vol. 3954, pp. 568–580. Springer, Heidelberg (2006).  https://doi.org/10.1007/11744085_44CrossRefGoogle Scholar
  55. 55.
    He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)Google Scholar
  56. 56.
    Wang, J., Chen, K., Xu, R., Liu, Z., Loy, C.C., Lin, D.: Carafe: content-aware reassembly of features. arXiv preprint arXiv:1905.02188 (2019)
  57. 57.
    Woo, S., Park, J., Lee, J.Y., So Kweon, I.: Cbam: convolutional block attention module. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 3–19 (2018)Google Scholar
  58. 58.
    Hu, J., Shen, L., Sun, G.: Squeeze-and-excitation networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7132–7141 (2018)Google Scholar
  59. 59.
    Wang, X., Girshick, R., Gupta, A., He, K.: Non-local neural networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7794–7803 (2018)Google Scholar
  60. 60.
    Guo, J., Zhu, X., Yang, Y., Yang, F., Lei, Z., Li, S.Z.: Towards fast, accurate and stable 3D dense face alignment. In: Proceedings of the European Conference on Computer Vision (ECCV) (2020)Google Scholar
  61. 61.
    Feng, Y., Wu, F., Shao, X., Wang, Y., Zhou, X.: Joint 3D face reconstruction and dense alignment with position map regression network. In: Proceedings of the European Conference on Computer Vision (ECCV) (2017)Google Scholar
  62. 62.
    Zhang, X., Ng, R., Chen, Q.: Single image reflection separation with perceptual losses. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4786–4794 (2018)Google Scholar
  63. 63.
    Boulkenafet, Z., Komulainen, J., Li, L., Feng, X., Hadid, A.: OULU-NPU: a mobile face presentation attack database with real-world variations. In: FGR, pp. 612–618 (2017)Google Scholar
  64. 64.
    Zhang, Z., Yan, J., Liu, S., Lei, Z., Yi, D., Li, S.Z.: A face antispoofing database with diverse attacks. In: ICB, pp. 26–31 (2012)Google Scholar
  65. 65.
    Chingovska, I., Anjos, A., Marcel, S.: On the effectiveness of local binary patterns in face anti-spoofing. In: Biometrics Special Interest Group, pp. 1–7 (2012)Google Scholar
  66. 66.
    Wen, D., Han, H., Jain, A.K.: Face spoof detection with image distortion analysis. IEEE Trans. Inf. Forensics Secur. 10(4), 746–761 (2015)CrossRefGoogle Scholar
  67. 67.
    International Organization for Standardization: ISO/IEC JTC 1/SC 37 biometrics: information technology biometric presentation attack detection part 1: Framework (2016). https://www.iso.org/obp/ui/iso
  68. 68.
    Boulkenafet, Z., et al.: A competition on generalized software-based face presentation attack detection in mobile scenarios. In: 2017 IEEE International Joint Conference on Biometrics (IJCB), pp. 688–696. IEEE (2017)Google Scholar
  69. 69.
    Arashloo, S.R., Kittler, J., Christmas, W.: An anomaly detection approach to face spoofing detection: a new formulation and evaluation protocol. IEEE Access 5, 13868–13882 (2017)CrossRefGoogle Scholar
  70. 70.
    Xiong, F., AbdAlmageed, W.: Unknown presentation attack detection with face RGB images. In: 2018 IEEE 9th International Conference on Biometrics Theory, Applications and Systems (BTAS), pp. 1–9. IEEE (2018)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  1. 1.Center for Machine Vision and Signal AnalysisUniversity of OuluOuluFinland
  2. 2.Key Laboratory of Intelligent Information Processing of Chinese Academy of Sciences (CAS), Institute of Computing Technology, CASBeijingChina
  3. 3.University of Chinese Academy of SciencesBeijingChina
  4. 4.School of Software EngineeringXi’an Jiaotong UniversityXi’anChina

Personalised recommendations