Abstract
In this paper, we propose a novel approach for face spoofing detection using a combination of color texture descriptors with a new convolutional neural network (CNN) architecture. The proposed approach is based on a new convolutional neural network architecture composed of two CNN parallel branches. The first branch is fed with complementary shallow local phase quantization (LPQ) invariant descriptors that result from joint color texture information from the hue, saturation, and value (HSV) color space to accurately capture the reflection properties of the face. Combining the HSV color space with LPQ is known to significantly improve performance. The second branch of the CNN takes an RGB image directly as input, effectively separating chromatic (color-related) information from achromatic (brightness-related) information in order to extract crucial facial color features. Each branch of the CNN produces a vector of deep features that are extracted. To effectively concatenate the deep features from the two output branches, we employ an attention mechanism based combination method. This method captures the complementarity of the two branches, improving the accuracy and robustness of the model. The combined feature vectors form an input vector for the next Dense layer, where the model can distinguish between live and spoofed faces. Our method detects 2D facial spoofing attacks involving printed photos and replayed videos. We showcase the effectiveness and superior performance of our approach through a series of experiments conducted on both the CASIA-FASD and Replay-Attack datasets. Our results are promising and surpassing those of other state-of-the-art methods on both used datasets in terms of 9 performance metrics.
Similar content being viewed by others
Data Availability
We have not utilized any proprietary data and we have provided comprehensive references for the publicly accessible datasets discussed in our paper.
References
Anjos A, Marcel S (2011) Counter-measures to photo attacks in face recognition: a public database and a baseline. In: 2011 international joint conference on Biometrics (IJCB). IEEE, pp 1-7. https://doi.org/10.1109/IJCB.2011.6117503
Galbally J, Marcel S, Fierrez J (2014) Biometric antispoofing methods: A survey in face recognition. IEEE Access 2:1530–1552. https://doi.org/10.1109/ACCESS.2014.2381273
Hadid A, Evans N, Marcel S, Fierrez J (2015) Biometrics systems under spoofing attack: An evaluation methodology and lessons learned. IEEE Signal Process Mag 32(5):20–30. https://doi.org/10.1109/MSP.2015.2437652
Li Y, Xu K, Yan Q, Li Y, Deng RH (2014) Understanding OSN-based facial disclosure against face authentication systems. In: Proceedings of the 9th ACM symposium on Information, computer and communications security. pp 413–424. https://doi.org/10.1145/2590296.2590315
Zhang Z, Yan J, Liu S, Lei Z, Yi D, Li SZ (2012) A face antispoofing database with diverse attacks. In: 2012 5th IAPR international conference on Biometrics (ICB). IEEE, pp 26–31. https://doi.org/10.1109/ICB.2012.6199754
Cortes C, Vapnik V (1995) Support-vector networks. Mach Learn 20:273–297. https://doi.org/10.1007/BF00994018
Tharwat A, Gaber T, Ibrahim A, Hassanien AE (2017) Linear discriminant analysis: A detailed tutorial. AI Commun 30(2):169–190. https://doi.org/10.3233/AIC-170729
Määttä J, Hadid A, Pietikäinen M (2011) Face spoofing detection from single images using micro-texture analysis. IEEE International Joint Conference on Biometrics (IJCB). pp 1–7. https://doi.org/10.1109/IJCB.2011.6117510
Yang J, Lei Z, Liao S and Li SZ (2013) Face liveness detection with component dependent descriptor. International Conference on Biometrics (ICB). pp 1–6. https://doi.org/10.1109/ICB.2013.6612955
Dalal N, Triggs B (2005) Histograms of oriented gradients for human detection. IEEE computer society conference on computer vision and pattern recognition (CVPR’05), vol. 1. pp 886–893. https://doi.org/10.1109/CVPR.2005.177
Ojansivu V, Heikkilä J (2008) Blur insensitive texture classification using local phase quantization. In: International conference on image and signal processing. Springer, pp 236–243. https://doi.org/10.1007/978-3-540-69905-7_27
Lazebnik S, Schmid C, Ponce J (2006) Beyond bags of features: Spatial pyramid matching for recognizing natural scene categories. IEEE computer society conference on computer vision and pattern recognition (CVPR’06), vol. 2. pp 2169–2178. https://doi.org/10.1109/CVPR.2006.68
Chingovska I, Anjos A, Marcel S (2012) On the effectiveness of local binary patterns in face anti-spoofing. In: 2012 BIOSIG-proceedings of the international conference of biometrics special interest group (BIOSIG). IEEE, pp 1–7
Boulkenafet Z, Komulainen J, Hadid A (2015) Face anti-spoofing based on color texture analysis. IEEE international conference on image processing (ICIP). pp 2636–2640. https://doi.org/10.1109/ICIP.2015.7351280
Boulkenafet Z, Komulainen J, Hadid A (2016) Face spoofing detection using colour texture analysis. IEEE Trans Inf Forensics Secur 11(8):1818–1830. https://doi.org/10.1109/TIFS.2016.2555286
Boulkenafet Z, Komulainen J, Hadid A (2016) Face antispoofing using speeded-up robust features and fisher vector encoding. IEEE Signal Process Lett 24(2):141–145. https://doi.org/10.1109/LSP.2016.2630740
Wen D, Han H, Jain AK (2015) Face spoof detection with image distortion analysis. IEEE Trans Inf Forensics Secur 10(4):746–761. https://doi.org/10.1109/TIFS.2015.2400395
Singh AK, Joshi P, Nandi GC (2014) Face recognition with liveness detection using eye and mouth movement. IEEE international conference on signal propagation and computer technology (ICSPCT 2014). pp. 592–597. https://doi.org/10.1109/ICSPCT.2014.6884911
Jain A, Nandakumar K, Ross A (2005) Score normalization in multimodal biometric systems. Elsevier Pattern Recognit 38(12):2270–2285. https://doi.org/10.1016/j.patcog.2005.01.012
George A, Marcel S (2019) Deep pixel-wise binary supervision for face presentation attack detection. In 2019 International Conference on Biometrics (ICB). IEEE, pp 1–8. https://doi.org/10.1109/ICB45273.2019.8987370
Huang G, Liu Z, Van Der Maaten L, Weinberger KQ (2017) Densely connected convolutional networks. In Proceedings of the IEEE conference on computer vision and pattern recognition. pp 4700–4708. https://doi.org/10.48550/arXiv.1608.06993
Abdullakutty F, Johnston P, Elyan E (2022) Fusion Methods for Face Presentation Attack Detection. Sensors 22(14):5196. https://doi.org/10.3390/s22145196
Abdullakutty F, Elyan E, Johnston P, Ali-Gombe A (2022) Deep transfer learning on the aggregated dataset for face presentation attack detection. Cogn Comput 14(6):2223–2233. https://doi.org/10.1007/s12559-022-10037-z
Satapathy A, Livingston LM, Jenila (2021) A lite convolutional neural network built on permuted Xceptio-inception and Xceptio-reduction modules for texture based facial liveness recognition. Multimed Tools Appl 80:10441–10472. https://doi.org/10.1007/s11042-020-10181-4
Gwyn T, Roy K (2022) Examining gender bias of convolutional neural networks via facial recognition. Fut Intern 14(12):375. https://doi.org/10.3390/fi14120375
Wang D, Ma G, Liu X (2022) An intelligent recognition framework of access control system with anti-spoofing function. AIMS Math 7(6):10495–10512. https://doi.org/10.3934/math.2022585
Li L, Feng X, Boulkenafet Z, Xia Z, Li M, Hadid A (2016) An original face anti-spoofing approach using partial convolutional neural network. In 2016 Sixth International Conference on Image Processing Theory, Tools and Applications (IPTA). IEEE, pp 1–6. https://doi.org/10.1109/IPTA.2016.7821013
Yang X, Luo W, Bao L, Gao Y, Gong D, Zheng S, Li Z, Liu W (2019) Face anti-spoofing: Model matters, so does data. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp 3507–3516. https://doi.org/10.1109/CVPR.2019.00362
Deb D, Jain AK (2020) Look locally infer globally: A generalizable face anti-spoofing approach. IEEE Trans Inf Forensics Secur 16:1143–1157. https://doi.org/10.1109/TIFS.2020.3029879
Shao R, Lan X, Li J, Yuen PC (2019) Multi-adversarial discriminative deep domain generalization for face presentation attack detection. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp 10023–10031. https://doi.org/10.1109/CVPR.2019.01026
de Souza GB, Papa JP, Marana AN (2018) On the learning of deep local features for robust face spoofing detection. In: 2018 31st SIBGRAPI Conference on Graphics, Patterns and Images (SIBGRAPI). IEEE, pp 258–265. https://doi.org/10.1109/SIBGRAPI.2018.00040
Sun CY, Chen SL, Li XJ, Chen F, Yin XC (2022) Danet: Dynamic attention to spoof patterns for face anti-spoofing. In: 2022 26th International Conference on Pattern Recognition (ICPR). IEEE, pp 1929–1936. https://doi.org/10.1109/ICPR56361.2022.9956725
Kong Y, Li X, Hao G, Liu C (2022) Face Anti-Spoofing Method Based on Residual Network with Channel Attention Mechanism. J Electron 11(19):3056. https://doi.org/10.3390/electronics11193056
Liu Y, Jourabloo A, Liu X (2018) Learning deep models for face anti-spoofing: Binary or auxiliary supervision. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp 389–398. https://doi.org/10.48550/arXiv.1803.11097
da Silva VL, Lérida JL, Sarret M, Valls M, Giné F (2023) Residual spatiotemporal convolutional networks for face anti-spoofing. Journal of Visual Communication and Image Representation. Elsevier, page 103744. https://doi.org/10.1016/j.jvcir.2022.103744
Xu Z, Li S, Deng W (2015) Learning temporal features using LSTM-CNN architecture for face anti-spoofing. In 2015 3rd IAPR asian conference on pattern recognition (ACPR). IEEE, pp 141–145. https://doi.org/10.1109/ACPR.2015.7486482
Guo J, Zhu X, Xiao J, Lei Z, Wan G, Li SZ (2019) Improving face anti-spoofing by 3d virtual synthesis. In: 2019 International Conference on Biometrics (ICB). IEEE, pp 1–8. https://doi.org/10.1109/ICB45273.2019.8987415
Hashemifard S, Akbari M (2021) A compact deep learning model for face spoofing detection. arXiv:2101.04756, https://doi.org/10.48550/arXiv.2101.04756
Khammari M (2019) Robust face anti-spoofing using CNN with LBP and WLD. IET Image Proc 13(11):1880–1884. https://doi.org/10.1049/iet-ipr.2018.5560
Patel K, Han H, Jain AK (2016) Cross-database face antispoofing with robust feature representation. Biometric Recognition: 11th Chinese Conference, CCBR 2016, Chengdu, China, October 14-16, 2016, Proceedings 11. Springer, pp 611–619. https://doi.org/10.1007/978-3-319-46654-5_67
Atoum Y, Liu Y, Jourabloo A, Liu X (2017) Face anti-spoofing using patch and depth-based CNNs. In 2017 IEEE International Joint Conference on Biometrics (IJCB). IEEE, pp 319–328. https://doi.org/10.1109/BTAS.2017.8272713
Chen H, Hu G, Lei Z, Chen Y, Robertson NM, Li SZ (2019) Attention-based two-stream convolutional networks for face spoofing detection. IEEE Trans Inf Forensics Secur 15:578–593. https://doi.org/10.1109/TIFS.2019.2922241
Wang Y, Nian F, Li T, Meng Z, Wang K (2017) Robust face anti-spoofing with depth information. J Vis Commun Image Represent 49:332–337. https://doi.org/10.1016/j.jvcir.2017.09.002
Asim M, Ming Z, Javed MY (2017) CNN based spatio-temporal feature extraction for face anti-spoofing. In: 2017 2nd International Conference on Image, Vision and Computing (ICIVC). IEEE, pp 234-238. https://doi.org/10.1109/ICIVC.2017.7984552
Antil A, Dhiman C (2023) A two stream face anti-spoofing framework using multi-level deep features and ELBP features. Multimedia Systems. Springer, pp 1–16. https://doi.org/10.1007/s00530-023-01060-7
Feng L, Po LM, Li Y, Xu X, Yuan F, Cheung TCH, Cheung KW (2016) Integration of image quality and motion cues for face anti-spoofing: A neural network approach. J Vis Commun Image Represent 38:451–460. https://doi.org/10.1016/j.jvcir.2016.03.019
Shu X, Li X, Zuo X, Xu D, Shi J (2023) Face spoofing detection based on multi-scale color inversion dual-stream convolutional neural network. Expert Syst Appl 224:119988. https://doi.org/10.1016/j.eswa.2023.119988
Zhang K, Zhang Z, Li Z, Qiao Y (2016) Joint face detection and alignment using multitask cascaded convolutional networks. IEEE Signal Process Lett 23(10):1499–1503. https://doi.org/10.1109/LSP.2016.2603342
Bargshady G, Zhou X, Deo RC, Soar J, Whittaker F, Wang H (2020) The modeling of human facial pain intensity based on temporal convolutional networks trained with video frames in HSV color space. Appl Soft Comput 97:106805. https://doi.org/10.1016/j.asoc.2020.106805
Rahman MA, Purnama IKE, Purnomo MH (2014) Simple method of human skin detection using HSV and YCbCr color spaces. In: 2014 international conference on intelligent autonomous agents, networks and systems. IEEE, pp 58–61. https://doi.org/10.1109/INAGENTSYS.2014.7005726
Xiao Y, Cao Z, Wang L, Li T (2017) Local phase quantization plus: A principled method for embedding local phase quantization into fisher vector for blurred image recognition. Inf Sci 420:77–95. https://doi.org/10.1016/j.ins.2017.08.059
Ramachandran P, Zoph B, Le QV (2017) Searching for activation functions. Technical report, 7(1):5. arXiv:1710.05941, https://doi.org/10.48550/arXiv.1710.05941
Nair V, Hinton GE (2010) Rectified linear units improve restricted boltzmann machines. In: Proceedings of the 27th international conference on machine learning (ICML-10). pp 807–814
Iliev A, Kyurkchiev N, Markov S (2017) On the approximation of the step function by some sigmoid functions. Math Comput Simul 133:223–234. https://doi.org/10.1016/j.matcom.2015.11.005
Chollet F (2017) Xception: Deep learning with depthwise separable convolutions. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp 1251–1258. https://doi.org/10.48550/arXiv.1610.02357
Lanjewar MG, Morajkar P, P P, (2023) Modified transfer learning frameworks to identify potato leaf diseases. Multimed Tools Appl 1–23. https://doi.org/10.1007/s11042-023-17610-0
Lanjewar MG, Gurav OL (2022) Convolutional Neural Networks based classifications of soil images. Multimed Tools Appl 81(7):10313–10336. https://doi.org/10.1007/s11042-022-12200-y
Abadi M, Agarwal A, Barham P, Brevdo E, Chen Z, Citro C, Corrado GS, Davis A, Dean J, Devin M, et al. (2016) Tensorflow: Large-scale machine learning on heterogeneous distributed systems. arXiv:1603.04467, https://doi.org/10.48550/arXiv.1603.04467
Ho Y, Wookey S (2019) The real-world-weight cross-entropy loss function: Modeling the costs of mislabeling. IEEE Access 8:4806–4813. https://doi.org/10.1109/ACCESS.2019.2962617
Kingma DP, Ba J (2014) Adam: A method for stochastic optimization. arXiv:1412.6980, https://doi.org/10.48550/arXiv.1412.6980
Acknowledgements
This work received support from the General Directorate for Scientific Research and Technological Development within the Ministry of Higher Education and Scientific Research (DGRSDT) in Algeria.
Funding
This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflicts of interest
The authors declare that they have no conflict of interest.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Madi, M., Khammari, M. & Larabi, MC. CNN-LPQ: convolutional neural network combined to local phase quantization based approach for face anti-spoofing. Multimed Tools Appl (2024). https://doi.org/10.1007/s11042-024-18880-y
Received:
Revised:
Accepted:
Published:
DOI: https://doi.org/10.1007/s11042-024-18880-y