Skip to main content
SpringerLink
Log in

Facial depth forgery detection based on image gradient

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

With the widespread application of deep learning, many artificially generated fake images and videos appear on the Internet. However, it is difficult for people to distinguish the real from the fake ones, making the research on detecting and recognizing fake images or videos receive significant attention. Since new forgery techniques can reduce the effectiveness of specific detection methods or even make them ineffective, research on detecting facial depth forgery needs to be continuously developed. To defend against the onslaught of new facial depth forgery methods, we proposed an image gradient-based approach to transform the facial depth forgery detection problem into the recognition and analysis of video frames. Specifically, there are two key components in this approach: (1) we capture images from videos and crop the face section, which dramatically reduces the amount of data; (2) we use the image gradient operator to process the face image that extracts image features for detection and recognition. After these, we have conducted extensive experiments on different facial depth forgery datasets. Experimental results demonstrated that using our image gradient approach could effectively detect facial depth forgery and achieve excellent detection and identification performance.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Algorithm 1
Algorithm 2
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

Notes

  1. https://www.electionguard.vote/

  2. https://www.kaggle.com/c/deepfake-detection-challenge/

  3. https://www.kaggle.com/competitions/deepfake-detection-challenge/leaderboard/

References

  1. Afchar D, Nozick V, Yamagishi J, Echizen I (2018) Mesonet: a compact facial video forgery detection network. In: IEEE international workshop on information forensics and security (WIFS), pp 1–7, DOI https://doi.org/10.1109/WIFS.2018.8630761

  2. Agarwal S, Farid H, Gu Y, He M, Nagano K, Li H (2019) Protecting world leaders against deep fakes. In: 2019 IEEE/CVF Conference on computer vision and pattern recognition workshops (CVPRW)

  3. Bonettini N, Cannas ED, Mandelli S, Bondi L, Bestagini P, Tubaro S (2021) Video face manipulation detection through ensemble of CNNS. In: 2020 25th International conference on pattern recognition (ICPR), pp 5012–5019, DOI https://doi.org/10.1109/ICPR48806.2021.9412711

  4. Chesney B, Citron D (2019) Deep fakes: a looming challenge for privacy, democracy, and national security. Calif L Rev 107:1753. https://doi.org/10.15779/Z38RV0D15J

    Article  Google Scholar 

  5. Dang H, Liu F, Stehouwer J, Liu X, Jain AK (2020) On the detection of digital face manipulation. In: 2020 IEEE/CVF conference on computer vision and pattern recognition (CVPR), pp 5780–5789, DOI https://doi.org/10.1109/CVPR42600.2020.00582

  6. Dolhansky B, Bitton J, Pflaum B, Lu J, Howes R, Wang M, Ferrer CC (2020) The deepfake detection challenge (dfdc) dataset. arXiv:2006.07397

  7. Fei J, Xia Z, Yu P, Xiao F (2021) Exposing ai-generated videos with motion magnification. Multimed Tools Appl 80(20):30789–30802. https://doi.org/10.1007/s11042-020-09147-3

    Article  Google Scholar 

  8. Gonzalez RC (2009) Digital image processing. Pearson Education India, New York

    Google Scholar 

  9. Güera D, Delp EJ (2018) Deepfake video detection using recurrent neural networks. In: 2018 15th IEEE international conference on advanced video and signal based surveillance (AVSS), pp 1–6, DOI https://doi.org/10.1109/AVSS.2018.8639163

  10. Guo Z, Hu L, Xia M, Yang G (2021) Blind detection of glow-based facial forgery. Multimed Tools Appl 80(5):7687–7710. https://doi.org/10.1007/s11042-020-10098-y

    Article  Google Scholar 

  11. Ha S, Kersner M, Kim B, Seo S, Kim D (2020) Marionette: few-shot face reenactment preserving identity of unseen targets. Proc AAAI Conf Artif Intell 34(07):10893–10900. https://doi.org/10.1609/aaai.v34i07.6721

    Google Scholar 

  12. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: 2016 IEEE conference on computer vision and pattern recognition (CVPR), pp 770–778. https://doi.org/10.1109/CVPR.2016.90

  13. Huang G, Liu Z, Van Der Maaten L, Weinberger KQ (2017) Densely connected convolutional networks. In: 2017 IEEE conference on computer vision and pattern recognition (CVPR), pp 2261–2269, DOI https://doi.org/10.1109/CVPR.2017.243

  14. Jeon H, Bang Y, Woo SS (2020) Fdftnet: facing off fake images using fake detection fine-tuning network. In: IFIP international conference on ICT systems security and privacy protection, pp 416–430, DOI https://doi.org/10.1007/978-3-030-58201-2_28

  15. Johnston P, Elyan E, Jayne C (2020) Video tampering localisation using features learned from authentic content. Neural Comput Appl 32(16):12243–12257. https://doi.org/10.1007/s00521-019-04272-z

    Article  Google Scholar 

  16. King DE (2009) Dlib-ml: A machine learning toolkit. J Mach Learn Res 10:1755–1758. https://doi.org/10.5555/1577069.1755843

    Article  Google Scholar 

  17. Kingma DP, Dhariwal P (2018) Glow: generative flow with invertible 1 × 1 convolutions. In: Proceedings of the 32nd international conference on neural information processing systems. NIPS’18. Curran Associates Inc., Red Hook, NY, pp 10236–10245, DOI https://doi.org/10.5555/3327546.3327685

  18. Korshunova I, Shi W, Dambre J, Theis L (2017) Fast face-swap using convolutional neural networks. In: 2017 IEEE international conference on computer vision (ICCV), pp 3697–3705. https://doi.org/10.1109/ICCV.2017.397

  19. Li L, Bao J, Zhang T, Yang H, Chen D, Wen F, Guo B (2020) Face x-ray for more general face forgery detection. In: 2020 IEEE/CVF conference on computer vision and pattern recognition (CVPR), pp 5001-5010, DOI https://doi.org/10.1109/CVPR42600.2020.00505

  20. Li Y, Chang M-C, Lyu S (2018) In ictu oculi: exposing ai created fake videos by detecting eye blinking. In: 2018 IEEE international workshop on information forensics and security (WIFS), pp 1–7, DOI https://doi.org/10.1109/WIFS.2018.8630787

  21. Li Y, Lyu S (2018) Exposing deepfake videos by detecting face warping artifacts. arXiv:1811.00656

  22. Li Y, Yang X, Sun P, Qi H, Lyu S (2020) Celeb-df: a large-scale challenging dataset for deepfake forensics. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 3207–3216, DOI https://doi.org/10.1109/cvpr42600.2020.00327

  23. Liu H, Li X, Zhou W, Chen Y, He Y, Xue H, Zhang W, Yu N (2021) Spatial-phase shallow learning: rethinking face forgery detection in frequency domain. In: IEEE Conference on computer vision and pattern recognition (CVPR), pp 772–781, DOI https://doi.org/10.1109/CVPR46437.2021.00083

  24. Liu S, Liu D, Srivastava G, Połap D, Woźniak M (2020) Overview and methods of correlation filter algorithms in object tracking. Compl Intell Syst 7(4):1895–1917. https://doi.org/10.1007/s40747-020-00161-4

    Article  Google Scholar 

  25. Liu Z, Qi X, Torr PHS (2020) Global texture enhancement for fake face detection in the wild. In: 2020 IEEE/CVF conference on computer vision and pattern recognition (CVPR), pp 8057–8066, DOI https://doi.org/10.1109/CVPR42600.2020.00808

  26. Liu S, Wang S, Liu X, Gandomi AH, Daneshmand M, Muhammad K, De Albuquerque VHC (2021) Human memory update strategy: a multi-layer template update mechanism for remote visual monitoring. IEEE Trans Multimed 23:2188–2198. https://doi.org/10.1109/TMM.2021.3065580

    Article  Google Scholar 

  27. Liu S, Wang S, Liu X, Lin C-T, Lv Z (2021) Fuzzy detection aided real-time and robust visual tracking under complex environments. IEEE Trans Fuzzy Syst 29(1):90–102. https://doi.org/10.1109/TFUZZ.2020.3006520

    Article  Google Scholar 

  28. Masi I, Killekar A, Mascarenhas RM, Gurudatt SP, AbdAlmageed W (2020) Two-branch recurrent network for isolating deepfakes in videos. In: The European conference on computer vision (ECCV). computer vision – ECCV 2020, pp 667–684, DOI https://doi.org/10.1007/978-3-030-58571-6_39

  29. Mirsky Y, Lee W (2021) The creation and detection of deepfakes: a survey. ACM Comput Surv 54(1):1–41. https://doi.org/10.1145/3425780

    Article  Google Scholar 

  30. Nguyen HH, Fang F, Yamagishi J, Echizen I (2019) Multi-task learning for detecting and segmenting manipulated facial images and videos. In: IEEE 10th international conference on biometrics theory, applications and systems (BTAS), pp 1–8, DOI https://doi.org/10.1109/BTAS46853.2019.9185974

  31. Nguyen HH, Yamagishi J, Echizen I (2019) Capsule-forensics: using capsule networks to detect forged images and videos. In: IEEE international conference on acoustics, speech and signal processing (ICASSP), pp 2307–2311, DOI https://doi.org/10.1109/ICASSP.2019.8682602

  32. Pan D, Sun L, Wang R, Zhang X, Sinnott RO (2020) Deepfake detection through deep learning. In: 2020 IEEE/ACM international conference on big data computing, applications and technologies (BDCAT), pp 134–143, DOI https://doi.org/10.1109/BDCAT50828.2020.00001

  33. Pumarola A, Agudo A, Martinez AM, Sanfeliu A, Moreno-Noguer F (2020) Ganimation: one-shot anatomically consistent facial animation. Int J Comput Vis 128(3):698–713. https://doi.org/10.1007/s11263-019-01210-3

    Article  Google Scholar 

  34. Qian Y, Yin G, Sheng L, Chen Z, Shao J (2020) Thinking in frequency: face forgery detection by mining frequency-aware clues. In: The European conference on computer vision (ECCV). Computer vision – ECCV 2020, pp 86–103, DOI https://doi.org/10.1007/978-3-030-58610-2_6

  35. Rössler A, Cozzolino D, Verdoliva L, Riess C, Thies J, Niessner M (2019) Faceforensics++: learning to detect manipulated facial images. In: IEEE international conference on computer vision (ICCV), pp 1–11, DOI https://doi.org/10.1109/ICCV.2019.00009

  36. Ruiz N, Bargal SA, Sclaroff S (2020) Disrupting deepfakes: adversarial attacks against conditional image translation networks and facial manipulation systems. In: European conference on computer vision, pp 236–251, DOI https://doi.org/10.1007/978-3-030-66823-5_14

  37. Shang Z, Xie H, Zha Z, Yu L, Li Y, Zhang Y (2021) Prrnet: pixel-region relation network for face forgery detection. Pattern Recogn 116:107950. https://doi.org/10.1016/j.patcog.2021.107950

    Article  Google Scholar 

  38. Singh A, Saimbhi AS, Singh N, Mittal M (2020) Deepfake video detection: a time-distributed approach. SN Comput Sci 1(4):1–8. https://doi.org/10.1007/s42979-020-00225-9

    Article  Google Scholar 

  39. Tan M, Le Q (2019) Efficientnet: rethinking model scaling for convolutional neural networks. In: International conference on machine learning, PMLR, pp 6105–6114

  40. Tolosana R, Vera-Rodriguez R, Fierrez J, Morales A, Ortega-Garcia J (2020) Deepfakes and beyond: a survey of face manipulation and fake detection. Inf Fusion 64:131–148. https://doi.org/10.1016/j.inffus.2020.06.014

    Article  Google Scholar 

  41. Wang M, Deng W (2021) Deep face recognition: a survey. Neurocomputing 429:215–244. https://doi.org/10.1016/j.neucom.2020.10.081

    Article  Google Scholar 

  42. Wang W, Dong J, Tan T (2010) Tampered region localization of digital color images based on jpeg compression noise. In: Proceedings of the 9th international conference on digital watermarking, Springer, pp 120–133, DOI https://doi.org/10.5555/1946180.1946190

  43. Wang Z, Yu Z, Zhao C, Zhu X, Qin Y, Zhou Q, Zhou F, Lei Z (2020) Deep spatial gradient and temporal depth learning for face anti-spoofing. In: 2020 IEEE/CVF conference on computer vision and pattern recognition (CVPR), pp 5041–5050, DOI https://doi.org/10.1109/CVPR42600.2020.00509

  44. Zakharov E, Shysheya A, Burkov E, Lempitsky V (2019) Few-shot adversarial learning of realistic neural talking head models. In: 2019 IEEE/CVF international conference on computer vision (ICCV), pp 9458–9467, DOI https://doi.org/10.1109/ICCV.2019.00955

  45. Zhang J, Zeng X, Wang M, Pan Y, Liu L, Liu Y, Ding Y, Fan C (2020) Freenet: multi-identity face reenactment. In: 2020 IEEE/CVF conference on computer vision and pattern recognition (CVPR), pp 5325–5334, DOI https://doi.org/10.1109/CVPR42600.2020.00537

  46. Zhang W, Zhao C, Li Y (2020) A novel counterfeit feature extraction technique for exposing face-swap images based on deep learning and error level analysis. Entropy 22(2):249. https://doi.org/10.3390/e22020249

    Article  MathSciNet  Google Scholar 

  47. Zhao H, Zhou W, Chen D, Wei T, Zhang W, Yu N (2021) Multi-attentional deepfake detection. In: IEEE Conference on computer vision and pattern recognition (CVPR), pp 2185–2194. https://doi.org/10.1109/CVPR46437.2021.00222

  48. Zhou P, Han X, Morariu VI, Davis LS (2017) Two-stream neural networks for tampered face detection. In: 2017 IEEE conference on computer vision and pattern recognition workshops (CVPRW), pp 1831–1839, DOI https://doi.org/10.1109/cvprw.2017.229

  49. Zhu B, Fang H, Sui Y, Li L (2020) Deepfakes for medical video de-identification: privacy protection and diagnostic information preservation. In: Proceedings of the AAAI/ACM conference on ai, ethics, and society. https://doi.org/10.1145/3375627.3375849. Association for Computing Machinery, New York, pp 414?-420

Download references

Acknowledgements

This work was supported by the Natural Science Foundation of Anhui Province of China under Grant No.2008085MF220, and the School Foundation of Anhui University of Science and Technology under Grant No.2021CX2102.

Author information

Authors and Affiliations

  1. School of Computer Science and Engineering, Anhui University of Science and Technology, Huainan, China

    Kun Xu, Gaoming Yang & Xianjin Fang

  2. Department of Mathematics and Computing, University of Southern Queensland, Toowoomba, Queensland, Australia

    Ji Zhang

Authors
  1. Kun Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Gaoming Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xianjin Fang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Ji Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Gaoming Yang.

Ethics declarations

Conflict of Interests

The authors declare that they have no conflict of interest.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Xu, K., Yang, G., Fang, X. et al. Facial depth forgery detection based on image gradient. Multimed Tools Appl 82, 29501–29525 (2023). https://doi.org/10.1007/s11042-023-14626-4

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-023-14626-4

Keywords

Search

Navigation