Skip to main content
SpringerLink
Log in

Light-weight 3DCNN for DeepFakes, FaceSwap and Face2Face facial forgery detection

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Detecting facial forgery in real time application such as video conferencing, netbanking, verification of identity etc. is a major concern. Highly accurate facial forgery detectors will be incompatible with limited computation and storage space devices like smartphones, tablets, PCs due to their deep complex architectures. Therefore, we propose a light weight 3D convolutional neural networks (3DCNN) to detect popular facial forgeries namely, DeepFakes, Face2Face and FaceSwap. The proposed 3DCNN is a five layered architecture with only 2.69M trainable parameters. The proposed network learns spatio-temporal features to detect forged videos. Further, to understand the detection process of 3DCNN, activation maps are studied in detail. The popular FaceForensic++ dataset is employed to train and test the proposed method. In addition, the effectiveness of the proposed method is evaluated and compared with state-of-the art methods in terms of both, number of trainable parameters and binary detection accuracy.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  1. Amerini I, Galteri L, Caldelli R, Del Bimbo A (2019) Deepfake video detection through optical flow based cnn. In: 2019 IEEE/CVF International conference on computer vision workshop (ICCVW), pp 1205–1207. https://doi.org/10.1109/ICCVW.2019.00152

  2. Amirkhani D, Bastanfard A (2021) An objective method to evaluate exemplar-based inpainted images quality using jaccard index. Multimedia Tools and Applications 80. https://doi.org/10.1007/s11042-021-10883-3

  3. Baek J, Yoo Y, Bae S (2020) Generative adversarial ensemble learning for face forensics. IEEE Access 8:45421–45431

    Article  Google Scholar 

  4. Bastanfard A, Bastanfard O, Takahashi H, Nakajima M (2004) Toward anthropometrics simulation of face rejuvenation and skin cosmetic: research articles. Comput Animat Virtual Worlds 15(3–4):347–352

    Article  Google Scholar 

  5. Bastanfard A, Takahashi H, Nakajima M (2004) Toward e-appearance of human face and hair by age, expression and rejuvenation. In: 2004 International conference on cyberworlds, pp 306–311. https://doi.org/10.1109/CW.2004.65

  6. Chollet F (2017) Xception: Deep learning with depthwise separable convolutions. In: 2017 IEEE Conference on computer vision and pattern recognition (CVPR), pp 1800–1807. https://doi.org/10.1109/CVPR.2017.195

  7. Cozzolino D, Poggi G, Verdoliva L (2017) Recasting residual-based local descriptors as convolutional neural networks: An application to image forgery detection. In: Proceedings of the 5th ACM workshop on information hiding and multimedia security, IHMMSec ’17. https://doi.org/10.1145/3082031.3083247. Association for Computing Machinery, New York, pp 159–164

  8. Deepfakes github. https://github.com/deepfakes/faceswap. Accessed: 2020-02-01

  9. Dosovitskiy A, Fischer P, Ilg E, Häusser P, Hazirbas C, Golkov V, Smagt PVD, Cremers D, Brox T (2015) Flownet: Learning optical flow with convolutional networks. In: 2015 IEEE International conference on computer vision (ICCV), pp 2758–2766. https://doi.org/10.1109/ICCV.2015.316

  10. Faceswap github. https://github.com/MarekKowalski/FaceSwap. Accessed: 2020-02-01

  11. Fridrich J, Kodovsky J (2012) Rich models for steganalysis of digital images. IEEE Transactions on Information Forensics and Security 7(3):868–882. https://doi.org/10.1109/TIFS.2012.2190402

    Article  Google Scholar 

  12. Hara K, Kataoka H, Satoh Y (2018) Can spatiotemporal 3d cnns retrace the history of 2d cnns and imagenet?. https://doi.org/10.1109/CVPR.2018.00685

  13. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: 2016 IEEE conference on computer vision and pattern recognition (CVPR), pp 770–778. https://doi.org/10.1109/CVPR.2016.90

  14. Howard AG, Zhu M, Chen B, Kalenichenko D, Wang W, Weyand T, Andreetto M, Adam H (2017) Mobilenets: Efficient convolutional neural networks for mobile vision applications

  15. Iandola FN, Han S, Moskewicz MW, Ashraf K, Dally WJ, Keutzer K (2016) Squeezenet: Alexnet-level accuracy with 50x fewer parameters and < 0.5mb model size. Cite arXiv:1602.07360. Trans Inf Forensic Secur Comment: In ICLR Format

  16. Kohli A, Gupta A (2021) Detecting deepfake, faceswap and face2face facial forgeries using frequency cnn. Multimedia Tools and Applications pp 1–20. https://doi.org/10.1007/s11042-020-10420-8

  17. Rahmouni N, Nozick V, Yamagishi J, Echizen I (2017) Distinguishing computer graphics from natural images using convolution neural networks. In: 2017 IEEE Workshop on information forensics and security (WIFS), pp 1–6. https://doi.org/10.1109/WIFS.2017.8267647

  18. Rössler A, Cozzolino D, Verdoliva L, Riess C, Thies J, Nießner M. (2019) FaceForensics++: Learning to detect manipulated facial images. In: International conference on computer vision (ICCV)

  19. Sandler M, Howard AG, Zhu M, Zhmoginov A, Chen LC (2018) Mobilenetv2: Inverted residuals and linear bottlenecks. In: 2018 IEEE/CVF conference on computer vision and pattern recognition pp 4510–4520

  20. Schroff F, Kalenichenko D, Philbin J (2015) Facenet: A unified embedding for face recognition and clustering. https://doi.org/10.1109/CVPR.2015.7298682

  21. Simonyan K, Zisserman A (2015) Very deep convolutional networks for large-scale image recognition. In: International conference on learning representations

  22. Sun D, Yang X, Liu MY, Kautz J (2018) PWC-Net: CNNs for optical flow using pyramid, warping and cost volume

  23. Szegedy C, Liu W, Jia Y, Sermanet P, Reed S, Anguelov D, Erhan D, Vanhoucke V, Rabinovich A (2015) Going deeper with convolutions. https://doi.org/10.1109/CVPR.2015.7298594

  24. Thies J, Zollhofer M, Stamminger M, Theobalt C, NieBner M (2016) Face2face: Real-time face capture and reenactment of rgb videos. pp 2387–2395. https://doi.org/10.1109/CVPR.2016.262

  25. Viola P, Jones M (2001) Rapid object detection using a boosted cascade of simple features. In: Proceedings of the 2001 IEEE computer society conference on computer vision and pattern recognition. CVPR 2001, vol 1, pp I–I. https://doi.org/10.1109/CVPR.2001.990517

  26. Wang Y, Dantcheva A (2020) A video is worth more than 1000 lies. comparing 3dcnn approaches for detecting deepfakes. In: 2020 15Th IEEE international conference on automatic face and gesture recognition (FG 2020), pp 515–519. https://doi.org/10.1109/FG47880.2020.00089

  27. Yang Z, Moczulski M, Denil M, De Freitas N, Song L, Wang Z (2015) Deep fried convnets. In: 2015 IEEE International conference on computer vision (ICCV), pp 1476–1483. https://doi.org/10.1109/ICCV.2015.173

  28. Zhou B, Khosla A, Lapedriza A, Oliva A, Torralba A (2016) Learning deep features for discriminative localization. In: 2016 IEEE Conference on computer vision and pattern recognition (CVPR), pp 2921–2929. https://doi.org/10.1109/CVPR.2016.319

Download references

Acknowledgements

The authors would thank Council of Scientific and Industrial Research, India (CSIR) for providing financial support to conduct research work, file No. 09/1132(0010)/2019 EMR-I.

Author information

Authors and Affiliations

  1. Department of Electronics and Communication Engineering, Jaypee Institute of Information Technology, A-10, Sector-62, Noida, Uttar Pradesh, 201309, India

    Aditi Kohli

  2. RBS Services India Private Limited, NatWest Group, Candor Techspace, Sector-21, Dundahera, Gurugram, Haryana, 122016, India

    Abhinav Gupta

Authors
  1. Aditi Kohli
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Abhinav Gupta
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Abhinav Gupta.

Ethics declarations

Conflict of Interests

Aditi Kohli and Abhinav Gupta declare that they have no conflict of interest. Aditi Kohli receives research grant, file No. 09/1132(0010)/2019 EMR-I, from Council of Scientific and Industrial Research, India (CSIR).

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kohli, A., Gupta, A. Light-weight 3DCNN for DeepFakes, FaceSwap and Face2Face facial forgery detection. Multimed Tools Appl 81, 31391–31403 (2022). https://doi.org/10.1007/s11042-022-12778-3

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-022-12778-3

Keywords

Search

Navigation