Skip to main content
SpringerLink
Log in

Obstructing DeepFakes by Disrupting Face Detection and Facial Landmarks Extraction

  • Chapter
  • First Online:
Deep Learning-Based Face Analytics

Part of the book series: Advances in Computer Vision and Pattern Recognition ((ACVPR))

Abstract

Recent years have seen fast development in synthesizing realistic human faces using AI technologies. AI-synthesized fake faces can be weaponized to cause negative personal and social impact. In this work, we develop technologies to defend individuals from becoming victims of recent AI-synthesized fake videos by sabotaging would-be training data. This is achieved by disrupting deep neural network (DNN)-based face detection and facial landmark extraction method with specially designed imperceptible adversarial perturbations to reduce the quality of the detected faces. We empirically show the effectiveness of our methods in disrupting state-of-the-art DNN-based face detectors and facial landmark extractors on several datasets.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 149.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 199.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 199.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    Since ground truth faces are not labeled in 300-W, we use the detection results of Dlib as the ground truth detection, which is also the protocol used in a compared work [5].

References

  1. Afchar D, Nozick V, Yamagishi J, Echizen I (2018) MesoNet: a compact facial video forgery detection network. In: IEEE international workshop on information forensics and security (WIFS)

    Google Scholar 

  2. Baluja S, Fischer I (2018) Learning to attack: adversarial transformation networks. In: Association for the advancement of artificial intelligence (AAAI)

    Google Scholar 

  3. Bansal A, Nanduri A, Castillo CD, Ranjan R., Chellappa R (2016) Umdfaces: an annotated face dataset for training deep networks. arXiv preprint arXiv:1611.01484v2

  4. Berthelot D, Schumm T, Metz L (2017) Began: boundary equilibrium generative adversarial networks. arXiv preprint arXiv:1703.10717

  5. Bose AJ, Aarabi P (2018) Adversarial attacks on face detectors using neural net based constrained optimization. In: IEEE international workshop on multimedia signal processing (MMSP)

    Google Scholar 

  6. Bulat A, Tzimiropoulos G (2017) How far are we from solving the 2D & 3D face alignment problem? (and a dataset of 230,000 3D facial landmarks). In: ICCV

    Google Scholar 

  7. Carlini N, Wagner D (2017) Towards evaluating the robustness of neural networks. In: IEEE symposium on security and privacy (sp)

    Google Scholar 

  8. Chan C, Ginosar S, Zhou T, Efros AA (2018) Everybody dance now. arXiv preprint arXiv:1808.07371

  9. Chan C, Ginosar S, Zhou T, Efros AA (2019) Everybody dance now. In: IEEE international conference on computer vision (ICCV)

    Google Scholar 

  10. Chen ST, Cornelius C, Martin J, Chau DH (2018) Robust physical adversarial attack on faster R-CNN object detector. arXiv preprint arXiv:1804.05810

  11. Chesney R, Citron DK, Deep fakes: a looming challenge for privacy, democracy, and national security. 107 California Law Review (2019, Forthcoming); U of Texas Law, Public Law Research Paper No. 692; U of Maryland Legal Studies Research Paper No. 2018-21

    Google Scholar 

  12. Dalal N, Triggs B (2005) Histograms of oriented gradients for human detection. In: IEEE conference on computer vision and pattern recognition (CVPR)

    Google Scholar 

  13. Denton EL, Chintala S, Fergus R, et al (2015) Deep generative image models using a laplacian pyramid of adversarial networks. In: Conference on neural information processing systems (NeurIPS)

    Google Scholar 

  14. Dong Y, Liao F, Pang T, Su H, Zhu J, Hu X, Li J (2018) Boosting adversarial attacks with momentum. In: IEEE conference on computer vision and pattern recognition (CVPR)

    Google Scholar 

  15. Everingham M, Eslami SA, Van Gool L, Williams CK, Winn J, Zisserman A (2015) The pascal visual object classes challenge: a retrospective. Int J Comput Vis (IJCV)

    Google Scholar 

  16. Eykholt K, Evtimov I, Fernandes E, Li B, Rahmati A, Tramer F, Prakash A, Kohno T, Song D (2018) Physical adversarial examples for object detectors. arXiv preprint arXiv:1807.07769

  17. Farfade SS, Saberian MJ, Li LJ (2015) Multi-view face detection using deep convolutional neural networks. In: ACM on international conference on multimedia retrieval

    Google Scholar 

  18. Goodfellow I, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville A, Bengio Y (2014) Generative adversarial nets. In: Conference on neural information processing systems (NeurIPS)

    Google Scholar 

  19. Goodfellow IJ, Shlens J, Szegedy C (2015) Explaining and harnessing adversarial examples. In: International conference on learning representations (ICLR)

    Google Scholar 

  20. Güera D, Delp EJ (2018) Deepfake video detection using recurrent neural networks. In: AVSS

    Google Scholar 

  21. Gulrajani I, Ahmed F, Arjovsky M, Dumoulin V, Courville AC (2017) Improved training of wasserstein gans. In: Conference on neural information processing systems (NeurIPS)

    Google Scholar 

  22. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: IEEE conference on computer vision and pattern recognition (CVPR)

    Google Scholar 

  23. Hu T, Qi H, Xu J, Huang Q (2018) Facial landmarks detection by self-iterative regression based landmarks-attention network. In: AAAI

    Google Scholar 

  24. Isola P, Zhu JY, Zhou T, Efros AA (2017) Image-to-image translation with conditional adversarial networks. In: IEEE conference on computer vision and pattern recognition (CVPR)

    Google Scholar 

  25. Jiang H, Learned-Miller E (2017) Face detection with the faster R-CNN. In: IEEE international conference on automatic face & gesture recognition (FG)

    Google Scholar 

  26. Karras T, Aila T, Laine S, Lehtinen J (2018) Progressive growing of GANs for improved quality, stability, and variation. In: International conference on learning representations (ICLR)

    Google Scholar 

  27. Karras T, Laine S, Aila T (2018) A style-based generator architecture for generative adversarial networks. arXiv preprint arXiv:1812.04948

  28. Karras T, Laine S, Aila T (2019) A style-based generator architecture for generative adversarial networks. In: IEEE conference on computer vision and pattern recognition (CVPR)

    Google Scholar 

  29. Kazemi V, Sullivan J (2014) One millisecond face alignment with an ensemble of regression trees. In: CVPR

    Google Scholar 

  30. Kim H, Carrido P, Tewari A, Xu W, Thies J, Niessner M, Pérez P, Richardt C, Zollhöfer M, Theobalt C (2018) Deep video portraits. ACM Trans Graph (TOG)

    Google Scholar 

  31. King DE (2009) Dlib-ml: a machine learning toolkit. J Mach Learn Res 10:1755–1758

    Google Scholar 

  32. Kurakin A, Goodfellow I, Bengio S (2017) Adversarial examples in the physical world. In: International conference on learning representations (ICLR)

    Google Scholar 

  33. Li H, Lin Z, Shen X, Brandt J, Hua G (2015) A convolutional neural network cascade for face detection. In: IEEE conference on computer vision and pattern recognition (CVPR)

    Google Scholar 

  34. Li J, Wang T, Zhang Y (2011) Face detection using surf cascade. In: IEEE international conference on computer vision workshops (ICCV Workshops)

    Google Scholar 

  35. Li J, Zhang Y (2013) Learning surf cascade for fast and accurate object detection. In: IEEE conference on computer vision and pattern recognition (CVPR)

    Google Scholar 

  36. Li Y, Chang MC, Lyu S (2018) In Ictu Oculi: exposing AI generated fake face videos by detecting eye blinking. In: IEEE international workshop on information forensics and security (WIFS)

    Google Scholar 

  37. Li Y, Lyu S (2019) Exposing deepfake videos by detecting face warping artifacts. In: IEEE conference on computer vision and pattern recognition workshops (CVPRW)

    Google Scholar 

  38. Li Y, Tian D, Chang M, Bian X, Lyu S (2018) Robust adversarial perturbation on deep proposal-based models. In: British machine vision conference (BMVC)

    Google Scholar 

  39. Li Y, Yang X, Sun P, Qi H, Lyu S (2020) Celeb-DF: a large-scale challenging dataset for deepfake forensics. In: CVPR

    Google Scholar 

  40. Liu B, Ding M, Zhu T, Xiang Y, Zhou W (2018) Using adversarial noises to protect privacy in deep learning era. In: IEEE global communications conference (GLOBECOM)

    Google Scholar 

  41. Liu MY, Breuel T, Kautz J (2017) Unsupervised image-to-image translation networks. In: Conference on neural information processing systems (NeurIPS)

    Google Scholar 

  42. Liu W, Anguelov D, Erhan D, Szegedy C, Reed S, Fu CY, Berg AC (2016) SSD: single shot multibox detector. In: European conference on computer vision (ECCV)

    Google Scholar 

  43. Liu Y, Zhang W, Yu N (2017) Protecting privacy in shared photos via adversarial examples based stealth. Secur Commun Netw

    Google Scholar 

  44. Lu J, Sibai H, Fabry E (2017) Adversarial examples that fool detectors. arXiv:1712:02494

  45. Luo B, Liu Y, Wei L, Xu Q (2018) Towards imperceptible and robust adversarial example attacks against neural networks. In: Association for the advancement of artificial intelligence (AAAI)

    Google Scholar 

  46. Ma L, Jia X, Sun Q, Schiele B, Tuytelaars T, Van Gool L (2017) Pose guided person image generation. In: NeurIPS

    Google Scholar 

  47. Matern F, Riess C, Stamminger M (2019) Exploiting visual artifacts to expose deepfakes and face manipulations. In: IEEE winter applications of computer vision workshops (WACVW)

    Google Scholar 

  48. Moosavi-Dezfooli SM, Fawzi A, Fawzi O, Frossard P (2017) Universal adversarial perturbations. In: IEEE conference on computer vision and pattern recognition (CVPR)

    Google Scholar 

  49. Moosavi-Dezfooli SM, Fawzi A, Frossard P (2016) Deepfool: a simple and accurate method to fool deep neural networks. In: IEEE conference on computer vision and pattern recognition (CVPR)

    Google Scholar 

  50. Najibi M, Samangouei P, Chellappa R, Davis LS (2017) SSH: single stage headless face detector. In: IEEE international conference on computer vision (ICCV)

    Google Scholar 

  51. Oh SJ, Fritz M, Schiele B (2017) Adversarial image perturbation for privacy protection a game theory perspective. In: IEEE international conference on computer vision (ICCV)

    Google Scholar 

  52. Ojala T, Pietikäinen M, Mäenpää T (2002) Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Trans Pattern Anal Mach Intell (TPAMI)

    Google Scholar 

  53. Papernot N, McDaniel P, Jha S, Fredrikson M, Celik ZB, Swami A (2016) The limitations of deep learning in adversarial settings. In: EuroS&P

    Google Scholar 

  54. Qian S, Sun K, Wu W, Qian C, Jia J (2019) Aggregation via separation: boosting facial landmark detector with semi-supervised style translation. In: ICCV

    Google Scholar 

  55. Ramanan D, Zhu X (2012) Face detection, pose estimation, and landmark localization in the wild. In: IEEE conference on computer vision and pattern recognition (CVPR)

    Google Scholar 

  56. Ranjan R, Patel VM, Chellappa R (2015) A deep pyramid deformable part model for face detection. In: IEEE international conference on biometrics theory, applications and systems (BTAS)

    Google Scholar 

  57. Ranjan, R., Patel VM, Chellappa R (2019) Hyperface: a deep multi-task learning framework for face detection, landmark localization, pose estimation, and gender recognition. IEEE Trans Pattern Anal Mach Intell (TPAMI)

    Google Scholar 

  58. Ranjan, R., Sankaranarayanan, S., Castillo, C.D., Chellappa, R.: An all-in-one convolutional neural network for face analysis. In: IEEE international conference on automatic face & gesture recognition (FG)

    Google Scholar 

  59. Ren S, He K, Girshick R, Sun J (2017) Faster R-CNN: towards real-time object detection with region proposal networks. IEEE Trans Pattern Anal Mach Intell (TPAMI)

    Google Scholar 

  60. Ruiz N, Sclaroff S (2020) Disrupting deepfakes: adversarial attacks against conditional image translation networks and facial manipulation systems. arXiv preprint arXiv:2003.01279

  61. Rumelhart DE, Hinton GE, Williams RJ, et al (1988) Learning representations by back-propagating errors. Cognit Model

    Google Scholar 

  62. Sagonas C, Tzimiropoulos G, Zafeiriou S, Pantic M (2013) 300 faces in-the-wild challenge: the first facial landmark localization challenge. In: IEEE international conference on computer vision workshops (ICCVW)

    Google Scholar 

  63. Salimans T, Goodfellow I, Zaremba W, Cheung V, Radford A, Chen X (2016) Improved techniques for training GANS. In: Conference on neural information processing systems (NeurIPS)

    Google Scholar 

  64. Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556

  65. Sun K, Xiao B, Liu D, Wang J (2019) Deep high-resolution representation learning for human pose estimation. In: CVPR

    Google Scholar 

  66. Sun X, Wu P, Hoi SC (2018) Face detection using deep learning: an improved faster RCNN approach. Neurocomputing

    Google Scholar 

  67. Suwajanakorn S, Seitz SM, Kemelmacher-Shlizerman I (2017) Synthesizing obama: learning lip sync from audio. ACM Trans Graph (TOG)

    Google Scholar 

  68. Szegedy C, Zaremba W, Sutskever I, Bruna J, Erhan D, Goodfellow I, Fergus R (2013) Intriguing properties of neural networks. arXiv 1312:6199

    Google Scholar 

  69. Szegedy C, Zaremba W, Sutskever I, Bruna J, Erhan D, Goodfellow I, Fergus R (2014) Intriguing properties of neural networks. In: ICLR

    Google Scholar 

  70. Taigman Y, Polyak A, Wolf L (2016) Unsupervised cross-domain image generation. arXiv preprint arXiv:1611.02200

  71. Tang X, Du DK, He Z, Liu J (2018) Pyramidbox: a context-assisted single shot face detector. In: European conference on computer vision (ECCV)

    Google Scholar 

  72. Thies J, Zollhofer M, Stamminger M, Theobalt C, Niessner M (2016) Face2Face: real-time face capture and reenactment of RGB videos. In: IEEE conference on computer vision and pattern recognition (CVPR)

    Google Scholar 

  73. Viola P, Jones M (2001) Rapid object detection using a boosted cascade of simple features. In: IEEE conference on computer vision and pattern recognition (CVPR)

    Google Scholar 

  74. Wang H, Li Z, Ji X, Wang Y (2017) Face R-CNN. arXiv preprint arXiv:1706.01061

  75. Wang Z, Bovik AC, Sheikh HR, Simoncelli EP, et al (2004) Image quality assessment: from error visibility to structural similarity. IEEE Trans Image Process (TIP)

    Google Scholar 

  76. Wu W, Qian C, Yang S, Wang Q, Cai Y, Zhou Q (2018) Look at boundary: a boundary-aware face alignment algorithm. In: CVPR

    Google Scholar 

  77. Wu W, Zhang Y, Li C, Qian C, Change Loy C (2018) Reenactgan: learning to reenact faces via boundary transfer. In: ECCV

    Google Scholar 

  78. Xie C, Wang J, Zhang Z, Zhou Y, Xie L, Yuille A (2017) Adversarial examples for semantic segmentation and object detection. In: IEEE international conference on computer vision (ICCV)

    Google Scholar 

  79. Xie C, Zhang Z, Zhou Y, Bai S, Wang J, Ren Z, Yuille AL (2019) Improving transferability of adversarial examples with input diversity. In: CVPR

    Google Scholar 

  80. Yang B, Yan J, Lei Z, Li SZ (2015) Convolutional channel features. In: IEEE international conference on computer vision (ICCV)

    Google Scholar 

  81. Yang S, Luo P, Loy CC, Tang X (2015) From facial parts responses to face detection: a deep learning approach. In: IEEE international conference on computer vision (ICCV)

    Google Scholar 

  82. Yang S, Luo P, Loy CC, Tang X (2016) Wider face: a face detection benchmark. In: IEEE Conference on computer vision and pattern recognition (CVPR)

    Google Scholar 

  83. Yang S, Xiong Y, Loy CC, Tang X (2017) Face detection through scale-friendly deep convolutional networks. arXiv preprint arXiv:1706.02863

  84. Yang X, Dong Y, Pang T, Zhu J, Su H (2020) Towards privacy protection by generating adversarial identity masks. arXiv preprint arXiv:2003.06814

  85. Yang, X., Li, Y., Lyu, S.: Exposing deep fakes using inconsistent head poses. In: ICASSP (2019)

    Google Scholar 

  86. Zeng X, Liu C, Qiu W, Xie L, Tai YW, Tang CK, Yuille AL (2017) Adversarial attacks beyond the image space. arXiv:1711.07183

  87. Zhang S, Zhu X, Lei Z, Shi H, Wang X, Li SZ (2017) S3FD: single shot scale-invariant face detector. In: IEEE international conference on computer vision (ICCV)

    Google Scholar 

  88. Zou X, Zhong S, Yan L, Zhao X, Zhou J, Wu Y (2019) Learning robust facial landmark detection via hierarchical structured ensemble. In: ICCV

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. University at Buffalo, State University of New York, Buffalo, USA

    Yuezun Li & Siwei Lyu

Authors
  1. Yuezun Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Siwei Lyu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yuezun Li .

Editor information

Editors and Affiliations

  1. Department of Computer Science and Engineering, University at Buffalo-SUNY, Buffalo, NY, USA

    Nalini K Ratha

  2. Department of Electrical and Computer Engineering, Johns Hopkins University, Baltimore, MD, USA

    Vishal M. Patel

  3. Departments of Electrical and Computer Engineering (Whiting School of Engineering) and Biomedical Engineering (School of Medicine), Johns Hopkins University, Baltimore, MD, USA

    Rama Chellappa

Rights and permissions

Reprints and permissions

Copyright information

© 2021 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Li, Y., Lyu, S. (2021). Obstructing DeepFakes by Disrupting Face Detection and Facial Landmarks Extraction. In: Ratha, N.K., Patel, V.M., Chellappa, R. (eds) Deep Learning-Based Face Analytics. Advances in Computer Vision and Pattern Recognition. Springer, Cham. https://doi.org/10.1007/978-3-030-74697-1_12

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-74697-1_12

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-74696-4

  • Online ISBN: 978-3-030-74697-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation