Abstract
This paper presents a novel Machine Learning (ML)-based DeepFake detection technology named CHIEFS (Corneal-Specular Highlights Imaging for Enhancing Fake-Face Spotter). We focus on the most reflective area of a human face, the eyes, upon the hypothesis that the existing DeepFake creation methods fail to coordinate their counterfeits with the reflective components. In addition to the traditional checking of the reflection shape similarity (RSS), we detect various corneal-specular highlights features, such as color components and textures, to find corneal-specular highlights consistency (CHC). Furthermore, we inspect the ensemble of the highlights with the surrounding environmental factors (SEF), including the light settings, directions, and strength. We designed and built them as modular features and have conducted extensive experiments with different combinations of the components using various input parameters and Deep Neural Network (DNN) architectures on Generative Adversarial Network (GAN)-based DeepFake datasets. The empirical results show that CHIEFS with three modules improves the accuracy from 86.05% (with the RSS alone) to 99.00% with the ResNet-50-V2 architecture.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Chiefs-DFD dataset (2022). https://github.com/READFake/CHIEFS-DFD-Dataset. Accessed 26 Nov 2022
Lexica stable diffusion search engine (2022). https://lexica.art/. Accessed 24 Nov 2022
Chollet, F.: Xception: deep learning with depthwise separable convolutions. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1251–1258 (2017)
Chollet, F.: Deep Learning with Python. Simon and Schuster, New York (2021)
Dolhansky, B., et al.: The DeepFake detection challenge (DFDC) dataset. arXiv preprint arXiv:2006.07397 (2020)
Dutta, A., Zisserman, A.: The VIA annotation software for images, audio and video. In: Proceedings of the 27th ACM International Conference on Multimedia. MM 2019, ACM, New York, NY, USA (2019). https://doi.org/10.1145/3343031.3350535
Guo, H., Hu, S., Wang, X., Chang, M.C., Lyu, S.: Eyes tell all: irregular pupil shapes reveal GAN-generated faces (2021)
He, K., Zhang, X., Ren, S., Sun, J.: Identity mappings in deep residual networks. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9908, pp. 630–645. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46493-0_38
Hu, S., Li, Y., Lyu, S.: Exposing GAN-generated faces using inconsistent corneal specular highlights. arXiv preprint arXiv:2009.11924 (2020)
Huang, G., Liu, Z., van der Maaten, L., Weinberger, K.Q.: Densely connected convolutional networks (2016). https://doi.org/10.48550/ARXIV.1608.06993, https://arxiv.org/abs/1608.06993
Icons8: Face Swapper (2022). https://icons8.com/swapper
Karras, T., Aila, T., Laine, S., Lehtinen, J.: Progressive growing of GANs for improved quality, stability, and variation. CoRR abs/1710.10196 (2017). http://arxiv.org/abs/1710.10196
Karras, T., et al.: Alias-free generative adversarial networks. In: Proceedings of the NeurIPS (2021)
Karras, T., Laine, S., Aila, T.: A style-based generator architecture for generative adversarial networks. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4401–4410 (2019)
Karras, T., Laine, S., Aittala, M., Hellsten, J., Lehtinen, J., Aila, T.: Analyzing and improving the image quality of StyleGAN. In: Proceedings of the CVPR (2020)
Korshunova, I., Shi, W., Dambre, J., Theis, L.: Fast face-swap using convolutional neural networks. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 3677–3685 (2017)
Le, B.M., Woo, S.S.: Exploring the asynchronous of the frequency spectra of GAN-generated facial images. CoRR abs/2112.08050 (2021). https://arxiv.org/abs/2112.08050
Li, L., Bao, J., Yang, H., Chen, D., Wen, F.: FaceShifter: towards high fidelity and occlusion aware face swapping (2020)
Li, Y., Chang, M.C., Lyu, S.: In Ictu oculi: Exposing AI generated fake face videos by detecting eye blinking. arXiv preprint arXiv:1806.02877 (2018)
Li, Y., Yang, X., Sun, P., Qi, H., Lyu, S.: Celeb-DF: A large-scale challenging dataset for DeepFake forensics. In: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3204–3213 (2020). https://doi.org/10.1109/CVPR42600.2020.00327
Liu, H., et al.: Spatial-phase shallow learning: Rethinking face forgery detection in frequency domain (2021)
Matern, F., Riess, C., Stamminger, M.: Exploiting visual artifacts to expose DeepFakes and face manipulations. In: 2019 IEEE Winter Applications of Computer Vision Workshops (WACVW), pp. 83–92. IEEE (2019)
Mo, H., Chen, B., Luo, W.: Fake faces identification via convolutional neural network. In: Proceedings of the 6th ACM Workshop on Information Hiding and Multimedia Security, p. 43–47. IH & MMSec 2018, Association for Computing Machinery, New York, NY, USA (2018). https://doi.org/10.1145/3206004.3206009
Nguyen, H.H., Fang, F., Yamagishi, J., Echizen, I.: Multi-task learning for detecting and segmenting manipulated facial images and videos. In: 2019 IEEE 10th International Conference on Biometrics Theory, Applications and Systems (BTAS), pp. 1–8 (2019). https://doi.org/10.1109/BTAS46853.2019.9185974
Nirkin, Y., Keller, Y., Hassner, T.: FSGAN: subject agnostic face swapping and reenactment. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 7184–7193 (2019)
Petrov, I., et al.: DeepFaceLab: a simple, flexible and extensible face swapping framework. arXiv preprint arXiv:2005.05535 (2020)
Rossler, A., Cozzolino, D., Verdoliva, L., Riess, C., Thies, J., Nießner, M.: FaceForensics++: learning to detect manipulated facial images. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 1–11 (2019)
Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., Chen, L.C.: MobileNetV2: inverted residuals and linear bottlenecks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4510–4520 (2018)
Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014)
Sun, Z., Han, Y., Hua, Z., Ruan, N., Jia, W.: Improving the efficiency and robustness of DeepFakes detection through precise geometric features (2021)
Tariq, S., Lee, S., Kim, H., Shin, Y., Woo, S.S.: Detecting both machine and human created fake face images in the wild. In: Proceedings of the 2nd International Workshop on Multimedia Privacy and Security, pp. 81–87 (2018)
Vasiljevic, I., et al.: DIODE: a dense indoor and outdoor depth dataset. CoRR abs/1908.00463 (2019). http://arxiv.org/abs/1908.00463
Woo, S., et al.: Add: frequency attention and multi-view based knowledge distillation to detect low-quality compressed DeepFake images. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 36, pp. 122–130 (2022)
Yang, X., Li, Y., Lyu, S.: Exposing deep fakes using inconsistent head poses. In: 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). ICASSP 2019, pp. 8261–8265. IEEE (2019)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Mohzary, M., Almalki, K., Choi, BY., Song, S. (2023). CHIEFS: Corneal-Specular Highlights Imaging for Enhancing Fake-Face Spotter. In: Jourdan, GV., Mounier, L., Adams, C., Sèdes, F., Garcia-Alfaro, J. (eds) Foundations and Practice of Security. FPS 2022. Lecture Notes in Computer Science, vol 13877. Springer, Cham. https://doi.org/10.1007/978-3-031-30122-3_10
Download citation
DOI: https://doi.org/10.1007/978-3-031-30122-3_10
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-30121-6
Online ISBN: 978-3-031-30122-3
eBook Packages: Computer ScienceComputer Science (R0)