Abstract
In this work, we will develop a simple convolutional neural network to detect deepfakes in videos on a frame-by-frame level, focusing on the region around the eyes. Since deepfakes are increasingly being created using forms of CNN, it should be possible to also detect deepfakes using CNN. OpenCV allows for frame extraction from videos, while also allowing image cropping. The well-developed Multitask Cascade Neural Network (MTCNN) is a stacked neural network for face detection and alignment. MTCNN is used for high accuracy face detection to greatly reduce false positive images in the dataset. Finally, a region around both eyes are cropped, with extra padding, to be used as input to train a CNN, using returned coordinates from MTCNN for the eyes. This research will focus on measuring if the eye region can be a useful area of interest for comparing original videos to deepfake videos.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Rössler, A., et al.: FaceForensics++: learning to detect manipulated facial images. In:International Conference on Computer Vision (ICCV) (2019)
Burroughs, S., Roy, K., Gokaraju, B., Luu, K.: Detection analysis of deepfake technology by reverse engineering approach (DREA) of feature matching. In: Agrawal, S., Kumar Gupta, K., H. Chan, J., Agrawal, J., Gupta, M. (eds.) Machine Intelligence and Smart Systems. AIS, pp. 421–430. Springer, Singapore (2021). https://doi.org/10.1007/978-981-33-4893-6_36
Wodajo, D., Atnafu, S.: Deepfake video detection using convolutional vision transformer, 11 March 2021. https://arxiv.org/pdf/2102.11126.pdf
Dufour, N., Gully, A., Jigsaw: Contributing data to deepfake detection research. Google Blog. Google, 24 September 2019. https://ai.googleblog.com/2019/09/contributing-data-to-deepfake-detection.html
Mo, H., Chen, B., Luo, W.: Fake faces identification via convolutional neural network. In: Proceedings of the 6th ACM Workshop on Information Hiding and Multimedia Security, IH&MMSec 2018, pp. 43–47. Association for Computing Machinery, Innsbruck (2018)
derpfakes. Nicolas Cage | Mega Mix Two | Derpfakes. YouTube. 02 February 2019. https://www.youtube.com/watch?v=_Kuf1DLcXeo
Buzzfeed Video, Jordan Peele. You Won’t Believe What Obama Says in This Video! YouTube, 17 April 2018. https://www.youtube.com/watch?v=cQ54GDm1eL0
Karandikar, A., Deshpande, V., Singh, S., Nagbhidkar, S., Agrawal, S.: Deepfake video detection using convolutional neural network. In: International Journal of Advanced Trends in Computer Science and Engineering (2020)
Python. https://www.python.org/
OpenCV. https://opencv.org/
Zhang, K., Zhang, Z., Li, Z., Qiao, Y.: Joint face detection and alignment using multi-task cascaded convolutional networks (2016)
Tensorflow. https://www.tensorflow.org/
Rossler, A., et al.: FaceForensics++: learning to detect manipulated facial images. In: The CVF (2019). https://openaccess.thecvf.com/content_ICCV_2019/papers/Rossler_FaceForensics_Learning_to_Detect_Manipulated_Facial_Images_ICCV_2019_paper.pdf
Dataset, FaceForensics++. GitHub. https://github.com/ondyari/FaceForensics
Acknowledgements
This research is supported by National Science Foundation (NSF). Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of NSF.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 Springer Nature Switzerland AG
About this paper
Cite this paper
Johnson, D., Gwyn, T., Qingge, L., Roy, K. (2022). Deepfake Detection Using CNN Trained on Eye Region. In: Fujita, H., Fournier-Viger, P., Ali, M., Wang, Y. (eds) Advances and Trends in Artificial Intelligence. Theory and Practices in Artificial Intelligence. IEA/AIE 2022. Lecture Notes in Computer Science(), vol 13343. Springer, Cham. https://doi.org/10.1007/978-3-031-08530-7_37
Download citation
DOI: https://doi.org/10.1007/978-3-031-08530-7_37
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-08529-1
Online ISBN: 978-3-031-08530-7
eBook Packages: Computer ScienceComputer Science (R0)