Skip to main content
SpringerLink
Log in

Masked Face Recognition Using Generative Adversarial Networks by Restoring the Face Closed Part

  • Published:
Optical Memory and Neural Networks Aims and scope Submit manuscript

Abstract

In recent years, many authors intensively develop systems allowing one to identify a person when something (a mask) covers a large part of his face. Most of the existing approaches use different forms of analysis of the visible facial features and apply the obtained results to solve the problem. In this article, we propose a fundamentally new approach based on the image segmentation to erase the mask from the face. After erasing the mask, we restore the image of the face under the mask and take an advantage of the existing face recognition methods. To reconstruct the covered part of the face we use the generative adversarial networks. We show that with the aid of the proposed approach it is possible to improve the quality of recognition of masked faces. We compare the effectiveness of our approach and the algorithm based on the MobileNetV2 and show that our method improves the recognition accuracy. We give some examples and appropriate recommendations.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1.
Fig. 2.
Fig. 3.
Fig. 4.
Fig. 5.
Fig. 6.
Fig. 7.
Fig. 8.
Fig. 9.

REFERENCES

  1. Ahmad Alzu’bi, Firas Albalas, Tawfik AL-Hadhrami, Lojin Bani Younis and Amjad Bashayreh. Masked Face Recognition Using Deep Learning: A Review. Electronics, 2021, 10 (21), 2666.

  2. Singh, S., Ahuja, U., Kumar, M. et al. Face mask detection using YOLOv3 and faster R-CNN models: COVID-19 environment. Multimed Tools Appl 80, 19753–19768 (2021).

    Article  Google Scholar 

  3. MFR 2021: Masked Face Recognition Competition—https://arxiv.org/pdf/2106.15288.pdf.

  4. Yaaseen Muhammad Saib, Sameerchand Pudaruth—Is Face Recognition with Masks Possible? (IJACSA) International Journal of Advanced Computer Science and Applica-tions, Vol. 12, No. 7, 2021, pp. 43–50.

    Google Scholar 

  5. Burak. Pins Face Recognition. October 30, 2020, from Kaggle: https://www.kaggle.com/hereisburak/pins-face-recognition.

  6. Pedro C. Neto, Fadi Boutros, Joao Ribeiro Pinto, Mohsen Saffari, Naser Damer, Ana F. Sequeira, Jaime S. Cardoso—My Eyes Are Up Here: Promoting Focus on Uncovered Re-gions in Masked Face Recognition. arXiv:2108.00996v3 [cs.CV] 18 Aug 2021.

  7. Pedro C. Neto, Fadi Boutros, Joao Ribeiro Pinto, Naser Damer, Ana F. Sequeira and Jaime S. Cardoso—FocusFace: Multi-task Contrastive Learning for Masked Face Recog-nition. arXiv:2110.14940v2 [cs.CV] 1 Nov 2021.

  8. Baojin Huang, Zhongyuan Wang, Guangcheng Wang, Kui Jiang, Zheng He, Hua Zou, Qin Zou: Masked Face Recognition Datasets and Validation. In Proceedings of the Inter-national Conference on Computer Vision Workshops, 2021, p. 1487–1491.

  9. Hariri, W. Efficient masked face recognition method during the COVID-19 pandem-ic. Signal, Image and Video Processing, volume 16, p. 605–612 (2022).

    Article  Google Scholar 

  10. Vu, H.N., Nguyen, M.H. & Pham, C. Masked face recognition with convolutional neural networks and local binary patterns. Appl Intell 52, 5497–5512 (2022).

    Article  Google Scholar 

  11. Prasad, S.; Li, Y.; Lin, D.; Sheng, D. MaskedFaceNet: A Progressive Semi-Supervised Masked Face Detector. In Proceedings of the IEEE/CVF Winter Conference on Applica-tions of Computer Vision, Waikoloa, HI, USA, 3–8 January 2021; pp. 3389–3398.

  12. Sikha, O.K., Bharath, B. VGG16-random Fourier hybrid model for masked face recognition. Soft Computing (2022). Published: 10 July 2022. https://doi.org/10.1007/s00500-022-07289-0

  13. Olaf Ronneberger, Philipp Fischer, and Thomas Brox-U-Net: Convolutional Networks for Biomedical Image Segmentation. arXiv:1505.04597v1 [cs.CV] 18 May 2015.

  14. Dong Yi, Zhen Lei, Shengcai Liao and Stan Z. Li—Learning Face Representation from Scratch. arXiv:1411.7923v1 [cs.CV] 28 November 2014.

  15. Ke Gong, Xiaodan Liang, Yicheng Li, Yimin Chen, Ming Yang, and Liang Lin—Instance-level Human Parsing via Part Grouping Network. arXiv:1808.00157v1 [cs.CV] 1 Aug 2018.

  16. AIM 2020 Challenge on Image Extreme Inpainting. arXiv:2010.01110v1 [cs.CV] 2 Oct 2020.

  17. Hui, Z., Li, J., Wang, X., Gao, X.: Image fine-grained inpainting. arXiv:2002.02609v1 [cs.CV] 7 Feb 2020.

Download references

Funding

The work was partially supported by Public Welfare Technology Applied Research Program of Zhejiang Province (LGF19F020016) and the National High-end Foreign Experts Program (G2021016028L, G2021016002L, G2021016001L).

Author information

Authors and Affiliations

  1. School of Information Science and Technology, Zhejiang Shuren University, 310015, Hangzhou, China

    Chaoxiang Chen

  2. International Science and Technology Cooperation Base of Zhejiang Province: Remote Sensing Image Processing and Application, 310015, Hangzhou, China

    Chaoxiang Chen

  3. Belarusian State University, 220030, Minsk, Republic of Belarus

    I. Kurnosov & S. Ablameyko

  4. EarthView Image Inc., 313200, Huzhou, China

    Guangdi Ma & Yang Weichen

  5. United Institute for Informatics Problems, National Academy of Sciences of Belarus, 220012, Minsk, Republic of Belarus

    S. Ablameyko

Authors
  1. Chaoxiang Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. I. Kurnosov
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Guangdi Ma
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yang Weichen
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. S. Ablameyko
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to S. Ablameyko.

Ethics declarations

COMPLIANCE WITH ETHICAL STANDARDS

This article is a completely original work of its authors; it has not been published before and will not be sent to other publications until the PRIA editorial board decides not to accept it for publication. Informed consent was obtained from all individual participants involved in the study.

CONFLICT OF INTERESTS

The process of writing and the content of the article does not give grounds for raising the issue of a conflict of interest.

The authors declare that they have no conflicts of interest.

ETHICS DECLARATIONS

All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki Declaration and its later amendments or comparable ethical standards.

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chen, C., Kurnosov, I., Ma, G. et al. Masked Face Recognition Using Generative Adversarial Networks by Restoring the Face Closed Part. Opt. Mem. Neural Networks 32, 1–13 (2023). https://doi.org/10.3103/S1060992X23010022

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.3103/S1060992X23010022

Keywords:

Search

Navigation