Skip to main content
SpringerLink
Log in

Occlusion recovery face recognition based on information reconstruction

  • Original Paper
  • Published:
Machine Vision and Applications Aims and scope Submit manuscript

Abstract

The face data in the wild have a face occlusion, complex background, and inestimable posture, leading to more sample classification errors. To alleviate the problem of face occlusion, we apply facial information reconstruction to occluded face recognition. Our model processes face images with variable poses and reconstructs face information to compensate for information loss due to occlusion. In this context, we propose an unconstrained face recognition method based on information reconstruction and occlusion dictionary learning. One is that 3D facial information reconstruction makes up for the lost part of self-occluded face images. The other is that the Gabor-based occlusion dictionary learning method increases feature diversity and better represents occluded faces. We conduct experiments on the occluded AR dataset to verify the robustness of occluded dictionary learning. Compared with the classical algorithm, the accuracy of the CAS-PEAL dataset improves by 40.3%. The experimental results of the LFW dataset are 5.57–12.21% higher than those of the state-of-the-art algorithm, which indicates that the proposed method realizes effective occlusion face recognition.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

References

  1. Beck, A., Teboulle, M.: A fast iterative shrinkage-thresholding algorithm for linear inverse problems. SIAM J. Imaging Sci. 2(1), 183–202 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  2. Booth, J., Roussos, A., Zafeiriou, S., Ponniah, A., Dunaway, D.: A 3d morphable model learnt from 10,000 faces. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2016)

  3. Boyd, S., Parikh, N., Hu, E.C., Peleato, B., Ec Kstein, J.: Distributed optimization and statistical learning via the alternating direction method of multipliers. Found. Trends Mach. Learn. 3(1), 1–122 (2010)

    Article  MATH  Google Scholar 

  4. Cai, J.F., Candès, E., Shen, Z.: A singular value thresholding algorithm for matrix completion. SIAM J. Optim. 20(4), 1956–1982 (2018)

    Article  MathSciNet  MATH  Google Scholar 

  5. Cai, S., Zuo, W., Zhang, L., Feng, X., Wang, P.: Support vector guided dictionary learning. In: European Conference on Computer Vision (2014)

  6. Cao, J., Hu, Y., Zhang, H., He, R., Sun, Z.: Towards high fidelity face frontalization in the wild. Int. J. Comput. Vis. 128(5), 1485–1504 (2020)

    Article  Google Scholar 

  7. Cen, F., Wang, G.: Dictionary representation of deep features for occlusion-robust face recognition. IEEE Access 7, 26595–26605 (2019)

    Article  Google Scholar 

  8. Deng, J., Cheng, S., Xue, N., Zhou, Y., Zafeiriou, S.: UV-GAN: Adversarial facial UV map completion for pose-invariant face recognition. In: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (2018)

  9. Gatys, L.A., Ecker, A.S., Bethge, M.: A neural algorithm of artistic style. J. Vis. (2015). https://doi.org/10.1167/16.12.326

    Article  Google Scholar 

  10. Gerig, T., Morel-Forster, A., Blumer, C., Egger, B., Luthi, M., Schoenborn, S., Vetter, T.: Morphable face models—an open framework. In: IEEE International Conference on Automatic Face & Gesture Recognition, pp. 75–82 (2018)

  11. Goodfellow, I.J., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., Bengio, Y.: Generative adversarial networks. Adv. Neural Inf. Process. Syst. 3, 2672–2680 (2014)

    Google Scholar 

  12. Hassner, T., Harel, S., Paz, E., Enbar, R.: Effective face frontalization in unconstrained images. In: 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 4295–4304 (2015). https://doi.org/10.1109/CVPR.2015.7299058

  13. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2016)

  14. Hu, Y., Wu, X., Yu, B., He, R., Sun, Z.: Pose-guided photorealistic face rotation. In: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (2018)

  15. Huang, G.B., Mattar, M., Berg, T., Learned-Miller, E.: Labeled faces in the wild: a database for studying face recognition in unconstrained environments. Month (2008)

  16. Huang, R., Zhang, S., Li, T., He, R.: Beyond face rotation: global and local perception GAN for photorealistic and identity preserving frontal view synthesis. In: 2017 IEEE International Conference on Computer Vision (ICCV) (2017)

  17. Johnson, J., Alahi, A., Fei-Fei, L.: Perceptual losses for real-time style transfer and super-resolution. In: Computer Vision—ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11–14, 2016, Proceedings, Part II 14, pp. 694–711. Springer (2016)

  18. Jiang, Z., Lin, Z., Davis, L.S.: Label consistent K-SVD: learning a discriminative dictionary for recognition. IEEE Trans. Pattern Anal. Mach. Intell. 35(11), 2651–2664 (2013)

    Article  Google Scholar 

  19. Lahasan, B., Lutfi, S.L., San-Segundo, R.: A survey on techniques to handle face recognition challenges: occlusion, single sample per subject and expression. Artif. Intell. Rev. 52, 949–979 (2019)

    Article  Google Scholar 

  20. Ledig, C., Theis, L., Huszar, F., Caballero, J., Cunningham, A., Acosta, A., Aitken, A., Tejani, A., Totz, J., Wang, Z.: Photo-realistic single image super-resolution using a generative adversarial network. In: IEEE Computer Society (2016)

  21. Li, Y., Feng, J.: Reconstruction based face occlusion elimination for recognition. Neurocomputing 101(FEB.4), 68–72 (2013)

    Article  Google Scholar 

  22. Liu, S., Wang, Y., Wu, X., Li, J., Lei, T.: Discriminative dictionary learning algorithm based on sample diversity and locality of atoms for face recognition. J. Vis. Commun. Image Represent. 71, 102763 (2020)

    Article  Google Scholar 

  23. Liu, W., Wen, Y., Yu, Z., Li, M., Raj, B., Song, L.: Sphereface: deep hypersphere embedding for face recognition. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2017)

  24. Liu, Z., Luo, P., Wang, X., Tang, X.: Deep learning face attributes in the wild. In: Proceedings of International Conference on Computer Vision (ICCV) (2015)

  25. Luan, T., Xi, Y., Liu, X.: Disentangled representation learning GAN for pose-invariant face recognition. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2017)

  26. Mairal, J., Bach, F., Ponce, J., Sapiro, G.: Online learning for matrix factorization and sparse coding. J. Mach. Learn. Res. 11(1), 19–60 (2009)

    MathSciNet  MATH  Google Scholar 

  27. Martínez, A., Benavente, R.: The AR face database. CVC Technical Report, vol. 24 (1998)

  28. Meng, Y., Lei, Z., Shiu, S., Zhang, D.: Gabor feature based robust representation and classification for face recognition with Gabor occlusion dictionary. Pattern Recognit. 46(7), 1865–1878 (2013)

    Article  Google Scholar 

  29. Ramirez, I., Sprechmann, P., Sapiro, G.: Classification and clustering via dictionary learning with structured incoherence and shared features. In: 2010 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2010)

  30. Shu, K., Wang, D.: A dictionary learning approach for classification: separating the particularity and the commonality. In: European Conference on Computer Vision (2012)

  31. Song, L., Gong, D., Li, Z., Liu, C., Liu, W.: Occlusion robust face recognition based on mask learning with pairwise differential siamese network. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 773–782 (2019)

  32. Vu, T.H., Monga, V.: Learning a low-rank shared dictionary for object classification. In: 2016 IEEE International Conference on Image Processing (ICIP). IEEE, pp. 4428–4432 (2016)

  33. Wang, C.P., Wei, W., Zhang, J.S., Song, H.B.: Robust face recognition via discriminative and common hybrid dictionary learning. Appl. Intell. Int. J. Artif. Intell. Neural Netw. Complex Probl. Solving Technol. 48, 156–165 (2018)

    Google Scholar 

  34. Wang, M., Hu, Z., Sun, Z., Zhao, S., Sun, M.: Varying face occlusion detection and iterative recovery for face recognition. J. Electron. Imaging 26(3), 033009 (2017)

    Article  Google Scholar 

  35. Wang, X., Yu, K., Wu, S., Gu, J., Liu, Y., Dong, C., Loy, C.C., Qiao, Y., Tang, X.: ESRGAN: enhanced super-resolution generative adversarial networks. In: European Conference on Computer Vision (2018)

  36. Wei, Y., Liu, M., Wang, H., Zhu, R., Hu, G., Zuo, W.: Learning flow-based feature warping for face frontalization with illumination inconsistent supervision. In: European Conference on Computer Vision, pp. 558–574. Springer (2020)

  37. Wen, G., Bo, C., Shiguang, S., Xilin, C., Delong, Z.: The CAS-PEAL large-scale Chinese face database and baseline evaluations. IEEE Trans. Syst. Man Cybern. Part A 38(1), 149–161 (2008)

    Article  Google Scholar 

  38. Wright, J., Ganesh, A., Zhou, Z., Wagner, A., Yi, M.: Demo: robust face recognition via sparse representation. In: 8th IEEE International Conference on Automatic Face & Gesture Recognition, 2008. FG’08 (2008)

  39. Yang, M., Zhang, L., Feng, X., Zhang, D.: Fisher discrimination dictionary learning for sparse representation. In: IEEE International Conference on Computer Vision, ICCV 2011, Barcelona, Spain, November 6–13, 2011 (2011)

  40. Yi, D., Lei, Z., Liao, S., Li, S.Z.: Learning face representation from scratch. arXiv preprint arXiv:1411.7923 (2014)

  41. Yim, J., Jung, H., Yoo, B.I., Choi, C., Kim, J.: Rotating your face using multi-task deep neural network. In: 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2015)

  42. Zeng, D., Veldhuis, R., Spreeuwers, L.: A survey of face recognition techniques under occlusion. IET Biom. 10(6), 581–606 (2021)

    Article  Google Scholar 

  43. Zhang, G., Porikli, F., Sun, H., Sun, Q., Xia, G., Zheng, Y.: Cost-sensitive joint feature and dictionary learning for face recognition. Neurocomputing 391, 177–188 (2020)

    Article  Google Scholar 

  44. Zhang, L., Yang, M., Feng, X.: Sparse representation or collaborative representation: which helps face recognition? In: International Conference on Computer Vision (2012)

  45. Zhao, Z., Shen, Q., Feng, G., Zhu, J.: Collaborative coding and dictionary learning for nearest subspace classification. Soft Comput. 25, 1–17 (2021)

    Article  Google Scholar 

  46. Zhou, H., Liu, J., Liu, Z., Liu, Y., Wang, X.: Rotate-and-render: unsupervised photorealistic face rotation from single-view images (2020)

  47. Zhu, X., Zhen, L., Yan, J., Dong, Y., Li, S.Z.: High-fidelity pose and expression normalization for face recognition in the wild. In: 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2015)

  48. Dhariwal, P., Nichol, A.: Diffusion models beat gans on image synthesis[J]. Adv Neural Inf. Process. Syst. 34, 8780–8794 (2021)

Download references

Acknowledgements

This study is supported by Open Project of Key Laboratory of Ministry of Public Security for Road Traffic Safety, No. 2021ZDSYSKFKT04.

Author information

Authors and Affiliations

  1. School of Computer Science and Artificial Intelligence, Changzhou University, Changzhou, 213164, China

    Huanjie He, Jiuzhen Liang, Zhenjie Hou, Hao Liu & Xinwen Zhou

Authors
  1. Huanjie He
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jiuzhen Liang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zhenjie Hou
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Hao Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Xinwen Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jiuzhen Liang.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

He, H., Liang, J., Hou, Z. et al. Occlusion recovery face recognition based on information reconstruction. Machine Vision and Applications 34, 74 (2023). https://doi.org/10.1007/s00138-023-01423-0

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s00138-023-01423-0

Keywords

Search

Navigation