Advertisement

Image gradient orientations embedded structural error coding for face recognition with occlusion

  • Xiao-Xin LiEmail author
  • Pengyi Hao
  • Lin He
  • Yuanjing Feng
Original Research

Abstract

Partially occluded faces are very common in automatic face recognition (FR) in the real world. We explore the problem of FR with occlusion by embedding Image Gradient Orientations (IGO) into robust error coding. The existing works usually put stress on the error distribution in the non-occluded region but neglect the one in the occluded region due to its unpredictability incurred by irregular occlusion. However, in the IGO domain, the error distribution in the occluded region can be built simply and elegantly by a uniform distribution in the interval \(\left[ -\pi ,\pi \right)\), and the one in the occluded region can be well built by a weight-conditional Gaussian distribution. By incorporating the two error distributions and a Markov random field for the priori distribution of the occlusion support, we propose a joint probabilistic generative model for a novel IGO-embedded Structural Error Coding (IGO-SEC) model. Two methods, a new reconstruction method and a new robust structural error metric, are further presented to boost the performance of IGO-SEC. Extensive experiments on 8 popular robust FR methods and 4 benchmark face databases demonstrate the effectiveness and robustness of IGO-SEC in dealing with facial occlusion and occlusion-like variations.

Keywords

Unconstrained face recognition Face occlusion Image gradient orientations Structural error coding Markov random field 

Notes

Acknowledgements

This work is partially supported by Natural Science Foundation of Zhejiang Province (LY18F020031), National Natural Science Foundation of China (61379020, 61402411, 61802347).

References

  1. Ahonen T, Hadid A, Pietikainen M (2006) Face description with local binary patterns: application to face recognition. IEEE Trans Pattern Anal Mach Intell 28:2037–2041CrossRefzbMATHGoogle Scholar
  2. Chan TH, Jia K, Gao S, Lu J, Zeng Z, Ma Y (2015) PCANet: a simple deep learning baseline for image classification? IEEE Trans Image Process 24:5017–5032MathSciNetCrossRefzbMATHGoogle Scholar
  3. Chen SS, Donoho DL, Saunders MA (2001) Atomic decomposition by basis pursuit. SIAM Rev 43:129–159MathSciNetCrossRefzbMATHGoogle Scholar
  4. Colombo A, Cusano C, Schettini R (2011) UMB-DB: a database of partially occluded 3d faces. In: IEEE International Conference on Computer Vision Workshops, Barcelona, Spain, pp 2113–2119Google Scholar
  5. Georghiades AS, Belhumeur PN, Kriegman DJ (2001) From few to many: illumination cone models for face recognition under variable lighting and pose. IEEE Trans Pattern Anal Mach Intell 23:643–660CrossRefGoogle Scholar
  6. Ghazi MM, Ekenel HK (2016) A comprehensive analysis of deep learning based representation for face recognition. In: IEEE Conference on Computer Vision and Pattern Recognition Workshops. Las Vegas, USA, pp 34–41Google Scholar
  7. He R, Zheng W, Hu B (2011) Maximum correntropy criterion for robust face recognition. IEEE Trans Pattern Anal Mach Intell 33:1561–1576CrossRefGoogle Scholar
  8. He R, Zheng WS, Tan T, Sun Z (2014) Half-quadratic-based iterative minimization for robust sparse representation. IEEE Trans Pattern Anal Mach Intell 36:261–275CrossRefGoogle Scholar
  9. Hua G, Yang MH, Learned-Miller E, Ma Y, Turk M, Kriegman DJ, Huang TS (2011) Introduction to the special section on real-world face recognition. IEEE Trans Pattern Anal Mach Intell 33:1921–1924CrossRefGoogle Scholar
  10. Huang GB, Mattar M, Berg T, Learned-Miller E (2007) Labeled faces in the wild: a database for studying face recognition in unconstrained environments. Technical report 07–49, University of MassachusettsGoogle Scholar
  11. Huang GB, Mattar M, Lee H, Learned-Miller E (2012) Learning to align from scratch. Advances in neural information processing systems. Lake Tahoe, Nevada, USA, pp 764–772Google Scholar
  12. Kolmogorov V, Zabih R (2004) What energy functions can be minimized via graph cuts? IEEE Trans Pattern Anal Mach Intell 26:147–159CrossRefzbMATHGoogle Scholar
  13. Lee K, Ho J, Kriegman D (2005) Acquiring linear subspaces for face recognition under variable lighting. IEEE Trans Pattern Anal Mach Intell 27:684–698CrossRefGoogle Scholar
  14. Li XX, Dai DQ, Zhang XF, Ren CX (2013) Structured sparse error coding for face recognition with occlusion. IEEE Trans Image Process 22:1889–1900MathSciNetCrossRefzbMATHGoogle Scholar
  15. Li XX, He L, Hao P, Liu Z, Li J (2017) Adaptive weberfaces for occlusion-robust face representation and recognition. IET Image Process 11:964–975CrossRefGoogle Scholar
  16. Liang R, Li XX (2015) Mixed error coding for face recognition with mixed occlusions. In: International Joint Conference on Artificial Intelligence. Buenos Aires, Argentina, pp 3657–3663Google Scholar
  17. Lin D, Tang X (2007) Quality-driven face occlusion detection and recovery. In: IEEE Conference on Computer Vision and Pattern Recognition. Minneapolis, USA, pp 1–7Google Scholar
  18. Liu W, Pokharel PP, Principe JC (2007) Correntropy: properties and applications in non-gaussian signal processing. IEEE Trans Signal Process 55:5286–5298MathSciNetCrossRefzbMATHGoogle Scholar
  19. Luo L, Yang J, Qian J, Tai Y (2015) Nuclear-l1 norm joint regression for face reconstruction and recognition with mixed noise. Pattern Recog 48:3811–3824CrossRefGoogle Scholar
  20. Martinez AM (1998) The AR face database. Technical Report 24, Computer Visual Center, Ohio State UniversityGoogle Scholar
  21. Parkhi OM, Vedaldi A, Zisserman A (2015) Deep face recognition. In: British Machine Vision Conference, London: BMVA Press, Swansea, UK, pp 41.1–41.12Google Scholar
  22. Portugal L, Judice J, Vicente L (1994) A comparison of block pivoting and interior-point algorithms for linear least squares problems with nonnegative variables. Math Comput 63:625–644MathSciNetCrossRefzbMATHGoogle Scholar
  23. Qian J, Yang J, Xu Y (2013) Local structure-based image decomposition for feature extraction with applications to face recognition. IEEE Trans Image Process 22:3591–3603CrossRefGoogle Scholar
  24. Qian J, Luo L, Yang J, Zhang F, Lin Z (2015) Robust nuclear norm regularized regression for face recognition with occlusion. Pattern Recog 48:3145–3159CrossRefGoogle Scholar
  25. Sun Y, Wang X, Tang X (2015) Deeply learned face representations are sparse, selective, and robust. In: IEEE Conference on Computer Vision and Pattern Recognition. MA, USA, Boston, pp 2892–2900Google Scholar
  26. Sun Y, Wang X, Tang X (2016) Sparsifying neural network connections for face recognition. In: IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, USA, pp 4856–4864Google Scholar
  27. Taigman Y, Yang M, Ranzato M, Wolf L (2015) Web-scale training for face identification. In: IEEE Conference on Computer Vision and Pattern Recognition, Boston, USA, pp 2746–2754Google Scholar
  28. Tzimiropoulos G, Zafeiriou S, Pantic M (2012) Subspace learning from image gradient orientations. IEEE Trans Pattern Anal Mach Intell 34:2454–66CrossRefGoogle Scholar
  29. Wang JJY, Wang (2013) Non-negative matrix factorization by maximizing correntropy for cancer clustering. BMC Bioinform 14:1–11Google Scholar
  30. Wei X, Li CT, Hu Y (2012) Robust face recognition under varying illumination and occlusion considering structured sparsity. In: International conference on digital image computing techniques and applications. Fremantle, Australia, pp 1–7Google Scholar
  31. Wright J, Ma Y (2010) Dense error correction via \(\ell ^1\)-minimization. IEEE Trans Inf Theory 56:3540–3560MathSciNetCrossRefzbMATHGoogle Scholar
  32. Wright J, Yang A, Ganesh A, Sastry S, Ma Y (2009) Robust face recognition via sparse representation. IEEE Trans Pattern Anal Mach Intell 31:210–227CrossRefGoogle Scholar
  33. Wu X, He R, Sun Z (2015) A lightened CNN for deep face representation. CoRR abs/1511.02683. arXiv:1511.02683
  34. Yang J, Luo L, Qian J, Tai Y, Zhang F, Xu Y (2017) Nuclear norm based matrix regression with applications to face recognition with occlusion and illumination changes. IEEE Trans Pattern Anal Mach Intell 39:156–171CrossRefGoogle Scholar
  35. Yang M, Zhang L, Yang J, Zhang D (2011) Robust sparse coding for face recognition. In: IEEE Conference on Computer Vision and Pattern Recognition. Colorado Springs, USA, pp 625–632Google Scholar
  36. Yang M, Zhang L, Shiu SC, Zhang D (2013a) Gabor feature based robust representation and classification for face recognition with gabor occlusion dictionary. Pattern Recog. 46:1865–1878CrossRefGoogle Scholar
  37. Yang M, Zhang L, Shiu SK, Zhang D (2013b) Robust kernel representation with statistical local features for face recognition. IEEE Trans Neural Netw Learn Syst 24:900–912CrossRefGoogle Scholar
  38. Yang M, Zhang L, Yang J, Zhang D (2013c) Regularized robust coding for face recognition. IEEE Trans Image Process 22:1753–1766MathSciNetCrossRefzbMATHGoogle Scholar
  39. Zhang L, Yang M, Feng X (2011) Sparse representation or collaborative representation: Which helps face recognition? In: IEEE International Conference on Computer Vision, Barcelona, Spain, pp 471–478Google Scholar
  40. Zhang T, Tang YY, Fang B, Shang Z, Liu X (2009) Face recognition under varying illumination using gradientfaces. IEEE Trans Image Process 18:2599–2606MathSciNetCrossRefzbMATHGoogle Scholar
  41. Zhao F, Feng J, Zhao J, Yang W, Yan S (2018) Robust lstm-autoencoders for face de-occlusion in the wild. IEEE Trans Image Process 27:778–790MathSciNetCrossRefzbMATHGoogle Scholar
  42. Zhou Z, Wagner A, Mobahi H, Wright J, Ma Y (2009) Face recognition with contiguous occlusion using markov random fields. In: IEEE International Conference on Computer Vision. Kyoto, Japan, pp 1050–1057Google Scholar

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature 2019

Authors and Affiliations

  • Xiao-Xin Li
    • 1
    Email author
  • Pengyi Hao
    • 1
  • Lin He
    • 2
  • Yuanjing Feng
    • 1
  1. 1.College of Computer Science and TechnologyZhejiang University of TechnologyHangzhouPeople’s Republic of China
  2. 2.School of Automation Science and EngineeringSouth China University of TechnologyGuangzhouPeople’s Republic of China

Personalised recommendations