Advertisement

A Robust Face Recognition Method Combining LBP with Multi-mirror Symmetry for Images with Various Face Interferences

  • Shui-Guang Tong
  • Yuan-Yuan Huang
  • Zhe-Ming TongEmail author
Research Article

Abstract

Face recognition (FR) is a practical application of pattern recognition (PR) and remains a compelling topic in the study of computer vision. However, in real-world FR systems, interferences in images, including illumination condition, occlusion, facial expression and pose variation, make the recognition task challenging. This study explored the impact of those interferences on FR performance and attempted to alleviate it by taking face symmetry into account. A novel and robust FR method was proposed by combining multi-mirror symmetry with local binary pattern (LBP), namely multi-mirror local binary pattern (MMLBP). To enhance FR performance with various interferences, the MMLBP can 1) adaptively compensate lighting under heterogeneous lighting conditions, and 2) generate extracted image features that are much closer to those under well-controlled conditions (i.e., frontal facial images without expression). Therefore, in contrast with the later variations of LBP, the symmetrical singular value decomposition representation (SSVDR) algorithm utilizing the facial symmetry and a state-of-art non-LBP method, the MMLBP method is shown to successfully handle various image interferences that are common in FR applications without preprocessing operation and a large number of training images. The proposed method was validated with four public data sets. According to our analysis, the MMLBP method was demonstrated to achieve robust performance regardless of image interferences.

Keywords

Face recognition (FR) local binary pattern (LBP) facial symmetry image interferences multi-mirror average 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Notes

Acknowledgements

The work was supported by National Natural Science Foundation of China (No. 51305392), Youth Funds of the State Key Laboratory of Fluid Power Transmission and Control (No. SKLoFP_QN_1501), Zhejiang Provincial Natural Science Foundation of China (Nos. LY17E050009 and LZ15E050001), and the Fundamental Rsesearch Funds for the Central Universities (No. 2018QNA4008).

References

  1. [1]
    J. V. Haxby, M. I. Gobbini, M. L. Furey, A. Ishai, J. L. Schouten, P. Pietrini. Distributed and overlapping representations of faces and objects in ventral temporal cortex. Science, vol. 293, no. 5539, pp. 2425–2430, 2001. DOI:  https://doi.org/10.1126/science.1063736.CrossRefGoogle Scholar
  2. [2]
    X. F. He, S. C. Yan, Y. X. Hu, P. Niyogi, H. J. Zhang. Face recognition using Laplacianfaces. IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 27, no. 3, pp. 328–340, 2005. DOI:  https://doi.org/10.1109/TPAMI.2005.55.CrossRefGoogle Scholar
  3. [3]
    J. Wright, A. Y. Yang, A. Ganesh, S. S. Sastry, Y. Ma. Robust face recognition via sparse representation. IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 31, no. 2, pp. 210–227, 2009. DOI:  https://doi.org/10.1109/TPAMI.2008.79.CrossRefGoogle Scholar
  4. [4]
    I. Naseem, R. Togneri, M. Bennamoun. Linear regression for face recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 32, no. 11, pp. 2106–2112, 2010. DOI:  https://doi.org/10.1109/TPAMI.2010.128.CrossRefGoogle Scholar
  5. [5]
    L. S. Qiao, S. C. Chen, X. Y. Tan. Sparsity preserving projections with applications to face recognition. Pattern Recognition, vol. 43, no. 1, pp. 331–341, 2010. DOI:  https://doi.org/10.1016/j.patcog.2009.05.005.CrossRefzbMATHGoogle Scholar
  6. [6]
    H. S. Du, Q. P. Hu, D. F. Qiao, I. Pitas. Robust face recognition via low–rank sparse representation–based classification. International Journal of Automation and Computing, vol. 12, no. 6, pp. 579–587, 2015. DOI:  https://doi.org/10.1007/s11633-015-0901-2.CrossRefGoogle Scholar
  7. [7]
    X. Geng, Z. H. Zhou, K. Smith–Miles. Automatic age estimation based on facial aging patterns. IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 29, no. 12, pp. 2234–2240, 2007. DOI:  https://doi.org/10.1109/TPAMI.2007.70733.CrossRefGoogle Scholar
  8. [8]
    U. Park, Y. Y. Tong, A. K. Jain. Age–invariant face recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 32, no. 5, pp. 947–954, 2010. DOI:  https://doi.org/10.1109/TPAMI.2010.14.CrossRefGoogle Scholar
  9. [9]
    X. Geng, C. Yin, Z. H. Zhou. Facial age estimation by learning from label distributions. IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 35, no. 10, pp. 2401–2412, 2013. DOI:  https://doi.org/10.1109/TPAMI.2013.51.CrossRefGoogle Scholar
  10. [10]
    K. Jia, S. G. Gong. Hallucinating multiple occluded face images of different resolutions. Pattern Recognition Letters, vol. 27, no. 15, pp. 1768–1775, 2006. DOI:  https://doi.org/10.1016/j.patrec.2006.02.009.CrossRefGoogle Scholar
  11. [11]
    C. X. Ren, D. Q. Dai, H. Yan. Coupled kernel embedding for low–resolution face image recognition. IEEE Transactions on Image Processing, vol. 21, no. 8, pp. 3770–3783, 2012. DOI:  https://doi.org/10.1109/TIP.2012.2192285.MathSciNetCrossRefzbMATHGoogle Scholar
  12. [12]
    W. W. W. Zou, P. C. Yuen. Very low resolution face recognition problem. IEEE Transactions on Image Processing, vol. 21, no. 1, pp. 327–340, 2012. DOI:  https://doi.org/10.1109/TIP.2011.2162423.MathSciNetCrossRefzbMATHGoogle Scholar
  13. [13]
    N. Alyuz, B. Gokberk, L. Akarun. Regional registration for expression resistant 3D face recognition. IEEE Transactions on Information Forensics and Security, vol. 5, no. 3, pp. 425–440, 2010. DOI:  https://doi.org/10.1109/TIFS.2010.2054081.CrossRefGoogle Scholar
  14. [14]
    H. Drira, B. Ben Amor, A. Srivastava, M. Daoudi, R. Slama. 3D face recognition under expressions, occlusions, and pose variations. IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 35, no. 9, pp. 2270–2283, 2013. DOI:  https://doi.org/10.1109/TPAMI.2013.48.CrossRefGoogle Scholar
  15. [15]
    F. K. Zaman, A. A. Shafie, Y. M. Mustafah. Robust face recognition against expressions and partial occlusions. International Journal of Automation and Computing, vol. 13, no. 4, pp. 319–337, 2016. DOI:  https://doi.org/10.1007/s11633-016-0974-6.CrossRefGoogle Scholar
  16. [16]
    X. Z. Zhang, Y. S. Gao. Face recognition across pose: A review. Pattern Recognition, vol. 42, no. 11, pp. 2876–2896, 2009. DOI:  https://doi.org/10.1016/j.patcog.2009.04.017.CrossRefGoogle Scholar
  17. [17]
    G. Passalis, P. Perakis, T. Theoharis, I. A. Kakadiaris. Using facial symmetry to handle pose variations in real–world 3D face recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 33, no. 10, pp. 1938–1951, 2011. DOI:  https://doi.org/10.1109/TPAMI.2011.49.CrossRefGoogle Scholar
  18. [18]
    H. C. Zhang, Y. N. Zhang, T. S. Huang. Pose–robust face recognition via sparse representation. Pattern Recognition, vol.46, no. 5, pp. 1511–1521, 2013. DOI:  https://doi.org/10.1016/j.patcog.2012.10.025.Google Scholar
  19. [19]
    K. C. Lee, J. Ho, D. J. Kriegman. Acquiring linear subspaces for face recognition under variable lighting. IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 27, no. 5, pp. 684–698, 2005. DOI:  https://doi.org/10.1109/TPAMI.2005.92.CrossRefGoogle Scholar
  20. [20]
    W. C. Kao, M. C. Hsu, Y. Y. Yang. Local contrast enhancement and adaptive feature extraction for illumination–invariant face recognition. Pattern Recognition, vol.43, no. 5, pp. 1736–1747, 2010. DOI:  https://doi.org/10.1016/j.patcog.2009.11.016.Google Scholar
  21. [21]
    X. Y. Tan, B. Triggs. Enhanced local texture feature sets for face recognition under difficult lighting conditions. IEEE Transactions on Image Processing, vol. 19, no. 6, pp. 1635–1650, 2010. DOI:  https://doi.org/10.1109/TIP.2010.2042645.MathSciNetCrossRefzbMATHGoogle Scholar
  22. [22]
    V. Struc, J. Zibert, N. Pavesic. Histogram remapping as a preprocessing step for robust face recognition. WSEAS Transactions on Information Science and Applications, vol. 6, no. 3, pp. 520–529, 2009.Google Scholar
  23. [23]
    P. H. Lee, S. W. Wu, Y. P. Hung. Illumination compensation using oriented local histogram equalization and its application to face recognition. IEEE Transactions on Image Processing, vol. 21, no. 9, pp. 4280–4289, 2012. DOI:  https://doi.org/10.1109/TIP.2012.2202670.MathSciNetCrossRefzbMATHGoogle Scholar
  24. [24]
    Y. Cheng, Y. K. Hou, C. X. Zhao, Z. Y. Li, Y. Hu, C. L. Wang. Robust face recognition based on illumination invariant in nonsubsampled contourlet transform domain. Neurocomputing, vol. 73, no. 10–12, pp. 2217–2224, 2010. DOI:  https://doi.org/10.1016/j.neucom.2010.01.012.CrossRefGoogle Scholar
  25. [25]
    Y. Adini, Y. Moses, S. Ullman. Face recognition: The problem of compensating for changes in illumination direction. IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 19, no. 7, pp. 721–732, 1997. DOI:  https://doi.org/10.1109/34.598229.CrossRefGoogle Scholar
  26. [26]
    A. S. Georghiades, P. N. Belhumeur, and D. J. Kriegman. From few to many: Illumination cone models for face recognition under variable lighting and pose. IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 23, no. 6, pp. 643–660, 2001. DOI:  https://doi.org/10.1109/34.927464.CrossRefGoogle Scholar
  27. [27]
    J. Y. Zhu, W. S. Zheng, F. Lu, J. H. Lai. Illumination invariant single face image recognition under heterogeneous lighting condition. Pattern Recognition, vol. 66, pp. 313–327, 2017. DOI:  https://doi.org/10.1016/j.patcog.2016.12.029.CrossRefGoogle Scholar
  28. [28]
    A. Shashua, T. Riklin–Raviv. The quotient image: Classbased re–rendering and recognition with varying illuminations. IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 23, no. 2, pp. 129–139, 2001. DOI:  https://doi.org/10.1109/34.908964.CrossRefGoogle Scholar
  29. [29]
    H. T. Wang, S. Z. Li, Y. S. Wang. Face recognition under varying lighting conditions using self quotient image. In Proceedings of the 6th IEEE International Conference on Automat ic Face and Gesture Recognition, IEEE, Seoul, South Korea, pp. 819–824, 2004. DOI:  https://doi.org/10.1109/AFGR.2004.1301635.Google Scholar
  30. [30]
    T. P. Zhang, Y. Y. Tang, B. Fang, Z. W. Shang, X. Y. Liu. Face recognition under varying illumination using gradientfaces. IEEE Transactions on Image Processing, vol. 18, no. 11, pp. 2599–2606, 2009. DOI:  https://doi.org/10.1109/TIP.2009.2028255.MathSciNetCrossRefzbMATHGoogle Scholar
  31. [31]
    J. Kim, J. Choi, J. Yi, M. Turk. Effective representation using ICA for face recognition robust to local distortion and partial occlusion. IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 27, no. 12, pp. 1977–1981, 2005. DOI:  https://doi.org/10.1109/TPAMI.2005.242.CrossRefGoogle Scholar
  32. [32]
    K. Wang, X. X. Long, R. F. Li, L. J. Zhao. A discriminative algorithm for indoor place recognition based on clustering of features and images. International Journal of Automation and Computing, vol. 14, no. 4, pp. 407–419, 2017. DOI:  https://doi.org/10.1007/s11633-017-1081-z.CrossRefGoogle Scholar
  33. [33]
    Y. Tai, J. Yang, Y. G. Zhang, L. Luo, J. J. Qian, Y. Chen. Face recognition with pose variations and misalignment via orthogonal Procrustes regression. IEEE Transactions on Image Processing, vol. 25, no. 6, pp. 2673–2683, 2016. DOI:  https://doi.org/10.1109/TIP.2016.2551362.MathSciNetCrossRefGoogle Scholar
  34. [34]
    B. Zhao, J. S. Feng, X. Wu, S. C. Yan. A survey on deep learning–based fine–grained object classification and semantic segmentation. International Journal of Automation and Computing, vol. 14, no. 2, pp. 119–135, 2017. DOI:  https://doi.org/10.1007/s11633-017-1053-3.CrossRefGoogle Scholar
  35. [35]
    U. Prabhu, J. Heo, M. Savvides. Unconstrained pose–invariant face recognition using 3D generic elastic models. IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 33, no. 10, pp. 1952–1961, 2011. DOI:  https://doi.org/10.1109/TPAMI.2011.123.CrossRefGoogle Scholar
  36. [36]
    M. Merras, S. El Hazzat, A. Saaidi, K. Satori, A. G. Nazih. 3D face reconstruction using images from cameras with varying parameters. International Journal of Automation and Computing, vol. 14, no. 6, pp. 661–671, 2017. DOI:  https://doi.org/10.1007/s11633-016-0999-x.CrossRefGoogle Scholar
  37. [37]
    T. Ojala, M. Pietikäinen, D. Harwood. A comparative study of texture measures with classification based on feature distributions. Pattern Recognition, vol. 29, no. 1, pp. 51–59, 1996. DOI:  https://doi.org/10.1016/0031-3203(95)00067-4.CrossRefGoogle Scholar
  38. [38]
    T. Ahonen, A. Hadid, M. Pietikäinen. Face description with local binary patterns: Application to face recognition. IEEE Transactions on Pattern Analysis & Machine Intelligence, vol. 28, no. 12, pp. 2037–2041, 2006. DOI:  https://doi.org/10.1109/TPAMI.2006.244.CrossRefzbMATHGoogle Scholar
  39. [39]
    T. Ahonen, A. Hadid, M. Pietikäinen. Face recognition with local binary patterns. In Proceedings of European Conference on Computer Vision, Prague, Czech Republic, pp. 469–481, 2004. DOI:  https://doi.org/10.1007/978-3-540-24670-1-36.Google Scholar
  40. [40]
    H. L. Jin, Q. S. Liu, H. Q. Lu, X. F. Tong. Face detection using improved LBP under bayesian framework. In Proceedings of the 3rd International Conference on Image and Graphics, IEEE, Hong Kong, China, pp. 306–309, 2004. DOI:  https://doi.org/10.1109/ICIG.2004.62.Google Scholar
  41. [41]
    S. C. Liao, X. X. Zhu, Z. Lei, L. Zhang, S. Z. Li. Learning multi–scale block local binary patterns for face recognition. In Proceedings of International Conference on Biometrcs, Seoul, Korea, pp. 828–837, 2007. DOI:  https://doi.org/10.1007/978-3-540-74549-5-87.Google Scholar
  42. [42]
    Y. H. Chen, S. G. Tong, F. Y. Cong, J. Xu. Symmetrical singular value decomposition representation for pattern recognition. Neurocomputing, vol. 214, pp. 143–154, 2016. DOI:  https://doi.org/10.1016/j.neucom.2016.05.075.CrossRefGoogle Scholar
  43. [43]
    C. X. Ding, J. Choi, D. C. Tao, L. S. Davis. Multi–directional multi–level dual–cross patterns for robust face recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 38, no. 3, pp. 518–531, 2016. DOI:  https://doi.org/10.1109/TPAMI.2015.2462338.CrossRefGoogle Scholar
  44. [44]
    P. J. Phillips, H. Moon, S. A. Rizvi, P. J. Rauss. The FERET evaluation methodology for face–recognition algorithms. IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 22, no. 10, pp. 1090–1104, 2000. DOI:  https://doi.org/10.1109/34.879790.CrossRefGoogle Scholar
  45. [45]
    B. Yang, S. C. Chen. A comparative study on local binary pattern (LBP) based face recognition: LBP histogram versus LBP image. Neurocomputing, vol. 120, pp. 365–379, 2013. DOI:  https://doi.org/10.1016/j.neucom.2012.10.032.CrossRefGoogle Scholar
  46. [46]
    Z. Wang, A. C. Bovik, H. R. Sheikh, E. P. Simoncelli. Image quality assessment: From error visibility to structural similarity. IEEE Transactions on Image Processing, vol. 13, no. 4, pp. 600–612, 2004. DOI:  https://doi.org/10.1109/TIP.2003.819861.CrossRefGoogle Scholar

Copyright information

© Institute of Automation, Chinese Academy of Sciences and Springer-Verlag GmbH Germany, part of Springer Nature 2018

Authors and Affiliations

  1. 1.State Key Laboratory of Fluid Power and Mechatronic SystemsZhejiang UniversityHangzhouChina
  2. 2.School of Mechanical EngineeringZhejiang UniversityHangzhouChina

Personalised recommendations