Skip to main content
SpringerLink
Log in

A Robust Face Recognition Method Combining LBP with Multi-mirror Symmetry for Images with Various Face Interferences

  • Research Article
  • Published:
International Journal of Automation and Computing Aims and scope Submit manuscript

Abstract

Face recognition (FR) is a practical application of pattern recognition (PR) and remains a compelling topic in the study of computer vision. However, in real-world FR systems, interferences in images, including illumination condition, occlusion, facial expression and pose variation, make the recognition task challenging. This study explored the impact of those interferences on FR performance and attempted to alleviate it by taking face symmetry into account. A novel and robust FR method was proposed by combining multi-mirror symmetry with local binary pattern (LBP), namely multi-mirror local binary pattern (MMLBP). To enhance FR performance with various interferences, the MMLBP can 1) adaptively compensate lighting under heterogeneous lighting conditions, and 2) generate extracted image features that are much closer to those under well-controlled conditions (i.e., frontal facial images without expression). Therefore, in contrast with the later variations of LBP, the symmetrical singular value decomposition representation (SSVDR) algorithm utilizing the facial symmetry and a state-of-art non-LBP method, the MMLBP method is shown to successfully handle various image interferences that are common in FR applications without preprocessing operation and a large number of training images. The proposed method was validated with four public data sets. According to our analysis, the MMLBP method was demonstrated to achieve robust performance regardless of image interferences.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. J. V. Haxby, M. I. Gobbini, M. L. Furey, A. Ishai, J. L. Schouten, P. Pietrini. Distributed and overlapping representations of faces and objects in ventral temporal cortex. Science, vol. 293, no. 5539, pp. 2425–2430, 2001. DOI: https://doi.org/10.1126/science.1063736.

    Article  Google Scholar 

  2. X. F. He, S. C. Yan, Y. X. Hu, P. Niyogi, H. J. Zhang. Face recognition using Laplacianfaces. IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 27, no. 3, pp. 328–340, 2005. DOI: https://doi.org/10.1109/TPAMI.2005.55.

    Article  Google Scholar 

  3. J. Wright, A. Y. Yang, A. Ganesh, S. S. Sastry, Y. Ma. Robust face recognition via sparse representation. IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 31, no. 2, pp. 210–227, 2009. DOI: https://doi.org/10.1109/TPAMI.2008.79.

    Article  Google Scholar 

  4. I. Naseem, R. Togneri, M. Bennamoun. Linear regression for face recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 32, no. 11, pp. 2106–2112, 2010. DOI: https://doi.org/10.1109/TPAMI.2010.128.

    Article  Google Scholar 

  5. L. S. Qiao, S. C. Chen, X. Y. Tan. Sparsity preserving projections with applications to face recognition. Pattern Recognition, vol. 43, no. 1, pp. 331–341, 2010. DOI: https://doi.org/10.1016/j.patcog.2009.05.005.

    Article  MATH  Google Scholar 

  6. H. S. Du, Q. P. Hu, D. F. Qiao, I. Pitas. Robust face recognition via low–rank sparse representation–based classification. International Journal of Automation and Computing, vol. 12, no. 6, pp. 579–587, 2015. DOI: https://doi.org/10.1007/s11633-015-0901-2.

    Article  Google Scholar 

  7. X. Geng, Z. H. Zhou, K. Smith–Miles. Automatic age estimation based on facial aging patterns. IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 29, no. 12, pp. 2234–2240, 2007. DOI: https://doi.org/10.1109/TPAMI.2007.70733.

    Article  Google Scholar 

  8. U. Park, Y. Y. Tong, A. K. Jain. Age–invariant face recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 32, no. 5, pp. 947–954, 2010. DOI: https://doi.org/10.1109/TPAMI.2010.14.

    Article  Google Scholar 

  9. X. Geng, C. Yin, Z. H. Zhou. Facial age estimation by learning from label distributions. IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 35, no. 10, pp. 2401–2412, 2013. DOI: https://doi.org/10.1109/TPAMI.2013.51.

    Article  Google Scholar 

  10. K. Jia, S. G. Gong. Hallucinating multiple occluded face images of different resolutions. Pattern Recognition Letters, vol. 27, no. 15, pp. 1768–1775, 2006. DOI: https://doi.org/10.1016/j.patrec.2006.02.009.

    Article  Google Scholar 

  11. C. X. Ren, D. Q. Dai, H. Yan. Coupled kernel embedding for low–resolution face image recognition. IEEE Transactions on Image Processing, vol. 21, no. 8, pp. 3770–3783, 2012. DOI: https://doi.org/10.1109/TIP.2012.2192285.

    Article  MathSciNet  MATH  Google Scholar 

  12. W. W. W. Zou, P. C. Yuen. Very low resolution face recognition problem. IEEE Transactions on Image Processing, vol. 21, no. 1, pp. 327–340, 2012. DOI: https://doi.org/10.1109/TIP.2011.2162423.

    Article  MathSciNet  MATH  Google Scholar 

  13. N. Alyuz, B. Gokberk, L. Akarun. Regional registration for expression resistant 3D face recognition. IEEE Transactions on Information Forensics and Security, vol. 5, no. 3, pp. 425–440, 2010. DOI: https://doi.org/10.1109/TIFS.2010.2054081.

    Article  Google Scholar 

  14. H. Drira, B. Ben Amor, A. Srivastava, M. Daoudi, R. Slama. 3D face recognition under expressions, occlusions, and pose variations. IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 35, no. 9, pp. 2270–2283, 2013. DOI: https://doi.org/10.1109/TPAMI.2013.48.

    Article  Google Scholar 

  15. F. K. Zaman, A. A. Shafie, Y. M. Mustafah. Robust face recognition against expressions and partial occlusions. International Journal of Automation and Computing, vol. 13, no. 4, pp. 319–337, 2016. DOI: https://doi.org/10.1007/s11633-016-0974-6.

    Article  Google Scholar 

  16. X. Z. Zhang, Y. S. Gao. Face recognition across pose: A review. Pattern Recognition, vol. 42, no. 11, pp. 2876–2896, 2009. DOI: https://doi.org/10.1016/j.patcog.2009.04.017.

    Article  Google Scholar 

  17. G. Passalis, P. Perakis, T. Theoharis, I. A. Kakadiaris. Using facial symmetry to handle pose variations in real–world 3D face recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 33, no. 10, pp. 1938–1951, 2011. DOI: https://doi.org/10.1109/TPAMI.2011.49.

    Article  Google Scholar 

  18. H. C. Zhang, Y. N. Zhang, T. S. Huang. Pose–robust face recognition via sparse representation. Pattern Recognition, vol.46, no. 5, pp. 1511–1521, 2013. DOI: https://doi.org/10.1016/j.patcog.2012.10.025.

    MATH  Google Scholar 

  19. K. C. Lee, J. Ho, D. J. Kriegman. Acquiring linear subspaces for face recognition under variable lighting. IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 27, no. 5, pp. 684–698, 2005. DOI: https://doi.org/10.1109/TPAMI.2005.92.

    Article  Google Scholar 

  20. W. C. Kao, M. C. Hsu, Y. Y. Yang. Local contrast enhancement and adaptive feature extraction for illumination–invariant face recognition. Pattern Recognition, vol.43, no. 5, pp. 1736–1747, 2010. DOI: https://doi.org/10.1016/j.patcog.2009.11.016.

    MATH  Google Scholar 

  21. X. Y. Tan, B. Triggs. Enhanced local texture feature sets for face recognition under difficult lighting conditions. IEEE Transactions on Image Processing, vol. 19, no. 6, pp. 1635–1650, 2010. DOI: https://doi.org/10.1109/TIP.2010.2042645.

    Article  MathSciNet  MATH  Google Scholar 

  22. V. Struc, J. Zibert, N. Pavesic. Histogram remapping as a preprocessing step for robust face recognition. WSEAS Transactions on Information Science and Applications, vol. 6, no. 3, pp. 520–529, 2009.

    Google Scholar 

  23. P. H. Lee, S. W. Wu, Y. P. Hung. Illumination compensation using oriented local histogram equalization and its application to face recognition. IEEE Transactions on Image Processing, vol. 21, no. 9, pp. 4280–4289, 2012. DOI: https://doi.org/10.1109/TIP.2012.2202670.

    Article  MathSciNet  MATH  Google Scholar 

  24. Y. Cheng, Y. K. Hou, C. X. Zhao, Z. Y. Li, Y. Hu, C. L. Wang. Robust face recognition based on illumination invariant in nonsubsampled contourlet transform domain. Neurocomputing, vol. 73, no. 10–12, pp. 2217–2224, 2010. DOI: https://doi.org/10.1016/j.neucom.2010.01.012.

    Article  Google Scholar 

  25. Y. Adini, Y. Moses, S. Ullman. Face recognition: The problem of compensating for changes in illumination direction. IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 19, no. 7, pp. 721–732, 1997. DOI: https://doi.org/10.1109/34.598229.

    Article  Google Scholar 

  26. A. S. Georghiades, P. N. Belhumeur, and D. J. Kriegman. From few to many: Illumination cone models for face recognition under variable lighting and pose. IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 23, no. 6, pp. 643–660, 2001. DOI: https://doi.org/10.1109/34.927464.

    Article  Google Scholar 

  27. J. Y. Zhu, W. S. Zheng, F. Lu, J. H. Lai. Illumination invariant single face image recognition under heterogeneous lighting condition. Pattern Recognition, vol. 66, pp. 313–327, 2017. DOI: https://doi.org/10.1016/j.patcog.2016.12.029.

    Article  Google Scholar 

  28. A. Shashua, T. Riklin–Raviv. The quotient image: Classbased re–rendering and recognition with varying illuminations. IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 23, no. 2, pp. 129–139, 2001. DOI: https://doi.org/10.1109/34.908964.

    Article  Google Scholar 

  29. H. T. Wang, S. Z. Li, Y. S. Wang. Face recognition under varying lighting conditions using self quotient image. In Proceedings of the 6th IEEE International Conference on Automat ic Face and Gesture Recognition, IEEE, Seoul, South Korea, pp. 819–824, 2004. DOI: https://doi.org/10.1109/AFGR.2004.1301635.

    Google Scholar 

  30. T. P. Zhang, Y. Y. Tang, B. Fang, Z. W. Shang, X. Y. Liu. Face recognition under varying illumination using gradientfaces. IEEE Transactions on Image Processing, vol. 18, no. 11, pp. 2599–2606, 2009. DOI: https://doi.org/10.1109/TIP.2009.2028255.

    Article  MathSciNet  MATH  Google Scholar 

  31. J. Kim, J. Choi, J. Yi, M. Turk. Effective representation using ICA for face recognition robust to local distortion and partial occlusion. IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 27, no. 12, pp. 1977–1981, 2005. DOI: https://doi.org/10.1109/TPAMI.2005.242.

    Article  Google Scholar 

  32. K. Wang, X. X. Long, R. F. Li, L. J. Zhao. A discriminative algorithm for indoor place recognition based on clustering of features and images. International Journal of Automation and Computing, vol. 14, no. 4, pp. 407–419, 2017. DOI: https://doi.org/10.1007/s11633-017-1081-z.

    Article  Google Scholar 

  33. Y. Tai, J. Yang, Y. G. Zhang, L. Luo, J. J. Qian, Y. Chen. Face recognition with pose variations and misalignment via orthogonal Procrustes regression. IEEE Transactions on Image Processing, vol. 25, no. 6, pp. 2673–2683, 2016. DOI: https://doi.org/10.1109/TIP.2016.2551362.

    Article  MathSciNet  MATH  Google Scholar 

  34. B. Zhao, J. S. Feng, X. Wu, S. C. Yan. A survey on deep learning–based fine–grained object classification and semantic segmentation. International Journal of Automation and Computing, vol. 14, no. 2, pp. 119–135, 2017. DOI: https://doi.org/10.1007/s11633-017-1053-3.

    Article  Google Scholar 

  35. U. Prabhu, J. Heo, M. Savvides. Unconstrained pose–invariant face recognition using 3D generic elastic models. IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 33, no. 10, pp. 1952–1961, 2011. DOI: https://doi.org/10.1109/TPAMI.2011.123.

    Article  Google Scholar 

  36. M. Merras, S. El Hazzat, A. Saaidi, K. Satori, A. G. Nazih. 3D face reconstruction using images from cameras with varying parameters. International Journal of Automation and Computing, vol. 14, no. 6, pp. 661–671, 2017. DOI: https://doi.org/10.1007/s11633-016-0999-x.

    Article  Google Scholar 

  37. T. Ojala, M. Pietikäinen, D. Harwood. A comparative study of texture measures with classification based on feature distributions. Pattern Recognition, vol. 29, no. 1, pp. 51–59, 1996. DOI: https://doi.org/10.1016/0031-3203(95)00067-4.

    Article  Google Scholar 

  38. T. Ahonen, A. Hadid, M. Pietikäinen. Face description with local binary patterns: Application to face recognition. IEEE Transactions on Pattern Analysis & Machine Intelligence, vol. 28, no. 12, pp. 2037–2041, 2006. DOI: https://doi.org/10.1109/TPAMI.2006.244.

    Article  MATH  Google Scholar 

  39. T. Ahonen, A. Hadid, M. Pietikäinen. Face recognition with local binary patterns. In Proceedings of European Conference on Computer Vision, Prague, Czech Republic, pp. 469–481, 2004. DOI: https://doi.org/10.1007/978-3-540-24670-1-36.

    Google Scholar 

  40. H. L. Jin, Q. S. Liu, H. Q. Lu, X. F. Tong. Face detection using improved LBP under bayesian framework. In Proceedings of the 3rd International Conference on Image and Graphics, IEEE, Hong Kong, China, pp. 306–309, 2004. DOI: https://doi.org/10.1109/ICIG.2004.62.

    Google Scholar 

  41. S. C. Liao, X. X. Zhu, Z. Lei, L. Zhang, S. Z. Li. Learning multi–scale block local binary patterns for face recognition. In Proceedings of International Conference on Biometrcs, Seoul, Korea, pp. 828–837, 2007. DOI: https://doi.org/10.1007/978-3-540-74549-5-87.

    Google Scholar 

  42. Y. H. Chen, S. G. Tong, F. Y. Cong, J. Xu. Symmetrical singular value decomposition representation for pattern recognition. Neurocomputing, vol. 214, pp. 143–154, 2016. DOI: https://doi.org/10.1016/j.neucom.2016.05.075.

    Article  Google Scholar 

  43. C. X. Ding, J. Choi, D. C. Tao, L. S. Davis. Multi–directional multi–level dual–cross patterns for robust face recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 38, no. 3, pp. 518–531, 2016. DOI: https://doi.org/10.1109/TPAMI.2015.2462338.

    Article  Google Scholar 

  44. P. J. Phillips, H. Moon, S. A. Rizvi, P. J. Rauss. The FERET evaluation methodology for face–recognition algorithms. IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 22, no. 10, pp. 1090–1104, 2000. DOI: https://doi.org/10.1109/34.879790.

    Article  Google Scholar 

  45. B. Yang, S. C. Chen. A comparative study on local binary pattern (LBP) based face recognition: LBP histogram versus LBP image. Neurocomputing, vol. 120, pp. 365–379, 2013. DOI: https://doi.org/10.1016/j.neucom.2012.10.032.

    Article  Google Scholar 

  46. Z. Wang, A. C. Bovik, H. R. Sheikh, E. P. Simoncelli. Image quality assessment: From error visibility to structural similarity. IEEE Transactions on Image Processing, vol. 13, no. 4, pp. 600–612, 2004. DOI: https://doi.org/10.1109/TIP.2003.819861.

    Article  Google Scholar 

Download references

Acknowledgements

The work was supported by National Natural Science Foundation of China (No. 51305392), Youth Funds of the State Key Laboratory of Fluid Power Transmission and Control (No. SKLoFP_QN_1501), Zhejiang Provincial Natural Science Foundation of China (Nos. LY17E050009 and LZ15E050001), and the Fundamental Rsesearch Funds for the Central Universities (No. 2018QNA4008).

Author information

Authors and Affiliations

  1. State Key Laboratory of Fluid Power and Mechatronic Systems, Zhejiang University, No.38, Zheda Road, Hangzhou, 310027, China

    Shui-Guang Tong, Yuan-Yuan Huang & Zhe-Ming Tong

  2. School of Mechanical Engineering, Zhejiang University, No.38, Zheda Road, Hangzhou, 310027, China

    Shui-Guang Tong, Yuan-Yuan Huang & Zhe-Ming Tong

Authors
  1. Shui-Guang Tong
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yuan-Yuan Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zhe-Ming Tong
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zhe-Ming Tong.

Additional information

Recommended by Associate Editor De Xu

Shui-Guang Tong received the M. Sc. degree in mechanical engineering from Nanjing University of Science and Technology, China in 1987, and the Ph. D. degree in mechanical engineering from Zhejiang University, China in 1991. Since 1996, he has been a professor at Zhejiang University, China. He has published about 100 refereed journal and conference papers. He is the professor of Mechanical Engineering, Zhejiang University, deputy dean of the Industrial Technology Research Institute, Zhejiang University, director of the Institute of Mechanical Design, Zhejiang University, China.

His research interests include fault diagnosis, structure optimization, and pattern recognition.

Yuan-Yuan Huang received the B. Sc. degree in mechanical engineering and automation from North China Electric Power University, China in 2016. Now, she is a Ph. D. degree candidate in State Key Laboratory of Fluid Power and Mechatronic Systems, Zhejiang University, China.

Her research interests include pattern recognition, fault diagnosis and signal processing.

Zhe-Ming Tong received the B. Sc. degree in mechanical engineering from University of Wisconsin-Madison, USA in 2010, the M Sc. and Ph. D. degrees in mechanical engineering from Cornell University, USA in 2015. Prior to joining Zhejiang University, he was a research associate at the Center for Green Buildings and Cities, Harvard University, USA. He is a research professor (tenure-track) at School of Mechanical Engineering, Zhejiang University, China. He was selected to “The 1000-talents Plan for Distinguished Young Scholars” by the Government of China, and “The Bairen Distinguished Young Faculty Program” by Zhejiang University, China in 2017. Dr. Tong′s research group primarily focuses on advanced characterization of vehicle emission, sustainable built environments, and cabin environmental quality of motor vehicles. In the past five years, he has authored more than 20 technical papers, most of which were published in top international journals, such as Environmental Science Technology, Environment International, and Applied Energy. Several of his articles were selected as ESI hot and highly cited papers that received a much higher than average number of citations. He is a member of American Association for Aerosol Research (AAAR), American Society of Heating, Refrigerating and Air Conditioning Engineer (ASHRAE), and American Chemical Society (ACS). He also regularly serves as a peer reviewer for more than 20 international journals. He also holds a number of patents through industrial collaborations.

His research interests include man-machine engineering, energy optimization management of new energy vehicles and multiphase flow numerical simulation technology.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Tong, SG., Huang, YY. & Tong, ZM. A Robust Face Recognition Method Combining LBP with Multi-mirror Symmetry for Images with Various Face Interferences. Int. J. Autom. Comput. 16, 671–682 (2019). https://doi.org/10.1007/s11633-018-1153-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11633-018-1153-8

Keywords

Search

Navigation