Skip to main content
SpringerLink
Log in

A review on face recognition systems: recent approaches and challenges

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Face recognition is an efficient technique and one of the most preferred biometric modalities for the identification and verification of individuals as compared to voice, fingerprint, iris, retina eye scan, gait, ear and hand geometry. This has over the years necessitated researchers in both the academia and industry to come up with several face recognition techniques making it one of the most studied research area in computer vision. A major reason why it remains a fast-growing research lies in its application in unconstrained environments, where most existing techniques do not perform optimally. Such conditions include pose, illumination, ageing, occlusion, expression, plastic surgery and low resolution. In this paper, a critical review on the different issues of face recognition systems are presented, and different approaches to solving these issues are analyzed by presenting existing techniques that have been proposed in the literature. Furthermore, the major and challenging face datasets that consist of the different facial constraints which depict real-life scenarios are also discussed stating the shortcomings associated with them. Also, recognition performance on the different datasets by researchers are also reported. The paper is concluded, and directions for future works are highlighted.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

References

  1. Abate AF, Nappi M, Riccio D, Sabatino G (2007) 2D and 3D face recognition: a survey. Pattern Recogn Lett 28:1885–1906

    Google Scholar 

  2. Ali ASO, Sagayan V, Malik A, Aziz A (2016) Proposed face recognition system after plastic surgery. IET Comput Vis 10:344–350

    Google Scholar 

  3. Alkkiomaki O, Kyrki V, Liu Y, Handroos H, and Kalviainen H (2009) Multi-modal force/vision sensor fusion in 6-DOF pose tracking," in Advanced Robotics. ICAR 2009. International conference on 2009,, pp. 1–8.

  4. Angadi SA, Kagawade VC (2017) A robust face recognition approach through symbolic modeling of polar FFT features. Pattern Recogn 71:235–248

    Google Scholar 

  5. Bartlett MS, Movellan JR, Sejnowski TJ (2002) Face recognition by independent component analysis. IEEE Trans Neural Netw 13:1450–1464

    Google Scholar 

  6. Belahcene M, Chouchane A, and Ouamane H (2014) 3D face recognition in presence of expressions by fusion regions of interest," in 2014 22nd Signal Processing and Communications Applications Conference (SIU), pp. 2269–2274.

  7. Bhat FA, Wani MA (2016) Elastic bunch graph matching based face recognition under varying lighting, pose, and expression conditions. IAES International Journal of Artificial Intelligence (IJ-AI) 3:177–182

    Google Scholar 

  8. Bolme DS (2003) Elastic bunch graph matching. Colorado State University

  9. Bowyer KW, Chang K, Flynn P (2006) A survey of approaches and challenges in 3D and multi-modal 3D+ 2D face recognition. Comput Vis Image Underst 101:1–15

    Google Scholar 

  10. Breiman L (2001) Random forests. Mach Learn 45:5–32

    MATH  Google Scholar 

  11. Brunelli R, Poggio T (1993) Face recognition: features versus templates. IEEE Trans Pattern Anal Mach Intell 15:1042–1052

    Google Scholar 

  12. Cao X, Shen W, Yu L, Wang Y, Yang J, Zhang Z (2012) Illumination invariant extraction for face recognition using neighboring wavelet coefficients. Pattern Recogn 45:1299–1305

    Google Scholar 

  13. Chen L, Liang M, Song W, and Xiao K (2018) A multi-scale parallel convolutional neural network based intelligent human identification using face information. Journal of Information Processing Systems, vol. 14.

  14. Cheng EJ, Chou KP, Jin S, Tanveer M, Lin CT, Young KY, Lin WC, Prasad M (2019) Deep sparse representation classifier for facial recognition and detection system. Pattern Recogn Lett 125:71–77

    Google Scholar 

  15. Chihaoui M, Elkefi A, Bellil W, Ben Amar C (2016) A survey of 2D face recognition techniques. Computers 5:21

    Google Scholar 

  16. Chu Y, Ahmad T, Bebis G, Zhao L (2017) Low-resolution face recognition with single sample per person. Signal Process 141:144–157

    Google Scholar 

  17. Chude-Olisah CC, Sulong G, Chude-Okonkwo UA, Hashim SZ (2014) Face recognition via edge-based Gabor feature representation for plastic surgery-altered images. EURASIP Journal on Advances in Signal Processing 2014:102

    Google Scholar 

  18. Delac K, Grgic M, Grgic S (2005) Independent comparative study of PCA, ICA, and LDA on the FERET data set. Int J Imaging Syst Technol 15:252–260

    Google Scholar 

  19. Deng W, Jiani H, Jun G (2017) Face recognition via collaborative representation: its discriminant nature and superposed representation. IEEE Transaction on pattern analysis and machine intelligence 40:2513–2521

    Google Scholar 

  20. Ding C, Tao D (2017) Pose-invariant face recognition with homography-based normalization. Pattern Recogn 66:144–152

    Google Scholar 

  21. Ding C, Hu Z, Karmoshi S, Zhu M (2017) A novel two-stage learning pipeline for deep neural networks. Neural processing letters

  22. Drira H, Amor BB, Srivastava A, Daoudi M, Slama R (2013) 3D face recognition under expressions, occlusions, and pose variations. IEEE Trans Pattern Anal Mach Intell 35:2270–2283

    Google Scholar 

  23. Feng Z-H, Kittler J, Awais M, Huber P, and Wu X-J (2017) Face detection, bounding box aggregation and pose estimation for robust facial landmark localisation in the Wild, arXiv preprint arXiv:1705.02402.

  24. Fu Y, Wu X, Wen Y, Xiang Y (2017) Efficient locality-constrained occlusion coding for face recognition. Neurocomputing 260:104–111

    Google Scholar 

  25. Gao G, Yang J, Jing X-Y, Shen F, Yang W, Yue D (2017) Learning robust and discriminative low-rank representations for face recognition with occlusion. Pattern Recogn 66:129–143

    Google Scholar 

  26. Gao C-z, Cheng Q, He P, Susilo W, Li J (2018) Privacy-preserving naive Bayes classifiers secure against the substitution-then-comparison attack. Inf Sci 444:72–88

    MathSciNet  MATH  Google Scholar 

  27. Ghiass RS, Arandjelović O, Bendada A, Maldague X (2014) Infrared face recognition: a comprehensive review of methodologies and datasets. Pattern Recogn 47:2807–2824

    Google Scholar 

  28. Goyal SJ, Upadhyay AK, Jadon R, and Goyal R (2018) Real-life facial expression recognition systems: a review," in Smart Computing and Informatics, ed: Springer, pp. 311–331.

  29. Guo Y, Zhang L, Hu Y, He X, and Gao J (2016) Ms-celeb-1m: A dataset and benchmark for large-scale face recognition, in European Conference on Computer Vision, pp. 87–102.

  30. Hanmandlu M, Gupta D, and Vasikarla S (2013) Face recognition using Elastic bunch graph matching. in Applied Imagery Pattern Recognition Workshop (AIPR): Sensing for Control and Augmentation, 2013 IEEE, pp. 1–7.

  31. Heo J, Marios S (2008) Face recognition across pose using view based active appearance models on CMU multi-PIE dataset. In Proceeding of International Conference on Computer Vision Systems, May, pp 527–535

    Google Scholar 

  32. Hijazi S, Kumar R, and Rowen C (2015) Using convolutional neural networks for image recognition, ed.

  33. Ho C, Morgado P, Persekian A, Vasconcelos N (2019) "PIEs: pose invariant Embeddings," IEEE/CVF conference on computer vision and pattern recognition (CVPR). Long Beach, CA, USA, pp 12369–12378. https://doi.org/10.1109/CVPR.2019.01266

    Book  Google Scholar 

  34. Hsu G-SJ, Ambikapathi A, Chung S-L, Shie H-C (2018) Robust cross-pose face recognition using landmark oriented depth warping. J Vis Commun Image Represent 53:273–280

    Google Scholar 

  35. Hu H (2008) ICA-based neighborhood preserving analysis for face recognition. Comput Vis Image Underst 112:286–295

    Google Scholar 

  36. Huang D, Shan C, Ardabilian M, Wang Y, Chen L (2011) Local binary patterns and its application to facial image analysis: a survey. IEEE Trans Syst Man Cybern Part C Appl Rev 41:765–781

    Google Scholar 

  37. Jia S, Lansdall-Welfare T, and Cristianini N (2016) Gender classification by deep learning on millions of weakly labelled images, in Data Mining Workshops (ICDMW), 2016 IEEE 16th International Conference on, pp. 462–467.

  38. Jiang L, Li C, Wang S, Zhang L (2016) Deep feature weighting for naive Bayes and its application to text classification. Eng Appl Artif Intell 52:26–39

    Google Scholar 

  39. Jin X, Tan X (2017) Face alignment in-the-wild: a survey. Comput Vis Image Underst 162:1–22

    Google Scholar 

  40. Jin T, Liu Z, Yu Z, Min X, Li L (2017) Locality preserving collaborative representation for face recognition. Neural Process Lett 45:967–979

    Google Scholar 

  41. Kakadiaris IA, Toderici G, Evangelopoulos G, Passalis G, Chu D, Zhao X, Shah SK, Theoharis T (2017) 3D-2D face recognition with pose and illumination normalization. Comput Vis Image Underst 154:137–151

    Google Scholar 

  42. Karamizadeh S, Abdullah SM, Zamani M, Shayan J, and Nooralishahi P (2017) Face recognition via taxonomy of illumination normalization," in Multimedia Forensics and Security, ed: Springer, pp. 139–160.

  43. Kim P (2017) Convolutional Neural Network, in MATLAB Deep Learning, ed: Springer, pp. 121–147.

  44. Kotropoulos C, Pitas I, Fischer S, and Duc B (1997) Face authentication using morphological dynamic link architecture," in Audio-and Video-based Biometric Person Authentication, pp. 169–176.

  45. Lades M, Vorbruggen JC, Buhmann J, Lange J, von der Malsburg C, Wurtz RP, Konen W (1993) Distortion invariant object recognition in the dynamic link architecture. IEEE Trans Comput 42:300–311

    Google Scholar 

  46. Lahasan BM, Venkat I, Al-Betar MA, Lutfi SL, De Wilde P (2016) Recognizing faces prone to occlusions and common variations using optimal face subgraphs. Appl Math Comput 283:316–332

    MathSciNet  MATH  Google Scholar 

  47. Le QV (2013) Building high-level features using large scale unsupervised learning, in Acoustics, Speech and Signal Processing (ICASSP), 2013 IEEE International Conference on, pp. 8595–8598.

  48. Lei G, Li X-h, Zhou J-l, and Gong X-g (2009) Geometric feature based facial expression recognition using multiclass support vector machines," in Granular Computing, 2009, GRC'09. IEEE International Conference on, pp. 318–321.

  49. Li L-y, Li D-r (2010) Research on particle swarm optimization in remote sensing image enhancement [J]. Journal of Geomatics Science and Technology 2:012

    Google Scholar 

  50. Li M, Yuan B (2005) 2D-LDA: a statistical linear discriminant analysis for image matrix. Pattern Recogn Lett 26:527–532

    Google Scholar 

  51. Li Z, Gong D, Li X, Tao D (2016) Aging face recognition: a hierarchical learning model based on local patterns selection. IEEE Trans Image Process 25:2146–2154

    MathSciNet  MATH  Google Scholar 

  52. Li Y, Wang Y, Liu J, Hao W (2018) Expression-insensitive 3D face recognition by the fusion of multiple subject-specific curves. Neurocomputing 275:1295–1307

    Google Scholar 

  53. Liao S, Lei Z, Yi D, and Li SZ (2014) A benchmark study of large-scale unconstrained face recognition," in Biometrics (IJCB), 2014 IEEE International Joint Conference on, pp. 1–8.

  54. Liu H-D, Yang M, Gao Y, Cui C (2014) Local histogram specification for face recognition under varying lighting conditions. Image Vis Comput 32:335–347

    Google Scholar 

  55. Long Y, Zhu F, Shao L, and Han J (2017) Face recognition with a small occluded training set using spatial and statistical pooling. Inf Sci.

  56. Lopes AT, de Aguiar E, De Souza AF, Oliveira-Santos T (2017) Facial expression recognition with convolutional neural networks: coping with few data and the training sample order. Pattern Recogn 61:610–628

    Google Scholar 

  57. Luan X, Fang B, Liu L, Yang W, Qian J (2014) Extracting sparse error of robust PCA for face recognition in the presence of varying illumination and occlusion. Pattern Recogn 47:495–508

    Google Scholar 

  58. Ma X, Song H, Qian X (2015) Robust framework of single-frame face Superresolution across head pose, facial expression, and illumination variations. IEEE Transactions on Human-Machine Systems 45:238–250

    Google Scholar 

  59. Manjani I, Sumerkan H, Flynn PJ, and Bowyer KW (2016) Template aging in 3D and 2D face recognition," in Biometrics Theory, Applications and Systems (BTAS), 2016 IEEE 8th International Conference on, pp. 1–6.

  60. Martinez AM (1998) The AR face dataset, CVC technical report, vol. 24.

  61. Martins JA, Lam R, Rodrigues J, du Buf J (2018) Expression-invariant face recognition using a biological disparity energy model. Neurocomputing 297:82–93

    Google Scholar 

  62. Masi L, Rawls S, Medioni G, and Natarajan P (2016) Pose-aware face recognition in the wild. In Proceedings of the IEEE Conference on computer vision and pattern recognition, pp. 4838–4846.

  63. Mi J-X, Liu T (2016) Multi-step linear representation-based classification for face recognition. IET Comput Vis 10:836–841

    Google Scholar 

  64. Nappi M, Ricciardi S, Tistarelli M (2016) Deceiving faces: when plastic surgery challenges face recognition. Image Vis Comput 54:71–82

    Google Scholar 

  65. Oloyede MO, Hancke GP (2016) Unimodal and multimodal biometric sensing systems: a review. IEEE Access 4:7532–7555

    Google Scholar 

  66. Oloyede MO, Hancke GP, and Kapileswar N (2017) Evaluating the effect of occlusion in face recognition systems, In Proceedings of IEEE Africon Conference, pp. 1547–1551.

  67. Oloyede MO, Hancke GP, and Myburgh HC (2018) Improving face recognition systems using a new image enhancement technique, hybrid features and the convolutional neural network. IEEE Access, pp. 1–11.

  68. Oloyede MO, Hancke GP, Myburgh HC, and Onumanyi AJ (2019) A new evaluation function for face image in unconstrained environments using metaheuristic algorithms. Eurasip Journal on Image and Video Processing, pp. 1–18.

  69. Ouyang S, Hospedales T, Song Y-Z, Li X, Loy CC, Wang X (2016) A survey on heterogeneous face recognition: sketch, infra-red, 3d and low-resolution. Image Vis Comput 56:28–48

    Google Scholar 

  70. Patacchiola M, Cangelosi A (2017) Head pose estimation in the wild using convolutional neural networks and adaptive gradient methods. Pattern Recogn 71:132–143

    Google Scholar 

  71. Pereira JF, Barreto RM, Cavalcanti GD, and Tsang R (2011) A robust feature extraction algorithm based on class-modular image principal component analysis for face verification, in Acoustics, Speech and Signal Processing (ICASSP), 2011 IEEE International Conference on, pp. 1469–1472.

  72. Petpairote C, Madarasmi S, Chamnongthai K (2017) A pose and expression face recognition method using transformation based on single face neutral reference. In Proceedings of IEEE Internationl Conference on Global Wireless Summit, October:123–126

  73. Qi Z, Tian Y, Shi Y (2013) Robust twin support vector machine for pattern classification. Pattern Recogn 46:305–316

    MATH  Google Scholar 

  74. Qian Y, Deng W, and Hu J (2019) Unsupervised face normalization with extreme pose and expressionin the wild , In proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 9851–9858.

  75. Rakshit RD, Kisku DR (2020) Face identification via strategic combination of local features. In Proceedings of Computational Intelligencein Pattern Recognition:207–217

  76. Rasti P, Uiboupin T, Escalera S, and Anbarjafari G (2016) Convolutional neural network super resolution for face recognition in surveillance monitoring, in International Conference on Articulated Motion and Deformable Objects, pp. 175–184.

  77. Rehman A, Saba T (2014) Neural networks for document image preprocessing: state of the art. Artif Intell Rev 42:253–273

    Google Scholar 

  78. Revina IM, Emmanuel WS (2018) Face expression recognition using LDN and dominant gradient local ternary pattern descriptors. Journal of King Saud University-Computer and Information Sciences

  79. Sabharwal T, Rashimi G (2019) Human identification after plastic surgery using region based score level fusion of local facial features. Journal of information security and application 48:102373

    Google Scholar 

  80. Sable AH, Talbar SN, Dhirbasi HA (2017) Recognition of plastic surgery faces and the surgery types: An approach with entropy based scale invariant features. Journal of King Saud University-Computer and Information Sciences

  81. Sariyanidi E, Gunes H, Cavallaro A (2015) Automatic analysis of facial affect: a survey of registration, representation, and recognition. IEEE Trans Pattern Anal Mach Intell 37:1113–1133

    Google Scholar 

  82. Savran A, Sankur B (2017) Non-rigid registration based model-free 3D facial expression recognition. Comput Vis Image Underst 162:146–165

    Google Scholar 

  83. Suri S, Sankaran A, Vasta M, Singh R (2018) On matching faces with alterations due to plastic surgery and disguise. In Proceedings of IEEE Conference on Biometrics Theory, Applications and Systems, pp 1–7

  84. Tan S, Xi S, Wenato C, Lei Q, Ling S (2017) Robust face recognition with kernalized locality-sensitive group sparsity representation. IEEE Transaction on image processing 26:4661–4668

    Google Scholar 

  85. Tefas A, Kotropoulos C, and Pitas I (1998) Variants of dynamic link architecture based on mathematical morphology for frontal face authentication, in Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231), pp. 814–819.

  86. Tong Z, Aihara K, and Tanaka G (2016) A hybrid pooling method for convolutional neural networks, in International Conference on Neural Information Processing, pp. 454–461.

  87. Tsai H-H, Chang Y-C (2017) Facial expression recognition using a combination of multiple facial features and support vector machine. Soft Comput:1–17

  88. Turk MA and Pentland AP (1991) Face recognition using eigenfaces, in Computer Vision and Pattern Recognition. Proceedings CVPR'91., IEEE Computer Society Conference on, 1991, pp. 586–591.

  89. Wang K, Chen Z, Wu QJ, Liu C (2017) Illumination and pose variable face recognition via adaptively weighted ULBP_MHOG and WSRC. Signal Process Image Commun 58:175–186

    Google Scholar 

  90. Wang J-W, Le NT, Lee J-S, Wang C-C (2017) Illumination compensation for face recognition using adaptive singular value decomposition in the wavelet domain. Inf Sci

  91. Xanthopoulos P, Pardalos PM, and Trafalis TB (2013) Linear discriminant analysis, in Robust data mining, ed: Springer, pp. 27–33.

  92. Xu C, Liu Q, Ye M (2017) Age invariant face recognition and retrieval by coupled auto-encoder networks. Neurocomputing 222:62–71

    Google Scholar 

  93. Yang J, Luo L, Qian J, Tai Y, Zhang F, Xu Y (2017) Nuclear norm based matrix regression with applications to face recognition with occlusion and illumination changes. IEEE Trans Pattern Anal Mach Intell 39:156–171

    Google Scholar 

  94. Yang J, Ren P, Zhang D, Chen D, Wen F, Li H, and Hua G (2017) Neural aggregation network for video face recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 4362–4371.

  95. Yu Y-F, Dai D-Q, Ren C-X, Huang K-K (2017) Discriminative multi-layer illumination-robust feature extraction for face recognition. Pattern Recogn 67:201–212

    Google Scholar 

  96. Zafeiriou S, Zhang C, Zhang Z (2015) A survey on face detection in the wild: past, present and future. Comput Vis Image Underst 138:1–24

    Google Scholar 

  97. Zeng S, Jianping G, Deng L (2017) An antinoise sparse representation method for robust face recognition via joint l1and l2 regularization. Expert System with Application 82:1–9

    Google Scholar 

  98. Zhang P, Ben X, Jiang W, Yan R, Zhang Y (2015) Coupled marginal discriminant mappings for low-resolution face recognition. Optik-International Journal for Light and Electron Optics 126:4352–4357

    Google Scholar 

  99. Zhang Y, Lu Y, Wu H, Wen C, and Ge C (2016) Face occlusion detection using cascaded convolutional neural network, in Chinese Conference on Biometric Recognition, pp. 720–727.

  100. Zhang D-x, An P, Zhang H-x (2018) Application of robust face recognition in video surveillance systems. Optoelectron Lett 14:152–155

    Google Scholar 

  101. Zhang MM, Shang K, Wu H (2019) Learning deep discriminative face features by customized weighted constraint. Nuerocomputing 332:71–79

    Google Scholar 

  102. Zhao S (2018) Pixel-level occlusion detection based on sparse representation for face recognition. Optik 168:920–930

    Google Scholar 

  103. Zhao K, Jingyl X, and Cheng MM (2019) Regukarface: Deep face recognition via exclusive regularization”, In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1136–1144.

  104. Zhou H, Lam K-M (2018) Age-invariant face recognition based on identity inference from appearance age. Pattern Recogn 76:191–202

    Google Scholar 

  105. Zhou Z, Wagner A, Mobahi H, Wright J, and Ma Y (2009) Face recognition with contiguous occlusion using markov random fields, in Computer Vision, 2009 IEEE 12th International Conference on, pp. 1050–1057.

  106. Zhou L-F, Du Y-W, Li W-S, Mi J-X, Luan X (2018) Pose-robust face recognition with Huffman-LBP enhanced by divide-and-rule strategy. Pattern Recogn

  107. Zhou Q, Zhang C, Yu W, Fan Y, Zhu H, Xiaofu W (2018) Face recognition via fast dense correspondence. Multimed Tools Appl 77:10501–10519

    Google Scholar 

  108. Zhuang L, Chan T-H, Yang AY, Sastry SS, Ma Y (2015) Sparse illumination learning and transfer for single-sample face recognition with image corruption and misalignment. Int J Comput Vis 114:272–287

    MathSciNet  MATH  Google Scholar 

Download references

Acknowledgments

This work was supported by the Council for Scientific and Industrial Research (CSIR), South Africa.

[ICT: Meraka].

Author information

Authors and Affiliations

  1. Department of Information and Communication Science, University of Ilorin, Ilorin, Nigeria

    Muhtahir O. Oloyede

  2. Department of Electrical, Electronic and Computer Engineering, University of Pretoria, Pretoria, South Africa

    Muhtahir O. Oloyede, Gerhard P. Hancke & Hermanus C. Myburgh

Authors
  1. Muhtahir O. Oloyede
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Gerhard P. Hancke
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hermanus C. Myburgh
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Muhtahir O. Oloyede.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Oloyede, M.O., Hancke, G.P. & Myburgh, H.C. A review on face recognition systems: recent approaches and challenges. Multimed Tools Appl 79, 27891–27922 (2020). https://doi.org/10.1007/s11042-020-09261-2

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-020-09261-2

Keywords

Search

Navigation