Computer Vision

Living Edition

Deep CNN-Based Face Recognition

  • Ankan BansalEmail author
  • Rajeev Ranjan
  • Carlos D. Castillo
  • Rama Chellappa
Living reference work entry
DOI: https://doi.org/10.1007/978-3-030-03243-2_880-1
  • 45 Downloads

Synonyms

Definition

Automatic face recognition is the problem of identifying a person from an image or a video. The problem of face recognition can be divided into face identification and face verification. The standard approach for training a CNN for solving these problems include four steps: face detection, alignment, representation, and classification (Fig. 1). Identification is the problem of assigning an identity to an image from a list of identities. From another perspective, this can be considered as trying to retrieve the best matching face from a gallery for a given probe image. On the other hand, face verification involves verifying whether two face images are of the same person. This is usually performed by computing the similarity between feature representations of the two faces. Both identification and verification have benefited immensely from developments in deep learning algorithms and more advanced CNN...
This is a preview of subscription content, log in to check access.

Notes

Acknowledgements

This research is based upon work supported by the Office of the Director of National Intelligence (ODNI), Intelligence Advanced Research Projects Activity (IARPA), via IARPA R&D Contract No. 2014-14071600012.

References

  1. 1.
    Huang GB, Mattar M, Berg T, Learned-Miller E (2008) Labeled faces in the wild: a database forstudying face recognition in unconstrained environments. In: Workshop on faces in real-life images: detection, alignment, and recognitionGoogle Scholar
  2. 2.
    Liu Z, Luo P, Wang X, Tang X (2015) Deep learning face attributes in the wild. In: IEEE international conference on computer vision, pp 3730–3738Google Scholar
  3. 3.
    Yi D, Lei Z, Liao S, Li SZ (2014) Learning face representation from scratch. arXiv preprint arXiv:1411.7923Google Scholar
  4. 4.
    Bansal A, Nanduri A, Castillo C, Ranjan R, Chellappa R (2016) UMDFaces: an annotated face dataset for training deep networks. arXiv preprint arXiv:1611.01484Google Scholar
  5. 5.
    Guo Y, Zhang L, Hu Y, He X, Gao J (2016) MS-celeb-1M: a dataset and benchmark for large-scale face recognition. In: European conference on computer vision. Springer, pp 87–102Google Scholar
  6. 6.
    Parkhi OM, Vedaldi A, Zisserman A (2015) Deep face recognition. In: BMVC, vol 1, p 6Google Scholar
  7. 7.
    Cao Q, Shen L, Xie W, Parkhi OM, Zisserman A (2018) VGGFace2: a dataset for recognising faces across pose and age. In: Automatic face & gesture recognition (FG 2018), 2018 13th IEEE international conference on. IEEE, pp 67–74Google Scholar
  8. 8.
    Kushwaha V, Singh M, Singh R, Vatsa M, Ratha N, Chellappa R (2018) Disguised faces in the wild. In: IEEE conference on computer vision and pattern recognition workshops, vol 8Google Scholar
  9. 9.
    Klare BF, Taborsky E, Blanton A, Cheney J, Allen K, Grother P, Mah A, Burge M, Jain AK (2015) Pushing the frontiers of unconstrained face detection and recognition: Iarpa Janus benchmark a. In: IEEE conference on computer vision and pattern recognition (CVPR), 13:4Google Scholar
  10. 10.
    Whitelam C, Taborsky E, Blanton A, Maze B, Adams JC, Miller T, Kalka ND, Jain AK, Duncan JA, Allen K et al (2017) Iarpa Janus benchmark-b face dataset. In: CVPR workshops, vol 2, p 6Google Scholar
  11. 11.
    Maze B, Adams J, Duncan JA, Kalka N, Miller T, Otto C, Jain AK, Niggel WT, Anderson J, Cheney J et al (2018) Iarpa Janus benchmark–c: face dataset and protocol. In: 11th IAPR international conference on biometricsGoogle Scholar
  12. 12.
    Kalka ND, Maze B, Duncan JA, O’Connor K, Elliott S, Hebert K, Bryan J, Jain AK (2018) IJB–S: Iarpa Janus surveillance video benchmark. In: 2018 IEEE 9th international conference on biometrics theory, applications and systems (BTAS). IEEE, pp 1–9Google Scholar
  13. 13.
    Kemelmacher-Shlizerman I, Seitz SM, Miller D, Brossard E (2016) The megaface benchmark: 1 million faces for recognition at scale. In: IEEE conference on computer vision and pattern recognition (CVPR), pp 4873–4882Google Scholar
  14. 14.
    Wolf L, Hassner T, Maoz I (2011) Face recognition in unconstrained videos with matched background similarity. In: Computer vision and pattern recognition (CVPR), 2011 IEEE conference on. IEEE, pp 529–534Google Scholar
  15. 15.
    Bansal A, Castillo C, Ranjan R, Chellappa R (2017) The do’s and don’ts for CNN-based face verification. In: Proceedings of the IEEE international conference on computer vision, pp 2545–2554Google Scholar
  16. 16.
    Yang S, Luo P, Loy C-C, Tang X (2016) Wider face: a face detection benchmark. In: IEEE conference on computer vision and pattern recognition, pp 5525–5533Google Scholar
  17. 17.
    Jain V, Learned-Miller E (2010) FDDB: a benchmark for face detection in unconstrained settings. Technical Report UM-CS-2010-009, University of Massachusetts, AmherstGoogle Scholar
  18. 18.
    Zhu X, Ramanan D (2012) Face detection, pose estimation, and landmark localization in the wild. In: IEEE conference on computer vision and pattern recognition, pp 2879–2886, June 2012Google Scholar
  19. 19.
    Sagonas C, Tzimiropoulos G, Zafeiriou S, Pantic M (2013) 300 faces in-the-wild challenge: the first facial landmark localization challenge. In: Proceedings of the IEEE international conference on computer vision workshops, pp 397–403Google Scholar
  20. 20.
    Belhumeur PN, Jacobs DW, Kriegman DJ, Kumar N (2011) Localizing parts of faces using a consensus of exemplars. In: IEEE conference on computer vision and pattern recognition, pp 545–552Google Scholar
  21. 21.
    Kostinger M, Wohlhart P, Roth PM, Bischof H (2011) Annotated facial landmarks in the wild: a large-scale, real-world database for facial landmark localization. In: IEEE international conference on computer vision workshops, pp 2144–2151, Nov 2011Google Scholar
  22. 22.
    Faltemier TC, Bowyer KW, Flynn PJ (2007) Using a multi-instance enrollment representation to improve 3D face recognition. In: Biometrics: theory, applications, and systems, 2007. BTAS 2007. First IEEE international conference on. IEEE, pp. 1–6Google Scholar
  23. 23.
    Levi G, Hassner T (2015) Age and gender classification using convolutional neural networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition workshops, pp 34–42Google Scholar
  24. 24.
    Rothe R, Timofte R, Van Gool L (2015) Dex: deep expectation of apparent age from a single image. In: IEEE international conference on computer vision workshop on ChaLearn looking at people, pp 10–15Google Scholar
  25. 25.
    Hand EM, Castillo C, Chellappa R (2018) Doing the best we can with what we have: multi-label balancing with selective learning for attribute prediction. In: AAAI conference on artificial intelligence. AAAIGoogle Scholar
  26. 26.
    Bansal A, Venkatesh KS (2015) People counting in high density crowds from still images. arXiv preprint arXiv:1507.08445Google Scholar
  27. 27.
    Sindagi VA, Patel VM (2017) CNN-based cascaded multi-task learning of high-level prior and density estimation for crowd counting. In: Advanced video and signal based surveillance (AVSS), 2017 14th IEEE international conference on. IEEE, pp 1–6Google Scholar
  28. 28.
    Uijlings JR, van de Sande KE, Gevers T, Smeulders AW (2013) Selective search for object recognition. Int J Comput Vis 104(2):154–171CrossRefGoogle Scholar
  29. 29.
    Zitnick CL, Dollár P (2014) Edge boxes: locating object proposals from edges. In: European conference on computer vision. Springer, pp 391–405Google Scholar
  30. 30.
    Ren S, He K, Girshick R, Sun J (2015) Faster R-CNN: towards real-time object detection with region proposal networks. In: Advances in neural information processing systems, pp 91–99Google Scholar
  31. 31.
    Ranjan R, Sankaranarayanan S, Castillo CD, Chellappa R (2017) An all-in-one convolutional neural network for face analysis. In: IEEE international conference on automatic face and gesture recognition (FG)Google Scholar
  32. 32.
    Ranjan R, Patel V, Chellappa R (2016) Hyperface: a deep multi-task learning framework for face detection, landmark localization, pose estimation, and gender recognition. arXiv preprint arXiv:1603.01249Google Scholar
  33. 33.
    Hu P, Ramanan D (2016) Finding tiny faces. arXiv preprint arXiv:1612.04402Google Scholar
  34. 34.
    Chen D, Hua G, Wen F, Sun J (2016) Supervised transformer network for efficient face detection. In: European conference on computer vision. Springer, pp 122–138Google Scholar
  35. 35.
    Liu W, Anguelov D, Erhan D, Szegedy C, Reed S, Fu C-Y, Berg AC (2016) SSD: single shot multibox detector. In: European conference on computer vision (ECCV), pp 21–37CrossRefGoogle Scholar
  36. 36.
    Redmon J, Divvala S, Girshick R, Farhadi A (2015) You only look once: unified, real-time object detection. arXiv preprint arXiv:1506.02640Google Scholar
  37. 37.
    Redmon J, Farhadi A (2017) Yolo9000: better, faster, stronger. arXiv preprintGoogle Scholar
  38. 38.
    Ranjan R, Bansal A, Zheng J, Xu H, Gleason J, Lu B, Nanduri A, Chen J-C, Castillo CD, Chellappa R (2018) A fast and accurate system for face detection, identification, and verification. arXiv preprint arXiv:1809.07586Google Scholar
  39. 39.
    Najibi M, Samangouei P, Chellappa R, Davis L (2017) SSH: single stage headless face detector. arXiv preprint arXiv:1708.03979Google Scholar
  40. 40.
    Li H, Lin Z, Shen X, Brandt J, Hua G (2015) A convolutional neural network cascade for face detection. In: IEEE conference on computer vision and pattern recognition (CVPR), pp 5325–5334Google Scholar
  41. 41.
    Yang S, Xiong Y, Loy CC, Tang X (2017) Face detection through scale-friendly deep convolutional networks. arXiv preprint arXiv:1706.02863Google Scholar
  42. 42.
    Zhang S, Zhu X, Lei Z, Shi H, Wang X, Li SZ (2017) S3fd: single shot scale-invariant face detector. arXiv preprint arXiv:1708.05237Google Scholar
  43. 43.
    Wang N, Gao X, Tao D, Yang H, Li X (2017) Facial feature point detection: a comprehensive survey. Neurocomputing 275:50–65CrossRefGoogle Scholar
  44. 44.
    Ranjan R, Bansal A, Xu H, Sankaranarayanan S, Chen J-C, Castillo CD, Chellappa R (2018) Crystal loss and quality pooling for unconstrained face verification and recognition. arXiv preprint arXiv:1804.01159Google Scholar
  45. 45.
    Liu W, Wen Y, Yu Z, Li M, Raj B, Song L (2017) Sphereface: deep hypersphere embedding for face recognition. In: IEEE international conference on computer vision and pattern recognition (CVPR)Google Scholar
  46. 46.
    Wang H, Wang Y, Zhou Z, Ji X, Gong D, Zhou J, Li Z, Liu W (2018) Cosface: large margin cosine loss for deep face recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 5265–5274Google Scholar
  47. 47.
    Deng J, Guo J, Zafeiriou S (2018) Arcface: additive angular margin loss for deep face recognition. arXiv preprint arXiv:1801.07698Google Scholar
  48. 48.
    Chen B, Deng W, Du J (2017) Noisy softmax: improving the generalization ability of DCNN via postponing the early softmax saturation. In: The IEEE conference on computer vision and pattern recognition (CVPR)Google Scholar
  49. 49.
    Wen Y, Zhang K, Li Z, Qiao Y (2016) A discriminative feature learning approach for deep face recognition. In: European conference on computer vision (ECCV), pp 499–515CrossRefGoogle Scholar
  50. 50.
    Wu Y, Liu H, Li J, Fu Y (2017) Deep face recognition with center invariant loss. In: Proceedings of the on thematic workshops of ACM multimedia 2017. ACM, pp 408–414Google Scholar
  51. 51.
    Zhang X, Fang Z, Wen Y, Li Z, Qiao Y (2017) Range loss for deep face recognition with long-tailed training data. In: 2017 IEEE international conference on computer vision (ICCV). IEEE, pp 5419–5428Google Scholar
  52. 52.
    Qi X, Zhang L (2018) Face recognition via centralized coordinate learning. arXiv preprint arXiv:1801.05678Google Scholar
  53. 53.
    Zheng Y, Pal DK, Savvides M (2018) Ring loss: convex feature normalization for face recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 5089–5097Google Scholar
  54. 54.
    Schroff F, Kalenichenko D, Philbin J (2015) Facenet: a unified embedding for face recognition and clustering. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 815–823Google Scholar
  55. 55.
    Sun Y, Wang X, Tang X (2014) Deep learning face representation from predicting 10000 classes. In: IEEE conference on computer vision and pattern recognition (CVPR), pp 1891–1898Google Scholar
  56. 56.
    Taigman Y, Yang M, Ranzato M, Wolf L (2014) Deepface: closing the gap to human-level performance in face verification. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1701–1708Google Scholar
  57. 57.
    Liu J, Deng Y, Bai T, Wei Z, Huang C (2015) Targeting ultimate accuracy: face recognition via deep embedding. arXiv preprint arXiv:1506.07310Google Scholar
  58. 58.
    Bansal A, Ranjan R, Castillo CD, Chellappa R (2018) Deep features for recognizing disguised faces in the wild. In: 2018 IEEE/CVF conference on computer vision and pattern recognition workshops (CVPRW). IEEE, pp 10–106Google Scholar
  59. 59.
    Lin W-A, Chen J-C, Ranjan R, Bansal A, Sankaranarayanan S, Castillo CD, Chellappa R (2018) Proximity-aware hierarchical clustering of unconstrained faces. Image Vis Comput 77:33–44CrossRefGoogle Scholar
  60. 60.
    Sun Y, Chen Y, Wang X, Tang X (2014) Deep learning face representation by joint identification-verification. In: Advances in neural information processing systems, pp 1988–1996Google Scholar
  61. 61.
    Masi I, Tran AT, Leksut JT, Hassner T, Medioni G (2016) Do we really need to collect millions of faces for effective face recognition? arXiv preprint arXiv:1603.07057Google Scholar
  62. 62.
    Wang D, Otto C, Jain AK (2015) Face search at scale: 80 million gallery. arXiv preprint arXiv:1507.07242Google Scholar
  63. 63.
    Masi I, Rawls S, Medioni G, Natarajan P (2016) Pose-aware face recognition in the wild. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 4838–4846Google Scholar
  64. 64.
    AbdAlmageed W, Wu Y, Rawls S, Harel S, Hassne T, Masi I, Choi J, Lekust J, Kim J, Natarajana P, Nevatia R, Medioni G (2016) Face recognition using deep multi-pose representations. In: IEEE winter conference on applications of computer vision (WACV)Google Scholar
  65. 65.
    Yang J, Ren P, Chen D, Wen F, Li H, Hua G (2016) Neural aggregation network for video face recognition. arXiv preprint arXiv:1603.05474Google Scholar
  66. 66.
    Chang F-J, Tran AT, Hassner T, Masi I, Nevatia R, Medioni G (2017) Faceposenet: making a case for landmark-free face alignment. In: Computer vision workshop (ICCVW), 2017 IEEE international conference on. IEEE, pp 1599–1608Google Scholar
  67. 67.
    Xiong L, Jayashree K, Zhao J, Feng J, Pranata S, Shen S (2017) A good practice towards top performance of face recognition: transferred deep feature fusion. arXiv preprint arXiv:1704.00438Google Scholar
  68. 68.
    Wu X, He R, Sun Z, Tan T (2018) A light CNN for deep face representation with noisy labels. IEEE Trans Inf Forensics Secur 13(11):2884–2896CrossRefGoogle Scholar
  69. 69.
    Merler M, Ratha N, Feris RS, Smith JR (2019) Diversity in faces. arXiv preprint arXiv:1901.10436Google Scholar

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  • Ankan Bansal
    • 1
    Email author
  • Rajeev Ranjan
    • 3
  • Carlos D. Castillo
    • 1
    • 2
  • Rama Chellappa
    • 1
  1. 1.University of MarylandCollege ParkUSA
  2. 2.Department of Computer ScienceUniversity of MarylandCollege ParkUSA
  3. 3.AmazonSeattleUSA

Section editors and affiliations

  • Rama Chellappa
    • 1
  1. 1.University of MarylandCollege ParkUnited States