Do We Really Need to Collect Millions of Faces for Effective Face Recognition?

Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9909)

Abstract

Face recognition capabilities have recently made extraordinary leaps. Though this progress is at least partially due to ballooning training set sizes – huge numbers of face images downloaded and labeled for identity – it is not clear if the formidable task of collecting so many images is truly necessary. We propose a far more accessible means of increasing training data sizes for face recognition systems: Domain specific data augmentation. We describe novel methods of enriching an existing dataset with important facial appearance variations by manipulating the faces it contains. This synthesis is also used when matching query images represented by standard convolutional neural networks. The effect of training and testing with synthesized images is tested on the LFW and IJB-A (verification and identification) benchmarks and Janus CS2. The performances obtained by our approach match state of the art results reported by systems trained on millions of downloaded images.

References

  1. 1.
    AbdAlmageed, W., Wu, Y., Rawls, S., Harel, S., Hassner, T., Masi, I., Choi, J., Leksut, J., Kim, J., Natarajan, P., Nevatia, R., Medioni, G.: Face recognition using deep multi-pose representations. In: Winter Conference on Applications of Computer Vision (2016)Google Scholar
  2. 2.
    Baltrusaitis, T., Robinson, P., Morency, L.P.: Constrained local neural fields for robust facial landmark detection in the wild. In: Proceedings of the International Conference on Computer Vision Workshops (2013)Google Scholar
  3. 3.
    Chatfield, K., Simonyan, K., Vedaldi, A., Zisserman, A.: Return of the devil in the details: Delving deep into convolutional nets. In: Proceedings of the British Machine Vision Conference (2014)Google Scholar
  4. 4.
    Chen, J.C., Sankaranarayanan, S., Patel, V.M., Chellappa, R.: Unconstrained face verification using fisher vectors computed from frontalized faces. In: International Conference on Biometrics: Theory, Applications and Systems (2015)Google Scholar
  5. 5.
    Chen, J.C., Patel, V.M., Chellappa, R.: Unconstrained face verification using deep cnn features. In: Winter Conference on Applications of Computer Vision (2016)Google Scholar
  6. 6.
    Ding, C., Xu, C., Tao, D.: Multi-task pose-invariant face recognition. Trans. Image Process. 24(3), 980–993 (2015)MathSciNetCrossRefGoogle Scholar
  7. 7.
    Eigen, D., Fergus, R.: Predicting depth, surface normals and semantic labels with a common multi-scale convolutional architecture. In: Proceedings of the International Conference on Computer Vision, pp. 2650–2658 (2015)Google Scholar
  8. 8.
    Ferrari, C., Lisanti, G., Berretti, S., Del Bimbo, A.: Dictionary learning based 3D morphable model construction for face recognition with varying expression and pose. In: 3DV (2015)Google Scholar
  9. 9.
    Hartley, R., Zisserman, A.: Multiple View Geometry in Computer Vision. Cambridge University Press, Cambridge (2003)MATHGoogle Scholar
  10. 10.
    Hassner, T.: Viewing real-world faces in 3d. In: Proceedings of the International Conference on Computer Vision, pp. 3607–3614 (2013)Google Scholar
  11. 11.
    Hassner, T., Harel, S., Paz, E., Enbar, R.: Effective face frontalization in unconstrained images. In: Proceedings of the International Conference on Computer Vision Pattern Recognition (2015)Google Scholar
  12. 12.
    Hassner, T., Masi, I., Kim, J., Choi, J., Harel, S., Natarajan, P., Medioni, G.: Pooling faces: template based face recognition with pooled face images. In: Proceedings of the International Conference on Computer Vision Pattern Recognition Workshops, June 2016Google Scholar
  13. 13.
    Huang, G.B., Ramesh, M., Berg, T., Learned-Miller, E.: Labeled faces in the wild: a database for studying face recognition in unconstrained environments. Technical report 07-49, UMass, Amherst, October 2007Google Scholar
  14. 14.
    Kemelmacher-Shlizerman, I., Seitz, S.M., Miller, D., Brossard, E.: The MegaFace benchmark: 1 million faces for recognition at scale. In: Proceedings of the International Conference on Computer Vision Pattern Recognition (2016)Google Scholar
  15. 15.
    Kemelmacher-Shlizerman, I., Suwajanakorn, S., Seitz, S.M.: Illumination-aware age progression. In: Proceedings of the International Conference on Computer Vision Pattern Recognition, pp. 3334–3341. IEEE (2014)Google Scholar
  16. 16.
    Klare, B.F., Klein, B., Taborsky, E., Blanton, A., Cheney, J., Allen, K., Grother, P., Mah, A., Burge, M., Jain, A.K.: Pushing the frontiers of unconstrained face detection and recognition: IARPA Janus benchmark A. In: Proceedings of the International Conference on Computer Vision Pattern Recognition, pp. 1931–1939 (2015)Google Scholar
  17. 17.
    Klontz, J., Klare, B., Klum, S., Taborsky, E., Burge, M., Jain, A.K.: Open source biometric recognition. In: International Conference on Biometrics: Theory, Applications and Systems (2013)Google Scholar
  18. 18.
    Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: Neural Information Processing Systems, pp. 1097–1105 (2012)Google Scholar
  19. 19.
    Lawrence, S., Giles, C.L., Tsoi, A.C., Back, A.D.: Face recognition: a convolutional neural-network approach. Trans. Neural Netw. 8(1), 98–113 (1997)CrossRefGoogle Scholar
  20. 20.
    Levi, G., Hassner, T.: Age and gender classification using convolutional neural networks. In: Proceedings of the International Conference on Computer Vision Pattern Recognition Workshops, June 2015. http://www.openu.ac.il/home/hassner/projects/cnn_agegender
  21. 21.
    Lewis, J.P., Anjyo, K., Rhee, T., Zhang, M., Pighin, F., Deng, Z.: Practice and theory of blendshape facial models. In: Eurographics 2014 (2014)Google Scholar
  22. 22.
    Li, H., Hua, G., Lin, Z., Brandt, J., Yang, J.: Probabilistic elastic matching for pose variant face verification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3499–3506 (2013)Google Scholar
  23. 23.
    Masi, I., Ferrari, C., Del Bimbo, A., Medioni, G.: Pose independent face recognition by localizing local binary patterns via deformation components. In: International Conference on Pattern Recognition (2014)Google Scholar
  24. 24.
    Masi, I., Rawls, S., Medioni, G., Natarajan, P.: Pose-aware face recognition in the wild. In: Proceedings of the International Conference on Computer Vision and Pattern Recognition (2016)Google Scholar
  25. 25.
    McLaughlin, N., Martinez Del Rincon, J., Miller, P.: Data-augmentation for reducing dataset bias in person re-identification. In: International Conference on Advanced Video and Signal Based Surveillance. IEEE (2015)Google Scholar
  26. 26.
    Nguyen, M.H., Lalonde, J.F., Efros, A.A., De la Torre, F.: Image-based shaving. Comput. Graph. Forum 27(2), 627–635 (2008)CrossRefGoogle Scholar
  27. 27.
    Parkhi, O.M., Simonyan, K., Vedaldi, A., Zisserman, A.: A compact and discriminative face track descriptor. In: Proceedings of the International Conference on Computer Vision and Pattern Recognition (2014)Google Scholar
  28. 28.
    Parkhi, O.M., Vedaldi, A., Zisserman, A.: Deep face recognition. In: Proceedings of the British Machine Vision Conference (2015)Google Scholar
  29. 29.
    Paysan, P., Knothe, R., Amberg, B., Romdhani, S., Vetter, T.: A 3d face model for pose and illumination invariant face recognition. In: Sixth IEEE International Conference on Advanced Video and Signal Based Surveillance, AVSS 2009, pp. 296–301, September 2009Google Scholar
  30. 30.
    Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al.: Imagenet large scale visual recognition challenge. Int. J. Comput. Vis. 115(3), 211–252 (2015)MathSciNetCrossRefGoogle Scholar
  31. 31.
    Sánchez, J., Perronnin, F., Mensink, T., Verbeek, J.: Image classification with the fisher vector: theory and practice. Int. J. Comput. Vis. 105(3), 222–245 (2013)MathSciNetCrossRefMATHGoogle Scholar
  32. 32.
    Sankaranarayanan, S., Alavi, A., Chellappa, R.: Triplet similarity embedding for face verification (2016). arXiv preprint: arXiv:1602.03418
  33. 33.
    Schroff, F., Kalenichenko, D., Philbin, J.: Facenet: a unified embedding for face recognition and clustering. In: Proceedings of the International Conference on Computer Vision and Pattern Recognition, pp. 815–823 (2015)Google Scholar
  34. 34.
    Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. In: International Conference on Learning Representations (2015)Google Scholar
  35. 35.
    Su, H., Maji, S., Kalogerakis, E., Learned-Miller, E.: Multi-view convolutional neural networks for 3d shape recognition. In: Proceedings of the International Conference on Computer Vision, pp. 945–953 (2015)Google Scholar
  36. 36.
    Sun, Y., Chen, Y., Wang, X., Tang, X.: Deep learning face representation by joint identification-verification. In: Neural Information Processing System, pp. 1988–1996 (2014)Google Scholar
  37. 37.
    Sun, Y., Liang, D., Wang, X., Tang, X.: Deepid3: face recognition with very deep neural networks (2015). arXiv preprint: arXiv:1502.00873
  38. 38.
    Sun, Y., Wang, X., Tang, X.: Deep learning face representation from predicting 10,000 classes. In: Proceedings of the International Conference on Computer Vision Pattern Recognition. IEEE (2014)Google Scholar
  39. 39.
    Taigman, Y., Yang, M., Ranzato, M., Wolf, L.: Deepface: Closing the gap to human-level performance in face verification. In: Proceedings of the International Conference on Computer Vision and Pattern Recognition, pp. 1701–1708. IEEE (2014)Google Scholar
  40. 40.
    Taigman, Y., Yang, M., Ranzato, M., Wolf, L.: Web-scale training for face identification. In: Proceedings of the International Conference on Computer Vision and Pattern Recognition (2015)Google Scholar
  41. 41.
    Wang, D., Otto, C., Jain, A.K.: Face search at scale: 80 million gallery (2015). arXiv preprint: arXiv:1507.07242
  42. 42.
    Wolf, L., Hassner, T., Maoz, I.: Face recognition in unconstrained videos with matched background similarity. In: Proceedings of the International Conference on Computer Vision Pattern Recognition, pp. 529–534. IEEE (2011)Google Scholar
  43. 43.
    Xie, S., Tu, Z.: Holistically-nested edge detection. In: Proceedings of the International Conference on Computer Vision (2015)Google Scholar
  44. 44.
    Yang, H., Patras, I.: Mirror, mirror on the wall, tell me, is the error small? In: Proceedings of the International Conference on Computer Vision Pattern Recognition (2015)Google Scholar
  45. 45.
    Yi, D., Lei, Z., Li, S.: Towards pose robust face recognition. In: Proceedings of the International Conference on Computer Vision Pattern Recognition, pp. 3539–3545 (2013)Google Scholar
  46. 46.
    Yi, D., Lei, Z., Liao, S., Li, S.Z.: Learning face representation from scratch (2014). arXiv preprint: arXiv:1411.7923, http://www.cbsr.ia.ac.cn/english/CASIA-WebFace-Database.html
  47. 47.
    Zhou, E., Cao, Z., Yin, Q.: Naive-deep face recognition: touching the limit of LFW benchmark or not? (2015). arXiv preprint: arXiv:1501.04690

Copyright information

© Springer International Publishing AG 2016

Authors and Affiliations

  1. 1.Institute for Robotics and Intelligent Systems, USCLos AngelesUSA
  2. 2.Information Sciences Institute, USCLos AngelesUSA
  3. 3.The Open University of IsraelRa’ananaIsrael

Personalised recommendations