Advertisement

Computational Visual Media

, Volume 3, Issue 4, pp 359–368 | Cite as

Face image retrieval based on shape and texture feature fusion

  • Zongguang LuEmail author
  • Jing Yang
  • Qingshan Liu
Open Access
Research Article

Abstract

Humongous amounts of data bring various challenges to face image retrieval. This paper proposes an efficient method to solve those problems. Firstly, we use accurate facial landmark locations as shape features. Secondly, we utilise shape priors to provide discriminative texture features for convolutional neural networks. These shape and texture features are fused to make the learned representation more robust. Finally, in order to increase efficiency, a coarse-tofine search mechanism is exploited to efficiently find similar objects. Extensive experiments on the CASIAWebFace, MSRA-CFW, and LFW datasets illustrate the superiority of our method.

Keywords

face retrieval convolutional neural networks (CNNs) coarse-to-fine 

References

  1. [1]
    Chan, C. H.; Tahir, M. A.; Kittler, J.; Pietikainen, M. Multiscale local phase quantization for robust component-based face recognition using kernel fusion of multiple descriptors. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 35, No. 5, 1164–1177, 2013.CrossRefGoogle Scholar
  2. [2]
    Wu, Z.; Ke, Q.; Sun, J.; Shum, H.-Y. Scalable face image retrieval with identity-based quantization and multireference reranking. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 33, No. 10, 1991–2001, 2011.CrossRefGoogle Scholar
  3. [3]
    Bach, J. R.; Paul, S.; Jain, R. A visual information management system for the interactive retrieval of faces. IEEE Transactions on Knowledge and Data Engineering Vol. 5, No. 4, 619–628, 1993.CrossRefGoogle Scholar
  4. [4]
    Eickeler, S. Face database retrieval using pseudo 2D hidden Markov models. In: Proceedings of the 5th IEEE International Conference on Automatic Face Gesture Recognition, 58–63, 2002.Google Scholar
  5. [5]
    Gudivada, V. N.; Raghavan, V. V. Modeling and retrieving images by content. Information Processing & Management Vol. 33, No. 4, 427–452, 1997.CrossRefGoogle Scholar
  6. [6]
    Wang, X.; Zhang, C.; Zhang, Z. Boosted multitask learning for face verification with applications to web image and video search. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 142–149, 2009.Google Scholar
  7. [7]
    Ojala, T.; Pietikainen, M.; Maenpaa, T. Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 24, No. 7, 971–987, 2002.CrossRefzbMATHGoogle Scholar
  8. [8]
    Parkhi, O. M.; Vedaldi, A.; Zisserman, A. Deep face recognition. In: Proceedings of the British Machine Vision Conference, 2015. Available at http://www.bmva.org/bmvc/2015/papers/paper041/abstract041.pdf.Google Scholar
  9. [9]
    Schroff, F.; Kalenichenko, D.; Philbin, J. FaceNET: A unified embedding for face recognition and clustering. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 815–823, 2015.Google Scholar
  10. [10]
    Sun, Y.; Chen, Y.; Wang, X.; Tang, X. Deep learning face representation by joint identification-verification. In: Proceedings of the Advances in Neural Information Processing Systems 27, 1988–1996, 2014.Google Scholar
  11. [11]
    Sun, Y.; Wang, X.; Tang, X. Hybrid deep learning for face verification. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 38, No. 10, 1997–2009, 2016.CrossRefGoogle Scholar
  12. [12]
    Taigman, Y.; Yang, M.; Ranzato, M.; Wolf, L. DeepFace: Closing the gap to human-level performance in face verification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 1701–1708, 2014.Google Scholar
  13. [13]
    Wen, Y.; Li, Z.; Qiao, Y. Latent factor guided convolutional neural networks for age-invariant face recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 4893–4901, 2016.Google Scholar
  14. [14]
    Chopra, S.; Hadsell, R.; Lecun, Y. Learning a similarity metric discriminatively, with application to face verification. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Vol. 1, 539–546, 2005.Google Scholar
  15. [15]
    Sun, Y.; Wang, X.; Tang, X. Deep learning face representation from predicting 10,000 classes. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 1891–1898, 2014.Google Scholar
  16. [16]
    Sun, Y.; Wang, X.; Tang, X. Deeply learned face representations are sparse, selective, and robust. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2892–2900, 2015.Google Scholar
  17. [17]
    Liu, J.; Deng, Y.; Bai, T.; Wei, Z.; Huang, C. Targeting ultimate accuracy: Face recognition via deep embedding. arXiv preprint arXiv:1506.07310, 2015.Google Scholar
  18. [18]
    Xiong, X.; la Torre, F. D. Supervised descent method and its applications to face alignment. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 532–539, 2013.Google Scholar
  19. [19]
    Szegedy, C.; Liu, W.; Jia, Y.; Sermanet, P.; Reed, S.; Anguelov, D.; Erhan, D.; Vanhoucke, V.; Rabinovich, A. Going deeper with convolutions. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 1–9, 2015.Google Scholar
  20. [20]
    Szegedy, C.; Vanhoucke, V.; Ioffe, S.; Shlens, J.; Wojna, Z. Rethinking the inception architecture for computer vision. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2818–2826, 2016.Google Scholar
  21. [21]
    Lin, M.; Chen, Q.; Yan, S. Network in network. arXiv preprint arXiv:1312.4400, 2013.Google Scholar
  22. [22]
    Yi, D.; Lei, Z.; Liao, S.; Li, S. Z. Learning face representation from scratch. arXiv preprint arXiv:1411.7923, 2014.Google Scholar
  23. [23]
    MSRA-CFW: Data set of celebrity faces on the web. Available at https://www.microsoft.com/en-us/research/project/msra-cfw-data-set-of-celebrity-faceson- the-web/.Google Scholar
  24. [24]
    Huang, G. B.; Mattar, M.; Berg, T.; Learned-Miller, E. Labeled faces in the wild: A database for studying face recognition in unconstrained environments. In: Proceedings of the Workshop on Faces in “Real-Life” Images: Detection, Alignment, and Recognition, 2008.Google Scholar
  25. [25]
    Cao, Z.; Yin, Q.; Tang, X.; Sun, J. Face recognition with learning-based descriptor. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2707–2714, 2010.Google Scholar
  26. [26]
    Kumar, N.; Berg, A. C.; Belhumeur, P. N.; Nayar, S. K. Attribute and simile classifiers for face verification. In: Proceedings of the IEEE 12th International Conference on Computer Vision, 365–372, 2009.Google Scholar
  27. [27]
    Yin, Q.; Tang, X.; Sun, J. An associate-predict model for face recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 497–504, 2011.Google Scholar
  28. [28]
    Taigman, Y.; Wolf, L. Leveraging billions of faces to overcome performance barriers in unconstrained face recognition. arXiv preprint arXiv:1108.1122, 2011.Google Scholar
  29. [29]
    Chen, D.; Cao, X.; Wang, L.; Wen, F.; Sun, J. Bayesian face revisited: A joint formulation. In: Computer Vision–ECCV 2012. Fitzgibbon, A.; Lazebnik, S.; Perona, P.; Sato, Y.; Schmid, C. Eds. Springer Berlin Heidelberg, 566–579, 2012.CrossRefGoogle Scholar
  30. [30]
    Berg, T.; Belhumeur, P. N. POOF: Part-based onevs.- one features for fine-grained categorization, face verification, and attribute estimation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 955–962, 2013.Google Scholar
  31. [31]
    Berg, T.; Belhumeur, P. N. Tom-vs-Pete classifiers and identity-preserving alignment for face verification. In: Proceedings of the British Machine Vision Conference, 2012. Available at http://www.bmva.org/bmvc/2012/BMVC/paper129/paper129.pdf.Google Scholar
  32. [32]
    Chen, D.; Cao, X.; Wen, F.; Sun, J. Blessing of dimensionality: High-dimensional feature and its efficient compression for face verification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 3025–3032, 2013.Google Scholar
  33. [33]
    Cao, X.; Wipf, D.; Wen, F.; Duan, G.; Sun, J. A practical transfer learning algorithm for face verification. In: Proceedings of the IEEE International Conference on Computer Vision, 3208–3215, 2013.Google Scholar
  34. [34]
    Zhu, Z.; Luo, P.; Wang, X.; Tang, X. Recover canonical-view faces in the wild with deep neural networks. arXiv preprint arXiv:1404.3543, 2014.Google Scholar
  35. [35]
    TCIT. Available at http://www.tcit-us.com/.Google Scholar
  36. [36]
    INSKY. Available at http://www.insky.so/.Google Scholar
  37. [37]
    DaHua-FaceImage. Available at http://www. dahuatech.com/.Google Scholar

Copyright information

© The Author(s) 2017

Open Access The articles published in this journal are distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

Other papers from this open access journal are available free of charge from http://www.springer.com/journal/41095. To submit a manuscript, please go to https://www.editorialmanager.com/cvmj.

Authors and Affiliations

  1. 1.School of Information and Control EngineeringNanjing University of Information Science and TechnologyNanjingChina

Personalised recommendations