Advertisement

A pyramid multi-level face descriptor: application to kinship verification

  • A. Moujahid
  • F. DornaikaEmail author
Article
  • 58 Downloads

Abstract

Texture descriptors such as Local Binary Pattern (LBP), Local Phase Quantization (LPQ), and Histogram of Oriented Gradients (HOG) have been widely used for face image analysis. This work introduces a novel framework for image-based kinship verification able to efficiently combine local and global facial information extracted from diverse descriptors. The proposed scheme relies on two main points: (1) we model the face images using a Pyramid Multi-level (PML) representation where local descriptors are extracted from several blocks at different resolution scales; (2) we compute the covariance (second-order statistics) between diverse local features characterizing each individual block in the PML representation. This gives rise to a face descriptor with two interesting properties: (i) thanks to the PML representation, scales and face parts are explicitly encoded in the final descriptor without having to detect the facial landmarks; (ii) the covariance descriptor encodes spatial features of any type allowing the integration of several state-of-the-art texture and color features. Experiments conducted on three public kinship databases show that the proposed descriptor can outperform many state-of-the-art kinship verification algorithms and descriptors including those that are based on deep Convolutional Neural Nets.

Keywords

Kinship verification Face descriptor Multi-scale representation Covariance descriptor Feature selection Classifier 

Notes

References

  1. 1.
    Ahonen T, Hadid A, Pietikainen M (2006) Face description with local binary patterns: application to face recognition. IEEE Trans Pattern Anal Mach Intell 28 (12):2037–2041CrossRefzbMATHGoogle Scholar
  2. 2.
    Alirezazadeh P, Fathi A, Abdali-Mohammadi F (2015) A genetic algorithm-based feature selection for kinship verification. IEEE Signal Process Lett 22(12):2459–2463CrossRefGoogle Scholar
  3. 3.
    Bekhouche S, Ouafi A, Benlamoudi A, Taleb-Ahmed A, Hadid A (2015) Automatic age estimation and gender classification in the wild. In: Proceeding of the international conference on automatic control, telecommunications and signals ICATS’15Google Scholar
  4. 4.
    Bekhouche S, Ouafi A, Dornaika F, Taleb-Ahmed A, Hadid A (2017) Pyramid multi-level features for facial demographic estimation. Expert Syst Appl 80 (Supplement C):297–310CrossRefGoogle Scholar
  5. 5.
    Bianconi F, Bello R, Napoletano P, Maria FD (2017) Improved opponent colour local binary patterns for colour texture classification. In: Workshop computational color imaging workshop, CCIWGoogle Scholar
  6. 6.
    Cao Z, Yin Q, Tang X, Sun J (2010) Face recognition with learning-based descriptor. In: 2010 IEEE Computer society conference on computer vision and pattern recognition, pages 2707–2714Google Scholar
  7. 7.
    Dalal N, Triggs B (2005) Histograms of oriented gradients for human detection IEEE computer society conference on computer vision and pattern recognition, 2005. CVPR 2005. vol 1. IEEE, pp 886–893Google Scholar
  8. 8.
    Davarpanah SH, Khalid F, Nurliyana Abdullah L, Golchin M (2016) A texture descriptor: background local binary pattern (bglbp). Multimedia Tools Appl 75 (11):6549–6568CrossRefGoogle Scholar
  9. 9.
    DeBruine L, Jones B, Little A, Perrett D (2008) Social perception of facial resemblance in humans. Arch Sex Behav 37(1):64–77CrossRefGoogle Scholar
  10. 10.
    DeBruine LM, Smith FG, Jones BC, Roberts SC, Petrie M, Spector TD (2009) Kin recognition signals in adult faces. Vis Res 49(1):38–43CrossRefGoogle Scholar
  11. 11.
    Dehghan A, Ortiz EG, Villegas R, Shah M (2014) Who do i look like? Determining parent-offspring resemblance via gated autoencoders. In: IEEE conference on computer vision and pattern recognition (CVPR), pp 1757–1764Google Scholar
  12. 12.
    Dornaika F, Moujahid A, Merabet YE, Ruichek Y (2016) Building detection from orthophotos using a machine learning approach: an empirical study on image segmentation and descriptors. Expert Syst Appl 58:130–142CrossRefGoogle Scholar
  13. 13.
    Fang R, Tang KD, Snavely N, Chen T (2010) Towards computational models of kinship verification. In: 2010 17th IEEE international conference on image processing (ICIP), pp 1577–1580Google Scholar
  14. 14.
    Fang Y, Chen YYS, Wang H, Shu C (2016) Sparse similarity metric learning for kinship verification. In: Visual communications and image processing (VCIP), 2016. IEEE, pp 1–4Google Scholar
  15. 15.
    Guan D, Yuan W, Lee Y-K, Najeebullah K, Rasel MK (2014) A review of ensemble learning based feature selection. IETE Tech Rev 31(3):190–198CrossRefGoogle Scholar
  16. 16.
    Guo Y, Dibeklioglu H, van der Maaten L (2014) Graph-based kinship recognition. In: 2014 22nd international conference on pattern recognition (ICPR), IEEE, pp 4287–4292Google Scholar
  17. 17.
    Hu J, Lu J, Yuan J, Tan Y-P (2014) Large margin multi-metric learning for face and kinship verification in the wild. In: ACCV (3), pp 252–267Google Scholar
  18. 18.
    Hussain SU, Napoléon T, Jurie F (2012) Face recognition using local quantized patterns. In: British machive vision conference Guildford, United Kingdom, p 11Google Scholar
  19. 19.
    Kabbai L, Azaza A, Abdellaoui M, Douik A (2015) Image matching based on lbp and sift descriptor. In: 2015 IEEE 12th international multi-conference on systems, signals and devices (SSD15), pp 1–6Google Scholar
  20. 20.
    Kannala J, Rahtu E (2012) Bsif: Binarized statistical image features. In: 2012 21st international conference on pattern recognition (ICPR), pp 1363–1366Google Scholar
  21. 21.
  22. 22.
    Kohli N, Vatsa M, Singh R, Noore A, Majumdar A (2017) Hierarchical representation learning for kinship verification. IEEE Trans Image Process 26(1):289–302MathSciNetCrossRefGoogle Scholar
  23. 23.
    Kumar CR (2017) Harmonic rule for measuring the facial similarities among relatives. Trans Mach Learn Artif Intell 4(6):29Google Scholar
  24. 24.
    Lan R, Zhou Y (2016) Quaternion-michelson descriptor for color image classification. IEEE Trans Image Process 25(11):5281–5292MathSciNetCrossRefGoogle Scholar
  25. 25.
    Lan R, Zhou Y, Tang YY (2016) Quaternionic local ranking binary pattern: a local descriptor of color images. IEEE Trans Image Process 25(2):566–579MathSciNetCrossRefGoogle Scholar
  26. 26.
    Lan R, Zhou Y, Tang YY (2017) Quaternionic weber local descriptor of color images. IEEE Trans Circuits Syst Video Technol 27(2):261–274CrossRefGoogle Scholar
  27. 27.
    Lan R, Zhou Y, Tang YY, Chen CLP (2014) Person reidentification using quaternionic local binary pattern. In: 2014 IEEE international conference on multimedia and expo (ICME), pp 1–6Google Scholar
  28. 28.
    Li L, Feng X, Wu X, Xia Z, Hadid A (2016) Kinship verification from faces via similarity metric based convolutional neural network. In: International conference image analysis and recognition. Springer, pp 539–548Google Scholar
  29. 29.
    Liu Q, Puthenputhussery A, Liu C (2016) A novel inheritable color space with application to kinship verification. In: 2016 IEEE winter conference on applications of computer vision (WACV). IEEE, pp 1–9Google Scholar
  30. 30.
    Liu Y, Nie L, Liu L, Rosenblum DS (2016) From action to activity: sensor-based activity recognition. Neurocomputing 181:108–115. Big Data Driven Intelligent Transportation SystemsCrossRefGoogle Scholar
  31. 31.
    Liu Y, Zhang L, Nie L, Yan Y, Rosenblum DS (2016) Fortune teller: predicting your career path. In: 13th AAAI conference on artificial intelligenceGoogle Scholar
  32. 32.
    López MB, Boutellaa E, Hadid A (2016) Comments on the “kinship face in the wild” data sets. IEEE Trans Pattern Anal Mach Intell 38(11):2342–2344CrossRefGoogle Scholar
  33. 33.
    Lu J, Hu J, Liong VE, Zhou X, Bottino A, Islam IU, Vieira TF, Qin X, Tan X, Chen S, Mahpod S, Keller Y, Zheng L, Idrissi K, Garcia C, Duffner S, Baskurt A, Castrillón-Santana M, Lorenzo-Navarro J (2015) The fg 2015 kinship verification in the wild evaluation. In: 2015 11th IEEE international conference and workshops on automatic face and gesture recognition (FG), vol 1, pp 1–7Google Scholar
  34. 34.
    Lu J, Liong VE, Zhou X, Zhou J (2015) Learning compact binary face descriptor for face recognition. IEEE Trans Pattern Anal Mach Intell 37(10):2041–2056CrossRefGoogle Scholar
  35. 35.
    Lu J, Zhou X, Tan Y-P, Shang Y, Zhou J (2014) Neighborhood repulsed metric learning for kinship verification. IEEE Trans Pattern Anal Mach Intell 36(2):331–345CrossRefGoogle Scholar
  36. 36.
    Ma Y, Wu X, Yu G, Xu Y, Wang Y (2016) Pedestrian detection and tracking from low-resolution unmanned aerial vehicle thermal imagery. Sensors 16(4)Google Scholar
  37. 37.
    Nguyen DT, Cho SR, Shin KY, Bang JW, Park KR (2014) Comparative study of human age estimation with or without preclassification of gender and facial expression. Sci World J 2014:15Google Scholar
  38. 38.
    Ojala T, Pietikäinen M, Maenpaa T (2002) Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Trans Pattern Anal Mach Intell 24:971–987CrossRefzbMATHGoogle Scholar
  39. 39.
    Ojala T, Pietikainen M, Maenpaa T (2002) Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Trans Pattern Anal Mach Intell 24(7):971–987CrossRefzbMATHGoogle Scholar
  40. 40.
    Patel B, Maheshwari R, Raman B (2017) Evaluation of periocular features for kinship verification in the wild. Comput Vis Image Underst 160(Supplement C):24–35CrossRefGoogle Scholar
  41. 41.
    Puthenputhussery A, Liu Q, Liu C (2016) Sift flow based genetic fisher vector feature for kinship verification. In: 2016 IEEE international conference on image processing (ICIP), IEEE, pp 2921–2925Google Scholar
  42. 42.
    Qin X, Tan X, Chen S (2016) Mixed bi-subject kinship verification via multi-view multi-task learning. Neurocomputing 214:350–357CrossRefGoogle Scholar
  43. 43.
    Shao M, Xia S, Fu Y (2011) Genealogical face recognition based on ub kinface database. In: CVPR 2011 WORKSHOPS, pp 60–65Google Scholar
  44. 44.
    Silva C, Bouwmans T, Frélicot C (2015) An extended center-symmetric local binary pattern for background modeling and subtraction in videos. In: Proceedings of the 10th international conference on computer vision theory and applications - volume 1: VISAPP, (VISIGRAPP 2015), pp 395–402Google Scholar
  45. 45.
    Szeliski R (2011) Computer vision algorithms and applications. Springer, New YorkzbMATHGoogle Scholar
  46. 46.
    Takala V, Ahonen T, Pietik?inen M (2005) Block-based methods for image retrieval using local binary patterns. In: Image analysis, SCIA, volume LNCS, 3540Google Scholar
  47. 47.
    Tan X, Triggs B (2010) Enhanced local texture feature sets for face recognition under difficult lighting conditions. IEEE Trans Image Processing 19:1635–1650MathSciNetCrossRefzbMATHGoogle Scholar
  48. 48.
    Tuzel O, Porikli F, Meer P (2006) A fast descriptor for detection and classification. In: European conf. on computer vision, pp 589–600Google Scholar
  49. 49.
    Vinay K, Shreyas B (2006) Face recognition using gabor wavelets. In: 14th asilomar conference on signals, systems and computers, pp 593–597Google Scholar
  50. 50.
    Wang W, Chen W, Xu D (2011) Pyramid-based multi-scale lbp features for face recognition. In: 2011 international conference on multimedia and signal processing (CMSP), vol 1, pp 151–155Google Scholar
  51. 51.
    Wolf L, Hassner T, Taigman Y (2008) Descriptor based methods in the wild. In: Faces in real-life images workshop in ECCVGoogle Scholar
  52. 52.
    Xia S, Shao M, Fu Y (2011) Kinship verification through transfer learning. In: Proceedings of the twenty-second international joint conference on artificial intelligence - volume three, IJCAI’11. AAAI Press, pp 2539–2544Google Scholar
  53. 53.
    Xia S, Shao M, Luo J, Fu Y (2012) Understanding kin relationships in a photo. IEEE Trans Multimedia 14(4):1046–1056CrossRefGoogle Scholar
  54. 54.
    Xia S, Shao M, Luo J, Fu Y (2012) Understanding kin relationships in a photo. IEEE Trans Multimedia 14(4):1046–1056CrossRefGoogle Scholar
  55. 55.
    Yan H (2017) Kinship verification using neighborhood repulsed correlation metric learning. Image Vis Comput 60:91–97CrossRefGoogle Scholar
  56. 56.
    Yan H, Lu J, Deng W, Zhou X (2014) Discriminative multimetric learning for kinship verification. IEEE Trans Inf Forensics Secur 9(7):1169–1178CrossRefGoogle Scholar
  57. 57.
    Yan H, Lu J, Deng W, Zhou X (2014) Discriminative multimetric learning for kinship verification. IEEE Trans Inf Forensics Secur 9(7):1169–1178CrossRefGoogle Scholar
  58. 58.
    Yan H, Lu J, Zhou X (2015) Prototype-based discriminative feature learning for kinship verification. IEEE Trans Cybern 45(11):2535–2545CrossRefGoogle Scholar
  59. 59.
    Yang B, Chen S (2013) A comparative study on local binary pattern (LBP) based face recognition LBP histogram versus LBP image. Neurocomputing 120:365–379CrossRefGoogle Scholar
  60. 60.
    Zhang K, Huang Y, Song C, Wu H, Wang L (2015) Kinship verification with deep convolutional neural networks. In: Proceedings of the British machine vision conference (BMVC). BMVA Press, pp 148.1–148.12Google Scholar
  61. 61.
    Zhou X, Hu J, Lu J, Shang Y, Guan Y (2011) Kinship verification from facial images under uncontrolled conditions. In: Proceedings of the 19th ACM international conference on multimedia. ACM, pp 953–956Google Scholar
  62. 62.
    Zhou X, Shang Y, Yan H, Guo G (2016) Ensemble similarity learning for kinship verification from facial images in the wild. Information Fusion 32:40–48CrossRefGoogle Scholar
  63. 63.
    Zhou X, Yan H, Shang Y (2016) Kinship verification from facial images by scalable similarity fusion. Neurocomputing 197:136–142CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  1. 1.University of the Basque Country UPV/EHUSan SebastianSpain
  2. 2.IKERBASQUEBasque Foundation for ScienceBilbaoSpain

Personalised recommendations