Skip to main content
SpringerLink
Log in

Multilinear subspace learning using handcrafted and deep features for face kinship verification in the wild

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

In this paper, we propose a new multilinear and multiview subspace learning method called Tensor Cross-view Quadratic Discriminant Analysis for face kinship verification in the wild. Most of the existing multilinear subspace learning methods straightforwardly focus on learning a single set of projection matrices, making it difficult to separate different classes. To address this issue, the proposed approach mutually learns multi-view representations for multidimensional cross-view matching. In order to decrease the effect of the within class variations for each mode of the tensor data, the proposed approach integrates the Within Class Covariance Normalization. Moreover, we propose a new tensor face descriptor based on the Gabor wavelets. Besides, we investigate the complementarity of handcrafted and deep face tensor features via their fusion at score level using the Logistic Regression method. Our extensive experiments demonstrate that the proposed kinship verification framework outperforms the state of the art, achieving 95.14%, 91.83% and 93.58% verification accuracies on Cornell KinFace, UB KinFace and TSKinFace face kinship datasets, respectively.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  1. Barkan O, Weill J, Wolf L, Aronowitz H (2013) Fast high dimensional vector multiplication face recognition. In: Proceedings of the IEEE international conference on computer vision, pp. 1960–1967

  2. Bessaoudi M, Belahcene M, Ouamane A, Chouchane A, Bourennane S (2019) Multilinear enhanced fisher discriminant analysis for robust multimodal 2d and 3d face verification. Appl Intell 49(4):1339–1354

    Article  Google Scholar 

  3. Bessaoudi M, Ouamane A, Belahcene M, Chouchane A, Boutellaa E, Bourennane S (2019) Multilinear side-information based discriminant analysis for face and kinship verification in the wild. Neurocomputing 329:267–278

    Article  Google Scholar 

  4. Boutellaa E, López MB, Ait-Aoudia S, Feng X, Hadid A (2017) Kinship verification from videos using spatio-temporal texture features and deep learning

  5. Dal Martello MF, Maloney LT (2006) Where are kin recognition signals in the human face? J Vis 6(12):2–2

    Article  Google Scholar 

  6. Dehak N, Kenny PJ, Dehak R, Dumouchel P, Ouellet P (2010) Front-end factor analysis for speaker verification. IEEE Transactions on Audio Speech, and Language Processing 19(4):788–798

    Article  Google Scholar 

  7. Fang L, Li S (2015) Face recognition by exploiting local gabor features with multitask adaptive sparse representation. IEEE Trans Instrum Meas 64(10):2605–2615

    Article  Google Scholar 

  8. Fang R, Tang KD, Snavely N, Chen T (2010) Towards computational models of kinship verification. In: 2010 IEEE International conference on image processing, pp. 1577–1580. IEEE

  9. Guo G, Wang X (2012) Kinship measurement on salient facial features. IEEE Trans Instrum Meas 61(8):2322–2325

    Article  Google Scholar 

  10. Harrell Jr FE (2015) Regression modeling strategies: with applications to linear models, logistic and ordinal regression, and survival analysis Springer

  11. Iman RL, Conover WJ (1982) A distribution-free approach to inducing rank correlation among input variables. Communications in Statistics-Simulation and Computation 11(3):311–334

    Article  Google Scholar 

  12. Kaminski G, Dridi S, Graff C, Gentaz E (2009) Human ability to detect kinship in strangerb faces: effects of the degree of relatedness. Proc R Soc B Biol Sci 276(1670):3193–3200

    Article  Google Scholar 

  13. Kaminski G, Ravary F, Graff C, Gentaz E (2010) Firstborns’ disadvantage in kinship detection. Psychological science 21(12):1746–1750

    Article  Google Scholar 

  14. Kannala J, Rahtu E (2012) Bsif: Binarized statistical image features. In: Proceedings of the 21st international conference on pattern recognition (ICPR2012), pp. 1363–1366. IEEE

  15. Kofman A (2016) The troubling rise of rapid dna testing. goo gl/uuhZSN

  16. Laiadi O, Ouamane A, Benakcha A, Taleb-Ahmed A, Hadid A (2019) Learning multi-view deep and shallow features through new discriminative subspace for bi-subject and tri-subject kinship verification. Appl Intell 49(11):3894–3908

    Article  Google Scholar 

  17. Laiadi O, Ouamane A, Benakcha A, Taleb-Ahmed A, Hadid A (2020) Tensor cross-view quadratic discriminant analysis for kinship verification in the wild. Neurocomputing 377:286–300

    Article  Google Scholar 

  18. Laiadi O, Ouamane A, Boutellaa E, Benakcha A, Taleb-Ahmed A, Hadid A (2019) Kinship verification from face images in discriminative subspaces of color components. Multimedia Tools and Applications 78(12):16465–16487

    Article  Google Scholar 

  19. Lee TS (1996) Image representation using 2d gabor wavelets. IEEE Transactions on pattern analysis and machine intelligence 18(10):959–971

    Article  Google Scholar 

  20. Li C, Huang Y, Xue Y (2019) Dependence structure of gabor wavelets based on copula for face recognition. Expert Syst Appl 137:453–470

    Article  Google Scholar 

  21. Li W, Wu Y, Li J (2017) Re-identification by neighborhood structure metric learning. Pattern Recogn 61:327–338

    Article  Google Scholar 

  22. Liang J, Hu Q, Dang C, Zuo W (2018) Weighted graph embedding-based metric learning for kinship verification. IEEE Trans Image Process 28(3):1149–1162

    Article  MathSciNet  Google Scholar 

  23. Liao S, Hu Y, Zhu X, Li SZ (2015) Person re-identification by local maximal occurrence representation and metric learning. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 2197–2206

  24. Liong VE, Lu J, Ge Y (2015) Regularized local metric learning for person re-identification. Pattern Recogn Lett 68:288–296

    Article  Google Scholar 

  25. Lu H, Plataniotis KN, Venetsanopoulos AN (2008) Mpca: Multilinear principal component analysis of tensor objects. IEEE transactions on Neural Networks 19(1):18–39

    Article  Google Scholar 

  26. Lu H, Plataniotis KN, Venetsanopoulos AN (2008) Uncorrelated multilinear discriminant analysis with regularization and aggregation for tensor object recognition. IEEE Transactions on Neural Networks 20(1):103–123

    Google Scholar 

  27. Lu J, Hu J, Tan YP (2017) Discriminative deep metric learning for face and kinship verification. IEEE Trans Image Process 26(9):4269–4282

    Article  MathSciNet  Google Scholar 

  28. Lu J, Zhou X, Tan YP, Shang Y, Zhou J (2013) Neighborhood repulsed metric learning for kinship verification. IEEE transactions on pattern analysis and machine intelligence 36(2):331–345

    Google Scholar 

  29. Mahpod S, Keller Y (2018) Kinship verification using multiview hybrid distance learning. Comput Vis Image Underst 167:28–36

    Article  Google Scholar 

  30. Nguyen HV, Bai L (2010) Cosine similarity metric learning for face verification. In: Asian conference on computer vision, pp. 709–720. Springer

  31. Ojansivu V, Heikkilä J. (2008) Blur insensitive texture classification using local phase quantization. In: International conference on image and signal processing, pp. 236–243. Springer

  32. Ouamane A, Chouchane A, Boutellaa E, Belahcene M, Bourennane S, Hadid A (2017) Efficient tensor-based 2d + 3d face verification. IEEE Transactions on Information Forensics and Security 12 (11):2751–2762

    Article  Google Scholar 

  33. Pang Y, Wang S, Yuan Y (2014) Learning regularized lda by clustering. IEEE transactions on neural networks and learning systems 25(12):2191–2201

    Article  Google Scholar 

  34. Parkhi OM, Vedaldi A, Zisserman A (2015) Deep face recognition. In: British machine vision conference

  35. Qin X, Tan X, Chen S (2015) Tri-subject kinship verification: Understanding the core of a family. IEEE Transactions on Multimedia 17(10):1855–1867

    Article  Google Scholar 

  36. Xia S, Shao M, Luo J, Fu Y (2012) Understanding kin relationships in a photo. IEEE Transactions on Multimedia 14(4):1046–1056

    Article  Google Scholar 

  37. Yan H (2017) Kinship verification using neighborhood repulsed correlation metric learning. Image Vis Comput 60:91–97

    Article  Google Scholar 

  38. Yan H, Lu J, Deng W, Zhou X (2014) Discriminative multimetric learning for kinship verification. IEEE Transactions on Information forensics and security 9(7):1169–1178

    Article  Google Scholar 

  39. Yan S, Xu D, Yang Q, Zhang L, Tang X, Zhang HJ (2006) Multilinear discriminant analysis for face recognition. IEEE Trans Image Process 16(1):212–220

    Article  MathSciNet  Google Scholar 

  40. Yang D, Jiao L, Gong M, Liu F (2011) Artificial immune multi-objective sar image segmentation with fused complementary features. Inf Sci 181(13):2797–2812

    Article  Google Scholar 

  41. Yang Y, Sun J (2010) Face recognition based on gabor feature extraction and fractal coding. In: 2010 Third international symposium on electronic commerce and security, pp. 302–306. IEEE

  42. Yu H, Chung C, Wong K, Lee H, Zhang J (2009) Probabilistic load flow evaluation with hybrid latin hypercube sampling and cholesky decomposition. IEEE Transactions on Power Systems 24(2):661–667

    Article  Google Scholar 

  43. Yuan S, Mao X, Chen L (2017) Multilinear spatial discriminant analysis for dimensionality reduction. IEEE Trans Image Process 26(6):2669–2681

    Article  MathSciNet  Google Scholar 

  44. Zhang L, Duan Q, Zhang D, Jia W, Wang x. (2020) Advkin: Adversarial convolutional network for kinship verification. IEEE Transactions on Cybernetics

  45. Zhou X, Jin K, Xu M, Guo G (2019) Learning deep compact similarity metric for kinship verification from face images. Information Fusion 48:84–94

    Article  Google Scholar 

  46. Zhou X, Lu J, Hu J, Shang Y (2012) Gabor-based gradient orientation pyramid for kinship verification under uncontrolled environments. In: Proceedings of the 20th ACM international conference on Multimedia, pp. 725–728

Download references

Author information

Authors and Affiliations

  1. Laboratory of LI3C, University of Biskra, Biskra, Algeria

    Mohcene Bessaoudi & Abdelmalik Ouamane

  2. University of Yahia Fares Medea, Médéa, Algeria

    Ammar Chouchane

  3. Telecommunication division, Centre de Développement des Technologies Avancées, Algiers, Algeria

    Elhocine Boutellaa

Authors
  1. Mohcene Bessaoudi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ammar Chouchane
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Abdelmalik Ouamane
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Elhocine Boutellaa
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mohcene Bessaoudi.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Bessaoudi, M., Chouchane, A., Ouamane, A. et al. Multilinear subspace learning using handcrafted and deep features for face kinship verification in the wild. Appl Intell 51, 3534–3547 (2021). https://doi.org/10.1007/s10489-020-02044-0

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-020-02044-0

Keywords

Search

Navigation