Skip to main content

Advertisement

SpringerLink
Log in

A novel factor analysis-based metric learning method for kinship verification

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

This paper presents a novel factor analysis-based metric learning (FAML) method for kinship verification. While metric learning has achieved reasonably good performance in kinship verification, most existing metric learning methods ignore to discover semantically meaningful similarity-patterns for kinship pairwise data. To address this, we propose a FAML method to seek hidden local similarity metrics, under which kin pairs would be relatively correlated and similar in certain facial regions. Particularly, to learn such local similarity metrics, we apply a series of transformations such as orthogonal rotation and thresholding to the factor loading matrix obtained through factor analysis. Moreover, instead of only seeking metrics in a local sense, we aim to simultaneously learn a set of distance metrics to integrate the locality with the globality, thus being more robust for diversified similarity-patterns that kin pairs contain. To jointly perform genetic characteristics exploiting and metric learning, we present an efficient algorithm that employs alternating optimization to integrate prior knowledge about genetic characteristics into metric learning, such that more discriminative information can be exploited in more fine-grained details for verification. Experiments are carried out on three face kinship datasets, and the results achieved clearly demonstrate the effectiveness of the proposed method.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. Ahonen T, Hadid A, Pietikainen M (2006) Face description with local binary patterns: Application to face recognition. IEEE Trans Pattern Anal Mach Intell 28(12):2037–2041

    Article  Google Scholar 

  2. Basilevsky A T (2009) Statistical factor analysis and related methods: theory and applications, vol 418. Wiley

  3. Bessaoudi M, Chouchane A, Ouamane A, Boutellaa E (2020) Multilinear subspace learning using handcrafted and deep features for face kinship verification in the wild. Appl Intell (4)

  4. Chechik G, Sharma V, Shalit U, Bengio S (2009) Large scale online learning of image similarity through ranking, pp 1109–1135

  5. Chowdary M K, Nguyen T N, Hemanth D J (2021) Deep learning-based facial emotion recognition for human ccomputer interaction applications. Neural Comput Appl:1–18

  6. Dehghan A, Ortiz E G, Villegas R, Shah M (2014) Who do i look like? determining parent-offspring resemblance via gated autoencoders. In: 2014 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE

  7. Deng J, Berg A C, Fei-Fei L (2011) Hierarchical semantic indexing for large scale image retrieval, pp 785–792

  8. Dibeklioglu H, Salah A A, Gevers T (2013) Like father, like son: Facial expression dynamics for kinship verification. In: 2013 IEEE International Conference on Computer Vision(ICCV), pp 1497–1504

  9. Dornaika F, Arganda-Carreras I, Serradilla O (2019) Transfer learning and feature fusion for kinship verification. Neural Comput Appl

  10. Fang R, Tang K D, Snavely N, Chen T (2010) Towards computational models of kinship verification. In: 2010 IEEE International Conference on Image Processing (ICIP). IEEE, pp 1577–1580

  11. Fang Y, Chen Y Y S, Wang H, Shu C (2017) Sparse similarity metric learning for kinship verification. In: Visual Communications and Image Processing, pp 1–4

  12. Gao X, Hoi S C H, Zhang Y, Wan J, Li J (2014) Soml: Sparse online metric learning with application to image retrieval. Proceedings of AAAI

  13. Gupta S, Mohan N, Kumar M A study on source device attribution using still images. Arch Comput Methods Eng:1–15

  14. Hu J, Lu J, Liu L, Zhou J (2019) Multi-view geometric mean metric learning for kinship verification. In: 2019 IEEE International Conference on Image Processing (ICIP)

  15. Hu J, Lu J, Tan Y P (2017) Sharable and individual multi-view metric learning. IEEE Trans Pattern Anal Mach Intell PP(99):1–1

    Google Scholar 

  16. Hu J, Lu J, Tan Y P, Yuan J, Zhou J (2017) Local large-margin multi-metric learning for face and kinship verification. IEEE Trans Circ Syst Video Technol PP(99):1–1

    Google Scholar 

  17. Hu J, Lu J, Yuan J, Tan Y-P (2014) Large margin multimetric learning for face and kinship verification in the wild. In: Proceedings of ACCV

  18. Kohli N, Singh R, Vatsa M (2012) Self-similarity representation of weber faces for kinship classification. In: 2012 IEEE Fifth International Conference on Biometrics: Theory, Applications and Systems (BTAS) . IEEE, pp 245–250

  19. Kou L, Zhou X, Xu M, Shang Y (2015) Learning a genetic measure for kinship verification using facial images. Math Probl Eng

  20. Kumar M, Bansal M, Kumar M (2020) 2d object recognition techniques: State-of-the-art work. Arch Comput Methods Eng (5)

  21. Kumar M, Chhabra P, Garg N K (2018) An efficient content based image retrieval system using bayesnet and k-nn. Multimed Tools Appl 77(16):1–14

    Article  Google Scholar 

  22. Laiadi O, Ouamane A, Benakcha A, Taleb-Ahmed A, Hadid A (2019) Learning multi-view deep and shallow features through new discriminative subspace for bi-subject and tri-subject kinship verification. Appl Intell (6)

  23. Laiadi O, Ouamane A, Benakcha A, Taleb-Ahmed A, Hadid A (2020) Tensor cross-view quadratic discriminant analysis for kinship verification in the wild. Neurocomputing 377:286–300

    Article  Google Scholar 

  24. Liang J, Hu Q, Dang C, Zuo W (2019) Weighted graph embedding-based metric learning for kinship verification. IEEE Trans Image Process 28 (3):1149–1162

    Article  MathSciNet  Google Scholar 

  25. Liu K H, Liua T J (2019) A structure-based human facial age estimation framework under a constrained condition. IEEE Trans Image Process PP (99):1–1

    MathSciNet  Google Scholar 

  26. Liu Q, Puthenputhussery A, Liu C (2016) A novel inheritable color space with application to kinship verification. In: 2016 IEEE Winter Conference on Applications of Computer Vision (WACV). IEEE, pp 1–9

  27. Lowe D G (2004) Distinctive image features from scale-invariant keypoints. Inte J Comput Vis 60(2):91–110

    Article  Google Scholar 

  28. Lu J, Hu J, Tan Y P (2017) Discriminative deep metric learning for face and kinship verification. IEEE Trans Image Process 26(9):4269–4282

    Article  MathSciNet  Google Scholar 

  29. Lu J, Hu J, Liong V E, Zhou X, Bottino A, Islam I U, Vieira T F, Qin X, Tan X, Keller Y et al (2015) The fg 2015 kinship verification in the wild evaluation. 2015 IEEE International Conference on Automatic Face and Gesture Recognition(FG)

  30. Lu J, Lu J, Deng W, Zhou X (2014) Discriminative multimetric learning for kinship verification. IEEE Trans Inf Forensic Secur 9(7):1169–1178

    Article  Google Scholar 

  31. Lu J, Zhou X, Tan Y-P, Shang Y, Zhou J (2014) Neighborhood repulsed metric learning for kinship verification. IEEE Trans Pattern Anal Mach Intell 36(2):331–345

    Article  Google Scholar 

  32. Mahpod S, Keller Y (2017) Kinship verification using multiview hybrid distance learning. Computer Vision & Image Understanding

  33. Parkhi O M, Vedaldi A, Zisserman A (2015) Deep face recognition. In: British Machine Vision Conference, vol 1, p 6

  34. Qin X, Liu D, Wang D (2018) Heterogeneous similarity learning for more practical kinship verification. Neural Process Lett 47(3):1253–1269

    Article  Google Scholar 

  35. Qin X, Liu D, Wang D (2019) Social relationships classification using social contextual features and svdd-based metric learning. Appl Soft Comput 77:344–355

    Article  Google Scholar 

  36. Qin X, Liu D, Wang D (2020) A literature survey on kinship verification through facial images. Neurocomputing 377:213–224

    Article  Google Scholar 

  37. Qin X, Tan X, Chen S (2015) Tri-subject kinship verification: Understanding the core of a family. IEEE Trans Multimed 17(10):1855–1867

    Article  Google Scholar 

  38. Qin X, Tan X, Chen S (2016) Mixed bi-subject kinship verification via multi-view multi-task learning. Neurocomputing

  39. Sellam A, Azzoune H (2020) Neighborhood min distance descriptor for kinship verification. Multimed Tools Appl 79(12)

  40. Shao M, Kit D, Fu Y (2014) Generalized transfer subspace learning through low-rank constraint. Int J Comput Vis 109(1-2):74–93

    Article  MathSciNet  Google Scholar 

  41. Shao M, Xia S, Fu Y (2014) Identity and kinship relations in group pictures. In: Human-Centered Social Media Analytics. Springer, pp 175–190

  42. Syed N, Shareq B K P P Q et al Understanding familial relationship in an image. Int J Sci Res Educ 2014,2(06)

  43. Wang S, Ding Z, Yun F (2018) Cross-generation kinship verification with sparse discriminative metric. IEEE Trans Pattern Anal Mach Intell PP:1–1

    Google Scholar 

  44. Wang S, Yan H (2020) Discriminative sampling via deep reinforcement learning for kinship verification. Pattern Recogn Lett

  45. Xia S, Shao M, Fu Y (2011) Kinship verification through transfer learning. In: Proceedings of the Twenty-Second international joint conference on Artificial Intelligence-Volume Volume Three. AAAI Press, pp 2539–2544

  46. Xia S, Shao M, Fu Y (2012) Toward kinship verification using visual attributes. In: 2012 International Conference on Pattern Recognition (ICPR). IEEE, pp 549–552

  47. Xia S, Shao M, Luo J, Fu Y (2012) Understanding kin relationships in a photo. IEEE Trans Multimed 14(4):1046–1056

    Article  Google Scholar 

  48. Xiaoqian Qin B G D W (2020) New metric learning model using statistical inference for kinship verification. Appl Soft Comput 95:1

    Google Scholar 

  49. Xu M, Shang Y (2016) Kinship verification using facial images by robust similarity learning. Math Probl Eng 2016:1–8

    Google Scholar 

  50. Xu M, Shang Y (2017) Kinship measurement on face images by structured similarity fusion. IEEE Access 4(99):10280–10287

    Google Scholar 

  51. Yan H, Song C (2020) Multi-scale deep relational reasoning for facial kinship verification. Pattern Recogn 110(2):107541

    Google Scholar 

  52. Yan H (2017) Kinship verification using neighborhood repulsed correlation metric learning. Image Vis Comput 60:91–97

    Article  Google Scholar 

  53. Yan H, Lu J, Zhou X (2014) Prototype-based discriminative feature learning for kinship verification. IEEE Trans Cybern PP(99):1

    Google Scholar 

  54. Zhang Z, Chen Y, Saligrama V (2015) Group membership prediction. arXiv:1509.04783

  55. Zhou X, Hu J, Lu J, Shang Y, Guan Y (2011) Kinship verification from facial images under uncontrolled conditions. In: Proceedings of the 19th ACM international conference on Multimedia. ACM, pp 953–956

  56. Zhou X, Jin K, Xu M, Guo G (2019) Learning deep compact similarity metric for kinship verification from face images. Inf Fusion 48:84–94

    Article  Google Scholar 

  57. Zhou X, Lu J, Hu J, Shang Y (2012) Gabor-based gradient orientation pyramid for kinship verification under uncontrolled environments. In: Proceedings of the 20th ACM international conference on Multimedia. ACM, pp 725–728

Download references

Acknowledgements

We thank the anonymous reviewers for your in-depth comments, suggestions, and corrections, which have greatly improved the manuscript. This work was supported by the National Natural Science Foundation of China(Grant No.61803170, No.62006046), the Natural Science Foundation of Jiangsu Province, China(Grant No.BK20181067), and the Humanities and Social Sciences Planning Foundation from the Ministry of Education of China(Grant No.18YJAZH070).

Author information

Authors and Affiliations

  1. Huaiyin Normal University, No.111 Changjiang West Road, Huai’an, 223300, China

    Xiaoqian Qin

  2. Yancheng Institute of Technology, Hope road No.1 Middle Road, Yancheng, 224051, China

    Dakun Liu

  3. Guangdong University of Technology, No. 100 Waihuan Xi Road, Guangzhou, 510006, China

    Dong Wang

Authors
  1. Xiaoqian Qin
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Dakun Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Dong Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xiaoqian Qin.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Qin, X., Liu, D. & Wang, D. A novel factor analysis-based metric learning method for kinship verification. Multimed Tools Appl 81, 11049–11070 (2022). https://doi.org/10.1007/s11042-022-12032-w

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-022-12032-w

Keywords

Search

Navigation