Neural Processing Letters

, Volume 47, Issue 3, pp 1253–1269 | Cite as

Heterogeneous Similarity Learning for More Practical Kinship Verification

  • Xiaoqian QinEmail author
  • Dakun Liu
  • Dong Wang


Kinship verification via facial images is a relatively new and challenging problem in computer vision. Prior studies in the literature have focused solely on gender-fixed kin relation, i.e., on the question of whether one gender-fixed kin relationship between given subjects can be established. In practice, however, large scale gender annotation is time-consuming and expensive. Instead, we propose in this paper to learn and predict with gender-unknown kin relations. To address this, we present a novel heterogeneous similarity learning (HSL) method. Motivated by the fact that different kinship relations may not only share some common genetic characteristics but also have its own inherited traits from parents to offspring, we aim to learn a similarity function under which the commonality among different kinship relations are captured and the geometry of each relation is preserved, simultaneously. We further derive a multi-view HSL method by optimal fusion of the similarity models from multiple feature representations, such that the complementary knowledge in multi-view kin data can be leveraged to obtain refined information. Experimental results demonstrate the effectiveness of our proposed methods.


Kinship verification Kin similarity Metric learning Multi-view learning 


  1. 1.
    Ahonen T, Hadid A, Pietikainen M (2006) Face description with local binary patterns: application to face recognition. IEEE Trans Pattern Anal Mach Intell 28(12):2037–2041CrossRefzbMATHGoogle Scholar
  2. 2.
    Belkin M, Niyogi P, Sindhwani V (2006) Manifold regularization: a geometric framework for learning from labeled and unlabeled examples. J Mach Learn Res 7(1):2399–2434MathSciNetzbMATHGoogle Scholar
  3. 3.
    Benavent X, Garcia-Serrano A, Granados R, Benavent J, de Ves E (2013) Multimedia information retrieval based on late semantic fusion approaches: experiments on a Wikipedia image collection. IEEE Trans Multimed 15(8):2009–2021CrossRefGoogle Scholar
  4. 4.
    Chechik G, Sharma V, Shalit U, Bengio S (2009) Large scale online learning of image similarity through ranking. J Mach Learn Res 11:1109–1135MathSciNetzbMATHGoogle Scholar
  5. 5.
    Dalal N, Triggs B (2005) Histograms of oriented gradients for human detection. In: Proceedings of the 2005 IEEE computer society conference on computer vision and pattern recognition (CVPR’05), vol 1. IEEE Computer Society, Washington, DC, pp 886–893Google Scholar
  6. 6.
    Dehghan A, Ortiz EG, Villegas R, Shah M (2014) Who do i look like? Determining parent-offspring resemblance via gated autoencoders. In: 2014 IEEE conference on computer vision and pattern recognition (CVPR). IEEEGoogle Scholar
  7. 7.
    Deng J, Berg AC, Fei-Fei L (2011) Hierarchical semantic indexing for large scale image retrieval. In: CVPR ’11 Proceedings of the 2011 IEEE conference on computer vision and pattern recognition. IEEE Computer Society, Washington, DC, pp 785–792Google Scholar
  8. 8.
    Dibeklioglu H, Salah AA, Gevers T (2013) Like father, like son: facial expression dynamics for kinship verification. In: 2013 IEEE international conference on computer vision (ICCV), pp 1497–1504Google Scholar
  9. 9.
    Fang R, Tang KD, Snavely N, Chen T (2010) Towards computational models of kinship verification. In: 2010 IEEE international conference on image processing (ICIP). IEEE, pp 1577–1580Google Scholar
  10. 10.
    Gao X, Hoi SCH, Zhang Y, Wan J, Li J (2014) SOML: sparse online metric learning with application to image retrieval. In: Proceedings of AAAIGoogle Scholar
  11. 11.
    Ghahramani M, Yau WY, Teoh EK (2014) Family verification based on similarity of individual family members facial segments. Mach Vis Appl 25(4):919–930CrossRefGoogle Scholar
  12. 12.
    Guo G, Wang X (2012) Kinship measurement on salient facial features. IEEE Trans Instrum Meas 61(8):2322–2325CrossRefGoogle Scholar
  13. 13.
    Hu J, Lu J, Yuan J, Tan YP (2014) Large margin multimetric learning for face and kinship verification in the wild. In: Proceeding of ACCVGoogle Scholar
  14. 14.
    Jiang YG, Wang J, Xue X, Chang SF (2013) Query-adaptive image search with hash codes. IEEE Trans Multimed 15(2):442–453CrossRefGoogle Scholar
  15. 15.
    Kohli N, Singh R, Vatsa M (2012) Self-similarity representation of weber faces for kinship classification. In: 2012 IEEE fifth international conference on biometrics: theory, applications and systems (BTAS). IEEE, pp 245–250Google Scholar
  16. 16.
    Lowe DG (2004) Distinctive image features from scale-invariant keypoints. Int J Comput Vis 60(2):91–110CrossRefGoogle Scholar
  17. 17.
    Lu J, Hu J, Liong VE, Zhou X, Bottino A, Islam IU, Vieira TF, Qin X, Tan X, Keller Y et al (2015) The fg 2015 kinship verification in the wild evaluation. In: 2015 IEEE international conference on automatic face and gesture recognition (FG)Google Scholar
  18. 18.
    Lu J, Zhou X, Tan YP, Shang Y, Zhou J (2014) Neighborhood repulsed metric learning for kinship verification. IEEE Trans Pattern Anal Mach Intell 36(2):331–345CrossRefGoogle Scholar
  19. 19.
    Ma Z, Yang Y, Sebe N, Zheng K, Hauptmann AG (2013) Multimedia event detection using a classifier-specific intermediate representation. IEEE Trans Multimed 15(7):1628–1637CrossRefGoogle Scholar
  20. 20.
    Platek SM, Raines DM, Gallup GG Jr, Mohamed FB, Thomson JW, Myers TE, Panyavin IS, Levin SL, Davis JA, Fonteyn L et al (2004) Reactions to children’s faces: males are more affected by resemblance than females are, and so are their brains. Evol Hum Behav 25(6):394–405CrossRefGoogle Scholar
  21. 21.
    Qin X, Tan X, Chen S (2015) Tri-subject kinship verification: understanding the core of a family. IEEE Trans Multimed 17(10):1855–1867CrossRefGoogle Scholar
  22. 22.
    Qin X, Tan X, Chen S (2016) Mixed bi-subject kinship verification via multi-view multi-task learning. Neurocomputing 214:350–357CrossRefGoogle Scholar
  23. 23.
    Salter F (1996) Carrier females and sender males: an evolutionary hypothesis linking female attractiveness, family resemblance, and paternity confidence. Ethol Sociobiol 17(4):211–220CrossRefGoogle Scholar
  24. 24.
    Shao M, Kit D, Fu Y (2014) Generalized transfer subspace learning through low-rank constraint. Int J Comput Vis 109(1–2):74–93MathSciNetCrossRefzbMATHGoogle Scholar
  25. 25.
    Shao M, Xia S, Fu Y (2014) Identity and kinship relations in group pictures. In: Fu Y (ed) Human-centered social media analytics. Springer, Cham, pp 175–190Google Scholar
  26. 26.
    Spyromitros-Xioufis E, Papadopoulos S, Kompatsiaris I, Tsoumakas G, Vlahavas I (2014) A comprehensive study over vlad and product quantizationin large-scale image retrieval. IEEE Trans Multimed 16(6):1713–1728CrossRefGoogle Scholar
  27. 27.
    Sulem P, Gudbjartsson DF, Stacey SN, Helgason A, Rafnar T, Magnusson KP, Manolescu A, Karason A, Palsson A, Thorleifsson G et al (2007) Genetic determinants of hair, eye and skin pigmentation in europeans. Nat Genet 39(12):1443–1452CrossRefGoogle Scholar
  28. 28.
    Syed N, Patil BK, Shareq Mohd Q (2014) Understanding familial relationship in an image. Int J Sci Res Educ 2(6):1037–1045Google Scholar
  29. 29.
    Wang G, Gallagher A, Luo J, Forsyth D (2010) Seeing people in social context: recognizing people and social relationships. In: Computer vision–ECCV 2010. Springer, pp 169–182Google Scholar
  30. 30.
    Xia S, Shao M, Fu Y (2011) Kinship verification through transfer learning. In: Proceedings of the twenty-second international joint conference on artificial intelligence, vol 3. AAAI Press, pp 2539–2544Google Scholar
  31. 31.
    Xia S, Shao M, Fu Y (2012) Toward kinship verification using visual attributes. In: 2012 international conference on pattern recognition (ICPR). IEEE, pp 549–552Google Scholar
  32. 32.
    Xia S, Shao M, Luo J, Fu Y (2012) Understanding kin relationships in a photo. IEEE Trans Multimed 14(4):1046–1056CrossRefGoogle Scholar
  33. 33.
    Xu M, Shang Y (2016) Kinship verification using facial images by robust similarity learning. Math Probl Eng 2016:1–8Google Scholar
  34. 34.
    Xu Z, Zhang Y, Cao L (2014) Social image analysis from a non-iid perspective. IEEE Trans Multimed 16(7):1986–1998CrossRefGoogle Scholar
  35. 35.
    Yan H, Lu J, Deng W, Zhou X (2014) Discriminative multi-metric learning for kinship verification. IEEE Trans Inf Forensics Secur 9(7):1169–1178CrossRefGoogle Scholar
  36. 36.
    Yan H, Lu J, Zhou X (2014) Prototype-based discriminative feature learning for kinship verification. IEEE Trans Cybern 45:2535–2545CrossRefGoogle Scholar
  37. 37.
    Zhang Z, Chen Y, Saligrama V (2015) Group membership prediction. arXiv preprint arXiv:1509.04783
  38. 38.
    Zhou X, Hu J, Lu J, Shang Y, Guan Y (2011) Kinship verification from facial images under uncontrolled conditions. In: Proceedings of the 19th ACM international conference on multimedia. ACM, pp 953–956Google Scholar
  39. 39.
    Zhou X, Lu J, Hu J, Shang, Y (2012) Gabor-based gradient orientation pyramid for kinship verification under uncontrolled environments. In: Proceedings of the 20th ACM international conference on multimedia. ACM, pp 725–728Google Scholar
  40. 40.
    Zhou X, Yan H, Shang Y (2016) Kinship verification from facial images by scalable similarity fusion. Neurocomputing 197:136–142CrossRefGoogle Scholar
  41. 41.
    Zoidi O, Tefas A, Nikolaidis N, Pitas I (2014) Person identity label propagation in stereo videos. IEEE Trans Multimed 16(5):1358–1368CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2017

Authors and Affiliations

  1. 1.Huaiyin Normal UniversityHuai’anChina
  2. 2.Yancheng Institute of TechnologyYanchengChina
  3. 3.Nanjing University of Aeronautics and AstronauticsNanjingChina

Personalised recommendations