Multimedia Tools and Applications

, Volume 76, Issue 3, pp 4105–4122 | Cite as

Kinship verification in multi-linear coherent spaces

  • Xiaojing Chen
  • Le An
  • Songfan Yang
  • Weimin Wu


Discovering kinship relations from face images in the wild has become an interesting and important problem in multimedia and computer vision. Despite the rapid advances in face analysis in unconstrained environment, kinship verification still remains a challenging problem as the subtle kinship relation is difficult to discover and changes in pose and lighting condition further complicate this task. In this paper, we propose a kinship verification approach based on multi-linear coherent space learning. Local image patches at different scales are independently projected into their corresponding coherent spaces learned by robust canonical correlation analysis such that patch pairs with kinship relations have improved correlation. In addition, most discriminative patches for verification are selected via constrained linear programming. Experimental results on two widely used kinship verification datasets show that the proposed method can effectively identify different kinship relations in image pairs. Compared to state-of-the-art techniques, the proposed method achieves very competitive performance with the use of simple feature descriptors.


Kinship verification Multi-linear coherent space learning Patch selection 


  1. 1.
    Ahonen T, Hadid A, Pietikainen M (2006) Face description with local binary patterns: application to face recognition. IEEE Trans Pattern Anal Mach Intell 28(12):2037–2041CrossRefzbMATHGoogle Scholar
  2. 2.
    Alvergne A, Oda R, Faurie C, Matsumoto-Oda A, Durand V, Raymond M (2009) Cross-cultural perceptions of facial resemblance between kin. J Vis 9(6)Google Scholar
  3. 3.
    An L, Bhanu B (2014) Face image super-resolution using 2D CCA. Signal Process 103:184–194CrossRefGoogle Scholar
  4. 4.
    An L, Yang S, Bhanu B (2015) Person re-identification by robust canonical correlation analysis. IEEE Sign Process Lett 22(8):1103–1107CrossRefGoogle Scholar
  5. 5.
    Arashloo S, Kittler J (2014) Class-specific kernel fusion of multiple descriptors for face verification using multiscale binarised statistical image features. IEEE Trans Inf Forensic Secur 9(12):2100–2109CrossRefGoogle Scholar
  6. 6.
    Chan CH, Kittler J, Poh N, Ahonen T, Pietikainen M (2009) (multiscale) local phase quantisation histogram discriminant analysis with score normalisation for robust face recognition. In: IEEE 12th international conference on computer vision workshops (ICCV Workshops), pp 633–640Google Scholar
  7. 7.
    Chang K, Chen C (2015) A learning framework for age rank estimation based on face images with scattering transform. IEEE Trans Image Process 24(3):785–798MathSciNetCrossRefGoogle Scholar
  8. 8.
    Chen BC, Chen YY, Kuo YH, Hsu W (2013) Scalable face image retrieval using attribute-enhanced sparse codewords. IEEE Trans Multimed 15(5):1163–1173CrossRefGoogle Scholar
  9. 9.
    Chen X, An L, Bhanu B (2013) Improving large-scale face image retrieval using multi-level features. In: IEEE international conference on image processing (ICIP), pp 4367–4371Google Scholar
  10. 10.
    Chen Y, Wiesel A, Eldar Y, Hero A (2010) Shrinkage algorithms for MMSE covariance estimation. IEEE Trans Signal Process 58(10):5016–5029MathSciNetCrossRefGoogle Scholar
  11. 11.
    Dal Martello MF, Maloney LT (2006) Where are kin recognition signals in the human face? J Vis 6(12)Google Scholar
  12. 12.
    Dal Martello MF, Maloney LT (2010) Lateralization of kin recognition signals in the human face. J Vis 10(8)Google Scholar
  13. 13.
    Dèniz O, Bueno G, Salido J, la Torre FD (2011) Face recognition using histograms of oriented gradients. Pattern Recogn Lett 32(12):1598–1603CrossRefGoogle Scholar
  14. 14.
    Dibeklioglu H, Salah A, Gevers T (2013) Like father, like son: facial expression dynamics for kinship verification. In: IEEE international conference on computer vision (ICCV), pp 1497–1504Google Scholar
  15. 15.
    Eleftheriadis S, Rudovic O, Pantic M (2015) Discriminative shared gaussian processes for multiview and view-invariant facial expression recognition. IEEE Trans Image Process 24(1):189–204MathSciNetCrossRefGoogle Scholar
  16. 16.
    Gao Y, Ji R, Liu W, Dai Q, Hua G (2014) Weakly supervised visual dictionary learning by harnessing image attributes. IEEE Trans Image Process 23(12):5400–5411MathSciNetCrossRefGoogle Scholar
  17. 17.
    Gao Y, Wang M, Zha ZJ, Shen J, Li X, Wu X (2013) Visual-textual joint relevance learning for tag-based social image search. IEEE Trans Image Process 22(1):363–376MathSciNetCrossRefGoogle Scholar
  18. 18.
    Guo Y, Dibeklioglu H, Van Der Maaten L (2014) Graph-based kinship recognition. In: International Conference on Pattern Recognition (ICPR), pp 4287–4292Google Scholar
  19. 19.
    He X, Yan S, Hu Y, Niyogi P, Zhang HJ (2005) Face recognition using laplacianfaces. IEEE Trans Pattern Anal Mach Intell 27(3):328–340CrossRefGoogle Scholar
  20. 20.
    Hotelling H (1936) Relations between two sets of variates. Biometrika 28 (3/4):321–377CrossRefzbMATHGoogle Scholar
  21. 21.
    Hu J, Lu J, Yuan J, Tan YP (2014) Large margin multi-metric learning for face and kinship verification in the wild. In: Asian conference on computer vision (ACCV)Google Scholar
  22. 22.
    Huang H, He H, Fan X, Zhang J (2010) Super-resolution of human face image using canonical correlation analysis. Pattern Recog 43(7):2532–2543CrossRefzbMATHGoogle Scholar
  23. 23.
    Kafai M, An L, Bhanu B (2014) Reference face graph for face recognition. IEEE Trans Inf Forensic Secur 9(12):2132–2143CrossRefGoogle Scholar
  24. 24.
    Kafai M, Eshghi K, Bhanu B (2014) Discrete cosine transform locality-sensitive hashes for face retrieval. IEEE Trans Multimed 16(4):1090–1103CrossRefGoogle Scholar
  25. 25.
    Kaminski G, Dridi S, Graff C, Gentaz E (2009) Human ability to detect kinship in strangers’ faces: effects of the degree of relatedness. Proc R Soc Lond B Biol Sci 276(1670):3193–3200CrossRefGoogle Scholar
  26. 26.
    Ledoit O, Wolf M (2012) Nonlinear shrinkage estimation of large-dimensional covariance matrices. Ann Stat 40(2):1024–1060MathSciNetCrossRefzbMATHGoogle Scholar
  27. 27.
    Lei Z, Li SZ (2012) Fast multi-scale local phase quantization histogram for face recognition. Pattern Recogn Lett 33(13):1761–1767CrossRefGoogle Scholar
  28. 28.
    Lu J, Hu J, Liong VE, Zhou X, Bottino A, Islam IU, Vieira TF, Qin X, Tan X, Chen S, Mahpod S, Keller Y, Zheng L, Idrissi K, Garcia C, Duffner S, Baskurt A, Castrillon-Santana M, Lorenzo-Navarro J (2015) The FG 2015 kinship verification in the wild evaluation. In: IEEE international conference on automatic face and gesture recognition (FG)Google Scholar
  29. 29.
    Lu J, Hu J, Zhou X, Shang Y, Tan YP, Wang G (2012) Neighborhood repulsed metric learning for kinship verification. In: IEEE conference on computer vision and pattern recognition (CVPR), pp 2594– 2601Google Scholar
  30. 30.
    Lu J, Hu J, Zhou X, Zhou J, Castrillon-Santana M, Lorenzo-Navarro J, Kou L, Shang Y, Bottino A, Vieira TF (2014) Kinship verification in the wild: The first kinship verification competition. In: IEEE international joint conference on biometrics (IJCB), pp 1–6Google Scholar
  31. 31.
    Lu J, Tan YP (2013) Ordinary preserving manifold analysis for human age and head pose estimation. IEEE Transactions on Human-Machine Systems 43(2):249–258CrossRefGoogle Scholar
  32. 32.
    Lu J, Zhou X, Tan YP, Shang Y, Zhou J (2014) Neighborhood repulsed metric learning for kinship verification. IEEE Trans Pattern Anal Mach Intell 36(2):331–345CrossRefGoogle Scholar
  33. 33.
    Meina Kan Shiguang Shan DX, Chen X (2011) Side-information based linear discriminant analysis for face recognition. In: British machine vision conference, pp 125.1–125.0Google Scholar
  34. 34.
    Somanath G, Kambhamettu C (2012) Can faces verify blood-relations? In: IEEE international conference on biometrics: theory, applications and systems (BTAS), pp 105–112Google Scholar
  35. 35.
    Taigman Y, Yang M, Ranzato M, Wolf L (2014) Deepface: closing the gap to human-level performance in face verification. In: IEEE conference on computer vision and pattern recognition (CVPR), pp 1701–1708Google Scholar
  36. 36.
    Tao D, Jin L, Wang Y, Yuan Y, Li X (2013) Person re-identification by regularized smoothing KISS metric learning. IEEE Trans Circuits Syst Video Technol 23(10):1675–1685CrossRefGoogle Scholar
  37. 37.
    Wang H (2010) Local two-dimensional canonical correlation analysis. IEEE Signal Proc Lett 17(11):921–924CrossRefGoogle Scholar
  38. 38.
    Wilson D, Weisburd D, McClure D (2011) Use of dna testing in police investigative work for increasing offender identification, arrest, conviction and case clearance. Campbell Syst Rev 7Google Scholar
  39. 39.
    Wu L, Jin R, Jain A (2013) Tag completion for image retrieval. IEEE Trans Pattern Anal Mach Intell 35(3):716–727CrossRefGoogle Scholar
  40. 40.
    Yan H, Lu J, Deng W, Zhou X (2014) Discriminative multimetric learning for kinship verification. IEEE Trans Inf Forensic Secur 9(7):1169–1178CrossRefGoogle Scholar
  41. 41.
    Yan H, Lu J, Zhou X (2014) Prototype-based discriminative feature learning for kinship verification. IEEE Trans Cybern PP(99):1–1Google Scholar
  42. 42.
    Yang S, Bhanu B (2012) Understanding discrete facial expressions in video using an emotion avatar image. IEEE Trans Syst Man Cybern B Cybern 42(4):980–992CrossRefGoogle Scholar
  43. 43.
    Yang W, Yi D, Lei Z, Sang J, Li S (2008) 2D-3D face matching using CCA. In: IEEE international conference on automatic face gesture recognition, pp 1–6Google Scholar
  44. 44.
    Zhang K, Tao D, Gao X, Li X, Xiong Z (2015) Learning multiple linear mappings for efficient single image super-resolution. IEEE Trans Image Process 24(3):846–861MathSciNetCrossRefGoogle Scholar
  45. 45.
    Zheng L, Idrissi K, Garcia C, Duffner S, Baskurt A (2015) Triangular similarity metric learning for face verification. In: IEEE international conference on automatic face and gesture recognition (FG)Google Scholar
  46. 46.
    Zheng W, Zhou X, Zou C, Zhao L (2006) Facial expression recognition using kernel canonical correlation analysis (KCCA). IEEE Trans Neural Networks 17 17(1):233–238Google Scholar
  47. 47.
    Zhou X, Hu J, Lu J, Shang Y, Guan Y (2011) Kinship verification from facial images under uncontrolled conditions. In: ACM international conference on multimedia, pp 953–956Google Scholar
  48. 48.
    Zhou X, Lu J, Hu J, Shang Y (2012) Gabor-based gradient orientation pyramid for kinship verification under uncontrolled environments. In: ACM international conference on multimedia, pp 725– 728Google Scholar

Copyright information

© Springer Science+Business Media New York 2015

Authors and Affiliations

  1. 1.Department of Computer Science and EngineeringUniversity of CaliforniaRiversideUSA
  2. 2.Department of Electrical and Computer EngineeringUniversity of CaliforniaRiversideUSA
  3. 3.College of Electronics and Information EngineeringSichuan UniversityChengduChina
  4. 4.School of Electronic Information and CommunicationsHuazhong University of Science and TechnologyWuhanChina

Personalised recommendations