Skip to main content
SpringerLink
Log in

Family classification and kinship verification from facial images in the wild

  • Original Paper
  • Published:
Machine Vision and Applications Aims and scope Submit manuscript

Abstract

Kinship verification from facial images in the wild based on one-to-one classification has gathered a promising attention by image processing and computer vision researchers. While family classification based on one-to-many classification is relatively the least explored domain in computer vision. This paper first performs family classification on different family-sets based on number of family members. Second, we perform kinship verification on different kinship relations covering parent–child and siblings. We present a new kinship database named KinIndian dedicated for these two tasks of family classification and kinship verification. KinIndian database comprises 1926 images of 813 individuals from 230 unique Indian families with 2–7 members. KinIndian is designed into two levels: the first is family-level for family classification, and the second is photo-level for kinship verification. We propose a novel weighted nearest member metric leaning (WNMML) method to evaluate family classification on different family-sets. Proposed WNMML method is based on minimizing intraclass separation by characterizing compactness for positive families and maximizing interclass separation by pushing members of negative families as far as possible. WNMML achieves competitive accuracy on different family-sets and hence shows that WNMML could be effectively used in real-world scenarios. Furthermore, we also perform kinship verification on KinIndian using baseline multimetric learning methods and achieves promising and encouraging kinship accuracy.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

Notes

  1. KinIndian database has single image for each member, which means, \(M=1\) and \(L=1\) for KinIndian.

  2. Family 101 has very less families with F3, F4, F5, F6 have 8, 13, 22, 24, respectively.

References

  1. Goyal, A., Meenpal, T.: Patch-based dual-tree complex wavelet transform for kinship recognition. IEEE Trans. Image Process. 30, 191–206 (2020)

    Article  Google Scholar 

  2. Goyal, A., Meenpal, T.: Eccentricity based kinship verification from facial images in the wild. Pattern Anal. Appl. 1–26 (2020)

  3. Robinson, J.P., Shao, M., Wu, Y., Liu, H., Gillis, T., Fu, Y.: Visual kinship recognition of families in the wild. IEEE Trans. Pattern Anal. Mach. Intell. 40(11), 2624–2637 (2018)

    Article  Google Scholar 

  4. Liu, W., Wen, Y., Yu, Z., Li, M., Raj, B., Song, L.: Sphereface: deep hypersphere embedding for face recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 212–220 (2017)

  5. Luo, Z., Zhang, Z., Xu, Z., Che, L.: Challenge report recognizing families in the wild data challenge. In: 2020 15th IEEE International Conference on Automatic Face and Gesture Recognition (FG 2020), pp. 868–871. IEEE (2020)

  6. Yu, J., Xie, G., Li, M., Hao, X.: Retrieval of family members using siamese neural network. In: 2020 15th IEEE International Conference on Automatic Face and Gesture Recognition (FG 2020), pp. 882–886. IEEE (2020)

  7. Robinson, J.P., Khan, Z., Yin, Y., Shao, M., Fu, Y.: Families in wild multimedia (fiw mm): a multimodal database for recognizing kinship. IEEE Trans. Multimed. (2021)

  8. Shadrikov, A.: Achieving better kinship recognition through better baseline. In: 2020 15th IEEE International Conference on Automatic Face and Gesture Recognition (FG 2020), pp. 872–876. IEEE (2020)

  9. Fang, R., Tang, K.D., Snavely, N., Chen, T.: Towards computational models of kinship verification. In: 2010 IEEE International Conference on Image Processing, pp. 1577–1580. IEEE (2010)

  10. Lu, J., Zhou, X., Tan, Y.P., Shang, Y., Zhou, J.: Neighborhood repulsed metric learning for kinship verification. IEEE Trans. Pattern Anal. Mach. Intell. 36(2), 331–345 (2014)

    Article  Google Scholar 

  11. Xia, S., Shao, M., Luo, J., Fu, Y.: Understanding kin relationships in a photo. IEEE Trans. Multimed. 14(4), 1046–1056 (2012)

    Article  Google Scholar 

  12. Fang, R., Gallagher, A.C., Chen, T., Loui, A.: Kinship classification by modeling facial feature heredity. In: 2013 20th IEEE International Conference on Image Processing (ICIP), pp. 2983–2987. IEEE (2013)

  13. Qin, X., Tan, X., Chen, S.: Tri-subject kinship verification: understanding the core of a family. IEEE Trans. Multimed. 17(10), 1855–1867 (2015)

    Article  Google Scholar 

  14. Shao, M., Xia, S., Fu, Y.: Genealogical face recognition based on ub kinface database. In: 2011 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 60–65. IEEE (2011)

  15. Zhou, X., Hu, J., Lu, J., Shang, Y., Guan, Y.: Kinship verification from facial images under uncontrolled conditions. In: Proceedings of the 19th ACM International Conference on Multimedia, pp. 953–956. ACM (2011)

  16. Guo, G., Wang, X.: Kinship measurement on salient facial features. IEEE Trans. Instrum. Meas. 61(8), 2322–2325 (2012)

    Article  Google Scholar 

  17. Kohli, N., Singh, R., Vatsa, M.: Self-similarity representation of weber faces for kinship classification. In: 2012 IEEE Fifth International Conference on Biometrics: Theory, Applications and Systems (BTAS), pp. 245–250. IEEE (2012)

  18. Zhou, X., Lu, J., Hu, J., Shang, Y.: Gabor-based gradient orientation pyramid for kinship verification under uncontrolled environments. In: Proceedings of the 20th ACM international conference on Multimedia, pp. 725–728. ACM (2012)

  19. Yan, H., Lu, J., Deng, W., Zhou, X.: Discriminative multimetric learning for kinship verification. IEEE Trans. Inf. Forensics Secur. 9(7), 1169–1178 (2014)

    Article  Google Scholar 

  20. Yan, H., Lu, J., Zhou, X.: Prototype-based discriminative feature learning for kinship verification. IEEE Trans. Cybern. 45(11), 2535–2545 (2015)

    Article  Google Scholar 

  21. Zhang, K., Huang, Y., Song, C., Wu, H., Wang, L.: Kinship verification with deep convolutional neural networks. In: Proceedings of the British Machine Vision Conference (BMVC), pp. 148.1–148.12. BMVA Press (2015)

  22. Wang, M., Li, Z., Shu, X., Tang, J., et al.: Deep kinship verification. In: 2015 IEEE 17th International Workshop on Multimedia Signal Processing (MMSP), pp. 1–6. IEEE (2015)

  23. Yan, H., Zhou, X., Ge, Y.: Neighborhood repulsed correlation metric learning for kinship verification. In: Visual Communications and Image Processing (VCIP), 2015, pp. 1–4. IEEE (2015)

  24. Zhou, X., Shang, Y., Yan, H., Guo, G.: Ensemble similarity learning for kinship verification from facial images in the wild. Inf. Fusion 32, 40–48 (2016)

    Article  Google Scholar 

  25. Xu, M., Shang, Y.: Kinship measurement on face images by structured similarity fusion. IEEE Access 4, 10280–10287 (2016)

    Article  Google Scholar 

  26. López, M.B., Boutellaa, E., Hadid, A.: Comments on the “kinship face in the wild” data sets. IEEE Trans. Pattern Anal. Mach. Intell. 38(11), 2342–2344 (2016)

  27. Zhang, J., Xia, S., Pan, H., Qin, A.: A genetics-motivated unsupervised model for tri-subject kinship verification. In: 2016 IEEE International Conference on Image Processing (ICIP), pp. 2916–2920. IEEE (2016)

  28. Liu, Q., Puthenputhussery, A., Liu, C.: A novel inheritable color space with application to kinship verification. In: 2016 IEEE Winter Conference on Applications of Computer Vision (WACV), pp. 1–9. IEEE (2016)

  29. Zhou, X., Yan, H., Shang, Y.: Kinship verification from facial images by scalable similarity fusion. Neurocomputing 197, 136–142 (2016)

    Article  Google Scholar 

  30. Puthenputhussery, A., Liu, Q., Liu, C.: Sift flow based genetic fisher vector feature for kinship verification. In: 2016 IEEE International Conference on Image Processing (ICIP), pp. 2921–2925 (Sept 2016)

  31. Kohli, N., Vatsa, M., Singh, R., Noore, A., Majumdar, A.: Hierarchical representation learning for kinship verification. IEEE Trans. Image Process. 26(1), 289–302 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  32. Hu, J., Lu, J., Tan, Y.P., Yuan, J., Zhou, J.: Local large-margin multi-metric learning for face and kinship verification. IEEE Trans. Circ. Syst. Video Technol. 28(8), 1875–1891 (2017)

    Article  Google Scholar 

  33. Patel, B., Maheshwari, R., Raman, B.: Evaluation of periocular features for kinship verification in the wild. Comput. Vis. Image Underst. 160, 24–35 (2017)

    Article  Google Scholar 

  34. Lu, J., Hu, J., Tan, Y.P.: Discriminative deep metric learning for face and kinship verification. IEEE Trans. Image Process. 26(9), 4269–4282 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  35. Qin, X., Liu, D., Wang, D.: Heterogeneous similarity learning for more practical kinship verification. Neural Process. Lett. 1–17 (2017)

  36. Zhao, Y.G., Song, Z., Zheng, F., Shao, L.: Learning a multiple kernel similarity metric for kinship verification. Inf. Sci. 430, 247–260 (2018)

    Article  MathSciNet  Google Scholar 

  37. Liang, J., Hu, Q., Dang, C., Zuo, W.: Weighted graph embedding-based metric learning for kinship verification. IEEE Trans. Image Process. (2018)

  38. Yan, H.: Learning discriminative compact binary face descriptor for kinship verification. Pattern Recogn. Lett. 117, 146–152 (2018)

    Article  Google Scholar 

  39. Moujahid, A., Dornaika, F.: A pyramid multi-level face descriptor: application to kinship verification. Multimed. Tools Appl. 78(7), 9335–9354 (2019)

    Article  Google Scholar 

  40. Zhou, X., Jin, K., Xu, M., Guo, G.: Learning deep compact similarity metric for kinship verification from face images. Inf. Fusion 48, 84–94 (2019)

    Article  Google Scholar 

  41. Wei, Z., Xu, M., Geng, L., Liu, H., Yin, H.: Adversarial similarity metric learning for kinship verification. IEEE Access 7, 100029–100035 (2019)

    Article  Google Scholar 

  42. Laiadi, O., Ouamane, A., Benakcha, A., Taleb-Ahmed, A., Hadid, A.: Kinship verification based deep and tensor features through extreme learning machine. In: 2019 14th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2019), pp. 1–4. IEEE (2019)

  43. Lu, J., Hu, J., Liong, V.E., Zhou, X., Bottino, A., Islam, I.U., Vieira, T.F., Qin, X., Tan, X., Chen, S., Mahpod, S., Keller, Y., Zheng, L., Idrissi, K., Garcia, C., Duffner, S., Baskurt, A., Castrillón-Santana, M., Lorenzo-Navarro, J.: The fg 2015 kinship verification in the wild evaluation. In: 2015 11th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG), vol. 1, pp. 1–7 (2015)

  44. Hermans, A., Beyer, L., Leibe, B.: In defense of the triplet loss for person re-identification. arXiv:1703.07737 (2017)

  45. Viola, P., Jones, M.: Rapid object detection using a boosted cascade of simple features. In: Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2001. CVPR 2001, vol. 1. IEEE (2001)

  46. Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2005. CVPR 2005, vol. 1, pp. 886–893. IEEE (2005)

  47. Ahonen, T., Hadid, A., Pietikainen, M.: Face description with local binary patterns: application to face recognition. IEEE Trans. Pattern Anal. Mach. Intell. 12, 2037–2041 (2006)

    Article  MATH  Google Scholar 

  48. Parkhi, O.M., Vedaldi, A., Zisserman, A., et al.: Deep face recognition. BMVC 1, 6 (2015)

    Google Scholar 

  49. Wolf, L., Hassner, T., Taigman, Y.: Descriptor based methods in the wild. In: Workshop on Faces in’Real-Life’Images: Detection, Alignment, and Recognition (2008)

  50. Liu, C.: Gabor-based kernel pca with fractional power polynomial models for face recognition. IEEE Trans. Pattern Anal. Mach. Intell. 26(5), 572–581 (2004)

    Article  Google Scholar 

Download references

Acknowledgements

We are thankful to Science and Engineering Research Board (SERB), Department of Science and Technology (DST), Government of India for research grant. The research work is sanctioned project titled as “Design and development of an Automatic Kinship Verification system for Indian faces with possible integration of AADHAR Database” with reference no. ECR/2016/001659.

Author information

Authors and Affiliations

  1. Department of Electronics and Communication, National Institute of Technology Raipur, Raipur, Chhattisgarh, 492010, India

    Aarti Goyal, Toshanlal Meenpal & Moumita Mukherjee

Authors
  1. Aarti Goyal
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Toshanlal Meenpal
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Moumita Mukherjee
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Toshanlal Meenpal.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Goyal, A., Meenpal, T. & Mukherjee, M. Family classification and kinship verification from facial images in the wild. Machine Vision and Applications 33, 88 (2022). https://doi.org/10.1007/s00138-022-01341-7

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s00138-022-01341-7

Keywords

Search

Navigation