Skip to main content
SpringerLink
Log in

Siamese convolutional neural network and fusion of the best overlapping blocks for kinship verification

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Analysis of facial images decoding familial features has been attracting the attention of researchers to develop a computerized system interested in determining whether a pair of facial images have a biological kin relationship or not. Given that not all regions of an image are useful to determine the kin relation, thus it is possible to obtain irrelevant and inaccurate information of kinship clues, resulting in false matched kinship. Thus, combining all these regions together will likely produces redundant, irrelevant and deceptive information of kinship, along with higher dimensional space. Motivated by the fact that the facial resemblance among the members in a family can be presented separately in different regions of facial images, where each independent region renders different familial features, there is a high probability that selecting and fusing only the most informative local regions and removing the irrelevant can obtain complementary information for further enhanced accuracy. To this end, unlike other methods, the Fusion of the Best Overlapping Blocks with Siamese Convolutional Neural Network (SCNN-FBOB) is an enhanced method for kinship verification in this paper. This method aimed to simultaneously remove the weak local blocks of the image from a set of overlapping local blocks that achieved low accuracy and only retain the local blocks that achieved high accuracy. Extensive experiments conducted on the benchmark KinFaceW-I and KinFaceW-II databases show highly competitive results over many other state-of-the-art methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

References

  1. Abdellah S, Hamid A (2018) Towards a better training for siamese cnns on kinship verefication. In: International symposium on modelling and implementation of complex systems. Springer, pp 230–242

  2. Akerkar R (2014) Introduction to artificial intelligence. PHI Learning Pvt. Ltd., New Delhi

    Google Scholar 

  3. Aliradi R, Belkhir A, Ouamane A, Elmaghraby A S (2018) Dieda: discriminative information based on exponential discriminant analysis combined with local features representation for face and kinship verification. Multimedia Tools and Applications, pp 1–18

  4. Alirezazadeh P, Fathi A, Abdali-Mohammadi F (2015) A genetic algorithm-based feature selection for kinship verification. IEEE Signal Process Lett 22 (12):2459–2463

    Article  Google Scholar 

  5. Alirezazadeh P, Fathi A, Abdali-Mohammadi F (2016) Effect of purposeful feature extraction in high-dimensional kinship verification problem. Journal of Computing Security 3(3):183–191

    Google Scholar 

  6. Almuashi M, Hashim SZM, Mohamad D, Alkawaz MH, Ali A (2017) Automated kinship verification and identification through human facial images: a survey. Multimedia Tools and Applications 76(1):265–307

    Article  Google Scholar 

  7. Bastanfard A, Bastanfard O, Takahashi H, Nakajima M (2004) Toward anthropometrics simulation of face rejuvenation and skin cosmetic. Computer Animation and Virtual Worlds 15(3-4):347–352

    Article  Google Scholar 

  8. Bastanfard A, Takahashi H, Nakajima M (2004) Toward e-appearance of human face and hair by age, expression and rejuvenation. In: 2004 international conference on cyberworlds, pp 306–311

  9. Beham M P, Roomi S M M (2013) A review of face recognition methods. International Journal of Pattern Recognition Artificial Intelligence 27 (04):1356005

    Article  Google Scholar 

  10. Bottino A, De Simone M, Laurentini A, Vieira TF (2012) A new problem in face image analysis-finding kinship clues for siblings pairs. In: International conference on pattern recognition applications and methods ICPRAM, pp 405–410

  11. Bukovčiková Z, Sopiak D, Oravec M, Pavlovičová J (2017) Face verification using convolutional neural networks with siamese architecture. In: 2017 international symposium ELMAR. IEEE, pp 205–208

  12. Chen X, An L, Yang S, Wu W (2017) Kinship verification in multi-linear coherent spaces. Multimedia Tools and Applications 76(3):4105–4122

    Article  Google Scholar 

  13. Chergui A, Ouchtati S, Mavromatis S, Bekhouche S E, Sequeira J (2019) Investigating deep cnns models applied in kinship verification through facial images. In: 2019 5th international conference on frontiers of signal processing (ICFSP). IEEE, pp 82–87

  14. Chergui A, Ouchtati S, Mavromatis S, Bekhouche SE, Sequeira J, Zerrari H (2019) Kinship verification using mixed descriptors and multi block face representation. In: 2019 international conference on networking and advanced systems (ICNAS). IEEE, pp 1–6

  15. Chergui A, Ouchtati S, Sequeira J, Bekhouche S E, Bougourzi F, Telli H (2019) Deep features for kinship verification from facial images. In: 2019 international conference on advanced systems and emergent technologies (ICASET). IEEE, pp 64–67

  16. Chergui A, Ouchtati S, Telli H, Bougourzi F, Bekhouche SE (2018) Lpq and ldp descriptors with ml representation for kinship verification. In: The second edition of the international workshop on signal processing applied to rotating machinery diagnostics (SIGPROMDâĂŹ2018). Springer, pp 1–10

  17. Chollet F (2015) Keras documentation. keras. io, 33

  18. Chung D, Tahboub K, Delp EJ (2017) A two stream siamese convolutional neural network for person re-identification. In: Proceedings of the IEEE international conference on computer vision. IEEE, pp 1983–1991

  19. Chung D, Tahboub K, Delp E J (2017) A two stream siamese convolutional neural network for person re-identification. In: Proceedings of the IEEE international conference on computer vision, pp 1983–1991

  20. Cui L, Ma B (2017) Adaptive feature selection for kinship verification. In: 2017 IEEE international conference on multimedia and expo (ICME). IEEE, pp 751–756

  21. Dahan E, Keller Y (2020) A unified approach to kinship verification. IEEE Transactions on Pattern Analysis and Machine Intelligence

  22. Dandekar AR, Nimbarte MS (2014) A survey: Verification of family relationship from parents and child facial images. In: 2014 international conference on electrical engineering and computer science (ICEECS). IEEE, pp 1–6

  23. Das T K, Chowdhary C L, Gao XZ (2020) Chest x-ray investigation: A convolutional neural network approach. Journal of Biomimetics, Biomaterials and Biomedical Engineering 45:57–70

    Article  Google Scholar 

  24. Dehshibi MM, Shanbehzadeh J (2019) Cubic norm and kernel-based bi-directional pca: Toward age-aware facial kinship verification. Vis Comput 35(1):23–40

    Article  Google Scholar 

  25. Dibeklioglu H, Ali Salah A, Gevers T (2013) Like father, like son: Facial expression dynamics for kinship verification. In: Proceedings of the IEEE international conference on computer vision. IEEE, pp 1497–1504

  26. Dornaika F, Arganda-Carreras I, Serradilla O (2020) Transfer learning and feature fusion for kinship verification. Neural Computing Applications 32 (11):7139–7151

    Article  Google Scholar 

  27. Duan Q, Zhang L, Jia W (2017) Adv-kin: An adversarial convolutional network for kinship verification, vol 10568. Springer, Cham, USA, pp 48–57

    Google Scholar 

  28. Duan Q, Zhang L, Zuo W (2017) From face recognition to kinship verification: An adaptation approach. In: Proceedings of the IEEE international conference on computer vision workshops, pp 1590–1598

  29. Duan X, Tan Z-H (2015) A feature subtraction method for image based kinship verification under uncontrolled environments. In: 2015 IEEE international conference on image processing (ICIP). IEEE, pp 1573–1577

  30. Fang R, Tang KD, Snavely N, Chen T (2010) Towards computational models of kinship verification. In: 2010 IEEE international conference on image processing. IEEE, pp 1577–1580

  31. Georgopoulos M, Panagakis Y, Pantic M (2018) Modeling of facial aging and kinship: A survey. Image and Vision Computing 80:58–79

    Article  Google Scholar 

  32. Goyal A, Meenpal T (2019) Kinship verification from facial images using feature descriptors. Springer, Berlin, pp 371–380

    Google Scholar 

  33. Goyal A, Meenpal T (2020) Template matching for kinship verification in the wild. Springer, Berlin, pp 255–265

    Google Scholar 

  34. Guo G, Wang X (2012) Kinship measurement on salient facial features. IEEE Transactions on Instrumentation Measurement 61(8):2322–2325

    Article  Google Scholar 

  35. Guo Q, Ma B, Lan T (2018) Ensemble learning based on convolutional kernel networks features for kinship verification. In: 2018 IEEE international conference on multimedia and expo (ICME). IEEE, pp 1–6

  36. Hu J, Lu J, Tan Y-P, Yuan J, Zhou J (2017) Local large-margin multi-metric learning for face and kinship verification. IEEE Transactions on Circuits Systems for Video Technology 28(8):1875–1891

    Article  Google Scholar 

  37. Hu J, Lu J, Yuan J, Tan Y-P (2014) Large margin multi-metric learning for face and kinship verification in the wild. In: Asian conference on computer vision. Springer, pp 252–267

  38. Jain A, Bhagat N, Srivastava V, Tyagi P, Jain P (2020) A feature-based kinship verification technique using convolutional neural network. Springer, Berlin, pp 353–362

    Google Scholar 

  39. Kämäräinen J-K, Hadid A, Pietikäinen M (2011) Local representation of facial features. Springer, Berlin, pp 79–108

    Google Scholar 

  40. Koch G, Zemel R, Salakhutdinov R (2015) Siamese neural networks for one-shot image recognition. In: ICML deep learning workshop, vol 2. Lille

  41. Kohli N, Singh R, Vatsa M (2012) Self-similarity representation of weber faces for kinship classification. In: 2012 IEEE Fifth international conference on biometrics: theory, applications and systems (BTAS). IEEE, pp 245–250

  42. Kohli N, Vatsa M, Singh R, Noore A, Majumdar A (2016) Hierarchical representation learning for kinship verification. IEEE Trans Image Process 26(1):289–302

    Article  MathSciNet  MATH  Google Scholar 

  43. Laiadi O, Ouamane A, Benakcha A, Taleb-Ahmed A, Hadid A (2019) Kinship verification based deep and tensor features through extreme learning machine. In: 2019 14th IEEE international conference on automatic face and gesture recognition (FG 2019). IEEE, pp 1–4

  44. Laiadi O, Ouamane A, Boutellaa E, Benakcha A, Taleb-Ahmed A, Hadid A (2019) Kinship verification from face images in discriminative subspaces of color components. Multimedia Tools and Applications 78(12):16465–16487

    Article  Google Scholar 

  45. Li L, Feng X, Wu X, Xia Z, Hadid A (2016) Kinship verification from faces via similarity metric based convolutional neural network. In: International conference on image analysis and recognition. Springer, pp 539–548

  46. Li Y, Zeng J, Zhang J, Dai A, Kan M, Shan S, Chen X (2017) Kinnet: Fine-to-coarse deep metric learning for kinship verification. In: Proceedings of the 2017 workshop on recognizing families in the wild. ACM, pp 13–20

  47. Liang J, Hu Q, Dang C, Zuo W (2018) Weighted graph embedding-based metric learning for kinship verification. IEEE Trans Image Process 28 (3):1149–1162

    Article  MathSciNet  Google Scholar 

  48. Liang J, Guo J, Lao S, Li J (2017) Using deep relational features to verify kinship. In: CCF Chinese conference on computer vision. Springer, pp 563–573

  49. Liu H, Zhu C (2017) Status-aware projection metric learning for kinship verification. In: 2017 IEEE international conference on multimedia and expo (ICME). IEEE, pp 319–324

  50. Lopez M B, Hadid A, Boutellaa E, Goncalves J, Kostakos V, Hosio S (2018) Kinship verification from facial images and videos: human versus machine. Machine Vision Applications 29(5):873–890

    Article  Google Scholar 

  51. Lu J, Hu J, Liong V E, Zhou X, Bottino A, Islam I U, Vieira T F, Qin X, Tan X, Chen S (2015) The fg 2015 kinship verification in the wild evaluation. In: 2015 11th IEEE international conference and workshops on automatic face and gesture recognition (FG), vol 1. IEEE, pp 1–7

  52. Lu J, Hu J, Tan Y-P (2017) Discriminative deep metric learning for face and kinship verification. IEEE Trans Image Process 26(9):4269–4282

    Article  MathSciNet  MATH  Google Scholar 

  53. Lu J, Hu J, Zhou X, Zhou J, Castrillón-Santana M, Lorenzo-Navarro J, Kou L, Shang Y, Bottino A, Vieira T F (2014) Kinship verification in the wild: The first kinship verification competition. In: IEEE international joint conference on biometrics, IEEE, pp 1–6

  54. Lu J, Zhou X, Tan Y-P, Shang Y, Zhou J (2013) Neighborhood repulsed metric learning for kinship verification. IEEE Trans Pattern Anal Mach Intell 36(2):331–345

    Google Scholar 

  55. Mahpod S, Keller Y (2018) Kinship verification using multiview hybrid distance learning. Computer Vision Image Understanding 167:28–36

    Article  Google Scholar 

  56. Mohamed O, Mohammed O, Brahim A, et al. (2017) Content-based image retrieval using convolutional neural networks. In: First international conference on real time intelligent systems. Springer, pp 463–476

  57. Moujahid A, Dornaika F (2019) A pyramid multi-level face descriptor: application to kinship verification. Multimedia Tools and Applications 78(7):9335–9354

    Article  Google Scholar 

  58. Najafabadi M M, Villanustre F, Khoshgoftaar T M, Seliya N, Wald R, Muharemagic E (2015) Deep learning applications and challenges in big data analytics. Journal of Big Data 2(1):1

    Article  Google Scholar 

  59. Nandy A, Mondal SS (2019) Kinship verification using deep siamese convolutional neural network. In: 2019 14th IEEE international conference on automatic face and gesture recognition (FG 2019). IEEE, pp 1–5

  60. Nguyen T-DH, Nguyen H-NH, Dao H (2020) Recognizing families through images with pretrained encoder. arXiv preprint arXiv:2005.11811

  61. Patel B, Maheshwari RP, Raman B (2017) Evaluation of periocular features for kinship verification in the wild. Computer Vision Image Understanding 160:24–35

    Article  Google Scholar 

  62. Qin X, Liu D, Wang D (2020) A literature survey on kinship verification through facial images. Neurocomputing 377:213–224

    Article  Google Scholar 

  63. Qin X, Tan X, Chen S (2015) Tri-subject kinship verification: Understanding the core of a family. IEEE Transactions on Multimedia 17(10):1855–1867

    Article  Google Scholar 

  64. Riccio D, Casanova A, Fenu G (2014) Local vs. global: Intelligent local face recognition. IGI Global, Pennsylvania, pp 187–205

    Google Scholar 

  65. Robinson JP, Shao M, Fu Y (2020) Visual kinship recognition: A decade in the making. arXiv preprint arXiv:2006.16033

  66. Robinson JP, Shao M, Wu Y, Liu H, Gillis T, Fu Y (2018) Visual kinship recognition of families in the wild. IEEE Trans Pattern Anal Mach Intell 40(11):2624–2637

    Article  Google Scholar 

  67. Ross A, Jain A (2003) Information fusion in biometrics. Pattern Recogn Lett 24(13):2115–2125

    Article  Google Scholar 

  68. Shao M, Xia S, Fu Y (2011) Genealogical face recognition based on ub kinface database. In: CVPR 2011 workshops. IEEE, pp 60–65

  69. Shao M, Xia S, Fu Y (2014) Identity and kinship relations in group pictures. Springer, Berlin, pp 175–190

    Google Scholar 

  70. Sharma S, Soni A, Malviya V (2019) Face recognition based on convolution neural network (cnn) applications in image processing: A survey. In: Proceedings of recent advances in interdisciplinary trends in engineering & applications (RAITEA)

  71. Somanath G, Kambhamettu C (2012) Can faces verify blood-relations?. In: 2012 IEEE Fifth international conference on biometrics: theory, applications and systems (BTAS). IEEE, pp 105–112

  72. Taigman Y, Yang M, Ranzato M, Wolf L (2014) Deepface: Closing the gap to human-level performance in face verification. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1701–1708

  73. Tidjani A, Taleb-Ahmed A, Samai D, Eddine AK (2018) Deep learning features for robust facial kinship verification. IET Image Process 12(12):2336–2345

    Article  Google Scholar 

  74. Van TN, Hoang VT (2019) Early and late features fusion for kinship verification based on constraint selection. In: 2019 25th Asia-Pacific conference on communications (APCC). IEEE, pp 116–121

  75. Van TN, Hoang VT (2019) Kinship verification based on local binary pattern features coding in different color space. In: 2019 26th international conference on telecommunications (ICT). IEEE, pp 376–380

  76. Vieira TF, Bottino A, Islam IU (2013) Automatic verification of parent-child pairs from face images. In: Iberoamerican congress on pattern recognition. Springer, pp 326–333

  77. Vyas AS, Prajapati HB, Dabhi VK (2019) Survey on face expression recognition using cnn. In: 2019 5th international conference on advanced computing & communication systems (ICACCS). IEEE, pp 102–106

  78. Wang M, Zechao L, Xiangbo S, Jingdong, Tang J (2015) Deep kinship verification. In: 2015 IEEE 17th international workshop on multimedia signal processing (MMSP). IEEE, pp 1–6

  79. Wang M, Shu X, Feng J, Wang X, Tang J (2020) Deep multi-person kinship matching and recognition for family photos. Pattern Recogn, pp 107342

  80. Wang S, Yan H (2020) Discriminative sampling via deep reinforcement learning for kinship verification. Pattern Recogn Lett 138:38–43

    Article  Google Scholar 

  81. Wang S, Robinson JP, Fu Y (2017) Kinship verification on families in the wild with marginalized denoising metric learning. In: 2017 12th IEEE international conference on automatic face and gesture recognition (FG 2017). IEEE, pp 216–221

  82. Wang W, You S, Karaoglu S, Gevers T (2020) Kinship identification through joint learning using kinship verification ensemble. arXiv preprint arXiv:2004.06382

  83. Wei Z, Xu M, Geng L, Liu H, Yin H (2019) Adversarial similarity metric learning for kinship verification. IEEE Access 7:100029–100035

    Article  Google Scholar 

  84. Wu X, Feng X, Boutellaa E, Hadid A (2018) Kinship verification using color features and extreme learning machine. In: 2018 IEEE 3rd international conference on signal and image processing (ICSIP). IEEE, pp 187–191

  85. Xia C, Xia S, Shao M, Fu Y (2018) Album to family tree: A graph based method for family relationship recognition. In: Asian conference on computer vision. Springer, pp 496–510

  86. Xia S, Shao M, Fu Y (2011) Kinship verification through transfer learning. In: Twenty-Second international joint conference on artificial intelligence. ACM, pp 2539–2544

  87. Xia S, Shao M, Fu Y (2012) Toward kinship verification using visual attributes. In: Proceedings of the 21st international conference on pattern recognition (ICPR2012). IEEE, pp 549–552

  88. Xia S, Shao M, Luo J, Fu Y (2012) Understanding kin relationships in a photo. IEEE Transactions on Multimedia 14(4):1046–1056

    Article  Google Scholar 

  89. Xu M, Shang Y (2016) Kinship measurement on face images by structured similarity fusion. IEEE Access 4:10280–10287

    Article  Google Scholar 

  90. Xu M, Shang Y (2016) Kinship verification using facial images by robust similarity learning. Math Probl Eng 2016:1–9

    Google Scholar 

  91. Yadav N, Goyal A, Meenpal T (2019) A feature averaging method for kinship verification. Springer, Berlin, pp 381–391

    Google Scholar 

  92. Yan H (2019) Learning discriminative compact binary face descriptor for kinship verification. Pattern Recogn Lett 117:146–152

    Article  Google Scholar 

  93. Yan H, Lu J (2017) Feature learning for facial kinship verification. Springer, Berlin, pp 7–36

    Google Scholar 

  94. Yan H, Wang S (2019) Learning part-aware attention networks for kinship verification. Pattern Recogn Lett 128:169–175

    Article  Google Scholar 

  95. Yan K, Huang S, Song Y, Liu W, Fan N (2017) Face recognition based on convolution neural network. In: 2017 36th Chinese control conference (CCC). IEEE, pp 4077–4081

  96. Yang Y, Wu Q (2017) A novel kinship verification method based on deep transfer learning and feature nonlinear mapping. In: 2nd international conference on artificial intelligence and engineering applications (AIEA 2017) ). DEStech Publication Inc., pp 928–937

  97. Yu J, Li M, Hao X, Xie G (2020) Deep fusion siamese network for automatic kinship verification. arXiv preprint arXiv:2006.00143

  98. Yu J, Xie G, Li M, Hao X (2020) Retrieval of family members using siamese neural network. arXiv preprint arXiv:2006.00174

  99. Zhang J, Xia S, Pan H, Qin AK (2016) A genetics-motivated unsupervised model for tri-subject kinship verification. In: 2016 IEEE international conference on image processing (ICIP). IEEE, pp 2916–2920

  100. Zhang L, Duan Q, Zhang D, Jia W, Wang X (2020) Advkin: Adversarial convolutional network for kinship verification. IEEE Trans Cybern

  101. Zhang12 K, Huang Y, Song C, Wu H, Wang L (2015) Kinship verification with deep convolutional neural networks. In: Xie X, Jones MW, Tam GKL (eds) British machine vision conference. BMVA Press

  102. Zhao Y-G, Song Z, Zheng F, Shao L (2018) Learning a multiple kernel similarity metric for kinship verification. Inf Sci 430:247–260

    Article  MathSciNet  Google Scholar 

  103. Zhou X, Hu J, Lu J, Shang Y, Guan Y (2011) Kinship verification from facial images under uncontrolled conditions. In: Proceedings of the 19th ACM international conference on Multimedia. ACM, pp 953–956

  104. Zhou X, Jin K, Xu M, Guo G (2019) Learning deep compact similarity metric for kinship verification from face images. Inform Fusion 48:84–94

    Article  Google Scholar 

  105. Zhou X, Lu J, Hu J, Shang Y (2012) Gabor-based gradient orientation pyramid for kinship verification under uncontrolled environments. In: Proceedings of the 20th ACM international conference on Multimedia. ACM, pp 725–728

Download references

Author information

Authors and Affiliations

  1. School of Computing, Universiti Teknologi Malaysia, 81310, Skudai, Johor, Malaysia

    Mohammed Almuashi & Siti Zaiton Mohd Hashim

  2. Institute For Artificial Intelligence and Big Data Pengkalan Chepa, Universiti Malaysia Kelantan, Bharu, Malaysia

    Nooraini Yusoff & Khairul Nizar Syazwan

  3. College of Computer Science and Engineering, Taibah University, Medina, Saudi Arabia

    Fahad Ghabban

Authors
  1. Mohammed Almuashi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Siti Zaiton Mohd Hashim
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Nooraini Yusoff
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Khairul Nizar Syazwan
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Fahad Ghabban
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mohammed Almuashi.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Almuashi, M., Hashim, S.Z.M., Yusoff, N. et al. Siamese convolutional neural network and fusion of the best overlapping blocks for kinship verification. Multimed Tools Appl 81, 39311–39342 (2022). https://doi.org/10.1007/s11042-022-12735-0

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-022-12735-0

Keywords

Search

Navigation