From Same Photo: Cheating on Visual Kinship Challenges

Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11363)


With the propensity for deep learning models to learn unintended signals from data sets there is always the possibility that the network can “cheat” in order to solve a task. In the instance of data sets for visual kinship verification, one such unintended signal could be that the faces are cropped from the same photograph, since faces from the same photograph are more likely to be from the same family. In this paper we investigate the influence of this artefactual data inference in published data sets for kinship verification.

To this end, we obtain a large data set, and train a CNN classifier to determine if two faces are from the same photograph or not. Using this classifier alone as a naive classifier of kinship, we demonstrate near state of the art results on five public benchmark data sets for kinship verification – achieving over \(90\%\) accuracy on one of them. Thus, we conclude that faces derived from the same photograph are a strong inadvertent signal in all the data sets we examined, and it is likely that the fraction of kinship explained by existing kinship models is small.


Kinship verification Data set bias Deep learning Convolutional neural network 

Supplementary material

484517_1_En_41_MOESM1_ESM.pdf (29 kb)
Supplementary material 1 (pdf 29 KB)


  1. 1.
    Aliradi, R., Belkhir, A., Ouamane, A., Elmaghraby, A.S.: DIEDA: discriminative information based on exponential discriminant analysis combined with local features representation for face and kinship verification. Multimedia Tools Appl. 1–18 (2018).
  2. 2.
    Chatfield, K., Simonyan, K., Vedaldi, A., Zisserman, A.: Return of the devil in the details: delving deep into convolutional nets. arXiv preprint arXiv:1405.3531 (2014)
  3. 3.
    Chen, X., An, L., Yang, S., Wu, W.: Kinship verification in multi-linear coherent spaces. Multimedia Tools Appl. 76(3), 4105–4122 (2017)CrossRefGoogle Scholar
  4. 4.
  5. 5.
    Dahan, E., Keller, Y., Mahpod, S.: Kin-verification model on FIW dataset using multi-set learning and local features. In: Proceedings of the 2017 Workshop on Recognizing Families In the Wild, pp. 31–35. ACM (2017)Google Scholar
  6. 6.
    Doersch, C., Gupta, A., Efros, A.A.: Unsupervised visual representation learning by context prediction. In: International Conference on Computer Vision (ICCV) (2015)Google Scholar
  7. 7.
    Duan, Q., Zhang, L.: AdvNet: adversarial contrastive residual net for 1 million kinship recognition. In: Proceedings of the 2017 Workshop on Recognizing Families In the Wild, pp. 21–29. ACM (2017)Google Scholar
  8. 8.
    Fang, R., Gallagher, A.C., Chen, T., Loui, A.: Kinship classification by modeling facial feature heredity. In: 2013 20th IEEE International Conference on Image Processing (ICIP), pp. 2983–2987. IEEE (2013)Google Scholar
  9. 9.
    Fang, R., Tang, K.D., Snavely, N., Chen, T.: Towards computational models of kinship verification. In: 2010 17th IEEE International Conference on Image Processing (ICIP), pp. 1577–1580. IEEE (2010)Google Scholar
  10. 10.
    King, D.E.: Dlib-ml: a machine learning toolkit. J. Mach. Learn. Res. 10, 1755–1758 (2009)Google Scholar
  11. 11.
    Kohli, N., Vatsa, M., Singh, R., Noore, A., Majumdar, A.: Hierarchical representation learning for kinship verification. IEEE Trans. Image Process. 26(1), 289–302 (2017)MathSciNetCrossRefGoogle Scholar
  12. 12.
    Kohli, N., Yadav, D., Vatsa, M., Singh, R., Noore, A.: Supervised mixed normautoencoder for kinship verification in unconstrained videos. IEEE Trans. Image Process. 28(3), 1329–1341 (2018)CrossRefGoogle Scholar
  13. 13.
    Laiadi, O., Ouamane, A., Benakcha, A., Taleb-Ahmed, A.: RFIW 2017: LPQ-SIEDA for large scale kinship verification. In: Proceedings of the 2017 Workshop on Recognizing Families In the Wild, pp. 37–39. ACM (2017)Google Scholar
  14. 14.
    Li, Y., et al.: KinNet: fine-to-coarse deep metric learning for kinship verification. In: Proceedings of the 2017 Workshop on Recognizing Families In the Wild, pp. 13–20. ACM (2017)Google Scholar
  15. 15.
    Liu, Q., Puthenputhussery, A., Liu, C.: A novel inheritable color space with application to kinship verification. In: 2016 IEEE Winter Conference on Applications of Computer Vision (WACV), pp. 1–9. IEEE (2016)Google Scholar
  16. 16.
    López, M.B., Boutellaa, E., Hadid, A.: Comments on the “kinship face in the wild” data sets. IEEE Trans. Pattern Anal. Mach. Intell. 38(11), 2342–2344 (2016)CrossRefGoogle Scholar
  17. 17.
    Lopez, M.B., Hadid, A., Boutellaa, E., Goncalves, J., Kostakos, V., Hosio, S.: Kinship verification from facial images and videos: human versus machine. Mach. Vis. Appl. 29(5), 873–890 (2018)CrossRefGoogle Scholar
  18. 18.
    Lu, J., Hu, J., Tan, Y.P.: Discriminative deep metric learning for face and kinship verification. IEEE Trans. Image Process. 26(9), 4269–4282 (2017)MathSciNetCrossRefGoogle Scholar
  19. 19.
    Lu, J., Zhou, X., Tan, Y.P., Shang, Y., Zhou, J.: Neighborhood repulsed metric learning for kinship verification. IEEE Trans. Pattern Anal. Mach. Intell. 36(2), 331–345 (2014)CrossRefGoogle Scholar
  20. 20.
    Mahpod, S., Keller, Y.: Kinship verification using multiview hybrid distancelearning. Comput. Vis. Image Underst. 167, 28–36 (2017)CrossRefGoogle Scholar
  21. 21.
    Moujahid, A., Dornaika, F.: A pyramid multi-level face descriptor: application to kinship verification. Multimedia Tools Appl. 78(7), 9335–9354 (2019). Scholar
  22. 22.
    Patel, B., Maheshwari, R., Raman, B.: Evaluation of periocular features for kinship verification in the wild. Comput. Vis. Image Underst. 160, 24–35 (2017)CrossRefGoogle Scholar
  23. 23.
    Qin, X., Tan, X., Chen, S.: Tri-subject kinship verification: understanding the core of a family. IEEE Trans. Multimedia 17(10), 1855–1867 (2015)CrossRefGoogle Scholar
  24. 24.
    Robinson, J.P., Shao, M., Wu, Y., Fu, Y.: Families in the wild (FIW): large-scale kinship image database and benchmarks. In: Proceedings of the 2016 ACM on Multimedia Conference, pp. 242–246. ACM (2016)Google Scholar
  25. 25.
    Robinson, J.P., Shao, M., Wu, Y., Liu, H., Gillis, T., Fu, Y.: Visual kinshiprecognition of families in the wild. IEEE Trans. Pattern Anal. Mach. Intell. 40(11), 2624–2637 (2018)CrossRefGoogle Scholar
  26. 26.
    Wang, S., Ding, Z., Fu, Y.: Cross-generation kinship verification with sparsediscriminative metric. IEEE Trans. Pattern Anal. Mach. Intell. 1 (2018).
  27. 27.
    Wang, S., Robinson, J.P., Fu, Y.: Kinship verification on families in the wild with marginalized denoising metric learning. In: 2017 12th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2017), pp. 216–221. IEEE (2017)Google Scholar
  28. 28.
    Wu, X., Boutellaa, E., López, M.B., Feng, X., Hadid, A.: On the usefulness of color for kinship verification from face images. In: 2016 IEEE International Workshop on Information Forensics and Security (WIFS), pp. 1–6. IEEE (2016)Google Scholar
  29. 29.
    Xia, C., Xia, S., Zhou, Y., Zhang, L., Shao, M.: Graph based family relationship recognition from a single image. In: Geng, X., Kang, B.-H. (eds.) PRICAI 2018. LNCS (LNAI), vol. 11012, pp. 310–320. Springer, Cham (2018). Scholar
  30. 30.
    Xia, S., Shao, M., Fu, Y.: Kinship verification through transfer learning. In: Proceedings of the Twenty-Second International Joint Conference on Artificial Intelligence - Volume Volume Three, IJCAI 2011, pp. 2539–2544. AAAI Press (2011)Google Scholar
  31. 31.
    Xu, M., Shang, Y.: Kinship verification using facial images by robust similarity learning. Math. Probl. Eng. 2016, 8 p. (2016). Article ID 4072323.
  32. 32.
    Yan, H.: Learning discriminative compact binary face descriptor for kinship verification. Pattern Recogn. Lett. 117, 146–152 (2018)CrossRefGoogle Scholar
  33. 33.
    Yang, Y., Wu, Q.: A novel kinship verification method based on deep transfer learning and feature nonlinear mapping. DEStech Trans. Comput. Sci. Eng. AIEA (2017).
  34. 34.
    Zhang, J., Xia, S., Pan, H., Qin, A.: A genetics-motivated unsupervised model for tri-subject kinship verification. In: 2016 IEEE International Conference on Image Processing (ICIP), pp. 2916–2920. IEEE (2016)Google Scholar
  35. 35.
    Zhao, Y.G., Song, Z., Zheng, F., Shao, L.: Learning a multiple kernel similarity metric for kinship verification. Inf. Sci. 430, 247–260 (2018)MathSciNetCrossRefGoogle Scholar
  36. 36.
    Zhou, X., Jin, K., Xu, M., Guo, G.: Learning deep compact similarity metric forkinship verification from face images. Inf. Fusion 48, 84–94 (2018)CrossRefGoogle Scholar
  37. 37.
    Zhou, X., Shang, Y., Yan, H., Guo, G.: Ensemble similarity learning for kinship verification from facial images in the wild. Inf. Fusion 32, 40–48 (2016)CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.Visual Geometry Group (VGG), Department of Engineering ScienceUniversity of OxfordOxfordUK
  2. 2.Nuffield Department of Women’s & Reproductive Health, Big Data Institute, IBMEUniversity of OxfordOxfordUK

Personalised recommendations