Identity and Kinship Relations in Group Pictures

  • Ming Shao
  • Siyu Xia
  • Yun Fu


This chapter studies the problem of identifying people in group pictures. That is, determining from a gallery of people who appear in a given picture. This is a well-studied problem that is becoming increasingly important given the recent explosion in usage of social networks. In this chapter we make two distinct contributions to this problem. First, we use novel kinship similarity to make better estimation of identity. Specifically, we use unary costs based on state-of-the-art face recognition algorithms and as pairwise cost we use the kinship similarity of the people in the image. Second, with these values we formulate a collection-specific MRF MAP estimation (labelling) problem and use existing MRF MAP estimation methods to solve it. To evaluate the proposed method, a family photo database is collected from the Internet. Experiments show that for group pictures of family members (family pictures) our method obtains the state-of-the-art performance, while performing competitively in nonfamily group pictures.


Support Vector Machine Face Recognition Kinship Similarity Stereo Match Kinship Relationship 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Besag, J.: On the statistical analysis of dirty pictures. J. Roy. Stat. Soc. B 48, 259–302 (1986)Google Scholar
  2. 2.
    Castillo, C.D., Jacobs, D.W.: Using stereo matching with general epipolar geometry for 2d face recognition across pose. IEEE Trans. Pattern Anal. Mach. Intell. 31(12), 2298–2304 (2009)CrossRefGoogle Scholar
  3. 3.
    Castillo, C.D., Jacobs, D.W.: Trainable 3d recognition using stereo matching. In: ICCV Workshop on 3D Representation and Recognition (3dRR), pp. 625–631 (2011)Google Scholar
  4. 4.
    Chen, H.F., Belhumeur, P.N., Jacobs, D.W.: In search of illumination invariants. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 1254–1261 (2000)Google Scholar
  5. 5.
    Fang, R., Gallagher, A.C., Chen, T., Loui, A.: Kinship classification by modeling facial feature heredity. In: IEEE International Conference on Image Processing. IEEE (2013)Google Scholar
  6. 6.
    Fang, R., Tang, K.D., Snavely, N., Chen, T.: Towards computational models of kinship verification. In: IEEE International Conference on Image Processing (2010)Google Scholar
  7. 7.
    Gallagher, A.C., Chen, T.: Understanding images of groups of people. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 256–263. IEEE (2009)Google Scholar
  8. 8.
    Huang, G.B., Ramesh, M., Berg, T., Learned-Miller, E.: Labeled faces in the wild: a database for studying face recognition in unconstrained environments. In: Faces in Real-Life Images Workshop in ECCV (2008)Google Scholar
  9. 9.
    Jesorsky, O., Kirchberg, K., Frischolz, R.: Robust face detection using the hausdorff distance. In: Audio and Video Based Person Authentication, pp. 90–95 (2001)Google Scholar
  10. 10.
    Kumar, N., Berg, A.C., Belhumeur, P.N., Nayar, S.K.: Attribute and simile classifiers for face verification. In: International Conference on Computer Vision, pp. 365–372 (2009)Google Scholar
  11. 11.
    Ling, H., Soatto, S., Ramanathan, N., Jacobs, D.W.: A study of face recognition as people age. In: International Conference on Computer Vision, pp. 1–8 (2007)Google Scholar
  12. 12.
    Shao, M., Xia, S., Fu, Y.: Genealogical face recognition based on ub kinface database. In: CVPR Workshop on Biometrics (BIOM) (2011)Google Scholar
  13. 13.
    Manyam, O.K., Kumar, N., Belhumeur, P.N., Kriegman, D.J.: Two faces are better than one: face recognition in group photographs. In: International Joint Conference on Biometrics (2011)Google Scholar
  14. 14.
    Moghaddam, B., Jebara, T., Pentland, A.: Bayesian modeling of facial similarity. In: Advances in Neural Information Processing Systems, pp. 910–916 (1999)Google Scholar
  15. 15.
    Phillips, P.J.: Support vector machines applied to face recognition. In: Advances in Neural Information Processing Systems 11, pp. 803–809. MIT Press (1998)Google Scholar
  16. 16.
    Phillips, P.J., Flynn, P.J., Scruggs, T., Bowyer, K.W., Worek, W.: Preliminary face recognition grand challenge results. In: IEEE International Conference on Automatic Face and Gesture Recognition, pp. 15–24 (2006)Google Scholar
  17. 17.
    Ramanathan, N., Chellappa, R.: Face verification across age progression. IEEE Trans. Image Process. 15(11), 3349–3362 (2006)CrossRefGoogle Scholar
  18. 18.
    Sim, T., Baker, S., Bsat, M.: The CMU pose, illumination, and expression database. IEEE Trans. Pattern Anal. Mach. Intell. 25(12), 1615–1618 (2003)CrossRefGoogle Scholar
  19. 19.
    Stone, Z., Zickler, T., Darrell, T.: Toward large-scale face recognition using social network context. Proc. IEEE 98(8), 1408–1415 (2010)CrossRefGoogle Scholar
  20. 20.
    Viola, P., Jones, M.J.: Robust real-time face detection. Int. J. Comput. Vis. 57(2), 137–154 (2004)Google Scholar
  21. 21.
    Wang, G., Gallagher, A.C., Luo, J., Forsyth, D.A.: Seeing people in social context: recognizing people and social relationships. In: European Conference on Computer Vision, pp. 169–182 (2010)Google Scholar
  22. 22.
    Xia, S., Shao, M., Fu, Y.: Kinship verification through transfer learning. In: International Joint Conference on Artificial Intelligence, pp. 2539–2544 (2011)Google Scholar
  23. 23.
    Xia, S., Shao, M., Luo, J., Fu, Y.: Understanding kin relationships in a photo. IEEE Trans. Multimedia 14(4), 1046–1056 (2012)CrossRefGoogle Scholar
  24. 24.
    Zhao, W., Chellappa, R., Phillips, P.J., Rosenfeld, A.: Face recognition: a literature survey. ACM Comput. Surv. 35(4), 399–458 (2003)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  1. 1.College of Computer and Information ScienceNortheastern UniversityBostonUSA
  2. 2.School of AutomationSoutheast UniversityNanjingChina
  3. 3.Department of Electrical and Computer EngineeringNortheastern UniversityBostonUSA

Personalised recommendations