Abstract
We propose to use discriminative subgraphs to discover family photos from group photos in an efficient and effective way. Group photos are represented as face graphs by identifying social contexts such as age, gender, and face position. The previous work utilized bag-of-word models and considered frequent subgraphs from all group photos as features for classification. This approach, however, produces numerous subgraphs, resulting in high dimensions. Furthermore, some of them are not discriminative. To solve these issues, we adopt a state-of-the-art, frequent subgraph mining method that removes nondiscriminative subgraphs. We also use TF-IDF normalization, which is more suitable for the bag-ofword model. To validate our method, we experiment in two datasets. Our method shows consistently better performance, higher accuracy in lower feature dimensions, compared to the previous method. We also integrate our method with the recent Microsoft face recognition API and release it in a public website.
Article PDF
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Avoid common mistakes on your manuscript.
References
Krizhevsky, A. Learning multiple layers of features from tiny images. Technical Report. University of Toronto, 2009.
Chen, Y.-Y.; Hsu, W. H.; Liao, H.-Y. M. Discovering informative social subgraphs and predicting pairwise relationships from group photos. In: Proceedings of the 20th ACM International Conference on Multimedia, 669–678, 2012.
Gallagher, A. C.; Chen, T. Understanding images of groups of people. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 256–263, 2009.
Murillo, A. C.; Kwak, I. S.; Bourdev, L.; Kriegman, D.; Belongie, S. Urban tribes: Analyzing group photos from a social perspective. In: Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, 28–35, 2012.
Wang, G.; Gallagher, A.; Luo, J.; Forsyth, D. Seeing people in social context: Recognizing people and social relationships. In: Lecture Notes in Computer Science, Vol. 6315. Daniilidis, K.; Maragos, P.; Paragios, N. Eds. Springer Berlin Heidelberg, 169–182, 2010.
Chiu, Y.-I.; Li, C.; Huang, C.-R.; Chung, P.-C.; Chen, T. Efficient graph based spatial face context representation and matching. In: Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing, 2001–2005, 2013.
Gallagher, A. C.; Chen, T. Finding rows of people in group images. In: Proceedings of IEEE International Conference on Multimedia and Expo, 602–605, 2009.
Choi, W.; Chao, Y.-W.; Pantofaru, C.; Savarese, S. Discovering groups of people in images. In: Lecture Notes in Computer Science, Vol. 8692. Fleet, D.; Pajdla, T.; Schiele, B.; Tuytelaars, T. Eds. Springer International Publishing, 417–433, 2014.
Shu, H.; Gallagher, A.; Chen, H.; Chen, T. Face–graph matching for classifying groups of people. In: Proceedings of IEEE International Conference on Image Processing, 2425–2429, 2013.
Chiu, Y.-I.; Hsu, R.-Y.; Huang, C.-R. Spatial face context with gender information for group photo similarity assessment. In: Proceedings of the 22nd International Conference on Pattern Recognition, 2673–2678, 2014.
Shimizu, K.; Nitta, N.; Nakai, Y.; Babaguchi, N. Classification based group photo retrieval with bag of people features. In: Proceedings of the 2nd ACM International Conference on Multimedia Retrieval, Article No. 6, 2012.
Zhang, T.; Chao, H.; Willis, C.; Tretter, D. Consumer image retrieval by estimating relation tree from family photo collections. In: Proceedings of the ACM International Conference on Image and Video Retrieval, 143–150, 2010.
Singla, P.; Kautz, H.; Luo, J.; Gallagher, A. Discovery of social relationships in consumer photo collections using Markov logic. In: Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern RecognitionWorkshops, 1–7, 2008.
Song, Z.; Wang, M.; Hua, X.-s.; Yan, S. Predicting occupation via human clothing and contexts. In: Proceedings of International Conference on Computer Vision, 1084–1091, 2011.
Lowe, D. G. Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision Vol. 60, No. 2, 91–110, 2004.
Oliva, A.; Torralba, A. Modeling the shape of the scene: A holistic representation of the spatial envelope. International Journal of Computer Vision Vol. 42, No. 3, 145–175, 2001.
Jiang, C.; Coenen, F.; Zito, M. A survey of frequent subgraph mining algorithms. The Knowledge Engineering Review Vol. 28, No. 1, 75–105, 2013.
Cook, S. A. The complexity of theorem-proving procedures. In: Proceedings of the 3rd Annual ACM Symposium on Theory of Computing, 151–158, 1971.
Wang, M.; Lai, Y.-K.; Liang, Y.; Martin, R. R.; Hu, S.-M. BiggerPicture: Data-driven image extrapolation using graph matching. ACM Transactions on Graphics Vol. 33, No. 6, Article No. 173, 2014.
Chen, X.; Zhou, B.; Guo, Y.; Xu, F.; Zhao, Q. Structure guided texture inpainting through multiscale patches and global optimization for image completion. Science China Information Sciences Vol. 57, No. 1, 1–16, 2014.
Li, H.; Wu, W.; Wu, E. Robust interactive image segmentation via graph-based manifold ranking. Computational Visual Media Vol. 1, No. 3, 183–195, 2015.
Hu, S.-M.; Zhang, F.-L.; Wang, M.; Martin, R. R.; Wang, J. PatchNet: A patch-based image representation for interactive library-driven image editing. ACM Transactions on Graphics Vol. 32, No. 6, Article No. 196, 2013.
Taigman, Y.; Yang, M.; Ranzato, M.; Wolf, L. DeepFace: Closing the gap to human-level performance in face verification. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 1701–1708, 2014.
Zhao, W.; Chellappa, R.; Phillips, P. J.; Rosenfeld, A. Face recognition: A literature survey. ACM Computing Surveys Vol. 35, No. 4, 399–458, 2003.
Yan, X.; Han, J. Gspan: Graph-based substructure pattern mining. In: Proceedings of the 2002 IEEE International Conference on Data Mining, 721–724, 2002.
Thoma, M.; Cheng, H.; Gretton, A.; Han, J.; Kriegel, H.-P.; Smola, A.; Song, L.; Yu, P. S.; Yan, X.; Borgwardt, K. M. Discriminative frequent subgraph mining with optimality guarantees. Statistical Analysis and Data Mining: The ASA Data Science Journal Vol. 3, No. 5, 302–318, 2010.
Jones, K. S. A statistical interpretation of term specificity and its application in retrieval. Journal of Documentation Vol. 28, No. 1, 11–21, 1972.
Author information
Authors and Affiliations
Corresponding author
Additional information
This article is published with open access at Springerlink.com
Changmin Choi is in a start-up company. He received his M.S. degree from School of Computing at Korea Advanced Institute of Science and Technology (KAIST), and B.A. degree from Business School at Hanyang University. His research interest is understanding group photos in social media.
YoonSeok Lee is a M.S. student in School of Computing at Korea Advanced Institute of Science and Technology (KAIST) and he received his B.S. degree in computer science from KAIST in 2014. His research interest lies in image classification, image representation, and hashing techniques.
Sung-Eui Yoon is currently an associate professor at Korea Advanced Institute of Science and Technology (KAIST). He received his B.S. and M.S. degrees in computer science from Seoul National University in 1999 and 2001, respectively. His main research interest is designing scalable graphics, image search, and geometric algorithms. He gave numerous tutorials on proximity queries and large-scale rendering at various conferences including ACM SIGGRAPH and IEEE Visualization. Some of his work received a distinguished paper award at Pacific Graphics, invitations to IEEE TVCG, an ACM student research competition award, and other domestic research-related awards. He is a senior member of IEEE, and a member of ACM and KIISE.
Open Access The articles published in this journal are distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.
Other papers from this open access journal are available free of charge from http://www.springer.com/journal/41095. To submit a manuscript, please go to https://www. editorialmanager.com/cvmj.
Rights and permissions
Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (https://creativecommons.org/licenses/by/4.0), which permits use, duplication, adaptation, distribution, and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.
About this article
Cite this article
Choi, C., Lee, Y. & Yoon, SE. Discriminative subgraphs for discovering family photos. Comp. Visual Media 2, 257–266 (2016). https://doi.org/10.1007/s41095-016-0054-4
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s41095-016-0054-4