Abstract
In this paper we present a method to automatically segment a photo sequence in groups containing the same persons. Many methods in literature accomplish to this task by adopting clustering techniques. We model the problem as the search for probable associations between faces detected in subsequent photos considering the mutual exclusivity constraint: a person can not be in a photo two times, nor two faces in the same photo can be assigned to the same group. Associations have been found considering face and clothing descriptions. In particular, a two level architecture has been adopted: at the first level, associations are computed within meaningful temporal windows (situations); at the second level, the resulting clusters are re-processed to find associations across situations. Experiments confirm our technique generally outperforms clustering methods. We present an analysis of the results on a public dataset, enabling future comparison, and on private collections.
Similar content being viewed by others
Notes
In Matlab this computation can be easily performed by means of the gamfit function.
References
Ahonen T, Hadid A, Pietikäinen M (2004) Face recognition with local binary patterns. In: Proc. of European Conference on Computer Vision (ECCV 2004), pp 469–481
Androutsos D, Plataniotiss K, Venetsanopoulos A (1998) Distance measures for color image retrieval. In: Proc. of International Conference on Image Processing (ICIP 98), vol 2. IEEE, pp 770–774
Ardizzone E, Cascia ML, Morana M, Vella F (2009) Clustering techniques for personal photo album management. J Electron Imaging 18(4):043014
Choi J, Yang S, Ro Y, Plataniotis K (2008) Face annotation for personal photos using context-assisted face recognition. In: Proc. of international conference on Multimedia Information Retrieval (MIR 2008), pp 44–51
Chu WT, Lee YL, Yu JY (2009) Using context information and local feature points in face clustering for consumer photos. In: Proc. of International Conference on Acoustics, Speech, and Signal Processing (ICASSP 2009), pp 1141–1144
Davis M, Smith M, Stentiford F, Bamidele A, Canny J, Good N, King S, Janakiraman R (2006) Using context and similarity for face and location identification. In: Proceedings of the IS&T/SPIE 18th annual symposium on electronic imaging science and technology, vol 6061. pp 119–127
Edmonds J, Karp R (1972) Theoretical improvements in algorithmic efficiency for network flow problems. JACM 19(2):248–264
El-Khoury E, Senac C, Joly P (2010) Face-and-clothing based people clustering in video content. In: Proc. of international conference on Multimedia Information Retrieval (MIR 2010), ACM, pp 1–10
Gallagher A, Chen T (2008) Clothing cosegmentation for recognizing people. In: Proc. of Computer Vision and Pattern Recognition (CVPR 2008). IEEE
Gross J, Yellen J (2006) Graph theory and its applications. CRC press
Heyer LJ, Kruglyak S, Yooseph S (1999) Exploring expression data: Identification and analysis of coexpressed genes. Genome Res 9(11):1106–1115. doi:10.1101/gr.9.11.1106
Huang GB, Jain V, Learned-Miller E (2007) Unsupervised joint alignment of complex images. In: Proc. of international conference on computer vision (ICCV 2007)
Huang T, Russell S (1997) Object identification in a bayesian context. Int Joint Conf on Artificial Intel 15:1276–1283
iPhoto (2002) http://www.apple.com/ilife/iphoto
Jain A, Murty M, Flynn P (1999) Data clustering: a review. ACM Comput Surv (CSUR) 31(3):264–323
Jain R, Sinha P (2010) Content without context is meaningless. In: Proc. of conference on Multimedia (MM 2010)
Kakumanu P, Makrogiannis S, Bourbakis N (2007) A survey of skin-color modeling and detection methods. Pattern Recogn 40(3):1106–1122
Kang H, Shneiderman B (2000) Visualization methods for personal photo collections: Browsing and searching in the photofinder. In: Proc. of International Conference on Multimedia & Expo (ICME 2000)
Kuhn H (1955) The Hungarian method for the assignment problem. Nav Res Logist Q 2(1–2):83–97
Lawless J (1982) Statistical models and methods for lifetime data. Wiley, New York
Lo Presti L, Morana M, La Cascia M (2010) A data association algorithm for people re-identification in photo sequences. In: Proc. of IEEE International Symposium on Multimedia (ISM 2010). IEEE
Ojala T, Pietikäinen M, Mäenpää T (2002) Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Trans Pattern Anal Mach Intell 24(7):971–987
Phillips P, Wechsler H, Huang J, Rauss P (1998) The FERET database and evaluation procedure for face-recognition algorithms. Image and Vision Computing 16(5):295–306
Phillips P, Moon H, Rizvi S, Rauss P (2002) The FERET evaluation methodology for face-recognition algorithms. IEEE Transactions on Pattern Analysis and Machine Intelligence 22(10):1090–1104
Picasa (2004) http://picasa.google.com
Rubner Y, Tomasi C, Guibas L (1998) A metric for distributions with applications to image databases. Proc of International Conference on Computer Vision, (ICCV 1998)
Sivic J, Zitnick C, Szeliski R (2006) Finding people in repeated shots of the same scene. In: Proc. of British Machine Vision Conference (BMVC 2006) 3:909–918
Song Y, Leung T (2006) Context-aided human recognition—clustering. In: Computer Vision—ECCV 2006, Lecture Notes in Computer Science, vol 3953. Springer, Berlin, Heidelberg, pp 382–395
Tan X, Triggs B (2007) Enhanced local texture feature sets for face recognition under difficult lighting conditions. In: Proc. of international conference on analysis and modeling of faces and gestures. Springer, pp 168–182
Turk M, Pentland A (1991) Eigenfaces for recognition. J Cogn Neurosci 3(1):71–86
Viola P, Jones MJ (2004) Robust real-time face detection. Int J Comput Vis 57(2):137–154
Zhang L, Chen L, Li M, Zhang H (2003) Automated annotation of human faces in family albums. In: Proc. of Conference on Multimedia (MM 2003), pp 355–358
Zhang L, Hu Y, Li M, Ma W, Zhang H (2004) Efficient propagation for face annotation in family albums. In: Proc of Conference on Multimedia (MM 2004), pp 716–723
Zhao M, Teo Y, Liu S, Chua TS, Jain R (2006) Automatic person annotation of family photo album. In: Image and video retrieval. Lecture notes in computer science, vol 4071. Springer, Berlin, Heidelberg, pp 163–172
Acknowledgement
We thank all the anonymous reviewers whose insightful comments led to significant improvements of the manuscript.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Lo Presti, L., Morana, M. & La Cascia, M. A data association approach to detect and organize people in personal photo collections. Multimed Tools Appl 61, 321–352 (2012). https://doi.org/10.1007/s11042-011-0839-5
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11042-011-0839-5