Abstract
This paper presents an automatic system that structures information in video surveillance systems based on the analysis of facial images. We describe the cluster analysis in video data using face detection in each video frame and feature extraction with pretrained deep convolutional neural networks. Different aggregation techniques to combine frame features into a single video descriptor are implemented to organize video data based on clustering techniques. An experimental study with the YouTube Faces dataset that demonstrates the most accurate algorithm matches normalized average frame feature vectors and group them with average linkage agglomerative clustering algorithm.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Sokolova, A.D., Kharchevnikova, A.S., Savchenko, A.V.: Organizing multimedia data in video surveillance systems based on face verification with convolutional neural networks. In: Proceedings of the International Conference on Analysis of Images, Social Networks and Texts (AIST), Springer, Cham, pp. 223–230 (2017)
Zhang, Y.J., Lu, H.B.: A hierarchical organization scheme for video data. Pattern Recognit. 35(11), 2381–2387 (2002)
Shan, C.: Face recognition and retrieval in video. In: Video Search and Mining. Springer, Berlin, Heidelberg, pp. 235–260 (2010)
Savchenko, A.V.: Search Techniques in Intelligent Classification Systems. Springer International Publishing, Heidelberg (2016)
Chen, J.C. Ranjan, R., Kumar, A., Chen, C.H., Patel, V.M., Chellappa, R.: An end-to-end system for unconstrained face verification with deep convolutional neural networks. In: Proceedings of the IEEE International Conference on Computer Vision Workshops, pp. 118–126 (2015)
Li, H., Hua, G., Shen, X., Lin, Z., Brandt, J.: Eigen-PEP for video face recognition. In: Cremers, D., Reid, I., Saito, H., Yang, M.H. (eds.), Proceedings of the Asian Conference on Computer Vision, Springer, Cham, vol. 9005, pp. 17–33 (2014)
Wolf, L., Hassner, T., Maoz, I.: Face recognition in unconstrained videos with matched background similarity. In: Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition (CVPR), pp. 529–534 (2011)
Yang, S., Luo, P., Loy, C.C., Tang, X.: Wider face: a face detection benchmark. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5525–5533 (2016)
Kaufman, L., Rousseeuw, P.J.: Finding Groups in Data: An Introduction to Cluster Analysis. Wiley (2009)
Savchenko, A.V.: Clustering and maximum likelihood search for efficient statistical classification with medium-sized databases. Optim. Lett. 11(2), 329–341 (2017)
Savchenko, A.V.: Efficient facial representations for age, gender and identity recognition in organizing photo albums using multi-output CNN. arXiv preprint arXiv:1807.07718 (2018)
Jia, Y., Shelhamer, E., Donahue, J., Karayev, S., Long, J., Girshick, R., Darrell, T.: Caffe: Convolutional architecture for fast feature embedding. In: Proceedings of the 22nd ACM International Conference on Multimedia, pp. 675–678 (2014)
Wu, X., He, R., Sun, Z., Tan, T.: A light CNN for deep face representation with noisy labels. Proc. IEEE Trans. Inf. Forensics Secur. 13(11), 2884–2896 (2018)
Goodfellow, I., Bengio, Y., Courville, A.: Deep Learning. MIT press (2016)
Sokolova, A.D., Savchenko, A.V.: Cluster analysis of facial video data in video surveillance systems using deep learning. In: Proceedings of the Computational Aspects and Applications in Large-Scale Networks (NET). Springer Proceedings in Mathematics and Statistics, vol. 247, pp. 113–120 (2018)
Rassadin, A., Gruzdev, A., Savchenko, A.V.: Group-level emotion recognition using transfer learning from face identification. In: Proceedings of the 19th ACM International Conference on Multimodal Interaction, pp. 544–548 (2017)
Miyamoto, S., Arai, K.: Different sequential clustering algorithms and sequential regression models. In: Proceedings of the IEEE International Conference on Fuzzy Systems, pp. 1107–1112 (2009)
Zhu, C., Wen, F., Sun, J.: A rank-order distance based clustering algorithm for face tagging. In: Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition (CVPR), pp. 481–488 (2011)
Schubert, E., Sander, J., Ester, M., Kriegel, H.P., Xu, X.: DBSCAN revisited, revisited: why and how you should (still) use DBSCAN. ACM Trans. Database Syst. (TODS), 42(3), 19 (2017)
Zhang, T., Ramakrishnan, R., Livny, M.: BIRCH: an efficient data clustering method for very large databases. ACM Sigmod Rec. (ACM) 25–2, 103–114 (1996)
Sculley, D.: Web-scale k-means clustering. In: Proceedings of the 19th International Conference on World Wide Web (ACM), pp. 1177–1178 (2010)
Rokach, L., Maimon, O.: Clustering methods. In: Data Mining and Knowledge Discovery Handbook. Springer, Boston, MA, pp. 321–352 (2005)
Parkhi, O.M., Vedaldi, A., Zisserman, A.: Deep face recognition. In: Proceedings of the British Machine Vision Conference (BMVC), vol. 1, no. 3, pp. 6–17 (2015)
Cao, Q., Shen, L., Xie, W., Parkhi, O. M., Zisserman, A.: VGGFace2: A dataset for recognising faces across pose and age. In: Proceedings of the 13th IEEE International Conference on Automatic Face and Gesture Recognition (FG), pp. 67–74 (2018)
Savchenko, A.V., Belova, N.S.: Unconstrained face identification using maximum likelihood of distances between deep off-the-shelf features. Expert Syst. Appl. 108, 170–182 (2018)
da Silva, S.F., Brandoli, B., Eler, D.M., Neto, J.B., Traina, A.J.: Silhouette-based feature selection for classification of medical images. In: Proceedings of the IEEE 23rd International Symposium on Computer-Based Medical Systems (CBMS), pp. 315–320 (2010)
Savchenko, A.V.: Sequential three-way decisions in multi-category image recognition with deep features based on distance factor. Inf. Sci. 489, 18–36 (2019)
Acknowledgements
The article was prepared within the framework of the Academic Fund Program at the National Research University Higher School of Economics (HSE University) in 2019 (grant No. 19-04-004) and within the framework of the Russian Academic Excellence Project “5–100”.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Sokolova, A.D., Savchenko, A.V. (2020). Facial Clustering in Video Data Using Deep Convolutional Neural Networks. In: Bychkov, I., Kalyagin, V., Pardalos, P., Prokopyev, O. (eds) Network Algorithms, Data Mining, and Applications. NET 2018. Springer Proceedings in Mathematics & Statistics, vol 315. Springer, Cham. https://doi.org/10.1007/978-3-030-37157-9_11
Download citation
DOI: https://doi.org/10.1007/978-3-030-37157-9_11
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-37156-2
Online ISBN: 978-3-030-37157-9
eBook Packages: Mathematics and StatisticsMathematics and Statistics (R0)