Abstract
Since image set classification has strong power to overcome various variations in illumination, expression, pose, and so on, it has drawn extensive attention in recent years. Noteworthily, the point-to-point distance-based methods have achieved the promising performance, which aim to compute the similarity between each gallery set and the probe set for classification purpose. Nevertheless, these existing methods have to face the following problems: (1) they do not take full advantage of the between-set discrimination information; (2) they ideally presume that the importance of different gallery sets is equal, whereas this always violates objective facts and may degenerate algorithm performance in practice; (3) they tend to have high computational cost and several parameters, though explicit sparsity can enhance discrimination. To address these problems, we propose a novel method for face image set classification, namely self-weighted latent sparse discriminative learning (SLSDL). Specifically, a novel self-weighted strategy guided discrimination term is proposed to largely boost the discrimination of different gallery sets, such that the effect of true sets can be boosted while the effect of false sets can be weakened or removed. Moreover, we propose a latent sparse normalization to reduce computational complexity as well as the number of trade-off parameters. In addition, we propose an efficient optimization algorithm to solve the final SLSDL. Comprehensive experiments on four public benchmark datasets demonstrate that SLSDL is superior to the state-of-the-art competitors.
Similar content being viewed by others
Notes
For image sets classification tasks [16], image set can be divided into two parts, including the gallery set and the probe set, where the gallery set with label is used to train and the probe set is used to test. Usually, each set may contain large variations in pose, illumination, and scale. Images in each set contained a same subject are collected from video-based face recognition systems, multiple cameras, or personal photo album.
References
Yang J, Liu Y (2019) Undersampled face recognition based on virtual samples and representation classification. Neural Comput Appl 31(7):2447–2453
Wei D, Shen X, Sun Q, Gao X, Yan W (2020) Prototype learning and collaborative representation using Grassmann manifolds for image set classification. Pattern Recogn 100:107123
Liu B, Jing L, Li J, Yu J, Gittens A, Mahoney MW (2019) Group collaborative representation for image set classification. Int J Comput Vis 127(2):181–206
Gao X, Sun Q, Xu H, Wei D, Gao J (2019) Multi-model fusion metric learning for image set classification. Knowl Based Syst 164:253–264
Moon HM, Seo CH, Pan SB (2017) A face recognition system based on convolution neural network using multiple distance face. Soft Comput 21(17):4995–5002
Ren Z, Sun Q, Yang C (2020) Nonnegative discriminative encoded nearest points for image set classification. Neural Comput Appl 32(13):9081–9092
Cevikalp H, Yavuz HS, Triggs B (2019) Face recognition based on videos by using convex hulls. IEEE Trans Circuits Syst Video Technol PP(99):1
Lei D, Jiang Z, Wu Y (2020) Weighted huber constrained sparse face recognition. Neural Comput Appl 32(9):5235–5253
Huang C, Li Y, Chen CL, Tang X (2019) Deep imbalanced learning for face recognition and attribute prediction. IEEE Trans Pattern Anal Mach Intell 42(11):2781–2794
Zhang Z, Jiang W, Qin J, Zhang L, Li F, Zhang M, Yan S (2017) Jointly learning structured analysis discriminative dictionary and analysis multiclass classifier. IEEE Trans Neural Netw Learn Syst 29(8):3798–3814
Zhang Z, Sun Y, Wang Y, Zhang Z, Zhang H, Liu G, Wang M (2020) Twin-incoherent self-expressive locality-adaptive latent dictionary pair learning for classification. IEEE Trans Neural Netw Learn Syst 99:1–15
Zhang Z, Jiang W, Zhang Z, Li S, Liu G, Qin J (2019a) Scalable block-diagonal locality-constrained projective dictionary learning. In: Proceedings of the 28th international joint conference on artificial intelligence, AAAI Press, pp 4376–4382
Zhang Z, Ren J, Jiang W, Zhang Z, Hong R, Yan S, Wang M (2019b) Joint subspace recovery and enhanced locality driven robust flexible discriminative dictionary learning. IEEE Trans Circuits Syst Video Technol 30:42
Sun Y, Zhang Z, Jiang W, Zhang Z, Zhang L, Yan S, Wang M (2020) Discriminative local sparse representation by robust adaptive dictionary pair learning. IEEE Trans Neural Netw Learn Syst 31(10):4303–4317
Cevikalp H, Triggs B (2010) Face recognition based on image sets. In: 2010 IEEE computer society conference on computer vision and pattern recognition, IEEE, pp 2567–2573
Hu Y, Mian AS, Owens R (2012) Face recognition using sparse approximated nearest points between image sets. IEEE Trans Pattern Anal Mach Intell 34(10):1992–2004
Yang M, Zhu P, Van Gool L, Zhang L (2013) Face recognition based on regularized nearest points between image sets. In: 2013 10th IEEE international conference and workshops on automatic face and gesture recognition (FG), IEEE, pp 1–7
Zhu P, Zuo W, Zhang L, Shiu SCK, Zhang D (2014) Image set-based collaborative representation for face recognition. IEEE Trans Inf Forensics Secur 9(7):1120–1132
Zhang L, Yang M, Feng X, Ma Y, Zhang D (2012) Collaborative representation based classification for face recognition. arXiv preprint arXiv:12042358
Wang W, Wang R, Shan S, Chen X (2016) Prototype discriminative learning for face image set classification. In: Asian conference on computer vision, Springer, pp 344–360
Feng Q, Zhou Y, Lan R (2016) Pairwise linear regression classification for image set retrieval. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 4865–4872
Yang M, Wang X, Liu W, Shen L (2017) Joint regularized nearest points for image set based face recognition. Image Vision Comput 58:47–60
Zheng P, Zhao ZQ, Gao J, Wu X (2017) Image set classification based on cooperative sparse representation. Pattern Recogn 63:206–217
Huang Z, Shan S, Wang R, Zhang H, Lao S, Kuerban A, Chen X (2015) A benchmark and comparative study of video-based face recognition on cox face database. IEEE Trans Image Process 24(12):5967–5981
Chen L, Hassanpour N (2017) Survey: how good are the current advances in image set based face identification?-experiments on three popular benchmarks with a naïve approach. Comput Vision Image Underst 160:1–23
Liu X, Guo Z, You J, Kumar BV (2019) Dependency-aware attention control for image set-based face recognition. IEEE Trans Inf Forensics Secur 15:1501–1512
Song Z, Cui K, Cheng G (2020) Image set face recognition based on extended low rank recovery and collaborative representation. Int J Mach Learn Cybern 11(1):71–80
Mian A, Hu Y, Hartley R, Owens R (2013) Image set based face recognition using self-regularized non-negative coding and adaptive distance metric learning. IEEE Trans Image Process 22(12):5252–5262
Wang G, Shi N (2020) Collaborative representation-based discriminant neighborhood projections for face recognition. Neural Comput Appl 32(10):5815–5832
Chen L (2014) Dual linear regression based classification for face cluster recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2673–2680
Ren CX, Luo YW, Xu XL, Dai DQ, Yan H (2019) Discriminative residual analysis for image set classification with posture and age variations. IEEE Trans Image Process 29:2875–2888
Hayat M, Bennamoun M, An S (2014) Deep reconstruction models for image set classification. IEEE Trans Pattern Anal Mach Intell 37(4):713–727
Sun Y, Ren Z, Yang C, Lei H (2020) Latent sparse discriminative learning for face image set classification. In: International conference on neural computing for advanced applications, Springer, pp 144–156
Ren Z, Sun Q (2020) Simultaneous global and local graph structure preserving for multiple kernel clustering. IEEE Trans Neural Netw Learn Syst PP:2
Nie F, Wang X, Jordan MI, Huang H (2016) The constrained Laplacian rank algorithm for graph-based clustering. In: Proceedings of the Thirtieth AAAI conference on artificial intelligence, pp 1969–1976
Wang H, Yang Y, Liu B, Fujita H (2019) A study of graph-based system for multi-view clustering. Knowl Based Syst 163:1009–1019
Ren Z, Yang SX, Sun Q, Wang T (2020a) Consensus affinity graph learning for multiple kernel clustering. IEEE Trans Cybern PP:1
Ren Z, Mukherjee M, Lloret J, Venu P (2020b) Multiple kernel driven clustering with locally consistent and selfish graph in industrial IoT. IEEE Trans Ind Inf PP:3
Ren Z, Wu B, Zhang X, Sun Q (2019a) Image set classification using candidate sets selection and improved reverse training. Neurocomputing 341:60–69
Ren Z, Sun Q, Wu B, Zhang X, Yan W (2019b) Learning latent low-rank and sparse embedding for robust image feature extraction. IEEE Trans Image Process 29(1):2094–2107
Learned-Miller E, Huang GB, RoyChowdhury A, Li H, Hua G (2016) Labeled faces in the wild: a survey. Advances in face detection and facial image analysis. Springer, Cham, pp 189–248
Taigman Y, Wolf L, Hassner T et al (2009) Multiple one-shots for utilizing class label information. BMVC 2:1–12
Cui H, Zhu L, Li J, Yang Y, Nie L (2019) Scalable deep hashing for large-scale social image retrieval. IEEE Trans Image Process 29:1271–1284
Acknowledgements
This research was supported by the Sichuan Science and Technology Program (Grant Nos. 2019ZDZX0043 and 2020ZDZX0014), the Key Lab of Film and TV Media Technology of Zhejiang Province (Grant No. 2020E10015), the Natural Science Foundation Project of CQ CSTC (Grant No. cstc2020jcyj-msxmX0473), the Major Cultivation Research Projects of Chongqing Three Gorges University (Grant No. 18ZDPY07), the Scientific Research Fund of Sichuan Provincial Education Department (Grant No. 17ZB0441), the Scientific Research Fund of Southwest University of Science and Technology (Grant No. 17zx7137), the Postgraduate Innovation Fund Project by Southwest University of Science and Technology (Grant No. 20ycx0001), and the Undergraduate Innovation and Entrepreneurship Training Program of Sichuan (Grant No. S202010619014).
Author information
Authors and Affiliations
Corresponding authors
Ethics declarations
Conflict of interest
There is no conflicts of interest.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Sun, Y., Ren, Z., Yang, C. et al. Face image set classification with self-weighted latent sparse discriminative learning. Neural Comput & Applic 35, 12283–12295 (2023). https://doi.org/10.1007/s00521-020-05479-1
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00521-020-05479-1