Skip to main content
SpringerLink
Log in

Face image set classification with self-weighted latent sparse discriminative learning

  • S.I. : New Trends of Neural Computing for Advanced Applications
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

Since image set classification has strong power to overcome various variations in illumination, expression, pose, and so on, it has drawn extensive attention in recent years. Noteworthily, the point-to-point distance-based methods have achieved the promising performance, which aim to compute the similarity between each gallery set and the probe set for classification purpose. Nevertheless, these existing methods have to face the following problems: (1) they do not take full advantage of the between-set discrimination information; (2) they ideally presume that the importance of different gallery sets is equal, whereas this always violates objective facts and may degenerate algorithm performance in practice; (3) they tend to have high computational cost and several parameters, though explicit sparsity can enhance discrimination. To address these problems, we propose a novel method for face image set classification, namely self-weighted latent sparse discriminative learning (SLSDL). Specifically, a novel self-weighted strategy guided discrimination term is proposed to largely boost the discrimination of different gallery sets, such that the effect of true sets can be boosted while the effect of false sets can be weakened or removed. Moreover, we propose a latent sparse normalization to reduce computational complexity as well as the number of trade-off parameters. In addition, we propose an efficient optimization algorithm to solve the final SLSDL. Comprehensive experiments on four public benchmark datasets demonstrate that SLSDL is superior to the state-of-the-art competitors.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Notes

  1. For image sets classification tasks [16], image set can be divided into two parts, including the gallery set and the probe set, where the gallery set with label is used to train and the probe set is used to test. Usually, each set may contain large variations in pose, illumination, and scale. Images in each set contained a same subject are collected from video-based face recognition systems, multiple cameras, or personal photo album.

References

  1. Yang J, Liu Y (2019) Undersampled face recognition based on virtual samples and representation classification. Neural Comput Appl 31(7):2447–2453

    Article  Google Scholar 

  2. Wei D, Shen X, Sun Q, Gao X, Yan W (2020) Prototype learning and collaborative representation using Grassmann manifolds for image set classification. Pattern Recogn 100:107123

    Article  Google Scholar 

  3. Liu B, Jing L, Li J, Yu J, Gittens A, Mahoney MW (2019) Group collaborative representation for image set classification. Int J Comput Vis 127(2):181–206

    Article  MATH  Google Scholar 

  4. Gao X, Sun Q, Xu H, Wei D, Gao J (2019) Multi-model fusion metric learning for image set classification. Knowl Based Syst 164:253–264

    Article  Google Scholar 

  5. Moon HM, Seo CH, Pan SB (2017) A face recognition system based on convolution neural network using multiple distance face. Soft Comput 21(17):4995–5002

    Article  Google Scholar 

  6. Ren Z, Sun Q, Yang C (2020) Nonnegative discriminative encoded nearest points for image set classification. Neural Comput Appl 32(13):9081–9092

    Article  Google Scholar 

  7. Cevikalp H, Yavuz HS, Triggs B (2019) Face recognition based on videos by using convex hulls. IEEE Trans Circuits Syst Video Technol PP(99):1

    Google Scholar 

  8. Lei D, Jiang Z, Wu Y (2020) Weighted huber constrained sparse face recognition. Neural Comput Appl 32(9):5235–5253

    Article  Google Scholar 

  9. Huang C, Li Y, Chen CL, Tang X (2019) Deep imbalanced learning for face recognition and attribute prediction. IEEE Trans Pattern Anal Mach Intell 42(11):2781–2794

    Article  Google Scholar 

  10. Zhang Z, Jiang W, Qin J, Zhang L, Li F, Zhang M, Yan S (2017) Jointly learning structured analysis discriminative dictionary and analysis multiclass classifier. IEEE Trans Neural Netw Learn Syst 29(8):3798–3814

    Article  MathSciNet  Google Scholar 

  11. Zhang Z, Sun Y, Wang Y, Zhang Z, Zhang H, Liu G, Wang M (2020) Twin-incoherent self-expressive locality-adaptive latent dictionary pair learning for classification. IEEE Trans Neural Netw Learn Syst 99:1–15

    Google Scholar 

  12. Zhang Z, Jiang W, Zhang Z, Li S, Liu G, Qin J (2019a) Scalable block-diagonal locality-constrained projective dictionary learning. In: Proceedings of the 28th international joint conference on artificial intelligence, AAAI Press, pp 4376–4382

  13. Zhang Z, Ren J, Jiang W, Zhang Z, Hong R, Yan S, Wang M (2019b) Joint subspace recovery and enhanced locality driven robust flexible discriminative dictionary learning. IEEE Trans Circuits Syst Video Technol 30:42

    Google Scholar 

  14. Sun Y, Zhang Z, Jiang W, Zhang Z, Zhang L, Yan S, Wang M (2020) Discriminative local sparse representation by robust adaptive dictionary pair learning. IEEE Trans Neural Netw Learn Syst 31(10):4303–4317

    Article  MathSciNet  Google Scholar 

  15. Cevikalp H, Triggs B (2010) Face recognition based on image sets. In: 2010 IEEE computer society conference on computer vision and pattern recognition, IEEE, pp 2567–2573

  16. Hu Y, Mian AS, Owens R (2012) Face recognition using sparse approximated nearest points between image sets. IEEE Trans Pattern Anal Mach Intell 34(10):1992–2004

    Article  Google Scholar 

  17. Yang M, Zhu P, Van Gool L, Zhang L (2013) Face recognition based on regularized nearest points between image sets. In: 2013 10th IEEE international conference and workshops on automatic face and gesture recognition (FG), IEEE, pp 1–7

  18. Zhu P, Zuo W, Zhang L, Shiu SCK, Zhang D (2014) Image set-based collaborative representation for face recognition. IEEE Trans Inf Forensics Secur 9(7):1120–1132

    Article  Google Scholar 

  19. Zhang L, Yang M, Feng X, Ma Y, Zhang D (2012) Collaborative representation based classification for face recognition. arXiv preprint arXiv:12042358

  20. Wang W, Wang R, Shan S, Chen X (2016) Prototype discriminative learning for face image set classification. In: Asian conference on computer vision, Springer, pp 344–360

  21. Feng Q, Zhou Y, Lan R (2016) Pairwise linear regression classification for image set retrieval. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 4865–4872

  22. Yang M, Wang X, Liu W, Shen L (2017) Joint regularized nearest points for image set based face recognition. Image Vision Comput 58:47–60

    Article  Google Scholar 

  23. Zheng P, Zhao ZQ, Gao J, Wu X (2017) Image set classification based on cooperative sparse representation. Pattern Recogn 63:206–217

    Article  Google Scholar 

  24. Huang Z, Shan S, Wang R, Zhang H, Lao S, Kuerban A, Chen X (2015) A benchmark and comparative study of video-based face recognition on cox face database. IEEE Trans Image Process 24(12):5967–5981

    Article  MathSciNet  MATH  Google Scholar 

  25. Chen L, Hassanpour N (2017) Survey: how good are the current advances in image set based face identification?-experiments on three popular benchmarks with a naïve approach. Comput Vision Image Underst 160:1–23

    Article  Google Scholar 

  26. Liu X, Guo Z, You J, Kumar BV (2019) Dependency-aware attention control for image set-based face recognition. IEEE Trans Inf Forensics Secur 15:1501–1512

    Article  Google Scholar 

  27. Song Z, Cui K, Cheng G (2020) Image set face recognition based on extended low rank recovery and collaborative representation. Int J Mach Learn Cybern 11(1):71–80

    Article  Google Scholar 

  28. Mian A, Hu Y, Hartley R, Owens R (2013) Image set based face recognition using self-regularized non-negative coding and adaptive distance metric learning. IEEE Trans Image Process 22(12):5252–5262

    Article  Google Scholar 

  29. Wang G, Shi N (2020) Collaborative representation-based discriminant neighborhood projections for face recognition. Neural Comput Appl 32(10):5815–5832

    Article  Google Scholar 

  30. Chen L (2014) Dual linear regression based classification for face cluster recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2673–2680

  31. Ren CX, Luo YW, Xu XL, Dai DQ, Yan H (2019) Discriminative residual analysis for image set classification with posture and age variations. IEEE Trans Image Process 29:2875–2888

    Article  MATH  Google Scholar 

  32. Hayat M, Bennamoun M, An S (2014) Deep reconstruction models for image set classification. IEEE Trans Pattern Anal Mach Intell 37(4):713–727

    Article  Google Scholar 

  33. Sun Y, Ren Z, Yang C, Lei H (2020) Latent sparse discriminative learning for face image set classification. In: International conference on neural computing for advanced applications, Springer, pp 144–156

  34. Ren Z, Sun Q (2020) Simultaneous global and local graph structure preserving for multiple kernel clustering. IEEE Trans Neural Netw Learn Syst PP:2

    Google Scholar 

  35. Nie F, Wang X, Jordan MI, Huang H (2016) The constrained Laplacian rank algorithm for graph-based clustering. In: Proceedings of the Thirtieth AAAI conference on artificial intelligence, pp 1969–1976

  36. Wang H, Yang Y, Liu B, Fujita H (2019) A study of graph-based system for multi-view clustering. Knowl Based Syst 163:1009–1019

    Article  Google Scholar 

  37. Ren Z, Yang SX, Sun Q, Wang T (2020a) Consensus affinity graph learning for multiple kernel clustering. IEEE Trans Cybern PP:1

    Google Scholar 

  38. Ren Z, Mukherjee M, Lloret J, Venu P (2020b) Multiple kernel driven clustering with locally consistent and selfish graph in industrial IoT. IEEE Trans Ind Inf PP:3

    Google Scholar 

  39. Ren Z, Wu B, Zhang X, Sun Q (2019a) Image set classification using candidate sets selection and improved reverse training. Neurocomputing 341:60–69

    Article  Google Scholar 

  40. Ren Z, Sun Q, Wu B, Zhang X, Yan W (2019b) Learning latent low-rank and sparse embedding for robust image feature extraction. IEEE Trans Image Process 29(1):2094–2107

    MATH  Google Scholar 

  41. Learned-Miller E, Huang GB, RoyChowdhury A, Li H, Hua G (2016) Labeled faces in the wild: a survey. Advances in face detection and facial image analysis. Springer, Cham, pp 189–248

    Google Scholar 

  42. Taigman Y, Wolf L, Hassner T et al (2009) Multiple one-shots for utilizing class label information. BMVC 2:1–12

    Google Scholar 

  43. Cui H, Zhu L, Li J, Yang Y, Nie L (2019) Scalable deep hashing for large-scale social image retrieval. IEEE Trans Image Process 29:1271–1284

    Article  MathSciNet  MATH  Google Scholar 

Download references

Acknowledgements

This research was supported by the Sichuan Science and Technology Program (Grant Nos. 2019ZDZX0043 and 2020ZDZX0014), the Key Lab of Film and TV Media Technology of Zhejiang Province (Grant No. 2020E10015), the Natural Science Foundation Project of CQ CSTC (Grant No. cstc2020jcyj-msxmX0473), the Major Cultivation Research Projects of Chongqing Three Gorges University (Grant No. 18ZDPY07), the Scientific Research Fund of Sichuan Provincial Education Department (Grant No. 17ZB0441), the Scientific Research Fund of Southwest University of Science and Technology (Grant No. 17zx7137), the Postgraduate Innovation Fund Project by Southwest University of Science and Technology (Grant No. 20ycx0001), and the Undergraduate Innovation and Entrepreneurship Training Program of Sichuan (Grant No. S202010619014).

Author information

Authors and Affiliations

  1. Department of National Defence Science and Technology, Southwest University of Science and Technology, Mianyang, 621010, China

    Yuan Sun, Zhenwen Ren, Chao Yang & Yanglong Ou

  2. Department of Computer Science and Engineering, Nanjing University of Science and Technology, Nanjing, 210094, China

    Zhenwen Ren & Quansen Sun

  3. Department of Electronic and Information Engineering, Chongqing Three Gorges University, Chongqing, 404199, China

    Liwan Chen

Authors
  1. Yuan Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zhenwen Ren
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Chao Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Quansen Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Liwan Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Yanglong Ou
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Zhenwen Ren or Chao Yang.

Ethics declarations

Conflict of interest

There is no conflicts of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Sun, Y., Ren, Z., Yang, C. et al. Face image set classification with self-weighted latent sparse discriminative learning. Neural Comput & Applic 35, 12283–12295 (2023). https://doi.org/10.1007/s00521-020-05479-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-020-05479-1

Keywords

Search

Navigation