Skip to main content
SpringerLink
Log in

Robust Discriminant Projection Via Joint Margin and Locality Structure Preservation

  • Published:
Neural Processing Letters Aims and scope Submit manuscript

Abstract

It is very challenging to obtain sufficiently discriminative features from the original data in real-world applications. Despite the multiplicity of researches on the linear discriminative analysis, most of them are sensitive to noise, outliers and the distribution of data, especially in the low sample size context. In this paper, we propose a novel image classification method, namely Margin and Locality Discriminant Projection, which simultaneously considers the margin and locality structure information based on low-rank and sparse representation. Specifically, the proposed method integrates marginal fisher analysis and neighborhood preserving embedding so as to preserve the intrinsic structure as well as enhance the discriminative ability, on account of which a more robust and comprehensive graph can be constructed to obtain sufficiently discriminative features. Meanwhile, the low-rank and sparsity constraints are introduced to compensate the noise. The proposed model can be solved efficiently using the linear alternative direction method with adaptive penalty and eigen-decomposition. Extensive experiments are conducted on four databases and the results demonstrate that the proposed method can achieve superior performance than other state-of-the-art algorithms.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15

Similar content being viewed by others

References

  1. Belhumeur PN, Hespanha JP, Kriegman DJ (1997) Eigenfaces vs. fisherfaces: recognition using class specific linear projection. IEEE Trans Pattern Anal Mach Intell 19(7):711–720

    Article  Google Scholar 

  2. Cai D, He X, Han J, Zhang HJ (2006) Orthogonal Laplacian faces for face recognition. IEEE Trans Image Process 15(11):3608–3614

    Article  Google Scholar 

  3. Cai D, He X, Hu Y, Han J, Huang T (2007) Learning a spatially smooth subspace for face recognition. In: 2007 IEEE conference on computer vision and pattern recognition. IEEE, pp 1–7

  4. Cai JF, Candès EJ, Shen Z (2010) A singular value thresholding algorithm for matrix completion. SIAM J Optim 20(4):1956–1982

    Article  MathSciNet  MATH  Google Scholar 

  5. Candès EJ, Li X, Ma Y, Wright J (2011) Robust principal component analysis? J ACM 58(3):11

    Article  MathSciNet  MATH  Google Scholar 

  6. Clemmensen L, Hastie T, Witten D, Ersbøll B (2011) Sparse discriminant analysis. Technometrics 53(4):406–413

    Article  MathSciNet  Google Scholar 

  7. Dong W, Wu XJ (2018) Robust low rank subspace segmentation via joint \(l_{21}\)-norm minimization. Neural Process Lett 48(1):299–312

    Google Scholar 

  8. Díaz-Vico D, Dorronsoro JR (2019) Deep least squares fisher discriminant analysis. IEEE Trans Neural Netw Learn Syst PP(99)

  9. Elhamifar E, Vidal R (2009) Sparse subspace clustering. In: 2009 IEEE conference on computer vision and pattern recognition. IEEE, pp 2790–2797

  10. Fang X, Xu Y, Li X, Lai Z, Wong WK (2015) Robust semi-supervised subspace clustering via non-negative low-rank representation. IEEE Trans Cybern 46(8):1828–1838

    Article  Google Scholar 

  11. Georghiades AS, Belhumeur PN, Kriegman DJ (2001) From few to many: illumination cone models for face recognition under variable lighting and pose. IEEE Trans Pattern Anal Mach Intell 23(6):643–660

    Article  Google Scholar 

  12. Guo T, Tan X, Zhang L, Liu Q, Deng L, Xie C (2019) Learning robust weighted group sparse graph for discriminant visual analysis. Neural Process Lett 49(1):203–226

    Article  Google Scholar 

  13. He X, Cai D, Yan S, Zhang HJ (2005) Neighborhood preserving embedding. In: Tenth IEEE international conference on computer vision (ICCV’05) Volume 1, vol 2. IEEE, pp 1208–1213

  14. He X, Niyogi P (2004) Locality preserving projections. In: Advances in neural information processing systems, pp 153–160

  15. He X, Yan S, Hu Y, Niyogi P, Zhang H (2005) Face recognition using Laplacianfaces. IEEE Trans Pattern Anal Mach Intell 27(3):328–340

    Article  Google Scholar 

  16. Huang H, Liu J, Pan Y (2012) Semi-supervised marginal fisher analysis for hyperspectral image classification. ISPRS Ann Photogramm Remote Sens Spatial Inform Sci 377–382

  17. Jin T, Liu Z, Yu Z, Min X, Li L (2017) Locality preserving collaborative representation for face recognition. Neural Process Lett 45(3):967–979

    Article  Google Scholar 

  18. Kokiopoulou E, Saad Y (2005) Orthogonal neighborhood preserving projections. In: Fifth IEEE international conference on data mining (ICDM’05). IEEE, p 8

  19. Krizhevsky A (2012) Learning multiple layers of features from tiny images. University of Toronto, Toronto

    Google Scholar 

  20. Lazar C, Taminau J, Meganck S, Steenhoff D, Coletta A, Molter C, De SV, Duque R, Bersini H, Nowé A (2012) A survey on filter techniques for feature selection in gene expression microarray analysis. IEEE/ACM Trans Comput Biol Bioinform 9(4):1106–1119

    Article  Google Scholar 

  21. Li F, Jiang M (2018) Tensor locality preserving sparse projection for image feature extraction. Neural Process Lett 47(2):661–678

    MathSciNet  Google Scholar 

  22. Li J, Chang H, Yang J (2014) Learning discriminative low-rank representation for image classification. In: 2014 international joint conference on neural networks (IJCNN). IEEE, pp 313–318

  23. Li J, Xuan J, Shi T (2013) Feature extraction based on semi-supervised kernel marginal fisher analysis and its application in bearing fault diagnosis. Mech Syst Sig Process 41(1–2):113–126

    Article  Google Scholar 

  24. Li S, Fu Y (2015) Learning robust and discriminative subspace with low-rank constraints. IEEE Trans Neural Netw Learn Syst 27(11):2160–2173

    Article  MathSciNet  Google Scholar 

  25. Lin Z, Chen M, Ma Y (2010) The augmented lagrange multiplier method for exact recovery of corrupted low-rank matrices. arXiv preprint arXiv:1009.5055

  26. Lin Z, Liu R, Su Z (2011) Linearized alternating direction method with adaptive penalty for low-rank representation. In: Advances in neural information processing systems, pp 612–620

  27. Liu G, Lin Z, Yan S, Sun J, Yu Y, Ma Y (2012) Robust recovery of subspace structures by low-rank representation. IEEE Trans Pattern Anal Mach Intell 35(1):171–184

    Article  Google Scholar 

  28. Liu G, Lin Z, Yong Y (2010) Robust subspace segmentation by low-rank representation. In: International conference on machine learning

  29. Liu G, Yan S (2011) Latent low-rank representation for subspace segmentation and feature extraction. In: 2011 international conference on computer vision. IEEE, pp 1615–1622

  30. Liu Q, Lu H, Ma S (2004) Improving kernel fisher discriminant analysis for face recognition. IEEE Trans Circuits Syst Video Technol 14(1):42–49

    Article  Google Scholar 

  31. Liu Y, Tang F, Zeng Z (2015) Feature selection based on dependency margin. IEEE Trans Cybern 45(6):1209–1221

    Article  Google Scholar 

  32. Liu Z, Wang J, Liu G, Pu J (2019) Sparse low-rank preserving projection for dimensionality reduction. IEEE Access 7:22941–22951

    Article  Google Scholar 

  33. Lu J, Gang W, Deng W, Moulin P, Jie Z (2015) Multi-manifold deep metric learning for image set classification. In: IEEE Conference on computer vision and pattern recognition

  34. Lu Y, Lai Z, Xu Y, Li X, Zhang D, Yuan C (2015) Low-rank preserving projections. IEEE Trans Cybern 46(8):1900–1913

    Article  Google Scholar 

  35. Maronidis A, Tefas A, Pitas I (2015) Subclass graph embedding and a marginal fisher analysis paradigm. Pattern Recogn 48(12):4024–4035

    Article  MATH  Google Scholar 

  36. Martínez AM, Kak AC (2001) Pca versus lda. IEEE Trans Pattern Anal Mach Intell 23(2):228–233

    Article  Google Scholar 

  37. Nene SA, Nayar SK, Murase H et al (1996) Columbia object image library (coil-20)

  38. Qiao Z, Zhou L, Huang JZ (2009) Sparse linear discriminant analysis with applications to high dimensional low sample size data. Int J Appl Math 39(1)

  39. Roweis ST, Saul LK (2000) Nonlinear dimensionality reduction by locally linear embedding. Science 290(5500):2323–2326

    Article  Google Scholar 

  40. Schölkopf B, Smola A, Müller KR (1998) Nonlinear component analysis as a kernel eigenvalue problem. Neural Comput 10(5):1299–1319

    Article  Google Scholar 

  41. Shailaja K, Anuradha B (2017) Effective face recognition using deep learning based linear discriminant classification. In: 2016 IEEE international conference on computational intelligence and computing research (ICCIC)

  42. Sim T, Baker S, Bsat M (2002) The cmu pose, illumination, and expression (pie) database. In: Proceedings of fifth IEEE international conference on automatic face gesture recognition. IEEE, pp 53–58

  43. Tang P, Wang X, Feng B, Liu W (2017) Learning multi-instance deep discriminative patterns for image classification. IEEE Trans Image Process 26(7):3385–3396

    Article  MathSciNet  MATH  Google Scholar 

  44. Turk MA, Pentland AP (1991) Face recognition using eigenfaces. In: Proceedings. 1991 IEEE computer society conference on computer vision and pattern recognition. IEEE, pp 586–591

  45. Wang P, Han B, Li J, Gao X (2019) Structural reweight sparse subspace clustering. Neural Process Lett 49(3):965–977

    Article  Google Scholar 

  46. Wang R, Chen C, Li J, Dai T (2016) Face recognition via non-negative sparse low-rank representation classification. In: 2016 12th international conference on natural computation, fuzzy systems and knowledge discovery (ICNC-FSKD). IEEE, pp 1609–1614

  47. Wang S, Xie D, Chen F, Gao Q (2018) Dimensionality reduction by lpp-l21. IET Comput Vis 12(5):659–665

    Article  Google Scholar 

  48. Wei L, Wang X, Wu A, Zhou R, Zhu C (2018) Robust subspace segmentation by self-representation constrained low-rank representation. Neural Process Lett 48(3):1671–1691

    Article  Google Scholar 

  49. Wen J, Fang X, Cui J, Fei L, Yan K, Chen Y, Xu Y (2018) Robust sparse linear discriminant analysis. IEEE Trans Circuits Syst Video Technol 29(2):390–403

    Article  Google Scholar 

  50. Wong WK, Lai Z, Wen J, Fang X, Lu Y (2017) Low-rank embedding for robust image feature extraction. IEEE Trans Image Process 26(6):2905–2917

    Article  MathSciNet  MATH  Google Scholar 

  51. Wright J, Yang AY, Ganesh A, Sastry SS, Ma Y (2008) Robust face recognition via sparse representation. IEEE Trans Pattern Anal Mach Intell 31(2):210–227

    Article  Google Scholar 

  52. Xie L, Yin M, Yin X, Liu Y, Yin G (2018) Low-rank sparse preserving projections for dimensionality reduction. IEEE Trans Image Process 27(11):5261–5274

    Article  MathSciNet  MATH  Google Scholar 

  53. Xu D, Yan S, Tao D, Lin S, Zhang HJ (2007) Marginal fisher analysis and its variants for human gait recognition and content- based image retrieval. IEEE Trans Image Process 16(11):2811–2821

    Article  MathSciNet  Google Scholar 

  54. Yan S, Xu D, Zhang B, Zhang H, Yang Q, Lin S (2007) Graph embedding and extensions: a general framework for dimensionality reduction. IEEE Trans Pattern Anal Mach Intell 29(1):40–51

    Article  Google Scholar 

  55. Ye J, Xiong T (2006) Null space versus orthogonal linear discriminant analysis. In: Proceedings of the 23rd international conference on Machine learning. ACM, pp 1073–1080

  56. Yin M, Gao J, Lin Z (2015) Laplacian regularized low-rank representation and its applications. IEEE Trans Pattern Anal Mach Intell 38(3):504–517

    Article  Google Scholar 

  57. Zhuang L, Gao H, Lin Z, Ma Y, Zhang X, Yu N (2012) Non-negative low rank and sparse graph for semi-supervised learning. In: 2012 IEEE conference on computer vision and pattern recognition. IEEE, pp 2328–2335

  58. Zou H, Hastie T, Tibshirani R (2006) Sparse principal component analysis. J Comput Gr Stat 15(2):265–286

    Article  MathSciNet  Google Scholar 

Download references

Acknowledgements

This work was supported in part by the National Key R&D Program of China under Grant 2018YFB1003201, in part by the National Natural Science Foundation of China under Grant 61702114 and Grant 61672171, in part by the High-level Talents Programme of Guangdong Province under Grant 2017GC010556, in part by the Natural Science Foundation of Guangdong Province under Grant 2020A1515011361, in part by the Guangdong Key R&D Project of China under Grants 2018B01010 7003, 2019B010121001 and in part by the Guangdong Natural Science foundation under Grant 2018B0303 11007.

Author information

Authors and Affiliations

  1. School of Computer Science, Guangdong University of Technology, Guangzhou, China

    Min Meng, Yu Liu & Jigang Wu

Authors
  1. Min Meng
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yu Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jigang Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Min Meng.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Meng, M., Liu, Y. & Wu, J. Robust Discriminant Projection Via Joint Margin and Locality Structure Preservation. Neural Process Lett 53, 959–982 (2021). https://doi.org/10.1007/s11063-020-10418-1

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11063-020-10418-1

Keywords

Search

Navigation