Skip to main content
SpringerLink
Log in

Adaptive unsupervised feature selection with robust graph regularization

  • Original Article
  • Published:
International Journal of Machine Learning and Cybernetics Aims and scope Submit manuscript

Abstract

Unsupervised feature selection, aiming at finding a refined representation of the original data by filtering out irrelevant and redundant features, has attracted intensive attention. Due to the dilemma of unavailable labels, existing methods select relevant features that preserve the intrinsic structure of data. Despite they are proven effective, the fixed metric is utilized to measure the distances from the projected samples to the target representation in the reconstruction term, which means that existing methods can not possess sufficient flexibility to adapt to different types of data sources. Besides, conventional methods utilize the \(l_2\) norm based Laplacian graph to preserve the local structure of data, which leads to the sensitivity to noisy data. Inspired by the effectiveness and flexibility of the \(l_{2,p}\) norm metric, we propose adaptive unsupervised feature selection with robust graph regularization (AUFS). Specifically, we impose the \(l_{2,p}\) norm on the feature reconstruction term, which enhance the adaptability of our method to different types of data sources by adjusting p. In addition, \(l_{2,1}\) norm based Laplacian graph is designed to alleviate the negative impact of noisy data. To solve the optimization problem, a unified iterative algorithm with guaranteed convergence is proposed. A large number of experimental results on several benchmark datasets demonstrate that our method outperforms some latest and related methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Data availability

Data will be made available on request.

References

  1. Luo F, Zhang L, Du B, Zhang L (2020) Dimensionality reduction with enhanced hybrid-graph discriminant learning for hyperspectral image classification. IEEE Trans Geosci Remote Sens 58(8):5336–5353

    Article  Google Scholar 

  2. Wong M, Hung C-C (2020) Dimensionality reduction with weighted k-means for hyperspectral image classification. In: IGARSS 2020-2020 IEEE International Geoscience and Remote Sensing Symposium, pp. 44–47

  3. Chantar H, Mafarja M, Alsawalqah H, Heidari AA, Aljarah I, Faris H (2020) Feature selection using binary grey wolf optimizer with elite-based crossover for arabic text classification. Neural Comput Appl 32:12201–12220

    Article  Google Scholar 

  4. Tang C, Liu X, Zhu X, Xiong J, Li M, Xia J, Wang X, Wang L (2019) Feature selective projection with low-rank embedding and dual laplacian regularization. IEEE Trans Knowl Data Eng 32(9):1747–1760

    Google Scholar 

  5. Zhang C, Fu H, Hu Q, Zhu P, Cao X (2017) Flexible multi-view dimensionality co-reduction. IEEE Trans Image Process 26(2):648–659

    Article  MathSciNet  Google Scholar 

  6. Lu J, Wang H, Zhou J, Chen Y, Lai Z, Hu Q (2021) Low-rank adaptive graph embedding for unsupervised feature extraction. Pattern Recogn 113:107758

    Article  Google Scholar 

  7. Yang Z, Wan M, Zhan T, Lai Z, Luo L, Huang P, Zhang J (2018) Unsupervised multi-manifold linear differential projection (umldp) for face recognition. Multimed Tools Appl 77:3795–3811

    Article  Google Scholar 

  8. Lai Z, Xu Y, Yang J, Shen L, Zhang D (2016) Rotational invariant dimensionality reduction algorithms. IEEE Trans Cybern 47(11):3733–3746

    Article  Google Scholar 

  9. Lai Z, Chen X, Zhang J, Kong H, Wen J (2023) Maximal margin support vector machine for feature representation and classification. IEEE Trans Cybern 2:2

    Google Scholar 

  10. Shang R, Kong J, Feng J, Jiao L (2022) Feature selection via non-convex constraint and latent representation learning with laplacian embedding. Expert Syst Appl 208:118179

    Article  Google Scholar 

  11. Shang R, Kong J, Zhang W, Feng J, Jiao L, Stolkin R (2022) Uncorrelated feature selection via sparse latent representation and extended olsda. Pattern Recogn 132:108966

    Article  Google Scholar 

  12. Nematzadeh H, Enayatifar R, Mahmud M, Akbari E (2019) Frequency based feature selection method using whale algorithm. Genomics 111(6):1946–1955

    Article  Google Scholar 

  13. Zhao Z, Liu H (2007) Spectral feature selection for supervised and unsupervised learning. In: Proceedings of the 24th International Conference on Machine Learning, pp. 1151–1157

  14. Tabakhi S, Moradi P, Akhlaghian F (2014) An unsupervised feature selection algorithm based on ant colony optimization. Eng Appl Artif Intell 32:112–123

    Article  Google Scholar 

  15. Zhang R, Li X (2020) Regularized regression with fuzzy membership embedding for unsupervised feature selection. IEEE Trans Fuzzy Syst 29(12):3743–3753

    Article  Google Scholar 

  16. He X, Cai D, Niyogi P (2005) Laplacian score for feature selection. Adv Neural Inf Process Syst 18:507–514

    Google Scholar 

  17. Lin Q, Men M, Yang L, Zhong P (2022) A supervised multi-view feature selection method based on locally sparse regularization and block computing. Inf Sci 582:146–166

    Article  MathSciNet  Google Scholar 

  18. Lin Q, Yang L, Zhong P, Zou H (2021) Robust supervised multi-view feature selection with weighted shared loss and maximum margin criterion. Knowl-Based Syst 229:107331

    Article  Google Scholar 

  19. Zhong J, Wang N, Lin Q, Zhong P (2019) Weighted feature selection via discriminative sparse multi-view learning. Knowl-Based Syst 178:132–148

    Article  Google Scholar 

  20. Shi C, Gu Z, Duan C, Tian Q (2020) Multi-view adaptive semi-supervised feature selection with the self-paced learning. Signal Process 168:107332

    Article  Google Scholar 

  21. Zhang H, Wu D, Nie F, Wang R, Li X (2021) Multilevel projections with adaptive neighbor graph for unsupervised multi-view feature selection. Inf Fusion 70:129–140

    Article  Google Scholar 

  22. Cao Z, Xie X, Sun F, Qian J (2023) Consensus cluster structure guided multi-view unsupervised feature selection. Knowl-Based Syst 2:110578

    Article  Google Scholar 

  23. Dong X, Zhu L, Song X, Li J, Cheng Z (2018) Adaptive collaborative similarity learning for unsupervised multi-view feature selection. In: International Joint Conference on Artificial Intelligence, pp. 2064–2070

  24. Shang R, Zhang W, Lu M, Jiao L, Li Y (2022) Feature selection based on non-negative spectral feature learning and adaptive rank constraint. Knowl-Based Syst 236:107749

    Article  Google Scholar 

  25. Zhu P, Hou X, Tang K, Liu Y, Zhao Y-P, Wang Z (2023) Unsupervised feature selection through combining graph learning and l2, 0-norm constraint. Inf Sci 622:68–82

    Article  Google Scholar 

  26. Li X, Zhang H, Zhang R, Nie F (2019) Discriminative and uncorrelated feature selection with constrained spectral analysis in unsupervised learning. IEEE Trans Image Process 29:2139–2149

    Article  MathSciNet  Google Scholar 

  27. Lin X, Guan J, Chen B, Zeng Y (2021) Unsupervised feature selection via orthogonal basis clustering and local structure preserving. IEEE Trans Neural Netw Learn Syst 33(11):6881–6892

    Article  MathSciNet  Google Scholar 

  28. Luo C, Zheng J, Li T, Chen H, Huang Y, Peng X (2022) Orthogonally constrained matrix factorization for robust unsupervised feature selection with local preserving. Inf Sci 586:662–675

    Article  Google Scholar 

  29. Liu Y, Ye D, Li W, Wang H, Gao Y (2020) Robust neighborhood embedding for unsupervised feature selection. Knowl-Based Syst 193:105462

    Article  Google Scholar 

  30. Zhang M, Yang Y, Zhang H, Shen F, Zhang D (2016) L2, p-norm and sample constraint based feature selection and classification for ad diagnosis. Neurocomputing 195:104–111

    Article  Google Scholar 

  31. Kodirov E, Xiang T, Fu Z, Gong S (2016) Learning robust graph regularisation for subspace clustering. In: BMVC, pp. 138–113812

  32. Tang C, Zhu X, Chen J, Wang P, Liu X, Tian J (2018) Robust graph regularized unsupervised feature selection. Expert Syst Appl 96:64–76

    Article  Google Scholar 

  33. Li X, Zhang H, Zhang R, Liu Y, Nie F (2018) Generalized uncorrelated regression with adaptive graph for unsupervised feature selection. IEEE Trans Neural Netw Learn Syst 30(5):1587–1595

    Article  MathSciNet  Google Scholar 

  34. Huang Y, Shen Z, Cai F, Li T, Lv F (2021) Adaptive graph-based generalized regression model for unsupervised feature selection. Knowl-Based Syst 227:107156

    Article  Google Scholar 

  35. Cai D, Zhang C, He X (2010) Unsupervised feature selection for multi-cluster data. In: Proceedings of the 16th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 333–342

  36. Zhu P, Zhu W, Hu Q, Zhang C, Zuo W (2017) Subspace clustering guided unsupervised feature selection. Pattern Recogn 66:364–374

    Article  Google Scholar 

  37. Zhu P, Zuo W, Zhang L, Hu Q, Shiu SC (2015) Unsupervised feature selection by regularized self-representation. Pattern Recogn 48(2):438–446

    Article  Google Scholar 

  38. Tang C, Bian M, Liu X, Li M, Zhou H, Wang P, Yin H (2019) Unsupervised feature selection via latent representation learning and manifold regularization. Neural Netw 117:163–178

    Article  Google Scholar 

  39. Shang R, Wang L, Shang F, Jiao L, Li Y (2021) Dual space latent representation learning for unsupervised feature selection. Pattern Recogn 114:107873

    Article  Google Scholar 

  40. Nie F, Zhu W, Li X (2019) Structured graph optimization for unsupervised feature selection. IEEE Trans Knowl Data Eng 33(3):1210–1222

    Google Scholar 

  41. Han D, Kim J (2015) Unsupervised simultaneous orthogonal basis clustering feature selection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5016–5023

  42. Liu X, Wang L, Zhang J, Yin J, Liu H (2013) Global and local structure preservation for feature selection. IEEE Trans Neural Netw Learn Syst 25(6):1083–1095

    Google Scholar 

  43. Nie F, Zhu W, Li X (2016) Unsupervised feature selection with structured graph optimization. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 30, pp. 1302–1308

  44. Parsa MG, Zare H, Ghatee M (2020) Unsupervised feature selection based on adaptive similarity learning and subspace clustering. Eng Appl Artif Intell 95:25

    Article  Google Scholar 

  45. Wang S, Xiao S, Zhu W, Guo Y (2022) Multi-view fuzzy clustering of deep random walk and sparse low-rank embedding. Inf Sci 586:224–238

    Article  Google Scholar 

Download references

Acknowledgements

This work is supported by National Natural Science Foundation of China (No. 61906101).

Author information

Authors and Affiliations

  1. School of Information Science and Engineering, Ningbo University, Ningbo, 315211, China

    Zhiwen Cao, Xijiong Xie & Feixiang Sun

Authors
  1. Zhiwen Cao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xijiong Xie
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Feixiang Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xijiong Xie.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Cao, Z., Xie, X. & Sun, F. Adaptive unsupervised feature selection with robust graph regularization. Int. J. Mach. Learn. & Cyber. 15, 341–354 (2024). https://doi.org/10.1007/s13042-023-01912-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13042-023-01912-8

Keywords

Search

Navigation