Skip to main content
SpringerLink
Log in

A new graph-preserving unsupervised feature selection embedding LLE with low-rank constraint and feature-level representation

  • Published:
Artificial Intelligence Review Aims and scope Submit manuscript

Abstract

Unsupervised feature selection is a powerful tool to process high-dimensional data, in which a subset of features is selected out for effective data representation. In this paper, we proposes a novel robust unsupervised features selection method based on graph-preserving feature selection embedding LLE. Specifically, we integrate the graph matrix learning and the low-dimensional space learning together to identify the correlation among both features and samples from the intrinsic low-dimensional space of original data. Also, the global and local correlation of features have been taken into consideration through the low-rank constraint and the feature-level representation property to find lower-dimensional representation which preserves not only the global and local correlation of features but also the global and local structure of training samples. Furthermore, we propose a new optimization algorithm to the resulting objective function, which iteratively updates the graph matrix and the intrinsic space in order to collaboratively improve each of them. Experimental analysis on 18 benchmark datasets verified that our proposed method outperformed the state-of-the-art feature selection methods in terms of classification and clustering performance.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  • Benabdeslem K, Hindawi M (2014) Efficient semi-supervised feature selection: constraint, relevance, and redundancy. IEEE Trans Knowl Data Eng 26(5):1131–1143

    Article  Google Scholar 

  • Cai D, Zhang C, He X (2010) Unsupervised feature selection for multi-cluster data. In: Proceedings of the 16th ACM SIGKDD international conference on Knowledge discovery and data mining. ACM, pp 333–342

  • Cai X, Ding C, Nie F et al (2013) On the equivalent of low-rank linear regressions and linear discriminant analysis based regressions. In: Proceedings of the 19th ACM SIGKDD international conference on knowledge discovery and data mining. ACM, pp 1124–1132

  • Chen L, Huang JZ (2012) Sparse reduced-rank regression for simultaneous dimension reduction and variable selection. J Am Stat Assoc 107(500):1533–1545

    Article  MathSciNet  Google Scholar 

  • Cheng D, Zhang S, Liu X et al (2017) Feature selection by combining subspace learning with sparse representation. Multimed Syst 23(3):285–291

    Article  Google Scholar 

  • Du L, Shen YD (2015) Unsupervised feature selection with adaptive structure learning. In: Proceedings of the 21th ACM SIGKDD international conference on knowledge discovery and data mining. ACM, pp 209–218

  • Du L, Shen Z, Li X et al (2013) Local and global discriminative learning for unsupervised feature selection. In: 2013 IEEE 13th international conference on data mining (ICDM). IEEE, pp 131–140

  • Du S, Wang W, Ma Y (2016) Low rank sparse preserve projection for face recognition. In: Control and decision conference (CCDC), 2016 Chinese. IEEE, pp 3822–3826

  • Gao S, Ver Steeg G, Galstyan A (2016) Variational information maximization for feature selection. In: Advances in neural information processing systems, pp 487–495

  • García-Torres M, Gómez-Vela F, Melián-Batista B et al (2016) High-dimensional feature selection via feature grouping: a variable neighborhood search approach. Inf Sci 326:102–118

    Article  MathSciNet  Google Scholar 

  • Guyon I, Elisseeff A (2003) An introduction to variable and feature selection. J Mach Learn Res 3(3):1157–1182

    MATH  Google Scholar 

  • Han Y, Xu Z, Ma Z et al (2013) Image classification with manifold learning for out-of-sample data. Signal Process 93(8):2169–2177

    Article  Google Scholar 

  • He X, Cai D, Niyogi P (2006) Laplacian score for feature selection. In: Weiss Y, Schölkopf B, Platt JC (eds) Advances in neural information processing systems. Neural information processing systems foundation, British Columbia, pp 507–514

    Google Scholar 

  • Jian L, Li J, Shu K et al (2016) Multi-label informed feature selection. In: IJCAI, pp 1627–1633

  • Jiang Y, Ren J (2011) Eigenvalue sensitive feature selection. In: Proceedings of the 28th international conference on machine learning (ICML-11), pp 89–96

  • Lan X, Ma AJ, Yuen PC et al (2015) Joint sparse representation and robust feature-level fusion for multi-cue visual tracking. IEEE Trans Image Process 24(12):5826–5841

    Article  MathSciNet  Google Scholar 

  • Lan X, Zhang S, Yuen PC (2016) Robust joint discriminative feature learning for visual tracking. In: IJCAI, pp 3403–3410

  • Li Z, Yang Y, Liu J et al (2012) Unsupervised feature selection using nonnegative spectral analysis. In: AAAI, vol 2, pp 1026–1032

  • Li J, Tang J, Liu H (2017a) Reconstruction-based unsupervised feature selection: an embedded approach. In: Proceedings of the 26th international joint conference on artificial intelligence. IJCAI/AAAI

  • Li J, Wu L, Zaïane OR et al (2017b) Toward personalized relational learning. In: Proceedings of the 2017 SIAM international conference on data mining. Society for Industrial and Applied Mathematics, pp 444–452

  • Liu M, Zhang D (2014) Sparsity score: a novel graph-preserving feature selection method. Int J Pattern Recognit Artif Intell 28(04):1450009

    Article  Google Scholar 

  • Liu G, Lin Z, Yu Y (2010) Robust subspace segmentation by low-rank representation. In: Proceedings of the 27th international conference on machine learning (ICML-10), pp 663–670

  • Luo M, Nie F, Chang X et al (2018a) Adaptive unsupervised feature selection with structure regularization. IEEE Trans Neural Netw Learn Syst 29(4):944–956

    Article  Google Scholar 

  • Luo M, Chang X, Nie L et al (2018b) An adaptive semisupervised feature analysis for video semantic recognition. IEEE Trans Cybern 48(2):648–660

    Article  Google Scholar 

  • Ma J, Zhou H, Zhao J et al (2015) Robust feature matching for remote sensing image registration via locally linear transforming. IEEE Trans Geosci Remote Sens 53(12):6469–6481

    Article  Google Scholar 

  • Mitra S, Kundu PP, Pedrycz W (2012) Feature selection using structural similarity. Inf Sci 198:48–61

    Article  Google Scholar 

  • Nie F, Huang H, Cai X et al (2010) Efficient and robust feature selection via joint ℓ2, 1-norms minimization. In: Lafferty JD, Williams CKI, Shawe-Taylor J, Zemel RS, Culotta A (eds) Advances in neural information processing systems. DBLP, British Columbia, pp 1813–1821

    Google Scholar 

  • Nie F, Wang X, Huang H (2014) Clustering and projected clustering with adaptive neighbors. In: Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining. ACM. pp 977–986

  • Nie F, Zhu W, Li X (2016) Unsupervised feature selection with structured graph optimization. In: AAAI, pp 1302–1308

  • Peng Y, Long X, Lu BL (2015) Graph based semi-supervised learning via structure preserving low-rank representation. Neural Process Lett 41(3):389–406

    Article  Google Scholar 

  • Qiao L, Chen S, Tan X (2010) Sparsity preserving projections with applications to face recognition. Pattern Recogn 43(1):331–341

    Article  Google Scholar 

  • Sheikhpour R, Sarram MA, Gharaghani S et al (2017) A survey on semi-supervised feature selection methods. Pattern Recogn 64:141–158

    Article  Google Scholar 

  • Shi L, Du L, Shen YD (2014) Robust spectral learning for unsupervised feature selection. In: 2014 IEEE international conference on data mining (ICDM). IEEE, pp 977–982

  • Shi X, Guo Z, Lai Z et al (2015) A framework of joint graph embedding and sparse regression for dimensionality reduction. IEEE Trans Image Process 24(4):1341–1355

    Article  MathSciNet  Google Scholar 

  • Tang J, Hu X, Gao H et al (2014) Discriminant analysis for unsupervised feature selection. In: Proceedings of the 2014 SIAM international conference on data mining. Society for Industrial and Applied Mathematics, pp 938–946

  • Tenenbaum JB, De Silva V, Langford JC (2000) A global geometric framework for nonlinear dimensionality reduction. Science 290(5500):2319–2323

    Article  Google Scholar 

  • Wang D, Nie F, Huang H (2014) Unsupervised feature selection via unified trace ratio formulation and k-means clustering (track). In: Joint European conference on machine learning and knowledge discovery in databases. Springer, Berlin, pp 306–321

  • Wang S, Tang J, Liu H (2015) Embedded unsupervised feature selection. In: AAAI, Citeseer, pp 470–476

  • Wei X, Philip SY (2016) Unsupervised feature selection by preserving stochastic neighbors. In: Gretton A, Robert CC (eds) Artificial intelligence and statistics. PMLR, Cadiz, pp 995–1003

    Google Scholar 

  • Yang Y, Zhuang YT, Wu F et al (2008) Harmonizing hierarchical manifolds for multimedia document semantics understanding and cross-media retrieval. IEEE Trans Multimed 10(3):437–446

    Article  Google Scholar 

  • Yao C, Liu YF, Jiang B et al (2017) LLE score: a new filter-based unsupervised feature selection method based on nonlinear manifold embedding and its application to image recognition. IEEE Trans Image Process 26(11):5257–5269

    Article  MathSciNet  Google Scholar 

  • Zhang L, Song M, Yang Y et al (2014) Weakly supervised photo cropping. IEEE Trans Multimed 16(1):94–107

    Article  Google Scholar 

  • Zhang L, Gao Y, Xia Y et al (2015) A fine-grained image categorization system by cellet-encoded spatial pyramid modeling. IEEE Trans Industr Electron 62(1):564–571

    Article  Google Scholar 

  • Zhang D, Han J, Jiang L et al (2017a) Revealing event saliency in unconstrained video collection. IEEE Trans Image Process 26(4):1746–1758

    Article  MathSciNet  Google Scholar 

  • Zhang Y, Wang Y, Jin J et al (2017b) Sparse Bayesian learning for obtaining sparsity of EEG frequency bands based feature vectors in motor imagery classification. Int J Neural Syst 27(02):1650032

    Article  Google Scholar 

  • Zhang S, Li X, Zong M et al (2017c) Learning k for knn classification. ACM Trans Intell Syst Technol 8(3):43

    Google Scholar 

  • Zhao Z, Wang L, Liu H (2010) Efficient spectral feature selection with minimum redundancy. In: AAAI, pp 673–678

  • Zhao Z, Wang L, Liu H et al (2013) On similarity preserving feature selection. IEEE Trans Knowl Data Eng 25(3):619–632

    Article  Google Scholar 

  • Zhu X, Zhang L, Huang Z (2014) A sparse embedding and least variance encoding approach to hashing. IEEE Trans Image Process 23(9):3737–3750

    Article  MathSciNet  Google Scholar 

  • Zhu P, Zuo W, Zhang L et al (2015) Unsupervised feature selection by regularized self-representation. Pattern Recogn 48(2):438–446

    Article  Google Scholar 

  • Zhu X, Li X, Zhang S (2016) Block-row sparse multiview multilabel learning for image classification. IEEE Trans Cybern 46(2):450–461

    Article  Google Scholar 

  • Zhu X, Li X, Zhang S et al (2017a) Graph PCA hashing for similarity search. IEEE Trans Multimed 19(9):2033–2044

    Article  Google Scholar 

  • Zhu X, Li X, Zhang S et al (2017b) Robust joint graph sparse coding for unsupervised spectral feature selection. IEEE Trans Neural Netw Learn Syst 28(6):1263–1275

    Article  MathSciNet  Google Scholar 

  • Zhu Y, Zhu X, Kim M et al (2017c) A novel dynamic hyper-graph inference framework for computer assisted diagnosis of neuro-diseases. In: International conference on information processing in medical imaging. Springer, Cham, pp 158–169

  • Zhu X, Suk HI, Wang L et al (2017d) A novel relational regularization feature selection method for joint regression and classification in AD diagnosis. Med Image Anal 38:205–214

    Article  Google Scholar 

  • Zhu X, Zhang S, Hu R et al (2018) Local and global structure preservation for robust unsupervised spectral feature selection. IEEE Trans Knowl Data Eng 30(3):517–529

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported by the Natural Science Foundation of Shanxi Province, China (Grant No. 201801D121136) and the Nation Natural Science Foundation of China (Grants Nos. 61872260 and 61772358).

Author information

Authors and Affiliations

  1. Taiyuan University of Technology, No. 79 Yingze West Street, Taiyuan, 030024, Shanxi, China

    Xiaohong Han, Ping Liu, Dengao Li & Li Wang

  2. Taiyuan Coal Gasification (Group) Co., Ltd., No. 29 Peace South Road, Taiyuan, 030024, Shanxi, China

    Haishui Chai

Authors
  1. Xiaohong Han
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Haishui Chai
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ping Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Dengao Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Li Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xiaohong Han.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Han, X., Chai, H., Liu, P. et al. A new graph-preserving unsupervised feature selection embedding LLE with low-rank constraint and feature-level representation. Artif Intell Rev 53, 2875–2903 (2020). https://doi.org/10.1007/s10462-019-09749-w

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10462-019-09749-w

Keywords

Search

Navigation