Advertisement

Multimedia Tools and Applications

, Volume 77, Issue 17, pp 22099–22113 | Cite as

Fast unsupervised feature selection with anchor graph and 2,1-norm regularization

  • Haojie Hu
  • Rong Wang
  • Feiping Nie
  • Xiaojun Yang
  • Weizhong Yu
Article
  • 129 Downloads

Abstract

Graph-based unsupervised feature selection has been proven to be effective in dealing with unlabeled and high-dimensional data. However, most existing methods face a number of challenges primarily due to their high computational complexity. In light of the ever-increasing size of data, these approaches tend to be inefficient in dealing with large-scale data sets. We propose a novel approach, called Fast Unsupervised Feature Selection (FUFS), to efficiently tackle this problem. Firstly, an anchor graph is constructed by means of a parameter-free adaptive neighbor assignment strategy. Meanwhile, an approximate nearest neighbor search technique is introduced to speed up the anchor graph construction. The 2,1-norm regularization is then performed to select more valuable features. Experiments on several large-scale data sets demonstrate the effectiveness and efficiency of the proposed method.

Keywords

Unsupervised feature selection Anchor graph 2,1-norm 

Notes

Acknowledgements

This paper is supported in part by the National Natural Science Foundation of China under Grant 61401471, Grant 61772427 and Grant 61751202 and in part by the China Postdoctoral Science Foundation under Grant 2014M562636.

References

  1. 1.
    Cai D, Zhang C, He X (2010) Unsupervised feature selection for multi-cluster data. In: Proceedings of the conference on knowledge discovery and data mining, pp 333–342Google Scholar
  2. 2.
    Cheng Q, Zhou H, Cheng J (2011) The fisher-markov selector: fast selecting maximally separable feature subset for multiclass classification with applications to high-dimensional data. IEEE Trans Pattern Anal Mach Intell 33(6):1217–33CrossRefGoogle Scholar
  3. 3.
    Deng C, Ji R, Liu W, Tao D, Gao X (2013) Visual reranking through weakly supervised multi-graph learning. In: Proceedings of the 2013 IEEE international conference on computer vision, pp 2600–2607Google Scholar
  4. 4.
    Deng C, Ji R, Tao D, Gao X, Li X (2014) Weakly supervised multi-graph learning for robust image reranking. IEEE Trans Multimed 16(3):785–795CrossRefGoogle Scholar
  5. 5.
    Dy JG, Brodley CE (2004) Feature selection for unsupervised learning. J Mach Learn Res 5(4):845–889MathSciNetMATHGoogle Scholar
  6. 6.
    Freeman C, Kulic D, Basir O (2013) Feature-selected tree-based classification. IEEE Trans Cybern 43(6):1990–2004CrossRefGoogle Scholar
  7. 7.
    Guyon I, Elisseeff A (2003) An introduction to variable and feature selection. J Mach Learn Res 3:1157–1182MATHGoogle Scholar
  8. 8.
    Hancock T, Mamitsuka H (2012) Boosted network classifiers for local feature selection. IEEE Trans Neural Netw Learn Syst 23(11):1767–1778CrossRefGoogle Scholar
  9. 9.
    He X, Cai D, Niyogi P (2005) Laplacian score for feature selection. In: Proceedings of the conference on neural information processing systems, pp 507–514Google Scholar
  10. 10.
    Hou C, Nie F, Li X, Yi D, Wu Y (2014) Joint embedding learning and sparse regression: a framework for unsupervised feature selection. IEEE Trans Cybern 44(6):793CrossRefGoogle Scholar
  11. 11.
    Hou C, Nie F, Tao H, Yi D (2017) Multi-view unsupervised feature selection with adaptive similarity and view weight. IEEE Trans Knowl Data Eng 29(9):1998–2011CrossRefGoogle Scholar
  12. 12.
    Kokiopoulou E, Saad Y (2007) Orthogonal neighborhood preserving projections: a projection-based dimensionality reduction technique. IEEE Trans Pattern Anal Mach Intell 29(12):2143–2156CrossRefGoogle Scholar
  13. 13.
    Lai HJ, Pan Y, Tang Y, Yu R (2013) Fsmrank: feature selection algorithm for learning to rank. IEEE Trans Neural Netw Learn Syst 24(6):940–952CrossRefGoogle Scholar
  14. 14.
    Laporte L, Flamary R, Canu S, Djean S, Mothe J (2014) Nonconvex regularizations for feature selection in ranking with sparse svm. IEEE Trans Neural Netw Learn Syst 25(6):1118–1130CrossRefGoogle Scholar
  15. 15.
    Li Z, Yang Y, Liu J, Zhou X, Lu H (2012) Unsupervised feature selection using nonnegative spectral analysis. In: Proceedings of the AAAI conference on artificial intelligence, pp 1026–1032Google Scholar
  16. 16.
    Li Y, Si J, Zhou G, Huang S, Chen S (2015) Frel: a stable feature selection algorithm. IEEE Trans Neural Netw Learn Syst 26(7):1388MathSciNetCrossRefGoogle Scholar
  17. 17.
    Ling X, Qiang MA, Min Z (2013) Tensor semantic model for an audio classification system. Sci Chin 56(6):1–9MathSciNetGoogle Scholar
  18. 18.
    Liu W, He J, Chang SF (2010) Large graph construction for scalable semi-supervised learning. In: Proceedings of the international conference on machine learning, pp 679–686Google Scholar
  19. 19.
    Liu W, Wang J, Chang SF (2012) Robust and scalable graph-based semisupervised learning. Proc IEEE 100(9):2624–2638CrossRefGoogle Scholar
  20. 20.
    Luo M, Nie F, Chang X, Yi Y, Hauptmann AG, Zheng Q (2017) Adaptive unsupervised feature selection with structure regularization. IEEE Trans Neural Netw Learn Syst PP(99):1–13Google Scholar
  21. 21.
    Muja M, Lowe DG (2014) Scalable nearest neighbor algorithms for high dimensional data. IEEE Trans Pattern Anal Mach Intell 36(11):2227–2240CrossRefGoogle Scholar
  22. 22.
    Nie F, Huang H, Cai X, Ding CH (2010) Efficient and robust feature selection via joint 2,1-norms minimization. In: Proceedings of the conference on advances in neural information processing systems, pp 1813–1821Google Scholar
  23. 23.
    Nie F, Xiang S, Jia Y, Zhang C, Yan S (2008) Trace ratio criterion for feature selection. In: Proceedings of the AAAI conference on artificial intelligence, pp 671–676Google Scholar
  24. 24.
    Nie F, Wang X, Huang H (2014) Clustering and projected clustering with adaptive neighbors. In: Proceedings of the conference on knowledge discovery and data mining, pp 977–986Google Scholar
  25. 25.
    Nie F, Wang X, Jordan M, Huang H (2016) The constrained laplacian rank algorithm for graph-based clustering. In: Proceedings of the AAAI conference on artificial intelligence, pp 1969–1976Google Scholar
  26. 26.
    Nie F, Zhu W, Li X (2016) Unsupervised feature selection with structured graph optimization. In: Proceedings of the AAAI conference on artificial intelligence, pp 1302–1308Google Scholar
  27. 27.
    Peng Y, Lu BL (2017) Discriminative extreme learning machine with supervised sparsity preserving for image classification. NeurocomputingGoogle Scholar
  28. 28.
    Qian M, Zhai C (2013) Robust unsupervised feature selection. In: Proceedings of the international joint conference on artificial intelligence, pp 1621–1627Google Scholar
  29. 29.
    Romero E, Sopena JM (2008) Performing feature selection with multilayer perceptrons. IEEE Trans Neural Netw 19(3):431–41CrossRefGoogle Scholar
  30. 30.
    Strehl A, Ghosh J (2002) Cluster ensembles: a knowledge reuse framework for combining partitionings. In: Eighteenth national conference on artificial intelligence, pp 93–98Google Scholar
  31. 31.
    Wang R, Nie F, Yang X, Gao F, Yao M (2015) Robust 2DPCA with non-greedy 1-norm maximization for image analysis. IEEE Trans Cybern 45(5):1108–1112CrossRefGoogle Scholar
  32. 32.
    Wang R, Nie F, Hong R, Chang X, Yang X, Yu W (2017) Fast and orthogonal locality preserving projections for dimensionality reduction. IEEE Trans Image Process 26(10):5019–5030MathSciNetCrossRefGoogle Scholar
  33. 33.
    Xiang S, Nie F, Zhang C, Zhang C (2009) Nonlinear dimensionality reduction with local spline embedding. IEEE Trans Knowl Data Eng 21(9):1285–1298CrossRefGoogle Scholar
  34. 34.
    Xing L, Dong H, Jiang W, Tang K (2017) Nonnegative matrix factorization by joint locality-constrained and 2,1-norm regularization. Multimed Tools Appl.  https://doi.org/10.1007/s11042-017-4970-9
  35. 35.
    Yang Y, Shen HT, Ma Z, Huang Z, Zhou X (2011) 2,1-norm regularized discriminative feature selection for unsupervised learning., In: Proceedings of the international joint conference on artificial intelligence, pp 1589–1594Google Scholar
  36. 36.
    Yu Q, Wang R, Yang X, Li BN, Yao M (2016) Diagonal principal component analysis with non-greedy 1-norm maximization for face recognition. Neurocomputing 171:57–62CrossRefGoogle Scholar
  37. 37.
    Zhao Z, Liu H (2007) Spectral feature selection for supervised and unsupervised learning. In: Proceedings of the international conference on machine learning, pp 1151–1157Google Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  1. 1.The Xi’an Research Institute of Hi-TechXi’anChina
  2. 2.The Center for OPTical IMagery Analysis and Learning (OPTIMAL)Northwestern Polytechnical UniversityXi’anChina
  3. 3.The School of Information EngineeringGuangdong University of TechnologyGuangzhouChina
  4. 4.The School of Electronic and Information EngineeringXi’an Jiaotong UniversityXi’anChina

Personalised recommendations