Skip to main content
SpringerLink
Log in

Weighted structure preservation and redundancy minimization for feature selection

  • Methodologies and Application
  • Published:
Soft Computing Aims and scope Submit manuscript

Abstract

The recent literature indicates that structure preserving is of great importance for feature selection and many existing selection criteria essentially work in this way. In this paper, we argue that the Eigen value decomposition of global pair wise similarity matrix should be weighted, and the redundancy among the features should be minimized. In order to show this, we propose a weighted structure preservation and features redundancy minimization framework for feature selection. In this framework, the Eigen vector obtained by the Eigen decomposition of global pair wise similarity matrix is weighted by the corresponding Eigen value, and the cosine distance between two features together with the L2,1 norm of these two features are used to evaluate the degree of redundancy between these two features. A comprehensive experimental study is then conducted in order to compare our feature selection algorithms with many state-of-the art ones in supervised learning scenarios. The conducted experiments validate the effectiveness of our feature selection.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  • Benabdeslem K, Hindawi M (2014) Efficient semi-supervised feature selection: constraint, relevance and redundancy. IEEE Trans Knowl Data Eng 26(5):1131–1143

    Article  Google Scholar 

  • Burkhardt F, Paeschke A, Rolfes M, Sendlmeier WF, Weiss B (2005) A database of German emotional speech. In: Proceedings of INTERSPEECH, Lisbon, pp 1517–1520

  • Cai D, Zhang C, He X (2010) Unsupervised feature selection for multi-cluster data. In: Proceedings of the 16th ACM SIGKDD international conference on Knowledge discovery and data mining, pp 333–342

  • Chen M, Tsang IW, Tan M, Cham TJ (2015) A unified feature selection framework for graph embedding on high dimensional data. IEEE Trans Knowl Data Eng 27(6):1465–477

    Article  Google Scholar 

  • Chen J, Jiao L, Wen Z (2016) High-level feature selection with dictionary learning for unsupervised SAR imagery terrain classification. IEEE J Sel Top Appl Earth Obs Remote Sens pp(99):1–16

    Google Scholar 

  • Du X, Yan Y, Pan P, Long G, Zhao L (2016) Multiple graph unsupervised feature selection. Signal Processing 120:754–760

    Article  Google Scholar 

  • Eyben F, Wöllmer M, Schuller B (2010) OpenSMILE —the munich versatile and fast open-source audio feature extractor. In: Proceedings of the ACM multimedia (MM), Florence, Italy, pp 1459–1462

  • Fang X, Xu Y, Li X, Fan Z, Liu H, Chen Y (2014) Locality and similarity preserving embedding for feature selection. IEEE Trans Cybern 128:304–315

    Google Scholar 

  • Han J, Sun Z, Hao H (2016) \(l_{0}\)-norm based structural sparse least square regression for feature selection. Pattern Recognition 48(12):3927–3940

    Article  Google Scholar 

  • Haq S, Jackson PJB (2009) Speaker-dependent audio–visual emotion recognition. In: Proceedings of AVSP, pp 53–58

  • He X, Niyogi P (2003) Locality preserving projections. In: Proceedings of the advances in neural information processing systems (NIPS), Vancouver, pp 585–591

  • Hou C, Nie F, Li X, Yi D, Wu Y (2014) Joint embedding learning and sparse regression: a framework for unsupervised feature selection. IEEE Trans Cybern 44(6):793–804

    Article  Google Scholar 

  • Lee K-C, Ho J, Kriegman D (2005) Acquiring linear subspaces for face recognition under variable lighting. IEEE Trans Pattern Anal Mach Intell 27(5):684–698

    Article  Google Scholar 

  • Li Z, Tang J (2015) Unsupervised feature selection via nonnegative spectral analysis and redundancy control. IEEE Trans Image Process 24(12):5343–5355

    Article  MathSciNet  Google Scholar 

  • Liang Y, Liao S, Wang L, Zou B (2011) Exploring regularized feature selection for person specific face verification. In: Proceedings of the international conference on computer vision, pp 1676–1683

  • Liao Y, Vemuri VR (2002) Use of K-nearest neighbor classifier for intrusion detection. Comput Secur 21(5):439–448

    Article  Google Scholar 

  • Liu X, Wang L, Zhang J, Yin J, Liu H (2015) Global and local structure preservation for feature selection. IEEE Trans Cybern 25(6):1083–1095

    Google Scholar 

  • Martinez AM, Kak AC (2001) PCA versus LDA. IEEE Trans Pattern Anal Mach Intell 23(2):228–233

    Article  Google Scholar 

  • Mohsenzadeh Y, Sheikhzadeh H, Reza AM, Bathaee N, Kalayeh MM (2013) The relevance sample-feature machine: a sparse Bayesian learning approach to joint feature-sample selection. IEEE Trans Cybern 43(6):2241–2254

    Article  Google Scholar 

  • Nene SA, Nayar SK, Murase H (Feb 1996) Columbia object image library (COIL-20), Technical Report CUCS-005-96

  • Nie F, Huang H, Cai X, Ding C (2010) Efficient and robust feature selection via joint \(L_{2,1}\)-norms minimization. In: Proceedings of the neural information processing systems

  • Nie F, Huang H, Cai X, Ding C (2011) Structured sparse model based feature selection and classification for hyperspectral imagery. In: Proceedings of the IEEE international geoscience and remote sensing symposium, pp 1771–1774

  • ORLface database. http://www.cl.cam.ac.uk/research/dtg/attarchive/facedatabase.html

  • Peng H, Long F, Ding C (2005) Feature selection based on mutual information: criteria of max-dependency, max-relevance, and min-redundancy. IEEE Trans Pattern Anal Mach Intell 27(8):1226–1238

    Article  Google Scholar 

  • Roweis S, Saul L (2000) Nonlinear dimensionality reduction by locally linear embedding. Science 290(5500):2323–2326

    Article  Google Scholar 

  • Shi C, Ruan Q, An G (2014) Sparse feature selection based on graph Laplacian for web image annotation. Image Vis Comput 32(3):189–201

    Article  Google Scholar 

  • Shi C, Ruan Q, An G, Ge C (2015) Semi-supervised sparse feature selection based on multi-view Laplacian regularization. Image Vis Comput 41:1–10

    Article  Google Scholar 

  • Sim T, Barker S, Bsat M (2013) The CMU pose, illumination, and expression database. IEEE Trans Pattern Anal Mach Intell 25(12):1615–1618

    Google Scholar 

  • Sun Y, Wen G (2015) Adaptive feature transformation for classification with sparse representation. Int J Light Electron Opt 126(23):4452–4459

    Article  Google Scholar 

  • Sun Y, Todorovic S, Goodison S (2010) Local-learning-based feature selection for high-dimensional data analysis. IEEE Trans Pattern Anal Mach Intell 32(9):1610–1626

    Article  Google Scholar 

  • The selected speech emotion database of Institute of Automation Chinese Academy of Sciences (CASIA). http://www.datatang.com/data/39277

  • The web page of Cai. http://www.cad.zju.edu.cn/home/dengcai/Data/

  • Tsimpiris A, Vlachos I, Kugiumtzis D (2014) Nearest neighbor estimate of conditional mutual information in feature selection. Expert Syst Appl 39:12697–12708

    Article  Google Scholar 

  • UCI datasets. http://archive.ics.uci.edu/ml/

  • USPS database. http://www-i6.informatik.rwth-aachen.de/~keysers/usps.html

  • Wang C, Cao L, Miao B (2013) Optimal feature selection for sparse linear discriminant analysis and its applications in gene expression data. Comput Stat Data Anal 66:140–149

    Article  MathSciNet  Google Scholar 

  • Wang JJ-Y, Bensmail H, Gao X (2014) Feature selection and multi-kernel learning for sparse representation on a manifold. Neural Netw 51:9–16

    Article  Google Scholar 

  • Wang D, Nie F, Huang H (2015) Feature selection via global redundancy minimization. IEEE Trans Knowl Data Eng 27(10):2743–2755

    Article  Google Scholar 

  • Wanga J, Wua L, Kong J, Li Y, Zhang B (2013) Maximum weight and minimum redundancy: a novel framework for feature subset selection. Pattern Recognit 181:1616–1623

    Article  Google Scholar 

  • Wu Y, Wang C, Bu J, Chen C (2016) Group sparse feature selection on local learning based clustering. Neurocomputing 171:1118–1130

    Article  Google Scholar 

  • Xiang S, Nie F, Meng G, Pan C, Zhang C (2012) Discriminative least squares regression for multiclass classification and feature selection. IEEE Trans Neural Netw Learn Syst 23(11):1738–1754

    Article  Google Scholar 

  • Xie Z, Xu Y (2014) Sparse group LASSO based uncertain feature selection. Int J Mach Learn Cybern 5(2):201–210

    Article  Google Scholar 

  • Yale face database. http://vision.ucsd.edu/content/yale-face-database

  • Zhang Q, Tian Y, Yang Y, Pan C (2015) Automatic spatial–spectral feature selection for hyperspectral image via discriminative sparse multimodal learning. IEEE Trans Geosci Remote Sens 53(1):261–279

    Article  Google Scholar 

  • Zhang J, Yu J, Wan J, Zeng Z (2015) l2,1 norm regularized fisher criterion for optimal feature selection. Neurocomputing 166:455–463

    Article  Google Scholar 

  • Zhang Q, Tian Y, Yang Y, Pan C (2015) Automatic spatial–spectral feature selection for hyperspectral image via discriminative sparse multimodal learning. IEEE Trans Cybern 128:261–279

    Google Scholar 

  • Zhao Z, Wang L, Liu H, Ye J (2012) Web image annotation via subspace-sparsity collaborated feature selection. IEEE Trans Multimed 14(4):1021–1031

    Article  Google Scholar 

  • Zhao Z, Wang L, Liu H, Ye J (2013) On similarity preserving feature selection. IEEE Trans Knowl Data Eng 25(3):619–632

    Article  Google Scholar 

  • Zhao Z, He X, Cai D, Zhang L, Ng W, Zhuang Y (2016) Graph regularized feature selection with data reconstruction. IEEE Trans Knowl Data Eng 28(3):689–700

    Article  Google Scholar 

  • Zhou N, Yangyang X, Cheng H, Fang J, Pedrycz W (2016) Global and local structure preserving sparse subspace learning: an iterative approach to unsupervised feature selection. Pattern Recognit 53:87–101

    Article  Google Scholar 

  • Zhu Y, Zhong Z, Cao W, Cheng D (2016) Graph feature selection for dementia diagnosis. Neurocomputing 195:19–22

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported by China National Science Foundation under Grants 61273363, 61003174, State Key Laboratory of Brain and Cognitive Science under Grants 08B12. Jiaxing National Science Foundation under Grants 2016AY13013.

Author information

Authors and Affiliations

  1. College of Mathematics Physics and Information Engineering, Jiaxing University, Jiaxing, 314001, China

    Qing Ye & Yaxin Sun

Authors
  1. Qing Ye
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yaxin Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Qing Ye.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Communicated by V. Loia.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ye, Q., Sun, Y. Weighted structure preservation and redundancy minimization for feature selection. Soft Comput 22, 7255–7268 (2018). https://doi.org/10.1007/s00500-017-2727-z

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00500-017-2727-z

Keywords

Search

Navigation