Skip to main content
SpringerLink
Log in

Robust Feature Selection with Feature Correlation via Sparse Multi-Label Learning

  • MATHEMATICAL THEORY OF PATTERN RECOGNITION
  • Published:
Pattern Recognition and Image Analysis Aims and scope Submit manuscript

Abstract

The multi-label feature selection that is regarded as a special case of multi-task learning has received much attention in recent years. In this paper, we propose a novel robust and pragmatic multi-label feature selection method, in which the joint l2,1-norm minimizations of loss function and regularization are emphasized. Specifically, the loss function based on the l2,1-norm is robust to outliers, and the l2,1-norm regularization selects features across all samples with joint sparsity. Besides, the feature information inherent in the data is used to construct the correlation matrix, which explores the correlation between features so as to remove the redundant features. An efficient algorithm based on the augmented Lagrangian multiplier method is proposed to solve the objective function. The extensive experiments compared with several state-of-the-art methods are performed on the multi-label datasets to show the effectiveness of the proposed method.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1.
Fig. 2.
Fig. 3.
Fig. 4.
Fig. 5.

Similar content being viewed by others

REFERENCES

  1. C. Ding, M. Zhao, J. Lin, and J. Jiao, “Multi-objective iterative optimization algorithm based optimal wavelet filter selection for multi-fault diagnosis of rolling element bearings,” ISA Trans. 88, 199–215 (2019).

    Article  Google Scholar 

  2. M. Labani, P. Moradi, F. Ahmadizar, and M. Jalili, “A novel multivariate filter method for feature selection in text classification problems,” Eng. Appl. Artif. Intell. 70, 25–37 (2018).

    Article  Google Scholar 

  3. C. Yao, Y.-F. Liu, B. Jiang, J. Han, and J. Han, “LLE SCORE: A new filter-based unsupervised feature selection method based on nonlinear manifold embedding and its application to image recognition,” IEEE Trans. Image Process. 26 (11), 5257–5269 (2017).

    Article  MathSciNet  Google Scholar 

  4. J. González, J. Ortega, M. Damas, et al., “A new multi-objective wrapper method for feature selection— Accuracy and stability analysis for BCI,” Neurocomput. 333, 407–418 (2019).

    Article  Google Scholar 

  5. S. Jadhav, H. He, and K. Jenkins, “Information gain directed genetic algorithm wrapper feature selection for credit rating,” Appl. Soft Comput. 69, 541–553 (2018).

    Article  Google Scholar 

  6. S. Maldonado and J. López, “Dealing with high-dimensional class-imbalanced datasets: Embedded feature selection for SVM classification,” Appl. Soft Comput. 67, 94–105 (2018).

    Article  Google Scholar 

  7. Y. Kong and T. Yu, “A graph-embedded deep feedforward network for disease outcome classification and feature selection using gene expression data,” Bioinf. 34 (21), 3727–3737 (2018).

    Article  Google Scholar 

  8. K. A. Taher, B. M. Yasin Jisan, and M. Rahman, “Network intrusion detection using supervised machine learning technique with feature selection,” in Proc. 1st Int. Conf. on Robotics, Electrical, and Signal Processing Techniques (ICREST) (Dhaka, Bangladesh, 2019), IEEE, pp. 643–646.

  9. H. Wang, F. Nie, H. Huang, S. L. Risacher, A. J. Saykin, et al., “Identifying disease sensitive and quantitative trait-relevant biomarkers from multidimensional heterogeneous imaging genetics data via sparse multimodal multitask learning,” Bioinf. 28 (12), i127– i136 (2012).

    Article  Google Scholar 

  10. Y. Li, X. Shi, C. Du, Y. Liu, and Y. Wen, “Manifold regularized multi-view feature selection for social image annotation,” Neurocomput. 204, 135–141 (2016).

    Article  Google Scholar 

  11. C. Shi, C. Duan, Z. Gu, et al., “Semi-supervised feature selection analysis with structured multi-view sparse regularization,” Neurocomput. 330, 412–424 (2019).

    Article  Google Scholar 

  12. Y.-M. Xu, C.-D. Wang, and J.-H. Lai, “Weighted multi-view clustering with feature selection,” Pattern Recognit. 53, 25–35 (2016).

    Article  Google Scholar 

  13. S. Wang and H. Wang, “Unsupervised feature selection via low-rank approximation and structure learning,” Knowl.-Based Syst. 124, 70–79 (2017).

    Article  Google Scholar 

  14. G. Obozinski, B. Taskar, and M. I. Jordan, “Joint covariate selection and joint subspace selection for multiple classification problems,” Stat. Comput. 20 (2), 231–252 (2010).

    Article  MathSciNet  Google Scholar 

  15. J. Liu, S. Ji, and J. Ye, “Multi-task feature learning via efficient l2,1-norm minimization,” in Proc. 25th Conference on Uncertainty in Artificial Intelligence (UAI’09) (Montreal, Canada, 2009) (AUAI Press, Arlington, VA, 2009), pp. 339–348.

  16. F. Nie, H. Huang, X. Cai, and C. H. Ding, “Efficient and robust feature selection via joint l2,1-norms minimization,” in Advances in Neural Information Processing Systems 23: Proc. 24th Annual Conf. NIPS 2010 (Vancouver, Canada, 2006) (Curran Associates, Red Hook, NY, 2010), pp. 1813–1821.

  17. H. Wang, F. Nie, H. Huang, S. Risacher, C. Ding, et al., “Sparse multi-task regression and feature selection to identify brain imaging predictors for memory performance,” in Proc. 2011 IEEE Int. Conf. on Computer Vision (ICCV 2011) (Barcelona, Spain, 2011), IEEE, pp. 557–562.

  18. J. Tang and H. Liu, “Feature selection with linked data in social media,” in Proc. 12th SIAM Int. Conf. on Data Mining (SDM 2012) (Anaheim, CA, 2012), SIAM, pp. 118–128.

  19. R. Zhang, F. Nie, and X. Li, “Self-weighted supervised discriminative feature selection,” IEEE Trans. Neural Networks Learn. Syst. 29 (8), 3913–3918 (2018).

    Article  Google Scholar 

  20. W. Gao, L. Hu, P. Zhang, and F. Wang, “Feature selection by integrating two groups of feature evaluation criteria,” Expert Syst. Appl. 110, 11–19 (2018).

    Article  Google Scholar 

  21. Z. Zhou and M. Zhang, “Multiinstance multi-label learning with application to scene classification,” in Advances in Neural Information Processing Systems 19: Proc. 20th Annual Conf. NIPS 2006 (Vancouver, Canada, 2006) (MIT Press, Cambridge, 2007), pp. 1609–1616.

  22. G. Qi, X.-S. Hua, Y. Rui, J. Tang, T. Mei, and H.‑J. Zhang, “Correlative multi-label video annotation,” in Proc. 15th ACM Int. Conf. on Multimedia (MM’07) (Augsburg, Germany, 2007) (ACM, New York, 2007), pp. 17–26.

  23. R. E. Schapire and Y. Singer, “BoosTexter: A boosting-based system for text categorization,” Mach. Learn. 39 (2–3), 135–168 (2000).

    Article  Google Scholar 

  24. A. Elisseeff and J. Weston, “A kernel method for multi-labelled classification,” in Advances in Neural Information Processing Systems 14: Proc. 2001 NIPS Conf. (Vancouver, Canada, 2006) (MIT Press, Cambridge, 2002), Vol. 1, pp. 681–687.

  25. X. Chang, F. Nie, Y. Yang, and H. Huang, “A convex formulation for semi-supervised multi-label feature selection,” in Proc. 28th AAAI Conference on Artificial Intelligence (AAAI-14) (Québec City, Québec, Canada, 2014) (AAAI Press, Palo Alto, CA, 2014), pp. 1171–1177.

  26. L. Huang, J. Tang, S. Chen, C. Ding, and B. Luo, “An efficient algorithm for feature selection with feature correlation,” in Intelligent Science and Intelligent Data Engineering, Third Sino-Foreign-Interchange Workshop, IScIDE 2012, Revised Selected Papers, Ed. By J. Yang, F. Fang, and C. Sun, Lecture Notes in Computer Science (Springer, Berlin, Heidelberg, 2013), Vol. 7751, pp. 639–646.

  27. Y. Guo and W. Xue, “Probabilistic multi-label classification with sparse feature learning,” in Proc. 23rd Int. Joint Conf. on Artificial Intelligence (IJCAI-13) (Beijing, China, 2013) (AAAI Press, Palo Alto, CA, 2013), Vol. 2, pp. 1373–1379.

  28. S. Ji, L. Tang, S. Yu, and J. Ye, “A shared-subspace learning framework for multi-label classification,” ACM Trans. Knowl. Discovery Data 4 (2), Article No. 8, 1–29 (2010).

    Google Scholar 

  29. Z. Ma, F. Nie, Y. Yang, et al., “Web image annotation via subspace-sparsity collaborated feature selection,” IEEE Trans. Multimedia 14 (4), 1021–1030 (2012).

    Article  Google Scholar 

  30. X. Zhu, X. Li, and S. Zhang, “Block-row sparse multiview multilabel learning for image classification,” IEEE Trans. Cybern. 46 (2), 450–461 (2016).

    Article  Google Scholar 

Download references

Funding

The work is supported by the College Student Research and Career-creation Program of Beijing City (no. 2018bj170).

Author information

Authors and Affiliations

  1. College of Science, China Agricultural University, 100083, Beijing, China

    Jiangjiang Cheng, Junmei Mei, Min Men & Ping Zhong

  2. College of Information and Electrical Engineering, China Agricultural University, 100083, Beijing, China

    Jing Zhong

Authors
  1. Jiangjiang Cheng
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Junmei Mei
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jing Zhong
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Min Men
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Ping Zhong
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ping Zhong.

Ethics declarations

The authors declare that they have no conflicts of interest.

Additional information

Jiangjiang Cheng was born in China, in 1998. He is majoring in Mathematics and Applied Mathematics in China Agricultural University, Beijing, China.

Jing Zhong was born in China, in 1996. She has received the B.S. degree from Anhui Jianzhu University in 2017. Now she is studying for a master’s degree in China Agricultural University. Her research interests include machine learning and data mining.

Ping Zhong is a professor and PhD supervisor in College of Science, China Agricultural University. She has published many papers. Her research interests include machine learning and support vector machines.

Min Men was born in China, in 1997. She has received the B.S. degree from China Agricultural University in 2018. Now she is studying for a master’s degree in China Agricultural University. Her research interests include machine learning and data mining.

Junmei Mei was born in China, in 1998. She is majoring in Mathematics and Applied Mathematics in China Agricultural University, Beijing, China.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Cheng, J., Mei, J., Zhong, J. et al. Robust Feature Selection with Feature Correlation via Sparse Multi-Label Learning. Pattern Recognit. Image Anal. 30, 52–62 (2020). https://doi.org/10.1134/S1054661820010034

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1134/S1054661820010034

Keywords:

Search

Navigation