Skip to main content
SpringerLink
Log in

Robust multi-label feature selection with shared label enhancement

  • Regular Paper
  • Published:
Knowledge and Information Systems Aims and scope Submit manuscript

Abstract

Feature selection has attracted considerable attention due to the wide application of multi-label learning. However, previous methods do not fully consider the relationship between feature sets and label sets but devote attention to either of them. Furthermore, conventional multi-label learning utilizes logical labels to estimate relevance between feature sets and label sets so that the importance of corresponding labels cannot be well reflected. Additionally, numerous irrelevant and redundant labels degrade the classification performance of models. To this end, we propose a multi-label feature selection method named Robust multi-label Feature Selection with shared Label Enhanced (RLEFS). First, we obtain a robust label enhancement term by reconstructing labels from logical labels to numerical labels and imposing \(l_{2,1}\)-norm onto the label enhancement term. Second, RLEFS utilizes the robust label enhancement term to share the similar latent semantic structure between feature matrix and label matrix. Third, local structure is considered to ensure the consistency of label information during the feature selection process. Finally, we integrate the above terms into one joint learning framework, and then, a simple but effective optimization method with provable convergence is proposed to solve RLEFS. Experimental results demonstrate the classification superiority of RLEFS in comparison with seven state-of-the-art methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. Li J, Cheng K, Wang S, Morstatter F, Trevino RP, Tang J, Liu H (2018) Feature selection: A data perspective. ACM Comput Surv 50(6):94. https://doi.org/10.1145/3136625

    Article  Google Scholar 

  2. Zhang P, Liu G, Gao W (2019) Distinguishing two types of labels for multi-label feature selection. Pattern Recogn 95:72–82. https://doi.org/10.1016/j.patcog.2019.06.004

    Article  Google Scholar 

  3. Huang J, Li G, Huang Q, Wu X (2016) Learning label-specific features and class-dependent labels for multi-label classification. IEEE Trans Knowl Data Eng 28(12):3309–3323. https://doi.org/10.1109/TKDE.2016.2608339

    Article  Google Scholar 

  4. Zhang J, Luo Z, Li C, Zhou C, Li S (2019) Manifold regularized discriminative feature selection for multi-label learning. Pattern Recogn 95:136–150. https://doi.org/10.1016/j.patcog.2019.06.003

    Article  Google Scholar 

  5. Zhu R, Dornaika F, Ruichek Y (2019) Learning a discriminant graph-based embedding with feature selection for image categorization. Neural Netw 111:35–46. https://doi.org/10.1016/j.neunet.2018.12.008

    Article  MATH  Google Scholar 

  6. Bolón-Canedo V, Sánchez-Maroño N, Alonso-Betanzos A (2016) Feature selection for high-dimensional data. Progr Artif Intell 5(2):65–75. https://doi.org/10.1007/s10287-008-0070-7

    Article  Google Scholar 

  7. Ditzler G, Polikar R, Rosen G (2017) A sequential learning approach for scaling up filter-based feature subset selection. IEEE Trans Neural Netw Learn Syst 29(6):2530–2544. https://doi.org/10.1109/TNNLS.2017.2697407

    Article  MathSciNet  Google Scholar 

  8. Wang A, An N, Chen G, Li L, Alterovitz G (2015) Accelerating wrapper-based feature selection with k-nearest-neighbor. Knowl-Based Syst 83:81–91. https://doi.org/10.1016/j.knosys.2015.03.009

    Article  Google Scholar 

  9. Hu L, Li Y, Gao W, Zhang P, Hu J (2020) Multi-label feature selection with shared common mode. Pattern Recogn 104:107344. https://doi.org/10.1016/j.patcog.2020.107344

    Article  Google Scholar 

  10. Yuan H, Li J, Lai LL, Tang YY (2019) Joint sparse matrix regression and nonnegative spectral analysis for two-dimensional unsupervised feature selection. Pattern Recogn 89:119–133. https://doi.org/10.1016/j.patcog.2019.01.014

    Article  Google Scholar 

  11. Zhang R, Nie F, Li X (2018) Self-weighted supervised discriminative feature selection. IEEE Trans Neural Netw Learn Syst 29(8):3913–3918. https://doi.org/10.1109/TNNLS.2017.2740341

    Article  Google Scholar 

  12. Huang J, Li G, Huang Q, Wu X (2018) Joint feature selection and classification for multilabel learning. IEEE Trans Cybern 48(3):876–889. https://doi.org/10.1109/TCYB.2017.2663838

    Article  Google Scholar 

  13. Jian L, Li J, Shu K, Liu H (2016) Multi-label informed feature selection. In: IJCAI, pp. 1627–1633 . https://doi.org/10.5555/3060832.3060848

  14. Cai Z, Zhu W (2018) Multi-label feature selection via feature manifold learning and sparsity regularization. Int J Mach Learn Cybern 9(8):1321–1334. https://doi.org/10.1007/s13042-017-0647-y

    Article  Google Scholar 

  15. Xu Y, Wang J, An S, Wei J, Ruan J (2018) Semi-supervised multi-label feature selection by preserving feature-label space consistency. In: Proceedings of the 27th ACM international conference on information and knowledge management, pp 783–792. https://doi.org/10.1145/3269206.3271760

  16. Saul LK, Roweis ST (2003) Think globally, fit locally: unsupervised learning of low dimensional manifolds. J Mach Learn Res 4(Jun):119–155. https://doi.org/10.1162/153244304322972667

    Article  MathSciNet  MATH  Google Scholar 

  17. Zhu X, Li X, Zhang S, Xu Z, Yu L, Wang C (2017) Graph pca hashing for similarity search. IEEE Trans Multimedia 19(9):2033–2044. https://doi.org/10.1109/TMM.2017.2703636

    Article  Google Scholar 

  18. Zhu Y, Kwok JT, Zhou Z-H (2017) Multi-label learning with global and local label correlation. IEEE Trans Knowl Data Eng 30(6):1081–1094. https://doi.org/10.1109/TKDE.2017.2785795

    Article  Google Scholar 

  19. Chen G, Song Y, Wang F, Zhang C (2008) Semi-supervised multi-label learning by solving a sylvester equation. In: Proceedings of the 2008 SIAM international conference on data mining, pp 410–419 . https://doi.org/10.1137/1.9781611972788.37

  20. Nie F, Huang H, Cai X, Ding CH (2010) Efficient and robust feature selection via joint l2, 1-norms minimization. In: Advances in neural information processing systems, pp 1813–1821. https://doi.org/10.5555/2997046.2997098

  21. Hou P, Geng X, Zhang M-L (2016) Multi-label manifold learning. In: Thirtieth AAAI conference on artificial intelligence, pp 1680–1686. https://doi.org/10.5555/3016100.3016134

  22. Shao R, Xu N, Geng X (2018) Multi-label learning with label enhancement. In: 2018 IEEE international conference on data mining (ICDM), pp 437–446. https://doi.org/10.1109/ICDM.2018.00059

  23. Zhang M-L, Zhou Z-H (2014) A review on multi-label learning algorithms. IEEE Trans Knowl Data Eng 26(8):1819–1837. https://doi.org/10.1109/TKDE.2013.39

    Article  Google Scholar 

  24. Boutell MR, Luo J, Shen X, Brown CM (2004) Learning multi-label scene classification. Pattern Recogn 37(9):1757–1771. https://doi.org/10.1016/j.patcog.2004.03.009

    Article  Google Scholar 

  25. Zhang J, Lin Y, Jiang M, Li S, Tang Y, Tan KC (2020) Multi-label feature selection via global relevance and redundancy optimization. In: Proceedings of 29th international joint conference artificial intelligence, pp 2512–2518. https://doi.org/10.24963/ijcai.2020/348

  26. Huang R, Jiang W, Sun G (2018) Manifold-based constraint laplacian score for multi-label feature selection. Pattern Recogn Lett 112:346–352. https://doi.org/10.1016/j.patrec.2018.08.021

    Article  Google Scholar 

  27. Read J (2008) A pruned problem transformation method for multi-label classification. In: Proceedings of the 2008 New Zealand computer science research student conference (NZCSRS 2008), vol. 143150, p 41

  28. Doquire G, Verleysen M (2011) Feature selection for multi-label classification problems. In: International work-conference on artificial neural networks, pp 9–16. https://doi.org/10.1007/978-3-642-21501-8_2

  29. Cai X, Nie F, Huang H (2013) Exact top-k feature selection via l2, 0-norm constraint. In: Twenty-third international joint conference on artificial intelligence, pp 1240–1246 . https://doi.org/10.5555/2540128.2540307

  30. Lin Y, Hu Q, Liu J, Duan J (2015) Multi-label feature selection based on max-dependency and min-redundancy. Neurocomputing 168:92–103. https://doi.org/10.1016/j.neucom.2015.06.010

    Article  Google Scholar 

  31. Braytee A, Liu W, Catchpoole DR, Kennedy PJ (2017) Multi-label feature selection using correlation information. In: Proceedings of the 2017 ACM on conference on information and knowledge management, pp 1649–1656 . https://doi.org/10.1145/3132847.3132858

  32. Gao W, Li Y, Hu L (2021) Multilabel feature selection with constrained latent structure shared term. IEEE Trans Neural Netw Learn Syst 1:1–10. https://doi.org/10.1109/TNNLS.2021.3105142

    Article  Google Scholar 

  33. Von Luxburg U (2007) A tutorial on spectral clustering. Stat Comput 17(4):395–416. https://doi.org/10.1007/s11222-007-9033-z

    Article  MathSciNet  Google Scholar 

  34. Hashemi A, Dowlatshahi MB, Nezamabadi-pour H (2021) An efficient pareto-based feature selection algorithm for multi-label classification. Inf Sci 581:428–447. https://doi.org/10.1016/j.ins.2021.09.052

    Article  MathSciNet  Google Scholar 

  35. Paniri M, Dowlatshahi MB, Nezamabadi-pour H (2021) Ant-td: Ant colony optimization plus temporal difference reinforcement learning for multi-label feature selection. Swarm Evol Comput 64:100892. https://doi.org/10.1016/j.swevo.2021.100892

    Article  Google Scholar 

  36. Hashemi A, Dowlatshahi MB, Nezamabadi-Pour H (2021) A bipartite matching-based feature selection for multi-label learning. Int J Mach Learn Cybern 12(2):459–475. https://doi.org/10.1016/j.knosys.2022.109085

    Article  Google Scholar 

  37. Hashemi A, Dowlatshahi MB, Nezamabadi-Pour H (2020) Mfs-mcdm: Multi-label feature selection using multi-criteria decision making. Knowl-Based Syst 206:106365. https://doi.org/10.1016/j.knosys.2020.106365

    Article  Google Scholar 

  38. Fan Y, Liu J, Liu P, Du Y, Lan W, Wu S (2021) Manifold learning with structured subspace for multi-label feature selection. Pattern Recogn 120:108169. https://doi.org/10.1016/j.patcog.2021.108169

    Article  Google Scholar 

  39. Fan Y, Liu J, Weng W, Chen B, Chen Y, Wu S (2021) Multi-label feature selection with constraint regression and adaptive spectral graph. Knowl-Based Syst 212:106621. https://doi.org/10.1016/j.knosys.2020.106621

    Article  Google Scholar 

  40. Liu J, Lin Y, Wu S, Wang C (2018) Online multi-label group feature selection. Knowl-Based Syst 143:42–57. https://doi.org/10.1016/j.knosys.2017.12.008

    Article  Google Scholar 

  41. Huang S-J, Zhou Z-H (2012) Multi-label learning by exploiting label correlations locally. In: Twenty-sixth AAAI conference on artificial intelligence. https://doi.org/10.5555/2900728.2900863

  42. Yan H, Yang J, Yang J (2016) Robust joint feature weights learning framework. IEEE Trans Knowl Data Eng 28(5):1327–1339. https://doi.org/10.1109/TKDE.2016.2515613

    Article  Google Scholar 

  43. Dempster AP, Laird NM, Rubin DB (1977) Maximum likelihood from incomplete data via the em algorithm. J Roy Stat Soc: Ser B (Methodol) 39(1):1–22. https://doi.org/10.1111/j.2517-6161.1977.tb01600.x

    Article  MathSciNet  MATH  Google Scholar 

  44. Cai D, He X, Han J, Huang TS (2010) Graph regularized nonnegative matrix factorization for data representation. IEEE Trans Pattern Anal Mach Intell 33(8):1548–1560. https://doi.org/10.1109/TPAMI.2010.231

    Article  Google Scholar 

  45. Tsoumakas G, Spyromitros-Xioufis E, Vilcek J, Vlahavas I (2011) Mulan: A java library for multi-label learning. J Mach Learn Res 12(7):2411–2414

    MathSciNet  MATH  Google Scholar 

  46. Huang K-H, Lin H-T (2017) Cost-sensitive label embedding for multi-label classification. Mach Learn 106(9–10):1725–1746. https://doi.org/10.1007/s10994-017-5659-z

    Article  MathSciNet  MATH  Google Scholar 

  47. Zhang Y, Miao D, Pedrycz W, Zhao T, Xu J, Yu Y (2020) Granular structure-based incremental updating for multi-label classification. Knowl-Based Syst 189:105066. https://doi.org/10.1016/j.knosys.2019.105066

    Article  Google Scholar 

  48. Jeong J-Y, Kang J-S, Jun C-H (2020) Regularization-based model tree for multi-output regression. Inf Sci 507:240–255. https://doi.org/10.1016/j.ins.2019.08.034

    Article  MathSciNet  MATH  Google Scholar 

  49. Klimt B, Yang Y (2004) The enron corpus: A new dataset for email classification research. In: European conference on machine learning, pp 217–226 . https://doi.org/10.1007/978-3-540-30115-8_22

  50. Zhang M-L, Zhou Z-H (2007) Ml-knn: A lazy learning approach to multi-label learning. Pattern Recogn 40(7):2038–2048. https://doi.org/10.1016/j.patcog.2006.12.019

    Article  MATH  Google Scholar 

  51. Yu K, Yu S, Tresp V (2005) Multi-label informed latent semantic indexing. In: Proceedings of the 28th Annual International ACM SIGIR conference on research and development in information retrieval, pp 258–265 . https://doi.org/10.1145/1076034.1076080

Download references

Acknowledgements

This work is funded by Postdoctoral Innovative Talents Support Program under Grant No. BX20190137, and National Key R &D Plan of China under Grant No. 2017YFA0604500, and by National Sci-Tech Support Plan of China under Grant No. 2014BAH02F00, and by National Natural Science Foundation of China under Grant No. 61701190, and by Youth Science Foundation of Jilin Province of China under Grant No. 20160520011JH & 20180520021JH, and by Youth Sci-Tech Innovation Leader and Team Project of Jilin Province of China under Grant No. 20170519017JH, and by Key Technology Innovation Cooperation Project of Government and University for the whole Industry Demonstration under Grant No. SXGJSF2017-4, and by Key scientific and technological R &D Plan of Jilin Province of China under Grant No. 20180201103GX, Project of Jilin Province Development and Reform Commission No. 2019FGWTZC001.

Author information

Author notes

  1. Juncheng Hu and Wanfu Gao have contributed equally to this work.

Authors and Affiliations

  1. College of Computer Science and Technology, Jilin University, Changchun, 130012, China

    Yonghao Li, Juncheng Hu & Wanfu Gao

  2. Key Laboratory of Symbolic Computation and Knowledge Engineering of Ministry of Education, Jilin University, Changchun, 130012, China

    Yonghao Li, Juncheng Hu & Wanfu Gao

Authors
  1. Yonghao Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Juncheng Hu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Wanfu Gao
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

YL mainly collected and processed data. He proposed algorithm framework. WG and JH conceived and designed the analysis, and then design contrast experiment. YL prepared the figures and experimental data. YL and JH wrote the main manuscript text. WG completed the audit and proofreading. JH is responsible for the delivery of final manuscript.

Corresponding author

Correspondence to Juncheng Hu.

Ethics declarations

Competing interests

The authors declare no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Li, Y., Hu, J. & Gao, W. Robust multi-label feature selection with shared label enhancement. Knowl Inf Syst 64, 3343–3372 (2022). https://doi.org/10.1007/s10115-022-01747-9

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10115-022-01747-9

Keywords

Search

Navigation