Skip to main content
SpringerLink
Log in

Ensemble Feature Selection for Multi-label Classification: A Rank Aggregation Method

  • Conference paper
  • First Online:
International Conference on Cyber Security, Privacy and Networking (ICSPN 2022) (ICSPN 2021)

Part of the book series: Lecture Notes in Networks and Systems ((LNNS,volume 599))

Included in the following conference series:

  • 347 Accesses

Abstract

Multi-label classification is an important task in machine learning applications. However, today, these methods have more challenges due to high-dimensional data. Existing methods in the literature have not yet reached an acceptable performance, and as the number of labels in the dataset increases, their performance becomes weaker. Thus providing effective feature selection methods is necessary and is suitable for all data. Ensemble feature selection is also a new approach based on combining the results of several feature selection methods. Ensemble approaches have been used in many single-label applications and have provided adequate results. However, this technique is less considered in multi-label feature selection. For this purpose, in this article, we have presented a method based on an ensemble of feature selection methods. This paper presents an ensemble multi-label feature method using rank aggregation algorithms. Three filter rankers with different structures are utilized to obtain the final ranking. This rank aggregation process is treated like an election where the rankers are assumed to be the voters, and the features are the alternatives. The Weighted Borda Count (WBC) method conducts the election process. Finally, the performance and results obtained by the proposed method are compared with the nine filter-based algorithms used based on six multi-label datasets. Most of the selected datasets have a high number of class labels, and the proposed method has recorded the best classification performance in all these datasets. Also, the results show that this method is not well affected by the adverse effects of the algorithms used.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 189.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 249.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Bayati, H., Dowlatshahi, M.B., Hashemi, A.: MSSL: a memetic-based sparse subspace learning algorithm for multi-label classification. Int. J. Mach. Learn. Cybern. (2022). https://doi.org/10.1007/s13042-022-01616-5

    Article  Google Scholar 

  2. Dalvand, A., Dowlatshahi, M.B., Hashemi, A.: SGFS: a semi-supervised graph-based feature selection algorithm based on the PageRank algorithm. In: 2022 27th International Computer Conference, Computer Society of Iran (CSICC), pp. 1–6. IEEE (2022)

    Google Scholar 

  3. Hancer, E., Xue, B., Zhang, M.: A survey on feature selection approaches for clustering. Artif. Intell. Rev. 53(6), 4519–4545 (2020). https://doi.org/10.1007/s10462-019-09800-w

    Article  Google Scholar 

  4. Hashemi, A., Dowlatshahi, M.B.: An ensemble of feature selection algorithms using OWA operator. In: 2022 9th Iranian Joint Congress on Fuzzy and Intelligent Systems (CFIS), pp 1–6. IEEE (2022)

    Google Scholar 

  5. Hashemi, A., Dowlatshahi, M.B., Nezamabadi-pour, H.: MGFS: a multi-label graph-based feature selection algorithm via PageRank centrality. Expert Syst. Appl. 142, 113024 (2020). https://doi.org/10.1016/j.eswa.2019.113024

    Article  Google Scholar 

  6. Hashemi, A., Dowlatshahi, M.B., Nezamabadi-pour, H.: MFS-MCDM: multi-label feature selection using multi-criteria decision making. Knowl. Based Syst. 206, 106365 (2020). https://doi.org/10.1016/j.knosys.2020.106365

    Article  Google Scholar 

  7. Hashemi, A., Dowlatshahi, M.B., Nezamabadi-pour, H.: An efficient Pareto-based feature selection algorithm for multi-label classification. Inf. Sci. (NY) 51, 428–447 (2021). https://doi.org/10.1016/j.ins.2021.09.052

    Article  MathSciNet  Google Scholar 

  8. Hashemi, A., Dowlatshahi, M.B., Nezamabadi-Pour, H.: Minimum redundancy maximum relevance ensemble feature selection: a bi-objective Pareto-based approach. J. Soft Comput. Inf. Technol. (2021)

    Google Scholar 

  9. Hashemi, A., Dowlatshahi, M.B., Nezamabadi-pour, H.: VMFS: a VIKOR-based multi-target feature selection. Expert Syst. Appl. 182, 115224 (2021). https://doi.org/10.1016/j.eswa.2021.115224

    Article  Google Scholar 

  10. Hashemi, A., Dowlatshahi, M.B., Nezamabadi-pour, H.: A Pareto-based ensemble of feature selection algorithms. Expert Syst. Appl. 180, 115130 (2021). https://doi.org/10.1016/j.eswa.2021.115130

    Article  Google Scholar 

  11. Hashemi, A., Dowlatshahi, M.B., Nezamabadi-Pour, H.: A bipartite matching-based feature selection for multi-label learning. Int. J. Mach. Learn. Cybern. 12(2), 459–475 (2020). https://doi.org/10.1007/s13042-020-01180-w

    Article  Google Scholar 

  12. Hashemi, A., Dowlatshahi, M.B., Nezamabadi-pour, H.: Ensemble of feature selection algorithms: a multi-criteria decision-making approach. Int. J. Mach. Learn. Cybern. 13(1), 49–69 (2021). https://doi.org/10.1007/s13042-021-01347-z

    Article  Google Scholar 

  13. Hashemi, A., Joodaki, M., Joodaki, N.Z., Dowlatshahi, M.B.: Ant Colony Optimization equipped with an ensemble of heuristics through multi-criteria decision making: a case study in ensemble feature selection. Appl. Soft. Comput. 109046 (2022). https://doi.org/10.1016/j.asoc.2022.109046

  14. Almomani, A., et al.: Phishing website detection with semantic features based on machine learning classifiers: a comparative study. Int. J. Semant. Web Inf. Syst. (IJSWIS) 18(1), 1–24 (2022)

    Article  MathSciNet  Google Scholar 

  15. Hashemi, A., Pajoohan, M.-R., Dowlatshahi, M.B.: Online streaming feature selection based on Sugeno fuzzy integral. In: 2022 9th Iranian Joint Congress on Fuzzy and Intelligent Systems (CFIS), pp. 1–6. IEEE (2022)

    Google Scholar 

  16. Huang, R., Jiang, W., Sun, G.: Manifold-based constraint Laplacian score for multi-label feature selection. Pattern Recognit. Lett. 112, 346–352 (2018). https://doi.org/10.1016/j.patrec.2018.08.021

    Article  Google Scholar 

  17. Kashef, S., Nezamabadi-pour, H.: A label-specific multi-label feature selection algorithm based on the Pareto dominance concept. Pattern Recognit. 88, 654–667 (2019). https://doi.org/10.1016/j.patcog.2018.12.020

    Article  Google Scholar 

  18. Kashef, S., Nezamabadi-pour, H., Nikpour, B.: Multilabel feature selection: a comprehensive review and guiding experiments. Wiley Interdiscip. Rev. Data Min. Knowl. Discov. 8, e1240 (2018). 10.1002/widm.1240

    Google Scholar 

  19. Miri, M., Dowlatshahi, M.B., Hashemi, A.: Evaluation multi label feature selection for text classification using weighted Borda count approach. In: 2022 9th Iranian Joint Congress on Fuzzy and Intelligent Systems (CFIS), pp. 1–6. IEEE (2022)

    Google Scholar 

  20. Paniri, M., Dowlatshahi, M.B., Nezamabadi-pour, H.: MLACO: a multi-label feature selection algorithm based on ant colony optimization. Knowl. Based Syst. 192, 105285 (2020). https://doi.org/10.1016/j.knosys.2019.105285

    Article  Google Scholar 

  21. Pan, X., Yamaguchi, S., Kageyama, T., Kamilin, M.H.B.: Machine-learning-based white-hat worm launcher in botnet defense system. Int. J. Softw. Sci. Comput. Intell. (IJSSCI) 14(1), 1–14 (2022)

    Article  Google Scholar 

  22. Reyes, O., Morell, C., Ventura, S.: Scalable extensions of the ReliefF algorithm for weighting and selecting features on the multi-label learning context. Neurocomputing 161 (2015). https://doi.org/10.1016/j.neucom.2015.02.045

  23. Sebastiani, F.: Machine learning in automated text categorization. ACM Comput. Surv. (CSUR) 34, 1–47 (2002)

    Article  Google Scholar 

  24. Sheikhpour, R., Sarram, M.A., Gharaghani, S., Chahooki, M.A.Z.: A Survey on semi-supervised feature selection methods. Pattern Recognit. 64, 141–158 (2017). https://doi.org/10.1016/j.patcog.2016.11.003

    Article  MATH  Google Scholar 

  25. Gaurav, A., et al.: A comprehensive survey on machine learning approaches for malware detection in IoT-based enterprise information system. Enterp. Inf. Syst. 1–25 (2022)

    Google Scholar 

  26. Solorio-Fernández, S., Carrasco-Ochoa, J.A., Martínez-Trinidad, J.F.: A review of unsupervised feature selection methods. Artif. Intell. Rev. 53(2), 907–948 (2019). https://doi.org/10.1007/s10462-019-09682-y

    Article  Google Scholar 

  27. Cvitić, I., Peraković, D., Periša, M., Gupta, B.: Ensemble machine learning approach for classification of IoT devices in smart home. Int. J. Mach. Learn. Cybern. 12(11), 3179–3202 (2021). https://doi.org/10.1007/s13042-020-01241-0

    Article  Google Scholar 

  28. Tsoumakas, G., Katakis, I.: Multi-label classification: an overview. Int. J. Data Warehous. Min. (IJDWM) 3, 1–13 (2007)

    Article  Google Scholar 

  29. Zhang, J., Luo, Z., Li, C., Zhou, C., Li, S.: Manifold regularized discriminative feature selection for multi-label learning. Pattern Recognit. 95, 136–150 (2019). https://doi.org/10.1016/j.patcog.2019.06.003

    Article  Google Scholar 

  30. Zhang, M.L., Zhou, Z.H.: ML-KNN: a lazy learning approach to multi-label learning. Pattern Recognit. 40, 2038–2048 (2007). https://doi.org/10.1016/j.patcog.2006.12.019

    Article  MATH  Google Scholar 

  31. Brdesee, H.S., Alsaggaf, W., Aljohani, N., Hassan, S.U.: Predictive model using a machine learning approach for enhancing the retention rate of students at-risk. Int. J. Semant. Web Inf. Syst. (IJSWIS) 18(1), 1–21 (2022)

    Article  Google Scholar 

  32. Zhang, P., Liu, G., Gao, W.: Distinguishing two types of labels for multi-label feature selection. Pattern Recognit. 95, 72–82 (2019). https://doi.org/10.1016/j.patcog.2019.06.004

    Article  Google Scholar 

  33. Zhang, R., Nie, F., Li, X., Wei, X.: Feature selection with multi-view data: a survey. Inf. Fusion 50, 158–167 (2019). https://doi.org/10.1016/j.inffus.2018.11.019

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Engineering, Faculty of Engineering, Lorestan University, Khorramabad, Iran

    Amin Hashemi & Mohammad Bagher Dowlatshahi

  2. Department of Computer Science, Shahid Bahonar University of Kerman, Kerman, Iran

    Marjan Kuchaki Rafsanjani

  3. College of Information and Electrical Engineering, Asia University, Taichung, Taiwan

    Ching-Hsien Hsu

Authors
  1. Amin Hashemi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mohammad Bagher Dowlatshahi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Marjan Kuchaki Rafsanjani
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Ching-Hsien Hsu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Marjan Kuchaki Rafsanjani .

Editor information

Editors and Affiliations

  1. Electronics Engineering and Telecom, State University of Rio de Janeiro, Rio de Janeiro, Brazil

    Nadia Nedjah

  2. University of Murcia, Murcia, Spain

    Gregorio Martínez Pérez

  3. Asia University, Taichung, Taiwan

    B. B. Gupta

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Hashemi, A., Dowlatshahi, M.B., Rafsanjani, M.K., Hsu, CH. (2023). Ensemble Feature Selection for Multi-label Classification: A Rank Aggregation Method. In: Nedjah, N., Martínez Pérez, G., Gupta, B.B. (eds) International Conference on Cyber Security, Privacy and Networking (ICSPN 2022). ICSPN 2021. Lecture Notes in Networks and Systems, vol 599. Springer, Cham. https://doi.org/10.1007/978-3-031-22018-0_14

Download citation

Publish with us

Policies and ethics

Search

Navigation