A New Way for Combining Filter Feature Selection Methods

  • Waad Bouaguel
  • Mohamed Limam
Conference paper
Part of the Smart Innovation, Systems and Technologies book series (SIST, volume 43)


This study investigates the issue of obtaining stable ranking from the fusion of the result of multiple filtering methods. Rank aggregation is the process of performing multiple runs of feature selection and then aggregating the results into a final ranked list. However, a fundamental question of is how to aggregate the individual results into a single robust ranked feature list. There are a number of available methods, ranging from simple to complex. Hence we present a new rank aggregation approach. The proposed approach is composed of two stages: in the first we evaluate he similarity and stability of single filtering methods then, in the second we aggregate the results of the stable ones. The obtained results on the Australian and German credit datasets using support vector machine and decision tree confirms that ensemble feature ranking have a major impact in the performance improvement.


Feature selection Credit scoring 


  1. 1.
    Ben Brahim, A., Bouaguel, W., Limam, M.: Feature selection aggregation versus classifiers aggregation for several data dimensionalities. In: Proceedings of the International Conference on Control, Engineering & Information Technology (CEIT13) (2013)Google Scholar
  2. 2.
    Ben brahim, A., Bouaguel, W., Limam, M.: Combining feature selection and data classification using ensemble approaches: application to cancer diagnosis and credit scoring. In: Francisr, T. (ed.) Case Studies in Intelligent Computing: Achievements and Trendss. CRC Press, Boca Raton (2013)Google Scholar
  3. 3.
    Fernandez, G.: Statistical data mining using SAS applications. In; Chapman & Hall/Crc: Data Mining and Knowledge Discovery. Taylor and Francis, Boca Raton (2010)Google Scholar
  4. 4.
    Forman, G.: BNS feature scaling: an improved representation over TF-IDF for SVM text classification. In: Proceedings of the 17th ACM Conference on Information and Knowledge Mining, pp. 263–270. ACM, New York, NY, USA (2008)Google Scholar
  5. 5.
    Rodriguez, I., Huerta, R., Elkan, C., Cruz, C.S.: Quadratic programming feature selection. J. Mach. Learn. Res. 11(4), 1491–1516 (2010)MATHMathSciNetGoogle Scholar
  6. 6.
    Saeys, Y., Inza, I.N., Larrañaga, P.: A review of feature selection techniques in bioinformatics. Bioinformatics 23(19), 2507–2517 (2007)Google Scholar
  7. 7.
    Bouaguel, W., Bel Mufti, G., Limam, M.: A new feature selection technique applied to credit scoring data using a rank aggregation approach based on: optimization, genetic algorithm and similarity. In: Francisr, T. (ed.) Knowledge Discovery & Data Mining (KDDM) for Economic Development: Applications, Strategies and Techniques. CRC Press, Chicago (2014)Google Scholar
  8. 8.
    Wu, O., Zuo, H., Zhu, M., Hu, W., Gao, J., Wang, H.: Rank aggregation based text feature selection. In: Proceedings of the Web Intelligence, pp. 165–172. (2009)Google Scholar
  9. 9.
    Wang, C.M., Huang, W.F.: Evolutionary-based feature selection approaches with new criteria for data mining: a case study of credit approval data. Expert Syst. Appl. 36(3), 5900–5908 (2009)CrossRefGoogle Scholar
  10. 10.
    Bouaguel, W., Bel Mufti, G.: An improvement direction for filter selection techniques using information theory measures and quadratic optimization. Int. J. Adv. Res. Artif. Intell. 1(5), 7–11 (2012)Google Scholar
  11. 11.
    Dittman, D.J., Khoshgoftaar, T.M., Wald, R., Napolitano, A.: Classification performance of rank aggregation techniques for ensemble gene selection. In: Boonthum-Denecke, C., Youngblood, G.M. (eds.) Proceedings of the International Conference of the Florida Artificial Intelligence Research Society (FLAIRS), AAAI Press, Coconut Grove (2013)Google Scholar
  12. 12.
    Saeys, Y., Abeel, T., Peer, Y.: Robust feature selection using ensemble feature selection techniques. In: Proceedings of the European conference on Machine Learning and Knowledge Discovery in Databases—Part II. ECML PKDD ‘08, pp. 313–325. Springer, Berlin, Heidelberg (2008)Google Scholar
  13. 13.
    Molina, L.C., Belanche, L., Nebot, A.: Feature selection algorithms: a survey and experimental evaluation. In: Proceedings of the IEEE International Conference on Data Mining, pp. 306–313. IEEE Computer Society (2002)Google Scholar
  14. 14.
    Dash, M., Liu, H.: Consistency-based search in feature selection. Artif. Intell. 151(1–2), 155–176 (2003)MATHMathSciNetCrossRefGoogle Scholar
  15. 15.
    Krishnaiah, P., Kanal, L.: Preface. In: Krishnaiah, P., Kanal, L. (eds.) Classification Pattern Recognition and Reduction of Dimensionality. Handbook of Statistics, vol. 2, pp. v–ix. Elsevier (1982)Google Scholar
  16. 16.
    Guyon, I., Elisseeff, A.: An introduction to variable and feature selection. J. Mach. Learn. Res. 3(9), 1157–1182 (2003)MATHGoogle Scholar
  17. 17.
    Hastie, T., Tibshirani, R., Friedman, J.: The Elements of Statistical Learning. Springer Series in Statistics. Springer New York Inc, New York (2001)MATHCrossRefGoogle Scholar
  18. 18.
    Prati, R.C.: Combining feature ranking algorithms through rank aggregation. In: The 2012 International Joint Conference on Neural Networks (IJCNN), pp. 1–8. Brisbane, Australia, 10–15 June 2012Google Scholar
  19. 19.
    Kalousis, A., Prados, J., Hilario, M.: Stability of feature selection algorithms: a study on high-dimensional spaces. Knowl. Inf. Syst. 12(1), 95–116 (2007)CrossRefGoogle Scholar
  20. 20.
    Pihur, V., Datta, S., Datta, S.: RankAggreg, an R package for weighted rank aggregation. BMC Bioinform. 10(1), 62–72 (2009)CrossRefGoogle Scholar
  21. 21.
    Mak, M.W., Kung, S.Y.: Fusion of feature selection methods for pairwise scoring svm. Neurocomputing 71(16–18), 3104–3113 (2008)CrossRefGoogle Scholar
  22. 22.
    Bouckaert, R.R., Frank, E., Hall, M., Kirkby, R., Reutemann, P., Seewald, A., Scuse, D.: Weka manual (3.7.1) (2009)Google Scholar
  23. 23.
    Kolde, R., Laur, S., Adler, P., Vilo, J.: Robust rank aggregation for gene list integration and meta-analysis. Bioinformatics 28(4), 573–580 (2012)CrossRefGoogle Scholar

Copyright information

© Springer India 2016

Authors and Affiliations

  1. 1.LARODEC, ISGUniversity of TunisTunisTunisia
  2. 2.Dhofar UniversitySalalahOman

Personalised recommendations