Swarm MeLiF: Feature Selection with Filter Combination Found via Swarm Intelligence

  • Ivan Smetannikov
  • Evgeniy Varlamov
  • Andrey Filchenkov
Conference paper
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 449)


Combination of algorithms being called ensemble is a widely used machine learning technique. In this paper we propose a new method Swarm MeLiF which aims to find the best combination of basic filters and uses swarm optimization methods for this purpose. In this work we combine filters by combining the measures they use to evaluate feature importance. Thus, the problem of filter ensemble learning is reduced to finding a linear combination of these measures. We applied several swarm optimization methods and found that Particle Swarm Optimization shows the best results and outperforms the original MeLiF.


Swarm intelligence Biologically-inspired meta-heuristics Feature selection Attribute selection Ensemble learning 


  1. 1.
    Abeel, T., Helleputte, T., Van de Peer, Y., Dupont, P., Saeys, Y.: Robust biomarker identification for cancer diagnosis with ensemble feature selection methods. Bioinformatics 26(3), 392–398 (2010)CrossRefGoogle Scholar
  2. 2.
    Binitha, S., Sathya, S.S.: A survey of bio inspired optimization algorithms. Int. J. Soft Comput. Eng. 2(2), 137–151 (2012)Google Scholar
  3. 3.
    Bolón-Canedo, V., Sánchez-Maroño, N., Alonso-Betanzos, A.: An ensemble of filters and classifiers for microarray data classification. Pattern Recogn. 45(1), 531–539 (2012)CrossRefGoogle Scholar
  4. 4.
    Burges, C.J.: A tutorial on support vector machines for pattern recognition. Data Min. Knowl. Disc. 2(2), 121–167 (1998)CrossRefGoogle Scholar
  5. 5.
    Eggensperger, K., Feurer, M., Hutter, F., Bergstra, J., Snoek, J., Hoos, H., Leyton-Brown, K.: Towards an empirical foundation for assessing bayesian optimization of hyperparameters. In: NIPS Workshop on Bayesian Optimization in Theory and Practice (2013)Google Scholar
  6. 6.
    Filchenkov, A., Dolganov, V., Smetannikov, I.: Pca-based algorithm for constructing ensembles of feature ranking filters. In: proceedings of ESANN Conference, pp. 201–206 (2015)Google Scholar
  7. 7.
    Kononenko, I.: Estimating attributes: analysis and extensions of relief. In: Machine Learning: ECML-94, pp. 171–182. Springer (1994)Google Scholar
  8. 8.
    Pedersen, M.E.H.: A hybrid glowworm swarm optimization algorithm for constrained engineering design problems. Hvass Laboratories Technical Report HL1001 (2010)Google Scholar
  9. 9.
    Pedersen, M.E.H.: Tuning and simplifying heuristical optimization. Ph.D. thesis, University of Southampton (2010)Google Scholar
  10. 10.
    Saeys, Y., Inza, I., Larrañaga, P.: A review of feature selection techniques in bioinformatics. Bioinformatics 23(19), 2507–2517 (2007)Google Scholar
  11. 11.
    Smetannikov, I., Filchenkov, A.: MeLiF: filter ensemble learning algorithm for gene selection. In: Advanced Science Letters. American Scientific Publisher (2016, to appear)Google Scholar
  12. 12.
    Sun, Y.: Feature weighting through local learning. In: Computational Methods of Feature Selection, p. 233 (2008)Google Scholar
  13. 13.
    Yang, X.S.: Firefly algorithm, Lévy flights and global optimization. In: Research and Development in Intelligent Systems XXVI, pp. 209–218. Springer (2010)Google Scholar
  14. 14.
    Yang, X.S.: Review of meta-heuristics and generalised evolutionary walk algorithm. Int. J. Bio-Inspired Comput. 3(2), 77–84 (2011)CrossRefGoogle Scholar
  15. 15.
    Yu, L.: Feature selection for genomic data analysis. In: Computational Methods of Feature Selection, pp. 337–353 (2008)Google Scholar
  16. 16.
    Zhou, Y., Zhou, G., Zhang, J.: Good parameters for particle swarm optimization. Appl. Math. Inf. Sci. 7, 379 (2013)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  • Ivan Smetannikov
    • 1
  • Evgeniy Varlamov
    • 1
  • Andrey Filchenkov
    • 1
  1. 1.ITMO UniversitySt. PetersburgRussia

Personalised recommendations