Evolvable Deep Features

  • Jakub NalepaEmail author
  • Grzegorz Mrukwa
  • Michal Kawulok
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10784)


Feature extraction is the first step in building real-life classification engines—it aims at elaborating features to characterize objects that are to be labeled by a trained model. Time-consuming feature extraction requires domain expertise to effectively design features. Deep neural networks (DNNs) appeared as a remedy in this context—their shallow layers perform representation learning, being an automated discovery of various-level features that robustly represent objects. However, the representations that are being learnt are still extremely difficult to interpret, and DNNs are prone to memorizing small datasets. In this paper, we introduce evolvable deep features (EDFs)—a DNN is used to extract automatic features that undergo genetic feature selection. Such evolved features are fed into a supervised learner. The experiments, backed up with statistical tests, performed on multi- and binary-class sets showed that our approach automatically learns object representations, greatly reduces the number of features without deteriorating the performance of trained models, and can even boost their classification performance.


Deep learning Genetic algorithm Feature selection 



JN was supported by the Polish National Centre for Research and Development under the Innomed grant POIR.01.02.00-00-0030/15. JN and MK were supported by the National Science Centre, Poland, under Research Grant No. DEC-2017/25/B/ST6/00474.


  1. 1.
    Sheikhpour, R., Sarram, M.A., Gharaghani, S., Chahooki, M.A.Z.: A survey on semi-supervised feature selection methods. Pattern Recogn. 64, 141–158 (2017)CrossRefGoogle Scholar
  2. 2.
    Chandrashekar, G., Sahin, F.: A survey on feature selection methods. Comput. Electr. Eng. 40(1), 16–28 (2014)CrossRefGoogle Scholar
  3. 3.
    Dy, J.G., Brodley, C.E.: Feature selection for unsupervised learning. J. Mach. Learn. Res. 5, 845–889 (2004)MathSciNetzbMATHGoogle Scholar
  4. 4.
    Wang, X., Zhang, X., Zeng, Z., Wu, Q., Zhang, J.: Unsupervised spectral feature selection with l 1-norm graph. Neurocomputing 200, 47–54 (2016)CrossRefGoogle Scholar
  5. 5.
    Shi, H., Li, Y., Han, Y., Hu, Q.: Cluster structure preserving unsupervised feature selection for multi-view tasks. Neurocomputing 175, 686–697 (2016)CrossRefGoogle Scholar
  6. 6.
    Nakariyakul, S., Casasent, D.P.: An improvement on floating search algorithms for feature subset selection. Pattern Recogn. 42(9), 1932–1940 (2009)CrossRefzbMATHGoogle Scholar
  7. 7.
    Chakravarty, K., Das, D., Sinha, A., Konar, A.: Feature selection by differential evolution algorithm - a case study in personnel identification. In: Proceedings of IEEE CEC, pp. 892–899 (2013)Google Scholar
  8. 8.
    Alba, E., Garcia-Nieto, J., Jourdan, L., Talbi, E.G.: Gene selection in cancer classification using PSO/SVM and GA/SVM hybrid algorithms. In: Proceedings of IEEE CEC, pp. 284–290 (2007)Google Scholar
  9. 9.
    Lee, J., Kim, D.W.: Memetic feature selection algorithm for multi-label classification. Inf. Sci. 293, 80–96 (2015)CrossRefGoogle Scholar
  10. 10.
    Goodfellow, I., Bengio, Y., Courville, A.: Deep Learning. MIT Press, Cambridge (2016)zbMATHGoogle Scholar
  11. 11.
    Zhou, B., Lapedriza, A., Xiao, J., Torralba, A., Oliva, A.: Learning deep features for scene recognition using places database. In: Proceedings of NIPS, pp. 487–495 (2014)Google Scholar
  12. 12.
    Poria, S., Cambria, E., Gelbukh, A.: Deep convolutional neural network textual features and multiple kernel learning for utterance-level multimodal sentiment analysis. In: Proceedings of EMNLP, pp. 2539–2544 (2015)Google Scholar
  13. 13.
    Sun, Y., Chen, Y., Wang, X., Tang, X.: Deep learning face representation by joint identification-verification. In: Proceedings of NIPS, pp. 1988–1996 (2014)Google Scholar
  14. 14.
    Nezhad, M.Z., Zhu, D., Li, X., Yang, K., Levy, P.: Safs: a deep feature selection approach for precision medicine. In: Proceedings of IEEE BIBM, pp. 501–506. IEEE (2016)Google Scholar
  15. 15.
    Li, Y., Chen, C.Y., Wasserman, W.W.: Deep feature selection: theory and application to identify enhancers and promoters. J. Comput. Biol. 23(5), 322–336 (2016)CrossRefGoogle Scholar
  16. 16.
    Zou, Q., Ni, L., Zhang, T., Wang, Q.: Deep learning based feature selection for remote sensing scene classification. IEEE Geosci. Remote Sens. Lett. 12(11), 2321–2325 (2015)CrossRefGoogle Scholar
  17. 17.
    Kowaliw, T., Banzhaf, W., Doursat, R.: Networks of transform-based evolvable features for object recognition. In: Proceedings of GECCO, USA, pp. 1077–1084. ACM (2013)Google Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Silesian University of TechnologyGliwicePoland
  2. 2.Future ProcessingGliwicePoland

Personalised recommendations