Evolvable Deep Features
Feature extraction is the first step in building real-life classification engines—it aims at elaborating features to characterize objects that are to be labeled by a trained model. Time-consuming feature extraction requires domain expertise to effectively design features. Deep neural networks (DNNs) appeared as a remedy in this context—their shallow layers perform representation learning, being an automated discovery of various-level features that robustly represent objects. However, the representations that are being learnt are still extremely difficult to interpret, and DNNs are prone to memorizing small datasets. In this paper, we introduce evolvable deep features (EDFs)—a DNN is used to extract automatic features that undergo genetic feature selection. Such evolved features are fed into a supervised learner. The experiments, backed up with statistical tests, performed on multi- and binary-class sets showed that our approach automatically learns object representations, greatly reduces the number of features without deteriorating the performance of trained models, and can even boost their classification performance.
KeywordsDeep learning Genetic algorithm Feature selection
JN was supported by the Polish National Centre for Research and Development under the Innomed grant POIR.01.02.00-00-0030/15. JN and MK were supported by the National Science Centre, Poland, under Research Grant No. DEC-2017/25/B/ST6/00474.
- 7.Chakravarty, K., Das, D., Sinha, A., Konar, A.: Feature selection by differential evolution algorithm - a case study in personnel identification. In: Proceedings of IEEE CEC, pp. 892–899 (2013)Google Scholar
- 8.Alba, E., Garcia-Nieto, J., Jourdan, L., Talbi, E.G.: Gene selection in cancer classification using PSO/SVM and GA/SVM hybrid algorithms. In: Proceedings of IEEE CEC, pp. 284–290 (2007)Google Scholar
- 11.Zhou, B., Lapedriza, A., Xiao, J., Torralba, A., Oliva, A.: Learning deep features for scene recognition using places database. In: Proceedings of NIPS, pp. 487–495 (2014)Google Scholar
- 12.Poria, S., Cambria, E., Gelbukh, A.: Deep convolutional neural network textual features and multiple kernel learning for utterance-level multimodal sentiment analysis. In: Proceedings of EMNLP, pp. 2539–2544 (2015)Google Scholar
- 13.Sun, Y., Chen, Y., Wang, X., Tang, X.: Deep learning face representation by joint identification-verification. In: Proceedings of NIPS, pp. 1988–1996 (2014)Google Scholar
- 14.Nezhad, M.Z., Zhu, D., Li, X., Yang, K., Levy, P.: Safs: a deep feature selection approach for precision medicine. In: Proceedings of IEEE BIBM, pp. 501–506. IEEE (2016)Google Scholar
- 17.Kowaliw, T., Banzhaf, W., Doursat, R.: Networks of transform-based evolvable features for object recognition. In: Proceedings of GECCO, USA, pp. 1077–1084. ACM (2013)Google Scholar