Memetic Evolution of Classification Ensembles

  • Szymon Piechaczek
  • Michal Kawulok
  • Jakub NalepaEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11454)


Creating classification ensembles may be perceived as a regularization technique which aims at improving the generalization capabilities of a classifier. In this paper, we introduce a multi-level memetic algorithm for evolving classification ensembles (they can be either homo- or heterogeneous). First, we evolve the content of such ensembles, and then we optimize the weights (both for the classifiers and for different classes) exploited while voting. The experimental study showed that our memetic algorithm retrieves high-quality heterogeneous ensembles, and can effectively deal with small training sets in multi-class classification.


Ensemble classifier Memetic algorithm Classification 



This work was supported by the National Science Centre, Poland, under Research Grant No. DEC-2017/25/B/ST6/00474, and JN was partially supported by the Silesian University of Technology under the Grant for young researchers (BKM-556/RAU2/2018).


  1. 1.
    Dietterich, T.G.: Ensemble methods in machine learning. In: Kittler, J., Roli, F. (eds.) MCS 2000. LNCS, vol. 1857, pp. 1–15. Springer, Heidelberg (2000). Scholar
  2. 2.
    Olague, G.: Evolutionary Computer Vision - The First Footprints. Natural Computing Series. Springer, Heidelberg (2016). Scholar
  3. 3.
    Woźniak, M., Graña, M., Corchado, E.: A survey of multiple classifier systems as hybrid systems. Inf. Fusion 16, 3–17 (2014)CrossRefGoogle Scholar
  4. 4.
    Claesen, M., De Smet, F., Suykens, J.A., De Moor, B.: Ensemblesvm: a library for ensemble learning using support vector machines. JMLR 15(1), 141–145 (2014)zbMATHGoogle Scholar
  5. 5.
    Nguyen, T.T., Nguyen, M.P., Pham, X.C., Liew, A.W.C.: Heterogeneous classifier ensemble with fuzzy rule-based meta learner. Inf. Sci. 422, 144–160 (2018)CrossRefGoogle Scholar
  6. 6.
    Kuncheva, L.I.: Clustering-and-selection model for classifier combination. In: 2000 Proceedings Fourth International Conference on Knowledge-Based Intelligent Engineering Systems and Allied Technologies, vol. 1, pp. 185–188. IEEE (2000)Google Scholar
  7. 7.
    Britto Jr., A.S., Sabourin, R., Oliveira, L.E.: Dynamic selection of classifiers—a comprehensive review. Pattern Recogn. 47(11), 3665–3680 (2014)CrossRefGoogle Scholar
  8. 8.
    Cruz, R.M., Sabourin, R., Cavalcanti, G.D.: Dynamic classifier selection: recent advances and perspectives. Inf. Fusion 41, 195–216 (2018)CrossRefGoogle Scholar
  9. 9.
    Woloszynski, T., Kurzynski, M.: A probabilistic model of classifier competence for dynamic ensemble selection. Pattern Recogn. 44(10–11), 2656–2668 (2011)CrossRefGoogle Scholar
  10. 10.
    Dos Santos, E.M., Sabourin, R., Maupin, P.: A dynamic overproduce-and-choose strategy for the selection of ensembles. Pattern Recogn. 41(10), 2993–3009 (2008)CrossRefGoogle Scholar
  11. 11.
    Oliveira, D.V., Cavalcanti, G.D., Sabourin, R.: Online pruning of base classifiers for dynamic ensemble selection. Pattern Recogn. 72, 44–58 (2017)CrossRefGoogle Scholar
  12. 12.
    Zhang, Z.L., Luo, X.G., García, S., Tang, J.F., Herrera, F.: Exploring the effectiveness of dynamic ensemble selection in the one-versus-one scheme. Knowl.-Based Syst. 125, 53–63 (2017)CrossRefGoogle Scholar
  13. 13.
    García, S., Zhang, Z.L., Altalhi, A., Alshomrani, S., Herrera, F.: Dynamic ensemble selection for multi-class imbalanced datasets. Inf. Sci. 445, 22–37 (2018)MathSciNetCrossRefGoogle Scholar
  14. 14.
    Gabrys, B., Ruta, D.: Genetic algorithms in classifier fusion. Appl. Soft Comput. 6(4), 337–347 (2006)CrossRefGoogle Scholar
  15. 15.
    Onan, A., Korukoğlu, S., Bulut, H.: A multiobjective weighted voting ensemble classifier based on differential evolution algorithm for text sentiment classification. Expert Syst. Appl. 62, 1–16 (2016)CrossRefGoogle Scholar
  16. 16.
    Lones, M.A., Lacy, S.E., Smith, S.L.: Evolving ensembles: what can we learn from biological mutualisms? In: Lones, M., Tyrrell, A., Smith, S., Fogel, G. (eds.) IPCAT 2015. LNCS, vol. 9303, pp. 52–60. Springer, Cham (2015). Scholar
  17. 17.
    Gao, Q., Lim, S., Jia, X.: Hyperspectral image classification using convolutional neural networks and multiple feature learning. Remote Sens. 10(2), 299 (2018)CrossRefGoogle Scholar
  18. 18.
    Ribalta, P., Marcinkiewicz, M., Nalepa, J.: Segmentation of hyperspectral images using quantized convolutional neural nets. In: Proceedings of IEEE DSD, pp. 260–267 (2018)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.Silesian University of TechnologyGliwicePoland

Personalised recommendations