Pruning GP-Based Classifier Ensembles by Bayesian Networks

  • C. De Stefano
  • G. Folino
  • F. Fontanella
  • A. Scotto di Freca
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7491)


Classifier ensemble techniques are effectively used to combine the responses provided by a set of classifiers. Classifier ensembles improve the performance of single classifier systems, even if a large number of classifiers is often required. This implies large memory requirements and slow speeds of classification, making their use critical in some applications. This problem can be reduced by selecting a fraction of the classifiers from the original ensemble. In this work, it is presented an ensemble-based framework that copes with large datasets, however selecting a small number of classifiers composing the ensemble. The framework is based on two modules: an ensemble-based Genetic Programming (GP) system, which produces a high performing ensemble of decision tree classifiers, and a Bayesian Network (BN) approach to perform classifier selection. The proposed system exploits the advantages provided by both techniques and allows to strongly reduce the number of classifiers in the ensemble. Experimental results compare the system with well-known techniques both in the field of GP and BN and show the effectiveness of the devised approach. In addition, a comparison with a pareto optimal strategy of pruning has been performed.


Bayesian Network Pareto Front Direct Acyclic Graph Classifier Ensemble Pruning Strategy 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Banfield, R., Hall, L., Bowyer, K., Kegelmeyer, W.: Ensembles diversity measures and their application to thinning. Information Fusion 6, 49–62 (2005)CrossRefGoogle Scholar
  2. 2.
    Cantú-Paz, E., Kamath, C.: Inducing oblique decision trees with evolutionary algorithms. IEEE Trans. on Evolutionary Computation 7(1), 54–68 (2003)CrossRefGoogle Scholar
  3. 3.
    Cooper, G.F., Herskovits, E.: A bayesian method for the induction of probabilistic networks from data. Machine Learning 9(4), 309–347 (1992)zbMATHGoogle Scholar
  4. 4.
    De Stefano, C., D’Elia, C., Scotto di Freca, A., Marcelli, A.: Classifier combination by bayesian networks for handwriting recognition. Int. Journal of Pattern Rec. and Artif. Intell. 23(5), 887–905 (2009)CrossRefGoogle Scholar
  5. 5.
    De Stefano, C., Fontanella, F., Marrocco, C., Scotto di Freca, A.: A Hybrid Evolutionary Algorithm for Bayesian Networks Learning: An Application to Classifier Combination. In: Di Chio, C., Cagnoni, S., Cotta, C., Ebner, M., Ekárt, A., Esparcia-Alcazar, A.I., Goh, C.-K., Merelo, J.J., Neri, F., Preuß, M., Togelius, J., Yannakakis, G.N. (eds.) EvoApplicatons 2010, Part I. LNCS, vol. 6024, pp. 221–230. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  6. 6.
    Folino, G., Pizzuti, C., Spezzano, G.: Gp ensembles for large-scale data classification. IEEE Trans. on Evolutionary Computation 10(5), 604–616 (2006)CrossRefGoogle Scholar
  7. 7.
    Folino, G., Pizzuti, C., Spezzano, G.: Training distributed gp ensemble with a selective algorithm based on clustering and pruning for pattern classification. IEEE Trans. Evolutionary Computation 12(4), 458–468 (2008)CrossRefGoogle Scholar
  8. 8.
    Freund, Y., Shapire, R.: Proceedings of the 13th Int. Conference on Machine LearningGoogle Scholar
  9. 9.
    Gagné, C., Sebag, M., Schoenauer, M., Tomassini, M.: Ensemble learning for free with evolutionary algorithms? In: GECCO, pp. 1782–1789 (2007)Google Scholar
  10. 10.
    Kuncheva, L.I.: Combining Pattern Classifiers: Methods and Algorithms. Wiley-Interscience (2004)Google Scholar
  11. 11.
    Pearl, J.: Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference. Morgan Kaufmann (1988)Google Scholar
  12. 12.
    Zhou, Z.H., Tang, W.: Selective Ensemble of Decision Trees. In: Wang, G., Liu, Q., Yao, Y., Skowron, A. (eds.) RSFDGrC 2003. LNCS (LNAI), vol. 2639, pp. 476–483. Springer, Heidelberg (2003)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • C. De Stefano
    • 1
  • G. Folino
    • 2
  • F. Fontanella
    • 1
  • A. Scotto di Freca
    • 1
  1. 1.Università di Cassino e del Lazio MeridionaleItaly
  2. 2.ICAR-CNR Istituto di Calcolo e Reti ad Alte PrestazioniItaly

Personalised recommendations