Pruning Adaptive Boosting Ensembles by Means of a Genetic Algorithm

  • Daniel Hernández-Lobato
  • José Miguel Hernández-Lobato
  • Rubén Ruiz-Torrubiano
  • Ángel Valle
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4224)

Abstract

This work analyzes the problem of whether, given a classification ensemble built by Adaboost, it is possible to find a subensemble with lower generalization error. In order to solve this task a genetic algorithm is proposed and compared with other heuristics like Kappa pruning and Reduce-error pruning with backfitting. Experiments carried out over a wide variety of classification problems show that the genetic algorithm behaves better than, or at least, as well as the best of those heuristics and that subensembles with similar and sometimes better prediction accuracy can be obtained.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. In: Proc. 2nd European Conference on Computational Learning Theory, pp. 23–37 (1995)Google Scholar
  2. Quinlan, J.R.: C4.5 programs for machine learning. Morgan Kaufmann, San Francisco (1993)Google Scholar
  3. Breiman, L., Friedman, J.H., Olshen, R.A., Stone, C.J.: Classification and Regression Trees. Chapman & Hall, New York (1984)MATHGoogle Scholar
  4. Quinlan, J.R.: Bagging, boosting, and C4.5. In: Proc. 13th National Conference on Artificial Intelligence, Cambridge, MA, pp. 725–730 (1996)Google Scholar
  5. Freund, Y., Schapire, R.E.: Experiments with a new boosting algorithm. In: International Conference on Machine Learning, pp. 148–156 (1996)Google Scholar
  6. Dietterich, T.G.: An experimental comparison of three methods for constructing ensembles of decision trees: Bagging, boosting, and randomization. Machine Learning 40(2), 139–157 (2000)CrossRefGoogle Scholar
  7. Breiman, L.: Arcing classifiers. The Annals of Statistics 26(3), 801–849 (1998)MATHCrossRefMathSciNetGoogle Scholar
  8. Zhou, Z.H., Wu, J., Tang, W.: Ensembling neural networks: Many could be better than all. Artificial Intelligence 137(1-2), 239–263 (2002)MATHCrossRefMathSciNetGoogle Scholar
  9. Martínez-Muñoz, G., Suárez, A.: Pruning in ordered bagging ensembles. International Conference on Machine Learning, 609–616 (2006)Google Scholar
  10. Breiman, L.: Bagging predictors. Machine Learning 24(2), 123–140 (1996)MATHMathSciNetGoogle Scholar
  11. Margineantu, D.D., Dietterich, T.G.: Pruning adaptive boosting. In: Proc. 14th International Conference on Machine Learning, pp. 211–218. Morgan Kaufmann, San Francisco (1997)Google Scholar
  12. Tamon, C., Xiang, J.: On the boosting pruning problem. In: Lopez de Mantaras, R., Plaza, E. (eds.) ECML 2000. LNCS (LNAI), vol. 1810, pp. 404–412. Springer, Heidelberg (2000)CrossRefGoogle Scholar
  13. Garey, M.R., Johnson, D.S.: Computers and Intractability. In: A Guide to the Theory of NP-Completeness, W. H. Freeman & Co, New York (1990)Google Scholar
  14. Goldberg, D.E.: Genetic Algorithms in Search, Optimization and Machine Learning. Addison-Wesley Longman Publishing Co., Inc., Boston (1989)MATHGoogle Scholar
  15. Blake, C.L., Merz, C.J.: UCI repository of machine learning databases (1998)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Daniel Hernández-Lobato
    • 1
  • José Miguel Hernández-Lobato
    • 1
  • Rubén Ruiz-Torrubiano
    • 1
  • Ángel Valle
    • 2
  1. 1.Escuela Politécnica SuperiorUniversidad Autónoma de MadridMadridSpain
  2. 2.CognodataMadridSpain

Personalised recommendations