Ensemble Techniques for Parallel Genetic Programming Based Classifiers
An extension of Cellular Genetic Programming for data classifiation to induce an ensemble of predictors is presented. Each classifier is trained on a different subset of the overall data, then they are combined to classify new tuples by applying a simple majority voting algorithm, like bagging. Preliminary results on a large data set show that the ensemble of classifiers trained on a sample of the data obtains higher accuracy than a single classifier that uses the entire data set at a much lower computational cost.
KeywordsGenetic Programming Main Memory Parallel Implementation Lower Computational Cost Island Model
Unable to display preview. Download preview PDF.
- 1.Eric Bauer and Ron Kohavi. An empirical comparison of voting classification algorithms: Bagging, boosting, and variants. Machine Learning, (36):105–139, 1999.Google Scholar
- 5.P. K. Chan and S. J. Stolfo. A comparative evaluation of voting and meta-learning on partitioned data. In International Conference on Machine Learning ICML95, pages 90–98, 1995.Google Scholar
- 6.N. Chawla, T. E. Moore, W. Bowyer K, L. O. Hall, C. Springer, and P. Kegelmeyer. Bagging-like effects for decision trees and neural nets in protein secondary structure prediction. In BIOKDD01: Workshop on Data mining in Bioinformatics (SIGKDD01), 2001.Google Scholar
- 7.Thomas G. Dietterich. An experimental comparison of three methods for costructing ensembles of decision trees: Bagging, boosting, and randomization. Machine Learning, (40):139–157, 2000.Google Scholar
- 8.U. M. Fayyad, G. Piatesky-Shapiro, and P. Smith. From data mining to knowledge discovery: an overview. In U. M. Fayyad & al. (Eds), editor, Advances in Knowledge Discovery and Data Mining, pages 1–34. AAAI/MIT Press, 1996.Google Scholar
- 9.G. Folino, C. Pizzuti, and G. Spezzano. A cellular genetic programming approach to classification. In Proc. Of the Genetic and Evolutionary Computation Conference GECCO99, pages 1015–1020, Orlando, Florida, July 1999. Morgan Kaufmann.Google Scholar
- 10.G. Folino, C. Pizzuti, and G. Spezzano. Genetic programming and simulated annealing: A hybrid method to evolve decision trees. In Riccardo Poli, Wolfgang Banzhaf, William B. Langdon, Julian Miller, Peter Nordin, and Terence C. Fogarty, editors, Proceedings of EuroGP’2000, volume 1802 of LNCS, pages 294–303, Edinburgh, UK, 15–16 April 2000. Springer-Verlag.Google Scholar
- 11.G. Folino, C. Pizzuti, and G. Spezzano. Cage: A tool for parallel genetic programming applications. In Julian F. Miller, Marco Tomassini, Pier Luca Lanzi, Conor Ryan, Andrea G. B. Tettamanzi, and William B. Langdon, editors, Proceedings of EuroGP’2001, volume 2038 of LNCS, pages 64–73, Lake Como, Italy, 18–20 April 2001. Springer-Verlag.Google Scholar
- 12.G. Folino, C. Pizzuti, and G. Spezzano. Parallel genetic programming for decision tree induction. In Proceedings of the 13th IEEE International Conference on Tools with Artificial Intelligence ICTAI01, pages 129–135. IEEE Computer Society, 2001.Google Scholar
- 13.A. A. Freitas. A genetic programming framework for two data mining tasks: Classification and generalised rule induction. In Proceedings of the 2nd Int. Conference on Genetic Programming, pages 96–101. Stanford University, CA, USA, 1997.Google Scholar
- 14.Y. Freund and R. Scapire. Experiments with a new boosting algorithm. In Proceedings of the 13th Int. Conference on Machine Learning, pages 148–156, 1996.Google Scholar
- 15.Hitoshi Iba. Bagging, boosting, and bloating in genetic programming. In Proc. Of the Genetic and Evolutionary Computation Conference GECCO99, pages 1053–1060, Orlando, Florida, July 1999. Morgan Kaufmann.Google Scholar
- 17.R. E. Marmelstein and G. B. Lamont. Pattern classification using a hybbrid genetic program-decision tree approach. In Proceedings of the Third Annual Conference on Genetic Programming, Morgan Kaufmann, 1998.Google Scholar
- 18.C. J. Merz and P. M. Murphy. In UCI repository of Machine Learning, http://www.ics.uci/mlearn/MLRepository.html, 1996.
- 19.N. I. Nikolaev and V. Slavov. Inductive genetic programming with decision trees. In Proceedings of the 9th International Conference on Machine Learning, Prague, Czech Republic, 1997.Google Scholar
- 20.J. Ross Quinlan. C4.5 Programs for Machine Learning. Morgan Kaufmann, San Mateo, Calif., 1993.Google Scholar
- 21.J. Ross Quinlan. Bagging, boosting, and c4.5. In Proceedings of the 13th National Conference on Artificial Intelligence AAAI96, pages 725–730. Mit Press, 1996.Google Scholar
- 22.M. D. Ryan and V. J. Rayward-Smith. The evolution of decision trees. In Proceedings of the Third Annual Conference on Genetic Programming, Morgan Kaufmann, 1998.Google Scholar
- 23.M. Tomassini. Parallel and distributed evolutionary algorithms: A review. In P. Neittaanmki K. Miettinen, M. Mkel and J. Periaux, editors, Evolutionary Algorithms in Engineering and Computer Science, J. Wiley and Sons, Chichester, 1999.Google Scholar