Adaptive Boosting: Dividing the Learning Set to Increase the Diversity and Performance of the Ensemble
As shown in the bibliography, Boosting methods are widely used to build ensembles of neural networks. This kind of methods increases the performance with respect to a single network. Since Freund and Schapire introduced Adaptive Boosting in 1996 some authors have studied and improved Adaboost. In this paper we present Cross Validated Boosting a method based on Adaboost and Cross Validation. We have applied Cross Validation to the learning set in order to get an specific training set and validation set for each network. With this procedure the diversity increases because each network uses an specific validation set to finish its learning. Finally, we have performed a comparison among Adaboost and Crossboost on eight databases from UCI, the results show that Crossboost is the best performing method.
KeywordsMinimum Mean Square Error Error Reduction Size Ensemble Single Network Wisconsin Breast Cancer
Unable to display preview. Download preview PDF.
- 2.Raviv, Y., Intratorr, N.: Bootstrapping with noise: An effective regularization technique. Connection Science, Special issue on Combining Estimators 8, 356–372 (1996)Google Scholar
- 4.Freund, Y., Schapire, R.E.: Experiments with a new boosting algorithm. In: International Conference on Machine Learning, pp. 148–156 (1996)Google Scholar
- 9.Newman, D.J., Hettich, S., Blake, C.L., Merz, C.J.: UCI repository of machine learning databases (1998)Google Scholar