Adaptive Boosting: Dividing the Learning Set to Increase the Diversity and Performance of the Ensemble

  • Joaquín Torres-Sospedra
  • Carlos Hernández-Espinosa
  • Mercedes Fernández-Redondo
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4232)


As shown in the bibliography, Boosting methods are widely used to build ensembles of neural networks. This kind of methods increases the performance with respect to a single network. Since Freund and Schapire introduced Adaptive Boosting in 1996 some authors have studied and improved Adaboost. In this paper we present Cross Validated Boosting a method based on Adaboost and Cross Validation. We have applied Cross Validation to the learning set in order to get an specific training set and validation set for each network. With this procedure the diversity increases because each network uses an specific validation set to finish its learning. Finally, we have performed a comparison among Adaboost and Crossboost on eight databases from UCI, the results show that Crossboost is the best performing method.


Minimum Mean Square Error Error Reduction Size Ensemble Single Network Wisconsin Breast Cancer 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Tumer, K., Ghosh, J.: Error correlation and error reduction in ensemble classifiers. Connection Science 8(3-4), 385–403 (1996)CrossRefGoogle Scholar
  2. 2.
    Raviv, Y., Intratorr, N.: Bootstrapping with noise: An effective regularization technique. Connection Science, Special issue on Combining Estimators 8, 356–372 (1996)Google Scholar
  3. 3.
    Verikas, A., Lipnickas, A., Malmqvist, K., Bacauskiene, M., Gelzinis, A.: Soft combination of neural classifiers: A comparative study. Pattern Recognition Letters 20(4), 429–444 (1999)CrossRefGoogle Scholar
  4. 4.
    Freund, Y., Schapire, R.E.: Experiments with a new boosting algorithm. In: International Conference on Machine Learning, pp. 148–156 (1996)Google Scholar
  5. 5.
    Breiman, L.: Arcing classifiers. The Annals of Statistics 26(3), 801–849 (1998)MATHCrossRefMathSciNetGoogle Scholar
  6. 6.
    Kuncheva, L., Whitaker, C.J.: Using diversity with three variants of boosting: Aggressive. In: Roli, F., Kittler, J. (eds.) MCS 2002. LNCS, vol. 2364, p. 81. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  7. 7.
    Oza, N.C.: Boosting with averaged weight vectors. In: Windeatt, T., Roli, F. (eds.) MCS 2003. LNCS, vol. 2709, pp. 15–24. Springer, Heidelberg (2003)CrossRefGoogle Scholar
  8. 8.
    Drucker, H., Cortes, C., Jackel, L.D., LeCun, Y., Vapnik, V.: Boosting and other ensemble methods. Neural Computation 6(6), 1289–1301 (1994)MATHCrossRefGoogle Scholar
  9. 9.
    Newman, D.J., Hettich, S., Blake, C.L., Merz, C.J.: UCI repository of machine learning databases (1998)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Joaquín Torres-Sospedra
    • 1
  • Carlos Hernández-Espinosa
    • 1
  • Mercedes Fernández-Redondo
    • 1
  1. 1.Departamento de Ingenieria y Ciencia de los ComputadoresUniversitat Jaume ICastellonSpain

Personalised recommendations