Improving Adaptive Boosting with k-Cross-Fold Validation

  • Joaquín Torres-Sospedra
  • Carlos Hernández-Espinosa
  • Mercedes Fernández-Redondo
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4113)


As seen in the bibliography, Adaptive Boosting (Adaboost) is one of the most known methods to increase the performance of an ensemble of neural networks. We introduce a new method based on Adaboost where we have applied Cross-Validation to increase the diversity of the ensemble. We have used Cross-Validation over the whole learning set to generate an specific training set and validation set for each network of the committee. We have tested Adaboost and Crossboost with seven databases from the UCI repository. We have used the mean percentage of error reduction and the mean increase of performance to compare both methods, the results show that Crossboost performs better.


Ensemble Method Error Reduction Single Network Wisconsin Breast Cancer Previous Network 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Tumer, K., Ghosh, J.: Error Correlation and Error Reduction in Ensemble Classifiers. Connection Science 8(3-4), 385–403 (1996)CrossRefGoogle Scholar
  2. 2.
    Raviv, Y., Intratorr, N.: Bootstrapping with Noise: An Effective Regularization Technique. Connection Science, Special issue on Combining Estimators 8, 356–372 (1996)Google Scholar
  3. 3.
    Hernandez, E.C., Fernandez, R.M., Torres, S.J.: Ensembles of Multilayer Feedforward for Classification Problems. In: Pal, N.R., Kasabov, N., Mudi, R.K., Pal, S., Parui, S.K. (eds.) ICONIP 2004. LNCS, vol. 3316, pp. 744–749. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  4. 4.
    Hernandez, E.C., Torres, S.J., Fernandez, R.M.: New Experiments on Ensembles of Multilayer Feedforward for Classification Problems. In: Proceedings of International Conference on Neural Networks, IJCNN 2005, Montreal, Canada, pp. 1120–1124 (2005)Google Scholar
  5. 5.
    Verikas, A., Lipnickas, A., Malmqvist, K., Bacauskiene, M., Gelzinis, A.: Soft Combination of Neural Classifiers: A Comparative Study. Pattern Recognition Letters 20(4), 429–444 (1999)CrossRefGoogle Scholar
  6. 6.
    Freund, Y., Schapire, R.E.: Experiments with A New Boosting Algorithm. In: International Conference on Machine Learning, pp. 148–156 (1996)Google Scholar
  7. 7.
    Kuncheva, L., Whitaker, C.J.: Using Diversity with Three Variants of Boosting: Aggressive. In: Roli, F., Kittler, J. (eds.) MCS 2002. LNCS, vol. 2364. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  8. 8.
    Oza, N.C.: Boosting with Averaged Weight Vectors. In: Windeatt, T., Roli, F. (eds.) MCS 2003. LNCS, vol. 2709, pp. 15–24. Springer, Heidelberg (2003)CrossRefGoogle Scholar
  9. 9.
    Breiman, L.: Arcing Classifiers. The Annals of Statistics 26(3), 801–849 (1998)zbMATHCrossRefMathSciNetGoogle Scholar
  10. 10.
    Drucker, H., Cortes, C., Jackel, L.D., LeCun, Y., Vapnik, V.: Boosting and Other Ensemble Methods. Neural Computation 6(6), 1289–1301 (1994)zbMATHCrossRefGoogle Scholar
  11. 11.
    Newman, D.J., Hettich, S., Blake, C.L., Merz, C.J.: UCI Repository of Machine Learning Databases (1998)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Joaquín Torres-Sospedra
    • 1
  • Carlos Hernández-Espinosa
    • 1
  • Mercedes Fernández-Redondo
    • 1
  1. 1.Departamento de Ingenieria y Ciencia de los ComputadoresUniversitat Jaume ICastellonSpain

Personalised recommendations