Robust Ensemble Learning for Data Mining

  • Gunnar Rätsch
  • Bernhard Schölkopf
  • Alexander Johannes Smola
  • Sebastian Mika
  • Takashi Onoda
  • Klaus-Robert Müller
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 1805)


We propose a new boosting algorithm which similarly to v-Support-Vector Classification allows for the possibility of a pre-specified fraction v of points to lie in the margin area or even on the wrong side of the decision boundary. It gives a nicely interpretable way of controlling the trade-off between minimizing training error and capacity. Furthermore, it can act as a filter for finding and selecting informative patterns from a database.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    L. Breiman. Prediction games and arcing algorithms. Technical Report 504, Statistics Department, University of California, December 1997.Google Scholar
  2. 2.
    H. Drucker, R. Schapire, and P. Simard. Boosting performance in neural networks. Int. Journal of Pattern Recognition and Artificial Intelligence, 7:705–719, 1993.CrossRefGoogle Scholar
  3. 3.
    Y. LeCun et al. Learning algorithms for classification: A comparison on handwritten digit recognition. Neural Networks, pages 261–276, 1995.Google Scholar
  4. 4.
    A. Grove and D. Schuurmans. Boosting in the limit: Maximizing the margin of learned ensembles. In Proc. of the 15th Nat. Conf. on AI, pages 692–699, 1998.Google Scholar
  5. 5.
    C. J. Merz and P. M. Murphy. UCI repository of machine learning databases, 1998. []. Irvine, CA.
  6. 6.
    J. R. Quinlan. Boosting first-order learning (invited lecture). Lecture Notes in Computer Science, 1160:143, 1996.Google Scholar
  7. 7.
    G. Rätsch, T. Onoda, and K.-R. Müller. Soft margins for AdaBoost. Technical Report NC-TR-1998-021, NeuroColt, 1998. To appear in Machine Learning.Google Scholar
  8. 8.
    G. Rätsch, B. Schökopf, A. Smola, S. Mika, T. Onoda, and K.-R. Müller. Robust ensemble learning. In A.J. Smola, P.L. Bartlett, B. Schölkopf, and D. Schuurmans, editors, Advances in Large Margin Classifiers, pages 207–219. MIT Press, Cambridge, MA, 1999.Google Scholar
  9. 9.
    R. Schapire, Y. Freund, P. L. Bartlett, and W. Sun Lee. Boosting the margin: A new explanation for the effectiveness of voting methods. Annals of Statistics, 26(5):1651–1686, 1998.zbMATHCrossRefMathSciNetGoogle Scholar
  10. 10.
    B. Schölkopf, A. Smola, R. C. Williamson, and P. L. Bartlett. New support vector algorithms. Neural Computation, 12:1083–1121, 2000.CrossRefGoogle Scholar
  11. 11.
    H. Schwenk and Y. Bengio. Training methods for adaptive boosting of neural networks. In Michael I. Jordan, Michael J. Kearns, and Sara A. Solla, editors, Advances in Neural Inf. Processing Systems, volume 10. The MIT Press, 1998.Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2000

Authors and Affiliations

  • Gunnar Rätsch
    • 1
  • Bernhard Schölkopf
    • 2
  • Alexander Johannes Smola
    • 3
  • Sebastian Mika
    • 1
  • Takashi Onoda
    • 4
  • Klaus-Robert Müller
    • 1
  1. 1.GMD FIRSTBerlinGermany
  2. 2.Microsoft ResearchCambridgeUK
  3. 3.Dep. of EngineeringANUCanberraAustralia
  4. 4.CIRL CRIEPITokyoJapan

Personalised recommendations