Ensemble Methods for Noise Elimination in Classification Problems

  • Sofie Verbaeten
  • Anneleen Van Assche
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 2709)


Ensemble methods combine a set of classifiers to construct a new classifier that is (often) more accurate than any of its component classifiers. In this paper, we use ensemble methods to identify noisy training examples. More precisely, we consider the problem of mislabeled training examples in classification tasks, and address this problem by pre-processing the training set, i.e. by identifying and removing outliers from the training set. We study a number of filter techniques that are based on well-known ensemble methods like cross-validated committees, bagging and boosting. We evaluate these techniques in an Inductive Logic Programming setting and use a first order decision tree algorithm to construct the ensembles.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    H. Blockeel and L. De Raedt. Top-down induction of first order logical decision trees. Artificial Intelligence, 101(1–2):285–297, 1998.zbMATHCrossRefMathSciNetGoogle Scholar
  2. 2.
    M. Bongard. Pattern Recognition. Spartan Books, 1970.Google Scholar
  3. 3.
    C.E. Brodley and M.A. Friedl. Identifying mislabeled training data. Journal of Artificial Intelligence Research, 11:131–167, 1999.zbMATHGoogle Scholar
  4. 4.
    T.G. Dietterich. Ensemble methods in machine learning. In J. Kittler and F. Roli, editors, Multiple Classifier Systems, First International Workshop, volume 1857 of Lecture Notes in Computer Science, pages 1–15. Springer, 2000.CrossRefGoogle Scholar
  5. 5.
    T.G. Dietterich. An experimental comparison of three methods for constructing ensembles of decision trees: Bagging, boosting, and randomization. Machine Learning, 40(2):139–157, 2000.CrossRefGoogle Scholar
  6. 6.
    Y. Freund and R.E. Schapire. Experiments with a new boosting algorithm. In L. Saitta, editor, Proceedings of the Thirteenth International Conference on Machine Learning, pages 148–156. Morgan Kaufmann, 1996.Google Scholar
  7. 7.
    D. Gamberger, N. Lavrač, and S. Džeroski. Noise detection and elimination in data preprocessing: experiments in medical domains. Applied Artificial Intelligence, 14:205–223, 2000.CrossRefGoogle Scholar
  8. 8.
    G.H. John. Robust decision trees: Removing outliers from databases. In U.M. Fayyad and R. Uthurusamy, editors, Proceedings of the First International Conference on Knowledge Discovery and Data Mining, pages 174–179. AAAI Press, 1995.Google Scholar
  9. 9.
    N. Lavrač and S. Džeroski. Inductive Logic Programming: Techniques and Applications. Ellis Horwood, 1994.Google Scholar
  10. 10.
    R.S. Michalski and J.B. Larson. Inductive inference of VL decision rules. Paper presented at Workshop in Pattern-Directed Inference Systems, Hawaii, 1977. SIGART Newsletter, ACM, 63, 38–44.Google Scholar
  11. 11.
    J.R. Quinlan. C4.5: Programs for Machine Learning. Morgan Kaufmann series in machine learning. Morgan Kaufmann, 1993.Google Scholar
  12. 12.
    S. Verbaeten. Identifying mislabeled training examples in ILP classification problems. In M. Wiering and W. de Back, editors, Twelfth Dutch-Belgian Conference on Machine Learning, pages 1–8, 2002.Google Scholar
  13. 13.
    S. Verbaeten and A. Van Assche. Ensemble methods for noise elimination in classification problems. Technical report, Department of Computer Science, K.U.Leuven, Belgium,, 2003.Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2003

Authors and Affiliations

  • Sofie Verbaeten
    • 1
  • Anneleen Van Assche
    • 1
  1. 1.Department of Computer ScienceKatholieke Universiteit LeuvenHeverleeBelgium

Personalised recommendations