Ensemble Methods for Noise Elimination in Classification Problems
Ensemble methods combine a set of classifiers to construct a new classifier that is (often) more accurate than any of its component classifiers. In this paper, we use ensemble methods to identify noisy training examples. More precisely, we consider the problem of mislabeled training examples in classification tasks, and address this problem by pre-processing the training set, i.e. by identifying and removing outliers from the training set. We study a number of filter techniques that are based on well-known ensemble methods like cross-validated committees, bagging and boosting. We evaluate these techniques in an Inductive Logic Programming setting and use a first order decision tree algorithm to construct the ensembles.
Unable to display preview. Download preview PDF.
- 2.M. Bongard. Pattern Recognition. Spartan Books, 1970.Google Scholar
- 6.Y. Freund and R.E. Schapire. Experiments with a new boosting algorithm. In L. Saitta, editor, Proceedings of the Thirteenth International Conference on Machine Learning, pages 148–156. Morgan Kaufmann, 1996.Google Scholar
- 8.G.H. John. Robust decision trees: Removing outliers from databases. In U.M. Fayyad and R. Uthurusamy, editors, Proceedings of the First International Conference on Knowledge Discovery and Data Mining, pages 174–179. AAAI Press, 1995.Google Scholar
- 9.N. Lavrač and S. Džeroski. Inductive Logic Programming: Techniques and Applications. Ellis Horwood, 1994.Google Scholar
- 10.R.S. Michalski and J.B. Larson. Inductive inference of VL decision rules. Paper presented at Workshop in Pattern-Directed Inference Systems, Hawaii, 1977. SIGART Newsletter, ACM, 63, 38–44.Google Scholar
- 11.J.R. Quinlan. C4.5: Programs for Machine Learning. Morgan Kaufmann series in machine learning. Morgan Kaufmann, 1993.Google Scholar
- 12.S. Verbaeten. Identifying mislabeled training examples in ILP classification problems. In M. Wiering and W. de Back, editors, Twelfth Dutch-Belgian Conference on Machine Learning, pages 1–8, 2002.Google Scholar
- 13.S. Verbaeten and A. Van Assche. Ensemble methods for noise elimination in classification problems. Technical report, Department of Computer Science, K.U.Leuven, Belgium, http://www.cs.kuleuven.ac.be/publicaties/rapporten/cw/CW358.abs.html, 2003.Google Scholar