Abstract
The role of abstaining from prediction by component classifiers in rule ensembles is discussed. We consider bagging and Ivotes approaches to construct such ensembles. In our proposal, component classifiers are based on unordered sets of rules with a classification strategy that solves ambiguous matching of the object’s description to the rules. We propose to induce rule sets by a sequential covering algorithm and to apply classification strategies using either rule support or discrimination measures. We adopt the classification strategies to abstaining by not using partial matching. Another contribution of this paper is an experimental evaluation of the effect of the abstaining on performance of ensembles. Results of comprehensive comparative experiments show that abstaining rule sets classifiers improve the accuracy, however this effect is more visible for bagging than for Ivotes.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Aijun, A.: Learning classification rules from data. Computers and Mathematics with Applications 45, 737–748 (2003)
Breiman, L.: Bagging predictors. Machine Learning 24(2), 123–140 (1996)
Breiman, L.: Pasting small votes for classification in large databases and on-line. Machine Learning 36, 85–103 (1999)
Cohen, W., Singer, Y.: A simple, fast and effective rule learner. In: Proc. of the 16th National Conference on Artificial Intelligence AAAI 1999, pp. 335–342 (1999)
Freund, Y., Schapire, R.E., Singer, Y., Warmuth, M.K.: Using and combining predictors that specialize. In: Proceedings of the 29th ACM symposium on Theory of Computing, pp. 334–343 (1997)
Grzymala-Busse, J.W.: Managing uncertainty in machine learning from examples. In: Proc. 3rd Int. Symp. in Intelligent Systems, pp. 70–84 (1994)
Kononenko, I., Kukar, M.: Machine Learning and Data Mining. Horwood Pub., England (2007)
Kuncheva, L.: Combining Pattern Classifiers. Methods and Algorithms. Wiley, Chichester (2004)
Mease, D., Wyner, A.: Evidence Contrary to the Statistical View of Boosting. Journal of Machine Learning Research 9, 131–156 (2008)
Pietraszek, T.: Optimizing abstaining classifiers using ROC analysis. In: Proc. of the 22nd Int. Conf. on Machine Learning, ICML 2005, pp. 665–672 (2005)
Quinlan, J.R.: C4.5: Programs for Machine Learning. Morgan Kaufmann, San Francisco (1992)
Rucket, U., Kramer, S.: Towards tight bounds for rule learning. In: Proc. of the 21st Int. Conf. on Machine Learning, ICML 2004, pp. 711–718 (2004)
Stefanowski, J.: The rough set based rule induction technique for classification problems. In: Proc. of the 6th European Conf. on Intelligent Techniques and Soft Computing EUFIT 1998, pp. 109–113 (1998)
Stefanowski, J.: On combined classifiers, rule induction and rough sets. In: Peters, J.F., Skowron, A., Düntsch, I., Grzymała-Busse, J.W., Orłowska, E., Polkowski, L. (eds.) Transactions on Rough Sets VI. LNCS, vol. 4374, pp. 329–350. Springer, Heidelberg (2007)
Weiss, S.M., Indurkhya, N.: Lightweight rule induction. In: Proc. of the 17st Int. Conf. on Machine Learning, ICML 2000, pp. 1135–1142 (2000)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2009 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Błaszczyński, J., Stefanowski, J., Zając, M. (2009). Ensembles of Abstaining Classifiers Based on Rule Sets. In: Rauch, J., Raś, Z.W., Berka, P., Elomaa, T. (eds) Foundations of Intelligent Systems. ISMIS 2009. Lecture Notes in Computer Science(), vol 5722. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-04125-9_41
Download citation
DOI: https://doi.org/10.1007/978-3-642-04125-9_41
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-04124-2
Online ISBN: 978-3-642-04125-9
eBook Packages: Computer ScienceComputer Science (R0)