Rule Stacking: An Approach for Compressing an Ensemble of Rule Sets into a Single Classifier
In this paper, we present an approach for compressing a rule-based pairwise classifier ensemble into a single rule set that can be directly used for classification. The key idea is to re-encode the training examples using information about which of the original rules of the ensemble cover the example, and to use them for training a rule-based meta-level classifier. We not only show that this approach is more accurate than using the same rule learner at the base level (which could have been expected for such a variant of stacking), but also demonstrate that the resulting meta-level rule set can be straight-forwardly translated back into a rule set at the base level. Our key result is that the rule sets obtained in this way are of comparable complexity to those of the original rule learner, but considerably more accurate.
Unable to display preview. Download preview PDF.
- 2.Asuncion, A., Newman, D.J.: UCI machine learning repository (2007), http://www.ics.uci.edu/~mlearn/MLRepository.html
- 3.van den Bosch, A.: Using induced rules as complex features in memory-based language learning. In: Proceedings of the 2nd Workshop on Learning Language in Logic and the 4th Conference on Computational Natural Language Learning, pp. 73–78. Association for Computational Linguistics, Morristown (2000)CrossRefGoogle Scholar
- 9.Domingos, P.: Metacost: A general method for making classifiers cost-sensitive. In: Proceedings of the 5th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD 1999), pp. 155–164. ACM, San Diego (1999)Google Scholar
- 12.Fürnkranz, J.: Round robin classification. Journal of Machine Learning Research 2, 721–747 (2002), http://www.ai.mit.edu/projects/jmlr/papers/volume2/fuernkranz02a/html/ MathSciNetMATHGoogle Scholar
- 14.Loza Mencía, E., Fürnkranz, J.: Efficient multilabel classification algorithms for large-scale problems in the legal domain. In: Francesconi, E., Montemagni, S., Peters, W., Tiscornia, D. (eds.) Semantic Processing of Legal Texts. LNCS (LNAI), vol. 6036, pp. 192–215. Springer, Heidelberg (2010)CrossRefGoogle Scholar
- 15.Seewald, A.K.: How to make stacking better and faster while also taking care of an unknown weakness. In: Sammut, C., Hoffmann, A.G. (eds.) Proceedings of the 19th International Conference (ICML 2002), pp. 554–561. Morgan Kaufmann, Sydney (2002)Google Scholar