A Further Comparison of Simplification Methods for Decision-Tree Induction
This paper presents an empirical investigation of eight well-known simplification methods for decision trees induced from training data. Twelve data sets are considered to compare both the accuracy and the complexity of simplified trees. The computation of optimally pruned trees is used in order to give a clear definition of bias of the methods towards overpruning and underpruning. The results indicate that the simplification strategies which exploit an independent pruning set do not perform better than the others. Furthermore, some methods show an evident bias towards either underpruning or overpruning.
Unable to display preview. Download preview PDF.
- [BFOS84]L. Breiman, J. Friedman, R. Olshen, and C. Stone. Classification and regression trees. Wadsworth International, Belmont, CA, 1984.Google Scholar
- [BN92]W. Buntine and T. Niblett. A further comparison of splitting rules for decision-tree induction. Machine Learning, 8:75–85, 1992.Google Scholar
- [CB91]B. Cestnik and I. Bratko. On estimating probabilities in tree pruning. In Proceedings of the EWSL-91, pages 138–150, 1991.Google Scholar
- [EMS93]F. Esposito, D. Malerba, and G. Semeraro. Decision tree pruning as a search in the state space. In P. Brazdil, editor, Machine Learning: ECML-93. Springer-Verlag, Berlin, 1993.Google Scholar
- [Fis92]D. H. Fisher. Pessimistic and optimistic induction. Department of Computer Science, Vanderbilt University, 1992.Google Scholar
- [MA94]P. M. Murphy and D. W. Aha. UCI repository of machine learning databases [machine-readable data repository]. Department of Information and Computer Science, University of California, Irvine, 1994.Google Scholar
- [NB86]T. Niblett and I. Bratko. Learning decision rules in noisy domains. In Proceedings of Expert Systems 86, Cambridge, 1986. Cambridge University Press.Google Scholar
- [Nib87]T. Niblett. Constructing decision trees in noisy domains. In I. Bratko and N. Lavrac, editors, Progress in Machine Learning. Sigma Press, Wilmslow, 1987.Google Scholar
- [Qui93]J. R. Quinlan. C.1.5: Programs for Machine Learning. Morgan Kaufmann, San Mateo, 1993.Google Scholar
- [Sch93]C. Schaffer. Overfitting avoidance as bias. Machine Learning, 10:153–178, 1993.Google Scholar