Abstract
This paper proposes a method to improve ID5R, an incremental TDIDT algorithm. The new method evaluates the quality of attributes selected at the nodes of a decision tree and estimates a minimum number of steps for which these attributes are guaranteed such a selection. This results in reducing overheads during incremental learning. The method is supported by theoretical analysis and experimental results.
Article PDF
Similar content being viewed by others
Avoid common mistakes on your manuscript.
References
Breiman, L., Friedman, J.H., Olshen, R.A., and Stone, C.J. (1984). Classification and Regression Trees. Wadsworth, Belmont, CA.
Brodley, C.E. and Utgoff, P.E. (1995). Multivariate decision trees. Machine Learning, 19:45–77.
Catlett, J. (1992). Peepholing: Choosing attributes effectively for megainduction. In Proceedings of the 9th International Workshop on Machine Learning, pages 49–54, Aberdeen, Scotland.
Cockett, J.R.B. (1987). Discrete decision theory: Manipulations. Theoretical Computer Science, 54:215–236.
Cockett, J.R.B. and Zhu, Y. (1989). A new incremental technique for decision trees with thresholds. Proceedings of the SPIE, 1095:804–811.
de Mántaras, R.L. (1991). A distance-based attribute selection measure for decision tree induction. Machine Learning, 4:81–92.
Fisher, D. (1987). Knowledge acquisition via incremental conceptual clustering. Machine Learning, 2:139–172.
Hyafil, L. and Rivest, R.L. (1976) Constructing optimal binary decision trees is NP-complete. Information Processing Letters, 5(1):15–17.
Moret, B.M.E. (1982). Decision trees and diagrams. ACM Computing Surveys, 14:593–623.
Murphy, P. and Aha, D. (1995). UCI repository of machine learning databases. Machine readable data repository, (ftp to) ics.uci.edu, (cd to) pub/machine-learning-databases, University of California, Irvine.
Musick, R., Catlett, J., and Russell, S. (1993). Decision theoretic subsampling for induction on large databases. In Proceedings of the 10th International Conference on Machine Learning, pages 212–219, Amherst, MA.
Naumov, G.E. (1991). NP-completeness of problems of construction of optimal decision trees. Soviet Physics: Doklady, 36(4):270–271.
Pagallo, G. (1989). Learning DNF by decision trees. In Proceedings of the 11th Joint International Conference on Artificial Intelligence, volume 1, pages 639–644, Detroit, MI.
Quinlan, J.R. (1983). Learning efficient classification procedures and their application to chess end games. In R.S. Michalski, J.G. Carbonell, and T.M. Mitchell, editors, Machine Learning: an Artificial Intelligence Approach, pages 463–482. Tioga Publishing, Palo Alto, CA.
Quinlan, J.R. (1986). Induction of decision trees. Machine Learning, 1(1):81–106.
Quinlan, J.R. (1993). C4.5 Programs for Machine Learning. Morgan Kaufmann, San Mateo, CA.
Schlimmer, J.C. and Fisher, D. (1986). A case study of incremental concept induction. In Proceedings of the 5th National Conference on Artificial Intelligence, pages 496–501, Philadelphia, PA. Morgan Kaufman.
Utgoff, P.E. (1988). ID5: An incremental ID3. In Proceedings of the 5th International Conference on Machine Learning, pages 107–120, Ann Arbor, MI.
Utgoff, P.E. (1989). Incremental induction of decision trees. Machine Learning, 4(2):161–186.
Utgoff, P.E. (1994). An improved algorithm for incremental induction of decision trees. In W. Cohen and H. Hirsh, editors, Proceedings of the 11th International Conference on Machine Learning, New Brunswick, NJ. Morgan Kaufmann.
van de Velde, W. (1990). Incremental induction of topologically minimal decision trees. In Proceedings of the 7th International Conference on Machine Learning, pages 66–74, Austin, TX.
Rights and permissions
About this article
Cite this article
Kalles, D., Morris, T. Efficient Incremental Induction of Decision Trees. Machine Learning 24, 231–242 (1994). https://doi.org/10.1023/A:1018295910873
Issue Date:
DOI: https://doi.org/10.1023/A:1018295910873