Abstract
This article presents an incremental algorithm for inducing decision trees equivalent to those formed by Quinlan's nonincremental ID3 algorithm, given the same training instances. The new algorithm, named ID5R, lets one apply the ID3 induction process to learning tasks in which training instances are presented serially. Although the basic tree-building algorithms differ only in how the decision trees are constructed, experiments show that incremental training makes it possible to select training instances more carefully, which can result in smaller decision trees. The ID3 algorithm and its variants are compared in terms of theoretical complexity and empirical behavior.
Article PDF
Similar content being viewed by others
References
Barto, A. G. (1985). Learning by statistical cooperation of self-interested neuron-like computing elements. Human Neurobiology, 4, 229–256.
Breiman, L., Friedman, J. H., Olshen, R. A., and Stone, C. J. (1984). Classification and regression trees. Belmont, CA: Wadsworth International Group.
Buchanan, B. G., and Mitchell, T. M. (1978). Model-directed learning of production rules. In D. A. Waterman and F. Hayes-Roth (Eds.), Pattern-directed inference systems. New York: Academic Press.
Cockett, J. R. B. (1987). Discrete decision theory: Manipulations. Theoretical Computer Science, 54, 215–236.
Fisher, D. H. (1987). Knowledge acquisition via incremental conceptual clustering. Machine Learning, 2, 139–172.
Gallant, S. I. (1988). Connectionist expert systems. Communications of the ACM, 31, 152–169.
Lewis, P. M. (1962). The characteristic selection problem in recognition systems. IRE Transactions on Information Theory, IT-8, 171–178.
Michalski, R. S., and Chilausky, R. L. (1980). Learning by being told and learning from examples: An experimental comparison of the two methods of knowledge acquisition in the context of developing an expert system for soybean disease diagnosis. Policy Analysis and Information Systems, 4, 125–160.
Michalski, R. S. (1983). A theory and methodology of inductive learning. In R. S. Michalski, J. G. Carbonell, and T. M. Mitchell (Eds.), Machine learning: An artificial intelligence approach. San Mateo, CA: Morgan Kaufmann.
Mitchell, T. M. (1978). Version spaces: An approach to concept learning. Doctoral dissertation, Department of Electrical Engineering, Stanford University, Palo Alto, CA.
Moret, B. M. E. (1982). Decision trees and diagrams. Computing Surveys, 14, 593–623.
Quinlan, J. R. (1983). Learning efficient classification procedures and their application to chess end games. In R. S. Michalski, J. G. Carbonell, and T. M. Mitchell (Eds.), Machine learning: An artificial intelligence approach. San Mateo, CA: Morgan Kaufmann.
Quinlan, J. R. (1986). Induction of decision trees. Machine Learning, 1, 81–106.
Quinlan, J. R. (1988). An empirical comparison of genetic and decision-tree classifiers. Proceedings of the Fifth International Conference on Machine Learning (pp. 135–141). Ann Arbor, MI: Morgan Kaufmann.
Schlimmer, J. C., and Fisher, D. (1986). A case study of incremental concept induction. Proceedings of the Fifth National Conference on Artificial Intelligence (pp. 496–501). Philadelphia, PA: Morgan Kaufmann.
Schlimmer, J. C., and Granger, R. H., Jr. (1986). Incremental learning from noisy data. Machine Learning, 1, 317–354.
Shannon, C. E. (1948). A mathematical theory of communication. Bell System Technical Journal, 27, 379–423.
Utgoff, P. E. (1988). ID5: An incremental ID3. Proceedings of the Fifth International Conference on Machine Learning (pp. 107–120). Ann Arbor, MI: Morgan Kaufmann.
Utgoff, P. E. (1989). Improved training via incremental learning. Proceedings of the Sixth International Workshop on Machine Learning. Ithaca, NY: Morgan Kaufmann.
Vere, S. A. (1980). Multilevel counterfactuals for generalizations of relational concepts and productions. Artificial Intelligence, 14, 138–164.
Watanabe, S. (1985). Pattern recognition: Human and mechanical. New York: Wiley and Sons.
Wilson, S. W. (1987). Classifier systems and the animal problem. Machine Learning, 2, 199–228.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Utgoff, P.E. Incremental Induction of Decision Trees. Machine Learning 4, 161–186 (1989). https://doi.org/10.1023/A:1022699900025
Issue Date:
DOI: https://doi.org/10.1023/A:1022699900025