Abstract
We present a result applicable to classification learning algorithms that generate decision trees or rules using the information entropy minimization heuristic for discretizing continuous-valued attributes. The result serves to give a better understanding of the entropy measure, to point out that the behavior of the information entropy heuristic possesses desirable properties that justify its usage in a formal sense, and to improve the efficiency of evaluating continuous-valued attributes for cut value selection. Along with the formal proof, we present empirical results that demonstrate the theoretically expected reduction in evaluation effort for training data sets from real-world domains.
Article PDF
Similar content being viewed by others
References
Breiman, L., Friedman, J.H., Olshen, R.A., & Stone, C.J. (1984).Classification and regression trees. Monterey, CA: Wadsworth & Brooks.
Cheng, J., Fayyad, U.M., Irani, K.B., & Qian, Z. (1988). Improved decision trees: A generalized version of ID3.Proceedings of the Fifth International Conference on Machine Learning (pp. 100–108). San Mateo, CA: Morgan Kaufmann.
Clark, P. & Niblett, T. (1989). The CN2 induction algorithm.Machine Learning, 3, 261–284.
Fayyad, U.M. & Irani, K.B. (1990). What should be minimized in a decision tree?Proceedings of the Eighth National Conference on Artificial Intelligence AAAI-90 (pp. 749–754). Cambridge, MA: MIT Press.
Fayyad, U.M. & Irani, K.B. (1991). A machine learning algorithm (GID3*) for automated knowledge acquisition: Improvements and extensions. (General Motors Research Report CS-634). Warren, MI: GM Research Labs.
Fayyad, U.M. (1991).On the induction of decision trees for multiple concept learning. Doctoral dissertation, EECS Department, The University of Michigan.
Fisher, R.A. (1936). The use of multiple measurements in taxonomic problems.Annual Eugenics, 7, Part II, 179–188.
Gelfand, S., Ravishankar, C. & Delp, E. (1991). An iterative growing and pruning algorithm for classification tree design.IEEE Transactions on Pattern Analysis and Machine Intelligence, 13:2, 163–174.
Irani, K.B., Cheng, J., Fayyad, U.M., & Qian, Z. (1990). Applications of machine learning techniques in semiconductor manufacturing.Proceedings of The S.P.I.E. Conference on Applications of Artificial Intelligence VIII (pp. 956–965). Bellingham, WA: SPIE: The International Society for Optical Engineers.
Lewis, P.M. (1962). The characteristic selection problem in recognition systems.IRE Transactions on Information Theory, IT-8, 171–178.
Luenberger, D.G. (1973).Introduction to linear and nonlinear programming. Reading, MA: Addison-Wesley.
Quinlan, J.R. (1986). Induction of decision trees.Machine Learning 1, 81–106.
Quinlan, J.R. (1990). Probabilistic decision trees. In Y. Kodratoff & R. Michalski (Eds.),Machine learning: An artificial intelligence approach, volume III. San Mateo, CA: Morgan Kaufmann.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Fayyad, U.M., Irani, K.B. On the handling of continuous-valued attributes in decision tree generation. Mach Learn 8, 87–102 (1992). https://doi.org/10.1007/BF00994007
Issue Date:
DOI: https://doi.org/10.1007/BF00994007