Machine Learning

, Volume 8, Issue 1, pp 87–102 | Cite as

On the Handling of Continuous-Valued Attributes in Decision Tree Generation

  • Usama M. Fayyad
  • Keki B. Irani


We present a result applicable to classification learning algorithms that generate decision trees or rules using the information entropy minimization heuristic for discretizing continuous-valued attributes. The result serves to give a better understanding of the entropy measure, to point out that the behavior of the information entropy heuristic possesses desirable properties that justify its usage in a formal sense, and to improve the efficiency of evaluating continuous-valued attributes for cut value selection. Along with the formal proof, we present empirical results that demonstrate the theoretically expected reduction in evaluation effort for training data sets from real-world domains.

Induction empirical concept learning decision trees information entropy minimization discretization classification 


  1. Breiman, L., Friedman, J.H., Olshen, R.A., & Stone, C.J. (1984). Classification and regression trees. Monterey, CA: Wadsworth & Brooks.Google Scholar
  2. Cheng, J., Fayyad, U.M., Irani, K.B., & Qian, Z. (1988). Improved decision trees: A generalized version of ID3. Proceedings of the Fifth International Conference on Machine Learning (pp. 100–108). San Mateo, CA: Morgan Kaufmann.Google Scholar
  3. Clark, P. & Niblett, T. (1989). The CN2 induction algorithm. Machine Learning, 3, 261–284.Google Scholar
  4. Fayyad, U.M. & Irani, K.B. (1990). What should be minimized in a decision tree? Proceedings of the Eighth National Conference on Artificial Intelligence AAAI-90 (pp. 749–754). Cambridge, MA: MIT Press.Google Scholar
  5. Fayyad, U.M. & Irani, K.B. (1991). A machine learning algorithm (GID3*) for automated knowledge acquisition: Improvements and extensions. (General Motors Research Report CS-634). Warren, MI: GM Research Labs.Google Scholar
  6. Fayyad, U.M. (1991). On the induction of decision trees for multiple concept learning. Doctoral dissertation, EECS Department, The University of Michigan.Google Scholar
  7. Fisher, R.A. (1936). The use of multiple measurements in taxonomic problems. Annual Eugenics, 7, Part II, 179–188.Google Scholar
  8. Gelfand, S., Ravishankar, C. & Delp, E. (1991). An iterative growing and pruning algorithm for classification tree design. IEEE Transactions on Pattern Analysis and Machine Intelligence, 13:2, 163–174.Google Scholar
  9. Irani, K.B., Cheng, J., Fayyad, U.M., & Qian, Z. (1990). Applications of machine learning techniques in semiconductor manufacturing. Proceedings of The S.P.I.E. Conference on Applications of Artificial Intelligence VIII (pp. 956–965). Bellingham, WA: SPIE: The International Society for Optical Engineers.Google Scholar
  10. Lewis, P.M. (1962). The characteristic selection problem in recognition systems. IRE Transactions on Information Theory, IT-8, 171–178.Google Scholar
  11. Luenberger, D.G. (1973). Introduction to linear and nonlinear programming. Reading, MA: Addison-Wesley.Google Scholar
  12. Quinlan, J.R. (1986). Induction of decision trees. Machine Learning 1, 81–106.Google Scholar
  13. Quinlan, J.R. (1990). Probabilistic decision trees. In Y. Kodratoff & R. Michalski (Eds.), Machine learning: An artificial intelligence approach, volume III. San Mateo, CA: Morgan Kaufmann.Google Scholar

Copyright information

© Kluwer Academic Publishers 1992

Authors and Affiliations

  • Usama M. Fayyad
    • 1
    • 2
  • Keki B. Irani
  1. 1.Artificial Intelligence Laboratory, Electrical Engineering and Computer Science DepartmentThe University of MichiganAnn Arbor
  2. 2.AI Group, M/S 525-3660, Jet Propulsion LabCalifornia Institute of TechnologyPasadena

Personalised recommendations