Advances in Web Intelligence and Data Mining pp 285-294 | Cite as
Look-Ahead Mechanism Integration in Decision Tree Induction Models
Abstract
Most of decision tree induction algorithms use a greedy splitting criterion. One of the possible solutions to avoid this greediness is looking ahead to make better splits. Look-Ahead has not been used in most decision tree methods primarily because of its high computational complexity and its questionable contribution to predictive accuracy. In this paper we describe a new Look-Ahead approach to induction of decision tree models. We present a computationally efficient algorithm which evaluates quality of subtrees of variable-depth in order to determine the best split attribute out of a set of candidate attributes with a splitting criterion statistically indifferent from the best one.
Preview
Unable to display preview. Download preview PDF.
References
- [1]Breslow, L. A. and Aha, D. W. 1997. “Simplifying decision trees: A survey.” Knowl. Eng. Rev. 12,1 (Jan. 1997), 1–40.CrossRefGoogle Scholar
- [2]J. R. Quinlan. “C4.5: Programs for Machine Learning.” Morgan Kaufmann Publishers Inc., 1993.Google Scholar
- [3]Murthy, S. and Salzberg, S. (1995). “Lookahead and pathology in decision tree induction, in C. S.” Mellish, ed., ‘Proceedings of the 14th International Joint Conference on Artificial Intelligence’, Morgan Kaufmann, pp. 1025–1031.Google Scholar
- [4]T. Elomaa. “On look-ahead and pathology in decision tree learning.” Journal of Experimental and Theoretical Artificial Intelligence 17,1–2 (Jan. 2005) 19–33. With T. Malinen.Google Scholar
- [5]T. Elomaa, T. Malinen. “On Lookahead Heuristics in Decision Tree Learning.” IS MIS 2003: 445–453.Google Scholar
- [6]Mitchell, T.M. “Machine Learning.” McGraw-Hill, New York (1997).MATHGoogle Scholar
- [7]T. G. Dietterich and E. B. Kong. “Machine learning bias, statistical bias, and statistical variance of decision tree algorithms.” Technical report, Department of Computer Science, Oregon State University, 1995.Google Scholar
- [8]Breiman, L., Friedman, J. H., Olshen, R. A., and Stone, C. J. (1984). “Classification and Regression Trees.” Wadsworth, Belmont, CAGoogle Scholar
- [9]J. R. Quinlan and R. M. Cameron-Jones. “Oversearching and layered search in empirical learning.” In IJCAI’95, pages 1019–1024. Morgan Kaufmann, 1995.Google Scholar
- [11]Esmeir, S. and Markovitch, S. 2004. “Lookahead-based algorithms for anytime induction of decision trees.” In Proceedings of the Twenty-First international Conference on Machine Learning (Banff, Alberta, Canada, July 04–08, 2004). ICML’ 04, vol. 69. ACM Press, New York, NY, 33.CrossRefGoogle Scholar
- [12]I. Witten and E. Frank. “Data Mining: Practical Machine Learning Tools and Techniques, 2nd Edition.” Morgan Kaufmann, ISBN 0120884070, 20Google Scholar
- [13]Oded Z. Maimon, Mark Last, Knowledge Discovery and Data Mining: The Info-Fuzzy Network (Ifn) Methodology, Kluwer Academic Publishers, Norwell, MA, 2001Google Scholar