New Options for Hoeffding Trees
Hoeffding trees are state-of-the-art for processing high-speed data streams. Their ingenuity stems from updating sufficient statistics, only addressing growth when decisions can be made that are guaranteed to be almost identical to those that would be made by conventional batch learning methods. Despite this guarantee, decisions are still subject to limited lookahead and stability issues. In this paper we explore Hoeffding Option Trees, a regular Hoeffding tree containing additional option nodes that allow several tests to be applied, leading to multiple Hoeffding trees as separate paths. We show how to control tree growth in order to generate a mixture of paths, and empirically determine a reasonable number of paths. We then empirically evaluate a spectrum of Hoeffding tree variations: single trees, option trees and bagged trees. Finally, we investigate pruning. We show that on some datasets a pruned option tree can be smaller and more accurate than a single tree.
Unable to display preview. Download preview PDF.
- 2.Ali, K.: Learning Probabilistic Relational Concept Descriptions. PhD thesis, University of California, Irvine (1996), http://www.isle.org/~ali/phd/thesis.ps.Z
- 3.Blake, C.L., Merz, C.J.: UCI repository of machine learning databases (1998)Google Scholar
- 5.Buntine, W.: Learning classification trees. In: Hand, D.J. (ed.) Artificial Intelligence frontiers in statistics, pp. 182–201. Chapman & Hall, London (1993)Google Scholar
- 6.Domingos, P., Hulten, G.: Mining high-speed data streams. Knowledge Discovery and Data Mining, 71–80 (2000)Google Scholar
- 9.Kohavi, R., Kunz, C.: Option decision trees with majority votes. In: Fisher, D. (ed.) Machine Learning. Proceedings of the Fourteenth International Conference, Morgan Kaufmann, San Francisco (1997)Google Scholar
- 10.Oza, N.C., Russell, S.: Online bagging and boosting. In: Artificial Intelligence and Statistics 2001, pp. 105–112. Morgan Kaufmann, San Francisco (2001)Google Scholar