New Options for Hoeffding Trees

  • Bernhard Pfahringer
  • Geoffrey Holmes
  • Richard Kirkby
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4830)


Hoeffding trees are state-of-the-art for processing high-speed data streams. Their ingenuity stems from updating sufficient statistics, only addressing growth when decisions can be made that are guaranteed to be almost identical to those that would be made by conventional batch learning methods. Despite this guarantee, decisions are still subject to limited lookahead and stability issues. In this paper we explore Hoeffding Option Trees, a regular Hoeffding tree containing additional option nodes that allow several tests to be applied, leading to multiple Hoeffding trees as separate paths. We show how to control tree growth in order to generate a mixture of paths, and empirically determine a reasonable number of paths. We then empirically evaluate a spectrum of Hoeffding tree variations: single trees, option trees and bagged trees. Finally, we investigate pruning. We show that on some datasets a pruned option tree can be smaller and more accurate than a single tree.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Agrawal, R., Imielinski, T., Swami, A.: Database mining: A performance perspective. IEEE Transactions on Knowledge and Data Engineering 5(6), 914–925 (1993)CrossRefGoogle Scholar
  2. 2.
    Ali, K.: Learning Probabilistic Relational Concept Descriptions. PhD thesis, University of California, Irvine (1996),
  3. 3.
    Blake, C.L., Merz, C.J.: UCI repository of machine learning databases (1998)Google Scholar
  4. 4.
    Breiman, L.: Bagging predictors. Machine Learning 24(2), 123–140 (1996)zbMATHMathSciNetGoogle Scholar
  5. 5.
    Buntine, W.: Learning classification trees. In: Hand, D.J. (ed.) Artificial Intelligence frontiers in statistics, pp. 182–201. Chapman & Hall, London (1993)Google Scholar
  6. 6.
    Domingos, P., Hulten, G.: Mining high-speed data streams. Knowledge Discovery and Data Mining, 71–80 (2000)Google Scholar
  7. 7.
    Gama, J., Rocha, R., Medas, P.: Accurate decision trees for mining high-speed data streams. In: KDD 2003. Proceedings of the ninth ACM SIGKDD international conference on Knowledge discovery and data mining, pp. 523–528. ACM Press, New York (2003)CrossRefGoogle Scholar
  8. 8.
    Holmes, G., Kirkby, R., Pfahringer, B.: Stress-testing hoeffding trees. In: Jorge, A.M., Torgo, L., Brazdil, P.B., Camacho, R., Gama, J. (eds.) PKDD 2005. LNCS (LNAI), vol. 3721, Springer, Heidelberg (2005)CrossRefGoogle Scholar
  9. 9.
    Kohavi, R., Kunz, C.: Option decision trees with majority votes. In: Fisher, D. (ed.) Machine Learning. Proceedings of the Fourteenth International Conference, Morgan Kaufmann, San Francisco (1997)Google Scholar
  10. 10.
    Oza, N.C., Russell, S.: Online bagging and boosting. In: Artificial Intelligence and Statistics 2001, pp. 105–112. Morgan Kaufmann, San Francisco (2001)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2007

Authors and Affiliations

  • Bernhard Pfahringer
    • 1
  • Geoffrey Holmes
    • 1
  • Richard Kirkby
    • 1
  1. 1.Department of Computer Science, University of Waikato, HamiltonNew Zealand

Personalised recommendations