Machine Learning

, Volume 41, Issue 3, pp 295–313

Enlarging the Margins in Perceptron Decision Trees

  • Kristin P. Bennett
  • Nello Cristianini
  • John Shawe-Taylor
  • Donghui Wu
Article

DOI: 10.1023/A:1007600130808

Cite this article as:
Bennett, K.P., Cristianini, N., Shawe-Taylor, J. et al. Machine Learning (2000) 41: 295. doi:10.1023/A:1007600130808

Abstract

Capacity control in perceptron decision trees is typically performed by controlling their size. We prove that other quantities can be as relevant to reduce their flexibility and combat overfitting. In particular, we provide an upper bound on the generalization error which depends both on the size of the tree and on the margin of the decision nodes. So enlarging the margin in perceptron decision trees will reduce the upper bound on generalization error. Based on this analysis, we introduce three new algorithms, which can induce large margin perceptron decision trees. To assess the effect of the large margin bias, OC1 (Journal of Artificial Intelligence Research, 1994, 2, 1–32.) of Murthy, Kasif and Salzberg, a well-known system for inducing perceptron decision trees, is used as the baseline algorithm. An extensive experimental study on real world data showed that all three new algorithms perform better or at least not significantly worse than OC1 on almost every dataset with only one exception. OC1 performed worse than the best margin-based method on every dataset.

capacity controldecision treesperceptronlearning theorylearning algorithm
Download to read the full article text

Copyright information

© Kluwer Academic Publishers 2000

Authors and Affiliations

  • Kristin P. Bennett
    • 1
  • Nello Cristianini
    • 2
  • John Shawe-Taylor
    • 3
  • Donghui Wu
    • 4
  1. 1.Dept of Mathematical SciencesRensselaer Polytechnic InstituteTroyUSA
  2. 2.Dept of Computer Science, Royal HollowayUniversity of London, EghamSurreyUK
  3. 3.Dept of Computer Science, Royal HollowayUniversity of London, EghamSurreyUK
  4. 4.Dept of Mathematical SciencesRensselaer Polytechnic InstituteTroyUSA