Generalization Performance of Classifiers in Terms of Observed Covering Numbers

  • John Shawe-Taylor
  • Robert C. Williamson
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 1572)

Abstract

It is known that the covering numbers of a function class on a double sample (length 2m) can be used to bound the generalization performance of a classifier by using a margin based analysis. In this paper we show that one can utilize an analogous argument in terms of the observed covering numbers on a single m-sample (being the actual observed data points). The significance of this is that for certain interesting classes of functions, such as support vector machines, there are new techniques which allow one to find good estimates for such covering numbers in terms of the speed of decay of the eigenvalues of a Gram matrix. These covering numbers can be much less than a priori bounds indicate in situations where the particular data received is “easy”. The work can be considered an extension of previous results which provided generalization performance bounds in terms of the VC-dimension of the class of hypotheses restricted to the sample, with the considerable advantage that the covering numbers can be readily computed, and they often are small.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Copyright information

© Springer-Verlag Berlin Heidelberg 1999

Authors and Affiliations

  • John Shawe-Taylor
    • 1
  • Robert C. Williamson
    • 2
  1. 1.Department of Computer Science, Royal HollowayUniversity of LondonEghamUK
  2. 2.Department of EngineeringAustralian National UniversityCanberraAustralia

Personalised recommendations