Machine Learning

, Volume 5, Issue 2, pp 197–227

The strength of weak learnability

  • Robert E. Schapire
Article

DOI: 10.1007/BF00116037

Cite this article as:
Schapire, R.E. Mach Learn (1990) 5: 197. doi:10.1007/BF00116037

Abstract

This paper addresses the problem of improving the accuracy of an hypothesis output by a learning algorithm in the distribution-free (PAC) learning model. A concept class islearnable (orstrongly learnable) if, given access to a source of examples of the unknown concept, the learner with high probability is able to output an hypothesis that is correct on all but an arbitrarily small fraction of the instances. The concept class isweakly learnable if the learner can produce an hypothesis that performs only slightly better than random guessing. In this paper, it is shown that these two notions of learnability are equivalent.

A method is described for converting a weak learning algorithm into one that achieves arbitrarily high accuracy. This construction may have practical applications as a tool for efficiently converting a mediocre learning algorithm into one that performs extremely well. In addition, the construction has some interesting theoretical consequences, including a set of general upper bounds on the complexity of any strong learning algorithm as a function of the allowed error ο.

Keywords

Machine learninglearning from exampleslearnability theoryPAC learningpolynomial-time identification
Download to read the full article text

Copyright information

© Kluwer Academic Publishers 1990

Authors and Affiliations

  • Robert E. Schapire
    • 1
  1. 1.MIT Laboratory for Computer ScienceCambridge