Alternate Models of Learning

Abstract

Up to now, we have examined a more or less standard model of learning, which has three characteristic features:

• 1. The “algorithm” used to map the data into the hypothesis is viewed merely as some function mapping an appropriate “data space” into the hypothesis class. In particular, no restrictions are placed on the nature of this function, for example, requiring that the function be efficiently computable.

• 2. The data that forms the input to the algorithm is assumed to be generated at random according to some (possibly unknown) probability measure. In particular, the learner is “passive,” and does not have the option of choosing the next input to the oracle, with a view towards speeding up the learning process.

• 3. The efficacy of learning, as measured by the quantity r(m,ε) defined in (3.2.1), is essentially a worst-case estimate, since a supremum is taken both with respect to the target concept T as well as the probability measure P. This definition of the speed of learning does not cater to the situation where there exists a prior probability distribution on the target concepts themselves, and a learning algorithm works reasonably well for “most” target concepts.