Dynamics of learning and generalization in a binary perceptron model

Abstract

Learning from examples by perceptron withN binary synaptic couplings is investigated within dynamic mean field theory. This applies to learning by simulated annealing, which is a polynomial algorithm for finite cooling rates. For examples created by a teacher perceptron of the same type, a discontinuous freezing transition occurs for a training set of size αN with α<1.58 at a temperature where the entropy is still positive. The resulting perceptrons have finite training and generalization error. For α>1.58 the couplings of the teacher are found by the above process. This work extends previous investigations on a binary perceptron trained with random patterns.