Computational Suspiciousness of Learning in Artificial Neural Nets

Abstract

In this paper we study the time complexity of learning in terms of continuous-parametric representations (CPR) of concept classes, that underly most of the connectionist approaches to machine learning. We introduce the notion of non-suspect concept classes in CPR. Suspiciousness essentially qualifies the shape of the error function associated to a CP representation when the learner adopts an optimization algorithm for searching in the parameter space. We show that a concept class F is non-suspect in a CPR only if F is efficiently learnable. Under some further assumptions, that are met for example in the case of the class of linearly separable patterns, we claim that there exists an optimal algorithm for learning non-suspect concept classes in the representation of multilayered neural networks, whose time complexity is Θ(mp), being m the number of training examples and p the dimension of the parameter space.