Representational Capabilities and Learning of Bithreshold Neural Networks

Abstract

The paper deals with questions related to the ability of real-weighted bithreshold neurons and neural networks to solve the classification tasks. We study how many partitions of a finite set in n-dimensional vector space can be computed by using bithreshold neurons. The upper and lower bounds on the number of bithreshold dichotomies are proved. Our main theoretical result states that there exists $\mathrm {\Theta }\left( m^{n+1}\right)$ bithreshold dichotomies of the set of m points in general position in n-dimensional space. We also estimate the Vapnik-Chervonenkis dimension of the class of bithreshold neurons. The problem of the synthesis of 2-layer bithreshold neural network for binary classification is considered. We demonstrate that an arbitrary dichotomy of m points in general position in n-dimensional space can be realized by 2-layer neural network containing at most $\left\lceil m/(2n)\right\rceil$ bithreshold neurons in hidden layer and a single output threshold neuron. The learning algorithm with computational complexity $O\left( m{{n}^{2}}+{{m}^{2}} \right)$ is proposed for such networks. We give simulation results of the network performance on real data sets and discuss the influence of algorithm parameters on the quality of learning and the representational capabilities of bithreshold networks.