An analysis of the equilibria of neural networks with linear interconnections

Abstract

In this paper, we analyse the equilibria of neural networks which consist of a set of sigmoid nonlinearities with linear interconnections,without assuming that the interconnections are symmetric or that there are no self-interactions. By eliminating these assumptions, we are able to study the effects of imperfect implementation on the behaviour of Hopfield networks. If one views the neural network as evolving on the openn-dimensional hypercubeH = (0, 1)ⁿ, we have the following conclusions as the neural characteristics become steeper and steeper: (i) There is at most one equilibrium in any compact subset ofH, and under mild assumptions this equilibrium is unstable. In fact, the dimension of the stable manifold of this equilibrium is the same as the number of eigenvalues of the interconnection matrix with negative real parts. (ii) There might be some equilibria in the faces ofH, and under mild conditions these are always unstable. Moreover, it is easy to compute the dimension of the stable manifold of each such equilibrium. (iii) A systematic procedure is given for determining which corners of the hypercubeH contain equilibria, and it is shown that all equilibria in the corners ofH are asymptotically stable.