Mutual Information and Topology 1: Asymmetric Neural Network

Abstract

An infinite range neural network works as an associative memory device if both the learning storage and attractor abilities are large enough. This work deals with the search of an optimal topology, varying the (small-world) parameters: the average connectivity γ ranges from the fully linked to a extremely diluted network; the randomness ω ranges from purely neighbor links to a completely random network. The network capacity is measured by the mutual information, MI, between patterns and retrieval states. It is found that MI is optimized at a certain value γ _o for a given 0 < ω< 1 if the network is asymmetric.