Mathematical aspects of outer-product asynchronous content-addressable memories

Conclusions

We have given evidence by mathematical analysis and by example that the construction of CAM's as quasineural networks based on the twin principles of an outer-product weight matrix and a random asynchronous single-neuron dynamics encounters two obstacles to good performance which appear to be inherent. It is desirable to have the stored memory vectors of a CAM as mutually far apart as possible in order to have unambiguous retrieval with as large a fraction of initial errors (minrad) as possible. For a given number,m, of memory vectors to be stored this requires that their dimension,n, be larger thanm and that the memory vectors be nearly mutually orthogonal (except for complementary pairs). In HOCAM's, it does not seem possible to have both a large minrad and an efficient ratiom/n. Attempts to increasem/n are likely to introduce extraneous fixed-points which reduce minrad appreciably. We have demonstrated this phenomenon in several cases for a particular mode of constructing CAM's of arbitrary size which have a desirable spacing between memory vectors. We conjecture that it is present also in HOCAM's having a random selection of memory vectors. (A mathematical proof of this conjecture now seems possible.) This may account for the rather pessimistic results on capacity obtained by mathematical analysis here and in Cottrell (1988), by a probabilistic analysis in Posner (1987) and by simulation in Hopfield (1982). Further, in Cottrell (1988) there is evidence that outer-product weights are near optimal with respect to minrad, so that otherW may not improve matters.

We have left to another paper a study of other approaches to content-addressable memories of which we are aware, but which are not focused on asynchronous dynamics; e.g. computer CAM's as in Kohonen (1977) and biological memory models as in Little (1974); Palm (1980) and Little and Shaw (1978). We have not considered the learning, or adaptive, aspects of CAM's. However, insofar as learning is Hebbian and leads to outer-product weights, our analysis has implications for the effectiveness of learned weights, as may be inferred from our results on ambiguous retrieval.