Information efficiency of the associative net at arbitrary coding rates
The associative net is a neural network model of associative memory that is unusual in having binary-valued connections between units. This net can work with high information efficiency, but only if the patterns to be stored are extremely sparse. In this paper we report how the efficiency of the net can be improved for more dense coding rates by using a partially-connected net. The information efficiency can be maintained at a high level over a 2–3 order of magnitude variation in the degree of pattern sparseness.
Unable to display preview. Download preview PDF.
- Amaral, D., Ishizuka, N., & Claiborne, B. (1990). Neurons, numbers and the hippocampal network. In Storm-Mathisen, J., Zimmer, J., & Ottersen, O. (Eds.), Progress in Brain Research, Vol.83, pp. 1–11. Elsevier Science Publishers B.V.Google Scholar
- Buckingham, J., & Willshaw, D. (1992). Performance characteristics of the associative net. Network, 3, 407–414.Google Scholar
- Buckingham, J., & Willshaw, D. (1993). On setting unit thresholds in an incompletely connected associative net. Network, 4, 441–459.Google Scholar
- Gardner, E. (1987). Maximum storage capacity in neural networks. Europhysics Letters, 4, 481.Google Scholar
- Graham, B., & Willshaw, D. (1995). Improving recall from an associative memory. Biol. Cybern., 72, 337–346.Google Scholar
- Graham, B., & Willshaw, D. (1994). Capacity and information efficiency of a brain-like associative net. In Tesauro, G., Touretzky, D., & Leen, T. (Eds.), Neural Information Processing Systems 7, pp. 513–520. MIT Press.Google Scholar
- Nadal, J.-P., & Toulouse, G. (1990). Information storage in sparsely coded memory nets. Network, 1, 61–74.Google Scholar
- Willshaw, D. (1971). Models of distributed associative memory. Ph.D. thesis, University of Edinburgh.Google Scholar
- Willshaw, D., Buneman, O., & Longuet-Higgins, H. (1969). Non-holographic associative memory. Nature, 222, 960–962.Google Scholar