A Comparative Study of Sparse Associative Memories
- 172 Downloads
We study various models of associative memories with sparse information, i.e. a pattern to be stored is a random string of 0s and 1s with about \(\log N\) 1s, only. We compare different synaptic weights, architectures and retrieval mechanisms to shed light on the influence of the various parameters on the storage capacity.
KeywordsNeural networks Associative memory Sparse patterns Storage capacity Exponential inequalities
Mathematics Subject ClassificationPrimary: 82C32 60K35 Secondary: 68T05 92B20
We are very grateful to two anonymous referees for a very careful reading of a first version of the manuscript and valuable remarks that helped to improve its readability significantly.
- 11.Jarollahi, H., Gripon, V., Onizawa, N., Gross, W.J.: Algorithm and architecture for a low-power content-addressable memory based on sparse-clustered networks. IEEE Trans. Very Large Scale Integr. Syst. 27(2), 375–387 (2016)Google Scholar
- 12.Jarollahi, H., Onizawa, N., Gripon, V., Gross, W.J.: Algorithm and architecture of fully-parallel associative memories based on sparse clustered networks. J. Signal Process. Syst. 76(3), 235–247 (2014)Google Scholar
- 22.Palm, G.: Neural associative memories and sparse coding. Neural Netw., 37(0), 165–171 (2013) (Twenty-fifth Anniversay Commemorative Issue)Google Scholar
- 25.Yao, Z., Gripon, V., Rabbat, M.: A GPU-based Associative Memory using Sparse Neural Networks. In: Proceedings of the PCNN-14 conference, pp. 688–692 (2014)Google Scholar