Augmented Spatial Pooling
Abstract
It is a widely held view in contemporary computational neuroscience that the brain responds to sensory input by producing sparse distributed representations. In this paper we investigate a brain-inspired spatial pooling algorithm that produces such sparse distributed representations by modelling the formation of proximal dendrites associated with neocortical minicolumns. In this approach, distributed representations are formed out of a competitive process of inter-column inhibition and subsequent learning. Specifically, we evaluate the performance of a recently proposed binary spatial pooling algorithm on a well-known benchmark of greyscale natural images. Our main contribution is to augment the algorithm to handle greyscale images, and to produce better quality encodings of binary images. We also show that the augmented algorithm produces superior population and lifetime kurtosis measures in comparison to a number of other well-known coding schemes.
Keywords
Independent Component Analysis Image Patch Sparse Code Active Input Proximal DendritePreview
Unable to display preview. Download preview PDF.
References
- 1.Chikkerur, S., Serre, T., Tan, C., Poggio, T.: What and where: A Bayesian inference theory of attention. Vision Research 50(22), 2233–2247 (2010)CrossRefGoogle Scholar
- 2.George, D., Hawkins, J.: A hierarchical Bayesian model of invariant pattern recognition in the visual cortex. In: Proceedings of the International Joint Conference on Neural Networks (IJCNN 2005), pp. 1812–1817 (2005)Google Scholar
- 3.Hawkins, J., Blakeslee, S.: On intelligence. Henry Holt, New York (2004)Google Scholar
- 4.Hyvärinen, A., Karhunen, J., Oja, E.: Independent Components Analysis. John Wiley and Sons, Inc., New York (2001)CrossRefGoogle Scholar
- 5.Kohonen, T.: Self-organization and associative memory. Springer, Berlin (1989)CrossRefzbMATHGoogle Scholar
- 6.Lee, T.S., Mumford, D.: Hierarchical Bayesian inference in visual cortex. Journal of the Optical Society of America A 20(7), 1434–1448 (2003)CrossRefGoogle Scholar
- 7.Mountcastle, V.B.: Introduction to the special issue on computation in cortical columns. Cerebral Cortex 13(1), 2–4 (2003)CrossRefGoogle Scholar
- 8.Numenta Inc.: Hierarchical temporal memory including HTM cortical learning algorithms. Tech. rep., Numenta, Inc, Palto Alto (2010), http://www.numenta.com/htm-overview/education/HTM_CorticalLearningAlgorithms.pdf
- 9.Olshausen, B.A.: Sparse codes and spikes. In: Rao, R.P.N., Olshausen, B.A., Lewicki, M.S. (eds.) Probabilistic Models of the Brain: Perception and Neural Function, pp. 257–272. MIT Press, Cambridge (2002)Google Scholar
- 10.Stuart, G., Spruston, N., Häusser, M.: Dendrites. OUP, New York (2008)Google Scholar
- 11.Willmore, B., Tolhurst, D.J.: Characterizing the sparseness of neural codes. Network: Computational Neural Systems 12, 255–270 (2001)CrossRefGoogle Scholar