Abstract
We briefly review the “sparse coding” principle employed in the sensory information processing system of mammals and focus on the phenomenon that such principle is realized through over-complete representation strategy in primary sensory cortical areas (V1). Considering the lack of quantitative analysis of how many gains in sparsenality the over-complete representation strategy brings in neuroscience, in this paper, we give a quantitative analysis from the viewpoint of nonlinear approximation. The result shows that the over-complete strategy can provide sparser representation than the complete strategy.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Olshausen, B.A., Field, D.J.: Sparse Coding of Sensory Inputs. Current Opinion in Neurobiology 14, 481–487 (2004)
Willshaw, D.J., Buneman, O.P., Longuet-Higgins, H.C.: Nonholographic Associative Memory. Nature 222, 960–962 (1969)
Barlow, H.B.: Possible Principles Underlying the Transformation of Sensory Messages. In: Rosenblith, W.A. (ed.) Sensory Communication, pp. 217–234. MIT Press, Cambridge (1961)
Attwell, D., Laughlin, S.B.: An Energy Budget for Signaling in The Grey Matter of The Brain. J. Cereb. Blood Flow Metab. 21, 1133–1145 (2001)
Lennie, P.: The Cost of Cortical Computation. Curr. Biol. 13, 493–497 (2003)
Olshausen, B.A.: Principles of Image Representation in Visual Cortex. In: Chalupa, L.M., Werner, J.S., Boston, M.A. (eds.) The Visual Neurosciences, pp. 1603–1615. MIT Press, Cambridge (2003)
Lee, K.S., Pedersen, D., Mumford: The Nonlinear Statistics of High-contrast Patches in Natural Images. Int. J. Comput. Vis. 54, 83–103 (2003)
Roweis, S.T., Saul, L.L.: Nonlinear Dimensionality Reduction By Locally Linear Embedding. Science 290, 2323–2326 (2000)
Wiskott, L., Sejnowski, L.: Slow Feature Analysis: Unsupervised Learning of Invariances. Neural Comput. 14, 715–770 (2002)
Olshausen, B.A., Field, D.J.: Sparse Coding With An Overcomplete Basis Set: A Strategy Employed By V1? Vision Res. 37, 3311–3325 (1997)
Lewicki, M.S., Sejnowski, T.J.: Learning Overcomplete Representations. Neural Comput. 12, 337–365 (2000)
De Vore, R.A.: Nonlinear Approximation. Acta Numerica. Cambridge University Press, Cambridge (1998)
Donoho, D.L., Elad, M.: Optimally Sparse Representation in General (Non-orthogonal) Dictionaries Via Minimization. Proc. Nat. Aca. Sci. 100, 2197–2202 (2003)
Feuer, N.A.: On Sparse Representations in Pairs of Bases. IEEE Trans. Inform. Theory 49, 1579–1581 (2003)
Starck, J.L., Elad, M., Donoho, D.L.: Image Decomposition Via the Compination of Sparse Representations And A Variational Approach. Submitted to IEEE. IP (2004)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2005 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Tan, S., Zhang, X., Wang, S., Jiao, L. (2005). Image Representation in Visual Cortex and High Nonlinear Approximation. In: Wang, J., Liao, X., Yi, Z. (eds) Advances in Neural Networks – ISNN 2005. ISNN 2005. Lecture Notes in Computer Science, vol 3496. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11427391_8
Download citation
DOI: https://doi.org/10.1007/11427391_8
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-25912-1
Online ISBN: 978-3-540-32065-4
eBook Packages: Computer ScienceComputer Science (R0)