On Shift-Invariant Sparse Coding
The goals of this paper are: 1) the introduction of a shift-invariant sparse coding model together with learning rules for this model; 2) the comparison of this model to the traditional sparse coding model; and 3) the analysis of some limitations of the newly proposed approach. To evaluate the model we will show that it can learn features from a toy problem as well as note-like features from a polyphonic piano recording. We further show that the shift-invariant model can help in overcoming some of the limitations of the traditional model which occur when learning less functions than are present in the true generative model. We finally show a limitation of the proposed model for problems in which mixtures of continuously shifted functions are used.
KeywordsSparse Code Blind Source Separation Neural Information Processing System Inference Process Sparse Component
Unable to display preview. Download preview PDF.
- 1.Abdallah, S.: Towards Music Perception by Redundancy Reduction and Unsupervised Learning in Probabilistic Models. PhD thesis, King’s College London (February 2003)Google Scholar
- 2.Olshausen, B.A., Millman, K.: Learning sparse codes with a mixture-of- Gaussians prior. Advances in Neural Information Processing Systems (2000)Google Scholar
- 3.Sallee, P., Olshausen, B.A.: Learning sparse multiscale image representations. Advances in Neural Information Processing Systems (2003)Google Scholar
- 4.Olshausen, B.A., Field, D.J.: Emergence of simple-cell receptive field properties by learning a sparse code for natural images. Nature (381), 607–609 (1995)Google Scholar
- 5.Lewicki, M.S., Sejnowski, T.J.: Learning overcomplete representations. Neural Computation (12), 337–365 (2000)Google Scholar
- 9.Li, Y., Cichocki, A., Amari, S.: Sparse component analysis for blind source separation with less sensors than sources. In: ICA 2003, pp. 89–94 (2003)Google Scholar
- 10.Blumensath, T., Davies, M.: Unsupervised learning of sparse and shift-invariant decompositions of polyphonic music. In: IEEE International Conference on Acoustics, Speech, and Signal Processing (2004)Google Scholar