The goals of this paper are: 1) the introduction of a shift-invariant sparse coding model together with learning rules for this model; 2) the comparison of this model to the traditional sparse coding model; and 3) the analysis of some limitations of the newly proposed approach. To evaluate the model we will show that it can learn features from a toy problem as well as note-like features from a polyphonic piano recording. We further show that the shift-invariant model can help in overcoming some of the limitations of the traditional model which occur when learning less functions than are present in the true generative model. We finally show a limitation of the proposed model for problems in which mixtures of continuously shifted functions are used.
- Sparse Code
- Blind Source Separation
- Neural Information Processing System
- Inference Process
- Sparse Component
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
This is a preview of subscription content, access via your institution.
Tax calculation will be finalised at checkout
Purchases are for personal use onlyLearn about institutional subscriptions
Unable to display preview. Download preview PDF.
Abdallah, S.: Towards Music Perception by Redundancy Reduction and Unsupervised Learning in Probabilistic Models. PhD thesis, King’s College London (February 2003)
Olshausen, B.A., Millman, K.: Learning sparse codes with a mixture-of- Gaussians prior. Advances in Neural Information Processing Systems (2000)
Sallee, P., Olshausen, B.A.: Learning sparse multiscale image representations. Advances in Neural Information Processing Systems (2003)
Olshausen, B.A., Field, D.J.: Emergence of simple-cell receptive field properties by learning a sparse code for natural images. Nature (381), 607–609 (1995)
Lewicki, M.S., Sejnowski, T.J.: Learning overcomplete representations. Neural Computation (12), 337–365 (2000)
Kreutz-Delgado, K., Murray, J.F., Rao, B.D., Engan, K., Lee, T.-W., Sejnowski, T.J.: Dictionary learning algorithms for sparse representation. Neural Computation 15, 349–396 (2003)
Figueiredo, M.A.T., Jain, A.K.: Bayesian learning of sparse classifiers. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition - CVPR 2001, Hawaii, December 2001, vol. 1, pp. 35–41. IEEE, Los Alamitos (2001)
Chen, S.S., Donoho, D.L., Saunders, M.A.: Atomic decomposition by basis pursuit. SIAM Journal of Scientific Computing 20(1), 33–61 (1998)
Li, Y., Cichocki, A., Amari, S.: Sparse component analysis for blind source separation with less sensors than sources. In: ICA 2003, pp. 89–94 (2003)
Blumensath, T., Davies, M.: Unsupervised learning of sparse and shift-invariant decompositions of polyphonic music. In: IEEE International Conference on Acoustics, Speech, and Signal Processing (2004)
Editors and Affiliations
© 2004 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Blumensath, T., Davies, M. (2004). On Shift-Invariant Sparse Coding. In: Puntonet, C.G., Prieto, A. (eds) Independent Component Analysis and Blind Signal Separation. ICA 2004. Lecture Notes in Computer Science, vol 3195. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-30110-3_152
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-23056-4
Online ISBN: 978-3-540-30110-3
eBook Packages: Springer Book Archive