Skip to main content

On Shift-Invariant Sparse Coding

Part of the Lecture Notes in Computer Science book series (LNCS,volume 3195)


The goals of this paper are: 1) the introduction of a shift-invariant sparse coding model together with learning rules for this model; 2) the comparison of this model to the traditional sparse coding model; and 3) the analysis of some limitations of the newly proposed approach. To evaluate the model we will show that it can learn features from a toy problem as well as note-like features from a polyphonic piano recording. We further show that the shift-invariant model can help in overcoming some of the limitations of the traditional model which occur when learning less functions than are present in the true generative model. We finally show a limitation of the proposed model for problems in which mixtures of continuously shifted functions are used.


  • Sparse Code
  • Blind Source Separation
  • Neural Information Processing System
  • Inference Process
  • Sparse Component

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

This is a preview of subscription content, access via your institution.

Buying options

USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
USD   74.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Abdallah, S.: Towards Music Perception by Redundancy Reduction and Unsupervised Learning in Probabilistic Models. PhD thesis, King’s College London (February 2003)

    Google Scholar 

  2. Olshausen, B.A., Millman, K.: Learning sparse codes with a mixture-of- Gaussians prior. Advances in Neural Information Processing Systems (2000)

    Google Scholar 

  3. Sallee, P., Olshausen, B.A.: Learning sparse multiscale image representations. Advances in Neural Information Processing Systems (2003)

    Google Scholar 

  4. Olshausen, B.A., Field, D.J.: Emergence of simple-cell receptive field properties by learning a sparse code for natural images. Nature (381), 607–609 (1995)

    Google Scholar 

  5. Lewicki, M.S., Sejnowski, T.J.: Learning overcomplete representations. Neural Computation (12), 337–365 (2000)

    Google Scholar 

  6. Kreutz-Delgado, K., Murray, J.F., Rao, B.D., Engan, K., Lee, T.-W., Sejnowski, T.J.: Dictionary learning algorithms for sparse representation. Neural Computation 15, 349–396 (2003)

    CrossRef  Google Scholar 

  7. Figueiredo, M.A.T., Jain, A.K.: Bayesian learning of sparse classifiers. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition - CVPR 2001, Hawaii, December 2001, vol. 1, pp. 35–41. IEEE, Los Alamitos (2001)

    Google Scholar 

  8. Chen, S.S., Donoho, D.L., Saunders, M.A.: Atomic decomposition by basis pursuit. SIAM Journal of Scientific Computing 20(1), 33–61 (1998)

    CrossRef  MathSciNet  Google Scholar 

  9. Li, Y., Cichocki, A., Amari, S.: Sparse component analysis for blind source separation with less sensors than sources. In: ICA 2003, pp. 89–94 (2003)

    Google Scholar 

  10. Blumensath, T., Davies, M.: Unsupervised learning of sparse and shift-invariant decompositions of polyphonic music. In: IEEE International Conference on Acoustics, Speech, and Signal Processing (2004)

    Google Scholar 

Download references

Author information

Authors and Affiliations


Editor information

Editors and Affiliations

Rights and permissions

Reprints and Permissions

Copyright information

© 2004 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Blumensath, T., Davies, M. (2004). On Shift-Invariant Sparse Coding. In: Puntonet, C.G., Prieto, A. (eds) Independent Component Analysis and Blind Signal Separation. ICA 2004. Lecture Notes in Computer Science, vol 3195. Springer, Berlin, Heidelberg.

Download citation

  • DOI:

  • Published:

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-23056-4

  • Online ISBN: 978-3-540-30110-3

  • eBook Packages: Springer Book Archive