Skip to main content
SpringerLink
Log in

Sparse Coding and Selected Applications

  • Fachbeitrag
  • Published:
KI - Künstliche Intelligenz Aims and scope Submit manuscript

Abstract

Sparse coding has become a widely used framework in signal processing and pattern recognition. After a motivation of the principle of sparse coding we show the relation to Vector Quantization and Neural Gas and describe how this relation can be used to generalize Neural Gas to successfully learn sparse coding dictionaries. We explore applications of sparse coding to image-feature extraction, image reconstruction and deconvolution, and blind source separation.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

References

  1. Aharon M, Elad M, Bruckstein A (2006) K-SVD: an algorithm for designing overcomplete dictionaries for sparse representation. IEEE Trans Signal Process 54(11):4311–4322

    Article  Google Scholar 

  2. Barlow HB (1961) Possible principles underlying the transformation of sensory messages. Sens Commun pp 217–234

  3. Bell A, Sejnowski T (1997) The “independent components” of natural scenes are edge filters. Vis Res 37(23):3327–3338

    Article  Google Scholar 

  4. Bruckstein AM, Donoho DL, Elad M (2009) From sparse solutions of systems of equations to sparse modeling of signals and images. SIAM Rev 51(1):34–81

    Article  MathSciNet  MATH  Google Scholar 

  5. Candès EJ, Tao T (2005) Decoding by linear programming. IEEE Trans Inf Theory 51(12):4203–4215

    Article  Google Scholar 

  6. Chen SS, Donoho DL, Saunders MA (1998) Atomic decomposition by basis pursuit. SIAM J Sci Comput 20(1):33–61

    Article  MathSciNet  Google Scholar 

  7. Comon P (1994) Independent component analysis, a new concept? Signal Process 36(3):287–314

    Article  MATH  Google Scholar 

  8. Davis G, Mallat S, Avellaneda M (1997) Greedy adaptive approximation. Constr Approx 13:57–89

    MathSciNet  MATH  Google Scholar 

  9. Deerwester S, Dumais S, Furnas G, Landauer T, Harshman R (1990) Indexing by latent semantic analysis. J Am Soc Inf Sci 41(6):391–407

    Article  Google Scholar 

  10. Ding C, Li T, Peng W (2006) Nonnegative matrix factorization and probabilistic latent semantic indexing: equivalence chi-square statistic, and a hybrid method. In: Proceedings of the national conference on artificial intelligence, vol 21, p 342

    Google Scholar 

  11. Donoho DL (2006) Compressed sensing. IEEE Trans Inf Theory 52(4):1289–1306

    Article  MathSciNet  Google Scholar 

  12. Elad M, Yavneh I (2009) A plurality of sparse representations is better than the sparsest one alone. IEEE Trans Inf Theory 55:4701–4714

    Article  MathSciNet  Google Scholar 

  13. Engan K, Aase SO, Hakon Husoy J (1999) Method of optimal directions for frame design. In: ICASSP’99: proceedings of the acoustics, speech, and signal processing, pp 2443–2446. 1999

    Google Scholar 

  14. Field DJ (1994) What is the goal of sensory coding? Neural Comput 6(4):559–601

    Article  MathSciNet  Google Scholar 

  15. Haykin S, Chen Z (2005) The cocktail party problem. Neural Comput 17(9):1875–1902

    Article  Google Scholar 

  16. Hofmann T (1999) Probabilistic latent semantic indexing. In: Proceedings of the 22nd annual international ACM SIGIR conference on research and development in information retrieval. ACM, New York, pp 50–57

    Chapter  Google Scholar 

  17. Hoyer P (2002) Non-negative sparse coding. In: Proceedings of the IEEE workshop on neural networks for signal processing XII, pp 557–565

    Chapter  Google Scholar 

  18. Labusch K, Barth E, Martinetz T (2008) Simple method for high-performance digit recognition based on sparse coding. IEEE Trans Neural Netw 19(11):1985–1989

    Article  Google Scholar 

  19. Labusch K, Barth E, Martinetz T (2009) Approaching the time dependent cocktail party problem with online sparse coding neural gas. In: Advances in self-organizing maps—WSOM 2009, vol 5629, pp 145–153

    Chapter  Google Scholar 

  20. Labusch K, Barth E, Martinetz T (2009) Demixing jazz-music: sparse coding neural gas for the separation of noisy overcomplete sources. Neural Netw World 19(5):561–579

    Google Scholar 

  21. Labusch K, Barth E, Martinetz T (2009) Sparse coding neural gas: learning of overcomplete data representations. Neurocomputing 72(7–9):1547–1555

    Article  Google Scholar 

  22. Labusch K, Barth E, Martinetz T (2011) Robust and fast learning of sparse codes with stochastic gradient descent. IEEE Trans Sel Top Signal Process 5(5):1048–1060

    Article  Google Scholar 

  23. Labusch K, Barth E, Martinetz T (2011) Soft-competitive learning of sparse codes and its application to image reconstruction. Neurocomputing 74(9):1418–1428

    Article  Google Scholar 

  24. LeCun Y MNIST handwritten digit database, NEC research institute. http://yann.lecun.com/exdb/mnist/

  25. Lee H, Battle A, Raina R, Ng AY (2007) Efficient sparse coding algorithms. Adv Neural Inf Process Syst 19:801–808

    Google Scholar 

  26. Lewicki M et al. (2002) Efficient coding of natural sounds. Nat Neurosci 5(4):356–363

    Article  Google Scholar 

  27. Martinetz T, Berkovich S, Schulten K (1993) “Neural-gas” network for vector quantization and its application to time-series prediction. IEEE Trans Neural Netw 4(4):558–569

    Article  Google Scholar 

  28. Martinetz T, Schulten K (1991) A “neural-gas network” learns topologies. Artif Neural Netw I:397–402

    Google Scholar 

  29. Olshausen BA, Field DJ (1996) Emergence of simple-cell receptive field properties by learning a sparse code for natural images. Nature 381:607–609

    Article  Google Scholar 

  30. Paatero P, Tapper U (1994) Positive matrix factorization: a non-negative factor model with optimal utilization of error estimates of data values. Environmetrics 5(2):111–126

    Article  Google Scholar 

  31. Pati Y, Rezaiifar R, Krishnaprasad P (1993) Orthogonal matching pursuit: recursive function approximation with applications to wavelet decomposition. In: Proceedings of the 27th annual asilomar conference on signals, systems, and computers

    Google Scholar 

  32. Rebollo-Neira L, Lowe D (2002) Optimized orthogonal matching pursuit approach. IEEE Signal Process Lett 9(4):137–140

    Article  Google Scholar 

  33. Shashanka M, Raj B, Smaragdis P (2007) Sparse overcomplete decomposition for single channel speaker separation. In: IEEE international conference on acoustics, speech, and signal processing, vol 2, pp 641–644

    Google Scholar 

  34. Villmann T, Hammer B (2009) Functional principal component learning using oja’s method and Sobolev norms. In: Advances in self-organizing maps—WSOM 2009, pp 325–333

    Chapter  Google Scholar 

  35. Zetzsche C, Barth E, Wegmann B (1993) The importance of intrinsically two-dimensional image features in biological vision and picture coding. In: Digital images and human vision, pp 109–138

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Institut für Neuro- und Bioinformatik, Universität Lübeck, Ratzeburger Allee 160, 23562, Lübeck, Germany

    Jens Hocke, Erhardt Barth & Thomas Martinetz

  2. Acrolinx, Rosenstraße 2, 10178, Berlin, Germany

    Kai Labusch

Authors
  1. Jens Hocke
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kai Labusch
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Erhardt Barth
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Thomas Martinetz
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jens Hocke.

Additional information

Supported by the DFG, grant number MA 2401/2-1.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Hocke, J., Labusch, K., Barth, E. et al. Sparse Coding and Selected Applications. Künstl Intell 26, 349–355 (2012). https://doi.org/10.1007/s13218-012-0197-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13218-012-0197-0

Keywords

Search

Navigation