The Quest for a Dictionary

  • Michael Elad


A fundamental ingredient in the definition of Sparse-Land’s signals and its deployment to applications is the dictionary A. How can we wisely choose A to perform well on the signals in question? This is the topic of this chapter, and our emphasis is put on learning methods for dictionaries, based on a group of examples.


Sparse Representation Wavelet Packet Image Patch Sparse Code Blind Source Separation 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Further Reading

  1. 1.
    M. Aharon, M. Elad, and A.M. Bruckstein, K-SVD and its non-negative variant for dictionary design, Proceedings of the SPIE conference wavelets, Vol. 5914, July 2005.Google Scholar
  2. 2.
    M. Aharon, M. Elad, and A.M. Bruckstein, On the uniqueness of overcomplete dictionaries, and a practical way to retrieve them, Journal of Linear Algebra and Applications, 416(1):48–67, July 2006.zbMATHCrossRefMathSciNetGoogle Scholar
  3. 3.
    M. Aharon, M. Elad, and A.M. Bruckstein. K-SVD: An algorithm for designing of overcomplete dictionaries for sparse representation, IEEE Trans. on Signal Processing, 54(11):4311–4322, November 2006.CrossRefGoogle Scholar
  4. 4.
    E.J. Candès and D.L. Donoho, Recovering edges in ill-posed inverse problems: optimality of curvelet frames, Annals of Statistics, 30(3):784–842, 2000.Google Scholar
  5. 5.
    E.J. Candès and D.L. Donoho, New tight frames of curvelets and optimal representations of objects with piecewise-C2 singularities, Comm. Pure Appl. Math., 57:219–266, 2002.CrossRefGoogle Scholar
  6. 6.
    A. Cichocki, R. Zdunek, A.H. Phan, and S.I. Amari, Nonnegative Matrix and Tensor Factorizations Applications to Exploratory Multi-way Data Analysis and Blind Source Separation, Wiley, Tokyo, Japan, 2009.Google Scholar
  7. 7.
    V. Chandrasekaran, M. Wakin, D. Baron, R. Baraniuk, Surflets: a sparse representation for multidimensional functions containing smooth discontinuities, IEEE Symposium on Information Theory, Chicago, IL, 2004.Google Scholar
  8. 8.
    R.R. Coifman and M.V. Wickerhauser, Adapted waveform analysis as a tool for modeling, feature extraction, and denoising, Optical Engineering, 33(7):2170–2174, July 1994.CrossRefGoogle Scholar
  9. 9.
    M.N. Do and M. Vetterli, Rotation invariant texture characterization and retrieval using steerable wavelet-domain hidden Markov models, IEEE Trans. On Multimedia, 4(4):517–527, December 2002.CrossRefGoogle Scholar
  10. 10.
    M.N. Do and and M. Vetterli, The finite ridgelet transform for image representation, IEEE Trans. On Image Processing, 12(1):16–28, 2003.CrossRefMathSciNetGoogle Scholar
  11. 11.
    M.N. Do and and M. Vetterli, Framing pyramids, IEEE Trans. On Signal Processing, 51(9):2329–2342, 2003.CrossRefMathSciNetGoogle Scholar
  12. 12.
    M.N. Do and M. Vetterli, The contourlet transform: an efficient directional multiresolution image representation, IEEE Trans. Image on Image Processing, 14(12):2091–2106, 2005.CrossRefMathSciNetGoogle Scholar
  13. 13.
    D.L. Donoho and V. Stodden, When does non-negative matrix factorization give a correct decomposition into parts? Advances in Neural Information Processing 16 (Proc. NIPS 2003), MIT Press, 2004.Google Scholar
  14. 14.
    M. Elad and M. Aharon, Image denoising via learned dictionaries and sparse representation, International Conference on Computer Vision and pattern Recognition, New-York, June 17–22, 2006.Google Scholar
  15. 15.
    M. Elad and M. Aharon, Image denoising via sparse and redundant representations over learned dictionaries, IEEE Trans. on Image Processing 15(12):3736–3745, December 2006.CrossRefMathSciNetGoogle Scholar
  16. 16.
    K. Engan, S.O. Aase, and J.H. Husoy, Multi-frame compression: Theory and design, EURASIP Signal Processing, 80(10):2121–2140, 2000.zbMATHGoogle Scholar
  17. 17.
    R. Eslami and H. Radha, The contourlet transform for image de-noising using cycle spinning, in Proceedings of Asilomar Conference on Signals, Systems, and Computers, pp. 1982–1986, November 2003.Google Scholar
  18. 18.
    R. Eslami and H. Radha, Translation-invariant contourlet transform and its application to image denoising, IEEE Trans. on Image Processing, 15(11):3362–3374, November 2006.CrossRefGoogle Scholar
  19. 19.
    A. Gersho and R.M. Gray, Vector Quantization And Signal Compression, Kluwer Academic Publishers, Dordrecht, Netherlands, 1992.zbMATHGoogle Scholar
  20. 20.
    G.H. Golub and C.F. Van Loan, Matrix Computations, Johns Hopkins Studies in Mathematical Sciences, Third edition, 1996.Google Scholar
  21. 21.
    P.O. Hoyer, Non-negative matrix factorization with sparseness constraints, Journal of Machine Learning Research, 1457–1469, 2004.Google Scholar
  22. 22.
    K. Kreutz-Delgado, J.F. Murray, B.D. Rao, K. Engan, T-W, Lee, and T.J. Sejnowski, Dictionary learning algorithms for sparse representation, Neural Computation, 15(2)349–396, 2003.zbMATHCrossRefGoogle Scholar
  23. 23.
    D. Lee and H. Seung, Learning the parts of objects by non-negative matrix factorization, Nature, 788–791, 1999.Google Scholar
  24. 24.
    S. Lesage, R. Gribonval, F. Bimbot, and L. Benaroya, Learning unions of orthonormal bases with thresholded singular value decomposition, ICASSP 2005 (IEEE Conf. on Acoustics, Speech and Signal Proc.).Google Scholar
  25. 25.
    M.S. Lewicki and B.A. Olshausen, A probabilistic framework for the adaptation and comparison of image codes, Journal of the Optical Society of America A: Optics, Image Science and Vision, 16(7):1587–1601, 1999.CrossRefGoogle Scholar
  26. 26.
    M.S. Lewicki and T.J. Sejnowski, Learning overcomplete representations, Neural Computation, 12:337–365, 2000.CrossRefGoogle Scholar
  27. 27.
    Y. Li, A. Cichocki, and S.-i. Amari, Analysis of sparse representation and blind source separation, Neural Computation, 16(6):1193–1234, 2004.zbMATHCrossRefGoogle Scholar
  28. 28.
    F.G. Meyer, A. Averbuch, and J.O. Stromberg, Fast adaptive wavelet packet image compression, IEEE Trans. on Image Processing, 9(5):792–800, 2000.CrossRefGoogle Scholar
  29. 29.
    F.G. Meyer and R.R. Coifman, Brushlets: a tool for directional image analysis and image compression, Applied and Computational Harmonic Analysis, 4:147–187, 1997.zbMATHCrossRefMathSciNetGoogle Scholar
  30. 30.
    B.A. Olshausen and D.J. Field, Natural image statistics and efficient coding, Network - Computation in Neural Systems, 7(2):333–339, 1996.CrossRefGoogle Scholar
  31. 31.
    B.A. Olshausen and B.J. Field, Emergence of simple-cell receptive field properties by learning a sparse code for natural images Nature, 381(6583):607–609, 1996.CrossRefGoogle Scholar
  32. 32.
    B.A. Olshausen and B.J. Field, Sparse coding with an overcomplete basis set: A strategy employed by V1? Vision Research, 37(23):3311–3325, 1997.CrossRefGoogle Scholar
  33. 33.
    E.P. Simoncelli, W.T. Freeman, E.H. Adelson, and D.J. Heeger, Shiftable multiscale transforms, IEEE Trans. on Information Theory, 38(2):587–607, 1992.CrossRefMathSciNetGoogle Scholar
  34. 34.
    J.-L. Starck, E.J. Candès, and D.L. Donoho, The curvelet transform for image denoising, IEEE Trans. on Image Processing, 11:670–684, 2002.CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2010

Authors and Affiliations

  1. 1.Computer Science DepartmentThe Technion – Israel Institute of TechnologyHaifaIsrael

Personalised recommendations