From High Definition Image to Low Space Optimization

  • Micha Feigin
  • Dan Feldman
  • Nir Sochen
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6667)


Signal and image processing have seen in the last few years an explosion of interest in a new form of signal/image characterization via the concept of sparsity with respect to a dictionary. An active field of research is dictionary learning: Given a large amount of example signals/images one would like to learn a dictionary with much fewer atoms than examples on one hand, and much more atoms than pixels on the other hand. The dictionary is constructed such that the examples are sparse on that dictionary i.e each image is a linear combination of small number of atoms.

This paper suggests a new computational approach to the problem of dictionary learning. We show that smart non-uniform sampling, via the recently introduced method of coresets, achieves excellent results, with controlled deviation from the optimal dictionary. We represent dictionary learning for sparse representation of images as a geometric problem, and illustrate the coreset technique by using it together with the K–SVD method. Our simulations demonstrate gain factor of up to 60 in computational time with the same, and even better, performance. We also demonstrate our ability to perform computations on larger patches and high-definition images, where the traditional approach breaks down.


Sparsity dictionary learning K–SVD coresets 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Agarwal, P.K., Har-Peled, S., Varadarajan, K.R.: Approximating extent measures of points. Journal of the ACM 51(4), 606–635 (2004)CrossRefzbMATHMathSciNetGoogle Scholar
  2. 2.
    Agarwal, P.K., Har-Peled, S., Varadarajan, K.R.: Geometric approximations via coresets. Combinatorial and Computational Geometry - MSRI Publications 52, 1–30 (2005)zbMATHMathSciNetGoogle Scholar
  3. 3.
    Aharon, M., Elad, M., Bruckstein, A.: K-SVD: An Algorithm for Designing Overcomplete Dictionaries for Sparse Representation. IEEE Transactions on Signal Processing 54(11), 4311–4322 (2006)CrossRefzbMATHGoogle Scholar
  4. 4.
    Czumaj, A., Sohler, C.: Sublinear-time approximation algorithms for clustering via random sampling. Random Struct. Algorithms (RSA) 30(1-2), 226–256 (2007)CrossRefzbMATHMathSciNetGoogle Scholar
  5. 5.
    Elad, M., Aharon, M.: Image denoising via sparse and redundant representations over learned dictionaries. IEEE Trans. Image Processing 15(12), 3736–3745 (2006)CrossRefMathSciNetGoogle Scholar
  6. 6.
    Feldman, D., Fiat, A., Segev, D., Sharir, M.: Bi-criteria linear-time approximations for generalized k-mean/median/center. In: Proc. 23rd ACM Symp. on Computational Geometry (SOCG), pp. 19–26 (2007)Google Scholar
  7. 7.
    Feldman, D., Langberg, M.: A unified framework for approximating and clustering data (submitted, 2010) manuscriptGoogle Scholar
  8. 8.
    Feldman, D., Monemizadeh, M., Sohler, C.: A PTAS for k-means clustering based on weak coresets. In: Proc. 23rd ACM Symp. on Computational Geometry (SoCG), pp. 11–18 (2007)Google Scholar
  9. 9.
    Har-Peled, S.: Low rank matrix approximation in linear time (2006) manuscriptGoogle Scholar
  10. 10.
    Kreutz-Delgado, K., Murray, J.F., Rao, B.D., Engan, K., Lee, T.W., Sejnowski, T.J.: Dictionary learning algorithms for sparse representation. Neural Computation 15(2), 349–396 (2003)CrossRefzbMATHGoogle Scholar
  11. 11.
    Lesage, S., Gribonval, R., Bimbot, F., Benaroya, L.: Learning unions of orthonormal bases with thresholded singular value decomposition. In: IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2005, vol. 5, IEEE, Los Alamitos (2005)Google Scholar
  12. 12.
    Pati, Y.C., Rezaiifar, R., Krishnaprasad, P.S.: Orthogonal matching pursuit: Recursive function approximation with applications to wavelet decomposition. In: 1993 Conference Record of The Twenty-Seventh Asilomar Conference on Signals, Systems and Computers, pp. 40–44. IEEE, Los Alamitos (2002)Google Scholar
  13. 13.
  14. 14.
    Vapnik, V.N., Chervonenkis, A.Y.: On the uniform convergence of relative frequencies of events to their probabilities. Theory of Probability and its Applications 16(2), 264–280 (1971)CrossRefzbMATHGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Micha Feigin
    • 1
  • Dan Feldman
    • 2
  • Nir Sochen
    • 1
  1. 1.Deparment of MathematicsTel-Aviv UniversityTel-AvivIsrael
  2. 2.Center for Mathematics of InformationCalifornia Institute of TechnologyPasadenaUSA

Personalised recommendations