Dictionary Learning for Sparse Representations: A Pareto Curve Root Finding Approach

  • Mehrdad Yaghoobi
  • Mike E. Davies
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6365)


A new dictionary learning method for exact sparse representation is presented in this paper. As the dictionary learning methods often iteratively update the sparse coefficients and dictionary, when the approximation error is small or zero, algorithm convergence will be slow or non-existent. The proposed framework can be used in such a setting by gradually increasing the fidelity of the approximation. This technique has previously been used for the convex sparse representations. It has been extended here to the non-convex dictionary learning problem by allowing the dictionary be modified.


Sparse Representation Sparse Code Gradient Projection Dictionary Learning Gradient Projection Method 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Mallat, S., Zhang, Z.: Matching Pursuits with time frequency dictionaries. IEEE Trans. on Signal Processing 41(12), 3397–3415 (1993)zbMATHCrossRefGoogle Scholar
  2. 2.
    Chen, S.S., Donoho, D.L., Saunders, M.A.: Atomic Decomposition by Basis Pursuit. SIAM Journal on Scientific Computing 20(1), 33–61 (1998)CrossRefMathSciNetGoogle Scholar
  3. 3.
    Yaghoobi, M., Daudet, L., Davies, M.: Parametric Dictionary Design for Sparse Coding. IEEE Trans. on Signal Processing 57(12), 4800–4810 (2009)CrossRefGoogle Scholar
  4. 4.
    Olshausen, B.A., Field, D.J.: Sparse coding with an overcomplete basis set: a strategy employed by V1? Vision Research 37(23), 3311–3325 (1997)CrossRefGoogle Scholar
  5. 5.
    Lewicki, M.S., Sejnowski, T.J.: Learning Overcomplete Representations. Neural Comp. 12(2), 337–365 (2000)Google Scholar
  6. 6.
    Engan, K., Aase, S.O., Hakon-Husoy, J.: Method of optimal directions for frame design. In: International Conference on Acoustics, Speech, and Signal Processing (ICASSP), pp. 2443–2446 (1999)Google Scholar
  7. 7.
    Aharon, M., Elad, E., Bruckstein, A.M.: K-SVD: an algorithm for designing of over complete dictionaries for sparse representation. IEEE Trans. on Signal Processing 54(11), 4311–4322 (2006)CrossRefGoogle Scholar
  8. 8.
    Yaghoobi, M., Blumensath, T., Davies, M.: Dictionary Learning for Sparse Approximations with the Majorization Method. IEEE Trans. on Signal Processing 57(6), 2178–2191 (2009)CrossRefMathSciNetGoogle Scholar
  9. 9.
    Georgiev, P., Theis, F., Cichocki, A.: Sparse component analysis and blind source separation of underdetermined mixtures. IEEE Trans. on Neural Networks 16(4), 992–996 (2005)CrossRefGoogle Scholar
  10. 10.
    Aharon, M., Elad, M., Bruckstein, A.M.: On the uniqueness of overcomplete dictionaries and a practical way to retrieve them. Journal of Linear Algebra and Applications 416, 48–67 (2006)zbMATHCrossRefMathSciNetGoogle Scholar
  11. 11.
    Gribonval, R., Schnass, K.: Some Recovery Conditions for Basis Learning by L1-Minimization. In: International Symposium on Communications, Control and Signal Processing, ISCCSP (2008)Google Scholar
  12. 12.
    Gribonval, R., Schnass, K.: Dictionary Identification: Sparse Matrix-Factorisation via ℓ1 Minimisation. IEEE Trans. on Information Theory 56(7), 3523–3539 (2010)CrossRefGoogle Scholar
  13. 13.
    Plumbley, M.D.: Dictionary Learning for l1-Exact Sparse Coding. In: International Conference on Independent Component Analysis and Signal Separation (ICA), pp. 406–413 (2007)Google Scholar
  14. 14.
    Van den Berg, E., Friedlander, M.P.: Probing the Pareto frontier for basis pursuit solutions. SIAM Journal on Scientific Computing 31(2), 890–912 (2008)zbMATHCrossRefMathSciNetGoogle Scholar
  15. 15.
    Goldstein, A.A.: Convex programming in Hilbert space. Bulletin of the American Mathematical Society 70(5), 709–710 (1964)zbMATHCrossRefMathSciNetGoogle Scholar
  16. 16.
    Tibshirani, R.: Regression shrinkage and selection via the Lasso. Journal of Royal Statistical Society Series B 58, 267–288 (1996)zbMATHMathSciNetGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Mehrdad Yaghoobi
    • 1
  • Mike E. Davies
    • 1
  1. 1.Institute for Digital Communications (IDCom)the University of EdinburghUK

Personalised recommendations