Evaluation of K-SVD Embedded with Modified \(\ell _{1}\)-Norm Sparse Representation Algorithm

  • Meixi Wang
  • Jingjing Liu
  • Shiwei Ma
  • Wanquan Liu
Conference paper
Part of the Communications in Computer and Information Science book series (CCIS, volume 762)


The K-SVD algorithm aims to find an adaptive dictionary for a set of signals by using the sparse representation optimization and constrained singular value decomposition. In this paper, firstly, the original K-SVD algorithm, as well as some sparse representation algorithms including \(\ell _{0}\)-norm OMP and \(\ell _{1}\)-norm Lasso were reviewed. Secondly, the revised Lasso algorithm was embedded into the K-SVD process and a new different K-SVD algorithms with \(\ell _{1}\)-norm Lasso embedded in (RL-K-SVD algrithm) was established. Finally, extensive experiments had been completed on necessary parameters determination, further on the performance compare of recovery error and recognition for the original K-SVD and RL-K-SVD algorithms. The results indicate that within a certain scope of parameter settings, the RL-K-SVD algorithm performs better on image recognition than K-SVD; the time cost for training sample number is lower for RL-K-SVD in case that the sample number is increased to a certain extend.


K-SVD algorithm Sparse representation Dictionary Pursuit methods 



This work was supported by the National Science Foundation of China (Nos. 61171145, 61671285).


  1. 1.
    Mairal, J., Sapiro, G., Elad, M.: Learning multiscale sparse representations for image and video restoration. Multiscale Model. Simul. 7(1), 214–241 (2008)MathSciNetCrossRefzbMATHGoogle Scholar
  2. 2.
    Davis, G., Mallat, S., Avellaneda, M.: Adaptive greedy approximations. Constr. Approx. 13(1), 57–98 (1997)MathSciNetCrossRefzbMATHGoogle Scholar
  3. 3.
    Meinshausen, N., Yu, B.: Lasso-type recovery of sparse representations for high-dimensional data. Ann. Stat. 37, 246–270 (2009)MathSciNetCrossRefzbMATHGoogle Scholar
  4. 4.
    Aharon, M., Elad, M., Bruckstein, A.: \( rm k \)-SVD: an algorithm for designing overcomplete dictionaries for sparse representation. IEEE Trans. Sig. Process. 54(11), 4311–4322 (2006)CrossRefGoogle Scholar
  5. 5.
    Jiang, Z., Lin, Z., Davis, L.S.: Learning a discriminative dictionary for sparse coding via label consistent K-SVD. In: 2011 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1697–1704. IEEE (2011)Google Scholar
  6. 6.
    Zhang, H., Yang, J., Zhang, Y., et al.: Close the loop: joint blind image restoration and recognition with sparse representation prior. In: 2011 IEEE International Conference on Computer Vision (ICCV), pp. 770–777. IEEE (2011)Google Scholar
  7. 7.
    Xie, S., Rahardja, S.: An alternating direction method for frame-based image deblurring with balanced regularization. In: 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 1061–1064. IEEE (2012)Google Scholar
  8. 8.
    Studer, C., Baraniuk, R.G.: Dictionary learning from sparsely corrupted or compressed signals. In: 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 3341–3344. IEEE (2012)Google Scholar
  9. 9.
    Wright, J., Yang, A.Y., Ganesh, A., et al.: Robust face recognition via sparse representation. IEEE Trans. Pattern Anal. Mach. Intell. 31(2), 210–227 (2009)CrossRefGoogle Scholar
  10. 10.
    Zhang, Q., Li, B.: Discriminative K-SVD for dictionary learning in face recognition. In: 2010 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2691–2698. IEEE (2010)Google Scholar
  11. 11.
    Mallat, S.G., Zhang, Z.: Matching pursuits with time-frequency dictionaries. IEEE Trans. Sig. Process. 41(12), 3397–3415 (1993)CrossRefzbMATHGoogle Scholar
  12. 12.
    Needell, D., Tropp, J.A.: CoSaMP: iterative signal recovery from incomplete and inaccurate samples. Appl. Comput. Harmon. Anal. 26(3), 301–321 (2009)MathSciNetCrossRefzbMATHGoogle Scholar
  13. 13.
    Chen, S.S., Donoho, D.L., Saunders, M.A.: Atomic decomposition by basis pursuit. SIAM Rev. 43(1), 129–159 (2001)MathSciNetCrossRefzbMATHGoogle Scholar
  14. 14.
    Gorodnitsky, I.F., Rao, B.D.: Sparse signal reconstruction from limited data using FOCUSS: a re-weighted minimum norm algorithm. IEEE Trans. Sig. Process. 45(3), 600–616 (1997)CrossRefGoogle Scholar
  15. 15.
    Rao, B.D., Kreutz-Delgado, K.: An affine scaling methodology for best basis selection. IEEE Trans. Sig. Process. 47(1), 187–200 (1999)MathSciNetCrossRefzbMATHGoogle Scholar
  16. 16.
    Rao, B.D., Engan, K., Cotter, S.F., et al.: Subset selection in noise based on diversity measure minimization. IEEE Trans. Sig. Process. 51(3), 760–770 (2003)CrossRefGoogle Scholar
  17. 17.
    Jung, H., Park, J., Yoo, J., et al.: Radial k-t FOCUSS for high resolution cardiac cine MRI. Magn. Reson. Med. 63(1), 68–78 (2010)Google Scholar
  18. 18.
    Aad, G., Abat, E., Abbott, B., et al.: Charged-particle multiplicities in pp interactions at measured with the ATLAS detector at the LHC. Phys. Lett. B 688(1), 21–42 (2010)CrossRefGoogle Scholar
  19. 19.
    He, Z., Xie, S., Zhang, L., et al.: A note on Lewicki-Sejnowski gradient for learning overcomplete representations. Neural Comput. 20(3), 636–643 (2008)MathSciNetCrossRefzbMATHGoogle Scholar
  20. 20.
    Blumensath, T., Davies, M.E.: Iterative hard thresholding for compressed sensing. Appl. Comput. Harmon. Anal. 27(3), 265–274 (2009)MathSciNetCrossRefzbMATHGoogle Scholar
  21. 21.
    Blumensath, T., Davies, M.E.: Iterative thresholding for sparse approximations. J. Fourier Anal. Appl. 14(5–6), 629–654 (2008)MathSciNetCrossRefzbMATHGoogle Scholar
  22. 22.
    Blumensath, T., Davies, M.E.: Normalized iterative hard thresholding: guaranteed stability and performance. IEEE J. Sel. Top. Sig. Process. 4(2), 298–309 (2010)CrossRefGoogle Scholar
  23. 23.
    Tibshirani, R.: Regression shrinkage and selection via the lasso. J. R. Stat. Soc. Ser. B (Methodol.) 58, 267–288 (1996)MathSciNetzbMATHGoogle Scholar
  24. 24.
    Jiang, Z., Lin, Z., Davis, L.S.: Label consistent K-SVD: learning a discriminative dictionary for recognition. IEEE Trans. Pattern Anal. Mach. Intell. 35(11), 2651–2664 (2013)CrossRefGoogle Scholar

Copyright information

© Springer Nature Singapore Pte Ltd. 2017

Authors and Affiliations

  • Meixi Wang
    • 1
  • Jingjing Liu
    • 1
  • Shiwei Ma
    • 1
  • Wanquan Liu
    • 2
  1. 1.School of Mechatronic Engineering and AutomationShanghai UniversityShanghaiChina
  2. 2.Department of Computer ScienceCurtin UniversityPerthAustralia

Personalised recommendations