Skip to main content
SpringerLink
Log in

Learning block-structured incoherent dictionaries for sparse representation

用于稀疏表示的块结构非相干字典学习方法

  • Research Paper
  • Published:
Science China Information Sciences Aims and scope Submit manuscript

Abstract

Dictionary learning is still a challenging problem in signal and image processing. In this paper, we propose an efficient block-structured incoherent dictionary learning algorithm for sparse representations of image signals. The constrained minimization of dictionary learning is achieved by iteratively alternating between sparse coding and dictionary update. Without relying on any prior knowledge of the group structure for the input data, we develop a two-stage clustering method that identifies the underlying block structure of the dictionary under certain restricted constraints. The two-stage clustering method mainly consists of affinity propagation and agglomerative hierarchical clustering. To meet the conditions of both the upper bound and the lower bound of the mutual coherence of dictionary atoms, we introduce a regularization term for the objective function to adjust the block coherence of the overcomplete dictionary. The experiments on synthetic data and real images demonstrate that the proposed dictionary learning algorithm has lower representation error, higher visual quality and better reconstructed results than most of the state-of-the-art methods.

中文摘要

字典学习是信号和图像处理领域中的一个挑战性问题。本文提出一种有效的块结构化非相干性的字典学习算法来提高图像信号的稀疏表示效果, 采用交替迭代最小化方法来求解字典学习优化的目标函数。通过稀疏编码和字典更新两个步骤的交替迭代来实现约束字典学习的优化设计。不依赖于输入图像信号的先验知识, 我们设计一种两阶段聚类分析方法来识别学习字典的潜在块结构。为了满足字典原子相干性的约束条件, 我们对字典学习优化的目标函数, 引入一种正则项用于调节超完备字典的块相干性。模拟数据和真实图像的实验结果表明, 与现有的绝大数最先进的方法相比, 本文所提的字典学习算法能够获得较小的表示误差、较高的视觉质量和较好的重建效果。

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Mairal J, Elad M, Sapiro G. Sparse representation for color image restoration. IEEE Trans Image Process, 2008, 17: 53–69

    Article  MathSciNet  Google Scholar 

  2. Zhang Z Y. Community structure detection in social networks based on dictionary learning. Sci China Inf Sci, 2013, 56: 078103

    MathSciNet  Google Scholar 

  3. Olshausen B A, Field D J. Emergence of simple-cell receptive field properties by learning a sparse code for natural images. Nature, 1996, 381: 607–609

    Article  Google Scholar 

  4. Vinje W E, Gallant J L. Sparse coding and decorrelation in primary visual cortex during natural vision. Science, 2000, 287: 1273–1276

    Article  Google Scholar 

  5. Nirenberg S, Carcieri S M, Jacobs A L, et al. Retinal ganglion cells act largely as independent encoders. Nature, 2001, 411: 698–701

    Article  Google Scholar 

  6. Fadili M J, Starck J L, Bobin J, et al. Image decomposition and separation using sparse representations: an overview. Proc IEEE, 2010, 98: 983–994

    Article  Google Scholar 

  7. Elad M, Milanfar P, Rubinstein R. Analysis versus synthesis in signal priors. Inverse Probl, 2007, 23: 947–968

    Article  MATH  MathSciNet  Google Scholar 

  8. Bruckstein A M, Donoho D L, Elad M. From sparse solutions of systems of equations to sparse modeling of signals and images. SIAM Rev, 2009, 51: 34–81

    Article  MATH  MathSciNet  Google Scholar 

  9. Elad M. Sparse and redundant representation modeling-What next? IEEE Signal Process Lett, 2012, 19: 922–928

    Article  Google Scholar 

  10. Mallat S, Zhang Z. Matching pursuits with time-frequency dictionaries. IEEE Trans Signal Process, 1993, 41: 3397–3415

    Article  MATH  Google Scholar 

  11. Tropp J A. Greed is good: algorithmic results for sparse approximation. IEEE Trans Inf Theory, 2004, 50: 2231–2242

    Article  MATH  MathSciNet  Google Scholar 

  12. Cai T T, Wang L. Orthogonal matching pursuit for sparse signal recovery with noise. IEEE Trans Inf Theory, 2011, 57: 4680–4688

    Article  MathSciNet  Google Scholar 

  13. Rubinstein R, Zibulevsky M, Elad M. Efficient implementation of the K-SVD algorithm using batch orthogonal matching pursuit. Technical Report, CS, Technion, 2008

    Google Scholar 

  14. Eldar Y C, Kuppinger P, Bolcskei H. Block-sparse signals: uncertainty relations and efficient recovery. IEEE Trans Signal Process, 2010, 58: 3042–3054

    Article  MathSciNet  Google Scholar 

  15. Donoho D L, Tsaig Y, Drori I, et al. Sparse solution of underdetermined linear equations by stagewise orthogonal matching pursuit. IEEE Trans Inf Theory, 2012, 58: 1094–1121

    Article  MathSciNet  Google Scholar 

  16. Zhang W H, Zhou T, Huang B X. Outlier deletion based improvement on the StOMP algorithm for sparse solution of large-scale underdetermined problems. Sci China Inf Sci, 2014, 57: 092203

    MathSciNet  Google Scholar 

  17. Needell D, Tropp J A. Cosamp: iterative signal recovery from incomplete and inaccurate samples. Appl Comput Harmonic Anal, 2009, 26: 301–321

    Article  MATH  MathSciNet  Google Scholar 

  18. Efron B, Hastie T, Johnstone I, et al. Least angle regression. Ann Statist, 2004, 32: 407–499

    Article  MATH  MathSciNet  Google Scholar 

  19. Dai W, Milenkovic O. Subspace pursuit for compressive sensing signal reconstruction. IEEE Trans Inf Theory, 2009, 55: 2230–2249

    Article  MathSciNet  Google Scholar 

  20. Blumensath T, Davies M E. Gradient pursuits. IEEE Trans Signal Process, 2008, 56: 2370–2382

    Article  MathSciNet  Google Scholar 

  21. Chen S S, Donoho D L, Saunders M A. Atomic decomposition by basis pursuit. SIAM Rev, 2001, 43: 129–159

    Article  MATH  MathSciNet  Google Scholar 

  22. Eldar Y C, Mishali M. Robust recovery of signals from a structured union of subspaces. IEEE Trans Inf Theory, 2009, 55: 5302–5316

    Article  MathSciNet  Google Scholar 

  23. Gorodnitsky I F, Rao B D. Sparse signal reconstruction from limited data using FOCUSS: a re-weighted minimum norm algorithm. IEEE Trans Signal Process, 1997, 45: 600–616

    Article  Google Scholar 

  24. Wright M H. The interior-point revolution in optimization: history, recent developments, and lasting consequences. Bull Amer Math Soc, 2004, 42: 39–56

    Article  Google Scholar 

  25. Elad M. Why simple shrinkage is still relevant for redundant representations. IEEE Trans Inf Theory, 2006, 52: 5559–5569

    Article  MATH  MathSciNet  Google Scholar 

  26. Bioucas-Dias J M, Figueiredo M A T. A new TwIST: two-step iterative shrinkage/thresholding algorithms for image restoration. IEEE Trans Image Process, 2007, 16: 2992–3004

    Article  MathSciNet  Google Scholar 

  27. Ahmed N, Natarajan T, Rao K R. Discrete cosine transform. IEEE Trans Comput, 1974, C-23: 90–93

    Article  MathSciNet  Google Scholar 

  28. Mallat S. A Wavelet Tour of Signal Processing, 3rd Edition: the Sparse Way. New York: Academic Press, 2008

    Google Scholar 

  29. Ma J, Plonka G. The curvelet transform. IEEE Signal Process Mag, 2010, 27: 118–133

    Article  Google Scholar 

  30. Do M N, Vetterli M. The contourlet transform: an efficient directional multiresolution image representation. IEEE Trans Image Process, 2005, 14: 2091–2106

    Article  MathSciNet  Google Scholar 

  31. Do M N, Vetterli M. The finite ridgelet transform for image representation. IEEE Trans Image Process, 2003, 12: 16–28

    Article  MATH  MathSciNet  Google Scholar 

  32. Le Pennec E, Mallat S. Sparse geometric image representations with bandelets. IEEE Trans Image Process, 2005, 14: 423–438

    Article  MathSciNet  Google Scholar 

  33. Easley G, Labate D, Lim W Q. Sparse directional image representations using the discrete shearlet transform. Appl Comput Harmonic Anal, 2008, 25: 25–46

    Article  MATH  MathSciNet  Google Scholar 

  34. Tosic I, Frossard P. Dictionary Learning: what is the right representation for my signal? IEEE Signal Process Mag, 2011, 28: 27–38

    Article  Google Scholar 

  35. Olshausen B A, Field D J. Sparse coding with an overcomplete basis set: a strategy employed by V1? Vis Res, 1997, 37: 3311–3325

    Article  Google Scholar 

  36. Engan K, Aase S O, Hakon Husoy J. Method of optimal directions for frame design. In: Proceedings of IEEE International Conference on Acoustics, Speech, and Signal Processing, Phoenix, 1999. 2443–2446

    Google Scholar 

  37. Kreutz-Delgado K, Murray J, Rao B, et al. Dictionary learning algorithms for sparse representation. Neural Comput, 2003, 15: 349–396

    Article  MATH  Google Scholar 

  38. Yaghoobi M, Blumensath T, Davies M E. Dictionary learning for sparse approximations with the majorization method. IEEE Trans Signal Process, 2009, 57: 2178–2191

    Article  MathSciNet  Google Scholar 

  39. Mairal J, Bach F, Ponce J, et al. Online learning for matrix factorization and sparse coding. J Mach Learn Res, 2010, 11: 19–60

    MATH  MathSciNet  Google Scholar 

  40. Mairal J, Bach F, Ponce J. Task-driven dictionary learning. IEEE Trans Patt Anal Mach Intell, 2012, 34: 791–804

    Article  Google Scholar 

  41. Schmid-Saugeon P, Zakhor A. Dictionary design for matching pursuit and application to motion-compensated video coding. IEEE Trans Circ Syst Video Technol, 2004, 14: 880–886

    Article  Google Scholar 

  42. Aharon M, Elad M, Bruckstein A. K-SVD: an algorithm for designing overcomplete dictionaries for sparse representation. IEEE Trans Signal Process, 2006, 54: 4311–4322

    Article  Google Scholar 

  43. Stewart G W. On the early history of the singular value decomposition. SIAM Rev, 1993, 35: 551–566

    Article  MATH  MathSciNet  Google Scholar 

  44. Zhang Y Q, Ding Y, Xiao J S, et al. Visibility enhancement using an image filtering approach. EURASIP J Appl Signal Process, 2012, 220: 1–6

    Article  Google Scholar 

  45. Zhang Y Q, Ding Y, Liu J Y, et al. Guided image filtering using signal subspace projection. IET Image Process, 2013, 7: 270–279

    Article  MathSciNet  Google Scholar 

  46. Zelnik-Manor L, Rosenblum K, Eldar Y C. Dictionary optimization for block-sparse representations. IEEE Trans Signal Process, 2012, 60: 2386–2395

    Article  MathSciNet  Google Scholar 

  47. Eksioglu E M. A clustering based rramework for dictionary block structure identification. In: Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing, Prague, 2011. 4044–4047

    Google Scholar 

  48. Oiwa H, Matsushima S, Nakagawa H. Feature-aware regularization for sparse online learning. Sci China Inf Sci, 2014, 57: 052104

    Article  MathSciNet  Google Scholar 

  49. Barchiesi D, Plumbley M D. Learning incoherent dictionaries for sparse approximation using iterative projections and rotations. IEEE Trans Signal Process, 2013, 61: 2055–2065

    Article  Google Scholar 

  50. Zhang Y Q, Liu J Y, Li M D, et al. Joint image denoising using adaptive principal component analysis and selfsimilarity. Inf Sci, 2014, 259: 128–141

    Article  MathSciNet  Google Scholar 

  51. Gribonval R, Nielsen M. Sparse representations in unions of bases. IEEE Trans Inf Theory, 2003, 49: 3320–3325

    Article  MATH  MathSciNet  Google Scholar 

  52. Engan K, Skretting K, Husoy J H. Family of iterative LS-based dictionary learning algorithms, ILS-DLA, for sparse signal representation. Digit Signal Process, 2007, 17: 32–49

    Article  Google Scholar 

  53. Chi Y T, Ali M, Rajwade A, et al. Block and group regularized sparse modeling for dictionary learning. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, Portland, 2013. 1–8

    Google Scholar 

  54. Mairal J, Sapiro G, Elad M. Learning multiscale sparse representations for image and video restoration. Multiscale Model Simul, 2008, 7: 214–241

    Article  MATH  MathSciNet  Google Scholar 

  55. Ophir B, Lustig M, Elad M. Multi-scale dictionary learning using wavelets. IEEE J-STSP, 2011, 5: 1014–1024

    Google Scholar 

  56. Barthelemy Q, Larue A, Mayoue A, et al. Shift boolean and 2D rotation invariant sparse coding for multivariate signals. IEEE Trans Signal Process, 2012, 60: 1597–1611

    Article  MathSciNet  Google Scholar 

  57. Meng D Y, Zhao Q, Leung Y, et al. Learning dictionary from signals under global sparsity constraint. Neurocomputing, 2013, 119: 308–318

    Article  Google Scholar 

  58. Peng Y, Meng D Y, Xu Z B, et al. Decomposable nonlocal tensor dictionary learning for multispectral image denoising. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, Columbus, 2014. 2949–2956

    Google Scholar 

  59. Bengio S, Pereira F, Singer Y, et al. Group sparse coding. In: Proceedings of Advances in Neural Information Processing Systems, Vancouver, 2009. 82–89

    Google Scholar 

  60. Gao S, Chia L T, Tsang I W. Multi-layer group sparse coding—for concurrent image classification and annotation. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, Providence, 2011. 2809–2816

    Google Scholar 

  61. Elhamifar E, Vidal R. Block-sparse recovery via convex optimization. IEEE Trans Signal Process, 2012, 60: 4094–4107

    Article  MathSciNet  Google Scholar 

  62. Strohmer T, Heath Jr R W. Grassmannian frames with applications to coding and communication. Appl Comput Harmonic Anal, 2003, 14: 257–275

    Article  MATH  Google Scholar 

  63. Eldar Y C, Bolcskei H. Block-sparsity: coherence and efficient recovery. In: Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing, Taipei, 2009. 2885–2888

    Google Scholar 

  64. Murtagh F, Contreras P. Algorithms for hierarchical clustering: an overview. Data Min Knowl Discov, 2012, 2: 86–97

    Article  Google Scholar 

  65. Frey B J, Dueck D. Clustering by passing messages between data points. Science, 2007, 315: 972–976

    Article  MATH  MathSciNet  Google Scholar 

  66. Wang C D, Lai J H, Suen C Y, et al. Multi-exemplar affinity propagation. IEEE Trans Patt Anal Mach Intell, 2013, 35: 2223–2237

    Article  Google Scholar 

  67. Sigg C D, Dikk T, Buhmann J M. Learning dictionaries with bounded self-coherence. IEEE Signal Process Lett, 2012, 19: 861–864

    Article  Google Scholar 

  68. Wang Z, Bovik A C, Sheikh H R, et al. Image quality assessment: from error visibility to structural similarity. IEEE Trans Image Process, 2004, 13: 600–612

    Article  Google Scholar 

  69. Huynh-Thu Q, Ghanbari M. Scope of validity of PSNR in image/video quality assessment. Electron Lett, 2008, 44: 800–801

    Article  Google Scholar 

  70. Duarte-Carvajalino J M, Sapiro G. Learning to sense sparse signals: simultaneous sensing matrix and sparsifying dictionary optimization. IEEE Trans Image Process, 2009, 18: 1395–1408

    Article  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Information Science and Technology, Northwest University, Xi’an, 710127, China

    YongQin Zhang

  2. School of Electronic Information, Wuhan University, Wuhan, 430079, China

    JinSheng Xiao

  3. College of Computer and Information Engineering, Henan University of Economics and Law, Zhengzhou, 450002, China

    ShuHong Li

  4. Shenzhen Key Laboratory for MRI, Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences, Shenzhen, 518055, China

    CaiYun Shi & GuoXi Xie

Authors
  1. YongQin Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. JinSheng Xiao
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. ShuHong Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. CaiYun Shi
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. GuoXi Xie
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to JinSheng Xiao or GuoXi Xie.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhang, Y., Xiao, J., Li, S. et al. Learning block-structured incoherent dictionaries for sparse representation. Sci. China Inf. Sci. 58, 1–15 (2015). https://doi.org/10.1007/s11432-014-5258-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11432-014-5258-6

Keywords

关键词

Search

Navigation