Soft Computing

, Volume 22, Issue 10, pp 3331–3342 | Cite as

DUK-SVD: dynamic dictionary updating for sparse representation of a long-time remote sensing image sequence

  • Lizhe Wang
  • Peng LiuEmail author
  • Weijing Song
  • Kim-Kwang Raymond Choo
Methodologies and Application


Sparse representations of data or signals have drawn considerable attentions in the past decade. In this paper, we focus on the problem of training high-efficacy dictionaries for remote sensing images of massive long-time sequences. By extending the classical K-SVD, we propose a new dictionaries learning algorithm. Different from K-SVD, in the proposed incremental K-SVD algorithm, we selectively train a certain number of atoms when each new batch of sample data are added into the training process; current dictionary are replenished by the selected and enhanced atoms. The new atoms are initialized by information entropy. Meanwhile, we introduce an uncertainty metric to determine whether or not new atoms should be added into the current dictionary. To efficiently and sparsely represent the long-time sequence data set, we also de-correlate the dictionary based on new atoms by introducing a mutual coherence constraint into the atom updating stage. The method presented in this paper aims to adaptively and dynamically train the dictionary from big data. Two other state-of-the-art dictionary learning methods such as online dictionary learning (ODL) and recursive least squares dictionary learning algorithm (RLS-DLA) who also could train the dictionary using relatively large data, are comprehensively compared with the proposed algorithm in both sparse model and error model. In the sparse model, the reconstruction error of the DUK-SVD dictionary was smaller than ODL and RLS-DLA. In the error model, the sparsity of the DUK-SVD was higher than ODL and RLS-DLA. We can also observe that in the sparse model the proposed DUK-SVD often consume fewer computing time than ODL.


Long-time sequence Sparse representation Dictionary learning Remote sensing 



This work is supported by the National Natural Science Foundation of China (Nos. 41571413 and 41471368).

Compliance with ethical standards

Conflict of interest

All authors declare that they have no conflict of interest.

Ethical approval

This article does not contain any studies with human participants performed by any of the authors.


  1. Aharon M, Elad M, Bruckstein A (2006) K-SVD: an algorithm for designing overcomplete dictionaries for sparse representation. IEEE Trans Sig Process 54(11):4311–4322CrossRefzbMATHGoogle Scholar
  2. Barchiesi D, Plumbley MD (2013) Learning incoherent dictionaries for sparse approximation using iterative projections and rotations. IEEE Trans Sig Process 61(8):2055–2065CrossRefGoogle Scholar
  3. Bottou L, Bousquet O (2008) The tradeoffs of large scale learning. In: Platt JC, Koller D, Singer, Roweis S (eds) Advances in neural information processing systems, Vancouver, British Columbia, Canada, pp 161–168Google Scholar
  4. Chen D, Li X, Wang L, Khan SU, Wang J, Zeng K, Cai C (2015a) Fast and scalable multi-way analysis of massive neural data. IEEE Trans Comput 64(3):707–719MathSciNetCrossRefzbMATHGoogle Scholar
  5. Chen Y, Li F, Fan J (2015b) Mining association rules in big data with NGEP. Clust Comput 18(2):577–585CrossRefGoogle Scholar
  6. Chen D, Hu Y, Wang L, Zomaya A, Li X (2016) H-parafac: hierarchical parallel factor analysis of multidimensional big data. IEEE Trans Parallel Distrib Syst PP(99):1–1Google Scholar
  7. Davis G, Mallat S, Avellaneda M (1994) Adaptive nonlinear approximations. Technical report, New York UniversityGoogle Scholar
  8. Deng Z, Hu Y, Zhu M, Huang X, Du B (2015) A scalable and fast OPTICS for clustering trajectory big data. Clust Comput 18(2):549–562CrossRefGoogle Scholar
  9. Easley G, Labate D (2008) Sparse directional image representations using the discrete shearlet transform. Appl Comput Harmonic Anal 25(1):25–46MathSciNetCrossRefzbMATHGoogle Scholar
  10. Elad M, Aharon M (2006) Image denoising via sparse and redundant representations over learned dictionaries. IEEE Trans Image Process 15(12):3736–3745MathSciNetCrossRefGoogle Scholar
  11. Engan K, Aase SO, Husoy JH (1999) Method of optimal directions for frame design. In: ICASSP, vol 05, pp 2443–2446Google Scholar
  12. Gorodnitsky IF, Rao BD (1997) Sparse signal reconstruction from limited data using FOCUSS: a re-weighted minimum norm algorithm. IEEE Trans Sig Process 45:600–616CrossRefGoogle Scholar
  13. He Z, Wu C, Liu G, Zheng Z, Tian Y (2015) Decomposition tree: a spatio-temporal indexing method for movement big data. Clust Comput 18(4):1481–1492CrossRefGoogle Scholar
  14. Jia S, Xie Y, Tang G, Zhu J (2016) Spatial–spectral-combined sparse representation-based classification for hyperspectral imagery. Soft Comput 20(12):4659–4668CrossRefGoogle Scholar
  15. Jiang X, Gao J, Wang T, Zheng L (2012a) Supervised latent linear Gaussian process latent variable model for dimensionality reduction. IEEE Trans Syst Man Cybern Part B: Cybern 42(6):1620–1632CrossRefGoogle Scholar
  16. Jiang Z, Zhang G, Davis LS (2012b). Submodular dictionary learning for sparse coding. In: 2012 IEEE conference on computer vision and pattern recognition (CVPR). IEEE, pp 3418–3425Google Scholar
  17. Jiang Z, Lin Z, Davis LS (2013) Label consistent K-SVD: learning a discriminative dictionary for recognition. IEEE Trans Pattern Anal Mach Intell 35(11):2651–2664CrossRefGoogle Scholar
  18. Jiang J, Hu R, Wang Z, Han Z (2014a) Noise robust face hallucination via locality-constrained representation. IEEE Trans Multimed 16(5):1268–1281CrossRefGoogle Scholar
  19. Jiang X, Gao J, Wang T, Shi D (2014b) Tpslvm: a dimensionality reduction algorithm based on thin plate splines. IEEE Trans Cybern 44(10):1795–1807CrossRefGoogle Scholar
  20. Jiang J, Ma X, Cai Z, Hu R (2015) Sparse support regression for image super-resolution. IEEE Photonics J 7(5):1–11CrossRefGoogle Scholar
  21. Jiang J, Hu R, Wang Z, Han Z, Ma J (2016) Facial image hallucination through coupled-layer neighbor embedding. IEEE Trans Circuits Syst Video Technol 26(9):1674–1684CrossRefGoogle Scholar
  22. Jiang J, Chen C, Yu Y, Jiang X, Ma J (2017) Spatial-aware collaborative representation for hyperspectral remote sensing image classification. IEEE Geosci Remote Sens Lett 14:1–5CrossRefGoogle Scholar
  23. Le Pennec E, Mallat S (2005) Sparse geometric image representations with bandelets. IEEE Trans Image Process 14(4):423–438MathSciNetCrossRefGoogle Scholar
  24. Li X, Wang L (2015) On the study of fusion techniques for bad geological remote sensing image. J Ambient Intell Humaniz Comput 6(1):141–149CrossRefGoogle Scholar
  25. Li Z, Liu J, Yang Y, Zhou X, Lu H (2014) Clustering-guided sparse structural learning for unsupervised feature selection. IEEE Trans Knowl Data Eng 26(9):2138–2150CrossRefGoogle Scholar
  26. Li Z, Liu J, Tang J, Lu H-H (2015) Robust structured subspace learning for data representation. IEEE Trans Pattern Anal Mach Intell 37(10):2085–2098CrossRefGoogle Scholar
  27. Ma J, Zhou H, Zhao J, Gao Y, Jiang J, Tian J (2015) Robust feature matching for remote sensing image registration via locally linear transforming. IEEE Trans Geosci Remote Sens 53(12):6469–6481CrossRefGoogle Scholar
  28. Mailh B, Barchiesi D, Plumbley MD (2012) INK-SVD: learning incoherent dictionaries for sparse representations. In: Proc. IEEE int. conf. acoust., speech signal process. (ICASSP), pp 3573–3576Google Scholar
  29. Mairal J, Bach F, Ponce J, Sapiro G (2009) Online dictionary learning for sparse coding. In: Proceedings of the 26th annual international conference on machine learning, ICML ’09. ACM, New York, pp 689–696Google Scholar
  30. Mallat SG (1989) A theory for multiresolution signal decomposition: the wavelet representation. IEEE Trans Pattern Anal Mach Intell 11:674–693CrossRefzbMATHGoogle Scholar
  31. Mallat S (2009) Geometrical grouplets. Appl Comput Harmonic Anal 26(2):161–180MathSciNetCrossRefzbMATHGoogle Scholar
  32. Mallat SG, Zhang Z (1993) Matching pursuits with time-frequency dictionaries. IEEE Trans Sig Process 41(12):3397–3415CrossRefzbMATHGoogle Scholar
  33. Needell D, Tropp JA (2008) CoSaMP: iterative signal recovery from incomplete and inaccurate samples. Technical report, California Institute of Technology, PasadenaGoogle Scholar
  34. Olshausen BA, Field DJ (1996) Emergence of simple-cell receptive field properties by learning a sparse code for natural images. Nature 381:607–609CrossRefGoogle Scholar
  35. Palm G, Schwenker F, Sommer FT, Strey A (1993) Neural associative memories. Biol Cybern 36:36–19Google Scholar
  36. Pati YC, Rezaiifar R, Rezaiifar YC, Pati R, Krishnaprasad PS (1993) Orthogonal matching pursuit: recursive function approximation with applications to wavelet decomposition. In: Proceedings of the 27th annual Asilomar conference on signals, systems, and computers, pp 40–44Google Scholar
  37. Ramłrez I, Lecumberry F, Sapiro G (2009) Sparse modeling with universal priors and learned incoherent dictionaries inst. Technical report, University of MinnesotaGoogle Scholar
  38. Shen H, Li X, Zhang L, Tao D, Zeng C (2014) Compressed sensing-based inpainting of aqua moderate resolution imaging spectroradiometer band 6 using adaptive spectrum-weighted sparse bayesian dictionary learning. IEEE Trans Geosci Remote Sens 52(2):894–906CrossRefGoogle Scholar
  39. Skretting K, Engan K (2010) Recursive least squares dictionary learning algorithm. IEEE Trans Sig Process 58(4):2121–2130MathSciNetCrossRefGoogle Scholar
  40. Starck J-L, Candes EJ, Donoho DL (2002) The curvelet transform for image denoising. IEEE Trans Image Process 11(6):670–684MathSciNetCrossRefzbMATHGoogle Scholar
  41. Tropp JA (2004) Greed is good: algorithmic results for sparse approximation. IEEE Trans Inf Theory 50:2231–2242MathSciNetCrossRefzbMATHGoogle Scholar
  42. Tuia D, Ratle F, Pacifici F, Kanevski MF, Emery WJ (2009) Active learning methods for remote sensing image classification. IEEE Trans Geosci Remote Sens 47(7):2218–2232CrossRefGoogle Scholar
  43. Vetterli M, Velisavljevic VL, Beferull-Lozano B, Dragotti PL (2006) Directionlets: anisotropic multi-directional representation with separable filtering. IEEE Trans Image Process 15(7):1916–1933CrossRefGoogle Scholar
  44. Wang L, Lu K, Liu P, Ranjan R, Chen L (2014) IK-SVD: dictionary learning for spatial big data via incremental atom update. Comput Sci Eng 16(4):41–52CrossRefGoogle Scholar
  45. Wang Y, Liu Z, Liao H, Li C (2015) Improving the performance of GIS polygon overlay computation with mapreduce for spatial big data processing. Clust Comput 18(2):507–516CrossRefGoogle Scholar
  46. Wu Z, Yuan J, Zhang J, Huang H (2016) A hierarchical face recognition algorithm based on humanoid nonlinear least-squares computation. J Ambient Intell Humaniz Comput 7(2):229–238CrossRefGoogle Scholar
  47. Xin X, Yin X, Liu Q, Hu F, Cai T, Jiang N, Xiong H (2015) A study on fast sift image mosaic algorithm based on compressed sensing and wavelet transform. J Ambient Intell Humaniz Comput 6(6):835–843CrossRefGoogle Scholar
  48. Zeng K, Chen D, Ouyang G, Wang L, Liu X, Li X (2016) An EEMD-ICA approach to enhancing artifact rejection for noisy multivariate neural data. IEEE Trans Neural Syst Rehabil Eng 24(6):630–638CrossRefGoogle Scholar
  49. Zhou M, Chen H, Paisley J, Ren L, Li L, Xing Z, Dunson D, Sapiro G, Carin L (2012) Nonparametric bayesian dictionary learning for analysis of noisy and incomplete images. IEEE Trans Image Process 21(1):130–144MathSciNetCrossRefzbMATHGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2017

Authors and Affiliations

  1. 1.School of Computer ScienceChina University of GeosciencesWuhanPeople’s Republic of China
  2. 2.Institute of Remote Sensing and Digital EarthChinese Academy of SciencesBeijingPeople’s Republic of China
  3. 3.Department of Information Systems and Cyber SecurityUniversity of Texas at San AntonioSan AntonioUSA

Personalised recommendations