Superpixel guided structure sparsity for multispectral and hyperspectral image fusion over couple dictionary

  • Feng Zhang
  • Kai ZhangEmail author


This paper proposed a hyperspectral (HS) and multispectral (MS) image fusion method based on superpixel guided structure sparsity and couple dictionary (SGSSCD). It is assumed that the pixels in a homogeneous area of MS image are similar due to the consistent spatial consistency. Superpixel technique is used to find the similar pixels in MS image by considering the region homogeneity. Then these pixels naturally share the same atoms in low spectral resolution dictionary. In order to capture the similarity prior, structural sparsity is employed to find more efficient coding of the similar pixels in MS image over low spectral resolution dictionary. Finally, high spatial resolution HS image can be produced by combining the codes of MS image with high spectral resolution dictionary. Besides, the couple dictionary is learned from HS and low spatial resolution MS images to ensure the spectral correspondence, which can further improve the quality of fusion results. The experimental results on different datasets demonstrate the effectiveness of the proposed method when compared with some existing methods.


Structure sparse Superpixel Couple dictionary Hyperspectral Multispectral Image fusion 



  1. 1.
    Achanta R, Shaji A, Smith K et al (2012) SLIC superpixels compared to state-of-the-art superpixel methods. IEEE Trans Pattern Anal Mach Intell 11:2274–2282CrossRefGoogle Scholar
  2. 2.
    Aharon M, Elad M, Bruckstein A (2006) K-SVD: an algorithm for designing overcomplete dictionaries for sparse representation. IEEE Trans Signal Process 11:4311–4322CrossRefGoogle Scholar
  3. 3.
    Aiazzi B, Baronti S, Lotti F et al (2009) A comparison between global and context-adaptive pansharpening of multispectral images. IEEE Geosci Remote Sens Lett 2:302–306CrossRefGoogle Scholar
  4. 4.
    Akhtar N, Shafait F, Mian A (2014) Sparse spatio-spectral representation for hyperspectral image super-resolution. Proc ECCV, pp 63–78Google Scholar
  5. 5.
    Akhtar N, Shafait F, Mian A (2015) Bayesian sparse representation for hyperspectral image super resolution. Proc IEEE CVPR, pp 3631–3640Google Scholar
  6. 6.
    Amro I, Mateos J, Vega M et al (2011) A survey of classical methods and new trends in pansharpening of multispectral images. EURASIP J Advan Signal Process 2011(1):79CrossRefGoogle Scholar
  7. 7.
    Ankarao V, Sowmya V, Soman KP (2018) Multi-sensor data fusion using NIHS transform and decomposition algorithms. Multimed Tools Appl 23:30381–30402CrossRefGoogle Scholar
  8. 8.
    Bioucasdias J, Plaza A, Campsvalls G et al (2013) Hyperspectral remote sensing data analysis and future challenges. IEEE Geosci Remote Sens Mag 1(2):6–36CrossRefGoogle Scholar
  9. 9.
    Campsvalls G, Bruzzone L (2005) Kernel-basted methods for hyperspectral image classification. IEEE Trans Geosci Remote Sens 6:1351–1362CrossRefGoogle Scholar
  10. 10.
    Candès EJ, Li X, Ma Y, Wright J (2009) Robust principal component analysis? J ACM 1:1–37zbMATHGoogle Scholar
  11. 11.
    Chang X, Ma Z, Lin M et al (2017) Feature interaction augmented sparse learning for fast kinect motion detection. IEEE Trans Image Process 8:3911–3920MathSciNetCrossRefGoogle Scholar
  12. 12.
    Chavez PS, Sides SC, Anderson JA (1991) Comparison of three different methods to merge multiresolution and multispectral data: Landsat TM and SPOT panchromatic. Photogramm Eng Remote Sens 3:265–303Google Scholar
  13. 13.
    Chen HY, Leou JJ (2012) Multispectral and multiresolution image fusion using particle swarm optimization. Multimed Tools Appl 3:495–518CrossRefGoogle Scholar
  14. 14.
    Chen Z, Pu HY, Wang B et al (2014) Fusion of hyperspectral and multispectral images: a novel framework based on generalization of pan-sharpening methods. IEEE Geosci Remote Sens Lett 8:1418–1422CrossRefGoogle Scholar
  15. 15.
    Conn AR, Gould NIM, Toint PL (1991) A globally convergent augmented Lagrangian algorithm for optimization with general constraints and simple bounds. Siam J Num Anal 2:545–572MathSciNetCrossRefGoogle Scholar
  16. 16.
    Hancock GR, Freeman MJ (2001) Power and sample size for the root mean square error of approximation test of not close fit in structural equation modeling. Educ Psychol Meas 5:741–758MathSciNetCrossRefGoogle Scholar
  17. 17.
    Kanmani M, Narasimhan V (2016) An optimal weighted averaging fusion strategy for thermal and visible images using dual tree discrete wavelet transform and self tunning particle swarm optimization. Multimed Tools Appl 20:1–22Google Scholar
  18. 18.
    Kawakami R, Wright J, Tai YW (2011) High-resolution hyperspectral imaging via matrix factorization. Proc IEEE CVPR, pp 2329–2336Google Scholar
  19. 19.
    Kunkel B, Blechinger F, Viehmann D et al (1991) ROSIS imaging spectrometer and its potential for ocean parameter measurements (airborne and space-borne). Int J Remote Sens 4:753–761CrossRefGoogle Scholar
  20. 20.
    Li Z, Nie F, Chang X et al (2017) Beyond trace ratio: weighted harmonic mean of trace ratios for multiclass discriminant analysis. IEEE Trans Knowl Data Eng 10:2100–2110CrossRefGoogle Scholar
  21. 21.
    Lin B, Tao X, Duan Y et al (2018) Hyperspectral and multispectral image fusion based on low rank constrained gaussian mixture model. IEEE Access 6:16901–16910CrossRefGoogle Scholar
  22. 22.
    Liu M Y, Tuzel O, Ramalingam S et al (2011) Entropy rate superpixel segmentation. Proc IEEE CVPR, pp 2097–2104Google Scholar
  23. 23.
    Mangalraj P, Agrawal A (2017) Novel approach on fusion of multisensor images based on neutrosophic domain in consideration of regional relevance. Multimed Tools Appl 1:1–18Google Scholar
  24. 24.
    Nasrabadi NM (2013) Hyperspectral target detection: an overview of current and future challenges. IEEE Signal Process Mag 1:34–44Google Scholar
  25. 25.
    Nezhad ZH, Karami A, Heylen R et al (2016) Fusion of hyperspectral and multispectral images using spectral unmixing and sparse coding. IEEE J Sel Topics Appl Earth Obser Remote Sens 6:2377–2389CrossRefGoogle Scholar
  26. 26.
    Paramanandham N, Rajendiran K (2017) Multi sensor image fusion for surveillance applications using hybrid image fusion algorithm. Multimed Tools Appl 7:1–32Google Scholar
  27. 27.
    Schweizer SM, Moura JMF (2001) Efficient detection in hyperspectral imagery. IEEE Trans Image Process 4:584–597CrossRefGoogle Scholar
  28. 28.
    Selva M, Aiazzi B, Butera F et al (2017) Hyper-sharpening: a first approach on SIM-GA data. IEEE J Sel Topics Appl Earth Obser Remote Sens 6:3008–3024Google Scholar
  29. 29.
    Shen J, Du Y, Wang W et al (2014) Lazy random walks for superpixel segmentation. IEEE Trans Image Process 4:1451–1462MathSciNetCrossRefGoogle Scholar
  30. 30.
    Stein DWJ, Beaven SG, Hoff LE et al (2002) Anomaly detection from hyperspectral imagery. IEEE Signal Process Mag 1:58–69CrossRefGoogle Scholar
  31. 31.
    Veganzones MA, Simoes M, Licciardi G et al (2016) Hyperspectral super-resolution of locally low rank images from complementary multisource data. IEEE Trans Image Process 1:274–288MathSciNetCrossRefGoogle Scholar
  32. 32.
    Vivone G, Alparone L, Chanussot J et al (2015) A critical comparison among pansharpening algorithms. IEEE Trans Geosci Remote Sens 5:2565–2586CrossRefGoogle Scholar
  33. 33.
    Wald L (2000) Quality of high resolution synthesised images: is there a simple criterion? Proc. Int. Conf. Fusion Earth Data, Nice, France, pp 99–103Google Scholar
  34. 34.
    Wald L, Ranchin T, Mangolini M (1997) Fusion of satellite images of different spatial resolutions: assessing the quality of resulting images. Photogramm Eng Remote Sens 63:691–699Google Scholar
  35. 35.
    Wang Z, Bovik AC (2002) A universal image quality index. IEEE Signal Process Lett 3:81–84CrossRefGoogle Scholar
  36. 36.
    Wei J, Huang Y (2017) NMPE: a normalized metric for measuring generalized spatial distortion of multispectral panshapening fusion. Multimed Tools Appl 4:1–18Google Scholar
  37. 37.
    Wei Q, BioucasDias José M, Dobigeon N et al (2015) Hyperspectral and multispectral image fusion based on a sparse representation. IEEE Trans Geosci Remote Sens 7:3658–3668CrossRefGoogle Scholar
  38. 38.
    Wei Q, Dobigeon N, Tourneret JY (2015) Fast fusion of multi-band images based on solving a Sylvester equation. IEEE Trans Image Process 11:1632–1636MathSciNetGoogle Scholar
  39. 39.
    Yokoya N, Yairi T, Iwasaki A (2012) Coupled nonnegative matrix factorization unmixing for hyperspectral and multispectral data fusion. IEEE Trans Geosci Remote Sens 2:528–537CrossRefGoogle Scholar
  40. 40.
    Yuhas RH, Goetz AFH, Boardman JW (1992) Discrimination among semi-arid landscape endmembers using the Spectral Angle Mapper (SAM) algorithm. Proc. Summaries 4th JPL Airborne Earth Sci. Workshop, pp 147–149Google Scholar
  41. 41.
    Zhang K, Wang M, Yang S (2017) Multispectral and hyperspectral image fusion based on group spectral embedding and low-rank factorization. IEEE Trans Geosci Remote Sens 3:1363–1371CrossRefGoogle Scholar
  42. 42.
    Zhang K, Wang M, Yang S et al (2018) Spatial–spectral-graph-regularized low-rank tensor decomposition for multispectral and hyperspectral image fusion. IEEE J Sel Topics Appl Earth Obser Remote Sens 4:1030–1040CrossRefGoogle Scholar
  43. 43.
    Zhu XX, Bamler R (2013) A sparse image fusion algorithm with application to pan-sharpening. IEEE Trans Geosci Remote Sens 5:2827–2836CrossRefGoogle Scholar
  44. 44.
    Zhu XX, Claas G, Bamler R (2016) Exploiting joint sparsity for pansharpening: the J-SparseFI algorithm. IEEE Trans Geosci Remote Sens 5:2664–2681CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2019

Authors and Affiliations

  1. 1.School of Information Science and EngineeringShandong Normal UniversityJinanChina
  2. 2.China Mobile Online Services Company LimitedZhengzhouChina

Personalised recommendations