Multimedia Tools and Applications

, Volume 77, Issue 24, pp 32013–32040 | Cite as

Non-subsampled shearlet transform-based image fusion using modified weighted saliency and local difference

  • Amit Vishwakarma
  • M. K. Bhuyan
  • Yuji Iwahori


Existing image fusion methods can not efficiently capture significant edges, texture and fine details of the source images due to inefficient fusion framework. In addition, for objective evaluation of fusion algorithms, not much attention is given to simultaneously measure both texture and structural information of the source images which are preserved in the fused image. To address these issues, non-subsampled shearlet transform (NSST) is used to decompose pre-registered source images into low- and high-frequency components. These low- and high-frequency coefficients are fused by using our proposed modified weighted salience and local difference fusion rules, respectively. To enrich edge information in the fused image, Canny edge detector with scale multiplication is employed. Moreover, a metric QTS is proposed to jointly measure both texture and structural information present in the fused image. The proposed metric is formulated on the basis of local standard deviation filtering, local information entropy, and local difference filtering. Both subjective and objective results validate the proposed fusion framework and the metric QTS.


Image fusion NSST Canny edge detector Local difference Weighted salience 


  1. 1.
    Aishwarya N, Bennila Thangammal C (2017) An image fusion framework using novel dictionary based sparse representation. Multimed Tools Appl 76(21):21869–21888CrossRefGoogle Scholar
  2. 2.
    Bao P, Zhang L, Wu X (2005) Canny edge detection enhancement by scale multiplication. IEEE Trans Pattern Anal Mach Intell 27(9):1485–1490CrossRefGoogle Scholar
  3. 3.
    Bavirisetti DP, Dhuli R (2016) Fusion of infrared and visible sensor images based on anisotropic diffusion and Karhunen-Loeve transform. IEEE Sens J 16(1):203–209CrossRefGoogle Scholar
  4. 4.
    Bhateja V, Patel H, Krishn A, Sahu A, Lay-Ekuakille A (2015) Multimodal medical image sensor fusion framework using cascade of wavelet and contourlet transform domains. IEEE Sens J 15(12):6783–6790CrossRefGoogle Scholar
  5. 5.
    Bhatnagar G, Wu QJ, Liu Z (2013) Directive contrast based multimodal medical image fusion in nsct domain. IEEE Trans Multimed 15(5):1014–1024CrossRefGoogle Scholar
  6. 6.
    Burt PJ, Kolczynski RJ (1993) Enhanced image capture through fusion. In: Fourth international conference on computer vision, 1993. Proceedings. IEEE, pp 173–182Google Scholar
  7. 7.
    Candès E, Demanet L, Donoho D, Ying L (2006) Fast discrete curvelet transforms. Multiscale Model Simul 5(3):861–899MathSciNetCrossRefGoogle Scholar
  8. 8.
    Cao L, Jin L, Tao H, Li G, Zhuang Z, Zhang Y (2015) Multi-focus image fusion based on spatial frequency in discrete cosine transform domain. IEEE Signal Process Lett 22(2):220–224CrossRefGoogle Scholar
  9. 9.
    Cao L, Jin L, Tao H, Li G, Zhuang Z, Zhang Y (2015) Multi-focus image fusion based on spatial frequency in discrete cosine transform domain. IEEE Signal Process Lett 22(2):220–224CrossRefGoogle Scholar
  10. 10.
    Choi M, Kim RY, Kim MG (2004) The curvelet transform for image fusion. Int Soc Photogramm Remote Sens ISPRS 2004 35:59–64Google Scholar
  11. 11.
    Collignon A, Maes F, Delaere D, Vandermeulen D, Suetens P, Marchal G (1995) Automated multi-modality image registration based on information theory. In: Information processing in medical imaging, vol 3, pp 263–274Google Scholar
  12. 12.
    Do MN, Vetterli M (2005) The contourlet transform: an efficient directional multiresolution image representation. IEEE Trans Image Process 14(12):2091–2106CrossRefGoogle Scholar
  13. 13.
    Easley G, Labate D, Lim WQ (2008) Sparse directional image representations using the discrete shearlet transform. Appl Comput Harmon Anal 25(1):25–46MathSciNetCrossRefGoogle Scholar
  14. 14.
    Easley G, Labate D, Lim WQ (2008) Sparse directional image representations using the discrete shearlet transform. Appl Comput Harmon Anal 25(1):25–46MathSciNetCrossRefGoogle Scholar
  15. 15.
    Feng F, Ran Q, Li W (2017) Multi-level fusion of graph based discriminant analysis for hyperspectral image classification. Multimed Tools Appl 76(21):22959–22977CrossRefGoogle Scholar
  16. 16.
    Geng P, Huang M, Liu S, Feng J, Bao P (2016) Multifocus image fusion method of ripplet transform based on cycle spinning. Multimed Tools Appl 75(17):10583–10593CrossRefGoogle Scholar
  17. 17.
    Huang R (2008) Some inequalities for the hadamard product and the fan product of matrices. Linear Algebra Appl 428(7):1551–1559MathSciNetCrossRefGoogle Scholar
  18. 18.
    Ji X, Zhang G (2017) Image fusion method of sar and infrared image based on curvelet transform with adaptive weighting. Multimed Tools Appl 76(17):17633–17649CrossRefGoogle Scholar
  19. 19.
    Kadir T, Brady M (2001) Saliency, scale and image description. Int J Comput Vis 45(2):83–105CrossRefGoogle Scholar
  20. 20.
    Kanmani M, Narasimhan V (2017) An optimal weighted averaging fusion strategy for thermal and visible images using dual tree discrete wavelet transform and self tunning particle swarm optimization. Multimed Tools Appl 76(20):20989–21010CrossRefGoogle Scholar
  21. 21.
    Kong W, Liu J (2013) Technique for image fusion based on nonsubsampled shearlet transform and improved pulse-coupled neural network. Opt Eng 52(1):017001–017001CrossRefGoogle Scholar
  22. 22.
    Li H, Manjunath B, Mitra SK (1995) Multisensor image fusion using the wavelet transform. Graph Models Image Process 57(3):235–245CrossRefGoogle Scholar
  23. 23.
    Li S, Kwok JT, Wang Y (2002) Multifocus image fusion using artificial neural networks. Pattern Recogn Lett 23(8):985–997CrossRefGoogle Scholar
  24. 24.
    Li S, Kang X, Hu J (2013) Image fusion with guided filtering. IEEE Trans Image Process 22(7):2864–2875CrossRefGoogle Scholar
  25. 25.
    Lim WQ (2010) The discrete shearlet transform: a new directional transform and compactly supported shearlet frames. IEEE Trans Image Process 19(5):1166–1180MathSciNetCrossRefGoogle Scholar
  26. 26.
    Liu Y, Liu S, Wang Z (2015) A general framework for image fusion based on multi-scale transform and sparse representation. Inf Fusion 24:147–164CrossRefGoogle Scholar
  27. 27.
    Liu Y, Chen X, Peng H, Wang Z (2017) Multi-focus image fusion with a deep convolutional neural network. Inf Fusion 36:191–207CrossRefGoogle Scholar
  28. 28.
    Ma J, Zhao J, Ma Y, Tian J (2015) Non-rigid visible and infrared face registration via regularized gaussian fields criterion. Pattern Recognit 48(3):772–784CrossRefGoogle Scholar
  29. 29.
    Ma J, Chen C, Li C, Huang J (2016) Infrared and visible image fusion via gradient transfer and total variation minimization. Inf Fusion 31:100–109CrossRefGoogle Scholar
  30. 30.
    Ma J, Jiang J, Liu C, Li Y (2017) Feature guided gaussian mixture model with semi-supervised em and local geometric constraint for retinal image registration. Inf Sci 417:128–142MathSciNetCrossRefGoogle Scholar
  31. 31.
    Ma J, Ma Y, Li C (2019) Infrared and visible image fusion methods and applications: a survey. Inf Fusion 45:153–178CrossRefGoogle Scholar
  32. 32.
    Miao Q-G, Shi C, Xu PF, Yang M, Shi YB (2011) A novel algorithm of image fusion using shearlets. Opt Commun 284(6):1540–1547CrossRefGoogle Scholar
  33. 33.
    Mitashe MR, Habib ARB, Razzaque A, Tanima IA, Uddin J (2017) An adaptive digital image watermarking scheme with pso, dwt and xfcm. In: 2017 IEEE international conference on imaging, vision pattern recognition (icIVPR), pp 1–5Google Scholar
  34. 34.
    Mitianoudis N, Stathaki T (2008) Optimal contrast correction for ica-based fusion of multimodal images. IEEE Sens J 8(12):2016–2026CrossRefGoogle Scholar
  35. 35.
    Perona P, Malik J (1990) Scale-space and edge detection using anisotropic diffusion. IEEE Trans Pattern Anal Mach Intell 12(7):629–639CrossRefGoogle Scholar
  36. 36.
    Petrovic VS, Xydeas CS (2004) Gradient-based multiresolution image fusion. IEEE Trans Image Process 13(2):228–237CrossRefGoogle Scholar
  37. 37.
    Prabhakar S, Jain AK (2002) Decision-level fusion in fingerprint verification. Pattern Recognit 35(4):861–874CrossRefGoogle Scholar
  38. 38.
    Summers D (2003) Harvard whole brain atlas: J Neurol Neurosurg Psychiatry 74(3):288CrossRefGoogle Scholar
  39. 39.
    Wang R, Bu F, Jin H, Li L (2007) A feature-level image fusion algorithm based on neural networks. In: 2007 1st international conference on bioinformatics and biomedical engineering, pp 821–824Google Scholar
  40. 40.
    Wenjing T, Fei G, Renren D, Yujuan S, Ping L (2017) Face recognition based on the fusion of wavelet packet sub-images and fisher linear discriminant. Multimed Tools Appl 76(21):22725–22740CrossRefGoogle Scholar
  41. 41.
    Xydeas C, Petrovic V (2000) Objective image fusion performance measure. Electron Lett 36(4):308–309CrossRefGoogle Scholar
  42. 42.
    Yang C, Zhang JQ, Wang XR, Liu X (2008) A novel similarity based quality metric for image fusion. Inf Fusion 9(2):156–160CrossRefGoogle Scholar
  43. 43.
    Yang Y, Que Y, Huang S, Lin P (2016) Multimodal sensor medical image fusion based on type-2 fuzzy logic in nsct domain. IEEE Sensors J 16(10):3735–3745CrossRefGoogle Scholar
  44. 44.
    Yin M, Liu W, Zhao X, Yin Y, Guo Y (2014) A novel image fusion algorithm based on nonsubsampled shearlet transform. Optik - Int J Light Electron Opt 125(10):2274–2282CrossRefGoogle Scholar
  45. 45.
    Zhang X, Li X, Feng Y (2017) Image fusion based on simultaneous empirical wavelet transform. Multimed Tools Appl 76(6):8175–8193CrossRefGoogle Scholar
  46. 46.
    Zhao S, Chen X, Wang S, Li J, Yang W (2003) A new method of remote sensing image decision-level fusion based on support vector machine. In: Proceedings of international conference on recent advances in space technologies, 2003. RAST ’03, pp 91–96Google Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Department of Electronics and Electrical EngineeringIndian Institute of Technology GuwahatiGuwahatiIndia
  2. 2.Department of Computer ScienceChubu UniversityKasugaiJapan

Personalised recommendations