Advertisement

Multimedia Tools and Applications

, Volume 77, Issue 7, pp 8327–8358 | Cite as

Multi-focus image fusion based on nonsubsampled compactly supported shearlet transform

  • Chunyu Wei
  • Bingyin Zhou
  • Wei Guo
Article
  • 248 Downloads

Abstract

Multi-focus image fusion, which aims to combine multi-focus images of a scene to construct an all-in-focus image, has become a major topic in image processing. Different methods have been proposed in spatial or transform domain. But many methods usually suffer from fusion quality degradations, such as contrast reduction, artificial edges, and discontinuous phenomena at boundaries of focused regions, which may cause issues when going for further processing. In order to overcome these problems, we introduce a nonsubsampled compactly supported shearlet transform (NSCSST), which possesses multi-scale, multi-direction, translation invariance and spatial localization characteristics that are very important for image fusion in transform domain. The transform can be implemented sequentially by the shear transform and the separable anisotropic nonsubsampled wavelet transform (SANSWT). Furthermore, we propose a new image fusion method based on NSCSST. It consists of two aspects: multi-direction fusion and transform domain fusion, which respectively correspond to the shear transform and the SANSWT of NSCSST. For each sheared image pair, the SANSWT coefficients are firstly fused by the transform domain fusion rules. And then, the final fused image is obtained by the multi-direction fusion rules, ranging from the simple averaging method to the proposed complex genetic algorithm based method. Experimental results show that our method outperforms some other methods, such as the method based on bilateral gradient, the method based on nonsubsampled contourlet transform, the method based on simultaneous empirical wavelet transform, and the method based on guided filtering.

Keywords

Multi-focus image fusion Shearlet transform Compactly supported shearlet Translation invariance 

Notes

Acknowledgments

The authors would like to thank the editor and anonymous reviewers for their detailed review and valuable comments. This work was supported by the NSF of China (No. 11301137), the NSF of Hebei Province, China (No. A2014205100), the Educational Commission of Hebei Province, China (No. ZD2014062).

References

  1. 1.
    Aiazzi B, Alparone L, Barducci A, Baronti S, Pippi I (1999) Multispectral fusion of multisensor image data by the generalized Laplacian pyramid. In: Proceeding of 1999 IEEE international geoscience and remote sensing symposium, vol 2. IEEE, pp 1183–1185Google Scholar
  2. 2.
    Aslantas V, Kurban R (2009) A comparison of criterion functions for fusion of multi-focus noisy images. Opt Commun 282(16):3231–3242CrossRefGoogle Scholar
  3. 3.
    Aslantas V, Toprak AN (2014) A pixel based multi-focus image fusion method. Opt Commun 332(1):350–358CrossRefGoogle Scholar
  4. 4.
    Baradarani A, Wu QMJ, Ahmadi M, Mendapara P (2012) Tunable halfband-pair wavelet filter banks and application to multifocus image fusion. Pattern Recogn 45(2):657–671zbMATHCrossRefGoogle Scholar
  5. 5.
    Candès E, Demanet L, Donoho D, Ying L (2006) Fast discrete curvelet transforms. Multiscale Model Simul 5(3):861–899MathSciNetzbMATHCrossRefGoogle Scholar
  6. 6.
    Cao Y, Li S, Hu J (2011) Multi-focus image fusion by nonsubsampled shearlet transform. In: Proceedings of the 6th international conference on image and graphics. IEEE, pp 17–21Google Scholar
  7. 7.
    Chen L, Li J, Chen C (2013) Regional multifocus image fusion using sparse representation. Opt Express 21(4):5182–5197CrossRefGoogle Scholar
  8. 8.
    Cunha ALD, Zhou J, Do MN (2006) The nonsubsampled contourlet transform: theory, design, and applications. IEEE Trans Image Process 15(10):3089–3101CrossRefGoogle Scholar
  9. 9.
    Desai UY, Mizuki MM, Masaki I, Horn BKP (1996) Edge and mean based image compression. Technical Report AIM-1584, Massachusetts Institute of Technology, Cambridge, MA USAGoogle Scholar
  10. 10.
    Ding G, Guo Y, Zhou J, Gao Y (2016) Large-scale cross-modality search via collective matrix factorization hashing. IEEE Trans Image Process 25(11):5427–5440MathSciNetCrossRefGoogle Scholar
  11. 11.
    Do MN, Vetterli M (2002) Contourlets: a directional multiresolution image representation. In: Proceedings of 2002 international conference on image processing, vol 1. IEEE, pp 357–360Google Scholar
  12. 12.
    Do MN, Vetterli M (2005) The contourlet transform: an efficient directional multiresolution image representation. IEEE Trans Image Process 14(12):2091–2106CrossRefGoogle Scholar
  13. 13.
    Easley G, Labate D, Lim WQ (2008) Sparse directional image representations using the discrete shearlet transform. Appl Comput Harmon Anal 25(1):25–46MathSciNetzbMATHCrossRefGoogle Scholar
  14. 14.
    Eltoukhy HA, Kavusi S (2003) A computationally efficient algorithm for multi-focus image reconstruction. In: Proceedings of SPIE electronic imaging, pp 332–341Google Scholar
  15. 15.
    Eskicioglu AM, Fisher PS (1995) Image quality measures and their performance. IEEE Trans Commun 43(12):2959–2965CrossRefGoogle Scholar
  16. 16.
    Gao G, Xu L, Feng D (2013) Multi-focus image fusion based on non-subsampled shearlet transform. IET Image Process 7(6):633–639CrossRefGoogle Scholar
  17. 17.
    Gao Y, Wang M, Tao D, Ji R, Dai Q. (2012) 3-D object retrieval and recognition with hypergraph analysis. IEEE Trans Image Process 21(9):4290–4303MathSciNetzbMATHCrossRefGoogle Scholar
  18. 18.
    Geng P, Huang M, Liu S, Feng J, Bao P (2016) Multifocus image fusion method of ripplet transform based on cycle spinning. Multimed Tools Appl 75(17):10,583–10,593CrossRefGoogle Scholar
  19. 19.
    Goshtasby AA, Nikolov S (2007) Image fusion: advances in the state of the art. Information Fusion 8(2):114–118CrossRefGoogle Scholar
  20. 20.
    Guo D, Yan J, Qu X (2015) High quality multi-focus image fusion using self-similarity and depth information. Opt Commun 338:138–144CrossRefGoogle Scholar
  21. 21.
    Guo K, Kutyniok G, Labate D (2006) Sparse multidimensional representations using anisotropic dilation and shear operators. In: Wavelets and splines. Nashboro Press, Athens, GA, pp 189–201Google Scholar
  22. 22.
    Guo K, Labate D (2007) Optimally sparse multidimensional representation using shearlets. SIAM J Math Anal 39(1):298–318MathSciNetzbMATHCrossRefGoogle Scholar
  23. 23.
    Holland JH (1992) Adaptation in natural and artificial systems. MIT Press, Cambridge, MA USAGoogle Scholar
  24. 24.
    Holshneider M, Kronland-Martinet R, Morlet J, Tchamitchian P (1989) A real-time algorithm for signal analysis with the help of the wavelet transform. In: Combes JM, Grossmann A, Tchamitchian P (eds) Proceedings of the International conference time-frequency methods and phase space wavelets. Springer-Verlag, Berlin, pp 286–297Google Scholar
  25. 25.
    Huang W, Jing Z (2007) Evaluation of focus measures in multi-focus image fusion. Pattern Recogn Lett 28(4):493–500CrossRefGoogle Scholar
  26. 26.
    Ji X, Zhang G (2015) Image fusion method of SAR and infrared image based on curvelet transform with adaptive weighting. Multimed Tools Appl 1–17. doi: 10.1007/s11042-015-2879-8
  27. 27.
    Kutyniok G, Labate D (2009) Resolution of the wavefront set using continuous shearlets. Trans Am Math Soc 361(5):2719–2754MathSciNetzbMATHCrossRefGoogle Scholar
  28. 28.
    Kutyniok G, Lim WQ, Zhuang X (2012) Digital shearlet transforms. In: Kutyniok G, Labate D (eds) Shearlets: Multiscale analysis for multivariate data, chap 7.Birkhäuser Basel, pp 239–282Google Scholar
  29. 29.
    Labate D, Lim WQ, Kutyniok G, Weiss G (2005) Sparse multidimensional representation using shearlets. In: Papadakis M, Laine AF, Unser MA (eds) Proceedings of the SPIE wavelets XI, vol 5914, pp 254–262Google Scholar
  30. 30.
    Lewis JJ, O’callaghan RJ, Nikolov SG, Bull DR, Canagarajah N (2007) Pixel- and region-based image fusion with complex wavelets. Information Fusion 8(2):119–130CrossRefGoogle Scholar
  31. 31.
    Li C, Yang X, Chu B, Lu W, Pang L (2010) A new image fusion quality assessment method based on contourlet and SSIM. In: Proceedings of the 3rd IEEE international conference on computer science and information technology, vol 5. IEEE, pp 246–249Google Scholar
  32. 32.
    Li S, Kang X (2012) Fast multi-exposure image fusion with median filter and recursive filter. IEEE Trans Consum Electron 58(2):626–632CrossRefGoogle Scholar
  33. 33.
    Li S, Kang X, Hu J (2013) Image fusion with guided filtering. IEEE Trans Image Process 22(7):2864–2875CrossRefGoogle Scholar
  34. 34.
    Li S, Kwok JT, Wang Y (2001) Combination of images with diverse focuses using the spatial frequency. Information Fusion 2(3):169–176CrossRefGoogle Scholar
  35. 35.
    Li S, Kwok JT, Wang Y (2002) Multifocus image fusion using artificial neural networks. Pattern Recogn Lett 23(8):985–997zbMATHCrossRefGoogle Scholar
  36. 36.
    Li S, Yang B (2008) Multifocus image fusion using region segmentation and spatial frequency. Image Vis Comput 26(7):971–979CrossRefGoogle Scholar
  37. 37.
    Li S, Yang B, Hu J (2011) Performance comparison of different multi-resolution transforms for image fusion. Information Fusion 12(2):74–84CrossRefGoogle Scholar
  38. 38.
    Lim WQ (2010) The discrete shearlet transform: a new directional transform and compactly supported shearlet frames. IEEE Trans Image Process 19(5):1166–1180MathSciNetzbMATHCrossRefGoogle Scholar
  39. 39.
    Lin Z, Ding G, Han J, Wang J (2016) Cross-view retrieval via probability-based semantics-preserving hashing. IEEE Trans Cybern 1–14. doi: 10.1109/TCYB.2016.2608906
  40. 40.
    Lin Z, Ding G, Hu M, Lin Y, Ge SS (2014) Image tag completion via dual-view linear sparse reconstructions. Comput Vis Image Underst 124:42–60CrossRefGoogle Scholar
  41. 41.
    Liu X, Zhou Y, Wang J (2014) Image fusion based on shearlet transform and regional features. AEU-Int J Electron C 68(6):471–477CrossRefGoogle Scholar
  42. 42.
    Liu Y, Liu S, Wang Z (2015) A general framework for image fusion based on multi-scale transform and sparse representation. Information Fusion 24:147–164CrossRefGoogle Scholar
  43. 43.
    Miao Q, Shi C, Li W (2013) Image fusion based on shearlets. In: Miao Q (ed) New advances in image fusion, chap 7. Intech, pp 113–133Google Scholar
  44. 44.
    Miao Q, Shi C, Xu P, Yang M, Shi Y (2011) Multi-focus image fusion algorithm based on shearlets. Chin Opt Lett 9(4):1–5Google Scholar
  45. 45.
    Miao Q, Shi C, Xu P, Yang M, Shi Y (2011) A novel algorithm of image fusion using shearlets. Opt Commun 284(6):1540–1547CrossRefGoogle Scholar
  46. 46.
    Miao Q, Wang B (2005) A novel adaptive multi-focus image fusion algorithm based on PCNN and sharpness. In: Proceedings of SPIE, vol 5778. pp 704–712Google Scholar
  47. 47.
    Moonon AU, Hu J (2015) Multi-focus image fusion based on NSCT and NSST. Sensing and Imaging 16(1):1–16CrossRefGoogle Scholar
  48. 48.
    Naidu V, Raol J (2008) Pixel-level image fusion using wavelets and principal component analysis. Def Sci J 58(3):338–352CrossRefGoogle Scholar
  49. 49.
    Nayar SK, Nakagawa Y (1994) Shape from focus. IEEE Trans Pattern Anal Mach Intell 16(8):824–831CrossRefGoogle Scholar
  50. 50.
    Nejatia M, Samavi S, Shiranib S (2015) Multi-focus image fusion using dictionary-based sparse representation. Information Fusion 25:72–84CrossRefGoogle Scholar
  51. 51.
    Pajares G, de la Cruz JM (2004) A wavelet-based image fusion tutorial. Pattern Recogn 37(9):1855–1872CrossRefGoogle Scholar
  52. 52.
    Patel R, Rajput M, Parekh P (2015) Comparative study on multi-focus image fusion techniques in dynamic scene. Int J Comput Appl 109(6):5–9Google Scholar
  53. 53.
    Pennec EL, Mallat S (2005) Sparse geometric image representations with bandelets. IEEE Trans Image Process 14(4):423–438MathSciNetCrossRefGoogle Scholar
  54. 54.
    Peyré G, Mallat S (2005) Discrete bandelets with geometric orthogonal filters. In: Proceedings of 2005 IEEE International Conference on Image Processing, IEEE, pp i–65–8Google Scholar
  55. 55.
    Piella G, Heijmans H (2003) A new quality metric for image fusion. In: Proceedings of 2003 International Conference on Image Processing. IEEE, Barcelona, Spain, pp 173–176Google Scholar
  56. 56.
    Qu X, Yan J, Xiao H, Zhu Z (2008) Image fusion algorithm based on spatial frequency-motivated pulse coupled neural networks in nonsubsampled contourlet transform domain. Acta Automatica Sinica 34(12):1508–1514zbMATHCrossRefGoogle Scholar
  57. 57.
    Qu X, Yan J, Xie G, Zhu Z, Chen B (2007) A novel image fusion algorithm based on bandelet transform. Chin Opt Lett 5(10):569–572Google Scholar
  58. 58.
    Qu XB, Yan JW, Yang GD (2005) Sum-modified-Laplacian-based multifocus image fusion method in cycle spinning sharp frequency localized contourlet transform domain. Opt Precis Eng 13(2)Google Scholar
  59. 59.
    Sharma M (2016) A review: image fusion techniques and applications. Int J Comput Sci Inf Technol 7(3):1082–1085Google Scholar
  60. 60.
    Tian J, Chen L (2012) Adaptive multi-focus image fusion using a wavelet-based statistical sharpness measure. Signal Process 92(9):2137–2146CrossRefGoogle Scholar
  61. 61.
    Tian J, Chen L, Ma L, Yu W (2011) Multi-focus image fusion using a bilateral gradient-based sharpness criterion. Opt Commun 284(1):80–87CrossRefGoogle Scholar
  62. 62.
    Wang H, Nie C, Li Y, Zhang K, Chen L (2011) A novel fusion algorithm for multi-focus image. In: Zhang J (ed) Applied informatics and communication, communications in computer and information science, vol 227. Springer, Berlin Heidelberg, pp 641–647Google Scholar
  63. 63.
    Wang J, Wang W, Li B, Xu G, Zhang R, Zhang J (2016) Exposure fusion via sparse representation and shiftable complex directional pyramid transform. Multimed Tools Appl pp 1–21. doi: 10.1007/s11042-016-3868-2
  64. 64.
    Wang Z, Bovik AC (2002) A universal image quality index. IEEE Signal Process Lett 9(3):81–84CrossRefGoogle Scholar
  65. 65.
    Wang Z, Ma Y, Gu J (2010) Multi-focus image fusion using PCNN. Pattern Recogn 43(6):2003–2016zbMATHCrossRefGoogle Scholar
  66. 66.
    Xiang T, Yan L, Gao R (2015) A fusion algorithm for infrared and visible images based on adaptive dual-channel unit-linking PCNN in NSCT domain. Infrared Phys Technol 69:53–61CrossRefGoogle Scholar
  67. 67.
    Xu J, Yang L, Wu D (2011) Ripplet: a new transform for image processing. J Vis Commun Image Represent 21(7):627–639CrossRefGoogle Scholar
  68. 68.
    Yang L, Guo BL, Ni W (2008) Multimodality medical image fusion based on multiscale geometric analysis of contourlet transform. Neurocomputing 72(1-3):203–211CrossRefGoogle Scholar
  69. 69.
    Yang Y, Huang S, Gao J, Qian Z (2014) Multi-focus image fusion using an effective discrete wavelet transform based algorithm. Meas Sci Rev 14(2):102–108CrossRefGoogle Scholar
  70. 70.
    Yang Y, Tong S, Huang S, Lin P (2015) Multifocus image fusion based on NSCT and focused area detection. IEEE Sensors J 15(5):2824–2838Google Scholar
  71. 71.
    You X, Chen Q, Fang B, Tang YY (2006) Thinning character using modulus minima of wavelet transform. Int J Pattern Recognit Artif Intell 20(3):361–375CrossRefGoogle Scholar
  72. 72.
    Zhang D, You X, Wang P, Yanushkevich SN, Tang YY (2009) Facial biometrics using nontensor product wavelet and 2D discriminant techniques. Int J Pattern Recognit Artif Intell 23(3):521–543CrossRefGoogle Scholar
  73. 73.
    Zhang Q, Guo BL (2009) Multifocus image fusion using the nonsubsampled contourlet transform. Signal Process 89(7):1334–1346zbMATHCrossRefGoogle Scholar
  74. 74.
    Zhang X, Li X, Feng Y (2017) Image fusion based on simultaneous empirical wavelet transform. Multimed Tools Appl 76(6):8175–8193CrossRefGoogle Scholar
  75. 75.
    Zhao S, Yao H, Gao Y, Ding G, Chua TS (2016) Predicting personalized image emotion perceptions in social networks. IEEE Trans Affective Comput. doi: 10.1109/TAFFC.2016.2628787
  76. 76.
    Zhao S, Yao H, Gao Y, Ji R, Ding G (2017) Continuous probability distribution prediction of image emotions via multi-task shared sparse regression. IEEE Trans Multimedia 19(3):632–645CrossRefGoogle Scholar
  77. 77.
    Zhao S, Yao H, Yang Y, Zhang Y (2014) Affective image retrieval via multi-graph learning. In: Proceedings of the 22nd ACM international conference on multimedia. ACM, pp 1025–1028Google Scholar
  78. 78.
    Zhao S, Yao H, Zhang Y, Wang Y, Liu S (2015) View-based 3D object retrieval via multi-modal graph learning. Signal Process 112:110–118CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2017

Authors and Affiliations

  1. 1.College of Mathematics and Information SciencesHebei Normal UniversityShijiazhuangChina
  2. 2.Hebei Key Laboratory of Computational Mathematics and ApplicationsHebei Normal UniversityShijiazhuangChina

Personalised recommendations