Skip to main content

A Robust Pan-Sharpening Scheme for Improving Resolution of Satellite Images in the Domain of the Nonsubsampled Shearlet Transform

Sensing and Imaging volume 21, Article number: 3 (2020) Cite this article

Abstract

Pan-sharpening is a substantial post-processing task for captured multispectral remotely sensed satellite imagery. Its main purpose is to fuse the high spectral characteristics of the multispectral (MS) images with the high spatial information of the panchromatic (Pan) image to output a sharper MS image (pan-sharpened) that encompasses higher spectral and spatial resolutions. In this paper, we investigate a conception of a new pan-sharpening scheme using the pulse coupled neural network (PCNN) in the nonsubsampled shearlet transform (NSST) domain. This can be done based on two main steps. In the first step, the input MS and Pan images are individually decomposed into multi-scaled and multi-directional coefficients by NSST. Second, the PCNN is applied to the low-frequency coefficients, which are merged by a weighted firing energy fusion rule utilizing the PCNN firing times. The detail coefficients with higher matching value are chosen to be the fused detail coefficients. Lastly, the pan-sharpened image is generated by the inverse NSST. WorldView-2, GeoEye-1, and QuickBird satellite datasets are employed in the experiments which demonstrate that the investigated scheme gained the ability in preserving both the high spatial details and high spectral characteristics simultaneously without involving abundant computation time. In addition, various image quality metrics such as CC, RMSE, RASE, ERGAS, SAM, Q4, QNR, and SCC are adopted to assess the spectral and spatial qualities of the pan-sharpened image. The experimental results and performance analysis illustrated that our scheme improved performance efficiency and achieved superiority over other conventional techniques.

This is a preview of subscription content, access via your institution.

Access options

Buy single article

Instant access to the full article PDF.

USD 39.95

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

References

  1. Alparone, L., Baronti, S., Aiazzi, B., et al. (2016). Spatial methods for multispectral pansharpening: Multiresolution analysis demystified. IEEE Transactions on Geoscience and Remote Sensing, 54(5), 2563–2576.

    Article  Google Scholar 

  2. Yang, C., Zhan, Q., Liu, H., et al. (2018). An IHS-based pan-sharpening method for spectral fidelity improvement using ripplet transform and compressed sensing. Sensors (Basel), 18(11), 3624.

    Article  Google Scholar 

  3. Leung, Y., Liu, J., & Zhang, J. (2014). An improved adaptive intensity-hue saturation method for the fusion of remote sensing images. IEEE Geoscience and Remote Sensing Letters, 11(5), 985–989.

    Article  Google Scholar 

  4. Zhou, X., Liu, J., Liu, S., et al. (2014). A GIHS-based spectral preservation fusion method for remote sensing images using edge restored spectral modulation. ISPRS Journal of Photogrammetry and Remote Sensing, 88(2), 16–27.

    Article  Google Scholar 

  5. Zhou, Z. M., Ma, N., Li, Y. X., et al. (2014). Variational PCA fusion for pan-sharpening very high resolution imagery. Science China Information Sciences, 57(11), 1–10.

    Google Scholar 

  6. Zhang, Y. (2012). Problems in the fusion of commercial high-resolution satellite as well as Landsat 7 images and initial solutions. International Archives of Photogrammetry Remote Sensing and Spatial Information Sciences, 34(4), 587–592.

    Google Scholar 

  7. Laben, C. A., & Brower, B. V. (2000). Process for enhancing the spatial resolution of multispectral imagery using pan-sharpening. U.S. Patent 6011875, Eastman Kodak Company, Tech. Rep.

  8. Aiazzi, B., Alparone, L., Baronti, S., et al. (2002). Context-driven fusion of high spatial and spectral resolution images based on oversampled multiresolution analysis. IEEE Transactions on Geoscience and Remote Sensing, 40(10), 2300–2312.

    Article  Google Scholar 

  9. Li, S., Kwok, J. T., & Wang, Y. (2002). Using the discrete Wavelet frame transform to merge Landsat TM and SPOT panchromatic images. Information Fusion, 3(1), 17–23.

    Article  Google Scholar 

  10. Ioannidou, S., & Karathanassi, V. (2007). Investigation of the dual-tree complex and shift-invariant discrete Wavelet transforms on Quickbird image fusion. IEEE Geoscience and Remote Sensing Letters, 4(1), 166–170.

    Article  Google Scholar 

  11. Nunez, J., Otazu, X., Fors, O., et al. (1999). Multiresolution-based image fusion with additive wavelet decomposition. IEEE Transactions on Geoscience and Remote Sensing, 37(3), 1204–1211.

    Article  Google Scholar 

  12. Aiazzi, B., Alparone, L., Baronti, S., et al. (2012). Advantages of Laplacian pyramids over “à trous” wavelet transforms for pansharpening of multispectral images. In Image and signal processing for remote sensing XVIII (Vol. 8537, pp. 1–10).

  13. Zhong, B., Zhang, C., Liao, M. W., et al. (2018). Spectral preservation fusion for remote sensing images using focus measure operators based on fast discrete curvelet transform and hyperspherical color space. Journal of Applied Remote Sensing, 12(3), 035017.

    Article  Google Scholar 

  14. Helmy, A. K., & El-Tawel, Gh S. (2015). An integrated scheme to improve pan-sharpening visual quality of satellite images. Egyptian Informatics Journal, 16(1), 121–131.

    Article  Google Scholar 

  15. Shah, V. P., Younan, N. H., & King, R. L. (2008). An efficient pan-sharpening method via a combined adaptive-PCA approach and contourlets. IEEE Transactions on Geoscience and Remote Sensing, 46(5), 1323–1335.

    Article  Google Scholar 

  16. El-Mezouar, M. C., Kpalma, K., Taleb, N., et al. (2014). A pan-sharpening based on the non-subsampled Contourlet transform: Application to Worldview-2 imagery. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 7(5), 1806–1815.

    Article  Google Scholar 

  17. Shi, C., Liu, F., Li, L.-L., et al. (2014). Pansharpening algorithm to remove thin cloud via mask dodging and non-sampled shift invariant shearlet transform. Journal of Applied Remote Sensing, 8(1), 1–17.

    Article  Google Scholar 

  18. Biswasa, B., Sena, B. K., & Choudhuria, R. (2015). Remote sensing image fusion using PCNN model parameter estimation by Gamma distribution in shearlet domain. Procedia Computer Science, 70, 304–310.

    Article  Google Scholar 

  19. Easley, G., Labate, D., & Lim, W.-Q. (2008). Sparse directional image representations using the discrete ST. Applied and Computational Harmonic Analysis, 25(1), 25–46.

    Article  MathSciNet  MATH  Google Scholar 

  20. Lari, S. N., & Yazdi, M. (2016). Improved IHS pan-sharpening method based on adaptive injection of à trous wavelet decomposition. International Journal of Signal Processing, Image Processing and Pattern Recognition, 9(3), 291–308.

    Article  Google Scholar 

  21. Marcello, J., Medina, A., & Eugenio, F. (2013). Evaluation of spatial and spectral effectiveness of pixel-level fusion techniques. IEEE Geoscience and Remote Sensing Letters, 10(3), 432–436.

    Article  Google Scholar 

  22. Xiang, T., Yan, L., & Gao, R. (2015). A fusion algorithm for infrared and visible images based on adaptive dual-channel unit-linking PCNN in NSCT domain. Infrared Physics & Technology, 69, 53–61.

    Article  Google Scholar 

  23. Liu, S., Hu, S., & Xiao, Y. (2014). Image separation using wavelet-complex shearlet dictionary. Journal of Systems Engineering and Electronics, 25(2), 314–321.

    Article  Google Scholar 

  24. Lim, W.-Q. (2010). The discrete shearlets transform: A new directional transform and compactly supported shearlets frames. IEEE Transactions on Image Processing, 19(5), 1166–1180.

    Article  MathSciNet  MATH  Google Scholar 

  25. Kong, W., & Lei, Y. (2013). Technique for image fusion between gray-scale visual light and infrared images based on NSST and improved RF. Optik—International Journal of Light and Electron Optics, 124(23), 6423–6431.

    Article  Google Scholar 

  26. Kong, W. (2014). Technique for image fusion based on NSST domain INMF. Optik—International Journal of Light and Electron Optics, 125(11), 2716–2722.

    Article  Google Scholar 

  27. Johnson, J. L., & Padgett, M. L. (1999). PCNN models and applications. IEEE Transaction on Neural Networks, 10(3), 480–498.

    Article  Google Scholar 

  28. Qu, X.-B., Yan, J.-W., Xiao, H.-Z., et al. (2008). Image fusion algorithm based on spatial frequency-motivated pulse coupled neural networks in nonsubsampled contourlet transform domain. Acta Automatica Sinica, 34(12), 1508–1514.

    Article  MATH  Google Scholar 

  29. Piella, G. (2003). A general framework for multiresolution image fusion: From pixels to regions. Information Fusion, 4(4), 259–280.

    Article  Google Scholar 

  30. Moonon, A.-U., & Hu, J. (2015). Multi-focus image fusion based on NSCT and NSST. Sensing and Imaging, 16(1), 1–16.

    Article  Google Scholar 

  31. Vivone, G., Alparone, L., Chanussot, J., et al. (2015). A critical comparison among pansharpening algorithms. IEEE Transactions on Geoscience and Remote Sensing, 53(5), 2565–2586.

    Article  Google Scholar 

  32. Panchal, S., & Thakker, R. (2015). Implementation and comparative quantitative assessment of different multispectral image pansharpening approaches. Signal & Image Processing International Journal (SIPIJ), 6(5), 35–48.

    Article  Google Scholar 

  33. https://www.digitalglobe.com/resources/satellite-information. Accessed 8 Apr 2017.

  34. Wan, W., Yang, Y., & Lee, H. J. (2018). Practical remote sensing image fusion method based on guided filter and improved SML in the NSST domain. Signal, Image and Video Processing, 12(5), 959–966.

    Article  Google Scholar 

  35. Fu, X., Lin, Z., Huang, Y., et al. (2019). A variational pan-sharpening with local gradient constraints. In IEEE conference on computer vision and pattern recognition (CVPR) (pp. 10265–10274).

Download references

Acknowledgements

Funding was provided by Faculty of Computers and Information, Suez Canal University (EG).

Author information

Authors and Affiliations

  1. Mathematics Department, Faculty of Science, Arish University, Arish, Egypt

    Asmaa G. Sulaiman

  2. CS Department, Faculty of Computers and Informatics, Suez Canal University, Ismailia, Egypt

    Walaa H. Elashmawi & Ghada S. El-Tawel

Authors
  1. Asmaa G. Sulaiman
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Walaa H. Elashmawi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ghada S. El-Tawel
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Asmaa G. Sulaiman.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Sulaiman, A.G., Elashmawi, W.H. & El-Tawel, G.S. A Robust Pan-Sharpening Scheme for Improving Resolution of Satellite Images in the Domain of the Nonsubsampled Shearlet Transform. Sens Imaging 21, 3 (2020). https://doi.org/10.1007/s11220-019-0268-5

Download citation

  • Received:

  • Revised:

  • Published:

  • DOI: https://doi.org/10.1007/s11220-019-0268-5

Keywords

  • Pan-sharpening
  • Multispectral
  • Panchromatic
  • NSST
  • PCNN