Skip to main content

Advertisement

SpringerLink
Log in

Pansharpening of Satellite Images with Convolutional Sparse Coding and Adaptive PCNN-Based Approach

  • Research Article
  • Published:
Journal of the Indian Society of Remote Sensing Aims and scope Submit manuscript

Abstract

In remote sensing, Pansharpening process has great significance in many practical applications like map updating, hazard monitoring, target recognition and object classification. Satellite sensors capturing panchromatic and multispectral images with complementary characteristics due to tradeoff between IFOV (instantaneous field of view) and SNR (signal-to-noise ratio). Pansharpening is a process of combining PAN (panchromatic) image of high spatial resolution with MS (multispectral) image of high spectral resolution to get image of high spectral and spatial resolution. In Pansharpening, balancing between extraction of information and injection of information is crucial point; misbalancing can cause intensity distortion. Proposed method is a combination of CSC (convolution sparse coding) and adaptive PCNN (pulse coupled neural network) approach. NSST (non-sub-sampled shearlet transform) is used for band separation of PAN and MS image. CSC is used for fusing low pass sub-bands, and adaptive PCNN method is employed for fusing high pass sub-bands. Five datasets with different geographical areas like mountain, urban and vegetation area are used for experiment purpose. Visual results and quantitative index analysis reflect the superiority of proposed method in preserving spectral details in pansharpened image.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Code availability

GitHub—sergiovitale/pansharpening-cnn-matlab-version: This MATLAB implementation of the paper Target-Adaptive CNN-Based Pansharpening

https://openremotesensing.net/knowledgebase/a-critical-comparison-among-pansharpening-algorithms/

References

  • Aiazzi, B., Alparone, L., Baronti, S., Garzelli, A., & Selva, M. (2006). MTF-tailored multiscale fusion of high-resolution MS and Pan imagery. Photogrammetric Engineering & Remote Sensing, 72, 591–596.

    Article  Google Scholar 

  • Amro, I., Mateos, J., Vega, M., et al. (2011). A survey of classical methods and new trends in pansharpening of multispectral images. EURASIP J. Adv. Signal Process. https://doi.org/10.1186/1687-6180-2011-79

    Article  Google Scholar 

  • Ballester, C., et al. (2006). A variational model for P+XS image fusion. International Journal of Computer Vision, 69(1), 43–58. https://doi.org/10.1007/s11263-006-6852-x

    Article  Google Scholar 

  • Carper, W. J., Lillesand, T. M., & Kiefer, R. W. (1990). The use of intensity-hue-saturation transformations for merging SPOT panchromatic and multispectral image data. Photogrammetric Engineering & Remote Sensing, 56(4), 459–467.

    Google Scholar 

  • Chen, C., Meng, Y., Luo, Q., & Zhou, Z. (2018). A novel variational model for pan-sharpening based on L1 regularization. Remote Sensing Letters, 9(2), 170–179.

    Article  Google Scholar 

  • Chen, F., et al. (2012). Fusion of remote sensing images using improved ica mergers based on wavelet decomposition. Procedia Engineering, 29, 2938–2943.

    Article  Google Scholar 

  • Choi, J., Yu, K., & Kim, Y. (2011). A New Adaptive Component-Substitution-Based Satellite Image Fusion by Using Partial Replacement. IEEE Transactions on Geoscience and Remote Sensing, 49(1), 295–309. https://doi.org/10.1109/TGRS.2010.2051674

    Article  Google Scholar 

  • Chu, H., & Zhu, W. (2008). Fusion of IKONOS Satellite Imagery Using IHS Transform and Local Variation. IEEE Geoscience and Remote Sensing Letters, 5(4), 653–657. https://doi.org/10.1109/LGRS.2008.2002034

    Article  Google Scholar 

  • Deng, L., Vivone, G., Guo, W., Dalla, M. Mura, J. Chanussot, "A variational pansharpening approach based on reproducible kernel Hilbert space and heaviside function," 2017 IEEE International Conference on Image Processing (ICIP), Beijing, China, 2017, pp. 535–539

  • Do, M. N., & Vetterli, M. (2005). The contourlet transform: an efficient directional multiresolution image representation. IEEE Transactions on Image Processing, 14(12), 2091–2106.

    Article  Google Scholar 

  • Duran, J. (2017). A survey of pansharpening methods with a new band-decoupled variational model. ISPRS Journal of Photogrammetry and Remote Sensing, 125, 78–105. https://doi.org/10.1016/j.isprsjprs.2016.12.013

    Article  Google Scholar 

  • Fu, X., Lin, Z., Huang Y., Ding X., (2019). "A Variational Pan-Sharpening With Local Gradient Constraints," 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA, 2019, pp. 10257–10266, doi: https://doi.org/10.1109/CVPR.2019.01051.

  • Fang, F., Li, F., Shen, C., & Zhang, G. (2013). A variational approach for pan-sharpening. IEEE Transactions on Image Processing. https://doi.org/10.1109/TIP.2013.2258355

    Article  Google Scholar 

  • Fei, R., Zhang, J., Liu, J., Du, F., Chang, P., & Hu, J. (2019). Convolutional Sparse Representation of Injected Details for Pansharpening. IEEE Geoscience and Remote Sensing Letters, 16(10), 1595–1599. https://doi.org/10.1109/LGRS.2019.2904526

    Article  Google Scholar 

  • Ganasala, P., & Kumar, V. (2015). Feature-Motivated Simplified Adaptive PCNN-Based Medical Image Fusion Algorithm in NSST Domain. Journal of Digital Imaging, 29, 73–85.

    Article  Google Scholar 

  • Garzelli, A., Nencini, F., & Capobianco, L. (2008). Optimal MMSE pan sharpening of very high resolution multispectral images. IEEE Transactions on Geoscience and Remote Sensing, 46(1), 228–236.

    Article  Google Scholar 

  • Ghahremani, M., & Ghassemian, H. (2015). Remote sensing image fusion using ripplet transform and compressed sensing. IEEE Geoscience and Remote Sensing Letters, 12(3), 502–506. https://doi.org/10.1109/LGRS.2014.2347955

    Article  Google Scholar 

  • Gogineni, R., & Chaturvedi, A. (2018). Sparsity inspired pan-sharpening technique using multi-scale learned dictionary. ISPRS Journal of Photogrammetry and Remote Sensing, 146, 360–372.

    Article  Google Scholar 

  • Gogineni, R., & Chaturvedi, A. (2019). A Robust Pansharpening Algorithm Based on Convolutional Sparse Coding for Spatial Enhancement. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 12(10), 4024–4037.

    Article  Google Scholar 

  • Huang, W., Xiao, L., Wei, Z., Liu, H., & Tang, S. (2015). A New Pan-Sharpening Method With Deep Neural Networks. IEEE Geoscience and Remote Sensing Letters, 12(5), 1037–1041.

    Article  Google Scholar 

  • Jiang, Y., Ding, X., Zeng, D., Huang, Y., and Paisley, J. (2015). “Pan-sharpening with a hyper-Laplacian penalty. In Proc. Int. Conf. Comput. Vis. ICCV, pp. 540–548.

  • Joshi, M. V., Bruzzone, L., & Chaudhuri, S. (2006). A model-based approach to multiresolution fusion in remotely sensed images. IEEE Transactions on Geoscience and Remote Sensing, 44(9), 2549–2562. https://doi.org/10.1109/TGRS.2006.873340

    Article  Google Scholar 

  • Laben, C. A., Brower, B.V. (2000). Process for enhancing the spatial resolution of multispectral imagery using pan-sharpening, U.S. Patent 6011875 A 2000.

  • Li, S., & Yang, B. (2011). A new pan-sharpening method using a compressed sensing technique. IEEE Transactions on Geoscience and Remote Sensing, 49(2), 738–746. https://doi.org/10.1109/TGRS.2010.2067219

    Article  Google Scholar 

  • Lolli, S., Alparone, L., Garzelli, A., & Vivone, G. (2017). Haze correction for contrast-based multispectral pansharpening. IEEE Geoscience and Remote Sensing Letters, 14(12), 2255–2259.

    Article  Google Scholar 

  • Masi, G., et al. (2016). Pansharpening by convolutional neural networks. Remote Sensing, 8(7), 594. https://doi.org/10.3390/rs8070594

    Article  Google Scholar 

  • Möller, M., Wittman, T., Bertozzi, A. L., & Burger, M. (2012). A Variational Approach for Sharpening High Dimensional Images. SIAM Journal on Imaging Sciences, 5(1), 150–178. https://doi.org/10.1137/100810356

    Article  Google Scholar 

  • Nencini, F., Garzelli, A., Baronti, S., & Alparone, L. (2007). Remote sensing image fusion using the curvelet transform. Information Fusion, 8(2), 143–156. https://doi.org/10.1016/j.inffus.2006.02.001

    Article  Google Scholar 

  • Nünez, J., Otazu, X., Fors, O., Prades, A., Palà, V., & Arbiol, R. (1999). Multiresolution-based image fusion with additive wavelet decomposition. IEEE Transactions on Geoscience and Remote Sensing. https://doi.org/10.1109/36763274

    Article  Google Scholar 

  • Ophir, B. (2011). MIchael Lustig, and Michael Elad, “Multi-scale dictionary learning using wavelets”, IEEE Journal of Selected Topics in Sig. Proc., 5(5), 1014–1024.

    Google Scholar 

  • Otazu, X., et al. (2005). Introduction of sensor spectral response into image fusion methods. application to wavelet-based methods. IEEE Transactions on Geoscience and Remote Sensing, 43, 2376–2385.

    Article  Google Scholar 

  • Palsson, F., Sveinsson, J. R., & Ulfarsson, M. O. (2014). A New Pansharpening Algorithm Based on Total Variation. IEEE Geoscience and Remote Sensing Letters, 11(1), 318–322. https://doi.org/10.1109/LGRS.2013.2257669

    Article  Google Scholar 

  • Panchal, S., & Thakker, R. A. (2017). Improved image pansharpening technique using nonsubsampled contourlet transform with sparse representation. Journal of the Indian Society of Remote Sensing, 45(3), 385–394.

    Article  Google Scholar 

  • Rahmani, S., et al. (2010). An adaptive ihs pan-sharpening method. IEEE Geoscience and Remote Sensing Letters, 7(4), 746–750.

    Article  Google Scholar 

  • Sahu, A., Bhateja, V., Krishn A. and Himanshi, "Medical image fusion with Laplacian Pyramids," 2014 International Conference on Medical Imaging, m-Health and Emerging Communication Systems (MedCom), Greater Noida, 2014, pp. 448–453, doi: https://doi.org/10.1109/MedCom.2014.7006050.

  • Scarpa, G., Vitale, S., & Cozzolino, D. (2018). Target-adaptive CNN-based pansharpening. IEEE Transactions on Geoscience and Remote Sensing, 56(9), 5443–5457. https://doi.org/10.1109/TGRS.2018.2817393

    Article  Google Scholar 

  • Shah, V. P., Younan, N. H., & King, R. L. (2008). “An efficient pan-sharpening method via a combined adaptive PCA approach and contourlets”,. IEEE Transactions on Geoscience and Remote Sensing, 46(5), 1323–1335.

    Article  Google Scholar 

  • Vicinanza, M. R., Restaino, R., Vivone, G., Dalla Mura, M., & Chanussot, J. (2015). A Pansharpening Method Based on the Sparse Representation of Injected Details. IEEE Geoscience and Remote Sensing Letters, 12(1), 180–184.

    Article  Google Scholar 

  • Vivone, G., et al. (2015). A Critical Comparison Among Pansharpening Algorithms. IEEE Transactions on Geoscience and Remote Sensing, 53(5), 2565–2586. https://doi.org/10.1109/TGRS.2014.2361734

    Article  Google Scholar 

  • Wang, X., Bai, S., Li, Z., Song, R., & Tao, J. (2019). The PAN and MS image pansharpening algorithm based on adaptive neural network and sparse representation in the NSST domain. IEEE Access, 7, 52508–52521. https://doi.org/10.1109/ACCESS.2019.2910656

    Article  Google Scholar 

  • Yin, H. (2015). Sparse representation based pansharpening with details injection model. Signal Processing. https://doi.org/10.1016/j.sigpro.2014.12.017

    Article  Google Scholar 

  • Zhu, X., & Bamler, R. (2013). “A sparse image fusion algorithm with applicationto pan-sharpening”,IEEE Trans. Geosci. Remote Sens., 51(5), 2827–2836.

    Article  Google Scholar 

  • Zhu, X. X., Grohnfeldt, C., & Bamler, R. (2016). Exploiting joint sparsity for pansharpening: The J-SparseFI algorithm. IEEE Transactions on Geoscience and Remote Sensing, 54(5), 2664–2681.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Electronics and Communication Department, Vishwakarma Government Engineering College Chandkheda, Gujarat, 382424, India

    Dhara J. Sangani

  2. Department of ECE, Government Engineering College, Rajkot, India

    Rajesh A. Thakker

  3. Computer Engineering, Gujarat Technological University, Ahmedabad, India

    S. D. Panchal

  4. Department of Electronics & Communications Engineering, Dhanekula Institute of Engineering & Technology, Vijayawada, 521139, India

    Rajesh Gogineni

Authors
  1. Dhara J. Sangani
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Rajesh A. Thakker
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. S. D. Panchal
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Rajesh Gogineni
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Dhara J. Sangani.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Sangani, D.J., Thakker, R.A., Panchal, S.D. et al. Pansharpening of Satellite Images with Convolutional Sparse Coding and Adaptive PCNN-Based Approach. J Indian Soc Remote Sens 49, 2989–3004 (2021). https://doi.org/10.1007/s12524-021-01440-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12524-021-01440-4

Keywords

Search

Navigation