Skip to main content
SpringerLink
Log in

Assessment of generative adversarial networks for cloud occlusion removal on remotely sensed images

  • Original Paper
  • Published:
Arabian Journal of Geosciences Aims and scope Submit manuscript

Abstract

Remotely sensed images acquired by optical sensors are frequently affected by cloud cover, which can impede interpretation and analysis. Generative adversarial networks (GANs) have proved to be effective in removing cloud occlusions from such images. In this study, we compare the performance of two GAN-based algorithms, CycleGAN and Pix2Pix, for cloud cover removal on an open-source Remote sensing Image Cloud rEmoving (RICE) dataset. Our main objective is to evaluate the effectiveness of these models in terms of image quality metrics and training time. The obtained results show that the CycleGAN model achieved superior image quality, with a peak signal-to-noise ratio (PSNR) of 29.589 dB, mean square error (MSE) of 3.662, and structural similarity index (SSIM) of 0.907, with a training time of 5 h and 10 min. On the other hand, the Pix2Pix model, while achieving lower image quality metrics of 28.160 dB at 100 epochs, mean square error (MSE) of 46.760, and structural similarity index (SSIM) of 0.761, has a shorter training time of 2 h and 20 min, making it more suitable for applications that require faster processing. Our study provides valuable insights into the performance of deep learning models for cloud removal in remotely sensed images. The findings suggest that both CycleGAN and Pix2Pix models are promising approaches for cloud removal, with the potential for further optimization and development.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  • Ajayi OG, Ojima A (2022) Performance evaluation of selected cloud occlusion removal algorithms on remote sensing imagery. Remote Sens Appl: Soc Environ 25:1–17. https://doi.org/10.1016/j.rsase.2022.100700

    Article  Google Scholar 

  • Ajayi OG, Salubi AA, Angbas AF, Odigure MG (2017) Generation of accurate digital elevation models from UAV acquired low percentage overlapping images. Int J Remote Sens 8–10(38):3113–3134. https://doi.org/10.1080/01431161.2017.1285085

    Article  Google Scholar 

  • Ajayi OG, Palmer M (2020) Modelling 3D topography by comparing airborne LiDAR data with Unmanned Aerial System (UAS) photogrammetry under multiple imaging conditions. Journal of Geomatics and Planning 6(2):122–138. https://doi.org/10.14710/geoplanning.6.2.122-138

  • Ajayi OG, Palmer M, Salubi AA (2018) Modelling farmland topography for suitable site selection of dam construction using unmanned aerial vehicle (UAV) photogrammetry. Remote Sens Appl: Soc Environ 11:220–230. https://doi.org/10.1016/j.rsase.2018.07.007

    Article  Google Scholar 

  • Bertalmio M, Vese L, Sapiro G, Osher S (2003) Simultaneous structure and texture image inpainting. IEEE T Image Process 12:882–889

    Article  Google Scholar 

  • Chen S, Chen X, Chen X, Chen J, Cao X, Shen M, Yang W, Cui X (2018) A novel cloud removal method based on IHOT and the cloud trajectories for Landsat imagery. Remote Sens-Basel 10(7):1040. https://doi.org/10.3390/rs10071040

    Article  Google Scholar 

  • Chen Y, Tang L, Yang X, Fan R, Bilal M, Li Q (2020) Thick clouds removal from multitemporal ZY-3 satellite images using deep Learning. IEEE J Sel Top Appl 13:143–153

    Google Scholar 

  • Cihlar J, Howarth J (1994) Detection and removal of cloud contamination from AVHRR images. IEEE T Geosci Remote 32:83–589

    Article  Google Scholar 

  • Criminisi A, Perez P, Toyama K (2004) Region filling and object removal by exemplar-based image inpainting. IEEE T Image Process 13:1200–1212

    Article  Google Scholar 

  • Dai Y, Xu Y, Li Z, Chen X, Li J (2018) Cloud detection of remote sensing images based on deep convolutional neural network. Remote Sens-Basel 10(11):1704

    Google Scholar 

  • Daoyu L, Guangluan X, Xiaoke W, Yang W, Kun F (2019) A remote sensing image dataset for cloud removal. Computer Vision and Pattern Recognition 1:30–34

    Google Scholar 

  • Foody GM (2019) Earth observation science: making sense of our planet. Remote Sens Lett 10(7):597–606

    Google Scholar 

  • Gao J, Yuan Q, Li J, Zhang H, Su X (2019) Cloud removal with fusion of high resolution optical and SAR images using generative adversarial networks. Remote Sens-Basel 12(1):191. https://doi.org/10.3390/rs12010191

    Article  Google Scholar 

  • Ghasemian N, Akhoondzadeh M (2018) Introducing two random forest based methods for cloud detection in remote sensing images. Adv Space Res 62(2):288–303. https://doi.org/10.1016/j.asr.2018.04.030

    Article  Google Scholar 

  • Goodfellow I, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville A, Bengio Y (2014) Generative adversarial nets. In: NIPS'14: Proceedings of the 27th international conference on neural information processing systems - vol 2, pp 2672–2680

  • Grebo A, Krstulovi L, Domazet Z (2023) Thermal to digital image correlation image to image translation with CycleGAN and Pix2Pix. Mater Today-Proc 93(4):752–760. https://doi.org/10.1016/j.matpr.2023.06.219

    Article  Google Scholar 

  • Guillemot C, Le Meur O (2014) Image inpainting: overview and recent advances. IEEE Signal Proc Mag 31:127–144

    Article  Google Scholar 

  • He K, Sun J (2014) Image completion approaches using the statistics of similar patches. IEEE T Pattern Anal 36:2423–2435

    Article  Google Scholar 

  • Heng L (2020) Cloud removal from remote sensing images. Int J of Eng Res App 10(8):15–22

    Google Scholar 

  • Heng L, Zhang C, Wu L, He Y (2017) Cloud removal from remote sensing imagery using a compound method of CFAR and anisotropic diffusion. Remote Sens-Basel 9(10):1052. (https://geography.ecu.edu/ncview/comparison-of-algorithms-to-remove-thin-clouds-within-a-single-landsat-8-oli-image/)

    Google Scholar 

  • Isola P, Zhu JY, Zhou T, Efros AA (2017) Image-to-image translation with conditional adversarial networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), pp 1125–1134

  • Ji S, Dai P, Lu M, Zhang Y (2020) Simultaneous cloud detection and removal from bitemporal remote sensing images using cascade convolutional neural networks. IEEE T Geosci Remote 59(1):732–748. https://doi.org/10.1109/TGRS.2020.2994349

    Article  Google Scholar 

  • Kang Y, Gao S, Roth RE (2019) Transferring multiscale map styles using generative adversarial networks. Int J Cartogr 5(2–3):115–141. https://doi.org/10.1080/23729333.2019.1615729

    Article  Google Scholar 

  • Klages P, Benslimane I, Riyahi S, Jiang J, Hunt M, Joseph O, Deasy VH, Tyagi N (2020) Patch-based generative adversarial neural network models for head and neck MR-only planning. Med Phys 47(2):626–642. https://doi.org/10.1002/mp.13927

    Article  Google Scholar 

  • Li X, Shen H, Zhang L, Zhang H, Yuan Q, Yang G (2014) Recovering quantitative remote sensing products contaminated by thick clouds and shadows using multitemporal dictionary learning. IEEE T Geosci Remote 52:7086–7098

    Article  Google Scholar 

  • Lin E (2023) Comparative Analysis of Pix2Pix and CycleGAN for image-to-image translation. Highl Sci Eng Technol 39:915–925. https://doi.org/10.54097/hset.v39i.6676

    Article  Google Scholar 

  • Liu J, Zhang L, Li J (2021) A GAN-based approach for cloud removal in Sentinel-2 images. IEEE J Sel Top Appl 14:2906–2914

    Google Scholar 

  • Pan, H. (2020). Cloud removal for remote sensing imagery via spatial attention generative adversarial network. ArXiv./abs/2009.13015

  • Ronneberger O, Fischer P, Brox T (2015) U-Net: convolutional networks for biomedical image segmentation. In: Navab N, Hornegger J, Wells W, Frangi A (eds) Medical image computing and computer-assisted intervention – MICCAI 2015. MICCAI 2015, Lecture notes in computer science, vol 9351. Springer, Cham. https://doi.org/10.1007/978-3-319-24574-4_28

  • Rossi RE, Dungan JL, Beck LR (1994) Kriging in the shadows: geostatistical interpolation for remote sensing. Proc Spie 49:32–40

    Google Scholar 

  • Shen H, Wu J, Cheng Q, Aihemaiti M, Zhang C, Li Z (2019) A spatiotemporal fusion based cloud removal method for remote sensing images with land cover changes. IEEE J Sel Top Appl 12:862–874

    Google Scholar 

  • Shuai T, Zhang X, Wang S, Zhan L, Shang K, Chen X, Wang J (2014) A spectral angle distance-weighting reconstruction method for filled pixels of the MODIS Land Surface Temperature Product. IEEE Geosci Remote S 11:1514–1518

    Article  Google Scholar 

  • Singh P, Komodakis N (2018) Cloud-Gan: cloud removal for sentinel-2 imagery using a cyclic consistent generative adversarial networks. In: IEEE international geoscience and remote sensing symposium, Valencia, pp 1772–1775. https://doi.org/10.1109/IGARSS.2018.8519033

  • Siravenha AC, Sousa D, Bispo A, Pelaes E (2011) Evaluating inpainting methods to the satellite images clouds and shadows removing. In: Kim Th, Adeli H, Ramos C, Kang BH (eds) Signal processing, image processing and pattern recognition. SIP 2011. Communications in computer and information science, vol 260. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-27183-0_7

  • Tan C, Zhong X (2023) A rapid wind velocity prediction method in built environment based on cyclegan model. In: Yuan, P.F., Chai, H., Yan, C., Li, K., Sun, T. (eds) Hybrid Intelligence. Computational Design and Robotic Fabrication. Springer, Singapore. https://doi.org/10.1007/978-981-19-8637-6_22

  • Wang L, Zhang J, Yang G, Wang J, Feng X (2016) A thresholding method for cloud detection from remote sensing images. IEEE Geosci Remote S 13(4):544–548

    Google Scholar 

  • Xu M, Jia X, Pickering M, Plaza AJ (2016) Cloud removal based on sparse representation via multitemporal dictionary learning. IEEE T Geosci Remote 54:2998–3006

    Article  Google Scholar 

  • Yu Z, Lu H, Liu Y (2020) Cloud detection and removal in Sentinel-2 remote sensing images using a deep convolutional neural network. IEEE Access 8:152273–152281

    Google Scholar 

  • Yu C, Chen L, Su L, Fan M, & Li S (2011) Kriging interpolation method and its application in retrieval of MODIS aerosol optical depth. In 19th international conference on geoinformatics, 1–6. https://doi.org/10.1109/GEOINFORMATICS.2011.5981052. Accessed 30 Aug 2023

  • Yuan Y, Liu Y, Tao D, Li X (2018) Learning to detect deception from opinion divergence. IEEE T Neur Net Lear 29(8):3695–3706. https://doi.org/10.1109/TNNLS.2017.2776409

    Article  Google Scholar 

  • Zhang Y (2004) Calculation of radiative fluxes from the surface to top of atmosphere based on ISCCP and other global data sets: Refinements of the radiative transfer model and the input data. J Geophys Res 109:1–27

    Google Scholar 

  • Zhang C, Li W, Travis DJ (2009) Restoration of clouded pixels in multispectral remotely sensed imagery with cokriging. Int J Remote Sens 30:2173–2195

    Article  Google Scholar 

  • Zhang Q, Yuan Q, Zeng C, Li X, Wei Y (2018) Missing data reconstruction in remote sensing image with a unified spatial–temporal–spectral deep convolutional neural network. IEEE T Geosci Remote 56:4274–4288

    Article  Google Scholar 

  • Zhang X, Li X, Li Z, Xu Y, Li Q, Li W, Liu D (2020) Cloud removal for Landsat-8 images using generative adversarial network. Remote Sens-Basel 12(8):1253

    Google Scholar 

  • Zhu X, Gao F, Liu D, Chen J (2012) A modified neighborhood similar pixel interpolator approach for removing thick clouds in Landsat images. IEEE Geosci Remote S 9:520–525

    Article  Google Scholar 

  • Zhu JY, Park T, Isola P, Efros AA (2017) Unpaired image-to-image translation using cycle-consistent adversarial networks. In: IEEE international conference on computer vision (ICCV), vol 2017, Venice, pp 2242–2251. https://doi.org/10.1109/ICCV.2017.244

Download references

Author information

Authors and Affiliations

  1. Department of Land and Spatial Sciences, Namibia University of Science and Technology, Windhoek, Namibia

    Oluibukun Gbenga Ajayi

  2. Department of Surveying and Geoinformatics, Federal University of Technology, Minna, Nigeria

    Oluibukun Gbenga Ajayi & Daud Musa Olaide

Authors
  1. Oluibukun Gbenga Ajayi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Daud Musa Olaide
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Oluibukun Gbenga Ajayi.

Ethics declarations

Conflict of interest

The authors declare no conflict of interest.

Additional information

Responsible Editor: Biswajeet Pradhan

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ajayi, O.G., Olaide, D.M. Assessment of generative adversarial networks for cloud occlusion removal on remotely sensed images. Arab J Geosci 17, 141 (2024). https://doi.org/10.1007/s12517-024-11939-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s12517-024-11939-y

Keywords

Search

Navigation