Abstract
This study proposes a deep learning-based approach for automated cloud cover removal from optical temporal satellite imagery using generative adversarial networks (GANs) in an image restoration approach. The pix2pix GAN, which is a slight modification of the conditional GAN, is explored for cloud removal in optical satellite images by learning the mapping from cloudy image to cloud-free image. This study proposes a novel approach for training the pix2pix GAN solely on optical multispectral images, using a novel data augmentation approach. The model is tested on real as well as synthetic cloudy images, consisting of cloud-saturated pixels as well as hazy pixels. The generated cloud-free images are evaluated through qualitative metrics of Pearson correlation for each band, peak signal-to-noise ratio (PSNR) and structural similarity index measure (SSIM). The PSNR values show a significant increase in image information after cloud removal and the SSIM of 0.958 is recorded between the generated image and the target image. The generated images are further assessed for their utility of remote-sensing applications, such as land-cover classification, and favourable results are observed.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Sarukkai V, Jain A, Uzkent B, Ermon S (2020) Cloud removal in satellite images using spatiotemporal generative networks. In: Proceedings 2020 IEEE Winter Conference on Applied Computer Vision, WACV 2020, pp 1785–1794. https://doi.org/10.1109/wacv45572.2020.9093564
Lin C, Tsai P, Lai K, Chen J (2013) Images using information cloning 51(1):232–241
Shen H et al (2015) Missing information reconstruction of remote sensing data: a technical review. IEEE Geosci Remote Sens Mag 3(3):61–85. https://doi.org/10.1109/MGRS.2015.2441912
Li X, Wang L, Cheng Q, Wu P, Gan W, Fang L (2019) Cloud removal in remote sensing images using nonnegative matrix factorization and error correction. ISPRS J Photogramm Remote Sens 148:103–113. https://doi.org/10.1016/j.isprsjprs.2018.12.013
Lin CH, Lai KH, Bin Chen Z, Chen JY (2014) Patch-based information reconstruction of cloud-contaminated multitemporal images. IEEE Trans Geosci Remote Sens 52(1):163–174. https://doi.org/10.1109/tgrs.2012.2237408
Goodfellow I et al (2014) Generative adversarial nets. In: Advances in neural information processing systems, pp 2672–2680
Singh P, Komodakis N (2018) Cloud-GAN: cloud removal for sentinel-2 imagery using a cyclic consistent generative adversarial networks. Int Geosci Remote Sens Symp pp 1772–1775. https://doi.org/10.1109/igarss.2018.8519033
Enomoto K et al (2017) Filmy cloud removal on satellite imagery with multispectral conditional generative adversarial nets. IEEE Comput Soc Conf Comput Vis Pattern Recognit Work pp 1533–1541. https://doi.org/10.1109/cvprw.2017.197
Toizumi T, Zini S, Sagi K, Kaneko E, Tsukada M, Schettini R (2019) Artifact-free thin cloud removal using gans. In: Proceedings of International Conference on Image Processing ICIP, vol 2019-Septe, no c, pp 3596–3600. https://doi.org/10.1109/icip.2019.8803652
Pacot MPB, Marcos N (2020) Cloud removal from aerial images using generative adversarial network with simple image enhancement. ACM Int Conf Proceeding Ser pp 77–81. https://doi.org/10.1145/3383812.3383838
Grohnfeldt C, Schmitt M, Zhu X (2018) A conditional generative adversarial network to fuse SAR and multispectral optical data for cloud removal from Sentinel-2 images. Int Geosci Remote Sens Symp, pp 1726–1729. https://doi.org/10.1109/igarss.2018.8519215
Isola P, Zhu J-Y, Zhou T, Efros AA, Research BA (2017) Image-to-image translation with conditional adversarial networks
Mirza M, Osindero S (2014) Conditional generative adversarial nets
Ronneberger O, Fischer P, Brox T (2015) U-net: convolutional networks for biomedical image segmentation. In: Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) vol 9351, pp 234–241. 10.1007/978-3-319-24574-4_28
Perlin K (2002) Improving noise. ACM Trans Graph, pp 681–682. https://doi.org/10.1145/566570.566636
Benesty J, Chen J, Huang Y, Cohen I (2009) Pearson correlation coefficient. Springer, Berlin, Heidelberg, pp 1–4
Huynh-Thu Q, Ghanbari M (2008) Scope of validity of PSNR in image/video quality assessment. Electron Lett 44(13):800–801. https://doi.org/10.1049/el:20080522
Wang Z, Bovik AC, Sheikh HR, Simoncelli EP (2004) Image quality assessment: from error visibility to structural similarity. IEEE Trans Image Process 13(4):600–612. https://doi.org/10.1109/TIP.2003.819861
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Maniyar, C., Kumar, A. (2021). Generative Adversarial Network for Cloud Removal from Optical Temporal Satellite Imagery. In: Tiwari, A., Ahuja, K., Yadav, A., Bansal, J.C., Deep, K., Nagar, A.K. (eds) Soft Computing for Problem Solving. Advances in Intelligent Systems and Computing, vol 1393. Springer, Singapore. https://doi.org/10.1007/978-981-16-2712-5_39
Download citation
DOI: https://doi.org/10.1007/978-981-16-2712-5_39
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-16-2711-8
Online ISBN: 978-981-16-2712-5
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)