Skip to main content
SpringerLink
Log in

DerainAttentionGAN: unsupervised single-image deraining using attention-guided generative adversarial networks

  • Original Paper
  • Published:
Signal, Image and Video Processing Aims and scope Submit manuscript

Abstract

As an important task of computer vision, the single-image deraining (SID) methods tend to supervised learning in the previous research. However, most existing SID methods suffer from the inability to collect paired datasets needed by supervised learning in real scenarios. In this paper, we introduce a recent image translation model known as CycleGAN into SID and propose Derain Attention-Guide GAN (DerainAttentionGAN) that only requires unpaired datasets can effectively overcome the above limitation. The main work of this paper is as follows: We firstly inject an attention mechanism into the generator, which makes the rain-removing regions to be concentrated near the rain line to preserve background details. Secondly, a multiscale discriminator is used to discriminate the generated image from different scales to improve its quality. Finally, the perceptual-consistency loss and internal feature perceptual loss (interfeat loss) are introduced to reduce artificial features on the generated image and make it more realistic. Experiments results demonstrate that our work is superior to the current unsupervised learning methods in terms of both quantitative and qualitative, and have achieved comparable effects to other popular supervised learning methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  1. Chen, X., Xu, C., Yang, X., Tao, D.: Attention-gan for object transfiguration in wild images. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 164–180 (2018)

  2. Engin, D., Genç, A., Kemal Ekenel, H.: Cycle-dehaze: Enhanced cyclegan for single image dehazing. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 825–833 (2018)

  3. Fan, Z., Wu, H., Fu, X., Huang, Y., Ding, X.: Residual-guide network for single image deraining. In: Proceedings of the 26th ACM international conference on Multimedia, pp. 1751–1759 (2018)

  4. Fu, X., Huang, J., Ding, X., Liao, Y., Paisley, J.: Clearing the skies: a deep network architecture for single-image rain removal. IEEE Trans. Image Process. 26(6), 2944–2956 (2017)

    Article  MathSciNet  Google Scholar 

  5. Fu, X., Huang, J., Zeng, D., Huang, Y., Ding, X., Paisley, J.: Removing rain from single images via a deep detail network. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3855–3863 (2017)

  6. Huynh-Thu, Q., Ghanbari, M.: Scope of validity of psnr in image/video quality assessment. Electron. lett. 44(13), 800–801 (2008)

    Article  Google Scholar 

  7. Isola, P., Zhu, J.Y., Zhou, T., Efros, A.A.: Image-to-image translation with conditional adversarial networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 1125–1134 (2017)

  8. Johnson, J., Alahi, A., Fei-Fei, L.: Perceptual losses for real-time style transfer and super-resolution. In: European conference on computer vision, pp. 694–711. Springer (2016)

  9. Kang, L.W., Lin, C.W., Fu, Y.H.: Automatic single-image-based rain streaks removal via image decomposition. IEEE Trans. Image Process. 21(4), 1742–1755 (2011)

    Article  MathSciNet  Google Scholar 

  10. Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)

  11. Li, G., He, X., Zhang, W., Chang, H., Dong, L., Lin, L.: Non-locally enhanced encoder-decoder network for single image de-raining. In: Proceedings of the 26th ACM international conference on Multimedia, pp. 1056–1064 (2018)

  12. Li, X., Wu, J., Lin, Z., Liu, H., Zha, H.: Recurrent squeeze-and-excitation context aggregation net for single image deraining. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 254–269 (2018)

  13. Lin, J., Xia, Y., Qin, T., Chen, Z., Liu, T.Y.: Conditional image-to-image translation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5524–5532 (2018)

  14. Luo, Y., Xu, Y., Ji, H.: Removing rain from a single image via discriminative sparse coding. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 3397–3405 (2015)

  15. Van den Oord, A., Kalchbrenner, N., Espeholt, L., Vinyals, O., Graves, A., et al.: Conditional image generation with pixelcnn decoders. In: Advances in neural information processing systems, pp. 4790–4798 (2016)

  16. Ren, D., Zuo, W., Hu, Q., Zhu, P., Meng, D.: Progressive image deraining networks: A better and simpler baseline. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 3937–3946 (2019)

  17. Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014)

  18. Tang, H., Liu, H., Xu, D., Torr, P.H., Sebe, N.: Attentiongan: Unpaired image-to-image translation using attention-guided generative adversarial networks. arXiv preprint arXiv:1911.11897 (2019)

  19. Wang, C., Xu, C., Wang, C., Tao, D.: Perceptual adversarial networks for image-to-image transformation. IEEE Trans. Image Process. 27(8), 4066–4079 (2018)

  20. Wang, H., Xie, Q., Zhao, Q., Meng, D.: A model-driven deep neural network for single image rain removal. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3103–3112 (2020)

  21. Wang, T., Yang, X., Xu, K., Chen, S., Zhang, Q., Lau, R.W.: Spatial attentive single-image deraining with a high quality real rain dataset. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 12270–12279 (2019)

  22. Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: from error visibility to structural similarity. IEEE Trans Image Process 13(4), 600–612 (2004)

    Article  Google Scholar 

  23. Wei, Y., Zhang, Z., Fan, J., Wang, Y., Yan, S., Wang, M.: Deraincyclegan: An attention-guided unsupervised benchmark for single image deraining and rainmaking. arXiv preprint arXiv:1912.07015 (2019)

  24. Wei, Y., Zhang, Z., Zhang, H., Qin, J., Zhao, M.: Semi-deraingan: A new semi-supervised single image deraining network. arXiv preprint arXiv:2001.08388 (2020)

  25. Yang, W., Tan, R.T., Feng, J., Guo, Z., Yan, S., Liu, J.: Joint rain detection and removal from a single image with contextualized deep networks. IEEE Trans. Pattern Anal. Mach. Intell. 42(6), 1377–1393 (2019)

    Article  Google Scholar 

  26. Yang, W., Tan, R.T., Feng, J., Liu, J., Guo, Z., Yan, S.: Deep joint rain detection and removal from a single image. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1357–1366 (2017)

  27. Yang, W., Tan, R.T., Wang, S., Fang, Y., Liu, J.: Single image deraining: From model-based to data-driven and beyond. IEEE Transactions on Pattern Analysis and Machine Intelligence (2020)

  28. Zhang, H., Sindagi, V., Patel, V.M.: Image de-raining using a conditional generative adversarial network. IEEE transactions on circuits and systems for video technology (2019)

  29. Zhu, H., Peng, X., Zhou, J.T., Yang, S., Chanderasekh, V., Li, L., Lim, J.H.: Singe image rain removal with unpaired information: A differentiable programming perspective. In: Proceedings of the AAAI Conference on Artificial Intelligence 33, pp. 9332–9339 (2019)

  30. Zhu, J.Y., Park, T., Isola, P., Efros, A.A.: Unpaired image-to-image translation using cycle-consistent adversarial networks. In: Proceedings of the IEEE international conference on computer vision, pp. 2223–2232 (2017)

Download references

Acknowledgements

This work was supported by the National Natural Science Foundation of China (Grant No. U1833115).

Author information

Authors and Affiliations

  1. Sichuan University, Chengdu, China

    ZhaoKang Guo, Mingzheng Hou, Mingjun Sima & ZiLiang Feng

Authors
  1. ZhaoKang Guo
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mingzheng Hou
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mingjun Sima
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. ZiLiang Feng
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to ZhaoKang Guo.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Guo, Z., Hou, M., Sima, M. et al. DerainAttentionGAN: unsupervised single-image deraining using attention-guided generative adversarial networks. SIViP 16, 185–192 (2022). https://doi.org/10.1007/s11760-021-01972-9

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11760-021-01972-9

Keywords

Search

Navigation