Skip to main content
SpringerLink
Log in

Single Image Deraining Using Residual Channel Attention Networks

  • Regular Paper
  • Published:
Journal of Computer Science and Technology Aims and scope Submit manuscript

Abstract

Image deraining is a highly ill-posed problem. Although significant progress has been made due to the use of deep convolutional neural networks, this problem still remains challenging, especially for the details restoration and generalization to real rain images. In this paper, we propose a deep residual channel attention network (DeRCAN) for deraining. The channel attention mechanism is able to capture the inherent properties of the feature space and thus facilitates more accurate estimations of structures and details for image deraining. In addition, we further propose an unsupervised learning approach to better solve real rain images based on the proposed network. Extensive qualitative and quantitative evaluation results on both synthetic and real-world images demonstrate that the proposed DeRCAN performs favorably against state-of-the-art methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

References

  1. Zhang D, Zhang H, Tang J, Wang M, Hua X, Sun Q. Feature pyramid transformer. In Proc. the 16th European Conference on Computer Vision, August 2020, pp.323–339. DOI: 10.1007/978-3-030-58604-1_20.

  2. Tang H, Li Z, Peng Z, Tang J. BlockMix: Meta regularization and self-calibrated inference for metric-based meta-learning. In Proc. the 28th ACM International Conference on Multimedia, October 2020, pp.610–618. DOI: 10.1145/3394171.3413884.

  3. Zhang D, Zhang H, Tang J, Hua X, Sun Q. Causal intervention for weakly-supervised semantic segmentation. In Proc. Annual Conference on Neural Information Processing Systems, December 2020, pp.655–666. DOI: https://doi.org/10.5555/3495724.3495780.

  4. Li Z, Sun Y, Zhang L, Tang J. CTNet: Context-based tandem tetwork for semantic segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022, 44(12): 9904–9917. DOI: https://doi.org/10.1109/TPAMI.2021.3132068.

    Article  Google Scholar 

  5. Kang L, Lin C, Fu Y. Automatic single-image-based rain streaks removal via image decomposition. IEEE Transactions on Image Processing, 2012, 21(4): 1742–1755. DOI: https://doi.org/10.1109/TIP.2011.2179057.

    Article  MathSciNet  MATH  Google Scholar 

  6. Luo Y, Xu Y, Ji H. Removing rain from a single image via discriminative sparse coding. In Proc. the 15th International Conference on Computer Vision, April 2015, pp.3397–3405. DOI: 10.1109/ICCV.2015.388.

  7. Zhu L, Fu C, Lischinski D, Heng P. Joint bi-layer optimization for single-image rain streak removal. In Proc. the 16th International Conference on Computer Vision, October 2017, pp.2545–2553. DOI: 10.1109/ICCV.2017.276.

  8. Wang Y, Liu S, Chen C, Xie D, Zeng B. Rain removal by image quasisparsity priors. arXiv: 1812.08348, 2018. https://arxiv.org/abs/1812.08348, Dec. 2018.

  9. Deng L, Huang T, Zhao X, Jiang T. A directional global sparse model for single image rain removal. Applied Mathematical Modelling, 2018, 59(7): 662–679. DOI: https://doi.org/10.1016/j.apm.2018.03.001.

    Article  MathSciNet  MATH  Google Scholar 

  10. Du S, Liu Y, Ye M, Xu Z, Li J, Liu J. Single image deraining via decorrelating the rain streaks and background scene in gradient domain. Pattern Recognition, 2018, (79): 303–317. DOI: https://doi.org/10.1016/j.patcog.2018.02.016.

    Article  Google Scholar 

  11. Li Y, Tan T, Guo X, Lu J, Brown M. Rain streak removal using layer priors. In Proc. the 29th IEEE Conference on Computer Vision and Pattern Recognition, June 2016, pp.2736–2744. DOI: 10.1109/CVPR.2016.299.

  12. Fu X, Huang J, Ding X, Liao Y, Paisley J. Clearing the skies: A deep network architecture for single-image rain removal. IEEE Transactions on Image Processing, 2017, 26(6): 2944–2956. DOI: https://doi.org/10.1109/TIP.2017.2691802.

    Article  MathSciNet  MATH  Google Scholar 

  13. Pan J, Liu S, Sun D, Zhang J, Liu Y, Ren J, Li Z, Tang J, Lu H, Tai Y, Yang M. Learning dual convolutional neural networks for low-level vision. In Proc. the 31st IEEE Conference on Computer Vision and Pattern Recognition, June 2018, pp.3070–3079. DOI: 10.1109/CVPR.2018.00324.

  14. Wang Y, Zhao X, Jiang T, Deng L, Chang Y, Huang T. Rain streak removal for single image via kernel guided CNN. arXiv: 1808.08545, 2018. https://arxiv.org/abs/1808.08545, Aug. 2018.

  15. Ye H, Li X, Liu H, Shi W, Liu M, Sun Q. Self-rening deep symmetry enhanced network for rain removal. In Proc. the 25th IEEE International Conference on Image Processing, September 2018, pp.2786–2790. DOI: 10.1109/ICIP.2019.8803265.

  16. Li G, He X, Zhang W, Chang H, Dong L, Lin L. Non-locally enhanced encoder-decoder network for single image de-raining. In Proc. the 26th ACM International Conference on Multimedia, October 2018, 1056–1064. DOI: https://doi.org/10.1145/3240508.3240636.

  17. Li S, Ren W, Zhang J, Yu J, Guo X. Single image rain removal via a deep decomposition-composition network. Computer Vision and Image Understanding, 2019, 186: 48–57. DOI: https://doi.org/10.1016/j.cviu.2019.05.003.

    Article  Google Scholar 

  18. Yang W, Tan T, Feng J, Guo Z, Yan S, Liu J. Joint rain detection and removal from a single image with contextualized deep networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020, 42(6): 1377–1393. DOI: https://doi.org/10.1109/TPAMI.2019.2895793.

    Article  Google Scholar 

  19. Fu X, Huang J, Zeng D, Huang Y, Ding X, Paisley J. Removing rain from single images via a deep detail network. In Proc. the 30th IEEE Conference on Computer Vision and Pattern Recognition, June 2017, pp.1715–1723. DOI: 10.1109/CVPR.2017.186.

  20. Fu X, Liang B, Huang Y, Ding X, Paisley J. Lightweight pyramid networks for image deraining. IEEE Transactions on Neural Networks and Learning Systems, 2020, 31(6): 1794–1807. DOI: https://doi.org/10.1109/TNNLS.2019.2926481.

    Article  Google Scholar 

  21. Chen D, He M, Fan Q, Liao J, Zhang L, Hou D, Yuan L, Hua G. Gated context aggregation network for image dehazing and deraining. In Proc. the 2019 IEEE Winter Conference on Applications of Computer Vision, January 2019, pp.1375–1383. DOI: 10.1109/WACV.2019.00151.

  22. Fu X, Qi Q, Huang Y, Ding X, Wu F, Paisley J. A deep tree-structured fusion model for single image deraining. arXiv: 1811.08632, 2018. https://arxiv.org/abs/1811.08632, Nov. 2018.

  23. Ren D, Zuo W, Hu Q, Zhu P, Meng D. Progressive image deraining networks: A better and simpler baseline. In Proc. the 32nd IEEE Conference on Computer Vision and Pattern Recognition, June 2019, pp.3937–3946. DOI: 10.1109/CVPR.2019.00406.

  24. Li X, Wu J, Lin Z, Liu H, Zha H. Recurrent squeeze-andexcitation context aggregation net for single image deraining. In Proc. the 32nd European Conference on Computer Vision, September 2018, pp.262–277. DOI: 10.1007/978-3-030-01234-2_16.

  25. Zhang H, Patel V. Density-aware single image de-raining using a multi-stream dense network. In Proc. the 31st IEEE Conference on Computer Vision and Pattern Recognition, June 2018, pp.695–704. DOI: 10.1109/CVPR.2018.00079.

  26. Zhang H, Sindagi V, Patel V. Image de-raining using a conditional generative adversarial network. IEEE Transactions on Circuits and Systems for Video Technology, 2020, 30(11): 3943–3956. DOI: https://doi.org/10.1109/TCSVT.2019.2920407.

    Article  Google Scholar 

  27. Qian R, Tan T, Yang W, Su J, Liu J. Attentive generative adversarial network for raindrop removal from a single image. In Proc. the 31st Conference on Computer Vision and Pattern Recognition, September 2018, pp.2482–2491. DOI: 10.1109/CVPR.2018.00263.

  28. Pu J, Chen X, Zhang L, Zhou Q, Zhao Y. Removing rain based on a cycle generative adversarial network. In Proc. the 13th IEEE Conference on Industrial Electronics and Applications, May 31–June 2, 2018, pp.621–626. DOI: 10.1109/ICIEA.2018.8397790.

  29. Pan J, Liu Y, Dong J, Zhang J, Ren J, Tang J, Tai Y, Yang M. Physics-based generative adversarial models for image restoration and beyond. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021, 43(7): 2449–2462. DOI: https://doi.org/10.1109/TPAMI.2020.2969348.

    Article  Google Scholar 

  30. He K, Zhang X, Ren S, Sun J. Deep residual learning for image recognition. In Proc. the 29th IEEE Conference on Computer Vision and Pattern Recognition, June 2016, pp.770–778. DOI: 10.1109/CVPR.2016.90.

  31. Wang D, Tang H, Pan J, Tang J. Learning a tree-structured channel-wise refinement network for efficient image deraining. In Proc. IEEE International Conference on Multimedia and Expo, July 2021. DOI: https://doi.org/10.1109/ICME51207.2021.9428187.

  32. Fan Z, Wu H, Fu X, Huang Y, Ding X. Residual guide network for single image deraining. In Proc. the 26th ACM International Conference on Multimedia, October 2018, pp.1751–1759. DOI: 10.1145/3240508.3240694.

  33. Wei W, Meng D, Zhao Q, Xu Z, Wu Y. Semi-supervised transfer learning for image rain removal. In Proc. the 32nd IEEE Conference on Computer Vision and Pattern Recognition, June 2019, pp.3877–3886. DOI: 10.1109/CVPR.2019.00400.

  34. Lin H, Li Y, Fu X, Ding X, Huang Y, Paisley J. Rain O’er Me: Synthesizing real rain to derain with data distillation. IEEE Transactions on Image Processing, 29(6): 7668–7680. DOI: 10.1109/TIP.2020.3005517.

  35. Hu J, Shen L, Sun G. Squeeze-and-excitation networks. In Proc. the 31st IEEE Conference on Computer Vision and Pattern Recognition, June 2018, pp.7132–7141. DOI: 10.1109/CVPR.2018.00745.

  36. Woo S, Park J, Lee J, Kweon I. CBAM: Convolutional block attention module. In Proc. the 32nd European Conference on Computer Vision, September 2018, pp.3–19. DOI: 10.1007/978-3-030-01234-21.

  37. Ledig C, Theis L, Huszar F, Caballero J, Cunningham A, Acosta A, Aitken A, Tejani A, JohannesTotz, Wang Z, Shi W. Photo-realistic single image super-resolution using a generative adversarial network. In Proc. the 30th IEEE Conference on Computer Vision and Pattern Recognition, June 2017, pp.105–114. DOI: 10.1109/CVPR.2017.19.

  38. Russakovsky O, Deng J, Su H, Krause J, Satheesh S, Ma S, Huang Z, Karpathy A, Khosla A, Bernstein M, Berg A, Li F. Imagenet large scale visual recognition challenge. International Journal of Computer Vision, 2015, 115(3): 211–252. DOI: https://doi.org/10.1007/s11263-015-0816-y.

    Article  MathSciNet  Google Scholar 

  39. Ledig C, Theis L, Caballero J, Cunningham A, Acosta A, Tejani A, Totz J, Wang Z, Shi W. Photo-realistic single image super-resolution using a generative adversarial network. In Proc. the 30th IEEE Conference on Computer Vision and Pattern Recognition, June 2017, pp.105–114. DOI: 10.1109/CVPR.2017.19.

  40. Wang T, Yang X, Xu K, Chen S, Zhang Q, Lau R. Spatial attentive single-image deraining with a high quality real rain dataset. In Proc. the 32nd IEEE Conference on Computer Vision and Pattern Recognition, June 2019, pp.12270–12279. DOI: 10.1109/CVPR.2019.01255.

  41. Quan H, Mohammed G. Scope of validity of PSNR in image/video quality assessment. Electronics Letters, 2008, 44(13): 800–801. DOI: https://doi.org/10.1109/CVPR.2017.19.

    Article  Google Scholar 

  42. Wang Z, Simoncelli E, Bovik A. Multiscale structural similarity for image quality assessment. In Proc. the 37th Asilomar Conference on Signals, Systems & Computers, Nov. 2003, pp.1398–1402. DOI: 10.1109/ACSSC.2003.1292216.

  43. Kingma D P, Ba J. Adam: A method for stochastic optimization. arXiv: 1412.6980, 2014. https://arxiv.org/abs/1412.6980, Jan. 2017.

  44. He K, Sun J, Tang X. Single image haze removal using dark channel prior. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2011, 33(12): 2341–2353. DOI: https://doi.org/10.1109/TPAMI.2010.168.

    Article  Google Scholar 

  45. Li R, Cheong L, Tan T. Heavy rain image restoration: Integrating physics model and conditional adversarial learning. In Proc. the 32nd IEEE Conference on Computer Vision and Pattern Recognition, June 2020, pp.1633–1642. DOI: 10.1109/CVPR.2019.00173.

  46. Liu J, Yang W, Yang S, Guo Z. Erase or fill? Deep joint recurrent rain removal and reconstruction in videos. In Proc. the 31st IEEE Conference on Computer Vision and Pattern Recognition, June 2018, pp.3233–3242. DOI: 10.1109/CVPR.2018.00341.

  47. Tang H, Yuan C, Li Z, Tang J. Learning attention-guided pyramidal features for few-shot fine-grained recognition. Pattern Recognition, 2022. DOI: https://doi.org/10.1016/j.patcog.2022.108792. (preprint)

  48. Ren S, He K, Girshick R, Sun J. Faster R-CNN: Towards real-time object detection with region proposal networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(6): 1137–1149. DOI: https://doi.org/10.1109/TPAMI.2016.2577031.

    Article  Google Scholar 

  49. Lin T, Maire M, Belongie S, Hays J, Perona P, Ramanan D, Dollar P, Zitnick C. Microsoft COCO: Common objects in context. In Proc. the 13th European Conference on Computer Vision, September 2014, pp.740–755. DOI: 10.1007/978-3-319-10602-148.

  50. Everingham M, Gool L, Williams C, Winn J, Zisserman A. The pascal visual object classes (VOC) challenge. International Journal of Computer Vision, 2010, 88(2): 303–338. DOI: https://doi.org/10.1007/s11263-009-0275-4.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Computer Science and Engineering, Nanjing University of Science and Technology, Nanjing, 210094, China

    Di Wang, Jin-Shan Pan & Jin-Hui Tang

Authors
  1. Di Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jin-Shan Pan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jin-Hui Tang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jin-Shan Pan.

Supplementary Information

ESM 1

(PDF 583 kb)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, D., Pan, JS. & Tang, JH. Single Image Deraining Using Residual Channel Attention Networks. J. Comput. Sci. Technol. 38, 439–454 (2023). https://doi.org/10.1007/s11390-022-0979-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11390-022-0979-2

Keywords

Search

Navigation