Skip to main content
SpringerLink
Log in

Rain Removal from a Single Image Using Refined Inception ResNet v2

  • Published:
Circuits, Systems, and Signal Processing Aims and scope Submit manuscript

Abstract

The atmospheric conditions like rain degrade visibility, creating problems for computer vision applications. In single-image de-raining, the lack of temporal information creates challenges. Rain removal requires a high-frequency layer extraction, as rain affects an image’s high-frequency layer (detail layer). This article has proposed using a combination of image decomposition and Inception ResNet v2 (IR v2) network for single-image rain removal. A guided filter decomposes the rainy image into the base and detail layers. The use of the Inception ResNet v2 (IR v2) network is proposed to create the residual map of the rain streaks from the high-frequency layer of the input image. The de-rained image obtained by subtracting the residual from the original rainy image provides comparable results. SSIM is added to MSE to train the IR v2 network, which improves the network’s performance. The complexity of the network (IR v2) is high. A systematic reduction in the network is done first at the module level and later at convolution filters. The proposed refinement decreases network complexity and reduces execution time without degrading the performance. An extensive test using natural and artificial rainy images reveals that the proposed refined Inception ResNet v2 (rIR v2) favorably competes against the recent single-image de-raining techniques.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

Notes

  1. https://sites.google.com/view/more-results/single-image-rain-removal-results.

References

  1. N. Ahn, S.Y. Jo, S.-J. Kang, Eagnet: Elementwise attentive gating network-based single image de-raining with rain simplification. IEEE Trans. Circuits Syst. Video Technol. 32(2), 608–620 (2021)

    Article  Google Scholar 

  2. L. Cai, Y. Fu, T. Zhu, Y. Xiang, Y. Zhang, H. Zeng, Joint depth and density guided single image de-raining. IEEE Trans. Circuits Syst. Video Technol. 32(7), 4108–4121 (2021)

    Article  Google Scholar 

  3. D.-Y. Chen, C.-C. Chen, L.-W. Kang, Visual depth guided color image rain streaks removal using sparse coding. IEEE Trans. Circuits Syst. Video Technol. 24(8), 1430–1455 (2014)

    Article  Google Scholar 

  4. Y. Ding, M. Li, T. Yan, F. Zhang, Y. Liu, R.W. Lau, Rain streak removal from light field images. IEEE Trans. Circuits Syst. Video Technol. 32(2), 467–482 (2021)

    Article  Google Scholar 

  5. Y. Du, J. Xu, X. Zhen, M.-M. Cheng, L. Shao, Conditional variational image deraining. IEEE Trans. Image Process. 29, 6288–6301 (2020)

    Article  MATH  Google Scholar 

  6. Z. Fan, H. Wu, X. Fu, Y. Huang, X. Ding. Residual-guide network for single image deraining. in Proceedings of the 26th ACM International Conference on Multimedia, pp. 1751–1759 (2018)

  7. X. Fu, J. Huang, X. Ding, Y. Liao, J. Paisley, Clearing the skies: a deep network architecture for single-image rain removal. IEEE Trans. Image Process. 26(6), 2944–2956 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  8. X. Fu, J. Huang, D. Zeng, Y. Huang, X. Ding, J. Paisley, Removing rain from single images via a deep detail network. in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3855–3863 (2017b)

  9. I. Goodfellow, Y. Bengio, A. Courville, Deep Learning. MIT Press. http://www.deeplearningbook.org (2016)

  10. K. He, J. Sun, X. Tang, Guided image filtering. IEEE Trans. Pattern Anal. Mach. Intell. 35(6), 1397–1409 (2012)

    Article  Google Scholar 

  11. G.E. Hinton, N. Srivastava, A. Krizhevsky, I. Sutskever, R.R. Salakhutdinov, Improving neural networks by preventing co-adaptation of feature detectors. arXiv preprint arXiv:1207.0580 (2012)

  12. D.-A. Huang, L.-W. Kang, Y.-C.F. Wang, C.-W. Lin, Self-learning based image decomposition with applications to single image denoising. IEEE Trans. Multimedia 16(1), 83–93 (2013)

    Article  Google Scholar 

  13. K. Jiang, Z. Wang, P. Yi, C. Chen, Z. Han, T. Lu, B. Huang, J. Jiang, Decomposition makes better rain removal: an improved attention-guided deraining network. IEEE Trans. Circuits Syst. Video Technol. 31(10), 3981–3995 (2020)

    Article  Google Scholar 

  14. K.-H. Lee, E. Ryu, J.-O. Kim, Progressive rain removal via a recurrent convolutional network for real rain videos. IEEE Access 8, 203134–203145 (2020)

    Article  Google Scholar 

  15. X. Li, J. Wu, Z. Lin, H. Liu, H. Zha, Recurrent squeeze-and-excitation context aggregation net for single image deraining. In Proceedings of the European Conference on Computer Vision (ECCV), pp. 254–269 (2018)

  16. Y. Li, R.T. Tan, X. Guo, J. Lu, M.S. Brown. Rain streak removal using layer priors. in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2736–2744 (2016)

  17. Y. Li, R.T. Tan, X. Guo, J. Lu, M.S. Brown, Single image rain streak decomposition using layer priors. IEEE Trans. Image Process. 26(8), 3874–3885 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  18. Q. Lian, W. Yan, X. Zhang, S. Chen, Single image rain removal using image decomposition and a dense network. IEEE/CAA J. Autom. Sin. 6(6), 1428–1437 (2019)

    Google Scholar 

  19. X. Lin, L. Ma, B. Sheng, Z.-J. Wang, W. Chen, Utilizing two-phase processing with fbls for single image deraining. IEEE Trans. Multimedia 23, 664–676 (2020)

    Article  Google Scholar 

  20. J. Liu, W. Yang, S. Yang, Z. Guo, D3r-net: dynamic routing residue recurrent network for video rain removal. IEEE Trans. Image Process. 28(2), 699–712 (2019)

    Article  MathSciNet  MATH  Google Scholar 

  21. Y. Luo, J. Zhu, J. Ling, E. Wu, Fast removal of rain streaks from a single image via a shape prior. IEEE Access 6, 60069–60078 (2018)

    Article  Google Scholar 

  22. P. Mu, J. Chen, R. Liu, X. Fan, Z. Luo, Learning bilevel layer priors for single image rain streaks removal. IEEE Signal Process. Lett. 26(2), 307–311 (2018)

    Article  Google Scholar 

  23. P. Mu, J. Chen, R. Liu, X. Fan, Z. Luo, Learning bilevel layer priors for single image rain streaks removal. IEEE Signal Process. Lett. 26(2), 307–311 (2019)

    Article  Google Scholar 

  24. J. Pan, S. Liu, D. Sun, J. Zhang, Y. Liu, J. Ren, Z. Li, J. Tang, H. Lu, Y.-W. Tai, et al. Learning dual convolutional neural networks for low-level vision. in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3070–3079 (2018)

  25. L. Peng, A. Jiang, Q. Yi, M. Wang, Cumulative rain density sensing network for single image derain. IEEE Signal Process. Lett. 27, 406–410 (2020)

    Article  Google Scholar 

  26. Y. Que, S. Li, H.J. Lee, Attentive composite residual network for robust rain removal from single images. IEEE Trans. Multimedia 23, 3059–3072 (2020)

    Article  Google Scholar 

  27. D. Ren, W. Zuo, Q. Hu, P. Zhu, D. Meng, Progressive image deraining networks: a better and simpler baseline. in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3937–3946 (2019)

  28. Z. Shi, Y. Li, M. Zhao, Y. Feng, L. He, Multi-stage filtering for single rainy image enhancement. IET Image Proc. 12(10), 1866–1872 (2018)

    Article  Google Scholar 

  29. X. Song, P. Sun, S. Song, V. Stojanovic. Event-driven NN adaptive fixed-time control for nonlinear systems with guaranteed performance. J. Frankl. Inst. (2022)

  30. C. Szegedy, S. Ioffe, V. Vanhoucke, A.A. Alemim Inception-v4, inception-resnet and the impact of residual connections on learning. in Thirty-First AAAI Conference on Artificial Intelligence (2017)

  31. C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan, V. Vanhoucke, A. Rabinovich. Going deeper with convolutions. in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–9 (2015)

  32. H. Tang, L. Zhu, D. Zhang, X. Wang, Single image rain removal model using pure rain dictionary learning. IET Image Proc. 13(10), 1797–1804 (2019)

    Article  Google Scholar 

  33. A. Tripathi, S. Mukhopadhyay, Video post processing: low-latency spatiotemporal approach for detection and removal of rain. IET Image Proc. 6(2), 181–196 (2012)

    Article  MathSciNet  Google Scholar 

  34. A.K. Tripathi, S. Mukhopadhyay, Meteorological approach for detection and removal of rain from videos. IET Comput. Vision 7(1), 36–47 (2013)

    Article  Google Scholar 

  35. C. Wang, C. Xu, C. Wang, D. Tao, Perceptual adversarial networks for image-to-image transformation. IEEE Trans. Image Process. 27(8), 4066–4079 (2018)

    Article  MathSciNet  MATH  Google Scholar 

  36. M. Wang, L. Chen, Y. Liang, Y. Hao, H. He, C. Li, Single image rain removal with reusing original input squeeze-and-excitation network. IET Image Processing (2020a)

  37. T. Wang, X. Yang, K. Xu, S. Chen, Q. Zhang, R.W. Lau, Spatial attentive single-image deraining with a high quality real rain dataset. in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 12270–12279 (2019)

  38. Y. Wang, S. Liu, C. Chen, B. Zeng, A hierarchical approach for rain or snow removing in a single color image. IEEE Trans. Image Process. 26(8), 3936–3950 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  39. Y.-T. Wang, X.-L. Zhao, T.-X. Jiang, L.-J. Deng, Y. Chang, T.-Z. Huang, Rain streaks removal for single image via kernel-guided convolutional neural network. IEEE Trans. Neural Netw. Learn. Syst. 32(8), 3664–3676 (2020)

    Article  Google Scholar 

  40. Z. Wang, A. Bovik, A universal image quality index. IEEE Signal Process. Lett. 9(3), 81–84 (2002)

    Article  Google Scholar 

  41. Z. Wang, A. Bovik, H. Sheikh, E. Simoncelli, Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13(4), 600–612 (2004)

    Article  Google Scholar 

  42. W. Wei, D. Meng, Q. Zhao, Z. Xu, Y, Wu Semi-supervised transfer learning for image rain removal. in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3877–3886 (2019)

  43. Y. Wei, Z. Zhang, Y. Wang, M. Xu, Y. Yang, S. Yan, M. Wang, Deraincyclegan: rain attentive cyclegan for single image deraining and rainmaking. IEEE Trans. Image Process. 30, 4788–4801 (2021)

    Article  Google Scholar 

  44. H. Xia, R. Zhuge, H. Li, S. Song, F. Jiang, M. Xu, Single image rain removal via a simplified residual dense network. IEEE Access 6, 66522–66535 (2018)

    Article  Google Scholar 

  45. P. Xiang, L. Wang, F. Wu, J. Cheng, M. Zhou, Single-image de-raining with feature-supervised generative adversarial network. IEEE Signal Process. Lett. 26(5), 650–654 (2019)

    Article  Google Scholar 

  46. X. Xin, Y. Tu, V. Stojanovic, H. Wang, K. Shi, S. He, T. Pan, Online reinforcement learning multiplayer non-zero sum games of continuous-time markov jump linear systems. Appl. Math. Comput. 412, 126537 (2022)

    MathSciNet  MATH  Google Scholar 

  47. K. Xu, X. Tian, X. Yang, B. Yin, R.W. Lau, Intensity-aware single-image deraining with semantic and color regularization. IEEE Trans. Image Process. 30, 8497–8509 (2021)

    Article  Google Scholar 

  48. W. Yang, J. Liu, S. Yang, Z. Guo, Scale-free single image deraining via visibility-enhanced recurrent wavelet learning. IEEE Trans. Image Process. 28(6), 2948–2961 (2019)

    Article  MathSciNet  MATH  Google Scholar 

  49. W. Yang, R.T. Tan, J. Feng, J. Liu, Z. Guo, S. Yan, Deep joint rain detection and removal from a single image. in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1357–1366 (2017)

  50. Y. Yang, J. Guan, S. Huang, W. Wan, Y. Xu, J. Liu, End-to-end rain removal network based on progressive residual detail supplement. IEEE Trans. Multimedia 24, 1622–1636 (2021)

    Article  Google Scholar 

  51. R. Yasarla, V.M. Patel, Confidence measure guided single image de-raining. IEEE Trans. Image Process. 29, 4544–4555 (2020)

    Article  MATH  Google Scholar 

  52. C.-H. Yeh, C.-H. Huang, L.-W. Kang, Multi-scale deep residual learning-based single image haze removal via image decomposition. IEEE Trans. Image Process. 29, 3153–3167 (2019)

    Article  MATH  Google Scholar 

  53. H. Zhang, V.M. Patel, Density-aware single image de-raining using a multi-stream dense network. in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 695–704 (2018)

  54. K. Zhang, D. Li, W. Luo, W. Ren, Dual attention-in-attention model for joint rain streak and raindrop removal. IEEE Trans. Image Process. 30, 7608–7619 (2021)

    Article  Google Scholar 

  55. X. Zhao, P. Liu, J. Liu, X. Tang, Removal of dynamic weather conditions based on variable time window. IET Comput. Vision 7(4), 219–226 (2013)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Electronics and Electrical Communication Engineering, Indian Institute of Technology Kharagpur, Kharagpur, West Bengal, 721302, India

    Bijaylaxmi Das, Ayan Saha & Sudipta Mukhopadhyay

Authors
  1. Bijaylaxmi Das
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ayan Saha
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Sudipta Mukhopadhyay
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sudipta Mukhopadhyay.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Das, B., Saha, A. & Mukhopadhyay, S. Rain Removal from a Single Image Using Refined Inception ResNet v2. Circuits Syst Signal Process 42, 3485–3508 (2023). https://doi.org/10.1007/s00034-022-02279-x

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00034-022-02279-x

Keywords

Search

Navigation