Skip to main content
SpringerLink
Log in

Import vertical characteristic of rain streak for single image deraining

  • Regular Paper
  • Published:
Multimedia Systems Aims and scope Submit manuscript

Abstract

Recently, deep convolutional neural networks show good effect for single image deraining. These networks always adopt the conventional convolution method to extract features, which may neglect the characteristic of rain streak. A novelty vertical module is proposed to focus on the vertical characteristic of rain streak. Such module uses 1 \(\times X\) convolution kernel to extract the vertical information of rain streaks and a \(X \times X\) convolution kernel to keep relative location information. Use this module in the front of deraining network can better detach rain streaks from background. In addition, the contrastive learning is employed to improve the performance of the model. Extensive experimental results demonstrated the superiority of the deraining methods with the proposed methods in comparison with the base ones.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16

Similar content being viewed by others

References

  1. Ren, D., Zuo, W., Hu, Q., Zhu, P., Meng, D.: Progressive image deraining networks: a better and simpler baseline. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3937–3946 (2019)

  2. Khan, H., Xiao, B., Li, W., Muhammad, N.: Recent advancement in haze removal approaches. Multimed. Syst. 1–24 (2021)

  3. Ren, W., Zhou, L., Chen, J.: Unsupervised single image dehazing with generative adversarial network. Multimed. Syst. 1–11 (2022)

  4. Ren, D., Shang, W., Zhu, P., Hu, Q., Meng, D., Zuo, W.: Single image deraining using bilateral recurrent network. IEEE Trans. Image Process. 29, 6852–6863 (2020)

    Article  MATH  Google Scholar 

  5. Ding, J., Guo, H., Zhou, H., Yu, J., Jiang, B.: Distributed feedback network for single-image deraining. Inf. Sci. (2021)

  6. Wu, H., Qu, Y., Lin, S., Zhou, J., Qiao, R., Zhang, Z., Xie, Y., Ma, L.: Contrastive learning for compact single image dehazing. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 10551–10560 (2021)

  7. Khosla, P., Teterwak, P., Wang, C., Sarna, A., Tian, Y., Isola, P., Maschinot, A., Liu, C., Krishnan, D.: Supervised contrastive learning. Adv. Neural Inf. Process. Syst. 33, 18661–18673 (2020)

    Google Scholar 

  8. Dai, B., Lin, D.: Contrastive learning for image captioning, Advances in Neural Information Processing Systems, vol. 30 (2017)

  9. Garg, K., Nayar, S.K.: Detection and removal of rain from videos. In: Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004, vol. 1, pp. I–I. IEEE (2004)

  10. Jiang, T.-X., Huang, T.-Z., Zhao, X.-L., Deng, L.-J., Wang, Y.: A novel tensor-based video rain streaks removal approach via utilizing discriminatively intrinsic priors. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4057–4066 (2017)

  11. Jiang, T.-X., Huang, T.-Z., Zhao, X.-L., Deng, L.-J., Wang, Y.: Fastderain: a novel video rain streak removal method using directional gradient priors. IEEE Trans. Image Process. 28(4), 2089–2102 (2018)

    Article  MathSciNet  Google Scholar 

  12. Kim, J.-H., Sim, J.-Y., Kim, C.-S.: Video deraining and desnowing using temporal correlation and low-rank matrix completion. IEEE Trans. Image Process. 24(9), 2658–2670 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  13. Li, M., Xie, Q., Zhao, Q., Wei, W., Gu, S., Tao, J., Meng, D.: Video rain streak removal by multiscale convolutional sparse coding. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 6644–6653 (2018)

  14. Li, Y., Tan, R. T., Guo, X., Lu, J., Brown, M. S.: Rain streak removal using layer priors. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2736–2744 (2016)

  15. Luo, Y., Xu, Y., Ji, H.: Removing rain from a single image via discriminative sparse coding. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 3397–3405 (2015)

  16. Ren, D., Zuo, W., Zhang, D., Zhang, L., Yang, M.-H.: Simultaneous fidelity and regularization learning for image restoration. IEEE Trans. Pattern Anal. Mach. Intell. 43(1), 284–299 (2019)

    Article  Google Scholar 

  17. Fu, X., Huang, J., Ding, X., Liao, Y., Paisley, J.: Clearing the skies: a deep network architecture for single-image rain removal. IEEE Trans. Image Process. 26(6), 2944–2956 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  18. Fu, X., Huang, J., Zeng, D., Huang, Y., Ding, X., Paisley, J.: Removing rain from single images via a deep detail network. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3855–3863 (2017)

  19. Denton, E.L., Zaremba, W., Bruna, J., LeCun, Y., Fergus, R.: Exploiting linear structure within convolutional networks for efficient evaluation, Advances in Neural Information Processing Systems, vol. 27 (2014)

  20. Jaderberg, M., Vedaldi, A., Zisserman, A.: Speeding up convolutional neural networks with low rank expansions (2014). arXiv:1405.3866

  21. Jin, J., Dundar, A., Culurciello, E.: Flattened convolutional neural networks for feedforward acceleration (2014). arXiv:1412.5474

  22. Lo, S.-Y., Hang, H.-M., Chan, S.-W., Lin, J.-J.: Efficient dense modules of asymmetric convolution for real-time semantic segmentation. In: Proceedings of the ACM Multimedia Asia, pp. 1–6 (2019)

  23. Ding, X., Guo, Y., Ding, G., Han, J.: Acnet: Strengthening the kernel skeletons for powerful cnn via asymmetric convolution blocks. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 1911–1920 (2019)

  24. Sermanet, P., Lynch, C., Chebotar, Y., Hsu, J., Jang, E., Schaal, S., Levine, S., Brain, G.: Time-contrastive networks: self-supervised learning from video. In: 2018 IEEE International Conference on Robotics and Automation (ICRA), pp. 1134–1141. IEEE (2018)

  25. Tian, Y., Krishnan, D., Isola, P.: Contrastive multiview coding. In: European Conference on Computer Vision, pp. 776–794. Springer (2020)

  26. Henaff, O.: Data-efficient image recognition with contrastive predictive coding. In: International Conference on Machine Learning, pp. 4182–4192. PMLR (2020)

  27. He, K., Fan, H., Wu, Y., Xie, S., Girshick, R.: Momentum contrast for unsupervised visual representation learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9729–9738 (2020)

  28. Chen, T., Kornblith, S., Norouzi, M., Hinton, G.: A simple framework for contrastive learning of visual representations. In: International Conference on Machine Learning, pp. 1597–1607. PMLR (2020)

  29. Grill, J.-B., Strub, F., Altché, F., Tallec, C., Richemond, P., Buchatskaya, E., Doersch, C., Avila Pires, B., Guo, Z., Gheshlaghi Azar, M.: et al. Bootstrap your own latent-a new approach to self-supervised learning, Advances in Neural Information Processing Systems, vol. 33, pp. 21271–21284 (2020)

  30. Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition (2014). arXiv:1409.1556

  31. Zhao, H., Gallo, O., Frosio, I., Kautz, J.: Loss functions for image restoration with neural networks. IEEE Trans. Comput. Imaging 3(1), 47–57 (2016)

    Article  Google Scholar 

  32. Chen, Y., Dai, X., Liu, M., Chen, D., Yuan, L., Liu, Z.: Dynamic convolution: Attention over convolution kernels. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 11030–11039 (2020)

  33. Yang, W., Tan, R. T., Feng, J., Liu, J., Guo, Z., Yan, S.: Deep joint rain detection and removal from a single image. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1357–1366 (2017)

  34. Yang, W., Tan, R.T., Feng, J., Guo, Z., Yan, S., Liu, J.: Joint rain detection and removal from a single image with contextualized deep networks. IEEE Trans. Pattern Anal. Mach. Intell. 42(6), 1377–1393 (2019)

    Article  Google Scholar 

Download references

Acknowledgements

The authors acknowledge the financial supported by the Fundamental Research Funds for the Provincial Universities of Zhejiang (Grant No. GK219909299001-015), Natural Science Foundation of Zhejiang Province (Grant No. LY22F020028), National Undergraduate Training Program for Innovation and Entrepreneurship (Grant No. 202110336042) and Planted talent plan (Grant No. 2022R407A002).

Author information

Authors and Affiliations

  1. Zhuoyue Honors College, Hangzhou Dianzi University, Hangzhou, China

    Zhexin Zhang & Yiming Yuan

  2. Computer and Software School, Hangzhou Dianzi University, Hangzhou, China

    Jiajun Ding, Jun Yu & Jianping Fan

Authors
  1. Zhexin Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jiajun Ding
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jun Yu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yiming Yuan
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jianping Fan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jianping Fan.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhang, Z., Ding, J., Yu, J. et al. Import vertical characteristic of rain streak for single image deraining. Multimedia Systems 29, 105–115 (2023). https://doi.org/10.1007/s00530-022-00958-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00530-022-00958-y

Keywords

Search

Navigation