Skip to main content
SpringerLink
Log in

A two-stage network with wavelet transformation for single-image deraining

  • Original article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

Image deraining is still a vital and challenging low-level computer vision task, the purpose of which is to restore rain-free images from images degraded by rain streaks. Recently, many convolutional neural network (CNN)-based methods have made significant progress for the task of image deraining. However, there are still two issues with these methods: The rain streaks in heavy rain images cannot be effectively removed, and the high-quality images with clear details cannot be reconstructed. To solve the first issue, we adopt a two-stage structure to gradually remove rain streaks and use dilated convolution in the first stage network to rapidly enlarge the receptive field to capture the spatial characteristics of heavy rain streaks. For the latter issue, we introduced a structure-preserving network (SPN) without any up- and down-sampling and designed a feature extraction module based on wavelet transform in SPN to help SPN restore clear high-frequency details. In addition, we also designed a feature filter (FF) and multi-level feature fusion module (MLFFM), so that the valuable features in the first stage can be fully utilized in the second stage. Extensive experiments on six synthetic datasets and one real-world dataset indicate that our method can achieve excellent performance, compared with the current state-of-the-art methods. To further demonstrate the practical applicability of the proposed method, we use it as a semantic segmentation preprocessing step and display the semantic segmentation results to verify the effectiveness of our deraining method in downstream vision tasks. The source codes will be open at https://github.com/noxsine/WTSDNet.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

References

  1. Wang, H., Chen, Y., Wu, M., Zhang, X., Huang, Z., Mao, W.: Attentional and adversarial feature mimic for efficient object detection. Vis Comput (2022). https://doi.org/10.1007/s00371-021-02363-4

    Article  Google Scholar 

  2. Chen, G., Qin, H.: Class-discriminative focal loss for extreme imbalanced multiclass object detection towards autonomous driving. Vis Comput (2021). https://doi.org/10.1007/s00371-021-02067-9

    Article  Google Scholar 

  3. Devi, R.B., Chanu, Y.J., Singh, K.M.: Discriminative object tracking with subspace representation. Vis Comput 37, 1207–1219 (2021). https://doi.org/10.1007/s00371-020-01862-0

    Article  Google Scholar 

  4. Fan, J., Yang, X., Lu, R., Li, W., Huang, Y.: Long-term visual tracking algorithm for UAVs based on kernel correlation filtering and SURF features. Vis Comput (2022). https://doi.org/10.1007/s00371-021-02331-y

    Article  Google Scholar 

  5. Lin, Z., Sun, W., Tang, B., Li, J., Yao, X., Li, Y.: Semantic segmentation network with multi-path structure, attention reweighting and multi-scale encoding. Vis Comput (2022). https://doi.org/10.1007/s00371-021-02360-7

    Article  Google Scholar 

  6. He, J.Y., Liang, S.H., Wu, X., Zhao, B., Zhang, L.: MGSeg: Multiple Granularity-Based Real-Time Semantic Segmentation Network. IEEE Trans. Image Process. 30, 7200–7214 (2021)

    Article  Google Scholar 

  7. Yang, W., Tan, R. T., Feng, J., Liu, J., Guo, Z., Yan, S. Deep joint rain detection and removal from a single image. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1357–1366 (2017). https://doi.org/10.1109/CVPR.2017.183

  8. Kim, J. H., Lee, C., Sim, J. Y., Kim, C. S.: Single-image deraining using an adaptive nonlocal means filter. In: 2013 IEEE International Conference on Image Processing (ICIP), pp. 914–917 (2013). https://doi.org/10.1109/ICIP.2013.6738189

  9. Du, S., Liu, Y., Ye, M., Xu, Z., Li, J., Liu, J.: Single image deraining via decorrelating the rain streaks and background scene in gradient domain. Pattern Recogn. 79, 303–317 (2018). https://doi.org/10.1016/j.patcog.2018.02.016

    Article  Google Scholar 

  10. Zhu, L., Fu, C. W., Lischinski, D., Heng, P. A.: Joint bi-layer optimization for single-image rain streak removal. In: Proceedings of the IEEE International Conference on Computer Vision (ICCV), pp. 2526–2534 (2017). https://doi.org/10.1109/ICCV.2017.276

  11. Li, Y., Tan, R. T., Guo, X., Lu, J., Brown, M. S.: Rain streak removal using layer priors. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2736–2744 (2016). https://doi.org/10.1109/CVPR.2016.299

  12. Luo, Y., Xu, Y., Ji, H.: Removing rain from a single image via discriminative sparse coding. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 3397–3405 (2015). https://doi.org/10.1109/ICCV.2015.388

  13. Gu, S., Meng, D., Zuo, W., Zhang, L.: Joint convolutional analysis and synthesis sparse representation for single image layer separation. In Proceedings of the IEEE International Conference on Computer Vision, pp. 1708–1716 (2017). https://doi.org/10.1109/ICCV.2017.276

  14. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016). https://doi.org/10.1109/AIEA53260.2021.00077

  15. Liu, C., Chen, L. C., Schroff, F., Adam, H., Hua, W., Yuille, A. L., Fei-Fei, L.: Auto-deeplab: Hierarchical neural architecture search for semantic image segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 82–92 (2019). https://doi.org/10.1109/CVPR.2019.00017

  16. Wang, C., Xing, X., Yao, G.: Single image deraining via deep shared pyramid network. Vis Comput 37, 1851–1865 (2021). https://doi.org/10.1007/s00371-020-01944-z

    Article  Google Scholar 

  17. Sharma, P.K., Basavaraju, S., Sur, A.: Deep learning-based image de-raining using discrete Fourier transformation. Vis Comput 37, 2083–2096 (2021). https://doi.org/10.1007/s00371-020-01971-w

    Article  Google Scholar 

  18. Wang, G., Sun, C., Sowmya, A.: Context-enhanced representation learning for single image deraining. Int J Comput Vis 129, 1650–1674 (2021). https://doi.org/10.1007/s11263-020-01425-9

    Article  Google Scholar 

  19. Yasarla, R., Sindagi, V.A., Patel, V.M.: Semi-supervised image deraining using Gaussian processes. IEEE Trans. Image Process. 30, 6570–6582 (2021). https://doi.org/10.1109/TIP.2021.3096323

    Article  Google Scholar 

  20. Fu, X., Huang, J., Ding, X., Liao, Y., Paisley, J.: Clearing the skies: A deep network architecture for single-image rain removal. IEEE Trans. Image Process. 26(6), 2944–2956 (2017). https://doi.org/10.1109/TIP.2017.2691802

    Article  MathSciNet  MATH  Google Scholar 

  21. Ren, D., Zuo, W., Hu, Q., Zhu, P., Meng, D. Progressive image deraining networks: A better and simpler baseline. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3937–3946 (2019). https://doi.org/10.1109/CVPR.2019.00406

  22. Jiang, K., Wang, Z., Yi, P., Chen, C., Wang, Z., Wang, X., Lin, C.W.: Rain-free and residue hand-in-hand: a progressive coupled network for real-time image deraining. IEEE Trans. Image Process. 30, 7404–7418 (2021)

    Article  Google Scholar 

  23. Fu, X., Huang, J., Zeng, D., Huang, Y., Ding, X., Paisley, J. Removing rain from single images via a deep detail network. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3855–3863 (2017). https://doi.org/10.1109/CVPR.2017.186

  24. Zheng, Y., Yu, X., Liu, M., Zhang, S.: Single-image deraining via recurrent residual multiscale networks. IEEE Trans. Neural. Netw. Learn. Syst. (2020). https://doi.org/10.1109/TNNLS.2020.3041752

    Article  Google Scholar 

  25. Yang, Y., Ran, W., Lu, H. Rddan: A residual dense dilated aggregated network for single image deraining. In: 2020 IEEE International Conference on Multimedia and Expo (ICME), pp. 1–6 (2020) https://doi.org/10.1109/ICME46284.2020.9102945

  26. Zhang, H., Sindagi, V., Patel, V.M.: Image de-raining using a conditional generative adversarial network. IEEE Trans. Circuits Syst. Video Technol. 30(11), 3943–3956 (2019). https://doi.org/10.1109/TCSVT.2019.2920407

    Article  Google Scholar 

  27. Li, X., Wu, J., Lin, Z., Liu, H., Zha, H. Recurrent squeeze-and-excitation context aggregation net for single image deraining. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 254–269 (2018). https://doi.org/10.1007/978-3-030-01234-2_16

  28. Yang, Y., Lu, H. Single image deraining via recurrent hierarchy enhancement network. In: Proceedings of the 27th ACM International Conference on Multimedia, pp. 1814–1822 (2019). https://doi.org/10.1145/3343031.3351149

  29. Zamir, S. W., Arora, A., Khan, S., Hayat, M., Khan, F. S., Yang, M. H., Shao, L. Multi-stage progressive image restoration. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 14821–14831 (2021). https://doi.org/10.1109/CVPR46437.2021.

  30. Chen, L., Lu, X., Zhang, J., Chu, X., Chen, C. HINet: Half instance normalization network for image restoration. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 182–192 (2021) https://doi.org/10.1109/10.1109/CVPRW53098.2021. 00027

  31. Jiang, K., Wang, Z., Yi, P., Chen, C., Huang, B., Luo, Y., Jiang, J: Multi-scale progressive fusion network for single image deraining. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8346–8355 (2020). https://doi.org/10.1109/CVPR42600.2020.00837

  32. Chen, X., Huang, Y., Xu, L.: Multi-scale hourglass hierarchical fusion network for single image deraining. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 872–879 (2021). https://doi.org/10.1109/CVPRW53098.2021.00097

  33. Xu, J., Zhao, W., Liu, P., Tang, X.: Removing rain and snow in a single image using guided filter. In: 2012 IEEE International Conference on Computer Science and Automation Engineering (CSAE), pp. 304–307 (2012). https://doi.org/10.1109/CSAE.2012.6272780

  34. Kang, L. W., Lin, C. W., Fu, Y. H.: Automatic single-image-based rain streaks removal via image decomposition. IEEE Transactions on Image Processing, 21(4), 1742–1755 (2011)

  35. Chen, Y. L., Hsu, C. T.: A generalized low-rank appearance model for spatio-temporally correlated rain streaks. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 1968–1975 (2013). https://doi.org/10.1109/ICCV.2013.247

  36. Wang, Y., Liu, S., Chen, C., Zeng, B.: A hierarchical approach for rain or snow removing in a single color image. IEEE Trans. Image Process. 26(8), 3936–3950 (2017). https://doi.org/10.1109/TIP.2017.2708502

    Article  MathSciNet  MATH  Google Scholar 

  37. Ran, W., Yang, Y., Lu, H.: Single image rain removal boosting via directional gradient. In: 2020 IEEE International Conference on Multimedia and Expo (ICME), pp. 1–6 (2020). https://doi.org/10.1109/ICME46284.2020.9102800

  38. Vidanpathirana, M., Sudasingha, I., Vidanapathirana, J.: Tracking and frame-rate enhancement for real-time 2D human pose estimation. Vis Comput 36, 1501–1519 (2020). https://doi.org/10.1007/s00371-019-01757-9

    Article  Google Scholar 

  39. Li, B., Gou, Y., Gu, S.: You only look yourself: Unsupervised and untrained single image dehazing neural network. Int J Comput Vis 129, 1754–1767 (2021). https://doi.org/10.1007/s11263-021-01431-5

    Article  Google Scholar 

  40. Howard, A., Sandler, M., Chu, G., Chen, L. C., Chen, B., Tan, M., Adam, H.: Searching for mobilenetv3. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 1314–1324 (2019). https://doi.org/10.1109/ICCV.2019.00140

  41. Zhang, H., Patel, V. M.: Density-aware single image de-raining using a multi-stream dense network. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 695–704 (2018). https://doi.org/10.1109/CVPR.2018.00079

  42. Liu, X., Suganuma, M., Sun, Z., Okatani, T.: Dual residual networks leveraging the potential of paired operations for image restoration. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition pp. 7007–7016 (2019). https://doi.org/10.1109/CVPR.2019.00717

  43. Fu, X., Liang, B., Huang, Y., Ding, X., Paisley, J.: Lightweight pyramid networks for image deraining. IEEE Transactions on Neural Networks and Learning Systems 31(6), 1794–1807 (2019). https://doi.org/10.1109/TNNLS.2019.2926481

    Article  Google Scholar 

  44. Wang, T., Yang, X., Xu, K., Chen, S., Zhang, Q., Lau, R. W.: Spatial attentive single-image deraining with a high quality real rain dataset. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 12270–12279 (2019). https://doi.org/10.1109/CVPR.2019.01255

  45. Du, Y., Xu, J., Zhen, X., Cheng, M.M., Shao, L.: Conditional variational image deraining. IEEE Trans. Image Process. 29, 6288–6301 (2020). https://doi.org/10.1109/TIP.2020.2990606

    Article  MATH  Google Scholar 

  46. Yasarla, R., Patel, V.M.: Confidence measure guided single image de-raining. IEEE Trans. Image Process. 29, 4544–4555 (2020). https://doi.org/10.1109/TIP.2020.2973802

    Article  MATH  Google Scholar 

  47. Wang, H., Xie, Q., Zhao, Q., Meng, D.: A model-driven deep neural network for single image rain removal. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3103–3112 (2020). https://doi.org/10.1109/CVPR42600.2020.00317

  48. Yi, Q., Li, J., Dai, Q., Fang, F., Zhang, G., Zeng, T.: Structure-preserving deraining with residue channel prior guidance. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 42384247 (2021). URL https://openaccess.thecvf.com/content/ICCV2021/html/Yi_StructurePreserving_Deraining_With_Residue_Channel_Prior_Guidance_ICCV_2021_paper.html

  49. Fu, X., Qi, Q., Zha, Z. J., Zhu, Y., Ding, X. Rain streak removal via dual graph convolutional network. In Proc. AAAI Conf. Artif. Intell, pp. 1–9 (2021). https://ojs.aaai.org/index.php/AAAI/article/view/16224.

  50. Chen, C., Li, H.: Robust representation learning with feedback for single image deraining. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 7742–7751 (2021). https://doi.org/10.1109/CVPR46437.2021.00765

  51. Zhu, J. Y., Park, T., Isola, P., Efros, A. A.: Unpaired image-to-image translation using cycle-consistent adversarial networks. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2223–2232 (2017). https://doi.org/10.1109/ICCV.2017.244

  52. Wei, Y., Zhang, Z., Wang, Y., Xu, M., Yang, Y., Yan, S., Wang, M.: DerainCycleGAN: Rain attentive CycleGAN for single image deraining and rainmaking. IEEE Trans. Image Process. 30, 4788–4801 (2021). https://doi.org/10.1109/TIP.2021.3074804

    Article  Google Scholar 

  53. Charbonnier, P., Blanc-Feraud, L., Aubert, G., Barlaud, M.: Two deterministic half-quadratic regularization algorithms for computed imaging. In: Proceedings of 1st International Conference on Image Processing, Vol. 2, pp. 168–172 (1994). https://doi.org/10.1109/ICIP.1994.413553

  54. Huynh-Thu, Q., Ghanbari, M.: Scope of validity of PSNR in image/video quality assessment. Electron. Lett. 44(13), 800–801 (2008)

    Article  Google Scholar 

  55. Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13(4), 600–612 (2004)

    Article  Google Scholar 

  56. Zhang, L., Zhang, L., Mou, X., Zhang, D.: FSIM: A feature similarity index for image quality assessment. IEEE Trans. Image Process. 20(8), 2378–2386 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  57. Sheikh, H.R., Bovik, A.C.: Image information and visual quality. IEEE Trans. Image Process. 15(2), 430–444 (2006)

    Article  Google Scholar 

  58. Feng, Q., Fei, H., Wencheng, W.: Blind image deblurring with reinforced use of edges. Vis Comput 35, 1081–1090 (2019). https://doi.org/10.1007/s00371-019-01697-4

    Article  Google Scholar 

  59. Cho, S. J., Ji, S. W., Hong, J. P., Jung, S. W., Ko, S. J. Rethinking coarse-to-fine approach in single image deblurring. In Proceedings of the IEEE/CVF International Conference on Computer Vision pp. 4641–4650. (2021)

  60. Xu, H., Jia, X., Cheng, L., Huang, H. Affine non-local Bayesian image denoising algorithm. Visual Computer, 1–20 (2022). https://doi.org/10.1007/s00371-021-02316-x

  61. Ko, K., Koh, Y.J., Kim, C.S.: Blind and Compact Denoising Network Based on Noise Order Learning. IEEE Trans. Image Process. 31, 1657–1670 (2022). https://doi.org/10.1109/TIP.2022.3145160

    Article  Google Scholar 

Download references

Acknowledgements

The authors thank the editors and anonymous reviewers for their detailed reviews, valuable comments, and constructive suggestions for this study. This work was supported by National Natural Science Foundation of China (62066047, 61966037) and Yunnan University Postgraduate Practice Innovation Project (2021Y186).

Author information

Authors and Affiliations

  1. School of Information Science and Engineering, Yunnan University, Kunming, 650504, China

    Hao Yang, Dongming Zhou, Miao Li & Qian Zhao

Authors
  1. Hao Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Dongming Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Miao Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Qian Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Dongming Zhou.

Ethics declarations

Conflict of interest

The authors declared that they have no conflicts of interest to this work.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yang, H., Zhou, D., Li, M. et al. A two-stage network with wavelet transformation for single-image deraining. Vis Comput 39, 3887–3903 (2023). https://doi.org/10.1007/s00371-022-02533-y

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-022-02533-y

Keywords

Search

Navigation