Skip to main content

Rain streak removal based on non-negative matrix factorization

Multimedia Tools and Applications volume 77, pages 20001–20020 (2018)Cite this article

Abstract

A rain streak in an image can degrade visual quality of that image to the human eye. Unfortunately, removing the rain streak from a single image represents a very challenging task. In this paper, a single image rain removal process based on non-negative matrix factorization is proposed. First, the rain image is broken down into a low-frequency and high-frequency part by a Gaussian filter. Therefore, the rain component, which lies mostly in the middle frequency range, can be discarded in high and low frequency domains. Next, non-negative matrix factorization (NMF) method is applied to deal with the rain streak in the low frequency domain. Finally, Canny edge detection and block copy strategy are performed separately to remove the rain component in the high frequency domain to improve image quality. In comparison with state-of-the-art approaches, the proposed method achieves competitive results without the need for an extra image database to train the dictionary.

This is a preview of subscription content, access via your institution.

We’re sorry, something doesn't seem to be working properly.

Please try refreshing the page. If that doesn't work, please contact support so we can address the problem.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16

References

  1. Barnum PC, Narasimhan S, Kanade T (2010) Analysis of rain and snow in frequency space. Int J Comput Vis 86(2–3):256–274

    Article  Google Scholar 

  2. Bossu J, Hautière N, Tarel J-P (2011) Rain or snow detection in image sequences through use of a histogram of orientation of streaks. Int J Comput Vis 93(3):348–367

    Article  Google Scholar 

  3. Brewer N, Liu N (2008) Using the shape characteristics of rain to identify and remove rain from video. Lecture Notes in Computer Science, pp 451–458

  4. Chen DY, Chen CC, Kang LW (2014) Visual depth guided color image rain streaks removal using sparse coding. IEEE Trans Circuits Syst Video Technol 24(8):1430–1455

    Article  Google Scholar 

  5. Fang S, Xia X-S, Huo X, Chen C-W (2014) Image dehazing using polarization effects of objects and airlight. Opt Express 22(16):19523–19537

    Article  Google Scholar 

  6. Garg K, Nayar SK (2004) Detection and removal of rain from videos. Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp 528–535

  7. Garg K, Nayar SK (2005) When does a camera see rain?. Proceedings of IEEE International Conference on Computer Vision, pp 1067–1074

  8. Garg K, Nayar SK (2006) Photorealistic rendering of rain streaks. ACM Trans Graph 25(3):996–1002

    Article  Google Scholar 

  9. Garg K, Nayar SK (2007) Vision and rain. Int J Comput Vis 75(1):3–27

    Article  Google Scholar 

  10. Gregor K, LeCun Y (2010) Learning fast approximations of sparse coding. Proceedings of the 27th International Conference on Machine Learning, pp 399–406

  11. Grindlay G (2010) "NMFLib-Efficient Matlab library implementing a number of common NMF variants."http://www.ee.columbia.edu/~grindlay/code.html

  12. He K, Sun J, Tang X (2011) Single image haze removal using dark channel prior. IEEE Trans Pattern Anal Mach Intell 33(12):2341–2353

    Article  Google Scholar 

  13. Huang DA, Kang LW, Yang MC, Lin CW, Wang YCF (2012) Context-aware single image rain removal. IEEE International Conference on Multimedia and Expo, pp 164–169

  14. Kang LW, Lin CW, Fu YH (2012) Automatic single-image-based rain streaks removal via image decomposition. IEEE Trans Image Process 21(4):1742–1755

    Article  MathSciNet  MATH  Google Scholar 

  15. Kang L-W, Zhuang B, Hsu C-C, Lin C-W, Yeh C-H (2013) Self-learning-based low-quality single image super resolution. Proceedings of IEEE International Workshop on Multimedia Signal Processing

  16. Krishnan S, Venkataraman D (2012) Restoration of video by removing rain. Int J Comput Sci Eng Appl 2(2):19–28

    Google Scholar 

  17. Lee DD, Seung HS (1999) Learning the parts of objects by non-negative matrix factorization. Nature 401:788–791

    Article  MATH  Google Scholar 

  18. Lee DD, Seung HS (2001) Algorithms for non-negative matrix factorization. Advances in neural information processing systems

  19. Lin CJ (2007) Projected gradient methods for nonnegative matrix factorization. Neural Comput 19(10):2756–2779

    Article  MathSciNet  MATH  Google Scholar 

  20. Mairal J, Bach F, Ponce J, Sapiro G (2009) Online dictionary learning for sparse coding. Proceedings of International Conference on Machine Learning, pp 689–696

  21. Mairal J, Bach F, Ponce J, Sapiro G (2010) Online learning for matrix factorization and sparse coding. J Mach Learn Res 11:19–60

    MathSciNet  MATH  Google Scholar 

  22. Manu BN (2015) Rain removal from still images using L0 gradient minimization technique. IEEE International Conference on Information Technology and Electrical Engineering, pp 263–268

  23. Shi Z, Long J, Tang W, Zhang C (2014) Single image dehazing in inhomogeneous atmosphere. Int J Light Electron Opt 125(15):3868–3875

    Article  Google Scholar 

  24. Utaminingrum F, Somawirata I-K (2015) Rain streaks removal by using composite method. IEEE International Conference on Signal-Image Technology & Internet-Based Systems, pp 139–142

  25. Wixson L (2000) Detecting salient motion by accumulating directionally-consistent flow. IEEE Trans Pattern Anal Mach Intell 22(8):774–780

    Article  Google Scholar 

  26. Wren CR, Azarbayejani A, Darrell T, Pentland AP (1997) Pfinder: real-time tracking of the human body. IEEE Trans Pattern Anal Mach Intell 19(7):780–785

    Article  Google Scholar 

  27. Xue X, Jin X, Zhang C, Goto S (2012) Motion robust rain detection and removal from videos. IEEE 14th International Workshop on Multimedia Signal Processing, pp 170–174

  28. Yeh C-H, Kang L-W, Lin C-Y (2012) Image haze removal via haze density analysis based on pixel-based dark channel prior. Proceedings of Computer Vision & Graphic Image Processing

  29. Yeh CH, Kang L-W, Lee M-S, Lin C-Y (2013) Haze effect removal from image via haze density estimation in optical model. Opt Express 21(22):27127–27141

    Article  Google Scholar 

  30. Yeh C-H, Kang L-W, Chiou Y-W, Lin C-W, Jiang S-JF (2014) Self-learning-based post-processing for image/video deblocking via sparse representation. J Vis Commun Image Represent 25:891–903

    Article  Google Scholar 

  31. Zhang X, Li H, Qi Y, Leow WK, Ng TK (2006) Rain removal in video by combining temporal and chromatic properties. Proceedings of IEEE International Conference on Multimedia and Expo, pp 461–464

Download references

Acknowledgements

This work was supported by Ministry of Science and Technology, Taiwan, under Grants MOST 103-2221-E-468-007-MY2, MOST 105-2221-E-155-083, MOST 103-2221-E-110-045-MY3, and NSC 102-2221-E-110-032-MY3.

Author information

Authors and Affiliations

  1. Department of Electrical Engineering, National Taiwan Normal University, Taipei, Taiwan

    Chia-Hung Yeh

  2. Department of Electrical Engineering, National Sun Yat-sen University, Kaohsiung, Taiwan

    Chia-Hung Yeh & Pin-Hsian Liu

  3. Department of Communications Engineering, Yuan-Ze University, Taoyuan, Taiwan

    Chih-Yang Lin

  4. Department of Electrical and Computer Engineering, Syiah Kuala University, Banda Aceh, Indonesia

    Kahlil Muchtar

Authors
  1. Chia-Hung Yeh
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Chih-Yang Lin
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Kahlil Muchtar
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Pin-Hsian Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Chih-Yang Lin.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Yeh, CH., Lin, CY., Muchtar, K. et al. Rain streak removal based on non-negative matrix factorization. Multimed Tools Appl 77, 20001–20020 (2018). https://doi.org/10.1007/s11042-017-5430-2

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-017-5430-2

Keywords

  • Rain removal
  • Non-negative matrix factorization (NMF)
  • Canny edge detection