Using Motion Compensation and Matrix Completion Algorithm to Remove Rain Streaks and Snow for Video Sequences

Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 12046)


The current outdoor surveillance equipment and cameras are vulnerable to be influenced by rain, snow, and other inclement weather, reducing the performance of the surveillance systems. In this paper, we propose a method to detect and remove rain streaks even snow artifacts from video sequences, using motion compensation and low-rank matrix completion method. First, we adopt the optical flow estimation method between consecutive frames to get a warped frame and obtain an initial binary rain map. We further use morphological component analysis method to dilate the tiny rain streaks. Then we employ the online dictionary learning for sparse representation technique and SVM classifier to refine the rain map by getting rid of parts which are not rain streaks. Finally, we reconstruct the video sequence by using low-rank matrix completion techniques. The experimental results demonstrate the proposed algorithm and perform qualitatively as well as quantitatively better in terms of PSNR/SSIM.


Rain streaks removal Motion compensation Sparse representation technique Block matching estimation 


  1. 1.
    Abdel-Hakim, A.E.: A novel approach for rain removal from videos using low-rank recovery. In: 2014 5th International Conference on Intelligent Systems, Modelling and Simulation, pp. 351–356. IEEE (2014)Google Scholar
  2. 2.
    Barnum, P.C., Narasimhan, S.G., Kanade, T.: Analysis of rain and snow in frequency space. Int. J. Comput. Vision 86, 256–274 (2008)CrossRefGoogle Scholar
  3. 3.
    Bossu, J., Hautière, N., Tarel, J.P.: Rain or snow detection in image sequences through use of a histogram of orientation of streaks. Int. J. Comput. Vision 93(3), 348–367 (2011) CrossRefGoogle Scholar
  4. 4.
    Elad, M., Aharon, M.: Image denoising via sparse and redundant representations over learned dictionaries. IEEE Trans. Image Process. 15(12), 3736–3745 (2006)MathSciNetCrossRefGoogle Scholar
  5. 5.
    Fadili, M.J., Starck, J.L., Bobin, J., Moudden, Y.: Image decomposition and separation using sparse representations: an overview. Proc. IEEE 98(6), 983–994 (2009)CrossRefGoogle Scholar
  6. 6.
    Garg, K., Nayar, S.K.: Detection and removal of rain from videos. In: Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, CVPR 2004, vol. 1, p. I (2004)Google Scholar
  7. 7.
    Garg, K., Nayar, S.K.: When does a camera see rain? In: Tenth IEEE International Conference on Computer Vision (ICCV 2005) Volume 1, vol. 2, pp. 1067–1074 (2005)Google Scholar
  8. 8.
    Garg, K., Nayar, S.K.: Vision and rain. Int. J. Comput. Vision 75, 3–27 (2006)CrossRefGoogle Scholar
  9. 9.
    George, J., Bhavani, S., Jaya, J.: Certain explorations on removal of rain streaks using morphological component analysis. Int. J. Eng. Res. Technol. (IJERT) 2 (2013). ISSN: 2278-0181Google Scholar
  10. 10.
    Jiang, T.X., Huang, T.Z., Zhao, X.L., Deng, L.J., Wang, Y.: FastDeRain: a novel video rain streak removal method using directional gradient priors. IEEE Trans. Image Process. 28, 2089–2102 (2018)MathSciNetCrossRefGoogle Scholar
  11. 11.
    Kim, J.H., Sim, J.Y., Kim, C.S.: Video deraining and desnowing using temporal correlation and low-rank matrix completion. IEEE Trans. Image Process. 24, 2658–2670 (2015)MathSciNetCrossRefGoogle Scholar
  12. 12.
    Li, M., et al.: Video rain streak removal by multiscale convolutional sparse coding. In: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 6644–6653 (2018)Google Scholar
  13. 13.
    Li, R., Cheong, L.F., Tan, R.T.: Heavy rain image restoration: integrating physics model and conditional adversarial learning. arXiv abs/1904.05050 (2019)Google Scholar
  14. 14.
    Li, X., Wu, J., Lin, Z., Liu, H., Zha, H.: Recurrent squeeze-and-excitation context aggregation net for single image deraining. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11211, pp. 262–277. Springer, Cham (2018). Scholar
  15. 15.
    Liu, C., et al.: Beyond pixels: exploring new representations and applications for motion analysis. Ph.D. thesis, Massachusetts Institute of Technology (2009)Google Scholar
  16. 16.
    Liu, P., Xu, J., Liu, J., Tang, X.: Pixel based temporal analysis using chromatic property for removing rain from videos. Comput. Inf. Sci. 2, 53–60 (2009)Google Scholar
  17. 17.
    Mairal, J., Bach, F.R., Ponce, J., Sapiro, G.: Online learning for matrix factorization and sparse coding. J. Mach. Learn. Res. 11, 19–60 (2010)MathSciNetzbMATHGoogle Scholar
  18. 18.
    Mukhopadhyay, S., Tripathi, A.K.: Combating bad weather part i: rain removal from video. Synth. Lect. Image Video Multimedia Process. 7(2), 1–93 (2014)CrossRefGoogle Scholar
  19. 19.
    Qian, R., Tan, R.T., Yang, W., Su, J., Liu, J.: Attentive generative adversarial network for raindrop removal from a single image. In: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2482–2491 (2017)Google Scholar
  20. 20.
    Ren, D., Zuo, W., Hu, Q., Zhu, P., Meng, D.: Progressive image deraining networks: a better and simpler baseline. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2019Google Scholar
  21. 21.
    Tripathi, A.K., Mukhopadhyay, S.: Removal of rain from videos: a review. SIViP 8, 1421–1430 (2014)CrossRefGoogle Scholar
  22. 22.
    Tripathi, A., Mukhopadhyay, S.: Video post processing: low-latency spatiotemporal approach for detection and removal of rain. IET Image Proc. 6(2), 181–196 (2012)MathSciNetCrossRefGoogle Scholar
  23. 23.
    Wang, T., Yang, X., Xu, K., Chen, S., Zhang, Q., Lau, R.W.: Spatial attentive single-image deraining with a high quality real rain dataset. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2019Google Scholar
  24. 24.
    Wei, W., Yi, L., Xie, Q., Zhao, Q., Meng, D., Xu, Z.: Should we encode rain streaks in video as deterministic or stochastic? In: 2017 IEEE International Conference on Computer Vision (ICCV), pp. 2535–2544 (2017)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  1. 1.Ritsumeikan UniversityKusatsuJapan

Personalised recommendations