Raindrop Detection and Removal from Long Range Trajectories

  • Shaodi YouEmail author
  • Robby T. Tan
  • Rei Kawakami
  • Yasuhiro Mukaigawa
  • Katsushi Ikeuchi
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9004)


In rainy scenes, visibility can be degraded by raindrops which have adhered to the windscreen or camera lens. In order to resolve this degradation, we propose a method that automatically detects and removes adherent raindrops. The idea is to use long range trajectories to discover the motion and appearance features of raindrops locally along the trajectories. These motion and appearance features are obtained through our analysis of the trajectory behavior when encountering raindrops. These features are then transformed into a labeling problem, which the cost function can be optimized efficiently. Having detected raindrops, the removal is achieved by utilizing patches indicated, enabling the motion consistency to be preserved. Our trajectory based video completion method not only removes the raindrops but also complete the motion field, which benefits motion estimation algorithms to possibly work in rainy scenes. Experimental results on real videos show the effectiveness of the proposed method.


Optical Flow Motion Estimation Image Patch Label Problem Binary Label 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.



This work is supported by Next-generation Energies for Tohoku Recovery (NET), MEXT, Japan.

Supplementary material

Supplementary material (mp4 17,261 KB)


  1. 1.
    Tan, R.: Visibility in bad weather from a single image. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2008)Google Scholar
  2. 2.
    Fattal, R.: Single image dehazing. In: SIGGRAPH (2008)Google Scholar
  3. 3.
    He, K., Sun, J., Tang, X.: Single image haze removal using dark channel prior. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR) (2009)Google Scholar
  4. 4.
    Meng, G., Wang, Y., Duan, J., Xiang, S., Pan, C.: Efficient image dehazing with boundary constraint and contextual regularization. In: ICCV (2013)Google Scholar
  5. 5.
    Barnum, P., Narasimhan, S., Kanade, T.: Analysis of rain and snow in frequency space. Int. J. Comput. Vis. (IJCV) 86, 256–274 (2010)CrossRefGoogle Scholar
  6. 6.
    Garg, K., Nayar, S.: Vision and rain. Int. J. Comput. Vis. (IJCV) 75, 3–27 (2007)CrossRefGoogle Scholar
  7. 7.
    Chen, Y.L., Hsu, C.T.: A generalized low-rank appearance model for spatio-temporally correlated rain streaks. In: ICCV (2013)Google Scholar
  8. 8.
    Garg, K., Nayar, S.: Photometric model of a raindrop. CMU Technical report (2003)Google Scholar
  9. 9.
    Garg, K., Nayar, S.: Detection and removal of rain from video. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR) (2004)Google Scholar
  10. 10.
    Kang, L., Lin, C., Fu, Y.: Automatic single-image-based rain streaks removal via image decomposition. IEEE Trans. Image Process. (TIP) 21, 1742–1755 (2012)CrossRefMathSciNetGoogle Scholar
  11. 11.
    Eigen, D., Krishnan, D., Fergus, R.: Restoring an image taken through a window covered with dirt or rain. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 633–640 (2013)Google Scholar
  12. 12.
    Kurihata, H., Takahashi, T., Ide, I., Mekada, Y., Murase, H., Tamatsu, Y., Miyahara, T.: Rainy weather recognition from in-vehicle camera images for driver assistance. In: IEEE Intelligent Vehicles Symposium (2005)Google Scholar
  13. 13.
    Yamashita, A., Tanaka, Y., Kaneko, T.: Removal of adherent water-drops from images acquired with stereo camera. In: IROS (2005)Google Scholar
  14. 14.
    Yamashita, A., Fukuchi, I., Kaneko, T.: Noises removal from image sequences acquired with moving camera by estimating camera motion from spatio-temporal information. In: IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (2009)Google Scholar
  15. 15.
    Roser, M., Geiger, A.: Video-based raindrop detection for improved image registration. In: Workshops of IEEE International Conference on Computer Vision (2009)Google Scholar
  16. 16.
    Roser, M., Kurz, J., Geiger, A.: Realistic modeling of water droplets for monocular adherent raindrop recognition using bezier curves. In: Asian Conference on Computer Vision (ACCV) (2010)Google Scholar
  17. 17.
    You, S., Tan, R.T., Kawakami, R., Ikeuchi, K.: Adherent raindrop detection and removal in video. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR) (2013)Google Scholar
  18. 18.
    Wexler, Y., Shechtman, E., Irani, M.: Space-time video completion. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2004)Google Scholar
  19. 19.
    Willson, R.G., Maimone, M., Johnson, A., Scherr, L.: An Optical Model for Image Artifacts Produced by Dust Particles on Lenses. Jet Propulsion Laboratory, National Aeronautics and Space Administration, Pasadena, CA (2005)Google Scholar
  20. 20.
    Zhou, C., Lin, S.: Removal of image artifacts due to sensor dust. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2007, pp. 1–8. IEEE (2007)Google Scholar
  21. 21.
    Gu, J., Ramamoorthi, R., Belhumeur, P., Nayar, S.: Removing image artifacts due to dirty camera lenses and thin occluders. ACM Trans. Graph. (TOG) 28, 144 (2009)CrossRefGoogle Scholar
  22. 22.
    Sand, P., Teller, S.: Particle video: Long-range motion estimation using point trajectories. Int. J. Comput. Vis. (IJCV) 80, 72–91 (2008)CrossRefGoogle Scholar
  23. 23.
    Sundaram, N., Brox, T., Keutzer, K.: Dense point trajectories by GPU-accelerated large displacement optical flow. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010, Part I. LNCS, vol. 6311, pp. 438–451. Springer, Heidelberg (2010) CrossRefGoogle Scholar
  24. 24.
    Brox, T., Malik, J.: Large displacement optical flow: descriptor matching in variational motion estimation. IEEE Trans. Pattern Anal. Mach. Intell. (TPAMI) 33, 500–513 (2011)CrossRefGoogle Scholar
  25. 25.
    Volz, S., Bruhn, A., Valgaerts, L., Zimmer, H.: Modeling temporal coherence for optical flow. In: 2011 IEEE International Conference on Computer Vision (ICCV), pp. 1116–1123. IEEE (2011)Google Scholar
  26. 26.
    Rubinstein, M., Liu, C., Freeman, W.T.: Towards longer long-range motion trajectories. In: British Machine Vision Conference (BMVC), pp. 1–11 (2012)Google Scholar
  27. 27.
    Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60, 91–110 (2004)CrossRefGoogle Scholar
  28. 28.
    Fulkerson, B., Vedaldi, A., Soatto, S.: Class segmentation and object localization with superpixel neighborhoods. In: IEEE International Conference on Computer Vision (ICCV) (2009)Google Scholar
  29. 29.
    Boykov, Y., Kolmogorov, V.: An experimental comparison of min-cut/max-flow algorithms for energy minimization in vision. IEEE Trans. Pattern Anal. Mach. Intell. (TPAMI)s 26, 1124–1137 (2004)Google Scholar
  30. 30.
    Boykov, Y., Veksler, O., Zabih, R.: Efficient approximate energy minimization via graph cuts. IEEE Trans. Pattern Anal. Mach. Intell. (TPAMI) 20, 1222–1239 (2001)CrossRefGoogle Scholar
  31. 31.
    Kolmogorov, V., Zabih, R.: What energy functions can be minimized via graph cuts? IEEE Trans. Pattern Anal. Mach. Intell. (TPAMI) 26, 147–159 (2004)CrossRefGoogle Scholar
  32. 32.
    Geiger, A., Lenz, P., Urtasun, R.: Are we ready for autonomous driving? the kitti vision benchmark suite. In: Conference on Computer Vision and Pattern Recognition (CVPR) (2012)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  • Shaodi You
    • 1
    Email author
  • Robby T. Tan
    • 2
  • Rei Kawakami
    • 1
  • Yasuhiro Mukaigawa
    • 3
  • Katsushi Ikeuchi
    • 1
  1. 1.The University of TokyoTokyoJapan
  2. 2.SIM UniversitySingaporeSingapore
  3. 3.Nara Institute of Science and TechnologyNaraJapan

Personalised recommendations