Advertisement

Springer Nature is making Coronavirus research free. View research | View latest news | Sign up for updates

Reflection removal on the windshield during nighttime driving

Abstract

The in-vehicle black box camera has become a popular device in many countries for security monitoring and event capture. However, the content of the video is often degraded by the reflection appear on the windshield. The reflection is produced by light and dust outside. We focus on this problem and attempt to improve the readability of video content by removing the undesired reflection layer from the nighttime onboard video. We firstly apply gradient prior and the characteristics of reflection to automatically detect the reflection regions; the reflections in these regions are then classified into the major and minor reflection. Next, we suppress the major reflections by reducing the gradient strength of the reflection layer. Based on the different distribution of gradient histograms between the background layer and the reflection layer, we construct the objective function to remove the major reflection residue and minor reflection. In the reflection removal process, we introduce luminance and chrominance fidelity terms into the objective function to maintain the brightness and chrominance of the background layer. Finally, we obtain a reflection removal result. Evaluation of removal experiments on nighttime onboard images shows that our proposed algorithm may successfully remove the reflection layer on the windshield in the nighttime onboard video.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

References

  1. 1.

    Simon, C., Park, I.K.: Reflection removal for in-vehicle black box videos. In: 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, USA (2015)

  2. 2.

    Huang, Z., Xiong, B., Cao, T., et al.: Dust and reflection removal from videos captured in moving car. In: 2016 IEEE 13th International Conference on e-Business Engineering (ICEBE), Macau, China (2016)

  3. 3.

    Cheong, J.Y., Simon, C., Kim, C.S., et al.: Reflection removal under fast forward camera motion. IEEE Trans. Image Process. 26(12), 6061–6073 (2017)

  4. 4.

    Fischler, M.A.: Recovering Intrinsic Scene Characteristics from Images, Menlo Park, CA. Artificial Intelligence Center, Rep. (1982)

  5. 5.

    Schechner, Y.Y., Kiryati, N., et al.: Separation of transparent layers using focus. Int. J. Comput. Vis. 39(1), 25–39 (2000)

  6. 6.

    Agrawal, A., Raskar, R., Nayar, S.K., Li, Y.: Removing photography artifacts using gradient projection and flash-exposure sampling. ACM Trans. Graph. (TOG) 24(3), 828–835 (2005)

  7. 7.

    Wang, Q., Lin, H., Ma, Y., et al.: Automatic Layer Separation using Light Field Imaging. https://arxiv.org/abs/1506.04721

  8. 8.

    Li, Y., Brown, M.S.: Exploiting reflection change for automatic reflection removal. In: 2013 IEEE International Conference on Computer Vision (ICCV), Sydney, NSW, Australia (2013)

  9. 9.

    Sirinukulwattana, T., Choe, G., Kweon, I.S.: Reflection removal using disparity and gradient-sparsity via smoothing algorithm. In: 2015 IEEE International Conference on Image Processing (ICIP), Quebec City, QC, Canada (2015)

  10. 10.

    Xue, T., Rubinstein, M., Liu, C., Freeman, W.T.: A computational approach for obstruction-free photography. ACM Trans. Graph. 34(4), 79:1–79:11 (2015)

  11. 11.

    Sun, C., Liu, S., Yang, T., Zeng, B., Wang, Z., Liu, G.: Automatic reflection removal using gradient intensity and motion cues. In: Proceedings of the ACM Multimedia Conference, Amsterdam, The Netherlands (2016)

  12. 12.

    Levin, A., Weiss, Y.: User assisted separation of reflections from a single image using a sparsity prior. IEEE Trans. Pattern Anal. Mach. Intell. 29(3), 1647–1655 (2007)

  13. 13.

    Yu, L., Brown, M.S.: Single image layer separation using relative smoothness. In: 2014 IEEE Conference on Computer Vision & Pattern Recognition (CVPR), Columbus, OH, USA (2014)

  14. 14.

    Arvanitopoulos, N., Achanta, R., Susstrunk, S.: Single image reflection suppression. In: 2017 IEEE Conference on Computer Vision & Pattern Recognition (CVPR), Honolulu, HI, USA (2017)

  15. 15.

    Wan, R., Shi, B., Duan, L.Y., et al.: Region-aware reflection removal with unified content and gradient priors. IEEE Trans. Image Process. 27(6), 2927–2941 (2018)

  16. 16.

    Wan, R., Shi, B., Hwee, T.A., et al.: Depth of field guided reflection removal. In: 2016 IEEE International Conference on Image Processing, Phoenix, AZ, USA (2016)

  17. 17.

    Yan, Q., Xu, Y., Yang, X., et al.: Separation of weak reflection from a single superimposed image. IEEE Signal Process. Lett. 21(10), 1173–1176 (2014)

  18. 18.

    Wan, R., Shi, B., Duan, L.Y., et al.: Benchmarking single-image reflection removal algorithms. In: 2017 IEEE International Conference on Computer Vision (ICCV), Venice, Italy (2017)

Download references

Author information

Correspondence to Chunming Tang.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Tang, C., Sun, R. & Chen, C. Reflection removal on the windshield during nighttime driving. SIViP (2020). https://doi.org/10.1007/s11760-020-01655-x

Download citation

Keywords

  • Gradient prior
  • Nighttime onboard video
  • Reflection removal
  • Major reflection suppression