Single Image Dehazing via Multi-scale Convolutional Neural Networks

  • Wenqi Ren
  • Si Liu
  • Hua Zhang
  • Jinshan Pan
  • Xiaochun Cao
  • Ming-Hsuan Yang
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9906)

Abstract

The performance of existing image dehazing methods is limited by hand-designed features, such as the dark channel, color disparity and maximum contrast, with complex fusion schemes. In this paper, we propose a multi-scale deep neural network for single-image dehazing by learning the mapping between hazy images and their corresponding transmission maps. The proposed algorithm consists of a coarse-scale net which predicts a holistic transmission map based on the entire image, and a fine-scale net which refines results locally. To train the multi-scale deep network, we synthesize a dataset comprised of hazy images and corresponding transmission maps based on the NYU Depth dataset. Extensive experiments demonstrate that the proposed algorithm performs favorably against the state-of-the-art methods on both synthetic and real-world images in terms of quality and speed.

Keywords

Image dehazing Defogging Convolutional neural network 

Notes

Acknowledgements

This work is supported by National High-tech R&D Program of China (2014BAK11B03), National Basic Research Program of China (2013CB329305), National Natural Science Foundation of China (No. 61422213), “Strategic Priority Research Program” of the Chinese Academy of Sciences (XDA06010701), and National Program for Support of Top-notch Young Professionals. W. Ren is supported by a scholarship from China Scholarship Council. M.-H. Yang is supported in part by the NSF CAREER grant #1149783, and gifts from Adobe and Nvidia.

Supplementary material

419974_1_En_10_MOESM1_ESM.pdf (30.2 mb)
Supplementary material 1 (pdf 30972 KB)

References

  1. 1.
    He, K., Sun, J., Tang, X.: Single image haze removal using dark channel prior. TPAMI 33(12), 2341–2353 (2011)CrossRefGoogle Scholar
  2. 2.
    Tang, K., Yang, J., Wang, J.: Investigating haze-relevant features in a learning framework for image dehazing. In: CVPR (2014)Google Scholar
  3. 3.
    Tan, R.T., Pettersson, N., Petersson, L.: Visibility enhancement for roads with foggy or hazy scenes. In: Intelligent Vehicles Symposium (2007)Google Scholar
  4. 4.
    Hautière, N., Tarel, J.P., Aubert, D.: Towards fog-free in-vehicle vision systems through contrast restoration. In: CVPR (2007)Google Scholar
  5. 5.
    Caraffa, L., Tarel, J.P.: Markov random field model for single image defogging. In: Intelligent Vehicles Symposium (2013)Google Scholar
  6. 6.
    Fattal, R.: Dehazing using color-lines. TOG 34(1), 13 (2014)CrossRefGoogle Scholar
  7. 7.
    Li, Z., Tan, P., Tan, R.T., Zou, D., Zhou, S.Z., Cheong, L.F.: Simultaneous video defogging and stereo reconstruction. In: CVPR (2015)Google Scholar
  8. 8.
    Pei, S.C., Lee, T.Y.: Nighttime haze removal using color transfer pre-processing and dark channel prior. In: ICIP (2012)Google Scholar
  9. 9.
    Ancuti, C.O., Ancuti, C.: Single image dehazing by multi-scale fusion. TIP 22(8), 3271–3282 (2013)Google Scholar
  10. 10.
    Tan, R.T.: Visibility in bad weather from a single image. In: CVPR (2008)Google Scholar
  11. 11.
    Zhu, Q., Mai, J., Shao, L.: A fast single image haze removal algorithm using color attenuation prior. TIP 24(11), 3522–3533 (2015)MathSciNetGoogle Scholar
  12. 12.
    Eigen, D., Puhrsch, C., Fergus, R.: Depth map prediction from a single image using a multi-scale deep network. In: NIPS (2014)Google Scholar
  13. 13.
    Liu, S., Liang, X., Liu, L., Shen, X., Yang, J., Xu, C., Lin, L., Cao, X., Yan, S.: Matching-CNN meets KNN: quasi-parametric human parsing. In: CVPR, pp. 1419–1427 (2015)Google Scholar
  14. 14.
    Liang, X., Liu, S., Shen, X., Yang, J., Liu, L., Dong, J., Lin, L., Yan, S.: Deep human parsing with active template regression. PAMI 37(12), 2402–2414 (2015)CrossRefGoogle Scholar
  15. 15.
    Zhang, H., Liu, S., Zhang, C., Ren, W., Wang, R., Cao, X.: SketchNet: sketch classification with web images. In: CVPR (2016)Google Scholar
  16. 16.
    Silberman, N., Hoiem, D., Kohli, P., Fergus, R.: Indoor segmentation and support inference from RGBD images. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012. LNCS, vol. 7576, pp. 746–760. Springer, Heidelberg (2012). doi: 10.1007/978-3-642-33715-4_54 Google Scholar
  17. 17.
    Kopf, J., Neubert, B., Chen, B., Cohen, M., Cohen-Or, D., Deussen, O., Uyttendaele, M., Lischinski, D.: Deep photo: model-based photograph enhancement and viewing. In: SIGGRAPH Asia (2008)Google Scholar
  18. 18.
    Treibitz, T., Schechner, Y.Y.: Polarization: beneficial for visibility enhancement? In: CVPR (2009)Google Scholar
  19. 19.
    Narasimhan, S.G., Nayar, S.K.: Contrast restoration of weather degraded images. TPAMI 25(6), 713–724 (2003)CrossRefGoogle Scholar
  20. 20.
    Narasimhan, S.G., Nayar, S.K.: Chromatic framework for vision in bad weather. In: CVPR (2000)Google Scholar
  21. 21.
    Schechner, Y.Y., Narasimhan, S.G., Nayar, S.K.: Instant dehazing of images using polarization. In: CVPR (2001)Google Scholar
  22. 22.
    Shwartz, S., Namer, E., Schechner, Y.Y.: Blind haze separation. In: CVPR (2006)Google Scholar
  23. 23.
    Fattal, R.: Single image dehazing. In: SIGGRAPH (2008)Google Scholar
  24. 24.
    Kratz, L., Nishino, K.: Factorizing scene albedo and depth from a single foggy image. In: ICCV (2009)Google Scholar
  25. 25.
    Tarel, J.P., Hautière, N., Caraffa, L., Cord, A., Halmaoui, H., Gruyer, D.: Vision enhancement in homogeneous and heterogeneous fog. Intell. Transp. Syst. Mag. 4(2), 6–20 (2012)CrossRefGoogle Scholar
  26. 26.
    Nishino, K., Kratz, L., Lombardi, S.: Bayesian defogging. IJCV 98(3), 263–278 (2012)MathSciNetCrossRefGoogle Scholar
  27. 27.
    Meng, G., Wang, Y., Duan, J., Xiang, S., Pan, C.: Efficient image dehazing with boundary constraint and contextual regularization. In: ICCV (2013)Google Scholar
  28. 28.
    Tarel, J.P., Hautiere, N.: Fast visibility restoration from a single color or gray level image. In: ICCV (2009)Google Scholar
  29. 29.
    Gibson, K.B., Vo, D.T., Nguyen, T.Q.: An investigation of dehazing effects on image and video coding. TIP 21(2), 662–673 (2012)MathSciNetGoogle Scholar
  30. 30.
    He, K., Sun, J., Tang, X.: Guided image filtering. TPAMI 35(6), 1397–1409 (2013)CrossRefGoogle Scholar
  31. 31.
    Yuan, J., Ni, B., Kassim, A.A.: Half-CNN: a general framework for whole-image regression (2014). arXiv preprint: arXiv:1412.6885
  32. 32.
    LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proc. IEEE 86(11), 2278–2324 (1998)CrossRefGoogle Scholar
  33. 33.
    Khan, S.H., Bennamoun, M., Sohel, F., Togneri, R.: Automatic feature learning for robust shadow detection. In: CVPR (2014)Google Scholar
  34. 34.
    Scharstein, D., Szeliski, R.: A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. IJCV 47(1–3), 7–42 (2002)CrossRefMATHGoogle Scholar
  35. 35.
    Scharstein, D., Szeliski, R.: High-accuracy stereo depth maps using structured light. In: CVPR (2003)Google Scholar
  36. 36.
    Hirschmüller, H., Scharstein, D.: Evaluation of cost functions for stereo matching. In: CVPR (2007)Google Scholar
  37. 37.
    Dong, C., Loy, C.C., He, K., Tang, X.: Learning a deep convolutional network for image super-resolution. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8692, pp. 184–199. Springer, Heidelberg (2014). doi: 10.1007/978-3-319-10593-2_13 Google Scholar
  38. 38.
    Ancuti, C.O., Ancuti, C., Hermans, C., Bekaert, P.: A fast semi-inverse approach to detect and remove the haze from a single image. In: Kimmel, R., Klette, R., Sugimoto, A. (eds.) ACCV 2010. LNCS, vol. 6493, pp. 501–514. Springer, Heidelberg (2011). doi: 10.1007/978-3-642-19309-5_39 CrossRefGoogle Scholar
  39. 39.
    Li, Y., Tan, R.T., Brown, M.S.: Nighttime haze removal with glow and multiple light colors. In: ICCV (2015)Google Scholar
  40. 40.
    Zhang, J., Cao, Y., Wang, Z.: Nighttime haze removal based on a new imaging model. In: ICIP (2014)Google Scholar

Copyright information

© Springer International Publishing AG 2016

Authors and Affiliations

  • Wenqi Ren
    • 1
    • 3
  • Si Liu
    • 2
  • Hua Zhang
    • 2
  • Jinshan Pan
    • 3
  • Xiaochun Cao
    • 1
  • Ming-Hsuan Yang
    • 3
  1. 1.Tianjin UniversityTianjinChina
  2. 2.IIE, CASBeijingChina
  3. 3.University of California, MercedMercedUSA

Personalised recommendations