Abstract
Image dehazing is a challenging task of restoring a clear image from a haze-polluted image. However, most popular dehazing methods have color shift and overexposure problems owing to the inaccurate estimation of the transmission and atmospheric light in the atmospheric scattering model. In view of the existing problems, this paper proposes a simple but effective dehazing network by learning the residual image between the hazy image and haze-free image. A novel transitional convolution structure is constructed in this network, which contributes to utilizing shallow structure information to enhance the final residual image. Furthermore, to provide images with more realistic color information after dehazing, a novel color difference loss based on CIEDE2000 is designed as one term of the total loss function. In addition, for some real-world images that affect practical applications, a brightness enhancement module is also introduced to restore the luminance of the images. Experiments on synthetic datasets and real-world images demonstrate that the proposed method has clear advantages in both subjective and objective evaluation indicators compared to several existing advanced algorithms.
Similar content being viewed by others
References
Al-Sammaraie, M. F. (2015). Contrast enhancement of roads images with foggy scenes based on histogram equalization. In Proceedings of the 10th international conference on Computer Science & Education, Cambridge, UK (pp. 95–101).
Ancuti, C. O., & Ancuti, C. (2013). Single image dehazing by multi-scale fusion. IEEE Transactions on Image Processing, 22(8), 3271–3282.
Berman, D., Treibitz, T., & Avidan, S. (2016). Non-local image dehazing. In Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR) (pp. 1674–1682).
Busch, C., & Debes, E. (1998). Wavelet transform for analyzing fog visibility. IEEE Intelligent Systems, 13(6), 66–71.
Cai, B., Xu, X., Jia, K., Qing, C., & Tao, D. (2016). DehazeNet: An end-to-end system for single image haze removal. IEEE Transactions on Image Processing, 25(11), 5187–5198.
Chen, D., He, M., Fan, Q., Liao, J., Zhang, L., Hou, D., Yuan, L., & Hua, G. (2019). Gated context aggregation network for image dehazing and deraining. In 2019 IEEE winter conference on applications of computer vision (WACV), Waikoloa Village, HI, USA (pp. 1375–1383).
Dong, C., Loy, C. C., He, K., & Tang, X. (2016). Image super-resolution using deep convolutional networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 38(2), 295–307.
Fang, F., Wang, T., Wang, Y., Zeng, T., & Zhang, G. (2019). Variational single image dehazing for enhanced visualization. IEEE Transactions on Multimedia (earlyaccess). https://doi.org/10.1109/TMM.2019.2958755.
Fattal, R. (2008). Single image dehazing. ACM Transactions on Graphics, 27(3), 72.
Fu, X., Huang, J., Ding, X., Liao, Y., & Paisley, J. (2017). Clearing the skies: A deep network architecture for single-image rain removal. IEEE Transactions on Image Processing, 26(6), 2944–2956.
Galdran, A. (2018). Image dehazing by artificial multiple-exposure image fusion. Signal Processing, 149, 135–147.
Girshick, R., Donahue, J., Darrell, T., & Malik, J. (2014). Rich feature hierarchies for accurate object detection and semantic segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR) (pp. 580–587).
Gu, Y., Yang, X., & Gao, Y. (2019). A novel total generalized variation model for image dehazing. Journal of Mathematical Imaging and Vision, 61, 1329–1341.
He, K., Sun, J., & Tang, X. (2011). Single image haze removal using dark channel prior. IEEE Transactions on Pattern Analysis and Machine Intelligence, 33(12), 2341–2353.
He, K., Zhang, X., Ren, S., & Sun, J. (2016). Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR) (pp. 770–778).
Hu, W.-W., Wang, R.-G., Fang, S., & Hu, Q. (2010). Retinex algorithm for image enhancement based on bilateral filtering. Journal of Engineering Graphics, 31(2), 104–109.
Huynh-Thu, Q., & Ghanbari, M. (2008). Scope of validity of PSNR in image/video quality assessment. Electronics Letters, 44(13), 800–801.
Hyunchan, A., Keum, B., Kim, D., & Lee, H. (2013). Adaptive local tone mapping based on retinex for high dynamic range images. In International conference on consumer electronics (pp. 153–156).
Jia, J., & Yue, H. (2014). A wavelet-based approach to improve foggy image clarity. In Proceedings of IFAC world congress (pp. 930–935).
Jobson, D. J., Rahman, Z.-U., & Woodell, G. A. (1997). A multiscale retinex for bridging the gap between color images and the human observation of scenes. IEEE Transactions on Image Processing, 6(7), 965–976.
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2012). ImageNet classification with deep convolutional neural networks. In Proceedings of the advances in neural information processing systems (pp. 1097–1105).
Li, B., Peng, X., Wang, Z., Xu, J., & Feng, D. (2017). Aod-net: All-in-one dehazing network. In Proceedings of the IEEE international conference on computer vision (ICCV) (pp. 4780–4788).
Li, B., Ren, W., Fu, D., Tao, D., Feng, D., Zeng, W., et al. (2019). Benchmarking single-image dehazing and beyond. IEEE Transactions on Image Processing, 28(1), 492–505.
Liu, S., Liu, T., Gao, L., Li, H., Hu, Q., Zhao, J., et al. (2019a). Convolutional neural network and guided filtering for SAR image denoising. Remote Sensing, 11(6), 702–720.
Liu, Z., Xiao, B., Alrabeiah, M., Wang, K., & Chen, J. (2019b). Single image dehazing with a generic model-agnostic convolutional neural network. IEEE Signal Processing Letters, 26(6), 833–837.
Long, J., Shelhamer, E., & Darrell, T. (2015). Fully convolutional networks for semantic segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR) (pp. 3431–3440).
Meng, G., Wang, Y., Duan, J., Xiang, S., & Pan, C. (2013). Efficient image dehazing with boundary constraint and contextual regularization. In Proceedings of the IEEE international conference on computer vision (ICCV) (pp. 617–624).
Reinhard, E., Stark, M., Shirley, P., & Ferwerda, J. (2002). Photographic tone reproduction for digital images. ACM Transactions on Graphics, 21(3), 267–276.
Ren, S., He, K., Girshick, R., & Sun, J. (2017). Faster R-CNN: Towards real-time object detection with region proposal networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 39(6), 1137–1149.
Ren, W., Liu, S., Zhang, H., Pan, J., & Cao, X. (2016). Single image dehazing via multi-scale convolutional neural networks. In Proceedings of the European conference on computer vision (ECCV) (pp. 154–169).
Rong, Z., & Jun, W. L. (2014). Improved wavelet transform algorithm for single image dehazing. Optik, 125(13), 3064–3066.
Seow, M. J., & Asari, V. K. (2006). Ratio rule and homomorphic filter for enhancement of digital colour image. Neurocomputing, 69(7), 954–958.
Sharma, G., Wu, W., & Dalal, E. N. (2005). The CIEDE2000 color-difference formula: Implementation notes, supplementary test data, and mathematical observations. Color Research and Applications, 30(1), 21–30.
Sharma, R., & Chopra, V. (2014). A review on different image dehazing methods. International Journal of Computer Engineering and Applications, 6, 77–87.
Shu, Q., Wu, C., Xiao, Z., & Liu, R. W. (2019a). Variational regularized transmission refinement for image dehazing. In 2019 IEEE international conference on image processing (ICIP), Taipei, Taiwan (pp. 2781–2785).
Shu, Q., Wu, C., Zhong, Q., & Liu, R. (2019b). Alternating minimization algorithm for hybrid regularized variational image dehazing. Optik, 185, 943–956.
Sun, X., Liu, L., Li, Q., Dong, J., Lima, E., & Yin, R. (2019). Deep pixel-to-pixel network for underwater image enhancement and restoration. IET Image Processing, 13(3), 469–474.
Wang, A., Wang, W., Liu, J., & Gu, N. (2019a). AIPNet: Image-to-image single image dehazing with atmospheric illumination prior. IEEE Transactions on Image Processing, 28(1), 381–393.
Wang, R., Zhang, Q., Fu, C., Shen, X., Zheng, W., & Jia, J. (2019b). Underexposed photo enhancement using deep illumination estimation. In Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), USA (pp. 6849–6857).
Wang, W., & Yuan, X. (2017). Recent advances in image dehazing. IEEE/CAA Journal of Automatica Sinica, 4(3), 410–436.
Wang, Z., Bovik, A. C., Sheikh, H. R., & Simoncelli, E. P. (2004). Image quality assessment: From error visibility to structural similarity. IEEE Transactions on Image Processing, 13(4), 600–612.
Wen, Y., Sheng, B., Li, P., Lin, W., & Feng, D. D. (2019). Deep color guided coarse-to-fine convolutional network cascade for depth image super-resolution. IEEE Transactions on Image Processing, 28(2), 994–1006.
Xu, Z., Liu, X., & Ji, N. (2009). Fog removal from color images using contrast limited adaptive histogram equalization. In Proceedings of the 2nd international congress on image and signal processing (CISP) (pp. 1–5).
Yadav, G., Maheshwari, S., & Agarwal, A. (2014). Foggy image enhancement using contrast limited adaptive histogram equalization of digitally filtered image: Performance improvement. In Proceedings of the 2014 international conference on advances in computing, communications and informatics, New Delhi, India (pp. 2225–2231).
Yang, W., Wang, R., Fang, S., & Zhang, X. (2010). Variable filter Retinex algorithm for foggy image enhancement. Journal of Computer-Aided Design & Computer Graphics, 22(6), 965–971.
Yang, W., Zhang, X., Tian, Y., Wang, W., Xue, J., & Liao, Q. (2019). Deep learning for single image super-resolution: A brief review. IEEE Transactions on Multimedia, 21(12), 3106–3121.
Zhang, H., & Patel, V. M. (2018). Densely connected pyramid dehazing network. In Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR) (pp. 3194–3203).
Zhang, L., Zhang, L., & Bovik, A. C. (2015). A feature enriched completely blind image quality evaluator. IEEE Transactions on Image Processing, 24(8), 2579–2591.
Zhao, H., Gallo, O., Frosio, I., & Kautz, J. (2017). Loss functions for image restoration with neural networks. IEEE Transactions on Computational Imaging, 3(1), 47–57.
Zhao, S., Zhang, L., Huang, S., Shen, Y., Zhao, S., & Yang, Y. (2019). Evaluation of defogging: A real-world benchmark dataset, a new criterion and baselines. In 2019 IEEE international conference on multimedia and expo (ICME), Shanghai, China (pp. 1840–1845).
Acknowledgements
This work is supported by the National Natural Science Foundation of China (Nos. 61862030 and 61662026), by the Natural Science Foundation of Jiangxi Province (Nos. 20182BCB22006, 20181BAB202010, 20192ACB20002, and 20192ACBL21008), and by the Project of the Education Department of Jiangxi Province (Nos. GJJ170312 and GJJ170318).
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no conflict of interest.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Huang, S., Li, H., Yang, Y. et al. An end-to-end dehazing network with transitional convolution layer. Multidim Syst Sign Process 31, 1603–1623 (2020). https://doi.org/10.1007/s11045-020-00723-2
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11045-020-00723-2