Abstract
Natural images can suffer from non-uniform haze distributions in different regions. However, this important fact is hardly considered in existing supervised dehazing methods, in which all training patches are accounted for equally in the loss design. These supervised methods may fail in making promising recoveries on some regions contaminated by heavy hazes. Therefore, for a more reasonable dehazing losses design, the varying importance of different training patches should be taken into account. Such rationale is exactly in line with the process of human learning that difficult concepts always require more practice in learning. To this end, we propose a bi-level dehazing (BILD) framework by designing an internal loop for weighted supervised dehazing and an external loop for training patch reweighting. With simple derivations, we show the gradients of BILD exhibit natural connections with policy gradient and can thus explain the BILD objective by the rewarding mechanism in reinforcement learning. The BILD is not a new dehazing method per se, it is better recognized as a flexible framework that can seamlessly work with general supervised dehazing approaches for their performance boosting.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Ancuti, C.O., Ancuti, C., Timofte, R.: NH-HAZE: an image dehazing benchmark with non-homogeneous hazy and haze-free images. In: CVPRW (2020)
Ancuti, C.O., Ancuti, C., Timofte, R., Vleeschouwer, C.D.: O-haze: a dehazing benchmark with real hazy and haze-free outdoor images. In: CVPRW (2018)
Ancuti, C.O., Ancuti, C., Vasluianu, F.A., Timofte, R., et al.: NTIRE 2020 challenge on nonhomogeneous dehazing. In: CVPRW (2020)
Baker, B., Gupta, O., Naik, N., Raskar, R.: Designing neural network architectures using reinforcement learning. arXiv preprint arXiv:1611.02167 (2016)
Berman, D., Treibitz, T., Avidan, S.: Air-light estimation using haze-lines. In: ICCP, pp. 1–9 (2017)
Berman, D., Treibitz, T., Avidan, S.: Single image dehazing using haze-lines. IEEE TPAMI 42(3), 720–734 (2018)
Bochkovskiy, A., Wang, C.Y., Liao, H.Y.M.: Yolov4: Optimal speed and accuracy of object detection. arXiv preprint arXiv:2004.10934 (2020)
Cai, B., Xu, X., Jia, K., Qing, C., Tao, D.: Dehazenet: An end-to-end system for single image haze removal. IEEE TIP 25(11), 5187–5198 (2016)
Chen, Y., Li, W., Sakaridis, C., Dai, D., Van Gool, L.: Domain adaptive faster r-cnn for object detection in the wild. In: CVPR, pp. 3339–3348 (2018)
Cubuk, E.D., Zoph, B., Mane, D., Vasudevan, V., Le, Q.V.: Autoaugment: Learning augmentation strategies from data. In: CVPR, pp. 113–123 (2019)
Deng, Z., et al.: Deep multi-model fusion for single-image dehazing. In: ICCV, pp. 2453–2462 (2019)
Dong, H., Pan, J., Xiang, L., Hu, Z., Zhang, X., Wang, F., Yang, M.H.: Multi-scale boosted dehazing network with dense feature fusion. In: CVPR, pp. 2157–2167 (2020)
Dong, J., Pan, J.: Physics-based feature dehazing networks. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12375, pp. 188–204. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58577-8_12
Franceschi, L., Niepert, M., Pontil, M., He, X.: Learning discrete structures for graph neural networks. In: ICML, pp. 1972–1982 (2019)
Gao, Y., Yang, H., Zhang, P., Zhou, C., Hu, Y.: Graphnas: Graph neural architecture search with reinforcement learning. arXiv preprint arXiv:1904.09981 (2019)
Grazzi, R., Franceschi, L., Pontil, M., Salzo, S.: On the iteration complexity of hypergradient computation. In: ICML, pp. 3748–3758 (2020)
Haarnoja, T., Tang, H., Abbeel, P., Levine, S.: Reinforcement learning with deep energy-based policies. In: ICML, pp. 1352–1361 (2017)
Haarnoja, T., Zhou, A., Abbeel, P., Levine, S.: Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor. In: ICML, pp. 1861–1870 (2018)
Haarnoja, T., et al.: Soft actor-critic algorithms and applications. arXiv preprint arXiv:1812.05905 (2018)
He, K., Sun, J., Tang, X.: Single image haze removal using dark channel prior. IEEE TPAMI 33(12), 2341–2353 (2010)
Hong, M., Xie, Y., Li, C., Qu, Y.: Distilling image dehazing with heterogeneous task imitation. In: CVPR, pp. 3462–3471 (2020)
Kohavi, R., et al.: A study of cross-validation and bootstrap for accuracy estimation and model selection. In: IJCAI, pp. 1137–1145 (1995)
Li, B., Peng, X., Wang, Z., Xu, J., Feng, D.: Aod-net: All-in-one dehazing network. In: ICCV, pp. 4770–4778 (2017)
Li, B., et al.: Benchmarking single-image dehazing and beyond. IEEE TIP 28(1), 492–505 (2019)
Liu, R., Liu, J., Jiang, Z., Fan, X., Luo, Z.: A bilevel integrated model with data-driven layer ensemble for multi-modality image fusion. IEEE TIP 30, 1261–1274 (2020)
Liu, X., Ma, Y., Shi, Z., Chen, J.: Griddehazenet: Attention-based multi-scale network for image dehazing. In: ICCV, pp. 7314–7323 (2019)
Liu, Z., Wei, P., Jiang, J., Cao, W., Bian, J., Chang, Y.: Mesa: Boost ensemble imbalanced learning with meta-sampler. In: NeurIPS, vol. 33 (2020)
Meng, G., Wang, Y., Duan, J., Xiang, S., Pan, C.: Efficient image dehazing with boundary constraint and contextual regularization. In: ICCV, pp. 617–624 (2013)
Mounsaveng, S., Laradji, I., Ben Ayed, I., Vázquez, D., Pedersoli, M.: Learning data augmentation with online bilevel optimization for image classification. In: WACV, pp. 1691–1700 (2021)
Narasimhan, S.G., Nayar, S.K.: Chromatic framework for vision in bad weather. In: CVPR, pp. 598–605 (2000)
Narasimhan, S.G., Nayar, S.K.: Vision and the atmosphere. IJCV 48(3), 233–254 (2002)
Qin, X., Wang, Z., Bai, Y., Xie, X., Jia, H.: Ffa-net: Feature fusion attention network for single image dehazing. In: AAAI, pp. 11908–11915 (2020)
Qu, Y., Chen, Y., Huang, J., Xie, Y.: Enhanced pix2pix dehazing network. In: CVPR, pp. 8160–8168 (2019)
Ren, W., Liu, S., Zhang, H., Pan, J., Cao, X., Yang, M.-H.: Single image dehazing via multi-scale convolutional neural networks. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9906, pp. 154–169. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46475-6_10
Ren, W., Ma, L., Zhang, J., Pan, J., Cao, X., Liu, W., Yang, M.H.: Gated fusion network for single image dehazing. In: CVPR, pp. 3253–3261 (2018)
Shao, Y., Li, L., Ren, W., Gao, C., Sang, N.: Domain adaptation for image dehazing. In: CVPR, pp. 2808–2817 (2020)
Shu, J., et al.: Meta-weight-net: Learning an explicit mapping for sample weighting. arXiv preprint arXiv:1902.07379 (2019)
Silver, D., Lever, G., Heess, N., Degris, T., Wierstra, D., Riedmiller, M.: Deterministic policy gradient algorithms. In: ICML, pp. 387–395 (2014)
Sun, M., Dou, H., Li, B., Yan, J., Ouyang, W., Cui, L.: Autosampling: Search for effective data sampling schedules. In: ICML, pp. 9923–9933 (2021)
Sutton, R.S., McAllester, D.A., Singh, S.P., Mansour, Y.: Policy gradient methods for reinforcement learning with function approximation. In: NeurIPS, pp. 1057–1063 (2000)
Tian, K., Lin, C., Sun, M., Zhou, L., Yan, J., Ouyang, W.: Improving auto-augment via augmentation-wise weight sharing. arXiv preprint arXiv:2009.14737 (2020)
Wang, X., Pham, H., Michel, P., Anastasopoulos, A., Carbonell, J., Neubig, G.: Optimizing data usage via differentiable rewards. In: ICML, pp. 9983–9995 (2020)
Wu, H., et al.: Contrastive learning for compact single image dehazing. In: CVPR, pp. 10551–10560 (2021)
Wu, X., et al.: Temporally correlated task scheduling for sequence learning. In: ICML, pp. 11274–11284 (2021)
Zhang, H., Patel, V.M.: Densely connected pyramid dehazing network. In: CVPR, pp. 3194–3203 (2018)
Zheng, Z., et al.: Ultra-high-definition image dehazing via multi-guided bilateral learning. In: CVPR, pp. 16180–16189 (2021)
Zhu, Q., Mai, J., Shao, L.: A fast single image haze removal algorithm using color attenuation prior. IEEE TIP 24(11), 3522–3533 (2015)
Acknowledgement
This research is supported by National Natural Science Foundation of China (Grant No.62031001, Grant No.61971020), National Key Research and Development Program of China (No.2020AAA0105502).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Jiang, X., Dou, H., Fu, C., Dai, B., Xu, T., Deng, Y. (2022). Boosting Supervised Dehazing Methods via Bi-level Patch Reweighting. In: Avidan, S., Brostow, G., Cissé, M., Farinella, G.M., Hassner, T. (eds) Computer Vision – ECCV 2022. ECCV 2022. Lecture Notes in Computer Science, vol 13678. Springer, Cham. https://doi.org/10.1007/978-3-031-19797-0_4
Download citation
DOI: https://doi.org/10.1007/978-3-031-19797-0_4
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-19796-3
Online ISBN: 978-3-031-19797-0
eBook Packages: Computer ScienceComputer Science (R0)