Advertisement

URIE: Universal Image Enhancement for Visual Recognition in the Wild

Conference paper
  • 824 Downloads
Part of the Lecture Notes in Computer Science book series (LNCS, volume 12354)

Abstract

Despite the great advances in visual recognition, it has been witnessed that recognition models trained on clean images of common datasets are not robust against distorted images in the real world. To tackle this issue, we present a Universal and Recognition-friendly Image Enhancement network, dubbed URIE, which is attached in front of existing recognition models and enhances distorted input to improve their performance without retraining them. URIE is universal in that it aims to handle various factors of image degradation and to be incorporated with any arbitrary recognition models. Also, it is recognition-friendly since it is optimized to improve the robustness of following recognition models, instead of perceptual quality of output image. Our experiments demonstrate that URIE can handle various and latent image distortions and improve the performance of existing models for five diverse recognition tasks where input images are degraded.

Keywords

Visual recognition Image enhancement 

Notes

Acknowledgement

This work was supported by Samsung Research Funding & Incubation Center of Samsung Electronics under Project Number SRFC-IT1801-05.

Supplementary material

504446_1_En_43_MOESM1_ESM.pdf (2.3 mb)
Supplementary material 1 (pdf 2314 KB)

References

  1. 1.
    Chen, L.C., Zhu, Y., Papandreou, G., Schroff, F., Adam, H.: Encoder-decoder with atrous separable convolution for semantic image segmentation. In: Proceedings of European Conference on Computer Vision (ECCV) (2018)Google Scholar
  2. 2.
    Chen, Y., Li, W., Sakaridis, C., Dai, D., Van Gool, L.: Domain adaptive faster r-cnn for object detection in the wild. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2018)Google Scholar
  3. 3.
    Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Fei-Fei, L.: ImageNet: a large-scale hierarchical image database. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2009)Google Scholar
  4. 4.
    Diamond, S., Sitzmann, V., Boyd, S.P., Wetzstein, G., Heide, F.: Dirty pixels: Optimizing image classification architectures for raw sensor data. arXiv preprint arXiv:1701.06487 (2017)
  5. 5.
    Dong, C., Loy, C.C., He, K., Tang, X.: Image super-resolution using deep convolutional networks. IEEE Trans. Pattern Anal. Mach. Intell (TPAMI) 38(2), 295–307 (2016)CrossRefGoogle Scholar
  6. 6.
    Everingham, M., et al.: The pascal visual object classes (VOC) challenge. Int. J. Comput. Vis. 88, 303–338 (2010).  https://doi.org/10.1007/s11263-009-0275-4CrossRefGoogle Scholar
  7. 7.
    Geirhos, R., Temme, C.R.M., Rauber, J., Schütt, H.H., Bethge, M., Wichmann, F.A.: Generalisation in humans and deep neural networks. In: Proceedings of Neural Information Processing Systems (NeurIPS) (2018)Google Scholar
  8. 8.
    Gomez, R., Zhang, Z., González-Jiménez, J., Scaramuzza, D.: Learning-based image enhancement for visual odometry in challenging hdr environments. In: Proceedings of International Conference on Robatics and Automation (ICRA) (2018)Google Scholar
  9. 9.
    Gopalan, R., Taheri, S., Turaga, P., Chellappa, R.: A blur-robust descriptor with applications to face recognition. IEEE Trans. Pattern Anal. Mach. Intell. (TPAMI) 34(6), 1220–1226 (2012)CrossRefGoogle Scholar
  10. 10.
    He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2016)Google Scholar
  11. 11.
    Hendrycks, D., Dietterich, T.: Benchmarking neural network robustness to common corruptions and perturbations. In: Proceedings of International Conference on Learning Representations (ICLR) (2019)Google Scholar
  12. 12.
    Ioffe, S., Szegedy, C.: Batch normalization: Accelerating deep network training by reducing internal covariate shift. In: Proceedings of International Conference on Machine Learning (ICML) (2015)Google Scholar
  13. 13.
    Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. In: Proceedings of International Conference on Learning Representations (ICLR) (2015)Google Scholar
  14. 14.
    Lee, S., et al.: VPGNet: vanishing point guided network for lane and road marking detection and recognition. In: Proceedings of IEEE International Conference on Computer Vision (ICCV) (2017)Google Scholar
  15. 15.
    Li, B., Peng, X., Wang, Z., Xu, J., Feng, D.: Aod-net: all-in-one dehazing network. In: Proceedings of IEEE International Conference on Computer Vision (ICCV) (2017)Google Scholar
  16. 16.
    Li, S., et al.: Single image deraining: a comprehensive benchmark analysis. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2019)Google Scholar
  17. 17.
    Li, X., Wang, W., Hu, X., Yang, J.: Selective kernel networks. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2019) 9Google Scholar
  18. 18.
    Liu, D., Wen, B., Fan, Y., Loy, C.C., Huang, T.S.: Non-local recurrent network for image restoration. In: Proceedings of Neural Information Processing Systems (NeurIPS) (2018)Google Scholar
  19. 19.
    Liu, D., Wen, B., Liu, X., Wang, Z., Huang, T.S.: When image denoising meets high-level vision tasks: a deep learning approach. In: Proceedings of International Joint Conference on Artificial Intelligence (IJCAI) (2018)Google Scholar
  20. 20.
    Liu, W., et al.: SSD: Single shot multibox detector. In: Proceedings of European Conference on Computer Vision (ECCV) (2016)Google Scholar
  21. 21.
    Nah, S., Hyun Kim, T., Mu Lee, K.: Deep multi-scale convolutional neural network for dynamic scene deblurring. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2017)Google Scholar
  22. 22.
    Noh, H., Hong, S., Han, B.: Learning deconvolution network for semantic segmentation. In: Proceedings of IEEE International Conference on Computer Vision (ICCV) (2015)Google Scholar
  23. 23.
    Oquab, M., Bottou, L., Laptev, I., Sivic, J.: Learning and transferring mid-level image representations using convolutional neural networks. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2014)Google Scholar
  24. 24.
    Pei, Y., Huang, Y., Zou, Q., Lu, Y., Wang, S.: Does haze removal help cnn-based image classification? In: Proceedings of European Conference on Computer Vision (ECCV) (2018)Google Scholar
  25. 25.
    Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional networks for biomedical image segmentation. In: Proceedings of Medical Image Computing and Computer-Assisted Intervention (MICCAI) (2015)Google Scholar
  26. 26.
    Sakaridis, C., Dai, D., Gool, L.V.: Guided curriculum model adaptation and uncertainty-aware evaluation for semantic nighttime image segmentation. In: Proceedings of IEEE International Conference on Computer Vision (ICCV) (2019)Google Scholar
  27. 27.
    Sakaridis, C., Dai, D., Hecker, S., Van Gool, L.: Model adaptation with synthetic and real data for semantic dense foggy scene understanding. In: Proceedings of European Conference on Computer Vision (ECCV) (2018)Google Scholar
  28. 28.
    Selvaraju, R.R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., Batra, D.: Grad-cam: Visual explanations from deep networks via gradient-based localization. In: Proceedings of IEEE International Conference on Computer Vision (ICCV) (2017)Google Scholar
  29. 29.
    Sharma, V., Diba, A., Neven, D., Brown, M.S., Van Gool, L., Stiefelhagen, R.: Classification-driven dynamic image enhancement. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2018)Google Scholar
  30. 30.
    Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. In: Proceedings of International Conference on Learning Representations (ICLR) (2015)Google Scholar
  31. 31.
    Singh, M., Nagpal, S., Singh, R., Vatsa, M.: Dual directed capsule network for very low resolution image recognition. In: Proceedings of IEEE International Conference on Computer Vision (ICCV) (2019)Google Scholar
  32. 32.
    Suganuma, M., Liu, X., Okatani, T.: Attention-based adaptive selection of operations for image restoration in the presence of unknown combined distortions. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2019)Google Scholar
  33. 33.
    Tai, Y., Yang, J., Liu, X., Xu, C.: Memnet: A persistent memory network for image restoration. In: Proceedings of International Conference on Computer Vision (ICCV). pp. 4539–4547 (2017)Google Scholar
  34. 34.
    Ulyanov, D., Vedaldi, A., Lempitsky, V.: Instance normalization: The missing ingredient for fast stylization. arXiv preprint arXiv:1607.08022 (2016) 5
  35. 35.
    Vidal, R.G., Banerjee, S., Grm, K., Struc, V., Scheirer, W.J.: Ug\(^{2}\): a video benchmark for assessing the impact of image restoration and enhancement on automatic visual recognition. In: Proceedings of IEEE Winter Conference on Applications of Computer Vision (WACV) (2018)Google Scholar
  36. 36.
    Wang, Z., Chang, S., Yang, Y., Liu, D., Huang, T.S.: Studying very low resolution recognition using deep networks. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2016)Google Scholar
  37. 37.
    Welinder, P., et al.: Caltech-UCSD Birds 200. Technical Report CNS-TR-2010-001, California Institute of Technology (2010)Google Scholar
  38. 38.
    Wu, Y., Ling, H., Yu, J., Li, F., Mei, X., Cheng, E.: Blurred target tracking by blur-driven tracker. In: Proceedings of IEEE International Conference on Computer Vision (ICCV) (2011)Google Scholar
  39. 39.
    Wu, Z., Suresh, K., Narayanan, P., Xu, H., Kwon, H., Wang, Z.: Delving into robust object detection from unmanned aerial vehicles: a deep nuisance disentanglement approach. In: Proceedings of IEEE International Conference on Computer Vision (ICCV) (2019)Google Scholar
  40. 40.
    Yasarla, R., Patel, V.M.: Uncertainty guided multi-scale residual learning-using a cycle spinning cnn for single image de-raining. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2019)Google Scholar
  41. 41.
    Yu, K., Dong, C., Lin, L., Change Loy, C.: Crafting a toolchain for image restoration by deep reinforcement learning. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2018)Google Scholar
  42. 42.
    Zendel, O., Honauer, K., Murschitz, M., Steininger, D., Fernandez Dominguez, G.: Wilddash - creating hazard-aware benchmarks. In: Proceedings of European Conference on Computer Vision (ECCV) (2018)Google Scholar
  43. 43.
    Zhang, K., Zuo, W., Chen, Y., Meng, D., Zhang, L.: Beyond a gaussian denoiser: residual learning of deep cnn for image denoising. IEEE Trans. Image Process. (TIP) 26(7), 3142–3155 (2017)MathSciNetCrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  1. 1.Department of Computer Science and EngineeringPOSTECH, PohangKorea
  2. 2.Graduate School of Artificial Intelligence, POSTECHPohangKorea

Personalised recommendations