Abstract
The task of recalibrating the illumination settings in an image to a target configuration is known as relighting. Relighting techniques have potential applications in digital photography, gaming industry and in augmented reality. In this paper, we address the one-to-one relighting problem where an image at a target illumination settings is predicted given an input image with specific illumination conditions. To this end, we propose a wavelet decomposed RelightNet called WDRN which is a novel encoder-decoder network employing wavelet based decomposition followed by convolution layers under a muti-resolution framework. We also propose a novel loss function called gray loss that ensures efficient learning of gradient in illumination along different directions of the ground truth image giving rise to visually superior relit images. The proposed solution won the first position in the relighting challenge event in advances in image manipulation (AIM) 2020 workshop which proves its effectiveness measured in terms of a Mean Perceptual Score which in turn is measured using SSIM and a Learned Perceptual Image Patch Similarity score.
D. Puthussery and P.S. Hrishikesh—Equal contribution.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Bychkovsky, V., Paris, S., Chan, E., Durand, F.: Learning photographic global tonal adjustment with a database of input/output image pairs. In: CVPR 2011, pp. 97–104 (2011)
Chen, Y., Wang, Y., Kao, M., Chuang, Y.: Deep photo enhancer: unpaired learning for image enhancement from photographs with GANs. In: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 6306–6314 (2018)
Debevec, P., Hawkins, T., Tchou, C., Duiker, H.P., Sarokin, W., Sagar, M.: Acquiring the reflectance field of a human face. In: Proceedings of the 27th Annual Conference on Computer Graphics and Interactive Techniques, SIGGRAPH 2000, pp. 145–156. ACM Press/Addison-Wesley Publishing Co., USA (2000). https://doi.org/10.1145/344779.344855
Dherse, A.P., Everaert, M.N., Gwizdała, J.J.: Scene relighting with illumination estimation in the latent space on an encoder-decoder scheme (2020)
El Helou, M., Zhou, R., Barthas, J., Süsstrunk, S.: VIDIT: virtual image dataset for illumination transfer. arXiv preprint arXiv:2005.05460 (2020)
El Helou, M., Zhou, R., Süsstrunk, S., Timofte, R., et al.: AIM 2020: Scene relighting and illumination estimation challenge. In: Bartoli, A., Fusiello, A. (eds.) ECCV 2020 Workshops. LNCS, vol. 12537, pp. 499–518 (2020)
Guo, X., Li, Y., Ling, H.: LIME: low-light image enhancement via illumination map estimation. IEEE Trans. Image Process. 26(2), 982–993 (2017)
Ignatov, A., Kobyshev, N., Timofte, R., Vanhoey, K., Gool, L.V.: WESPE: weakly supervised photo enhancer for digital cameras. CoRR abs/1709.01118 (2017). http://arxiv.org/abs/1709.01118
Ignatov, A., Kobyshev, N., Vanhoey, K., Timofte, R., Gool, L.V.: DSLR-quality photos on mobile devices with deep convolutional networks. CoRR abs/1704.02470 (2017). http://arxiv.org/abs/1704.02470
Land, E.H.: The retinex. Am. Sci. 52(2), 247–264 (1964)
Liu, P., Zhang, H., Zhang, K., Lin, L., Zuo, W.: Multi-level wavelet-CNN for image restoration. CoRR abs/1805.07071 (2018). http://arxiv.org/abs/1805.07071
Lore, K.G., Akintayo, A., Sarkar, S.: LLNet: a deep autoencoder approach to natural low-light image enhancement. CoRR abs/1511.03995 (2015). http://arxiv.org/abs/1511.03995
Matusik, W., Loper, M., Pfister, H.: Progressively-refined reflectance functions from natural illumination. In: Rendering Techniques (2004)
Reddy, D., Ramamoorthi, R., Curless, B.: Frequency-space decomposition and acquisition of light transport under spatially varying illumination. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012. LNCS, vol. 7577, pp. 596–610. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-33783-3_43
Sun, T., et al.: Single image portrait relighting. CoRR abs/1905.00824 (2019). http://arxiv.org/abs/1905.00824
Wang, R., Zhang, Q., Fu, C., Shen, X., Zheng, W., Jia, J.: Underexposed photo enhancement using deep illumination estimation. In: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 6842–6850 (2019)
Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: from error visibility to structural similarity. IEEE TIP 13(4), 600–612 (2004)
Xu, Z., Sunkavalli, K., Hadap, S., Ramamoorthi, R.: Deep image-based relighting from optimal sparse samples. ACM Trans. Graph. 37(4) (2018). https://doi.org/10.1145/3197517.3201313
Ying, Z., Li, G., Ren, Y., Wang, R., Wang, W.: A new low-light image enhancement algorithm using camera response model. In: 2017 IEEE International Conference on Computer Vision Workshops (ICCVW), pp. 3015–3022 (2017)
Yuan, S., Timofte, R., Leonardis, A., Slabaugh, G.: NTIRE 2020 challenge on image demoireing: methods and results. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, June 2020
Zhang, R., Isola, P., Efros, A.A., Shechtman, E., Wang, O.: The unreasonable effectiveness of deep features as a perceptual metric. In: CVPR, pp. 586–595 (2018)
Acknowledgements
We gratefully acknowledge the support of NVIDIA PSG Cluster and Trivandrum Engineering Science and Technology Research Park (TrEST) in providing the computational resource to conduct this research.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Puthussery, D., Panikkasseril Sethumadhavan, H., Kuriakose, M., Charangatt Victor, J. (2020). WDRN: A Wavelet Decomposed RelightNet for Image Relighting. In: Bartoli, A., Fusiello, A. (eds) Computer Vision – ECCV 2020 Workshops. ECCV 2020. Lecture Notes in Computer Science(), vol 12537. Springer, Cham. https://doi.org/10.1007/978-3-030-67070-2_31
Download citation
DOI: https://doi.org/10.1007/978-3-030-67070-2_31
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-67069-6
Online ISBN: 978-3-030-67070-2
eBook Packages: Computer ScienceComputer Science (R0)