Skip to main content
SpringerLink
Log in

Retinex Meets Transformer: Bridging Illumination and Reflectance Maps for Low-Light Image Enhancement

  • Conference paper
  • First Online:
Neural Information Processing (ICONIP 2023)

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 1966))

Included in the following conference series:

  • 491 Accesses

Abstract

Low-light image enhancement, which is also known as LLIE for short, aims to reconstruct the original normal image from its low-illumination counterpart. Recently, it has received increasingly attention in image restoration. In particular, with the success of deep convolutional neural network (CNN), Retinex-based approaches have emerged as a promising line of research in LLIE, since they can well transfer adequate prior knowledge from an image captured under sufficient illumination to its low-light version for image enhancement. However, existing Retinex-based approaches usually overlook the correlation between Illumination and Reflectance maps which are both derived from the same feature extractor, leading to sub-optimal reconstructed image quality. In this study, we propose a novel Transformer architecture for LLIE, termed Bridging Illumination and Reflectance maps Transformer which is shortly BIRT. It aims to estimate the correlation between Illumination and Reflectance maps derived from Retinex decomposition within a Transformer architecture via the Multi-Head Self-Attention mechanism. In terms of model structure, the proposed BIRT comprises Retinex-based and Transformer-based sub-networks, which allow our model to elevate the image quality by learning cross-feature dependencies and long-range details between Illumination and Reflectance maps. Experimental results demonstrate that the proposed BIRT model achieves competitive performance on par with the state-of-the-arts on the public benchmarking datasets for LLIE.

Supported by the National Natural Science Foundation of China under Grant 62173186 and 62076134.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 79.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 99.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Bandara, W.G.C., Patel, V.M.: Hypertransformer: a textural and spectral feature fusion transformer for pansharpening. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1767–1777 (2022)

    Google Scholar 

  2. Cai, Y., Bian, H., Lin, J., Wang, H., Timofte, R., Zhang, Y.: Retinexformer: one-stage retinex-based transformer for low-light image enhancement. arXiv preprint arXiv:2303.06705 (2023)

  3. Chen, H., et al.: Pre-trained image processing transformer. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 12299–12310 (2021)

    Google Scholar 

  4. Dabov, K., Foi, A., Katkovnik, V., Egiazarian, K.: Image denoising with block-matching and 3D filtering. In: Image Processing: Algorithms and Systems, Neural Networks, and Machine Learning, vol. 6064, pp. 354–365. SPIE (2006)

    Google Scholar 

  5. Guo, C., et al.: Zero-reference deep curve estimation for low-light image enhancement. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1780–1789 (2020)

    Google Scholar 

  6. Han, K., et al.: A survey on vision transformer. IEEE Trans. Pattern Anal. Mach. Intell. 45(1), 87–110 (2023). https://doi.org/10.1109/TPAMI.2022.3152247

    Article  Google Scholar 

  7. Huang, S.C., Cheng, F.C., Chiu, Y.S.: Efficient contrast enhancement using adaptive gamma correction with weighting distribution. IEEE Trans. Image Process. 22(3), 1032–1041 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  8. Jiang, Y., et al.: Enlightengan: deep light enhancement without paired supervision. IEEE Trans. Image Process. 30, 2340–2349 (2021)

    Article  Google Scholar 

  9. Johnson, J., Alahi, A., Fei-Fei, L.: Perceptual losses for real-time style transfer and super-resolution. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9906, pp. 694–711. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46475-6_43

    Chapter  Google Scholar 

  10. Land, E.H.: The retinex theory of color vision. Sci. Am. 237(6), 108–129 (1977)

    Article  Google Scholar 

  11. Li, C., et al.: Low-light image and video enhancement using deep learning: a survey. IEEE Trans. Pattern Anal. Mach. Intell. 44(12), 9396–9416 (2021)

    Article  Google Scholar 

  12. Li, C., Guo, C., Loy, C.C.: Learning to enhance low-light image via zero-reference deep curve estimation. IEEE Trans. Pattern Anal. Mach. Intell. 44(8), 4225–4238 (2021)

    Google Scholar 

  13. Liu, J., Xu, D., Yang, W., Fan, M., Huang, H.: Benchmarking low-light image enhancement and beyond. Int. J. Comput. Vision 129, 1153–1184 (2021)

    Article  Google Scholar 

  14. Liu, R., Ma, L., Zhang, J., Fan, X., Luo, Z.: Retinex-inspired unrolling with cooperative prior architecture search for low-light image enhancement. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 10561–10570 (2021)

    Google Scholar 

  15. Liu, Z., et al.: Video swin transformer. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3202–3211 (2022)

    Google Scholar 

  16. Lore, K.G., Akintayo, A., Sarkar, S.: Llnet: a deep autoencoder approach to natural low-light image enhancement. Pattern Recogn. 61, 650–662 (2017)

    Article  Google Scholar 

  17. Ma, L., Liu, R., Wang, Y., Fan, X., Luo, Z.: Low-light image enhancement via self-reinforced retinex projection model. IEEE Trans. Multimedia (2022)

    Google Scholar 

  18. Pizer, S.M., et al.: Adaptive histogram equalization and its variations. Comput. Vis. Graph. Image Process. 39(3), 355–368 (1987)

    Article  Google Scholar 

  19. Ren, S., He, K., Girshick, R., Sun, J.: Faster R-CNN: towards real-time object detection with region proposal networks. In: Advances in Neural Information Processing Systems, vol. 28 (2015)

    Google Scholar 

  20. Shao, H., Wang, L., Chen, R., Li, H., Liu, Y.: Safety-enhanced autonomous driving using interpretable sensor fusion transformer. In: Conference on Robot Learning, pp. 726–737. PMLR (2023)

    Google Scholar 

  21. Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014)

  22. Wang, R., Zhang, Q., Fu, C.W., Shen, X., Zheng, W.S., Jia, J.: Underexposed photo enhancement using deep illumination estimation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 6849–6857 (2019)

    Google Scholar 

  23. Wang, W., et al.: Internimage: exploring large-scale vision foundation models with deformable convolutions. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 14408–14419 (2023)

    Google Scholar 

  24. Wang, Z., Cun, X., Bao, J., Zhou, W., Liu, J., Li, H.: Uformer: a general U-shaped transformer for image restoration. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 17683–17693 (2022)

    Google Scholar 

  25. Wei, C., Wang, W., Yang, W., Liu, J.: Deep retinex decomposition for low-light enhancement. arXiv preprint arXiv:1808.04560 (2018)

  26. Wu, W., Weng, J., Zhang, P., Wang, X., Yang, W., Jiang, J.: Uretinex-net: retinex-based deep unfolding network for low-light image enhancement. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5901–5910 (2022)

    Google Scholar 

  27. Xu, X., Wang, R., Fu, C., Jia, J.: SNR-aware low-light image enhancement. In: 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 17693–17703 (2022)

    Google Scholar 

  28. Yang, S., Zhou, D., Cao, J., Guo, Y.: Lightingnet: an integrated learning method for low-light image enhancement. IEEE Trans. Comput. Imaging 9, 29–42 (2023)

    Article  Google Scholar 

  29. Yang, W., Wang, W., Huang, H., Wang, S., Liu, J.: Sparse gradient regularized deep retinex network for robust low-light image enhancement. IEEE Trans. Image Process. 30, 2072–2086 (2021)

    Article  Google Scholar 

  30. Yuan, L., et al.: Tokens-to-token VIT: training vision transformers from scratch on imagenet. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 558–567 (2021)

    Google Scholar 

  31. Zamir, S.W., Arora, A., Khan, S., Hayat, M., Khan, F.S., Yang, M.H.: Restormer: efficient transformer for high-resolution image restoration. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5728–5739 (2022)

    Google Scholar 

  32. Zhang, Y., Guo, X., Ma, J., Liu, W., Zhang, J.: Beyond brightening low-light images. Int. J. Comput. Vision 129, 1013–1037 (2021)

    Article  Google Scholar 

  33. Zhang, Y., Zhang, J., Guo, X.: Kindling the darkness: a practical low-light image enhancer. In: Proceedings of the 27th ACM International Conference on MultiMedia, pp. 1632–1640 (2019)

    Google Scholar 

  34. Zhu, X., Su, W., Lu, L., Li, B., Wang, X., Dai, J.: Deformable DETR: deformable transformers for end-to-end object detection. arXiv preprint arXiv:2010.04159 (2020)

Download references

Author information

Authors and Affiliations

  1. School of Computer and Electronic Information, Nanjing Normal University, Nanjing, 210023, China

    Yilong Cheng, Jun Li & Jianhua Xu

  2. School of Data Science and Engineering, East China Normal University, Shanghai, 200062, China

    Zhijian Wu

Authors
  1. Yilong Cheng
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zhijian Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jun Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jianhua Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jun Li .

Editor information

Editors and Affiliations

  1. School of Automation, Central South University, Changsha, China

    Biao Luo

  2. Institute of Automation, Chinese Academy of Sciences, Beijing, China

    Long Cheng

  3. Institute of Cyber-Systems and Control, Zhejiang University, Hangzhou, China

    Zheng-Guang Wu

  4. School of Automation, Guangdong University of Technology, Guangzhou, China

    Hongyi Li

  5. School of Electrical Engineering and Telecommunications, UNSW Sydney, Sydney, NSW, Australia

    Chaojie Li

Rights and permissions

Reprints and permissions

Copyright information

© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Cheng, Y., Wu, Z., Li, J., Xu, J. (2024). Retinex Meets Transformer: Bridging Illumination and Reflectance Maps for Low-Light Image Enhancement. In: Luo, B., Cheng, L., Wu, ZG., Li, H., Li, C. (eds) Neural Information Processing. ICONIP 2023. Communications in Computer and Information Science, vol 1966. Springer, Singapore. https://doi.org/10.1007/978-981-99-8148-9_31

Download citation

  • DOI: https://doi.org/10.1007/978-981-99-8148-9_31

  • Published:

  • Publisher Name: Springer, Singapore

  • Print ISBN: 978-981-99-8147-2

  • Online ISBN: 978-981-99-8148-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation