Skip to main content
SpringerLink
Log in

L\(^2\)DM: A Diffusion Model for Low-Light Image Enhancement

  • Conference paper
  • First Online:
Pattern Recognition and Computer Vision (PRCV 2023)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 14435))

Included in the following conference series:

Abstract

Low-light image enhancement is a challenging yet beneficial task in computer vision that aims to improve the quality of images captured under poor illumination conditions. It involves addressing difficulties such as color distortions and noise, which often degrade the visual fidelity of low-light images. Although tremendous CNN-based and ViT-based approaches have been proposed, the potential of diffusion models in this domain remains unexplored. This paper presents L\(^2\)DM, a novel framework for low-light image enhancement using diffusion models. Since L\(^2\)DM falls into the category of latent diffusion models, it can reduce computational requirements through denoising and the diffusion process in latent space. Conditioning inputs are essential for guiding the enhancement process, therefore, a new ViT-based network called ViTCondNet is introduced to efficiently incorporate conditioning low-light inputs into the image generation pipeline. Extensive experiments on benchmark LOL datasets demonstrate L\(^2\)DM’s state-of-the-art performance compared to diffusion-based counterparts. The L\(^2\)DM source code is available on GitHub for reproducibility and further research.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 109.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 139.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Arici, T., Dikbas, S., Altunbasak, Y.: A histogram modification framework and its application for image contrast enhancement. IEEE Trans. Image Process. 18(9), 1921–1935 (2009)

    Article  MathSciNet  Google Scholar 

  2. Charbonnier, P., Blanc-Feraud, L., Aubert, G., Barlaud, M.: Two deterministic half-quadratic regularization algorithms for computed imaging. In: Proceedings of IEEE International Conference on Image Processing, vol. 2, pp. 168–172. IEEE (1994)

    Google Scholar 

  3. Chen, C., Chen, Q., Xu, J., Koltun, V.: Learning to see in the dark. In: Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3291–3300 (2018)

    Google Scholar 

  4. Croitoru, F.A., Hondru, V., Ionescu, R.T., Shah, M.: Diffusion models in vision: a survey. IEEE Trans. Pattern Anal. Mach. Intelli. (2023)

    Google Scholar 

  5. Cui, Z., et al.: Illumination adaptive transformer. arXiv preprint arXiv:2205.14871 (2022)

  6. Dhariwal, P., Nichol, A.: Diffusion models beat GANs on image synthesis. Adv. Neural. Inf. Process. Syst. 34, 8780–8794 (2021)

    Google Scholar 

  7. Dong, X., et al.: Abandoning the Bayer-filter to see in the dark. In: Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 17431–17440 (2022)

    Google Scholar 

  8. Esser, P., Rombach, R., Ommer, B.: Taming transformers for high-resolution image synthesis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 12873–12883 (2021)

    Google Scholar 

  9. Fan, C.M., Liu, T.J., Liu, K.H.: Half wavelet attention on M-Net+ for low-light image enhancement. In: 2022 IEEE International Conference on Image Processing (ICIP), pp. 3878–3882. IEEE (2022)

    Google Scholar 

  10. Fan, M., Wang, W., Yang, W., Liu, J.: Integrating semantic segmentation and retinex model for low-light image enhancement. In: Proceedings of the 28th ACM International Conference on Multimedia (ACMMM). pp. 2317–2325 (2020)

    Google Scholar 

  11. Guo, C., et al.: Zero-reference deep curve estimation for low-light image enhancement. In: Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1780–1789 (2020)

    Google Scholar 

  12. Guo, X., Hu, Q.: Low-light image enhancement via breaking down the darkness. Int. J. Comput. Vision 131(1), 48–66 (2023)

    Article  Google Scholar 

  13. Guo, X., Li, Y., Ling, H.: LIME: low-light image enhancement via illumination map estimation. IEEE Trans. Image Process. 26(2), 982–993 (2016)

    Article  MathSciNet  Google Scholar 

  14. Ho, J., Jain, A., Abbeel, P.: Denoising diffusion probabilistic models. Adv. Neural. Inf. Process. Syst. 33, 6840–6851 (2020)

    Google Scholar 

  15. Isola, P., Zhu, J.Y., Zhou, T., Efros, A.A.: Image-to-image translation with conditional adversarial networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1125–1134 (2017)

    Google Scholar 

  16. Jiang, N., Lin, J., Zhang, T., Zheng, H., Zhao, T.: Low-light image enhancement via stage-transformer-guided network. IEEE Trans. Circuits Syst. Video Technol. (2023)

    Google Scholar 

  17. Jiang, Y., et al.: EnlightenGAN: deep light enhancement without paired supervision. IEEE Trans. Image Process. 30, 2340–2349 (2021)

    Article  Google Scholar 

  18. Jin, Y., Yang, W., Tan, R.T.: Unsupervised night image enhancement: when layer decomposition meets light-effects suppression. In: Avidan, S., Brostow, G., Cissé, M., Farinella, G.M., Hassner, T. (eds.) ECCV 2022, Part XXXVII. LNCS, vol. 13697, pp. 404–421. Springer, Cham (2022). https://doi.org/10.1007/978-3-031-19836-6_23

    Chapter  Google Scholar 

  19. Kawar, B., Elad, M., Ermon, S., Song, J.: Denoising diffusion restoration models. arXiv preprint arXiv:2201.11793 (2022)

  20. Kim, G., Kwon, D., Kwon, J.: Low-lightgan: low-light enhancement via advanced generative adversarial network with task-driven training. In: 2019 IEEE International Conference on Image Processing (ICIP), pp. 2811–2815. IEEE (2019)

    Google Scholar 

  21. Kim, H., Choi, S.M., Kim, C.S., Koh, Y.J.: Representative color transform for image enhancement. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 4459–4468 (2021)

    Google Scholar 

  22. Kosugi, S., Yamasaki, T.: Unpaired image enhancement featuring reinforcement-learning-controlled image editing software. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34, pp. 11296–11303 (2020)

    Google Scholar 

  23. Lee, C., Lee, C., Kim, C.S.: Contrast enhancement based on layered difference representation of 2D histograms. IEEE Trans. Image Process. 22(12), 5372–5384 (2013)

    Article  Google Scholar 

  24. Li, J., Li, J., Fang, F., Li, F., Zhang, G.: Luminance-aware pyramid network for low-light image enhancement. IEEE Trans. Multimedia 23, 3153–3165 (2020)

    Article  Google Scholar 

  25. Lim, S., Kim, W.: DSLR: deep stacked laplacian restorer for low-light image enhancement. IEEE Trans. Multimedia 23, 4272–4284 (2020)

    Article  Google Scholar 

  26. Liu, R., Ma, L., Zhang, J., Fan, X., Luo, Z.: Retinex-inspired unrolling with cooperative prior architecture search for low-light image enhancement. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 10561–10570 (2021)

    Google Scholar 

  27. Lu, C., Zhou, Y., Bao, F., Chen, J., Li, C., Zhu, J.: DPM-solver: a fast ode solver for diffusion probabilistic model sampling in around 10 steps. arXiv preprint arXiv:2206.00927 (2022)

  28. Ma, L., Ma, T., Liu, R., Fan, X., Luo, Z.: Toward fast, flexible, and robust low-light image enhancement. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5637–5646 (2022)

    Google Scholar 

  29. Mirza, M., Osindero, S.: Conditional generative adversarial nets. arXiv preprint arXiv:1411.1784 (2014)

  30. Moran, S., Marza, P., McDonagh, S., Parisot, S., Slabaugh, G.: DeepLPF: deep local parametric filters for image enhancement. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 12826–12835 (2020)

    Google Scholar 

  31. Park, T., Liu, M.Y., Wang, T.C., Zhu, J.Y.: Semantic image synthesis with spatially-adaptive normalization. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2337–2346 (2019)

    Google Scholar 

  32. Reed, S., Akata, Z., Yan, X., Logeswaran, L., Schiele, B., Lee, H.: Generative adversarial text to image synthesis. In: International Conference on Machine Learning, pp. 1060–1069. PMLR (2016)

    Google Scholar 

  33. Ren, W., et al.: Deep video dehazing with semantic segmentation. IEEE Trans. Image Process. 28(4), 1895–1908 (2018)

    Article  MathSciNet  Google Scholar 

  34. Rombach, R., Blattmann, A., Lorenz, D., Esser, P., Ommer, B.: High-resolution image synthesis with latent diffusion models. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 10684–10695 (2022)

    Google Scholar 

  35. Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24574-4_28

    Chapter  Google Scholar 

  36. Saharia, C., et al.: Palette: image-to-image diffusion models. In: ACM SIGGRAPH 2022 Conference Proceedings, pp. 1–10 (2022)

    Google Scholar 

  37. Sohl-Dickstein, J., Weiss, E., Maheswaranathan, N., Ganguli, S.: Deep unsupervised learning using nonequilibrium thermodynamics. In: International Conference on Machine Learning, pp. 2256–2265. PMLR (2015)

    Google Scholar 

  38. Sohn, K., Lee, H., Yan, X.: Learning structured output representation using deep conditional generative models. In: Advances in Neural Information Processing Systems, vol. 28 (2015)

    Google Scholar 

  39. Song, Y., Ermon, S.: Generative modeling by estimating gradients of the data distribution. In: Advances in Neural Information Processing Systems, vol. 32 (2019)

    Google Scholar 

  40. Van Den Oord, A., Vinyals, O., et al.: Neural discrete representation learning. In: Advances in Neural Information Processing Systems, vol. 30 (2017)

    Google Scholar 

  41. Vaswani, A., et al.: Attention is all you need. In: Advances in Neural Information Processing Systems, vol. 30 (2017)

    Google Scholar 

  42. Wang, H., Xu, K., Lau, R.W.: Local color distributions prior for image enhancement. In: Avidan, S., Brostow, G., Cissé, M., Farinella, G.M., Hassner, T. (eds.) ECCV 2022, Part XVIII. LNCS, vol. 13678, pp. 343–359. Springer, Cham (2022). https://doi.org/10.1007/978-3-031-19797-0_20

    Chapter  Google Scholar 

  43. Wang, R., Zhang, Q., Fu, C.W., Shen, X., Zheng, W.S., Jia, J.: Underexposed photo enhancement using deep illumination estimation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 6849–6857 (2019)

    Google Scholar 

  44. Wang, T., Li, Y., Peng, J., Ma, Y., Wang, X., Song, F., Yan, Y.: Real-time image enhancer via learnable spatial-aware 3D lookup tables. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 2471–2480 (2021)

    Google Scholar 

  45. Wang, Y., Wan, R., Yang, W., Li, H., Chau, L.P., Kot, A.: Low-light image enhancement with normalizing flow. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 36, pp. 2604–2612 (2022)

    Google Scholar 

  46. Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13(4), 600–612 (2004)

    Article  Google Scholar 

  47. Wei, C., Wang, W., Yang, W., Liu, J.: Deep retinex decomposition for low-light enhancement. arXiv preprint arXiv:1808.04560 (2018)

  48. Wu, W., Weng, J., Zhang, P., Wang, X., Yang, W., Jiang, J.: URetinex-Net: Retinex-based deep unfolding network for low-light image enhancement. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5901–5910 (2022)

    Google Scholar 

  49. Wu, X., Liu, X., Hiramatsu, K., Kashino, K.: Contrast-accumulated histogram equalization for image enhancement. In: 2017 IEEE International Conference on Image Processing (ICIP), pp. 3190–3194. IEEE (2017)

    Google Scholar 

  50. Xu, K., Yang, X., Yin, B., Lau, R.W.: Learning to restore low-light images via decomposition-and-enhancement. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2281–2290 (2020)

    Google Scholar 

  51. Xu, K., Yang, X., Yin, B., Lau, R.W.: Learning to restore low-light images via decomposition-and-enhancement. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2281–2290 (2020)

    Google Scholar 

  52. Xu, W., Dong, X., Ma, L., Teoh, A.B.J., Lin, Z.: Rawformer: an efficient vision transformer for low-light raw image enhancement. IEEE Signal Process. Lett. 29, 2677–2681 (2022)

    Article  Google Scholar 

  53. Xu, X., Wang, R., Fu, C.W., Jia, J.: SNR-aware low-light image enhancement. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 17714–17724 (2022)

    Google Scholar 

  54. Xu, X., Wang, R., Lu, J.: Low-light image enhancement via structure modeling and guidance. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9893–9903 (2023)

    Google Scholar 

  55. Yang, W., Wang, S., Fang, Y., Wang, Y., Liu, J.: From fidelity to perceptual quality: a semi-supervised approach for low-light image enhancement. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3063–3072 (2020)

    Google Scholar 

  56. Yang, W., Wang, S., Fang, Y., Wang, Y., Liu, J.: Band representation-based semi-supervised low-light image enhancement: bridging the gap between signal fidelity and perceptual quality. IEEE Trans. Image Process. 30, 3461–3473 (2021)

    Article  Google Scholar 

  57. Yang, W., Wang, W., Huang, H., Wang, S., Liu, J.: Sparse gradient regularized deep retinex network for robust low-light image enhancement. IEEE Trans. Image Process. 30, 2072–2086 (2021)

    Article  Google Scholar 

  58. Yu, J., et al.: Vector-quantized image modeling with improved VQGAN. arXiv preprint arXiv:2110.04627 (2021)

  59. Yuan, N., et al.: Low-light image enhancement by combining transformer and convolutional neural network. Mathematics 11(7), 1657 (2023)

    Article  Google Scholar 

  60. Yuan, Y., et al.: Learning to kindle the starlight. arXiv preprint arXiv:2211.09206 (2022)

  61. Zamir, S.W., et al.: Learning enriched features for real image restoration and enhancement. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12370, pp. 492–511. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58595-2_30

    Chapter  Google Scholar 

  62. Zeng, H., Cai, J., Li, L., Cao, Z., Zhang, L.: Learning image-adaptive 3D lookup tables for high performance photo enhancement in real-time. IEEE Trans. Pattern Anal. Mach. Intell. 44(4), 2058–2073 (2020)

    Google Scholar 

  63. Zhang, R., Isola, P., Efros, A.A., Shechtman, E., Wang, O.: The unreasonable effectiveness of deep features as a perceptual metric. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 586–595 (2018)

    Google Scholar 

  64. Zhang, Y., Guo, X., Ma, J., Liu, W., Zhang, J.: Beyond brightening low-light images. Int. J. Comput. Vision 129, 1013–1037 (2021)

    Article  Google Scholar 

  65. Zhang, Y., Zhang, J., Guo, X.: Kindling the darkness: a practical low-light image enhancer. In: Proceedings of the 27th ACM International Conference on Multimedia (ACMMM), pp. 1632–1640 (2019)

    Google Scholar 

  66. Zhao, L., Lu, S.P., Chen, T., Yang, Z., Shamir, A.: Deep symmetric network for underexposed image enhancement with recurrent attentional learning. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 12075–12084 (2021)

    Google Scholar 

  67. Zheng, C., Shi, D., Shi, W.: Adaptive unfolding total variation network for low-light image enhancement. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 4439–4448 (2021)

    Google Scholar 

  68. Zhu, M., Pan, P., Chen, W., Yang, Y.: EEMEFN: low-light image enhancement via edge-enhanced multi-exposure fusion network. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34, pp. 13106–13113 (2020)

    Google Scholar 

Download references

Acknowledgments

This work was supported by the National Natural Science Foundation of China (Grant Nos. 62376003, 62306003, 62372004, 62302005).

Author information

Authors and Affiliations

  1. Anhui Provincial International Joint Research Center for Advanced Technology in Medical Imaging, School of Computer Science and Technology, Anhui University, Hefei, 230093, China

    Xingguo Lv, Hui Zhang, Siyi Song & Xuejun Li

  2. Anhui Provincial Key Laboratory of Secure Artificial Intelligence, School of Artificial Intelligence, Anhui University, Hefei, 230093, China

    Xingbo Dong & Zhe Jin

Authors
  1. Xingguo Lv
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xingbo Dong
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zhe Jin
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Hui Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Siyi Song
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Xuejun Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xingbo Dong .

Editor information

Editors and Affiliations

  1. Nanjing University of Information Science and Technology, Nanjing, China

    Qingshan Liu

  2. Xiamen University, Xiamen, China

    Hanzi Wang

  3. Beijing University of Posts and Telecommunications, Beijing, China

    Zhanyu Ma

  4. Sun Yat-sen University, Guangzhou, China

    Weishi Zheng

  5. Peking University, Beijing, China

    Hongbin Zha

  6. Chinese Academy of Sciences, Beijing, China

    Xilin Chen

  7. Chinese Academy of Sciences, Beijing, China

    Liang Wang

  8. Xiamen University, Xiamen, China

    Rongrong Ji

Rights and permissions

Reprints and permissions

Copyright information

© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Lv, X., Dong, X., Jin, Z., Zhang, H., Song, S., Li, X. (2024). L\(^2\)DM: A Diffusion Model for Low-Light Image Enhancement. In: Liu, Q., et al. Pattern Recognition and Computer Vision. PRCV 2023. Lecture Notes in Computer Science, vol 14435. Springer, Singapore. https://doi.org/10.1007/978-981-99-8552-4_11

Download citation

  • DOI: https://doi.org/10.1007/978-981-99-8552-4_11

  • Published:

  • Publisher Name: Springer, Singapore

  • Print ISBN: 978-981-99-8551-7

  • Online ISBN: 978-981-99-8552-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation