Skip to main content
SpringerLink
Log in

Low-light image enhancement base on brightness attention mechanism generative adversarial networks

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

With the development of the field of deep learning, image recognition, enhancement and other technologies have been widely used.However, dark lighting environments in reality, such as insufficient light at night, cause or block photographic images in low brightness, severe noise, and a large number of details are lost, resulting in a huge loss of image content and information, which hinders further analysis and use. Such problems not only exist in the traditional deep learning field, but also exist in criminal investigation, scientific photography and other fields, such as the accuracy of low-light image. However, in the current research results, there is no perfect means to deal with the above problems. Therefore, the study of low-light image enhancement has important theoretical significance and practical application value for the development of smart cities. In order to improve the quality of low-light enhanced images, this paper tries to introduce the luminance attention mechanism to improve the enhancement efficiency. The main contents of this paper are summarized as follows: using the attention mechanism, we proposed a method of low-light image enhancement based on the brightness attention mechanism and generative adversarial networks . This method uses brightness attention mechanism to predict the illumination distribution of low-light image and guides the enhancement network to enhance the image adaptiveness in different luminance regions. At the same time, u-NET network is designed and constructed to improve the modeling process of low-light image. We verified the performance of the algorithm on the synthetic data set and compared it with traditional image enhancement methods (HE, SRIE) and deep learning methods (DSLR). The experimental results show that our proposed network model has relatively good enhancement quality for low-light images, and improves the overall robustness, which has practical significance for solving the problem of low-light image enhancement.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17

Similar content being viewed by others

Availibility of data and materials

No suitable materials are available.

Abbreviations

CGAN:

Conditional Generative adversarial networks

GAN:

Generative adversarial networks

HE:

Histogram equalization

SRIE:

Reflected illumination estimation

PSNR:

Peak signal-to-noise ratio

SSIM:

Structural similarity

References

  1. Chen C, Chen Q, Xu J, et al (2023) Learning to see in the dark[C]. Proc IEEE Conf Comput Vis Pattern Recognit 2018:3291–3300

  2. Denton EL, Chintala S, Fergus R (2015) Deep generative image models using a laplacian pyramid of adversarial networks[C]. Advances in neural information processing systems 2015:1486–1494

  3. Gharbi M, Chen J, Barron JT et al (2017) Deep bilateral learning for real-time image enhancement[J]. ACM Transactions on Graphics (TOG) 36(4):118

    Article  Google Scholar 

  4. Goodfellow IJ, Pouget-Abadie J, Mirza M, et al (2014) Generative adversarial nets[C]. Proceedings of the 27th International Conference on Neural Information Processing Systems, Montreal, Dec 8-13. Cambridge: MIT Press 2014:2672–2680

  5. Howard AG, Zhu M, Chen B, et al (2017) MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications[J]

  6. Ioffe S, Szegedy C (2015) Batch normalization: Accelerating deep network training by reducing internal covariate shift[J]. arXiv preprint arXiv:1502.03167

  7. Jaderberg M, Simonyan K, Zisserman A (2015) Spatial transformer networks[C]. Advances in neural information processing systems 2015:2017–2025

  8. Kumar A, Rastogi P, Srivastava P (2015) Design and FPGA Implementation of DWT, Image Text Extraction Technique[J]. Procedia Computer Science 57:1015–1025

    Article  Google Scholar 

  9. Li, Huafeng, He, Xiaoge, Tao, Dapeng, Tang, Yuanyan, Wang, Ruxin (2018) Joint medical image fusion, denoising and enhancement via discriminative low-rank sparse dictionaries learning[J]. Pattern Recognition the Journal of the Pattern Recognition Society

  10. Li, Huafeng, He, Xiaoge, Tao, Dapeng, Tang, Yuanyan, Wang, Ruxin (2018) Joint medical image fusion, denoising and enhancement via discriminative low-rank sparse dictionaries learning[J]. Pattern Recognition the Journal of the Pattern Recognition Society

  11. Ma N, Zhang X, Zheng HT, et al (2018) ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Design[J]

  12. Mirza M, Osindero S (2014) Conditional generative adversarial nets[J]. arXiv preprint arXiv:1411.1784

  13. Radford A, Metz L, Chintala S (2015) Unsupervised representation learning with deep convolutional generative adversarial networks[J]. arXiv preprint arXiv:1511.06434

  14. Ronneberger O, Fischer P, Brox T (2015) U-net: Convolutional networks for biomedical image segmentation[C]. International Conference on Medical image computing and computer-assisted intervention 2015:234–241. Springer, Cham

  15. Sandler, Mark, Howard AG, Zhu M, Zhmoginov A, Chen L-C (2018) MobileNetV2: Inverted Residuals and Linear Bottlenecks. 2018 IEEE/CVF Conf Comput Vis Pattern Recognit (2018):4510–4520

  16. Yan L, Jairun F, Wang C, Ye Z, Chen H, Ling H (2021) Enhanced network optimized generative adversarial network for image enhancement[J]. Multimedia Tools and Applications 1

  17. Zhang R , Isola P , Efros AA , et al (2018) The Unreasonable Effectiveness of Deep Features as a Perceptual Metric[J]. IEEE

  18. Zhang Y (2019) Research on population Counting Algorithm based on attention convolutional neural network[J]. Shandong university

  19. Zhang Y (2019) Research on population Counting Algorithm based on attention convolutional neural network[J]. Shandong university

  20. Zhang Z (2017) Study on weather classification and Low-quality Image Enhancement for outdoor monitoring scenes[J]. Beijing University of Posts and Telecommunications

  21. Zhang Z (2017) Study on weather classification and Low-quality Image Enhancement for outdoor monitoring scenes[J]. Beijing University of Posts and Telecommunications

  22. Zhang H, Goodfellow I, Metaxas D, et al (2018) Self-attention generative adversarial networks[J]. arXiv preprint arXiv:1805.08318

  23. Zhang Y, Zhang J, Guo X (2019) Kindling the Darkness: A Practical Low-light Image Enhancer[C]. Proceedings of the 27th ACM International Conference on Multimedia 2019:1632–1640

  24. Zhang X, Zhou X, Lin M, et al (2017) ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices[J]

  25. Zhou Z, Feng Z, Liu J, Hao S (2018) Single-image low-light enhancement via generating and fusing multi?ple sources[J]. Neural Comput & Applic

  26. Zhou Z, Feng Z, Liu J, Hao S (2018) Single-image low-light enhancement via generating and fusing multiple sources[J]. Neural Comput & Applic

Download references

Funding

Project supported by the National Natural Science Foundation (61502155,61772180);

Author information

Authors and Affiliations

  1. School of Computer Science, Hubei University of Technology, Wuhan, China

    Jiarun Fu, Lingyu Yan, KunPeng Zheng & Rong Gao

  2. Hunan ProvinciaMaternal and Child Health Care Hospital, Changsha, China

    Yulin Peng

  3. School of Computer Science, Huazhong University of Science and Technology, Wuhan, China

    HeFei Ling

Authors
  1. Jiarun Fu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Lingyu Yan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yulin Peng
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. KunPeng Zheng
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Rong Gao
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. HeFei Ling
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Jiarun.Fu and Lingyu.Yan conceived the study, participated in the design of its experiments, and drafted the manuscript. Yulin.Peng participated in the experimental part of the research, assisted in completing the experiment and made statistical analysis. Kunpeng.Zheng participated in the synthesis of the low-light image set and counted the experimental results. Gao.Rong and Hefei.Ling provided ideas for the use of the attention mechanism in the paper and participated in the construction of the experimental network. Lingyu.Yan provided financial support for the research of this paper and participated in the improvement of the paper manuscript. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Lingyu Yan.

Ethics declarations

Competing interests

The authors do not have any possible conflicts of interest. All authors read and approved the final manuscript.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendix

Appendix

1.1 Figure legends

This section will introduce the images used in the article.

Figure 1:Smart city image recognition application.

Figure 1 detailed legend:This picture shows an example of the application of image recognition technology in smart cities, specifically the detection and recognition of vehicles by the traffic monitoring system. The blue box is the recognized vehicle. We can see that the quality of the image is slightly blurred, which corresponds to the point that the image quality of smart city recognition described in the text is limited by hardware conditions.

Figure 2:Low light environment photo instance.

Figure 2 detailed legend:This picture shows a specific example of a low-light image that needs to be enhanced. The picture shows the result of taking pictures of several people under street lights with mobile phone cameras in the dark. It can be seen that although there is a light source, due to the lack of light in the overall environment, the characters in the picture are basically completely black. And this is the dilemma that low-light image enhancement algorithms are trying to solve.

Figure 3:Example of image enhancement for mobile phone photography.

Figure 3 detailed legend:This picture shows the actual image enhancement algorithm to enhance the results of mobile phone camera photos. The specific content on the way is a busy street, and we can see that various colors have become more vivid due to enhanced enhancement algorithms. And the influence of the light source on the image quality has also been well controlled. This picture is also the goal pursued by the low-light enhancement algorithm.

Figure 4:Low light image dilemma in the field of criminal investigation.

Figure 4 detailed legend:This picture shows the dilemma of image recognition technology in the field of criminal investigation photography. As you can see in the picture, although the blue frame contains all vehicle targets, because the lighting environment is too dark, only a group of "dark" results can be seen. Obviously, this is fatal to the technical field of criminal investigation, which requires precision. The problem in this figure is also the goal our algorithm aims to solve.

Figure 5:Network structure diagram of CGAN.

Figure 5 detailed legend:This picture is a specific explanation of CGAN in the reference. We can see that the figure includes two parts: generator and discriminator. In each part, a different prototype part is used to explain the generation process of the generation result X and the discrimination result Z respectively. Supplementary explanation is provided for the above formula.

Figure 6:Self-attention module network structure diagram.

Figure 6 detailed legend:This picture is a specific explanation of SA-GAN in the reference. The attention mechanism in this picture is the focus of this article. The f(x) and g(x) in the upper part of the figure are the specific functions of the attention mechanism, and h(x) is the normal convolutional neural network. Its specific action process is described in detail in the text.

Figure 7:Spatial Transformer application example diagram.

Figure 7 detailed legend:This picture shows a concrete example of low-light image enhancement. The content in the figure is the specific process of the enhancement of the handwritten digit set. The specific process explanation has been described in detail in the text.

Figure 8:Standard U-net structure diagram.

Figure 8 detailed legend:This picture is a specific description of U-net. Because U-net is the prototype of the network used in this article. Therefore, the various parts of U-net are marked in different colors in the local area, and their principles and names are explained in detail in the figure. Other detailed descriptions have been given in the text.

Figure 9:Schematic diagram of generator network structure.

Figure 9 detailed legend:This picture is an explanation of the structure of the generator in this article. The specific content in the figure is that the input low-light image is processed by the attention network composed of the orange and blue sampling layers to generate a clearer image result.

Figure 10:Schematic diagram of discriminator network structure.

Figure 10 detailed legend:This picture is an explanation of the structure of the discriminator in the network constructed in this article. The specific content in the figure is that after the input low-light image is processed by the U-net-like network, a clearer image result is generated after inputting the fully connected layer and the Softmax layer.

Figure 11:Schematic diagram of generator network structure.

Figure 11 detailed legend:The content of this figure is basically the same as in Fig. 14. The purpose is to further explain the same problem with different colors.

Figure 12:Original DIV2K image set.

Figure 12 detailed legend:This picture shows the Original DIV2K image set. The specific content in the figure is nine images that have not been preprocessed, which is explained with the preprocessed images below.

Figure 13:Div2k image set after low light preprocessing.

Figure 13 detailed legend:This picture shows the DIV2K image set that has undergone low-light processing. The specific content in the picture is nine preprocessed images. We can see from the figure that after preprocessing, the image is relatively dark compared to Fig18. This result accomplishes our purpose of artificially constructing low-light images.

Figure 14:Image results of the experiment.

Figure 14 detailed legend:This picture is a concrete display of the experimental results. The figure includes three comparison algorithms, our algorithm, the original image and the image after low-light preprocessing. The specific contents are human faces, oranges and an oil painting. This picture intuitively shows the visual effects of various images.

Figure 15:Ablation experiment result - Less noise.

Figure 15 detailed legend:This is the first step in our ablation experiments, selecting low-light images with less noise to demonstrate the effect of the luminance attention mechanism.

Figure 16:Ablation experiment result-noise.

Figure 16 This is the second step in our ablation experiments, selecting noisy low-light images to demonstrate the effect of the luminance attention mechanism..

Figure 17:Ablation experiment result.

Figure 17 detailed legend:This picture shows the effect of the brightness attention mechanism in the experiment. The figure includes three types of low-light images, the brightness attention map generated by the algorithm, and the final image generated by the algorithm. After comparison, we can find that the image of the brightness attention mechanism plays a very good auxiliary effect on the enhancement of low-light images, and finally helps our algorithm generate excellent results.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Fu, J., Yan, L., Peng, Y. et al. Low-light image enhancement base on brightness attention mechanism generative adversarial networks. Multimed Tools Appl 83, 10341–10365 (2024). https://doi.org/10.1007/s11042-023-15815-x

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-023-15815-x

Keywords

Search

Navigation