Abstract
The poor quality of images recorded in low-light environments affects their further applications. To improve the visibility of low-light images, we propose a recurrent network based on filter-cluster attention (FCA), the main body of which consists of three units: difference concern, gate recurrent, and iterative residual. The network performs multi-stage recursive learning on low-light images, and then extracts deeper feature information. To compute more accurate dependence, we design a novel FCA that focuses on the saliency of feature channels. FCA and self-attention are used to highlight the low-light regions and important channels of the feature. We also design a dense connection pyramid (DenCP) to extract the color features of the low-light inversion image, to compensate for the loss of the image’s color information. Experimental results on six public datasets show that our method has outstanding performance in subjective and quantitative comparisons.
摘要
在低光环境下拍摄的图像质量不佳, 影响其进一步应用. 为提升低光图像可视性, 提出一种基于过滤—群聚注意力(FCA)的递归网络, 其中主体由3个单元组成: 差异关注、 门控递归以及迭代残差. 该网络对低光图像进行多阶段递归学习, 进而提取更深层次特征信息. 为算得更加精确的相关性, 设计了一种关注特征通道突出性的FCA. FCA与自注意力被用以突出特征的低光区域与重要通道. 此外, 设计了密集连接金字塔(DenCP)来提取低光反转图的色彩特征, 使图像的色彩信息损失得以补偿. 在6种公开数据集上的实验结果表明, 本文方法在视觉和指标上有着突出表现.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Data availability
The data that support the findings of this study are available from the corresponding author upon reasonable request.
References
Abdullah-Al-Wadud M, Kabir H, Dewan MAA, et al., 2007. A dynamic histogram equalization for image contrast enhancement. IEEE Trans Consum Electron, 53(2):593–600. https://doi.org/10.1109/TCE.2007.381734
Aradi S, 2022. Survey of deep reinforcement learning for motion planning of autonomous vehicles. IEEE Trans Intell Transp Syst, 23(2):740–759. https://doi.org/10.1109/TITS.2020.3024655
Bychkovsky V, Paris S, Chan E, et al., 2011. Learning photographic global tonal adjustment with a database of input/output image pairs. IEEE/CVF Conf on Computer Vision and Pattern Recognition, p.97–104. https://doi.org/10.1109/CVPR.2011.5995413
Celik T, Tjahjadi T, 2011. Contextual and variational contrast enhancement. IEEE Trans Image Process, 20(12):3431–3441. https://doi.org/10.1109/TIP.2011.2157513
Chen BH, Wu YL, Shi LF, 2019. A fast image contrast enhancement algorithm using entropy-preserving mapping prior. IEEE Trans Circ Syst Video Technol, 29(1):38–49. https://doi.org/10.1109/TCSVT.2017.2773461
Cheng HD, Shi XJ, 2004. A simple and effective histogram equalization approach to image enhancement. Dig Signal Process, 14(2):158–170. https://doi.org/10.1016/j.dsp.2003.07.002
Cho K, van Merriënboer B, Gulcehre C, et al., 2014. Learning phrase representations using RNN encoder-decoder for statistical machine translation. Proc Conf on Empirical Methods in Natural Language Processing, p.1724–1734. https://doi.org/10.3115/v1/D14-1179
Guo CL, Li CY, Guo JC, et al., 2020. Zero-reference deep curve estimation for low-light image enhancement. IEEE/CVF Conf on Computer Vision and Pattern Recognition, p.1780–1789. https://doi.org/10.1109/CVPR42600.2020.00185
Guo XJ, Li Y, Ling HB, 2017. LIME: low-light image enhancement via illumination map estimation. IEEE Trans Image Process, 26(2):982–993. https://doi.org/10.1109/TIP.2016.2639450
Hao SJ, Han X, Guo YR, et al., 2020. Low-light image enhancement with semi-decoupled decomposition. IEEE Trans Multim, 22(12):3025–3038. https://doi.org/10.1109/TMM.2020.2969790
Hochreiter S, Schmidhuber J, 1997. Long short-term memory. Neur Comput, 9(8):1735–1780. https://doi.org/10.1162/neco.1997.9.8.1735
Huang G, Liu Z, van der Maaten L, et al., 2017. Densely connected convolutional networks. IEEE/CVF Conf on Computer Vision and Pattern Recognition, p.2261–2269. https://doi.org/10.1109/CVPR.2017.243
Huang ZX, Li JJ, Hua Z, et al., 2022. Underwater image enhancement via adaptive group attention-based multi-scale cascade transformer. IEEE Trans Instrum Meas, 71:5015618. https://doi.org/10.1109/TIM.2022.3189630
Jiang YF, Gong XY, Liu D, et al., 2021. EnlightenGAN: deep light enhancement without paired supervision. IEEE Trans Image Process, 30:2340–2349. https://doi.org/10.1109/TIP.2021.3051462
Jung E, Yang N, Cremers D, 2020. Multi-frame GAN: image enhancement for stereo visual odometry in low light. Proc 3rd Annual Conf on Robot Learning, p.651–660.
Kingma DP, Ba J, 2014. Adam: a method for stochastic optimization. Proc 3rd Int Conf on Learning Representations.
Lee C, Lee C, Kim CS, 2012. Contrast enhancement based on layered difference representation. 19th IEEE Int Conf on Image Processing, p.965–968. https://doi.org/10.1109/ICIP.2012.6467022
Lee C, Lee C, Kim CS, 2013. Contrast enhancement based on layered difference representation of 2D histograms. IEEE Trans Image Process, 22(12):5372–5384. https://doi.org/10.1109/TIP.2013.2284059
Li CL, Tang SQ, Yan JW, et al., 2020. Low-light image enhancement based on quasi-symmetric correction functions by fusion. Symmetry, 12(9):1561. https://doi.org/10.3390/sym12091561
Li JJ, Feng XM, Hua Z, 2021. Low-light image enhancement via progressive-recursive network. IEEE Trans Circ Syst Video Technol, 31(11):4227–4240. https://doi.org/10.1109/TCSVT.2021.3049940
Li L, Wang RG, Wang WM, et al., 2015. A low-light image enhancement method for both denoising and contrast enlarging. IEEE Int Conf on Image Processing, p.3730–3734. https://doi.org/10.1109/ICIP.2015.7351501
Li MD, Liu JY, Yang WH, et al., 2018. Structure-revealing low-light image enhancement via robust retinex model. IEEE Trans Image Process, 27(6):2828–2841. https://doi.org/10.1109/TIP.2018.2810539
Li PL, Liang JL, Zhang MH, 2021. A degradation model for simultaneous brightness and sharpness enhancement of low-light image. Signal Process, 189:108298. https://doi.org/10.1016/j.sigpro.2021.108298
Lim KL, Jiang XD, Yi CY, 2020. Deep clustering with variational autoencoder. IEEE Signal Process Lett, 27:231–235. https://doi.org/10.1109/LSP.2020.2965328
Liu L, Ouyang WL, Wang XG, et al., 2020. Deep learning for generic object detection: a survey. Int J Comput Vis, 128(2):261–318. https://doi.org/10.1007/s11263-019-01247-4
Liu RS, Ma L, Zhang JA, et al., 2021. Retinex-inspired unrolling with cooperative prior architecture search for low-light image enhancement. IEEE/CVF Conf on Computer Vision and Pattern Recognition, p.10556–10565. https://doi.org/10.1109/CVPR46437.2021.01042
Liu YJ, Wang ZN, Zeng Y, et al., 2021. PD-GAN: perceptual-details GAN for extremely noisy low light image enhancement. IEEE Int Conf on Acoustics, Speech and Signal Processing, p.1840–1844. https://doi.org/10.1109/ICASSP39728.2021.9413433
Loh YP, Chan CS, 2019. Getting to know low-light images with the exclusively dark dataset. Comput Vis Image Underst, 178:30–42. https://doi.org/10.1016/j.cviu.2018.10.010
Lore KG, Akintayo A, Sarkar S, 2017. LLNet: a deep autoencoder approach to natural low-light image enhancement. Patt Recogn, 61:650–662. https://doi.org/10.1016/j.patcog.2016.06.008
Lv FF, Li Y, Lu F, 2021. Attention guided low-light image enhancement with a large scale low-light simulation dataset. Int J Comput Vis, 129(7):2175–2193. https://doi.org/10.1007/s11263-021-01466-8
Ma L, Liu RS, Zhang JA, et al., 2022. Learning deep context-sensitive decomposition for low-light image enhancement. IEEE Trans Neur Netw Learn Syst, 33(10):5666–5680. https://doi.org/10.1109/TNNLS.2021.3071245
Mittal A, Soundararajan R, Bovik AC, 2013. Making a “completely blind” image quality analyzer. IEEE Signal Process Lett, 20(3):209–212. https://doi.org/10.1109/LSP.2012.2227726
Peng T, Su LL, Zhang RH, et al., 2020. A new safe lane-change trajectory model and collision avoidance control method for automatic driving vehicles. Expert Syst Appl, 141:112953. https://doi.org/10.1016/j.eswa.2019.112953
Ren WQ, Liu SF, Ma L, et al., 2019. Low-light image enhancement via a deep hybrid network. IEEE Trans Image Process, 28(9):4364–4375. https://doi.org/10.1109/TIP.2019.2910412
Ren XT, Li MD, Cheng WH, et al., 2018. Joint enhancement and denoising method via sequential decomposition. IEEE Int Symp on Circuits and Systems, p.1–5. https://doi.org/10.1109/ISCAS.2018.8351427
Shiau YH, Chen PY, Yang HY, et al., 2015. A low-cost hardware architecture for illumination adjustment in real-time applications. IEEE Trans Intell Transp Syst, 16(2):934–946. https://doi.org/10.1109/TITS.2014.2347701
Singh H, Kumar A, Balyan LK, et al., 2017. A novel optimally gamma corrected intensity span maximization approach for dark image enhancement. 22nd Int Conf on Digital Signal Processing, p.1–5. https://doi.org/10.1109/ICDSP.2017.8096035
Singh N, Bhandari AK, 2021. Principal component analysis-based low-light image enhancement using reflection model. IEEE Trans Instrum Meas, 70:70:5012710. https://doi.org/10.1109/TIM.2021.3096266
Wang LW, Liu ZS, Siu WC, et al., 2020. Lightening network for low-light image enhancement. IEEE Trans Image Process, 29:7984–7996. https://doi.org/10.1109/TIP.2020.3008396
Wang QL, Wu BG, Zhu PF, et al., 2020. ECA-Net: efficient channel attention for deep convolutional neural networks. IEEE/CVF Conf on Computer Vision and Pattern Recognition, p.11531–11539. https://doi.org/10.1109/CVPR42600.2020.01155
Wang SH, Luo G, 2018. Naturalness preserved image enhancement using a priori multi-layer lightness statistics. IEEE Trans Image Process, 27(2):938–948. https://doi.org/10.1109/TIP.2017.2771449
Wang SH, Zheng J, Hu HM, et al., 2013. Naturalness preserved enhancement algorithm for non-uniform illumination images. IEEE Trans Image Process, 22(9):3538–3548. https://doi.org/10.1109/TIP.2013.2261309
Wang W, Sun N, Ng MK, 2019. A variational gamma correction model for image contrast enhancement. Inv Probl Imag, 13(3):461–478. https://doi.org/10.3934/ipi.2019023
Wang YF, Liu HM, Fu ZW, 2019. Low-light image enhancement via the absorption light scattering model. IEEE Trans Image Process, 28(11):5679–5690. https://doi.org/10.1109/TIP.2019.2922106
Wei C, Wang WJ, Yang WH, et al., 2018. Deep retinex decomposition for low-light enhancement. British Machine Vision Conf, Article 155.
Wu XM, Liu XH, Hiramatsu K, et al., 2017. Contrast-accumulated histogram equalization for image enhancement. IEEE Int Conf on Image Processing, p.3190–3194. https://doi.org/10.1109/ICIP.2017.8296871
Xie EZ, Ding J, Wang WH, et al., 2021. DetCo: unsupervised contrastive learning for object detection. IEEE/CVF Int Conf on Computer Vision, p.8372–8381. https://doi.org/10.1109/ICCV48922.2021.00828
Xu CR, Peng ZZ, Hu XZ, et al., 2020. FPGA-based low-visibility enhancement accelerator for video sequence by adaptive histogram equalization with dynamic clip-threshold. IEEE Trans Circ Syst I Regul Papers, 67(11):3954–3964. https://doi.org/10.1109/TCSI.2020.3010634
Xu K, Yang X, Yin BC, et al., 2020. Learning to restore low-light images via decomposition-and-enhancement. IEEE/CVF Conf on Computer Vision and Pattern Recognition, p.2278–2287. https://doi.org/10.1109/CVPR42600.2020.00235
Xu YD, Yang C, Sun BB, et al., 2021. A novel multi-scale fusion framework for detail-preserving low-light image enhancement. Inform Sci, 548:378–397. https://doi.org/10.1016/j.ins.2020.09.066
Yan XA, Liu Y, Jia MP, 2020a. Multiscale cascading deep belief network for fault identification of rotating machinery under various working conditions. Knowl-Based Syst, 193:105484. https://doi.org/10.1016/j.knosys.2020.105484
Yan XA, Liu Y, Xu YD, et al., 2020b. Multistep forecasting for diurnal wind speed based on hybrid deep learning model with improved singular spectrum decomposition. Energy Conv Manag, 225:113456. https://doi.org/10.1016/j.enconman.2020.113456
Yang B, Cao XL, Yuen C, et al., 2021. Offloading optimization in edge computing for deep-learning-enabled target tracking by Internet of UAVs. IEEE Int Things J, 8(12):9878–9893. https://doi.org/10.1109/JIOT.2020.3016694
Yang WH, Wang WJ, Huang HF, et al., 2021a. Sparse gradient regularized deep retinex network for robust low-light image enhancement. IEEE Trans Image Process, 30:2072–2086. https://doi.org/10.1109/TIP.2021.3050850
Yang WH, Wang SQ, Fang YM, et al., 2021b. Band representation-based semi-supervised low-light image enhancement: bridging the gap between signal fidelity and perceptual quality. IEEE Trans Image Process, 30:3461–3473. https://doi.org/10.1109/TIP.2021.3062184
Ying ZQ, Li G, Ren YR, et al., 2017. A new low-light image enhancement algorithm using camera response model. IEEE Int Conf on Computer Vision Workshops, p.3015–3022. https://doi.org/10.1109/ICCVW.2017.356
Yu SY, Zhu H, 2019. Low-illumination image enhancement algorithm based on a physical lighting model. IEEE Trans Circ Syst Video Technol, 29(1):28–37. https://doi.org/10.1109/TCSVT.2017.2763180
Zamir SW, Arora A, Khan S, et al., 2020. Learning enriched features for real image restoration and enhancement. Proc 16th European Conf on Computer Vision, p.492–511. https://doi.org/10.1007/978-3-030-58595-2_30
Zhang L, Zhang L, Mou XQ, et al., 2011. FSIM: a feature similarity index for image quality assessment. IEEE Trans Image Process, 20(8):2378–2386. https://doi.org/10.1109/TIP.2011.2109730
Zhang TL, Li JJ, Fan H, 2022. Progressive edge-sensing dynamic scene deblurring. Comput Visual Media, 8(3):495–508. https://doi.org/10.1007/s41095-021-0246-4
Zhang YH, Zhang JW, Guo XJ, 2019. Kindling the darkness: a practical low-light image enhancer. Proc 27th ACM Int Conf on Multimedia, p.1632–1640. https://doi.org/10.1145/3343031.3350926
Author information
Authors and Affiliations
Contributions
Jinjiang LI designed the research. Zhixiong HUANG designed the software. Zhen HUA and Linwei FAN processed the data. Zhixiong HUANG drafted the paper. Jinjiang LI helped organize the paper. Zhixiong HUANG and Linwei FAN revised and finalized the paper.
Corresponding author
Ethics declarations
Zhixiong HUANG, Jinjiang LI, Zhen HUA, and Linwei FAN declare that they have no conflict of interest.
Additional information
Project supported by the National Natural Science Foundation of China (Nos. 61772319, 62002200, and 62202268), the Shandong Natural Science Foundation of China (Nos. ZR2021QF134 and ZR2021MF107), the Shandong Provincial Science and Technology Support Program for Youth Innovation Team in Colleges (Nos. 2021KJ069 and 2019KJN042), and the Yantai Science and Technology Innovation Development Plan (No. 2022JCYJ031)
Rights and permissions
About this article
Cite this article
Huang, Z., Li, J., Hua, Z. et al. Filter-cluster attention based recursive network for low-light enhancement. Front Inform Technol Electron Eng 24, 1028–1044 (2023). https://doi.org/10.1631/FITEE.2200344
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1631/FITEE.2200344