Skip to main content
SpringerLink
Log in

Contrast enhancement based on discriminative co-occurrence statistics

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Despite recent advances in contrast enhancement, it remains difficult for existing methods to simultaneously achieve consistent improvements in image brightness and contrast in both low-light and normal-light images. To address this issue, we revisit 2D histogram equalization methods and extend them to accommodate more diversified and poor lighting conditions. The extension is based on the observation that the contrast needs to be improved by increasing the intensity difference between spatially neighboring pixels, while the degree of increase should be proportional to the difference in their reflectances. On the basis of this observation, we propose to embed the inter-pixel contextual information of image reflectance into the 2D histogram of intensity co-occurrence. An intensity mapping function can thus be derived by solving optimization problems formulated with the 2D histogram, leading to two novel contrast enhancement methods. Qualitative and quantitative evaluations on more than 600 images showed that the proposed methods are superior to state-of-the-art contrast enhancement methods. It was also shown that the reflectance of an image provides important visual information on the significance and objectness of local image areas. Using this reflectance as a clue, the degree of contrast enhancement can be adaptively derived to achieve sufficient brightness improvement in low-light images and to avoid excessive enhancement in normal-light images.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18

Similar content being viewed by others

Notes

  1. sipi.usc.edu/database

  2. sites.google.com/site/vonikakis/datasets

References

  1. Agaian SS, Silver B, Panetta KA (2007) Transform coefficient histogram-based image enhancement algorithms using contrast entropy. IEEE Trans Image Process 16(3):741–758

    Article  MathSciNet  Google Scholar 

  2. Arbelaez P, Maire M, Fowlkes CC, Malik J (2011) Contour detection and hierarchical image segmentation. IEEE Trans Pattern Anal Mach Intell 33(5):898–916

    Article  Google Scholar 

  3. Arici T, Dikbas S, Altunbasak Y (2009) A histogram modification framework and its application for image contrast enhancement. IEEE Trans Image Process 18(9):1921–1935

    Article  MathSciNet  Google Scholar 

  4. Balado F (2018) Optimum exact histogram specification. In: ICASSP, pp 1413–1417

  5. Çelik T (2012) Two-dimensional histogram equalization and contrast enhancement. Pattern Recognit 45(10):3810–3824

    Article  Google Scholar 

  6. Çelik T, Tjahjadi T (2011) Contextual and variational contrast enhancement. IEEE Trans Image Process 20(12):3431–3441

    Article  MathSciNet  Google Scholar 

  7. Chen Z, Abidi BR, Page DL, Abidi MA (2006) Gray-level grouping (GLG): an automatic method for optimized image contrast enhancement—part i: the basic method. IEEE Trans Image Process 15(8):2290–2302

    Article  Google Scholar 

  8. Chen Q, Xu J, Koltun V (2017) Fast image processing with fully-convolutional networks. In: ICCV, pp 2516–2525

  9. Chen Y, Wang Y, Kao M, Chuang Y (2018) Deep photo enhancer: unpaired learning for image enhancement from photographs with GANs. In: CVPR, pp 6306–6314

  10. Dabov K, Foi A, Katkovnik V, Egiazarian KO (2007) Image denoising by sparse 3-D transform-domain collaborative filtering. IEEE Trans Image Process 16(8):2080–2095

    Article  MathSciNet  Google Scholar 

  11. Durand F, Dorsey J (2002) Fast bilateral filtering for the display of high-dynamic-range images. ACM Trans Graph 21(3):257–266

    Article  Google Scholar 

  12. Eilertsen G, Mantiuk RK, Unger J (2015) Real-time noise-aware tone mapping. ACM Trans Graph 34(6):198:1–198:15

    Article  Google Scholar 

  13. Farbman Z, Fattal R, Lischinski D, Szeliski R (2008) Edge-preserving decompositions for multi-scale tone and detail manipulation. ACM Trans Graph 27(3):67.1–67.10

    Article  Google Scholar 

  14. Fu X, Zeng D, Huang Y, Zhang X, Ding X (2016) A weighted variational model for simultaneous reflectance and illumination estimation. In: CVPR, pp 2782–2790

  15. Gharbi M, Chen J, Barron JT, Hasinoff SW, Durand F (2017) Deep bilateral learning for real-time image enhancement. ACM Trans Graph 36 (4):118:1–118:12

    Article  Google Scholar 

  16. Guo X, Li Y, Ling H (2017) LIME: low-light image enhancement via illumination map estimation. IEEE Trans Image Process 26(2):982–993

    Article  MathSciNet  Google Scholar 

  17. Hautière N, Tarel JP, Aubert D, Dumont E (2008) Blind contrast enhancement assessment by gradient ratioing at visible edges. Image Anal Stereol 27(2):87–95

    Article  MathSciNet  Google Scholar 

  18. He K, Sun J, Tang X (2013) Guided image filtering. IEEE Trans Pattern Anal Mach Intell 35(6):1397–1409

    Article  Google Scholar 

  19. Hu Y, He H, Xu C, Wang B, Lin S (2018) Exposure: a white-box photo post-processing framework. ACM Trans Graph 37(2):26:1–26:17

    Article  Google Scholar 

  20. Ignatov A, Kobyshev N, Timofte R, Vanhoey K, Gool LV (2017) DSLR-quality photos on mobile devices with deep convolutional networks. In: ICCV, pp 3297–3305

  21. Jobson DJ, Rahman Z, Woodell GA (1997a) A multiscale Retinex for bridging the gap between color images and the human observation of scenes. IEEE Trans Image Process 6(7):965–976

    Article  Google Scholar 

  22. Jobson DJ, Rahman Z, Woodell GA (1997b) Properties and performance of a center/surround Retinex. IEEE Trans Image Process 6(3):451–462

    Article  Google Scholar 

  23. Kinoshita Y, Kiya H (2019) Scene segmentation-based luminance adjustment for multi-exposure image fusion. IEEE Trans Image Process 28(8):4101–4116

    Article  MathSciNet  Google Scholar 

  24. Lecca M (2018) STAR: a segmentation-based approximation of point-based sampling Milano Retinex for color image enhancement. IEEE Trans Image Process 27(12):5802–5812

    Article  MathSciNet  Google Scholar 

  25. Lee C, Lee C, Kim C (2013) Contrast enhancement based on layered difference representation of 2D histograms. IEEE Trans Image Process 22 (12):5372–5384

    Article  Google Scholar 

  26. Li J, Liao X, Hu R, Liu X (2018a) Detectability of the image operation order: upsampling and mean filtering. In: APSIPA, pp 1544–1549

  27. Li M, Liu J, Yang W, Sun X, Guo Z (2018b) Structure-revealing low-light image enhancement via robust Retinex model. IEEE Trans Image Process 27(6):2828–2841

    Article  MathSciNet  Google Scholar 

  28. Liao X, Li K, Zhu X, Liu KJR (2020) Robust detection of image operator chain with two-stream convolutional neural network. IEEE J Sel Top Signal Process 14(5):955–968

    Article  Google Scholar 

  29. Liu X, Tanaka M, Okutomi M (2013) Single-image noise level estimation for blind denoising. IEEE Trans Image Process 22(12):5226–5237

    Article  Google Scholar 

  30. Liu X, Tanaka M, Okutomi M (2014) Practical signal-dependent noise parameter estimation from a single noisy image. IEEE Trans Image Process 23(10):4361–4371

    Article  MathSciNet  Google Scholar 

  31. Mittal A, Soundararajan R, Bovik AC (2013) Making a completely blind image quality analyzer. IEEE Signal Process Lett 20(3):209–212

    Article  Google Scholar 

  32. Park J, Lee J, Yoo D, Kweon IS (2018) Distort-and-recover: color enhancement using deep reinforcement learning. In: CVPR, pp 5928–5936

  33. Pu Y, Siarry P, Chatterjee A, Wang Z, Yi Z, Liu Y, Zhou J, Wang Y (2018) A fractional-order variational framework for Retinex: fractional-order partial differential equation-based formulation for multi-scale nonlocal contrast enhancement with texture preserving. IEEE Trans Image Process 27(3):1214– 1229

    Article  MathSciNet  Google Scholar 

  34. Ramponi G (1998) A cubic unsharp masking technique for contrast enhancement. Signal Process 67(2):211–222

    Article  Google Scholar 

  35. Ren Y, Ying Z, Li TH, Li G (2019) LECARM: low-light image enhancement using the camera response model. IEEE Trans Circ Syst Video Technol 29(4):968–981

    Article  Google Scholar 

  36. Ren X, Yang W, Cheng W, Liu J (2020) LR3M: robust low-light enhancement via low-rank regularized Retinex model. IEEE Trans Image Process 29:5862–5876

    Article  MathSciNet  Google Scholar 

  37. Shen X, Zhou C, Xu L, Jia J (2017) Mutual-structure for joint filtering. Int J Comput Vis 125(1-3):19–33

    Article  MathSciNet  Google Scholar 

  38. Shibata T, Tanaka M, Okutomi M (2016) Gradient-domain image reconstruction framework with intensity-range and base-structure constraints. In: CVPR, pp 2745–2753

  39. Shu X, Wu X (2018) Locally adaptive rank-constrained optimal tone mapping. ACM Trans Graph 37(3):38:1–38:10

    Article  Google Scholar 

  40. Su H, Jung C (2017) Low light image enhancement based on two-step noise suppression. In: ICASSP, pp 1977–1981

  41. Tang M, Xie F, Zhang R, Jiang Z, Bovik AC (2020) A local flatness based variational approach to Retinex. IEEE Trans Image Process 29:7217–7232

    Article  MathSciNet  Google Scholar 

  42. Wang S, Luo G (2018) Naturalness preserved image enhancement using a priori multi-layer lightness statistics. IEEE Trans Image Process 27(2):938–948

    Article  MathSciNet  Google Scholar 

  43. Wang S, Zheng J, Hu H, Li B (2013) Naturalness preserved enhancement algorithm for non-uniform illumination images. IEEE Trans Image Process 22(9):3538–3548

    Article  Google Scholar 

  44. Wang R, Zhang Q, Fu C, Shen X, Zheng W, Jia J (2019) Underexposed photo enhancement using deep illumination estimation. In: CVPR, pp 6849–6857

  45. Wu X, Liu X, Hiramatsu K, Kashino K (2017) Contrast-accumulated histogram equalization for image enhancement. In: ICIP, pp 3190–3194

  46. Xiao B, Tang H, Jiang Y, Li W, Wang G (2018) Brightness and contrast controllable image enhancement based on histogram specification. Neurocomputing 275:2798–2809

    Article  Google Scholar 

  47. Xu J, Hou Y, Ren D, Liu L, Zhu F, Yu M, Wang H, Shao L (2020) STAR: a structure and texture aware Retinex model. IEEE Trans Image Process 29:5022–5037

    Article  Google Scholar 

  48. Xu L, Lu C, Xu Y, Jia J (2011) Image smoothing via L0 gradient minimization. ACM Trans Graph 30(6):174

    Google Scholar 

  49. Xu L, Yan Q, Xia Y, Jia J (2012) Structure extraction from texture via relative total variation. ACM Trans Graph 31(6):139:1–139:10

    Google Scholar 

  50. Xu L, Ren JSJ, Yan Q, Liao R, Jia J (2015) Deep edge-aware filters. In: ICML, pp 1669–1678

  51. Yan Z, Zhang H, Wang B, Paris S, Yu Y (2016) Automatic photo adjustment using deep neural networks. ACM Trans Graph 35(2):11:1–11:15

    Article  Google Scholar 

  52. Zhang Q, Shen X, Xu L, Jia J (2014a) Rolling guidance filter. In: ECCV, pp 815–830

  53. Zhang Q, Xu L, Jia J (2014b) 100+ times faster weighted median filter (WMF). In: CVPR, pp 2830–2837

  54. Zhang Y, Zhang J, Guo X (2019) Kindling the darkness: a practical low-light image enhancer. In: ACM multimedia, pp 1632–1640

Download references

Author information

Authors and Affiliations

  1. Communication Science Laboratories, NTT Corporation, 3-1 Morinosato Wakamiya, Atsugi-shi, Kanagawa, 243-0198, Japan

    X. Wu, T. Kawanishi & K. Kashino

  2. Media Intelligence Laboratories, NTT Corporation, 1-1 Hikarinooka, Yokosuka-shi, Kanagawa, 239-0847, Japan

    Y. Sun

Authors
  1. X. Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Y. Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. T. Kawanishi
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. K. Kashino
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to X. Wu.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wu, X., Sun, Y., Kawanishi, T. et al. Contrast enhancement based on discriminative co-occurrence statistics. Multimed Tools Appl 80, 6413–6442 (2021). https://doi.org/10.1007/s11042-020-09948-6

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-020-09948-6

Keywords

Search

Navigation