Advertisement

Multimedia Tools and Applications

, Volume 78, Issue 22, pp 32137–32158 | Cite as

Nighttime visual refinement techniques for surveillance video: a review

  • Soumya TEmail author
  • Sabu M. Thampi
Article
  • 47 Downloads

Abstract

Video surveillance systems substitute manual efforts in various safety critic domains such as border area, assisted living, banking, service stations, and transportation. The multimedia-based surveillance system has a significant role in security and forensic systems because people tend to be easily convinced after observing voice, image, and video. Hence, these videos are strong evidence in the forensic investigation. However, most of the criminal activities such as ATM robbery and assassination are occur at nighttime because of the crime supporting dark environment. Many of the night surveillance systems in military, as well as commercial applications, are equipped with infrared and thermal based night vision systems. Its poor capability of texture and color interpretations are the major issues to ensure secure nighttime video monitoring. Specifically, visual refinements of nighttime surroundings and foreground objects provide a valuable assistance in the nighttime security system. In this scenario, it is highly recommended a review of the state-of-the-art nighttime visual refinement approaches. We conducted an extensive literature review and classified the nighttime visual refinement approaches into nighttime restoration and enhancement. This comparative literary analysis identified the research gap fields to explore future research directions in nighttime visual enhancement techniques. Finally, we discussed various open issues and future directions in the context enhancement based nighttime enhancement research.

Keywords

Video surveillance Night video enhancement Night video restoration Context enhancement 

Notes

Acknowledgments

The authors would like to thank University of Kerala, LBS Centre for Science and Technology, College of Engineering Trivandrum, College of Engineering Perumon and Centre for Engineering Research and Development for providing research facilities.

References

  1. 1.
    Al-Ameen Z (2019) Nighttime image enhancement using a new illumination boost algorithm. IET Image ProcessGoogle Scholar
  2. 2.
    Benoit A, Caplier A, Durette B, Hérault J (2010) Using human visual system modeling for bio-inspired low level image processing. Comput Vis Image Underst 114(7):758–773CrossRefGoogle Scholar
  3. 3.
    Cai L, Qian J (2009) Night color image enhancement using fuzzy set. In: 2nd International congress on image and Signal Processing, 2009. CISP’09. IEEE, pp 1–4Google Scholar
  4. 4.
    Celik T (2013) Spatio-temporal video contrast enhancement. IET Image Process 7(6):543–555CrossRefGoogle Scholar
  5. 5.
    Chen Y, Lin W, Zhang C, Chen Z, Xu N, Xie J (2013) Intra-and-inter-constraint-based video enhancement based on piecewise tone mapping. IEEE Trans Circuits Syst Video Technol 23(1):74–82CrossRefGoogle Scholar
  6. 6.
    Cheng H-Y, Yu C-C (2014) Nighttime traffic flow analysis for rain-drop tampered cameras. In: 2014 22nd International conference on pattern recognition (ICPR). IEEE, pp 714–719Google Scholar
  7. 7.
    Chouhan R, Biswas PK, Jha RK (2015) Enhancement of low-contrast images by internal noise-induced fourier coefficient rooting. Signal Image Video Process 9(1):255–263CrossRefGoogle Scholar
  8. 8.
    Chouhan R, Jha RK, Biswas PK (2013) Enhancement of dark and low-contrast images using dynamic stochastic resonance. IET Image Process 7(2):174–184MathSciNetCrossRefGoogle Scholar
  9. 9.
    Chouhan R, Jha RK, Biswas PK (2013) Noise-enhanced contrast stretching of dark images in svd-dwt domain. In: 2013 Annual IEEE India conference (INDICON). IEEE, pp 1–6Google Scholar
  10. 10.
    Dong X, Li W, Wang G, Lu Y, Meng W, et al. (2011) An efficient and integrated algorithm for video enhancement in challenging lighting conditions. arXiv:1102.3328
  11. 11.
    Dong X, Wang G, Pang Y, Li W, Wen J, Meng W, Lu Y (2011) Fast efficient algorithm for enhancement of low lighting video. In: 2011 IEEE international conference on multimedia and expo (ICME). IEEE, pp 1–6Google Scholar
  12. 12.
    Fan X, Wang L (2019) Image defogging approach based on incident light frequency, Multimed Tools Appl. [Online]. Available:  https://doi.org/10.1007/s11042-018-7103-1 CrossRefGoogle Scholar
  13. 13.
    Fu X, Zeng D, Huang Y, Liao Y, Ding X, Paisley J (2016) A fusion-based enhancing method for weakly illuminated images. Signal Process 129:82–96CrossRefGoogle Scholar
  14. 14.
    Honda H, Timofte R, Van Gool L (2015) Make my day-high-fidelity color denoising with near-infrared. In: Proceedings of the IEEE conference on computer vision and pattern recognition workshops, pp 82–90Google Scholar
  15. 15.
    Hu J, Hu R, Wang Z, Gong Y, Duan M (2013) Kinect depth map based enhancement for low light surveillance image. In: 2013 20th IEEE International conference on image processing (ICIP). IEEE, pp 1090–1094Google Scholar
  16. 16.
    Hu Y, Shang Y, Fu X, Ding H (2015) A low illumination video enhancement algorithm based on the atmospheric physical model. In: 2015 8th International congress on image and signal processing (CISP). IEEE, pp 119–124Google Scholar
  17. 17.
    Ilie A, Raskar R, Yu J (2005) Gradient domain context enhancement for fixed cameras. Int J Pattern Recognit Artif Intell 19(04):533–549CrossRefGoogle Scholar
  18. 18.
    Jha RK, Chouhan R, Biswas PK, Aizawa K (2012) Internal noise-induced contrast enhancement of dark images. In: 2012 19th IEEE International conference on image processing (ICIP). IEEE, pp 973– 976Google Scholar
  19. 19.
    Jiang X, Yao H, Liu D (2019) Nighttime image enhancement based on image decomposition. SIViP 13(1):189–197CrossRefGoogle Scholar
  20. 20.
    Jiang X, Yao H, Zhang S, Lu X, Zeng W (2013) Night video enhancement using improved dark channel prior. In: 2013 IEEE International conference on image processing. IEEE, pp 553–557Google Scholar
  21. 21.
    Jung C, Yang Q, Sun T, Fu Q, Song H (2017) Low light image enhancement with dual-tree complex wavelet transform. J Vis Commun Image Represent 42:28–36CrossRefGoogle Scholar
  22. 22.
    Kim M, Park D, Han DK, Ko H (2015) A novel approach for denoising and enhancement of extremely low-light video. IEEE Trans Consum Electron 61(1):72–80CrossRefGoogle Scholar
  23. 23.
    Lee S. -W., Maik V, Jang J, Shin J, Paik J (2005) Noise-adaptive spatio-temporal filter for real-time noise removal in low light level images. IEEE Trans Consum Electron 51(2):648–653CrossRefGoogle Scholar
  24. 24.
    Li J, Li SZ, Pan Q, Yang T (2005) Illumination and motion-based video enhancement for night surveillance. In: 2nd Joint IEEE International workshop on visual surveillance and performance evaluation of tracking and surveillance, pp 169–175Google Scholar
  25. 25.
    Li J, Yang T, Pan Q, Cheng Y (2009) Combining scene model and fusion for night video enhancement. J Electron (China) 26(1):88–93CrossRefGoogle Scholar
  26. 26.
    Li Y, Lu J, Wang J, Miao Z, Xu W (2013) Night vision image contrast enhancement base on adaptive dynamic histogram. In: 2013 Fourth international conference on digital manufacturing automation, pp 823–828Google Scholar
  27. 27.
    Li Y, Tan RT, Brown MS (2015) Nighttime haze removal with glow and multiple light colors. In: Proceedings of the IEEE international conference on computer vision, pp 226–234Google Scholar
  28. 28.
    Ling Z, Liang Y, Wang Y, Shen H, Lu X (2015) Adaptive extended piecewise histogram equalisation for dark image enhancement. IET Image Process 9 (11):1012–1019CrossRefGoogle Scholar
  29. 29.
    Łoza A, Bull DR, Hill PR, Achim AM (2013) Automatic contrast enhancement of low-light images based on local statistics of wavelet coefficients. Digital Signal Process 23(6):1856–1866CrossRefGoogle Scholar
  30. 30.
    Makwana I, Zaveri T, Gupta V (2011) Efficient color transfer method based on colormap clustering for night vision applications. In: 2011 Third national conference on computer vision, pattern recognition, image processing and graphics (NCVPRIPG). IEEE, pp 196–199Google Scholar
  31. 31.
    Malm H, Oskarsson M, Warrant E, Clarberg P, Hasselgren J, Lejdfors C (2007) Adaptive enhancement and noise reduction in very low light-level video. In: IEEE 11th International conference on computer vision, 2007. ICCV 2007. IEEE, pp 1–8Google Scholar
  32. 32.
    Meng Y, Kong D, Zhu Z, Zhao Y (2019) From night to day: Gans based low quality image enhancement, Neural Process Lett. [Online]. Available:  https://doi.org/10.1007/s11063-018-09968-2 CrossRefGoogle Scholar
  33. 33.
    Pan J, Sun D, Pfister H, Yang M-H (2016) Blind image deblurring using dark channel prior. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1628–1636Google Scholar
  34. 34.
    Pei S-C, Lee T-Y (2012) Night time haze removal using color transfer pre-processing and dark channel prior. In: 2012 19th IEEE International conference on image processing. IEEE, pp 957–960Google Scholar
  35. 35.
    Quevedo E, De La Cruz J, Callico GM, Tobajas F, Sarmiento R (2014) Video enhancement using spatial and temporal super-resolution from a multi-camera system. IEEE Trans Consum Electron 60(3):420–428CrossRefGoogle Scholar
  36. 36.
    Rao Y, Lin W, Chen L (2010) Image-based fusion for video enhancement of night-time surveillance. Optical Eng 49(12):120 501–120 501CrossRefGoogle Scholar
  37. 37.
    Rao Y, Chen Z, Sun M-T, Hsu Y-F, Zhang Z (2011) An effecive night video enhancement algorithm. In: 2011 IEEE visual communications and image processing (VCIP). IEEE, pp 1–4Google Scholar
  38. 38.
    Rao Y, Hou L, Wang Z, Chen L (2014) Illumination-based nighttime video contrast enhancement using genetic algorithm. Multimed Tools Appl 70(3):2235–2254CrossRefGoogle Scholar
  39. 39.
    Rivera AR, Ryu B, Chae O (2012) Content-aware dark image enhancement through channel division. IEEE Trans Image Process 21(9):3967–3980MathSciNetCrossRefGoogle Scholar
  40. 40.
    Soumya T, Thampi SM (2015) Day color transfer based night video enhancement for surveillance system. In: 2015 IEEE International conference on signal processing, informatics, communication and energy systems (SPICES). IEEE, pp 1–5Google Scholar
  41. 41.
    Soumya T, Thampi SM (2016) Recolorizing dark regions to enhance night surveillance video. Multimed Tools Appl: 1–17Google Scholar
  42. 42.
    Soumya T, Thampi SM (2017) A fuzzy fusion approach to enlighten the illuminated regions of night surveillance videos. J Intell Fuzzy Syst 32(4):3143–3149CrossRefGoogle Scholar
  43. 43.
    Soumya T, Thampi SM (2017) Self-organized night video enhancement for surveillance systems. SIViP 11(1):57–64CrossRefGoogle Scholar
  44. 44.
    Su H, Jung C, Wang L, Wang S, Du Y (2019) Adaptive tone mapping for display enhancement under ambient light using constrained optimization. Displays 56:11–22CrossRefGoogle Scholar
  45. 45.
    Wang Q, Ward RK (2007) Fast image/video contrast enhancement based on weighted thresholded histogram equalization. IEEE Trans Consum Electron 53(2)CrossRefGoogle Scholar
  46. 46.
    Wang W, Chen Z, Yuan X, Wu X (2019) Adaptive image enhancement method for correcting low-illumination images. Inform SciGoogle Scholar
  47. 47.
    Warrant E, Oskarsson M, Malm H (2014) The remarkable visual abilities of nocturnal insects: neural principles and bioinspired night-vision algorithms. Proc IEEE 102(10):1411–1426CrossRefGoogle Scholar
  48. 48.
    Xiao H, Rao Y (2015) Research on deep auto-encoder network for nighttime video enhancement. J Inf Comput Sci 12(10):4125–4136CrossRefGoogle Scholar
  49. 49.
    Xu Q, Jiang H, Scopigno R, Sbert M (2014) A novel approach for enhancing very dark image sequences. Signal Process 103:309–330CrossRefGoogle Scholar
  50. 50.
    Yamasaki A, Takauji H, Kaneko S, Kanade T, Ohki H (2008) Denighting: enhancement of nighttime images for a surveillance camera. In: 19th International conference on pattern recognition (ICPR), pp 1–4Google Scholar
  51. 51.
    Yu J, Liao Q (2010) Color constancy-based visibility enhancement in low-light conditions. In: 2010 International conference on digital image computing: techniques and applications (DICTA). IEEE, pp 441–446Google Scholar
  52. 52.
    Zhang X, Shen P, Luo L, Zhang L, Song J (2012) Enhancement and noise reduction of very low light level images. In: 2012 21st International conference on pattern recognition (ICPR). IEEE, pp 2034–2037Google Scholar
  53. 53.
    Zhang J, Cao Y, Wang Z (2014) Nighttime haze removal based on a new imaging model. In: 2014 IEEE International conference on image processing (ICIP). IEEE, pp 4557–4561Google Scholar
  54. 54.
    Zhang J, Cao Y, Wang Z (2016) Nighttime haze removal with illumination correction. arXiv:1606.01460
  55. 55.
    Zhang Q, Nie Y, Zhang L, Xiao C (2016) Underexposed video enhancement via perception-driven progressive fusion. IEEE Trans Vis Comput Graph 22(6):1773–1785CrossRefGoogle Scholar
  56. 56.
    Zhang C, Shivakumara P, Xue M, Zhu L, Lu T, Pal U (2018) New fusion based enhancement for text detection in night video footage. In: Pacific rim conference on multimedia. Springer, pp 46–56Google Scholar
  57. 57.
    Zhuo L, Hu X, Li J, Zhang J, Li X (2019) A naturalness-preserved low-light enhancement algorithm for intelligent analysis. Chin J Electron 28(2):316–324CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2019

Authors and Affiliations

  1. 1.University of Kerala, College of Engineering PerumonKeralaIndia
  2. 2.Indian Institute of Information Technology and ManagementKeralaIndia

Personalised recommendations