Skip to main content
SpringerLink
Log in

A Universal Foreground Segmentation Technique using Deep-Neural Network

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Background subtraction is generally used for foreground segmentation (moving object detection) from video sequences. Several background subtraction methods have been proposed for visual surveillance applications. However, the existing methods fail in case of real-surveillance challenges such as camouflage, sudden illumination variation, hard shadow, camera-jitter, non-static background, etc. A deep-neural network based background subtraction model is presented for flexible foreground segmentation. In addition to background subtraction model, a novel background modeling technique is also proposed for flexible background subtraction process. The presented deep-neural network architecture performs the background subtraction operation using the non-handcrafted features. The proposed method uses optical-flow details to make use of temporal information. This temporal information and spatial information (from the background image and current processing frame) are used for the training purpose. The model is trained using randomly selected images and its ground truth images from CDnet-2014 dataset. The presented model is evaluated using CDnet-2014 dataset, and it gives significant results compared to the existing background subtraction methods in terms of qualitative and quantitative analyzes.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  1. Allebosch G, Deboeverie F, Veelaert P, Philips W (2015) Efic: edge based foreground background segmentation and interior classification for dynamic camera viewpoints. In: International conference on advanced concepts for intelligent vision systems, pp. 130–141. Springer

  2. Azab MM, Shedeed HA, Hussein AS (2010) A new technique for background modeling and subtraction for motion detection in real-time videos. In: Image processing ICIP, 2010 17th IEEE international conference on, pp. 3453–3456. IEEE

  3. Babaee M, Dinh DT, Rigoll G (2018) A deep convolutional neural network for video sequence background subtraction. Pattern Recogn 76:635–649

    Article  Google Scholar 

  4. Bouwmans T (2011) Recent advanced statistical background modeling for foreground detection-a systematic survey. Recent Patents on Computer Science 4(3):147–176

    Google Scholar 

  5. Bouwmans T, Porikli F, Höferlin B., Vacavant A (2014) Background modeling and foreground detection for video surveillance CRC press

  6. Braham M, Van Droogenbroeck M (2016) Deep background subtraction with scene-specific convolutional neural networks. In: Systems, signals and image processing IWSSIP, 2016 international conference on, pp. 1–4. IEEE

  7. Chang O, Constante P, Gordon A, Singana M (2017) A novel deep neural network that uses space-time features for tracking and recognizing a moving object. Journal of Artificial Intelligence and Soft Computing Research 7(2):125–136

    Article  Google Scholar 

  8. Chen Y, Wang J, Lu H (2015) Learning sharable models for robust background subtraction. In: Multimedia and expo ICME, 2015 IEEE international conference on, pp. 1–6. IEEE

  9. Chiranjeevi P, Sengupta S (2017) Interval-valued model level fuzzy aggregation-based background subtraction. IEEE transactions on cybernetics 47 (9):2544–2555

    Article  Google Scholar 

  10. Culibrk D, Marques O, Socek D, Kalva H, Furht B (2007) Neural network approach to background modeling for video object segmentation. IEEE Transactions on Neural Networks 18(6):1614–1627

    Article  Google Scholar 

  11. De Gregorio M, Giordano M (2016) Wisardrp for change detection in video sequences Submitted to CVPR’16 Google Scholar

  12. Fu K, Zhao Q, Gu IYH, Yang J (2019) Deepside: a general deep framework for salient object detection. Neurocomputing 356:69–82

    Article  Google Scholar 

  13. Gao F, Wu T, Li J, Zheng B, Ruan L, Shang D, Patel B (2018) Sd-cnn: a shallow-deep cnn for improved breast cancer diagnosis. Comput Med Imaging Graph 70:53–62

    Article  Google Scholar 

  14. Ghosh S, Pal A, Jaiswal S, Santosh K, Das N, Nasipuri M (2019) Segfast-v2: Semantic image segmentation with less parameters in deep learning for autonomous driving. International Journal of Machine Learning and Cybernetics 10 (11):3145–3154

    Article  Google Scholar 

  15. He K, Zhang X, Ren S, Sun J (2015) Delving deep into rectifiers: Surpassing human-level performance on imagenet classification. In: Proceedings of the IEEE international conference on computer vision, pp. 1026–1034

  16. Hu G, Peng X, Yang Y, Hospedales TM, Verbeek J (2017) Frankenstein: Learning deep face representations using small data. IEEE Trans Image Process 27 (1):293–303

    Article  MathSciNet  Google Scholar 

  17. Hu W, Tan T, Wang L, Maybank S (2004) A survey on visual surveillance of object motion and behaviors. IEEE T Syst Man Cy C 34(3):334–352

    Article  Google Scholar 

  18. Jain SD, Xiong B, Grauman K (2017) Fusionseg:, Learning to combine motion and appearance for fully automatic segmention of generic objects in videos, vol 2. arXiv:1701.05384

  19. Jian X, Xiao-qing D, Sheng-jin W, You-shou W (2008) Background subtraction based on a combination of texture, color and intensity. In: Signal processing, 2008 ICSP 2008. 9th international conference on, pp. 1400–1405 IEEE

  20. Jiang S, Lu X (2017) Wesambe: A weight-sample-based method for background subtraction IEEE Transactions on Circuits and Systems for Video Technology

  21. Kang K, Li H, Yan J, Zeng X, Yang B, Xiao T, Zhang C, Wang Z, Wang R, Wang X, et al. (2017) T-cnn: tubelets with convolutional neural networks for object detection from videos IEEE Transactions on Circuits and Systems for Video Technology

  22. Khaire P, Kumar P, Imran J (2018) Combining cnn streams of rgb-d and skeletal data for human activity recognition. Pattern Recogn Lett 115:107–116

    Article  Google Scholar 

  23. Khatami A, Babaie M, Tizhoosh HR, Khosravi A, Nguyen T, Nahavandi S (2018) A sequential search-space shrinking using cnn transfer learning and a radon projection pool for medical image retrieval. Expert Syst Appl 100:224–233

    Article  Google Scholar 

  24. Kim IS, Choi HS, Yi KM, Choi JY, Kong SG (2010) Intelligent visual surveillance—a survey International Journal of Control. Automation and Systems 8 (5):926–939

    Article  Google Scholar 

  25. Kim K, Chalidabhongse TH, Harwood D, Davis L (2005) Real-time foreground–background segmentation using codebook model. Real-time imaging 11 (3):172–185

    Article  Google Scholar 

  26. Kingma DP, Ba J (2014) Adam:, A method for stochastic optimization. arXiv:1412.6980

  27. Krungkaew R, Kusakunniran W (2016) Foreground segmentation in a video by using a novel dynamic codebook. In: Electrical engineering/electronics, computer, telecommunications and information technology ECTI - CON , 2016 13th international conference on, pp. 1–6. IEEE

  28. Lim LA, Keles HY (2018) Foreground segmentation using a triplet convolutional neural network for multiscale feature encoding. arXiv:1801.02225

  29. Maddalena L, Petrosino A (2010) A fuzzy spatial coherence-based approach to background/foreground separation for moving object detection. Neural Comput & Applic 19(2):179–186

    Article  Google Scholar 

  30. Maddalena L, Petrosino A, et al. (2008) A self-organizing approach to background subtraction for visual surveillance applications. IEEE Trans Image Process 17(7):1168

    Article  MathSciNet  Google Scholar 

  31. Martins I, Carvalho P, Corte-Real L, Alba-Castro JL (2017) Bmog: boosted gaussian mixture model with controlled complexity. In: Iberian conference on pattern recognition and image analysis, pp. 50–57 Springer

  32. Pathak D, Girshick R, Dollár P., Darrell T, Hariharan B (2017) Learning features by watching objects move. In: Computer vision and pattern recognition CVPR

  33. Paul M, Haque SM, Chakraborty S (2013) Human detection in surveillance videos and its applications-a review. EURASIP J Adv Sig Pr 2013(1):176

    Article  Google Scholar 

  34. Radenović F., Tolias G, Chum O (2018) Fine-tuning cnn image retrieval with no human annotation. IEEE transactions on pattern analysis and machine intelligence 41(7):1655–1668

    Article  Google Scholar 

  35. Ramírez-Alonso G, Chacón-Murguía MI (2016) Auto-adaptive parallel som architecture with a modular analysis for dynamic object segmentation in videos. Neurocomputing 175:990–1000

    Article  Google Scholar 

  36. Ruder S (2016) An overview of gradient descent optimization algorithms. arXiv:1609.04747

  37. Sajid H, Cheung SCS (2015) Background subtraction for static & moving camera. In: Image processing ICIP, 2015 IEEE international conference on, pp. 4530–4534. IEEE

  38. Seeliger K, Fritsche M, Güċlü U., Schoenmakers S, Schoffelen JM, Bosch S, van Gerven M (2018) Convolutional neural network-based encoding and decoding of visual object recognition in space and time. NeuroImage 180:253–266

    Article  Google Scholar 

  39. Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv:1409.1556

  40. St-Charles PL, Bilodeau GA, Bergevin R (2015) A self-adjusting approach to change detection based on background word consensus. In: 2015 IEEE Winter conference on applications of computer vision WACV, pp. 990–997. IEEE

  41. St-Charles PL, Bilodeau GA, Bergevin R (2015) Subsense: a universal change detection method with local adaptive sensitivity. IEEE Trans Image Process 24 (1):359–373

    Article  MathSciNet  Google Scholar 

  42. St-Charles PL, Bilodeau GA, Bergevin R (2016) Universal background subtraction using word consensus models. IEEE Trans Image Process 25(10):4768–4781

    Article  MathSciNet  Google Scholar 

  43. Stauffer C, Grimson WEL (1999) Adaptive background mixture models for real-time tracking. In: Computer vision and pattern recognition, 1999. IEEE computer society conference on., vol. 2, pp. 246–252 IEEE

  44. Varghese A, Sreelekha G (2017) Sample-based integrated background subtraction and shadow detection. IPSJ Transactions on Computer Vision and Applications 9(1):25

    Article  Google Scholar 

  45. Vijayan M, Mohan R (2018) A novel foreground segmentation method using convolutional neural network. In: International conference on recent trends in image processing and pattern recognition, pp. 27–40. Springer

  46. Wang K, Gou C, Wang FY (2018) M4cd: a robust change detection method for intelligent visual surveillance. IEEE Access 6:15505–15520

    Article  Google Scholar 

  47. Wang N, Yeung DY (2013) Learning a deep compact image representation for visual tracking. In: Advances in neural information processing systems, pp. 809–817

  48. Wang Y, Jodoin PM, Porikli F, Konrad J, Benezeth Y, Ishwar P (2014) Cdnet 2014: an expanded change detection benchmark dataset. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 387–394

  49. Wang Y, Luo Z, Jodoin PM (2017) Interactive deep learning method for segmenting moving objects. Pattern Recogn Lett 96:66–75

    Article  Google Scholar 

  50. Yang L, Li J, Luo Y, Zhao Y, Cheng H, Li J (2018) Deep background modeling using fully convolutional network. IEEE Trans Intell Transp Syst 19(1):254–262

    Article  Google Scholar 

  51. Zeng Z, Jia J, Yu D, Chen Y, Zhu Z (2017) Pixel modeling using histograms based on fuzzy partitions for dynamic background subtraction. IEEE Trans Fuzzy Syst 25(3):584–593

    Article  Google Scholar 

Download references

Acknowledgements

The authors would like to thank CDnet-2014 [48] benchmark for providing the segmentation results and quantitative results of few existing methods, which helped us to perform qualitative and quantitative comparisons.

Author information

Authors and Affiliations

  1. Department of Computer Science and Engineering, National Institute of Technology, Tiruchirappalli, 620015, India

    Midhula Vijayan & R Mohan

Authors
  1. Midhula Vijayan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. R Mohan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Midhula Vijayan.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Vijayan, M., Mohan, R. A Universal Foreground Segmentation Technique using Deep-Neural Network. Multimed Tools Appl 79, 34835–34850 (2020). https://doi.org/10.1007/s11042-020-08977-5

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-020-08977-5

Keywords

Search

Navigation