Skip to main content
SpringerLink
Log in

Recommendations for evaluating the performance of background subtraction algorithms for surveillance systems

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Background subtraction is a prerequisite for a wide range of applications, including video surveillance systems. A significant number of algorithms are often developed and published in different publication mediums in the area, such as workshops, symposiums, conferences, and journals. An important task in presenting a new background subtraction algorithms is to clearly show that its performance outperforms the performance of the state-of-the-art algorithms. In this paper, we present recommendations on how to evaluate the performance of background subtraction algorithms for surveillance systems. We identified, through a systematic mapping, the key steps and components of this evaluation process – procedures, methods, and tools – most used by the authors in each of these steps. Considering this statistical analysis, we perform a theoretical analysis of the most used key components to identify their pros and cons. Then, we define a set of recommendations that aim to standardize and clarify the performance evaluation process of a new background subtraction algorithm.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18

Similar content being viewed by others

References

  1. Ahn JH (2014) Fast adaptive robust subspace tracking for online background subtraction. In: Proceedings of international conference on pattern recognition, IEEE, pp 2555–2559, https://doi.org/10.1109/ICPR.2014.441, (to appear in print)

  2. Akilan T, Jonathan Wu QM, Jiang W, Safaei A, Huo J (2018) New trend in video foreground detection using deep learning. In: 2018 IEEE 61St international midwest symposium on circuits and systems (MWSCAS), pp 889–892, https://doi.org/10.1109/MWSCAS.2018.8623825, (to appear in print)

  3. Akilan T, Wu QJ, Safaei A, Huo J, Yang Y (2020) A 3D CNN-LSTM-based image-to-image foreground segmentation. IEEE Trans Intell Transp Syst 21(3):959–971. https://doi.org/10.1109/TITS.2019.2900426

    Article  Google Scholar 

  4. Alvar M, Rodriguez-Calvo A, Sanchez-Miralles A, Arranz A (2014) Mixture of merged gaussian algorithm using RTDENN. Mach Vis Appl 25(5):1133–1144. https://doi.org/10.1007/s00138-013-0550-9

    Article  Google Scholar 

  5. Alvarez-Meza AM, Molina-Giraldo S, Castellanos-Dominguez G (2014) Correntropy-based adaptive learning to support video surveillance systems. In: Proceedings of international conference on pattern recognition, IEEE, pp 2590–2595, https://doi.org/10.1109/ICPR.2014.447, (to appear in print)

  6. Azzam R, Kemouche MS, Aouf N, Richardson M (2016) Efficient visual object detection with spatially global Gaussian mixture models and uncertainties. J Vis Commun Image Represent 36:90–106. https://doi.org/10.1016/j.jvcir.2015.11.009

    Article  Google Scholar 

  7. Babaee M, Dinh DT, Rigoll G (2017) A deep convolutional neural network for video sequence background subtraction. Pattern Recogn 76:635–649. https://doi.org/10.1016/j.patcog.2017.09.040

    Article  Google Scholar 

  8. Balcilar M, Sonmez AC (2016) Background estimation method with incremental iterative Re-weighted least squares. SIViP 10(1):85–92. https://doi.org/10.1007/s11760-014-0705-9

    Article  Google Scholar 

  9. Barnich O, Van Droogenbroeck M (2011) Vibe: a universal background subtraction algorithm for video sequences. IEEE Trans Image Process 20 (6):1709–1724. https://doi.org/10.1109/TIP.2010.2101613

    Article  MathSciNet  MATH  Google Scholar 

  10. Berjón D, Cuevas C, Morán F, García N (2018) Real-time nonparametric background subtraction with tracking-based foreground update. Pattern Recogn 74:156–170. https://doi.org/10.1016/j.patcog.2017.09.009

    Article  Google Scholar 

  11. Bispo J, Reis L, Cardoso JMP (2015) Techniques for efficient matlab-to-c compilation. In: Proceedings of the 2nd ACM SIGPLAN international workshop on libraries, languages, and compilers for array programming, ARRAY 2015, association for Computing machinery, New York, NY, USA, https://doi.org/10.1145/2774959.2774961, (to appear in print)

  12. Bloisi DD, Pennisi A, Iocchi L (2014) Background modeling in the maritime domain. Mach Vis Appl 25(5):1257–1269. https://doi.org/10.1007/s00138-013-0554-5

    Article  Google Scholar 

  13. Bouwmans T (2011) Recent advanced statistical background modeling for foreground detection - a systematic survey. Recent Patents Comput Sci 4(3):147–176. https://doi.org/10.2174/2213275911104030147

    Article  Google Scholar 

  14. Bouwmans T, Javed S, Sultana M, Jung SK (2019) Deep neural network concepts for background subtraction:a systematic review and comparative evaluation. Neural Netw 117:8–66. https://doi.org/10.1016/j.neunet.2019.04.024

    Article  Google Scholar 

  15. Bouwmans T, Porikli F, Hferlin B, Vacavant A (2014) Background modeling and foreground detection for video surveillance, 1st edn. Chapman & hall/CRC, UK

    Book  Google Scholar 

  16. Braham M, Van Droogenbroeck M (2016) Deep background subtraction with scene-specific convolutional neural networks. In: 2016 international conference on systems, signals and image processing (IWSSIP), pp 1–4, https://doi.org/10.1109/IWSSIP.2016.7502717, (to appear in print)

  17. Cao W, Wang Y, Sun J, Meng D, Yang C, Cichocki A, Xu Z (2016) Total variation regularized tensor RPCA for background subtraction from compressive measurements. IEEE Trans Image Process 25(9):4075–4090. https://doi.org/10.1109/TIP.2016.2579262

    Article  MathSciNet  MATH  Google Scholar 

  18. Chan KL (2015) Detection of foreground in dynamic scene via two-step background subtraction. Mach Vis Appl 26(6):723–740. https://doi.org/10.1007/s00138-015-0696-8

    Article  Google Scholar 

  19. Chan KL (2018) Segmentation of moving objects in image sequence based on perceptual similarity of local texture and photometric features. Eurasip J Image Video Process 2018(1), https://doi.org/10.1186/s13640-018-0308-4

  20. Chen ATY, Biglari-Abhari M, Wang KI (2018) SuperBE: computationally light background estimation with superpixels. J Real-Time Image Proc 7:1–17. https://doi.org/10.1007/s11554-018-0750-7

    Article  Google Scholar 

  21. Chen BH, Huang SC, Yen JY (2018) Counter-propagation artificial neural network-based motion detection algorithm for static-camera surveillance scenarios. Neurocomputing 273:481–493. https://doi.org/10.1016/j.neucom.2017.08.002

    Article  Google Scholar 

  22. Chen X, Xi C, Cao J (2015) Research on moving object detection based on improved mixture Gaussian model. Optik 126(20):2256–2259. https://doi.org/10.1016/j.ijleo.2015.05.122

    Article  Google Scholar 

  23. Chen Z, Ellis T (2014) A self-adaptive Gaussian mixture model. Comput Vis Image Underst 122:35–46. https://doi.org/10.1016/j.cviu.2014.01.004

    Article  Google Scholar 

  24. Chiu WY, Tsai DM (2014) Dual-mode detection for foreground segmentation in low-contrast video images. J Real-Time Image Proc 9(4):647–659. https://doi.org/10.1007/s11554-011-0240-7

    Article  Google Scholar 

  25. Cocorullo G, Corsonello P, Frustaci F, Guachi-Guachi LA, Perri S (2016) Multimodal background subtraction for high-performance embedded systems. J Real-Time Image Process pp 1–17, https://doi.org/10.1007/s11554-016-0651-6

  26. Dou J, Li J (2014) Modeling the background and detecting moving objects based on Sift flow. Optik 125(1):435–440. https://doi.org/10.1016/j.ijleo.2013.06.079

    Article  Google Scholar 

  27. Dou J, Li J, Qin Q, Tu Z (2015) Moving object detection based on incremental learning low rank representation and spatial constraint. Neurocomputing 168:382–400. https://doi.org/10.1016/j.neucom.2015.05.088

    Article  Google Scholar 

  28. Dou J, Qin Q, Tu Z (2017) Background subtraction based on circulant matrix. SIViP 11(3):407–414. https://doi.org/10.1007/s11760-016-0975-5

    Article  Google Scholar 

  29. Duan L, Hu X (2020) Multiscale refinement network for water-body segmentation in high-resolution satellite imagery. IEEE Geosci Remote Sens Lett 17 (4):686–690. https://doi.org/10.1109/LGRS.2019.2926412

    Article  Google Scholar 

  30. Elgammal A (2014) Background subtraction: theory and practice. Morgan & Claypool Publishers, New York

    Google Scholar 

  31. Elgammal A, Duraiswami R, Harwood D, Davis LS (2002) Background and foreground modeling using nonparametric kernel density estimation for visual surveillance. Proc IEEE 90(7):1151–1163. https://doi.org/10.1109/JPROC.2002.801448

    Article  Google Scholar 

  32. Elgammal A, Harwood D, Davis L (2000) Non-parametric model for background subtraction. In: Vernon D. (ed) Computer vision — ECCV 2000. Springer, Berlin, pp 751–767, https://doi.org/10.1007/3-540-45053-X_48, (to appear in print)

  33. Elguebaly T, Bouguila N (2014) Background subtraction using finite mixtures of asymmetric Gaussian distributions and shadow detection. Mach Vis Appl 25(5):1145–1162. https://doi.org/10.1007/s00138-013-0568-z

    Article  Google Scholar 

  34. Elharrouss O, Moujahid D, Tairi H (2015) Motion detection based on the combining of the background subtraction and the structure–texture decomposition. Optik 126(24):5992–5997. https://doi.org/10.1016/j.ijleo.2015.08.084

    Article  Google Scholar 

  35. Erichson NB, Donovan C (2016) Randomized low-rank dynamic mode decomposition for motion detection. Comput Vis Image Underst 146:40–50. https://doi.org/10.1016/j.cviu.2016.02.005

    Article  Google Scholar 

  36. Ferryman J, Shahrokni A (2009) Pets2009: dataset and challenge. In: 2009 Twelfth IEEE international workshop on performance evaluation of tracking and surveillance, pp 1–6, https://doi.org/10.1109/PETS-WINTER.2009.5399556, (to appear in print)

  37. Gao Z, Zhang H, Dong S, Sun S, Wang X, Yang G, Wu W, Li S, de Albuquerque VHC (2020) Salient object detection in the distributed cloud-edge intelligent network. IEEE Netw 34(2):216–224. https://doi.org/10.1109/MNET.001.1900260

    Article  Google Scholar 

  38. Ge W, Dong Y, Guo Z, Chen Y (2014) Background subtraction with dynamic noise sampling and complementary learning. In: 2014 22nd international conference on pattern recognition, IEEE, pp 2341–2346, https://doi.org/10.1109/ICPR.2014.406, (to appear in print)

  39. Ge W, Guo Z, Dong Y, Chen Y (2016) Dynamic background estimation and complementary learning for pixel-wise foreground/background segmentation. Pattern Recogn pp 112–125, https://doi.org/10.1016/j.patcog.2016.01.031

  40. Gemignani G, Rozza A (2016) A robust approach for the background subtraction based on multi-layered self-organizing maps. IEEE Trans Image Process 25(11):5239–5251. https://doi.org/10.1109/TIP.2016.2605004

    Article  MathSciNet  MATH  Google Scholar 

  41. Gonzalez RC, Woods RE (2001) Digital image processing, 2nd edn. Addison-Wesley Longman Publishing Co., Inc., USA

    Google Scholar 

  42. Goyette N, Jodoin PM, Porikli F, Konrad J, Ishwar P (2012) Changedetection.net: a new change detection benchmark dataset. In: 2012 IEEE Computer society conference on computer vision and pattern recognition workshops, pp 1–8, https://doi.org/10.1109/CVPRW.2012.6238919, (to appear in print)

  43. Gregorio MD, Giordano M (2014) Change detection with weightless neural networks. In: 2014 IEEE conference on computer vision and pattern recognition workshops, pp 409–413, https://doi.org/10.1109/CVPRW.2014.66, (to appear in print)

  44. Guo C, Liu D, Guo Y, Sun Y (2014) An adaptive graph cut algorithm for video moving objects detection. Multimed Tools Appl 72(3):2633–2652. https://doi.org/10.1007/s11042-013-1566-x

    Article  Google Scholar 

  45. Heikkila M, Pietikainen M (2006) A texture-based method for modeling the background and detecting moving objects. IEEE Trans Pattern Anal Mach Intell 28(4):657–662. https://doi.org/10.1109/TPAMI.2006.68

    Article  Google Scholar 

  46. Hernandez-Lopez FJ, Rivera M (2014) Change detection by probabilistic segmentation from monocular view. Mach Vis Appl 25(5):1175–1195. https://doi.org/10.1007/s00138-013-0564-3

    Article  Google Scholar 

  47. Hofmann M, Tiefenbacher P, Rigoll G (2012) Background segmentation with feedback: the pixel-based adaptive segmenter. In: 2012 IEEE Computer society conference on computer vision and pattern recognition workshops, pp 38–43, https://doi.org/10.1109/CVPRW.2012.6238925, (to appear in print)

  48. Holtzhausen PJ, Crnojevic V, Herbst BM (2015) An illumination invariant framework for real-time foreground detection. J Real-Time Image Proc 10(2):423–433. https://doi.org/10.1007/s11554-012-0287-0

    Article  Google Scholar 

  49. Huynh-The T, Banos O, Lee S, Kang BH, Kim E, Le-Tien T (2017) NIC: A robust background extraction algorithm for foreground detection in dynamic scenes. IEEE Trans Circ Syst Video Technol 27(1-2):1478–1490. https://doi.org/10.1109/TCSVT.2016.2543118

    Article  Google Scholar 

  50. Jeeva S, Sivabalakrishnan M (2019) Twin background model for foreground detection in video sequence. Clust Comput 22:11659–11668. https://doi.org/10.1007/s10586-017-1446-7

    Article  Google Scholar 

  51. Jeyabharathi D (2018) Dejey: cut set-based dynamic key frame selection and adaptive layer-based background modeling for background subtraction. J Vis Commun Image Represent 55(1):434–446. https://doi.org/10.1016/j.jvcir.2018.06.024

    Article  Google Scholar 

  52. Jeyabharathi D, Dejey D (2016) A novel rotational symmetry dynamic texture (rsdt) based sub space construction and scd (similar-congruent-dissimilar) based scoring model for background subtraction in real time videos. Multimed Tools Appl 75(24):17617–17645. https://doi.org/10.1007/s11042-016-3772-9

    Article  Google Scholar 

  53. Ji Z, Wang W (2014) Detect foreground objects via adaptive fusing model in a hybrid feature space. Pattern Recogn 47(9):2952–2961. https://doi.org/10.1016/j.patcog.2014.03.016

    Article  Google Scholar 

  54. Jian M, Lam KM, Dong J (2014) Illumination-insensitive texture discrimination based on illumination compensation and enhancement. Inf Sci 269:60–72. https://doi.org/10.1016/j.ins.2014.01.019

    Article  MathSciNet  Google Scholar 

  55. Jian M, Yin Y, Dong J, Zhang W (2018) Comprehensive assessment of non-uniform illumination for 3D heightmap reconstruction in outdoor environments. Comput Ind 99:110–118. https://doi.org/10.1016/j.compind.2018.03.034

    Article  Google Scholar 

  56. Kaewtrakulpong P, Bowden R (2002) An improved adaptive background mixture model for real-time tracking with shadow detection. Springer, Boston, pp 135–144

    Google Scholar 

  57. Karadag OO, Erdaş O. (2018) Evaluation of the robustness of deep features on the change detection problem. In: 2018 26Th signal processing and communications applications conference (SIU), pp 1–4, https://doi.org/10.1109/SIU.2018.8404636, (to appear in print)

  58. Kermani E, Asemani D (2014) A robust adaptive algorithm of moving object detection for video surveillance. Eurasip J Image Video Process 2014:1–9. https://doi.org/10.1186/1687-5281-2014-27

    Article  Google Scholar 

  59. Khraief C, Benzarti F, Amiri H (2020) Elderly fall detection based on multi-stream deep convolutional networks. Multimed Tools Appl 1:1–24. https://doi.org/10.1007/s11042-020-08812-x

    Article  Google Scholar 

  60. Kim K., Chalidabhongse T.H., Harwood D., Davis L. (2005) Real-time foreground–background segmentation using codebook model. Real-Time Imaging 11(3):172–185. https://doi.org/10.1016/j.rti.2004.12.004. Special issue on video object processing

    Article  Google Scholar 

  61. Kim W (2018) Background subtraction with variable illumination in outdoor scenes. Multimed Tools Appl 77(15):19439–19454. https://doi.org/10.1007/s11042-017-5410-6

    Article  Google Scholar 

  62. Kryjak T, Komorkiewicz M, Gorgon M (2014) Real-time background generation and foreground object segmentation for high-definition colour video stream in FPGA device. J Real-Time Image Proc 9(1):61–77. https://doi.org/10.1007/s11554-012-0290-5

    Article  Google Scholar 

  63. Kushwaha AKS, Srivastava R (2016) Automatic moving object segmentation methods under varying illumination conditions for video data: comparative study, and an improved method. Multimed Tools Appl 75(23):16209–16264. https://doi.org/10.1007/s11042-015-2927-4

    Article  Google Scholar 

  64. Lee S, Lee C (2014) Low-complexity background subtraction based on spatial similarity. Eurasip J Image Video Process 2014(1):1–16. https://doi.org/10.1186/1687-5281-2014-30

    Article  Google Scholar 

  65. Li L, Huang W, Gu IYH, Tian Q (2004) Statistical modeling of complex backgrounds for foreground object detection. Trans Img Proc 13(11):1459–1472. https://doi.org/10.1109/TIP.2004.836169

    Article  Google Scholar 

  66. Li X, Li G, Huang Q, Wang Z, Yu Z (2018) An adaptive background extraction method in traffic scenes. Optik 156(1):659–671. https://doi.org/10.1016/j.ijleo.2017.11.174

    Article  Google Scholar 

  67. Liang D, Kaneko S, Hashimoto M, Iwata K, Zhao X (2015) Co-occurrence probability-based pixel pairs background model for robust object detection in dynamic scenes. Pattern Recogn 48(4):1374–1390. https://doi.org/10.1016/j.patcog.2014.10.020

    Article  Google Scholar 

  68. Lim K, Jang W, Kim C (2017) Background subtraction using encoder-decoder structured convolutional neural network. In: 2017 14th IEEE international conference on advanced video and signal based surveillance (AVSS), pp 1–6, https://doi.org/10.1109/AVSS.2017.8078547, (to appear in print)

  69. Lin L, Xu Y, Liang X, Lai J (2014) Complex background subtraction by pursuing dynamic spatio-temporal models. IEEE Trans Image Process 23(7):3191–3202. https://doi.org/10.1109/TIP.2014.2326776

    Article  MathSciNet  MATH  Google Scholar 

  70. Lin Z, Chen M, Ma Y (2010) The augmented lagrange multiplier method for exact recovery of corrupted low-rank matrices, arXiv:1009.5055, Accessed 2 Sep 2019

  71. Ling Q, Yan J, Li F, Zhang Y (2014) A background modeling and foreground segmentation approach based on the feedback of moving objects in traffic surveillance systems. Neurocomputing 133:32–45. https://doi.org/10.1016/j.neucom.2013.11.034

    Article  Google Scholar 

  72. López-Rubio FJ, López-Rubio E (2015) Features for stochastic approximation based foreground detection. Comput Vis Image Underst 133:30–50. https://doi.org/10.1016/j.cviu.2014.12.007

    Article  Google Scholar 

  73. Luo J, Wang J, Xu H, Lu H (2016) Real-time people counting for indoor scenes. Signal Process 124:27–35. https://doi.org/10.1016/j.sigpro.2015.10.036. Big data meets multimedia analytics

    Article  Google Scholar 

  74. Ma M, Hu R, Chen S, Xiao J, Wang Z (2018) Robust background subtraction method via low-rank and structured sparse decomposition. China Commun 15(7):156–167. https://doi.org/10.1109/CC.2018.8424611

    Article  Google Scholar 

  75. Maddalena L, Petrosino A (2008) A self-organizing approach to background subtraction for visual surveillance applications. IEEE Trans Image Process 17(7):1168–1177. https://doi.org/10.1109/TIP.2008.924285

    Article  MathSciNet  Google Scholar 

  76. Maddalena L, Petrosino A (2010) A fuzzy spatial coherence-based approach to background/foreground separation for moving object detection. Neural Comput. Appl. 19(2):179–186. https://doi.org/10.1007/s00521-009-0285-8

    Article  Google Scholar 

  77. Maddalena L, Petrosino A (2012) The sobs algorithm: what are the limits?. In: 2012 IEEE computer society conference on computer vision and pattern recognition workshops, pp 21–26, https://doi.org/10.1109/CVPRW.2012.6238922, (to appear in print)

  78. Maddalena L, Petrosino A (2014) The 3dSOBS+ algorithm for moving object detection. Comput Vis Image Underst 122:65–73. https://doi.org/10.1016/j.cviu.2013.11.006

    Article  Google Scholar 

  79. Microsoft Corporation (2019) Test images for wallflower paper. https://www.microsoft.com/en-us/download/details.aspx?id=54651, Accessed 9 Aug 2019

  80. Nakagawa E, Scannavino K, Fabbri S, Ferrari F (2017) Revisáo sistemática da Literatura em Engenharia de software: Teoria e prática. Elsevier Editora Ltda, New York

    Google Scholar 

  81. Nguyen TP, Pham CC, Ha SVU, Jeon JW Change detection by training a triplet network for motion feature extraction. IEEE Trans Circ Syst Video Technol pp 1–14. https://doi.org/10.1109/TCSVT.2018.2795657

  82. OpenCV team (2019) OpenCV. https://opencv.org/, Accessed 26 Aug 2019

  83. Pal S, Petrosino A, Maddalena L (2012) Handbook on soft computing for video surveillance. CRC Press, USA

    Book  Google Scholar 

  84. Panda DK, Meher S (2018) A new Wronskian change detection model based codebook background subtraction for visual surveillance applications. J Vis Commun Image Represent 56:52–72. https://doi.org/10.1016/j.jvcir.2018.07.014

    Article  Google Scholar 

  85. Parsa AB, Movahedi A, Taghipour H, Derrible S, Mohammadian AK (2020) Toward safer highways, application of xgboost and shap for real-time accident detection and feature analysis. Accident Anal Prevent 105405:136. https://doi.org/10.1016/j.aap.2019.105405

    Article  Google Scholar 

  86. Petersen K, Feldt R, Mujtaba S, Mattsson M (2008) Systematic mapping studies in software engineering. In: Proceedings of the 12th international conference on evaluation and assessment in software engineering, EASE’08, pp. 68–77. British computer society

  87. Qin L, Sheng B, Lin W, Wu W, Shen R (2015) GPU-accelerated video background subtraction using gabor detector. J Vis Commun Image Represent 32:1–9. https://doi.org/10.1016/j.jvcir.2015.07.010

    Article  Google Scholar 

  88. Quach KG, Duong CN, Luu K, Bui TD (2017) Non-convex online robust PCA: Enhance sparsity via ρp-norm minimization. Comput Vis Image Underst 158:126–140. https://doi.org/10.1016/j.cviu.2017.03.002

    Article  Google Scholar 

  89. Raman R, Choudhury SK, Bakshi S (2018) Spatiotemporal optical blob reconstruction for object detection in grayscale videos. Multimed Tools Appl 77(1):741–762. https://doi.org/10.1007/s11042-016-4234-0

    Article  Google Scholar 

  90. Ramírez-Alonso G, Chacón-Murguía MI (2016) Auto-adaptive parallel SOM architecture with a modular analysis for dynamic object segmentation in videos. Neurocomputing 175:990–1000. https://doi.org/10.1016/j.neucom.2015.04.118

    Article  Google Scholar 

  91. Ramirez-Quintana JA, Chacon-Murguia MI (2015) Self-adaptive SOM-CNN neural system for dynamic object detection in normal and complex scenarios. Pattern Recogn 48(4):1137–1149. https://doi.org/10.1016/j.patcog.2014.09.009

    Article  Google Scholar 

  92. Rashid ME, Thomas V (2016) A background foreground competitive model for background subtraction in dynamic background. Procedia Technol 25:536–543. https://doi.org/10.1016/j.protcy.2016.08.142

    Article  Google Scholar 

  93. Roberto R, Lima JP, Teichrieb V (2016) Tracking for mobile devices: a systematic mapping study. Comput Graph 56:20–30. https://doi.org/10.1016/j.cag.2016.02.002

    Article  Google Scholar 

  94. Sakkos D, Liu H, Han J, Shao L (2018) End-to-end video background subtraction with 3d convolutional neural networks. Multimed Tools Appl 77(17):23023–23041. https://doi.org/10.1007/s11042-017-5460-9

    Article  Google Scholar 

  95. Salvadori C, Petracca M, del Rincon JM, Velastin SA, Makris D (2017) An optimisation of Gaussian mixture models for integer processing units. J Real-Time Image Proc 13(2):273–289. https://doi.org/10.1007/s11554-014-0402-5

    Article  Google Scholar 

  96. Sanches SRR, Oliveira C, Sementille AC, Freire V (2019) Challenging situations for background subtraction algorithms. Appl Intell 49(5):1771–1784. https://doi.org/10.1007/s10489-018-1346-4

    Article  Google Scholar 

  97. Sanches SRR, Sementille AC, Tori R, Nakamura R, Freire V PAD: A perceptual application-dependent metric for quality assessment of segmentation algorithms. Multimed Tools Appl 78(22). https://doi.org/10.1007/s11042-019-07958-7

  98. Savaş M., Demirel H, Erkal B (2018) Moving object detection using an adaptive background subtraction method based on block-based structure in dynamic scene. Optik 168:605–618. https://doi.org/10.1016/j.ijleo.2018.04.047

    Article  Google Scholar 

  99. Schick A, Bäuml M, Stiefelhagen R (2012) Improving foreground segmentations with probabilistic superpixel markov random fields. In: 2012 IEEE computer society conference on computer vision and pattern recognition workshops, pp 27–31, https://doi.org/10.1109/CVPRW.2012.6238923, (to appear in print)

  100. Seidel F, Hage C, Kleinsteuber M (2014) Prost: a smoothed ρp-norm robust online subspace tracking method for background subtraction in video. Mach Vis Appl 25 (5):1227–1240. https://doi.org/10.1007/s00138-013-0555-4

    Article  Google Scholar 

  101. Seo JW, Kim SD (2016) Dynamic background subtraction via sparse representation of dynamic textures in a low-dimensional subspace. SIViP 10(1):29–36. https://doi.org/10.1007/s11760-014-0697-5

    Article  Google Scholar 

  102. Shah M, Deng JD, Woodford BJ (2014) Video background modeling: Recent approaches, issues and our proposed techniques. Mach Vis Appl 25(5):1105–1119. https://doi.org/10.1007/s00138-013-0552-7

    Article  Google Scholar 

  103. Shah N, Pingale A, Patel V, George NV (2018) An adaptive background subtraction scheme for video surveillance systems. In: 2017 IEEE international symposium on signal processing and information technology ISSPIT, vol 2017, pp 13–17, https://doi.org/10.1109/ISSPIT.2017.8388311

  104. Shakeri M, Zhang H (2016) COROLA: A sequential solution to moving object detection using low-rank approximation. Comput Vis Image Underst 146:27–39. https://doi.org/10.1016/j.cviu.2016.02.009

    Article  Google Scholar 

  105. Shi G, Huang T, Dong W, Wu J, Xie X (2018) Robust foreground estimation via structured Gaussian scale mixture modeling. IEEE Trans Image Process 27(10):4810–4824. https://doi.org/10.1109/TIP.2018.2845123

    Article  MathSciNet  MATH  Google Scholar 

  106. Shimada A, Nonaka Y, Nagahara H, Taniguchi RI (2014) Case-based background modeling: associative background database towards low-cost and high-performance change detection. Mach Vis Appl 25(5):1121–1131. https://doi.org/10.1007/s00138-013-0563-4

    Article  Google Scholar 

  107. Silva C, Bouwmans T, Frelicot C (2017) Online weighted one-class ensemble for feature selection in background/foreground separation. In: Proceedings - international conference on pattern recognition, IEEE, pp 2216–2221, https://doi.org/10.1109/ICPR.2016.7899965, (to appear in print)

  108. Sobral A, Bouwmans T, ZahZah E (2015) Double-constrained rpca based on saliency maps for foreground detection in automated maritime surveillance. In: 2015 12th IEEE international conference on advanced video and signal based surveillance (AVSS), pp 1–6, https://doi.org/10.1109/AVSS.2015.7301753, (to appear in print)

  109. Sobral A, Vacavant A (2014) A comprehensive review of background subtraction algorithms evaluated with synthetic and real videos. Comput Vis Image Underst 122(2014):4–21. https://doi.org/10.1016/j.cviu.2013.12.005

    Article  Google Scholar 

  110. Spampinato C, Palazzo S, Kavasidis I (2014) A texton-based kernel density estimation approach for background modeling under extreme conditions. Comput Vis Image Underst 122:74–83. https://doi.org/10.1016/j.cviu.2013.12.003

    Article  Google Scholar 

  111. St-Charles PL, Bilodeau GA, Bergevin R (2014) Flexible background subtraction with self-balanced local sensitivity. In: IEEE Computer society conference on computer vision and pattern recognition workshops, IEEE, pp 414–419, https://doi.org/10.1109/CVPRW.2014.67, (to appear in print)

  112. St-Charles PL, Bilodeau GA, Bergevin R (2015) SuBSENSE: A universal change detection method with local adaptive sensitivity. IEEE Trans Image Process 24(1):359–373. https://doi.org/10.1109/TIP.2014.2378053

    Article  MathSciNet  MATH  Google Scholar 

  113. St-Charles PL, Bilodeau GA, Bergevin R (2016) Universal background subtraction using word consensus models. IEEE Trans Image Process 25 (10):4768–4781. https://doi.org/10.1109/TIP.2016.2598691

    Article  MathSciNet  MATH  Google Scholar 

  114. Stauffer C, Grimson W (1999) Adaptive background mixture models for real-time tracking. In: Proceedings. 1999 IEEE computer society conference on computer vision and pattern recognition (cat. no PR00149), IEEE comput. Soc, vol 2, pp 246–252, https://doi.org/10.1109/CVPR.1999.784637

  115. Stauffer C, Grimson WEL (2000) Learning patterns of activity using real-time tracking. IEEE Trans Pattern Anal Mach Intell 22(8):747–757. https://doi.org/10.1109/34.868677

    Article  Google Scholar 

  116. Subudhi BN, Ghosh S, Nanda PK, Ghosh A (2017) Moving object detection using spatio-temporal multilayer compound markov random field and histogram thresholding based change detection. Multimed Tools Appl 76 (11):13511–13543. https://doi.org/10.1007/s11042-016-3698-2

    Article  Google Scholar 

  117. Sultana M, Mahmood A, Javed S, Jung SK (2019) Unsupervised deep context prediction for background estimation and foreground segmentation. Mach Vis Appl 30(3):375–395. https://doi.org/10.1007/s00138-018-0993-0

    Article  Google Scholar 

  118. Sun L, Sheng W, Liu Y (2015) Background modeling and its evaluation for complex scenes. Multimed Tools Appl 74(11):3947–3966. https://doi.org/10.1007/s11042-013-1806-0

    Article  Google Scholar 

  119. Tsai T, Lin C (2010) Markov random field based background subtration method for foreground detection under moving background scene. In: 2010 Fourth international conference on genetic and evolutionary computing, pp 691–694, https://doi.org/10.1109/ICGEC.2010.176, (to appear in print)

  120. Université de Sherbrooke (2018) ChangeDetection.NET – a video database for testing change detection algorithms. http://www.changedetection.net. Accessed 22 July 2018

  121. University of Naples Parthenope (2019) SceneBackgroundModeling.net.NET – a video database for testing background estimation algorithms. http://scenebackgroundmodeling.net. Accessed 24 July 2019

  122. Vacavant A, Chateau T, Wilhelm A, Lequièvre L. (2012) A benchmark dataset for outdoor foreground/background extraction. In: Proceedings of the 11th international conference on computer vision - Volume Part I, ACCV’12. Springer, Berlin, pp 291–300, https://doi.org/10.1007/978-3-642-37410-4_25, (to appear in print)

  123. Van Droogenbroeck M., Paquot O. (2012) Background subtraction: experiments and improvements for vibe. In: 2012 IEEE Computer society conference on computer vision and pattern recognition workshops, pp 32–37, https://doi.org/10.1109/CVPRW.2012.6238924, (to appear in print)

  124. Varadarajan S, Miller P, Zhou H (2015) Region-based Mixture of Gaussians modelling for foreground detection in dynamic scenes. Pattern Recogn 48(11):3488–3503. https://doi.org/10.1016/j.patcog.2015.04.016

    Article  MATH  Google Scholar 

  125. Varadarajan S, Wang H, Miller P, Zhou H (2015) Fast convergence of regularised Region-based Mixture of Gaussians for dynamic background modelling. Comput Vis Image Underst 136:45–58. https://doi.org/10.1016/j.cviu.2014.12.004

    Article  Google Scholar 

  126. Varghese A, Sreelekha G (2015) Background subtraction for vehicle detection. In: Global conference on communication technologies, GCCT 2015, gcct, IEEE, pp 380–382, https://doi.org/10.1109/GCCT.2015.7342688, (to appear in print)

  127. Varghese A, Sreelekha G (2017) Sample-based integrated background subtraction and shadow detection. IPSJ Trans Comput Vision Appl 9(1):1–12. https://doi.org/10.1186/s41074-017-0036-1

    Article  Google Scholar 

  128. Vijayan M, Ramasundaram M (2018) Moving object detection using vector image model. Optik 168:963–973. https://doi.org/10.1016/j.ijleo.2018.05.012

    Article  Google Scholar 

  129. Wang B, Liu Y, Xu W, Wang W, Zhang M (2014) Background subtraction using spatiotemporal condition information. Optik 125(3):1406–1411. https://doi.org/10.1016/j.ijleo.2013.08.034

    Article  Google Scholar 

  130. Wang R, Bunyak F, Seetharaman G, Palaniappan K (2014) Static and moving object detection using flux tensor with split gaussian models. In: 2014 IEEE conference on computer vision and pattern recognition workshops, pp 420–424, https://doi.org/10.1109/CVPRW.2014.68, (to appear in print)

  131. Wang Y, Jodoin P, Porikli F, Konrad J, Benezeth Y, Ishwar P (2014) Cdnet 2014: an expanded change detection benchmark dataset. In: 2014 IEEE conference on computer vision and pattern recognition workshops, pp 393–400, https://doi.org/10.1109/CVPRW.2014.126, (to appear in print)

  132. Wang Y, Luo Z, Jodoin PM (2017) Interactive deep learning method for segmenting moving objects. Pattern Recogn Lett 96:66–75. https://doi.org/10.1016/j.patrec.2016.09.014

    Article  Google Scholar 

  133. Wang Y, Luo Z, Jodoin PM (2017) Interactive deep learning method for segmenting moving objects. Pattern Recogn Lett 96:66–75. https://doi.org/10.1016/j.patrec.2016.09.014

    Article  Google Scholar 

  134. Wen J, Xu Y, Tang J, Zhan Y, Lai Z, Guo X (2014) Joint video frame set division and low-rank decomposition for background subtraction. IEEE Trans Circ Syst Video Technol 24(12):2034–2048. https://doi.org/10.1109/TCSVT.2014.2333132

    Article  Google Scholar 

  135. Wren CR, Azarbayejani A, Darrell T, Pentland AP (1997) Pfinder: real-time tracking of the human body. IEEE Trans Pattern Anal Mach Intell 19(7):780–785. https://doi.org/10.1109/34.598236

    Article  Google Scholar 

  136. Wu X, Lu X (2019) Adaptive pixel-block based background subtraction using low-rank and block-sparse matrix decomposition. Multimed Tools Appl 78(12):16507–16526. https://doi.org/10.1007/s11042-018-7037-7

    Article  Google Scholar 

  137. Xia H, Song S, He L (2016) A modified Gaussian mixture background model via spatiotemporal distribution with shadow detection. SIViP 10 (2):343–350. https://doi.org/10.1007/s11760-014-0747-z

    Article  Google Scholar 

  138. Xiao H, Liu Y, Zhang M (2016) Fast ρ1-minimization algorithm for robust background subtraction. Eurasip J Image Video Process 2016(1). https://doi.org/10.1186/s13640-016-0150-5

  139. Xue Y, Guo X, Cao X (2012) Motion saliency detection using low-rank and sparse decomposition. In: 2012 IEEE international conference on acoustics, speech and signal processing (ICASSP), pp 1485–1488, https://doi.org/10.1109/ICASSP.2012.6288171, (to appear in print)

  140. Yang B, Zou L (2015) Robust foreground detection using block-based RPCA. Optik 126(23):4586–4590. https://doi.org/10.1016/j.ijleo.2015.08.064

    Article  Google Scholar 

  141. Yang MH, Huang CR, Liu WC, Lin SZ, Chuang KT (2015) Binary descriptor based nonparametric background modeling for foreground extraction by using detection theory. IEEE Trans Circ Syst Video Technol 25(4):595–608. https://doi.org/10.1109/TCSVT.2014.2361418

    Article  Google Scholar 

  142. Yang SC, Lin GC, Wang CM (2018) Foreground detection using texture-based codebook method for monitoring systems. SIViP 12 (4):693–701. https://doi.org/10.1007/s11760-017-1209-1

    Article  Google Scholar 

  143. Ye X, Yang J, Sun X, Li K, Hou C, Wang Y (2015) foreground-background separation from video clips via motion-assisted matrix restoration. IEEE Trans Circ Syst Video Technol 25(11):1721–1734. https://doi.org/10.1109/TCSVT.2015.2392491

    Article  Google Scholar 

  144. Yoshinaga S, Shimada A, Nagahara H, Taniguchi R (2014) Object detection based on spatiotemporal background models. Comput Vis Image Underst 122:84–91. https://doi.org/10.1016/j.cviu.2013.10.015

    Article  Google Scholar 

  145. Zeng Z, Jia J, Zhu Z, Yu D (2016) Adaptive maintenance scheme for codebook-based dynamic background subtraction. Comput Vis Image Underst 152:58–66. https://doi.org/10.1016/j.cviu.2016.08.009

    Article  Google Scholar 

  146. Zhang C, Zheng J, Zhang Y, Han M, Li B (2017) Moving object detection algorithm based on pixel spatial sample difference consensus. Multimed Tools Appl 76(21):22077–22093. https://doi.org/10.1007/s11042-017-4802-y

    Article  Google Scholar 

  147. Zhang R, Liu X, Hu J, Chang K, Liu K (2017) A fast method for moving object detection in video surveillance image. SIViP 11(5):841–848. https://doi.org/10.1007/s11760-016-1030-2

    Article  Google Scholar 

  148. Zhang X, He H, Cao S, Liu H (2015) Flow field texture representation-based motion segmentation for crowd counting. Mach Vis Appl 26(7):871–883. https://doi.org/10.1007/s00138-015-0703-0

    Article  Google Scholar 

  149. Zheng Z, Hong P (2018) Robust detection of adversarial attacks by modeling the intrinsic properties of deep neural networks. In: Proceedings of the 32nd international conference on neural information processing systems, NIPS’18, Curran Associates Inc., Red Hook, NY, USA, pp 7924–7933

  150. Zhou T, Tao D (2011) Godec: Randomized low-rank & sparse matrix decomposition in noisy case. In: Proceedings of the 28th international conference on international conference on machine learning, ICML’11. Omnipress, USA, pp 33–40

  151. Zhou X, Yang C, Yu W (2013) Moving object detection by detecting contiguous outliers in the low-rank representation. IEEE Trans Pattern Anal Mach Intell 35(3):597–610. https://doi.org/10.1109/TPAMI.2012.132

    Article  Google Scholar 

  152. Zhu T, Zeng P (2016) Background subtraction based on non-parametric model. In: Proceedings of 2015 4th international conference on computer science and network technology, ICCSNT 2015, IEEE, vol 01, pp 1379–1382, https://doi.org/10.1109/ICCSNT.2015.7490985

  153. Zivkovic Z (2004) Improved adaptive gaussian mixture model for background subtraction. In: Proceedings of the 17th international conference on pattern recognition, 2004. ICPR 2004., vol. 2, pp. 28–31 Vol.2, https://doi.org/10.1109/ICPR.2004.1333992, (to appear in print)

  154. Zivkovic Z, van der Heijden F (2006) Efficient adaptive density estimation per image pixel for the task of background subtraction. Pattern Recogn Lett 27(7):773–780. https://doi.org/10.1016/j.patrec.2005.11.005

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Universidade Tecnológica Federal do Paraná, Cornélio Procópio, Brazil

    Silvio Ricardo Rodrigues Sanches

  2. Universidade Estadual Paulista “Júlio de Mesquita Filho”, Bauru, Brazil

    Antonio Carlos Sementille

  3. Simon Fraser University, Burnaby, Canada

    Ivan Abdo Aguilar

  4. Universidade de São Paulo, São Paulo, Brazil

    Valdinei Freire

Authors
  1. Silvio Ricardo Rodrigues Sanches
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Antonio Carlos Sementille
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ivan Abdo Aguilar
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Valdinei Freire
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Silvio Ricardo Rodrigues Sanches.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Sanches, S.R.R., Sementille, A.C., Aguilar, I.A. et al. Recommendations for evaluating the performance of background subtraction algorithms for surveillance systems. Multimed Tools Appl 80, 4421–4454 (2021). https://doi.org/10.1007/s11042-020-09838-x

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-020-09838-x

Keywords

Search

Navigation