Skip to main content
SpringerLink
Log in

Method for selecting representative videos for change detection datasets

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

In evaluating the change detection algorithms, the algorithm evaluated must show a superior performance than the state-of-the-art algorithms. The evaluation process steps comprise executing a new algorithm to segment a set of videos from a dataset and compare the results regarding a ground truth. In this paper, we propose using additional information in evaluating change detection algorithms: the level of difficulty in classifying a pixel. First, for each video frame used in the evaluation, we created a difficulty map structure, which stores values representing the level of difficulty required by an algorithm to classify each pixel of that frame. Second, we developed a metric to estimate each dataset video’s difficulty based on our difficulty maps. Third, we applied the metric to selecting the more representative videos from the dataset based on their difficulty level. Finally, to demonstrate the method’s contribution, we evaluated it using all videos from the CDNet 2014 dataset. The results showed that a subset of videos selected by our method has the same potential as the original CDNet 2014 dataset. Hence, a new change detection algorithm can be evaluated more quickly using our subset of videos selected.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  1. Allebosch G, Deboeverie F, Veelaert P, Philips W (2015) Efic: edge based foreground background segmentation and interior classification for dynamic camera viewpoints. In: Battiato S, Blanc-Talon J, Gallo G, Philips W, Popescu D, Scheunders P (eds) Advanced concepts for intelligent vision systems. Springer International Publishing, Cham, pp 130–141. https://doi.org/10.1007/978-3-319-25903-1_12

  2. Allebosch G, Van Hamme D, Deboeverie F, Veelaert P, Philips W (2016) C-efic: color and edge based foreground background segmentation with interior classification. In: Braz J, Pettré J, Richard P, Kerren A, Linsen L, Battiato S, Imai F (eds) Computer vision, imaging and computer graphics theory and applications. Springer International Publishing, Cham, pp 433–454. https://doi.org/10.1007/978-3-319-29971-6_23

  3. Babaee M, Dinh DT, Rigoll G (2018) A deep convolutional neural network for video sequence background subtraction. Pattern Recogn 76:635–649. https://doi.org/10.1016/j.patcog.2017.09.040

    Article  Google Scholar 

  4. Benezeth Y, Jodoin PM, Emile B, Laurent H, Rosenberger C (2010) Comparative study of background subtraction algorithms. Journal of Electronic Imaging 19(3):1–12. https://doi.org/10.1117/1.3456695

    Article  Google Scholar 

  5. Bianco S, Ciocca G, Schettini R (2017) Combination of video change detection algorithms by genetic programming. IEEE Transactions on Evolutionary Computation 21(6):914–928. https://doi.org/10.1109/TEVC.2017.2694160

    Article  Google Scholar 

  6. Bianco S, Ciocca G, Schettini R (2017b) How far can you get by combining change detection algorithms? In: Battiato S, Gallo G, Schettini R, Stanco F (eds) Image analysis and processing - ICIAP 2017. Springer International Publishing, Cham, pp 96–107. https://doi.org/10.1007/978-3-319-68560-1_9

  7. Braham M, Pierard S, Droogenbroeck MV (2017) Semantic background subtraction. In: 2017 IEEE International Conference on Image Processing (ICIP), pp 4552–4556

  8. Chan YT (2019) Deep learning-based scene-awareness approach for intelligent change detection in videos. Journal of Electronic Imaging 28(1):1–12. https://doi.org/10.1117/1.JEI.28.1.013038

    Article  Google Scholar 

  9. Chen Y, Wang J, Lu H (2015) Learning sharable models for robust background subtraction. In: 2015 IEEE International Conference on Multimedia and Expo (ICME), pp 1–6. https://doi.org/10.1109/ICME.2015.7177419

  10. Elgammal A, Harwood D, Davis L (2000) Non-parametric model for background subtraction. In: Vernon D (ed) Computer vision — ECCV 2000. Springer, Berlin, pp 751–767.https://doi.org/10.1007/3-540-45053-X_48

  11. Fisher R (2019) CAVIAR test case scenarios. http://groups.inf.ed.ac.uk/vision/CAVIAR Accessed 24 Sep 2019

  12. Goyette N, Jodoin PM, Porikli F, Konrad J, Ishwar P (2012) Changedetection.net: a new change detection benchmark dataset. In: 2012 IEEE computer society conference on computer vision and pattern recognition workshops, pp 1–8. https://doi.org/10.1109/CVPRW.2012.6238919

  13. Gregorio MD, Giordano M (2014) Change detection with weightless neural networks. In: 2014 IEEE conference on computer vision and pattern recognition workshops, pp 409–413. https://doi.org/10.1109/CVPRW.2014.66

  14. Gregorio MD, Giordano M (2017) Wisardrp for change detection in video sequences. In: 25th European symposium on artificial neural networks, computational intelligence and machine learning (ESANN 2017), pp 453–458

  15. Isik S, Özkan K, Günal S, Gerek ON (2018) Swcd: a sliding window and self-regulated learning-based background updating method for change detection in videos. Journal of Electronic Imaging 27(2):1–11. https://doi.org/10.1117/1.JEI.27.2.023002

    Article  Google Scholar 

  16. Jiang S, Lu X (2018) Wesambe: a weight-sample-based method for background subtraction. IEEE Transactions on Circuits and Systems for Video Technology 28(9):2105–2115. https://doi.org/10.1109/TCSVT.2017.2711659

    Article  Google Scholar 

  17. Kalsotra R, Arora S (2019) A comprehensive survey of video datasets for background subtraction. IEEE Access 7:59143–59171. https://doi.org/10.1109/ACCESS.2019.2914961

    Article  Google Scholar 

  18. Krungkaew R, Kusakunniran W (2016) Foreground segmentation in a video by using a novel dynamic codebook. 2016 13th International Conference on Electrical Engineering/Electronics. Computer, Telecommunications and Information Technology (ECTI-CON), pp 1–6

  19. Lee Sh, Gc Lee, Yoo J, Kwon S (2019) Wisenetmd: motion detection using dynamic background region analysis. Symmetry 11(5):1–15. https://doi.org/10.3390/sym11050621

    Article  Google Scholar 

  20. Liang D, Kaneko S, Hashimoto M, Iwata K, Zhao X (2015) Co-occurrence probability-based pixel pairs background model for robust object detection in dynamic scenes. Pattern Recogn 48(4):1374–1390. https://doi.org/10.1016/j.patcog.2014.10.020

    Article  Google Scholar 

  21. Lim LA, Keles HY (2018) Foreground segmentation using convolutional neural networks for multiscale feature encoding. Pattern Recogn Lett 112:256–262. https://doi.org/10.1016/j.patrec.2018.08.002

    Article  Google Scholar 

  22. Lim LA, Keles HY (2019) Learning multi-scale features for foreground segmentation. Pattern Analysis and Applications. https://doi.org/10.1007/s10044-019-00845-9

    Article  Google Scholar 

  23. Lu X (2014) A multiscale spatio-temporal background model for motion detection. In: 2014 IEEE International Conference on Image Processing (ICIP), pp 3268–3271. https://doi.org/10.1109/ICIP.2014.7025661

  24. Maddalena L, Petrosino A (2010) A fuzzy spatial coherence-based approach to background/foreground separation for moving object detection. Neural Computing and Applications 19(2):179–186. https://doi.org/10.1007/s00521-009-0285-8

    Article  Google Scholar 

  25. Maddalena L, Petrosino A (2012) The sobs algorithm: what are the limits? In: 2012 IEEE computer society conference on computer vision and pattern recognition workshops, pp 21–26. https://doi.org/10.1109/CVPRW.2012.6238922

  26. Martins I, Carvalho P, Corte-Real L, Alba-Castro JL (2017) Bmog: boosted gaussian mixture model with controlled complexity. In: Alexandre LA, Salvador Sánchez J, Rodrigues JMF (eds) Pattern recognition and image analysis. Springer International Publishing, Cham, pp 50–57. https://doi.org/10.1007/978-3-319-58838-4_6

  27. Microsoft Corporation (2019) Test images for wallflower paper. https://www.microsoft.com/en-us/download/details.aspx?id=54651. Accessed 9 Aug 2019

  28. Miron A, Badii A (2015) Change detection based on graph cuts. In: 2015 International conference on systems, signals and image processing (IWSSIP), pp 273–276. https://doi.org/10.1109/IWSSIP.2015.7314229

  29. OpenCV team (2019) OpenCV. https://opencv.org/. Accessed 24 Sep 2019

  30. Ramírez-Alonso G, Chacon-Murguia MI (2016) Auto-adaptive parallel som architecture with a modular analysis for dynamic object segmentation in videos. Neurocomputing 175:990–1000. https://doi.org/10.1016/j.neucom.2015.04.118

    Article  Google Scholar 

  31. Russel J, Cohn R (2013) Interquartile range. Tbilisi State University

  32. Sajid H, Cheung SS (2017) Universal multimode background subtraction. IEEE Transactions on Image Processing 26(7):3249–3260. https://doi.org/10.1109/TIP.2017.2695882

    Article  MathSciNet  MATH  Google Scholar 

  33. Sanches SRR, Oliveira C, Sementille AC, Freire V (2019) Challenging situations for background subtraction algorithms. Applied Intelligence 49(5):1771–1784. https://doi.org/10.1007/s10489-018-1346-4

    Article  Google Scholar 

  34. Sanches SRR, Sementille AC, Aguilar IA, Freire V (2021) Recommendations for evaluating the performance of background subtraction algorithms for surveillance systems. Multimed Tools Applic 80(3):4421–4454. https://doi.org/10.1007/s11042-020-09838-x

    Article  Google Scholar 

  35. Sedky M, Moniri M, Chibelushi CC (2014) Spectral-360: a physics-based technique for change detection. In: 2014 IEEE conference on computer vision and pattern recognition workshops, pp 405–408. https://doi.org/10.1109/CVPRW.2014.65

  36. Sobral A, Vacavant A (2014) A comprehensive review of background subtraction algorithms evaluated with synthetic and real videos. Computer Vision and Image Understanding 122:4–21. https://doi.org/10.1016/j.cviu.2013.12.005

    Article  Google Scholar 

  37. Soomro K, Shah M (2012) Ucf101: a dataset of 101 human action classes from videos in the wild. Tech. rep., CRCV-TR-12-01

  38. St-Charles P, Bilodeau G, Bergevin R (2015a) A self-adjusting approach to change detection based on background word consensus. In: 2015 IEEE winter conference on applications of computer vision, pp 990–997 https://doi.org/10.1109/WACV.2015.137

  39. St-Charles P, Bilodeau G, Bergevin R (2015) Subsense: a universal change detection method with local adaptive sensitivity. IEEE Transactions on Image Processing 24(1):359–373. https://doi.org/10.1109/TIP.2014.2378053

    Article  MathSciNet  MATH  Google Scholar 

  40. Stauffer C, Grimson WEL (1999) Adaptive background mixture models for real-time tracking. In: Proceedings. 1999 IEEE computer society conference on computer vision and pattern recognition (Cat. No PR00149), vol 2, 246–252. https://doi.org/10.1109/CVPR.1999.784637

  41. Toyama K, Krumm J, Brumitt B, Meyers B (1999) Wallflower: principles and practice of background maintenance. In: Proceedings of the seventh IEEE international conference on computer vision, vol 1, pp 255–261. https://doi.org/10.1109/ICCV.1999.791228

  42. Université de Sherbrooke (2019) ChangeDetection.NET – a video database for testing change detection algorithms. http://www.changedetection.net. Accessed 22 Jul 2018

  43. University of Naples Parthenope (2019) SceneBackgroundModeling.net.NET – a video database for testing background estimation algorithms. http://scenebackgroundmodeling.net. Accessed 24 Jul 2019

  44. Vacavant A, Chateau T, Wilhelm A, Lequiévre L (2013) A benchmark dataset for outdoor foreground/background extraction. Springer, Berlin, pp 291–300. https://doi.org/10.1007/978-3-642-37410-4_25

  45. Varadarajan S, Miller P, Zhou H (2013) Spatial mixture of gaussians for dynamic background modelling. In: 2013 10th IEEE international conference on advanced video and signal based surveillance, pp 63–68. https://doi.org/10.1109/AVSS.2013.6636617

  46. Varghese AGS (2017) Sample-based integrated background subtraction and shadow detection. IPSJ Transactions on Computer Vision and Applications 9(1):25. https://doi.org/10.1186/s41074-017-0036-1

    Article  Google Scholar 

  47. Wang B, Dudek P (2014) A fast self-tuning background subtraction algorithm. In: 2014 IEEE conference on computer vision and pattern recognition workshops, pp 401–404. https://doi.org/10.1109/CVPRW.2014.64

  48. Wang K, Gou C, Wang FY (2018) M4cd: A robust change detection method for intelligent visual surveillance. arXiv:1802.04979. Cornell University. Accessed 12 Nov 2019

  49. Wang R, Bunyak F, Seetharaman G, Palaniappan K (2014) Static and moving object detection using flux tensor with split gaussian models. In: 2014 IEEE conference on computer vision and pattern recognition workshops, pp 420–424. https://doi.org/10.1109/CVPRW.2014.68

  50. Wang Y, Luo Z, Jodoin PM (2017) Interactive deep learning method for segmenting moving objects. Pattern Recognition Letters 96:66–75 https://doi.org/10.1016/j.patrec.2016.09.014

  51. Yilmaz AA, Guzel MS, Bostanci E, Askerzade I (2020) A novel action recognition framework based on deep-learning and genetic algorithms. IEEE Access 8:100631–100644. https://doi.org/10.1109/ACCESS.2020.2997962

    Article  Google Scholar 

  52. Young DP, Ferryman JM (2005) Pets metrics: on-line performance evaluation service. In: 2005 IEEE international workshop on visual surveillance and performance evaluation of tracking and surveillance, pp 317–324. https://doi.org/10.1109/VSPETS.2005.1570931

  53. Zheng W, Wang K, Wang FY (2019) A novel background subtraction algorithm based on parallel vision and bayesian gans. Neurocomputing. https://doi.org/10.1016/j.neucom.2019.04.088

    Article  Google Scholar 

  54. Zheng W, Wang K, Wang FY (2020) A novel background subtraction algorithm based on parallel vision and bayesian gans. Neurocomputing 394:178–200. https://doi.org/10.1016/j.neucom.2019.04.088

    Article  Google Scholar 

  55. Zivkovic Z (2004) Improved adaptive gaussian mixture model for background subtraction. In: Proceedings of the 17th international conference on pattern recognition, 2004. ICPR 2004, vol 2, 28–31. https://doi.org/10.1109/ICPR.2004.1333992

Download references

Author information

Authors and Affiliations

  1. Universidade Tecnológica Federal do Paraná, Cornélio Procópio, Brasil

    Claudinei M. Silva, Katharina A. I. Rosa, Pedro H. Bugatti, Priscila T. M. Saito, Cléber G. Corrêa & Silvio R. R. Sanches

  2. Universidade Federal do ABC, Santo André, SP, Brasil

    Roberto S. Yokoyama

Authors
  1. Claudinei M. Silva
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Katharina A. I. Rosa
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Pedro H. Bugatti
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Priscila T. M. Saito
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Cléber G. Corrêa
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Roberto S. Yokoyama
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Silvio R. R. Sanches
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Silvio R. R. Sanches.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Silva, C.M., Rosa, K.A.I., Bugatti, P.H. et al. Method for selecting representative videos for change detection datasets. Multimed Tools Appl 81, 3773–3791 (2022). https://doi.org/10.1007/s11042-021-11640-2

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-021-11640-2

Keywords

Search

Navigation