Machine Vision and Applications

, Volume 25, Issue 5, pp 1105–1119 | Cite as

Video background modeling: recent approaches, issues and our proposed techniques

  • Munir Shah
  • Jeremiah D. Deng
  • Brendon J. Woodford
Special Issue Paper

Abstract

Effective and efficient background subtraction is important to a number of computer vision tasks. We introduce several new techniques to address key challenges for background modeling using a Gaussian mixture model (GMM) for moving objects detection in a video acquired by a static camera. The novel features of our proposed model are that it automatically learns dynamics of a scene and adapts its parameters accordingly, suppresses ghosts in the foreground mask using a SURF features matching algorithm, and introduces a new spatio-temporal filter to further refine the foreground detection results. Detection of abrupt illumination changes in the scene is dealt with by a model shifting-based scheme to reuse already learned models and spatio-temporal history of foreground blobs is used to detect and handle paused objects. The proposed model is rigorously tested and compared with several previous models and has shown significant performance improvements.

Keywords

Background subtraction Online learning Feature matching Gaussian mixture model 

References

  1. 1.
    Bay, H., Ess, A., Tuytelaars, T., Van Gool, L.: Speeded-up robust features (SURF). Comput. Vis. Image Underst. 110(3), 346–359 (2008). doi: 10.1016/j.cviu.2007.09.014 CrossRefGoogle Scholar
  2. 2.
    Benedek, C., Sziranyi, T.: Study on color space selection for detecting cast shadows in video surveillance. Int. J. Imaging Syst. Technol. 17(3), 190–201 (2007)CrossRefGoogle Scholar
  3. 3.
    Blauensteiner, P., Wildenauer, H., Hanbury, A., Kampel, M.: On color spaces for change detection and shadows suppression. In: Proceedings of the Computer Vision Winter Workshop. Czech Pattern Recognition Society (2007)Google Scholar
  4. 4.
    Bouwmans, T.: Subspace learning for background modeling: a survey. Recent Patent Comput. Sci. 2(3), 223–234 (2009)CrossRefGoogle Scholar
  5. 5.
    Bouwmans, T.: Recent advanced statistical background modeling for foreground detection: a systematic survey. Recent Patents Comput. Sci. 4(3), 147–176 (2011)Google Scholar
  6. 6.
    Bouwmans, T., Baf, F.E., Vachon, B.: Statistical Background Modeling for Foreground Detection: A Survey, pp. 181–199. World Scientific Publishing, Singapore (2010)Google Scholar
  7. 7.
    Bucak, S., Gunsel, B., Gursoy, O.: Incremental nonnegative matrix factorization for background modeling in surveillance video. In: Proceeedings of the 15th IEEE Conference on Signal Processing and Communications Applications, pp. 1–4. IEEE, New York (2007). doi: 10.1109/SIU.2007.4298684
  8. 8.
    Chang, R., Gandhi, T., Trivedi, M.: Vision modules for a multi-sensory bridge monitoring approach. In: Proceedings of the 7th International IEEE Conference on Intelligent Transportation Systems, pp. 971–976. IEEE, New York (2004)Google Scholar
  9. 9.
    Cheng, J., Yang, J., Zhou, Y., Cui, Y.: Flexible background mixture models for foreground segmentation. Image Vis. Comput. 24(5), 473–482 (2006)CrossRefGoogle Scholar
  10. 10.
    Davis, J.W., Sharma, V.: Background-subtraction using contour-based fusion of thermal and visible imagery. Comput. Vis. Image Underst. 106(2–3), 162–182 (2007). doi: 10.1016/j.cviu.2006.06.010 CrossRefGoogle Scholar
  11. 11.
    Elgammal, A.M., Harwood, D., Davis, L.S.: Non-parametric model for background subtraction. In: Proceedings of the 6th European Conference on Computer Vision, ECCV’00, pp. 751–767. Springer, London (2000)Google Scholar
  12. 12.
    Evangelio, R.H., Ptzold, M., Sikora, T.: Splitting Gaussians in mixture models. In: Proceedings of the 9th IEEE International Conference on Advanced Video and Signal-Based Surveillance, AVSS’2012, pp. 300–305. IEEE Computer Society, New York (2012)Google Scholar
  13. 13.
    Evangelio, R.H., Senst, T., Sikora, T.: Detection of static objects for the task of video surveillance. In: Proceedings of the IEEE Workshop on Applications of Computer Vision, WACV’11, pp. 534–540. IEEE Computer Society, Washington, DC (2011). doi: 10.1109/WACV.2011.5711550
  14. 14.
    Figueiredo, M.A.T., Jain, A.K.: Unsupervised learning of finite mixture models. IEEE Trans. Pattern Analy. Mach. Intell. 24(3), 381–396 (2000)CrossRefGoogle Scholar
  15. 15.
    Fischler, M.A., Bolles, R.C.: Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Commun. ACM 24(6), 381–395 (1981)Google Scholar
  16. 16.
    Friedman, N., Russell, S.: Image segmentation in video sequences: a probabilistic approach. In: Proceedings of the 13th Conference on Uncertainty in Artificial Intelligence, UAI’97, pp. 175–181. Morgan Kaufmann Publishers Inc., San Francisco (1997)Google Scholar
  17. 17.
    Goyette, N., Jodoin, P., Porikli, F., Konrad, J., Ishwar, P.: Changedetection.net: A new change detection benchmark dataset. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, pp. 1–8. IEEE Computer Society, New York (2012). doi: 10.1109/CVPRW.2012.6238919
  18. 18.
    Han, B., Davis, L.S.: Density-based multifeature background subtraction with support vector machine. IEEE Trans. Pattern Anal. Mach. Intell. 34, 1017–1023 (2012)CrossRefGoogle Scholar
  19. 19.
    Heikkila, M., Pietikainen, M.: A texture-based method for modeling the background and detecting moving objects. IEEE Trans. Pattern Anal. Mach. Intell. 28(4), 657–662 (2006). doi: 10.1109/TPAMI.2006.68 CrossRefGoogle Scholar
  20. 20.
    Hofmann, M., Tiefenbacher, P., Rigoll, G.: Background Segmentation with feedback: the pixel-based adaptive segmenter. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition—Change Detection Workshop, pp. 38–43. IEEE, New York (2012)Google Scholar
  21. 21.
    Huang, T., Qiu, J., Ikenaga, T.: A foreground extraction algorithm based on adaptively adjusted Gaussian mixture models. In: Proceedings of the 5th International Joint Conference on INC, IMS and IDC, pp. 1662–1667. IEEE Computer Society, New York (2009)Google Scholar
  22. 22.
    Zheng, J., Wang, Y., Nihan, N., Hallenbeck, E.: Extracting roadway background image a model based approach. Transp. Res. Rep. 1944, 82–88 (2006)Google Scholar
  23. 23.
    KaewTraKulPong, P., Rowden, R.: An improved adaptive background mixture model for real-time tracking with shadow detection. In: Proceedings of the Second European Workshop on Advanced Video Based Surveillance Systems, pp. 149–158 (2001)Google Scholar
  24. 24.
    Kim, K., Chalidabhongse, T.H., Harwood, D., Davis, L.: Real-time foreground–background segmentation using codebook model. Real-Time Imaging 11(3), 172–185 (2005)CrossRefGoogle Scholar
  25. 25.
    Lee, D.S.: Effective Gaussian mixture learning for video background subtraction. IEEE Trans. Pattern Anal. Mach. Intell. 27(5), 827–832 (2005)CrossRefGoogle Scholar
  26. 26.
    McFarlane, N.J.B., Schofield, C.P.: Segmentation and tracking of piglets in images. Mach. Vis. Appl. 8, 187–193 (1995). doi: 10.1007/BF01215814 CrossRefGoogle Scholar
  27. 27.
    McHugh, J., Konrad, J., Saligrama, V., Jodoin, P.: Foreground-adaptive background subtraction. IEEE Signal Process. Lett. (IEEE-SPL) 16(5), 390–393 (2009)Google Scholar
  28. 28.
    Mittal, A., Paragios, N.: Motion-based background subtraction using adaptive kernel density estimation. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, CVPR’2004, 2, pp. 302–309 (2004). doi: 10.1109/CVPR.2004.1315179
  29. 29.
    Noriega, P., Bernier, O.: Real time illumination invariant background subtraction using local kernel histograms. In: Proceedings of the British Machine Vision Conference, vol. 3, pp. 979–988 (2006)Google Scholar
  30. 30.
    Piccardi, M.: Background Subtraction techniques: a review. In: Proceedings of the IEEE International Conference on Systems, Man and Cybernetics, vol. 4, pp. 3099–3104. IEEE, New York (2004)Google Scholar
  31. 31.
    Shah, M., Deng, J., Woodford., B.: Enhancing the mixture of Gaussians background model with local matching and local adaptive learning. In: Proceedings of the 27th Conference on Image and Vision Computing New Zealand, IVCNZ ’12, pp. 103–108. ACM, New York (2012). doi: 10.1145/2425836.2425859
  32. 32.
    Shan, Y., Yang, F., Wang, R.: Color Space Selection for Moving Shadow Elimination. In: Proceedings of the 4th International Conference on Image and Graphics, ICIG’2007, pp. 496–501. IEEE, New York (2007). doi: 10.1109/ICIG.2007.54
  33. 33.
    Shimada, A., Arita, D., ichiro Taniguchi, R.: Dynamic Control of Adaptive Mixture-of-Gaussians background model. In: Proceedings of the IEEE Conference on Advanced Video and Signal Based Surveillance, vol. 0, p. 5. IEEE Computer Society, Los Alamitos (2006)Google Scholar
  34. 34.
    Shoushtarian, B., Ghasem-aghaee, N.: A practical approach for real-time dynamic background generation based A temporal median filter. J. Sci. 14(4), 351–362 (2003)Google Scholar
  35. 35.
    Smids, M.: Background Subtraction for Urban Traffic Monitoring using Webcams. Master thesis, Universiteit van Amsterdam, FNWI, The Netherlands (2006)Google Scholar
  36. 36.
    Stauffer, C., Grimson, W.: Adaptive background mixture models for real-time tracking. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, CVPR’99, vol. 2, pp. 246–252. IEEE Computer Society, Los Alamitos (1999)Google Scholar
  37. 37.
    Stefano, M., Maria, M.C., Nicola, S., Michele, Z.: A kalman filter based background updating algorithm robust to sharp illumination changes. In: Proceedings of the 13th International Conference on Image Analysis and Processing, ICIAP’05, pp. 163–170. Springer, Berlin (2005)Google Scholar
  38. 38.
    Toyama, K., Krumm, J., Brumitt, B., Meyers, B.: Wallflower: Principles and practice of background maintenance. In: Proceedings of the IEEE International Conference on Computer Vision, ICCV’99, vol. 1, p. 255. IEEE Computer Society, Los Alamitos (1999)Google Scholar
  39. 39.
    Van Droogenbroeck, M., Paquot, O.: Background subtraction: experiments and improvements for ViBe. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, pp. 32–37. IEEE, New York (2012)Google Scholar
  40. 40.
    Wang, L., Pan, C.: Effective multi-resolution background subtraction. In: Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP’2011, pp. 909–912. IEEE, New York (2011)Google Scholar
  41. 41.
    Wayne, P., Johann, P., Schoonees, A.: Understanding background mixture models for foreground. Techncial Report, Industrial Research Limited, Auckland, New Zealand (2002)Google Scholar
  42. 42.
    White, B., Shah, M.: Automatically tunings background subtraction parameters using particle swarm optimization. In: Proceedings of the IEEE International Conference on Multimedia and Expo, pp. 1826–1829. IEEE, New York (2005)Google Scholar
  43. 43.
    Xu, Z., Shi, P., Gu, I.Y.H.: An eigenbackground subtraction method using recursive error compensation. In: Proceedings of the 7th Pacific Rim conference on Advances in Multimedia Information Processing, PCM’06, pp. 779–787. Springer, Berlin (2006)Google Scholar
  44. 44.
    Yamazaki, M., Xu, G., Chen, Y.W.: Detection of moving objects by independent component analysis. In: Proceedings of the 7th Asian conference on Computer Vision, ACCV’06, pp. 467–478. Springer, Berlin (2006)Google Scholar
  45. 45.
    Yu, X., Chen, X., Zhang, H.: Accurate motion detection in dynamic scenes based on ego-motion estimation and optical flow segmentation combined method. In: Proceedings of the Symposium on Photonics and Optoelectronics, SOPO’2011, pp. 1–4. IEEE, New York (2011)Google Scholar
  46. 46.
    Zhang, E., Chen, F., Zhang, W.: A novel particle filter based background subtraction method. In: Proceedings of the International Conference on Computational Intelligence and Security, vol. 2, pp. 1837–1840 (2006)Google Scholar
  47. 47.
    Zhong, J., Sclaroff, S.: Segmenting foreground objects from a dynamic textured background via a robust Kalman filter. In: Proceediings of the 9th IEEE International Conference on Computer Vision, ICCV’03, p. 44. IEEE Computer Society, Washington, DC (2003)Google Scholar
  48. 48.
    Zivkovic, Z.: Improved adaptive Gaussian mixture model for background subtraction. In: Proceedings of the 17th International Conference on Pattern Recognition, ICPR’04, vol. 2, pp. 28–31. IEEE Computer Society, New York (2004)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  • Munir Shah
    • 1
  • Jeremiah D. Deng
    • 1
  • Brendon J. Woodford
    • 1
  1. 1.Department of Information ScienceUniversity of OtagoDunedinNew Zealand

Personalised recommendations