Signal, Image and Video Processing

, Volume 8, Issue 6, pp 1015–1029 | Cite as

Neighborhood-level learning techniques for nonparametric scene models

Original Paper

Abstract

A new stochastic learning algorithm for use in nonparametric pixel-level background models is presented in this paper. For the first time, we propose the use of kernel density estimation techniques in the model update step to identify outliers within the pixel-level sample collections and replace them with recently observed background pixel values. A neighborhood diffusion process that improves on recently reported scene model learning techniques is presented, wherein information sharing between similarly structured adjacent background models is encouraged to promote spatial consistency within localized image regions. We demonstrate the superiority of the proposed algorithm in comparison with the state-of-the-art visual background extraction system using the well-known percentage correct classification statistic and a new figure of merit, probability correct classification, presented here for the first time.

Keywords

Video segmentation Scene modeling Background modeling 

References

  1. 1.
    Adam, A., Kimmel, R., Rivlin, E.: On scene segmentation and histograms-based curve evolution. IEEE Trans. Pattern Anal. Mach. Intell. 31(9), 1708–1714 (2009)CrossRefGoogle Scholar
  2. 2.
    Barnich, O., Van Droogenbroeck, M.: ViBe: a universal background subtraction algorithm for video sequences. IEEE Trans. Image Process. 20(6), 1709–1724 (2011)CrossRefMathSciNetGoogle Scholar
  3. 3.
    Benedek, C., Sziranyi, T.: Bayesian foreground and shadow detection in uncertain frame rate surveillance videos. IEEE Trans. Image Process. 17(4), 608–621 (2008)CrossRefMathSciNetGoogle Scholar
  4. 4.
    Cucchiara, R., Piccardi, M., Prati, A.: Detecting moving objects, ghosts, and shadows in video streams. IEEE Trans. Pattern Anal. Mach. Intell. 25(10), 1337–1342 (2003)CrossRefGoogle Scholar
  5. 5.
    Elgammal, A., Duraiswami, R., Harwood, D., Davis, L.: Background and foreground modeling using nonparametric kernel density estimation for visual surveillance. Proc. IEEE 90(7), 1151–1163 (2002)CrossRefGoogle Scholar
  6. 6.
    Kim, K., Thanarat, T., Chalidabbhognse, H., Harwood, D., Davis, L.: Real time foreground-background segmentation using codebook model. Real-Time Imaging 11, 172–185 (2005)CrossRefGoogle Scholar
  7. 7.
    Friedman, N., Russell, S.: Image segmentation in video sequences: a probabilistic approach. In: Proceedings of the Thirteenth Conference Uncertainty in Artificial Intelligence, pp. 1–3 (1997)Google Scholar
  8. 8.
    Kato, J., Watanabe, T., Joga, S., Rittscher, J., Blake, A.: An HMM-based segmentation method for traffic monitoring movies. IEEE Trans. Pattern Anal. Mach. Intell. 24(9), 1291–1296 (2002)CrossRefGoogle Scholar
  9. 9.
    Haritaoglu, I., Harwood, D., Davis, L.S.: Real-time surveillance of people and their activities. IEEE Trans. Pattern Anal. Mach. Intell. 22(8), 809–830 (2000)CrossRefGoogle Scholar
  10. 10.
    Heikkila, M., Pietikainen, M.: A texture-based method for modeling the background and detecting moving objects. IEEE Trans. Pattern Anal. Mach. Intell. 28(4), 657–662 (2006)CrossRefGoogle Scholar
  11. 11.
    Wren, C.R., Azarbayejani, A., Darrell, T., Pentland, A.P.: Real-time tracking of the human body. IEEE Trans. Pattern Anal. Mach. Intell. 19(7), 780–785 (1997)CrossRefGoogle Scholar
  12. 12.
    Briassouli, A., Kompatsiaris, I.: Robust temporal activity templates using higher order statistics. IEEE Trans. Image Process. 18(12), 2756–2768 (2009)CrossRefMathSciNetGoogle Scholar
  13. 13.
    Stauffer, C., Grimson, W.E.L.: Learning patterns of activity using real-time tracking. IEEE Trans. Pattern Anal. Mach. Intell. 22(8), 747–757 (2000)CrossRefGoogle Scholar
  14. 14.
    Patwardhan, K., Sapiro, G., Morellas, V.: Robust foreground detection in video using pixel layers. IEEE Trans. Pattern Anal. Mach. Intell. 30(4), 746–751 (2008)CrossRefGoogle Scholar
  15. 15.
    Bleicher, A.: Eyes in the sky that see too much. IEEE Spectr. 47(10), 16–16 (2010)CrossRefGoogle Scholar
  16. 16.
    Cristani, M., Farenzena, M., Bloisi, D. and Murino, V.: Background subtraction for automated multisensor surveillance: a comprehensive review. EURASIP J. Adv. Signal Process. 2010(43), 1–24 (2010)Google Scholar
  17. 17.
    Bouwmans, T.: Recent advanced statistical background modeling for foreground detection: a systematic survey. Recent Patents Comput. Sci. 4(3), 147–176 (2011)Google Scholar
  18. 18.
    Junejo, I.N., Bhutta, A.A., Foroosh, H.: Single-class SVM for dynamic scene modeling. Signal Image Video Process. 7(1), 45–52 (2013)CrossRefGoogle Scholar
  19. 19.
    Trivedi, M.M., Doshi, A.: Satellite imagery based adaptive background models and shadow suppression. Signal Image Video Process. 1(2), 119–132 (2007)CrossRefMATHGoogle Scholar
  20. 20.
    Donohoe, G.W., Hush, D.R. Ahmed, N.: Change detection for target detection and classification in video sequences. In: Proceedings of the IEEE International Conference on Acoustic Speech and Signal Processing, pp. 1084–1087 (1988)Google Scholar
  21. 21.
    Ridder, C., Munkelt, O., Kirchner, H.: Adaptive background estimation and foreground detection using Kalman-filtering. In: Proceedings of the International Conference on Recent Advances in Mechatronics, pp. 193–199 (1995)Google Scholar
  22. 22.
    Karmann, K.-P., von Brandt, A., Gerl, R.: Moving object segmentation based on adaptive reference images. In: Proceedings of the Fifth European Signal Processing Conference, pp. 951–954. Barcelona, Spain, Sept 18–21 (1990)Google Scholar
  23. 23.
    Toyama, K., Krumm, J., Brumitt, B., Meyers, B.: Wallflower: principles and practice of background maintenance. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 255–261. Kerkyra, Greece, Sept 20–27 (1999)Google Scholar
  24. 24.
    Aach, T., Kaup, A.: Statistical model-based change detection in moving video. Signal Process. 31(2), 165–180 (1993)CrossRefMATHGoogle Scholar
  25. 25.
    Bichsel, M.: Segmenting simply connected moving objects in a static scene. IEEE Trans. Pattern Anal. Mach. Intell. 16(11), 1135–1142 (1994)CrossRefGoogle Scholar
  26. 26.
    Jabri, S., Duric, Z., Wechsler, H., Rosenfeld, A.: Detection and location of people in video images using adaptive fusion of color and edge information. In: Proceedings of the IEEE International Conference on Pattern Recognition, vol. 4, pp. 627–630 (2000)Google Scholar
  27. 27.
    McKenna, S.J., Jabri, S., Duric, Z., Rosenfeld, A., Wechsler, H.: Tracking groups of people. Comput. Vis. Image Underst. 80, 42–56 (2000)CrossRefMATHGoogle Scholar
  28. 28.
    Ohta, N.: A statistical approach to background subtraction for surveillance systems. In: Proceedings of the IEEE International Conference on Computer Vision, vol. 2, pp. 481–486. Vancouver, BC, CA, July 7–14 (2001)Google Scholar
  29. 29.
    Wang, Y., Loe, K.-F., Wu, J.-K.: A dynamic conditional random field model for foreground and shadow segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 28(2), 279–289 (2006)CrossRefGoogle Scholar
  30. 30.
    Davis, J.W., Sharma, V.: Background-subtraction in thermal imagery using contour saliency. Int’l. J. Comput. Vis. 71(2), 161–181 (2007)CrossRefGoogle Scholar
  31. 31.
    Raja, Y., McKenna, S.J., Gong, S.: Colour model selection and adaptation in dynamic scenes. In: Proceedings of the European Conference on Computer Vision, pp. 460–474 (1998)Google Scholar
  32. 32.
    Stauffer, C., Grimson, W.E.L.: Adaptive mixture models for real-time tracking. In: Proceedings of the IEEE International Conference on Computer Vision Pattern Recognition, vol. 2. Fort Collins CO, USA, June 23–25 (1999)Google Scholar
  33. 33.
    Javed, O., Shafique, K., Shah, M.: A hierarchical approach to robust background subtraction using color and gradient information. In: Proceedings of the IEEE Workshop on Motion, Video, Computing, pp. 22–27, Dec 5–6 (2002)Google Scholar
  34. 34.
    Harville, M.: A framework for high-level feedback to adaptive, per-pixel, mixture-of-Gaussian background models. In: Proceedings of the European Conference on Computer Vision, pp. 37–49 (2002)Google Scholar
  35. 35.
    Withagen, P., Schutte, K., Groen, F.: Likelihood-based object detection and object tracking using color histograms and EM. In: Proceedings of the IEEE International Conference on Image Processing, pp. 589–592 (2002)Google Scholar
  36. 36.
    Hayman, E., Eklundh, J.-O.: Statistical background subtraction for a mobile observer. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 67–74. Nice, France, Oct. 13–16 (2003)Google Scholar
  37. 37.
    Zivkovic, Z.: Improved adaptive Gaussian mixture model for background subtraction. In: Proceedings of the IEEE International Conference on Pattern Recognition, vol. 2, pp. 28–31 (2004)Google Scholar
  38. 38.
    Zivkovic, Z., van der Heijden, F.: Recursive unsupervised learning of finite mixture models. IEEE Trans. Pattern Anal. Mach. Intell. 26(5), 651–656 (2004)CrossRefGoogle Scholar
  39. 39.
    Zang, Q., Klette, R.: Robust background subtraction and maintenance. In: Proceedings of the IEEE International Conference on Pattern Recognition, vol. 2, pp. 90–93 (2004)Google Scholar
  40. 40.
    Zhou, D., Zhang, H.: Modified GMM background modeling and optical flow for detection of moving objects. Proc. IEEE Intl. Conf. Syst. Man Cybern. 3, 2224–2229 (2006)Google Scholar
  41. 41.
    Lee, D.-A.: Effective gaussian mixture learning for video background subtraction. IEEE Trans. Pattern Anal. Mach. Intell. 27(5), 827–832 (2005)CrossRefGoogle Scholar
  42. 42.
    Huang, S.-S., Fu, L.-C., Hsiao, P.-Y.: Region-level motion-based background modeling and subtraction using MRFs. IEEE Trans. Image Process. 16(5), 1446–1456 (2007)CrossRefMathSciNetGoogle Scholar
  43. 43.
    Joshi, A.J., Papanikolopoulos, N.P.: Learning to detect moving shadows in dynamic environments. IEEE Trans. Pattern Anal. Mach. Intell. 30(11), 2055–2063 (2008)CrossRefGoogle Scholar
  44. 44.
    Elgammal, A.M., Harwood, D., Davis, L.: Non-parametric model for background subtraction. Proc. Eur. Conf. Comput. Vis. 1843, 751–767 (2000)Google Scholar
  45. 45.
    Seki, M., Wada, T., Fuliwara, H., Sumi, K.: Background subtraction based on cooccurrence of image variations. In: Proceedings of the IEEE International Conference on Computer Vision, Pattern Recognition, vol. 2, pp. 65–72. Madison, WI, USA, June 18–20 (2003)Google Scholar
  46. 46.
    Mittal, A., Paragios, N.: Motion-based background subtraction using adaptive kernel density estimation. In: Proceedings of the IEEE International Conference on Computer Vision, Pattern Recognition, vol. 2, pp. 302–309. Washington, DC, USA, June 27–July 2 (2004)Google Scholar
  47. 47.
    Han, B., Comaniciu, D., Davis, L.: Sequential kernel density approximation through mode propagation. In: Proceedings of the Asian Conference on Computer Vision. Jeju Island, Korea (2004)Google Scholar
  48. 48.
    Sheikh, Y., Shah, M.: Bayesian modeling of dynamic scenes for object detection. IEEE Trans. Pattern Anal. Mach. Intell. 27(11), 1778–1792 (2005)CrossRefGoogle Scholar
  49. 49.
    Parag, T., Elgammal, A., Mittal, A.: A framework for feature selection for background subtraction. In: Proceedings IEEE Internationl Conference on Computer Vision, Pattern Recognition, vol. 2, pp. 1916–1923. New York, NY, USA, June 17–22 (2006)Google Scholar
  50. 50.
    Skifstad, K., Jain, R.: Illumination independent change detection from real world image sequence. Comput. Vis. Graph. Image Process. 46, 387–399 (1989)CrossRefGoogle Scholar
  51. 51.
    Karmann, K.-P., von Brandt, A.: Moving object recognition using adaptive background memory. In: Time-Varying Image Processing and Moving Object Recognition 2: Proceedings of the Third Internationl Workshop, pp. 289–296. Florence, Italy, May 29–31 (1989)Google Scholar
  52. 52.
    Koller, D., Weber, J., Malik, J., Ogasawara, G., Rao, B., Russell, S.: Towards robust automatic traffic scene analysis in real-time. In: Proceedings of the IEEE Internationl Conference on Pattern Recognition, pp. 126–131. Jerusalem, Israel, Oct 9 (1994)Google Scholar
  53. 53.
    Horprasert, T., Harwood, D., Davis, L.S.: A statistical approach for real-time background subtraction and shadow detection. In: Proceedings of the IEEE Internationl Conference on Computer Vision, pp. 1–19 (1999)Google Scholar
  54. 54.
    Rota, N., Thonnat, M.: Video sequence interpretation for visual surveillance. In: Proceedings of the IEEE Workshop Visual Surveillance, pp. 325–332. Dublin, IL, Aug 6 (2000)Google Scholar
  55. 55.
    Li, L., Huang, W., Yu-Hua Gu, I., Tian, Q.: Statistical modeling of complex backgrounds for foreground object detection. IEEE Trans. Image Process. 13(11), 1459–1472 (2004)CrossRefGoogle Scholar
  56. 56.
    Jain, R., Militzer, D., Nagel. H.H.: Separating non-stationary from stationary scene components in a sequence of real world tv-images. In: Proceedings of the International Joint Conference on Artificial Intelligence, pp. 612–618. Cambridge, MA, USA (1977)Google Scholar
  57. 57.
    Heikkila, M., Pietikainen, M., Heikkila, J.: A texture-based method for detecting moving objects. Proc. British Mach. Vis. Conf. 1, 187–196 (2004)Google Scholar
  58. 58.
    Manzanera, A.: \(\Sigma -\Delta \) background subtraction and the Zipf law. Prog. Pattern Recogn. Image Anal. Appl. Springer 4756, 42–51 (2007)Google Scholar
  59. 59.
    Li, L., Huang, W., Gu, I., Tian, Q.: Foreground object detection from videos containing complex background. In: ACM International Conference on Multimedia, pp. 2–10. Berkeley, CA (2003)Google Scholar
  60. 60.
    KaewTraKulPong, P., Bowden, R.: An improved adaptive background mixture model for real-time tracking with shadow detection. In: Proceedings of the European Workshop Advanced Video Based Surveillance Systems (2001)Google Scholar
  61. 61.
    Brutzer, S., Hoferlin, B., Heidemann, G.: Evaluation of background subtraction techniques for video surveillance. In: Proceedings of the IEEE International Conference on Computer Vision, Pattern Recognition, pp. 1937–1944. Colorado Springs CO, (2011)Google Scholar
  62. 62.
    McFarlane, N., Schofield, C.: Segmentation and tracking of piglets in images. Mach. Vis. Appl. 8, 187–193 (1995)CrossRefGoogle Scholar
  63. 63.
    Oliver, N.M., Rosario, B., Pentland, A.P.: A Bayesian computer vision system for modeling human interactions. IEEE Trans. Pattern Anal. Mach. Intell. 22(8), 831–843 (2000)CrossRefGoogle Scholar
  64. 64.
    Maddalena, L., Petrosino, A.: A self-organizing approach to background subtraction for visual surveillance applications. IEEE Trans. Image Process. 17(7), 1168–1177 (2008)CrossRefMathSciNetGoogle Scholar
  65. 65.
    Mould, N., Havlicek, J.P.: A conservative scene model update policy. In: Proceedings of the IEEE Southwest Symposium Image Analysis Interpretation. Santa Fe, NM, Apr 22–24 (2012)Google Scholar
  66. 66.
    Mould, N., Havlicek, J.P.: A stochastic learning algorithm for pixel-level background models. In: Proceedings of the IEEE Internationl Conference Image Proceedings. Orlando, FL, Sep 30–Oct. 3 (2012)Google Scholar
  67. 67.
    Kim, K., Chalidabhongse, T.H., Harwood, D., Davis, L.: Background modeling and subtraction by codebook construction. In: Proceedings of the IEEE International Conference Image Processing, vol. 5, pp. 3061–3064, Oct 24–27 (2004)Google Scholar
  68. 68.
    Seki, M., Fujiwara, H., Sumi, K.: A robust background subtraction method for changing background. In: Proceedings of the IEEE Workshop Applications of Computer Vision, pp. 207–213 (2000)Google Scholar
  69. 69.
    Monnet, A., Mittal, A., Paragios, N., Visvanathan, R.: Background modeling and subtraction of dynamic scenes. In: Proceedings of the IEEE International Conference on Computer Vision, vol. 2, pp. 1305–1312. Nice, France, Oct 13–16 (2003)Google Scholar
  70. 70.
    Zhong, B., Liu, S., Yao, H.: Local spatial co-occurrence for background subtraction via adaptive binned kernel estimation. Proc. Asian Conf. Comput. Vis. 5996, 152–161 (2010)Google Scholar
  71. 71.
    Goyette, N., Jodoin, P.-M., Porikli, F., Konrad, J., Ishwar, P.: Changedetection.net: A new change detection benchmark dataset. In: Proceedings of the IEEE Workshop on Change Detection (CDW12) at CVPR12. Providence, RI, June 16–20 (2012) Google Scholar
  72. 72.
    Computer Vision Pattern Recognition (CVPR). Performance evaluation of tracking and surveillance (PETS). Website: ftp://ftp.pets.rdg.ac.uk/PETS2001/ (2001)
  73. 73.
    Mahadevan, Vijay, Vasconcelos, Nuno: Spatiotemporal saliency in dynamic scenes. IEEE Trans. Pattern Anal. Mach. Intell. 32(1), 171–177 (2010)CrossRefGoogle Scholar
  74. 74.
    Elhabian, S., El-Sayed, K., Ahmed, S.: Moving object detection in spatial domain using background removal techniques—state-of-art. Recent Patents Comput. Sci. 1, 32–54 (2008)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag London 2013

Authors and Affiliations

  1. 1.School of Electrical and Computer EngineeringThe University of OklahomaNormanUSA

Personalised recommendations