Springer Nature is making SARS-CoV-2 and COVID-19 research free. View research | View latest news | Sign up for updates

Real-time background subtraction-based video surveillance of people by integrating local texture patterns


This paper presents a real-time surveillance system for detecting and tracking people, which takes full advantage of local texture patterns, under a stationary monocular camera. A novel center-symmetric scale invariant local ternary pattern feature is put forward to combine with pattern kernel density estimation for building a pixel-level-based background model. The background model is then used to detect moving foreground objects on every newly captured frame. A variant of a fast human detector that utilizes local texture patterns is adopted to look for human objects from the foreground regions, and it is assisted by a head detector, which is proposed to find in advance the candidate locations of human, to reduce computational costs. Each human object is given a unique identity and is represented by a spatio-color-texture object model. The real-time performance of tracking is achieved by a fast mean-shift algorithm coupled with several efficient occlusion-handling techniques. Experiments on challenging video sequences show that the proposed surveillance system can run in real-time and is quite robust in segmenting and tracking people in complex environments that include appearance changes, abrupt motion, occlusions, illumination variations and clutter.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7


  1. 1.

    Allili, M., Ziou, D.: Active contours for video object tracking using region, boundary and shape information. Signal Image Video Process. 1(2), 101–117 (2007)

  2. 2.

    Babenko, B., Yang, M.H., Belongie, S.: Robust object tracking with online multiple instance learning. IEEE Trans. Pattern Anal. Mach. Intell. 33(8), 1619–1632 (2011)

  3. 3.

    Barnich, O., Van Droogenbroeck, M.: ViBe: a universal background subtraction algorithm for video sequences. IEEE Trans. Image Process. 20(6), 1709–1724 (2011)

  4. 4.

    Comaniciu, D., Ramesh, V., Meer, P.: Kernel-based object tracking. IEEE Trans. Pattern Anal. Mach. Intell. 25(5), 564–575 (2003)

  5. 5.

    Fisher, R.B.: The PETS04 surveillance ground-truth data sets. In: Proceedings of IEEE Workshop on Performance Evaluation of Tracking and Surveillance, pp. 1–5 (2004)

  6. 6.

    Haritaoglu, I., Harwood, D., Davis, L.S.: \( \text{ W }^4\): real-time surveillance of people and their activities. IEEE Trans. Pattern Anal. Mach. Intell. 22(8), 809–830 (2000)

  7. 7.

    Heikkilä, M., Pietikäinen, M., Schmid, C.: Description of interest regions with local binary patterns. Pattern Recognit. 42(3), 425–436 (2009)

  8. 8.

    KaewTraKulPong, P., Bowden, R.: An improved adaptive background mixture model for real-time tracking with shadow detection. In: Proceedings of 2nd European Workshop on Advanced Video-Based Surveillance Systems, pp. 1–5 (2001)

  9. 9.

    Kalal, Z., Mikolajczyk, K., Matas, J.: Tracking–learning-detection. IEEE Trans. Pattern Anal. Mach. Intell. 34(7), 1409–1422 (2012)

  10. 10.

    Kim, K., Chalidabhongse, T., Harwood, D., Davis, L.: Real-time foreground–background segmentation using codebook model. Real-Time Imaging 11(3), 172–185 (2005)

  11. 11.

    Kwon, J., Lee, K.M.: Tracking by sampling trackers. In: Proceedings of the IEEE International Conference Computer Vision, pp. 1195–1202 (2011)

  12. 12.

    Li, L., Huang, W., Gu, I., Tian, Q.: Foreground object detection from videos containing complex background. In: Proceedings of the ACM Conference Multimedia, pp. 2–10 (2003)

  13. 13.

    Liao, S., Zhao, G., Kellokumpu, V., Pietikainen, M., Li, S.Z.: Modeling pixel process with scale invariant local patterns for background subtraction in complex scenes. In: Proceedings of IEEE Conference Computer Vision and Pattern Recognition, pp. 1301–1306 (2010)

  14. 14.

    Lipton, A.J., Fujiyoshi, H., Patil, R.S.: Moving target classification and tracking from real-time video. In: Proceedings of IEEE Workshop on Applications of Computer Vision, pp. 8–14 (1998)

  15. 15.

    Mahadevan, V., Vasconcelos, N.: Spatiotemporal saliency in dynamic scenes. IEEE Trans. Pattern Anal. Mach. Intell. 32(1), 171–177 (2010)

  16. 16.

    Narayana, M., Hanson, A., Learned-Miller, E.: Background modeling using adaptive pixelwise kernel variances in a hybrid feature space. In: Proceedings of IEEE Conference Computer Vision and Pattern Recognition, pp. 2104–2111 (2012)

  17. 17.

    Ojala, T., Pietikäinen, M., Mäenpää, T.: Multi-resolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Trans. Pattern Anal. Mach. Intell. 24(7), 971–987 (2002)

  18. 18.

    Shan, D., Zhang, C.: Visual tracking using IPCA and sparse representation. Signal Image Video Process (2013). doi:10.1007/s11760-013-0525-3. http://link.springer.com/article/10.1007/s11760-013-0525-3

  19. 19.

    Siebel, N.T., Maybank, S.J.: Fusion of multiple tracking algorithms for robust people tracking. In: Proceedings of the European Conference Computer Vision, pp. 373–387 (2002)

  20. 20.

    Varma, M., Zisserman, A.: A statistical approach to texture classification from single images. Int. J. Comput. Vis. 62(1), 61–81 (2005)

  21. 21.

    Wu, J., Geyer, C., Rehg, J.M.: Real-time human detection using contour cues. In: Proceedings of IEEE International Conference Robotics and Automation, pp. 860–867 (2011)

  22. 22.

    Wu, J., Rehg, J.M.: CENTRIST: a visual descriptor for scene categorization. IEEE Trans. Pattern Anal. Mach. Intell. 33(8), 1489–1501 (2011)

  23. 23.

    Wu, Y., Lim, J., Yang, M.H.: Online object tracking: a benchmark. In: Proceedings of IEEE Conference Computer Vision and Pattern Recognition (2013)

  24. 24.

    Zabih, R., Woodfill, J.: Non-parametric local transforms for computing visual correspondence. In: Proceedings of the European Conference Computer Vision, pp. 151–158 (1994)

Download references


This research is supported by NSFC-Guangdong Joint Fund (U1135003), the Natural Science Foundation of China (61370186, 6110008), the Industry-academy-research Project of Guangdong (2012B091000104, 2012B091100410), the Special Foundation of Industry Development for Biology, Internet, New Energy and New Material of Shenzhen (JC201104220324A), and the Fundamental Research Funds for the Central Universities (2010620003161035).

Author information

Correspondence to Ning Liu.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Wu, H., Liu, N., Luo, X. et al. Real-time background subtraction-based video surveillance of people by integrating local texture patterns. SIViP 8, 665–676 (2014). https://doi.org/10.1007/s11760-013-0576-5

Download citation


  • Local texture patterns
  • Background modeling
  • Human detection
  • People tracking
  • Spatio-color-texture representation