Background Subtraction Based on Fusion of Color and Local Patterns

  • Md Rifat Arefin
  • Farkhod Makhmudkhujaev
  • Oksam Chae
  • Jaemyun KimEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11366)


Segmentation of foreground objects using background subtraction methods is popularly used in a wide variety of application areas such as surveillance, tracking, and human pose estimation. Many of the background subtraction methods construct a background model in a pixel-wise manner using color information that is sensitive to illumination variations. In the recent past, a number of local feature descriptors have been successfully applied to overcome such issues. However, these descriptors still suffer from over-sensitivity and sometimes unable to differentiate local structures. In order to tackle the aforementioned problems of existing descriptors, we propose a novel edge based descriptor, Local Top Directional Pattern (LTDP), that represents local structures in a pattern form with aid of compass masks providing information of top local directional variations. Moreover, to strengthen the robustness of the pixel-wise background model and get benefited from each other, we combine both color and LTDP features. We evaluate the performance of our method on the publicly available change detection datasets. The results of extensive experiments demonstrate the better performance of our method compared to other state-of-the-art unsupervised methods.


Background subtraction Directional Pattern Foreground segmentation 


  1. 1.
    Aggarwal, A., Biswas, S., Singh, S., Sural, S., Majumdar, A.K.: Object tracking using background subtraction and motion estimation in MPEG videos. In: Narayanan, P.J., Nayar, S.K., Shum, H.-Y. (eds.) ACCV 2006. LNCS, vol. 3852, pp. 121–130. Springer, Heidelberg (2006). Scholar
  2. 2.
    Babaee, M., Dinh, D.T., Rigoll, G.: A deep convolutional neural network for video sequence background subtraction. Pattern Recogn. 76, 635–649 (2018)CrossRefGoogle Scholar
  3. 3.
    Barnich, O., Van Droogenbroeck, M.: ViBe: a universal background subtraction algorithm for video sequences. IEEE Trans. Image Process. 20(6), 1709–1724 (2011)MathSciNetCrossRefGoogle Scholar
  4. 4.
    Bilodeau, G.A., Jodoin, J.P., Saunier, N.: Change detection in feature space using local binary similarity patterns. In: 2013 International Conference on Computer and Robot Vision (CRV), pp. 106–112. IEEE (2013)Google Scholar
  5. 5.
    Bouwmans, T., Silva, C., Marghes, C., Zitouni, M.S., Bhaskar, H., Frelicot, C.: On the role and the importance of features for background modeling and foreground detection. Comput. Sci. Rev. 28, 26–91 (2018)MathSciNetCrossRefGoogle Scholar
  6. 6.
    De Gregorio, M., Giordano, M.: A WiSARD-based approach to CDnet. In: 2013 BRICS Congress on Computational Intelligence and 11th Brazilian Congress on Computational Intelligence (BRICS-CCI & CBIC), pp. 172–177. IEEE (2013)Google Scholar
  7. 7.
    Elgammal, A., Duraiswami, R., Harwood, D., Davis, L.S.: Background and foreground modeling using nonparametric Kernel density estimation for visual surveillance. Proc. IEEE 90(7), 1151–1163 (2002)CrossRefGoogle Scholar
  8. 8.
    Goyette, N., Jodoin, P.M., Porikli, F., Konrad, J., Ishwar, P.: Changedetection. net: a new change detection benchmark dataset. In: 2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 1–8. IEEE (2012)Google Scholar
  9. 9.
    Haines, T.S., Xiang, T.: Background subtraction with dirichletprocess mixture models. IEEE Trans. Pattern Anal. Mach. Intell. 36(4), 670–683 (2014)CrossRefGoogle Scholar
  10. 10.
    Han, B., Davis, L.S.: Density-based multifeature background subtraction with support vector machine. IEEE Trans. Pattern Anal. Mach. Intell. 34(5), 1017–1023 (2012)CrossRefGoogle Scholar
  11. 11.
    Heikkila, M., Pietikainen, M.: A texture-based method for modeling the background and detecting moving objects. IEEE Trans. Pattern Anal. Mach. Intell. 28(4), 657–662 (2006)CrossRefGoogle Scholar
  12. 12.
    Hofmann, M., Tiefenbacher, P., Rigoll, G.: Background segmentation with feedback: the pixel-based adaptive segmenter. In: 2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 38–43. IEEE (2012)Google Scholar
  13. 13.
    KaewTraKulPong, P., Bowden, R.: An improved adaptive background mixture model for real-time tracking with shadow detection. In: Remagnino, P., Jones, G.A., Paragios, N., Regazzoni, C.S. (eds.) Video-Based Surveillance Systems, pp. 135–144. Springer, Boston (2002). Scholar
  14. 14.
    Kalaivani, P., Vimala, D.: Human action recognition using background subtraction method. Int. Res. J. Eng. Technol. (IRJET) 2(3), 1032–1035 (2015)Google Scholar
  15. 15.
    Kim, J., Ramirez Rivera, A., Ryu, B., Chae, O.: Simultaneous foreground detection and classification with hybrid features. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 3307–3315 (2015)Google Scholar
  16. 16.
    Kim, J., Rivera, A.R., Kim, B., Roy, K., Chae, O.: Background modeling using adaptive properties of hybrid features. In: 2017 14th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS), pp. 1–6. IEEE (2017)Google Scholar
  17. 17.
    Kim, K., Chalidabhongse, T.H., Harwood, D., Davis, L.: Real-time foreground-background segmentation using codebook model. Real-time Imaging 11(3), 172–185 (2005)CrossRefGoogle Scholar
  18. 18.
    Lee, D.S.: Effective Gaussian mixture learning for video background subtraction. IEEE Trans. Pattern Anal. Mach. Intell. 27(5), 827–832 (2005)CrossRefGoogle Scholar
  19. 19.
    Liao, S., Zhao, G., Kellokumpu, V., Pietikäinen, M., Li, S.Z.: Modeling pixel process with scale invariant local patterns for background subtraction in complex scenes. In: 2010 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1301–1306. IEEE (2010)Google Scholar
  20. 20.
    McIvor, A.M.: Background subtraction techniques. Proc. Image Vis. Comput. 4, 3099–3104 (2000)Google Scholar
  21. 21.
    Roy, K., Kim, J., Iqbal, M.T.B., Makhmudkhujaev, F., Ryu, B., Chae, O.: An adaptive fusion scheme of color and edge features for background subtraction. In: 2017 14th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS), pp. 1–6. IEEE (2017)Google Scholar
  22. 22.
    Sajid, H., Cheung, S.C.S.: Universal multimode background subtraction. IEEE Trans. Image Process. 26(7), 3249–3260 (2017)MathSciNetCrossRefGoogle Scholar
  23. 23.
    Senior, A.W., Tian, Y.L., Lu, M.: Interactive motion analysis for video surveillance and long term scene monitoring. In: Koch, R., Huang, F. (eds.) ACCV 2010. LNCS, vol. 6468, pp. 164–174. Springer, Heidelberg (2011). Scholar
  24. 24.
    St-Charles, P.L., Bilodeau, G.A.: Improving background subtraction using local binary similarity patterns. In: 2014 IEEE Winter Conference on Applications of Computer Vision (WACV), pp. 509–515. IEEE (2014)Google Scholar
  25. 25.
    St-Charles, P.L., Bilodeau, G.A., Bergevin, R.: SuBSENSE: a universal change detection method with local adaptive sensitivity. IEEE Trans. Image Process. 24(1), 359–373 (2015)MathSciNetCrossRefGoogle Scholar
  26. 26.
    Stauffer, C., Grimson, W.E.L.: Adaptive background mixture models for real-time tracking. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 2, pp. 246–252. IEEE (1999)Google Scholar
  27. 27.
    Tian, Y., Senior, A., Lu, M.: Robust and efficient foreground analysis in complex surveillance videos. Mach. Vis. Appl. 23(5), 967–983 (2012)CrossRefGoogle Scholar
  28. 28.
    Van Droogenbroeck, M., Paquot, O.: Background subtraction: experiments and improvements for viBe. In: 2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 32–37. IEEE (2012)Google Scholar
  29. 29.
    Wang, Y., Jodoin, P.M., Porikli, F., Konrad, J., Benezeth, Y., Ishwar, P.: CDnet 2014: an expanded change detection benchmark dataset. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 387–394 (2014)Google Scholar
  30. 30.
    Wren, C.R., Azarbayejani, A., Darrell, T., Pentland, A.P.: Pfinder: real-time tracking of the human body. IEEE Trans. Pattern Anal. Mach. Intell. 19(7), 780–785 (1997)CrossRefGoogle Scholar
  31. 31.
    Zeng, D., Zhu, M., Zhou, T., Xu, F., Yang, H.: Robust background subtraction via the local similarity statistical descriptor. Appl. Sci. 7(10), 989 (2017)CrossRefGoogle Scholar
  32. 32.
    Zivkovic, Z.: Improved adaptive Gaussian mixture model for background subtraction. In: Proceedings of the 17th International Conference on Pattern Recognition, ICPR 2004, vol. 2, pp. 28–31. IEEE (2004)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.Department of Computer Science and EngineeringKyung Hee UniversityYongin-siRepublic of Korea

Personalised recommendations