An Efficient Combination of RGB and Depth for Background Subtraction

  • Van-Toi NguyenEmail author
  • Hai Vu
  • Thanh-Hai Tran
Conference paper
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 341)


This paper describes a new method for background subtraction using RGB and depth data from a Microsoft Kinect sensor. In the first step of the proposed method, noises are removed from depth data using the proposed noise model. Denoising procedure help improving the performance of background subtraction and also avoids major limitations of RGB mostly when illumination changes. Background subtraction then is solved by combining RGB and depth features instead of using individual RGB or depth data. The fundamental idea in our combination strategy is that when depth measurement is reliable, the background subtraction from depth taken priority over all. Otherwise, RGB is used as alternative. The proposed method is evaluated on a public benchmark dataset which is suffered from common problems of the background subtraction such as shadows, reflections and camouflage. The experimental results show better performances in comparing with state-of-the-art. Furthermore, the proposed method is successful with a challenging task such as extracting human fall-down event in a RGB-D image sequence. Therefore, the foreground segmentation is feasibility for the further task such as tracking and recognition.


Microsoft Kinect Background subtractions Color segmentation Depth in use RBG-D combinations 



The research leading to this paper was supported by the National Project B2013.01.41 “Study and develop an abnormal event recognition system based on computer vision techniques”. We would like to thank the project and people involved in this project.


  1. 1.
    Grimson, W.E.L., Stauffer, C., Romano, R., Lee, L.: Using adaptive tracking to classify and monitor activities in a site. In: Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 22–29. IEEE (1998)Google Scholar
  2. 2.
    Stauffer, C., Grimson, W.E.L.: Adaptive background mixture models for real-time tracking. In: Computer Vision and Pattern Recognition (1999)Google Scholar
  3. 3.
    Microsoft: Microsoft kinect: (2013)
  4. 4.
    Camplani, M., Salgado, L.: Background foreground segmentation with RGB-D Kinect data: an efficient combination of classifiers. J. Vis. Commun. Image Represent. (2014)Google Scholar
  5. 5.
    Schiller, I., Koch, R.: Improved Video Segmentation by Adaptive Combination of Depth Keying. Lecture Notes in Computer Science (Image Analysis), vol. 6688, pp. 59–68 (2011)Google Scholar
  6. 6.
    Gordon, G., Darrell, T., Harville, M., Woodfill, J.: Background estimation and removal based on range and color. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Number June (1999)Google Scholar
  7. 7.
    Bouwmans, T.: Recent advanced statistical background modeling for foreground detection—a systematic survey. RPCS 4(3), 147–176 (2011)Google Scholar
  8. 8.
    Goyette, N., Jodoin, P., Porikli, F., Konrad, J., Ishwar, P.: Changedetection .net: a new change detection benchmark dataset. In: Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 1–8 (2012)Google Scholar
  9. 9.
    McFarlane, N.J.B., Schofield, C.P.: Segmentation and tracking of piglets in images. Mach. Vis. Appl. 8(3), 187–193 (1995)CrossRefGoogle Scholar
  10. 10.
    Zheng, J., Wang, Y., Nihan, N., Hallenbeck, M.: Extracting roadway background image: mode-based approach. Transp. Res. Rec. 1944(1), 82–88 (2006)CrossRefGoogle Scholar
  11. 11.
    Wren, C., Azarbayejani, A., Darrell, T., Pentland, A.: Pfinder: real-time tracking of the human body. IEEE Trans. Pattern Anal. Mach. Intell. 19(7), 780–785 (1997)CrossRefGoogle Scholar
  12. 12.
    Sigari, M.H., Mozayani, N., Pourreza, H.R.: Fuzzy running average and fuzzy background subtraction: concepts and application. Int. J. Comput. Sci. Netw. Secur. 8(2), 138–143 (2008)Google Scholar
  13. 13.
    El Baf, F., Bouwmans, T., Vachon, B.: Type-2 fuzzy mixture of Gaussians model: application to background modeling. In: Advances in Visual Computing, pp. 772–781. Springer (2008)Google Scholar
  14. 14.
    Zhang, H., Xu, D.: Fusing color and texture features for background model. In: Fuzzy Systems and Knowledge Discovery: Third International Conference, vol. 3, pp. 887–893 (2006)Google Scholar
  15. 15.
    El Baf, F., Bouwmans, T., Vachon, B.: Fuzzy integral for moving object detection. In: 2008 IEEE International Conference on Fuzzy Systems (IEEE World Congress on Computational Intelligence), June 2008, vol. 1, pp. 1729–1736. IEEE (2008)Google Scholar
  16. 16.
    Butler, D., Sridharan, S., Jr., V.M.B.: Real-time adaptive background segmentation. In: IEEE International Conference on Acoustics, Speech, and Signal Processing, III-349 (2003)Google Scholar
  17. 17.
    Kim, K., Chalidabhongse, T.H., Harwood, D., Davis, L.: Real-time foreground-background segmentation using codebook model. Real-Time Imaging 11(3), 172–185 (2005)CrossRefGoogle Scholar
  18. 18.
    Culibrk, D., Marques, O., Socek, D., Kalva, H., Furht, B.: Neural network approach to background modeling for video object segmentation. IEEE Trans. Neural Netw./A Publ. IEEE Neural Netw. Counc. 18(6), 1614–1627 (2007)CrossRefGoogle Scholar
  19. 19.
    Messelodi, S., Modena, C.M., Segata, N., Zanin, M.: A Kalman filter based background updating algorithm robust to sharp illumination changes. In: Image Analysis and Processing, ICIAP 2005, pp. 163–170 (2005)Google Scholar
  20. 20.
    Toyama, K., Krumm, J., Brumitt, B., Meyers, B.: Wallflower: principles and practice of background maintenance. In: The Proceedings of the Seventh IEEE International Conference on Computer Vision, pp. 255–261 (1999)Google Scholar
  21. 21.
    Stormer, A., Hofmann, M., Rigoll, G.: Depth gradient based segmentation of overlapping foreground objects in range images. In: 13th Conference on Information Fusion (FUSION), pp. 1–4 (2010)Google Scholar
  22. 22.
    Frick, A., Kellner, F., Bartczak, B., Koch, R.: Generation of 3D-TV LDV-content with time-of-flight camera. In: 3DTV Conference: The True Vision-Capture, Transmission and Display of 3D Video, vol. 2, pp. 1–4 (2009)Google Scholar
  23. 23.
    Fernandez-Sanchez, E.J., Diaz, J., Ros, E.: Background subtraction based on color and depth using active sensors. Sensors (Basel, Switzerland) 13(7), 8895–8915 (2013)Google Scholar
  24. 24.
    McGuinness, K., OConnor, N.E.: A comparative evaluation of interactive segmentation algorithms. Pattern Recognit. 43(2), 434–444 (2010)CrossRefzbMATHGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  1. 1.International Research Institute MICA, HUST - CNRS/UMI-2954 - Grenoble INPHanoi University of Science & TechnologyHanoiVietnam
  2. 2.University of Information and Communication TechnologyThai Nguyen UniversityThai NguyenVietnam

Personalised recommendations