Detection and Segmentation of Moving Objects from Dynamic RGB and Depth Images

  • Naotomo TatematsuEmail author
  • Jun Ohya
  • Larry Davis
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8971)


This paper proposes a method that combines Temporal Modified-RANSAC(TMR) with a fixation-based segmentation algorithm for reconstructing the 3D structure of moving and still objects that are tracked in video and depth image sequences acquired by moving Kinect© and/or range finders First, we compute 3D optical flow of feature points. Second, TMR classifies all flows into consistent 3D flow sets for the background and each moving object; simultaneously, the rotation matrix and the translation vector for each 3D flow set are computed. Then, an improved fixation-based segmentation method segments each object’s area. Finally, dense 3D models for the background and each moving object are constructed along with each object’s rotation matrix and translation vector in each frame. Experiments using multiple moving objects in color and depth image sequences acquired by Kinect(c) demonstrate the effectiveness of our proposed method.


Fixation-based segmentation Kinect Segmentation Temporal modified RANSAC Reconstruction 


  1. 1.
    Heikkila, M., Pietikainen, M.: A texture-based method for modeling the background and detecting moving objects. IEEE Trans. Pattern Anal. Mach. Intell. 28, 657–662 (2006)CrossRefGoogle Scholar
  2. 2.
    Sugaya, Y., Kanatani, K.: Extracting moving objects from a moving camera video sequence. In: Proceedings of the 10th Symposium on Sensing via Image Information, pp. 279–284 (2004)Google Scholar
  3. 3.
    Ogale, A., Fermuller, C., Aloimonos, Y.: Motion segmentation using occlusions. IEEE Trans. Pattern Anal. Mach. Intell. 27, 988–992 (2005)CrossRefGoogle Scholar
  4. 4.
    Demirdjian, D., Darrell, T.: Motion estimation from disparity images. In: Proceedings of International Conference on Computer Vision, vol. 1, pp. 213–218 (2001)Google Scholar
  5. 5.
    Agrawal, M., Knolige, K.: Real-time localization in outdoor environments using stereo vision and inexpensive gps. In: 18th International Conference on Pattern Recognition, vol. 3. pp. 1063–1068 (2006)Google Scholar
  6. 6.
    Agrawal, M., Konolige, K., Iocchi, L.: Real-time detection of independent motion using stereo. In: IEEE Workshop on WACV/MOTION (2005)Google Scholar
  7. 7.
    Xie, Y., Ohya, J.: A method for detecting multiple independently moving objects from the sequences acquired by active stereo cameras and estimating the cameras’ egomotion. J. Inst. Image Electron. Eng. Jpn. 39(39), 163–174 (2010)Google Scholar
  8. 8.
    Tatematsu, N., Ohya, J.: Study of temporal modified-ransac based method for the extraction and 3d shape reconstruction of moving objects from dynamic stereo images and for estimating the camera pose. In: IS&T-SPIE Electronic Imaging 2011. volume 7878, pp. E1–E9 (2011)Google Scholar
  9. 9.
    Tatematsu, N., Ohya, J.: Accurate dense 3d reconstruction of moving and still objects from dynamic stereo sequences based on temporal modified-ransac and feature-cut. In: IS&T-SPIE Electronic Imaging 2012. volume 8301, pp. 05–15 (2012)Google Scholar
  10. 10.
    Mishra, A.K., Aloimonos, Y.: Visual segmentation of “simple” objects for robots. In: Robotics Science and Systems Conference (2011)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2015

Authors and Affiliations

  1. 1.Waseda UniversityTokyoJapan
  2. 2.The University of MarylandCollege ParkUSA

Personalised recommendations