Abstract
To improve the robustness and efficiency of RGBD video segmentation, we propose a novel video segmentation method combining multi-scale plane extraction and hierarchical graph-based video segmentation. Firstly, to reduce depth data noise, we extract plane structures of 3D RGBD point clouds in three levels including voxel, pixel and neighborhood with geometry and color features. To solve uneven distribution of depth data and object occlusion problem, we further propose multi-scale voxel based plane fusion algorithm and use amodal completion strategy to improve plane extraction performance. Then hierarchical graph-based RGBD video segmentation is used to segment the rest of the non-plane pixels. Finally, we fuse above plane extraction and video segmentation results to get final RGBD video scene segmentation results. The qualitative and quantitative results of plane extraction and RGBD scene video segmentation show the effectiveness of proposed methods.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Paris, S., Durand, F.: A topological approach to hierarchical segmentation using mean shift. In: CVPR (2007)
Shi, J., Malik, J.: Normalized cuts and image segmentation. TPAMI 22(8), 888–905 (2000)
Sharon, E., Galun, M., Sharon, D., Basri, R., Brandt, A.: Hierarchy and adaptivity in segmenting visual scenes. Nature 442(7104), 810–813 (2006)
Felzenszwalb, P.F., Huttenlocher, D.P.: Efficient graph-based image segmentation. IJCV 2(59), 167–181 (2004)
Stuckler, J., Behnke, S.: Efficient dense rigid-body motion segmentation and estimation in RGB-D video. IJCV 113(3), 233–245 (2015)
Song, J.K., Gao, L.L., Pusca M.M., et al.: Joint graph learning and video segmentation via multiple cues and topology calibration. In: ACM MM (2016)
Xu, C., Corso, J.J.: Evaluation of super-voxel methods for early video processing. In: CVPR (2012)
Corso, J.J., Sharon, E., et al.: Efficient multilevel brain tumor segmentation with integrated Bayesian model classification. IEEE Trans. Med. Imaging 27(5), 629–640 (2008)
Felzenszwalb, P.F., Huttenlocher, D.P.: Efficient graph-based image segmentation. IJCV 59(2), 167–181 (2004)
Gupta, S., Arbeláez, P., Malik, J.: Perceptual organization and recognition of indoor scenes from RGB-D images. In: CVPR (2013)
Grundmann, M., Kwatra, V., et al.: Efficient hierarchical graph-based video segmentation. In: CVPR (2010)
Fowlkes, C., Belongie, S., et al.: Spectral grouping using the Nystrom method. TPAMI 26(2), 214–225 (2004)
Steven, H., Stan B., et al.: Efficient hierarchical graph-based segmentation of RGBD videos. In: CVPR (2014)
Wang, Z., Liu, H., Qian, Y., Xu, T.: Real-time plane segmentation and obstacle detection of 3D point clouds for indoor scenes. In: Fusiello, A., Murino, V., Cucchiara, R. (eds.) ECCV 2012. LNCS, vol. 7584, pp. 22–31. Springer, Heidelberg (2012). doi:10.1007/978-3-642-33868-7_3
Wang, Z., Liu, H., Wang, X.D., Qian, Y.L.: Segment and label indoor scene based on RGB-D for the visually impaired. In: MMM (2014)
Holz, D., Holzer, S., Rusu, R.B., Behnke, S.: Real-time plane segmentation using RGB-D cameras. In: Röfer, T., Mayer, N.,Michael, Savage, J., Saranlı, U. (eds.) RoboCup 2011. LNCS (LNAI), vol. 7416, pp. 306–317. Springer, Heidelberg (2012). doi:10.1007/978-3-642-32060-6_26
Dube, D., Zell, A.: Real-time plane extraction from depth images with the randomized Hough transform. In: ICCV Workshops (2011)
Liu, H., Wang, J., Qian, Y. L., Wang, X.D.: iSee: obstacle detection and feedback system for the blind. In: UbiComp (2015)
Acknowledgments
This work is supported in part by Beijing Natural Science Foundation (4142051) and National Key Technology R&D Program of China (2014BAK15B02).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this paper
Cite this paper
Liu, H., Wang, J., Wang, X., Qian, Y. (2017). Efficient Multi-scale Plane Extraction Based RGBD Video Segmentation. In: Amsaleg, L., Guðmundsson, G., Gurrin, C., Jónsson, B., Satoh, S. (eds) MultiMedia Modeling. MMM 2017. Lecture Notes in Computer Science(), vol 10132. Springer, Cham. https://doi.org/10.1007/978-3-319-51811-4_50
Download citation
DOI: https://doi.org/10.1007/978-3-319-51811-4_50
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-51810-7
Online ISBN: 978-3-319-51811-4
eBook Packages: Computer ScienceComputer Science (R0)