Abstract
Applications based on colored 3-D data sequences suffer from lack of efficient algorithms for transformation estimation and key points extraction to perform accurate registration and sensor localization either in the 2-D or 3-D domain. Therefore, we propose a real-time RGB-D registration pipeline, named R3P, presented in processing layers. In this paper, we present an evaluation of several algorithm combinations for each layer, to optimize the registration and sensor localization for specific applications. The resulting dynamic reconfigurability of R3P makes it suitable as a front-end system for any SLAM reconstruction algorithm. Evaluation results on several public datasets reveal that R3P delivers real-time registration with 59 fps and high accuracy with the relative pose error (for a time span of 40 frames) for rotation and translation of \(0.5^\circ \) and 8 mm, respectively. All the heterogeneous dataset and implementations are publicly available under an open-source license [21].
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
The Kinect device is commercially available from Microsoft and the professional laser scanner is offered by FARO Corporation.
References
Javan Hemmat, H., et al.: Real-time planar segmentation of depth images: from 3D edges to segmented planes. J. Electron. Imaging 24(5), 051008 (2015)
Mur-Artal, R., Montiel, J.M.M., Tards, J.D.: ORB-SLAM: a versatile and accurate monocular SLAM system. IEEE Trans. Robot. 31(5), 1147–1163 (2015)
Lu, Y., Song, D.: Visual navigation using heterogeneous landmarks and unsupervised geometric constraints. IEEE Trans. Robot. 31(3), 736–749 (2015)
Bouaziz, S., Pauly, M.: Dynamic 2D/3D registration for the kinect. In: ACM SIGGRAPH Courses, SIGGRAPH 2013, pp. 21:1–21:14. ACM, New York (2013)
Henry, P., et al.: RGB-D mapping: using kinect-style depth cameras for dense 3D modeling of indoor environments. Int. J. Robot. Res. 31(5), 647–663 (2012)
Newcombe, R.A., Lovegrove, S., Davison, A.: DTAM: dense tracking and mapping in real-time. In: IEEE International Conference on Computer Vision, pp. 2320–2327 (2011)
Labbe, M., Michaud, F.: Appearance-based loop closure detection for online large-scale and long-term operation. IEEE Trans. Robot. 29(3), 734–745 (2013)
Endres, F., Hess, J., Sturm, J., Cremers, D., Burgard, W.: 3-D mapping with an RGB-D camera. IEEE Trans. Robot. 30(1), 177–187 (2014)
Hu, G., Huang, S., Zhao, L., Alempijevic, A., Dissanayake, G.: A robust RGB-D slam algorithm. In: International Conference on Intelligent Robots and Systems (IROS), pp. 1714–1719 (2012)
Andreasson, H., Stoyanov, T.: Real time registration of RGB-D data using local visual features and 3D-NDT registration. In: International Conference on Robotics and Automation (2012)
Stckler, J., Behnke, S.: Model learning and real-time tracking using multiresolution surfel maps. In: Proceedings of the AAAI Conference on Artificial Intelligence (2012)
Steinbrucker, F., Sturm, J., Cremers, D.: Real-time visual odometry from dense RGB-D images. In: ICCV Workshops, pp. 719–722. IEEE (2011)
Dib, A., et al.: A real time visual slam for RGB-D cameras based on chamfer distance and occupancy grid. In: International Conference on Advanced Intelligent Mechatronics, pp. 652–657 (2014)
Kerl, C., Sturm, J., Cremers, D.: Robust odometry estimation for RGB-D cameras. In: ICRA (2013)
Choi, C., Trevor, A.J.B., Christensen, H.I.: RGB-D edge detection and edge-based registration. In: IROS, pp. 1568–1575. IEEE (2013)
Lu, Y., Song, D.: Robust RGB-D odometry using point and line features. In: IEEE International Conference on Computer Vision, ICCV (2015)
Morell-Gimenez, V., et al.: A comparative study of registration methods for RGB-D video of static scenes. Sensors 14(5), 8547 (2014)
Seifi, S., et al.: DeReEs: real-time registration of RGB-D images using image-based feature detection and robust 3D correspondence estimation and refinement. In: 29th International Conference on Image and Vision Computing, pp. 136–141 (2014)
Yousif, K., et al.: A real-time RGB-D registration and mapping approach by heuristically switching between photometric and geometric information. In: International Conference on Information Fusion (2014)
Figueroa, N., Dong, H., Saddik, A.E.: A combined approach toward consistent reconstructions of indoor spaces based on 6D RGB-D odometry and kinect fusion. ACM Trans. Intell. Syst. Technol. 6(2), 14:1–14:10 (2015)
R3P-related codes, docs and dataset. https://gitlab.com/HaniJH/R3P/tree/master. Accessed 9 Sept 2016
Bradski, G.: OpenCV: an open source computer vision library. Dr. Dobbs Journal of Software Tools (2000)
Rusu, R., Cousins, S.: 3D is here: point cloud library (PCL). In: IEEE International Conference on Robotics and Automation (ICRA), pp. 1–4 (2011)
Handa, A., Whelan, T., McDonald, J., Davison, A.: A benchmark for RGB-D visual odometry, 3D reconstruction and SLAM. In: IEEE ICRA (2014)
Sturm, J., Engelhard, N., Endres, F., Burgard, W., Cremers, D.: A benchmark for the evaluation of RGB-D slam systems. In: Intelligent Robot Systems (IROS) (2012)
Shotton, J., et al.: Scene coordinate regression forests for camera relocalization in RGB-D images. In: Computer Vision and Pattern Recognition (IEEE CVPR) (2013)
Javan Hemmat, H., Bondarev, E., Dubbelman, G., de With, P.H.N.: Improved ICP-based pose estimation by distance-aware 3D mapping. In: 9th International Conference on Computer Vision Theory and Applications, pp. 360–367 (2014)
Javan Hemmat, H., Bondarev, E., de With, P.H.N.: Exploring distance-aware weighting strategies for accurate reconstruction of voxel-based 3D synthetic models. In: Gurrin, C., Hopfgartner, F., Hurst, W., Johansen, H., Lee, H., O’Connor, N. (eds.) MMM 2014. LNCS, vol. 8325, pp. 412–423. Springer, Heidelberg (2014). doi:10.1007/978-3-319-04114-8_35
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer International Publishing AG
About this paper
Cite this paper
Javan Hemmat, H., Bondarev, E., de With, P.H.N. (2016). R\(^{3}\)P: Real-time RGB-D Registration Pipeline. In: Blanc-Talon, J., Distante, C., Philips, W., Popescu, D., Scheunders, P. (eds) Advanced Concepts for Intelligent Vision Systems. ACIVS 2016. Lecture Notes in Computer Science(), vol 10016. Springer, Cham. https://doi.org/10.1007/978-3-319-48680-2_34
Download citation
DOI: https://doi.org/10.1007/978-3-319-48680-2_34
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-48679-6
Online ISBN: 978-3-319-48680-2
eBook Packages: Computer ScienceComputer Science (R0)