Abstract
In this paper we present a novel feature-based RGB-D camera pose optimization algorithm for real-time 3D reconstruction systems. During camera pose estimation, current methods in online systems suffer from fast-scanned RGB-D data, or generate inaccurate relative transformations between consecutive frames. Our approach improves current methods by utilizing matched features across all frames and is robust for RGB-D data with large shifts in consecutive frames. We directly estimate camera pose for each frame by efficiently solving a quadratic minimization problem to maximize the consistency of 3D points in global space across frames corresponding to matched feature points. We have implemented our method within two state-of-the-art online 3D reconstruction platforms. Experimental results testify that our method is efficient and reliable in estimating camera poses for RGB-D data with large shifts.
Article PDF
Similar content being viewed by others
Avoid common mistakes on your manuscript.
References
Keller, M.; Lefloch, D.; Lambers, M.; Izadi, S.; Weyrich, T.; Kolb, A. Real-time 3D reconstruction in dynamic scenes using point-based fusion. In: Proceedings of the International Conference on 3D Vision, 1–8, 2013.
Nießner, M.; Zollhöfer, M.; Izadi, S.; Stamminger, M. Real-time 3D reconstruction at scale using voxel hashing. ACM Transactions on Graphics Vol. 32, No. 6, Article No. 169, 2013.
Zhou, Q.-Y.; Koltun, V. Color map optimization for 3D reconstruction with consumer depth cameras. ACM Transactions on Graphics Vol. 33, No. 4, Article No. 155, 2014.
Newcombe, R. A.; Izadi, S.; Hilliges, O.; Molyneaux, D.; Kim, D.; Davison, A. J.; Kohi, P.; Shotton, J.; Hodges, S.; Fitzgibbon, A. KinectFusion: Real-time dense surface mapping and tracking. In: Proceedings of the 10th IEEE International Symposium on Mixed and Augmented Reality, 127–136, 2011.
Whelan, T.; Kaess, M.; Johannsson, H.; Fallon, M.; Leonard, J.; McDonald, J. Real-time large-scale dense RGB-D slam with volumetric fusion. The International Journal of Robotics Research Vol. 34, Nos. 4–5, 598–626, 2015.
Peasley, B.; Birchfield, S. Replacing projective data association with Lucas–Kanade for KinectFusion. In: Proceedings of the IEEE International Conference on Robotics and Automation, 638–645, 2013.
Henry, P.; Krainin, M.; Herbst, E.; Ren, X.; Fox, D. RGB-D mapping: Using Kinect-style depth cameras for dense 3D modeling of indoor environments. The International Journal of Robotics Research Vol. 31, No. 5, 647–663, 2012.
Newcombe, R. A.; Lovegrove, S. J.; Davison, A. J. DTAM: Dense tracking and mapping in real-time. In: Proceedings of the IEEE International Conference on Computer Vision, 2320–2327, 2011.
Whelan, T.; Leutenegger, S.; Salas-Moreno, R.; Glocker, B.; Davison, A. ElasticFusion: Dense SLAM without a pose graph. In: Proceedings of Robotics: Science and Systems, 11, 2015.
Whelan, T.; Johannsson, H.; Kaess, M.; Leonard, J. J.; McDonald, J. Robust real-time visual odometry for dense RGB-D mapping. In: Proceedings of the IEEE International Conference on Robotics and Automation, 5724–5731, 2013.
Zhang, K.; Zheng, S.; Yu, W.; Li, X. A depthincorporated 2D descriptor for robust and efficient 3D environment reconstruction. In: Proceedings of the 10th International Conference on Computer Science & Education, 691–696, 2015.
Endres, F.; Hess, J.; Engelhard, N.; Sturm, J.; Cremers, D.; Burgard, W. An evaluation of the RGB-D SLAM system. In: Proceedings of the IEEE International Conference on Robotics and Automation, 1691–1696, 2012.
Huang, A. S.; Bachrach, A.; Henry, P.; Krainin, M.; Maturana, D.; Fox, D.; Roy, N. Visual odometry and mapping for autonomous flight using an RGBD camera. In: Robotics Research. Christensen, H. I.; Khatib, O.; Eds. Springer International Publishing, 235–252, 2011.
Xiao, J.; Owens, A.; Torralba, A. SUN3D: A database of big spaces reconstructed using SfM and object labels. In: Proceedings of the IEEE International Conference on Computer Vision, 1625–1632, 2013.
Kümmerle, R.; Grisetti, G.; Strasdat, H.; Konolige, K.; Burgard, W. G2o: A general framework for graph optimization. In: Proceedings of the IEEE International Conference on Robotics and Automation, 3607–3613, 2011.
Roth, H.; Vona, M. Moving volume KinectFusion. In: Proceedings of British Machine Vision Conference, 1–11, 2012.
Zeng, M.; Zhao, F.; Zheng, J.; Liu, X. Octreebased fusion for realtime 3D reconstruction. Graphical Models Vol. 75, No. 3, 126–136, 2013.
Whelan, T.; Johannsson, H.; Kaess, M.; Leonard, J. J.; McDonald, J. Robust tracking for real-time dense RGB-D mapping with Kintinuous. Computer Science and Artificial Intelligence Laboratory Technical Report, MIT-CSAIL-TR-2012-031, 2012.
Henry, P.; Fox, D.; Bhowmik, A.; Mongia, R. Patch volumes: Segmentation-based consistent mapping with RGBD cameras. In: Proceedings of the International Conference on 3D Vision, 398–405, 2013.
Chen, J.; Bautembach, D.; Izadi, S. Scalable real-time volumetric surface reconstruction. ACM Transactions on Graphics Vol. 32, No. 4, Article No. 113, 2013.
Steinbrucker, F.; Kerl, C.; Cremers, D. Large-scale multiresolution surface reconstruction from RGB-D sequences. In: Proceedings of the IEEE International Conference on Computer Vision, 3264–3271, 2013.
Sturm, J.; Engelhard, N.; Endres, F.; Burgard, W.; Cremers, D. A benchmark for the evaluation of RGBD SLAM systems. In: Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems, 573–580, 2012.
Stückler, J.; Behnke, S. Multi-resolution surfel maps for efficient dense 3D modeling and tracking. Journal of Visual Communication and Image Representation Vol. 25, No. 1, 137–147, 2014.
Zhou, Q.-Y.; Koltun, V. Dense scene reconstruction with points of interest. ACM Transactions on Graphics Vol. 32, No. 4, Article No. 112, 2013.
Zhou, Q.-Y.; Miller, S.; Koltun, V. Elastic fragments for dense scene reconstruction. In: Proceedings of the IEEE International Conference on Computer Vision, 473–480, 2013
Choi, S.; Zhou, Q.-Y.; Koltun, V. Robust reconstruction of indoor scenes. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 5556–5565, 2015.
Steinbrücker, F.; Sturm, J.; Cremers, D. Realtime visual odometry from dense RGB-D images. In: Proceedings of the IEEE International Conference on Computer Vision Workshops, 719–722, 2011.
H¨ansch, R.; Weber, T.; Hellwich, O. Comparison of 3D interest point detectors and descriptors for point cloud fusion. ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences Vol. 2, No. 3, 57, 2014.
Juan, L.; Gwun, O. A comparison of SIFT, PCA-SIFT and SURF. International Journal of Image Processing Vol. 3, No. 4, 143–152, 2009.
Yan, W.; Shi, X.; Yan, X.; Wan, L. Computing openSURF on openCL and general purpose GPU. International Journal of Advanced Robotic Systems Vol. 10, No. 10, 375, 2013.
Hartley, R.; Zisserman, A. Multiple View Geometry in Computer Vision. Cambridge University Press, 2003.
Arun, K. S.; Huang, T. S.; Blostein, S. D. Leastsquares fitting of two 3-D point sets. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. PAMI-9, No. 5, 698–700, 1987.
Author information
Authors and Affiliations
Corresponding author
Additional information
This article is published with open access at Springerlink.com
Chao Wang is currently a Ph.D. candidate in the Department of Computer Science at the University of Texas at Dallas. Before that, he received his M.S. degree in computer science in 2012, and B.S. degree in automation in 2009, both from Tsinghua University. His research interests include geometric modeling, spectral geometric analysis, and 3D reconstruction of indoor environments.
Xiaohu Guo received his Ph.D. degree in computer science from Stony Brook University in 2006. He is currently an associate professor of computer science at the University of Texas at Dallas. His research interests include computer graphics and animation, with an emphasis on geometric modeling and processing, mesh generation, centroidal Voronoi tessellation, spectral geometric analysis, deformable models, 3D and 4D medical image analysis, etc. He received a prestigious National Science Foundation CAREER Award in 2012.
Other papers from this open access journal are available free of charge from http://www.springer.com/journal/41095. To submit a manuscript, please go to https://www.editorialmanager.com/cvmj.
Rights and permissions
Open Access The articles published in this journal are distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.
About this article
Cite this article
Wang, C., Guo, X. Feature-based RGB-D camera pose optimization for real-time 3D reconstruction. Comp. Visual Media 3, 95–106 (2017). https://doi.org/10.1007/s41095-016-0072-2
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s41095-016-0072-2