Abstract
This article concerns the problem of a dense mapping system for a robot exploring a new environment. In this scenario, a robot equipped with an RGB-D camera uses RGB and range data to build a consistent model of the environment. Firstly, dense mapping requires the selection of the data representation. Secondly, the dense mapping system has to deal with localization drift which can be corrected when loop closure is detected. In this article, we deal with both of these problems, and we make several technical contributions. We define local maps which use the Normal Distribution Transform (NDT) stored in the 2D structures to represent the local scene with varying 3D resolution. This method directly utilizes the uncertainty model of the range sensor and provides information about the accuracy of the data in the map. We also propose an architecture that utilizes pose and covisibility graphs to correct a global model of the environment after loop closure detection. We show how to integrate the dense local mapping with the pose graph and keyframes management system in the ORB-SLAM2 localization. Finally, we show the advantages of the view-dependent model over the methods that uniformly divide the space to represent objects in the environment.
Article PDF
Similar content being viewed by others
Avoid common mistakes on your manuscript.
Code Availability
Modified version of ORB-SLAM2 is available here: https://github.com/mikolajnowaczyk/ORB_SLAM2
References
Amigoni, F., Yu, W., Andre, T., Holz, D., Magnusson, M., Matteuci, M., Moon, H., Yokotsuka, M., Biggs, G., Madhavan, R.: A standard for map data representation. IEEE Robot. Autom. Mag. 25(1), 65–76 (2018)
Belter, D., Nowicki, M., Skrzypczyński, P.: Modeling spatial uncertainty of point features in feature-based RGB-d SLAM. Mach. Vis. Appl. 29(5), 827–844 (2018)
Belter, D., Piaskowski, K., Staszak, R.: Keyframe-Based local normal distribution transform occupancy maps for environment mapping. In: IEEE 23rd International Conference on Emerging Technologies and Factory Automation (ETFA), pp. 706–712 (2018)
Belter, D., Wietrzykowski, J., Skrzypczyński, P.: Employing natural terrain semantics in motion planning for a multi-legged robot. J. Intell. Robot. Syst. 93, 723–743 (2019)
Campos, C., Elvira, R., Rodríguez, J.J.G., Montiel, J.M.M., Tardós, J.D.: ORB-SLAM3: An accurate open-source library for visual visual-inertial and multi-map SLAM (2020)
Canelhas, D.R., Stoyanov, T., Lilienthal, A.J.: Sdf Tracker: A parallel algorithm for on-line pose estimation and scene reconstruction from depth images. In: IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 3671–3676 (2013)
Dai, A., Nießner, M., Zollhöfer, M., Izadi, S., Theobalt, C.: Bundlefusion: Real-time globally consistent 3D reconstruction using on-the-fly surface reintegration. ACM Trans. Graph. 36(3) (2017)
Droeschel, D., Schwarz, M., Behnke, S.: Continuous mapping and localization for autonomous navigation in rough terrain using a 3D laser scanner. Robot. Auton. Syst. 88, 104–115 (2017)
Dryanovski, I., Morris, W., Xiao, J.: Multi-volume occupancy grids: an efficient probabilistic 3D mapping model for micro aerial vehicles. In: Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 1553–1559 (2010)
Engel, J., Schöps, T., Cremers, D. Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.): LSD-SLAM: large-scale direct monocular SLAM. Springer, Cham (2014)
Fankhauser, P., Bloesch, M., Gehring, C., Hutter, M., Siegwart, R.: Robot-centric elevation mapping with uncertainty estimates. In: Kozłowski, K. (ed.) Robotics, Mobile Service, pp 433–440. World-Scientific (2014)
Fankhauser, P., Bloesch, M., Hutter, M.: Probabilistic terrain mapping for mobile robots with uncertain localization. IEEE Robot. Autom. Lett. 3(4), 3019–3026 (2018)
Grinvald, M., Furrer, F., Novkovic, T., Chung, J.J., Cadena, C., Siegwart, R., Nieto, J.: Volumetric instance-aware semantic mapping and 3D object discovery. IEEE Robot. Autom. Lett. 4 (3), 3037–3044 (2019)
Halmetschlager-Funek, G., Suchi, M., Kampel, M., Vincze, M.: An empirical evaluation of ten depth cameras. IEEE Robot. Autom. Mag. 26(1), 67–77 (2018)
Handa, A., Whelan, T., McDonald, J., Davison, A.J.: A Benchmark for RGB-D Visual Odometry, 3D Reconstruction and SLAM. In: 2014 IEEE International Conference on Robotics and Automation (ICRA), pp. 1524–1531 (2014)
Hebert, M., Caillas, C., Krotkov, E., Kweon, I.: Terrain mapping for a roving planetary explorer. In: Proceedings of IEEE International Conference on Robotics and Automation, pp. 997–1002 (1989)
Hess, W., Kohler, D., Rapp, H., Andor, D.: Real-time loop closure in 2D LIDAR SLAM. In: Proceedings of IEEE International Conference on Robotics and Automation, pp. 1271–1278 (2016)
Ho, B.J., Sodhi, P., Teixeira, P., Hsiao, M., Kusnur, T., Kaess, M.: Virtual occupancy grid map for submap-based pose graph SLAM and planning in 3D environments. In: Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 2175–2182 (2018)
Hornung, A., Wurm, K.M., Bennewitz, M., Stachniss, C., Burgard, W.: Octomap: an efficient probabilistic 3D mapping framework based on octrees. Auton. Robot. 34(3), 189–206 (2013)
Kähler, O., Adrian Prisacariu, V., Yuheng Ren, C., Sun, X., Torr, P., Murray, D.: Very high frame rate volumetric integration of depth images on mobile devices. IEEE Trans. Vis. Comput. Graph. 21 (11), 1241–1250 (2015). https://doi.org/10.1109/TVCG.2015.2459891
Kähler, O., Prisacariu, V.A., Murray, D.W. Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.): Real-time large-scale dense 3D reconstruction with loop closure. Springer International Publishing, Cham (2016)
Konolige, K., Bowman, J., Chen, J., Mihelich, P., Calonder, M., Lepetit, V., Fua, P.: View-based maps. Int. J. Robot. Res. 29(8), 941–957 (2010). https://doi.org/10.1177/0278364910370376
Kulecki, B., Mlodzikowski, K., Staszak, R., Belter, D.: Practical aspects of detection and grasping objects by a mobile manipulating robot. Industrial Robot. https://doi.org/10.1108/IR-10-2020-0242(2021)
Kweon, I., Kanade, T.: High-resolution terrain map from multiple sensor data. IEEE Trans. Pattern Anal. Mach. Intell. 14(2), 278–292 (1992)
McCormac, J., Clark, R., Bloesch, M., Davison, A., Leutenegger, S.: Fusion++: Volumetric Object-Level SLAM. In: 2018 International Conference on 3D Vision (3DV), pp. 32–41. https://doi.org/10.1109/3DV.2018.00015(2018)
McCormac, J., Handa, A., Davison, A., Leutenegger, S.: Semanticfusion: Dense 3D semantic mapping with convolutional neural networks. In: 2017 IEEE International Conference on Robotics and Automation (ICRA), pp. 4628–4635. https://doi.org/10.1109/ICRA.2017.7989538 (2017)
Meilland, M., Comport, A.I.: On unifying key-frame and voxel-based dense visual SLAM at large scales. In: Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems, pp 3677–3683, Tokyo, Japan (2013)
Millane, A., Taylor, Z., Oleynikova, H., Nieto, J., Siegwart, R., Cadena, C.: C-Blox: A scalable and consistent Tsdf-based dense mapping approach. In: 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 995–1002. https://doi.org/10.1109/IROS.2018.8593427 (2018)
Mur-Artal, R., Tardós, J.: Visual-inertial monocular SLAM with map reuse. IEEE Robot. Autom. Lett. 2(2), 796–803 (2016). https://doi.org/10.1109/LRA.2017.2653359
Mur-Artal, R., Tardós, J.: ORB-SLAM2: An open-source SLAM system for monocular, stereo and RGB-d cameras. IEEE Trans. Robot. 33(5), 1255–1262 (2017)
Oleynikova, H., Taylor, Z., Fehr, M., Siegwart, R., Nieto, J.: Voxblox: Incremental 3D Euclidean signed distance fields for on-board MAV planning. In: 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 1366–1373. https://doi.org/10.1109/IROS.2017.8202315 (2017)
Pan, J., Chitta, S., Manocha, D.: Fcl: A general purpose library for collision and proximity queries. In: 2012 IEEE International Conference on Robotics and Automation, pp. 3859–3866 (2012)
Park, J.H., Shin, Y.D., Bae, J.H., Baeg, M.H.: Spatial uncertainty model for visual features using a Kinect sensor. Sensors 12(7), 8640–8662 (2012)
Pfaff, P., Triebel, R., Burgard, W.: An efficient extension to elevation maps for outdoor terrain mapping and loop closing. Int. J. Robot. Res. 26(2), 217–230 (2007)
Saarinen, J., Andreasson, H., Stoyanov, T., Ala-Luhtala, J., Lilienthal, A.: Normal distributions transform occupancy maps: Application to large-scale online 3D mapping. In: Proceedings of the IEEE International Conference on Robotics and Automation, pp. 2233–2238 (2013)
Saarinen, J., Andreasson, H., Stoyanov, T., Lilienthal, A.J.: 3D normal distributions transform occupancy maps: an efficient representation for mapping in dynamic environments. Int. J. Robot. Res. 32(14), 1627–1644 (2013)
Steinbruecker, F., Sturm, J., Cremers, D.: Volumetric 3D mapping in real-time on a CPU. In: Proceedings of the IEEE International Conference on Robotics and Automation, pp. 2021–2028 (2014)
Strom, J., Olson, E.: Occupancy grid rasterization in large environments for teams of robots. In: Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 4271–4276 (2011)
Stückler, J., Behnke, S.: Multi-resolution surfel maps for efficient dense 3D modeling and tracking. J. Vis. Commun. Image Represent. 25(1), 137–147 (2014)
Sturm, J., Engelhard, N., Endres, F., Burgard, W., Cremers, D.: A benchmark for the evaluation of RGB-d SLAM systems. In: Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 573–580 (2012)
Sungjoon, C., Zhou, Q., Koltun, V.: Robust reconstruction of indoor scenes. In: 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5556–5565. https://doi.org/10.1109/CVPR.2015.7299195(2015)
Wang, J., Song, J., Zhao, L., Huang, S. Hutter, M., Siegwart, R. (eds.): A Submap joining based RGB-D SLAM algorithm using planes as features. Springer International Publishing, Cham (2018)
Weilharter, R., Schenk, F., Fraundorfer, F.: Globally consistent dense real-time 3D reconstruction from RGBD data. In: Welk, M., Urschler, M., Roth, P. (eds.) Proceedings of the OAGM Workshop 2018, pp. 121–127. Verlag der Technischen Universität Graz. https://doi.org/10.3217/978-3-85125-603-1-25 (2018)
Whelan, T., Leutenegger, S., Salas-Moreno, R., Glocker, B., Davison, A.J.: Elasticfusion: Dense SLAM without a Pose Graph. In: Robotics: Science and Systems (2015)
Whelan, T., McDonald, J., Kaess, M., Fallon, M., Johannsson, H., Leonard, J.: Kintinuous: Spatially Extended KinectFusion. In: RSS Workshop on RGB-D: Advanced Reasoning with Depth Cameras. Sydney, Australia (2012)
Whelan, T., Salas-Moreno, R.F., Glocker, B., Davison, A.J., Leutenegger, S.: Elasticfusion: Real-time dense SLAM and light source estimation. Int. J. Robot. Res. 35(14), 1697–1716 (2016). https://doi.org/10.1177/0278364916669237
Zieliński, K., Belter, D.: Keyframe-based dense mapping with the graph of view-dependent local maps. In: Proceedings of the IEEE International Conference on Robotics and Automation, pp. 10,744–10,750 (2020)
Funding
This work was supported by the National Centre for Research and Development (NCBR) through project LIDER/33/0176/L-8/16/NCBR/2017
Author information
Authors and Affiliations
Contributions
Dominik Belter contributed to the study conception and design and was responsible for funding acquisition. Rafal Staszak was working with the initial version of the mapping system. Local view-dependent NDT-based mapping was implemented by Krzysztof Zieliński. Mikolaj Nowaczyk integrated the local mapping system with ORB-SLAM2. Data collection and analysis were performed by all authors. All authors prepared the first draft, read and approved the final manuscript.
Corresponding author
Ethics declarations
Consent for Publication
The authors and relevant institutions approve publishing this research.
Conflict of Interests
The authors have no conflict of interests to declare.
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Electronic supplementary material
Below is the link to the electronic supplementary material.
(MP4 126 MB)
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Zieliński, K., Staszak, R., Nowaczyk, M. et al. 3D Dense Mapping with the Graph of Keyframe-Based and View-Dependent Local Maps. J Intell Robot Syst 103, 28 (2021). https://doi.org/10.1007/s10846-021-01476-1
Received:
Accepted:
Published:
DOI: https://doi.org/10.1007/s10846-021-01476-1