Online 3D Reconstruction and 6-DoF Pose Estimation for RGB-D Sensors

Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8925)


In this paper, we propose an approach to Simultaneous Localization and Mapping (SLAM) for RGB-D sensors. Our system computes 6-DoF pose and sparse feature map of the environment. We propose a novel keyframe selection scheme based on the Fisher information, and new loop closing method that utilizes feature-to-landmark correspondences inspired by image-based localization. As a result, the system effectively mitigates drift that is frequently observed in visual odometry system. Our approach gives lowest relative pose error amongst any other approaches tested on public benchmark dataset. A set of 3D reconstruction results on publicly available RGB-D videos are presented.


Simultaneous Localization and Mapping RGB-D SLAM 

Supplementary material

336125_1_En_16_MOESM1_ESM.mp4 (4 mb)
Supplementary material (MP4 4,068 KB)


  1. 1.
    Abt, M., Welch, W.J.: Fisher information and maximum-likelihood estimation of covariance parameters in gaussian stochastic processes. Canadian Journal of Statistics 26(1), 127–137 (1998)MathSciNetCrossRefzbMATHGoogle Scholar
  2. 2.
    Bylow, E., Sturm, J., Kerl, C., Kahl, F., Cremers, D.: Real-time camera tracking and 3d reconstruction using signed distance functions. In: RSS (2013)Google Scholar
  3. 3.
    Cummins, M., Newman, P.: Fab-map: Probabilistic localization and mapping in the space of appearance. The International Journal of Robotics Research 27(6), 647–665 (2008)CrossRefGoogle Scholar
  4. 4.
    Eade, E., Fong, P., Munich, M.E.: Monocular graph slam with complexity reduction. In: IROS, pp. 3017–3024. IEEE (2010)Google Scholar
  5. 5.
    Endres, F., Hess, J., Engelhard, N., Sturm, J., Cremers, D., Burgard, W.: An evaluation of the rgb-d slam system. In: ICRA, pp. 1691–1696. IEEE (2012)Google Scholar
  6. 6.
    Henry, P., Krainin, M., Herbst, E., Ren, X., Fox, D.: Rgb-d mapping: Using kinect-style depth cameras for dense 3d modeling of indoor environments. The International Journal of Robotics Research 31(5), 647–663 (2012)CrossRefGoogle Scholar
  7. 7.
    Irschara, A., Zach, C., Frahm, J.M., Bischof, H.: From structure-from-motion point clouds to fast location recognition. In: CVPR, pp. 2599–2606. IEEE (2009)Google Scholar
  8. 8.
    Kerl, C., Sturm, J., Cremers, D.: Dense visual slam for rgb-d cameras. In: Proc. of the Int. Conf. on Intelligent Robot Systems (IROS) (2013)Google Scholar
  9. 9.
    Kerl, C., Sturm, J., Cremers, D.: Robust odometry estimation for rgb-d cameras. In: ICRA (2013)Google Scholar
  10. 10.
    Klein, G., Murray, D.: Parallel tracking and mapping for small ar workspaces. In: ISMAR, pp. 225–234. IEEE (2007)Google Scholar
  11. 11.
    Konolige, K., Agrawal, M.: Frameslam: From bundle adjustment to real-time visual mapping. IEEE Transactions on Robotics 24(5), 1066–1077 (2008)CrossRefGoogle Scholar
  12. 12.
    Konolige, K., Bowman, J.: Towards lifelong visual maps. In: IROS, pp. 1156–1163. IEEE (2009)Google Scholar
  13. 13.
    Li, Y., Snavely, N., Huttenlocher, D.P.: Location recognition using prioritized feature matching. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010, Part II. LNCS, vol. 6312, pp. 791–804. Springer, Heidelberg (2010) CrossRefGoogle Scholar
  14. 14.
    Lim, H., Sinha, S.N., Cohen, M.F., Uyttendaele, M.: Real-time image-based 6-dof localization in large-scale environments. In: CVPR, pp. 1043–1050. IEEE (2012)Google Scholar
  15. 15.
    Lim, J., Frahm, J.M., Pollefeys, M.: Online environment mapping. In: CVPR, pp. 3489–3496. IEEE (2011)Google Scholar
  16. 16.
    Lowe, D.G.: Object recognition from local scale-invariant features. In: ICCV, vol. 2, pp. 1150–1157. IEEE (1999)Google Scholar
  17. 17.
    Newcombe, R.A., Davison, A.J., Izadi, S., Kohli, P., Hilliges, O., Shotton, J., Molyneaux, D., Hodges, S., Kim, D., Fitzgibbon, A.: Kinectfusion: real-time dense surface mapping and tracking. In: ISMAR, pp. 127–136. IEEE (2011)Google Scholar
  18. 18.
    Nister, D., Stewenius, H.: Scalable recognition with a vocabulary tree. In: CVPR, vol. 2, pp. 2161–2168. IEEE (2006)Google Scholar
  19. 19.
    Sattler, T., Leibe, B., Kobbelt, L.: Fast image-based localization using direct 2d-to-3d matching. In: ICCV, pp. 667–674. IEEE (2011)Google Scholar
  20. 20.
    Schindler, G., Brown, M., Szeliski, R.: City-scale location recognition. In: CVPR, pp. 1–7. IEEE (2007)Google Scholar
  21. 21.
    Snavely, N., Seitz, S.M., Szeliski, R.: Skeletal graphs for efficient structure from motion. In: CVPR (2008)Google Scholar
  22. 22.
    Strasdat, H., Davison, A.J., Montiel, J., Konolige, K.: Double window optimisation for constant time visual slam. In: ICCV, pp. 2352–2359. IEEE (2011)Google Scholar
  23. 23.
    Stuckler, J., Behnke, S.: Integrating depth and color cues for dense multi-resolution scene mapping using rgb-d cameras. In: IEEE Conf. on Multisensor Fusion and Integration for Intelligent Systems, pp. 162–167. IEEE (2012)Google Scholar
  24. 24.
    Sturm, J., Engelhard, N., Endres, F., Burgard, W., Cremers, D.: A benchmark for the evaluation of rgb-d slam systems. In: IROS, October 2012Google Scholar
  25. 25.
    Triggs, B., McLauchlan, P.F., Hartley, R.I., Fitzgibbon, A.W.: Bundle adjustment – a modern synthesis. In: Triggs, B., Zisserman, A., Szeliski, R. (eds.) ICCV-WS 1999. LNCS, vol. 1883, pp. 298–372. Springer, Heidelberg (2000) CrossRefGoogle Scholar
  26. 26.
    Wendel, A., Irschara, A., Bischof, H.: Natural landmark-based monocular localization for mavs. In: ICRA, pp. 5792–5799. IEEE (2011)Google Scholar
  27. 27.
    Zhang, G., Dong, Z., Jia, J., Wong, T.-T., Bao, H.: Efficient non-consecutive feature tracking for structure-from-motion. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010, Part V. LNCS, vol. 6315, pp. 422–435. Springer, Heidelberg (2010) CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  1. 1.Seoul National UniversitySeoulKorea
  2. 2.Hanyang UniversitySeoulKorea

Personalised recommendations