Submap-Based Bundle Adjustment for 3D Reconstruction from RGB-D Data

Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8753)


The key contribution of this paper is a novel submapping technique for RGB-D-based bundle adjustment. Our approach significantly speeds up 3D object reconstruction with respect to full bundle adjustment while generating visually compelling 3D models of high metric accuracy. While submapping has been explored previously for mono and stereo cameras, we are the first to transfer and adapt this concept to RGB-D sensors and to provide a detailed analysis of the resulting gain. In our approach, we partition the input data uniformly into submaps to optimize them individually by minimizing the 3D alignment error. Subsequently, we fix the interior variables and optimize only over the separator variables between the submaps. As we demonstrate in this paper, our method reduces the runtime of full bundle adjustment by 32 % on average while still being able to deal with real-world noise of cheap commodity sensors. We evaluated our method on a large number of benchmark datasets, and found that we outperform several state-of-the-art approaches both in terms of speed and accuracy. Furthermore, we present highly accurate 3D reconstructions of various objects to demonstrate the validity of our approach.


Base Node Bundle Adjustment Detect Loop Closure Bundle Adjustment Method Sparse Bundle Adjustment 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Agarwal, S., Mierle, K.: Ceres Solver: Tutorial & Reference. Google Inc.
  2. 2.
    Agarwal, S., Snavely, N., Simon, I., Seitz, S., Szeliski, R.: Building rome in a day. In: ICCV (2009)Google Scholar
  3. 3.
    Arun, K., Huang, T., Blostein, S.: Least-squares fitting of two 3-D point sets. IEEE Trans. Pattern Anal. Mach. Intell. 5, 698–700 (1987)CrossRefGoogle Scholar
  4. 4.
    Bylow, E., Sturm, J., Kerl, C., Kahl, F., Cremers, D.: Real-time camera tracking and 3D reconstruction using signed distance functions. In: RSS (2013)Google Scholar
  5. 5.
    Chiuso, A., Favaro, P., Jin, H., Soatto, S.: 3-D motion and structure from 2-D motion causally integrated over time: implementation. In: Vernon, D. (ed.) ECCV 2000. LNCS, vol. 1843, pp. 734–750. Springer, Heidelberg (2000)CrossRefGoogle Scholar
  6. 6.
    Curless, B., Levoy, M.: A volumetric method for building complex models from range images. In: SIGGRAPH (1996)Google Scholar
  7. 7.
    Davison, A.: Real-time simultaneous localisation and mapping with a single camera. In: ICCV (2003)Google Scholar
  8. 8.
    Endres, F., Hess, J., Engelhard, N., Sturm, J., Cremers, D., Burgard, W.: An evaluation of the RGB-D SLAM system. In: ICRA (2012)Google Scholar
  9. 9.
    Fuhrmann, S., Goesele, M.: Fusion of depth maps with multiple scales. ACM Trans. Graph. 30(6), 148 (2011)CrossRefGoogle Scholar
  10. 10.
    Henry, P., Krainin, M., Herbst, E., Ren, X., Fox, D.: RGB-D mapping: Using depth cameras for dense 3D modeling of indoor environments. ISER 20, 22–25 (2010)Google Scholar
  11. 11.
    Henry, P., Krainin, M., Herbst, E., Ren, X., Fox, D.: RGB-D mapping: using kinect-style depth cameras for dense 3D modeling of indoor environments. Int. J. Rob. Res. (IJRR) 31(5), 647–663 (2012)CrossRefGoogle Scholar
  12. 12.
    Kaess, M., Johannsson, H., Roberts, R., Ila, V., Leonard, J., Dellaert, F.: iSAM2: Incremental smoothing and mapping using the Bayes tree. Int. J. Rob. Res. (IJRR) 31, 217–236 (2012)Google Scholar
  13. 13.
    Klein, G., Murray, D.: Parallel tracking and mapping for small AR workspaces. In: ISMAR (2007)Google Scholar
  14. 14.
    Konolige, K.: Large-scale map-making. In: AAAI (2004)Google Scholar
  15. 15.
    Lim, J., Frahm, J.M., Pollefeys, M.: Online environment mapping. In: CVPR (2011)Google Scholar
  16. 16.
    Lourakis, M., Argyros, A.: SBA: a software package for generic sparse bundle adjustment. ACM Trans. Math. Softw. (TOMS) 36(1), 2 (2009)MathSciNetCrossRefGoogle Scholar
  17. 17.
    Newcombe, R., Davison, A., Izadi, S., Kohli, P., Hilliges, O., Shotton, J., Molyneaux, D., Hodges, S., Kim, D., Fitzgibbon, A.: KinectFusion: real-time dense surface mapping and tracking. In: ISMAR (2011)Google Scholar
  18. 18.
    Ni, K., Dellaert, F.: Multi-level submap based SLAM using nested dissection. In: IROS (2010)Google Scholar
  19. 19.
    Ni, K., Dellaert, F.: HyperSfM. In: 3DIMPVT (2012)Google Scholar
  20. 20.
    Ni, K., Steedly, D., Dellaert, F.: Out-of-core bundle adjustment for large-scale 3D reconstruction. In: ICCV (2007)Google Scholar
  21. 21.
    Pinies, P., Paz, L., Haner, S., Heyden, A.: Decomposable bundle adjustment using a junction tree. In: ICRA (2012)Google Scholar
  22. 22.
    Scherer, S., Dube, D., Zell, A.: Using depth in visual simultaneous localisation and mapping. In: ICRA (2012)Google Scholar
  23. 23.
    Sibley, G., Mei, C., Reid, I., Newman, P.: Adaptive relative bundle adjustment. In: RSS (2009)Google Scholar
  24. 24.
    Steinbruecker, F., Kerl, C., Sturm, J., Cremers, D.: Large-scale multi-resolution surface reconstruction from RGB-D sequences. In: ICCV (2013)Google Scholar
  25. 25.
    Sturm, J., Engelhard, N., Endres, F., Burgard, W., Cremers, D.: A benchmark for the evaluation of RGB-D SLAM systems. In: IROS (2012)Google Scholar
  26. 26.
    Triggs, B., McLauchlan, P.F., Hartley, R.I., Fitzgibbon, A.W.: Bundle adjustment – a modern synthesis. In: Triggs, B., Zisserman, A., Szeliski, R. (eds.) ICCV-WS 1999. LNCS, vol. 1883, pp. 298–372. Springer, Heidelberg (2000)CrossRefGoogle Scholar
  27. 27.
    Whelan, T., Kaess, M., Leonard, J., McDonald, J.: Deformation-based loop closure for large scale dense RGB-D SLAM. In: IROS (2013)Google Scholar
  28. 28.
    Wurm, K., Hornung, A., Bennewitz, M., Stachniss, C., Burgard, W.: OctoMap: a probabilistic, flexible, and compact 3D map representation for robotic systems. In: ICRA Workshop on Best Practice in 3D Perception and Modeling for Mobile Manipulation (2010)Google Scholar
  29. 29.
    Zhou, Q.Y., Koltun, V.: Dense scene reconstruction with points of interest. In: SIGGRAPH (2013)Google Scholar
  30. 30.
    Zhou, Q.Y., Miller, S., Koltun, V.: Elastic fragments for dense scene reconstruction. In: ICCV (2013)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  1. 1.TU MunichMunichGermany

Personalised recommendations