RGB-D Mapping: Using Depth Cameras for Dense 3D Modeling of Indoor Environments

  • Peter Henry
  • Michael Krainin
  • Evan Herbst
  • Xiaofeng Ren
  • Dieter Fox
Part of the Springer Tracts in Advanced Robotics book series (STAR, volume 79)

Abstract

RGB-D cameras are novel sensing systems that capture RGB images along with per-pixel depth information. In this paper we investigate how such cameras can be used in the context of robotics, specifically for building dense 3D maps of indoor environments. Such maps have applications in robot navigation, manipulation, semantic mapping, and telepresence. We present RGB-D Mapping, a full 3D mapping system that utilizes a novel joint optimization algorithm combining visual features and shape-based alignment. Visual and depth information are also combined for view-based loop closure detection, followed by pose optimization to achieve globally consistent maps.We evaluate RGB-D Mapping on two large indoor environments, and show that it effectively combines the visual and shape information available from RGB-D cameras.

Keywords

Point Cloud Indoor Environment Scale Invariant Feature Transform Iterative Close Point Iterative Close Point Algorithm 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Akbarzadeh, A., Frahm, J.M., Mordohai, P., Clipp, B., Engels, C., Gallup, D., Merrell, P., Phelps, M., Sinha, S., Talton, B., Wang, L., Yang, Q., Stewenius, H., Yang, R., Welch, G., Towles, H., Nistér, D., Pollefeys, M.: Towards urban 3D reconstruction from video. In: Proc. of the Third International Symposium on 3D Data Processing, Visualization and Transmission (3DPVT) (2006)Google Scholar
  2. 2.
    Besl, P.J., McKay, N.D.: A method for registration of 3-d shapes. IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI) 14(2) (1992)Google Scholar
  3. 3.
  4. 4.
    Chen, Y., Medioni, G.: Object modeling by registration of multiple range images. Image Vision Comput. 10(3), 145–155 (1992)CrossRefGoogle Scholar
  5. 5.
    Clemente, L., Davison, A., Reid, I., Neira, J., Tardós, J.: Mapping large loops with a single hand-held camera. In: Proc. of Robotics: Science and Systems, RSS (2007)Google Scholar
  6. 6.
    Debevec, P., Taylor, C.J., Malik, J.: Modeling and rendering architecture from photographs: A hybrid geometryand image-based approach. In: SIGGRAPH (1996)Google Scholar
  7. 7.
    Furukawa, Y., Curless, B., Seitz, S., Szeliski, R.: Reconstructing building interiors from images. In: Proc. of the International Conference on Computer Vision, ICCV (2009)Google Scholar
  8. 8.
    Furukawa, Y., Ponce, J.: Patch-based multi-view stereo software (PMVS), http://grail.cs.washington.edu/software/pmvs/
  9. 9.
    Grisetti, G., Grzonka, S., Stachniss, C., Pfaff, P., Burgard, W.: Estimation of accurate maximum likelihood maps in 3D. In: Proc. of the IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS (2007)Google Scholar
  10. 10.
    Grisetti, G., Stachniss, C., Grzonka, S., Burgard, W.: A tree parameterization for efficiently computing maximum likelihood maps using gradient descent. In: Proc. of Robotics: Science and Systems, RSS (2007)Google Scholar
  11. 11.
    Horn, B.K.P.: Closed-form solution of absolute orientation using unit quaternions. J. Opt. Soc. Am. A 4(4), 629–642 (1987)MathSciNetCrossRefGoogle Scholar
  12. 12.
  13. 13.
    Johnson, A., Kang, S.B.: Registration and integration of textured 3-d data. In: International Conference on Recent Advances in 3-D Digital Imaging and Modeling (3DIM 1997), pp. 234–241 (May 1997)Google Scholar
  14. 14.
    Kim, Y.M., Theobalt, C., Diebel, J., Kosecka, J., Micusik, B., Thrun, S.: Multi-view image and ToF sensor fusion for dense 3D reconstruction. In: Workshop on 3-D Digital Imaging and Modeling (3DIM) (2009)Google Scholar
  15. 15.
    Konolige, K.: Projected texture stereo. In: Proc. of the IEEE International Conference on Robotics & Automation, ICRA (2010)Google Scholar
  16. 16.
    Konolige, K., Agrawal, M.: FrameSLAM: From bundle adjustment to real-time visual mapping. IEEE Transactions on Robotics 25(5) (2008)Google Scholar
  17. 17.
    Krainin, M., Henry, P., Ren, X., Fox, D.: Manipulator and object tracking for in hand 3D object modeling. Technical Report UW-CSE-10-09-01, University of Washington (2010), http://www.cs.washington.edu/ai/Mobile_Robotics/projects/hand_tracking/
  18. 18.
    Lowe, D.: Discriminative image features from scale-invariant keypoints. International Journal of Computer Vision 60(2) (2004)Google Scholar
  19. 19.
    May, S., Dröschel, D., Holz, D., Fuchs, E., Malis, S., Nüchter, A., Hertzberg, J.: Three-dimensional mapping with time-of-flight cameras. Journal of Field Robotics (JFR) 26(11-12) (2009)Google Scholar
  20. 20.
  21. 21.
    Newman, P., Sibley, G., Smith, M., Cummins, M., Harrison, A., Mei, C., Posner, I., Shade, R., Schröter, D., Murphy, L., Churchill, W., Cole, D., Reid, I.: Navigating, recognising and describing urban spaces with vision and laser. International Journal of Robotics Research (IJRR) 28(11-12) (2009)Google Scholar
  22. 22.
    Nister, D.: An efficient solution to the five-point relative pose problem. IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI) 26(6), 756–777 (2004)CrossRefGoogle Scholar
  23. 23.
    Pfister, H., Zwicker, M., van Baar, J., Gross, M.: Surfels: Surface elements as rendering primitives. In: ACM Transactions on Graphics (Proc. of SIGGRAPH) (2000)Google Scholar
  24. 24.
    Pollefeys, M., Nister, D., Frahm, J.-M., Akbarzadeh, A., Mordohai, P., Clipp, B., Engels, C., Gallup, D., Kim, S.-J., Merrell, P., Salmi, C., Sinha, S., Talton, B., Wang, L., Yang, Q., Stewenius, H., Yang, R., Welch, G., Towles, H.: Detailed real-time urban 3D reconstruction from video. International Journal of Computer Vision 72(2), 143–167 (2008)CrossRefGoogle Scholar
  25. 25.
  26. 26.
    Ramos, F., Fox, D., Durrant-Whyte, H.: CRF-matching: Conditional random fields for feature-based scan matching. In: Proc. of Robotics: Science and Systems, RSS (2007)Google Scholar
  27. 27.
    Rusinkiewicz, S., Levoy, M.: Efficient variants of the ICP algorithm. In: Third International Conference on 3D Digital Imaging and Modeling (2001)Google Scholar
  28. 28.
    Segal, A., Haehnel, D., Thrun, S.: Generalized-ICP. In: Proc. of Robotics: Science and Systems, RSS (2009)Google Scholar
  29. 29.
    Sharp, G.C., Lee, S.W., Wehe, D.K.: ICP registration using invariant features. IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI) 24(1), 90–102 (2002)CrossRefGoogle Scholar
  30. 30.
    Snavely, N., Seitz, S., Szeliski, R.: Photo tourism: Exploring photo collections in 3D. In: ACM Transactions on Graphics (Proc. of SIGGRAPH) (2006)Google Scholar
  31. 31.
    Thrun, S., Burgard, W., Fox, D.: A real-time algorithm for mobile robot mapping with applications to multi-robot and 3D mapping. In: Proc. of the IEEE International Conference on Robotics & Automation, ICRA (2000)Google Scholar
  32. 32.
    Triebel, R., Burgard, W.: Improving simultaneous mapping and localization in 3D using global constraints. In: Proc. of the National Conference on Artificial Intelligence, AAAI (2005)Google Scholar
  33. 33.
    Wu, C.: SiftGPU: A GPU implementation of scale invariant feature transform, SIFT (2007), http://cs.unc.edu/~ccwu/siftgpu

Copyright information

© Springer-Verlag GmbH Berlin Heidelberg 2014

Authors and Affiliations

  • Peter Henry
    • 1
  • Michael Krainin
    • 1
  • Evan Herbst
    • 1
  • Xiaofeng Ren
    • 2
  • Dieter Fox
    • 1
  1. 1.Department of Computer Science & EngineeringUniversity of WashingtonSeattleUSA
  2. 2.Intel Labs SeattleSeattleUSA

Personalised recommendations