Real-Time Plane Segmentation Using RGB-D Cameras

  • Dirk Holz
  • Stefan Holzer
  • Radu Bogdan Rusu
  • Sven Behnke
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7416)


Real-time 3D perception of the surrounding environment is a crucial precondition for the reliable and safe application of mobile service robots in domestic environments. Using a RGB-D camera, we present a system for acquiring and processing 3D (semantic) information at frame rates of up to 30Hz that allows a mobile robot to reliably detect obstacles and segment graspable objects and supporting surfaces as well as the overall scene geometry. Using integral images, we compute local surface normals. The points are then clustered, segmented, and classified in both normal space and spherical coordinates. The system is tested in different setups in a real household environment.

The results show that the system is capable of reliably detecting obstacles at high frame rates, even in case of obstacles that move fast or do not considerably stick out of the ground. The segmentation of all planes in the 3D data even allows for correcting characteristic measurement errors and for reconstructing the original scene geometry in far ranges.


Point Cloud Collision Avoidance Latent Dirichlet Allocation Integral Image Distance Space 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Droeschel, D., Holz, D., Stückler, J., Behnke, S.: Using Time-of-Flight Cameras with Active Gaze Control for 3D Collision Avoidance. In: Proc. of the IEEE International Conference on Robotics and Automation, ICRA, pp. 4035–4040 (2010)Google Scholar
  2. 2.
    Endres, F., Plagemann, C., Stachniss, C., Burgard, W.: Unsupervised discovery of object classes from range data using latent dirichlet allocation. In: Proc. of Robotics: Science and Systems (2009)Google Scholar
  3. 3.
    Holz, D., Lörken, C., Surmann, H.: Continuous 3D Sensing for Navigation and SLAM in Cluttered and Dynamic Environments. In: Proc. of the International Conference on Information Fusion (FUSION), pp. 1469–1475 (2008)Google Scholar
  4. 4.
    Holz, D., Schnabel, R., Droeschel, D., Stückler, J., Behnke, S.: Towards Semantic Scene Analysis with Time-of-Flight Cameras. In: Ruiz-del-Solar, J., Chown, E., Ploeger, P.G. (eds.) RoboCup 2010. LNCS (LNAI), vol. 6556, pp. 121–132. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  5. 5.
    Lai, K., Fox, D.: 3D laser scan classification using web data and domain adaptation. In: Proc. of Robotics: Science and Systems (2009)Google Scholar
  6. 6.
    May, S., Droeschel, D., Holz, D., Fuchs, S., Malis, E., Nüchter, A., Hertzberg, J.: Three-dimensional mapping with time-of-flight cameras. Journal of Field Robotics, Special Issue on Three-Dimensional Mapping, Part 2 26(11-12), 934–965 (2009)Google Scholar
  7. 7.
    Memarsadeghi, N., Mount, D.M., Netanyahu, N.S., Moigne, J.L.: A fast implementation of the isodata clustering algorithm. International Journal of Computational Geometry and Applications 17, 71–103 (2007)MathSciNetzbMATHCrossRefGoogle Scholar
  8. 8.
    Nüchter, A., Hertzberg, J.: Towards semantic maps for mobile robots. Robotics and Autonomous Systems 56(11), 915–926 (2008)CrossRefGoogle Scholar
  9. 9.
    Rusu, R.B., Blodow, N., Marton, Z.C., Beetz, M.: Close-range Scene Segmentation and Reconstruction of 3D Point Cloud Maps for Mobile Manipulation in Human Environments. In: Proc. of the IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS, pp. 1–6 (2009)Google Scholar
  10. 10.
    Steder, B., Rusu, R.B., Konolige, K., Burgard, W.: NARF: 3D range image features for object recognition. In: Workshop on Defining and Solving Realistic Perception Problems in Personal Robotics at the IEEE/RSJ Int. Conf. on Intelligent Robots and Systems, IROS (2010)Google Scholar
  11. 11.
    Triebel, R., Shin, J., Siegwart, R.: Segmentation and unsupervised part-based discovery of repetitive objects. In: Proc. of Robotics: Science and Systems (2010)Google Scholar
  12. 12.
    Wulf, O., Arras, K.O., Christensen, H.I., Wagner, B.: 2D Mapping of Cluttered Indoor Environments by Means of 3D Perception. In: Proc. of the IEEE Intl. Conf. on Robotics and Automation, ICRA, pp. 4204–4209 (2004)Google Scholar
  13. 13.
    Yuan, F., Swadzba, A., Philippsen, R., Engin, O., Hanheide, M., Wachsmuth, S.: Laser-based navigation enhanced with 3D time-of-flight data. In: Proc. of the IEEE Intl. Conf. on Robotics and Automation (ICRA), pp. 2231–2237 (2008)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Dirk Holz
    • 1
  • Stefan Holzer
    • 2
  • Radu Bogdan Rusu
    • 3
  • Sven Behnke
    • 1
  1. 1.Autonomous Intelligent Systems GroupUniversity of BonnGermany
  2. 2.Department of Computer ScienceTechnical University of Munich (TUM)Germany
  3. 3.Willow Garage, Inc.Menlo ParkUSA

Personalised recommendations