Towards Semantic Scene Analysis with Time-of-Flight Cameras

  • Dirk Holz
  • Ruwen Schnabel
  • David Droeschel
  • Jörg Stückler
  • Sven Behnke
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6556)


For planning grasps and other object manipulation actions in complex environments, 3D semantic information becomes crucial. This paper focuses on the application of recent 3D Time-of-Flight (ToF) cameras in the context of semantic scene analysis. For being able to acquire semantic information from ToF camera data, we a) pre-process the data including outlier removal, filtering and phase unwrapping for correcting erroneous distance measurements, and b) apply a randomized algorithm for detecting shapes such as planes, spheres, and cylinders. We present experimental results that show that the robustness against noise and outliers of the underlying RANSAC paradigm allows for segmenting and classifying objects in 3D ToF camera data captured in natural mobile manipulation setups.


Point Cloud Planar Model Object Candidate Phase Unwrap Laser Range Scan 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Capel, D.P.: An effective bail-out test for RANSAC consensus scoring. In: Proc. British Machine Vision Conf., pp. 629–638 (2005)Google Scholar
  2. 2.
    Fischler, M.A., Bolles, R.C.: Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Commun. ACM 24(6), 381–395 (1981)MathSciNetCrossRefGoogle Scholar
  3. 3.
    Frey, B.J., Koetter, R., Petrovic, N.: Very Loopy Belief Propagation for Unwrapping Phase Images. In: Neural Information Processing Systems Conference (NIPS), Algorithms & Architectures (2001)Google Scholar
  4. 4.
    Graham, R.L.: An Efficient Algorithm for Determining the Convex Hull of a Finite Planar Set. Information Processing Letters 1(4), 132–133 (1972)CrossRefzbMATHGoogle Scholar
  5. 5.
    Matas, J., Chum, O.: Randomized RANSAC with t(d, d) test. In: Proc. of the British Machine Vision Conference 2002, BMVC (2002)Google Scholar
  6. 6.
    May, S., Droeschel, D., Holz, D., Fuchs, S., Malis, E., Nüchter, A., Hertzberg, J.: Three-dimensional mapping with time-of-flight cameras. Journal of Field Robotics, Special Issue on Three-Dimensional Mapping, Part 2 26(11-12), 934–965 (2009)Google Scholar
  7. 7.
    Mount, D., Arya, S.: ANN: A library for approximate nearest neighbor searching. In: Proc. of the 2nd Annual Fall Workshop on Computational Geometry (1997)Google Scholar
  8. 8.
    Nüchter, A., Hertzberg, J.: Towards semantic maps for mobile robots. Robotics and Autonomous Systems 56(11), 915–926 (2008)CrossRefGoogle Scholar
  9. 9.
    Ohno, K., Nomura, T., Tadokoro, S.: Real-time robot trajectory estimation and 3d map construction using 3d camera. In: Proc. of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (2006)Google Scholar
  10. 10.
    Rusu, R.B., Blodow, N., Marton, Z.C., Beetz, M.: Close-range Scene Segmentation and Reconstruction of 3D Point Cloud Maps for Mobile Manipulation in Human Environments. In: Proc. of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (2009)Google Scholar
  11. 11.
    Schnabel, R., Wahl, R., Klein, R.: Efficient RANSAC for Point-Cloud Shape Detection. Computer Graphics Forum 26(2), 214–226 (2007)CrossRefGoogle Scholar
  12. 12.
    Sheh, R., Kadous, M.W., Sammut, C.: On building 3d maps using a range camera: Applications to rescue robotics. Technical report, UNSW, Sydney, Australia (2006)Google Scholar
  13. 13.
    Torr, P.H., Zisserman, A.: MLESAC: a new robust estimator with application to estimating image geometry. Computer Vision and Image Understanding 78(1), 138–156 (2000)CrossRefGoogle Scholar
  14. 14.
    Weingarten, J.W., Grüner, G., Siegwart, R.: A State-of-the-Art 3D Sensor for Robot Navigation. In: Proc. of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (2004)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • Dirk Holz
    • 1
  • Ruwen Schnabel
    • 2
  • David Droeschel
    • 1
  • Jörg Stückler
    • 1
  • Sven Behnke
    • 1
  1. 1.Institute of Computer Science VIUniversity of BonnGermany
  2. 2.Institute of Computer Science IIUniversity of BonnGermany

Personalised recommendations