Machine Vision and Applications

, Volume 29, Issue 2, pp 313–327 | Cite as

Geometric primitive refinement for structured light cameras

  • Peter FuersattelEmail author
  • Simon Placht
  • Andreas Maier
  • Christian Riess
Original Paper


Three-dimensional camera systems are useful sensors for several higher level vision tasks like navigation, environment mapping or dimensioning. However, the raw 3-D data is for many algorithms not the best representation. Instead, many methods rely on a more abstract scene description, where the scene is represented as a collection of geometric primitives like planes, spheres, or even more complex models. These primitives are commonly estimated on individual point measurements, which are directly affected by the measurement errors of the sensor. This paper proposes a method for refining the parameters of geometric primitives for structured light cameras with spatially varying patterns. In contrast to fitting the model to a set of 3-D point measurements, we propose to use all information that belongs to a particular object simultaneously to directly fit the model to the image, without the detour of calculating disparities. To this end, we propose a novel calibration procedure which recovers the unknown internal parameters of the range sensors and reconstructs the unknown projected pattern. This is particularly necessary for consumer-structured light sensors whose internals are not available to the user. After calibration, a coarse model fit is considerably refined by comparing the observed structured light dot pattern with a predicted virtual view of the projected virtual pattern. The calibration and the refinement methods are evaluated on three geometric primitives: planes, spheres, and cuboids. The orientations of the plane normals are improved by more than 60%, and plane distances by more than 30% compared to the baseline. Furthermore, the initial parameters of spheres and cuboids are refined by more than 50 and 30%. The method also operates robustly on highly textured plane segments, and at ranges that have not been considered during calibration.


Structured light Range imaging Geometric primitives 


  1. 1.
    Bergamasco, F., Albarelli, A., Rodola, E., Torsello, A.: Rune-tag: a high accuracy fiducial marker with strong occlusion resilience. In: 2011 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 113–120 (2011)Google Scholar
  2. 2.
    Bird, N., Papanikolopoulos, N.: Optimal image-based euclidean calibration of structured light systems in general scenes. IEEE Trans. Autom. Sci. Eng. 8(4), 815–823 (2011)CrossRefGoogle Scholar
  3. 3.
    Birk, A., Pathak, K., Vaskevicius, N., Pfingsthorn, M., Poppinga, J., Schwertfeger, S.: Surface representations for 3d mapping. Künstl. Intell. 24(3), 249–254 (2010)CrossRefGoogle Scholar
  4. 4.
    Biswas, J., Veloso, M.: Depth camera based indoor mobile robot localization and navigation. In: IEEE International Conference on Robotics and Automation (ICRA), pp. 1697–1702 (2012)Google Scholar
  5. 5.
    Borrmann, D., Elseberg, J., Lingemann, K., Nüchter, A.: The 3d hough transform for plane detection in point clouds: a review and a new accumulator design. 3D Res 2(2), 3 (2011)CrossRefGoogle Scholar
  6. 6.
    Feng, C., Taguchi, Y., Kamat, V.R.: Fast plane extraction in organized point clouds using agglomerative hierarchical clustering. In: IEEE International Conference on Robotics and Automation (ICRA), pp. 6218–6225 (2014)Google Scholar
  7. 7.
    Fernndez-Moral, E., Mayol-Cuevas, W., Arvalo, V., Gonzlez-Jimnez, J.: Fast place recognition with plane-based maps. In: 2013 IEEE International Conference on Robotics and Automation, pp. 2719–2724 (2013)Google Scholar
  8. 8.
    Fuersattel, P., Placht, S., Balda, M., Schaller, C., Hofmann, H., Maier, A., Riess, C.: A comparative error analysis of current time-of-flight sensors. IEEE Trans. Comput. Imaging 2(1), 27–41 (2016)MathSciNetCrossRefGoogle Scholar
  9. 9.
    Geiger, A., Moosmann, F., Car, Ö., Schuster, B.: Automatic camera and range sensor calibration using a single shot. In: IEEE International Conference on Robotics and Automation (ICRA), pp. 3936–3943. IEEE (2012)Google Scholar
  10. 10.
    Georgiev, K., Creed, R.T., Lakaemper, R.: Fast plane extraction in 3d range data based on line segments. In: IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 3808–3815 (2011)Google Scholar
  11. 11.
    Herrera, D., Kannala, J., Heikkilä, J.: Accurate and practical calibration of a depth and color camera pair. In: International Conference on Computer Analysis of Images and Patterns, pp. 437–445. Springer (2011)Google Scholar
  12. 12.
    Holz, D., Behnke, S.: Approximate triangulation and region growing for efficient segmentation and smoothing of range images. Robot. Auton. Syst. 62(9), 1282–1293 (2014)CrossRefGoogle Scholar
  13. 13.
    Horn, B.K.P.: Closed-form solution of absolute orientation using unit quaternions. J. Opt. Soc. Am. 4(4), 629 (1987)CrossRefGoogle Scholar
  14. 14.
    Khoshelham, K., Elberink, S.O.: Accuracy and resolution of kinect depth data for indoor mapping applications. Sensors 12(2), 1437–1454 (2012)CrossRefGoogle Scholar
  15. 15.
    McIlroy, P., Izadi, S., Fitzgibbon, A.: Kinectrack: 3d pose estimation using a projected dense dot pattern. IEEE Trans. Vis. Comput. Graph. 20(6), 839–851 (2014)CrossRefGoogle Scholar
  16. 16.
    Moreno, D., Taubin, G.: Simple, accurate, and robust projector-camera calibration. In: 2012 Second International Conference on 3D Imaging, Modeling, Processing, Visualization and Transmission (3DIMPVT), pp. 464–471 (2012)Google Scholar
  17. 17.
    Mörwald, T., Richtsfeld, A., Prankl, J., Zillich, M., Vincze, M.: Geometric data abstraction using b-splines for range image segmentation. In: IEEE International Conference on Robotics and Automation (ICRA), pp. 148–153 (2013)Google Scholar
  18. 18.
    Nguyen, A., Le, B.: 3d point cloud segmentation: a survey. In: 6th International Conference on Robotics, Automation and Mechatronics (RAM), pp. 225–230 (2013)Google Scholar
  19. 19.
    Placht, S., Fürsattel, P., Mengue, E.A., Hofmann, H., Schaller, C., Balda, M., Angelopoulou, E.: Rochade: robust checkerboard advanced detection for camera calibration. In: Computer Vision—ECCV 2014, Lecture Notes in Computer Science, vol. 8692, pp. 766–779. Springer (2014)Google Scholar
  20. 20.
    Poppinga, J., Vaskevicius, N., Birk, A., Pathak, K.: Fast plane detection and polygonalization in noisy 3d range images. In: IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 3378–3383 (2008)Google Scholar
  21. 21.
    Ryan Fanello, S., Rhemann, C., Tankovich, V., Kowdle, A., Orts Escolano, S., Kim, D., Izadi, S.: Hyperdepth: learning depth from structured light without matching. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2016)Google Scholar
  22. 22.
    Salas-Moreno, R.F., Glocken, B., Kelly, P.H.J., Davison, A.J.: Dense planar slam. In: IEEE International Symposium on Mixed and Augmented Reality (ISMAR), pp. 157–164 (2014)Google Scholar
  23. 23.
    Taguchi, Y., Jian, Y.D., Ramalingam, S., Feng, C.: Point-plane slam for hand-held 3d sensors. In: IEEE International Conference on Robotics and Automation (ICRA), pp. 5182–5189 (2013)Google Scholar
  24. 24.
    Trevor, A.J.B., Gedikli, S., Rusu, R.B., Christensen, H.I.: Efficient organized point cloud segmentation with connected components. In: Semantic Perception Mapping and Exploration (SPME) (2013)Google Scholar
  25. 25.
    Weingarten, J., Siegwart, R.: 3d slam using planar segments. In: IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 3062–3067 (2006)Google Scholar
  26. 26.
    Yamazaki, S., Mochimaru, M., Kanade, T.: Simultaneous self-calibration of a projector and a camera using structured light. In: 2011 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops (CVPR Workshops), pp. 60–67 (2011)Google Scholar
  27. 27.
    Ye, Y., Song, Z.: A practical means for the optimization of structured light system calibration parameters. In: 2016 IEEE International Conference on Image Processing (ICIP), pp. 1190–1194 (2016)Google Scholar
  28. 28.
    Zhang, Z.: A flexible new technique for camera calibration. IEEE Trans. Pattern Anal. Mach. Intell. 22(11), 1330–1334 (2000)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature 2017

Authors and Affiliations

  1. 1.Friedrich-Alexander-University Erlangen-NürnbergErlangenGermany
  2. 2.Metrilus GmbHErlangenGermany

Personalised recommendations