Ground Truth for Evaluating Time of Flight Imaging

  • Rahul Nair
  • Stephan Meister
  • Martin Lambers
  • Michael Balda
  • Hannes Hofmann
  • Andreas Kolb
  • Daniel Kondermann
  • Bernd Jähne

Abstract

In this work, we systematically analyze how good ground truth (GT) datasets for evaluating methods based on Time-of-Flight (ToF) imaging data should look like. Starting from a high level characterization of the application domains and requirements they typically have, we characterize how good datasets should look like and discuss how algorithms can be evaluated using them. Furthermore, we discuss the two different ways of obtaining ground truth data: By measurement and by simulation.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Boehler, W., Vicent, M.B., Marbs, A.: Investigating laser scanner accuracy. The International Archives of Photogrammetry, Remote Sensing and Spatial Information Sciences 34(Part 5), 696–701 (2003)Google Scholar
  2. 2.
    Geiger, A., Lenz, P., Urtasun, R.: Are we ready for autonomous driving? the kitti vision benchmark suite. In: Computer Vision and Pattern Recognition (CVPR), Providence, USA (June 2012)Google Scholar
  3. 3.
    Reynolds, M., Doboš, J., Peel, L., Weyrich, T., Brostow, G.J.: Capturing time-of-flight data with confidence. In: CVPR (2011)Google Scholar
  4. 4.
    Clark, J., Robson, S.: Accuracy of measurements made with a cyrax 2500 laser scanner against surfaces of known colour. Survey Review 37(294), 626–638 (2004)CrossRefMATHGoogle Scholar
  5. 5.
    Soudarissanane, S., Lindenbergh, R., Menenti, M., Teunissen, P.: Incidence angle influence on the quality of terrestrial laser scanning points. In: ISPRS Workshop Laserscanning (2009)Google Scholar
  6. 6.
    Lichti, D.D.: Error modelling, calibration and analysis of an am–cw terrestrial laser scanner system. ISPRS Journal of Photogrammetry and Remote Sensing 61(5), 307–324 (2007)CrossRefGoogle Scholar
  7. 7.
    Newcombe, R.A., Izadi, S., Hilliges, O., Molyneaux, D., Kim, D., Davison, A.J., Kohli, P., Shotton, J., Hodges, S., Fitzgibbon, A.: KinectFusion: Real-time dense surface mapping and tracking. In: 2011 10th IEEE International Symposium on Mixed and Augmented Reality, vol. 7, pp. 127–136 (2011)Google Scholar
  8. 8.
    Izadi, S., Newcombe, R.A., Kim, D., Hilliges, O., Molyneaux, D., Hodges, S., Kohli, P., Shotton, J., Davison, A.J., Fitzgibbon, A.: KinectFusion: Real-time dynamic 3D surface reconstruction and interaction. In: ACM SIGGRAPH 2011 Talks, p. 23. ACM (2011)Google Scholar
  9. 9.
    Khoshelham, K.: Accuracy analysis of kinect depth data. In: ISPRS Workshop Laser Scanning, vol. 38, p. 1 (2011)Google Scholar
  10. 10.
    Meister, S., Izadi, S., Kohli, P., Hämmerle, M., Rother, C., Kondermann, D.: When can we use kinectfusion for ground truth acquisition? In: 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Workshops & Tutorials (2012)Google Scholar
  11. 11.
    Besl, P.J., McKay, N.D.: Method for registration of 3-d shapes. In: Robotics-DL Tentative, International Society for Optics and Photonics, pp. 586–606 (1992)Google Scholar
  12. 12.
    Bradski, G.: The OpenCV Library. Dr. Dobb’s Journal of Software Tools (2000)Google Scholar
  13. 13.
    Maier-Hein, L., Schmidt, M., Franz, A.M., dos Santos, T.R., Seitel, A., Jähne, B., Fitzpatrick, J.M., Meinzer, H.P.: Accounting for anisotropic noise in fine registration of time-of-flight range data with high-resolution surface data. In: Jiang, T., Navab, N., Pluim, J.P.W., Viergever, M.A. (eds.) MICCAI 2010, Part I. LNCS, vol. 6361, pp. 251–258. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  14. 14.
    Schmidt, M.: Analysis, Modeling and Dynamic Optimization of 3D Time-of-Flight Imaging Systems. PhD thesis, University of Heidelberg (20011)Google Scholar
  15. 15.
    Meister, S., Kondermann, D.: Real versus realistically rendered scenes for optical flow evaluation. In: Proceedings of 14th ITG Conference on Electronic Media Technology (2011)Google Scholar
  16. 16.
    Keller, M., Kolb, A.: Real-time simulation of time-of-flight sensors. J. Simulation Practice and Theory 17, 967–978 (2009)CrossRefGoogle Scholar
  17. 17.
    Research, L.: Optics software for layout and optimization (oslo), http://www.lambdares.com
  18. 18.
    Zemax, R.: Zemax, optical design software, http://www.radiantzemax.com
  19. 19.
    Schmidt, M., Jähne, B.: A physical model of time-of-flight 3D imaging systems, including suppression of ambient light. In: Kolb, A., Koch, R. (eds.) Dyn3D 2009. LNCS, vol. 5742, pp. 1–15. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  20. 20.
    Schmidt, M.: Analysis, Modeling and Dynamic Optimization of 3D Time-of-Flight Imaging Systems. PhD thesis, University of Heidelberg (2011)Google Scholar
  21. 21.
    Geiger, A., Lenz, P., Urtasun, R.: Are we ready for autonomous driving? the kitti vision benchmark suite. In: Conference on Computer Vision and Pattern Recognition (CVPR) (2012)Google Scholar
  22. 22.
    Dal Mutto, C., Zanuttigh, P., Mattoccia, S., Cortelazzo, G.: Locally consistent toF and stereo data fusion. In: Fusiello, A., Murino, V., Cucchiara, R. (eds.) ECCV 2012 Ws/Demos, Part I. LNCS, vol. 7583, pp. 598–607. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  23. 23.
    Zhang, L., Curless, B., Seitz, S.M.: Spacetime stereo: Shape recovery for dynamic scenes. In: Proceedings of the 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 2, pp. II–367. IEEE (2003)Google Scholar
  24. 24.
    Haeusler, R., Nair, R., Kondermann, D.: (Ensemble learning for confidence measures in stereo vision)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  • Rahul Nair
    • 1
    • 2
  • Stephan Meister
    • 1
    • 2
  • Martin Lambers
    • 3
  • Michael Balda
    • 4
  • Hannes Hofmann
    • 4
  • Andreas Kolb
    • 3
  • Daniel Kondermann
    • 1
    • 2
  • Bernd Jähne
    • 1
  1. 1.Heidelberg Collaboratory for Image Processing (HCI)Heidelberg UniversityGermany
  2. 2.Intel Visual Computing InstituteSaarland UniversityGermany
  3. 3.Computer Graphics GroupUniversity of SiegenGermany
  4. 4.Metrilus GmbHErlangenGermany

Personalised recommendations