TOF Cameras and Stereo Systems: Comparison and Data Fusion

  • Carlo Dal  Mutto
  • Pietro Zanuttigh
  • Guido M. Cortelazzo


Time-Of-Flight range cameras and stereo vision systems (for simplicity called TOF cameras and stereo systems now on) are both depth acquisition devices capable to collect 3M information of dynamic scenes. In spite they can be used for similar tasks in many applications, it would not be appropriate to view the two systems as alternate or even competitive choices, since their characteristics and actual capability are markedly different. Indeed synergically combining together TOF cameras and stereo systems is a rather intriguing and useful option. This chapter firstly compares Time-Of-Flight range cameras and stereo vision systems, and then addresses the problem of fusing the data produced by the two systems. Because of the many aspects involved, the comparison is all but straightforward and could be certainly organized in different ways. The proposed one represents a systematic approach.


Stereo Vision Conjugate Point Match Cost Depth Discontinuity Markov Random Field Model 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    T. Luhmann, S. Robson, S. Kyle, I. Harley, Close Range Photogrammetry: Principles, Techniques and Applications, (Wiley, Chichester, 2007), pp. 528 ISBN 978-0-47010-633-4 Google Scholar
  2. 2.
    E.M. Mikhail, J.S. Bethel, J.C. McGlone, Introduction to Modern Photogrammetry, (Wiley, New York, 2001), pp. 496 ISBN 978-0-47130-924-6Google Scholar
  3. 3.
    Point Grey Research, Inc.,
  4. 4.
    TYZX, Inc.,
  5. 5.
    Middlebury Stereo Vision Page,
  6. 6.
    R. Szeliski, Computer Vision, Algorithms and Applications, Series: Texts in Computer Science, 1st edn. (Springer, New York, 2011) pp. 812 ISBN 978-1-84882-934-3Google Scholar
  7. 7.
    G. Bradski, A. Kaehler, Learning OpenCV: Computer Vision with the OpenCV Library, 1st edn. (O’Reilly Media, Sebastopol, 2008) pp. 555 ISBN 978-0596516130Google Scholar
  8. 8.
    J.-Y. Bouguet, Camera Calibration Toolbox for Matlab,
  9. 9.
    OpenCV, Open Source Computer Vision library,
  10. 10.
    A. Fusiello, E. Trucco, A. Verri, A compact algorithm for rectification of stereo pairs, machine vision and applications 12(1) (Springer Berlin/Heidelberg, 2000) pp. 16–22 ISSN: 0932-8092Google Scholar
  11. 11.
    D. Scharstein, R. Szeliski, A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. Int. J. Comput. Vis. 47(1–3) (Kluwer Academic Publishers, Hingham, 2002), pp. 7–42 ISSN 0920-5691Google Scholar
  12. 12.
    A.W. Gruen, Adaptive least squares correlation: a powerful image matching technique. South African J Photogramm, Remote Sens. Cartogr. 14, 175–187 (1985)Google Scholar
  13. 13.
    F.C. Crow, Summed-area tables for texture mapping, Proceedings of the 11th annual conference on Computer graphics and interactive techniques, SIGGRAPH 84, (ACM, New York, 1984), pp. 207–212 ISBN 0-89791-138-5Google Scholar
  14. 14.
    M.J. McDonnell, Box-filtering techniques, computer graphics and image processing, 17(1), 65–70 (1981). ISSN 0146-664X,  10.1016/S0146-664X(81)80009-3
  15. 15.
    T. Kanade, M. Okutomi, A stereo matching algorithm with an adaptive window: theory and experiment, IEEE transactions on pattern analysis and machine intelligence, 16, N. 1 920–932, (1994)Google Scholar
  16. 16.
    A. Fusiello, V. Roberto, E. Trucco, Symmetric stereo with multiple windowing. Int. J. Pattern Recognit. Artif. Intell. 14, 1053–1066 (2000)Google Scholar
  17. 17.
    K.-J. Yoon, I.S. Kweon, Adaptive support-weight approach for correspondence search. IEEE Trans. Pattern Anal. Mach. Intell. 28(4), 650–656 (2006)CrossRefGoogle Scholar
  18. 18.
    F. Tombari, S. Mattoccia, L. Di Stefano, Segmentation-based adaptive support for accurate stereo correspondence, in Proceedings of IEEE Pacific-Rim Symposium on Image and Video Technology, vol 1 December 17–19 (Santiago, Chile, 2007), pp. 427–438, SpringerGoogle Scholar
  19. 19.
    J. Sun, N.-N. Zheng, H.-Y. Shum, Stereo matching using belief propagation. IEEE Trans. Pattern Anal. Mach. Intell. 25(7), 787–800 (2003)CrossRefGoogle Scholar
  20. 20.
    R. Szeliski, R. Zabih, D. Scharstein, O. Veksler, A. Agarwala, C. Rother, A comparative study of energy minimization methods for Markov random fields, ECCV, 2006, pp. 16–29Google Scholar
  21. 21.
    S.Z. Li, Markov random field modelling in image analysis (Advances in Pattern Recognition), 3rd edn. (Springer, London, 2009). ISBN 1848002785Google Scholar
  22. 22.
    I.J. Cox, S.L. Hingorani, S.B. Rao, B.M. Maggs, A maximum likelihood stereo algorithm. Comput. Vis. Image Underst. 63, 542–567 (1996)CrossRefGoogle Scholar
  23. 23.
    H. Hirschmuller, Stereo processing by semiglobal matching and mutual information. IEEE Trans. Pattern Anal. Mach. Intell. 30(2), 328–341 (2008)CrossRefGoogle Scholar
  24. 24.
    M. Burns, G.W. Roberts, Mixed-Signal IC test and measurement (The Oxford series in electrical and computer engineering) (Oxford University Press, New York, 2001)Google Scholar
  25. 25.
    Evaluation of measurement data—Guide to the expression of uncertainty in measurement, Joint Committee for Guides in Metrology, JCGM 100:2008Google Scholar
  26. 26.
    T. Kahlmann, H. Ingensand, Calibration and development for increased accuracy of 3D range imaging cameras, J. Appl. Geodesy, 2(1), 1–11 (2008), ISSN (Online) 1862-9024, ISSN (Print) 1862-9016Google Scholar
  27. 27.
  28. 28.
    J. Davis, D. Nehab, R. Ramamoorthi, S. Rusinkiewicz, Spacetime stereo: A unifying framework for depth from triangulation, CVPR, 359–366 (2003)Google Scholar
  29. 29.
    L. Zhang, B. Curless, S. M. Seitz, Spacetime stereo: Shape recovery for dynamic scenes. In IEEE Computer Society, CVPR, 2003, pp. 367–374Google Scholar
  30. 30.
    C. Dal Mutto, P. Zanuttigh, G.M. Cortelazzo, A Probabilistic Approach to TOF and stereo data fusion, 3DPVT (France, Paris, 2010)Google Scholar
  31. 31.
    J. Zhu, L. Wang, R. Yang, J. Davis, Fusion of Time-Of-Flight depth and stereo for high accuracy depth maps. CVPR, 23–28 June 2008Google Scholar
  32. 32.
    B.K.P. Horn, Closed-form solution of absolute orientation using unit quaternions. J. Opt. Soc. America 4(4), 629–642 (1987)CrossRefGoogle Scholar
  33. 33.
    M.A. Fischler, R.C. Bolles, Random Sample Consensus: A paradigm for model fitting with applications to image analysis and automated cartography. Comm. ACM 24, 381–395 (1981)CrossRefGoogle Scholar
  34. 34.
    K.-D. Kuhnert, M. Stommel, Fusion of stereo-camera and PMD-camera data for real-time suited precise 3D environment reconstruction, IEEE/RSJ international conference on intelligent robots and systems, 9–15 Oct 2006, pp. 4780–4785Google Scholar
  35. 35.
    Q. Yang, K.-H. Tan, B. Culbertson, J. Apostolopoulos, Fusion of active and passive sensors for fast 3D capture, IEEE international workshop on multimedia signal processing (MMSP), pp. 69–74, 4–6 Oct 2010Google Scholar
  36. 36.
    J. Zhu, L. Wang, R. Yang, J. E Davis, Z. Pan, Reliability fusion of Time-Of-Flight depth and stereo geometry for high quality depth maps, IEEE Trans. Pattern Anal. Mach. Intell. 33(7), ISSN 0162-8828, (IEEE Computer Society, Washington, 2011) pp. 1400–1414Google Scholar
  37. 37.
    J. Zhu, L. Wang, J. Gao, R. Yang, Spatial-temporal fusion for high accuracy depth maps using dynamic MRFs. IEEE Trans. Pattern Anal. Mach. Intell. 32(5), 899–909 (2010)CrossRefGoogle Scholar
  38. 38.
    S. A. Gudmundsson, H. Aanaes, R. Larsen, Fusion of Stereo Vision and Time-Of-Flight Imaging for Improved 3D Estimation. Int. J. Intell. Syst. Technol. Appl. 5(3/4), 425–433, (2008) ISSN 1740–8865, (Inderscience Publishers, Geneva, Switzerland 2008)Google Scholar
  39. 39.
    J. Fischer, G. Arbeiter, A. Verl, Combination of Time-Of-Flight depth and stereo using semiglobal optimization, IEEE international conference on robotics and automation (ICRA), pp. 3548–3553, 9–13 May 2011Google Scholar
  40. 40.
    S. Birchfield, C. Tomasi, A pixel dissimilarity measure that is insensitive to image sampling, IEEE Trans. Pattern Anal. Mach. Intell. 20(4), 401–406, N. 6 (1998), ISSN 0162-8828Google Scholar
  41. 41.
    C.M. Bishop, Pattern recognition and machine learning (Information Science and Statistics) (Springer, New York, 2006). ISBN 0387310738, 9780387310732Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  • Carlo Dal  Mutto
    • 1
  • Pietro Zanuttigh
    • 1
  • Guido M. Cortelazzo
    • 1
  1. 1.University of PaduaPaduaItaly

Personalised recommendations