3D Depth Cameras in Vision: Benefits and Limitations of the Hardware

With an Emphasis on the First- and Second-Generation Kinect Models
  • Achuta KadambiEmail author
  • Ayush Bhandari
  • Ramesh Raskar
Part of the Advances in Computer Vision and Pattern Recognition book series (ACVPR)


The second-generation Microsoft Kinect uses time-of-flight technology, while the first-generation Kinect uses structured light technology. This raises the question whether one of these technologies is “better” than the other. In this chapter, readers will find an overview of 3D camera technology and the artifacts that occur in depth maps.


Point Cloud Depth Image Bilateral Filter Structure Light Depth Camera 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Banno A, Ikeuchi K (2011) Disparity map refinement and 3d surface smoothing via directed anisotropic diffusion. Comput Vis Image Underst 115(5):611–619CrossRefGoogle Scholar
  2. 2.
    Bhandari A, Kadambi A, Whyte R, Barsi C, Feigin M, Dorrington A, Raskar R (2014) Resolving multipath interference in time-of-flight imaging via modulation frequency diversity and sparse regularization. Opt Lett 39(6):1705–1708CrossRefGoogle Scholar
  3. 3.
    Birchfield S, Tomasi C (1999) Depth discontinuities by pixel-to-pixel stereo. Int J Comput Vis 35(3):269–293CrossRefGoogle Scholar
  4. 4.
    Camplani M, Salgado L (2012) Efficient spatio-temporal hole filling strategy for kinect depth maps. In: Proceedings of SPIE, vol 8920Google Scholar
  5. 5.
    Chen L, Lin H, Li S (2012) Depth image enhancement for kinect using region growing and bilateral filter. In: 21st international conference on pattern recognition (ICPR), 2012, pp 3070–3073. IEEEGoogle Scholar
  6. 6.
    Durand F, Dorsey J (2002) Fast bilateral filtering for the display of high-dynamic-range images. In: ACM transactions on graphics (TOG), vol 21, pp 257–66. ACMGoogle Scholar
  7. 7.
    Grimson WEL (1985) Computational experiments with a feature based stereo algorithm. IEEE Trans Pattern Anal Mach Intell 1:17–34CrossRefGoogle Scholar
  8. 8.
    Heide F, Hullin MB, Gregson J, Heidrich W (2013) Low-budget transient imaging using photonic mixer devices. ACM Trans Graph (TOG) 32(4):45Google Scholar
  9. 9.
    Henry P, Krainin M, Herbst E, Ren X, Fox D (2014) RGB-D mapping: using depth cameras for dense 3D modeling of indoor environments. In: Experimental robotics, pp 477–491. Springer, BerlinGoogle Scholar
  10. 10.
    Henry P, Krainin M, Herbst E, Ren X, Fox D (2012) RGB-D mapping: using kinect-style depth cameras for dense 3D modeling of indoor environments. Int J Robot Res 31(5):647–63Google Scholar
  11. 11.
    Hoegg T, Lefloch D, Kolb A (2013) Real-time motion artifact compensation for PMD-ToF images. In: Time-of-flight and depth imaging. Sensors, algorithms, and applications, pp 273–288. Springer, BerlinGoogle Scholar
  12. 12.
    Horn BKP (1970) Shape from shading: a method for obtaining the shape of a smooth opaque object from one viewGoogle Scholar
  13. 13.
    Izadi S, Kim D, Hilliges O, Molyneaux D, Newcombe R, Kohli P, Shotton J, et al (2011) KinectFusion: real-time 3D reconstruction and interaction using a moving depth camera. In: Proceedings of the 24th annual ACM symposium on user interface software and technology, pp 559–568. ACMGoogle Scholar
  14. 14.
    Jones DG, Malik J (1992) Computational framework for determining stereo correspondence from a set of linear spatial filters. Image Vis Comput 10(10):699–708Google Scholar
  15. 15.
    Kadambi A, Bhandari A, Whyte R, Dorrington A, Raskar R (2014) Demultiplexing illumination via low cost sensing and nanosecond coding. In: 2014 IEEE international conference on computational photography (ICCP). IEEEGoogle Scholar
  16. 16.
    Kadambi A, Whyte R, Bhandari A, Streeter L, Barsi C, Dorrington A, Raskar R (2013) Coded time of flight cameras: sparse deconvolution to address multipath interference and recover time profiles. ACM Trans Graph (TOG) 32(6):167Google Scholar
  17. 17.
    Kanade T, Okutomi M (1994) A stereo matching algorithm with an adaptive window: theory and experiment. IEEE Trans Pattern Anal Mach Intell 16(9):920–932CrossRefGoogle Scholar
  18. 18.
    Kauff P, Atzpadin N, Fehn C, Müller M, Schreer O, Smolic A, Tanger R (2007) Depth map creation and image-based rendering for advanced 3DTV services providing interoperability and scalability. Sig Process Image Commun 22(2):217–234Google Scholar
  19. 19.
    Klaus A, Sormann M, Karner K (2006) Segment-based stereo matching using belief propagation and a self-adapting dissimilarity measure. In: 18th international conference on pattern recognition, 2006. ICPR 2006, vol 3, pp 15–18. IEEEGoogle Scholar
  20. 20.
    Lenzen F, Schäfer H, Garbe C (2011) Denoising time-of-flight data with adaptive total variation. In: Advances in visual computing, pp 337–346. Springer, BerlinGoogle Scholar
  21. 21.
    Li H, Vouga E, Gudym A, Luo L, Barron JT, Gusev G (2013) 3D self-portraits. ACM Trans Graph (TOG) 32(6):187Google Scholar
  22. 22.
    Marr D, Poggio T (1976) Cooperative computation of stereo disparity. Science 194(4262):283–287CrossRefGoogle Scholar
  23. 23.
    Naik N, Zhao S, Velten A, Raskar R, Bala K (2011) Single view reflectance capture using multiplexed scattering and time-of-flight imaging. In: ACM Transactions on Graphics (TOG), vol 30, p 171. ACMGoogle Scholar
  24. 24.
    Newcombe RA, Davison AJ, Izadi S, Kohli P, Hilliges O, Shotton J, Molyneaux D, Hodges S, Kim D, Fitzgibbon A (2011) KinectFusion: real-time dense surface mapping and tracking. In: 2011 10th IEEE international symposium on mixed and augmented reality (ISMAR), pp 127–136. IEEEGoogle Scholar
  25. 25.
    Ohta Y, Kanade T (1985) Stereo by intra-and inter-scanline search using dynamic programming. IEEE Trans Pattern Anal Mach Intell 2:139–154CrossRefGoogle Scholar
  26. 26.
    Penne J, Schaller C, Hornegger J, Kuwert T (2008) Robust real-time 3D respiratory motion detection using time-of-flight cameras. Int J Comput Assist Radiol Surg 3(5):427–431CrossRefGoogle Scholar
  27. 27.
    Shi B, Tan P, Matsushita Y, Ikeuchi K (2014) Bi-polynomial modeling of low-frequency reflectances. IEEE Trans Pattern Anal Mach Intell 36(6):1078–1091Google Scholar
  28. 28.
    Silberman N, Hoiem D, Kohli P, Fergus R (2012) Indoor segmentation and support inference from RGBD images. In: Computer vision-ECCV 2012, pp 746–760. Springer, BerlinGoogle Scholar
  29. 29.
    Velten A, Wu D, Jarabo A, Masia B, Barsi C, Joshi C, Lawson E, Bawendi M, Gutierrez D, Raskar R (2013) Femto-photography: capturing and visualizing the propagation of light. ACM Trans Graph (TOG) 32(4):44Google Scholar
  30. 30.
    Woodham RJ (1980) Photometric method for determining surface orientation from multiple images. Opt Eng 19(1):191139CrossRefGoogle Scholar
  31. 31.
    Ye C, Hegde GPM (2009) Robust edge extraction for SwissRanger SR-3000 range images. In: IEEE international conference on robotics and automation, 2009. ICRA’09, pp 2437–2442. IEEEGoogle Scholar
  32. 32.
    Ye G, Liu Y, Hasler N, Ji X, Dai Q, Theobalt C (2012) Performance capture of interacting characters with handheld kinects. In: Computer vision-ECCV 2012, pp 828–841. Springer, BerlinGoogle Scholar
  33. 33.
    Yoon K-J, Kweon IS (2006) Adaptive support-weight approach for correspondence search. IEEE Trans Pattern Anal Mach Intell 28(4):650–656Google Scholar
  34. 34.
    Zhang C, Zhang Z (2011) Calibration between depth and color sensors for commodity depth cameras. In: 2011 IEEE international conference on multimedia and expo (ICME), pp 1–6. IEEEGoogle Scholar
  35. 35.
    Zhang Z (2000) A flexible new technique for camera calibration. IEEE Trans Pattern Anal Mach Intell 22(11):1330–1334CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  1. 1.Massachusetts Institute of TechnologyCambridgeUSA

Personalised recommendations