Skip to main content
SpringerLink
Log in

3D Depth Cameras in Vision: Benefits and Limitations of the Hardware

With an Emphasis on the First- and Second-Generation Kinect Models

  • Chapter
  • First Online:
Computer Vision and Machine Learning with RGB-D Sensors

Part of the book series: Advances in Computer Vision and Pattern Recognition ((ACVPR))

Abstract

The second-generation Microsoft Kinect uses time-of-flight technology, while the first-generation Kinect uses structured light technology. This raises the question whether one of these technologies is “better” than the other. In this chapter, readers will find an overview of 3D camera technology and the artifacts that occur in depth maps.

We thank the following people at the Massachusetts Institute of Technology for their contributions to the chapter: Nikhil Naik, Boxin Shi, Ameya Joshi, Genzhi Ye, Amol Mahurkar, Julio Estrada, Hisham Bedri, and Rohan Puri.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 54.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    This is an open question.

  2. 2.

    A related question: what impact will the new Kinect have on the accuracy of current scene-understanding techniques?

References

  1. Banno A, Ikeuchi K (2011) Disparity map refinement and 3d surface smoothing via directed anisotropic diffusion. Comput Vis Image Underst 115(5):611–619

    Article  Google Scholar 

  2. Bhandari A, Kadambi A, Whyte R, Barsi C, Feigin M, Dorrington A, Raskar R (2014) Resolving multipath interference in time-of-flight imaging via modulation frequency diversity and sparse regularization. Opt Lett 39(6):1705–1708

    Article  Google Scholar 

  3. Birchfield S, Tomasi C (1999) Depth discontinuities by pixel-to-pixel stereo. Int J Comput Vis 35(3):269–293

    Article  Google Scholar 

  4. Camplani M, Salgado L (2012) Efficient spatio-temporal hole filling strategy for kinect depth maps. In: Proceedings of SPIE, vol 8920

    Google Scholar 

  5. Chen L, Lin H, Li S (2012) Depth image enhancement for kinect using region growing and bilateral filter. In: 21st international conference on pattern recognition (ICPR), 2012, pp 3070–3073. IEEE

    Google Scholar 

  6. Durand F, Dorsey J (2002) Fast bilateral filtering for the display of high-dynamic-range images. In: ACM transactions on graphics (TOG), vol 21, pp 257–66. ACM

    Google Scholar 

  7. Grimson WEL (1985) Computational experiments with a feature based stereo algorithm. IEEE Trans Pattern Anal Mach Intell 1:17–34

    Article  Google Scholar 

  8. Heide F, Hullin MB, Gregson J, Heidrich W (2013) Low-budget transient imaging using photonic mixer devices. ACM Trans Graph (TOG) 32(4):45

    Google Scholar 

  9. Henry P, Krainin M, Herbst E, Ren X, Fox D (2014) RGB-D mapping: using depth cameras for dense 3D modeling of indoor environments. In: Experimental robotics, pp 477–491. Springer, Berlin

    Google Scholar 

  10. Henry P, Krainin M, Herbst E, Ren X, Fox D (2012) RGB-D mapping: using kinect-style depth cameras for dense 3D modeling of indoor environments. Int J Robot Res 31(5):647–63

    Google Scholar 

  11. Hoegg T, Lefloch D, Kolb A (2013) Real-time motion artifact compensation for PMD-ToF images. In: Time-of-flight and depth imaging. Sensors, algorithms, and applications, pp 273–288. Springer, Berlin

    Google Scholar 

  12. Horn BKP (1970) Shape from shading: a method for obtaining the shape of a smooth opaque object from one view

    Google Scholar 

  13. Izadi S, Kim D, Hilliges O, Molyneaux D, Newcombe R, Kohli P, Shotton J, et al (2011) KinectFusion: real-time 3D reconstruction and interaction using a moving depth camera. In: Proceedings of the 24th annual ACM symposium on user interface software and technology, pp 559–568. ACM

    Google Scholar 

  14. Jones DG, Malik J (1992) Computational framework for determining stereo correspondence from a set of linear spatial filters. Image Vis Comput 10(10):699–708

    Google Scholar 

  15. Kadambi A, Bhandari A, Whyte R, Dorrington A, Raskar R (2014) Demultiplexing illumination via low cost sensing and nanosecond coding. In: 2014 IEEE international conference on computational photography (ICCP). IEEE

    Google Scholar 

  16. Kadambi A, Whyte R, Bhandari A, Streeter L, Barsi C, Dorrington A, Raskar R (2013) Coded time of flight cameras: sparse deconvolution to address multipath interference and recover time profiles. ACM Trans Graph (TOG) 32(6):167

    Google Scholar 

  17. Kanade T, Okutomi M (1994) A stereo matching algorithm with an adaptive window: theory and experiment. IEEE Trans Pattern Anal Mach Intell 16(9):920–932

    Article  Google Scholar 

  18. Kauff P, Atzpadin N, Fehn C, Müller M, Schreer O, Smolic A, Tanger R (2007) Depth map creation and image-based rendering for advanced 3DTV services providing interoperability and scalability. Sig Process Image Commun 22(2):217–234

    Google Scholar 

  19. Klaus A, Sormann M, Karner K (2006) Segment-based stereo matching using belief propagation and a self-adapting dissimilarity measure. In: 18th international conference on pattern recognition, 2006. ICPR 2006, vol 3, pp 15–18. IEEE

    Google Scholar 

  20. Lenzen F, Schäfer H, Garbe C (2011) Denoising time-of-flight data with adaptive total variation. In: Advances in visual computing, pp 337–346. Springer, Berlin

    Google Scholar 

  21. Li H, Vouga E, Gudym A, Luo L, Barron JT, Gusev G (2013) 3D self-portraits. ACM Trans Graph (TOG) 32(6):187

    Google Scholar 

  22. Marr D, Poggio T (1976) Cooperative computation of stereo disparity. Science 194(4262):283–287

    Article  Google Scholar 

  23. Naik N, Zhao S, Velten A, Raskar R, Bala K (2011) Single view reflectance capture using multiplexed scattering and time-of-flight imaging. In: ACM Transactions on Graphics (TOG), vol 30, p 171. ACM

    Google Scholar 

  24. Newcombe RA, Davison AJ, Izadi S, Kohli P, Hilliges O, Shotton J, Molyneaux D, Hodges S, Kim D, Fitzgibbon A (2011) KinectFusion: real-time dense surface mapping and tracking. In: 2011 10th IEEE international symposium on mixed and augmented reality (ISMAR), pp 127–136. IEEE

    Google Scholar 

  25. Ohta Y, Kanade T (1985) Stereo by intra-and inter-scanline search using dynamic programming. IEEE Trans Pattern Anal Mach Intell 2:139–154

    Article  Google Scholar 

  26. Penne J, Schaller C, Hornegger J, Kuwert T (2008) Robust real-time 3D respiratory motion detection using time-of-flight cameras. Int J Comput Assist Radiol Surg 3(5):427–431

    Article  Google Scholar 

  27. Shi B, Tan P, Matsushita Y, Ikeuchi K (2014) Bi-polynomial modeling of low-frequency reflectances. IEEE Trans Pattern Anal Mach Intell 36(6):1078–1091

    Google Scholar 

  28. Silberman N, Hoiem D, Kohli P, Fergus R (2012) Indoor segmentation and support inference from RGBD images. In: Computer vision-ECCV 2012, pp 746–760. Springer, Berlin

    Google Scholar 

  29. Velten A, Wu D, Jarabo A, Masia B, Barsi C, Joshi C, Lawson E, Bawendi M, Gutierrez D, Raskar R (2013) Femto-photography: capturing and visualizing the propagation of light. ACM Trans Graph (TOG) 32(4):44

    Google Scholar 

  30. Woodham RJ (1980) Photometric method for determining surface orientation from multiple images. Opt Eng 19(1):191139

    Article  Google Scholar 

  31. Ye C, Hegde GPM (2009) Robust edge extraction for SwissRanger SR-3000 range images. In: IEEE international conference on robotics and automation, 2009. ICRA’09, pp 2437–2442. IEEE

    Google Scholar 

  32. Ye G, Liu Y, Hasler N, Ji X, Dai Q, Theobalt C (2012) Performance capture of interacting characters with handheld kinects. In: Computer vision-ECCV 2012, pp 828–841. Springer, Berlin

    Google Scholar 

  33. Yoon K-J, Kweon IS (2006) Adaptive support-weight approach for correspondence search. IEEE Trans Pattern Anal Mach Intell 28(4):650–656

    Google Scholar 

  34. Zhang C, Zhang Z (2011) Calibration between depth and color sensors for commodity depth cameras. In: 2011 IEEE international conference on multimedia and expo (ICME), pp 1–6. IEEE

    Google Scholar 

  35. Zhang Z (2000) A flexible new technique for camera calibration. IEEE Trans Pattern Anal Mach Intell 22(11):1330–1334

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Massachusetts Institute of Technology, Cambridge, MA, USA

    Achuta Kadambi, Ayush Bhandari & Ramesh Raskar

Authors
  1. Achuta Kadambi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ayush Bhandari
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ramesh Raskar
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Achuta Kadambi .

Editor information

Editors and Affiliations

  1. University of Sheffield, United Kingdom

    Ling Shao

  2. Civolution Technology, Eindhoven, The Netherlands

    Jungong Han

  3. Microsoft Research, Cambridge, United Kingdom

    Pushmeet Kohli

  4. Microsoft Research, Redmond, Washington, USA

    Zhengyou Zhang

Rights and permissions

Reprints and permissions

Copyright information

© 2014 Springer International Publishing Switzerland

About this chapter

Cite this chapter

Kadambi, A., Bhandari, A., Raskar, R. (2014). 3D Depth Cameras in Vision: Benefits and Limitations of the Hardware. In: Shao, L., Han, J., Kohli, P., Zhang, Z. (eds) Computer Vision and Machine Learning with RGB-D Sensors. Advances in Computer Vision and Pattern Recognition. Springer, Cham. https://doi.org/10.1007/978-3-319-08651-4_1

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-08651-4_1

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-08650-7

  • Online ISBN: 978-3-319-08651-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation