Skip to main content
SpringerLink
Log in

Using ToF and RGBD cameras for 3D robot perception and manipulation in human environments

  • Original Research Paper
  • Published:
Intelligent Service Robotics Aims and scope Submit manuscript

Abstract

Robots, traditionally confined into factories, are nowadays moving to domestic and assistive environments, where they need to deal with complex object shapes, deformable materials, and pose uncertainties at human pace. To attain quick 3D perception, new cameras delivering registered depth and intensity images at a high frame rate hold a lot of promise, and therefore many robotics researchers are now experimenting with structured-light RGBD and Time-of-Flight (ToF) cameras. In this paper both technologies are critically compared to help researchers to evaluate their use in real robots. The focus is on 3D perception at close distances for different types of objects that may be handled by a robot in a human environment. We review three robotics applications. The analysis of several performance aspects indicates the complementarity of the two camera types, since the user-friendliness and higher resolution of RGBD cameras is counterbalanced by the capability of ToF cameras to operate outdoors and perceive details.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Notes

  1. http://opencv.org/.

  2. http://pointclouds.org/.

References

  1. Alenyà G, Dellen B, Torras C (2011) 3D modelling of leaves from color and tof data for robotized plant measuring. In: Proceedings of IEEE international conference on robotics and automation, Shanghai, pp 3408–3414

  2. Alenyà G, Dellen B, Foix S, Torras C (2013) Robotized plant probing: leaf segmentation utilizing time-of-flight data. IEEE Robot Autom Mag 20(3):50–59

    Article  Google Scholar 

  3. Alenyà G, Foix S, Torras C (2014) ToF cameras for active vision in robotics. Sens Actuators Physl 218:10–22

    Article  Google Scholar 

  4. Chiabrando F, Chiabrando R, Piatti D, Rinaudo F (2009) Sensors for 3D imaging: metric evaluation and calibration of a CCD/CMOS time-of-flight camera. Sensors 9(12):10080–10096

    Article  Google Scholar 

  5. Foix S, Alenyà G, Andrade-Cetto J, Torras C (2010) Object modeling using a ToF camera under an uncertainty reduction approach. In: Proceedings of IEEE international conference on robotics and automation, Anchorage, pp 1306–1312

  6. Foix S, Alenyà G, Torras C (2011) Lock-in time-of-flight (ToF) cameras: a survey. IEEE Sensors J 11(9):1917–1926

    Article  Google Scholar 

  7. Fossati A, Gall J, Grabner H, Ren X, Konolige K (eds) (2013) Consumer Depth cameras for computer vision. Advances in computer vision and pattern recognition series. Springer, Berlin

  8. Fuchs S, May S (2008) Calibration and registration for precise surface reconstruction with time of flight cameras. Int J Intell Syst Technol Appl 5(3/4):274–284

    Google Scholar 

  9. Ghobadi S, Loepprich O, Ahmadov F, Bernshausen J (2008) Real time hand based robot control using multimodal images. Int J Comput Sci 35(4):500–505

    Google Scholar 

  10. Janoch A, Karayev S, Jia Y, Barron JT, Fritz M, Saenko K, Darrell T (2013) A category-level 3d object dataset: putting the Kinect to work. In:Fossati A, Gall J, Grabner H, Ren X, Konolige K (eds) Consumer depth cameras for computer vision, Springer, London, pp 141–165

  11. Kahn S, Bockholt U, Kuijper A, Fellner DW (2013) Towards precise real-time 3D difference detection for industrial applications. Comput Indus 64(9):1115–1128

  12. Kazmi W, Foix S, Alenyà G, Andersen HJ (2014) Indoor and outdoor depth imaging of leaves with time-of-flight and stereo vision sensors: analysis and comparison. ISPRS J Photogramm Remote Sens 88:128–146

    Article  Google Scholar 

  13. Khoshelham K, Elberink SO (2012) Accuracy and resolution of Kinect depth data for indoor mapping applications. Sensors 12(2):1437–1454

    Article  Google Scholar 

  14. Lange R, Seitz P (2001) Solid-state time-of-flight range camera. IEEE J Quantum Electron 37(3):390–397

    Article  Google Scholar 

  15. Lefloch D, Nair R, Lenzen F, Schäfer H, Streeter L, Cree M, Koch R, Kolb A. (2013) Technical foundation and calibration methods for time-of-flight cameras. In: Grzegorzek M, Theobalt C, Koch R, Kolb A (eds) Time-of-flight and depth imaging: sensors, algorithms, and applications, LNCS 8200, Springer

  16. Lindner M, Schiller I, Kolb A, Koch R (2010) Time-of-flight sensor calibration for accurate range sensing. Comput Vis Image Underst 114(12):1318–1328

    Article  Google Scholar 

  17. May S, Fuchs S, Droeschel D, Holz D, Nuchter A (2009) Robust 3D-mapping with time-of-flight cameras. In: Proceedings of IEEE international conference in intelligent robots and systems. Saint Louis, pp 1673–1679

  18. Ramisa A, Alenyà G, Moreno-Noguer F, Torras C (2012) Using depth and appearance features for informed robot grasping of highly wrinkled clothes. In: Proceedings of IEEE international conference on robotics and automation, Saint Paul, pp 1703–1708

  19. Shotton J, Fitzgibbon A, Cook M, Sharp T, Finocchio M, Moore R, Kipman A, Blake A (2011) Real-time human pose recognition in parts from a single depth image. In: Proceedings of 25th IEEE conference on computer vision and pattern recognition, Colorado Springs

  20. Zhang Z (2012) Microsoft Kinect sensor and its effect. IEEE Multimedia 19(2):4–10

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Institut de Robòtica i Informàtica Industrial, CSIC-UPC, Llorens i Artigas 4-6, 08028, Barcelona, Spain

    G. Alenyà, S. Foix & C. Torras

Authors
  1. G. Alenyà
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. S. Foix
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. C. Torras
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to G. Alenyà.

Additional information

This research is partially funded by the EU GARNICS project FP7-247947, by CSIC project MANIPlus 201350E102, by the Spanish Ministry of Science and Innovation under project PAU+ DPI2011-27510, and the Catalan Research Commission under Grant SGR-155.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Alenyà, G., Foix, S. & Torras, C. Using ToF and RGBD cameras for 3D robot perception and manipulation in human environments. Intel Serv Robotics 7, 211–220 (2014). https://doi.org/10.1007/s11370-014-0159-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11370-014-0159-5

Keywords

Search

Navigation