Skip to main content

Roboscan: a combined 2D and 3D vision system for improved speed and flexibility in pick-and-place operation

Abstract

We describe Roboscan, a Robot cell that combines 2D and 3D vision in a simple device, to aid a Robot manipulator in pick-and-place operations in a fast and accurate way. The optical head of Roboscan combines the two vision systems: the camera is used “stand-alone” in the 2D system, and combined to a laser slit projector in the 3D system, which operates in the triangulation mode. The 2D system, using suitable libraries, provides the preliminary 2D information to the 3D system to perform in a very fast, flexible and robust way the point cloud segmentation and fitting. Roboscan is mounted onto an anthropomorphic, 6-DOF Robot manipulator. The most innovative part of the system is represented by the use of robust 2D geometric template matching as a means to classify 3D objects. In this way, we avoid time-consuming 3D point cloud segmentation and 3D object classification, using 3D data only for estimating pose and orientation of the robot gripper. In addition, a novel approach to the template definition in the 2D geometric template matching is proposed, where the influence of surface reflectance and colour of the objects over the definition of the template geometry is minimized. We describe the procedures for 2D and 3D vision of Roboscan, together with the calibration procedures that have been implemented. We also present a set of tests that show the performance of the system and its effectiveness in a number of pick-and-place operations.

This is a preview of subscription content, access via your institution.

References

  1. 1.

    Brogardh T (2007) Present and future robot control development—an industrial perspective. Annual Rev Control 31:69–79

    Article  Google Scholar 

  2. 2.

    Xiong Y and Quek F (2002) Machine Vision for 3D mechanical part recognition in intelligent manufacturing environments. In: Proceedings of the 3rd International Workshop on Robotic Motion and Control (RoMo-Co’02), pp 441–446

  3. 3.

    Sakakibara S (2006) The robot cell as a re-configurable machining system. In: Dashchenko AI (ed) Reconfigurable manufacturing systems and transformable factories. Springer, Berlin, pp 259–272

  4. 4.

    Tudorie CR (2010) Different approaches in feeding of a flexible manufacturing cell. In: Simulation, modeling, and programming for autonomous robots. Springer-Verlag, Heidelberg, pp 509–520

  5. 5.

    Steger C, Ulrich M, Wiedemann C (2008) Machine vision algorithms and applications. Wiley, Weinheim

    Google Scholar 

  6. 6.

    Blais F (2004) A review of 20 years of range sensors development. J Electron Imag 13(1):231–240

    Article  Google Scholar 

  7. 7.

    Sumi Y, Kawai Y, Yoshimi T, Tomita F (2002) 3D object recognition in cluttered environments by segment-based stereo vision. Int J Comput Vision 46(1):5–23

    Article  MATH  Google Scholar 

  8. 8.

    Rossi NV, Savino C (2010) A new real-time shape acquisition with a laser scanner: first test results. Robot Comp Int Man 26:543–550

    Article  Google Scholar 

  9. 9.

    Rahayem M, Kjellander JAP (2011) Quadric segmentation and fitting of data captured by a laser profile scanner mounted on an industrial robot. Int J Adv Manuf Technol 52:155–169

    Article  Google Scholar 

  10. 10.

    Parker JR (2010) Algorithms for image processing and computer vision. Wiley, New York

    Google Scholar 

  11. 11.

    Aqsense SAL3D, http://www.aqsense.com/products/sal3d. Accessed 1 July 2013

  12. 12.

    Rusu RB, Cousins S (2011) 3D is here: Point Cloud Library (PCL). Proceedings of the IEEE International Conference on Robotics and Automation 2011:305–309

    Google Scholar 

  13. 13.

    MVTec Software GmbH, Halcon—the power of machine vision—HDevelop User’s Guide, München, 2009, pp 185–188

  14. 14.

    Zhao D, Li S (2005) A 3D image processing method for manufacturing process automation. Comput Ind 56:975–985

    Article  Google Scholar 

  15. 15.

    Biegelbauer G, Vincze M, Wohlkinger W (2010) Model-based 3D object detection: efficient approach using superquadrics. Mach Vision Appl 21:497–516

    Article  Google Scholar 

  16. 16.

    Richtsfeld M, Vincze M (2009) Robotic grasping of unknown objects, contemporary robotics—challenges and solutions. A D Rodić (ed.). ISBN: 978-953-307-038-4, InTech, DOI: 10.5772/7805. Available from: http://www.intechopen.com/books/contemporary-robotics-challenges-and-solutions/robotic-grasping-of-unknown-objects. Accessed 1 July 2013

  17. 17.

    Klinger T (2003) Image processing with labVIEW and Imaq Vision. Prentice-Hall, USA

    Google Scholar 

  18. 18.

    Gruen A, Huang TS (2001) Calibration and orientation of cameras in computer vision. Springer, Berlin

    Book  MATH  Google Scholar 

  19. 19.

    Sansoni G, Bellandi P, Docchio F (2011) Design and development of a 3D system for the measurement of tube eccentricity. Meas Sci Technol 22:075302. doi:10.1088/0957-0233/22/7/075302

    Article  Google Scholar 

  20. 20.

    Moeslund TB (2012) Introduction to video and image processing. Springer, London

    Book  MATH  Google Scholar 

  21. 21.

    Sibiryakov A et al (2008) Statistical template matching under geometric transformations. In: Coeurjolly D (ed) Discrete geometry for computer imagery. Springer, Berlin, pp 225–237

    Chapter  Google Scholar 

  22. 22.

    Trucco A Verri E (1998) Introductory techniques for 3D computer vision. (Prentice-Hall, ISBN 0-13-261108-2)

  23. 23.

    Marquardt D (1963) An algorithm for least-squares estimation of nonlinear parameters. SIAM J Appl Math 11:431–441

    MathSciNet  Article  MATH  Google Scholar 

  24. 24.

    InnovMetric Software (2011) Polyworks modeler & inspector—user's guide. Ste-Foy, Quèbec

Download references

Author information

Affiliations

Authors

Corresponding author

Correspondence to Giovanna Sansoni.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Bellandi, P., Docchio, F. & Sansoni, G. Roboscan: a combined 2D and 3D vision system for improved speed and flexibility in pick-and-place operation. Int J Adv Manuf Technol 69, 1873–1886 (2013). https://doi.org/10.1007/s00170-013-5138-z

Download citation

Keywords

  • Machine vision
  • Robotic manipulation
  • Pattern matching
  • 2D and 3D calibration
  • Blob analysis
  • 3D segmentation