Smart Check 3D: An Industrial Inspection System Combining 3D Vision with Automatic Planning of Inspection Viewpoints

  • Nicola Carlon
  • Nicolò Boscolo
  • Stefano Tonello
  • Emanuele Menegatti
Part of the Advances in Computer Vision and Pattern Recognition book series (ACVPR)


In this chapter, we describe an industrial inspection system composed by a 3D vision system, mounted on a manipulator robot arm , able to perform quality and completeness inspection on a complex solid part. The novelty of the system is in the deep integration among three software modules: the visual inspection system , the 3D simulation software, and the motion planning engine of the manipulator robot. This enables an automatic off-line programming of the robot path by specifying in the system the desired inspection tasks. The system automatically generates the needed points of view in order to perform 3D reconstruction and automatic visual inspection. Moreover, the motion planning system can reorder the inspection points in order to optimize the inspection cycle time. The core of this system was developed in the European Project “Thermobot,” and currently, it is been engineered to be deployed in an industrial production plant.


Motion Planning Rapidly Explore Randomize Tree Inspection Task Inspection Point Automatic Visual Inspection 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.



This research has been funded by the European Unions 7th Framework (FP7/2007–2013) under Grant Agreement No. 284607, Thermobot project, and 3D Complete project (Grant Agreement No. 262009).


  1. 1.
    Malamas EN, Petrakis EGM, Zervakis M, Petit L, Legat JD (2003) A survey on industrial vision systems, applications and tools. Image Vis Comput 21(2):171–188CrossRefGoogle Scholar
  2. 2.
    So EWY, Munaro M, Michieletto S, Tonello S, Menegatti E (2013) 3Dcomplete: efficient completeness inspection using a 2.5D color scanner. Comput Ind 64(9):1237–1252CrossRefGoogle Scholar
  3. 3.
    Keyes D (2007) Petaflop/s, seriously. Lect Notes Comput Sci 4873:2CrossRefGoogle Scholar
  4. 4.
    Pretto A, Tonello S, Menegatti E (2013) Flexible 3D localization of planar objects for industrial bin-picking with monocamera vision system. In: IEEE international conference on automation science and engineering (CASE), 2013, pp 168–175Google Scholar
  5. 5.
    So E, Munaro M, Michieletto S, Menegatti E, Tonello S (2013) 3Dcomplete: efficient completeness inspection using a 2.5D color scanner. Comput Ind Spec Issue 3D Imaging Ind 64(9):1237–1252Google Scholar
  6. 6.
    Singh AP, Latombe JC, Brutlag DL (1999) A motion planning approach to flexible ligand binding. In: ISMB, pp 252–261Google Scholar
  7. 7.
    Lien JM, Bayazit B, Sowell RT, Rodriguez S, Amato NM (2004) Shepherding behaviors. In: Proceedings. ICRA’04. IEEE international conference on robotics and automation, 2004, vol 4, pp 4159–4164Google Scholar
  8. 8.
    Mettin U, Shiriaev AS, Freidovich LB, Sampei M (2010) Optimal ball pitching with an under actuated model of a human arm. In: IEEE international conference on robotics and automation (ICRA), 2010, pp 5009–5014Google Scholar
  9. 9.
    LaValle SM (2006) Planning algorithms. Cambridge University Press, CambridgeGoogle Scholar
  10. 10.
    Schwartz JT, Sharir M (1983) On the piano movers problem. II. General techniques for computing topological properties of real algebraic manifolds. Adv Appl Math 4(3):298–351zbMATHMathSciNetCrossRefGoogle Scholar
  11. 11.
    LaValle SM, Kuffner JJ (2001) Randomized kinodynamic planning. Int J Robot Res 20(5):378–400CrossRefGoogle Scholar
  12. 12.
    Kavraki LE, Svestka P, Latombe JC, Overmars MH (1996) Probabilistic roadmaps for path planning in high-dimensional configuration spaces. IEEE Trans Robot Autom 12(4):566–580CrossRefGoogle Scholar
  13. 13.
    Carpin S, Pillonetto G (2005) Motion planning using adaptive random walks. IEEE Trans Robot 21(1):129–136CrossRefGoogle Scholar
  14. 14.
    Karaman S, Frazzoli E (2011) Sampling-based algorithms for optimal motion planning. Int J Robot Res 30(7):846–894CrossRefGoogle Scholar
  15. 15.
    SM LaValle (2003) From dynamic programming to RRTs: algorithmic design of feasible trajectories. In: Control problems in robotics. Springer, Berlin, pp 19–37Google Scholar
  16. 16.
    Perez A, Platt R, Konidaris G, Kaelbling L, Lozano-Perez T (2012) LQR-RRT*: optimal sampling-based motion planning with automatically derived extension heuristics. In: IEEE international conference on robotics and automation (ICRA), 2012, pp 2537–2542Google Scholar
  17. 17.
    Glassman E, Tedrake R (2010) A quadratic regulator-based heuristic for rapidly exploring state space. In: IEEE international conference on robotics and automation (ICRA), 2010, pp 5021–5028Google Scholar
  18. 18.
    Aboaf EW, Drucker S, Atkeson CG (1989) Task-level robot learning: juggling a tennis ball more accurately. In: IEEE international conference on robotics and automation, 1989. Proceedings 1989, pp 1290–1295Google Scholar
  19. 19.
    Plaku E, Bekris KE, Chen BY, Ladd AM, Kavraki LE (2005) Sampling-based roadmap of trees for parallel motion planning. IEEE Trans Robot 21(4):597–608Google Scholar
  20. 20.
    Tonello S, Zanetti GP, Finotto M, Bortoletto R, Tosello E, Menegatti E (2012) Workcellsimulator: a 3D simulator for intelligent manufacturing. In: Simulation, modeling, and programming for autonomous robots. Springer, Berlin, pp 311–322Google Scholar
  21. 21.
    Bouguet J-Y, Perona P (1998) Camera calibration from points and lines in dual-space geometry. In: Proceedings of 5th European conference on computer vision, pp 2–6Google Scholar
  22. 22.
    Tsai RY (1987) A versatile camera calibration technique for high-accuracy 3D machine vision metrology using off-the-shelf tv cameras and lenses. IEEE J Robot Autom 3(4):323–344Google Scholar
  23. 23.
    So EWY, Michieletto S, Menegatti E (2012) Calibration of a dual-laser triangulation system for assembly line completeness inspection. In: IEEE international symposium on robotic and sensors environments, pp 138–143Google Scholar
  24. 24.
    Eitzinger C, Ghidoni S, Menegatti E (2013) Thermobot: towards semi-autonomous, thermographic detection of cracks. In: International conference on heating by electromagnetic sources, PaduaGoogle Scholar
  25. 25.
    Dorigo M, Maniezzo V, Colorni A (1996) Ant system: optimization by a colony of cooperating agents. IEEE Trans Syst Man, Cybern Part B Cybern 26(1):29–41CrossRefGoogle Scholar

Copyright information

© Springer-Verlag London (outside the USA) 2015

Authors and Affiliations

  • Nicola Carlon
    • 1
  • Nicolò Boscolo
    • 1
  • Stefano Tonello
    • 1
  • Emanuele Menegatti
    • 2
  1. 1.IT+Robotics SrlPadovaItaly
  2. 2.Intelligent Autonomous Systems LaboratoryUniversity of PadovaPadovaItaly

Personalised recommendations