Skip to main content
SpringerLink
Log in

Three-dimensional object recognition system for enhancing the intelligence of a KUKA robot

  • ORIGINAL ARTICLE
  • Published:
The International Journal of Advanced Manufacturing Technology Aims and scope Submit manuscript

Abstract

Machine intelligence has been a research hotspot in mechatronics in recent years. This research presents a 2D/3D object recognition system for enhancing the intelligence of an industrial robot (KUKA robot). The image processing and object recognition algorithms were developed using software packages VisionPro and LabVIEW. Experiments were carried out to verify the performance of the system. It can be concluded that the system is able to recognise any general 2D object within a time of six seconds. The performance of the system in 3D object recognition is slower compared to 2D objects, which is largely affected by the number of trained images stored in the database, the complexity of the object, and the presence of similar objects in the database. Despite the complexity of the objects being recognised, both the overall accuracy and success rate of the system are close to 100%. The developed system proved to be robust and allows for automatic recognition of objects in the manufacturing environment described in this paper.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Krar A, Gill S (2003) Exploring Advanced Manufacturing Technologies. Industrial Press Inc., New York

    Google Scholar 

  2. Abdullah M, Guan L, Lim K, Karim A (2004) The applications of computer vision system and tomographic radar imaging for assessing physical properties of food. J Food Eng 61:125–135, DOI 10.1016/S0260-8774(03)00194-8

    Article  Google Scholar 

  3. Billingsley J, Dunn M (2005) Unusual vision - machine vision applications at the ncea. Sens Rev 25(3):202–208, DOI 10.1108/02602280510606480

    Article  Google Scholar 

  4. Brosnan T, Sun DW (2004) Improving quality inspection of food products by computer vision - a review. J Food Eng 61:3–16, DOI 10.1016/S0260-8774(03)00183-3

    Article  Google Scholar 

  5. Gunasekaran S (1996) Computer vision technology for food quality assurance. Trends Food Sci Tech 7(8):245–256, DOI 10.1016/0924-2244(96)10028-5

    Article  Google Scholar 

  6. Burschka D, Li M, Taylor R, Hager GD (2004) Scale-invariant registration of monocular endoscopic images to ct-scans for sinus surgery. Lect Notes Comput Sc 3217(1):413–421, DOI 10.1016/j.media.2005.05.005

    Article  Google Scholar 

  7. Yaniv Z, Joskowicz L (2005) Precise robot-assisted guide positioning for distal locking of intramedullary nails. IEEE Trans Med Imaging 24(5):624–625, URL 10.1109/TMI.2005.844922

    Article  Google Scholar 

  8. Galantucci L, Percoco G, Spina R (2004) An artificial intelligence approach to registration of free-form shapes. CIRP Ann Manuf Technol 53(1):139–142

    Article  Google Scholar 

  9. Varady T, Martin RR, Cox J (1997) Reverse engineering of geometric models - an introduction. CAD Comput Aided Des 29(4):255–268, DOI 10.1016/S0010-4485(96)00054-1

    Article  Google Scholar 

  10. Zhang H, Zhang G, Shi Y, Zhao X (2005) Application of binocular vision probe on measurement of high-reflective metallic surface. Proc SPIE Int Soc Opt Eng 5633:333–338, DOI 10.1117/12.570778

    Google Scholar 

  11. Bowyer K, Chang K, Flynn P (2006) A survey of approaches and challenges in 3d and multi-modal 3d + 2d face recognition. Comp Vis Image Und 101(1):1–15, DOI 10.1016/j.cviu.2005.05.005

    Article  Google Scholar 

  12. Conde C, Rodriguez-Aragon LJ, Cabello E (2006) Automatic 3d face feature points extraction with spin images. Lect Notes Comput Sc 4142:317–328

    Article  Google Scholar 

  13. Hu W, Tan T, Wang L, Maybank S (2004) A survey on visual surveillance of object motion and behaviors. IEEE Trans Syst Man Cybern Pt C Appl Rev 34(3):334–352, DOI 10.1109/TSMCC.2004.829274

    Article  Google Scholar 

  14. Zubal G, Tagare H, Zhang L, Duncan J (1991) 3-d registration of intermodality medical images. Proc Annu Conf Eng Med Biol 13(1):293–294, DOI 10.1109/IEMBS.1991.683942

    Google Scholar 

  15. Lobo J, Dias J (2003) Vision and inertial sensor cooperation using gravity as a vertical reference. IEEE Trans Pattern Anal Mach Intell 25(12):1597–1608, DOI 10.1109/TPAMI.2003.1251152

    Article  Google Scholar 

  16. Moravec HP (1980) Obstacle avoidance and navigation in the real world by a seeing robot rover. PhD thesis, Carnegie-Mellon University

  17. Benhabib B (2003) Manufacturing: design, production, automation, and integration. CRC Press, Boca Raton, FL, USA

  18. Hornberg A (ed) (2006) Handbook of Machine Vision. Wiley-VCH, Weinheim, Germany

  19. (2006) Cognex machine vision systems and machine vision sensors! http://www.cognex.com/ Accessed September 15, 2006

  20. (2006) National instruments - test and measurement. http://www.ni.com/ Accessed September 15, 2006

  21. Wong A, Rong L, Liang X (1998) Robotic vision: 3d object recognition and pose determination. IEEE Int Conf Intell Rob Syst 2:1202–1209, DOI 10.1109/IROS.1998.727463

    Google Scholar 

  22. Chan V, Bradley C, Vickers G (2001) A multi-sensor approach to automating co-ordinate measuring machine-based reverse engineering. Comput Ind 44(2):105–115, DOI 10.1016/S0166-3615(00)00087-7

    Article  Google Scholar 

  23. Liao J, Wu M, Baines R (1999) Coordinate measuring machine vision system. Comput Ind 38(3):239–248, DOI 10.1016/S0166-3615(98)00093-1

    Article  Google Scholar 

  24. Brunelli R, Poggio T (1993) Face recognition: features versus templates. IEEE Trans Pattern Anal Mach Intell 15(10):1042–1052, DOI 10.1109/34.254061

    Article  Google Scholar 

  25. Zitová B, Flusser J (2003) Image registration methods: a survey. Image Vision Comput 21(11):977–1000, DOI 10.1016/S0262-8856(03)00137-9

    Article  Google Scholar 

  26. Chen HY, Li YF (2004) Non-model-based view planning for active vision. Proc Annu Conf Mechatronics Machine Vision Pract MViP2004 pp 7–15

  27. Gao J, Xu W, Geng J (2006) 3d shape reconstruction of teeth by shadow speckle correlation method. Opt Lasers Eng 44(5):455–465, DOI 10.1016/j.optlaseng.2005.04.013

    Article  Google Scholar 

  28. Mikolajczyk K, Schmid C (2005) A performance evaluation of local descriptors. Proc IEEE Comput Soc Conf Compu Vision Pattern Recognit 2:257–263

    Google Scholar 

  29. Mikolajczyk K, Tuytelaars T, Schmid C, Zisserman A, Matas J, Schaffalitzky F, Kadir T, Van Gool L (2005) A comparison of affine region detectors. Int J Comput Vision 65(1–2):43–72, DOI 10.1007/s11263-005-3848-x

    Article  Google Scholar 

  30. Brown LG (1992) A survey of image registration techniques. ACM Comput Surv 24(4):325–376, DOI 10.1145/146370.146374

    Article  Google Scholar 

  31. Goshtasby A (2005) 2-D and 3-D image registration. Wiley, Hoboken

  32. Mikolajczyk K, Schmid C (2004) Comparison of affine-invariant local detectors and descriptors. Proc 12th European Signal Processing Conference pp 1729–1732

  33. Büker U, Drüe S, Götze N, Hartmann G, Kalkreuter B, Stemmer R, Trapp R (2001) Vision-based control of an autonomous disassembly station. Robot Auton Syst 35(3–4):179–189, DOI 10.1016/S0921-8890(01)00121-X

    Article  MATH  Google Scholar 

  34. Jeong S, Chung J, Lee S, Suh IH, Choi B (2005) Design of a simultaneous mobile robot localization and spatial context recognition system. Lect Notes Comput Sc 3683 NAI:945–952, DOI 10.1007/11553939

    Article  Google Scholar 

  35. Harris C, Stephens M (1988) A combined corner and edge detector. Proceedings of the 4th Alvey Vision Conference pp 147–151

  36. Lowe DG (2004) Distinctive image features from scale-invariant keypoints. Int J Comput Vision 60(2):91–110, DOI 10.1023/B:VISI.0000029664.99615.94

    Article  Google Scholar 

  37. Peña Cabrera M, Lopez-Juarez I, Rios-Cabrera R, Corona-Castuera J (2005) Machine vision approach for robotic assembly. Assem Autom 25(3):204–216, DOI 10.1108/01445150510610926

    Article  Google Scholar 

  38. Abdullah M, Bharmal M, Sardi M (2005) High speed robot vision system with flexible end effector for handling and sorting of meat patties. 9th International Conference on Mechatronics Technology

  39. Hough P (1962) Method and means for recognizing complex patterns. Patent, patent number: 3069654

  40. (2005) KUKA robot manual. KUKA, http://www.kuka.com/usa/en/newsevents/downloads/ Accessed September 14, 2006

  41. Berge J (2005) Software for automation: architecture, integration, and security. ISA, Research Triangle Park, NC, USA

  42. Iwanitz F, Lange J (2006) OPC: fundamentals, implementation & application, 3rd edn. Hüthig Fachverlag, Heidelherg

  43. Held G (2003) The ABCs of TCP/IP. CRC Press

  44. (2005) PatMax and PatQuick. Cognex, One Vision Drive, Natick, MA 01760-2059, USA

  45. Hartley RI, Zisserman A (2003) Multiple View Geometry in Computer Vision, 2nd edn. Cambridge University Press, Cambridge

  46. Bay H, Tuytelaars T, Van Gool L (2006) Surf: speeded up robust features. Lect Notes Comput Sc 3951 NCS:404–417

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Mechanical Engineering, The University of Auckland, Private Bag 92019, Auckland, New Zealand

    S. Q. Xie, D. Cheng, S. Wong & E. Haemmerle

Authors
  1. S. Q. Xie
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. D. Cheng
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. S. Wong
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. E. Haemmerle
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to S. Q. Xie.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Xie, S.Q., Cheng, D., Wong, S. et al. Three-dimensional object recognition system for enhancing the intelligence of a KUKA robot. Int J Adv Manuf Technol 38, 822–839 (2008). https://doi.org/10.1007/s00170-007-1112-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00170-007-1112-y

Keywords

Search

Navigation