Skip to main content

Advertisement

SpringerLink
Log in

3D complex curve seam tracking using industrial robot based on CAD model and computer vision

  • Original Paper
  • Published:
International Journal on Interactive Design and Manufacturing (IJIDeM) Aims and scope Submit manuscript

Abstract

3D complex curve seam tracking using industrial robots applied in fields that affect human health such as welding, gluing, spraying, painting, etc. However, at present, manual performance by workers or teaching manipulators by using teach-pendant to recognize the planning trajectories curves, especially for 3D complex seam curves which often has complex shape, direction, small width are still the two main approaches. Through these two implementation methods, productivity is low and product quality is not uniform. So this study will present a practical method to overcome the above disadvantages by using a low-cost camera combining an image processing algorithm using a 3D CAD model and a pre-defined 3D curve profile that needs to be attached to that model to extract the contour on the object. This data is then transmitted to the industrial manipulator to perform the task of tracing the planned 3D complex curve. By using this method, the small width 3D complex curve is easily tracked and the system is not complicated. Moreover, the profile to be tracked can be modified easily and quickly by non-expert users with basic knowledge about CAD drawing and image processing. Through the result of the experiments, small error system and fast image processing time about 1.8 mm and 0.5 s respectively, the system proved that it meets the requirement of the production line and it can replace the worker. The system is also easy to maintain and setup.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

Code availability

Not applicable.

Data availability

Not applicable

References

  1. Liu, Y., Tang, Q., Tian, X.: A discrete method of sphere-pipe intersecting curve for robot welding by offline programming. Robot. Comput. Integr. Manuf. 57, 404–411 (2019)

    Article  Google Scholar 

  2. BenchCat, BenchCat Fluid Dispensing Robot Software, https://benchcat.com/benchcat-robot-software.html. Accessed: 2022-07-15.

  3. Yaskawa, Moto Sim EG-VRC https://www.yaskawa.eu.com/products/software/productdetail/product/motosim-eg-vrc_1686. Accessed: 2022-07-15

  4. Kuka, KUKA. Sim, https://www.kuka.com/en-de/products/robot-systems/software/planning-project-engineering-service-safety/kuka_sim. Accessed: 2022-07-15

  5. ABB, Weldguide IV, The most powerful robotic Thru-the-Arc joint tracking on the market, https://new.abb.com/products/robotics/application-equipment-and-accessories/arc-welding-equipment/process-support-tools/weldguide

  6. Arc Sensor by Daihen, USA, (https://www.daihen-usa.com/product/fd-tr-tig-arcsensor/). Accessed: 2018-06-26.

  7. Cook, G.E., Andersen, K., Fernandez, K.R., Shepard, M.E., Wells Jr., A.M.: “Electric arc sensing for robot positioning control”, Robotic Welding, pp. 181–216. IFS (Publications) Ltd, Bedford (2010)

    Google Scholar 

  8. Fridenfalk, M., Bolmsjo, G.: Design and validation of a universal 6D seam-tracking system in robotic welding using arc sensing. Adv. Robot. 18, 1–21 (2004)

    Article  Google Scholar 

  9. Suwanratchatamanee, K., Matsumoto, M., Hashimoto, S.: Robotic tactile sensor system and applications. IEEE Trans. Industr. Electron. 57(3), 1074–1087 (2010)

    Article  Google Scholar 

  10. Xu, Y., Lv, N., Zhong, J., Chen, H., Chen, S.: Research on the real-time tracking information of three-dimension welding seam in robotic GTAW process based on composite sensor technology. J. Intell. Robot. Syst. 68(2), 89–103 (2012)

    Article  Google Scholar 

  11. Silvers, G.A., Fang, G.: Human machine interface in weld seam detection using an RGB-D camera. Adv. Mater. Res. 875–877, 1967–1971 (2014)

    Article  Google Scholar 

  12. Wang, Q., Jiao, W., Yu, R., Johnson, M.T., Zhang, Y.M.: Virtual reality robot-assisted welding based on human intention recognition. IEEE Trans. Autom. Sci. Eng. 17(2), 799–808 (2020). https://doi.org/10.1109/TASE.2019.2945607

    Article  Google Scholar 

  13. Pablo, R.G., Manuel, R.M.: Weld bead detection based on 3D geometric featuresand machine learning approaches. IEEE Access 7, 14714–14727 (2019). https://doi.org/10.1109/ACCESS.2019.2891367

    Article  Google Scholar 

  14. Panasonic Industry, VRPS (Virtual Robot Programming System), https://industrial.panasonic.com/ww/products-fa/welding/industrial-robots/arc-welding-robot/vrps

  15. Xiong, J., Zou, S.: Active vision sensing and feedback control of back penetration for thin sheet aluminum alloy in pulsed MIG suspension welding. J. Process Control 77, 89–96 (2019)

    Article  Google Scholar 

  16. Xiao, R., Xu, Y., Hou, Z., Chen, C., Chen, S.: An adaptive feature extraction algo-rithm for multiple typical seam tracking based on vision sensor in robotic arcwelding. Sens. Actuators A 297, 111533 (2019). https://doi.org/10.1016/j.sna.2019.111533

    Article  Google Scholar 

  17. Xiong, J., Pi, Y., Chen, H.: Deposition height detection and feature point extrac-tion in robotic GTA-based additive manufacturing using passive vision sensing. Robot. Comput. Integr. Manuf. 59, 326–334 (2019). https://doi.org/10.1016/j.rcim.2019.05.006

    Article  Google Scholar 

  18. Yang, L., Liu, Y., Peng, J., Liang, Z.: A novel system for off-line 3D seam extraction and path planning based on point cloud segmentation for arc welding robot. Robot. Comput. Integr. Manuf. 64, 101929 (2020)

    Article  Google Scholar 

  19. Fan, J., Jing, F., Yang, L., Teng, L., Tan, M.: A precise initial weld point guidingmethod of micro-gap weld based on structured light vision sensor. IEEE Sens. J. 19, 322–331 (2019). https://doi.org/10.1109/JSEN.2018.2876144

    Article  Google Scholar 

  20. Hanh, L.D., Phuc, H.T.: Simultaneously extract 3D seam curve and weld head angle for robot arm using passive vision. Int. J. Interact. Des. Manuf. (2021). https://doi.org/10.1007/s12008-021-00801-0

    Article  Google Scholar 

  21. Liu, S., Lu, H., Liu, S., Zhang, Y., Wang, S.: A speed adaptive path planning method based on line structured light information for robotic multi-layer/multi-pass welding. J. Phys. Conf. Ser. (2022). https://doi.org/10.1088/1742-6596/2281/1/012001

    Article  Google Scholar 

  22. Hanh, L.D., Thien, C.V.: Visual guidance of a sealant dispensing robot for online detection of complex 3D-curve seams. Int. J. Interact. Des. Manuf. (2022). https://doi.org/10.1007/s12008-022-00843-y

    Article  Google Scholar 

  23. Zhang, Q., Wu, Z.: Three-dimensional imaging technique based on Gray-coded structured illumination. Infrared Laser Eng. 49(3), 0303004 (2020)

    Article  Google Scholar 

  24. Yang, L., Liu, Y.: A novel 3D seam extraction method based on multi-functional sensor for V-type weld seam. IEEE Access 7, 182415–182424 (2019). https://doi.org/10.1109/ACCESS.2019.2944884

    Article  Google Scholar 

  25. Yang, L., Liu, Y., Peng, J.: Advances techniques of the structured light sensing in intelligent welding robots: a review. Int. J. Adv. Manuf. Technol. 110, 1027–1046 (2020). https://doi.org/10.1007/s00170-020-05524-2

    Article  Google Scholar 

  26. Lu, Z., Fan, J., Hou, Z., Deng, S., Zhou, C., Jing, F.: Automatic 3D seam extraction method for welding robot based on monocular structured light. IEEE Sens. J. 21(14), 16359–16370 (2021). https://doi.org/10.1109/JSEN.2021.3076341

    Article  Google Scholar 

  27. Yang, L., Fan, J., Liu, Y., Li, E., Peng, J., Liang, Z.: Automatic detection and location of weld beads with deep convolutional neural networks. IEEE Trans. Instrum. Meas., vol. 70, pp. 1–12, 2021, Art no. 5001912, https://doi.org/10.1109/TIM.2020.3026514.

  28. Du, R.Q., Xu, Y., Hou, Z., Shu, J., Chen, S.: Strong noise image processing for vision-based seam tracking in robotic gas metal arc welding. Int. J. Adv. Manuf. Technol. 101, 2135–2149 (2019)

    Article  Google Scholar 

  29. Zou, Y., Lan, R., Wei, X., Chen, J.: Robust seam tracking via a deep learning framework combining tracking and detection. Appl. Opt. 59, 4321–4331 (2020)

    Article  Google Scholar 

  30. Li, Y., Hu, M., Taiyong, W.: Weld image recognition algorithm based on deep learning. Int. J. Pattern Recognit. Artif. Intell. 34(08), 2052004 (2020)

    Article  Google Scholar 

  31. Yang, L., Liu, Y., Peng, J.: An automatic detection and identification method of welded joints based on deep neural network. IEEE Access 7, 164952–164961 (2019). https://doi.org/10.1109/ACCESS.2019.2953313

    Article  Google Scholar 

  32. Rodríguez-Gonzálvez, P., Rodríguez-Martín, M.: Weld bead detection based on 3D geometric features and machine learning approaches. IEEE Access 7, 14714–14727 (2019). https://doi.org/10.1109/ACCESS.2019.2891367

    Article  Google Scholar 

  33. Hanh, L.D., Hieu, K.T.G.: 3D matching by combining CAD model and computer vision for autonomous bin picking. Int. J. Interact. Des. Manuf. 15, 239–247 (2021). https://doi.org/10.1007/s12008-021-00762-4

    Article  Google Scholar 

  34. Li, M., Hashimoto, K.: Curve set feature-based robust and fast pose estimation algorithm. Sensors (Basel). 17(8), 1782 (2017). https://doi.org/10.3390/s17081782. (PMID:28771216;PMCID:PMC5580083)

    Article  Google Scholar 

  35. Hanh, L.D., Nhat, N.T.: Autonomous gluing shoe sole based on pointcloud data. Int. J. Eng. Appl. (IREA) 10(1), 49–55 (2022). https://doi.org/10.15866/irea.v10i1.20674

    Article  Google Scholar 

  36. Hanh, L.D., Lin, C.-Y.: Autonomous gluing based on image based visual servoing. Int. J. Robot. Autom. 36(2), 119–127 (2021)

    Google Scholar 

Download references

Acknowledgement

We acknowledge Ho Chi Minh City University of Technology (HCMUT), VNU-HCM for supporting this study.

Funding

Not applicable.

Author information

Authors and Affiliations

  1. Department of Mechatronics, Falculty of Mechanical Engineering, Ho Chi Minh City University of Technology (HCMUT), 268 Ly Thuong Kiet Street, District 10, Ho Chi Minh City, Vietnam

    Le Duc Hanh & Nguyen Cong Luan

  2. Department of Industrial System Engineering, Falculty of Mechanical Engineering, Ho Chi Minh City University of Technology (HCMUT), 268 Ly Thuong Kiet Street, District 10, Ho Chi Minh City, Vietnam

    Le Duc Dao

  3. Vietnam National University, Ho Chi Minh City, Linh Trung Ward, Thu Duc City, Ho Chi Minh City, Vietnam

    Le Duc Hanh, Le Duc Dao & Nguyen Cong Luan

Authors
  1. Le Duc Hanh
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Le Duc Dao
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Nguyen Cong Luan
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

LDH: Writing and review. LDD: Coding and testing. NCL: Coding and testing.

Corresponding author

Correspondence to Le Duc Hanh.

Ethics declarations

Conflicts of interest

The authors declare that they have no conflict of interest.

Ethics approval

Not applicable.

Consent to participate

The authors consent to participate.

Consent for publication

The authors consent for publication.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Hanh, L.D., Dao, L.D. & Luan, N.C. 3D complex curve seam tracking using industrial robot based on CAD model and computer vision. Int J Interact Des Manuf 17, 1039–1046 (2023). https://doi.org/10.1007/s12008-022-01043-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12008-022-01043-4

Keywords

Search

Navigation