Vision-based deviation extraction for three-dimensional control in robotic welding with steel sheet

  • Zhuohua Yu
  • Yinshui He
  • Yanling Xu
  • Huabin Chen


Effective multidirectional extraction of the deviations between the welding torch and the tracking point is prerequisite for three-dimensional control in robotic automatic welding. This paper presents a vision-based method to determine the logical tracking point and further to extract the deviations of the welding torch in two directions for robotic pulse metal active gas welding with steel sheet in real time. Images from passive vision sensors include two edges of the groove, the joint line, and complete arc. A scheme is proposed to accurately discern the joint line based on the local grayscale maximum within the region of interest using orientation feature maps. The geometric center of complete arc is used to reflect the position of the welding torch, and the intersection of the extracted joint line and the line that passes through the geometric center and is vertical to the former is regarded as the desired tracking point. It is transformed to the three-dimensional world coordinate via vision calibration techniques, and the deviations in two directions are yielded when contrasted with the real-time, recorded coordinate of the welding torch from the robot control system. The experiment results show the effectiveness of the proposed method.


Deviation extraction Joint lines Robotic welding Steel sheet Orientation feature maps 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


Funding information

This work is supported by the National Natural Science Foundation of China under the Grant No. 51405298 and 51575348.


  1. 1.
    YL X, Fang G, Chen SB, Zou JJ, Ye Z (2014) Real-time image processing for vision-based weld seam tracking in robotic GMAW. Int J Adv Manuf Technol 73:1413–1425CrossRefGoogle Scholar
  2. 2.
    Gu WP, Xiong ZY, Wan W (2013) Autonomous seam acquisition and tracking system for multi-pass welding based on vision sensor. Int J Adv Manuf Technol 69(1-4):451–460. CrossRefGoogle Scholar
  3. 3.
    Gong Y, Dai X, Li X (2010) Structured-light based joint recognition using bottom-up and top-down combined visual processing. International Conference on Image Analysis and Signal Processing, Zhejiang, China, April 9-11, 2010 pp: 507–512Google Scholar
  4. 4.
    Fang ZJ, Xu D, Tan M (2013) Vision-based initial weld point positioning using the geometric relationship between two seams. Int J Adv Manuf Technol 66(9-12):1535–1543. CrossRefGoogle Scholar
  5. 5.
    Nguyen HC, Lee BR (2014) Laser-vision-based quality inspection system for small-bead laser welding. Int J Precis Eng Manuf 15(3):415–423. CrossRefGoogle Scholar
  6. 6.
    Wang XP, Bai RL, Liu ZT (2014) Weld seam detection and feature extraction based on laser vision. Control Conference, 33rd Chinese, Nanjing, China, July 28–30, 2014 pp: 8249–8252Google Scholar
  7. 7.
    Huang W, Kovacevic R (2012) Development of a real-time laser-based machine vision system to monitor and control welding processes. Int J Adv Manuf Technol 63(1-4):235–248. CrossRefGoogle Scholar
  8. 8.
    YL X, HW Y, Zhong JY, Lin T, Chen SB (2012) Real-time seam tracking control technology during welding robot GTAW process based on passive vision sensor. J Mater Process Technol 212:1654–1662CrossRefGoogle Scholar
  9. 9.
    Ye Z, Fang G, Chen SB, Zou JJ (2013) Passive vision based seam tracking system for pulse-MAG welding. Int J Adv Manuf Technol 67(9-12):1987–1996. CrossRefGoogle Scholar
  10. 10.
    Xu YL, Lv N, Fang G, Du SF, Zhao WJ, Ye Z, Chen SB (2017) Welding seam tracking in robotic gas metal arc welding. J Mater Process TechnolGoogle Scholar
  11. 11.
    Guo B, Shi YH, GQ Y, Liang B, Wang K (2016) Weld deviation detection based on wide dynamic range vision sensor in MAG welding process. Int J Adv Manuf Technol 87(9-12):3397–3410. CrossRefGoogle Scholar
  12. 12.
    Zou Y, Jiang LB, Li YH, Xue LH, Huang JF, Huang JQ (2016) Welding deviation detection algorithm based on extremum of molten pool image contour. CHN J Mech Eng 29:74–83CrossRefGoogle Scholar
  13. 13.
    Gao XD, Chen YQ, You DR, Xiao ZL, Chen XH (2017) Detection of micro gap weld joint by using magneto-optical imaging and Kalman filtering compensated with RBF neural network. Mech Syst Signal Process 84:570–583. CrossRefGoogle Scholar
  14. 14.
    Gao XD, Mo L, You DR, Li ZM (2017) Tight butt joint weld detection based on optical flow and particle filtering of magneto-optical imaging. Mech Syst Signal Process 96:16–30. CrossRefGoogle Scholar
  15. 15.
    He YS, YL X, Chen YX, Chen HB, Chen SB (2015) Weld seam profile detection and feature point extraction for multi-pass route planning based on visual attention model. Robot Comput Integr Manuf 37:251–261CrossRefGoogle Scholar
  16. 16.
    He YS, Chen HB, Huang YM, Wu D, Chen SB (2016) Parameter self-optimizing clustering for autonomous extraction of the weld seam based on orientation saliency in robotic MAG welding. J Intell Robot Syst 83(2):219–237. CrossRefGoogle Scholar
  17. 17.
    He YS, Chen YX, YL X, Huang YM, Chen SB (2016) Autonomous detection of weld seam profiles via a model of saliency-based visual attention for robotic arc welding. J Intell Robot Syst 81(3-4):395–406. CrossRefGoogle Scholar
  18. 18.
    He YS, Zhou H, Wang JW, Wu D, Chen SB (2016) Weld seam profile extraction of T-joints based on orientation saliency for path planning and seam tracking. Advanced Robotics and ITS Social Impacts IEEE, pp 110–115Google Scholar
  19. 19.
    Liu SY, Wang GR (2008) Fast calibration for robot welding system with laser vision. In Robotics, automation and mechatronics, 2008 I.E. Conference on, pp 706–710Google Scholar

Copyright information

© Springer-Verlag London Ltd., part of Springer Nature 2018

Authors and Affiliations

  • Zhuohua Yu
    • 1
  • Yinshui He
    • 2
  • Yanling Xu
    • 3
  • Huabin Chen
    • 3
  1. 1.Institute of TechnologyEast China Jiao Tong UniversityNanchangChina
  2. 2.School of Environment and Chemical EngineeringNanchang UniversityNanchangChina
  3. 3.School of Materials Science and EngineeringShanghai Jiao Tong UniversityShanghaiChina

Personalised recommendations