Skip to main content

Advertisement

SpringerLink
Log in

Advances techniques of the structured light sensing in intelligent welding robots: a review

  • ORIGINAL ARTICLE
  • Published:
The International Journal of Advanced Manufacturing Technology Aims and scope Submit manuscript

Abstract

With the rapid development of artificial intelligence and intelligent manufacturing, the traditional teaching-playback mode and the off-line programming (OLP) mode cannot meet the flexible and fast modern manufacturing mode. Therefore, the intelligent welding robots have been widely developed and applied into the industrial production lines to improve the manufacturing efficiency. The sensing system of welding robots is one of the key technologies to realize the intelligent robot welding. Due to its unique characteristics of good robustness and high precision, the structured light sensor has been widely developed in the intelligent welding robots. To adapt to different measurement tasks of the welding robots, many researchers have designed different structured light sensors and integrated them into the intelligent welding robots. Therefore, the latest research and application work about the structured light sensors in the intelligent welding robots is analyzed and summarized, such as initial weld position identification, parameter extraction, seam tracking, monitoring of welding pool, and welding quality detection, to provide a comprehensive reference for researchers engaged in these related research work as much as possible.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21

Similar content being viewed by others

References

  1. Mitsi S, Bouzakis K-D, Mansour G, Sagris D, Maliaris G (2005) Off-line programming of an industrial robot for manufacturing. Int J Adv Manuf Technol 26(3):262–267

    Google Scholar 

  2. Chan SF, Kwan R (2003) Post-processing methodologies for off-line robot programming within computer integrated manufacture. J Mater Process Tech 139(1–3):8–14

    Google Scholar 

  3. Maiolino P, Woolley R, Branson D, Benardos P, Popov A, Ratchev S (2017) Flexible robot sealant dispensing cell using RGB-D sensor and off-line programming. Robotics Comput Integr Manuf 48:188–195

    Google Scholar 

  4. Ong SK, Chong JWS, Nee AYC (2010) A novel AR-based robot programming and path planning methodology. Robotics Comput Integr Manuf 26(3):240–249

    Google Scholar 

  5. Le J, Zhang H, Xiao (2017) Circular fillet weld tracking in GMAW by robots based on rotating arc sensors. Int J Adv Manuf Technol 88(9-12):2705–2715

    Google Scholar 

  6. Le J, Zhang H, Chen XQ (2018) Realization of rectangular fillet weld tracking based on rotating arc sensors and analysis of experimental results in gas metal arc welding. Robotics Comput Integr Manuf 49:263–276

    Google Scholar 

  7. Zhang S, Shengsun Hu, Wang Z (2016) Weld penetration sensing in pulsed gas tungsten arc welding based on arc voltage. J Mater Process Tech 229:520–527

    Google Scholar 

  8. Russell AM, Becker AT, Chumbley LS, Enyart DA, Bowersox BL, Hanigan TW, Labbe JL, Moran JS, Spicher EL, Zhong L (2016) A survey of flaws near welds detected by side angle ultrasound examination of anhydrous ammonia nurse tanks. J Loss Prevent Proc 43:263–272

    Google Scholar 

  9. Chen C, Fan C, Lin S, Cai X, Yang C, Zhou L (2019) Influence of pulsed ultrasound on short transfer behaviors in gas metal arc welding. J Mater Process Tech 267:376–383

    Google Scholar 

  10. Petcher PA, Dixon S (2015) Weld defect detection using PPM EMAT generated shear horizontal ultrasound. NDT &E Int 74:58–65

    Google Scholar 

  11. Klimenov VA, Abzaev YuA, Potekaev AI, Vlasov VA, Klopotov AA, Zaitsev KV, Chumaevskii AV, Porobova SA, Grinkevich LS, Tazin ID (2016) Structural state of a weld formed in aluminum alloy by friction stir welding and treated by ultrasound. Russ Phys J + 59(7):971–977

    Google Scholar 

  12. Zhu J, Wang J, Su N, Xu G, Yang M (2017) An infrared visual sensing detection approach for swing arc narrow gap weld deviation. J Mater Process Tech 243:258–268

    Google Scholar 

  13. Yu P, Xu G, Gu X, Zhou G, Tian Y (2017) A low-cost infrared sensing system for monitoring the MIG welding process. Int J Adv Manuf Technol 92(9–12):4031–4038

    Google Scholar 

  14. Wikle Iii HC, Kottilingam S, Zee RH, Chin BA (2001) Infrared sensing techniques for penetration depth control of the submerged arc welding process. J Mater Process Tech 113(1-3):228–233

    Google Scholar 

  15. Bai P, Wang Z, Hu S, Ma S, Liang Y (2017) Sensing of the weld penetration at the beginning of pulsed gas metal arc welding. J Manuf Process 28:343–350

    Google Scholar 

  16. Bo C, Wang J, Chen S (2010) A study on application of multi-sensor information fusion in pulsed GTAW. Ind Robot 37(2):168–176

    Google Scholar 

  17. Pal K, Pal SK (2010) Study of weld joint strength using sensor signals for various torch angles in pulsed MIG welding. CIRP Ann-Manuf Techn 3(1):55–65

    Google Scholar 

  18. Bo C, Chen S (2010) Multi-sensor information fusion in pulsed gtaw based on fuzzy measure and fuzzy integral. Assembly Autom 30(3):276–285

    Google Scholar 

  19. Gao X, Liu Y, You D (2014) Detection of micro-weld joint by magneto-optical imaging. Opt Laser Technol 62:141–151

    Google Scholar 

  20. Gao X, Chen Y (2014) Detection of micro gap weld using magneto-optical imaging during laser welding. Int J Adv Manuf Technol 73(1–4):23–33

    Google Scholar 

  21. Gao X, Mo L, Xiao Z, Chen X, Katayama S (2016) Seam tracking based on Kalman filtering of micro-gap weld using magneto-optical image. Int J Adv Manuf Technol 83(1–4):21–32

    Google Scholar 

  22. Gao X, Zhen R, Xiao Z, Katayama S (2015) Modeling for detecting micro-gap weld based on magneto-optical imaging. J Manuf Syst 37:193–200

    Google Scholar 

  23. Sun J, Li C, Wu X, Palade V, Fang W (2019) An effective method of weld defect detection and classification based on machine vision. IEEE T Ind Inform

  24. Zhao Z, Deng L, Bai L, Yi Z, Han J (2019) Optimal imaging band selection mechanism of weld pool vision based on spectrum analysis. Opt Laser Technol 110:145–151

    Google Scholar 

  25. Xiong J, Zou S (2019) Active vision sensing and feedback control of back penetration for thin sheet aluminum alloy in pulsed MIG suspension welding. J Process Contr 77:89–96

    Google Scholar 

  26. Abu-Nabah BA, ElSoussi AO, Alami A, ElRahman KA (2018) Virtual laser vision sensor environment assessment for surface profiling applications. Measurement 113:148–160

    Google Scholar 

  27. Abu-Nabah BA, ElSoussi AO, Alami A, ElRahman KA (2016) Simple laser vision sensor calibration for surface profiling applications. Opt Laser Eng 84:51–61

    Google Scholar 

  28. Rout A, Deepak BBVL, Biswal BB (2019) Advances in weld seam tracking techniques for robotic welding: a review. Robotics Comput Integr Manuf 56:12–37

    Google Scholar 

  29. Wang X, Li B, Zhang T (2018) Robust discriminant correlation filter-based weld seam tracking system. Int J Adv Manuf Technol 98(9–12):3029–3039

    Google Scholar 

  30. Zhang Y-x, You D-y, Gao X-d, Na S-J (2018) Automatic gap tracking during high power laser welding based on particle filtering method and BP neural network. Int J Adv Manuf Technol 96(1–4):685–696

    Google Scholar 

  31. Xu Y, Gu F, Chen S, Ju JZ, Ye Z (2014) Real-time image processing for vision-based weld seam tracking in robotic GMAW. Int J Adv Manuf Technol 73(9–12):1413–1425

    Google Scholar 

  32. Zhang K, Yan M, Huang T, Zheng J, Li Z (2019) 3D reconstruction of complex spatial weld seam for autonomous welding by laser structured light scanning. J Manuf Process 39:200–207

    Google Scholar 

  33. He Y, Xu Y, Chen Y, Chen H, Chen S (2016) Weld seam profile detection and feature point extraction for multi-pass route planning based on visual attention model. Robotics Comput Integr Manuf 37:251–261

    Google Scholar 

  34. Xu Y, Gu F, Lv N, Chen S, Ju JZ (2015) Computer vision technology for seam tracking in robotic GTAW and GMAW. Robotics Comput Integr Manuf 32:25–36

    Google Scholar 

  35. Yang L, Li E, Long T, Fan J, Mao Y, Fang Z, Liang Z (2018) A welding quality detection method for arc welding robot based on 3D reconstruction with SFS algorithm. Int J Adv Manuf Technol 94 (1–4):1209–1220

    Google Scholar 

  36. Xiong J, Zou S (2019) Active vision sensing and feedback control of back penetration for thin sheet aluminum alloy in pulsed MIG suspension welding. J Process Contr 77:89–96

    Google Scholar 

  37. Zhang Z, Chen H, Xu Y, Zhong J, Lv N, Chen S (2015) Multisensor-based real-time quality monitoring by means of feature extraction, selection and modeling for Al alloy in arc welding. Mech Syst Signal Pr 60:151–165

    Google Scholar 

  38. Han Y, Fan J, Yang X (2020) A structured light vision sensor for on-line weld bead measurement and weld quality inspection. Int J Adv Manuf Technol 106(5):2065–2078

    Google Scholar 

  39. Ye D, Hong GS, Zhang Y, Zhu K, Fuh JYH (2018) Defect detection in selective laser melting technology by acoustic signals with deep belief networks. Int J Adv Manuf Technol 96(5–8):2791–2801

    Google Scholar 

  40. Lin J, Yu Y, Ma L, Wang Y (2018) Detection of a casting defect tracked by deep convolution neural network. Int J Adv Manuf Technol 97(1–4):573–581

    Google Scholar 

  41. Fan J, Jing F, Yang L, Long T, Tan M (2019) An initial point alignment method of narrow weld using laser vision sensor. Int J Adv Manuf Technol, 1–12

  42. Zhang L, Ye Q, Yang W, Jiao J (2014) Weld line detection and tracking via spatial-temporal cascaded hidden Markov models and cross structured light. IEEE T Instrum Meas 63(4):742–753

    Google Scholar 

  43. Zhang G, Yu S, Gu YF, Fan D (2017) Welding torch attitude-based study of human welder interactive behavior with weld pool in gtaw. Robotics Comput Integr Manuf 48:145–156

    Google Scholar 

  44. Xu P, Tang X, Yao S (2008) Application of circular laser vision sensor (CLVS) on welded seam tracking. J Mater Process Tech 205(1–3):404–410

    Google Scholar 

  45. Zhang C, Li H, Jin Z, Gao H (2017) Seam sensing of multi-layer and multi-pass welding based on grid structured laser. Int J Adv Manuf Technol 91(1–4):1103–1110

    Google Scholar 

  46. Liu YK, Zhang WJ, Zhang YuM (2014) A tutorial on learning human welder’s behavior: sensing, modeling, and control. J Manuf Process 16(1):123–136

    Google Scholar 

  47. Iakovou D, Aarts R, Meijer J (2005) Sensor integration for robotic laser welding processes. In: International Congress on Applications of Lasers & Electro-Optics, pp 2301–2309. LIA

  48. Xiao Z (2011) Research on a trilines laser vision sensor for seam tracking in welding. In: Robotic welding, intelligence and automation. Springer, pp 139–144

  49. Zhu YZh, Lin T, Piao YJ, Chen SB (2005) Recognition of the initial position of weld based on the image pattern match technology for welding robot. Int J Adv Manuf Technol 26(7–8):784–788

    Google Scholar 

  50. Chen X, Chen S, Lin T, Lei Y (2006) Practical method to locate the initial weld position using visual technology. Int J Adv Manuf Technol 30(7–8):663–668

    Google Scholar 

  51. Chen X, Chen S (2010) The autonomous detection and guiding of start welding position for arc welding robot. Ind Robot 37(1):70–78

    Google Scholar 

  52. Fang Z, Xu D, Tan M (2013) Vision-based initial weld point positioning using the geometric relationship between two seams. Int J Adv Manuf Technol 66(9–12):1535–1543

    Google Scholar 

  53. Liu FQ, Wang ZY, Yu J (2018) Precise initial weld position identification of a fillet weld seam using laser vision technology. Int J Adv Manuf Technol 99(5–8):2059–2068

    Google Scholar 

  54. Fan J, Jing F, Yang L, Teng Lo, Tan M (2018) A precise initial weld point guiding method of micro-gap weld based on structured light vision sensor. IEEE Sens J 19(1):322–331

    Google Scholar 

  55. Zhang L, Xu Y, Du S, Zhao W, Hou Z, Chen S (2018) Point cloud based three-dimensional reconstruction and identification of initial welding position. In: Transactions on intelligent welding manufacturing. Springer, pp 61–77

  56. Dinham M, Gu F (2013) Autonomous weld seam identification and localisation using eye-in-hand stereo vision for robotic arc welding. Robotics Comput Integr Manuf 29(5):288–301

    Google Scholar 

  57. Li J, Jing F, Li E (2016) A new teaching system for arc welding robots with auxiliary path point generation module. In: 2016 35th Chinese Control Conference (CCC). IEEE, pp 6217–6221

  58. Jin Z, Li H, Zhang C, Wang Q, Gao H (2017) Online welding path detection in automatic tube-to-tubesheet welding using passive vision. Int J Adv Manuf Technol 90(9–12):3075–3084

    Google Scholar 

  59. Yang L, Li E, Long T, Fan J, Liang Z (2019) A novel 3-D path extraction method for arc welding robot based on stereo structured light sensor. IEEE Sens J 19(2):763–773

    Google Scholar 

  60. Yang L, Li E, Long T, Fan J, Liang Z (2018) A high-speed seam extraction method based on the novel structured-light sensor for arc welding robot: a review. IEEE Sens J 18(21):8631–8641

    Google Scholar 

  61. Zeng J, Chang B, Du D, Peng G, Chang S, Hong Y, Li W, Shan J (2017) A vision-aided 3D path teaching method before narrow butt joint welding. Sensors 17(5):1099

    Google Scholar 

  62. Gu WP, Xiong ZY, Wan W (2013) Autonomous seam acquisition and tracking system for multi-pass welding based on vision sensor. Int J Adv Manuf Technol 69(1–4):451–460

    Google Scholar 

  63. Luo H, Chen X (2005) Laser visual sensing for seam tracking in robotic arc welding of titanium alloys. Int J Adv Manuf Technol 26(9–10):1012–1017

    Google Scholar 

  64. Shen H, Lin T, Chen S, Li L (2010) Real-time seam tracking technology of welding robot with visual sensing. J Intell Robot Syst 59(3–4):283–298

    MATH  Google Scholar 

  65. Gao X, You D, Katayama S (2012) Seam tracking monitoring based on adaptive Kalman filter embedded Elman neural network during high-power fiber laser welding. IEEE T Ind Electron 59(11):4315–4325

    Google Scholar 

  66. Fang Z, Xu D, Tan M (2011) A vision-based self-tuning fuzzy controller for fillet weld seam tracking. IEEE-ASME T Mech 16(3):540–550

    Google Scholar 

  67. Kiddee P, Fang Z, Tan M (2016) An automated weld seam tracking system for thick plate using cross mark structured light. Int J Adv Manuf Technol 87(9–12):3589–3603

    Google Scholar 

  68. Lü X, Gu D, Wang Y, Qu Y, Qin C, Huang F (2018) Feature extraction of welding seam image based on laser vision. IEEE Sens J 18(11):4715–4724

    Google Scholar 

  69. He Z, Yi S, Cheung Y-M, You X, Tang YY (2017) Robust object tracking via key patch sparse representation. IEEE T Cybernetics 47(2):354–364

    Google Scholar 

  70. Čehovin L, Leonardis A, Kristan M (2016) Visual object tracking performance measures revisited. IEEE T Image Process 25(3):1261–1274

    MathSciNet  MATH  Google Scholar 

  71. Comaniciu D, Ramesh V, Meer P (2003) Kernel-based object tracking. IEEE T Pattern Anal 25(5):564–575

    Google Scholar 

  72. Bolme DS, Ross Beveridge J, Draper BA, Lui YM (2010) Visual object tracking using adaptive correlation filters. In: 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. IEEE, pp 2544–2550

  73. Zou Y, Chen T (2018) Laser vision seam tracking system based on image processing and continuous convolution operator tracker. Opt Laser Eng 105:141–149

    Google Scholar 

  74. Zou Y, Wang Y, Zhou W, Chen X (2018) Real-time seam tracking control system based on line laser visions. Opt Laser Technol 103:182–192

    Google Scholar 

  75. Zou Y, Chen X, Gong G, Li J (2018) A seam tracking system based on a laser vision sensor. Measurement 127:489–500

    Google Scholar 

  76. Pinto-Lopera J, Motta JST, Alfaro SA (2016) Real-time measurement of width and height of weld beads in GMAW processes. Sensors 16(9):1500

    Google Scholar 

  77. Wang Z, Zhang YM, Yang R (2013) Analytical reconstruction of three-dimensional weld pool surface in GTAW. J Manuf Process 15(1):34–40

    Google Scholar 

  78. Zhang WJ, Liu YK, Wang X, Zhang YM (2012) Characterization of three dimensional weld pool surface in GTAW. Weld J 91(7):195s–203s

    Google Scholar 

  79. Zhang WJ, Zhang X, Yu MZ (2015) Robust pattern recognition for measurement of three dimensional weld pool surface in GTAW. J Intell Manuf 26(4):659–676

    MathSciNet  Google Scholar 

  80. Liu YK, Zhang YM (2017) Fusing machine algorithm with welder intelligence for adaptive welding robots. J Manuf Process 27:18–25

    Google Scholar 

  81. Li C, Yu S, Gu YF, Yuan P (2018) Monitoring weld pool oscillation using reflected laser pattern in gas tungsten arc welding. J Mater Process Tech 255:876–885

    Google Scholar 

  82. Yu S, Li C, Du L, Gu YF, Ming Z (2016) Frequency characteristics of weld pool oscillation in pulsed gas tungsten arc welding. J Manuf Process 24:145–151

    Google Scholar 

  83. He K, Li X (2016) A quantitative estimation technique for welding quality using local mean decomposition and support vector machine. J Intell Manuf 27(3):525–533

    MathSciNet  Google Scholar 

  84. Zhang H, Hou Y, Zhao J, Wang L, Xi T, Li Y (2017) Automatic welding quality classification for the spot welding based on the Hopfield associative memory neural network and Chernoff face description of the electrode displacement signal features. Mech Syst Signal Pr 85:1035–1043

    Google Scholar 

  85. Li Y, Li YF, Wang QL, Xu D, Tan M (2010) Measurement and defect detection of the weld bead based on online vision inspection. IEEE T Instrum Meas 59(7):1841–1849

    Google Scholar 

  86. Chu H-H, Wang Z-Y (2016) A vision-based system for post-welding quality measurement and defect detection. Int J Adv Manuf Technol 86(9–12):3007–3014

    Google Scholar 

  87. Rodríguez-Martín M, Rodríguez-Gonzálvez P, González-Aguilera D, Fernández-Hernández J (2017) Feasibility study of a structured light system applied to welding inspection based on articulated coordinate measure machine data. IEEE Sens J 17(13):4217–4224

    Google Scholar 

Download references

Acknowledgments

The authors would like to thank the anonymous referees for their valuable suggestions and comments.

Funding

This work was supported by the National Natural Science Foundation of China (No.61473265,61803344,61773351), Science and Technology Research Project in Henan Province of China (No.202102210098), Robot Perception and Control Support Program for Outstanding Foreign Scientists in Henan Province of China (NO.GZS201908), and Innovation Research Team of Science & Technology in Henan Province of China (No.17IRTSTHN013).

Author information

Authors and Affiliations

  1. School of Electrical Engineering, Zhengzhou University, Zhengzhou, 450001, China

    Lei Yang, Yanhong Liu & Jinzhu Peng

  2. Robot Perception and Control Engineering Laboratory, Zhengzhou, 450001, Henan Province, China

    Lei Yang, Yanhong Liu & Jinzhu Peng

Authors
  1. Lei Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yanhong Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jinzhu Peng
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yanhong Liu.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yang, L., Liu, Y. & Peng, J. Advances techniques of the structured light sensing in intelligent welding robots: a review. Int J Adv Manuf Technol 110, 1027–1046 (2020). https://doi.org/10.1007/s00170-020-05524-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00170-020-05524-2

Keywords

Search

Navigation