Weld bead geometry real-time control in gas metal arc welding processes using intelligent systems

Gas metal arc welding (GMAW) process is one of the most important in the industry, so different efforts have been made to anticipate the parameters to convert this process into a stable one capable of joining parts with minimum human interference. In this sense, controlling is essential for automated applications because properties such as the weld mechanical strength are defined by the metal composition, the microstructure, and the weld bead geometry. Nevertheless, performing this automatic control to guarantee quality characteristics similar to a human expert’s in mechanized welding systems is still tricky. Nowadays, although various sensors have been used in the monitoring for control, it is still hard to detect effective options to real-time identify geometry characteristics in the formation process of the welds. Furthermore, even today, a process much more complex is to control more than one parameter simultaneously or control the weld penetration using a single sensor. Then, this research describes two intelligence systems for real-time control of the weld bead geometry in the GMAW process. The first is a passive vision system with sensor fusion that controls the width and height; the second is an active vision system that controls the penetration. Results indicate that the proposed methodology can be applied to simultaneously control external geometrical parameters without a predefined model of the welding process. In the case of penetration, a fuzzy controller and a neural network-based model help the system adapt to input parameter variations throughout the welding process, thus correcting instabilities under changing operating conditions.


Introduction
The gas metal arc welding process is an extensively used process of industrial production based on its adaptability to automated processes and higher productivity compared to other processes, such as Gas Tungsten Arc Welding (GTAW) and Shielded Metal Arc Welding (SMAW). Notably, various process applications have been developed in recent years based on technological innovation and market demands. Its consolidation has been reinforced in the automotive industry, oil and gas sectors, and fabrication and recovery of parts and structures.
Furthermore, for each product derived using welding processes, bead geometry and technical specifications are guaranteed by correctly selecting input parameters. Nevertheless, such processes require the time and effort of engineers and technicians, who determine the adequate parameters that generate the results closer to the initially proposed requirements. Therefore, identifying the appropriate parameter combination guaranteeing the required technical specifications is the key to defining more efficient processes that produce defect-free weld beads, with the indicated geometry and minimum input waste.
In addition, the weld quality generally depends on its mechanical and metallurgical characteristics, which, in turn, depends on weld geometry. Therefore, several studies have been conducted regarding the weld bead geometry in the GMAW process to control the operative parameters that guarantee the required diverse characteristics, like width, height, and penetration, among the most important ones. Furthermore, as Dong et al. [1] highlighted, an expert welder can intuitively adapt the welding parameters during the bead forming process to guarantee suitable geometry and quality features. In the case of automated systems, performing this type of control, adapting to different conditions, and, at the same time, guaranteeing characteristics similar to a human expert's are still complex.
Moreover, the study of weld bead geometry is essential when there are specific technical requirements, for example, in coating processes [2], in production line cost study [3], in the control and decrease of input waste [4], and in the study of specific characteristics of the components to be welded [5,6]. Bestard and Alfaro [7] present a compendium of studies related to the measurement and estimation of weld bead geometry in arc welding processes for the past 50 years of development. In this sense, different advances have been made. In any case, Bestard [8] states that a correct choice of measurement techniques and algorithms is based on sensor fusion. Then, combined with indirect measurement techniques, it will reduce welding production costs and increase their productivity, thus reducing the number of rejected parts in final quality inspections and utilizing modern power sources and robotic systems.
In weld bead geometry control, width, height, and penetration are the principal characteristics considered in diverse processes. In general, a significant part of systems designed to control geometric parameters in the welding process is based on statistical and artificial intelligence models, which are usually linear regression and neural networks, respectively. When applying such methodologies, the control variables of such proposed systems mainly rely on the control variables of power sources. For example, the typical control variables are voltage and wire feed rate to manage the GMAW conventional process using a constant voltage-type power source. Other independent parameters of the power source may be involved additionally, such as the welding speed, contact tip-to-work piece distance, and shielding gas variations.
For example, in other welding techniques as the GTAW process, Li et al. [9] monitored the voltage and used the current as a control variable in penetration control. Kejie et al. [10], also in penetration control for GTAW, apply the welding speed as a control variable. They monitored the width and length of the weld pool observed infrared radiation on the opposite side of the workpiece. In other cases, as the Pulsed Gas Metal Arc Welding (GMAW-P) process, Wang et al. [11] monitored changes in arc voltage during peak current periods to handle weld penetration variations. Yan et al. [12] also controlled weld penetration for the GMAW-P, in this case, using the weld pool width estimated from the radiation provided at the opposite side of the workpiece and also measuring the width and height above the weld pool and using the process current as a control variable. In general, each proposed control system focuses on a single geometric parameter. Another example, in multilayer GMAW, Xiong et al. [13] use the welding speed and control the weld bead width.
In general, Bestard and Alfaro [14] introduce a vision of diverse techniques and methods used overtime to control the weld bead geometry in different kinds of welding processes, with proposals ranging from simple open-loop controllers to intelligent control algorithms. The notable proportional-integral-derivative stands out when combined with other techniques, adaptive methods, neural networks, and fuzzy logic systems. Still, they depend on specific models, so in general, there are no autonomous systems that can be adapted and thus emulate a human welder. In particular, no indicated systems control the width and height of weld beads simultaneously, nor do they show techniques that use a single sensor to monitor at the same time both exposed parameters.
In this respect, the accurate monitoring and measurement of the entailed parameters is an important challenge for geometric parameters' real-time control. For example, Xiong and Zhang [15] used two cameras for real-time measurement of the bead width and height in a computer vision system. The first camera is fixed on the welding gun rear and is used to achieve the width measurement; the second camera measures the height. It is set on the side of the welding gun, perpendicular to the welding direction and perpendicular to the torch.
Other proposals for monitoring external parameters of the weld bead are presented from stereoscopic systems, using two cameras or particular hardware to split the scene into two images, as Zhimin et al. [16]. In that case, for GMAW-P, out of the trouble of camera synchronization, finding reference points in the weld pool is more challenging. On the other hand, Huang et al. [17] use complex systems based on structured light and specular reflection to extract information from the weld pool in real time for GTAW. In general, in filler metal processes such as GMAW conventional, the disturbances that continually occur in the weld pool change considerably the shape of the specular surface and the directions of the reflected rays. So, mainly, it can say that these kinds of systems are limited to the GTAW or GMAW-P process and should be restricted to conditions that favor small fluctuations in the weld pool.
As a specific case, Pinto-Lopera et al. [18] introduced a passive vision system to concurrently monitor the width and height of the weld bead of GMAW processes in real time. In this case, they used a single camera and a longpass optical filter with a near-infrared cut-off to be used in any metal transfer mode of GMAW processes, which would not create occlusion problems. The measurement process consumes less than three milliseconds per image to attain more than 300 frames per second (fps) transfer rate. Thus, it can be applied in control systems of the stated parameters because a reasonable processing time might be required in the control stage.
On the other hand, real-time penetration monitoring is considered a significant challenge. This measurement generally relies on an indirect estimation based on diverse characteristics. For example, Zou et al. [19] promote studies of the weld pool oscillation frequency, attempting to obtain models that might characterize the dynamics of weld bead formation and penetration. Shi et al. [20] and Wang et al. [21], for example, used the frequency monitoring technique, such as Zou et al. [19], based on the specular reflection of a structured light pattern; in the first case, it was used for GTAW, in the second for GMAW-P. Here, it should note that, as indicated above, in this technique, the two-dimensional reflected pattern is affected negatively by strong fluctuations of the weld pool, which are typical in the GMAW process.
In contrast, and only for frequency monitoring, Ramos et al. [22] applied shadowgraphy. In this case, the information is one-dimensional; it is measured in the perpendicular axis of the workpiece surface. Then, it is more appropriate for the GMAW conventional process. Fundamentally, the shadowgraphy technique uses a collimated and expanded laser beam to illuminate the scene with the relevant objects (nozzle, wire, and weld pool) and a camera on the other side, with an optical bandpass filter, in agreement with laser beam wavelength [23]. This system captures images of the shadow produced by the illuminated elements with the laser beam, reflecting by the filter the light of the electric arc and allowing to observe the weld pool oscillation.
Therefore, based on computer vision, this paper presents two intelligent systems that allow real-time controlling the geometrical parameters of the weld beads in the GMAW process, in this case, focused on the short-circuit transfer mode in a flat position. Here, the width and height of the weld bead were concurrently monitored and controlled by the passive vision system used in Pinto-Lopera et al. [18]. The difficulty, in this case, is to use a single sensor to measure the width and height simultaneously and use sensor fusion with synchronized electrical signals to stabilize the process, driving two geometric parameters simultaneously. The penetration was checked separately using an active computer vision system based on the shadowgraphy technique. The challenge, in this case, was monitoring the weld pool oscillation frequency guaranteeing the parameter combination to favor penetration in different operating conditions.

Materials and monitoring methods
This study uses a Fronius TransPuls Synergic 5000 power source that operates the GMAW processes in constant voltage mode. In addition, a National Instrument USB 6009 to communicate the computer vision system with the interface for robots ROB 5000, which is in charge of conditioning signals, and communicate the control system with the welding source is used. As consumables, the experiments employed a 1.2 mm electrode wire, AWS A5.18 ER70S-6; 1020 steel plates as base material; a commercial mixture of 94% Ar + 6% CO 2 as shielding gas. The distance from the contact tip to the workpiece used was 15 mm. The control parameters were those provided by the power source and the workstation, in this case, the welding voltage, the wire feed rate, and the welding speed.
External parameters of the weld bead (width and height) were monitored in real time using the methodology presented by Pinto-Lopera et al. [18], here with a DALSA DS-21-001M150 high-speed camera. The passive vision system used an infrared longpass filter with an 800 nm cut-off  Moreover, in the camera lens, the diaphragm aperture used was f/8.0 with adequate depth of field to observe relevant objects in each image (weld pool and wire electrode). The camera used a frame rate of 100 fps with a spatial resolution of 200 rows by 416 columns per image. Thus,   Reference to measure the angle and estimate the signal behavior. This example for weld bead height Fig. 7 Flowchart for the control of the weld bead width the system captured a frame every 10 ms, which was sufficient to get control stage calculations, considering that the processing time required to measure the width and height of the weld bead does not exceed 3 ms. Figure 1 shows conventional images in a welding process with the used vision system, here Fig. 1(a) for a short-circuit period and Fig. 1(b) for an open-arc period.
In addition, the process stability was also evaluated in real time and considered in the control stage. In this case, Vilarinho Short-Circuit Transfer Regularity Index (IVcc) was used, and it allowed regulating the welding voltage adequately in the short-circuiting transfer mode. This index assumes that a scant variance in the short-circuit and openarc times produces process stability, as exposed by Rezende et al. [24]. For example, with this index, Souza et al. [25] study the regularity of metal transfer in the process based on welding voltage. Therefore, it is related to the shortcircuit time (t cc ) and open-arc time (t ab ), including the corresponding standard deviation (σ tcc , σ tab ), as follows: From the arc voltage oscillograms, the control system computed IVcc in real time and synchronized it with the images acquired by the camera on the main computer. Voltage signal acquisition was performed with a National Instrument USB 6353 device using an acquisition rate of 2.5 kHz. In this case, the system calculated the IVcc value for each data acquired in windows time of 200 ms. The process is in a region of instability if IVcc exceeds a threshold value of "1.0." It indicates that in at least one of the times, t cc or t ab , the coefficient of variation is greater than 50%, which generates instabilities.
On the other hand, penetration control considered the oscillation frequency of the weld pool. Therefore, to estimate this, the shadowgraphy technique was applied. In this case, the vision system employed the DALSA DS-21-001M150 camera, with exposure time set at 375 µs and a frame rate of 1000 fps. Each acquired image had a spatial resolution of 250 rows and 344 columns. The light source for the shadowgraphy technique was a He-Ne laser, which sets a wavelength of 632 nm and a 1.2 mm beam diameter. The beam expander collimator used a diverging lens of 12.5 mm focal length and a converging lens of 500 mm, producing a magnification factor of 40 × with a final 48 mm collimated beam. Finally, the bandpass filter agrees with the laser wavelength, 632 nm. Figure 2 presents a conventional frame captured by the system.
In this case, the digital image processing initially uses a mean filter and then a thresholding technique. Here, fix a preset threshold considering the advantage of high contrast between relevant objects (workpiece, weld pool, and wire electrode) and the background. Furthermore, to determine the oscillation of the weld pool, a row position and a column position are preset as part of the image. In this case, the row is preset at the workpiece surface position, and it is marked before the welding process starts. The vision system used a distance of 60 pixels from the wire electrode in the weld bead direction to place the column. Figure 3(a) shows a cross of straight lines to represent the preset row and column in an image after the thresholding process. Figure 3(b) presents the reference point estimated as the weld pool surface. Here, for each binarized image,  Subsequently, the tracking of surface reference points on the weld pool for each image in different time windows generated input signals to the system. Then, the computer vision program computed the Fourier spectrum of these signals. Here, with a frame rate of 1000 fps, each time window is composed of 350 frames (350 ms). Figure 4 presents an example of the point oscillation plot, represented by the signal, in a time window. Figure 5 shows the respective frequency spectrum diagram of Fig. 4 and highlights the most significant frequency at 38 Hz.

Width and height modeling and control
Initially, the proposal is to control the width and height of the weld bead in real time, without a predefined model of the welding process under consideration (consumables and fixed welding parameters). In this case, each control stage, one for width and one for height, is separately conducted every 200 ms. Here, the controller employs an indirect variable designated as the "behavior" of the signal, which evaluates the input signals of the width and height every 200 ms. In this case, the variable estimates whether the parameter increases or decreases according to the acquired signal inclination angle, as represented in Fig. 6 for weld bead height.
The angle in Fig. 6 is computed from a fundamental trigonometric relation, considering normalized values in the signal and the samples. The calculation of this angle helps determine if the controller should act or not. This fact is important because the weld bead formation dynamic from liquid metal can consider a delay compared with the controller time window. Here, if the signal values increase (or decrease) considerably, the control variables do not have to be modified. An angle of 30° is used as the control threshold when the signal value increases and − 30° when it decreases.
The selection of thresholds 30° and − 30° is arbitrary but considers practical experience. In this sense, Then, the computer vision system employs the IVcc variable. Thus, if IVcc exceeds the 1.0 value, the system increases the welding voltage by 0.5 V at the power source. In this case, the controller computes and uses the IVcc after the height control step by applying an additional time window of 100 ms. This time window is to wait for the machine and process reaction to each control stage. In this manner, a complete control step uses 500 ms: two windows of 200 ms, each one by each geometrical parameter, and the additional one of 100 ms for stability.
On the other hand, if no short circuit occurs within the IVcc measurement window, the standard deviations σ tcc and σ tab in Eq. (1) are zero, and IVcc is also zero. Therefore, IVcc is used to check if the metal transfer mode goes from short circuiting to globular. This manner allows correcting and getting back to the initial transfer mode.
Finally, the controller is a goal-based intelligent agent, with a first goal oriented to the width control and a second to the height. The welding speed is a control variable for the width, and the wire feed rate is a control variable for the weld bead height, as exposed in Figs. 7 and 8, which show the flowcharts to control the width and height.
To estimate the amount of increase or decrease in the variable value, the controller applies a sinusoid function based on the measurement errors, expressed as (2) x = sin error where allows varying the limit value of the permitted error, and the error value for each measured variable is computed as.
"Reference value" is the parameter target size to be measured and controlled; the user defines this according to the geometric characteristics desired in the weld bead, which, naturally, must be allowed by the process. Therefore, the control system introduces a heuristic whose basis is the action mode of a human welder. Thus, during the process, the controller can simultaneously consider the width and height of weld beads, and finally, it can correct control variables to estimate the geometric parameters of reference.
In addition, the complementary rules associated with the process stability are introduced based on the signal angle and the IVcc, as follows: • If the signal angle > 30°, do not change the control variable; • If the signal angle < − 30°, do not change the control variable; (3) error = Reference value − Measured value • If IVcc > 1, increase the voltage based on a sinusoidal inference function; • If IVcc = 0, the welding speed and wire feed rate values return to their initial values.
As additional limits for each control cycle, the system takes into account the following: • Wire feed rate changes should not surpass 1 m/min; • Welding speed changes should not surpass 1 mm/s; • The voltage value increase should not surpass 1 V.

Penetration modeling and control
For penetration depth, a direct measurement is not feasible. In this case, the system uses a fuzzy controller and an artificial neural network to define the input values of the controller and process, as shown in the control loop of Fig. 9. The control variables were the wire feed rate and the welding speed. The neural network also computes voltage values In this case, this research used a data set compiled from a three-variable central composite design (CCD) to train the neural network. Table 1 shows the variables and the values for each coded level in the CCD. Based on these values, 20 experiments were carried out. Table 2 shows the frequency ranges computed for each working point. The penetration values given in Table 2 were measured via destructive testing when each weld bead was cross-cut. In this case, initially, three cuts are made for each weld bead at a distance of 20 mm from the center.
The neural network architecture used has a feedforward topology with a four-neuron input layer (two for the required oscillation frequency and penetration measurements and two for standard deviations of such frequency and penetration). There are two hidden layers and an output layer with three neurons representing the process variables. The method used the backpropagation algorithm for training. Once the architecture has been defined and the training method of the artificial neural network has been established in order to predict the process variables, verification is carried out, which consists of validating the designed network with another set of data for which the results are known, in order to verify its efficiency through the analysis of the standard deviations. Figure 9 shows that the controller has the error values of frequency and oscillation range measurements as input. This error is computed using the reference value estimated using the neural network and the measurements obtained in real time using the proposed computer vision system. For these variables, three triangular and two trapezoidal membership functions were applied in the fuzzy system: high negative (HN), low negative (LN), null (N), low positive (LP), and high positive (HP). To calculate the wire feed rate output, two trapezoidal and three triangular functions were used, called quite low (QL), scantly low (SL), same value (SV), low increase (LI), and high increase (HI). To compute the welding speed, two trapezoidal and one triangular functions were used, called decrease (D), same value (SV), and increase (I). Table 3 shows the rules applied to the controller corresponding to Mamdani-type fuzzy inference rules. Fig. 11 Control signals at the initial working point: voltage, 20 V; wire feed rate, 6 m/min; welding speed, 7 mm/s; width reference value, 6 mm; height reference value, 1.8 mm

Results and discussion
Initially, for the weld bead width and height, Figs. 10, 11, and 12 show examples of signals acquired during different welding processes. The control stage starts 4 s after the welding process has begun. Dashed lines in plots of the width and height measurements indicate reference input values for each geometric parameter provided to the controller. Other graphs in Figs. 10, 11, and 12 show synchronized signals of the voltage working, welding speed, wire feed rate, and IVcc, which are helpful to observe the controller behavior, according to the reference values and the real-time calculations made by the computer system. Figures 10, 11, and 12 show that the controller increases the welding speed to diminish the width (and vice versa) and augment the wire feed rate to increase heights (and vice versa). Thus, reaching output values close to the reference values is possible, and it can verify the proper works of the controller. Moreover, it is possible to look as the control signals stabilize as the monitored signal becomes steady close to the reference input value.
On the other hand, despite the welding speed being used to control the width and wire feed rate for height control, both control variables affect the two geometric parameters. This fact is significant because it influences the formation of the weld bead and the stability of the process. Then, it is important to consider appropriated control times and the IVcc parameter. In the case given in Fig. 12, IVcc is unstable and two times surpasses the threshold value (one); here, the voltage correction process is executed twice for the same test, and the geometric reference parameters in the weld bead are attained at the end. Stable welding processes generate stable IVcc over time and IVcc values lower than one (Figs. 10 and 11).
IVcc tracks the stability in the welding process, and the convergence in the control variables, welding speed, and wire feed rate, as shown in Figs. 10, 11, and 12, tends to reach with the application of the control stages. Stable values in these control variables in the welding machine added to adequate stability of the process, given by IVcc, guarantee a stable geometry in the weld beads. Figures 13, 14, and 15 show the signals acquired in the welding process for three different examples for the penetration controller. Each one presents the oscillation frequency, frequency variation, wire feed rate, and   Initial voltage: 20.9 V; initial wire feed rate: 7.9 m/min; initial welding speed: 6.7 mm/s; penetration reference value: 2.5 mm without defects. These aspects are attributed to a good setup of the computer vision system and an accurate choice of parameter values in the control process.
Several efforts have been made to establish the relationships between process parameters and weld bead geometry in the arc welding process. These attempts have been divided in empirical methods and theoretical studies. In empirical methods, they usually used the statistical approach which sets the bases of the relationships between process parameters and bead geometry of GMAW. On the other hand, the theoretical studies are based on heat flow theory. The present work tries to generate a new perspective of control based on intelligent systems that allow reducing the efforts in the knowledge of the process parameters estimated from the parameters of the welding bead with a good level of precision. Although optimization is not guaranteed in the proposed methods, the results obtained are not trivial since it presents a methodology that minimizes the number of experimental results necessary to predict the values of voltage and wire speed from the geometric characteristics of the desired weld bead.
Finally, in the geometric parameter control, the time spent to stabilize the process and the parameter is an important factor and should be considered in the future. In any case, this study stage concentrated on generating the control of the geometric parameters under a new perspective, using intelligent and autonomous systems; this work did not study the stabilization time optimization.

Conclusions
Regarding the external geometry of the weld beads, this study introduces a computer vision system following a goal-based intelligent agent. This system allows to simultaneously control the width and height of weld beads for the GMAW process in a short-circuit transfer mode, without requiring a predefined model of the specific welding process. We concluded that the inference method used in the controller does not represent a mathematical model of the process, but it can increase or decrease control variable values in the appropriate percentage, depending on errors between monitored signals and reference values for each geometric parameter. Furthermore, the sensor fusion and the real-time monitoring of the Vilarinho Short-Circuit Transfer Regularity Index help correct the instabilities in the control processes.
In the case of penetration, the computer vision system, based on a single sensor, allows segmenting relevant objects in the scene and quickly and correctly extracting the required information, thus processing one frame each millisecond. Concerning the oscillation frequency of welding pools, results of the frequency spectra show adequate distributions and behavior according to the expectations. Thus, it can conclude that the methodology is appropriate for the monitoring system, and it validates the application of the fast Fourier transform as adequate in the acquisition of the oscillation frequency.
Accordingly, the developed modeling procedure is significant as a methodological approach, guaranteeing any parameter combination to favor penetration. Hence, the model presents a good appearance and quality welds, attaining the desired characteristics. Regarding the fuzzy controller, it is concluded that the technique allows the system to adapt immediately to variations in characteristics throughout the welding process. Then, appropriate robustness is achieved, and instabilities are corrected when operating conditions change.
Author contribution All authors contributed to the study conception and design. Software (penetration measurement mainly), formal analysis, investigation, data debugging, and writing-reviewing and editing were performed by Jorge Andres Girón-Cruz. Software (external parameter measurement mainly), formal analysis, investigation, data debugging, writing (original draft), and visualization were performed by Jesús Emilio Pinto-Lopera and the conceptualization, validation, supervision, and project administration were performed by Sadek C. A. Alfaro. All authors read and approved the final manuscript.
Funding Open Access funding provided by Colombia Consortium   Fig. 18 Cross-sectional photography for a 2.5 mm penetration reference value and corresponding direct measurement, derived from a controlled welding process: (a) 2.47 mm; (b) 2.56 mm; (c) 2.46 mm