1 Introduction

Welded regions in metal structures have a lower fatigue life than the base material. This is mainly caused by local stress concentrations along the weld toe arising from high tensile residual stresses appearing when the weld solidifies and cools down. Additionally, stress concentrations can be caused by notches appearing from the weld geometry, and weld imperfections [1]. Various post-weld techniques have been invented to improve the fatigue life of welds. These techniques apply standard industrial processes, such as burr grinding, tungsten inert gas (TIG) dressing, shot peening, laser peening and high-frequency mechanical impact (HFMI) techniques as ultrasonic impact treatment (UIT) and pneumatic impact treatment (PIT) [1,2,3,4,5,6,7,8]. The working principles of the treatment techniques are different, but in general they have one or more of the following aims;

  • relieving the weld’s tensile residual stress zones or even turning them into compressive stress zones,

  • smoothening notches,

  • suppressing or removing weld defects.

This paper focuses on HFMI treatment of welds for the offshore industry. The method can be adapted to other applications such as the automotive industry and other post-weld techniques, e.g. burr grinding. Burr grinding removes material on a pass along the weld toe, and by doing this, weld defects are removed and concurrently, the stress concentration factor is reduced. By using a burr grinder, the resulting grinding marks are parallel to the load direction, making them act as a prevention for crack initiation [1]. HFMI treatment works by hammering a pin in the weld toe region along the weld at a frequency of > 90 Hz causing plastic deformation of the weld toe, as illustrated in Fig. 1. This process reshapes notches into a smooth geometry along the weld toe and additionally relieves tensile residual stresses in the area and builds up compressive residual stresses [9]. HFMI has been proven to significantly increase fatigue life, regardless of the level of treatment [10]. In a study by E. Harati et al. [11], it was concluded that HFMI treatment could increase fatigue life by 26% in welds of ultra-high-strength steels, while other studies have found similar benefits in mild and high-strength steels [9, 12, 13].

Fig. 1
figure 1

Weld seen before and after HFMI treatment. It is observed how the treatment has plastically deformed the weld toe into a smooth path, removing sharp notches

Full size image

The typical industrial way of applying burr grinding and HFMI treatment is manual and based on user experience. Treating more complex geometries, including curvilinear weld toes, has proven to be very demanding and requires more expertise than treating linear welds to ensure an acceptable quality that adheres to the guidelines of G. Marquis and Z. Barsoum [14]. The HFMI treatment tool vibrates, the process is very noisy, and the tool is heavy; therefore, the working environment is not very pleasant for an operator through a continuous eight-hour work shift. Even though operation for eight continuous hours is generally permitted depending on the tool manufacturer, hearing and eye protection is mandatory. An automated solution for post-weld treatment would be highly interesting for the industry to overcome the working environment concerns, and could potentially be economically profitable [15].

Robotising the post-weld treatment is expected to result in higher reliability, repeatability and improved control of the process parameters when compared to manual treatment [16]. Moreover, robotising the treatment could in itself lead to a more efficient treatment as clamping the tool, or mounting it onto a robot, has shown to result in a scatter band of compressive forces that are 50% narrower than manual operation while maintaining or improving the impact force [16,17,18]. Utilising a robot further enables maintaining a constant speed and controlled impact angle along the weld.

Studies on robotising the post-weld treatment are yet limited. The authors of R. Yekta et al. [10] and K. Ghahremani et al. [19] compared robotic treatments along straight trajectories to manual reference treatments. The purpose of robotising the treatment was, in this case, not focused on improving treatment quality but to minimise bias in the treatments. The results showed that the measured radius of the treated area, the indentation depth (base metal side) and indentation depth (weld side) of the robotic treatment of R. Yekta et al. [10] had a scatter that was respectively ≈ 145%, ≈ 197% and ≈ 80% larger than that of the manual treatment. A similar result was achieved by K. Ghahremani et al. [19], where the scatter of the measured radius of the robotically treated area was more than 100% larger than the comparable manual treatment. This more considerable scatter is in both cases believed to be a result of the robot’s inability to adapt to the inconsistency of the weld like a manual operator would, which negatively affects the quality of the post-treatment.

Not adapting to the weld profile can further result in treating perpendicular to the base material. Therefore, hitting the sides of the weld creates folds, thus further degrading the fatigue performance of the weld [20]. Hence, the current solution for robotic HFMI treatment is inferior to manual treatment, indicating why a robotic solution has not yet been developed. By adapting to the weld and thereby not following a generic straight trajectory, it is possible to develop a solution that can reduce scatter in the treatments and achieve process stability comparable to or better than manual treatment. Improved process stability is here defined as a reduction in the variance of the quantitative measures, e.g. indent depth and groove width and radius of the treated area.

Several researchers have investigated methods for automatic trajectory generation to treat parts with nonconformities using a range of vision methodologies and setups. The authors of O. Gurdal et al. [21] have developed a solution for robotic finishing of friction stir-welded corner joints that utilises a laser line scanner mounted to an industrial robot to generate a 3D model of the part. The 3D model is further used to locate the workpiece and automatically generate a robot path to treat the component. H. Zhang et al. [22] have developed a hybrid approach that utilises a 2D camera in combination with a force sensor to automatically generate robot programs that can follow a free-form surface for manufacturing tasks. However, none of these methods have been utilised for automatic post-treatment of welds.

Performing the automated post-weld treatment itself is one challenge; performing automated quality assurance is another. For quality assurance of post-weld treatment, methods and guidelines for recommended execution exist, such as G. Marquis and Z. Barsoum [14] and G. Marquis and Z. Barsoum [15]. These guidelines for HFMI treatment include recommendations related to, e.g. working speed and travelling and working angles.

The quantitative measures are typically evaluated using simple manual gauges and follow the same principles that are applied, when evaluating weld geometries. These gauges can only measure a single point at a time and are often associated with significant variance. P. Hammersberg and H. Olsson [23] studied the resulting variance when using a gauge for measuring throat thickness and depth of undercut in welds. The authors was concluded that the gauge itself contributed to almost 60% of the total variance due to manual operation. When making binary Go/No Go decisions, a standard requirement states that the measurement system should not contribute with more than 9% of the total variation [24]. Hence, the need for a measurement system that does not rely on a simple manual gauge for accurate quality assurance of post-treated welds is clear.

Several researchers have studied methods for non-contact measurements of treated and non-treated welds with the purpose of quality assurance. L. Yang et al. [25] proposed a method for determining weld quality by state vector machine (SVM) classification on 3D images that have been reconstructed using the shape from shading (SFS) algorithm. This method does not perform any direct weld measurements but classifies the overall quality, e.g. incomplete fusion and inadequate penetration. T. Stenberg et al. [24] proposed a system for online quality control and assurance of welds using a laser line scanner. The system showed a significant improvement in measurement variation when compared to manual measuring methods. Other authors have applied laser line scanners to measure weld profiles and detect weld defects by extracting feature points using various methods, e.g. through second-order differentiation of the weld profile [26,27,28]. However, the above methods are only designed for quality assurance of welds and do not focus on post-treatment of welds and quality assurance thereof.

K. Ghahremani et al. [19] have utilised a handheld 3D scanner as a method for quality assurance of HFMI treatment to evaluate the quantitative measures as defined by G. Marquis and Z. Barsoum [15]. The authors found that point cloud based measurements of the indentation depth could successfully be used to determine the treatment level: undertreated, properly treated or overtreated. However, the proposed method is based on manual processing of the point cloud data to perform the measurements, which is unsuitable for an automated solution.

It can be concluded from the literature review that automated HFMI treatment shows excellent potential as a method for improving the stability of the post-weld treatment and the reliability of the subsequent quality assurance. However, the current solutions for robotic HFMI treatment are inferior to manual HFMI treatment. This is mainly due to the robotic treatment trajectory not being adapted to the weld but performed along a straight line with constant travelling and working angles. Automatic robotic trajectory generation has to the authors’ knowledge, currently not been developed for HFMI treatment of welded components. It can additionally be concluded that there exists a research gap in automated quality assurance for HFMI treatment, as the current methods rely either on simple manual gauges or manual processing of 3D point clouds of the weld surface. Nevertheless, methods have been proposed for evaluating the quality of the weld itself, however, not for post-weld treatment.

To overcome the challenges of the current solutions, this paper proposes an integrated software and hardware system that can automatically perform robotic post-weld treatment and post-treatment quality assurance based on 3D scanning. The intended purpose of the developed system is to perform automated flexible post-weld treatment of welds in an industrial setting which allows for improved process stability and quality assurance.

There are two main contributions of this paper. The first contribution is the development of an algorithm that can accurately and automatically locate the weld toe based on 3D surface information of the weld, such as the weld profile’s gradient and curvature. This information is used to generate an adaptive robotic trajectory for performing post-weld treatment, which follows the inconsistencies of the weld toe and adapts the travelling and working angle to the weld profile. The second contribution is further research of the algorithm such that it can locate the treated area and evaluate the treatment quality by determining the quantitative quality measures, such as indentation depth, groove width and deviation from the weld toe. The developed algorithm is implemented in a generalised software and hardware framework that combines automated post-weld treatment and quality assurance in a flexible production setup.

The proposed solution was experimentally verified by performing post-weld treatment on 600 mm of weld toe based on the adaptive robotic treatment trajectory generated by the developed system. To evaluate the performance of the system, comparable treatments were performed manually and using a linear robotic trajectory. As the developed system accurately followed and adapted the treatment trajectory to the weld toe, it showed a significant improvement in the overall process stability compared to the linear robotic treatment. Only moderate quality improvements were shown compared to the manual treatment performed by a human operator. However, due to the robot performing the treatment, the working environment was vastly improved for the human operator.

The paper is structured as follows: Section 2 gives an introduction to the developed system and architecture. Section 3 presents the methodology for filtering the raw point cloud from the 3D scanner for data processing. Section 4 presents the first main contribution; the method for locating the weld toe and generating the adaptive treatment trajectories. Section 5 presents the second main contribution; the proposed approach for determining the quantitative measures for the treated weld toe. Section 6 presents the experimental setup that is used to test and validate the developed system. Section 7 presents and discusses the obtained results, while Section 8 summarises the conclusions and reflects upon future work.

2 System description and architecture

The developed system relies on a combination of software and hardware operations, as shown in the flow chart, Fig. 2. The system takes the welded part as input and follows the five main steps, as described below:

  1. I

    A 3D scanner mounted onto an industrial manipulator is utilised to acquire a point cloud of the weld along a linear, manually programmed trajectory. The result is a 3D point cloud representing the surface of the sample as a closely spaced grid of x, y and z coordinates.

  2. II

    The acquired point cloud is analysed to locate the weld toes to treat. This information is used to generate a set of process instructions, such that the treatment is adapted to the inconsistencies of the weld and adheres to the guidelines for recommended execution [14].

  3. III

    Based on the generated process instructions, the post-weld treatment is automatically performed using an industrial manipulator.

  4. IV

    Following the treatment, the weld is rescanned using the same linear robotic trajectory as in step I.

  5. V

    The acquired point cloud is analysed to locate the treated groove. The quantitative measures of the treated groove, i.e. indentation depth, weld toe radius and deviation from weld toe, are evaluated, and quality documentation is generated.

Therefore, the system’s output is the treated part, documentation of the treatment and quantitative quality measures. The individual steps will be further described in the following sections.

Fig. 2
figure 2

Architecture of the developed system. Yellow arrows indicate flow of material. Black arrows indicate flow of data. Red boxes represent physical operations. Black boxes represent software for data processing

Full size image

3 Pre-processing of 3D scans

The system requires the scanned data to be arranged in a 2D grid form. Furthermore, the scan must be performed so that the grid lines are parallel or perpendicular to the weld. For this reason, a line scanner that works by applying the principles of laser triangulation is utilised. In the case of the line scanner, the scan should be performed along a linear trajectory at a constant speed. The scanner should be oriented such that the scan line (x-direction) is perpendicular to the length of the weld, while the scanning direction (y-direction) should be parallel to the length of the weld. The resulting z-direction is normal to the surface of the weld. An example of a scan is seen in Fig. 3.

Fig. 3
figure 3

Point cloud from a scanned weld before treatment consisting of closely spaced 2D weld profiles. A random weld profile is highlighted as the red line. Note that the parts of the point cloud that do not contain the weld have been removed; hence, the y-axis does not start from 0. Furthermore, the distance values of the z-axis indicate the distance to the origin of the scanner

Full size image

Due to the nature of the line scanner, the data is acquired as m number of scanned profiles, each consisting of n number of points with a distance between each point of dx; defined as the x-resolution. All of the scan lines are parallel with a homogeneous spacing of dy, defined as the y-resolution. As the grid does not determine the z-values, the spacing is continuous. To accurately locate the weld toe, the required x- and z-resolution of the system is < 50 μm, while the required resolution in the y-direction is < 1000 μm to compare the irregularities of the weld toe.

This means that each of the scan lines represents a 2D profile u of the weld, as illustrated by the red line in Fig. 3, which combined constitute the entire point cloud of the scan U. Finally, the scans have been performed such that each scan contains a single, continuous weld with both welds toes. To remove noise and outlying points from the point cloud, a 2D median filter is applied. The 2D median filter is a well known non-linear filter known to preserve edges while removing noise. It works by sliding a window over a neighbouring region and returning the median value of that region. To reduce numerical noise in the succeeding steps, a low-pass Butterworth filter is used independently on each acquired line to smooth the data further.

4 Developed algorithm for post-weld treatment planning

The weld toe, as seen in Fig. 1, is defined as the junction of base material to weld face, which can be empirically observed as a sudden change in the surface gradient along with the profile of the weld. To locate the weld toe in the point cloud of the scan, the gradients \(\boldsymbol {\theta }^{\prime }\) and curvatures \(\boldsymbol {\theta }^{\prime \prime }\) along and between each of the weld profiles are determined. This approach is inspired by Y. Li et al. [26] and Y. Han et al. [27], who both apply second-order information to perform weld bead measurement.

As each scan line represents a weld profile, each weld profile is described as a discrete set of points; therefore, \(\boldsymbol {\theta }^{\prime }\) and \(\boldsymbol {\theta }^{\prime \prime }\) are vectors that can be mathematically expressed by first and second-order finite approximations, defined by Eqs. 1 and 2. To minimise noise and acquire as much information as possible, central difference approximations using a 3-point 1D stencil are applied.

$$ \begin{array}{@{}rcl@{}} \theta^{\prime}_{i,j} = \frac{U_{i+1,j} - U_{i-1,j}}{2 dx} + \frac{U_{i,j+1} - U_{i,j-1}}{2 dy} \\ \text{for} \quad i = 2, 3, \ldots, n-1, j = 2, 3, \ldots, m-1 \end{array} $$
(1)
$$ \begin{array}{@{}rcl@{}} \theta^{\prime\prime}_{i,j} &=& \frac{U_{i+1,j} - 2 U_{i,j} + U_{i-1,j}}{dx^{2}} \\ &&+ \frac{U_{i,j+1} - 2 U_{i,j} + U_{i,j-1}}{dy^{2}} \\ &&\text{for} \quad i = 2, 3, \ldots, n-1, j = 2, 3, \ldots, m-1 \end{array} $$
(2)

As the point cloud consists of a set of closely spaced weld profiles, j defines the index of the weld profile in the y-direction, such that Uj+ 1 is the adjacent weld profile to Uj. i defines the index of the point in the x-direction along with the weld profile of Uj. The coordinate system is presented in Fig. 3. In the following, the proposed methods will focus on a single weld profile u at a random index j. However, these methods will be applied for all weld profiles U for j = 2, 3, … , m − 1, unless otherwise specified. Hence, the index j will be omitted for the sake of simplicity.

In an optimal case, the two weld toes will be represented as the two global negative peaks of the curvature \(\boldsymbol {\theta }^{\prime \prime }\) along with the weld profile. Though, as the weld quality, notches and other surface variations affect the weld toes’ prominence, these can result in a peak curvature more significant than that at the weld toes. The same problem applies for multi-pass welds, where the curvature at the interpasses can also be more significant than that at the weld toes. This issue is illustrated in Fig. 4, where the local peaks of the curvature \(\boldsymbol {\theta }^{\prime \prime }\) are plotted onto the scan of the weld.

Fig. 4
figure 4

The local peaks of the curvature \(\boldsymbol {\theta }^{\prime \prime }\) plotted across the entire point cloud. Scan noise and other surface irregularities will present themselves as randomly distributed points, while the distinct lines of connected and semi-clustered points in the scan are the weld toes and other distinct regions

Full size image

The challenge associated with determining which peak curvatures belong to the weld toes is further apparent from Fig. 4. The first step to resolve this issue is to assume that the gradient \(\boldsymbol {\theta }^{\prime }\) of the weld profile near the weld toe approaches zero. This is a valid approach, as it has been observed through more than 50 scanned weld toes following the above-mentioned scanning approach, that the weld toe has a semi-circular geometry in which the gradient of its tangent at some point along the weld toe geometry approaches zero. It has further been observed, that the approach is insensitive to small rotations around the y-axis of the local coordinate system (Fig. 4) of the point cloud.

The extracted weld profile further illustrates this in Fig. 5, middle graph, where it can be seen, that the gradient \(\boldsymbol {\theta }^{\prime }\) approaches zero, as we move closer to the weld toe. Using a normal probability density function, Eq. 3, it is possible to determine a set of weights wθ to penalise the curvature values according to their associated gradients.

$$ \begin{array}{@{}rcl@{}} {\boldsymbol{w}}_{\theta}= f\left( \boldsymbol{\theta}|\mu_{0},\sigma \right) = \frac{1}{\sigma \sqrt{2\pi}}\exp{\frac{{-(\boldsymbol{\theta}^{\prime}-\mu_{0})^{2}}}{2\sigma^{2}}} \end{array} $$
(3)

In Eq. 3, μ0 is mean of the applied density function, which is set to 0, as this is the gradient \(\boldsymbol {\theta }^{\prime }\) to aim for. σ is the standard deviation or spread of the density function, that determines how the gradients are penalised. As the total area under the probability density curve is equal to 1, the penalty values of wθ must be normalised to ensure that a gradient of 0 is not penalised (see Eq. 4):

$$ \tilde{\boldsymbol{w}}_{\theta} = {\frac{\boldsymbol{w}_{\theta}}{\sigma \sqrt{2\pi}}} $$
(4)

the penalised curvature \(\boldsymbol {\hat {\theta }}^{\prime \prime }\) can therefore be expressed by Eq. 5

$$ \boldsymbol{\hat{\theta}}^{\prime\prime}= \|\tilde{\boldsymbol{w}_{\theta}}\| \circ \boldsymbol{\theta}^{\prime\prime} $$
(5)

The bottom graph of Fig. 5 represents the penalised curvature \(\boldsymbol {\hat {\theta }}^{\prime \prime }\) as the blue line, where the two most significant negative peaks have been circled. In an optimal case, the most significant penalised curvature peaks will represent the weld toes. Though, due to scan noise and local surface variations, this is not always the case. Based on the assumption that the weld toe forms a continuous line in the y-direction of the scan, the peak penalised curvatures representing the weld toe should likewise form a continuous line in the scan. As some scanned profiles will not correctly represent the location of the weld toes due to noise, these must be omitted.

Fig. 5
figure 5

The processing stages of each of the scanned profiles. The weld toe location is illustrated on the weld profile as the circled points, which lie at the local negative peaks of the penalised curvature \(\boldsymbol {\hat {\theta }}^{\prime \prime }\). Note that using the largest peaks of the curvature \(\boldsymbol {\theta }^{\prime \prime }\) itself would lead to an incorrect estimation of the weld toe

Full size image

To ensure continuity in the determined peak penalised curvatures, several continuous reference regions (blue regions of Fig. 6) must be established to define in which areas the weld toes should be located. This is initially done by dividing the scan into several sections (divided by the red lines of Fig. 6), each having q number of weld profiles depending on the y-resolution of the scan. Computing the mean of the q number of weld profiles in each section will result in a mean profile of each section, where non-continuous local peak curvatures such as notches and other local surface variations are filtered out as a result.

Fig. 6
figure 6

The scan divided by the red lines into separate sections of q = 10 weld profiles. The vertical yellow lines define the continuous peaks of the curvature between the mean profiles. The blue areas are the reference regions in which the weld toes should be located

Full size image

A search algorithm based on the principles proposed by E. Moore [29] that finds the shortest path in a weighted graph is applied. Through this approach, it is possible to select the most significant peak curvatures of the mean profiles that form continuous lines through the scan with the smallest x-offset between them. The selected peak curvatures will act as the centres ρ (yellow lines of Fig. 6) of the reference regions. The results are illustrated in Fig. 6, where five continuous reference regions have been found. Curvatures \(\boldsymbol {\hat {\theta }}^{\prime \prime }\) outside of the reference regions will be ignored, as these are assumed to be a result of noise.

As each weld toe can be described by a point on the weld profile, a single point within each of the reference regions across each weld profile must be determined. In the case of Fig. 6, a total of five points per profile u must be located, as five continuous lines have been found through the scan. These five points are selected based on a weighting of the peak curvatures in each reference region and their corresponding distances Δp to the centre of that region, computed as a Euclidean norm. The weights are based on Δp and estimated following the same principles as Eq. 3, again setting μ0 = 0, as this is the distance to approach. As a result, the five points that have the best balance between penalised curvature \(\boldsymbol {\hat {\theta }}^{\prime \prime }\) and distance to the reference centres are selected. This operation is performed for all weld profiles U for j = 2, 3, … , m − 1, such that five continuous lines are located through the weld.

As there are only two weld toes, the remaining three obsolete lines must be removed. The mean values of the curvatures \(\boldsymbol {\hat {\theta }}^{\prime \prime }\) for all j = 2, 3, … , m − 1 are determined. Initially, the lines with a mean curvature below a pre-determined threshold value are removed. These are typically the lines in the inherently flat base material, as the curvature here will be low. The remaining lines will hence lay in or at the edge of the weld material. As the weld toe is characterised as the transition between base and weld material, the two lines that are the furthest away from the centre of the weld, one on each side of the weld, are considered the two weld toes. The result is plotted as the red and blue lines in Fig. 7. Smoothing splines are applied on the estimated positions to filter and further improve the robustness and accuracy of the algorithm, resulting in a m − 1 × 3 vector Ω of (x, y, z) coordinates for each weld toe for all weld profiles U. The result is plotted as the blue and oranges lines in Fig. 7.

Fig. 7
figure 7

The determined continuous lines through the scan. The blue and red lines are the determined weld toes that have been smoothed using a smoothing spline. Identical colour notation is used throughout the remainder of the paper, such that the red lines refer to the left weld toe, while the blue lines refer to the right

Full size image

4.1 Treatment trajectory generation

According to the guidelines proposed by G. Marquis and Z. Barsoum [15], the treatment must also be performed with a specified set of angles that have been defined relative to the surface of the weld and travelling direction (Fig. 8).

Fig. 8
figure 8

Illustration of the treatment angles. The working angle ϕ is defined with respect to the surface of the base metal. The travelling angle ψ is defined with respect to the direction of travel

Full size image

Therefore, the surrounding points around the weld toes are used to determine the required tool orientation along the treatment trajectory. The travelling angle ψ is defined with respect to the direction of travel and computed based on the change in curvature of the along with the weld toe (see Eq. 6). As the treatment must be performed with a travelling angle ψ within a specified interval, a reference angle ψ0 is set. The reference angle is based on operator experience and does generally not vary between similar welds.

$$ \psi_{i} = \psi_{0} + \operatorname {atan2}\left (\frac{{\varOmega}.x_{i} - {\varOmega}.x_{i-1}}{{\varOmega}.y_{i} - {\varOmega}.y_{i-1}} \right ) \quad i = 2, 3 {\ldots} m - 2 $$
(6)
$$ \begin{array}{@{}rcl@{}} \phi_{i} = \eta_{b_{i}} + \eta_{w_{i}} - \phi_{0} \quad \text{for} \quad i = 1,2 {\ldots} m-2 \end{array} $$
(7)

The working angle ϕ is defined with respect to the surface of the base metal is computed by fitting two linear lines starting from the weld toe and pointing in the opposite direction; one line in the direction of the base material and the other line in the direction of the weld. Each line is fitted with the number of points that transverse across a distance equivalent to the HFMI tool pin tip radius. This is done to ensure that the fitted lines represent the area of which the tool hits the weld. A reference angle of ϕ0 = 90° relative to the surface is used, as this is the angle to apply for a completely flat surface. The working angle is then computed as Eq. 7.

ηb is the angle of the fitted line across the base metal, while ηw is the angle of the fitted line across the weld metal. The computed angle ϕi is further constrained by the recommended range stated by the IIW recommendations [14]. The computed angles can be seen in Fig. 9.

Fig. 9
figure 9

The generated treatment angles based on the surface of the weld and the curvature of the weld toe. The computed working angle ϕ and travelling angle ψ for the two weld toes

Full size image

The computed working angles and travelling angles have been smoothed using a smoothing spline. This is done to minimise rapid changes in the angles during treatment that will be difficult for the robot to follow.

In order to generate the HFMI trajectory for the robot, the estimated positions of the weld toes and the treatment angles are transformed using a transformation matrix from the local coordinate system of the scanner to the global coordinate system of the robot. The required transformation matrix is determined through a simple calibration based on global reference points from the robot and local reference points from the scanner. By touching a set of reference points with the TCP of the HFMI tool on the robot and subsequently scanning the same points and locating them in the 3D scan, it is possible to apply a non-linear least-square optimisation to compute the transformation. The HFMI tool is also calibrated to the robot, which is done by applying a 4-point calibration procedure implemented in the robot’s controller.

This calibration operation must be performed, when changes or maintenance have been done to the scanner or tool. Having determined the global coordinates of the weld toe, a robot program can automatically be generated, hence limiting the amount of manual programming.

5 Developed algorithm for post-weld treatment quality inspection

As defined by G. Marquis and Z. Barsoum [15], the quality of the treatment and the resulting groove can be assessed based on two measures: qualitative (I) and quantitative (II).

  1. I

    The qualitative measures are based on a visual inspection and evaluate the quality of the groove. E.g. the groove must be continuous, smooth and shiny and without a visual presence of the original fusion line. These are clear visual differences between the levels of treatment: under, over and properly treated R. Yekta et al. [10]. Flaking type damage can also be observed in cases of high coverage rates, i.e. two passes or low-speed [20].

  2. II

    The quantitative measures, as illustrated in Fig. 14, are directly measurable. These include the indentation depth (base metal side) Db and groove width W, where the indentation depth has proven to be an excellent measure for quality assurance [19]. The optimal indentation depth (base metal side) Db and groove width W depend on the material and tool, but should be between 0.2 and 0.6 mm in depth and 3 and 6 mm in width. However, the minimum Db should be between 0.1 and 0.2 mm to ensure a complete HFMI treatment. Similar guidelines apply for the HFMI treatment groove location, which must be placed such that 25–75% of its width is in the weld material. It should be noted, that the recommendations serve as guidelines and depend on several factors, including the yield strength of the welded steel. [14].

The focus will be placed on the quantitative measures, as these can be directly measured using a 3D scanner. The weld is therefore rescanned post-treatment, utilising the same scanning trajectory and principles as the pre-treatment scan.

The scans must be aligned in the same coordinate system to compare the weld before and after treatment. Aligning two point clouds is done using the iterative closest point algorithm (ICP), originally proposed by K. Arun et al. [30], which searches for a set of reference points between two point clouds. A non-linear optimisation then follows to determine the transformation between the two 3D point clouds.

Pre-processing of the scan for determining the quantitative measures follows the same methods, as described in Section 3. The resulting scan of a peened weld is presented in Fig. 10.

Fig. 10
figure 10

Point cloud from a scanned weld post-treatment. A profile of the treated weld is highlighted as the red line. Note that the scanned section is that same as for the untreated weld in Fig. 7

Full size image

On an extracted single profile u, illustrated in Fig. 11, the peened grooves can be identified as two semi-circles, constrained by sharp edges on each side of the grooves. Mathematically, the edges are described as a sudden change in the surface gradient perpendicular to the length of the groove. Applying Eq. 1 to determine the curvature across the scanned surface is, therefore, the first step. However, as the curve that forms the groove edge points away from the weld, the curvature will have the opposite sign than the weld toe. Therefore, the aim is to find the most significant positive curvatures. In this case, the gradient is not used as a weighting parameter, as this has been shown to influence the results negatively.

Fig. 11
figure 11

The processing stages of each of the scanned profiles. The groove edges’ locations are highlighted on the weld profile with red cirlces and can be associated with the local peak of the curvature at the indicated points

Full size image

As with the weld toes, the four largest peak curvatures would represent the four edges of the two grooves in an optimal case. Yet, due to notches and other surface noise, the four largest peak curvatures might not always represent the peened grooves’ edges. The approach for selecting the correct peak curvatures on each weld profile follows the same method as for the weld toes, described in Section 4; hence a number of reference regions that form continuous lines through the scan are determined and then used to constrain the area in which the curvature peaks should be located. The result can be seen in Fig. 12.

Fig. 12
figure 12

The scan divided into separate sections of weld profiles by the red lines. The yellow lines define the continuous peaks of the curvature between the mean profiles, while the blue area is the reference region in which the groove edges should be located. In this case, there are seven references regions

Full size image

In this instance, seven continuous lines are formed through the reference regions. As there are only four edges for the two grooves, three lines must therefore be removed. The approach again follows the same principles as for the weld toes. In this case, it is the four lines furthest away from the weld with mean curvatures above a pre-determined threshold that define the groove edges; two on each side of the weld. Therefore, each peened groove will have m − 1 × 3 Γw and m − 1 × 3 Γb matrices containing the groove edge positions γw and γb for respectively the weld side and base metal side. These are plotted as the black lines in Fig. 13. The estimated groove positions, further described in Section 5.1, are plotted as the blue lines in Fig. 13. The position of the weld toes before treatment are plotted as the white lines in Fig. 13.

Fig. 13
figure 13

The located HFMI treatment grooves in the scanned point cloud. The black lines indicate the positions of each of the groove edges. The red and blue lines are the located groove positions used to determine the error 𝜖 for respectively the left and right weld toe. The whites line are the positions of the weld toes Ω before treatment

Full size image

5.1 Determining the quantitative measures

Having located each groove of the treated weld toes, the quantitative measures can be determined. The quantitative measures are illustrated in Fig. 14 and determined for all weld profiles U and both peened grooves. The groove for each weld toe is found on the weld profile u between the indexes Iw and Ib which, respectively, define the groove edge (weld side) and groove edge (base metal side).

Fig. 14
figure 14

The quantitative measures based on the IIW recommendation, G. Marquis and Z. Barsoum [14], for HFMI treatment and the measures proposed by K. Ghahremani et al. [19]. The measurements are performed on both the left and right cut out just mirrored. W is the groove width, Db the Indentation depth (base metal side), Dw the indentation depth (weld side), R is the weld toe radius and ω is the position of the weld toe before treatment

Full size image

Groove width W:

The indentation width is computed by fitting a line lb to the surface of the base metal and projecting the positions of the groove edge (weld side) γw and the groove edge (base metal side) γb onto the fitted line. If lb is described by two points, p0 and p1, the point γw can be projected onto Ib using a vector projection, presented by Eq. 8. The same principle applies to the projection of γb.

$$ \begin{array}{@{}rcl@{}} \gamma^{\prime}_{w} = {p_{0}} +\frac{\overrightarrow{{p_{0}\gamma_{w}}}\cdot \overrightarrow{{p_{0}p_{1}}}}{\overrightarrow{{p_{0}p_{1}}}\cdot \overrightarrow{{p_{0}p_{1}}}} \cdot \overrightarrow{{p_{0}p_{1}}} \end{array} $$
(8)

The Euclidean distance between the two projected points is, therefore, the groove width W (see Eq. 9).

$$ \begin{array}{@{}rcl@{}} W = \|\gamma_{w}^{\prime} - \gamma_{b}^{\prime} \| \end{array} $$
(9)

Indentation depth (base metal side) D b:

The Euclidean distance from the point pd in the groove at index Id with the largest perpendicular distance to the fitted line lb is used as the indentation depth Db of the base metal side. The perpendicular distances S for all points in the groove is determined through Eq. 10.

$$ \begin{array}{@{}rcl@{}} \boldsymbol{S} = {\frac{\sqrt{|{\overrightarrow{{p_{0}p_{1}}}}|^{2}|{\overrightarrow{{p_{0}u_{i}}}}|^{2} - (\overrightarrow{{p_{0}p_{1}}} \cdot \overrightarrow{{p_{0}u_{i}}})^{2}}}{\|\overrightarrow{{p_{0}p_{1}}} \|}} \\ \text{for} \quad i = 1,\ldots,n-1 \end{array} $$
(10)

Therefore, the indentation depth Db of the base metal side can be expressed by Eq. 11, while the corresponding index Id and point pd can be expressed by Eq. 12.

$$ \begin{array}{@{}rcl@{}} D_{b} = \max_{i}\{S_{i} : I_{b},\ldots,I_{w}\} \end{array} $$
(11)
$$ \begin{array}{@{}rcl@{}} I_{d} = \arg\max_{i}\{S_{i} : I_{b},\ldots,I_{w}\} \implies p_{d} = S_{I_{d}} \end{array} $$
(12)

Indentation depth (weld side) D w:

The indentation depth (weld side) is based on the metric presented in K. Ghahremani et al. [19]. A line lw is spanned between the point pd in the groove and the groove edge on the weld side γw. The Euclidean distance from the point in the groove, constrained by pd and γw, that has the largest perpendicular distance to lw is used as the indentation depth Ds of the weld side. Computation follows the same methods as for the indentation depth on the base metal sides; hence the indentation depth is expressed through Eq. 13.

$$ \begin{array}{@{}rcl@{}} D_{w} = \max_{i}\{S_{i} : I_{d},\ldots,I_{w}\} \end{array} $$
(13)

Weld toe radius R:

The weld toe radius R is determined by applying the Hyper fit method to fit a circle to the points along with the weld profile u, that constitutes the groove constrained by γw and γb. The Hyper fit method, proposed by Ali Al-Sharadqah and Nikolai Chernov [31], is an algebraic fit that is computationally inexpensive and not subject to divergence, opposite to a geometric fit. Additionally, the method benefits from having a higher accuracy and lower sensitivity to noise than other competing methods [31].

Groove vs weld toe (error) 𝜖:

To establish the groove deviation from the weld toe, a reference position for the groove must be determined. It is assumed that the contact point of the tool tip constitutes the deepest point pg in the groove, measured as the largest perpendicular distance from the line lg spanned by γw and γb. Determining pg follows the same principles as pd, and it is based on Eqs. 10, 11 and 12. By projecting both pg and the position of the corresponding weld toe ω onto the line lg, based on Eq. 8, the treatment deviation from the weld toe can be expressed as the difference between the two projected points \(p_{g}^{\prime }\) and \(\omega ^{\prime }\) (see Eq. 14).

$$ \begin{array}{@{}rcl@{}} \epsilon= \|p_{g}^{\prime} - \omega^{\prime} \| \cdot \text{sgn} (p_{g}^{\prime}.x - \omega^{\prime}.x) \end{array} $$
(14)

As the quantitative measures are automatically determined, a documentation report for the treatment is further automatically generated.

6 Experimental setup

The adaptive robotic HFMI treatment is evaluated by being compared to a linear robotic treatment and to a manual treatment. The adaptive HFMI treatment uses the developed system to automatically generate the treatment trajectory, such that the treatment accurately follows the weld toe and varies the working angle ϕ and travelling angle ψ along the trajectory. The linear HFMI treatment was programmed manually; therefore, the HFMI tool follows a linear trajectory based on the weld toe position at the start and end of the weld. For the linear trajectory, the travelling and working angles were kept constant throughout the treatment. All trajectories were programmed so that the robot moves in a continuous forward motion at a constant velocity.

The manual treatment is performed by a trained human operator with varying speed, treatment and working angles according to the operator’s assessment, who simultaneously aims to stay within the recommended limits stated by IIW [14]. Unlike the robot, the human operator moves in a back and forward motion when performing the treatment. It should be noted, that the chosen speed for all treatments was below the IIW recommendation, as it was concluded to be necessary to ensure a proper treatment depth.

The experimental setup for testing the system consists of a KUKA KR60-3 industrial robot used to manipulate both the 3D scanner and the PITEC Weld Line 10-06 HFMI tool. The 3D scanner is a Wenglor MLWL131 laser line scanner and works based on the principles of laser triangulation. The scans are performed along a linear trajectory parallel to the weld with a constant speed, orientation and distance to the workpiece. The equipment and parameters for the experiments are specified in Table 1.

Table 1 Parameters for scanning and HFMI treatment
Full size table

The treatments are performed on a sample of two S355 300 × 1500 mm steel plates, each with a thickness of 20 mm, that have been welded together using gas metal arc welding in a double-sided joint configuration. This creates a 1500-mm weld with two weld toes, each split into 50-mm sections with a spacing of 10 mm. The workpiece and welding equipment are further detailed in Table 2. The welding parameters are given in Table 3 along with the welding sequence, illustrated in Fig. 15. Each treatment type was performed on the two weld toes of six sections, resulting in 300 mm of weld or 600 mm of weld toe per treatment type. As the weld toes were less prominent on the root side of the weld, the treatments were performed on the root side making it more challenging for the developed system to correctly identify and locate the weld toes.

Fig. 15
figure 15

The welding sequence along with the groove geometry. Note that (C) specifies the ceramic backing. Further details are outlined in Table 3

Full size image
Table 2 Workpiece and equipment
Full size table
Table 3 Welding process parameters
Full size table

The experimental setup is illustrated in Figs. 16 and 17.

Fig. 16
figure 16

Scanning of the weld using an industrial robot. This is done before and after the treatment

Full size image
Fig. 17
figure 17

Performing robotised HFMI treatment of the welded sample

Full size image

7 Results and discussion

Application of the developed system results in a significant decrease in programming time for the robot than manually programming the HFMI treatment path. The proposed system only requires programming the robot scanning trajectory, which is identical pre- and post-treatment. Programming can be done through both online and offline programming based on a CAD model of the part. The total programming time per treated weld sample with the developed system was around 2 min, while manually programming the linear trajectory was more than 10 min for both toes of the weld. In cases where there are similar parts, the scanning trajectories for the developed system can be reused and thus the programming time is reduced further. This is not the case for the manually programmed treatment trajectory as no individual weld is unique; thus, the treatment trajectories must be reprogrammed for each weld.

Figure 18 shows an example of a weld, that has been treated using the developed system. It can be observed that the HFMI tool has adapted to the weld toe along the treatment trajectory. Figure 19 shows the resulting quantitative measures for the treated weld, which act as documentation for the treatment quality. It should be noted, that in this case, the indentation depth (base metal side) is not within the recommended limits set by IIW [14]. However, this problem could be solved by performing the treatment at a lower speed.

Fig. 18
figure 18

HFMI treated weld using an automatically generated treatment trajectory. Note how the the robot has adapted to the curvature of the weld using the information from the 3D scan. This acts as empirical validation for the performance of the developed system

Full size image
Fig. 19
figure 19

The quantitative measures plotted along the length of a random weld along with the associated IIW recommendation limits [14]. The IIW recommendation limits for the error 𝜖 are defined based on the position of the HFMI groove, which varies for the right and left weld toe. Twenty-five to 75% of the total groove width must be in the base metal, which is indicated by the upper and lower limits on the lowest graph

Full size image

7.1 Quantitative measurements of the treatments

The quantitative measurements have been evaluated based on scans before and after adaptive robotic, linear robotic and manual post-weld treatment.

ANOVA analysis combined with Tukey’s Honest Significant Difference (HSD) have been performed for each of the five quantitative measures across the three treatment types: adaptive robotic, linear robotic and manual, all scanned before and after treatment. The results show a statistically significant difference in the means across the treatments when comparing the quantitative measures (e.g. indentation depth and groove width). The only case where two groups were not statistically significant is the linear robotic treatment and the manual treatment, when observing the indentation depth (base metal side). Thus, the measured distribution of the indentation depth (base metal side) is comparable for the manual and linear robotic treatment. However, this could be due to the relatively large variance in the indentation depth (base metal side) of the linear treatment, as presented in the box plot, Fig. 20 and Table 4. Based on the ANOVA analysis, it can be concluded that the treatment types generally lead to different results and can, therefore, be further evaluated.

Fig. 20
figure 20

Box plot of the quantitative measures, illustrating the variance and median of each treatment type. Note the smaller variance of Adapt Rob (adaptive robotic) compared to Lin Rob (linear robotic) across the majority of metrics. This is a clear result of the robot successfully adapting to the weld toe. The circular points are statistical outliers

Full size image
Table 4 Results for comparison of adaptive robotic versus linear robotic and manual treatment
Full size table

Figure 20 illustrates a box plot that compares the quantitative measures between the three treatment types. The red line within each box illustrates the median values of the group, while the lower and upper bound of the boxes illustrate, respectively, the first quartile (25th percentile) and the third quartile (75th percentile). The box plot data have automatically been generated using the developed system and is composed of the combined data from all 600 mm of treated weld per treatment type as described in Section 6. Therefore, some of the variances in the measurements can be explained by the natural variance in the welds. The data, that is used to generate Fig. 20, is further summarised in Table 4, while related observations are presented below.

R. Yekta et al. [10] generally observed a significantly lower variance of the manual treatment across all metrics compared to a linear robotic treatment. K. Ghahremani et al. [19] also observed a significant scatter in the measured weld toe radius of the linear robotic treatments compared to the manual treatment. However, in the case of this paper, the standard deviation (square root of variance) of the weld toe radius and groove width of the linear robotic treatment is approximately reduced 50% compared to the manual treatment, as presented in Table 4. This could be explained by the homogeneity and linearity of the treated welds. As the adaptive robotic treatment adapts to the irregularity of the weld toe and adjusts the working and travelling angles relative to the weld profile, the adaptive robotic treatment achieves the lowest variance across almost all metrics except for the indentation depth (base metal and weld side), where the resulting standard deviation is, respectively, 25% and 15.3% higher compared to the manual treatment. Conversely, the absolute difference in the standard deviation of indentation depth (base metal and weld side) is 0.01 mm and 0.02 mm, which is deemed insignificant.

A relatively large mean value and scatter is observed in both the weld toe radius and groove width for the manual treatment. The back-and-forth motion that the operator performs when treating the weld can possibly explain this; hence treating the same area more than once results in an uneven treatment. The vibrations of the tool make it challenging to control, and as a result, the operator might not follow the same trajectory, when moving over a previously treated area.

According to the IIW recommendation for HFMI treatment, G. Marquis and Z. Barsoum [14], the minimum indentation depth (base metal side) should be between 0.1 and 0.2 mm to ensure a complete HFMI treatment. Although, the optimum HFMI treatment indentation depth (base metal side) is between 0.2 and 0.6 mm, while the groove depth should be between 3 and 6 mm. It must be noted that the recommendation stated by G. Marquis and Z. Barsoum [14] serve as guidelines and depend on several factors, including the yield strength of the welded steel. In Fig. 20 it can be observed, that the mean indentation depth (base metal side) for the adaptive robotic treatment of 0.16 mm is within the minimum limits.

As stated by R. Yekta et al. [10], directing the tool at a too high degree towards the weld, opposed to the base metal, equates to a lower indentation depth (base metal side) of the treatment. Reducing the treatment speed could furthermore result in a larger indentation depth (base metal side) of the treatment. The mean groove width of 2.92 mm of the adaptive robotic treatment (Table 4) is marginally below the recommended interval. This could probably be solved by using a larger tool pin tip radius. Factors that influence the measurements, such as the z-axis resolution and pooling of scanning coating in the treated groove, could also affect the results.

From Table 4, it can be observed that the mean error 𝜖 in the position of the groove vs the weld toe for the adaptive robotic, linear robotic and manual treatment are, respectively, 0.09, 0.65 and 0.02 mm. As the mean error for the adaptive robotic treatment is below 0.1 mm with a standard deviation of 0.39 mm, it is considered acceptable. This further underlines, that the adaptive robotic treatment precisely follows the weld toe. Comparing the mean error and standard deviation (Table 4) of the adaptive robotic to that of the linear robotic treatment illustrates that adaptive robotic treatment results in a significant mean error reduction of 86.2% (0.56 mm) and a reduction in the standard deviation of 45.8% (0.33 mm). As expected, the linear robotic trajectory does not follow the irregularity of the weld toe, which naturally results in positional errors. The mean error of the adaptive robotic treatment compared to the manual treatment is 350% (0.07 mm) larger, while the standard deviation is 64.1% (0.25 mm) lower. The larger mean error in the adaptive robotic treatment could result from calibration tolerances and be lowered with an improved calibration routine. However, the decrease in the error variance when comparing adaptive robotic to manual treatment indicates improved process stability. As an overall summary, the comparison of standard deviation across all metrics is averaged; indicating that the adaptive robotic treatment has a 26.6% and 31.9% overall lower standard deviation compared to, respectively, the manual treatment and the linear robotic treatment. In the case of a more consistent weld of improved appearance and quality it is expected that the standard deviation across all metrics of the adaptive robotic treatment would be further reduced.

It should be noted that the error measurements are based on the computed weld toe positions from the developed system and hence on the assumption that the calculated weld toe positions are correct. Furthermore, assuming that a trained operator can treat the weld toe accurately, a mean error of 0.02 mm would indicate that both the weld toes and the treated grooves are correctly located. Based on these assumptions, the performance of the developed system is validated.

In summary, the adaptive robotic treatment showed substantial improvements compared to the manually programmed linear robotic treatment, as both the mean error 𝜖 and the variance across all metrics were improved. The adaptive robotic treatment showed considerable potential compared to the manual treatment as the treatment variance across most metrics was improved. Determining the quantitative parameters using 3D scanning has been proven to be a fast and efficient method compared to manually determining the metrics. A large amount of measurements points (> 1000) is achieved with a single pass of the scanner across the weld (pre- and post-treatment) with minimum human interference. As the measurements are automatically evaluated based on the 3D scan, human bias is further eliminated.

8 Conclusion

An integrated software and hardware system has been developed for performing automated post-weld treatment and post-treatment quality inspection using 3D scanning. A 3D point cloud of the weld was used to compute the gradient and curvature across the weld surface, forming the basis for locating the weld toe. Knowledge of the weld toe position and the surrounding weld surface was used to generate an adaptive robotic treatment trajectory, that accurately follows the curvature of the weld toe and adapts the tool orientation to changes in the weld profile, hence improving process stability. Rescanning the weld post-treatment and further utilising the surface gradient and curvature allowed for extracting the quantitative measures of the treatment: indentation depth (base metal and weld side), groove width and weld toe radius. In addition to the quantitative measures as defined by IIW, G. Marquis and Z. Barsoum [14], the HFMI groove deviation from the located weld toe was further computed to evaluate the accuracy of the system.

The developed system was experimentally tested on welds of S355 mild construction steel. The results were compared to similar treatments performed manually by a human operator and by a robot following a manually programmed linear treatment trajectory. Each treatment type being applied on 600 mm of weld toe. The experimental results were evaluated using statistical metrics and ANOVA analysis in conjunction with Tukey’s HSD based on which the following conclusions can be made:

  • The ANOVA analysis and Tukey’s HSD indicate that the quantitative measurements from the treatment types are generally statistically significant; hence, they result in treatments that statistically differ.

  • The automatically generated treatment trajectory that is adapted to the weld toe resulted in a treatment variance, which was the lowest in the majority of the quantitative metrics. The exception was the indentation depth (base metal and weld side), where the standard deviation was 0.01 and 0.02 mm higher, which is deemed insignificant. The averaged standard deviation across all metrics was, respectively, 26.6% and 31.9% lower than manual and linear robotic treatment. In conclusion, the adaptive robotic treatment improved the process stability.

  • The manual treatment had a relatively large mean and scatter for both the weld toe radius and groove width. This could be attributed to the operator’s back and forth motion during treatment, resulting in uneven treatment and lower process stability.

  • The mean groove width for the adaptive robotic treatment was 2.92 mm, which is marginally below the 3 mm recommendation by IIW [14]. The groove width could likely be increased by using a larger tool pin tip radius.

  • The trajectory deviation (error) from the weld toe was respectively 0.09 mm, 0.65 mm and 0.02 mm for the adaptive robotic, linear robotic and manual treatment. As the operator is trained, the small error in the manual treatment is used as validation for the developed system’s ability to evaluate the quantitative measurements accurately. Additionally, it can be concluded that the adaptive trajectory follows the weld toe as the error is significantly smaller than that of the linear treatment trajectory.

Furthermore, it can be concluded that automatically generating the treatment trajectories based on the developed system is far more efficient than manual programming. The developed system only requires manual programming of the linear scanning program, which is the same for all comparable welds. The system performs as intended and shows an excellent potential for performing post-weld treatment and quality insurance with minimal human involvement.

For future work, it could be interesting to evaluate the quantitative measurements further to identify tool wear and weld defects.