In situ monitoring of build height during powder-based laser metal deposition

A powder-based laser metal deposition (LMD) system can fabricate customised three-dimensional (3D) parts, layer by layer, based upon a computer-aided design (CAD) model. However, the deposition will not always feature the expected geometry due to excessive heat input and inconsistent powder flow. Due to the layer-by-layer nature of LMD, geometrical error in one layer is compounded in all following layers and may result in a build failure. Thus, it is critical to monitor online the track and layer height. This study developed an in situ monitoring system integrating a webcam and a narrow bandpass filter. The laser/powder defocus distance was extracted from the melt pool images, and the track/layer height was calculated from the laser/powder defocusing distance and preprogrammed layer spacing. The presented approach does not need additional illumination sources and is a nonintrusive online method. Therefore, it is a potential precursor to a feedback build height control system. It also can be used for measuring omnidirectional height, i.e. height in different build directions relative to the substrate, which has been tested by fabricating two thin-wall structures with customised shapes. These online-measured height data were successfully validated against dimensional measurements from an offline 3D scanner, thus demonstrating the online system’s potential utility in a feedback control system for ensuring acceptable part geometrical accuracy.


Introduction
LMD is a metal-based additive manufacturing method [1] and is particularly competitive in achieving a high mass deposition rate and building flexibility. As a result, it is widely applied to the fabrication of customised 3D parts and part repair in the aviation, automotive, and biomedical sectors [2]. Metal powder or wire is fed to a region of the substrate surface (i.e. the melt pool) being melted by a moving laser beam. The feed material is melted within the melt pool and resolidified after the laser leaves the area. The laser path is carefully preprogrammed to generate deposition tracks in a track-by-track and layer-by-layer manner to deliver the desired component geometry [3].
Despite its ability to manufacture components with complex 3D geometry, powder-based LMD-manufactured builds still face quality issues, particularly nonuniform build geometry (particularly build height), even for a constant preprogrammed laser path and (other) process parameters. Three main factors affect the delivery of correct build geometry, namely.
1. Even for constant-valued process parameters, fluctuation still occurs in, e.g. powder feed rate, due to variation in incoming powder flow rate, resulting in uneven build height. The laser beam and powder jet become mutually defocused at convex (overly large build height) and concave positions (overly small build height). These build height errors propagate to subsequent layers and can significantly distort parts. 2. An uneven substrate surface will also affect overall build height, typically not compensated for during laser path planning. 3. During trajectory changes, the laser's acceleration and deceleration affect the amount of powder caught by the melt pool, affecting the overall build height [4,5].
This study focuses mainly on the online measurement of build height via in situ monitoring of the laser/powder defocusing distance to develop a precursor for a feedback control system for build height for LMD printers. In situ monitoring of an uneven substrate surface is briefly discussed but is not the current focus, as is neither laser acceleration nor deceleration.
The effect of the laser/powder defocusing distance on thin-wall structures was studied for single-track builds and reported elsewhere [5]. The largest single-track height obtained in this prior study occurred when the powder jet's focal plane was aligned to the melt pool and decreased when the powder focus moved above or below. However, the study concludes that the thin-wall build achieved better geometrical accuracy when the powder focal plane was slightly below the substrate plane. The preprogrammed layer spacing ( Δz ) in the laser's toolpath program also affects the laser/powder defocusing distance. The relationship between laser/powder defocusing distance and cladding layer height was reported in [1] and concluded that the layer spacing ( Δz ) should be kept in a suitable range of 0.4-0.7 times the cladding layer height for obtaining good part quality. However, even with such static optimisations to Δz [6][7][8], as long as the LMD system is running in an open-loop manner, the fabrication process is still not robust to disturbances (e.g. fluctuation of powder feed rate due to online variation in powder size), which therefore still leads to dimensional inaccuracy [9].
Before designing a feedback controller for layer height for the LMD process, it is necessary to acquire an in situ measured signal that reflects the build height accurately. Several articles have reported such techniques, e.g. via a 3D scanner or a LDS mounted off-axially to the laserpowder feed system [4,[9][10][11]. These systems are typically precise to micron-scale and scan the build after each layer is deposited to obtain a complete layer height profile. Thus, these methods introduce an interlayer pause, periodically interrupting and slowing the fabrication process. Moreover, interruptions can result in a coarser build surface and even cracks [12]. Triangulation methods for height monitoring have also been proposed by integrating camera(s) off-axially to the laser-powder feed system [13][14][15][16][17] or coaxially [18,19]. Compared to the scan-based method, off-axially integrated cameras have the advantage of not disturbing the laser by height measurement and (thus) real-time monitoring of build height. However, camera-based height measurement methods usually require high-quality images that require additional illumination and limited installation locations.
This study proposes a low-cost solution for in situ monitoring of build height, based on off-axially mounting a webcam and a narrow bandpass filter; see Fig. 1. Melt pool images are acquired from the webcam and processed using an adaptive thresholding method to estimate the location of the melt pool's centre. The laser/powder defocusing distance can be extracted from the melt pool position. The build's height can, in turn, be calculated based on the preprogrammed Δz and the extracted laser/powder defocusing distance. This calculated build height is compared to offline height profiles obtained from the finished build's 3D scanned point cloud data. The error between calculated height and measured height was found to lie within a reasonable range compared to the existing literature [14]. The results presented here thus prove the feasibility of implementing an off-axial camera to monitor build height online during continuous LMD. This information can then be used within a control system for delivering accurate build height during LMD.
The advantages of the presented in situ monitoring system are as follows: (1) The presented in situ monitoring system is simple, as it consists of one only camera, one narrow bandpass filter, and one cover slide. There are no other cameras (i.e. as for the trinocular-based method [13,17]) or illumination/probe sources needed, i.e. as in [14,18,19]. (2) As the camera was off-axially mounted on the laser head, it moved with the laser head during deposition. The presented approach can thus be used without stopping the process, e.g. unlike scanner-based monitoring systems [10,11,20]. 3) The presented approach has been tested by measuring the overall height of customised-shape samples, which reveals that it can successfully measure omnidirectional heights, i.e. in build directions non-perpendicular to the substrate. In contrast, most publication reviewed here tested their methods solely using single-track thin wall fabrication [9,14].

Experimental setup
The LMD system consisted of a 4 kW diode laser generator (LDF 4000-100, Laserline GmbH, Mülheim-Kärlich, Germany), a 0.6 mm diameter optical transmission fibre attached to the laser generator that worked in continuous wavelength mode (980-1030 nm), a 6-axis robotic arm (IRB2400, ABB Ltd, Zürich, Switzerland), a laser head attached to the end of the robotic arm, and a powder feeder system (Metco Twin 10-C, Sulzer Ltd, Winterthur, Switzerland). The laser head was equipped with a collimating lens (72 mm focal length) and focusing lens (300 mm focal length) to form a 2.5 mm diameter laser beam with a 'top hat' (or rectangular function) intensity distribution.
The laser beam and powder jets were set to focus at the same horizontal plane-the definition of the laser/powder defocusing distance ( d defocus ) is presented in Fig. 2. A consumer range webcam [21] (LifeCam Cinema, Microsoft Corporation, Redmond, Washington, USA) was mounted upon the laser head in an off-axial position, ~70° to the vertical direction and 150 mm horizontally away from the laser head ( Fig. 2). A low-cost narrow bandpass filter [22] (20BPF10-900, MKS Instruments, Inc., Andover, Massachusetts, USA) was placed in front of the webcam to monitor the melt pool by permitting only a narrow spectral band of near-infrared (central wavelength 900 nm, full-width half max 10 ± 2 nm).
A disposable transparent cover slide was mounted in front of the bandpass filter for protection purposes.
The optimum angle and distance between the camera and laser head were determined empirically. While their exact values are not critical, several guiding principles must be considered during selection. The angle cannot be too small (e.g. 0°) as the acquired images would show the melt pool in plan view, thus obscuring measurements of d defocus . The angle cannot be too large (i.e. near 90°) as the camera's view melt pool may be blocked by the height variations elsewhere along the track, particularly when used for omnidirectional height measurement, i.e. for build directions non-perpendicular to the substrate. The distance between the camera and laser head should be neither too small (heat conduction damages the camera) nor too large (decreased image resolution).

Materials
Commercial stainless steel (SS) 316L powder (SS316L-5320, Höganäs AB, Höganäs, Sweden) was used for all builds described in this study. The as-received particle size was in the range of 45-150 µm. Figure 3a presents a scanning electron microscope (SEM) image of the particles and their particle size distribution. The particles were heterogeneous, with their modal size bin at 71-106 μm as measured from SEM images (Fig. 3b). Regarding the substrate, rectangular mild steel plates (150 × 100 × 10 mm 3 ) were selected for all builds.

Camera calibration
As the camera was mounted on the laser head, it moved simultaneously. Thus, the melt pool should appear stationary within the acquired images, provided no laser/powder defocus. Once defocusing occurs, the melt pool changes in a vertical position relative to the camera, manifesting as a change in its apparent size within the image. This effect permitted a calibration experiment to be conducted to determine the effective image resolution of the camera for height measurement. The laser was deliberately moved The laser was switched on without powder flow throughout this movement, and the evolution of melt pool size with laser head distance was recorded. Specifically, the melt pool was detected and extracted from each image using the Otsu adaptive thresholding method [23] included within the OpenCV-Python v4.5.3.36 library (henceforth abbreviated to 'OpenCV'). The melt pool's boundary was then extracted and fitted with an ellipse. The relative y-coordinate of the ellipse centre was then extracted from the processed images and scaled to lie within [−10, +10] mm. The results are highly linear (Fig. 5) and permit the camera's effective image resolution ( d res ) to be calculated as 0.42 mm/pixel (2 significant figures-SF).

Image processing and height calculation
As the LMD process builds parts in a layer-by-layer manner, the user may naively expect the deposition height to match the preprogrammed layer spacing/thickness, thus obtaining a part geometry consistent with the simple virtual slicing of its designed geometry. However, due to the high-velocity fluid dynamics of the melt pool and unavoidable residual fluctuations in controlled process parameters, it is difficult to achieve a perfectly consistent deposited geometry.
When depositing a new layer atop its preceding layer, the laser beam and powder jet defocus at convex positions (i.e. relatively high positions) of the previous layer and similarly at concave positions (i.e. relatively low positions) of the previous layer. Within a limited range of d defocus , relatively small convex and concave regions of the preceding layer are compensated during the deposition of successive layers by the self-healing or self-regulation effect [5,16,24]. Thus, minor instances of defocusing may not lead to the part fabrication failure, but they still affect the part geometry. The deposition height monitoring system proposed here takes advantage of this behaviour, i.e. instead of monitoring deposition height directly, the laser/powder defocusing distance is observed instead, with the build height h(t) calculated via Eq. (1), where d defocus represents the laser/powder defocusing distance, t represents time, n represents layer number, and Δz represents the preprogramed layer spacing: Fig. 3 a SEM image of SS316L powder used for all builds described in this study and b corresponding particle size distribution, which is adapted from the vendor's datasheet and very similar to that obtained from the SEM images (not included for brevity) Fig. 4 Schematic of laser head movement during the camera resolution calibration experiment Two challenges must be addressed to extract robustly y centre from the melt pool images when processing the melt pool images of thin-wall structures. Firstly, as the webcam does not have zoom functionality, the melt pool occupies only a small proportion of its images, and the ellipse fitting will sometimes fail to identify y centre ; see Fig. 6a. Secondly, when printing thin-wall structures, the melt pool tail (Fig. 6b) increases in length and width as heat accumulates with successive layers. These challenges affect the accuracy of the algorithm for locating y centre as well as the calculation of d defocus if using just a singly run thresholding method to extract the melt pool boundary.
This study presents an adaptive thresholding method to address both the above challenges, thus improving the robustness of the image processing algorithm for identifying the y-coordinate (in pixels) of the centre of the melt pool's fitted ellipse ( y centre ); see Fig. 7. When a raw image is supplied, the region of interest (ROI) is cropped from the raw image. This ROI is zoomed to 5 × larger while maintaining apparent resolution using bicubic interpolation [25]. This zooming and interpolation prevent the failure of the automated melt pool boundary detection and ellipse fitting code by refining the apparent image resolution to 0.084 mm/pixel. This refined image is then binarized using an initial threshold suggested by the Otsu adaptive thresholding method within OpenCV for the analysed image set. Each binarized image is fitted with an ellipse outline via a preexisting OpenCV function [26]. If the major axis a of the ellipse is shorter than an empirical value ' l ', y n centre (t) was stored. If a ≥ l , the minimum threshold limit is increased by unity and the thresholding, binarization, and ellipse-fitting steps are rerun. This process is rerun until a meets the constraint above. The overall process described in Fig. 7 thus eliminates the undesirable inclusion of the melt pool tail within the melt pool boundary, and therefore the associated error during the determination of the y-coordinate (in pixels) of the centre of the melt pool's fitted ellipse ( y centre ).

Results
A triangle-shaped and an arrow-shaped thin-wall structure were built in continuous mode to test the in situ build height monitoring method presented here. The off-axial camera recorded the fabrication process during both builds. The process parameters for both builds are listed in Table 1. The parameter values were determined empirically from the authors' lab database of single-track builds using SS316L powder.    [27] function to artificially equate the lengths of h cal and h scan . The completed build's heights for positions A, B, and C were marked as dark blue crosses and, while near-symmetric around the isosceles triangle's symmetry plane, exceeded the mid-side heights. This result was due to the deceleration and acceleration of the laser's robotic arm as it approaches and leaves the corner positions, respectively, which results in greater heat build-up, a larger melt pool, greater powder capture, and a higher build-up rate. The error percentage ' ' was calculated using Eq. (3) and presented in Fig. 8e, where h is the expected value of the built height and ĥ is the built height calculated after the image processing steps described in Sect.   Table 2. These include the mean absolute error (MAE), maximum absolute error (MXAE), mean absolute percent error (MAPE), and maximum absolute percent error (MXAPE); see Eqs. (4)- (7), where m is the length of the variable arrays.

Case study-triangle-shaped thin wall build
The MAE and MXAE between h cal and h ref were 4.49% of MAPE and 14.84% of MXAPE, respectively. These relatively high absolute errors appear due to the aforementioned deceleration and acceleration of the laser's robotic arm at the build's corners, e.g. the peaks in in Fig. 8e near A and B. This may be related to errors in y centre -determination during image processing; when the laser beam passes the corners, especially for the last layer, the uneven surface at the corner may block the view of the melt pool. Thus, the camera would see only the portion of the tail up to the highest point of the wall, i.e. the preceding build corner. This would result in an abnormal y centre value that affects the accuracy of the h cal values calculated using Eq. (1). The MAE between h cal and h scan was 1.13 mm, which is below the MAE between h cal and h ref , and the MXAE between h cal and h scan was 2.62 mm. The MAPE and MXAPE between h cal and h scan are 3.17 and 7.12 < 10%, respectively, which are both acceptably low [28] and even slightly closer to as-built values than the results presented by [14].
The y centre vs. historical time profile extracted from the melt pool images is presented in Fig. 9, with the initial and final  . 9 y-coordinate of the melt pool centre ( y centre ) as extracted from the melt pool images in a triangle-shaped thin wall build. The initial and final 100 s of the fabrication process were magnified for clarity periods of the 100 s build magnified for clarity. The y centre of each edge can be distinguished as they have similar patterns that are repeated in the zoomed-in plots. The major peaks and valleys represent the corners and mid-sides of the triangle, respectively. The y centre of each edge in the initial depositing period is quasi-linear, which represents the height of each edge as linearly changing. The y centre for each edge in the final depositing period appears to 'vibrate', which represents the height profile of each edge as oscillating. Interestingly, for the first build layer, instead of y centre lying within a horizontal plane, y centre varied in vertical position quasi-linearly with laser displacement for each edge of the triangle. This is likely because the substrate plate's surface is not flat (see Appendix for evidence).

Case study-arrow-shaped thin wall build
An arrow-shaped thin wall build (Fig. 10a) was fabricated to test the height monitoring method's feasibility. Its plan-view dimensions are ~100 × 80 mm 2 . The scan path in Fig. 10c was built by setting the laser robot's tool path to follow the scan path clockwise through positions A-B-C-D-E-F-G-A. The y centre vs. time profile extracted from the resulting melt pool images are presented in Fig. 11, with the initial and final periods of the build (duration 100 s) magnified for clarity. In Fig. 10d, h scan was obtained using a 3D scanner (Sect. 3.1), h cal was obtained from y centre 's features as extracted from the height monitoring system presented here, and h ref was obtained using the preprogrammed Δz and layer number. The data analysis for h scan , h cal , and h ref follows the same logic as described for the triangle-shaped wall in Sect. 3.1 and Fig. 8d, e. h cal and h scan show good agreement, whereas h cal and h ref show relatively high differences, mainly at the corners of the build. The relative differences ( ) between h cal and the other profiles are presented in Fig. 10e, and the resulting error measures values are presented in Table 3.  and h scan is 0.71 mm, which is relatively small and demonstrates the feasibility of the height monitoring method presented here. The MXAE and MXAPE are 3.2 mm and 8.44%, respectively, which are also reasonably low. As for the triangle-shaped wall build (Table 2), the relatively high MXAEs and MXAPEs appear mainly at the corners, where the build is also higher than the mid-sides, as seen for the triangle build (Fig. 8d). Also noteworthy is that the arrow build is more asymmetric around its symmetry plane that the triangle build. The reasons for the higher corners than the mid-sides are the same as the triangle build due to the acceleration/deceleration of the robot and the accumulated heat. The high MXAE/MXAPE values at the corners can be caused by the quality of the melt pool image and the data interpolation method used in the present study, which causes a small relative shift between the peaks of the h cal and h scan profiles presented in Fig. 10d. The cause of the asymmetric height profile of the arrow build around its single vertical symmetry plane is believed to be the uneven surface of the substrate (note that the triangle and arrow builds were fabricated on different plates). The cause of this unevenness is likely due to residual stresses, i.e. from when the substrate was cut into blocks from its originating steel sheet. Evidence for this uneven surface is presented in the Appendix.

Conclusions and future work
This study demonstrates the design and implementation of a low-cost, in situ monitoring system for measuring the built height online during the LMD process. The low-cost system comprised an off-axial webcam and bandpass filter (< USD$1000). The built height was calculated based on the laser/powder defocusing distance and preprogrammed layer spacing. The former was obtained from the differential y-coordinates of the centre of the ellipse-fitted melt pool boundary. An adaptive thresholding method was presented to improve robustness that reduced the effects of the melt pool tail, which increases in length as heat accumulates with successive layers and thus would otherwise reduce the positioning accuracy of the melt pool centre.
The proposed height monitoring method was tested by fabricating two SS316L thin-wall builds (triangle shaped and arrow shaped). Both builds were 3D scanned offline using a portable 3D scanner to obtain their real overall height profiles. The calculated overall height based on the in situ monitoring system was compared with the 3D scanned overall height and the design height. The relative difference between each pair of profiles was calculated and summarised using various error measures. Key results and conclusions are as follows: • The built height can be acceptably precisely calculated based on the preprogrammed layer spacing and the differential y-coordinates of the centre of the ellipse-fitted melt pool, which was extracted from the presented in situ monitoring system. The system also proves the occurrence of laser/powder defocusing. • The presented in situ monitoring system is highly sensitive to both height deviations in the build and the uneven surface of the substrate (see Appendix). • The relatively high errors between the calculated height and design height usually appear at the build corners due to the deceleration of the laser and the resulting increase in heat input. • The relatively high errors between the calculated height and the 3D scanned height usually appear at the corners. The cause relates to the method of melt pool extraction and the error introduced when interpolating the image data to a higher resolution. • The MAEs between the calculated height and 3D scanned height for both builds are ~1 mm, with MAPEs < 5%. The MXAPEs between the calculated height and 3D scanned height for both builds are < 10%, comparable to the performance of state-of-the-art methods presented in the literature [14], but at a lower cost.
Despite the presented work providing reasonably accurate build height calculation, there are still several unaddressed aspects. (1) Image processing can be improved to extract a more accurate melt pool boundary location and thus further improve the accuracy of the calculated build height. (2) A cooling system would improve the camera's robustness over long build times. (3) The height monitoring system can monitor the built height in real-time non-intrusively. Thus, its output can be used as the measured signal to drive a feedback process control system to control build height for LMD systems, e.g. by adjusting the powder feed rate.
head was stopped to measure the standoff distance between the nozzle end and substrate surface (~2-min process). Standoff distance was then manually measured using gaugewidth leaves (Mitutoyo Corporation, Takatsuki, Kawasaki, Kanagawa, Japan) to ±0.25 mm.
Data analysis requires a reference height, for which at point 'A' was selected. Figure 12b presents the comparison between flatness calculated from the off-axial camera video and direct measurements using the gauge leaves. Presented  alongside is the resulting (small) error in flatness. These results demonstrate the resolution and robustness of the height monitoring system presented in this study. The results also show that B, C, and D lie below A, with C being the lowest point (~1.2 mm below A). Considering the size of the substrate plate (150 × 100 × 10 mm 3 ), ~1 mm deviation in flatness is considered here to be acceptably small for the current purposes.