Introduction

Within the last few years, there have been several steps towards further industrialization of the laser powder bed fusion (LPBF) process, which includes process qualifications,1,2,3 production line integration and new machine concepts, such as the EOS P500, for higher output at overall better quality and integrated process control and monitoring.

However, most systems lack the capabilities for process monitoring, even though it has started very early within the development of the technology. The first monitoring systems have been used to obtain better insights of the process and understanding the underlying principles. In 1994, a video microscopy was integrated into a machine for in situ investigations.4

Early, part-warpage detection is the objective of a monitoring set-up with a fringed light projector. The focus of this project is given by the investigation of curling based on different build temperatures and laser parameter settings as well as the material formation within the melt pool.5,6

Gardner at al. assembled an optical coherence tomography system within the laser path. The project demonstrated the possibilities of this technique for the detection of subsurface defects. An embedded temperature monitoring, even of the powder around the parts, is given as outlook.7,8

Today, thermography process monitoring is the most common monitoring technology for laser sintering. As the freshly applied powder has to be pre-heated via infrared radiators, a closed-loop temperature control is mandatory for every machine. In newer machines, thermography cameras are used because they have a high resolution of measurement points, allow in-depth information on individual part temperature histories, and open the door for modified laser control. In research, thermography has given insights into a better understanding of the process itself.9,10,11,12,13,14,15,16,17 Tyloret al. even predicted the break position of selective laser sintering (SLS) specimens, by identifying colder spots within one or several consecutive layers. It was possible to correlate thermography results to the fracture position with a precision of 45–80%. However, the correlation to the elongation at break as well as reduced ultimate strength is only possible with a high standard deviation.13

Other researchers have implemented a low-cost imaging process monitoring system. However, they aimed to control the energy input by gray scales on the image. Therefore, they have the same or a similar approach to the thermography image sensors.18 In March 2021, a machine-learning system for identifying powder spread defects was published. The system is installed on a S2 laser sintering system from Sintratec , and based on a workaround, as the 4K-in-house-camera seems not to be accessible. With a webcam, the life image of the Sintratec camera has been monitored and then analyzed. The Sintratec S2 is working with carbon black PA12 powder, which increases the contrast of molten und unmolten areas. The specific Sintratec S2 camera set-up and light situation within the build chamber is not described.19

Within this report, a retrofittable powder spread monitoring system is presented, which is capable of detecting powderbed imperfections even with low contrast materials. An automatic check for powder spread flaws, considering the intersection with any parts in the respective layer, allows a detailed quality report and early warning for the operator. Furthermore, it should be possible with this system to implement an additional monitoring on the recoater filling level and to use it for the no-curl-temperature search.

System Development

The hardware design comprises the camera assembly, the light assembly, and the trigger assembly, which includes a Z-axis trigger to identify the job start after the preheating phase and a trigger attached to the recoater in order to obtain a signal for each layer. The individual component assemblies and their position within the EOS P3 system are illustrated in Fig. 1. The key element is the camera assembly with an IDS GV-5890CP-M-GL C-mount camera (CMOS Mono 12MP sensor with the size of 1/1.7 and the ratio of 4:3) and a modified Kowa lens – LM5JC10M with f 1.8–16 and a fixed focal length of 5 mm. The lens is head-first positioned in the build chamber, replacing a halogen light, and is therefore only separated by a 2-mm window from an environmental temperature of up to 200°C. Hence, a proper cooling system is required. Preliminary tests have shown that a simple liquid cooling ring or block is not sufficient for temperature regulation of the lens, as the first glass of the lens exceeded the max. temperature limit within the first 30 min. of preheating. To overcome this issue, a selective laser melting camera housing with integrated cooling jackets and air circulation has been designed.

Fig. 1
figure 1

Overview of the LC-ProMo hardware assemblies and its location on a stripped-down EOS P3 machine.

Full size image

An overview of the camera assembly is given in Fig. 2. The camera window is shielded by a nitrogen gas flow. This is realized by several metal sheets, working as spacer, clamps, or guideways. A PTFE block is used to isolate the camera housing and to gain some more space for the lens, which must be in an exact position in order to have a total view of the build platform. Nine screws, including spring-loaded washers and nuts, attach the build chamber window assembly to the camera housing. Those washers secure a pressure regulation even if the materials exhibit thermal expansion.

Fig. 2
figure 2

(a) Cross-section view of the double-walled camera housing with coolant and air flow. (b) Detailed view of the N2 protection flow for the window fluidization within the build chamber.

Full size image

The light is positioned within the machine hub and in front of the build chamber. The light furthermore comes perpendicular to the recoater moving direction, which enlarges the shadows cast of powder failures created by agglomerates, snow balls, or coating attachments.

An overview of the information flow inside the monitoring system is given in Fig. 3. The LC_ProMo_Job application is running on a Raspberry Pi and controls all signals and the camera. The most important signals are the process information, here shown as trigger signals, and the capture image signal. If the Z-trigger is activated, the preheating phase is over and the recoater trigger is set to listening, hence the monitoring starts. As not only the freshly applied layer but also the recoater filling level must be monitored, the trigger has to be activated twice. Once, if the recoater is directly below the camera and a view on the inside is possible, and once for the platform powder spread analysis as soon as the recoating process is finished.

Fig. 3
figure 3

Information flow of the camera-based monitoring system.

Full size image

The images are sent to the PC over the Ethernet, including the current layer index given by the image name. On the PC, the EOS-Formats application generates a binary mask of the layer indicating the part position and exposure areas by using the .sli files from the build job. The image is then transformed using a perspective correction to gain a perpendicular bird’s-eye view, and a high pass filter for working out the fine details in the powder is applied. This image is afterwards sent to the detection module which contains the Mask Region-Based Convolutional Neural Network (Mask R-CNN). The detection result is super-positioned with the sli-based exposure mask to highlight the imperfections within the build parts. If one of the failures exceeds a severity alarm threshold, this layer is highlighted and an alert is given. Each failure, including the severity score and position, is furthermore logged in a job report file.

Detection Software

Detecting and localizing faults in the images translates to the task of object detection/instance segmentation in machine learning. For the project, a pre-trained Mask R-CNN20 engine, which is based on the Faster R-CNN21 framework, is used. The network has been trained on the COCO dataset,22,23 and is further refined for the new purpose. To this end, the pre-trained head of the network for bounding box, mask, and class prediction is replaced with a new one, but keeps the feature extraction backbone, as used in the architecture of the PyTorch tutorial.24

The annotation process has been carried out using the SuperAnnotate platform.25 A custom dataset adapter was developed which acts as an interface between the SuperAnnotate given .json format and the Mask R-CNN used COCO format, so that the images and annotations can be read by the network.

For the training, over 2500 images from more than 13 different build jobs (A–M) have been marked precisely with over 5500 imperfections and 6500 attributes. Each fault is defined by a specific location, shape (area, line, point) and severity (low, medium, high, critical). The annotations have been created by multiple people. Since, currently, it is not defined how the severities can be distinguished precisely and explicitly, the annotations have a subjective pillar. For more consistency, a quality check was made for every individual image by just one person.

The results of the first 1000 images have shown an over-sensitivity in the detection network; therefore, the low imperfection markers have been eliminated from the annotations. This greatly increased the annotation speed as over 50% of all annotations had been of low severity and could be observed in nearly every layer. Figure 4 shows the statistics in more detail. Relating the number of annotations to the total number of layers (2544), 46.6% of all layers show some line defects, of which most are classified as medium, with presumably no influence on the part properties. However, there are 4.8% with a critical severity, while 26.0% of all layers show point imperfections and 10% of the annotated features are area imperfections. In 5.4% of the annotations, a foreign object is seen in the powder spread. In all, 52.3 % of the layers are perfect, as long as the low imperfections are not considered. As more than one imperfection can be positioned within one layer, the sum of the percentual values is higher than 100%.

Fig. 4
figure 4

Number of annotations for line, point, and area failures with different severities related to all detections.

Full size image

The detection/segmentation problem is modeled as a two-class problem, with the classes failure and background. In the training process, the images and their annotations are randomly flipped horizontally to artificially create more diverse training data. The common way to measure the accuracy of an object detection system is the average precision (AP) and average recall (AR) based on correct detections of true positives (TP), wrong detections of false positives (FP), undetected objects, i.e., false negatives (FN) and true negatives (TN), for the background using the intersection over union (IoU) on the bounding boxes, i.e., masks. An object counts as a TP if the IoU is larger than a given threshold, and as FP otherwise. The IoU is defined as the ratio of the intersection of the bounding boxes and their union, as shown in Eq. (1).

(1)
$$ {\text{Precision}} = \frac{{{\text{TP}}}}{{{\text{TP}} + {\text{FP}}}} = \frac{{{\text{TP}}}}{{\text{all detections}}} $$
(2)
$$ {\text{Recall}} = \frac{{{\text{TP}}}}{{{\text{TP}} + {\text{FN}}}} = \frac{{{\text{TP}}}}{{\text{all objects}}} $$
(3)
$$ F1 = 2 \frac{AP*AR}{{AP + AR}} $$
(4)

Since the precision and recall can differ depending on the given threshold, the IoU is usually averaged over different thresholds, e.g., from 0.50 to 0.90. The AP and AR can then be calculated for each threshold of IoU by Eqs. II and III. Combining all the AP results in the mean AP and mean AR, if only 1 of 10 objects is detected by the network, the precision might be very good. However, in this case, the recall would be very low, with only 10%. By detecting more objects, the precision might decrease, as more detections do not match the given bounding boxes with the given threshold of the IoU. Hence, both values should be analyzed together.

Some layers have more than one entry that overlap with the real object. In such cases, the first entry is considered as TP, the others as FP. This rule is used with the PASCAL VOC 2012 metric:26,27 e.g. five detections (TP) of an item are counted as one correct detection and four as false alarms.28

For the interpretation of the mean average precision and recall, the relationships are important. Even the human eye will not always distinguish between a low severity powder bed imperfection and a failure, which has to be marked as medium. Some of those FPs might be failures with the severity low, as there is an anomaly visible, but the influence on the part is assumed to be neglectable. Most important is the FN score for the imperfections with high severity. Hence, a manual evaluation of the network is required.

To obtain an unbiased estimation of the accuracy, the data have been divided into training data, validation data, and testing data in a 7-fold cross-validation split. The training data are used to train the network and to adjust its weights, while the validation data are used for early stopping when the F1 score (Eq. (IV)) did not improve for 10 epochs. The F1 score is calculated on a maximum of 100 detections and averaged over IoU thresholds from 0.50 to 0.95 and all area types [@AP and AR (IoU = 0.50:0.95 | area=all | maxDets = 100)]. To avoid data leakage, the split is performed among the build jobs and not within. To analyze the influence of the preprocessing, the training has been performed once with the original dataset, hence without high pass and tonal value corrections. Those tests show that the engine itself is capable of detecting the failures even without the preprocessing. The human eye often has problems detecting contrasts in those image settings. However, the results without the preprocessing is ranked at the lower end of the scale compared to other training scenarios, which are shown in Fig. 5. Here, within one dataset, the larger bubble shows the test result, whereby the smaller datapoint represents the last epoch evaluation during the training.

Fig. 5
figure 5

Mean average precision and recall for all seven trained networks (with different cross-validation splits for training, testing, and final evaluation).

Full size image

The test splits show a wide range in the mean average precision (mAP) and recall (mAR). Most splits had better testing results than within the last epoch evaluation; however, sets V, VI, and VII show an opposite behavior. The training sets are very different in the part structure. Some test sets include thin walls oriented in the recoater direction, which might be detected as failure, if the engine was not trained to differentiate those parts from line defects. Hence, a comparable validation with a uniform test set is required. As double detections are punished with down-scoring the precision, this validation is made manually for a real build job, and is not based on the PASCAL VOC 2012 metric.

First, typical and atypical failure images are fed into the engines. Here, the individual scores for the different severities are measured and used for defining the alarm threshold. Then, those engines are applied to the manual test images. All failures need to be detected with as few false alerts as possible. Especially, the critical failures should not be missed. For the medium failures, the alarm threshold could be adapted further with the risk to miss out some of those detections. Overall, 16 layer images are fed to all seven networks. In total, 6 critical flaws, 7 high flaws, and 4 medium flaws should be detected by the networks. Those elements consist of 1 foreign object, 8 line flaws, and 1 massive area flaw, where the individual failures can not be distinguished. Net_I, _IV, _V, and _VII show no misses within the detection, but the false alerts range from 16.8% (Net_V) down to 1,2% (Net_VII). The Net_II, _III, and _VI have missed one or two medium flaws in this test. Hence, Net_VII is recommended for the LC-ProMo system, as it shows the overall best performance in the manual test. The manually checked precision and recall is the best in this test, whereby the PASCAL VOC 2012 is also good.

Imperfections versus Failures

Next, analyses have been carried out to differentiate between different failure intensities and their influences on the part properties. Eleven build jobs with 15 tensile test specimens flat on the platform (YXZ) have been built. To reduce the influence of inhomogeneous cooling times, the overall job is only 6.12 mm high. The specimens have been spread to increase the probability of having powder spread failures within the parts.

All the jobs have been built on the P395 with the layout presented in Fig. 6. Within those jobs, no failure has been intentionally applied. They have only been provoked by not cleaning the recoater after every job. However, all imperfections are random. The part properties profile for those jobs is the standard EOS default job PA2200_120_110_Balance (layer thickness 120 µm). The build job temperature is set to 178.5°C and the recoating velocity is 120 mm/s. With a 6-mm preheating base layer, 4-mm part free space and 3-mm cover layer, the overall job is just 19.12 mm high and has a build duration of approximately 20 min.

Fig. 6
figure 6

(a) Build job layout with DIN EN ISO 527 tensile test specimens type 1BA, with enlarged shoulders (total specimen is 100 mm long), all specimens start at 4 mm. (b) Monitoring image with critical point flaw in build job 7 on specimen 5.

Full size image

In total, there have been 81 imperfections with contact to 36 parts marked by the annotator. Those include 3 critical detections on two parts, 27 medium markers on 18 parts, and 57 with the severity attribute low affecting 26 parts. Most of the detections have been line imperfections, which are very often observed for several layers on the same position. One of the critical detections is shown in Fig. 6b. It is a random critical point failure. This failure is even visible on the next layer and also within the final part. It is expected that powder attached to the outer side or the top of the recoater is responsible for point defects in general. Most probably, this powder falls off from time to time, and results in this huge powder pile (Fig.6b), which is now visibly interfering with test specimen 5 from job-7-L. Overall, 11 jobs have been built to obtain a sufficient data background for the analysis.

When comparing the fracture position with the recoating-failure position, it is not possible to identify a direct correlation between powder spread imperfections and part fracture. In Fig. 7, the specimens with a failure intersection have been marked depending on the flaw severity: critical (red), medium (yellow) and low (blue). Those specimens are set to the X-axis of the diagram, defined by the job name and the part specimen number, which had the contact to the powder spread imperfection. For specimen 1 in build job 5, a very good correlation is visible. In job 7 on specimen 5, this correlation is not directly visible. The medium or low severity detections show a large variation, so it is very hard to indicate a trend. To further check the correlations, the elongation at break and the tensile strength have been analyzed.

Fig. 7
figure 7

Correlation of fracture distance and powder spread imperfection. critical (red), medium (yellow), or low (blue) (Color figure online).

Full size image

A box-plot diagram of the tensile strength per specimen position over all 11 build jobs is presented in Fig. 8a. The cross marks the average value, the first as well as the third quartile, and the median is given by the box and the line within the box, respectively. The whiskers indicating the range of the data without outlier datapoints. Outliers are marked in yellow, whereby measurements were considered to be outliers if the value differs from the respective quartile by more than 1.5 times the interquartile distance. The specimens with a critical powder spread detection are marked in red. The tensile strength of those specimens shows no significant deviation, as the variation in the measurements is already very high. Even for this shallow build job, the typical strength reduction towards the corners (specimens 1 and 15) is visible, which points towards inhomogeneous powder bed temperatures and laser power exposure. The same systematic for plotting the data is applied for the elongation at break in Fig. 8b. The reduced elongation at break for the corner position is also visible. Even if there are some outliers, specimens with the critical detections are within the normal measurement variation. All the outliers are positioned at the upper end of the testing range. Having a view of the specimens which show a very high elongation at break is a common observation in LS.29

Fig. 8
figure 8

Boxplots of 11 build jobs for the (a) tensile strength over all 15 specimen positions and (b) elongation at break over all 15 specimen positions (Color figure online).

Full size image

For a better understanding of the delamination, a CT scan of specimen 5 from job 7 was made. The defect itself is already 5 mm long and goes through the full part. The defect is positioned in the middle of the 2-mm-high part. Within the delamination, a shift of the layers towards the recoater movement direction is visible. It is expected that the defect has moved the molten mass a little, so the layers show a small displacement. In Fig. 9a, a part cross-section on top of the defect is shown, indicating that there is already some standard porosity. In Fig. 9b, all standard pores are blacked out, so only the delamination is shown. The red line indicates the fracture position, while the green line shows the measurement position for the defect on the part. The distance between those lines is approximately 3.5 mm. Considering the measurement failures and part shrinkage, as well as the scaling factors, the limits for the correlation should not be set to be too narrow. The part failure is even visible at the outside of the part, as shown in the fringed light surface topology measurement in Fig. 9c.

Fig. 9
figure 9

(a) Standard porosity of a no-defect layer. (b) Large, complex pores within the part job 7-L5 (large defect). c) Surface topology, side view of the specimen (Color figure online).

Full size image

Considering the measurement displacement and the large powder spread defect, the significant deviation between the fracture position and the powder spread defect position can be explained. The tensile strength and the elongation at break are also low, but no significant dependency can be derived from the data sets so far available. In build job 6, a specimen with an elongation at break of 0.03% points less than the defect part value was measured.

The fracture correlation of specimen 1 out of build job 5 was very good, but the tensile strength and elongation at break do not show any trend, as the properties are above or average.

Summary and Conclusion

A new monitoring system for automatic powder spread analysis and quality assurance has been developed, which can be retrofitted to existing EOS P39X-systems at an affordable price. The system can be installed and used fully autonomously, and can detect a wide variety of powder spread flaws.

In total, seven different powder spread imperfection detection networks have been trained and validated on a selection of layer images and one real build job. Here, the engine VII has shown the best overall performance and is therefore used for the system. In the tests so far, all failures have been detected and only three layers have been false positive. This performance of the system allows a reliable quality control of the build jobs, and will gain a better understanding of the powder spreading process itself in the future.

The influence of powder spread imperfections on part properties was preliminarily investigated. As only two specimens suffered from high or critical powder spread flaws, the dataset is not sufficient for reliable forecasts and predictions. For both specimens, the fracture position was close to the recoating failure. The tensile strength and the elongation at break do not show a significant deviation. Even a trend to lower properties is only shown in one of the specimens, where the imperfection is already visible on the surface of the part. Based on the recoating failure type, the affected part might delaminate, which has a more significant influence on the mechanical properties in z-direction than within the build plane. Thus, investigating the influence on different build directions is another requirement for further investigations.

Those results support the findings of Tylor,13 as she was able to predict the fracture position but it was hard to see a correlation to the mechanical properties. Thus, the remaining question is: what would be the elongation at break and tensile strength of these specimen without the defects?

In conclusion, the database is not large enough for secured assessments of the powder spread influence on the part properties. More specimens should be measured and analyzed. Using provoked and artificially introduced powder spread imperfections might lead to a better picture of the recoating process and its influence on the final part.

Furthermore, preliminary tests have shown the potential of the system for a recoater filling level detection, resulting in an early warning for the operator to adapt the powder fluidization, the dosing values, or refill the powder vessels. For a large job, this might save a lot of money. Additionally, preliminary investigations have shown that the system can detect part curling, which allows an objective solution for the no-curl-temperature search. Overall, the presented process monitoring system is the next step towards a consistent quality concept for laser sintering part manufacturing.