Introduction

The increasing availability of data from the first three paradigms of science (experiments, theory, and simulations), along with advances in artificial intelligence and machine learning (AI/ML) techniques has offered unprecedented opportunities for data-driven science and discovery, which is the fourth paradigm of science (Agrawal & Choudhary, 2016). Within the field of AI/ML, deep learning (DL) (LeCun et al., 2015) has emerged as a transformative technology in the last few years, and has been extensively used in materials science (Agrawal & Choudhary, 2019; Ge et al., 2020; Gupta et al., 2021, 2022; Jha et al., 2018, 2019, 2021, 2022; Li et al., 2019; Mater & Coote, 2019; Yang et al., 2018, 2019, 2020; Ziletti et al., 2018) to enhance materials property prediction, discovery, and design, as well as in additive manufacturing (AM) (Francis & Bian, 2019; Ghungrad et al., 2021; Imani et al., 2019; Li et al., 2020; Mozaffar et al., 2018; Qin et al., 2020; Scime & Beuth, 2018; Xie et al., 2021; Zhang et al., 2019). Laser Powder Bed Fusion (L-PBF) is a popular AM technique, widely researched in both academia and industries ranging from aerospace to healthcare. However, pores in an AM part lead to stress concentration contributing to the formation and propagation of cracks that are detrimental to mechanical properties, such as tensile strength and fatigue life (Sabzi et al., 2020). Porosity formation mechanism is still difficult to describe, although it is popularly investigated in not only one but also hybrid additive manufacturing process (Webster et al., 2021).

In-situ process monitoring of the melt-pool, laser track, or build layer during the AM process is critical in detecting potential porosity and controlling part quality. Commonly measured signatures are thermal (Bakish, 1983; Nath & Mahadevan, 2021) or acoustic features (Eschner et al., 2020), and optical images (Bugatti & Colosimo, 2021; Li et al., 2021; McCann et al., 2021) before or after powder recoating or powder fusion (Grasso & Colosimo, 2017). However, In-situ process monitoring, and porosity detection in current literature (Paulson et al., 2020; Scime et al., 2020; Zhang et al., 2020) can only detect porosity that has already occurred but cannot predict potential porosity which might occur in the future, a much-desired ability to guide real-time process parameter optimization for better part quality. One possible direction is to understand the process–structure–property relationships by exploring the effect of different processing parameters (i.e., laser power, scanning speed, hatch spacing, etc.) on the microstructure, mechanical properties, or porosity of the L-PBFed parts (Gawade et al., 2022; Liu & Zhang, 2021; Nath & Mahadevan, 2021; Tang et al., 2017). However, simulation methods require prior knowledge and experience from expertise when designing parameters according to different materials, set-ups, geometries, etc.

Given the limitations of the previous studies mentioned above and the idea that one can adjust laser process parameters in the next layer if the potential of porosity formation can be predicted, it is critical to build a porosity prediction system that can predict potential porosity occurrence. At the very least, if such a predictive model existed, the current build could be stopped before unallowable part defects occur using in-situ monitored signals to avoid more serious part porosity and save on machine time.

Inspired by advanced applications of deep learning methods in material science and additive manufacturing, a deep learning framework is proposed in this paper to predict potential porosity formation using in-situ monitored signatures. To the best of our knowledge, this is the first work that attempts porosity prediction before it happens based on in-situ thermal signatures using a deep learning approach. The proposed detection-prediction framework is composed of two modules, a detection and a prediction module. The detection module detects porosity based on thermal signatures while the prediction module predicts porosity in the next layer according to previous layers. Each module can be applied separately or in unison for different purposes.

The related works are reviewed in section “Related works”. The methods used in this work are introduced in section “Methods”, including the description of the dataset in section “Dataset preparation” and the proposed framework in section “Deep learning methods”. The experiments and results are presented in section “Results and discussion”. Finally, conclusions of this work and future work are discussed in section “Conclusions and future work”.

Related works

Simulation methods

Tang et al. (2017) proposed a geometry-based simulation method to predict the volume fraction of the unmelted material (i.e., lack of fusion porosity) in parts built by L-PBF by calculating the overlap between melt pools. It was assumed that melt pools are in the shape of dual half-ellipses on the cross-section along with the scanning and building directions. The inputs into the simulation model were melt-pool cross-sectional area, hatch spacing, and layer thickness. Melt-pool areas were estimated with the analytical Rosenthal equation in which material properties at room temperature need to be carefully collated. However, this simulation model does not incorporate any environmental variables (i.e., environmental temperature, humidity, airflow, etc.) that might lead to a discrepancy in the melt-pool sizes. Nath and Mahadevan (2021) updated the physics-based model at each layer using in situ monitoring data for porosity prediction at future layers. However, lots of prior knowledge is required to build the simulation model, such as heat transfer mechanism in different materials.

Artificial intelligence methods

Deep learning has become the AI/ML technique of choice in the last few years, due to its groundbreaking success in numerous real-world applications (Aghazadeh & Gharehchopogh, 2018; Asghari et al., 2021; Collobert & Weston, 2008; Khataei Maragheh et al., 2022; Krizhevsky et al., 2017; Lin et al., 2019; Niu et al., 2019). Four machine learning methods were adopted by Paulson et al. (2020) to model the relationships between thermal histories and surface porosity formation in Ti6Al4V during single line scans using L-PBF. Each run was equally separated into eight segments based on time and each run had eight data points corresponding to the eight-time segments. For each data point, the input is the thermal history (black-body radiance measured using infrared imaging) at the middle of each time segment and the output is the most serious porosity formation situation of any time segment during the whole run. In the study by Liu and Zhang (2021), physical parameters (i.e., laser energy density and laser radiation pressure) were first extracted from the original machine setting parameters (i.e., laser powder, laser angle, etc.) and used as inputs for six machine learning methods to predict maximum, mean, and median pore diameters as well as median pore spacing. The physics-based model was able to explain pore generation mechanisms and was validated on an L-PBF machine (i.e., Concept Laser M2 machine) with a porosity prediction error of 10–26%. Other significant physical effects influencing pore formation were not considered in their models, such as material energy absorption and heat transfer between material and air.

In the study by McGowan et al. (2022), experimental and simulated laser metal deposition data are applied to train a convolutional neural network (CNN)-based model with physics-informed loss functions for porosity prediction. The results show that the physics-informed model works better than the model without any prior physics knowledge. A CNN-based model named ResNet18, proposed by Zhang et al. (2020), achieved an accuracy higher than 99 % in lack of fusion and keyhole defect detection in L-PBFed single Ti6Al4V beads. Each sample was scanned for 32 s with a width of 200 mm. The inputs to the image classification model were 2D spectral images transformed from audio signals. In the study by Scime et al. (2020), in-situ optical images of the build plate and powder bed at each layer were captured and labeled pixel by pixel. Their results demonstrated that machine learning-based models (i.e., Dynamic Segmentation CNN, etc.) achieved real-time detection and classification of 12 types of defects (such as porosity, super-elevation, soot, etc.) in six different PBF machines. However, for the porosity detection problem, the model was validated only in one PBF machine with a true positive rate of only 78.4%. Although the study by Zhang et al. (2019) presented that their proposed CNN models achieved local volume porosity prediction, their models actually detected pore size using in-process cross-section images or tomography scans after the pores happened instead of predicting them ahead.

Methods

Dataset preparation

The proposed porosity detection-prediction framework will be used to examine the lack of fusion porosity. To obtain the dataset, a part named lack of fusion (LOF) with only lack of fusion porosity was manufactured using an L-PBF machine EOS M290 and Ti6Al4V powder with a size distribution of 23–50 µm. The part had 594 layers with a layer thickness of 30 µm, including 501 part layers, from the 85th to the 585th layer, while the layers below are for building the part supports and the layers above are for part annotations. The part, shown in Fig. 1c, was built in its axial direction as denoted by the red arrow. As shown in Fig. 1a and b, the imparted porosity regions are designed to be spherical, and their diameters decrease from 1.10 to 0.10 mm along the building direction. On a given building level, six porosity regions with the same diameter were designed and distributed in a hexagonal pattern. Fig. 1b shows the top view of parts with the designed six anomalies with a diameter of 1.10 mm. Layer ranges of each designed porosity region are listed in Table 2. For example, a porosity region with a diameter of 1.10 mm is designed to range from the 133th to the 170th layer. When the laser comes to the designed porous regions, less energy is applied to achieve a lack of fusion porosity as shown in Table 1. Optimal processing parameters obtained in preliminary experiments were applied to build the non-defective regions (Table 2).

Fig. 1
figure 1

a Front view; b Top view; c Part geometry (the building direction denoted by the red arrow); d STL file obtained by CT indicating lack of fusion anomalies

Full size image

The built part was tested using computed tomography (CT) and a Standard Tessellation Language (STL) file was created that indicates lack of fusion anomalies in three dimensions, as shown in Fig. 1d. The CT scan data was subsequently transformed into layer-wise labeled images indicating the location of the porosity in each layer of the part LOF. Each pixel in a labeled image has a value. As for a certain pixel in one part layer, if porosity appears at that pixel’s location according to the CT scan data, the label value of the pixel is set to 1, otherwise 0. These labeled images constitute the porosity ground truth for the proposed porosity detection-prediction framework. The supervised learning algorithms are applied in this study.

During the manufacturing process, six thermal features of the melt pool scanned by the moving laser were in-situ monitored by a Co-Axial Planck Thermometry system provided by Sigma Labs Inc. Planck Thermometry is based on Planck’s law. The six thermal features provided by the system are Thermal Energy Planck (TEP), TEP_sigma, Thermal Energy Density (TED), TED_sigma, counts, and TEP_sums, which will be explained as follows. TEP represents the temperature of the melt pool by taking the ratio of two radiation values at two wavelengths. TED is the integrated radiation taken in the whole near-infrared region. Thus, TEP is representative of the temperature of the energy deposited region while TED is representative of the input process parameters and material response. Counts indicate measurement times for each deposited region. The TEP_sums value is the sum of all TEP measurements of the corresponding deposited region. Thermal signatures of all part points are measured right after laser scanning and stitched as images, so that there are six images at each part layer. In other words, each pixel in each layer has six values that correspond to six thermal signatures.

All in all, each layer has six images which represent the six thermal signatures (i.e., TEP, TEP sigma, TED, TED sigma, counts, and TEP sums) and one image that represents the porosity labels. The size of all the images is 160 pixels \(\times \) 160 pixels with one pixel in the imaging system representing 100 µm in the part system. Fig. 2a–f shows six thermal signature images in colormaps and Fig. 2g shows the labeled image of the 185th layer. The values of these six thermal features vary from 10 to \(10^5\), thus, the value of each feature is normalized to 0–1 using the mean and standard deviation correspondingly, which is also applied in the study by Sorkhabi et al. (2020). All images of part LOF built using the EOS M290 machine are collected as a dataset named EOS-LOF (EOS denotes the machine name and LOF denotes the porosity type) with a dimension of 501 layers \(\times \) 7 images \(\times \) 160 pixels \(\times \) 160 pixels.

Table 1 Manufacturing process parameters
Full size table

Sigma Labs Inc. has claimed that the six thermal features described above can indicate four types of defects, including gas porosity, lack of fusion, keyhole, and tungsten inclusion (Betts & Anderson, 2020). In their statement, the average value of all pixels in the window (i.e., 5 pixels \(\times \) 5 pixels) centered at a certain pixel is considered as the feature value of that pixel. All six feature values at the current layer and the previous layer (total of 12 values) were input into ExtraTree Regressor models to predict the probability of each pixel being defective. Each model was trained only for detecting one type of defect. The probability is in the range from 0.0 to 1.0. But in their work, no threshold was applied to its result to actually classify each pixel into porous or non-porous categories, which is done in our study by identifying an appropriate threshold. Secondly, their model can only identify already existing defects rather than predicting potential defects in the future. Thirdly, their way of dividing the dataset into training and testing sets is different (randomly at pixel-level) than the way used in our study (temporally at layer-level, which is a more realistic and stringent evaluation of the predictive capability of the model).

Deep learning methods

A detection-prediction framework based on deep learning is proposed in this paper, which aimed at predicting porosity in the next layer based on the thermal signatures of previous layers is shown in Fig. 3. The framework includes two modules, a detection and a prediction module to be described in detail in section “Detection module” and “Prediction module”, respectively. The inputs are the thermal signatures of layers \(\{l_{i-k}, ..., l_{i-1}\}\), the outputs are the porosity labels in layer \(l_{i}\). k indicates the number of layers in history whose information is used for the integrated framework to predict the next layer’s porosity occurrence. H indicates the number of layers in history whose information is used for the porosity detection of one layer. j presents the number of layers whose porosity labeled images are used to predict the next layer’s porosity labels. Here, \(k=H+j\).

Table 2 Porosity distribution along the building direction in the designed part
Full size table

Detection module

Recently, convolutional neural networks (CNN) (LeCun et al., 1998) have led to great successes in a broad range of image-based applications, e.g., image segmentation (Schulz & Behnke, 2012), image classification (Sermanet & LeCun, 2011) and face recognition (Adjabi et al., 2020). For this module, we develop a CNN-based method to detect porosity according to thermal signatures in a pixel-wise fashion. The challenges of the porosity detection problem are limited and imbalanced data. Especially, for this dataset, some layers lack porosity. Thus, it is challenging to use semantic segmentation methods on this dataset. We consider the porosity detection problem as a regression problem, and take each pixel as a data point to predict the probability of this pixel being porous using CNN-based methods. If the probability of a pixel is larger than a threshold, the pixel is classified as porous.

A CNN model is usually comprised of three basic components: convolution layers, pooling layers, and fully connected layers. These components can be repeated multiple times in a CNN model. The convolution layer forms the core layer of CNN models, and its goal is to objectively extract important features from the input images. This is accomplished using a set of convolution kernels (Krizhevsky et al., 2012) in each convolution layer. 2-D kernels are usually used for image classification.

Fig. 2
figure 2

Example images of six features and the ground truth at the 185th layer in colormaps: a counts; b TED; c TED_sigma; d TEP; e TEP_sigma; f TEP_sums; g ground truth labels (the porosity regions are in light yellow and the part region is in light blue)

Full size image
Fig. 3
figure 3

The architecture of the porosity detection-prediction framework. To predict the porosity labels in layer \(l_{i}\) , the thermal signatures of layers \(\{l_{i-k} , ... , l_{i-1}\}\) are input to the detection module to obtain the detected porosity labels of layers \(\{l_{i-j}, ... , l_{i-1}\}\) which are the inputs into the prediction module to predict the porosity labels in layer \(l_{i}\). The dimensions of input and output are \(160 \times 160 \times k \) and \(160 \times 160\), respectively. The thermal signatures of layers \(\{l_{i-k}, ... , l_{i-j}\}\) are the input to the detection module to obtain the detected porosity labels of layer \(l_{i-j}\), and the thermal signatures of layers \(\{l_{i-(j-k+1)}, ... , l_{i-1}\}\) are the input to the detection module to obtain the detected porosity labels of layer \(l_{i-1}\)

Full size image

Considering not only the six thermal signatures of the target pixel, but also the information of surrounding neighboring pixels and the history of these pixels might be important to achieve good detection accuracy. The information of surrounding neighboring pixels and the history of these pixels in previous layers are composed of C channels of 2-D images for each data point as the input of a CNN model. For every target pixel in layer \(l_i\), we impose a \(D \times D\) square region in this layer, where the central pixel is the target pixel and other pixels are the surrounding neighbors of the target pixel. Each pixel has six thermal signatures, so we obtain six \(D \times D\) images in layer \(l_i\). The square regions at the same position in H previous layers include the historical information of these pixels. Similarly, we obtain six \(D \times D\) images for each previous layer. Hence, \(C=6 \times (H+1)\), and the input for each data point is \(C \times D \times D\) images. Figure 4 shows an example input, where \(D=5\). The output is the probability of being porous at the current layer at the location corresponding to the target pixel.

Fig. 4
figure 4

The input and output of the proposed CNN model

Full size image

A pooling layer is usually employed after one or several stacked convolution layers. The purpose of the pooling layer is to reduce the dimensionality of feature maps, which are the outputs of the convolution layers. The outputs obtained after employing stacks of convolution and pooling layers are flattened to a vector. This vector is then fed into a fully connected layer. The CNN architecture used in this work is shown in Fig. 5, and is designed by stacking several convolutional layers and pooling layers, and one fully connected layer at the end. In the convolutional layer, the kernel size is \(5\times 5\), \(stride=1\), \(padding=2\). In the pooling layer, the kernel size is \(5\times 5\), \(stride=2\). The first and second convolutional layers have 16 and 32 kernels, respectively. ReLU is used as the activation function for all convolutional (Conv) layers and the sigmoid function is used after the fully connected (FC) layer.

Fig. 5
figure 5

The architecture of the proposed CNN model. Conv denotes a convolutional layer, FC denotes a fully connected layer

Full size image

Prediction module

The objective of the prediction module is to predict porosity labels in the next layer according to the porosity labels of previous layers. Because both the current and history information about surrounding neighbors can impact the pixel-wise porosity in future layers, one needs to consider the correlation between local neighbors when forecasting. In other words, both spatial and temporal context information needs to be considered for prediction. Therefore, this problem can be formulated as a spatiotemporal sequence forecasting problem with the sequence of porosity labeled images in previous layers as input and the porosity labeled image in the next layer as output. When predicting the porosity labels in layer \(l_i\), the labeled images of previous layers \(\{l_{i-j}, ..., l_{i-1}\}\) are used to directly predict the labeled image in layer \(l_i\). The size of all the images is 160 \(\times \) 160.

In the study by Shi et al. (2015), a convolutional LSTM (ConvLSTM) network was proposed for a spatiotemporal sequence forecasting problem. In ConvLSTM, the inputs, cell outputs and states are all 3-D tensors whose last two dimensions are spatial dimensions (rows and columns). ConvLSTM includes two structures, the convolutional and the LSTM structure. The convolutional structures are in both the input-to-state and state-to-state transitions, that are capable of capturing local spatial features (e.g., correlation between neighbors). The LSTM can capture temporal autocorrelation in the data. Therefore, the ConvLSTM network has desirable properties for spatiotemporal sequence forecasting problems.

For the present problem, we develop an encoder–decoder model to predict porosity labeled images. Figure 6 shows the architecture of our model. The encoder–decoder model has two networks, an encoder and a decoder network. The encoder network compresses the whole input sequence into a hidden state tensor, where three ConvLSTM layers are stacked with 32 hidden states each. The decoder network unfolds this hidden state by three ConvLSTM layers with 32 hidden states and concatenates all the states. This is followed by one 3-D CNN layer to produce the final prediction. All the input-to-state and state-to-state kernels are of size 3 \(\times \) 3. After doing some hyper-parameter tuning, we use these hyper-parameters in our experiments. The model simultaneously reconstructs the input sequence images and predicts the future image. Both the input and output of the model are a sequence of images. The input consists of porosity labeled images in layers \(\{l_{i-j}, ..., l_{i-1}\}\). The output data consists of porosity labeled images in layers \(\{l_{i-j+1}, ..., l_{i}\}\). Here j represents the number of history layers that are used to predict layer \(l_{i}\). The model is thus expected to predict the porosity labeled image of layer \(l_{i}\) according to porosity labeled images of previous j layers.

Fig. 6
figure 6

The architecture of the proposed ConvLSTM encoder–decoder model. The dimensions of both input and output are \(160 \times 160 \times j\)

Full size image

Results and discussion

We split the dataset EOS-LOF into training and testing sets based on layer number in the ratio of 7:3 for all evaluations of different models. The training set is composed of the first 350 layers (85–434th) while the testing set is composed of the remaining 151 layers (435–585th), as shown in Table 3. Each pixel is taken as a data point. Several metrics to evaluate the effectiveness of our framework will be introduced in section “Evaluation metrics”. In section “Experiments”, the detection and the prediction module are first individually evaluated followed by the combined porosity detection-prediction framework on the testing set. We conducted all modeling experiments using PyTorch with an NVIDIA Tesla V100 PCIe 16 GB GPU. The code is available at https://github.com/MaoYuwei/AM-porosity-prediction.

Table 3 The division of dataset EOS-LOF
Full size table

Evaluation metrics

In this work, we use precision, recall, and the F1-score for the minority class, i.e., porous class to evaluate the models. Based on the confusion matrix, model results are divided into four categories: true positive (TP), false positive (FP), false negative (FN), and true negative (TN). As for the porous class, true positives are those porous part pixels that are correctly predicted as porous. False positives are those non-porous part pixels that are wrongly predicted as porous. False negatives are those porous part pixels that are predicted wrongly as non-porous. True negatives are those non-porous part pixels that are correctly predicted as non-porous.

Precision is calculated as \(TP/(TP+FP)\), while recall is calculated as \(TP/(TP+FN)\). F1-score is the harmonic mean of precision and recall, calculated as \( 2 \times precision \times recall/(precision+recall)\). The Recall is the proportion of actual porous part pixels that are correctly predicted porous, and precision is the proportion of predicted porous part pixels that are actually porous. From the perspective of engineering, both recall and precision are equally important, so the F1-score is utilized to comprehensively evaluate the prediction performance of the proposed method using a single metric, which by definition is good only when both precision and recall are good.

Experiments

Detection module

In this study, we choose a region \(15 \times 15\) in size, which means \(D=15\). The mean square error loss function is used with an Adam optimizer (Kingma & Ba, 2014). The learning rate is 0.00001. The batch size is 10,000. The number of epochs is 300.

The training set is further randomly divided into a training set and a validation set in the ratio of 7:3 for the detection module. The validation set is used for exploring the parameter H and the threshold. For each pixel, if the predicted probability is higher than the threshold, the pixel is predicted as porous.

We explore the history of up to three previous layers for detection, i.e., \(H =\, \)0, 1, 2, or 3, respectively. The receiver operating characteristics area under the curve (ROC AUC) and the precision recall area under the curve (PR AUC) can be used to evaluate classification performance without considering the threshold, which is used to select the best H values. The ROC AUC and PR AUC for the porous class are shown in Table 4, showing that utilizing information about the layers’ history facilitates better performance. The ROC AUC using different H values are similar and PR AUC is the best when \(H=1\), suggesting that using information from just one previous layer is sufficient for porosity detection. Therefore, we only use one previous layer, i.e., \(H=1\), in the later experiments.

Table 4 Results of of the detection module using different number of history layers on the validation set
Full size table

One of the main challenges of this detection problem is that only a small number of pixels are porous. The ratio of the number of non-porous pixels to that of porous ones in the dataset EOS-LOF is about 200:1, as shown in Table 3. Thus, porosity detection and prediction is an imbalanced data problem and the imbalance ratio (200:1) is very high. In binary classification problems, the probability threshold is generally chosen as 0.5. But a smaller threshold may be better for this imbalanced data problem. We thus explore a set of different thresholds on the validation set, as shown in Table 5. All the precision, recall, and F1-score are close to 1.00 for the non-porous class, so we do not show them on this table and the following tables. The threshold is set as 0.4 in the following experiments, because the results for the porous class are the best as shown in Table 5. In real applications, the threshold can be adjusted in accordance with the given tasks (depending on whether recall or precision is more important). When it is more desirable not to miss porous pixels, the recall is more important and a smaller threshold can be used. When the prediction of porous pixels is required to be with high confidence, the precision is more important and a larger threshold can be used.

Table 5 Results of the detection module with different probability thresholds for porous class on the validation set
Full size table

The proposed CNN model is compared to several machine learning (ML) algorithms well-known for classification problems, such as the Extremely Randomized Trees [or Extra Trees (ET)] (Geurts et al., 2006), Logistic Regression (LR), and the K-Nearest Neighbor algorithm (KNN) (Cover & Hart, 1967). In this study, all ML models are implemented within the Python scikit-learn package. The ML models are used here with hyper-parameter tuning using an extensive grid search for each of these models as shown in Appendix Table 9. The training set and testing set of ML models are the same as the proposed CNN model. And all models are using the same dimension of features, i.e., six thermal signatures of neighbors at the current and previous layers. So the input dimension for each data point is \(15\times 15\times 2\times 6 = 2700\), the output is 0 (non-porous) or 1 (porous).

Table 6 compares the results of these models for porous class. One can see that the proposed CNN model is better than the other ML models in detecting where the porosity occurs with higher precision, recall, and F1-score. The proposed CNN model can not only learn the information of surrounding neighboring pixels but also the relative position information of these pixels.

Table 6 Results of different porosity detection models for porous class on the testing set
Full size table

Prediction module

To assess the predictive ability of the proposed framework, j in Fig. 3 is set to 5, which means that porosity labeled images of the previous five layers are used to predict the next layer. A mean square error loss function is used with the Adam optimizer and a learning rate of 0.001. The batch size is 8. The number of epochs is 300. To evaluate the performance of this module in a stand-alone manner (i.e., without coupling it with the detection module), the raw testing set is used where the actual porosity labeled images are used as input to predict the output images with each pixel assigned the predicted probability of being porous, which are subsequently compared with the actual porosity labeled images for evaluation. We again use 0.4 as the probability threshold to convert the predicted probabilities into porous/non-porous predictions.

Table 7 Results of prediction module for different models on the testing set
Full size table
Table 8 Result of detection-prediction framework on the testing set
Full size table

Some ML models are used as baselines, which take each pixel as a data point and the label values of the previous five layers are features. Therefore, the dimension of features for a data point is five. The output is 0 (non-porous) or 1 (porous). A CNN model is also used as a baseline using labeled images of the previous five layers as the input to predict the next layer. The dimensions of input and output are \(160 \times 160 \times 5\) and \(160 \times 160\), respectively. The other parameters remain the same as for the ConvLSTM model. Table 7 shows the results of the proposed ConvLSTM model and some baselines. One can see that the results of the ConvLSTM model are better than other models, which indicates that the proposed ConvLSTM encoder–decoder model can accurately predict porosity labeled images in the next layer based on the actual porosity labeled images of previous layers.

Detection-prediction framework

The two detection and prediction models proposed in sections “Detection module” and “Prediction module” are combined into an integrated framework to predict porosity labeled images in the next layer according to only the thermal signatures of previous layers, i.e., without knowing the actual porosity labeled images in previous layers.

During a real manufacturing process, thermal signatures can be immediately obtained after processing each layer and used to label the images using the above-described detection model according to their thermal signatures. After obtaining and labeling the images of the previous layers, the prediction module can be used to predict the porosity labeled image of the next layer, but current porosity detection methods only detect the existing porosity of manufactured layers. The predicted porosity labels of the next layer can thus be used to decide whether or not to continue the manufacturing process depending on the severity of porosity. If the manufacturing process is continued, thermal signatures of the new layers can be collected for predicting the porosity labels of the next layer, and this can be continued until the end of the manufacturing process.

To evaluate the whole framework the process was simulated to the testing set. For each layer \(l_{i}\), the input of the integrated framework is the set of thermal signatures of the previous \(\{l_{i-k}, ..., l_{i-1}\}\) layers and the output is the predicted porosity label of layer \(l_{i}\) as shown in Fig. 3. The ground truth is the actual porosity labeled image. Here \(H =1\) as mentioned in section “Detection module”, which means one previous layer’s information is used in the detection module. Here \(j=5\) as mentioned in section “Prediction module”, which means five previous layers are used to predict the next layer in the prediction module. Thus, \(k=H+j\), i.e., k=6. Specifically, to predict porosity labels in layer \(l_{i}\), the set of thermal signatures of previous layers \(\{l_{i-6}, ..., l_{i-1}\}\) is input to the trained detection model, outputting a set of detected porosity labeled images of \(\{l_{i-5}, ..., l_{i-1}\}\) layers which becomes the input to the trained prediction model.

Some ML models were trained as baselines, which use six thermal signatures of the neighbors and the previous six layers as features, i.e., the feature dimension is \(15\times 15\times 6\times 6=8100\). Table 8 shows the results of the integrated detection-prediction framework and the ML models on the testing set. The results of the proposed framework are significantly better than those of the ML models, which shows that the proposed approach can predict porosity with reasonable accuracy, which may be useful to help prevent more severe part porosity.

Fig. 7
figure 7

Example prediction results on the testing set and corresponding groundtruth. The true or predicted porosity regions are in yellow and the part region is in light blue

Full size image

The layer range (i.e., from the 444th to the 461th layer) of the porosity region with a diameter of 0.50 mm are taken as examples for detailed results analysis and common conclusion. The same phenomenon is also seen in other layer ranges (i.e., 339–363th, etc.). Figure 7 shows some example predictions and the corresponding ground truth. More pairs of prediction and ground truth are provided in Appendix Fig. 8. The 445th and 459th layers are the beginning and the end layers in the analyzed layer range in the manufactured part LOF. While the 452th layer is in the middle of the layer range.

First, it can be observed that the proposed deep learning framework has a remarkable prediction accuracy during the period of porosity formation. As shown in the middle row of Fig. 7c and d, at the 452th layer, the results in the middle of the layer range are quite accurate. Second, we see that the proposed deep learning framework might struggle in making accurate predictions when the porosity formation starts or ends. As shown in Fig. 7a and b, at the 445th layer, there are six regions of lack of fusion porosity distributed hexagonally, but only one is predicted. Similar results are at the beginning of layer range 444–461th, several porous pixels are predicted as not porous. As shown in Fig. 7e and f, at the 459th layer, the performance is better in predicting porosity than at the beginning of the layer range (tiny porosity is predicted) but still worse than that in the middle layers.

We believe that the proposed method performs nicely during the porosity formation, because the porosity in the current layer and the previous layers is similar and the porosity is relatively obvious. But at the beginning or end of the porosity formation, large change and tiny size make the prediction much harder.

To analyze the practical application, the trained models can be generalized to predict porosity for new prints from the k + 1 layer (since the input of the proposed framework is k layers’ thermal signatures) to the last layer. Note that in order to make the first prediction from the trained model on a new print, we will need the first k layers manufactured (e.g., k = 6 in this work), but that is much smaller than the number of layers used in training (e.g., 350 in this work). Further, if more layers happen to be available for new prints, the trained model on old prints could be used as pre-trained models which can be finetuned using thermal signatures collected while building other prints.

Conclusions and future work

Current works can only detect already existing porosity based on in-situ monitored signals. To solve this problem, a detection-prediction framework is proposed based on deep learning methods for porosity prediction using in-situ monitored thermal signatures. The approach predicts lack of fusion in the next layer based on the thermal signatures of the previous layers. The precision, recall, and F1-score of the integrated framework for the porous class are 0.72, 0.78 and 0.75. The framework includes a detection and a prediction module, each of which can be applied separately for different purposes. For instance, the detection module can be adopted to localize the lack of fusion porosity at each layer of parts built via L-PBF. The prediction module can also be built upon to predict the thermal signature, melt pool geometries, properties, etc.

The main limitation of the proposed method is its limited generalization ability. Since the system learns from the dataset manufactured in the EOS M290 machine, it is unclear how it would perform for other L-PBF machines, like the DMG-MORI machines, etc. This work is validated in predicting lack of fusion porosity, but its performance is not clear for other types of porosity, like gas porosity. Moreover, the performance is also unclear for other types of materials, like In718. And predicting porosity in parts with other unintentional and complex geometries, like lattice structure, is much more challenging. It is unclear how the proposed framework will work in more complex situations. The second limitation is that the proposed method might struggle in predicting tiny porosity and when the porosity formation starts or ends. Size limitation is due to limited dimension accuracy of Co-Axial Planck Thermometry system (each pixel represents 100 µm) and the prediction accuracy of the proposed model (F1-score of 0.75).

The most significant step in the future is to build more parts of different geometries (e.g., cube, globe, etc.) with undesigned and tiny porosity using different L-PBF machines (e.g., DMG-MORI, EOS machine, etc) and collect thermal features during the manufacturing process for finetuning the proposed modules. We could further improve the proposed method based on the new larger dataset and improve generalizability and customize loss functions/models for the beginning and end layers. We could also further analyze the misclassification pixels of both the detection module and prediction module during processing.