Abstract
Selective laser melting is the most commonly used additive manufacturing technique for fabricating metal components. However, the SLMed part quality still largely suffered from the porosity defects that can significantly affect the mechanical properties. Recently, in situ monitoring based on machine learning has been recognized as an effective method to overcome this challenge. In this work, a deep learning method is developed for in situ part quality inspection. The layer-wise visual images are used as the inputs without manual feature extraction and a deep transfer learning (DTL) model combining deep convolutional neural network and transfer learning is creatively applied. First, an off-axial in situ monitoring system by a high-resolution digital camera is developed to capture the images of each deposited layer. Then, samples with different part quality levels are produced by varying process parameters. Thereafter, based on the porosity measurement results obtained by optical microscopy, each captured visual image is labeled. An image dataset associated with a label of three categories of poor, medium, and high quality is created. Finally, the proposed DTL is employed to perform the classification tasks, aiming to identify the part quality based on the layer-wise visual images. Results show that a 99.89% classification accuracy of the developed DTL was obtained, revealing the feasibility and effectiveness of using layer-wise visual images without manual feature extraction to realize quality inspection. Overall, the proposed DTL method provides a promising solution to monitor part quality and reduce porosity defects during the printing process.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Alfieri, V., Argenio, P., Caiazzo, F., & Sergi, V. (2017). Reduction of surface roughness by means of laser processing over additive manufacturing metal parts. Materials, 10(1), 12. https://doi.org/10.3390/ma10010030
Aminzadeh, M., & Kurfess, T. R. (2019). Online quality inspection using Bayesian classification in powder-bed additive manufacturing from high-resolution visual camera images. Journal of Intelligent Manufacturing, 30(6), 2505–2523. https://doi.org/10.1007/s10845-018-1412-0
Chen, Z., Gryllias, K., & Li, W. (2019). Intelligent fault diagnosis for rotary machinery using transferable convolutional neural network. IEEE Transactions on Industrial Informatics, 16(1), 339–349. https://doi.org/10.1109/TII.2019.2917233
de Terris, T., Andreau, O., Peyre, P., Adamski, F., Koutiri, I., Gorny, C., et al. (2019). Optimization and comparison of porosity rate measurement methods of Selective Laser Melted metallic parts. Additive Manufacturing, 28, 802–813. https://doi.org/10.1016/j.addma.2019.05.035
Ding, X., & He, Q. (2017). Energy-fluctuated multiscale feature learning with deep ConvNet for intelligent spindle bearing fault diagnosis. IEEE Transactions on Instrumentation and Measurement, 66(8), 1926–1935. https://doi.org/10.1109/TIM.2017.2674738
Donahue, J., Jia, Y., Vinyals, O., Hoffman, J., Zhang, N., Tzeng, E., et al. Decaf: A deep convolutional activation feature for generic visual recognition. In International conference on machine learning, 2014 (pp. 647–655). arXiv:1310.1531.
Gobert, C., Reutzel, E. W., Petrich, J., Nassar, A. R., & Phoha, S. (2018). Application of supervised machine learning for defect detection during metallic powder bed fusion additive manufacturing using high resolution imaging. Additive Manufacturing, 21, 517–528. https://doi.org/10.1016/j.addma.2018.04.005
Gong, H., Rafi, K., Gu, H., Starr, T., & Stucker, B. (2014). Analysis of defect generation in Ti–6Al–4V parts made using powder bed fusion additive manufacturing processes. Additive Manufacturing, 1–4, 87–98. https://doi.org/10.1016/j.addma.2014.08.002
Gu, J., Wang, Z., Kuen, J., Ma, L., Shahroudy, A., Shuai, B., et al. (2018). Recent advances in convolutional neural networks. Pattern Recognition, 77, 354–377. https://doi.org/10.1016/j.patcog.2017.10.013
Guo, C., Li, S., Shi, S., Li, X., Hu, X., Zhu, Q., et al. (2020). Effect of processing parameters on surface roughness, porosity and cracking of as-built IN738LC parts fabricated by laser powder bed fusion. Journal of Materials Processing Technology, 285, 116788. https://doi.org/10.1016/j.jmatprotec.2020.116788
Guo, N., & Leu, M. (2013). Additive manufacturing: Technology, applications and research needs. Frontiers in Mechanical Engineering, 8(3), 215–243. https://doi.org/10.1007/s11465-013-0248-8
He, K., Zhang, X., Ren, S., & Sun, J. Deep residual learning for image recognition. In 2016 IEEE conference on computer vision and pattern recognition (CVPR), 27–30 June 2016 2016 (pp. 770–778). https://doi.org/10.1109/CVPR.2016.90.
Hinton, G., Deng, L., Yu, D., Dahl, G., Mohamed, A.-R., Jaitly, N., et al. (2012). Deep neural networks for acoustic modeling in speech recognition: The shared views of four research groups. Signal Processing Magazine, IEEE, 29, 82–97. https://doi.org/10.1109/MSP.2012.2205597
Hinton, G. E., Srivastava, N., Krizhevsky, A., Sutskever, I., & Salakhutdinov, R. R. (2012). Improving neural networks by preventing co-adaptation of feature detectors. arXiv preprint arXiv:1207.0580.
Hojjatzadeh, S. M. H., Parab, N. D., Guo, Q., Qu, M., Xiong, L., Zhao, C., et al. (2020). Direct observation of pore formation mechanisms during LPBF additive manufacturing process and high energy density laser welding. International Journal of Machine Tools and Manufacture, 153, 103555. https://doi.org/10.1016/j.ijmachtools.2020.103555
Hojjatzadeh, S. M. H., Parab, N. D., Yan, W., Guo, Q., Xiong, L., Zhao, C., et al. (2019). Pore elimination mechanisms during 3D printing of metals. Nature Communications, 10(1), 3088. https://doi.org/10.1038/s41467-019-10973-9
Howard, A., Zhmoginov, A., Chen, L.-C., Sandler, M., & Zhu, M. (2018). Inverted residuals and linear bottlenecks: Mobile networks for classification, detection and segmentation. ArXiv, abs/1801.04381.
Huang, X., Lei, Q., Xie, T., Zhang, Y., Hu, Z., & Zhou, Q. (2020). Deep transfer convolutional neural network and extreme learning machine for lung nodule diagnosis on CT images. Knowledge-Based Systems, 204, 106230. https://doi.org/10.1016/j.knosys.2020.106230
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017). ImageNet classification with deep convolutional neural networks. Communications of the ACM, 60(6), 84–90. https://doi.org/10.1145/3065386
Leuders, S., Thöne, M., Riemer, A., Niendorf, T., Tröster, T., Richard, H. A., et al. (2013). On the mechanical behaviour of titanium alloy TiAl6V4 manufactured by selective laser melting: Fatigue resistance and crack growth performance. International Journal of Fatigue, 48, 300–307. https://doi.org/10.1016/j.ijfatigue.2012.11.011
Lough, C. S., Wang, X., Smith, C. C., Landers, R. G., Bristow, D. A., Drallmeier, J. A., et al. (2020). Correlation of SWIR imaging with LPBF 304L stainless steel part properties. Additive Manufacturing, 35, 101359. https://doi.org/10.1016/j.addma.2020.101359
Lu, Q. Y., Nguyen, N. V., Hum, A. J. W., Tran, T., & Wong, C. H. (2020). Identification and evaluation of defects in selective laser melted 316L stainless steel parts via in-situ monitoring and micro computed tomography. Additive Manufacturing, 35, 101287. https://doi.org/10.1016/j.addma.2020.101287
Peng, T., & Chen, C. (2018). Influence of energy density on energy demand and porosity of 316L stainless steel fabricated by selective laser melting. International Journal of Precision Engineering and Manufacturing-Green Technology, 5(1), 55–62. https://doi.org/10.1007/s40684-018-0006-9
Prashanth, K., Scudino, S., Maity, T., Das, J., & Eckert, J. (2017). Is the energy density a reliable parameter for materials synthesis by selective laser melting? Materials Research Letters, 5(6), 386–390. https://doi.org/10.1080/21663831.2017.1299808
Qi, X., Chen, G., Li, Y., Cheng, X., & Li, C. (2019). Applying neural-network-based machine learning to additive manufacturing: Current applications, challenges, and future perspectives. Engineering, 5(4), 721–729. https://doi.org/10.1016/j.eng.2019.04.012
Schmidt, M., Merklein, M., Bourell, D. L., Dimitrov, D., Hausotte, T., Wegener, K., et al. (2017). Laser based additive manufacturing in industry and academia. Cirp Annals-Manufacturing Technology, 66(2), 561–583. https://doi.org/10.1016/j.cirp.2017.05.011
Scipioni Bertoli, U., Wolfer, A. J., Matthews, M. J., Delplanque, J.-P.R., & Schoenung, J. M. (2017). On the limitations of volumetric energy density as a design parameter for selective laser melting. Materials & Design, 113, 331–340. https://doi.org/10.1016/j.matdes.2016.10.037
Shao, S., McAleer, S., Yan, R., & Baldi, P. (2018). Highly accurate machine fault diagnosis using deep transfer learning. IEEE Transactions on Industrial Informatics, 15(4), 2446–2455. https://doi.org/10.1109/TII.2018.2864759
Shevchik, S. A., Kenel, C., Leinenbach, C., & Wasmer, K. (2018). Acoustic emission for in situ quality monitoring in additive manufacturing using spectral convolutional neural networks. Additive Manufacturing, 21, 598–604. https://doi.org/10.1016/j.addma.2017.11.012
Simonyan, K., & Zisserman, A. (2015). Very deep convolutional networks for large-scale image recognition. CoRR, abs/1409.1556.
Snell, R., Tammas-Williams, S., Chechik, L., Lyle, A., Hernández-Nava, E., Boig, C., et al. (2020). Methods for rapid pore classification in metal additive manufacturing. JOM Journal of the Minerals Metals and Materials Society, 72(1), 101–109. https://doi.org/10.1007/s11837-019-03761-9
Snow, Z., Diehl, B., Reutzel, E. W., & Nassar, A. (2021). Toward in-situ flaw detection in laser powder bed fusion additive manufacturing through layerwise imagery and machine learning. Journal of Manufacturing Systems, 59, 12–26. https://doi.org/10.1016/j.jmsy.2021.01.008
Snow, Z., Nassar, A., & Reutzel, E. W. (2020). Review of the formation and impact of flaws in powder bed fusion additive manufacturing. Additive Manufacturing. https://doi.org/10.1016/j.addma.2020.101457
Sun, D., Gu, D., Lin, K., Ma, J., Chen, W., Huang, J., et al. (2019). Selective laser melting of titanium parts: Influence of laser process parameters on macro- and microstructures and tensile property. Powder Technology, 342, 371–379. https://doi.org/10.1016/j.powtec.2018.09.090
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., et al. Going deeper with convolutions. In Proceedings of the IEEE conference on computer vision and pattern recognition, 2015 (pp. 1–9). https://doi.org/10.1109/CVPR.2015.7298594.
Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., & Wojna, Z. Rethinking the inception architecture for computer vision. In Proceedings of the IEEE conference on computer vision and pattern recognition, 2016 (pp. 2818–2826). https://doi.org/10.1109/CVPR.2016.308.
Tan, C., Sun, F., Kong, T., Zhang, W., Yang, C., & Liu, C. A Survey on Deep Transfer Learning. In V. Kůrková, Y. Manolopoulos, B. Hammer, L. Iliadis, & I. Maglogiannis (Eds.), Artificial neural networks and machine learning–ICANN 2018, Cham, 2018// 2018 (pp. 270–279): Springer International Publishing.
Thompson, A., Maskery, I., & Leach, R. K. (2016). X-ray computed tomography for additive manufacturing: A review. Measurement Science and Technology, 27(7), 072001. https://doi.org/10.1088/0957-0233/27/7/072001
Wang, C., Tan, X., Tor, S. B., & Lim, C. (2020). Machine learning in additive manufacturing: State-of-the-art and perspectives. Additive Manufacturing. https://doi.org/10.1016/j.addma.2020.101538
Wang, P., Tan, X., He, C., Nai, M. L. S., Huang, R., Tor, S. B., et al. (2018). Scanning optical microscopy for porosity quantification of additively manufactured components. Additive Manufacturing, 21, 350–358. https://doi.org/10.1016/j.addma.2018.03.019
Wen, L., Gao, L., & Li, X. (2017). A new deep transfer learning based on sparse auto-encoder for fault diagnosis. IEEE Transactions on Systems, Man, and Cybernetics: Systems, 49(1), 136–144. https://doi.org/10.1109/TSMC.2017.2754287
Wen, L., Li, X., & Gao, L. (2019). A transfer convolutional neural network for fault diagnosis based on ResNet-50. Neural Computing and Applications. https://doi.org/10.1007/s00521-019-04097-w
Xu, G., Shen, X., Chen, S., Zong, Y., Zhang, C., Yue, H., et al. (2019). A deep transfer convolutional neural network framework for EEG signal classification. IEEE Access, 7, 112767–112776. https://doi.org/10.1109/ACCESS.2019.2930958
Yu, T. Y., Li, M., Breaux, A., Atri, M., Obeidat, S., & Ma, C. (2019). Experimental and numerical study on residual stress and geometric distortion in powder bed fusion process. Journal of Manufacturing Processes, 46, 214–224. https://doi.org/10.1016/j.jmapro.2019.09.010
Zhang, B., Liu, S., & Shin, Y. C. (2019). In-Process monitoring of porosity during laser additive manufacturing process. Additive Manufacturing, 28, 497–505. https://doi.org/10.1016/j.addma.2019.05.030
Acknowledgements
This work has been supported by the China Postdoctoral Science Foundation under Grant No. 2020M682397, No. 2020M682396, the National Natural Science Foundation of China (NSFC) under Grant No. 51805179, the National Defense Innovation Program under Grant No. 18-163-00-TS-004-033-01, and the Research Funds of the Maritime Defense Technologies Innovation.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no conflict of interest.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Li, J., Zhou, Q., Huang, X. et al. In situ quality inspection with layer-wise visual images based on deep transfer learning during selective laser melting. J Intell Manuf 34, 853–867 (2023). https://doi.org/10.1007/s10845-021-01829-5
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10845-021-01829-5