Abstract
This work presents an in-situ quality assessment and improvement technique using point cloud and AI for data processing and smart decision making in Additive Manufacturing (AM) fabrication to improve the quality and accuracy of fabricated artifacts. The top surface point-cloud containing top surface geometry and quality information is pre-processed and passed to an improved deep Hybrid Convolutional Auto-Encoder decoder (HCAE) model used to statistically describe the artifact's quality. The HCAE’s output is comprised of 9 × 9 segments, each including four channels with the segment's probability to contain one of four labels, Under-printed, Normally-printed, Over-printed, or Empty region. This data structure plays a significant role in command generation for fabrication process optimization. The HCAE’s accuracy and repeatability were measured by a multi-label multi-output metric developed in this study. The HCAE’s results are used to perform a real-time process adjustment by manipulating the future layer's fabrication through the G-code modification. By adjusting the machine's print speed and feed-rate, the controller exploits the subsequent layer’s deposition, grid-by-grid. The algorithm is then tested with two defective process plans: severe under-extrusion and over-extrusion conditions. Both test artifacts' quality advanced significantly and converged to an acceptable state by four iterations.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Akhavan, J., & Manoochehri, S. (2022). Sensory Data fusion using machine learning methods for in-situ defect registration in additive manufacturing: A review. In 2022 IEEE International IOT, Electronics and Mechatronics Conference, IEMTRONICS 2022. https://doi.org/10.1109/IEMTRONICS55184.2022.9795815
Alammar, A., Kois, J. C., Revilla-León, M., & Att, W. (2022). Additive manufacturing technologies: Current status and future perspectives. In Journal of Prosthodontics (Vol. 31, pp. 4–12). John Wiley and Sons Inc. https://doi.org/10.1111/jopr.13477
Askari, I., Badnava, B., Woodruff, T., Zeng, S., & Fang, H. (2022). Sampling-based nonlinear MPC of neural network dynamics with application to autonomous vehicle motion planning. In Proceedings of the American Control Conference, 2022-June, 2084–2090. https://doi.org/10.23919/ACC53348.2022.9867324
Azimirad, V., Ramezanlou, M. T., Sotubadi, S. V., & Janabi-Sharifi, F. (2022). A consecutive hybrid spiking-convolutional (CHSC) neural controller for sequential decision making in robots. Neurocomputing, 490, 319–336. https://doi.org/10.1016/j.neucom.2021.11.097
Bakhshi, A., Sedghi, R., & Hojati, M. (2021). A preliminary study on the mix design of 3D-printable engineered cementitious composite. In Tran-SET 2021-Proceedings of the Tran-SET Conference 2021. https://doi.org/10.1061/9780784483787.021
Baumann, F., & Roller, D. (2016). Vision based error detection for 3D printing processes. MATEC Web of Conferences. https://doi.org/10.1051/matecconf/20165906003
Borish, M., Post, B. K., Roschli, A., Chesser, P. C., & Love, L. J. (2020). Real-time defect correction in large-scale polymer additive manufacturing via thermal imaging and laser profilometer. Procedia Manufacturing. https://doi.org/10.1016/j.promfg.2020.05.091
Delli, U., & Chang, S. (2018). Automated process monitoring in 3D printing using supervised machine learning. Procedia Manufacturing, 26, 865–870. https://doi.org/10.1016/j.promfg.2018.07.111
Devesse, W., de Baere, D., Hinderdael, M., & Guillaume, P. (2016). Hardware-in-the-loop control of additive manufacturing processes using temperature feedback. Journal of Laser Applications, 28(2), 1022302. https://doi.org/10.2351/1.4943911
Dosovitskiy, A., Springenberg, J. T., Riedmiller, M., & Brox, T. (2014). Discriminative unsupervised feature learning with convolutional neural networks. Advances in Neural Information Processing Systems. https://doi.org/10.48550/arXiv.1406.6909
Durão, L. F. C. S., Barkoczy, R., Zancul, E., Lee Ho, L., & Bonnard, R. (2019). Optimizing additive manufacturing parameters for the fused deposition modeling technology using a design of experiments. Progress in Additive Manufacturing, 4(3), 291–313. https://doi.org/10.1007/s40964-019-00075-9
Farbiz, F., Habibullah, M. S., Hamadicharef, B., Maszczyk, T., & Aggarwal, S. (2022). Knowledge-embedded machine learning and its applications in smart manufacturing. Journal of Intelligent Manufacturing. https://doi.org/10.1007/s10845-022-01973-6
Fastowicz, J., Grudziński, M., Tecław, M., & Okarma, K. (2019). Objective 3D printed surface quality assessment based on entropy of depth maps. Entropy, 21(1), 1–13. https://doi.org/10.3390/e21010097
Gockel, J., Sheridan, L., Koerper, B., & Whip, B. (2019). The influence of additive manufacturing processing parameters on surface roughness and fatigue life. International Journal of Fatigue. https://doi.org/10.1016/j.ijfatigue.2019.03.025
Harbig, J., Wenzler, D. L., Baehr, S., Kick, M. K., Merschroth, H., Wimmer, A., Weigold, M., & Zaeh, M. F. (2022). Methodology to determine melt pool anomalies in powder bed fusion of metals using a laser beam by means of process monitoring and sensor data fusion. Materials. https://doi.org/10.3390/ma15031265
Heydarian, M., Doyle, T. E., & Samavi, R. (2022). MLCM: Multi-label confusion matrix. IEEE Access, 10, 19083–19095. https://doi.org/10.1109/ACCESS.2022.3151048
Ioffe, S., & Szegedy, C. (2015). Batch normalization: Accelerating deep network training by reducing internal covariate shift. In 32nd International Conference on Machine Learning, ICML 2015, 1. https://doi.org/10.48550/arXiv.1502.03167
Jin, Z., Zhang, Z., & Gu, G. X. (2020). Automated real-time detection and prediction of interlayer imperfections in additive manufacturing processes using artificial intelligence. Advanced Intelligent Systems, 2(1), 1900130. https://doi.org/10.1002/aisy.201900130
Kumar, S., Gopi, T., Harikeerthana, N., Gupta, M. K., Gaur, V., Krolczyk, G. M., & Wu, C. (2022). Machine learning techniques in additive manufacturing: A state of the art review on design, processes and production control. Journal of Intelligent Manufacturing. https://doi.org/10.1007/s10845-022-02029-5
Kwon, O., Kim, H. G., Ham, M. J., Kim, W., Kim, G. H., Cho, J. H., Kim, N., & il, & Kim, K. (2020). A deep neural network for classification of melt-pool images in metal additive manufacturing. Journal of Intelligent Manufacturing, 31(2), 375–386. https://doi.org/10.1007/s10845-018-1451-6
Larsen, S., & Hooper, P. A. (2022). Deep semi-supervised learning of dynamics for anomaly detection in laser powder bed fusion. Journal of Intelligent Manufacturing, 33(2), 457–471. https://doi.org/10.1007/s10845-021-01842-8
Li, J., Zhou, Q., Huang, X., Li, M., & Cao, L. (2021). In situ quality inspection with layer-wise visual images based on deep transfer learning during selective laser melting. Journal of Intelligent Manufacturing. https://doi.org/10.1007/s10845-021-01829-5
Li, X., Jia, X., Yang, Q., & Lee, J. (2020). Quality analysis in metal additive manufacturing with deep learning. Journal of Intelligent Manufacturing, 31(8), 2003–2017. https://doi.org/10.1007/s10845-020-01549-2
Li, Z., Zhang, Z., Shi, J., & Wu, D. (2019). Prediction of surface roughness in extrusion-based additive manufacturing with machine learning. Robotics and Computer-Integrated Manufacturing, 57(January), 488–495. https://doi.org/10.1016/j.rcim.2019.01.004
Luo, C., Wang, X., Migler, K. B., & Seppala, J. E. (2020). Effects of feed rates on temperature profiles and feed forces in material extrusion additive manufacturing. Additive Manufacturing. https://doi.org/10.1016/j.addma.2020.101361
Lyu, J., Boroujeni, J. A. T., & Manoochehri, S. (2021). In-situ laser-based process monitoring and in-plane surface anomaly identification for additive manufacturing using point cloud and machine learning. In International Design Engineering Technical Conferences and Computers and Information in Engineering Conference. https://doi.org/10.1115/DETC2021-69436
Lyu, J., Akhavan, J., & Manoochehri, S. (2022). Image-based dataset of artifact surfaces fabricated by additive manufacturing with applications in machine learning. Data in Brief. https://doi.org/10.1016/j.dib.2022.107852
Lyu, J., & Manoochehri, S. (2021). Online convolutional neural network-based anomaly detection and quality control for fused filament fabrication process. Virtual and Physical Prototyping, 16(2), 160–177. https://doi.org/10.1080/17452759.2021.1905858
Mahmoud, Y., & Manoochehri, S. (2021). In-situ temperature monitoring of ABS during Fused Filament Fabrication (FFF) process with varying process parameters. In Proceedings of the ASME Design Engineering Technical Conference, 3A-2021. https://doi.org/10.1115/DETC2021-69813
Mondal, S., Gwynn, D., Ray, A., & Basak, A. (2020). Investigation of melt pool geometry control in additive manufacturing using hybrid modeling. Metals, 10(5), 1–23. https://doi.org/10.3390/met10050683
Moretti, M., Rossi, A., & Senin, N. (2021). In-process monitoring of part geometry in fused filament fabrication using computer vision and digital twins. Additive Manufacturing, 37, 101609. https://doi.org/10.1016/j.addma.2020.101609
Obilanade, D., Dordlofva, C., & Törlind, P. (2021). Surface roughness considerations in design for additive manufacturing: A literature review. Proceedings of the Design Society. https://doi.org/10.1017/pds.2021.545
Pandiyan, V., Drissi-Daoudi, R., Shevchik, S., Masinelli, G., Le-Quang, T., Logé, R., & Wasmer, K. (2022). Deep transfer learning of additive manufacturing mechanisms across materials in metal-based laser powder bed fusion process. Journal of Materials Processing Technology. https://doi.org/10.1016/j.jmatprotec.2022.117531
Papon, E. A., Haque, A., & Sharif, M. A. R. (2021). Numerical study for the improvement of bead spreading architecture with modified nozzle geometries in additive manufacturing of polymers. Rapid Prototyping Journal. https://doi.org/10.1108/RPJ-05-2019-0142
Petsiuk, A. L., & Pearce, J. M. (2020). Open source computer vision-based layer-wise 3D printing analysis. Additive Manufacturing, 36(7), 101473. https://doi.org/10.1016/j.addma.2020.101473
Preissler, M., Broghammer, J., Rosenberger, M., & Notni, G. (2018). Inline process monitoring method for geometrical characteristics in additive manufacturing. Journal of Physics. https://doi.org/10.1088/1742-6596/1044/1/012035
Ravi, A. K., Deshpande, A., & Hsu, K. H. (2016). An in-process laser localized pre-deposition heating approach to inter-layer bond strengthening in extrusion based polymer additive manufacturing. Journal of Manufacturing Processes. https://doi.org/10.1016/j.jmapro.2016.08.007
Rusu, R. B., & Cousins, S. (2011). 3d is here: Point cloud library (pcl). IEEE International Conference on Robotics and Automation, 2011, 1–4. https://doi.org/10.1109/ICRA.2011.5980567
Suzuki, T., Fukushige, S., & Tsunori, M. (2020). Load path visualization and fiber trajectory optimization for additive manufacturing of composites. Additive Manufacturing. https://doi.org/10.1016/j.addma.2019.100942
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., & Rabinovich, A. (2015). Going deeper with convolutions. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 07-12-June-2015. https://doi.org/10.1109/CVPR.2015.7298594
Tercan, H., & Meisen, T. (2022). Machine learning and deep learning based predictive quality in manufacturing: a systematic review. In Journal of Intelligent Manufacturing (Vol. 33, Issue 7, pp. 1879–1905). Springer. https://doi.org/10.1007/s10845-022-01963-8
Tian, Q., Guo, S., Melder, E., Bian, L., & Guo, W. (2021). Deep learning-based data fusion method for in situ porosity detection in laser-based additive manufacturing. Journal of Manufacturing Science and Engineering, Transactions of the ASME, Doi, 10(1115/1), 4048957.
Xu, K., Lyu, J., & Manoochehri, S. (2022). In situ process monitoring using acoustic emission and laser scanning techniques based on machine learning models. Journal of Manufacturing Processes, 84, 357–374. https://doi.org/10.1016/j.jmapro.2022.10.002
Zhao, C., Du, S., Lv, J., Deng, Y., & Li, G. (2021). A novel parallel classification network for classifying three-dimensional surface with point cloud data. Journal of Intelligent Manufacturing. https://doi.org/10.1007/s10845-021-01802-2
Acknowledgements
This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors. The authors would like to thank Miss Youmna Mahmoud for many fruitful discussions and her extensive help with the write-up review.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
No potential conflict of interest was reported by the authors.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Akhavan, J., Lyu, J. & Manoochehri, S. A deep learning solution for real-time quality assessment and control in additive manufacturing using point cloud data. J Intell Manuf 35, 1389–1406 (2024). https://doi.org/10.1007/s10845-023-02121-4
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10845-023-02121-4