Skip to main content
Log in

Selecting subsets of source data for transfer learning with applications in metal additive manufacturing

  • Published:
Journal of Intelligent Manufacturing Aims and scope Submit manuscript

Abstract

Considering data insufficiency in metal additive manufacturing (AM), transfer learning (TL) has been adopted to extract knowledge from source domains (e.g., completed printings) to improve the modeling performance in target domains (e.g., new printings). Current applications use all accessible source data directly in TL with no regard to the similarity between source and target data. This paper proposes a systematic method to find appropriate subsets of source data based on similarities between the source and limited target datasets. Such similarity is characterized by the spatial and model distance metrics. A Pareto frontier-based source data selection method is developed, where the source data located on the Pareto frontier defined by two similarity distance metrics are selected iteratively. This method is integrated into an instance-based TL method (decision tree regression model) and a model-based TL method (fine-tuned artificial neural network). Both models are then tested on several regression tasks in metal AM. Comparison results demonstrate that (1) the source data selection method is general and supports integration with various TL methods and distance metrics, (2) compared with using all source data, the proposed method can find a subset of source data from the same domain with better TL performance in metal AM regression tasks involving different processes and machines, and (3) when multiple source domains exist, the source data selection method could find the subset from one source domain to obtain comparable or better TL performance than the model constructed using data from all source domains.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

(Adapted from Ding et al., 2022)

Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

Data availability

The datasets about the melt pool width in DED-LB/p and the relative densities of parts fabricated by different machines are open-accessed. The dataset about the melt pool width in DED-LB/w is available upon reasonable request from the authors.

References

  • Aboutaleb, A. M., Bian, L., Elwany, A., Shamsaei, N., Thompson, S. M., & Tapia, G. (2017). Accelerated process optimization for laser-based additive manufacturing by leveraging similar prior studies. IISE Transactions, 49(1), 31–44. https://doi.org/10.1080/0740817X.2016.1189629

    Article  Google Scholar 

  • Aharoni, R., & Goldberg, Y. (2020). Unsupervised domain clusters in pretrained language models. arXiv preprint: arXiv:2004.02105

  • Akhavan, J., Lyu, J., Mahmoud, Y., Xu, K., Vallabh, C. K. P., & Manoochehri, S. (2023). Dataset of in-situ coaxial monitoring and print’s cross-section images by direct energy deposition fabrication. Scientific Data, 10(1), 776. https://doi.org/10.1038/s41597-023-02672-4

    Article  Google Scholar 

  • Bischl, B., Binder, M., Lang, M., Pielok, T., Richter, J., Coors, S., et al. (2023). Hyperparameter optimization: Foundations, algorithms, best practices, and open challenges. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, 13(2), 1–43. https://doi.org/10.1002/widm.1484

    Article  Google Scholar 

  • Bisheh, M. N., Wang, X., Chang, S. I., Lei, S., & Ma, J. (2023). Image-based characterization of laser scribing quality using transfer learning. Journal of Intelligent Manufacturing, 34, 2307–2319. https://doi.org/10.1007/s10845-022-01926-z

    Article  Google Scholar 

  • Brion, D. A. J., Shen, M., & Pattinson, S. W. (2022). Automated recognition and correction of warp deformation in extrusion additive manufacturing. Additive Manufacturing, 56, 102838. https://doi.org/10.1016/j.addma.2022.102838

    Article  Google Scholar 

  • Cheng, G. H., Wang, G. G., & Hwang, Y. M. (2021). Multi-objective optimization for high-dimensional expensively constrained black-box problems. Journal of Mechanical Design. https://doi.org/10.1115/1.4050749

    Article  Google Scholar 

  • Dai, X., Karimi, S., Hachey, B., & Paris, C. (2019). Using similarity measures to select pretraining data for NER. arXiv preprint: arXiv:1904.00585

  • Dai, X., Karimi, S., Hachey, B., & Paris, C. (2020). Cost-effective selection of pretraining data: a case study of pretraining BERT on social media. arXiv preprint: arXiv:2010.01150

  • Ding, D., He, F., Yuan, L., Pan, Z., Wang, L., & Ros, M. (2021). The first step towards intelligent wire arc additive manufacturing: An automatic bead modelling system using machine learning through industrial information integration. Journal of Industrial Information Integration, 23, 100218. https://doi.org/10.1016/j.jii.2021.100218

    Article  Google Scholar 

  • Ding, Y., Ding, P., Zhao, X., Cao, Y., & Jia, M. (2022). Transfer learning for remaining useful life prediction across operating conditions based on multisource domain adaptation. IEEE/ASME Transactions on Mechatronics, 27(5), 4143–4152. https://doi.org/10.1109/TMECH.2022.3147534

    Article  Google Scholar 

  • Dongshang. (2023). Stainless steel density. http://www.dsstainlesssteel.com/stainless-steel-density/. Accessed 13 Oct 2023.

  • Ferreira, R. D. S. B., Sabbaghi, A., & Huang, Q. (2020). Automated geometric shape deviation modeling for additive manufacturing systems via Bayesian neural networks. IEEE Transactions on Automation Science and Engineering, 17(2), 584–598. https://doi.org/10.1109/TASE.2019.2936821

    Article  Google Scholar 

  • Gao, J., Zhao, X., Chen, B., Yan, F., Guo, H., & Tang, R. (2023). AutoTransfer: instance transfer for cross-domain recommendations. In SIGIR 2023—Proceedings of the 46th international ACM SIGIR conference on research and development in information retrieval, Taipei, Taiwan, China (pp. 1478–1487). Association for Computing Machinery. https://doi.org/10.1145/3539618.3591701

  • Guo, H., Pasunuru, R., & Bansal, M. (2019). AutoSEM: automatic task selection and mixing in multi-task learning. arXiv preprint: arXiv:1904.04153

  • Jamnikar, N., Liu, S., Brice, C., & Zhang, X. (2021). Machine learning based in situ quality estimation by molten pool condition-quality relations modeling using experimental data. arXiv preprint: arXiv:2103.12066

  • Kang, Z., Yang, B., Yang, S., Fang, X., & Zhao, C. (2020). Online transfer learning with multiple source domains for multi-class classification. Knowledge-Based Systems, 190, 105149. https://doi.org/10.1016/j.knosys.2019.105149

    Article  Google Scholar 

  • Karimi, S., Dai, X., Hassanzadeh, H., & Nguyen, A. (2017). Automatic diagnosis coding of radiology reports: a comparison of deep learning and conventional classification methods. In BioNLP 2017—SIGBioMed workshop on biomedical natural language processing, proceedings of the 16th BioNLP Workshop, Vancouver, Canada (pp. 328–332). Association for Computational Linguistics. https://doi.org/10.18653/v1/w17-2342

  • Kim, H., Lee, H., & Ahn, S. H. (2022). Systematic deep transfer learning method based on a small image dataset for spaghetti-shape defect monitoring of fused deposition modeling. Journal of Manufacturing Systems, 65, 439–451. https://doi.org/10.1016/j.jmsy.2022.10.009

    Article  Google Scholar 

  • Kitahara, A. R., & Holm, E. A. (2018). Microstructure cluster analysis with transfer learning and unsupervised learning. Integrating Materials and Manufacturing Innovation, 7(3), 148–156. https://doi.org/10.1007/s40192-018-0116-9

    Article  Google Scholar 

  • Knüttel, D., Baraldo, S., Valente, A., Wegener, K., & Carpanzano, E. (2022). Transfer learning of neural network based process models in direct metal deposition. Procedia CIRP, 107, 863–868. https://doi.org/10.1016/j.procir.2022.05.076

    Article  Google Scholar 

  • Lange, L., Strötgen, J., Adel, H., & Klakow, D. (2021). To share or not to share: predicting sets of sources for model transfer learning. arXiv preprint: arXiv:2104.08078

  • Leung, H. C., Leung, C. S., & Wong, E. W. M. (2019). Fault and noise tolerance in the incremental extreme learning machine. IEEE Access, 7, 155171–155183. https://doi.org/10.1109/ACCESS.2019.2948059

    Article  Google Scholar 

  • Li, Z., Liu, F., Yang, W., Peng, S., & Zhou, J. (2022). A survey of convolutional neural networks: Analysis, applications, and prospects. IEEE Transactions on Neural Networks and Learning Systems, 33(12), 6999–7019. https://doi.org/10.1109/TNNLS.2021.3084827

    Article  Google Scholar 

  • Lin, Y. H., Chen, C. Y., Lee, J., Li, Z., Zhang, Y., Xia, M., et al. (2019). Choosing transfer languages for cross-lingual learning. arXiv preprint: arXiv:1905.12688

  • Liu, M., Song, Y., Zou, H., & Zhang, T. (2020). Reinforced training data selection for domain adaptation. In ACL 2019—57th annual meeting of the association for computational linguistics, Florence, Italy (pp. 1957–1968). Association for Computational Linguistics. https://doi.org/10.18653/v1/p19-1189

  • Liu, R., Liu, S., & Zhang, X. (2021a). A physics-informed machine learning model for porosity analysis in laser powder bed fusion additive manufacturing. International Journal of Advanced Manufacturing Technology, 113(7–8), 1943–1958. https://doi.org/10.1007/s00170-021-06640-3

    Article  Google Scholar 

  • Liu, S., Stebner, A. P., Kappes, B. B., & Zhang, X. (2021b). Machine learning for knowledge transfer across multiple metals additive manufacturing printers. Additive Manufacturing, 39, 101877. https://doi.org/10.1016/j.addma.2021.101877

    Article  Google Scholar 

  • Lu, H., Wu, J., Ruan, Y., Qian, F., Meng, H., Gao, Y., & Xu, T. (2023). A multi-source transfer learning model based on LSTM and domain adaptation for building energy prediction. International Journal of Electrical Power and Energy Systems, 149, 109024. https://doi.org/10.1016/j.ijepes.2023.109024

    Article  Google Scholar 

  • MatWeb. (2023). Duplex stainless steel 2209. https://www.matweb.com/search/datasheet.aspx?matguid=e4df7ef1593f4f518bd3b26667a0aa56. Accessed 13 Oct 2023.

  • Mehta, M., & Shao, C. (2022). Federated learning-based semantic segmentation for pixel-wise defect detection in additive manufacturing. Journal of Manufacturing Systems, 64, 197–210. https://doi.org/10.1016/j.jmsy.2022.06.010

    Article  Google Scholar 

  • Mikolov, T., Grave, E., Bojanowski, P., Puhrsch, C., & Joulin, A. (2017). Advances in pre-training distributed word representations. arXiv preprint: arXiv:1712.09405

  • Milner, B. B., Gradl, P., Snedden, G., Brooks, M., Pitot, J., Lopez, E., et al. (2021). Metal additive manufacturing in aerospace: A review. Materials and Design, 209, 110008. https://doi.org/10.1016/j.matdes.2021.110008

    Article  Google Scholar 

  • Ngatchou, P., Zarei, A., & El-Sharkawi, M. A. (2005). Pareto multi objective optimization. In Proceedings of the 13th international conference on intelligent systems application to power systems, Arlington, VA, USA (pp. 84–91). IEEE. https://doi.org/10.1109/ISAP.2005.1599245

  • Olleak, A., & Xi, Z. (2020). Calibration and validation framework for selective laser melting process based on multi-fidelity models and limited experiment data. Journal of Mechanical Design, Transactions of the ASME, 142(8), 1–13. https://doi.org/10.1115/1.4045744

    Article  Google Scholar 

  • Ontañón, S. (2020). An overview of distance and similarity functions for structured data. Artificial Intelligence Review, 53(7), 5309–5351. https://doi.org/10.1007/s10462-020-09821-w

    Article  Google Scholar 

  • Pandita, P., Ghosh, S., Gupta, V. K., Meshkov, A., & Wang, L. (2022). Application of deep transfer learning and uncertainty quantification for process identification in powder bed fusion. ASCE-ASME Journal of Risk and Uncertainty in Engineering Systems, Part B: Mechanical Engineering, 8(1), 1–12. https://doi.org/10.1115/1.4051748

    Article  Google Scholar 

  • Pandiyan, V., Drissi-Daoudi, R., Shevchik, S., Masinelli, G., Le-Quang, T., Logé, R., & Wasmer, K. (2022). Deep transfer learning of additive manufacturing mechanisms across materials in metal-based laser powder bed fusion process. Journal of Materials Processing Technology. https://doi.org/10.1016/j.jmatprotec.2022.117531

    Article  Google Scholar 

  • Pang, Y., Cao, Y., Chu, Y., Liu, M., Snyder, K., MacKenzie, D., & Cao, C. (2020). Additive manufacturing of batteries. Advanced Functional Materials, 30(1), 1–22. https://doi.org/10.1002/adfm.201906244

    Article  Google Scholar 

  • Pardoe, D., & Stone, P. (2010). Boosting for regression transfer. In Proceedings of the 27th international conference on machine learning, Haifa, Israel (pp. 863–870). Association for Computing Machinery. https://doi.org/10.5555/3104322.3104432

  • Qin, J., Hu, F., Liu, Y., Witherell, P., Wang, C. C. L., Rosen, D. W., et al. (2022). Research and application of machine learning for additive manufacturing. Additive Manufacturing. https://doi.org/10.1016/j.addma.2022.102691

    Article  Google Scholar 

  • Rahmani Dehaghani, M., Sahraeidolatkhaneh, A., Nilsen, M., Sikström, F., Sajadi, P., Tang, Y., & Wang, G. G. (2024). System identification and closed-loop control of laser hot-wire directed energy deposition using the parameter-signature-property modeling scheme. Journal of Manufacturing Processes, 112, 1–13. https://doi.org/10.1016/j.jmapro.2024.01.029

    Article  Google Scholar 

  • Ren, J. (2018). Two-stage-TrAdaboost.R2. https://github.com/jay15summer/Two-stage-TrAdaboost.R2. Accessed 20 Jan 2020.

  • Ren, J., & Wang, H. (2019). Surface variation modeling by fusing multiresolution spatially nonstationary data under a transfer learning framework. Journal of Manufacturing Science and Engineering, Transactions of the ASME, 141(1), 011002. https://doi.org/10.1115/1.4041425

    Article  Google Scholar 

  • Ren, J., Wei, A. T., Jiang, Z., Wang, H., & Wang, X. (2021). Improved modeling of kinematics-induced geometric variations in extrusion-based additive manufacturing through between-printer transfer learning. IEEE Transactions on Automation Science and Engineering, 19(3), 2310–2321. https://doi.org/10.1109/TASE.2021.3063389

    Article  Google Scholar 

  • Ren, K., Chew, Y., Zhang, Y. F., Fuh, J. Y. H., & Bi, G. J. (2020). Thermal field prediction for laser scanning paths in laser aided additive manufacturing by physics-based machine learning. Computer Methods in Applied Mechanics and Engineering, 362, 112734. https://doi.org/10.1016/j.cma.2019.112734

    Article  Google Scholar 

  • Ruder, S., & Plank, B. (2017). Learning to select data for transfer learning with Bayesian optimization. arXiv preprint: arXiv:1707.05246

  • Sabbaghi, A., & Huang, Q. (2018). Model transfer across additive manufacturing processes via mean effect equivalence of lurking variables. The Annals of Applied Statistics, 12(4), 2409–2429. https://doi.org/10.1214/18-AOAS1158

    Article  Google Scholar 

  • Sabbaghi, A., Huang, Q., & Dasgupta, T. (2018). Bayesian model building from small samples of disparate data for capturing in-plane deviation in additive manufacturing. Technometrics, 60(4), 532–544. https://doi.org/10.1080/00401706.2017.1391715

    Article  Google Scholar 

  • Schönemann, P. H. (1966). A generalized solution of the orthogonal procrustes problem. Psychometrika, 31(1), 1–10. https://doi.org/10.1007/BF02289451

    Article  Google Scholar 

  • Senanayaka, A., Tian, W., Falls, T. C., & Bian, L. (2023). Understanding the effects of process conditions on thermal–defect relationship: a transfer machine learning approach. Journal of Manufacturing Science and Engineering. https://doi.org/10.1115/1.4057052

    Article  Google Scholar 

  • Shin, S. J., Lee, J. H., Sainand, J., & Kim, D. B. (2024). Material-adaptive anomaly detection using property-concatenated transfer learning in wire arc additive manufacturing. International Journal of Precision Engineering and Manufacturing, 25, 383–408. https://doi.org/10.1007/s12541-023-00924-2

    Article  Google Scholar 

  • Tang, Y., Rahmani Dehaghani, M., & Wang, G. G. (2023a). Comparison of transfer learning based additive manufacturing models via a case study. arXiv preprint: arXiv:2305.11181

  • Tang, Y., Rahmani Dehaghani, M., & Wang, G. G. (2023b). Review of transfer learning in modeling additive manufacturing processes. Additive Manufacturing, 61, 103357. https://doi.org/10.1016/j.addma.2022.103357

    Article  Google Scholar 

  • Tian, J., Han, D., Li, M., & Shi, P. (2022). A multi-source information transfer learning method with subdomain adaptation for cross-domain fault diagnosis. Knowledge-Based Systems, 243, 108466. https://doi.org/10.1016/j.knosys.2022.108466

    Article  Google Scholar 

  • Tian, Y., Sehovac, L., & Grolinger, K. (2019). Similarity-based chained transfer learning for energy forecasting with big data. IEEE Access, 7, 139895–139908. https://doi.org/10.1109/ACCESS.2019.2943752

    Article  Google Scholar 

  • Van Houdt, G., Mosquera, C., & Nápoles, G. (2020). A review on the long short-term memory model. Artificial Intelligence Review, 53(8), 5929–5955. https://doi.org/10.1007/s10462-020-09838-1

    Article  Google Scholar 

  • Vasco, J. C. (2021). Additive manufacturing for the automotive industry. In J. P. Davim & K. Gupta (Eds.), Additive manufacturing (pp. 505–530). Elsevier Inc. https://doi.org/10.1016/B978-0-12-818411-0.00010-0

    Chapter  Google Scholar 

  • Wang, B., Qiu, M., Wang, X., Li, Y., Gong, Y., Zeng, X., et al. (2019). A minimax game for instance based selective transfer learning. In Proceedings of the ACM SIGKDD international conference on knowledge discovery and data mining, Anchorage, AK, USA (pp. 34–43). Association for Computing Machinery. https://doi.org/10.1145/3292500.3330841

  • Wang, J., Li, Y., Gao, R. X., & Zhang, F. (2022a). Hybrid physics-based and data-driven models for smart manufacturing: Modelling, simulation, and explainability. Journal of Manufacturing Systems, 63, 381–391. https://doi.org/10.1016/j.jmsy.2022.04.004

    Article  Google Scholar 

  • Wang, Z., Yang, W., Liu, Q., Zhao, Y., Liu, P., Wu, D., et al. (2022b). Data-driven modeling of process, structure and property in additive manufacturing: A review and future directions. Journal of Manufacturing Processes, 77, 13–31. https://doi.org/10.1016/j.jmapro.2022.02.053

    Article  Google Scholar 

  • Willia, R. J. (1992). Simple statistical gradient-following algorithms for connectionist reinforcement learning. Machine Learning, 8(3), 229–256. https://doi.org/10.1023/A:1022672621406

    Article  Google Scholar 

  • Xie, Y., Li, B., Wang, C., Zhou, K., Wu, C. T., & Li, S. (2023). A Bayesian regularization network approach to thermal distortion control in 3D printing. Computational Mechanics, 72(1), 137–154. https://doi.org/10.1007/s00466-023-02270-6

    Article  Google Scholar 

  • Xie, Y., Li, S., Wu, C. T., Lyu, D., Wang, C., & Zeng, D. (2022). A generalized Bayesian regularization network approach on characterization of geometric defects in lattice structures for topology optimization in preliminary design of 3D printing. Computational Mechanics, 69(5), 1191–1212. https://doi.org/10.1007/s00466-021-02137-8

    Article  Google Scholar 

  • Yao, Y., & Doretto, G. (2010). Boosting for transfer learning with multiple sources. In Proceedings of the IEEE computer society conference on computer vision and pattern recognition, San Francisco, CA, USA (pp. 1855–1862). IEEE. https://doi.org/10.1109/CVPR.2010.5539857

  • Yuan, Y., Chen, Z., Wang, Z., Sun, Y., & Chen, Y. (2023). Attention mechanism-based transfer learning model for day-ahead energy demand forecasting of shopping mall buildings. Energy, 270, 126878. https://doi.org/10.1016/j.energy.2023.126878

    Article  Google Scholar 

  • Zhang, K., Xiong, C., Liu, Z., & Liu, Z. (2020). Selective weak supervision for neural information retrieval. In The web conference 2020—proceedings of the world wide web conference, WWW 2020, Taipei, Taiwan, China (pp. 474–485). Association for Computing Machinery. https://doi.org/10.1145/3366423.3380131

  • Zhang, H., Choi, J. P., Moon, S. K., & Ngo, T. H. (2021). A knowledge transfer framework to support rapid process modeling in aerosol jet printing. Advanced Engineering Informatics, 48, 101264. https://doi.org/10.1016/j.aei.2021.101264

    Article  Google Scholar 

  • Zhou, Z., Shen, H., Liu, B., Du, W., & Jin, J. (2021). Thermal field prediction for welding paths in multi-layer gas metal arc welding-based additive manufacturing: A machine learning approach. Journal of Manufacturing Processes, 64, 960–971. https://doi.org/10.1016/j.jmapro.2021.02.033

    Article  Google Scholar 

  • Zhu, J., Jiang, Q., Shen, Y., Qian, C., Xu, F., & Zhu, Q. (2022). Application of recurrent neural network to mechanical fault diagnosis: A review. Journal of Mechanical Science and Technology, 36(2), 527–542. https://doi.org/10.1007/s12206-022-0102-1

    Article  Google Scholar 

  • Zhu, X., Jiang, F., Guo, C., Xu, D., Wang, Z., & Jiang, G. (2023). Surface morphology inspection for directed energy deposition using small dataset with transfer learning. Journal of Manufacturing Processes, 93, 101–115. https://doi.org/10.1016/j.jmapro.2023.03.016

    Article  Google Scholar 

Download references

Acknowledgements

The authors gratefully acknowledge funding from the Natural Sciences and Engineering Research Council (NSERC) of Canada [Grant numbers: RGPIN-2019-06601] and the in-kind support of University West (Dr. Morgan Nilsen and Dr. Fredrik Sikström) under the Eureka! SMART project (S0410) titled “TANDEM: Tools for Adaptive and Intelligent Control of Discrete Manufacturing Processes.” Meanwhile, the authors acknowledge the Ph.D. candidate Javid Akhavan at Stevens Institute of Technology, United States, for his help in processing their image datasets.

Author information

Authors and Affiliations

Authors

Contributions

Yifan Tang: writing—original draft preparation, validation, methodology, investigation, formal analysis, conceptualization. Mostafa Rahmani Dehaghani: resources, methodology, formal analysis, conceptualization. Pouyan Sajadi: resources, methodology, formal analysis, conceptualization. G. Gary Wang: writing—review & editing, supervision, resources, project administration, methodology, funding acquisition, conceptualization.

Corresponding author

Correspondence to G. Gary Wang.

Ethics declarations

Conflict of interest

The authors declare that they have no conflicts of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Tang, Y., Rahmani Dehaghani, M., Sajadi, P. et al. Selecting subsets of source data for transfer learning with applications in metal additive manufacturing. J Intell Manuf (2024). https://doi.org/10.1007/s10845-024-02402-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10845-024-02402-6

Keywords

Navigation