Abstract
The booming development of emerging technologies and their integration in process planning provide new opportunities for solving the problems in traditional trial-and-error process planning. Combining digital twin with 3D computer vision, this paper defines a novel feature-level digital twin process model (FL-DTPM) by extracting machining features from model-based definition models. Firstly, a multi-dimensional FL-DTPM framework is defined by fusing on-site data, quality information, and process knowledge, where the synergistic mechanism of its virtual and physical processes is revealed. Then, 3D computer vision-enabled machining features extraction method is embedded into the FL-DTPM framework to support the reuse of process knowledge, which involves the procedures of data pre-processing, semantic segmentation, and instance segmentation. Finally, the effectiveness of the proposed features extraction method is verified and the application of FL-DTPM in machining process is presented. Oriented to the impeller process planning, a prototype of FL-DTPM is constructed to explore the potential application scenarios of the proposed method in intelligent process planning, which could provide insights into the industrial implementation of FL-DTPM for aerospace manufacturing enterprises.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Data availability
The data that support the findings of this paper are available on reasonable request to the corresponding author.
References
Aldoma, A., Marton, Z.-C., Tombari, F., Wohlkinger, W., Potthast, C., Zeisl, B., Rusu, R. B., Gedikli, S., & Vincze, M. (2012). Tutorial: Point cloud library: Three-dimensional object recognition and 6 DOF pose estimation. IEEE Robotics & Automation Magazine, 19(3), 80–91. https://doi.org/10.1109/mra.2012.2206675
Amashaa, M. A., khairya, D., Abougalalaa, R. A., Alkhalaf, S., & Areed, M. (2020). Python-based graphical user interface for automatic selection of data clustering algorithm. International Journal of Future Generation Communication and Networking, 13, 451–461.
Charles, R. Q., Su, H., Kaichun, M., & Guibas, L. J. (2017a). PointNet: Deep learning on point sets for 3D classification and segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), Honolulu. (pp.77–85). https://doi.org/10.1109/CVPR.2017.16
Charles, R. Q., Yi, L., Su, H., & Guibas, L. J. (2017b). PointNet ++: Deep hierarchical feature learning on point sets in a metric space. In Proceedings of the 31st International Conference on Neural Information Processing Systems (NIPS), Long Beach California. (pp.5105–5114). https://doi.org/10.5555/3295222.3295263
Ghadai, S., Balu, A., Sarkar, S., & Krishnamurthy, A. (2018). Learning localized features in 3D CAD models for manufacturability analysis of drilled holes. Computer Aided Geometric Design, 62, 263–275. https://doi.org/10.1016/j.cagd.2018.03.024
Hänel, A., Schnellhardt, T., Schnellhardt, T., Nestler, A., Brosius, A., Corinth, C., Fay, A., & Ihlenfeldt, S. (2020). The development of a digital twin for machining processes for the application in aerospace industry. Procedia CIRP, 93, 1399–1404. https://doi.org/10.1016/j.procir.2020.04.017
Hänel, A., Wenkler, E., Schnellhardt, T., Corinth, C., Brosius, A., Fay, A., & Nestler, A. (2019). Development of a method to determine cutting forces based on planning and process data as contribution for the creation of digital process twins. MM Science Journal. https://doi.org/10.17973/mmsj.2019_11_2019064
Kazemian, A., Yuan, X., Davtalab, O., & Khoshnevis, B. (2019). Computer vision for real-time extrusion quality monitoring and control in robotic construction. Automation in Construction, 101, 92–98. https://doi.org/10.1016/j.autcon.2019.01.022
Leo Kumar, S. P. (2017). Automation of tool path generation in multi-process micromachine tool for micromachining of prismatic and rotational parts. International Journal of Computer Integrated Manufacturing, 31(1), 49–70. https://doi.org/10.1080/0951192x.2017.1356471
Li, J., Zhou, G., & Zhang, C. (2021). Twin process model driven machining quality optimization for complex parts. 2021 International Conference on Digital Society and Intelligent Systems (DSInS), Chengdu, China. (pp.218-222). https://doi.org/10.1109/DSInS54396.2021.9670612
Li, J., Zhou, G., & Zhang, C. (2022). A twin data and knowledge-driven intelligent process planning framework of aviation parts. International Journal of Production Research, 60, 5217–5234. https://doi.org/10.1080/00207543.2021.1951869
Liang, X., Lin, L., Wei, Y., Shen, X., Yang, J., & Yan, S. (2018). Proposal-free network for instance-level object segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 40(12), 2978–2991. https://doi.org/10.1109/TPAMI.2017.2775623
Liu, C., Li, Y., & Li, Z. (2018a). A machining feature definition approach by using two-times unsupervised clustering based on historical data for process knowledge reuse. Journal of Manufacturing Systems, 49, 16–24. https://doi.org/10.1016/j.jmsy.2018.08.003
Liu, J., Liu, X., Cheng, Y., & Ni, Z. (2015). An approach to mapping machining feature to manufacturing feature volume based on geometric reasoning for process planning. Proceedings of the Institution of Mechanical Engineers, Part b: Journal of Engineering Manufacture, 231(7), 1204–1216. https://doi.org/10.1177/0954405415585377
Liu, J., & To, A. C. (2020). Computer-aided design-based topology optimization system with dynamic feature shape and modeling history evolution. Journal of Mechanical Design, 142(7), 071704. https://doi.org/10.1115/1.4045301
Liu, J., Zhou, H., Liu, X., Tian, G., Wu, M., Cao, L., & Wang, W. (2019). Dynamic evaluation method of machining process planning based on digital twin. IEEE Access, 7, 19312–19323. https://doi.org/10.1109/access.2019.2893309
Liu, J., Zhou, H., Tian, G., Liu, X., & Jing, X. (2018b). Digital twin-based process reuse and evaluation approach for smart process planning. The International Journal of Advanced Manufacturing Technology, 100(5–8), 1619–1634. https://doi.org/10.1007/s00170-018-2748-5
Liu, J., Zhao, P., Jing, X., Cao, X., Sheng, S., Zhou, H., Liu, X., & Feng, F. (2021a). Dynamic design method of digital twin process model driven by knowledge-evolution machining features. International Journal of Production Research, 60(7), 2312–2330. https://doi.org/10.1080/00207543.2021.1887531
Liu, S., Bao, J., Lu, Y., Li, J., Lu, S., & Sun, X. (2021a). Digital twin modeling method based on biomimicry for machining aerospace components. Journal of Manufacturing Systems, 58, 180–195. https://doi.org/10.1016/j.jmsy.2020.04.014
Liu, S., Lu, Y., Li, J., Song, D., Sun, X., & Bao, J. (2021b). Multi-scale evolution mechanism and knowledge construction of a digital twin mimic model. Robotics and Computer-Integrated Manufacturing. https://doi.org/10.1016/j.rcim.2021.102123
Liu, Y., Xu, H., Liu, D., & Wang, L. (2022). A digital twin-based sim-to-real transfer for deep reinforcement learning-enabled industrial robot grasping. Robotics and Computer-Integrated Manufacturing. https://doi.org/10.1016/j.rcim.2022.102365
Moretti, M., Rossi, A., & Senin, N. (2021). In-process monitoring of part geometry in fused filament fabrication using computer vision and digital twins. Additive Manufacturing. https://doi.org/10.1016/j.addma.2020.101609
Newman, S. T., Zhu, Z., Dhokia, V., & Shokrani, A. (2015). Process planning for additive and subtractive manufacturing technologies. CIRP Annals, 64(1), 467–470. https://doi.org/10.1016/j.cirp.2015.04.109
Onaji, I., Tiwari, D., Soulatiantork, P., Song, B., & Tiwari, A. (2022). Digital twin in manufacturing: Conceptual framework and case studies. International Journal of Computer Integrated Manufacturing, 35(8), 831–858. https://doi.org/10.1080/0951192x.2022.2027014
Rezaei Aderiani, A., Wärmefjord, K., & Söderberg, R. (2021). Evaluating different strategies to achieve the highest geometric quality in self-adjusting smart assembly lines. Robotics and Computer-Integrated Manufacturing. https://doi.org/10.1016/j.rcim.2021.102164
Riego, V., Castejón-Limas, M., Sánchez-González, L., Fernández-Robles, L., Perez, H., Diez-Gonzalez, J., & Guerrero-Higueras, Á. -M. (2021). Strong classification system for wear identification on milling processes using computer vision and ensemble learning. Neurocomputing, 456, 678–684. https://doi.org/10.1016/j.neucom.2020.07.131
Sarkar, A., & Šormaz, D. (2019). On semantic interoperability of model-based definition of product design. Procedia Manufacturing, 39, 513–523. https://doi.org/10.1016/j.promfg.2020.01.065
Shi, P., Qi, Q., Qin, Y., Scott, P. J., & Jiang, X. (2020). A novel learning-based feature recognition method using multiple sectional view representation. Journal of Intelligent Manufacturing, 31(5), 1291–1309. https://doi.org/10.1007/s10845-020-01533-w
Villalba-Diez, J., Schmidt, D., Gevers, R., Ordieres-Mere, J., Buchwitz, M., & Wellbrock, W. (2019). Deep learning for industrial computer vision quality control in the printing industry 4.0. Sensors (basel). https://doi.org/10.3390/s19183987
Wang, Z., Lai, Z., Li, C., Wang, C., Zheng, Y., Xu, W., Li, M., Gao, Y., Zhang, Z., Cao, Q., Han, X., & Li, L. (2022). Data-driven method for process optimization in electromagnetic-electrohydraulic hybrid high-velocity sheet metal forming. The International Journal of Advanced Manufacturing Technology, 121(7–8), 4355–4365. https://doi.org/10.1007/s00170-022-09621-2
Wenna, W., Weili, D., Changchun, H., Heng, Z., Haibing, F., & Yao, Y. (2022). A digital twin for 3D path planning of large-span curved-arm gantry robot. Robotics and Computer-Integrated Manufacturing. https://doi.org/10.1016/j.rcim.2022.102330
Xu, T., Li, J., & Chen, Z. (2022). Automatic machining feature recognition based on MBD and process semantics. Computers in Industry. https://doi.org/10.1016/j.compind.2022.103736
Ye, Y., Hu, T., Yang, Y., Zhu, W., & Zhang, C. (2018). A knowledge based intelligent process planning method for controller of computer numerical control machine tools. Journal of Intelligent Manufacturing, 31(7), 1751–1767. https://doi.org/10.1007/s10845-018-1401-3
Ye, Y., Hu, T., Zhang, C., & Luo, W. (2016). Design and development of a CNC machining process knowledge base using cloud technology. The International Journal of Advanced Manufacturing Technology, 94(9–12), 3413–3425. https://doi.org/10.1007/s00170-016-9338-1
Yeo, C., Cheon, S., & Mun, D. (2021). Manufacturability evaluation of parts using descriptor-based machining feature recognition. International Journal of Computer Integrated Manufacturing, 34(11), 1196–1222. https://doi.org/10.1080/0951192x.2021.1963483
Yi, Y., Yan, Y., Liu, X., Ni, Z., Feng, J., & Liu, J. (2021). Digital twin-based smart assembly process design and application framework for complex products and its case study. Journal of Manufacturing Systems, 58, 94–107. https://doi.org/10.1016/j.jmsy.2020.04.013
Zhang, C., Zhou, G., Hu, J., & Li, J. (2020). Deep learning-enabled intelligent process planning for digital twin manufacturing cell. Knowledge-Based Systems. https://doi.org/10.1016/j.knosys.2019.105247
Zhang, C., Zhou, G., Jing, Y., Wang, R., & Chang, F. (2022a). A digital twin-based automatic programming method for adaptive control of manufacturing cells. IEEE Access, 10, 80784–80793. https://doi.org/10.1109/access.2022.3195905
Zhang, C., Zhou, G., Li, J., Qin, T., Ding, K., & Chang, F. (2022b). KAiPP: An interaction recommendation approach for knowledge aided intelligent process planning with reinforcement learning. Knowledge-Based Systems. https://doi.org/10.1016/j.knosys.2022.110009
Zhang, C., Zhou, G., Lu, Q., & Chang, F. (2017). Graph-based knowledge reuse for supporting knowledge-driven decision-making in new product development. International Journal of Production Research, 55(23), 7187–7203. https://doi.org/10.1080/00207543.2017.1351643
Zhang, Z., Jaiswal, P., & Rai, R. (2018). FeatureNet: Machining feature recognition based on 3D convolution neural network. Computer-Aided Design, 101, 12–22. https://doi.org/10.1016/j.cad.2018.03.006
Zhao, Z., Wang, S., Wang, Z., Wang, S., Ma, C., & Yang, B. (2020). Surface roughness stabilization method based on digital twin-driven machining parameters self-adaption adjustment: A case study in five-axis machining. Journal of Intelligent Manufacturing, 33(4), 943–952. https://doi.org/10.1007/s10845-020-01698-4
Zheng, Y., Yang, S., & Cheng, H. (2019). An application framework of digital twin and its case study. Journal of Ambient Intelligence and Humanized Computing, 10(3), 1141–1153. https://doi.org/10.1007/s12652-018-0911-3
Zhou, G., Yang, X., Zhang, C., Li, Z., & Xiao, Z. (2019a). Deep learning enabled cutting tool selection for special-shaped machining features of complex products. Advances in Engineering Software, 133, 1–11. https://doi.org/10.1016/j.advengsoft.2019.04.007
Zhou, G., Zhang, C., Li, Z., Ding, K., & Wang, C. (2019b). Knowledge-driven digital twin manufacturing cell towards intelligent manufacturing. International Journal of Production Research, 58(4), 1034–1051. https://doi.org/10.1080/00207543.2019.1607978
Zhou, Z., Zhang, Y., & Foroosh, H. (2021). Panoptic-polarnet: Proposal-free LiDAR point cloud panoptic segmentation. 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition, (pp. 13189–13198). https://doi.org/10.1109/CVPR46437.2021.01299
Acknowledgements
This work was supported by the [National Key Research and Development Program of China] under Grant [No. 2021YFB3301400], and [National Natural Science Foundation of China] under Grant [No. 52105530], and [China National Postdoctoral Program for Innovative Talents] under Grant [No. BX2021244], and [China Postdoctoral Science Foundation] under Grant [No. 2021M692556]. Key Research and Development Project of Shaanxi Province under Grant [No. 2023-ZDLNY-71].
Author information
Authors and Affiliations
Contributions
Jingjing Li: conceptualization, methodology, software, investigation, writing—original draft. Guanghui Zhou: resources, writing—review & editing, supervision. Chao Zhang: resources, writing—review & editing, supervision. Junsheng Hu: validation, formal analysis, software, data curation. Fengtian Chang: writing—review & editing. Andrea Matta: review& editing.
Corresponding author
Ethics declarations
Conflict of interest
The authors declared that they have no conflicts of interest to this work.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Li, J., Zhou, G., Zhang, C. et al. Defining a feature-level digital twin process model by extracting machining features from MBD models for intelligent process planning. J Intell Manuf (2024). https://doi.org/10.1007/s10845-024-02406-2
Received:
Accepted:
Published:
DOI: https://doi.org/10.1007/s10845-024-02406-2