Introduction

Manufacturing is considered one of the most important engineering domains and one of the pillars of the industry (Lasi et al., 2014). Manufacturing intelligence is a key concept in the modernization of the industry and plays a significant role in Industry 4.0 (Zhong et al., 2017). In the context of Industry 4.0, innovative and pioneer companies should incorporate autonomous, smart, sustainable, and resource-efficient manufacturing operations (Ghobakhloo, 2020), leading to operational cost reduction and improvement in machining efficiency.

Specifically, the application of machine learning and deep learning algorithms in manufacturing favors complex manufacturing tasks (Harding et al., 2006). However, the geometrical complexity of the data to be analyzed following the high demand for Quality Assurance/Quality control (QA/QC) parameters is still a tough and difficult challenge (Bia and Wang, 2010; Altıparmak et al., 2021).

In this regard, recently in 3D data analysis, point clouds have become one of the most utilized data types. A point cloud is a collection of data that consists of a large number of points and represents a 3D object. Point clouds are normally represented with the x, y, and z coordinates for each point, although more information may be included such as color intensity or the surface normals of the geometric surface (Wang et al., 2020). Point cloud usage is growing in numerous disciplines, including manufacturing, since additional acquisition technologies, such as LIDARsFootnote 1 or 3D scanners, become accessible (Liang et al., 2021). The acquisition of 3D point cloud data from the aforementioned sensing technologies has great advantages over conventional measurement methods, such as the considerably greater measurement rate and better measurement accuracy (Wang et al., 2020).

However, the lack of specific structure and order in point cloud data, as well as the significantly redundant and inconsistent sample densities, makes it challenging to analyze (Nguyen and Le, 2013). Point clouds are characterized by their unstructured and irregular nature; thus, computational problems are frequently encountered (Guo et al., 2021). Even more, in the manufacturing process, point clouds with high point density are crucial to guarantee the QA/QC parameters. Besides, manufacturing is a dynamic process that produces data at different times generating temporal 3D point clouds (Hou et al., 2016). Therefore, in addition to the spatial dimension, we have also temporality. Depending on the time step, a workpiece in the manufacturing process will be geometrically less or more formed to the final product shape. In similar spatio-temporal intelligent processes, such as autonomous driving, machine, and deep learning algorithms use a wide assortment of data as input to achieve a somehow all-around intelligence (Zhang et al., 2022). Observing the success of such methods in autonomous driving, it is of utmost importance to assure that certain of those methods can also be applied in specific manufacturing tasks, such as the automatic identification of properties in machining processes, utilizing this challenging data type, i.e. 3D point clouds (Ma et al., 2019).

To address these issues, in this paper, we study the application of intelligent techniques in manufacturing utilizing temporal 3D point clouds. Specifically, our contribution is three-fold. First, we propose a novel process and develop guidelines for the task of identification of machining tools while having in possession temporal 3D point cloud data of a workpiece in progress. Additionally, as a second contribution, we provide to the research community two novel 3D point cloud datasets. Finally, we propose an end-to-end intelligent system for machining tool identification using the PointNet (Qi et al., 2017a) neural network. For clarification, in this initial study, even though our data consist of sequences of 3D point clouds, our modeling is based on single instance identification. Thus, we do not fully exploit the temporality of the provided datasets.

The paper is organized as follows. Section 2 portrays an overview of relevant studies and shows certain open challenges in 3D point cloud manufacturing in both industry and academic research. Section 3 displays the research and development of an intelligent machining tool identification framework and Sect. 4 describes our experiments and results. Finally, Sect. 6 summarizes our concluding remarks and provides future research lines.

Related work

Deep learning enables sophisticated analytics capabilities for analyzing large amounts of manufacturing data. A detailed assessment of frequently used deep learning algorithms is presented in (Wang et al., 2018). Also, deep learning is utilized to provide intelligence in specialized manufacturing processes, such as defects detection in products (Yang et al., 2020), fault diagnosis in motors (Shao et al., 2017), manufacturing cost estimation or part machining feature recognition (Ning et al., 2020).

Furthermore, nowadays, specialized research exists on tool monitoring (Nasir and Sassani, 2021). Specifically, researchers apply deep learning techniques to analyze the wear of a tool in high-speed (Zheng and Lin, 2019) while others apply more sophisticated physics-guided intelligent techniques to model the tool’s life (Karandikar et al., 2021). Also, Liu et al. (2020) develop an intelligent prognostic framework for remaining useful life prediction of machining tools under variable conditions improving the health assessment of machining tools by utilizing not only data from multiple sensors but also the spindle load.

Besides, in recent times, the conjunction of deep learning and computer vision methods draws a lot of attention. For instance, researchers focus on downstream tasks utilizing image data, such as feature points recognition of the machining tool path (Hu et al., 2022). Sequential pattern analysis and image processing are also employed. Others use time series of image data (frames) paired with Recurrent Neural Networks (RNNs) and Convolutional Neural Networks (CNNs) in a welding case study (Wang et al., 2021). Finally, a 3D vision system based on image data is developed to identify tool insert specifications, such as insert angles and cutting edge lengths (Ping et al., 2018). All these processes described above produce enormous amounts of data, ranging from time series to 3D shapes (Wang et al., 2018, 2021).

The analysis of 3D shapes in the manufacturing domain is mostly focused on using 3D CAD models instead of images because this type of data carries implicit accurate geometrical information. However, format translation from CAD files is essential because they cannot directly deliver volumetric information to a deep learning model (Peddireddy et al., 2020). Also, there are fewer studies, such as (Feng et al., 2019), that use mesh processing directly. However, meshes contain certain complexity and irregularity, and, thus, the majority of the current research on manufacturing uses mainly 3D point clouds and voxel-based data as inputs. It is worth mentioning that Cao et al. (2020) proposed a graph representation of 3D CAD models, which is more efficient than voxels for deep learning algorithms.

Point cloud data appear in today’s literature in studies that deal with additive manufacturing, i.e. 3D printing guidance (Ye et al., 2020), or automated bridge component recognition, i.e. damage observation in the bridges’ maintenance (Kim et al., 2020), among others. Also, Ma et al. (2019) use 3D point cloud data of part models with point-based neural networks and CNNs to provide automatic recognition of machining features, such as ring slot, triangular passage, etc. Recently, 3D point clouds are also utilized as input to develop efficient and effective detection of surface defects for remanufacturing (He et al., 2022).

However, complex analysis in manufacturing operations using point clouds implies certain challenges, which mainly belong to the heavy industry (Kumar et al., 2018; Moon et al., 2019). Machining manufacturing thrives for complex modeling in its operations due to the need for high-performance computations, which are resource hungry and as a result, their efficiency is more than a need (He et al., 2012). Machining is the removal of material from a workpiece in order to bring it into the desired shape. To perform machining operations on the workpiece a variety of specialized tools are used, such as drilling, milling, and turning tools. Initially, the workpiece is usually a piece of raw material or an existing product, and the machine setup is dictated by the type of material and the amount of material to be removed. During this process, machine tool accessories, precision measurement equipment, and advanced mathematical methods are utilized to ensure and guarantee optimal machining results (Ehmann et al., 1997).

Monitoring a workpiece in progress during a machining task is not a new concept and it aims to enhance the agility in manufacturing (Sun and Jiang, 2008). In complex machining tasks, such as turn-milling, the classification of the machining tools and their properties by understanding the impact of cutting angles and forces is critical. In such machining scenarios, the identification of the optimal tool and its posture could lead to high-efficiency machining and improve the life of the tools. During a machining process, Utsumi et al. (2020) model time series of cutting forces to produce a final point cloud of a workpiece. They represent a workpiece with a point cloud, with any points that interfere with the tool volume being eliminated when the tool edge passes over the workpiece’s surface, i.e. the swept volume. Besides, forecasting the swept volume of a workpiece during a machining process can be a key aid in diminishing the tool wear and making the necessary tool change during the machining process. Wiederkehr et al. (2018) developed stochastic modeling of tool wear in a grinding simulation utilizing point clouds of the swept volume. Finally, the analysis and prediction of the 3D surface topography of a workpiece in progress and its 3D shape evolution are considered a need in auto, aviation, and other industries that have high-performance requirements. However, in prediction tasks that involve five-axis machine tool processing, the process of machining becomes way more challenging due to the time-varying tool locations and orientations (Wang et al., 2020).

The main source of challenges and issues in the research of point clouds is the dimensionality of the data and especially the number of points contained in a point cloud. The data analysis and the ways to pre-process them require certain high computational power and expertise. The majority of the point-based neural network architectures, such as PointNet (Qi et al., 2017a), PointNet++ (Qi et al., 2017b), RSConv (Liu et al., 2019), PointMLP (Ma et al., 2022) operate in most cases with small in size point clouds, i.e. the number of points in each point cloud are normally 1024, 2048 or 4096. However, in real machining scenarios and even machining simulations, the extracted point clouds contain thousands or millions of points. Therefore, there are two ways of analyzing such data: (i) suitable oversampling or undersampling techniques should be considered depending on the size of extracted simulated data, and/or (ii) the input size scalability of the applied neural networks should be enhanced. Indeed, sampling and representation learning on 3D point clouds are challenging because of the irregular and unstructured nature of such data.

In summary, according to our analysis of the existing literature, we have identified certain needs and challenges of the current research on the intersection of temporal 3D point clouds extracted through machining processes and the modeling of them with deep learning algorithms. This fact leads us to perform a series of analysis tasks, provide meaningful insights and develop an end-to-end intelligent system in this field. There is a gap in tool identification methods and especially when working with temporal point clouds, i.e. point clouds extracted in time intervals with uniform or non-uniform sampling forming a sequence of temporal 3D point clouds. While the majority of the studies in the domain, focus on side data of the workpiece or the machining tool, such as cutting forces or other properties, to later form a final point cloud through modeling Utsumi et al. (2020), we focus directly on the point clouds of the workpiece and tools. We aim to provide end-to-end intelligence in such machining scenarios emphasizing the point clouds sampled through the whole machining process in different time steps.

The study of Ping et al. (2018) is somehow related to our study. However, they provide a solution to the problem of misplacement of tool inserts in machining tools with turning capability. Their proposed framework is able to identify insert angles, cutting edge lengths, and nose radii utilizing image data and not on 3D point clouds. Instead, we utilize 3D point clouds of the surface of an in-machining process workpiece and identify the tool that was utilized to machine its surface. Our work provides novelty in the analysis of manufacturing-related 3D point clouds. Machining tool identification may lead to the reverse engineering of workpieces. Additionally, incorporating a tool identification model into a machining decision support system may provide extra insights to avoid defects or resource waste due to non-optimal tool selection.

At the time of writing, there are no exact relative studies or well-known frameworks able to provide concrete end-to-end results and insights in the task of tool identification based on temporal 3D point cloud data of a workpiece in progress. Finally, the forecasting of the shape of the workpiece in progress and the swept volume during the machining process are two open issues of utmost importance for the manufacturing industry.

Case study: machining tool identification

This section details the designed machining tool identification case study. First, Sect. 3.1 defines the problem. Second, Sect. 3.2 details the overall process of machining tool identification. Third, Sect. 3.3 describes the complete data preparation and pre-processing steps to construct temporal 3D point clouds from a workpiece in progress. Fourth, Sect. 3.4 defines the modeling process, where the goal is to effectively classify the input 3D point clouds to identify the most utilized machining tool that shaped the workpiece. Finally, Sect. 3.5 shows the final intelligent system to identify machining tools.

Problem statement

Normally, the machining process starts with a workpiece of raw material in which different tools can be used to achieve the final product shape. In Fig. 1, we illustrate examples of the machining process of a raw material utilizing two different tools. In our case, we assume that all the present elements in a machining process, i.e. workpiece and machining tools, are modeled through a sampling process with 3D point clouds at different steps in time, resulting in temporal point clouds. In Eqs. 1 and 2, we formulate a point cloud and a series of temporal point clouds per machining tool respectively. Also, the total number of temporal point clouds per machining tool is formulated and explained in Eq. 3.

$$\begin{aligned} pc^{tool_i} = \bigg \{k_1, \dots , k_{npoints}\bigg \} \end{aligned}$$
(1)

where \(pc^{tool_i}\) denotes a point cloud of a workpiece shaped by a machining tool, \(tool_i\), and \(k_1, \dots , k_{npoints}\) represent the points contained in it, with npoints showing the maximum number of points.

$$\begin{aligned} p^{tool_i} = \bigcup _{t=0}^{T} pc_t^{tool_i} = \bigg \{ pc_0^{tool_i}, \dots , pc_T^{tool_i} \bigg \} \end{aligned}$$
(2)

where \(p^{tool_i}\) denotes a series of temporal point clouds (i.e. multiple temporal \(pc^{tool_i}\)) of a workpiece utilizing the machining tool, \(tool_i\). The union \(\bigcup _{t=0}^{T}\) includes all the point clouds from the time step \(t=0\) to \(t=T\). The T denotes the final time step of sampling.

$$\begin{aligned} P = \bigcup _{i=1}^{n} p^{tool_i} = \bigg \{p^{tool_1}, \dots , p^{tool_n} \bigg \} \end{aligned}$$
(3)

where P shows the total number of temporal point clouds obtained by utilizing all the available machining tools (n) and it represents the complete dataset. The union \(\bigcup _{i=1}^{n}\) includes all the temporal point clouds that belong to each tool ranging from the \(tool_1\) to \(tool_n\).

Fig. 1
figure 1

A machining process of a workpiece using two different machining tools (The figures were downloaded from https://www.grainger.com/category/machining. No copyright infringement intended)

Full size image

For completion, we also show three examples of temporal point clouds of a workpiece in progress in Fig. 2. Then, given a product (workpiece), the aim is to identify the machining tool with which it has been created. For this purpose, we utilize as input in our intelligent process these temporal point clouds of the workpiece, capturing all the machining process evolution, from the first step to the final product workpiece.

Fig. 2
figure 2

Temporal point clouds of the workpiece in progress. In this example, we show the application of tool 4 in three different time steps, \(t=0\), \(t=150\), \(t=300\) (from left to right) illustrating the \(pc_0^{tool_4}\), \(pc_{150}^{tool_4}\), \(pc_{300}^{tool_4}\) respectively. Please note that each point cloud has 1024 points, i.e. \(pc^{tool_4} = \bigg \{p1,\dots ,p1024\bigg \}\)

Full size image
Fig. 3
figure 3

Process of the intelligent tool identification framework

Full size image

Overall process

In this section, we describe the overall process of machining tool identification utilizing point clouds. An abstract overview of the development of an intelligent framework to identify machining tools is portrayed in Fig. 3. Actually, the knowledge in the intelligent system, depicted in Fig. 3, is provided by a model that is trained in a set of temporal workpieces resulting from different machining tools. In the first stage, namely data preparation and pre-processing, a set of temporal point clouds of the workpiece shaped with different tools is built. These temporal workpieces are categorized according to the machining tool that is used to remove raw material during the shaping process into the desired form. In the learning process, a deep learning model is trained to classify a temporal workpiece according to n different tool labels, where n denotes the number of available tools. Afterward, this trained model will be used by the intelligent system to classify unknown temporal workpieces, predicting with certain accuracy the machining tool that shaped the workpiece. It should be noted that, initially, we extract temporal 3D point clouds during a machining process of a workpiece with several tools. Then, the temporal point clouds of a workpiece are categorized and labeled according to the machining tool that was used during shaping.

Fig. 4
figure 4

Illustration of machining process and 3D point cloud sampling

Full size image

Data preparation and pre-processing

In our study, we acquire data through machining simulation software, which uses an implicit surface representation during geometric modeling. Then, in order to extract temporal 3D point clouds of the in-process workpiece geometry, a scanline-based approach is used for point sampling of the implicit surface representation, as displayed in Figs. 4 and 5.

The machining process is illustrated in four steps in Fig. 4. Steps (a), (b), and (c) refer to the machining simulation and the actions that take place before the process of data preparation. In the first step (a) we have the initial 3D mesh workpiece. Then we apply a machining tool, where the swept volume of a moving machining tool is depicted in yellow in (b) and we observe the material removal on the resulting workpiece in step (c). Finally, in step (d) we show the current tool engagement, where we apply a sampling process in the exact region in which the machining tool removed the material of the workpiece. Indeed, the data preparation of our proposal takes place in step (d).

Fig. 5
figure 5

3D point cloud sampling

Full size image

Once we obtain the region to be rasterized (Fig. 4 (d)), we proceed with the next process. Initially, three scanline directions in a geometrical box are used to scan the x, y, and z coordinated space. The scanlines are used to sample the implicit surface representation of the current workpiece geometry. Figure 5 details the sampling process according to which we acquired the 3D point clouds. To achieve a fixed and predefined number of sampled points, a recursive subdivision of scanlines using a breadth-first strategy is utilized. The levels of scanlines denote the recursive subdivision. The scanning process stops when the defined number of sampling points is acquired. The scanning takes place in different time steps during the machining process. It is worth noticing that, the sampled points acquired during each scanning dimension are merged together to form the final point cloud. Please note that, during the whole process, we sample the entry points of the implicit surface, or alternatively the first contact points, in which a scanline is passing through.

Fig. 6
figure 6

Point clouds obtained by the application of different tools on the workpiece in progress

Full size image
Fig. 7
figure 7

\(PointNet_{+1L}\) Architecture. We have added a layer of (1024, 1024) on the PointNet’s classification head (Qi et al., 2017a)

Full size image
Fig. 8
figure 8

\(PointNet_{l}^{v1}\) Architecture. We have decreased the number of neurons per layer on PointNet’s (Qi et al., 2017a) architecture to 1/4, both in the back-end feature extraction layers and classification head

Full size image
Table 1 Description of the two introduced point cloud datasets
Full size table

Therefore, by following the above-mentioned sampling process, we acquire two 3D point cloud datasets, taking into account different scanning resolutions. The first one, called 16 Tools Small Dataset, consists of point cloud objects having a fixed amount of sampled points (1024) in each point cloud. It contains 324 sequential (in time) temporal point cloud objects of the workpiece in each tool label while having in total 16 different tools (\(n=16\)). The total number of point clouds in 16 Tools Small Dataset is 5184. The second dataset, called 16 Tools Large Dataset, is obtained by adjusting the sampling density of the sampling process, which similarly to the 16 Tools Small Dataset consists of point cloud objects having a fixed amount of sampled points (1024) in each point cloud. However, in this case, the sampling process is done in a smaller region of the workpiece point cloud, which corresponds to a more compact geometrical sampling box. This region of points corresponds to the exact contact area of contact between the tool and the workpiece in progress, i.e. the bounding box of the scanning is more compact and focuses on a smaller region. This dataset contains 810 temporal point cloud objects for each tool label while having in total 16 different tools. The total number of point clouds in 16 Tools Large Dataset is 12960. In practice, the geometrical sampling box in the creation of 16 Tools Small Dataset covers a larger area of the implicit surface of the workpiece than the one in 16 Tools Large Dataset. We summarize the details of the two datasets in Table 1. Also, we portray example point clouds obtained by applying different machining tools on the workpiece in progress in Fig. 6.

For clarification purposes, we also calculate the maximum distance of each point from all the other points inside a point cloud in each dataset using the generalization of the Euclidean distance in 3D space. Thus, in 16 Tools Large Dataset the maximum distance of a point from its neighbors is approximately 1.77, while in 16 Tools Small Dataset it is approximately 1.86, showing that the point clouds contained in 16 Tools Small Dataset capture a larger area of the current tool engagement with 1024 points, while the ones contained in 16 Tools Large Dataset capture a smaller one with the same number of points (1024).

Fig. 9
figure 9

\(PointNet_{l}^{v2}\) Architecture. We have decreased the number of neurons per layer on PointNet’s (Qi et al., 2017a) architecture to 1/4 in the classification head while keeping the default backend feature extraction layers

Full size image

Modelling

During the learning process, depicted in Fig. 3, we train intelligent models to identify the tools that have been used in the machining process utilizing the temporal point clouds that we previously extracted from a workpiece, as explained in Sect. 3.2. We analyze the task of tool identification with four point-based deep learning models, the PointNet (Qi et al., 2017a) and three variants of it. These point-based neural networks are based on the pioneering deep learning architecture, PointNet. PointNet is a fundamental deep learning architecture that enables the direct analysis of point cloud instances. Also, we construct three variants of PointNet architecture in order to exploit its full capabilities in identifying underlying point cloud data relations and to create a baseline method for machining tool identification using 3D point clouds.

We decided to fully exploit the capabilities of the well-known point-based neural network architecture, PointNet, in order to create a baseline for future analyses, by modifying it in three ways: (i) extending its classification head by adding a linear layer, (ii) simplifying it by reducing the number of neurons in each layer in both backend feature extraction layers and classification head layers, (iii) simplifying it by reducing the number of neurons in each layer of the classification head but keeping the default backend feature extraction layers.

Firstly, we increase the complexity and the number of parameters in the default PointNet architecture in order to identify highly complex relations between the point cloud data. Thus, we experiment with the addition of a linear layer with an input and output size equal to the length of the global feature vector in PointNet architecture, i.e 1024. We name \(PointNet_{+1L}\) the neural network that includes this additional layer on top of PointNet and is depicted in Fig. 7. By increasing the network’s complexity, we expect to observe higher classification accuracy than the default architecture in the presence of highly complex underlying relations in the input point clouds.

Secondly, to reveal the effectiveness of a simpler neural network, we experiment with less complex PointNet architectures having fewer parameters than the default one, namely \(PointNet_{l}^{v1}\) and \(PointNet_{l}^{v2}\). In \(PointNet_{l}^{v1}\), we decrease the neurons in each layer (both backend and classification head) to 1/4 compared to the default PointNet architecture. In \(PointNet_{l}^{v2}\), we decrease the neurons in each layer of the classification head to 1/4 compared to the default PointNet architecture and keep the default backend feature extraction. Figures 8 and 9 display \(PointNet_{l}^{v1}\) and \(PointNet_{l}^{v2}\) respectively.

Final system

Finally, after the learning process, as shown in Fig. 3, an intelligent system is constructed by selecting the best-trained model. Figure 10 shows an example run of the intelligent system, where we have the input of a point cloud into the system. Then, the system utilizes a pre-trained neural network model to identify the machining tool that was used to shape the input workpiece.

Fig. 10
figure 10

Prototype intelligent system for tool identification. The input point cloud and output classification label of the intelligent system are examples

Full size image

Experiments

We detail our experimental setup in Sect. 4.1, and discuss our results in Sect. 4.2.

Table 2 Model comparisons in the test set of the two 16 Tools datasets
Full size table

Experimental setup

Here, we describe our experimental investigation on the development of an intelligent system capable of identifying machining tools with the extracted temporal 3D point clouds of a workpiece in progress.

Technically, we use Python 3.8.10, CUDA 10.2, and Pytorch 1.8.1 version. Our system configuration includes an Intel Core i9-10900 paired with 32GB of RAM and a Quadro RTX 5000 with 16 GB and the operating system is Ubuntu 20.04.

We utilize a standard protocol for training and testing the neural network models. Regarding the training and testing with the 16 Tools Small Dataset, we utilize 4672 temporal point clouds of a workpiece for training and 512 for testing. Concerning the training with 16 Tools Large Dataset, we utilize 11661 temporal point clouds for training and 1295 for testing. In both cases, 90% of the total point clouds is used for training and 10% for testing. Also, to assess the variability in our models we run the training of the models 4 times and we report the standard deviation of the results.

Moreover, although our data collection consists of artificial/synthetic data, we train the selected neural networks with certain data augmentation techniques in order to robustify their performance. Specifically, during training, we apply input noise in each point cloud, scaling, and random point dropout. Additionally, we train all the networks for 200 epochs utilizing the Adam optimizer with exponential learning rate decay and an initial learning rate of 0.001. Finally, we experiment with different batch size values (8, 16, 32, and 64).

Results

In this section, we show the results of our experiments in Table 2 and describe them onwards. As detailed in Sect. 4.1, we investigate different batch sizes for training the four neural network models, PointNet, \(PointNet_{+1L}\), \(PointNet_{l}^{v1}\) and \(PointNet_{l}^{v2}\). Besides, we run the training process for each model 4 times to estimate the models’ variability, and we include the mean learning curves, i.e. training and testing loss and accuracy curves, in the Supplementary material. Please recall that the models are trained with input noise in each point cloud, scaling, and random point dropout, which clearly enhances the generalizability of the models.

Also, as efficiency is a need in machining manufacturing systems (He et al., 2012), we report the time for a single instance inference (i.e. \(1 \times [1024, 3]\)) and the number of parameters of each model in Table 3. Please note that measuring inference time is closely related to the system setup, which is described previously in Sect. 4.1.

It can be observed that in the 16 Tools Small Dataset, the two best models are the ones that use fewer parameters, i.e. the most efficient ones (Table 3). \(PointNet_{l}^{v2}\) achieves a high classification accuracy of 0.77 with a batch size of 8. Also, both \(PointNet_{l}^{v2}\) and the most efficient model \(PointNet_{l}^{v1}\) achieve the second-best score of 0.68 with batch sizes equal to 32 and 16 respectively. However, \(PointNet_{l}^{v2}\) has a lower standard deviation in its best results, meaning that it has a lower variability in its predictions.

On the other hand, in 16 Tools Large Dataset, we observe that \(PointNet_{l}^{v2}\) using a batch size of 16 achieves the best accuracy of 0.95, with a very low standard deviation of 0.01, denoting that it has consistency in its best accuracy results in the 4 training sessions. PointNet achieves the second-best accuracy with a score of 0.84 with a batch size of 32. In fact, in both datasets, large batch sizes do not achieve high accuracy.

It should be noted that the accuracy scores of \(PointNet_{l}^{v1}\) and \(PointNet_{l}^{v2}\) in both datasets, reveal two promising models in terms of machining tool identification as the selected model for a tool identification system should also be highly efficient in its utilized resources. \(PointNet_{l}^{v1}\) has 0.17 M parameters while \(PointNet_{l}^{v2}\) has 2.94 M. Also, their inference times are lower than the rest of the models.

Consequently, due to its general high accuracy in both datasets and its efficiency, \(PointNet_{l}^{v2}\) is utilized to serve as a classifier of tools in our intelligent system for machining tool identification, presented in Fig. 3. However, the model selection could be changed according to the expert’s needs.

Table 3 Time and space efficiency of the utilized neural networks (NNs)
Full size table

Limitations and discussion

Machining tool identification based on 3D point clouds of a workpiece in-process could lead to reverse engineering of a final product shape and reduce material waste due to non-optimal tool selection. The purpose of this research is to provide a baseline framework and a prototype end-to-end intelligent system of machining tool identification based on 3D point cloud representation learning. Point-based neural networks have shown great potential in a wide variety of artificial and real datasets and successfully applied in other domains (Guo et al., 2021). In this study, we develop our framework with synthetic/artificial data due to non-publicly available real data for such tasks. We are aware that in a real machining environment the acquired 3D point cloud of the area by a LIDAR scanner will be highly saturated from noise and probably we will observe degraded accuracy performance of the trained neural network. In this case, data pre-processing is essential and should receive high attention. Noise filtering, outlier removal, and context-aware sampling methods should be employed prior to modeling. However, in an attempt to handle such issues, we apply data augmentation during training. Specifically, as mentioned in Sect. 4.1, we apply input noise in each point cloud, scaling, and random point dropout. Augmenting the input data during training is beneficial for a model to generalize better. It may strengthen its robustness to noise, outlier points, and rigid transformations to achieve similar performance in a real scenario. In the near future, we plan to utilize point-based neural networks to analyze 3D point clouds captured in the wild from real laser sensors. Finally, even though our data consist of sequences of 3D point clouds, our modeling is based on single instance identification and thus, we do not fully exploit the temporality of the provided datasets. For this, a future work direction of utmost importance is spatio-temporal forecasting of multiple machining tools using sequences of temporal 3D point clouds of a workpiece. Also, as an extension, potential research directions imply forecasting the whole shape of a workpiece in progress.

Conclusion

The analysis of point cloud data originating from manufacturing simulations implies certain challenges and research opportunities. In this paper, we propose a novel process and advance guidelines in the task of identification of machining tools while possessing temporal 3D point cloud data of a workpiece in progress. In addition, we provide to the research community two datasets, the 16 Tools Small Dataset and 16 Tools Large Dataset, both of them containing sequences of temporal 3D point clouds. The two datasets can be found in this data repository: https://doi.org/10.34810/data205. Finally, we propose a prototype intelligent system for machining tool identification using PointNet neural network, which is able to analyze the input 3D point clouds of a workpiece in progress and identify the utilized machining tool with high accuracy. Our research could be a source of interest for industrial companies and research institutes. The code associated with this study is available in the following GitHub repository: https://github.com/thzou/machining_tools_identification.