1 Introduction

In the era of transition of manufacturing towards mass customization [1] and mass personalization [2], there is a need to inspect each manufactured product, preferably through automated online inspection systems. Application of automated inspection systems is one of the key goals of manufacturing industry in transition to Industry 4.0 [3, 4]. Specifically, automated surface inspection systems require inputs of detailed digital representation of physical products to classify as within or outside specifications.

In recent years, the concept of Skin Model, and its operationalization through Skin Model Shapes, has been proposed as representation of non-ideal surfaces [5] and digital twin of manufactured parts [6]. Skin Model Shapes have applications in product development, manufacturing, and inspection [7] and in linking the associated tolerance-related interactions among these applications [8]. A single digital representation of a product can be used to update assumed geometric characteristics and perform tolerance analysis and finite element analysis [9]. Skin Model Shapes (hereafter SMS) are obtained by discretizing nominal CAD model into finer meshes or from tactile and optical devices in the form of point clouds. The discrete representation scheme enables capturing a more detailed geometric characteristic of manufacturing parts. The central focus of SMS is inclusion of form errors in Computer Aided Tolerancing, while conforming to Geometrical Product Specifications (GPS) standards. The classical methods of conformance to specification have been to fit primitive shapes that capture variation of simple shapes only.

Furthermore, for more complex shapes, like free-form shapes, primitive shape-based tolerance inspection cannot be applied [10]. As product profile becomes more complex, setting up an appropriate threshold, like a profile envelope, to identify unfamiliar variation within tolerance becomes a difficult task. This is one of the reasons that the manufacturing industry still applies human inspectors to perform implicit inspection to detect previously unseen but obviously defective feature characteristics such as poor finishes, dents, wrong orientations, and many other ill-defined faults [11]. However, relative to human inspectors, many automated inspection systems perform better in terms of accuracy, consistency and inspection speed, and the need to set a buffer for inspection, as well as reducing labor costs, e.g., [10,11,12,13,14]. The advantage of automated inspection system becomes more visible when the number of to be inspected features increases.

Nonetheless, the random scattering of form errors, difficulty of setting up appropriate thresholds for unfamiliar surface variations, and possible presence of previously unseen errors make it difficult to formulate exhaustive rules that can automatically capture the anomalous geometric characteristics. Hence, as an alternative, heuristic approaches can be applied [11].

One of the heuristic approaches specifically aimed at detecting anomalous behavior is the concept of anomaly detection. Anomaly detection falls within general purpose of inspection, which is to determine if a product deviates from a set of specifications [12] and obtain a pass/fail information [11]. In terms of machine learning approaches, anomaly detection refers to the process of finding patterns in data that do not conform to expected behavior [15]. In this paper, anomaly detection is applied in detecting deviations from the expected normal behavior, that is, the surface deviations of parts produced in a stable manufacturing line.

Specifically, surface variations display unique characteristics depending on whether they have systematic or random variations. Systematic variations tend to show wide, skewed, and multimodal distributions, while random variations exhibit narrow and normal Gaussian distribution. Hence, the classical machine learning classifiers can be applied to compare and categorize those distribution into normal (random variation) and anomalous (systematic variation) classes depending on the similarity to a baseline distribution. In this regard, the novelty of this paper lies in the methodology of detecting unfamiliar surface variation independent of shape complexity. The work extends the application of SMSs in quality control and automated inspection systems.

This paper is organized as follows. Section 2 presents related works. Section 3 introduces a methodology. Sections 4 through 6 introduce the concepts of surface characterization, deviation distributions, and supervised anomaly detection approaches. Section 7 presents an illustration case. Finally, Sections 8 and 9 present discussion and conclusion of the results of the proposed approach.

2 Related work

There is an extended interest in automated inspection systems that stems from the need to inspect 100% of produced parts and avoid statistical sampling approach. One of the commonly applied inspection systems is image based [16]. However, image-based approaches lack 3D representation and associated dimensional inspection capability.

Alternatively, point cloud data can be acquired from laser and structured light scanners that can capture 3D geometric information. Data acquisition methods for free-form have been discussed in [10]. Specifically, point cloud data-based identification has been applied in aircraft panel dent detection [17] and in engine surface inspection [18]. Scanned data has also been applied in surface defect detection through visual inspection of color difference in [19, 20]. However, these are manual-visual inspection approaches, which are relatively inefficient.

Recent efforts towards automated inspection system based on point cloud data include defect detection by dividing selectively growing region of interest and spatiotemporal control charting approach [14]. A similar approach in detecting perforated sheet metals based on machine learning techniques has been presented in [21]. Such approaches specifically focus on surface defect detection independent of process distribution.

Moreover, in an online inspection, change detection of process distributions is difficult to monitor using statistical process control methods, as the number of samples is significantly smaller than the number of surface points to extract an accurate process distribution [22]. Point cloud-based change detection approaches focus computing distance between two registered point clouds, e.g., [19, 23]. Nevertheless, comparison between two point clouds alone is insufficient to extract the manufacturing process trend. Explicit specification can be set using Geometric Dimensioning and Tolerancing (GD&T) scheme where a set of geometric limits is applied to control point deviations [24]. However, such approaches are not aimed at detecting process changes and surface defects within tolerance.

Alternatively, anomaly detection algorithms can be applied. Most of the anomaly detection approaches focus in intrusion detection, fraud detection, and mechanical fault detection [15]. The choice of these approaches depends on the set of attribute that describe data [15]. Moreover, some of the approaches use distance metric learning techniques, like learning from Mahalanobis distance, which has been applied in handwritten digit and face identification [25, 26], information retrieval and bioinformatics [27], and anomaly detection in network traffic [28]. In manufacturing, anomaly detection using a machine learning approach utilizing manufacturing inspection, and aftersales data to detect anomalous engines has been applied [29]. However, an anomaly detection approach aimed towards automated inspection of geometric and dimensional change in complex parts has not been extensively studied.

3 Methodology

The methodology reported herein attempts to detect anomalous geometric and dimensional deviations in SMSs of products produced in a stable production line. Figure 1 summarizes the proposed methodology. The methodology follows the classical supervised machine learning approaches. However, since machine learning approaches are data intensive, producing parts with different types of errors for the mere purpose of teaching a classifier is not practical. To overcome this drawback, point cloud data is generated with techniques and principles applied in prediction and simulation of SMSs, as reported in [7]. To generate an SMS, a nominal model is discretized into a mesh and manipulated so that the surface closely represents a part.

Fig. 1
figure 1

Methodology for detecting anomaly based on synthetic and real data

Full size image

Specifically, in this paper, five truck part samples from an industrial production line were first scanned and a region of interest was extracted from the point cloud data. The extracted point cloud is fitted with corresponding discretized nominal feature to acquire the orientation and positions of the scanned feature. The nominal feature has vertices approximately close to number of points of the scanned feature. Then waviness is added to the discretized feature by adding Gaussian random fields. Since each part is unique in terms of geometric characteristics, multiple SMSs are generated that closely match the orientation, position, and waviness of each scanned sample.

The generated SMS deviations from nominal model are estimated by computing Euclidian distance or Hausdorff distance [23]. The distance values are then organized in histograms, which exhibit different distribution profiles when they are from a defective part compared to a normal part. To estimate the extent to which the two histograms are different, Mahalanobis distance from a baseline histogram is computed.

Moreover, as the deviation profile changes, the mean, mode, and width of the histograms may change, which are constructed as features (hereafter predictors). Thus, using the four indicators as a machine learning feature vector (hereafter predictor vector), the standard class of classifiers is trained. The training and validation data set is acquired from generated SMSs and the test sets are obtained from a scanned model. When anomaly is detected, an alarm is raised, or the part is redirected to a more detailed inspection.

4 Surface characterization and extraction

4.1 Surface characterization

This work applies machine learning classifiers to detect unfamiliar surface variation of a part produced in a stable manufacturing line. However, machine learning models require a lot of data, with normal and anomalous class, to make accurate predictions. Producing physical parts with defects to just train a model is expensive and difficult. Furthermore, the trained model would be biased towards the normal behavior when excessive normal data, and fewer anomalous data is used, as the class imbalance reduces performance of the classical classifiers [30, 31]. The alternative is to generate synthetic data that closely represents a manufactured part through simulation. The synthetic data is then used to train a classifier and validate the classification results, and the real data to test the trained classifier.

To this end, surface characteristics that represent physical products can be generated through point deviation, displacement, and orientation, in line with SMS prediction and manipulation techniques discussed in [7]. ISO standard indicate that product geometric errors can be expressed as superimposition of lay, waviness, and roughness [32]. Surface roughness has a limited effect on the functional effectiveness and tolerance of a part [33]. The lay of the surface generally includes position and orientation errors, which can collectively be treated as systematic variation. The surface waviness is considered as set of form errors, which are randomly distributed along the surface at different sizes and shapes. The larger form errors have low waviness frequency and the smaller ones have high frequency. The low frequency form errors can be considered as systematic variations and the high frequency ones as the random variations.

To generate a systematic variation, a discretized nominal model is rotated and displaced, points of Gaussian random fields with large deviations are added, or a combination of all. To represent the random behavior of form errors, Gaussian random fields can be applied, by changing the correlation and scale of point deviation [7]. For equal number of points with the same x-y values, the z value can be added. For more complex shapes, interpolation methods can be applied [34]. Figure 2 shows the summation of a nominal shape with orientation error and generated using Gaussian random fields with equal number of points. The sum of data points can be referred as a digital twin of the physical feature. Since optical devices provide detailed geometrical, dimensional, and surface variation information, the derived vertices of SMSs from a point cloud data are theoretically the same. Arguably, the derived vertices can be used to closely approximate the physical part.

Fig. 2
figure 2

Summation of systematic deviation obtained from scanned data and Gaussian random fields. a Systematic deviation. b Random deviation. c Total deviation

Full size image

4.2 Procrustes registration

In the context of SMS, surface variations are captured using optical or tactile devices, whereby point cloud deviations from nominal model are used for further analysis. The scanned models can be repositioned through matching and registration steps of Generalized Procrustes Algorithm. A Procrustes-based registration matches a corresponding feature’s points, aka landmarks, by minimizing the distance between the points [35]. Specifically, these landmarks should be set where the fixtures are expected to come in contact so that the manufacturing error is captured in reference to those landmarks. As shown in Fig. 3, the manufacturing errors are pushed to the other end relative to the landmark. Mathematically, the Procrustes distance PD is estimated by,

$$ \boldsymbol{PD}\left(\boldsymbol{A},\boldsymbol{B}\right)=\sqrt{\sum {\left(\boldsymbol{A}-\boldsymbol{B}\right)}^2} $$
(1)
Fig. 3
figure 3

An illustration of an effect of registration at landmarks

Full size image

where A = {a1, …an} and B = {b1, …bn} are corresponding landmarks with n points.

Moreover, only few features are likely to be the objects of interest for inspection and analysis. Hence, the features of interest must be extracted from the part, which could follow different approaches of segmentation, e.g., [36]. Alternatively, for known poses of a part, partitioning regions of interest of the scanned models can be performed by including only the coordinates that cover areas of interest.

4.3 Hausdorff distance

The deviation of data point to nominal model can be estimated by Euclidean distance. However, there may not exist one to one correspondence between point clouds and nominal modal or between two-point clouds. Alternatively, the difference between nominal model and an SMS or between two SMSs can be estimated using Hausdorff distance [37]. Mathematically, for reference SMS vertices or point cloud A = {a1,  … , an} and observed point cloud B = {b1, …bn}, the Hausdorff distance is computed by:

$$ \boldsymbol{HD}\left(\boldsymbol{A},\mathbf{B}\right)=\underset{a\epsilon A}{\max}\underset{b\epsilon B}{\min}\parallel \boldsymbol{A}-\mathbf{B}\parallel $$
(2)

4.4 Geometric tolerances

The SMSs’ variation can be controlled explicitly through tolerance schema. According to ASME/ANSI standards’ GD&T schema, the variations are grouped into location, orientation and form errors, profile, and runout tolerances [32]. Location tolerance controls the variation the allowable displacement in terms of a position, concentricity, and symmetry tolerance. Orientational error is controlled by in angularity, parallelism, and perpendicularity tolerances [24, 32]. Angularity controls the angle of a toleranced feature from a datum feature. When the referenced feature is vertical or horizontal relative to the datum feature, perpendicularity or parallelism tolerances are applied, respectively. Form errors, waviness of surfaces, can be controlled by planarity or straightness, by placing two parallel planes or lines, respectively. Cylindricity tolerances also control waviness of the surface, by placing the surface between two concentric cylinders. The radial difference between the two cylinders is the measure of cylindricity [24, 32].

4.5 Free-form surfaces

The GD&T-based tolerance control methods have been applied as a de facto standard in quality control industry. However, these methods are limited to utilization of primitive shapes and cannot be applied to free-form surfaces [10], where products have a complex profile. These shapes could be shapes produced by additive manufacturing, similar to the shapes shown in Fig. 4. In such cases, inspection through geometric tolerance specifications is difficult to apply in controlling the anomalous variation. Instead, the inspection system can learn what normal variation is and what anomalous variation from deviation distributions is.

Fig. 4
figure 4

Point cloud of MathWorks® logo with induced systematic and random deviations

Full size image

5 Deviation distributions and similarities

5.1 Histogram-based deviation representation

As aforementioned, the deviations between data points and nominal surface can be estimated by computing Hausdorff or Euclidean distance. The distance values can be organized into a histogram by grouping range of the data into smaller ranges, aka bin(s). Histograms provide unique description of surface characteristics that can be used to differentiate anomalous deviations from normal ones.

The point deviations of parts produced in an ideal production process are small and found scattered around a nominal surface, like the green points shown in Fig. 5; hence, a normal Gaussian distribution of points around the position of the nominal model is expected. Since the deviation is small for such cases, the resulting histograms show narrow distributions. A similar distribution can be expected for small form errors, depending on the number and size of form errors.

Fig. 5
figure 5

A 2D illustration of scattering of random deviation (green) and systematic deviations (red) around nominal line for planar and cylindrical features

Full size image

By the same token, the position, orientation, and large form errors, on the other hand, are likely to have wider, skewed, mean-shifted distributions and a tendency to show abrupt changes between consecutive bins. Furthermore, orientation and large form errors have a tendency to change modality of the distribution as the error values increase. It is worth noting that in cases like the one shown in Fig. 5, where points with approximately equal deviations lie in both sides of a nominal surface, the mean of systematic and random deviations can be close to the same value. Figure 6 shows wider and shifted systematic deviation’s histogram relative to random deviation’s histogram, based on rotation and translation of a feature shown in Fig. 2. Similar distributions have been obtained in experiments with point clouds of the MathWorks® logo and hypothetical cylinders. In line with above argument, the following conditions hold:

Fig. 6
figure 6

Histogram of random deviations (green) and systematic deviations (red). a Wider distribution of systematic deviations due to change in orientation. b A mean shifted distribution of systematic deviations due to change in position

Full size image
$$ {\sum}_1^n\mid {d}_i^s\mid -{\sum}_1^n\mid {d}_i^r\mid >0 $$
(3)
$$ \mid \max \left({\boldsymbol{d}}^r\right)-\min \left({\boldsymbol{d}}^r\right)\mid \le \mid \max \left({\boldsymbol{d}}^s\right)-\min \left({\boldsymbol{d}}^s\right)\mid \kern0.5em $$
(4)

where \( {\boldsymbol{d}}^r=\left\{{d}_i^r\in \mathbb{R}:i=1,\dots, n\right\} \) is deviations of randomly scattered points and \( {\boldsymbol{d}}^s=\left\{{d}_i^s\in \mathbb{R}:i=1,\dots, n\right\} \) is deviations of systematically scattered points. Those deviations can either be Euclidean or Hausdorff distance-based estimates.

Moreover, the histogram-based representations of deviations are further summarized in Fig. 7. The figure illustrates correlation matrices of two features whose points were initially normally distributed around a hypothetical nominal plane (Fig. 7a) and cylinder (Fig. 7c), in which later deviations were induced. The point cloud of the plane was rotated so that it has a parallelism value of 1.5 mm, 1.8 mm, and 3.1 mm relative to a horizontal plane and displaced vertically so that it has location error of 0.5 mm (Fig. 7b). Similarly, the cylinder of radius of 50 mm (Fig. 7c) was manipulated so that it has a location error of 0.25 mm, 0.5 mm, and 1.44 mm, and radial error of 1 mm (Fig. 7d).

Fig. 7
figure 7

The correlation matrix of two features with and without systematic variations. a Planar feature. b Correlation matrix of the planar feature with cases of parallelism (Pa) 0 mm, 1.5 mm, 1.8 mm, and 3.1 mm, and position (P) of 0.5 mm. c Cylindrical feature (green random deviation and red systematic deviation). d Correlation matrix of the cylinder with a position error (P) of 0.0 mm, 0.25 mm, and 1.44 mm and radial error of 1 mm (R49)

Full size image

Both matrices display change in correlation at different location and orientation errors. Further, a change in surface characteristic is marked by a shift from unimodal to multimodal histograms as the deviation values increase from cell 1 to cell 3. Those similarities become a basis for differentiating normal and anomalous variation of geometric features, independent of shape complexity.

5.2 Estimating similarities between histograms

In line with the above behavior of point deviations, the similarity and closeness of two histograms can be quantified by estimating a distance metric. Distance measures like Euclidean and Mahalanobis distances can be used to measure distance between corresponding bins of histograms (e.g., [28]). The Euclidian distance function measures bin-to-bin distance without considering correlation between bins. Mahalanobis distance [38], on the other hand, is a cross-bin distance measure that takes into account the correlation between bins [39]. As shown in Fig. 7, the similarities and closeness between two histograms can be noticed by the extent of changes of correlation resulted due to difference in set of deviations. Hence, Mahalanobis distance can be applied to study the similarity of histograms of SMSs. Mathematically, for two histograms with bins P = {pi ∈ ℝm : i = 1,  … , n} and baseline bins Q = {qi ∈ ℝm : i = 1,  … , n}, and covariance matrix C ∈ ℝn × n, where n is the number of bins and m number of variables, Mahalanobis distance MD is defined as:

$$ {MD}^2\left(\boldsymbol{P},\boldsymbol{Q}\right)={\left(\boldsymbol{P}-\boldsymbol{Q}\right)}^T{C}^{-1}\left(\boldsymbol{P}-\boldsymbol{Q}\right) $$
(5)

The variables that define histograms are frequency of the deviation per bin and bin-edge values. The computation of the Mahalanobis distance using these variables, the extent to which two histograms are similar, can be estimated. The Mahalanobis distance values are then used as features of a supervised machine learning classification algorithm.

6 Supervised anomaly detection

6.1 Predictor vectors

Machine learning classification algorithms aim to categorize data organized in the form of a predictor vector into corresponding classes. Specifically, supervised anomaly detection techniques mainly apply the classical supervised machine learning approaches. This is expressed as follows: for input vector xi with m variables and labeled class yi, to which xi belongs to, the learning process for input space X and output space Y, seeks to learn function f : X ⟼ Y. Moreover, machine learning seeks to generalize from previously seen data such that for unseen data x, f(x) can predict the corresponding y value. For n number of samples, the data set is organized as:

$$ DS={\left\{\left({\boldsymbol{x}}_i,{y}_i\right):{x}_i\in {\mathbb{R}}^m,{y}_i\in \mathbb{Z}\right\}}_{i=1}^n $$
(6)

In this work, the classifiers seek to learn from Mahalanobis distance values, by treating each value as a predictor. For a histogram with m bins, there are m Mahalanobis distance points; the Mahalanobis distance from Eq. 7 then becomes a predictor.

$$ {\boldsymbol{x}}_i={MD}_i^2\left(\boldsymbol{P},\boldsymbol{Q}\right) $$
(7)

To compute Mahalanobis distance of random and systematic deviations’ histograms, three histograms are required: (1) a baseline histogram with N(0, σ2) distribution, where the variance is set to give an ideal variation, (2) a histogram of observed random deviation, and (3) a histogram of observed systematic deviation. Figure 8 shows the superimposition of these histograms, where it is programmatically difficult to delineate between yellow and red histograms. However, computation of the corresponding Mahalanobis distance from the baseline histogram makes it easier to delineate between the two as shown in Fig. 9.

Fig. 8
figure 8

Histograms generated from deviations induced to a MathWorks® logo. A baseline histogram (green), histogram of observed random deviation (yellow), and histogram of observed systematic deviations (red)

Full size image
Fig. 9
figure 9

Mahalanobis distance of random and systematic deviations’ histograms from a baseline histogram

Full size image

As pointed out, as the point deviations increase, the histograms’ mean and mode(s) shift and the width increases; hence, the change in distribution can be detected by tracking those indicators of the data. The mean shift tends to be caused by location error as all points are transformed in one direction, and change in modality is due to orientation errors as the points are distributed along both sides of a nominal surface.

Furthermore, another unique characteristic of surfaces is that the difference between two extreme ends of geometric features changes proportionally with rotation of the feature, which increases the width of the distribution. In this paper, the distribution width is referred as a delta predictor. The inclusion of the predictor significantly improves the prediction score related to orientation error. This predictor does not contribute to the score for the cases of deviations due translational displacement. Moreover, these parameters alone cannot be reliable indicators of change as these parameters can remain unchanged even though the distribution has changed.

Thus, for a baseline histogram N, random deviations’ histogram R and systematic deviations’ histogram S, the predictor vector of random deviations \( {x}_i^r \), predictor vector of systematic deviations \( {x}_i^s \), and the total data set DS become:

$$ {\boldsymbol{x}}_i^r=\left({MD}_i^2\left(\boldsymbol{N},\boldsymbol{R}\right),{\mu}_i^r,{M}_i^r,{\Delta}_i^r\right) $$
(8)
$$ {\boldsymbol{x}}_i^s=\left({MD}_i^2\left(\boldsymbol{N},\mathbf{S}\right),{\mu}_i^s,{M}_i^s,{\Delta}_i^s\right) $$
(9)

where μ, M, and Δ are mean, mode, and width of their corresponding distributions, respectively.

$$ DS=\left\{\begin{array}{c}\left({\boldsymbol{x}}_i^r,{y}_i=0\right),\kern0.5em i\le k\\ {}\left({\boldsymbol{x}}_i^s,{y}_i=1\right),\kern0.5em \mathrm{otherwise}\end{array}\right. $$
(10)

where k is the number of random deviations’ histograms.

6.2 Machine learning classifiers

There are many potentially applicable machine learning classifiers for anomaly detection. In this work, based on the methodology presented above, four classical classifiers that performed well are briefly introduced below.

6.2.1 Decision trees

Decision trees are classifiers that extract rules underlying training data by applying a sequence of Boolean logic to construct hierarchical and sequential structures [40]. To construct a tree, a predictor, selected as a root node, is iteratively split until further splitting is unnecessary. The termination criteria are either the outcome contains one class or all predictor nodes have been decomposed. The sample data is classified following the rules of the constructed tree [40].

6.2.2 Support vector machine

Support vector machine (SVM) is one of supervised machine learning algorithms that are used for linear and non-linear classification of given data. SVM seeks to find the largest margin that separates points of two classes between which an optimal hyperplane can be positioned. For high-dimensional predictors, kernels embedded in SVM can project a predictor into high dimensional feature space, where separation is easier. During evaluation and testing, samples point are scattered in the same feature space and classified based on which side of the hyperplane the samples fall in [41].

6.2.3 k-nearest neighbors

k-nearest neighbors (kNN) is a model free algorithm, in which sample data is classified based on closeness to the training data. A distance from a sample point to each training point is computed. The first k training points closest to the sample point are then collected and the sample point is categorized to the class of the most frequent points [42].

6.2.4 Ensemble

Ensemble learning combines multiple weak classifiers to improve classification performance of the classifier in the ensemble [43]. Class of algorithms are combined, or the alternative is to use a single classifier with many instances of the algorithm with different sort of training data [43]. The majority vote of individual classifiers is then selected as the outcome of learning. The common strategies are bagging and boosting. Bagging is a random selection training set with replacement. Boosting experiments sequence of algorithms by giving more weight to the misclassified data during previous iterations. These strategies almost always give better results than single classifiers [43].

7 Illustration case

To demonstrate the method, both SMS-based synthetic data and scanned data were considered. The scanned data was obtained from an industrial production line that produces a truck component shown in Fig. 10. This illustration focuses on extracted features B and C. The synthetic data was used to train machine learning models and the scanned data to test the models.

Fig. 10
figure 10

A drawing of a part of considered for illustration

Full size image

7.1 Training using synthetic data

To generate a synthetic data that is close to real data of a manufactured part, a feature of interest, feature B, was first extracted from the point cloud. Outliers were removed by fitting the point cloud in a 0.5-mm-thick box. Following this step, a discretized nominal model corresponding to feature B was best fitted to obtain orientational and position errors. Further, multiple point clouds based on Gaussian random fields were generated, and the waviness was scaled to approximate the planarity of scanned models by changing the correlation and scaling towards the z direction. Then new SMSs were generated by summation of the discretized nominal model and the generated Gaussian random fields, following the approach presented in Section 4.1. Figure 11 shows an example of generated SMS from its corresponding scanned data. Each sample was set to contain around 140,000 points.

Fig. 11
figure 11

Generated SMS corresponding to feature B

Full size image

The generated SMS orientation and position was further varied by adding small random orientation and position errors with standard deviation of 0.001° and 0.001 mm, respectively. This captures the presumed variation in the part produced in a stable manufacturing line. These SMSs were stored as normal training data and assigned to normal class. Furthermore, to generate SMSs for the anomalous class, the generated SMSs’ standard deviations were increased to a rotation of 0.1° and position deviation of 0.1 mm, and the Gaussian random fields were generated and scaled in the z direction so that the generated points will have planarity of more than 0.04 mm. In this way, 2100 SMSs with equal number of normal and anomalous classes were generated.

The generated SMSs’ deviations from the nominal model were computed and organized in their corresponding histograms of 20 bins. The Mahalanobis distances of the histograms from a baseline histogram of N(0, 0.0022), the mean, mode, and delta were computed for use as predictor vectors. The data was divided into 80% training and 20% validation sets. Six classical machine learning classifiers were evaluated, and four that consistently performed higher prediction accuracy were selected for further evaluation. The classifiers include decision tree, kNN, SVM, and Ensemble. The prediction score of the four classifiers on validation data are shown in the Table 1. In decision tree, a Fine Tree (FT) type with max splits of 100 was used. In kNN, weighted kNN type with 10 as the number of neighbors using Euclidean distance was applied. In SVM, cubic SVM type, and in Ensemble, bagging strategy with decision tree learners were used.

Table 1 Score of classifiers based on extracted data from generated SMSs
Full size table

Moreover, all the classifiers with principal component analysis (PCA)-enabled dimensional reduction did not have an effect on improving the prediction accuracy. Normalization of histograms did not contribute to a better result as the Mahalanobis distance was estimated from the frequency per bin, proportionally suppressing the difference between corresponding histogram frequencies. Further, all experiments reported in this work applied Matlab2018a’s implementation of the classifiers, using a machine with processor Intel® Core™ i7-5500U CPU @ 2.4GHz 8 GB RAM Windows 10 operating system.

7.2 Testing trained classifiers on real data

To test the classifiers, a scanned data with and without outliers from the industrial manufacturing line was used. The two extracted features, features B and C, were machined in two different setups, resulting to two different sets of variations. Those features had not been seen by the classifiers that were initially trained on synthetic data.

To predict classification of the real data, first, the deviation of points from nominal of the samples shown in Fig. 12 was organized in the histogram, and Mahalanobis distance, mean, mode, and delta values were computed. The trained classifiers applied these values to predict the class of each of the geometric features. The number of correct predictions by the classifiers on eight test sets of five samples are summarized in Table 2.

Fig. 12
figure 12

Five extracted data points. a Feature B. b Feature C

Full size image
Table 2 Number of correct predictions on data sets of five samples
Full size table

8 Discussion

In inspection, surface deviations are often controlled by GD&T schema that applies a set of primitive shapes. However, this does not apply when it comes to free-form surfaces and detecting changes within tolerance. In an attempt towards coming up with a generic anomaly detection method, which is independent of shape complexity, supervised anomaly detection approach was proposed. Mainly, the learning is based on a distribution change detection technique. We established that the unique distribution profiles are applicable in both large and small features as well as simple and complex features, such as MathWorks® logo.

Using the approach, the trained classifiers were found to be very sensitive in detecting anomaly related to planarity, parallelism, and position of the feature shown in Fig. 12a. Figure 13 summarizes the sensitivity of the trained classifiers. For the three error types, 10 separate data sets of 100 SMSs were used to extract variation information and train multiple classifiers. To represent acceptable normal variation, half of each data set was generated in such a way that they fall below a hypothetical threshold.

Fig. 13
figure 13

Sensitivity of trained classifiers to change in planarity, parallelism, and position. a Planarity change. b Parallelism change. c Position change

Full size image

The planarity value up to of 0.04 mm was considered as normal variation of the SMSs. Change of 0.02 mm in planarity due to form errors can be detected at a prediction accuracy of 100% as shown in Fig. 13a. When the features were rotated so that they have parallelism change shown in Fig. 13b, the classifiers were able to detect a change of 0.04 mm at a prediction accuracy of 100%. Similarly, data sets of SMS samples were displaced in z directions equivalent to make position errors shown in Fig. 13c. A change of 0.01 mm could be detected at a prediction accuracy of 100%.

The above results are based on synthetic training data specifically separated for parallelism, planarity, and position errors. Figure 14 shows the differences between random and systematic variations plotted with respect to the predictor vector. Few of the first, the last, and the middle Mahalanobis distances contribute in planarity and parallelism error detection. Delta has more contribution in planarity error detection. The mean and mode contribute more in position error detection. This is in line with an expected shift of mean and mode due to change in position error.

Fig. 14
figure 14

Examples of a parallel coordinate plot of training data with respect to a predictor vector. Random variation class (blue) and systematic variation class (orange). a Planarity error. b Parallelism error. c Position error

Full size image

However, when the training data is combined, the classifiers become less sensitive. This is due to the insensitivity of mean values to parallelism change and mode to planarity change. Hence, application of the three separate anomaly detection classifiers, specifically trained for those errors, in sequence is much likely to improve the sensitivity.

Moreover, the accuracy of the method is dependent on the size of region of interest and an accurate setting of the threshold value. When the focus is on a small region of a feature, the training can be performed for that region, similar to extraction regions of interest through small grid and adaptively growing grids (e.g., [14, 18]). Further, there was variation on the validation and test result, particularly with kNN and SVM classifiers. For instance, kNN prediction on a specific sample of feature B with outliers was treated as normal by a trained model and as anomalous by the subsequent trained model. The SVM prediction accuracy fluctuates between 97 and 99%. This could be attributed to the randomness induced by Gaussian random fields.

With a combined data of form, parallelism, and position error data, the classifiers took around 10 s to train and validate. Moreover, it took less than 1 s per feature to test on the real data. Thus, the classifiers can be applied for online detection without becoming a bottle neck to a production line that produce products similar to the one discussed in this work. This is aside from time spent in extracting the geometric features. Furthermore, it is the cautious belief of the authors that dummy histograms can be obtained by distribution-fitting of the real data, so that the steps of generation of SMSs are skipped.

9 Conclusion

In transition to Industry 4.0, automated inspection system is one of the core activities. In this work, we showed that unfamiliar orientation, position, and form errors can be detected by applying the classical machine learning classifiers, aimed for use in an automated inspection system. First, SMSs that represent normal variation were generated by summation of surface that capture orientation and position errors of scanned parts, and Gaussian random fields to approximate form errors. A similarly generated set of SMSs was manipulated so that they acquire different levels of planarity, parallelism, and position errors, representing anomalous deviations. From both normal and anomalous deviations, a predictor vector that includes Mahalanobis distances, mean, and mode and a range of deviations were derived. Based on the predictor vectors, four classifiers were trained and scored more than 97% accuracy in detecting change of 0.02 mm planarity, 0.04 mm parallelism, and 0.01 mm position. Specifically, the decision tree and ensemble classifiers scored 100% on a synthetic validation data and real industrial data. Since the proposed approach is independent of shape complexity, it can enable an automated inspection to be performed on free form surfaces.

The weakness of the approach is that the system is expected to learn threshold beyond which variation should not exceed. This may require lot of training data. In future, we will investigate how this approach can be applied for detection of surface defects on small regions of a feature and how the prediction accuracy changes based on variable bin number per histogram.