Keywords

1 Introduction

Despite recent advantages, modern aircraft production and maintenance are mainly performed by manual assembly [1]. Highest demands for complete traceability and quality control of each assembly step, require high efforts in process supervision. One of those supervision necessities is checking, whether the correct component is chosen for the next assembly step. Typically, such verifications can be digitized through the help of markers, e.g. RFID or 2D/1D-Codes. However, many components in the aviation industry either are not permitted to wear such codes or cannot bear such codes due to their surface area being a functional area. Therefore, manual identification of components is necessary for various stages of a (dis-)assembly. With manual identification being error-prone, this poses a great factor for process instability. This applies even more so if the components have a high degree of similarity to each other.

Due to the necessary waiver of markings, visual and markerless identification have to be employed. Such approaches are feasible for distinctive and feature-rich aviation components [3]. Within this work, we focus on components that wear little to no features, and have high similarities. Due to the highly individualistic nature of such identification problems, solutions are hardly transferable between problems. We, therefore, aim to contribute a transferable baseline for analysis of those problems. A methodology is presented that guides through the assessment of challenges for such identification systems. Special focus is given to the assessment of similarity and task identification complexity.

2 Related Work and State of the Art

Designing visual sensor applications for industrial processes is a task that requires expert knowledge both about sensor systems as well as the application domain [1, 6, 9]. To assist in the design process, methodologies have been developed that guide in the selection of sensors: The approaches of sensor planning methodologies augment the selection process towards configuration parameters such as the extrinsic pose of the camera and illumination of the scene. A sensor planning system outputs to the user camera pose, optical settings, and illuminator settings [4]. Sensor planning approaches have been developed for specific visual applications such as surface inspection [5]. They contribute a flexible yet automated planning pipeline that is motivated by the trend of individualistic customization of products.

In the domain of aircraft production [1] proposed an assistance approach to design visual sensor applications for assembly supervision. This specifically considers the assembly task-specific generation of viewpoint candidates that allow detecting the successful assembly operation. They augmented that approach by adding an automated enablement through AI-based processing trained with synthetic data. This synthetic data is generated with the same 3D models and camera-specifics used to configure the system. As they targeted assembly situations that follow pre-defined assembly patterns, the object recognition and localization tasks are well defined. Focusing on the distinction of different assembly situations [13] introduced a geometric analysis of assembly situations to derive view points for assembly supervision. Such approaches can be further improved by extending the metric for suitability of vision view-points towards similarity of considered components.

Focusing on the ability to detect aircraft components in production supplying logistic operations with delivery units, [2, 3] provide the capability to enable AI-based visual sensor applications with the help of synthetic training data. Such an approach can be incorporated into the design flow of this paper, however, is limited to components that can be differentiated through means of object detection and a top-view sensor configuration.

Addressing the challenges of identification of similar appearing car parts, [11] introduced a classification box that utilizes Deep-Learning based image processing, enabled by synthetic data. Focusing on the similarity of objects, [12] found the use of CNN-based image processing in principle applicable to such challenging situations. However, it is necessary to determine for which identification tasks special considerations have to be given. We, therefore, contribute an analysis methodology that focuses on similarity analysis. Assessment of similarities between 3D-shapes can be done by various methods [8]. We incorporate two approaches and derive a similarity metric.

3 Assessing Identification Challenges - Methodology

As shown in the previous sections, the selection of suitable technologies for identification tasks has to consider multiple parameters of each sensor and algorithm type. Mapping those parameters and the resulting abilities to individualistic identification problems, requires revealing the individualistic facets of each identification problem. For this analysis, a methodological approach is proposed that assesses requirements based on the component spectrum, reveals challenges, and defines the identification task complexity:

  1. 1.

    Revealing component features

  2. 2.

    Scale of geometric features

  3. 3.

    Assessing the similarity of components and their features

  4. 4.

    Using iterative subdivisioning to reduce the identification task complexity

The following sub-sections detail the steps of this methodical analysis approach. For each step several to be analyzed aspects are explained, and it is discussed how and when these aspects affect other steps or technology selection criteria.

Step 1: Revealing Component Features and their Value. Analyzing the component spectrum is the main driver of this methodological approach. As such the output of this step is the main input for subsequent analysis steps and may be re-visited whenever necessary information is missing in follow-up steps. With this step, highly individual outputs for each identification task are expected. It is therefore not possible to provide a comprehensive list that can be applied universally. In general, it can be differentiated between qualitative and quantitative or measurable features. Most of the qualitative features can be translated into one or more of several quantitative features (e.g. a button can be attributed to geometric and color features). Geometric features are mainly described by (1) surface features: planes, spheres, cylinders, cones, and free-form surfaces, and (2) curve features: lines, circles, ellipses, paraboles, splines [13]. This work addresses visual identification. The appearance of objects concerning features such as texture and colore schemes are mainly attributed to the surface of the object. Such features can be quantified through distribution maps and histogram analysis.

After features for each component are extracted, it may be beneficial to reflect those features against the entire component spectrum. This yields information with respect to the proper rating of suitability for different applications as well as the subsequent analysis. The main challenge, which is to be addressed in the following step, is whether these features can be detected by sensors and assessing the similarity of components.

Step 2: Scale of Geometric Features. After geometric features that allow for possible unique differentiation between components are identified, it has to be assessed whether those features can be detected and measured by sensors. Considering mainly geometric features, two parameters are relevant for that assessment, first the resolution r of the sensor and second the geometrical manufacturing tolerances of the component feature m. If the extreme values of the manufacturing tolerances and sensor resolution combined are greater than the range between two adjacent components feature values d, unique identification is not possible. Therefore the following has to hold:

$$\begin{aligned} \textrm{max}(r) + \textrm{max}(m) \le \frac{d}{2} \end{aligned}$$
(1)

The criterion to evaluate the suitability of sensors, therefore, is, whether this inequality holds.

Step 3: Assessing the Similarity of Components. The similarity of components is the main challenge for successful and unambiguous identification tasks. Considering the domain of the aviation industry, this poses a considerable challenge for modularized components. Choosing appropriate sensors and identification algorithms is a key factor to ensure that distinctive features are not only detected but also accordingly processed. It is, therefore, necessary to assess the distinctivity of each component with respect to a different component of the same spectrum. For this similarity measures may be applied. Two different methods for the assessment of component similarities are presented.

  1. 1.

    3D-Model-based analysis: under assumption of available 3D-models geometric analysis can be directly applied. Multiple approaches are viable for assessment of shape similarity [14]. Depending on requirements with respect to pose-invariance or scale suitable metrics have to be chosen. In the following examples, the Hausdorff-distance is used:

    $$\begin{aligned} \begin{aligned} \hat{d}_H(X,Y) =&\textrm{max}_{x\in X}\{\textrm{min}_{y\in Y}\{\Vert x,y\Vert \}\} \\ d_H(X,Y) =&\textrm{max}\{\hat{d}_H(X,Y), \hat{d}_H(Y,X)\}. \end{aligned} \end{aligned}$$
    (2)

    With the euclidean distance \(\Vert \cdot \Vert \), small values of the Hausdorff-distance denote that each of the elements x in a set X has an element \(y\in Y\) for which the distance is small. We use this distance to assess the similarity between two shapes:

    $$\begin{aligned} \mathcal {S} = 1 - d_H(X,Y) \in [0\,...\,1], \end{aligned}$$
    (3)

    where 0 denotes no similarity, and 1 denotes that both shapes are identical.

  2. 2.

    Feature-based analysis: under the assumption of a feature set that can be applied to describe the component spectrum, those feature-sets can be analyzed and similarities can be identified. In accordance with the previous step, both qualitative features can be analysed through histogram distributions. Consider a set \(\mathcal {F}\) of n features:

    $$\begin{aligned} \mathcal {F} = \{F_l\, \vert \,l = 1\,...\,n\}. \end{aligned}$$
    (4)

    For each feature \(F_l\) chose an appropriate bin width h. A narrower bin-width allows for good distinctivity but requires sufficiently accurate measurement capabilities. With the chosen bin width, calculate the histogram distribution \(\mathcal {H}\), which can be represented as set of bins:

    $$\begin{aligned} \mathcal {H}_{F_l} = \{B^l_i\,\vert \,i=1\,...\,k \}, \end{aligned}$$
    (5)

    with the number of bins calculated through the bin width h and max/min values x in each Feature range:

    $$\begin{aligned} k = \left\lceil \frac{\textrm{max}(x \in F_l) - \textrm{min}(x \in F_l)}{h}\right\rceil . \end{aligned}$$
    (6)

    The similarity between two components X and Y, is calculated with the number of occurrences the two components are listed in the same bin. This is denoted through the Kronecker delta \(\delta _{(X \in B^l_i),(Y\in B^l_i)}\):

    $$\begin{aligned} \mathcal {S} = \frac{\sum ^{n}_l \sum _i^k\delta _{(X\in B^l_i),(Y\in B^l_i)}}{n}. \end{aligned}$$
    (7)

    Again the similarity is ranged between 0 and 1, with the latter indicating that none of the features can be utilized to distinguish both components from another.

Step 4: Using Iterative Subdivisioning to Reduce the Identification Task Complexity. Within each identification task, the identification task complexity \(\mathcal {C}\) denotes from how many different components a specific component one has to be differentiated. An index notation is introduced \(\mathcal {C}_{\textrm{i, j}}\) that denotes the complexity for different subsets of the component spectrum.

Based on the previous step of feature analysis, similarity assessment, and sensor applicability assessment, suitable distinctive features can be chosen to subdivide the component spectrum alongside that feature (s. Fig. 1). Each subdivisionend spectrum contains very similar or same values of that specific feature, which therefore can be considered no longer useful for identification purposes within this smaller spectrum. This procedure may be applied iteratively until,

  1. a)

    spawning branches (s. Fig. 1) result in a complexity of \(\mathcal {C}_{i,j} = 1\) through feature measurement.

  2. b)

    AI-based or Template-Matching-based are deemed applied.

Fig. 1.
figure 1

Index notation of the identification task complexity value. Each subdivision step, reduces the complexity by utilizing one feature to subdivide the component spectrum into further sub-spectrums.

Full size image

4 Application of Methodology to Use-Cases

Two industrial application scenarios are used to demonstrate the presented analysis approach: (1) Identification of bushings and (2) Identification of tubes. Both Use-Cases originate from Aircraft Industry and are considered to this date unsolved.

4.1 Use-Case 1 - Type-Identification of Bushings

Aircraft systems that contain actuators or moving parts, often include bushings. For E.g. landing gear systems contain up to several hundred different bushings. Since the bearings consist mainly of functional areas, attaching an identification marker is impossible and visual identification is necessary. In the analysed component spectrum, 23 different bushings are considered.

Step 1 - Component Features: Based on the construction data, four features are relevant for the identification of bushings: (1) length overall [mm], (2) outer diameter of the main bushing body [mm], (3) inner diameter [mm], and (4) the collar outer diameter. The inner diameter is not considered a suitable identification feature, since the considered bushings share the same feature value. Value distributions for the other features are shown in Fig. 2.

Step 2 - Scale of Geometric Features: Besides the presence of a collar, which can be considered a binary feature, lengths and diameters are quantitative measurable features. Therefore it has to be assessed, how narrowly two entries of two geometric measurements are located. Considering a dimensional manufacturing tolerance of \(0.01\,\textrm{mm}\) and the minimal distance d between two feature measurements (\(0.03\,\textrm{mm}\)), the inequation 1 can be converted to yield the necessary resolution of \(r=0.005\,\textrm{mm}\) or \(5\,\textrm{m}\).

Step 3 - Similarities: Although the design of bushings is not limited to the above parameters, bushings share a distinct similarity. In the considered use-cases no oil outlets, inlet, or similar feature is considered on the bushings. However, each bushing is uniquely described by the geometric features length as well as the inner and outer diameter, and the presence of a collar and its outer diameter. The inner diameter is for all bushings equal. Assessing the similarity of the bushings through the presented feature-based approach yields the in Fig. 2 shown results. With the three resulting features, all bushings can be distinguished.

Fig. 2.
figure 2

Heat-map representation of bushing similarities. Due to the three distinguishing features, three similarity steps are calculated. Bushings without a collar, are represented in the histogram analysis with a collar diameter of 0.

Full size image

Step 4 - Subdivisioning: The entire component spectrum of 23 bushings can be divided by the binary feature of a collar. Assessing the subset of collared bushings, 8 bushings have to be uniquely identified. Considering the subset of bushings without a collar, 15 bushings have to be differentiated.

4.2 Use-Case 2 - Identification of Tubes

Tubes are omnipresent throughout aircraft systems, being present in actuator systems, fueling, lavatories, engines, and similar systems. With flown systems, handled tubes are often soiled when going into MRO procedures and MSN plaques are often unreadable. Visual identification is carried out by manual matching of components to reference images. In the considered Use-Case, nine tubes are to be differentiated.

Step 1 - Component Features: The considered tubes are of the same diameter and have no flanges or similar mounted features. Thus, only geometric features may be extracted. Out of the possible features, three describe the tubes best: (1) End-to-End length of each tube, (2) Bending radii over the curve of the tube, and (3) Distances between bends but also distances between a bend and the end.

Step 2 - Scale of Geometric Features: While the above features can be measured, adjacent feature values are above \(>1\,\textrm{mm}\). Thus, regular manufacturing tolerances and typical industry-standard sensors do not affect the measurement of these features.

Step 3 - Similarities: For this Use-Case the 3D Shape Similarity approach is used. The resulting similarity values are shown in Fig. 3. As intuitively visible, tubes 1 and 2 share a high similarity alongside tubes 2 and 3. Pairings 7 and 8 may also be identified as cumbersome. However, since the similarity with other tubes is low for both tubes 7 and 8, subdivisioning is a suitable approach for this pairing.

Step 4 - Subdivisioning: With the above-shown similarities between multiple pairs of tubes, it is recommendable to subdivide the tube identification. As such At least two sub-spectrums should be formed that mix tubes 1, 2, 3 accordingly to their similarity value.

Fig. 3.
figure 3

Representation of the nine considered tubes and their respective similarity values.

Full size image

5 Discussion

The above approach infers possible identification challenges. By Revealing those challenges it is possible to address those specific challenges in the selection of suitable identification algorithms and procedures. Considering the first presented use-case, the identification of bushings, feature selection allows presenting a set of distinctive bushing parameters. Calculating the similarity based on those features allows for selecting suitable identification strategies. Since the distinction between certain similar bushings is only possible based on a quantitative feature, measurement processes have to apply to accurately measure this specific feature.

This is in contrast to the second use case. There, the geometric similarity was not described by features but by the continuous similarity score. The derived subdivisions of the component spectrum can be used to derive classification tasks for AI-based object classification. For each subdivision, classification should be easier to train than for the component spectrum in its entirety. Utilizing this reduction in task complexity may yield more robust differentiation between those similar components.

Nevertheless, the presented approach relies on profound knowledge and understanding of the component spectrum. The limitation of the first similarity analysis is the resulting dependence on the quantified parameters of the component spectrum. Nevertheless, if distinctive parameters are found, the analysis can reveal whether the robust distinction between components is possible. If no such parameters are derived, the second similarity approach can be used. However, the geometric-based similarity analysis does not reveal which features can be measured or used to distinguish components from one another.

6 Conclusion and Outlook

This work presents a methodology to analyze similar and unmarked component spectrums with respect to identification challenges. Particular focus is given to similarities between components and revealing those similarities. A four-step methodology is presented that includes feature analysis, feature applicability assessment for sensors, similarity revealing, and identification of task complexity subdivision. The two presented methods for similarity revealing allow early discovery of possible mix-ups that pose threat to identification systems. By subdivision of identification task and reduction of the task complexity, this problem can be addressed in the design phase of the identification system.

Future work will address a combination of this approach with sensor selection and planning phases as well as further identification-algorithmic discussions.