Abstract
The increasing demand for miniaturization is having an immense impact on manufacturing technologies and is leading to the development of many novel and innovative production processes for highprecision micro parts.
Download chapter PDF
5.1 Introduction to Quality Control and Characterization
Peter Maaß and Iwona PiotrowskaKurczewski*
The increasing demand for miniaturization is having an immense impact on manufacturing technologies and is leading to the development of many novel and innovative production processes for highprecision micro parts. Industrially feasible micro production requires fast quality assurance control mechanisms, which need to take into account the microspecific properties.
In this chapter several innovative technologies for quality control of production processes for highprecision micro parts will be presented. The proposed innovative solutions also address the need to ensure the quality of final products, which should be manufactured with approved (specified) materials and within certain performance requirements. Therefore, careful quality control is required during each production step.
Micro production processes and related techniques for quality control and characterization need to take into account the size effects that occur at the micro level, which lead to different scaling of forces and additional manufacturing errors that do not occur in macro processes. As a consequence, the conventional control methods for macro processes may fail, or become much more complicated or unreliable when applied to micro processes. In order to guarantee efficient quality assurance, possible error classes must be redefined and evaluated.
Moreover, in order to achieve an upscaling in the number of produced parts, the semifinished products must be examined for mechanical properties. Micro production is characterized by a low aspect ratio of components to microstructure, which can dominate the mechanical behavior by means of the surface defects. It turns out that fatigue strength is an important component property.
This chapter presents some technical and methodological solutions for quality control and characterization measurement, with respect to size effects and upscaling in micro production.
Digital holographic measuring systems (DHM) have the potential for complete (100%) and fast quality testing. Such a system is described in Sect. 5.2 together with fast algorithms for geometric evaluation and surface defect detection. The proposed method is demonstrated by inspecting cold formed micro cups.
This system is suitable for the automated examination and classification of micro deep drawn parts in an industrial environment. Together with automatic error detection, which uses the knowledge of the shape of the object, it allows the reliable inspection of components with a measuring speed of about one part per second. Further reduction of time is possible by additional optimization of the error detection algorithm. Section 5.3 shows the inspection of the interior of micro parts in the industrial environment.
To guarantee consistent production parameters for semifinished products, which are subsequently used for the production of micro components, a characterization of their physical properties is of particular importance. Due to the often oligocrystalline character of these semifinished products and components, it is necessary to use a suitable testing technique for static and dynamic investigations, as the mechanical properties are not transferable from the macroscopic point of view.
In addition, the micro semifinished products and components often show inhomogeneities induced by the manufacturing process. On the one hand, these are directly reflected in the microstructure and on the other hand they have an effect on quantities such as hardness or residual stresses, which play a decisive role in the application. Mechanical testing, conventional metallography, SEM, EBSD, ultramicrohardness testing and Xray residual stress analysis were used as measuring and analysis techniques suitable for the submillimeter range. In Sect. 5.4 the possibilities and limitations of two of these methods are illustrated using the example of mechanical testing and electron backscatter diffraction (EBSD).
Micro forming tools with high strength and hardness can be produced by means of laser chemical machining (LCM). The in situ geometry measurement of microstructures during this process is based on the fluorescence of a liquid medium and principles of confocal microscopy. Section 5.5 focuses on that measurement technique, based on confocal fluorescence microscopy. It turns out that the modelbased approach is suitable to detect the geometry parameter step height with an uncertainty of 8.8 μm for a step submerged in a fluid layer with a thickness of 2.3 mm.
Acknowledgements The editors and authors of this book like to thank the Deutsche Forschungsgemeinschaft DFG (German Research Foundation) for the financial support of the SFB 747 “Mikrokaltumformen—Prozesse, Charakerisierung, Optimierung” (Collaborative Research Center “Micro Cold Forming—Processes, Characterization, Optimization”). We also like to thank our members and project partners of the industrial working group as well as our international research partners for their successful cooperation.
5.2 Quality Inspection and Logistic Quality Assurance of Micro Technical Manufacturing Processes
Mostafa Agour*, Axel von Freyberg, Benjamin Staar, Claas Falldorf, Andreas Fischer, Michael Lütjen, Michael Freitag, Gert Goch and Ralf B. Bergmann
Abstract Quality inspection is an essential tool for quality assurance during production. In the microscopic domain, where the manufactured objects have a size of less than 1 mm in at least two dimensions, very often mass production takes place with high demands regarding the failure rate, as micro components generally form the basis for larger assemblies. Especially when it comes to safetyrelevant parts, e.g. in the automobile or medical industry, a 100% quality inspection is mandatory. Here, we present a robust and precise metrology method comprised of a holographic contouring system with fast algorithms for geometric evaluation and surface defect detection that paves the way for inspecting cold formed micro parts in less than a second. Using a telecentric lens instead of a standard microscope objective, we compensate scaling effects and wave field curvature, which distort the reconstruction in digital holographic microscopy. To enhance the limited depth of focus of the microscope objective, depth information from different object layers is stitched together to yield 3D data of its complete geometry. The 3D data map is converted into a point cloud and processed by geometry and surface inspection. Thereby, the resulting point cloud data are automatically decomposed into geometric primitives in order to analyze geometric deviations. Additionally, the surface itself is checked for scratches and other defects by the use of convolutional neural networks. The developed machine learning algorithm makes it possible not only to distinguish between good and failed parts but also to show the defect area pixelwise. The methods are demonstrated by quality inspection of cold formed micro cups. Defects larger than 2 μm laterally and 5 μm axially can be detected.
Keywords Digital holographic microscopy · Geometry · Surface defect
5.2.1 Introduction
Quality inspection is an essential tool for quality assurance during production. Generally, the precise geometrical and surface inspection of a test object plays a decisive role for developing and/or optimizing the manufacturing process. In the microscopic domain, where the manufactured objects have a size of less than 1 mm in at least two dimensions, very often mass production takes place with high demands regarding the failure rate, as micro components generally form the basis for larger assemblies. Especially when it comes to safetyrelevant parts, e.g. in the automobile or medical industry, often a 100% quality inspection is mandatory [Kop13].
Achieving 100% quality inspection is especially challenging in the microdomain, e.g. when measurement uncertainty has to be in the range of a few microns, as there is usually a tradeoff between precision and speed [Ago17]. Processes like micro cold forming, however, allow for production rates of multiple parts per second [Flo14]. Presently, no solutions exist that are fast, precise and suitable for inprocess measurements at the same time [Ago17b]. Actually, most industrial quality inspection processes of micro parts rely on manual sampling using confocal microscopy. This means also that no automatic data processing regarding the geometry and surface inspection is applied because such measurements are affected by strong image noise.
Due to these industrial requirements, methods such as tactile, confocal microscopy and phase retrieval [Fal12a] by means of multiple illumination directions [Ago10] are not suited for the inspection task since they are much too slow. In contrast to these methods, interferometric methods are established for a full field measurement. Examples of interferometric techniques include digital holography (DH) [Sch94], whitelight interferometry (WLI) [Wya98] and computational shear interferometry (CoSI) [Fal15]. These methods are based on the determination of the optical path difference (OPD) of light scattered by or transmitted through a test object. Since available cameras can only measure intensities of light, the OPD must be encoded consequently. Based on the encoding technique utilized and the geometrical complexity of the test object, measurement uncertainties down to a fraction of the illumination’s wavelength can be achieved [Mar05].
Here, we present a robust, fast and precise metrology method comprised of a holographic contouring system, as well as fast algorithms for geometric evaluation and surface defect detection, that paves the way for the inspection of metallic micro cups in less than a second. The holographic system is composed of four twowavelength digital holographic microscopy setups. It uses four directions of illumination in order to enable the simultaneous observation of the whole object surface and for speckle noise reduction as well as twowavelength contouring to collect form data. Spatial multiplexing, coherent gating, is used to capture four holograms by each camera in a single shot [Ago17]. Moreover, it utilizes a telecentric microscope objective instead of a standard microscope objective to compensate scaling effects and wave field curvature, which distort reconstruction in digital holographic microscopy. In order to enhance the limited depth of focus of the microscope objective, depth information from different object layers is stitched together to yield 3D data of its geometry utilizing the autofocus approach presented in [Ago18].
The 3D data map is converted into a point cloud. The resulting point cloud data are automatically decomposed into simple geometric elements with a holistic approximation approach in order to analyze geometric deviations. In addition, surface defects are detected with a convolutional neural network. The measurement and data evaluation approaches are demonstrated by quality inspection of cold formed micro cups. As a result, defects larger than 2 μm in the lateral and 5 μm in the axial (depth) dimension can be detected.
The present section is organized in the following three subsections: Sect. 5.2.2 describes the measurement approach for dimensional inspection that allows the geometric features of deepdrawn micro cups to be characterized. Section 5.2.3 describes the dimensional inspection that allows for an automated dimensional analysis of prismatic surface data, where the surface can be a combination of different simple geometric base bodies (cylinder, plane, torus, etc.). Section 5.2.4 describes the detection of surface defects using convolutional neural networks (CNNs), which achieve high accuracy with only a few training samples.
5.2.2 Optical 3D Surface Recording of Micro Parts Using DHM
5.2.2.1 Holographic Contouring
Common engineering objects are much larger than the optical wavelength used for inspection, and the surfaces exhibit peaks and steps larger than one quarter of the wavelength of light. Such surfaces are therefore optically rough, and the measured phase values vary within the interval [−π, π]. The evaluation process becomes ambiguous and the measurement is not unique. The solution to this dilemma is the use of a synthetic wavelength much larger than the optical (natural) wavelength. In this approach, two phase distributions are retrieved from two measurements associated with two different wavelengths λ_{1} and λ_{2}. Calculating the phase difference Δφ between the reconstructed phase distributions corresponding to the two measurements, the 3D height map z_{p} of the test object can be directly calculated by utilizing [Fal12]
Here, α is the angle between the observation and illumination direction and Λ is the synthetic wavelength
The resulting phase difference for a single measurement using Λ is given by Eq. 5.1. A measurement that covers the sample surface results in a map that contains fringes and is referred to as the phase contouring map. Adapting the difference between the two wavelengths λ_{1} and λ_{2} is required to enable the investigation of objects with step heights of several millimeters.
5.2.2.2 Digital Holographic Microscopy
The digital holographic microscope schematically sketched in Fig. 5.1 is used to capture the required two holograms with two different wavelengths. The setup contains a longdistance microscope objective (LDM) with a magnification of 10×, a numerical aperture (NA) of 0.21 and a working distance (WD) of 51 mm. Optical fibers are used to illuminate the test object and to provide the required reference wave. A beam splitter (BS) is used to combine the object wave, light diffracted from the surface under investigation, and the reference wave, resulting in a hologram which is captured across the camera plane. It is noted that there exists an angle α between the observation and illumination direction (see Eq. 5.1). Numerically, the phase distribution which corresponds to each measurement is reconstructed utilizing the spatial phase shifting method, where the carrier frequency is controlled by shifting the position of the reference wave with respect to the optical axis [Ago17]. Thus, the setup is used to register the phase information in a plane which is close to the surface of the object under test.
In the following, the 3D surface measurements of the micro cup based on digital holographic microscopy will be discussed; this is the backbone of geometric inspection.
Figure 5.2 shows an image of an experimental setup consisting of four DHMs based on the sketch of the individual DHM shown in Fig. 5.1. Four laser diodes are used, two with \( \uplambda = 638.13 \) nm and the other two with \( \uplambda = 644.08 \) nm. According to Eq. 5.2, a synthetic wavelength of 69.07 µm results for the evaluation. Utilizing a fiber switch and a 1 × 2 fiber splitter, object and reference waves are formed. A digital hologram captured using the DHM is utilized for recovering the phase and the real amplitude of a monochromatic wave field across the object plane. The hologram generated across the output plane of the DHM is given by
where U_{O} and U_{R} represent the complex amplitudes of light diffracted from the test object and a reference wave across the recording plane. Here, X = (x_{i}, x_{j}) denotes the spatial coordinate vector at the recording plane. Equation 5.3 can be generalized as
representing the N holograms captured separately via the coherencegating principle [Ago17]. The intensity of the hologram recorded by the DHM’s camera is
Here, …^{2} denotes the intensity of the wave field
that represents the dc term which is the incoherent superposition of the object and reference waves and * refers to the complex conjugation. Using the spatial phaseshifting method, the complex amplitude which corresponds to each wavelength can be determined. Based on the shifted reference point sources with regard to the optical access, the diffraction terms of Eq. 5.5 are separated. To show this, it is necessary to transform the intensity from the spatial domain to the frequency domain by implementing the fast Fourier transform (FFT). Performing the FFT (\( {\Im } \)) on Eq. 5.5 results in
where υ = (υ_{i}, υ_{j}) is a vector in the frequency domain, δ(…) refers to the Dirac delta functions and indicates the corresponding shifted υ_{0,λn} across the Fourier domain, which shows the effect of the shifted reference point sources, and ⊗ is the convolution symbol.
Figure 5.3a shows an example of a hologram captured by illuminating a micro cup from four different directions using the DHM setup. And Fig. 5.3b shows the spectrum of the single hologram, which contains object information for the four directions of illumination. It is noted here that there is no crosstalk between the four different holograms, which are recorded in a single shot camera image. Each object’s information related to a certain illumination direction is shifted according to Eq. 5.7 to an exact position υ_{0,λn}.
It is noteworthy that the test object is illuminated from four different directions and four holograms are recorded on a single shot using four reference waves by applying the digital holography multiplexing principle [Ago17]. These holograms are used to reduce speckle noise in twowavelength contouring. Accordingly, each holographic unit from the four units will capture two successive multiplexed holograms. The two successive multiplexed holograms are captured, one for each wavelength. In the following, the results that were obtained using the four observation directions will be presented and discussed. The time required for the capturing process and for the switching between the two wavelengths is 120 ms. Using the spatial carrier frequency method [Ago15], one can numerically reconstruct the phase distributions ϕ_{λ1} and ϕ_{λ2,} which correspond to the two measurements.
Figure 5.4 shows the recovered complex amplitude. The phase difference Δ = ϕ_{λ2−}ϕ_{λ1} between the two reconstructed phase distributions across the capturing plane that represents the countering map across that plane is shown in Fig. 5.4b. As can be seen in Fig. 5.4b, fringes are only sharp across the area of the object in focus, which can be clearly recognized from the real amplitude shown in Fig. 5.4a. This is as expected, since the microscope objective has a limited depth of focus. In contrast to microscopy, digital holography offers the extension of the objective depth of focus, and since holograms give access to the complex amplitude, digital refocusing across the whole object by means of numerical propagation is possible. Thus, in order to completely reconstruct a sharp contouring propagation, autofocus algorithms are used. For fast evaluation, an automated process was proposed and implemented within a graphics processing unit (GPU). The autofocus algorithm is implemented by scanning within small windows throughout all the propagated planes to define where the object is in focus by estimating the standard deviations of Δ, which are relatively high within these windows, where the object is out of focus because of the speckle decorrelation. The result of this process is shown in Fig. 5.5 for only one observation direction. In both contouring maps, Figs. 5.4b and 5.5b, a surface defect (dent) is clearly shown.
The contouring phase map shown in Fig. 5.5b is then unwrapped. Then the values are substituted into Eq. 5.1 to determine the 3D height map. The result is shown in Fig. 5.6. Eleven seconds are required for data transfer and hologram analysis to obtain the 3D point cloud which is used as input for the geometry and surface defect analysis. Defects with lateral extensions from 2 µm and a minimal depth of at least 5 µm can be detected [Ago17]. Such a height map is converted to a 3D point cloud which is used as the input for both the dimensional and the surface inspection process, which will be discussed in the next section.
5.2.3 Dimensional Inspection
Dimensional inspection implies the evaluation of surface data with respect to dimensional, form and position deviations of certain geometric features. These deviations are compared to the specified tolerances in order to decide whether the workpiece meets the quality requirements or not. The following subsections give a brief survey of the state of the art in evaluating point clouds and present the holistic approximation as the method of choice for the dimensional inspection of optically acquired surfaces of micro parts.
5.2.3.1 State of the Art
Optical measurement data contain a high number of surface coordinates of one or multiple observation directions and represent either a freeform surface or a combination of several geometric elements. The evaluation of freeform surfaces, on the one hand, consists in aligning the measurement data to the nominal CAD data [Sav07] and calculating and visualizing the deviations of each measurement point. For this kind of quality inspection, several commercial solutions exist. On the other hand, in order to analyze the optical measurement data regarding geometric parameters like dimensional or shape deviations, the measured points have to be segmented. This means assigning the individual measurement points to the approximating geometric elements. However, a manual segmentation is timeconsuming and not suitable for automated analysis within a mass production. Only by an automated segmentation of the measurement data, the individual measuring points can be assigned in a reproducible and optimal way to the corresponding geometric elements. Two approaches are known for such an automated segmentation:

1.
Neighboring measurement points are rated based on their curvature and assigned to corresponding geometric elements [Wes06]. This method can provide accurate solutions, but it is sensitive to noisy data and not able to distinguish between spheres and cylinders with certain radii.

2.
A holistic approximation can evaluate a composed set of data under the present boundary conditions in a single approximation task [Goc91]. By the definition of separating functions, an optimal assignment of the measurement points to the corresponding geometric elements (segmentation) can be carried out simultaneously. The method is presented for different applications, e.g. for a 2D combination of lines and circles [Lüb10], or for micro punches as a 3D combination of a cylinder, a torus and a plane [Lüb12].
It was proved that the holistic approximation with automated segmentation (second approach) is only slightly sensitive regarding the initial values of the approximation and at the same time converges reliably within wide ranges [Lüb10]. Furthermore, this method was successfully tested for the evaluation of micromeasurements [Zha11], and it allows the automatic detection of outliers by a combination with statistical methods [Gru69]. Thus, the second approach is particularly suited for noisy optical measurement data. However, the algorithms have not yet been implemented for the evaluation of optical data acquired with DHM.
5.2.3.2 Method
The holistic approximation will be described for the evaluation of micro cups, whose surface is acquired by DHM. The part’s geometry is a combination of a cylinder with radius r_{c}, a (quarter) torus with wall radius r_{w} and a plane (see Fig. 5.7). These radii form a vector of shape parameters \( \overrightarrow {\varvec{a}}_{g} \), while the position of the elements is included in a transformation vector \( \overrightarrow {\varvec{a}}_{p} \). The detailed principle of the holistic approximation is described in [Goc91] for 2D combinations and in [Lüb12] for a 3D application. The approximation is performed by minimizing the L_{2} norm
where n_{cyl}, n_{tor} and n_{pla} are the numbers of points assigned to the cylinder, the torus and the plane, respectively. A single point has the index i, and its orthogonal distance to the assigned geometric element is d_{i}. During the approximation, not only the degrees of freedom (transformation \( \overrightarrow {\varvec{a}}_{p} \) and shape parameters \( \overrightarrow {\varvec{a}}_{g} \)) are optimized, but also the assignment of the measurement points to the geometric elements. This implies that the numbers of elements in Eq. 5.7 vary during the iterative calculation. The geometric assignment itself is based on a geometric model, which is presented as a crosssection in Fig. 5.7. It consists of a cylinder with radius r_{c}, whose axis represents the zaxis, a quarter of a torus in the x–yplane with wall radius r_{w} and ring radius r_{r} = r_{c} − r_{w} as well as a plane parallel to the x–yplane at z = −r_{w}. This model contains certain geometric constraints, e.g. coaxiality of the cylinder and the torus axis, which are again perpendicular to the plane, as well as tangential transitions between all elements. These constraints result from the workpiece design and reduce the degrees of freedom to five transformation parameters \( \overrightarrow {\varvec{a}}_{p} = \left[ {\Delta x,\Delta y,\Delta z,\varphi_{x} ,\varphi_{y} } \right] \) and two shape parameters \( \overrightarrow {\varvec{a}}_{g} = \left[ {r_{c} ,r_{w} } \right] \). As the geometry is axially symmetric, the rotation \( \varphi_{z} \) around the zaxis remains disregarded.
Out of the geometric model, the decision rules shown in Fig. 5.8 are derived and implemented in the algorithm. All transformed points with a positive zcoordinate belong to the cylinder. The remaining points are distinguished by their polar radius, points with a radius r_{i} < (r_{c} − r_{w}) are assigned to the plane, and the residual points are assigned to the torus.
5.2.3.3 Verification and Measurement Results
For the verification of the algorithms, the geometry of the measuring object was simulated as a combination of a cylinder, a torus and a plane. The cylinder radius was defined to r_{c,0} = 412 µm and the wall radius of the torus was r_{w,0} = 126 µm. These dimensions were chosen according to the application. The cylinder was formed by 300,000 equidistant points, the torus by approximately 100,000 points and the plane by 379,000 points, each element with a uniformly distributed noise in the normal direction of the nominal surface with different intervals [−a_{e}/2, a_{e}/2] in seven steps between a_{e} = 0.0 … 5.0 µm. Each case was simulated n = 100 times and automatically evaluated by the holistic approximation. The results of the holistic approximation of the simulated data are presented in Fig. 5.9.
To analyze systematic deviations, a one sample ttest is performed with the hypothesis that the approximated radii r_{c} and r_{w} are equal to the set values r_{c,0} and r_{w,0} in the simulation. For this purpose, the radius deviations \( \updelta\,r_{c} = {\text{r}}_{\text{c}}  r_{c,0} \) and \( \updelta\,r_{w} = r_{w}  r_{w,0} \) are introduced and the coverage factors
are calculated for the mean radius deviations \( \overline{{\delta r_{c} }} = \overline{{r_{c} }}  r_{c,0} \) and \( \overline{{\delta r_{w} }} = \overline{{r_{w} }}  r_{w,0} \), respectively, based on the mean approximated radii \( \overline{{r_{c} }} \), \( \overline{{r_{w} }} \) of the cylinder and the torus as well as their standard deviations σ_{c}, σ_{w}.
The maximum value for the cylinder radius is t_{c,max} = 0.74, whereas it is t_{w,max} = 0.92 for the torus radius. According to the tdistribution for a probability of 95% (α = 0.05) and a degree of freedom of f = n−1 = 99, the critical value is t_{crit} = 1.984. As both calculated coverage factors are below this critical value, it can be stated that the verification results do not disagree with the hypothesis with a probability of error of 5%. Thus, it can be assumed that no systematic influence within the holistic approximation leads to significant deviations of both the approximated radii.
The random deviations can be characterized by the standard deviations of the calculated radii. In absolute numbers, the standard deviation of the cylinder radius is σ_{c} < 22 nm in this simulation, while the standard deviation of the torus radius is σ_{w} < 2.87 µm. The random deviations of the torus are 2 orders of magnitude higher than those of the cylinder radius, which is assumed to result from the approximation of only a part of the geometric torus object and agrees with earlier findings, e.g., the error of a spherical center approximation depending on the size of the measured spherical cap [Bou93], or the increased diameter [Fla01] or center uncertainty [McC79] with decreasing arcs of a circle. A second reason for the increased standard deviation of the torus radius might be the number of evaluated points. The torus was simulated with approximately 100,000 points, which is only a third of the number of points on the cylinder. Nevertheless, for both parameters the standard deviation is only a fraction of the initial amplitude of noise due to the high number of data points available.
In order to demonstrate the holistic approximation, the data acquired with DHM were evaluated based on the geometric model in Fig. 5.7. The threedimensional approximation results with a measured cylinder radius of 497 µm and a torus radius of 234 µm are illustrated in Fig. 5.10 for a crosssection through the symmetry axis of the micro cup. Systematic deviations between the measured and approximated surface points occur, which are a result of real deviations of the cylindrical part of the threedimensional micro cup from the desired geometry. Hence, the holistic approximation allows the identification of geometric deviations and, thus, the automatic quality inspection of geometric features in micro production. However, a crucial point is a tailored geometric model. Depending on the inspection task, the degrees of freedom of the geometric model can include the desired geometric parameters only (workpiece quality) or additional parameters for quantifying typical manufacturing errors (manufacturing process).
5.2.4 Detection of Surface Defects
Surface defects such as scratches or dirt might be too small to cause a detectable change in the measured phase distribution. Hence, reliable detection necessitates additional methods, which incorporate the measured amplitude image.
5.2.4.1 State of the Art
Currently, algorithms for automatic surface inspection are to a large extent based on manually engineered features [Xie08], most commonly statistical and filterbased [Neo14]. While the introduction of expert knowledge often allows for the creation of powerful features, this process is laborious and might be necessary for each new product. General solutions that can automatically adapt to new problem sets could thus yield significant time and cost advantages. One such solution is convolutional neural networks (CNN). These have become the driving factor behind many recent innovations in the field of computer vision and allowed significant advances in various applications, such as object classification [Kri12] or semantic image segmentation [Yu15]. CNNs have also recently been successfully applied for industrial surface inspection [Wei16].
5.2.4.2 Methods
The core building block of CNNs is the convolutional layer. Instead of processing, e.g. an image all at once, it is divided into small (usually) overlapping windows and fed piecewise into a neural network. Each window is thus mapped to a vector of activations of a shared neural network. Convolutional layers hence automatically learn a set of filters in the form of the networks weights.
In the most common framework, convolutional layers are combined with pooling layers, usually maxpooling layers. Maxpooling layers summarize the extracted features by taking the maximum activation for each unit over a small area. Deep CNNs are built by stringing together multiple convolutional and pooling layers. With increasing depth, the network thus extracts increasingly complex features for increasingly large image areas or receptive fields. The application of maxpooling thereby yields multiple advantages. By scaling down the input, the number of parameters is decreased, which increases the computational efficiency. At the same time, the receptive field sizes are increased and hence the amount of context that can be integrated by each unit of the neural network. Additionally, the use of maxpooling yields a small degree of translation invariance, which increases the network’s robustness towards these operations. One disadvantage, however, is that, by scaling down the input, spatial resolution gets lost. This becomes an issue when the goal is spatially precise defect detection. To solve this problem, multiple solutions have been proposed in the field of semantic image segmentation, e.g. the use of dilated convolutions [Yu15], the UNet architecture [Ron15] and the LinkNet architecture [Cha17].
One solution is to augment the classical CNN architecture with a second network for upscaling the spatial resolution. Highlevel, lowresolution features are thereby upsampled and merged with the corresponding lowlevel, highresolution features. This architecture is largely known as UNet [Ron15]. The advantage is that it harnesses the benefits of maxpooling while still being able to give precise defect labels.
Here we implemented a modified version of the UNet. Aside from accurate defect detection, our objective was thereby to keep the hardware requirements and computing time of the network as low as possible. To achieve this goal, we employed three recently developed methods. Firstly, our architecture is heavily inspired by densely connected CNNs [Hua17]. Secondly, we opted for depthwise separable convolutions [Cho16]. Finally, our network takes inspiration from the LinkNet architecture [Cha17].
The basic idea behind densely connected CNNs is to feed into each new layer the activation of each previous layer. This allows a significant decrease of the number of connections in each layer as all the information from previous layers can be directly accessed instead of having to be repeated over each layer. Figure 5.11a illustrates the blocks of densely connected layers used in this work. Each layer propagates its activations to all the successive layers within a block. Filter sizes were chosen to be 5 × 5 for layers one and three in each block and 1 × 1 for layers two and four as well as the final output layer. Each layer thereby uses h units. In our experiments h was set equal to 12. As a bottleneck, we only concatenate the activations of all layers within the block to produce the blocks output, i.e. the input to a block is not propagated after the block. Implementation details are given in Table 5.1.
The second method for increasing the model efficiency is the use of depthwise separable convolutions [Cho16]. The principle behind depthwise separable convolutions is that, instead of performing convolutions over all channels within a spatial window simultaneously, the spatial and the depth/channelwise convolutions are performed separately. This allows for a significant decrease in the amount of network connections.
The third method for increasing the model efficiency takes inspiration from the LinkNet architecture [Cha17]. There are two ways in which LinkNet increases the efficiency of the standard UNet architecture. Firstly, high and lowlevel features are merged via addition instead of concatenation. Secondly, the number of features is also reduced before the summation. Our implementation of this procedure is shown in Fig. 5.11b.
The final network consists of a downsampling and upsampling part. The spatial resolution of the input image is thereby downsampled from 512 × 512 to 8 × 8 and then upsampled again to 512 × 512. The downsampling part consists of densely connected blocks followed by a 4 × 4 maxpooling operation and batch normalization [Iof15]. In the upsampling part, we use decoder blocks as illustrated in Fig. 5.12 to efficiently integrate low and highlevel features.
As activation functions, we used exponential linear units (elu) [Cle15] in all but the output layers. For the output layer, we used sigmoid units to constrain the output to the interval [0, 1]. The network was trained by minimizing binary crossentropy (also known as logloss) using the Adam optimizer [Kin14]. The learning rate was initialized at 0.001 and automatically reduced by a factor of 0.1 when no decrease in loss was observed for more than 10 epochs. The minibatch size was set to eight. All experiments were conducted using the keras library for Python.
To improve the defect labeling, we automatically created a mask for background subtraction for each measurement. The steps are shown in Fig. 5.13. First we applied a strong lowpass filter to the measurement (i.e. convolution with a 55 × 55 matrix of ones). We binarized the resulting image using its mean as a threshold. In the resulting binary matrix, the largest contour was detected using methods provided by the OpenCV software library [Bra00]. The background mask was directly fed into the CNN and multiplied with its output layer. This allowed the network to learn features only for the relevant parts of the image.
We evaluated our method by using 64 samples for training and the remaining 5 samples for evaluation. To increase the amount of training data artificially, we used the following operations: horizontal flipping, vertical flipping, random rotations, and scaling the size by a factor between 0.9 and 1.1 (cropping or adding the additional/missing pixels at the boundaries).
5.2.4.3 Validation
Figure 5.14 shows the resulting defect maps for the testing data. All defects are detected and marked correctly. However, it should also be noted that the defect labels are still rather coarse, especially around the borders.
Defect detection for a single input image takes <130 ms on our test system (AMD Ryzen Threadripper 1900X 8Core Processor × 16, 64 GB RAM, GeForce GTX 1080 Ti). Additional speed gains can be achieved by processing multiple images at once, as this would decrease the amount of data transfer towards and from the Graphics Processing Unit (GPU).
Acknowledgements The editors and authors of this book like to thank the Deutsche Forschungsgemeinschaft DFG (German Research Foundation) for the financial support of the SFB 747 “Mikrokaltumformen—Prozesse, Charakerisierung, Optimierung” (Collaborative Research Center “Micro Cold Forming—Processes, Characterization, Optimization”). We also like to thank our members and project partners of the industrial working group as well as our international research partners for their successful cooperation.
5.3 Inspection of Functional Surfaces on Micro Components in the Interior of Cavities
Aleksandar Simic*, Benjamin Staar, Claas Falldorf, Michael Lütjen, Michael Freitag and Ralf B. Bergmann
Abstract A fast and precise solution for the inspection of the interior of micro parts using digital holography is presented in this chapter. The system presented here is capable of operating in an industrial environment. For this purpose, a compact Michelson setup in front of the imaging optics is used, so that the light paths of the object and reference arm are almost identical. This makes the system less vulnerable to mechanical vibrations. A further improvement is obtained using the twoframe phaseshifting method for the recording of a complex wave field. This enables the usage of two cameras in order to allow the recording of a complex wave field in a single exposure. With the help of twowavelength contouring, optically rough objects with a synthetic wavelength of approximately 93 µm are investigated. The measurement results make it possible to determine the shape of the interior surface and faults such as scratches with a resolution of approximately 5 µm. In order to fully utilize the measurement speed of the setup, a fast and reliable solution for automatic defect detection is required. For a profitable industrial application, it is therefore crucial to reliably detect all defective parts while producing little to no false positives (i.e. pseudorejections). This is realized by utilizing prior knowledge about the object shapes to implement fast phase unwrapping for defect detection. Defects are then reliably detected by identifying consecutive areas of deviation in relative depth. The evaluation of measurements taken in an industrial environment shows that this approach reliably detects all defects with a falsepositive rate of less than one percent.
Keywords Quality control · Optical monitoring · Digital holography
5.3.1 Introduction
Micro cold formed parts are produced in high quantities, as many of such are incorporated inside a complete system. The mass production of micro parts can only be efficient if the quality inspection of these parts is incorporated within the production line. Optical metrology offers the opportunity to determine the shape of the structures of such parts and allows for quality control. Automated quality control with the help of an automated optical system within the industrial production line reduces the costs and time that would otherwise be required for a sophisticated manual inspection. Up to now, tactile methods have been used to inspect components on a sample basis, but these are not suitable for fast quality inspection in the production line as they are too slow and might alter the sample.
Among the nontactile methods, confocal microscopy is commonly used for inspection but is clearly too slow for an automated 100% inspection. An overview of such methods can be found in [Ber12, Kop13]. Alternatively, whitelight interferometry (WLI) is suitable, as it measures with high speed and is highly precise. For a review of WLI, see [Gro15]. However, WLI requires a comparatively large number of recordings, commonly by depth scanning, to capture depth information.
Digital holography (DH) is precise and only requires a small number of recordings to obtain the object shape, as shown in [Fal15]. This makes it a good candidate for the fast threedimensional inspection of micro parts. Usually DH uses the method of temporal phaseshifting for phase evaluation, which is generally realized with a piezoelectric device. To realize a system which exhibits an even higher robustness, the method of twoframe phaseshifting is used to measure the object shape in two consecutive exposures.
5.3.1.1 Digital Holography
Historically, an interference pattern of an object and reference wave was recorded on holographic plates and evaluated afterwards. However, such holographic plates can only be used a single time and require wet chemical processing. With the rise of computational methods and imageprocessing methods with the use of CCD and CMOS cameras, classical holography was replaced by digital holography and digital holographic microscopy (DHM) for the inspection of microscopic objects. An interference pattern arising from the light scattered by the investigated object and a reference wave is recorded in the CCD plane to extract the complex wave field of the object under investigation (see Fig. 5.15).
The intensity distribution of the interference pattern is given by
where \( {\vec{\text{r}}} \) is the position vector and \( {\text{U}}_{\text{O}} \left( {{\vec{\text{r}}}} \right) \) and \( {\text{U}}_{\text{R}} \left( {{\vec{\text{r}}}} \right) \) are the object and referencewave fields respectively, with \( {\text{U}}_{\text{O}}^{ *} \left( {{\vec{\text{r}}}} \right) \) and \( {\text{U}}_{\text{R}}^{ *} \left( {{\vec{\text{r}}}} \right) \) being the particular conjugated wave fields. When multiplying this interference pattern with the complex amplitude of the reference wave, the following fundamental equation is obtained:
The first term of Eq. 5.11 represents the DC term, which can be observed in the center of the recorded picture as the brightness of the image. The second term describes an inverted image \( {\text{U}}_{\text{O}}^{ *} \left( {{\vec{\text{r}}}} \right) \cdot {\text{U}}_{\text{R}} \left( {{\vec{\text{r}}}} \right)^{2} \) of the object. The last term represents a virtual image of the object. The phase Φ of the wave field \( {\text{U}}_{\text{O}} \left( {{\vec{\text{r}}}} \right) \) contains information on the form of the object. The height h of the observed object can be calculated with the phase Φ from
5.3.1.2 TwoWavelength Contouring
Twowavelength contouring is an established method for the form recognition of diffusely reflecting objects. Falldorf et al. managed to significantly enhance the signaltonoise ratio of the holographic measurement of optically rough objects with the help of this method [Fal12]. For objects that have a roughness larger than half of the wavelength of the illuminating light, there will be ambiguous results according to Eq. 5.12, as the phase only varies in the range of −π to π. In twowavelength contouring, two light sources are used to perform the measurement with slightly different wavelengths, \( \uplambda_{1} \) and \( \uplambda_{2} \). The corresponding phase distributions are subtracted and the resulting phase difference can be interpreted as a single measurement with a synthetic wavelength [Fal15] of
The synthetic wavelength can be chosen to be much larger than the surface roughness by adjusting \( \uplambda_{1} \) and \( \uplambda_{2} \) to resolve the ambiguity problem.
5.3.1.3 TwoFrame PhaseShifting
The recorded digital hologram from Eq. 5.11 only contains object information in the virtual image, and the remaining inverted image and DC term are generally not of interest. Therefore, only a small part of the camera resolution can be used with this method. To use the complete spatial bandwidth, the method of temporal phaseshifting is used, where the phase is shifted by a fractional amount of the wavelength to generate several equations and extract detailed phase information.
The recorded interference pattern from Eq. 5.10 can be expressed with the help of the phase difference \( \Delta\Phi \) of the involved wave fields, to extract 3Dinformation on the considered object by using
This equation contains the three unknown variables \( {\text{I}}_{\text{O}} \left( {{\vec{\text{r}}}} \right),{\text{I}}_{\text{R}} \left( {{\vec{\text{r}}}} \right), \Delta\Phi , \) which makes it impossible to extract the phase difference \( \Delta\Phi \). To solve this problem the phase difference \( \Delta\Phi \) of the interference pattern produced is shifted with a known factor \( \Delta \) to obtain a system of at least three equations from the corresponding recorded intensities. This process is generally accomplished with the help of a piezoelectric device in the setup. The resulting phase distribution is wrapped in the bounded interval [0, 2π] and has to be unwrapped to determine the continuous behavior.
To incorporate this method in an industrial environment, its robustness is improved by replacing the piezoelectric device. To maintain the complete space–bandwidth product of the detected signal, it is vital to find a way of using temporal phaseshifting in a single camera exposure. For this purpose, the temporal phaseshifting method is used with the help of only two recorded interference patterns. This is accomplished by using circular polarized light from the object and linear polarized light from the reference mirror. Nozawa et al. have already used this system for singleshot and highly accurate measurements of complex amplitude fields with a simple optical setup [Noz15].
For the recorded interference patterns in the CCD plane, the wave field \( {\text{U}}_{\text{G}} \left( {{\vec{\text{r}}} = {\text{z}}_{0} } \right) \) in the CCD plane can be written as
with \( I_{1} \) and \( I_{2} \) being the phaseshifted single recorded interference patterns shifted by \( \pi /2 \), respectively. Liu et al. showed that the DC term \( {\text{I}}_{0} \) is given by [Liu09]
with the amplitude of the reference wave R.
5.3.2 Experimental Alignment
For the inspection of the interior of a micro deep drawing component with DHM, a Michelson interferometer based setup, shown in Fig. 5.16, is constructed with the object in one arm and a plane mirror in the other arm to provide a reference wave. The Michelson interferometer has been placed as close to the object as possible in order to minimize the lengths of the separated light paths. The object is introduced to the setup with the help of a positioning apparatus, constructed by the industrial partner Stüken Corp. Furthermore, the usage of polarization optics and two cameras enables the simultaneous recording of two phaseshifted images. In this configuration, the setup is more stable with respect to exterior disturbances compared to a setup using a piezoelectric device.
After leaving the fiber switch, the light is parallelized with the help of a collimating lens and is linearly polarized. A λ/4 plate then polarizes the light circularly and illuminates the object through a beam splitter. At the same time, half of the intensity is redirected and linearly polarized again to illuminate the reference mirror. After traveling through a microscope with 5x magnification, the light is again divided with the help of a polarizationsensitive beam splitter to lead it to two camera targets at the same time. By using such a beam splitter, two interference patterns are projected on the camera targets, shifted by 90°.
As light sources, two diode lasers with output powers of \( P_{1} = 70 \) mW and \( {\text{P}}_{2} = 20 \) mW and with wavelengths \( \uplambda_{1} = 642.2 \) nm and \( \uplambda_{2} = 637.8 \) nm are used. With that, the camera exposure times were set to 8 ms to fully illuminate the camera targets. With the employed wavelengths, the synthetic wavelength amounts to \( \Lambda = 93.1 \) µm. To avoid coherent amplification and to minimize speckle noise, laser light with coherence lengths of less than 1 mm is used.
5.3.2.1 Experimental Results
Figure 5.17 shows a sketch of the investigated parts, which have a functionality area that is vital for the component. This area must not exhibit scratches or imperfections and therefore has to be inspected. 247 parts were inspected, of which 230 were defectfree and 17 were bad parts which were identified prior to the measurement.
The setup was implemented in the department for quality assurance of Stüken Corp., a producer of micro deep drawing parts. The recorded phase distribution displays the inner form of the object and allows for a classification of the functionality. Figure 5.18a shows the phase of the functional area of the component presented above. Two consecutive fringes indicate a height difference of Λ/2 on the object surface. Any deviation of the concentric inner form shows potential faults in the functional area, which can be seen in Fig. 5.18b, for example.
The phase can be unwrapped to obtain metric data with the help of Eq. 5.12. The unwrapped area around the defect can be seen in Fig. 5.19. Eight points on the dashed red line inside the defect were considered and compared with eight points on the parallel dashed red line outside the defect to calculate the mean depth of the error. Figure 5.19 contains measured values for the phase difference of the measured wave field. To convert the measured values into metric data, Eq. 5.12 is used with respect to a doubled light path as the light is reflected from the object. Then the mean depth d of the error can be calculated using the averages of the phase difference \( \Delta\Phi _{1} ,\Delta\Phi _{2} \) at the dotted lines of
With this system, a lateral as well as a depth resolution of 5 µm can be achieved.
5.3.2.2 Comparison with XRay Tomography
For validation, the functionality area of the bad part from the last section was inspected using Xray tomography. Figure 5.20 shows the measurement result, which depicts a scratch with a depth of 23 µm at the marked spot. With that, the result from Eq. 5.16 can be validated.
5.3.2.3 Different Batches of Material
Different batches of materials were investigated with the presented system to evaluate the method. These were glossy parts, oily parts and heattreated parts. A comparison of those three material types is shown in Fig. 5.21. Especially glossy parts do not scatter the incoming light and directly reflect most of it instead. Therefore, the signaltonoise ratio decreases especially for the glossy parts, making further evaluation therefore impossible.
5.3.3 Automatic Defect Detection
For the effective utilization of the setup’s measurement speed, manual evaluation of the measurements is not feasible. Hence a solution for automatic defect detection was developed. Thus the challenge was threefold: Firstly, the method had to be fast, as a slow algorithm would be detrimental to the fast measurement system. Secondly, defect detection had to be very accurate, with zero false negatives (undetected defects) and less than 4% false positives (intact parts falsely labeled as defective). Thirdly, due to the well optimized process, the number of defective samples was very small. Consequently, the application of stateoftheart machine learning methods, like e.g. convolutional neural networks, which have been applied by Ronneberger et al. [Ron15] and Weimar et al. [Wei16] for example, was not feasible.
The detection of the measured part is achieved by detecting circles in the phase distribution \( \Delta\Phi \). Potential defects are then filtered out by applying a lowpass filter (LPF) in a circular motion. The resulting prototype is then subtracted from the measured \( \Delta\Phi \) to identify deviations via the application of a threshold. The whole defect detection pipeline is schematically shown in Fig. 5.22.
5.3.3.1 Preprocessing
In order to decrease noise, \( \Delta\Phi \) was filtered by applying a sin/cos LPF. Since \( \Delta\Phi \) is a phase distribution, lowpass filtering in the complex plane may be applied, i.e. to the complex phasor \( {\text{f}}_{\text{c}} \left( {\text{x}} \right) = {\text{e}}^{{{\text{if}}\left( {\text{x}} \right)}} \) which exhibits unit amplitude and \( {\text{f}}\left( {\text{x}} \right) \) as the phase. This approach prevents filter artifacts at the phase transitions.
5.3.3.2 Part Detection
While objects are positioned close to the center of the measured image, there are significant deviations that do not allow the assumption of a fixed center without notable loss of accuracy. For the problem at hand, the measured part of the object is circular and the measurement is taken orthogonally. This is manifested in the resulting phase distributions as nearly concentric circles. The measured part’s center can hence be located by the detection of these circles. This was implemented by first applying a strong sin/cos LPF to \( \Delta\Phi \), yielding \( \Delta\Phi ^{{\prime }} \) and subsequently applying a binary threshold to \( \Delta\Phi ^{{\prime }} , \) resulting in two images \( {\text{T}}_{1} \) and \( {\text{T}}_{2} \) with:
An example is shown in Fig. 5.22b. Subsequently, contours in \( {\text{T}}_{1} \) and \( {\text{T}}_{2} \) are detected and sorted by area. The largest contours are then fitted by their minimal enclosing circle. As a robust estimate of the object’s center, the median of the resulting centers is taken, yielding the center \( ({\text{c}}_{\text{x}} ,{\text{c}}_{\text{y}} ) \).
5.3.3.3 Prototype Creation and Phase Unwrapping
In order to detect defects, the measured phase distribution \( \Delta\Phi \) is compared to an ideal prototype of that measurement. Expecting the measured part to have a smooth surface, prototype creation is achieved by the application of a sin/cos lowpass filter in a circular motion. One could think of this as virtually regrinding the object to smooth out defects. The detailed steps for this process are as follows:
First \( \Delta\Phi \) is mapped to polar coordinates with respect to the object’s center detected in the previous step. The result is an \( \upbeta \times {\text{r}} \) image \( \Delta\Phi ^{polar} \) whereby \( \upbeta \) marks the angular resolution and r the radius.
Lowpass filtering in a circular motion can thus be achieved by applying a sin/cos LPF to \( \Delta\Phi ^{polar} \), yielding \( \Delta\Phi ^{prototype} \) (for an example see Fig. 5.22c). The \( m \times n \) filter matrix was thereby chosen to be much larger in angular direction m than in radial direction n. The reasoning is that, due to phase transitions, highfrequency components are expected in the radial direction even for smooth surfaces. In the angular direction, however, a smooth surface should only exhibit lowfrequency components, as there should not be any phase transitions. Deviations in depth can thus be calculated via (Fig. 5.22d)
5.3.3.4 Defect Detection
Potential defects are marked by deviations from zero in \( \Delta {\bar{\text{d}}} \). Due to roughness of the measured part’s surface that lies within tolerance, there might exist multiple such areas, even for intact parts. To differentiate between this background noise and actual defects, two different features are used. Firstly, errors are assumed to be marked by larger connected areas of deviations from zero, i.e., the area of an actual defect exceeds a certain threshold. Secondly, it is assumed that for defective areas the mean deviation from the background exceeds a certain threshold.
Accordingly, the defect detection routine searches for connected areas of deviations from zero in \( \Delta {\bar{\text{d}}} \) with areas above an area threshold \( {\text{t}}_{\text{A}} \) where the mean deviation exceeds a depth threshold \( {\text{t}}_{\text{D}} \). An example is shown in Fig. 5.22e.
5.3.3.5 Detecting Loss of Focus
Larger defects in the measured part’s geometry as well as environmental influences might cause the measured phase distribution to be out of focus. However, the defect detection routine does not necessarily capture this case. Therefore, an additional method for detecting loss of focus was implemented. As a marker for focus, the orientation of gradients in \( \Delta\Phi ^{polar} \) is employed. The underlying assumption is that well focused parts of \( \Delta\Phi ^{polar} \) show homogeneous orientation of gradients, while unfocused areas show gradient orientations that are more or less random. Focused areas are hence marked by low standard deviation in the gradient orientations, while unfocused areas are marked by large standard deviation. The gradient orientation in \( \Delta\Phi ^{polar} \) is calculated in the following way. First, the Sobel derivatives [Sob90] are calculated by convolution with Sobel operators \( {\text{S}}_{\text{x}} \) and \( {\text{S}}_{\text{y}} \), resulting in
and
Large derivatives caused by phase transitions (Fig. 5.23b) are then removed by looking for entries in \( G_{x} \) and \( G_{y} \) with absolute values above five times the median absolute deviation (mad), i.e. \( \left {G_{x} } \right > 5mad\left( {\left {G_{x} } \right} \right) \) and \( \left {G_{y} } \right > 5mad\left( {\left {G_{y} } \right} \right) \) respectively. The identified values are then replaced by the respective median value (Fig. 5.23c).
Subsequently, the gradient orientation (Fig. 5.23d) is calculated via
Then the local standard deviation \( std\left(\Theta \right) \) of \( \Theta \) over windows of \( 25 \times 25 \) pixels is calculated
with a \( 25 \times 25 \) unit matrix \( K \) as the convolution kernel. The normalized sum f of the values in \( {\text{std}}\left(\Theta \right) \) then serves as an indicator value on how focused the image is (Fig. 5.23e):
If \( f \) exceeds a certain threshold \( {\text{t}}_{\text{focus}} \), the measurement is said to be out of focus.
5.3.3.6 Results
In total, this defect detection routine has the four parameters shown in Table 5.2 with the thresholds for area \( {\text{t}}_{\text{A}} \), depth \( {\text{t}}_{\text{D}} \) and focus \( {\text{t}}_{\text{focus}} \). After evaluating different settings manually, the set of parameters shown in Table 5.2 was used for further work
Defect detection was evaluated on 296 measurements of 247 parts (230 parts of those were previously inspected and found to be acceptable and 17 parts were identified as bad parts) in the department of quality assurance of Stüken Corp. with a measurement speed of approximately one part per second. Defective parts were all measured at least 3 times with different orientations to verify reproducible defect detection. Out of the 296 measurements, all the defective parts were reliably detected (true positive), while eleven intact parts were sorted out. Of these eleven parts, nine were correctly sorted out due to the measurement being out of focus, leaving two false positive detections.
Acknowledgements The editors and authors of this book like to thank the Deutsche Forschungsgemeinschaft DFG (German Research Foundation) for the financial support of the SFB 747 “Mikrokaltumformen—Prozesse, Charakerisierung, Optimierung” (Collaborative Research Center “Micro Cold Forming—Processes, Characterization, Optimization”). We also like to thank our members and project partners of the industrial working group as well as our international research partners for their successful cooperation.
5.4 In Situ Geometry Measurement Using Confocal Fluorescence Microscopy
Merlin Mikulewitsch* and Andreas Fischer
Abstract Due to the challenging environment of micro manufacturing processes such as laser chemical machining (LCM) where the workpiece is submerged in a fluid, a contactless in situ capable measurement is required for quality control. However, the in situ geometry measurement has several challenges for optical measurement systems because the high surface gradients of the micro geometries and the fluid environment complicate the use of conventional metrology. Confocal fluorescence microscopy allows for the determination of the surface position by adding an isotropically scattering fluorophore to the fluid and detecting the signal drop at the boundary layer between the measured object and the fluid. This technique, capable of improving the measurability of metallic surfaces with strong curvatures, is evaluated for suitability as an in situ measurement method for the LCM process. Unlike in thinner layers, however, the signal with fluid layers ≥1 mm, as needed for LCM in situ applications, shows strong dependencies on the fluorophore concentration and fluid depth. Thus, a physical model of the fluorescence intensity signal was developed for the evaluation of the surface position. To validate the method for the in situ measurement of geometry parameters, the step height of a submerged reference step was determined by measuring the surface positions along a line over the step. The step height measurement results in an uncertainty of 8.8 μm that is verified by deriving the potential measurement uncertainty of the modelbased measurement approach. Further investigation of the uncertainty budget will allow a reduction of the measurement uncertainty and enable in situ monitoring and control of the LCM process.
Keywords In situ measurement · Confocal microscopy · Signal modeling
5.4.1 Challenges of Optical Metrology for InProcess and in situ Measurements
Laser chemical machining (LCM) is a promising alternative process that allows for inexpensive manufacturing of micro geometries in hard metals, such as dies for micro forming, without heat damage or structural alterations to the material [Mik17]. Laser chemically machined geometries can reach structure sizes between \( 10\,\upmu{\text{m}} \) and \( 400\,\upmu{\text{m}} \), with steep slopes and a surface roughness of up to \( 0.3\,\upmu{\text{m}} \) [Ste10]. However, factors in the process environment, such as chaotic thermal interactions between the fluid and workpiece geometry, complicate the manufacture of a desired geometry, necessitating a closedloop quality control [Zha17] with an in situ measurement feedback to improve the manufacturing quality (see Sect. 4.3). The challenging conditions of the LCM process, such as the requirement that the workpiece needs to be submerged by a fluid layer (typically 1‒40 mm thick), hinder the in situ application of many measurement methods. The general lack of accessibility to the workpiece, for instance, requires the use of contactless measurement methods based on optical acquisition.
Conventional microtopography measurement techniques can be separated into interferometric methods (e.g. displacement interferometry, digital holography [Kop13]) and other techniques, such as conventional laserscanning confocal microscopy [Han06]. Conventional confocal microscopy is hindered, however, by the in situ conditions of high surface angles of the specimen [Liu16]. Interferometric methods were also investigated as a means of control feedback [Zha13], but were found to be unsuitable: The tested measurement systems integrated the interferometer directly into the machining head of the laser jet system as a twobeam interferometer according to the Michelson principle in order to increase the signal strength. The measuring beam was guided coaxially to the etchant and processing beam onto the surface of the workpiece. To obtain the path difference from the interferogram, the number of interference fringes was determined with phototransistors. If the measuring and reference arms are in different ambient media, a correction with the refractive indices of the media is also required. Evaluating the interferometer with samples of different surface roughness, it was determined that the measuring signal strength decreases with increasing surface roughness [Ger10]. In the end however, successful in situ measurement application proved to be unfeasible due to the formation of thermal gradients and gas bubbles that act as moving micro lenses and cause strong disruptions of the measuring beam [Ger10]. Thus, a suitable in situ measurement method capable of dealing with the processinduced currents, thermal gradients, and refractive index fluctuations is needed to improve the feasibility and acceptance of laser chemical machining as a competitive manufacturing process. A method based on the confocal detection of the fluorescence emitted by the fluid shows promise for in situ measurement application. The measurement is based on detecting the boundary position of the specimen surface and the fluid through the change in fluorescence signal while the confocal detection volume is scanned vertically through the fluid [Mic14].
5.4.2 Principle of Confocal Microscopy Based Measurement
The principle of measurement as shown in Fig. 5.24 is based on the detection of the fluorescence intensity emitted from the fluid covering the specimen using a confocal microscopy setup [Mik18]. The light of a green diode laser (λ = 532 nm) is expanded by a Keplerian beam expander and redirected by a beam splitter to the objective lens (NA = 0.42, WD = 20 mm), exciting the fluorescent fluid (aqueous solution of Rhodamine B) submerging the specimen. The specimen container is positioned using a 3axis linear stage to enable the scanning of the focus position through the fluid. Only the fluorescence light emitted by the fluid (λ = 565 nm) is collected by a chargecoupled device (CCD). The confocal principle causes light not originating from a volume around the focus of the objective (confocal volume) to contribute less to the detected fluorescence signal. Scanning the confocal volume of the excitation laser vertically (in the \( z \)direction) through the fluorescent fluid produces a characteristic fluorescence intensity signal (see Fig. 5.24, bottom right).
Since the excitation light is filtered out, only light that is emitted by the fluid inside the confocal volume is detected. Only for values of z inside the boundaries of the fluid (z_{0 } < z < z_{1}, see Fig. 5.24) will a significant signal be produced, since no fluorescent fluid is present to generate light when the confocal volume is fully located in either air or the specimen. The signal does not decay abruptly at the boundary but gradually, depending on the vertical extent of the confocal volume. The exact determination of the surface position z_{0} is not trivial, as opposed to the case of very thin fluid layers, where the depth response is more similar to that of conventional confocal microscopy where the intensity peak corresponds directly to the surface position. For the case of thicker fluid layers, the properties of the fluorescence signal depend strongly on the fluorophore concentration and the fluid depth. With high fluorophore concentrations or thick fluid layers, the Lambert–Beer law of absorption causes less excitation light to reach far into the fluid, resulting in the decay of the fluorescence signal before the confocal volume reaches the specimen surface. This effect is negligible for thin fluid layers, but needs to be taken into consideration when choosing the fluorophore concentration for measurements in thicker layers. For the purpose of determining the surface position of the specimen from the fluorescence signal, a physical model of the fluorescence signal is used.
5.4.2.1 Model Assumptions
In order to solve the inverse problem of how to determine the surface position of the specimen from the acquired fluorescence intensity signal, a model of the fluorescence signal formation was developed [Mik18]. The model is based on several assumptions:

1.
The detected fluorescence intensity is only generated in the confocal volume

2.
The shape of the confocal volume is simplified to a 3D Gaussian function

3.
The shape of the confocal volume is not affected by refraction

4.
The specimen surface is nonreflective

5.
The fluid surface does not move

6.
A constant and uniform fluorophore concentration

7.
A constant excitation power

8.
The confocal volume is cut off by the horizontal surface element
The model assumptions are the source of model uncertainties that propagate into the uncertainty of the geometry parameter determination. However, it could be shown [Mik18] that even this simplified model is capable of enabling the surface position to be determined within thick fluid layers. The advantage of these simplifications is the existence of a closed mathematical formula to describe the fluorescence intensity signal (see Eq. 5.27).
5.4.2.2 Model Description
To model the fluorescence signal, the confocal volume in which it is generated needs to be described first. Since the confocal microscope only detects light from inside the confocal volume, the signal rapidly decreases if this volume moves outside the fluid that generates the fluorescent light. The spatially distributed contribution of each infinitesimal volume element to the detected fluorescence light power can be characterized in a first approximation by the threedimensional Gaussian function [Rüt08]
\( \text{with}\,\varvec{r} = \left( {\begin{array}{*{20}c} x \\ y \\ \end{array} } \right). \)
This confocal volume function has a width of \( w_{0} \) in the xydirection and \( \kappa w_{0} \) in the zdirection, where \( \kappa > 1 \) is a constant factor dependent on the confocal setup. The parameter \( I_{0} \) describes the maximum fluorescence light power determined by the excitation power and fluorophore concentration. Because the signal is generated by scanning the confocal volume through the fluid, a weighting factor, which is zero outside the boundaries of the fluid and follows the Lambert–Beer law of absorption inside the fluorophore, needs to be considered. The fluorescence intensity signal \( I_{F} \left( z \right) \) detected at position z (see Fig. 5.25) is obtained by integrating the total contribution described by weighting the confocal volume function over all dimensions. The integral over the confocal volume function can be thought of as a vertical (z) convolution of the horizontal (x, y) integral \( \displaystyle\int \limits_{  \infty }^{\infty } I\left( {\varvec{r},z} \right){\text{d}}\varvec{r} \) with the weighting function \( \eta \left( z \right) \) of the fluid
Evaluating the convolution integral from Eq. 5.26 with the confocal volume function \( I\left( {\varvec{r},z} \right) \) from Eq. 5.25 gives the model function of the fluorescence intensity signal \( I_{F} \left( z \right) \) as
The surface position \( z_{0} \) is then determined by a nonlinear regression of the measured fluorescence intensity signal with the model function \( I_{F} \left( z \right) \) using a least squares method. The approximation parameters are the amplitude \( \tilde{I}_{0} \), the offset \( C \), the fluorophore concentrationdependent attenuation coefficient \( \varepsilon \), the confocal volume shape parameter \( \xi \) and the position parameters \( z_{1} \) (fluid surface) and \( z_{0} \) (specimen surface).
5.4.3 Experimental Validation
The in situ measurement technique was validated by measuring the geometry parameter step height of a referenced step object [Mik18]. The fluorescence intensity signal resulting from the measurement of a single point on the stepspecimen is shown in Fig. 5.25.
In order to obtain the geometry parameter step height, the fluorescence intensity signal (see Fig. 5.25 for the signal of a single point) of 23 points along a line perpendicular to the edge of the stepspecimen was acquired (i.e. performing a \( z \)scan for each \( \left( {x,y} \right) \)point). The surface position \( z_{0} \) was determined from each measured intensity signal, with a leastsquares approximation using Eq. 5.27. The resulting specimen surface positons \( z_{0} \) are shown in Fig. 5.26. After correcting the tilt of the step and the focus shift due to refraction at the fluid surface, the step height \( h \) was determined by the difference of the mean surface positions of each step surface
Comparing the step height result of \( h = \left( {258.2 \pm 8.8} \right)\,\upmu{\text{m}} \) with the tactile reference measurement of \( \left( {253.5 \pm 0.2} \right)\,\upmu{\text{m}} \) shows no significant systematic deviations. Since the positions on each step surface show a relatively large stochastic scattering (up to 20%), the uncertainties are most likely caused by the general surface condition or uncertainties in the fitting model. The measurement technique based on confocal fluorescence microscopy was thus shown to be capable of determining the geometry parameter step height for microstructures submerged in thick fluid layers >100 \( \upmu{\text{m}} \), which demonstrates the suitability of the modelbased approach for in situ application. However, the sources of the uncertainty of \( 8.8\,\upmu{\text{m}} \) need to be further characterized in order to reduce it to the desired \( 1 \,\upmu{\text{m}} \). In order to find the lower boundary of uncertainty for the confocal microscopybased geometry measurement of submerged microstructures, a determination of the measurement uncertainty with the approximation of the signal model is necessary.
5.4.4 Uncertainty Characterization
To obtain the fundamental uncertainty of the surface position \( \sigma_{{z_{0} }} = \sqrt {{\text{Var}}\left( {z_{0} } \right)} \) from the measurement with the nonlinear least squares approximation method, the covariance matrix of the estimator
based on the fit function \( I_{F} \left( z \right) \) (see Eq. 5.27) needs to be calculated. Applying an uncertainty propagation calculation to the least squares estimator gives the following relation for the estimator’s covariance matrix [Kay93]:
where \( H_{{I_{F} }} \) denotes the Jacobian matrix with the partial derivatives of the approximation function with respect to \( {{\hat{\uptheta }}}_{{I_{F} }} \) at each position \( z_{i} \) of the measured fluorescence intensity signal \( \tilde{I}_{F} \left( {z_{i} } \right) \), and \( {\text{C}}_{{\tilde{I}_{F} }} \) the covariance matrix whose main diagonal contains the variance \( \sigma_{{I_{\text{F}} }}^{2} \) of each \( z_{i} \) of the fluorescence signal \( \tilde{I}_{F} \left( {z_{i} } \right) \), since the covariance between the individual values is assumed to be zero.
To calculate the surface position uncertainty \( \sigma_{{z_{0} }} \) of the measurement using the nonlinear regression of the model function \( I_{F} \left( z \right) \) (see. Eq. 5.27), the average variance of the measured fluorescence signal around the model curve, i.e. \( \sigma_{{I_{F} }}^{2} = 1.26 \cdot 10^{9} \) (in units of detected photons), is used. The calculation results in an uncertainty of 8.56 μm for the surface position \( z_{0} \) and 3.75 μm if propagated into an uncertainty for the step height (based on 23 surface position measurements, according to the results from Fig. 5.26). The uncertainty of the experimental result of 8.8 μm is still larger than the calculated uncertainty by a factor of 2.3, suggesting varying conditions during the measurement, such as the form of the fluid surface or the micro topography of the specimen. To analyze the effects of the parameters fluid depth \( d_{f} = z_{1}  z_{0} \) and concentrationdependent attenuation coefficient \( \epsilon \), the uncertainty \( \sigma_{{z_{0} }} \) is determined as a function of the fluid depth \( d_{f} \) for different attenuation coefficients \( \epsilon \), shown in Fig. 5.27. The calculation reveals, that for each \( \epsilon \), a fluid depth exists where the uncertainty of the surface position \( \sigma_{{z_{0} }} \) is at a minimum (marked as a black cross).
The minimum decreases with higher attenuation coefficients \( \epsilon \) and lower fluid depths \( d_{f} \), which allows a suitable concentration (i.e. \( \epsilon \)) to be chosen for any particular applicationdependent fluid depth to achieve optimal uncertainty. A MonteCarlo (MC) simulation of the surface position determination (fitting an artificial signal with equivalent noise 10,000 times) was used to verify the results of the uncertainty calculations. To determine the theoretical limit of the achievable uncertainty, the variance \( \sigma_{{I_{F} }}^{2} \) of the fluorescence intensity is decreased to its theoretical minimum. For this, an ideal, shotnoise limited signal with a Poisson distributed variance is considered, resulting in a surface position uncertainty of \( \sigma_{{z_{0} }} = 0.07\,\upmu{\text{m}} \) for the 23 position measurements of the experiment (see Fig. 5.26). As a result, the shotnoise limited uncertainty is two orders of magnitude lower than the currently achieved measurement uncertainty. However, if the variance of the fit residuum is considered, the theoretically achievable uncertainty of the surface position for a fluid thickness of \( 2.6\,{\text{mm}} \) amounts to \( \sigma_{{z_{0} }} = 8.56\,\upmu{\text{m}} \), which corresponds to a step height uncertainty of \( \sigma_{h} = 3.75\,\upmu{\text{m}} \). These results demonstrate the untapped potential of the measurement technique and suggest that the step height uncertainty of the modelbased measurement approach is only limited by the natural variation of the surface or effects not considered in the model (model uncertainties). Hence, a submicrometer precision seems feasible with the proposed confocal fluorescence microscopy technique, if the model uncertainties are decreased and the signaltonoise ratio is maximized in future investigations.
Acknowledgements The editors and authors of this book like to thank the Deutsche Forschungsgemeinschaft DFG (German Research Foundation) for the financial support of the SFB 747 “Mikrokaltumformen—Prozesse, Charakerisierung, Optimierung” (Collaborative Research Center “Micro Cold Forming—Processes, Characterization, Optimization”). We also like to thank our members and project partners of the industrial working group as well as our international research partners for their successful cooperation.
5.5 Characterization of Semifinished Micro Products and Micro Components
Bernd Köhler, Brigitte Clausen* and HansWerner Zoch
Abstract The mechanical properties of semifinished micro products and micro components cannot easily be extrapolated from macro material. The effects of microstructure and surface conditions have a strong influence on the reliability of measuring results and on the material properties. To evaluate new approaches in micro wire and foil production new testing techniques had to be applied. Furthermore, the suitability of bonding techniques in production processes had to be tested dynamically. Finally, the mechanical properties of micro components produced with the newly developed techniques had to be validated.
Keywords Microstructure · Tensile strength · Fatigue
5.5.1 Introduction
Acquiring the mechanical properties of micro samples demands additional awareness concerning the effects of microstructure, surface, and alignment influences. In both tensile and fatigue tests of thin sheets, the edges of samples have to be very smooth and without micro notches to prevent them from early failure [Köh10b]. It is even more important to observe a correct alignment of the samples than for common samples, since eccentricity in the driving direction can lead to wrinkling and accelerate failure [Hon03]. Frames to place them into the testing device [Hon03] can additionally support especially sensitive samples. The ratio between the sample dimensions and the spatial extent of micro structural features causes size effects on, for example, the yield strength.
The strongest influence is caused by the inhibition of dislocation movement due to surface effects and a lack of dislocation sources. For sample sizes between 1 and 10 µm, this effect causes a strong increase of the yield strength. Furthermore, strain gradient plasticity effects influence the yield strength, due to additionally generated geometrically necessary dislocations (GND). Assuming a length scale of possible interactions with mobile dislocations between 10 nm and 100 µm, the plastic deformation in micro samples can be described satisfactorily [Vol09]. The surface grain model [Kal96] takes into account the difference between the influence of the surface grains and the influence of core grains on the yield strength. The plastic deformation of surface grains occurs more easily than for core grains due to a smaller contribution to strain hardening. The effect is measureable in sample sizes less than 20 times the average grain size [Kal96]. Among the different approaches to calculation, Köhler et al. [Köh13] summarized the yield strength dependencies from structural features. They show that for micro metal forming applications, the temperature and strain rate dependence on the yield strength can be neglected. For samples with less than 20 grains in the crosssection, an increasing yield strength effect in comparison to single crystals occurs due to the different orientation of grains towards the applied stress and the lack of strain continuity across grain boundaries. The increase can be calculated by a crystal orientation factor m_{or} [Tay56]. For samples with more than 20 grains in the crosssection, the Hall–Petch relation works well to calculate the grain size influence. An additional term taking the grain boundary resistance (GBR) into account effects a further increase of yield strength. Kim et al. [Kim07], using an extended Hall–Petch relation, provided a combination of both.
Since the local microstructure gains a higher influence on the mechanical properties, the natural scatter in the local microstructure results in larger scatter in the mechanical properties (see Sect. 1.4.2.2 Scatter).
5.5.2 Equipment for Testing Micro Samples
5.5.2.1 Mechanical Testing
The results depicted in the following were produced on an electrodynamic testing machine Instron® ElectroPuls™ E1000 type powered by a linear motor, described thoroughly by Köhler et al. [Köh10b]. Its control allows a wide range of specimens and specimen stiffnesses to be tested in dynamic mode with loads up to ±1000 N as well as in static mode with loads up to ±710 N. The cyclic tensile tests were carried out under sinusoidal load at a stress ratio R = 0.1 and a frequency f = 20 Hz. Spark erosion and an additional barrel finishing process could achieve the required high surface quality of dynamically tested sheets with a thickness of about 10 µm. The parameters of this finishing process are the results of internal investigations. The testing of micro wires required the development of new clamping systems, since a conventional clamping of wires would provoke an early failure caused by the notch effect. Quasistatic testing of soft samples allows the application of a reinforcement at the clamping ends to avoid breakage at the clamps. For samples with higher strength and dynamic tests, the demand was solved by a concept known from testing fibers. The incrementally formed wire is guided two turns around a roll at both ends and only mechanically clamped at the end (Fig. 5.28). In this way, the force is dissipated via the friction of the wire on the rollers and the notch effect at the end of the clamping can be neglected.
Ultramicro hardness measurements on the rotary swaged wires made of 304 stainless steel showed that they undergo a strong work hardening due to the forming process and have a surface hardness up to 550 HV 0.03 [Köh17]. To ensure a satisfactory endurance of the rollers, a nitriding steel (31CrMoV9) was chosen as material. After manufacturing, the rolls were quenched and tempered (870 °C 2 h/oil/550 °C 2 h) and nitrided (510 °C 24 h) up to a nitriding depth of about 0.3 mm. A surface hardness of 900 HV 0.1 or 65 HRC, respectively, was achieved.
5.5.2.2 Metallographic Investigations
The change of microstructure due to deformation and heat treatment has to be documented and interpreted to understand the mechanisms occurring in micro cold forming. Besides classic metallographic methods, an electron backscatter scanning diffractometer (EBSD) was used to obtain the grain sizes in aluminum sheets, recognize the anisotropic microstructure, and scan the phase distribution. For this purpose, a Philips XL30 scanning electron microscope (SEM) with an EBSD detector EDAX DigiView IV was available. The sample preparation for optimized EBSD imaging was dependent on the material to be analyzed. To avoid misinterpretation due to preparation failures, an investigation of the influence of different preparation techniques on the results of the EBSD micrographs was started. The results were published by Köhler et al. [Köh18].
5.5.3 Tensile Tests on Micro Samples
The first results from tensile tests on thin DC01 sheets with a chemical composition Fe0.027C0.047Al0.005Si0.190Mn0.007P0.009S (mass%) and a thickness of about 50 μm demanded an appropriate model to explain the unexpected dependence on the grain size [Köh10a]. The first received results are displayed in Fig. 5.29. The high yield strength in the cold worked state can easily be explained by the high dislocation density due to cold working in the forming process. The dislocation density was reduced by heat treatment for 15 min at 850 °C in a salt bath from 10^{10} to about 2 × 10^{8} m^{−2}. An elongation of the holding time at the heat treatment temperature caused no further reduction of the dislocation density.
To explain the deviance towards the Hall–Petch relation due to the small number of grains in the crosssection of the tensile test samples, a model inspired by Janssen [Jan07] was applied successfully [Köh11a]. A simple Taylor approach assigning different yield strengths to the core and surface provides a satisfactory description of the yield strength of the sheets:
with α representing the volume fraction of the surface grains. The yield strengths \( \sigma_{{{\text{p}}0.2}}^{\text{surface}} \) and \( \sigma_{{{\text{p}}0.2}}^{\text{core}} \) in the Hall–Petch relation are extended by the hardnessdependent contribution \( c \cdot \left( {HV  150} \right) \) (see Eqs. 5.32 and 5.33), wherein c is a materialdependent parameter. A hardness of 150 HV is assumed for the soft annealed state of the mild steel. The differing grain sizes d and grain boundaries resistances of surface and core grains, k^{surface} and k^{core}, as well as the friction stress σ_{0} are explicitly taken into account:
Using the measured values and values from the literature [Hut63], the relations could be completed for DC01sheets to:
With this newly developed relation, the dependence of the 0.2% yield strength of micro sheets on the grain size could be predicted with a satisfactory accuracy [Köh11a].
The characterization of the first thin films generated by physical vapor deposition required the simultaneous consideration of the test result and the microstructure [Sto10]. Typical deposition effects, like hillocks, explained the unusually high differences in the results of nominally equal batches. The enhancement of the process quality resulted in more homogeneous tensile test results, suitable to adjust the process parameters of the sputter process [Kov17].
A customized clamping technique for micro wires was successfully applied on the incrementally forged samples. As expected, the specimen failure occurred predominantly within the gauge length. In the tensile test results, the scatter was tolerable, though the results were sometimes unexpected.
Rotary swaging of micro wires produced from 304 stainless steel causes the transformationinduced formation of martensite [Kuh15]. The fraction of martensite increases with decreasing feed rate in the forming process. Though the hardness increase is in accordance with the martensite fraction, the tensile test results show an almost diametrical tendency [Köh17]. The ultimate strength and the yield strength increase slightly with the increasing feed rate (Fig. 5.30). The increase can be explained by the strength hardening of the samples due to the forming process. It should be mentioned that the scatter of the strength values of these samples is comparatively small.
5.5.4 Endurance Tests on Micro Samples
In contrast to the tensile test results, the results of the fatigue test on rotary swaged micro wires show a considerable influence of the feed rate [Köh17]. With 1 and 3 mm/s feed rates, the results show an increase of the endurance limit σ_{e} with an enormous scatter, which is represented by the scattering parameter T = σ_{e,10%}/ σ_{e,90%}. With 5 mm/s, the scatter and the endurance limit decrease significantly in accordance with the decrease in hardness (Fig. 5.31). The scatter of the results is significantly higher than in the initial material. A comparison of these results with results gained on macro swaged samples showed that, though the scatter of the results is much lower, the mean value of the macro swaged samples lies well below the results for the micro swaged samples [Gan96]. This result can be explained by the statistical influence of the appearance of failure on the fatigue values. The less commonly a critical failure appears in the material, the more the scatter and the mean value for the endurance limit σ_{e} increase, and the more samples have to be tested to receive a loadable result.
An additional interesting point is discovered in the origin of the failure of the fracture (Table 5.3). Stereo microscopy and additional scanning electron microscopic (SEM) investigations of the fracture surface showed that the fracture mainly does not start from a nonmetallic inclusion or from the surface, as normally expected, but from under the surface.
A comparison of the results of the hardness measurements, metallographic investigations and the failure origins position in the fractured samples showed that the fracture starts predominantly in areas with an increased martensite fraction and the highest hardness values. In Fig. 5.32 the micrographs of the wires are superimposed with the position of the highest probability of failure (colored rings = mean value of failure probability; surrounding dotted rings = mean value +/− standard deviation) and the hardness measured across the crosssection. Residual stress measurements revealed additionally an increase of residual stresses in the martensitic phase below the surface. The position of the maximum residual stresses correlates with the fracture initiation position of the dynamically tested samples (Fig. 5.33).
5.5.5 Microstructure Analysis with EBSD on Rotary Swaged Samples
Köhler et al. [Köh18] published the results of the comparison of preparation techniques for EBSD measurements. The most interesting result was that the indexability of grains depends not only on the quality of preparation but also on the degree of deformation of the grains due to the forming process. Any distortion to the crystal lattice within the diffracting volume produces a lower quality due to more diffuse diffraction patterns. This enables the parameter to provide a qualitative description of the strain distribution in the microstructure, if the underlying structures can still be dissolved in principle [War94]. However, severe deformation can also cause ultrafine grains, which can also be the reason for insufficient resolution of the diffraction patterns. An example of the effect of severe deformation on the image quality is shown in Figs. 5.34 and 5.35. Figure 5.34 shows the image quality and the EBSD image of the microstructure of a 304 stainless steel wire in the initial state before forming. The image quality is very good and there are only a few dots along the grain boundary that cannot be indexed correctly. Figure 5.35 shows the same steel wire after rotary swaging with a degree of deformation φ = −1.46. The grains have been deformed differently according to their alignment to the main shear stresses. Between the deformed grains, large black areas appear, which are not indexable and which indicate a severe deformation or the formation of nanograined structures.
Acknowledgements The editors and authors of this book like to thank the Deutsche Forschungsgemeinschaft DFG (German Research Foundation) for the financial support of the SFB 747 “Mikrokaltumformen—Prozesse, Charakerisierung, Optimierung” (Collaborative Research Center “Micro Cold Forming—Processes, Characterization, Optimization”). We also like to thank our members and project partners of the industrial working group as well as our international research partners for their successful cooperation.
References
[Ago10] Agour, M., Huke, P., von Kopylow, C., Falldorf, C.: Shape measurement by means of phase retrieval using a spatial light modulator. In: AIP Conference Proceedings, vol. 1236, pp. 265–270 (2010)
[Ago15] Agour, M., ElFarahaty, K., Seisa, E., Omar, E., Sokkar, T.: Singleshot digital holography for fast measuring optical properties of fibers. Appl. Opt. 54, E188–E195 (2015)
[Ago17] Agour, M., Klattenhoff, R., Falldorf, C., Bergmann, R.B.: Spatial multiplexing digital holography for speckle noise reduction in singleshot holographic twowavelength contouring. Opt. Eng. 56, 124101 (2017)
[Ago17b] Agour, M., Klattenhoff, R., Falldorf, C., Bergmann, R.B.: Speckle noise reduction in singleshot holographic twowavelength contouring. In: Proceedings of SPIE, vol. 10233, p. 102330R (2017)
[Ago18] Agour, M., Falldorf, C., Bergmann, R.B.: Fast inspection of microparts by utilizing spatial multiplexing and autofocus in holographic contouring. Opt. Express 26, 28576–28588 (2018)
[Ber12] Bergmann, R.B., Huke, P.: Advanced methods for optical nondestructive testing. In: Osten, W., Reingand, N. (eds.) Optical Imaging and Metrology: Advanced Technologies, pp. 393–412. WileyVCH Verlag GmbH & Co., Weinheim, Germany (2012)
[Bou93] Bourdet, P., Lartigue, C., Leveaux, F.: Effects of data point distribution and mathematical model on finding the bestfit sphere to data. Precis. Eng. 15, 150–157 (1993)
[Bra00] Bradski, G.: The OpenCV libarary. Dr. Dobb’s Journal of Software Tools, vol. 25, pp. 120–126 (2000)
[Cha17] Chaurasia, A., Culurciello, E.: LinkNet: exploiting encoder representations for efficient semantic segmentation (2017). arXiv:1707.03718
[Cho16] Chollet, F.: Xception: deep learning with depthwise separable convolutions (2016). arXiv:1610.02357
[Cle15] Clevert, D.A., Unterthiner, T., Hochreiter, S.: Fast and accurate deep network learning by exponential linear units (elus) (2015). arXiv:1511.07289
[Fal12] Falldorf, C., Huferathvon Luepke, S., von Kopylow, C., Bergmann, R.B.: Reduction of speckle noise in multiwavelength contouring. Appl. Opt. 51, 8211–8215 (2012)
[Fal12a] Falldorf, C., Agour, M., von Kopylow, C., Bergmann, R.B.: Phase retrieval for optical inspection of technical components. J. Opt. 14, 065701 (2012)
[Fal15] Falldorf, C., Agour, M., Bergmann, R.B.: Digital holography and quantitative phase contrast imaging using computational shear interferometry. Opt. Eng. 54, 24110 (2015)
[Fla01] Flack, D.: Measurement Good Practice Guide No. 41: CMM Measurement Strategies. National Physical Laboratory, London, UK (2001)
[Flo14] Flosky, H., Vollertsen, F.: Wear behaviour in a combined micro blanking and deep drawing process. CIRP Ann. Manuf. Technol. 63(1), 281–284 (2014)
[Gan96] Ganesh Sundara Raman, S., Padmanabhan, K.A.: Effect of prior cold work on the room temperature lowcycle fatigue behaviour of AISI 304LN stainless steel. Int. J. Fatigue 18(2), 71–79 (1996)
[Ger10] Gerhard, C., Stephen, A., Vollertsen, F.: Limits for interferometric measurements on rough surfaces in streaming inhomogeneous media. Prod. Eng. Res. Dev. 4(2), 141–146 (2010)
[Goc91] Goch, G.: Algorithm for the combined approximation of continuously differentiable profiles composed of straight lines and circle segments. CIRP Ann. 40(1), 499–502 (1991)
[Gro15] de Groot, P.: Principles of interference microscopy for the measurement of surface topography. Adv. Opt. Photon. 7, 1–65 (2015)
[Gru69] Grubbs, F.E.: Procedures for detecting outlying observations in samples. Technometrics 11, 1–21 (1969)
[Han06] Hansen, H., Carneiro, K., Haitjema, H., De Chiffre, L.: Dimensional micro and nano metrology. CIRP Ann. 55(2), 721–743 (2006)
[Hon03] Hong, S., Hoffmann, H.: Study of scaling effect on mechanical properties for milliforming of sheet metal–Tensile test of a very thin sheet. In: 1st Colloquium Process Scaling, pp. 145–151. BIAS Verlag, Bremen (2003)
[Hua17] Huang, G., Liu, Z., Weinberger, K.Q., van der Maaten, L.: Densely connected convolutional networks. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, vol. 1, p. 3 (2017)
[Hut63] Hutchison, M.M.: The temperature dependence of the yield stress of polycrystalline iron. Phil. Mag. 8(85), 121–127 (1963)
[Iof15] Ioffe, S., Szegedy, C.: Batch normalization: accelerating deep network training by reducing internal covariate shift (2015). arXiv:1502.03167
Jan07] Janssen, P.J.M.: Firstorder size effects in the mechanics of miniaturised components. Dissertation, Eindhoven University of Technology (2007)
[Kal96] Kals, R., Vollertsen, F., Geiger, M.: Scaling effects in sheet metal forming. In: Kals, H.J.J., Shirvani, B., Singh, U.P., Geiger, M. (eds.) 4th International Conference on Sheet Metal, pp. 65–75. University of Twente Enschede (1996)
[Kay93] Kay, S.M.: Fundamentals of Statistical Signal Processing: Estimation Theory, vol. 1, p. 1. Prentice Hall (1993)
[Kim07] Kim, G.Y., Koc, M., Ni, J.: Modeling of the size effects on the behavior of metals in microscale deformation processes. J. Manuf. Sci. Eng. 129, 470–476 (2007)
[Kin14] Kingma, D., Ba, J.: Adam: a method for stochastic optimization (2014). arXiv:1412.6980
[Köh10a] Köhler, B., Bomas, H., Hunkel, M., Lütjens, J., Zoch, H.W.: Yield strength behaviour of carbon steel microsheets after cold forming and after annealing. Scripta Mat. 62, 548–551 (2010)
[Köh10b] Köhler, B., Bomas, H., Zoch, H.W., Stalkopf, J.: Werkstoffprüfung an Mikroproben und –halbzeugen. MP Mater. Test. 52(11–12), 759–764 (2010)
[Köh11a] Köhler, B., Bomas, H., Zoch, H.W.: Bewertung von Zugversuchen an Mikroblechen des Stahls DC01. Tagungsband Koll. In: Kraft, O., Haug, A., Vollertsen, F., Büttgenbach, S. (eds.) Mikroproduktion und Abschlusskolloquium des SFB 499, Oct. 11–12, 2011, Karlsruhe, Deutschland, pp. 139–146. KIT Scientific Publication Karlsruhe (2011)
[Köh13] Köhler, B., Bomas, H.: Flow stress. In: Vollertsen, F. (ed.) Micro Metal Forming, pp. 69–89. Springer, Berlin (2013) ISBN: 9783642309151
[Köh17] Köhler, B., Clausen, B., Zoch, H.W.: Einfluss der Vorschubgeschwindigkeit beim Rundkneten des Stahls X5CrNi1810 (1.4301) auf deren mechanische Eigenschaften sowie Gefügeeigenschaften. In: Vollertsen, F., Hopmann, C., Schulze, V., Wulfsberg, J. (eds.) Tagungsband 8. Kolloquium Mikroproduktion, 27–28 Nov. 17, pp. 91–98. Bremen, Deutschland (2017)
[Köh18] Köhler, B., Clausen, B., Zoch, H.W.: Development and application of methods to characterize micro semifinishes products and micro components. In: Proceedings of 5th ICNFT 2018, Sep. 19–21, 2018, Bremen, Germany. MATEC Web of Conferences, vol. 190, p. 15012. https://doi.org/10.1051/matecconf/ (2018)
[Kop13] von Kopylow, C., Bergmann, R.B.: Optical metrology. In: Vollertsen, F. (ed.) Micro Metal Forming, pp. 392–404. Springer, Berlin (2013)
[Kov17] Kovac, J., Mehner, A., Köhler, B., Clausen, B., Zoch, H.W.: Mechanical properties, microstructure and phase composition of thin magnetron sputtered TWIP steel foils. HTM J. Heat Treat. Mater. 72(3), 168–174 (2017)
[Kri12] Krizhevsky, A., Sutskever, I., Hinton, G.E.: ImageNet classification with deep convolutional neural networks. In: Proceedings of 25th International Conference on Neural Information Processing Systems, vol. 1, pp. 1097–1105 (2012)
[Kuh15] Kuhfuss, B., Moumi, E., Clausen, B., Epp, J., Koehler, B.: Investigation of deformation induced martensitic transformation during incremental forming of 304 stainless steel wires. In: Proceedings of the 18th International ESAFORM Conference on Material Forming, ESAFORM 2015, Apr. 15–17, 2015, Graz, Austria. Key Engineering Materials, vol. 651–653, pp. 645–650 (2015)
[Liu09] Liu, J.P., Poon, J.P.: Twosteponly quadrature phaseshifting digital holography. Opt. Lett. 34, 250–252 (2009)
[Liu16] Liu, J., Liu, C., Tan, J., Yang, B., Wilson, T.: Superaperture metrology: overcoming a fundamental limit in imaging smooth highly curved surfaces. J. Microsc. 261, 300–306 (2016) (Wiley Online Library)
[Lüb10] Lübke, K., Sun, Z., Goch, G.: Ganzheitliche Approximation eines GeradeKreisGeradeProfils mit automatischer Trennung in Einzelprofile. In: Scholl, G. (ed.) XXIV. Messtechnisches Symposium des Arbeitskreises der Hochschullehrer für Messtechnik e.V. (AHMT), pp. 77–90. Shaker Verlag, Aachen (2010)
[Lüb12] Lübke, K., Sun, Z., Goch, G.: Threedimensional holistic approximation of measured points combined with an automatic separation algorithm. CIRP Ann. Manuf. Technol. 61(1), 499–502 (2012)
[Mar05] Marquet, P., Rappaz, B., Magistretti, P.J., Cuche, E., Emery, Y., Colomb, T., Depeursinge, C.: Digital holographic microscopy: a noninvasive contrast imaging technique allowing quantitative visualization of living cells with subwavelength axial accuracy. Opt. Lett. 30, 468–470 (2005)
[McC79] McCool, J.: Systematic and random errors in least squares estimation for circular contours. Precis. Eng. 1, 215–220 (1979)
[Mic14] Michihata, M., Fukui, A., Hayashi, T., Takaya, Y.: Sensing a vertical surface by measuring a fluorescence signal using a confocal optical system. Meas. Sci. Technol. 25, 064004 (2014)
[Mik17] Mikulewitsch, M., von Freyberg, A., Fischer, A.: Adaptive Qualitätsregelung für die laserchemische Fertigung von Mikroumformwerkzeugen. In: Vollertsen, F., Hopmann, C., Schulze, V., Wulfsberg, J. (eds.) Fachbeiträge 8. Kolloquium Mikroproduktion, Bremen, 27–28 Nov. 2017, pp. 21–26. BIAS Verlag (2017) (online)
[Mik18] Mikulewitsch, M., Auerswald, M., von Freyberg, A., Fischer, A.: Geometry measurement of submerged metallic microparts using confocal fluorescence microscopy. Nanomanufacturing Metrol. 1(3), 171–179 (2018)
[Neo14] Neogi, N., Mohanta, D.K., Dutta, P.K.: Review of visionbased steel surface inspection systems. EURASIP J. Image Video Process. 2014(1), 50 (2014)
[Noz15] Nozawa, J., Okamoto, A., Shibukawa, A., Takabayashi, M., Tomita, A.: Twochannel algorithm for singleshot highresolution measurement of optical wavefronts using two image sensors. Appl. Opt. 54, 8644–8652 (2015)
[Ron15] Ronneberger, O., Fischer, P., Brox, T.: Unet: convolutional networks for biomedical image segmentation. In: International Conference on Medical Image Computing and ComputerAssisted Intervention, vol. 9351, pp. 234–241. Springer (2015)
[Rüt08] Rüttinger, S., Buschmann, V., Krämner, B.: Comparison and accuracy of methods to determine the confocal volume for quantitative fluorescence correlation spectroscopy. J. Microsc. 232, 343–352 (2008)
[Sav07] Savio, E., De Chiffre, L., Schmitt, R.: Metrology of freeform shaped parts. CIRP Ann. Manuf. Technol. 56(2), 810–835 (2007)
[Sch94] Schnars, U., Jüptner, W.: Direct recording of holograms by a CCD target and numerical reconstruction. Appl. Opt. 33, 179–181 (1994)
[Sim17] Simic, A., Freiheit, H., Agour, M., Falldorf, C., Bergmann, R.B.: Inline quality control of micro parts using digital holography. In: Holography: Advances and Modern Trends V, Proceedings of SPIE, vol. 10233, pp. 1023311–1023316 (2017)
[Sob90] Sobel, I.: An isotropic 3 × 3 image gradient operator. In: Machine Vision for Threedimensional Scenes, pp. 376–379 (1990)
[Ste10] Stephen, A., Vollertsen, F.: Mechanisms and processing limits in laser thermochemical machining. CIRP Ann. Manuf. Technol. 59(1), 251–254 (2010)
[Sto10] Stock, H.R., Köhler, B., Bomas, H., Zoch, H.W.: Characteristics of aluminumscandium alloy thin sheets obtained by physical vapour deposition. Mater. Des. 31, 76–81 (2010)
[Tay56] Taylor, G.I.: Strain in crystalline aggregates. In: Proceedings of the Colloquium on Deformation and Flow of Solids, Madrid, vol. 1955, pp. 3–12. Springer, Berlin (1956)
[Vol09] Vollertsen, F., Biermann, D., Hansen, H.N., Jawahir, I.S., Kuzman, K.: Size effects in manufacturing of metallic components. CIRP Ann. Manuf. Technol. 58(2), 566–587 (2009)
[War94] Wardle, S.T., Lin, L.S., Cetel, A., Adams, B.L.: Orientation imaging microscopy: monitoring residual stress profiles in single crystals using an imaging quality parameter, IQ. In: Bailey, G.W., GarrattReed, A.J. (eds.) Proceedings of the 52nd Annual Meeting of the Microscopy Society of America, pp. 680–681 (1994)
[Wei16] Weimer, D., ScholzReiter, B., Shpitalni, M.: Design of deep convolutional neural network architectures for automated feature extraction in industrial inspection. CIRP Ann. Manuf. Technol. 65(1), 417–420 (2016)
[Wes06] Westkämper, E., Stotz, M., Effenberger, I.: Automatic segmentation of measurement point clouds to geometric primitives. Technisches Messen 73, 60–66 (2006)
[Wya98] Wyant, J.C.: White light interferometry. In: Proceedings of SPIE, vol. 4737, p. 98 (2002)
[Xie08] Xie, X.: A review of recent advances in surface defect detection using texture analysis techniques. Electron. Lett. Comput. Vis. Image Anal. 7, 1–22 (2008)
[Yu15] Yu, F., Koltun, V.: Multiscale context aggregation by dilated convolutions (2015). arXiv:1511.07122
[Zha13] Zhang, P., Mehrafsun, S., Goch, G., Vollertsen, F.: Automatisierung der laserchemischen Feinbearbeitung und Qualitätsprüfung mittels Interferometer. In 6. Kolloquium Mikroproduktion 10, B34 (2013)
[Zha11] Zhang, P., Mehrafsun, S., Lübke, K., Goch, G., Vollertsen, F.: Laserchemische Feinbearbeitung und Qualitätsprüfung von Mikrokaltumformwerkzeugen. In: Kraft, O., Haug, A., Vollertsen, F., Büttgenbach, S., 5. Kolloquium Mikroproduktion und Abschlusskolloquium SFB 499, Karlsruhe, vol. 7591, pp. 169—176. KIT Scientific Publishing (2011)
[Zha17] Zhang, P., von Freyberg, A., Fischer, A.: Closedloop quality control system for laser chemical machining in metal microproduction. Int. J. Adv. Manuf. Technol. 93, 3693 (2017)
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Open Access This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.
The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.
Copyright information
© 2020 The Author(s)
About this chapter
Cite this chapter
Maaß, P. et al. (2020). Quality Control and Characterization. In: Vollertsen, F., Friedrich, S., Kuhfuß, B., Maaß, P., Thomy, C., Zoch, HW. (eds) Cold Micro Metal Forming. Lecture Notes in Production Engineering. Springer, Cham. https://doi.org/10.1007/9783030112806_5
Download citation
DOI: https://doi.org/10.1007/9783030112806_5
Published:
Publisher Name: Springer, Cham
Print ISBN: 9783030112790
Online ISBN: 9783030112806
eBook Packages: EngineeringEngineering (R0)