Introduction

Tailored materials are the driving force for the introduction of new products to the market. According to [1], more than 60% of new products can be accredited to material developments. Development and qualification time of materials, therefore, often pose a limiting factor for industry. Multiple national research efforts such as the Material Genome Initiative [2] (United States of America), Materials Genome Engineering [3] (China), MaterialDigital [4] (Germany) address the digital transformation of materials aiming for accelerated materials design and optimization. Achieving these objectives requires digital representations of materials and sophisticated material models describing the materials state over its lifetime. Prospectively, this could ensure reliability of components with shortened development cycles and reduced testing sample sizes. Since fatigue of materials determines the product lifetime in many applications, accurate fatigue models predicting lifetime and its scatter are of particular importance. Despite its pivotal role for the product lifetime in many applications where components are loaded under very high cycle fatigue (VHCF) conditions, a comprehensive mechanistic understanding of initial fatigue damage evolution is lacking. This is owed to a multitude of reasons, but most importantly, the complexity of the fatigue feature space, diversity of materials, and experimental limitations. The latter in particular, results in a lack of open-access data capturing initial fatigue damage evolution, which in turn causes the absence of disclosed post-processing routines. Considering the inherent nature of fatigue crack initiation, data describing it will presumably always be sparse. In initial attempts to describe microstructural damage emergence by data-driven methods, the absence of representative data posed a substantial constraint [5, 6]. However, especially these data-driven methodologies hold the potential to attain models that generalize over a broader range of materials and processes.

To allow for modeling based on physical mechanisms, such data sets should comprise information on the surface topography [7], local strain distributions, three-dimensional microstructure [8, 9] and its evolution. It should resolve damage accumulation kinetics as well as crack initiation and propagation. The characterization method should provide the accumulation of cycles up to the VHCF regime in a reasonable time span. Furthermore, there are additional requirements that are set frequently, including non-intermittent and environmental condition experimentation. Satisfying these requirements would enable transcribing the relations into fatigue models.

However, there is no fatigue characterization technique available, which complies with these demands exhaustively. This includes techniques such as the near-field and far-field High Energy X-ray Diffraction Microscopy (HEDM) as well as diffraction contrast tomography (DCT), which arguably can address these demands to the largest extent. While they enabled in-situ insights into the 3D microstructure evolution, these aspiring methods, for now, hold limitations regarding availability, VHCF testing capability, and spatial resolution [10, 11]. Furthermore, while both HEDM and DCT score similar results, there are some inconsistencies concerning microstructure imaging [12] that are yet to be resolved. In contrast, the vast majority of reported experimental fatigue studies rely on the sequential characterization of microstructure, and fatigue damage evolution [13,14,15]. Both approaches involve elaborate post-processing routines to convert the raw data into interpretable information.

This work presents a workflow for the efficient establishment of multimodal fatigue data sets. It aims to provide methodological insights in fatigue characterization and ultimately address the present lack of data for a wide range of metals. A workflow is demonstrated, where fatigue characterization of mesoscale specimen extending upon [16] and [17] is embedded in a custom workflow consisting of complementary analytical characterization and data post-processing techniques. Throughout the workflow, the emphasis is placed on a high degree of automation and generalizability to generate multimodal data sets for various materials efficiently. As boundary conditions and specimen geometry are known and specimen volumes are manageable, the data sets provide straightforward integration into computational modeling efforts. For instance, these data sets facilitate the validation of computational micromechanical fatigue models such as crystal plasticity finite element (CPFEM) or related fast Fourier transform (FFT) based approaches. Moreover, given the improved degree of workflow automation, it is particularly suitable to produce highly quantitative data sets to build the foundation for data-driven fatigue damage evolution modeling or data mining to find sensitive features.

Experimental and Methods

Workflow Description

A schematic of the workflow is depicted in Fig. 1, where steps (1–9) are in the scope of this work.

Fig. 1
figure 1

Overview of the workflow including the experimental approach and the data handling

Full size image

The aforementioned fatigue testing (6) provides pronounced sensitivity regarding initial fatigue damage detection through a sophisticated control mechanism as well as the reduction of the highly-loaded volume through specimen miniaturization and bending resonant loading. Therefore, a comparatively high signal-to-noise ratio is obtained. Additionally, a bidirectional stroboscope illumination and image acquisition system enable localization of damage and tracking its cyclic evolution. To be specific, the resulting series of in-situ images allows detection of protrusion formation and crack initiation, both manifesting themselves as regions of elevated pixel intensity. Despite the inherent scarcity of damage locations, ascribed to their tendency to occur only at sites of defect-induced stress concentration, a variety of such damage emergence events can be captured. This is attained through specimen and testing setup design optimization, described in “Fatigue of Mesoscale Specimen”. Employing mesoscale specimens over notched macroscopic ones improves sensitivity regarding damage detection while rendering time-consuming characterization techniques applicable for the whole highly-loaded specimen surface/volume. In terms of complementary microstructure and topography characterization, 2D electron backscatter diffraction (EBSD) before fatigue (4) and topography contrast SEM stitch image acquisition before (3), and after (7) fatigue is applied, respectively. The undeformed reference state of the specimen, aside from indicating surface defects present before fatigue, allows correction of the imaging-based distinct distortions present in the different modalities. Subsequent data processing involves systematic multimodal registrations (spatial alignment) and data fusion of the individual collected data sets (9). Using an accordingly trained deep learning model (8) [18], damage locations in high-resolved SEM stitch images after fatigue are detected pixel-wise (semantic segmentation). Transcribing this information onto the in-situ image series potentially promotes the detection of fatigue damage in it and compensates for its underlying moderate resolving power. Moreover, accurate registration would enable the assignment of damage locations and their evolution kinetics to microstructural entities such as grains or their boundaries. In the following, the individual elements of this workflow are described.

Specimen Fabrication and Characterization

Exemplarily, the material EN 1.4003 (AISI 3Cr12), a ferritic stainless steel, is considered in this work. In Table 1 the average values of the alloying elements are summarized. Planar, mesoscale fatigue specimens were produced from as-received rod material with a diameter of 21 mm, which previously underwent hot rolling, grinding, cold drawing, and annealing. The specimen’s fabrication employed different processing steps, including electrical discharge machining to fabricate disks and laser cutting to define the specimen shape, shown in [16]. As opposed to the there described geometry, a thickness of \(340\,\mu\)m and minimal (center) beam width of \(480\,\mu\)m were targeted. Two planar specimens were extracted per disk such that the rod axis was oriented orthogonal to the specimen plane. The positions at which the specimens were extracted were chosen such that a fixed distance from the disk center is kept to avoid regions with higher segregation emergence in the core of the rod. Subsequently, silicon carbide paper grinding and electropolishing with an electrolyte composed of perchloric acid, 2-Butoxyethanol, ethanol, and water (Struers® A2) were performed. Latter was motivated in removing the laser cutting-induced heat-affected zones at the specimen sidewalls. The electropolishing process introduces a slight, beneficial specimen edge rounding circumventing crack initiation at otherwise sharp edges, facilitating inference of microstructure-related influence factors. To ensure appropriate surface conditions for electron diffraction techniques, the surface finish on both sides of the planar specimen was performed through rotational polishing, including 9 \(\upmu\)m, 3 \(\upmu\)m diamond suspensions as well as colloidal silica polishing (OPS).

Table 1 Nominal chemical composition of ferritic stainless steel 1.4003 in weight %
Full size table

As indicated in Fig. 1, different characterization steps were performed on the SEM, encompassing Everhart-Thorley secondary electron (SE2) (3) and EBSD data (4) acquisition. Either characterization technique is applied to the highly-loaded regime of both specimen sides. These measurements utilized a Zeiss Supra 40VP equipped with an EDAX TSL EBSD system. For EBSD data, the working distance (WD), scan step size, and aperture were chosen to be 18 mm, 0.6 \(\upmu\)m and 60 mm, respectively. Activating high current mode enabled high electron collection rates. Dynamic focusing ensured good pattern quality across the whole region of interest. As a part of the automated post-processing routine, EBSD pixels that exhibit confidence index (CI) below 0.05 are discarded, and the grains and grain boundary segments are reconstructed. Therefore, the Voronoi tessellation-based clustering algorithm following [19] implemented in the MTEX toolbox [20] is utilized. Pixel misorientations between adjacent EBSD pixels exceeding 5°were considered to separate grains. Grains consisting of less than eight indexed pixels are discarded. The EBSD data reveals an average equivalent grain diameter of about 30 \(\upmu\)m and single grains up to 100 \(\upmu\)m diameter. In longitudinal direction of the rod, the EBSD image displays slightly elongated grain shapes. Therefore, the specimen alignment relative to the rod results in elongated grains and elongated MnS inclusions along in the out-of-plane direction of the specimen. The collection of microtexture data is indispensable for the investigation of how microstructural features impact the damage evolution. Performing EBSD scans prior to as opposed to after fatigue holds the advantage that a comprehensive, undisturbed orientation mapping is possible.

However, a limiting factor in mapping EBSD onto other image-based data is posed by spatial distortions in the EBSD data. The 70°specimen tilt during the measurement resulting in a position-dependent WD, in conjunction with electron Larmor precession, causes such distortions [21]. These are amplified by specimen surface curvature that is prominent in the vicinity of specimen edges. The polishing introduces deviations up to 3°from an ideal planar specimen surface. Additionally to the spatial distortions, the orientation measurements are affected if large sample surfaces are scanned, see [22, 23]. Primarily, the spatial distortions in EBSD-inferred microstructures impede the correct assignment of damage derived from comparatively undistorted image sources to its underlying microstructural feature. Overcoming this challenge is an essential step towards performing reliable microstructure-property relationship analysis when dealing with large area EBSD scans. This motivated the acquisition of the preliminary undistorted and undeformed reference (Fig. 1, (3)) since it allows for a spatial distortion correction of EBSD data as proposed by [23,24,25]. To reduce spatial distortions in the reference SEM image (3) and SEM image after fatigue (7), high magnifications, long dwell times, and low working distances were utilized here as suggested by [26]. Tile images acquired in such a manner were stitched subsequently to account for the whole highly loaded beam section despite the high magnifications. For image stitching, the implementation [27] was used. This procedure allows resolving damage features appropriately and specimen-scale imaging without low magnifications, typically associated with pronounced optical distortions [28]. At the same time, drift distortions or scan line shifts were not apparent. The EBSD distortion correction approach is delineated in “Multimodal Image Registrations”.

While not part of the proposed workflow, a few atomic force microscopy (AFM) topographic maps were collected to evaluate the in-situ imaging during fatigue, described in the following section. Therefore, a Veeco® Dimension V AFM was utilized to conduct scans after fatigue in regions of protrusions at the specimen surface. A scan size of 55\(\times\)55 \(\upmu\)m was appropriate for all target protrusions, and the samples per scanned line were chosen to be 1024, resulting in a spatial resolution of \(r_{x/y}\) \(\approx\) 53 nm. For scanning a proportional gain of 30, interal gain of 15, and scan rate if 0.5 Hz was employed. The scans were performed in contact mode with a Bruker® RTESPA-300 silicon tip with a nominal spring constant of 40 N/m, nominal tip radius of 8 nm. A front, back, and side angle of 15 ± 2°, 25 ± 2°, and 17.5 ± 2°at the apex, respectively, affects which features are resolvable.

Fatigue of Mesoscale Specimen

Fatigue experimentation was conducted with a setup that extends upon Straub et. al [16]. The adapted fatigue setup (Fig. 2(a)) is briefly introduced in the following. It uses resonant multi-axial loading of one-sided clamped, mesoscale specimen (Fig. 2(a), blue). Apart from the reduced specimen volume, bending and torsion loading (Fig. 2(b)) resulted in a reduction of highly-loaded volume, hence improving the signal-to-noise ratio. In conjunction with a sensitive control mechanism, detectability of initial fatigue damage states is attained. In addition to that, slip band formation and crack initiation are measured sensitively through a resonant frequency change [16]. In this work, solely bending resonant fatigue was conducted. The control mechanism utilizes the phase response between sinusoidal piezo actuator excitation and the measured angle of rotation at the specimen’s unconstrained end. The time-resolved angle of rotation is measured through a laser reflected at the deflected specimen surface (at the unconstrained end of the beam), causing a displacement of the laser spot on a position-sensitive detector (PSD). By adapting the actuation frequency such that there is a 90° relative phase shift between the piezo excitation and PSD signal at all times in the course of cyclic loading, the first-order bending resonant frequency of the specimen is tracked. Resonant frequency changes are attributed to integral softening/hardening, damage evolution, or oxidation.

Fig. 2
figure 2

a) Overview of the fatigue setup illustrating the arrangement of piezo acuators (khaki), mesoscale sample (blue), laser (yellow), position sensitive device (green) and LEDs (black). b) Mesoscale specimen in a loaded state and the corresponding von Mises stress distribution for bending and torsional loading. c) and d) show the azimuthal and the polar plane of the spherical coordinate system to the sample surface orientation of the both LEDs, respectively. Moreover, the viewing direction (VD) of the camera in the polar coordinate system is indicated

Full size image

A fully reversed loading with a load ratio of R = -1 is applied. The present specimen geometry resulted in a resonant frequency of about \(f_0\) = 2 kHz at which it is cycled. Attaining \(N = 10^9\) cycles or a relative frequency decrease of \(\Delta f/f_0 = 0.1\%\) posed stopping criteria. In contrast to the vast majority of fatigue devices, this custom setup utilizes the angle of rotation as a control parameter for fatigue loading amplitude. Hence, before fatigue, the angle of rotation corresponding to a von Mises stress amplitude of \(\sigma _a\) = 270 MPa was computed by an iterative binary search scaling the results of a continuum FE modal analysis.

In order to observe local damage evolution, the setup was extended with a stroboscope illumination and camera system (Basler® beat beA4000-62km monochrome camera and Canon® MP-E 65mm f/2.8 1–5\(\times\) macro objective) capturing a series of in-situ light optical images [17]. Such an image series typically comprises several thousand images, and a subset of it is showcased in Fig. 3. In these inverted images, black and red arrow annotations indicate emerging and growing damage locations, respectively. Multiple protrusions, extrusions, and few cracks initiate across the specimen, and while some of the cracks stagnate, others manage to propagate.

Fig. 3
figure 3

Intensity-inverted subset of the light optical images acquired in the course of a fatigue experiment with annotated damage locations. The black and red annotations indicate emergence and growth of defects, respectively. The numbers at the bottom of individual images indicate the cycle number at which the images were acquired

Full size image

The image formation is based on light scattered at damage emerging from the specimen surface facing the camera. Therefore, the stroboscope utilizes two distinctly oriented light emitting diode (LED) spot lights (2\(\times\) CCS® HLV2-22BL-1220_3W LED spot light with its wavelength peak at 465 nm, see Fig. 2), to illuminate the specimen during the inflection point of the fully reversed bending motion (i.e., the non-deflected state). Note that the relatively high resonant frequencies raise the demand for accurate timing and short duration of the light pulses. The camera integrates over multiple cycles and multiple light pulses to retrieve a detectable scatter signal from the specimen surface. The spatial orientation of the LEDs, L1 axial and L2 transversal, is illustrated in Fig. 2(c), (d) in the azimuthal and polar plane of the spherical coordinate system. Their orientation is chosen such that the optical yield on the camera is maximized and the interference on the position-sensitive device is negligible. A configuration of nearly perpendicular azimuthal angles is employed to detect damage locations independent of their surface topography.

Data-Driven Localization of Fatigue Damage

A segmentation methodology was devised to detect damage locations pixel-wise in the high-resolved post-mortem SEM stitched images to assist the in-situ light optical imaging. This modality was chosen as it can capture the relevant features of damage locations. The deep learning methodology and results are reported in [18] and will be briefly summarized here. A U-Net architecture [29] was trained to detect the plasticity traces (protrusions) and cracks utilizing damage micrographs and their corresponding manually annotated damage maps. The considered data was of the same material and similar fatigue conditions as here. A set of image transformations were applied for data augmentation during training to account for the variances introduced due to specimen geometry, imaging, and operator subjectivity. The segmentation prediction quality was measured on an unseen test data set by comparing model predictions with expert annotations. Therefore, the intersection over union metric (IoU), also referred to as the Jaccard index,

$$\begin{aligned} IoU = \frac{TP}{TP+FP+FN}, \end{aligned}$$
(1)

was utilized, where TP, FP, and FN are true positive, false positive, and false negative pixel predictions, respectively. For the relevant material, a mean IoU (averaged over both damage classes and the background) of 0.85 was achieved. The trained network was applied to the full stitched SEM image, and the resulting damage map was manually inspected and slightly modified.

Multimodal Image Registrations

Correlative microscopy holds the potential to give access to previously inaccessible information [30]. For instance, investigating the local interdependencies between localized cyclic damage evolution data and microstructure information requires spatial alignment of both. While many commercial correlative microscopy tools exist, they often rely on vendor-specific specimen holders and do not correct for non-linear distortions in the data sets [31]. In contrast, we applied image registrations to match the coordinate systems and correct distortions of the data sets. Registration methods can be subdivided into feature-based and intensity-based techniques. While automated feature-based techniques require methods of feature detection, extraction, matching, and outlier exclusion, intensity-based ones rely on the correlation of intensity textures in images or sub-images, see [32]. Multimodal data registration refers to the spatial alignment of the source to target data sets and is accompanied by challenges concerning data heterogeneity. This comprises aspects such as different imaging modalities, physical pixel sizes, dimensionalities, and field of views.

With regard to this study, the image-based data sets depicted in Fig. 4 were registered, where (s) denotes source, (it) intermediate target, and (t) target data sets. The in-situ images were optimized to have minimal optical geometric distortions by considering solely the center of the frame where the highly loaded specimen region was positioned. Moreover, the relatively narrow spectrum of the LED spotlights circumvents chromatic aberrations. When it comes to SEM image data, it was claimed that distortion correction is required for accurate overlay [33]. However, in our case, appropriate imaging conditions for image tiles, their stitching (see “Data-Driven Localization of Fatigue Damage”), and the usage of a custom specimen holder throughout the whole fatigue and analytical process chain minimized distortions in SEM images and specimen alignment-induced relative distortions between the data sets. Therefore, the EBSD data was the only data set for which distortion correction was necessary.

In this multi-stage registration procedure, the undeformed and comparatively undistorted stitched SE2 SEM image of the specimen surface before fatigue acts as an intermediate registration target. This intermediate state facilitates subsequent collective transformation of all (s) and (it) data sets to the target EBSD stage by the same transformation. Hence, it could be ensured that all individual fairly undistorted source data sets are aligned well amongst each other. Note that using direct transformations to the distortion containing EBSD map possibly results in an erroneous alignment amongst the transformed source images. In literature, automatic feature detection and matching for light optical and SEM data was reported [34] employing Scale-Invariant Feature transform (SIFT) [35]. However, this proved inadequate for the data at hand since the data sets exhibit a wide range of physical pixel sizes and distinct grayscale textures. Therefore, a landmark approach using a selection of point correspondences is conducted to derive most affine transformations. Pores and static particles visible on the specimen surface in various modalities facilitate the user’s selection and matching of these features. Initially, the stitched SE2 image after fatigue and the in-situ light optical image time series are affine transformed to match the SE2 image before fatigue, see arrow (1) and (2) in Fig. 4, respectively. Additionally, a deep learning model-inferred segmentation map indicating damage locations is derived from the SE2 image after fatigue testing (see “Data-Driven Localization of Fatigue Damage”) and transformed accordingly. Thus, every experimental source data set is at the stage of the SE2 image before fatigue.

Fig. 4
figure 4

Overview of the registration process. The regular arrows and the one denoted with the asterisk describe affine transformations and distortion correction by an elastic transformation, respectively. a) Stitched SEM image after fatigue (s) for accurate damage localization. The DL-derived damage segmentation map is transformed accordingly. b) Stitched SEM image before fatigue (it) acting as spatial reference and indicating prior defects. c) In-situ light optical image series (s) containing information on cyclic damage evolution. d) Idealized geometry (s) and boundary conditions. e) Inverse polefigure map from EBSD (t) for microtexture information. f) elastic transformation to correct for spatial distortions in EBSD

Full size image

Carrying along the idealized geometry (without fabrication-induced variances except actual specimen dimensions) and loading conditions, together with the embedded microstructure enables straightforward incorporation in micromechanical simulations. The absence of defect-related features in the geometry necessitates a different approach for its registration. An intensity-based algorithm relying on shape-based features is used to transform the idealized specimen geometry represented by a binary mask onto a binary mask derived from the light optical images as indicated by the arrow (3) in Fig. 4, followed by the transformation (2). Therefore, the Mattes mutual information metric [36], which measures the relation between two-pixel distributions, was employed. In the case of image registration, the measure contains information on how well pixel samples from both images are mapping onto each other. During registration, the alignment is optimized by reducing the entropy between the two-pixel distributions. In most of the implementations, so-called image pyramids, relying on subsequent filter and downsampling operations, are employed, e.g., [37]. These ensure that larger-scale features such as the specimen shape are taken into account as well.

Subsequently, using the landmark approach, a transformation is inferred to commonly transform every data set from the intermediate target stage (SE2 before fatigue) to the target EBSD data stage (4). To derive this affine transformation, common image features related to volume defects in the SE2 before fatigue and the SEM signal channel of the EBSD data are utilized. Since the EBSD data contains the aforementioned distortions, the superposition after this affine transformation is non-ideal. Finally, the correction of the EBSD data (5) takes place by computing its elastic transformation field relative to the affine transformed SE2 image before fatigue, illustrated in Fig. 4(f). Therefore, a b-spline optimization incorporating landmark selection was performed, following [38]. This elastic transformation utilizes the minimization of an energy functional consisting of several weighted energy terms. Namely, the terms are represented by dissimilarity energy between the images, optional landmark constraints, regularization, and bidirectional consistency. For the purpose of registration, the dissimilarity term attempts to minimize the pixel intensity difference between the warped source image and the target image. The regularization term ensures gradual displacements without discontinuities in regions where no landmark information is available. Landmark information constraints the deformation depending on the weight applied for it. The bidirectional consistency energy refers to the inverse transformation and ensures the invertibility of the deformations. Images and displacements are constituted by cubic b-splines, as proposed by [39]. This information is translated to pixel-wise displacements to determine each sampled data point’s corrected positions on the hexagonal EBSD grid. The EBSD attributes (Euler angles, confidence index, image quality, etc.) are assigned to the closest grid point without altering the grid point positions. Subsequently, data cleaning and grain reconstruction as described in “Specimen Fabrication and Characterization” are applied.

Results and Discussion

This section aims to evaluate and discuss previously described imaging and registration methodologies as well as to showcase the information contained in the data sets. For the latter part, the data set and the associated analyses were created from a single fatigue experiment and subsequent post-processing.

Characterization of In-Situ Imaging

To characterize the in-situ imaging, the dependence of its resulting local grayscale image texture on the slip trace topography (i.e., extrusions, intrusions, protrusions) is investigated. Therefore, atomic force microscopy (AFM) maps are acquired. In Fig. 5(a), (b) a comparison of both modalities is illustrated for two instances of protrusions.

Fig. 5
figure 5

a+b) Two exemplary comparisons of protrusions in cleaned AFM topographic maps with corresponding light-optical images after fatigue (inlay). c) Correlation plot between accumulated intensity from the light-optical images and the projection of protrusion topography onto the emitting surface of the axial LED

Full size image

For a more quantitative understanding of the relationship between topography and resulting light optical image texture, multiple plasticity trace features were computed from both the topographic map and their corresponding elevated intensity region. Subsequently, these features were evaluated with regard to their correlation. Extracted features from the AFM information included various roughness metrics and geometrical quantities. The extruded maximum and average elevation, extrusion ground area, volume, and area originating from projecting the protrusion topography onto the axial LED’s emitting plane \(A_{AP}\) (scattering cross-section) were considered. These features were computed relative to the host grain background elevation. On the other hand, the set of features derived from the light optical representation comprised elevated intensity area and maximum, pixel averaged, and pixel accumulated intensity. Assessing both feature sets for eight distinct extrusions/protrusions indicated that \(A_{AP}\) correlates with the accumulated intensity most (see Fig. 4(c)). For this pair of variables, a Pearson correlation coefficient was R = 0.88.

It is plausible that the correlation of \(A_{AP}\) to the accumulated intensity is more robust than that of the extruded volume, for instance, since the accumulated intensity depends on the protrusion portion illuminated (scattering cross-section) by the LEDs. Nonetheless, it can be observed that the data points deviate from a linear behavior for larger projected areas. This can potentially be attributed to a more notable dependency of the accumulated intensity on the not considered extrusion surface orientation as protrusion size increases. The distinct shape in both modalities can be primarily ascribed to the light optical imaging’s bidirectional lighting conditions. The substantial impact of the azimuthal illumination angle on the imaging of a static object was demonstrated in [40]. It was shown that occlusion prevents the realistic imaging of three-dimensional objects when few illumination directions are applied. When temporal imaging of dynamically evolving surface topography such as protrusions under motion is concerned, this is further complicated. In large grain size and single-phase materials, these plasticity traces are typically extensive protrusions. Especially, in this case, pronounced topography changes in the course of cyclic loading occur. Then occlusions can sensitively impact the imaging and potentially lead to an overestimation of topography change. On the other hand, the distinction between microstructurally short cracks and plasticity traces by inspecting local intensity distributions in these light optical images is not straightforward. In conjunction with the fact that cracks in many materials originate at surface plasticity traces, this impairs the crack initiation life’s determination from these optical image series. Despite these non-ideal imaging characteristics of the optical setting, microstructural damage emergence can be detected reliably. Especially when data fusion with highly resolved, feature-rich modalities such as SEM and deduced damage segmentation maps is performed, many of the aforementioned challenges can be alleviated. Such damage maps reliably localize cracks, and surface plasticity traces pixel-wise and hence improve damage type distinction as well as counteract shading for improved damage connectivity inference.

Validation of Registration Methodology

To qualitatively assess the registration approach, a comparison between two inverse pole figure color-coded EBSD maps acquired before (Fig. 6(a)) and after fatigue (6(b)) is performed on the same specimen section. While the former contains the overlayed damage map (black) deduced from SEM imaging after fatigue and represented the proposed registration approach, the latter is superimposed with the intrinsic confidence index (CI) channel of the EBSD data. For straightforward comparison and illustration, both damage types, originally differentiated by the DL model, were merged in Fig. 6(a).

Fig. 6
figure 6

a) Inverse pole figure map before fatigue with the reference direction being the normal direction (ND), i.e. pointing out of the specimen plane. Damage locations from DL semantic segmentation in black from the SEM image after fatigue are registered and superimposed onto the the microstructure. The dashed box highlights a crack location investigated subsequently. b) Inverse pole figure map after fatigue in the same orientation setting. In this case the intrinsic confidence index channel of the EBSD data is superimposed as a gray value distribution indicating damage locations and polishing artifacts. The scale bar, the color code reference triangle and specimen coordinate system apply to both images

Full size image

From Fig. 6 it is evident that multiple fatigue-induced damage locations emerge across the specimen microstructure. Typical instances of microstructurally short cracks and protrusions developing the highly chromium-alloyed, ferritic EN 1.4003 steel are depicted in Fig. 7. In contrast to the literature, where most damage in the HCF/VHCF regime is reported to be associated with (internal) pore or inclusion defects [41], for the material and loading conditions at hand (low amplitude bending), damage emerged at and is localized to grain boundaries at the surface predominantly. An exception to this are few protrusions developing at surface-residing processing-induced pores and crack sections that propagate in a transcrystalline manner. Moreover, a comparison of both images in Fig. 6 shows that the positions of damage locations with respect to the microstructure are in accordance. This applies to the whole highly loaded specimen region illustrated here, including the vicinity of specimen edges. Differences in shape and extent of individual damage locations arise between both images.

From the agreement between both images to damage locations within the microstructure, it can be inferred that the applied affine and elastic transformations appropriately correct global alignment and trapezoidal distortion superimposed with further distortion effects. The proposed registration methodology, in contrast to grid calibration, in theory, provides the possibility to correct the specimen geometry-dependent part of the distortions as well. Indeed, even at specimen boundaries where such geometry-induced distortions arise due to edge rounding, the alignment of damage with microstructural features in both images is widely conformal. Discrepancies in the shape of damage locations are attributed to the fact that the DL segmentation model was trained to detect specific local and contextual image textures in topography-sensitive SE2 images rather than small-scale plasticity in the vicinity of slip traces that do not culminate in a surface change. Even though both damage types, predicted by the DL approach, were merged for visualization, the possibility to reliably discern damage types distinguishes the proposed multimodal workflow from the direct damage inference using EBSD data after fatigue. Furthermore, comprehensive microtexture information is accessible due to the absence of plastic deformation in the collected EBSD data before fatigue. In the future, automated alternatives for the multimodal registration process are required to avoid the manual selection of point correspondences between both images. This would render the post-processing chain fully automated. Recently, methodologies that employ convolutional neural networks for multimodal image registration were introduced [42, 43] that can potentially alleviate this issue.

Fig. 7
figure 7

SE2 SEM images of damage locations. Subfigures a) and b) – c) show a typical short crack and two protrusion area, respectively

Full size image

Fatigue Damage Evolution Analysis

Damage localization

As stated previously, Fig. 6 indicates that the damage is localizing to grain boundaries. Registration enables quantitative analysis of proximity between damage locations and characteristic microstructure or topography features. In this case study, specifically, the euclidean distance of damage to grain boundaries is investigated. The relative euclidean distance of every pixel containing damage to its nearest grain boundary segment neighbor is illustrated in Fig. 8 as a probability histogram (blue). The reference grain size to compute the relative euclidean distance was the halved minimal Feret diameter of the corresponding host grain. In order to compare the damage-GB distance distribution to a random case, two approaches were followed. The first artificial random distribution indicated by the plotted line was created by sampling from a unit circle-shaped grain with replacement (solid line). Second, damage pixels were randomly distributed among all EBSD pixels, followed by normalization of the resulting histogram (red). Latter reference considers the identical microstructure and the same computation of relative distances as for the real damage distribution relying on the halved minimal Feret diameter.

Fig. 8
figure 8

Probability histogram distribution of euclidean distance between damaged/statistical reference pixels and its nearest grain boundary segments. While the actual reference was computed by assigning damage to EBSD pixels randomly, the circular reference considered a single unit circle grain

Full size image

In the damage probability histogram (blue), the measured distribution is more focused towards small relative euclidean distances when compared with both random references. In the direct vicinity of the GBs, multiples of random (MOR) in the range 1.5–2.0 are demonstrated.

This confirms that damage localizes to grain boundaries well. Qualitatively, we arrive at a similar conclusion as [44, 45], who investigated the distance of stress hotspots derived from micromechanical simulations to microstructural features. In the future, systematic comparison of experiments and simulations can validate the mechanistic descriptions captured in fatigue models. Aside from GB segments, these euclidean distance probability distribution plots can be applied to GB triple points, inclusions, or pores to evaluate the distinct failure modes and their relevance in the specific material.

Load conversion to slip systems

This study aims to get an insight into how the imposed stress state translates into slip activation. While the proposed workflow provides all boundary conditions to conduct and incorporate micromechanical simulation results seamlessly, here, the assumption of a uniaxial stress state and absence of microstructure-induced stress modulation is made. Following this simplification, global tensile and compressive stresses act in the specimen axis direction, and its orientation can be evaluated concerning the grain orientations.

Therefore, multiple inverse pole figures (IPF) of specimen axis orientation distributions were computed, see Fig. 9, following [13]. The black dots in subfigures 9(a) and 9(b) describe the specimen axis orientation in the crystal coordinate systems of EBSD pixels for which damage was observed. Maximum Schmid factors for a range of grain orientations can be computed and illustrated as an isoline contour plot within the IPF (dashed lines), see Figs. 9(a), (b). During maximum Schmid factor computation in Fig. 9(a) only {110}\(\langle 111\rangle\) slip systems were accounted for, while isolines in Fig. 9(b) included {112} and {123} bcc slip systems as well. In this case neither a distinction between slip systems of different slip plane families nor asymmetries within specific slip planes is made. In Fig. 9(b) additionally an orientation density function (ODF) was estimated from the loading axis distribution at damage containing pixels applying a de la Vallée Poussin kernel [46] with a halfwidth of 2.9°. In contrast, Fig. 9(c) shows the ODF (halfwidth = 2.0°) for all pixels contained in the EBSD scan as a reference. Different kernel halfwidths were chosen to account for the distinct amount of data points in both scenarios. The ODFs are measured in multiples of random.

Fig. 9
figure 9

Inverse pole figures showing specimen axis [100] distributions. a) loading axis orientations at EBSD pixels which contain damage and isolines indicating the maximum Schmid factor considering bcc {110}\(\langle 111\rangle\) slip systems. b) shows the same as a) except that max. Schmid factor isolines were computed considering all three slip plane families. Additionally, an ODF is superimposed on b) for easier comparison with c), which illustrates the ODF of the specimen axis for all EBSD pixels. Some considerable resulting peaks in the ODFs are annotated with star symbols. Values on the colorbar indicate multiples of random

Full size image

From Fig. 9(a) it becomes apparent that indeed a large part of damage containing EBSD pixels exhibited orientations resulting in Schmid factors exceeding 0.48. This applies even more, when all three slip plane families are considered (Fig. 9(b)). The loading axis ODF considering only crystal orientations at damage locations in Fig. 9(b) shows three peaks, one pronounced peak (1) with MOR of approximately 6 and two minor peaks (2, 3) with MOR in the range 3–4. These peaks are situated in close proximity to the centers of the isolines of either slip system family associated with the highest Schmid factors.

In literature, surface plasticity traces and microstructurally short cracks were related to slip activation and therefore dependent on the specific crystallographic scenario [47]. In the case of loading axes situated at the center of the standard triangle, single slip was reported to be dominant [48]. When comparing the specimen axis [100] ODF at the damage locations to the overall specimen axis [100] ODF (Fig. 9(c)), one can recognize that the major peak (4) at low Schmid factors observed in Fig. 9(c) is not reproduced in Fig. 9(b). However, the second-highest peak (5) in 9(c) located in a higher Schmid factor regime can be associated with the major peak (1) in Fig. 9(b). In conclusion, damage locations are in grains with high Schmid factors, but it presumably represents only one amongst multiple decisive factors in this material. Extending upon this, in the future, anomalies for dislocation glide in bcc [49] can be investigated for various materials to get insights on slip sensitivity of different slip plane families, asymmetry of slip, and the pronounced influence of alloying elements on the slip. The contribution of protrusions and microstructurally short cracks to such specimen axis distributions can be broken down due to DL-deduced damage maps’ availability.

Cyclic damage evolution

In this analysis, the data set’s information content to cyclic damage evolution is evaluated. Therefore, the region annotated with a dashed box in Fig. 6, which contains a microstructurally short crack, is superimposed with the in-situ image series, see Fig. 10(a)–l. Moreover, the states before and after fatigue acquired with SEM in Fig. 10(n), (o), respectively, pose references. Figure 10(m) shows the DL-derived damage map downsampled to the resolution of EBSD and therefore appeared coarser than Fig. 10(o).

Fig. 10
figure 10

Subfigures 10(a)–l represent a time series of intensity evolution ascribed to topography changes from slip trace and crack formation and growth superimposed with an inverse polefigure map. The numerical value at the top of each image represents the corresponding cycle number at which the image was captured in [\(\cdot 10^6\)]. In Fig. 10(m) a multi-class segmentation map with cracks (green) and extrusions (blue) is shown, which was downsampled to the resolution of the EBSD. These are complemented with two SEM images captured prior to Fig. 10(n) and after fatigue 10(o). The SEM image prior fatigue contains grain numbers and the image after an illustration of the stages in the crack growth, where blue arrows indicate crack initiation locations and black arrows indicate crack growth. Moreover, green arrows point out a loop shaped plasticity trace

Full size image

From the post-mortem SEM in Fig. 10(o) it is observable that two separate crack branches are present. This can not be resolved in the later stages of the in-situ image series (10(e)–(l)). However, the two distinct elevated intensity regions emerging in Fig. 10(b) indicate that two distinct intergranular cracks originated in close vicinity (approximately 5-10 \(\upmu\)m gap). From the image series, it can be deduced that upon crack formation, both crack instances exhibit bidirectional crack growth. Subsequently, the cracks converge and, based on the in-situ image series, seemingly merge. However, from Fig. 10(o), it is uncertain whether the cracks merged or growth concludes with a stress-relaxed state that prevents the two cracks from fully merging. In contrast to this, the other ends of the cracks proceed to grow and are intermittent by microstructural defects. The upper crack part transitions from an intergranular crack to a transgranular crack and subsequently undergoes growth direction changes (see Fig. 10(o)).

In order to analyze the crack growth rates and barrier mechanisms, the crack branches were considered individually, as illustrated in Fig. 10(m). Therefore, the assumption is made that the elevated intensity region solely transcribes the crack growth, even though there are intensity contributions from the crack and the surrounding surface plasticity. The crack growth of the upper and lower crack branch is illustrated in Fig. 11(a), (b), respectively.

Fig. 11
figure 11

Cyclic crack length evolution demonstrating five instances of crack growth retardation. The insets 1–5 show the states at which the growth was retarded due to different phenomenons. A pair of dotted vertical lines indicate the run-up of the fatigue amplitude

Full size image

In the following, we describe the results by referring to specific grains and grain boundaries by means of the designations introduced in Fig. 10(n). Crack initiation and growth occurred shortly after reaching the target amplitude (vertical dotted line in Fig. 11(a), (b)). Both figures show distinct plateaus in the course of microstructural crack growth. At every plateau, the growth is obstructed for a few 10\(^5\) cycles. While the plateau #1 in Fig. 11(a) occurs at a surface residing grain boundary when transitioning from intercrystalline to transcrystalline crack growth, the crack resting at the other plateaus does not show any indication of superficial defect interaction. However, in the plateau #2 in Fig. 11(a) reorientation of the crack occurs, resulting in subsequent crack growth nearly orthogonal to the horizontally oriented specimen axis. In this section the growth rate reaches its upper limit of \(da/dN = 1.5 \cdot 10^{-4}\,\upmu {}\)m/cycle. A comparison of the upper crack path in the transgranular section (in G1) with potential slip planes indicated that some of the collinear crack segments with distinct directions approximately coincide with the potential {110}[111] trace orientations. The slip plane traces of the {110}, {112} and {123} slip plane families are provided in the Supplemental documents. The crack observed in the lower crack part is retarded when the intercrystalline crack at the G1-G2 boundary meets the G1-G2-G3 triple point and transitions into a transcrystalline crack in G3 (Fig. 11(b) #4) and when the crack transitions trough the G3-G4 border (Fig. 11(b) #5).

The growth of microstructurally short fatigue cracks has been studied in the literature extensively. Typically, microstructurally short crack growth is described as a slip-assisted process occurring in specific crystallographic directions (slip systems) [50,51,52]. This could be confirmed for the considered crack through a slip plane trace study. There are multiple reported crack retardation mechanisms, including crack branching [53] and impingement of the crack tip at grain or phase boundaries. Latter is sometimes associated with crack redirection or crack tip blunting through the dissemination of dislocations [54]. This blunting can be hampered by back stresses originating from dislocation pile-ups ahead of the crack tip [54]. The transition from the intergranular to the transgranular crack in the upper section (Fig. 11(a) #1) is potentially ascribed to the sharp GB trace direction change from approximately orthogonal to approximately parallel to the specimen axis. This transition is linked with a crack arrest which might occur due to crack reorientation to a specific slip plane when entering the transgranular mode. While propagating through G1, intermittent crack propagation (Fig. 11(a) #2, #3) occurs despite the absence of surface defects. However, retardation instance #2 is accompanied by a crack direction change towards the normal of the specimen axis. Such a behavior is often linked with the transition to macroscopic crack growth (stage II) [55]. Indeed, this transition occurs at shorter absolute crack length in miniaturized specimen [56]. Another explanation could be crack interaction with interfaces or redirection towards stress concentrations that reside subsurface. Further, it is noteworthy that presumably during crack growth, a plasticity trace loop develops annotated by the green arrows in Fig. 10(o). Since there was no in-situ intensity elevation observed corresponding to this surface plasticity, its cyclic emergence can not be traced back. However, it can be tentatively assumed that dislocations emitted from the crack’s plastic zone interacted with the G1-G6 grain boundary amongst others and thereby affected the crack growth.

From this study, it can be deduced that these data sets facilitate investigation on how specific microstructural features affect the stage I crack growth. Moreover, the accumulation of surface plasticity can be investigated, which was not shown here. However, the formation of surface plasticity and crack growth are fundamentally different at the mechanistic level, and hence their corresponding elevated intensities regions in the in-situ images evolve differently. While surface plasticity such as extrusions and, to some extent, protrusions are comparatively localized and can be characterized through their topographic evolution, cracks exhibit directional growth. This has the consequence that different metrics are required to measure both phenomenons. In case of surface plasticity, the pixel accumulated intensity metric (“Characterization of In-Situ Imaging”) is appropriate in most cases. From evaluating the cyclic accumulated intensity growth, it can be inferred that comparatively localized extrusions or protrusions exhibit an abrupt growth, which terminates within \(\Updelta N \approx 2\cdot 10^6\) cycles after reaching the target angular rotation amplitude under these loading conditions. In contrast, cracks do show pronounced variance in their growth behavior and directional propagation. Therefore, the accumulated intensity metric is inadequate in crack propagation analysis since growth direction information is omitted and temporary crack arrests are concealed. For the purpose of a crack analysis, morphological image operations are required after segmentation to estimate the crack length. The deep learning predicted segmentation mask derived from SEM after fatigue (e.g., Fig. 10(m)) assists not only in automated distinction of damage types and hence the decision of which damage metric to apply but also improves segmentation quality and connectivity assessment. In the future, the superimposed SEM data and damage maps can assist in identifying intensity contributions from cracks and surface plasticity in the in-situ images. Prospectively, a segmentation network can be trained directly on the in-situ images utilizing the high fidelity annotations provided from SEM. While these analyses were conducted on individual instances of cracks and protrusions, the proposed methodology to establish data set is predestined for statistical analyzes.

By virtue of combination of

  • a large field of view

  • easily accessible testing up to the VHCF regime

  • temporally high resolved detection of damage nucleation and crack growth despite the comparatively high resonant frequencies

  • dataset multimodality

  • high degree of workflow automation facilitated though deep learning-based damage detection, semi-automated multimodal registration, and microstructure analysis scripts

these datasets can provide means for data mining to find microstructural dependencies of HCF and VHCF damage evolution. Especially in these fatigue regimes, damage instances are scarce and arise only at critical microstructure configurations. To address this, datasets entailing few thousands of gains (where the damaged portion is loading-dependent) can be acquired within a week of testing and post-processing, majority of which is spent during cyclic loading. Therefore, prospectively, this data can be utilized in multilayer perceptrons, convolutional approaches, graph convolutional networks or combinations thereof to predict damage emergence or 2D microstructural crack growth paths. Recurrent neural networks can be employed to predict the kinetics of damage evolution.

Conclusions & Outlook

A workflow was developed that enables the efficient creation of multimodal fatigue crack initiation and growth data sets. The resulting data sets combine microstructural information with spatially resolved damage evolution information. The in-situ imaging captures the formation of damage with unprecedented temporal resolution despite high frequency cycling. These data sets are attained by linking a custom fatigue setup with post-processing methods, including deep learning semantic segmentation of damage locations, registration of heterogeneous image data, and data fusion.

This renders the data suitable to validate micromechanical simulation models. As damage locations are semantically segmented by deep learning, these data sets complement existing HR-DIC-based validation approaches. The deep learning semantic segmentation approach enables access to statistical validation of extrusion and crack formation events while complementary strain fields and slip bands can be obtained from HR-DIC. Apart from numerical modeling, these quantitative data sets can promote the development of data-driven modeling and data mining of microstructure-dependent extrusion formation and crack initiation. Here the availability of a considerable data quantity is a crucial prerequisite. This applies in particular due to the probabilistic nature of VHCF fatigue, since comprehensive descriptions of all influence factors are inaccessible. To this end, the high degree of automation of the workflow, high-frequency cycling, and the amount of corrected microstructure data can compensate for the inherent scarcity of damage formation in the VHCF regime to provide statistically meaningful data. This might enable to identify more correlations between degradation mechanisms and microstructural features.

The proposed workflow is assumed to apply to a multitude of materials with minimal grain sizes in the order of a few microns. The availability of damage data from diverse materials reported in [18] should allow the training of deep learning damage localization models for alternate material domains with minimal additional effort. In the future, we aim to further increase the degree of automation and generalizability through advanced registration methodologies that avoid the manual selection of point correspondences. Thereby we aspire to enable data-driven studies for a wide range of alloys irrespective of their specific characteristics.