Introduction

In material science and especially in the steel industry, rudimentary methods are still frequently used for segmentation, primarily threshold segmentation. However, when analyzing more complex microstructures in steel, traditional segmentation methods have their limitations. For example, bainitic structures, such as granular, upper or lower bainite, are difficult to segment because they differ only in the forms and arrangements of bainitic ferrite and the carbon-rich second phase, but not in their grayscale values. Thus, threshold segmentation cannot be used to separate different bainitic structures. Bainite is an essential constituent of modern high-strength steels, combining high strength and high toughness, making it interesting for many applications [1]. Despite many years of steel research, bainite is still a controversial topic. Its characterization is challenging because of the variety and number of involved phases as well as the fineness and complexity of the structures. Additionally, the lack of consensus among human experts in labeling and classifying bainitic structures further complicates this task.

As threshold segmentation is not applicable for bainitic structures anymore, other segmentation approaches must be found. Miyama et al. [2] distinguished upper and lower bainite by using morphological parameters of the cementite precipitates. However, images where both bainite types are present at once were not analyzed. Banerjee et al. [3] distinguished ferrite, bainite and martensite by using intensity values and the density of substructure particles, while Paul et al. [4] employed regional contour pattern and local entropy for segmenting ferrite, martensite and bainite in dual-phase steels. Gola et al. [5] presented a workflow for two-phase steels, in which after first segmenting the carbon-rich phase (pearlite, bainite or martensite) against the ferritic matrix by thresholding, morphological and textural parameters are used to classify pearlite, bainite and martensite. The applicability of textural parameters to distinguish different microstructures was also shown by Webel et al. [6] who distinguished pearlite, lower bainite and martensite with Haralick textural features or by Arivazhagan et al. [7] where local ternary patterns were used to differentiate low-, medium- and high-carbon steels.

Segmentation using machine learning methods is becoming more and more important, also in material science. In general, machine learning algorithms find and learn patterns in training data to build a mathematical model which is used to make predictions, e.g., classes that compose an image. In particular, deep learning segmentation [8] is popular, and although other research fields such as biology and medicine [9,10,11,12] are the driving forces, first applications in steels and metals research can be found in the literature. Azimi et al. [13] applied deep learning techniques to automatically segment and classify pearlite, bainite and martensite in two-phase steels. Deep learning methods were also used by DeCost et al. [14] for different segmentation tasks in high-carbon steels, e.g., the detection of cementite particles in a matrix or grain boundary carbides, or by Chowdhury et al. [15] who used them to recognize dendritic structures. Bulgarevich et al. [16] used machine learning techniques, i.e., trainable segmentation with a random forest classifier, to segment ferrite, pearlite and bainite in light microscopic images of three-phase steels. Komenda et al. [17] also used trainable segmentation to distinguish ferrite, pearlite, martensite as well as upper and lower bainite in sintered steel, yet the segmentation result is only shown in light microscopic image and differences between martensite, lower and upper bainite are not sufficiently discussed. An overview of applications of trainable segmentations on low-carbon steels can be found in Müller et al. [18]. Although deep learning segmentation is very promising, there are some drawbacks, primarily the large image sets required for training the neural networks and their difficult interpretability [19, 20]. For a more detailed description and discussion of deep learning segmentation, the reader is referred to [8, 21].

In most of the papers cited above, bainite subclasses or micrographs that simultaneously exhibit different bainite types are not considered, clearly showing the need for ongoing research in the segmentation and classification of bainitic microstructures.

Despite the progress in deep learning segmentation, classical image operations are still relevant. The Hough transform which is essential in processing and analyzing data from electron backscatter diffraction (EBSD) is a simple but prominent example [22]. In general image processing, many analysis methods rely on local operators. Some of them run on a single pixel basis, like the aforementioned binarization by thresholding, but many on local environments, in region sizes of 3 × 3 pixels, e.g., standard edge operators like Sobel or Prewitt, among others. Convolution operators, as used in this work, can be implemented for adjustable operator window sizes [23, 24]. The implemented operations are predominantly carried out on each image pixel. They are state of the art and can be found in open accessible image processing libraries [25]. The aim of these operators is to always achieve a suitable transformation of the image content, which helps to find, e.g., edges, objects or to determine characteristic features. For example, a locally adaptive binarization is implemented in the OpenCV library [25], in which a “sliding” operator window of a predetermined size runs over the image and performs the transformation of a gray value image into a binary image. In this way, the local calculation can, among other things, suppress undesirable phenomena such as shading effects. Other standard image operations to detect geometric figures such as lines, circles or periodic patterns include Fourier and Hough transformation [26, 27]. Methods for an analysis of directions and orientations can be found, especially for material-reinforcing fiber structures in concretes and plastics as well as for conglomerates of wood fiber structures [28,29,30]. These involve the characterization of individual objects (fibers and their conglomerates) regarding length, strength and orientation, as well as their distribution in the material. A directional characterization for a complete local window area of ​​ grayscale images cannot be found in the literature, in particular not an evaluation of the directional expression (directionality/ directional dominance), as in the approach that will be presented in this work. The characterization of local window areas with regard to statistical and direction-oriented parameters can be found in [31] where textured natural stone surfaces are evaluated and compared by using parameters from an operator window sliding across the image of the surface. Such parameters include mean value, standard deviation, entropy as well as local direction information.

This approach will be further developed to be used for microstructure images which mainly contain lath-like and granular structures, e.g., complex-phase steel microstructures with lath-like and granular bainite. Common image analysis operators will be suitably combined in a macro on the basis of library procedures [32] and applied to microstructures of complex-phase steels in order to mark lath-like bainite and distinguish it from granular bainite. In the following sections, first the image acquisition, starting with the sample material, sample preparation and imaging using light optical and scanning electron microscopy are described and the local orientation and direction analysis method will be explained. Then results of the analysis, including the influence of some parameters on the segmentation quality as well as several variations of this procedure, will be shown and discussed.

Experiments

Material

The materials used in this study are low-carbon complex-phase steels, taken from industrially produced heavy plates. Steels were thermo-mechanically rolled followed by controlled accelerated cooling. Typical microstructures of these steels consist of granular bainite and upper or degenerate upper bainite which exhibit a typical lath-like structure. Bainite types are named according to the bainite classification scheme suggested by Zajac et al. [33]. Later, the term lath-like bainite will be used to describe upper and degenerate upper bainite and differentiate it from granular bainite in the segmentation tasks.

Figure 1a shows a typical micrograph of bainitic structures in complex-phase steels. The microstructure consists mainly of granular bainite with regions of lath-like bainite, in this case upper bainite. For the segmentation task of distinguishing lath-like bainite from granular bainite, threshold segmentation cannot be used since both bainitic phases are composed of dark (bainitic ferrite) and bright (cementite) regions (Fig. 1b).

Fig. 1
figure 1

(a) Scanning electron microscope image of a complex-phase-steel microstructure consisting of granular bainite with regions of lath-like bainite. (b) Corresponding threshold segmentation which is not suitable to distinguish these bainite types because both are composed of dark (bainitic ferrite) and bright (cementite) constituents

Full size image

Sample Preparation

The samples were ground using 80–1200 grid SiC papers and then subjected to 6, 3 and finally 1 μm diamond polishing to obtain smooth surfaces for subsequent etching. For investigation under light optical microscope (LOM), metallographic etching was carried out by immersing polished sample surfaces into a mixture of ethanol and nitric acid (2 vol.%), also called “Nital” etching. For scanning electron microscope (SEM) examination, the samples were etched electrolytically using Struers electrolyte A2.

Microscopy

For light optical microscopy, a Zeiss Axio Imager.Z2m microscope was used. Images were obtained at a magnification of 500 × with an image size of 1388 × 1040 px2, equal to an area of 179.6 × 135.5 µm2. SEM images were recorded in a Zeiss Supra FEG-SEM using secondary electron contrast at a magnification of 2000 × with an image size of 2048 × 1536 px2, equal to 56.7 × 42.5 µm2. The SEM was operated at an acceleration voltage of 5 kV, an aperture of 30 µm to set the probe current and a working distance of 5 mm. All images were acquired with the same image contrast and brightness settings.

Local Orientation and Direction Analysis

The approach, from now on called local orientation and direction analysis, is aimed at the detection of lath-like structures, e.g., lath-like bainite, in suitably prepared images. It is assumed that corresponding structures in images typically manifest themselves in large areas. The procedure is divided into two steps: (1) calculation of directionalities to detect areas with lath-like structures and (2) region growing to expand these areas up to the next structural boundaries. The first step can also be run alone. To start the procedure, all images are scaled to 1024 × 768 px2. In this first step, a window (“sliding” region of interest, ROI) of suitable size is defined, e.g., 32 × 32, 48 × 48 or 64 × 64 px2. This window runs over the image step by step in x and y directions. The step size can be specified; this value is typically selected to be the window width. At each local position of the “sliding” window, the gradient direction and its magnitude are determined for each pixel based on a 3 × 3 convolution operator in x and y directions (here Prewitt operator [23], modifications are possible). For the angular range of 0°–180°, the magnitudes are summed up, resulting in a histogram of the “directional weights.” Lath-shaped structures should be distinguished in this histogram by one (or more) clear maxima. A smoothing of the histogram can be performed, but is not essential. To determine the “weight” of a direction (from now on called directionality), all the neighboring weights to the maximum are summed up in a defined range of the histogram, e.g., ± 10°. The value of the directionality for the current “sliding” window position is calculated from the obtained sum minus a value that would be achieved if the histogram was evenly distributed. The result is divided by the sum of all histogram values. If the resulting directionality value exceeds a predetermined “directionality threshold,” this area, i.e., the local window, is marked as a potential lath-like structure. In addition, a neighborhood analysis is performed. It is required that at least a given number of local windows adjacent in x or y are also marked as a potential lath-like structure with similar angular orientation. The term “similar” means that the difference between the angle of the considered window and the neighboring window must not exceed a given value. The experiments carried out assume exactly one preferred direction of the lath-like structures in the image. To take more predominant directions into account, adjustments must be made which will be presented and discussed in “Consideration of Several Predominant Lath Directions” section.

The second step corresponds to a region growing. For its implementation, it is assumed that there are local areas around the already detected lath-like bainite areas in which no structural boundaries occur. In other words, local gray value statistics show only small standard deviations. For an expansion of a lath-like bainite region, local windows of smaller sizes, e.g., 16 × 16 px2, are used in order to “carefully” approach the next structural boundary. It is evaluated consecutively in all four image directions at each not yet labeled window position whether an already labeled lath-like bainite region is directly adjacent. If so and if the standard deviation at this window position falls below a given value, this window is added as an extension of the existing lath-like region.

Figure 2 summarizes and illustrates the most important steps from this local orientation and direction analysis. As mentioned in the above sections, there are several parameters in the analysis that can, to varying degrees, influence the segmentation quality. Table 1 summarizes the most important parameters with typical values used in the analysis. Achieving optimum segmentation results demands experimenting with different parameter settings. Changing one parameter can also require a change in other parameters, a fact which complicates a systematic parametric study.

Fig. 2
figure 2

Flowchart illustrating different steps of the local orientation and direction analysis: A window of suitable size slides over the image → pixelwise calculation with 3 × 3 px2 convolution operator at the current window position → resulting directional histogram and calculation of directionality → as the directionality exceeds the threshold, the region is detected as a potential lath-like structure → overlay with all potential lath-like structures → final lath-like structures after neighborhood analysis → final lath-like structures after region growing. For better illustration, scale bars are omitted in the result images

Full size image
Table 1 Summary of different parameters that can be adjusted during local orientation and direction analysis
Full size table

Results and Discussion

Segmentation Tasks

Figure 3 shows two typical microstructure images of a complex-phase steel used for the segmentation experiments. Figure 3a shows an SEM image after electrolytic etching. The microstructure consists of granular bainite with some regions of lath-like bainite (green overlay). The granular bainite is composed of irregular ferrite with carbon-rich second phases distributed between irregular grains (Fig. 3b). In this case, the carbon-rich second phase is predominantly cementite, although some martensite–austenite (MA) particles can be found as well. The lath-like bainite in this case is upper bainite, consisting of lath-like ferrite with cementite on the lath boundaries (Fig. 3c). Figure 3d shows a LOM image after Nital etching. The microstructure consists of granular bainite, lath-like bainite (green overlay) and some polygonal ferrite (Fig. 3f). Granular bainite is made up of irregular ferrite plus cementite as the carbon-rich second phase (Fig. 3g). The lath-like bainite is upper bainite (Fig. 3e). The aim of the segmentation is to detect the regions of lath-like bainite and separate them from the “background” of granular bainite or granular bainite plus polygonal ferrite, respectively.

Fig. 3
figure 3

(a) SEM microstructure image of complex-phase-steels which is used for the segmentation trials, consisting of granular bainite plus lath-like bainite (green overlay). (b) Enlargement of granular bainite: It consists of irregular ferrite with cementite and MAs as the carbon-rich second phase. (c) Enlargement of lath-like bainite which can be classified as upper bainite with cementite on the lath boundaries. (d) LOM microstructure image of complex-phase-steels which is used for the segmentation trials, consisting of granular bainite and some polygonal ferrite plus lath-like bainite (green overlay). (e) Enlargement of lath-like bainite, i.e., upper bainite. (f) Enlargement of polygonal ferrite. (g) Enlargement of granular bainite, composed of irregular ferrite with cementite as the carbon-rich second phase

Full size image

Variation of Sliding Window Size

Figure 4 shows a comparison of the detected lath-like areas for different sliding window sizes of 32 × 32, 48 × 48 and 64 × 64 px2. All other parameters of the analysis are kept constant. To assess the influence of the window size, only the results of the first step of the directionality analysis are shown, without region growing. For better illustration, scale bars are omitted in all result images.

Fig. 4
figure 4

Detected lath-like bainite areas for different sliding window sizes: (a) sliding window size of 32 × 32 px2: many potential lath-like areas hit well, but very scattered, grainy impression and relatively many incorrect structures are detected. (b) Sliding window size of 48 × 48 px2: many, but not all potential lath-like areas hit well, less grainy impression, few incorrect structures are detected. (c) Sliding window size of 64 × 64 px2: lath-like areas are hit well, no incorrect structures detected. (d), (e) Details of (a) and (b): grain boundaries with debris of cementite that appear lath-like

Full size image

The window size of 64 × 64 px2 (Fig. 4c) achieves the best results. Lath-like structures of the microstructure are found, and no other structures are wrongly detected. In contrast, window sizes of 32 × 32 px2 (Fig. 4a) and to a lesser extent 48 × 48 px2 (Fig. 4b) also detect incorrect structures, i.e., grain boundaries or structures of the granular bainite (Fig. 4d). As the Nital etch attacks the ferritic grains [34], grain boundaries remain, and if they are too pronounced, they can appear similar to carbide laths of the upper bainite.

The smaller the window size, the more sensitive this analysis technique is to potential lath-like structures. With smaller window sizes, structures from granular bainite, e.g., second-phase particles that are thin and slender and especially grain boundaries can also appear like carbide laths. This means that the smaller window sizes of 32 × 32 px2 and to a lesser extent 48 × 48 px2 do not capture the representative area of the microstructure constituents because carbide laths of lath-like bainite and second-phase particles of granular bainite or grain boundaries may appear similar. On the other hand, the window size 64 × 64 px2 is consistent with the representative area of the microstructure constituents and correctly detects only lath-like bainitic structures.

Region Growing: Expansion of Lath-Like Areas Toward Structural Borders

Around many areas with lath-like bainite local areas in which no structural boundaries occur can be found. This means that local gray value statistics will show only small standard deviations. Local windows of smaller size, e.g., 16 × 16 px2, are used in order to “carefully” approach the next structural boundary for an expansion of a lath-like region. If an already labeled bainite region is directly adjacent and if the standard deviation at this window position falls below a given value, this window is added as an extension of the existing bainite region. This second stage can be run several times, whereby it should be noted that unwanted “finger-like structures” can occur (Fig. 5). Figure 5a–c shows the detected lath-like regions for the different sliding window sizes from Fig. 4a–c (white outlines) after the region growing (green outlines). Lath-like areas are now well detected for all three different moving window sizes, i.e., for window sizes 32 × 32 and 48 × 48 px2, lath-like regions could grow together. But still, window size 64 × 64 px2 gives the best results since no incorrect structures are detected.

Fig. 5
figure 5

Detected lath-like areas from Fig. 4a–c after the neighborhood analysis (white outlines) and region growing for different moving window sizes. Green outlines mark the additions by the region growing

Full size image

If holes with unlabeled structures appear within labeled lath-like bainite regions, they can be closed using morphological operations. If a representation without the “finger-like structures” is preferred, the result can be displayed as the convex envelope or as a moment-equivalent ellipse instead (Fig. 6a). The preferred segmentation representation can also be displayed as a binary image, which allows the calculation of the phase fraction of the lath-like bainite. To assess the segmentation quality, a comparison with human expert labeling is necessary. Typically, labeling is challenging even for human experts because there can be transitions between lath-like and granular shapes or boundaries of lath-like regions that are hard to recognize and can only be anticipated based on experience. Thus, there is usually a subjective component in labeling, and results will differ from expert to expert. For a well-founded comparison, labeling was performed by four different human experts. Results for phase fractions determined by human labeling are listed in Table 2. For illustration, the four human labels were combined, and the outlines of the minimum and maximum labeled lath-like regions are shown in Fig. 6b. Figure 6a shows different result representations, i.e., contours after region growing, the moment-equivalent ellipse and the convex envelope of the region growing contours. These three representations have different shapes and yield different results. As human labeling differs from expert to expert, the preferred results representation will also be different. Results from the local orientation and direction analysis for phase fraction of lath-like regions, for different results representations, are also listed in Table 2. In general, detected lath-like regions and calculated phase fractions are in good agreement with human expert labeling. In addition, the automatically determined fraction of the orientation and direction analysis has the substantial advantage of being more objective and more reproducible than human labeling.

Fig. 6
figure 6

(a) Comparison of human expert labeling: Blue markings represent the minimum labeled lath-like area by four human experts, red the maximum label lath-like area. (b) Turquoise overlay marks lath-like bainitic structures after the first step, and green overlay the regions added in the second step (region growing with two cycles). Yellow marking shows the convex envelope and red the moment-equivalent elliptic contour

Full size image
Table 2 Phase fractions of lath-like regions from human labeling and local orientation and direction analysis
Full size table

Another method to find an objective ground truth, instead of using the consensus of different experts, could be a correlative characterization of SEM/LOM plus EBSD. The additional information from EBSD, e.g., misorientation or grain boundary data could be used to define a lath-like bainite region [35]. Regarding further analyses of the segmented image, using the binary image as a mask to the original image allows a separate analysis of the lath-like bainite, e.g., to further distinguish upper and degenerate upper bainite. This could be accomplished by calculating textural parameters from these regions, as suggested by Müller et al. [36]. In addition, granular and lath-like regions can be analyzed independently, e.g., morphological parameters of the carbon-rich second phase such as size distributions, particle shapes and mean free distance between particles for microstructure–properties correlations can be calculated for each type of bainite instead of calculating it for the entire image.

Application of a Fixed Parameter Set to Unknown Images

To demonstrate the general applicability and the suitability for automation of this segmentation approach, the optimal parameters found for one specific image (Fig. 7a) are applied to similar, but unknown images from the same microstructure type (Fig. 7b–d). The parameter settings are summarized in Table 3, and the segmentation results are shown in Fig. 7. White marking denotes lath-like bainitic areas found after step 1 from the local orientation and direction analysis, green marking the extensions added by the region growing of step 2. Lath-like bainitic regions are mostly segmented correctly and agree with human expert labeling. This demonstrates that once an optimal set of parameters is found, it can be successfully applied to unknown, but similar images from the same microstructure type, allowing an automated segmentation.

Fig. 7
figure 7

(a) Image that was used for finding optimal parameters for detecting lath-like bainite. (b–d) Parameters are applied to three unknown images from the same sample resp. microstructure type. White marking characterizes lath-like bainitic areas found after step 1 from the local orientation and direction analysis, green marking the extensions added by region growing

Full size image
Table 3 Summary of fixed parameters used for the local orientation and direction analysis for the set of images from Fig. 7
Full size table

Combination of Different Sliding Window Sizes

As shown in the section about variation of the sliding window size, smaller windows can misidentify granular structures or grain boundaries as lath-like bainite. On the other hand, they are more sensitive about lath structures than larger window sizes so they can correctly detect small lath-like bainite regions that a larger sliding window would potentially miss. Therefore, a combination of two different sliding window sizes was considered.

First, the orientation and direction analysis are separately performed on one image with two different sliding window sizes, in this case 48 × 48 px2 and 64 × 64 px2. The analysis also includes the neighboring analysis and region growing. Figure 8a and b shows the results for 48 × 48 and 64 × 64 px2, respectively. The two result images will be binarized and combined by logical “and,” meaning that only regions detected by both sliding window sizes will be kept as lath-like bainitic areas. This is followed by a modified region growing, analogous to the region growing procedure described above. A local window of smaller size, here 16 × 16 px2, is used to consecutively evaluate at each not yet labeled window position in all of the four image directions whether an already labeled lath-like bainite region is directly adjacent. If so, the 48 × 48 px2 and 64 × 64 px2 images are examined for labeled positions. If so, this position is labeled in the combination image.

Fig. 8
figure 8

(a) Result image from sliding window size of 48 × 48 px2 (white outlines) and after region growing (green outlines). (b) Result image from sliding window size of 64 × 64 px2 (white outlines) and after region growing (green outlines). (c) Result image from the combination of sliding window sizes of 48 × 48 and 64 × 64 px2 (white outlines), also with moment-equivalent ellipse (red outlines)

Full size image

The result images of the combination of 48 × 48 and 64 × 64 px2 window size (Fig. 8c) only differ slightly from the result images of just the 64 × 64 px2 window size. Phase fractions of the lath-like bainite are 20.6% for 64 × 64 px2 and 21.9% for the combination image. Differences mainly occur with respect to the contours of the lath-like bainite regions, and in particular, the contour of the smaller second lath-like region in the lower right of Fig. 8c looks more reasonable in the combination image. Additionally, by combining the “vote” of two different sliding window sizes, the confidence of the detection is increased. As the appearance of the contours is subject to human expert preference, this variation of the analysis could be used as an alternative if results regarding contours from one sliding window size are not satisfying.

Consideration of Several Predominant Lath Directions

The findings from SEM images as presented in the previous sections are also valid for LOM images. Micrographs that have been analyzed so far were recorded at higher magnifications and only one predominant lath direction was present. However, when larger areas are imaged, e.g., the LOM image at a magnification of 500 × in this case (Fig. 9a), the image can exhibit different predominant directions of the lath-like bainite. With the standard procedure, only lath-like regions with the predominant direction are detected (Fig. 9a). White outlines mark the lath-like regions found by combining sliding window sizes of 48 × 48 px2 and 64 × 64 px2 with the approach presented in “Combination of Different Sliding Window Sizes” section. The parameters used for the separate analysis with 48 × 48 px2 and 64 × 64 px2 sliding window sizes are summarized in Table 4. Orange markings in Fig. 9a show lath-like regions which have differing orientations from the dominant direction and were not detected.

Fig. 9
figure 9

(a) Thick white outlines mark the lath-like regions found by combining sliding window sizes of 48 × 48 px2 and 64 × 64 px2 with the approach presented in “Combination of Different Sliding Window Sizes” section. Orange markings show examples of lath-like region which orientations differ from the dominant direction and were not detected. (b) Segmentation result after a second predominant lath direction is considered. Thick white outlines mark the lath-like regions from (a) and red the corresponding moment-equivalent ellipses. Thin white outlines mark regions for the second considered predominant direction and yellow the corresponding moment-equivalent ellipses. Green overlay shows lath-like regions from human expert labeling which agree quite well with the detected lath-like regions

Full size image
Table 4 Summary of fixed parameters used for the local orientation and direction analysis for the image from Fig. 9
Full size table

This means that the procedure must be adjusted so it can take into account more than one predominant direction. For the investigated image, an automated detection of the second predominant direction is not possible because of the amount of differing structures appearing in the image which produce several irrelevant peaks in the angle histogram. However, it is possible to manually determine the second predominant direction. Segmentation results after the consideration of this direction are shown in Fig. 9b. Yellow ellipses mark the newfound lath-like structures. Most regions showing pronounced lath-like structures are found, and there is a good agreement with human expert labeling. However, it should be noted that this complex-phase steel exhibits several structures that fall into a “transition region” from granular to lath-like structure where human experts usually disagree about labeling them as lath-like or not. That is why the segmentation quality could be assessed differently depending on the observer. Still, this approach is more objective and more reproducible than manual human expert labeling as it does not rely on the visual appearance of the structures, but on relevant features extracted during image processing.

Conclusion

In this work, we suggest a method of analyzing local orientations and directions in an image in order to detect and segment lath-like structures. The applicability of the approach was shown on microstructures of complex-phase steels where lath-like bainite and granular bainite were to be distinguished. However, it can be universally applied for the segmentation of lath-like structures, independent of the material or type of microstructure.

Comparing this approach with deep learning segmentation, which could also perform this segmentation task, but requires large image sets, the main advantage is its use for individual images. It is also more human-interpretable and, because it is based on common image operations, this approach allows easy access to the segmentation process and therefore a targeted adjustment of parameters to achieve the best possible segmentation result in agreement with human experts labeling. This is what makes this approach more flexible when it should be applied to different microstructures as it can be adjusted, whereas deep learning segmentation would require a new image set for training.

For the segmentation of lath-like bainite, very good results in agreement with manual segmentation by human experts are achieved. Additionally, segmentation based on the proposed method is more objective and reproducible than manual human labeling. So far, no methods are known to the authors which allow an easy, fast and automated segmentation of these types of complex-phase steel microstructures. The approach suggested in this work is a promising method and a first step in this direction. Regarding further analyses, this segmentation approach allows the determination of phase fractions of granular and lath-like bainite and, by using the segmentation result as a binary mask, a separate analysis of granular and lath-like bainite regions. Therefore, type, amount and morphological parameters (e.g., size distributions, mean free distance between particles etc.) of carbon-rich second phases can be measured individually for each bainite type and be used for correlations with mechanical properties. In addition, lath-like bainite could be further differentiated into upper and degenerate upper bainite.

Regarding the general application of the suggested approach, some recommendations can be derived:

  • Optimal parameters can vary for different images. However, the parameters listed in Tables 3 and 4 are considered as reasonable starting points when analyzing new images.

  • The sliding window size should capture all relevant features of the microstructure constituents.

  • Once an optimal set of parameters is found, it can be successfully applied to unknown, but similar images of the same microstructure type.

  • In order to reach an agreement with human labeling of the lath-like bainite regions, a region growing after the orientation and direction analysis was performed.

  • Combining different sliding window sizes can increase the confidence of lath detection and yield contours that are in better agreement with human labeling.

  • To account for more than one predominant lath direction in the analyzed image, it can be necessary to manually determine the other predominant directions.

Future work could include improvements to the region growing procedures, e.g., methods for smoothing or removing unwanted “fingerlike” structures or methods for joining small regions which are not included by the region growing procedure, but in the human labeling. Regions could also be allowed to grow until a pre-defined border is reached, e.g., grain boundaries reconstructed by euclidean distance transform and watershed. Furthermore, the suggested approach could also be attempted using other edge detectors or other operators like Fourier and Hough transform. By adding other features to the convolution operator, e.g., textural features as described in [36], the segmentation could be expanded to simultaneously perform a microstructure classification.