Machine learning for reconstruction of highly porous structures from FIB-SEM nano-tomographic data (cid:2)

. Reconstruction of highly porous structures from FIB-SEM image stacks is a diﬃcult segmentation task. Supervised machine learning approaches demand large amounts of labeled data for training, that are hard to get in this case. A way to circumvent this problem is to train on simulated images. Here, we report on segmentation results derived by training a convolutional neural network solely on simulated FIB-SEM image stacks of realizations of a variety of stochastic geometry models.


Introduction
The micro-structure of materials influences their macroscopic properties decisively. 3D images of the micro-structure yield deeper insight into the microstructure's geometric features and can be used for numeric simulations of materials properties like mechanical strength, filtration properties or thermal conductivity. Combined with stochastic geometry models [10,16], they are the basis for optimizing the micro-structure -so-called virtual material design.
Serial slicing by a focused ion beam (FIB) and subsequent imaging by scanning electron microscopy (SEM) is a versatile source for high-quality 3D images of materials structures at the scale of 3-200 nm. For highly porous structures however, reconstruction of the 3D structure from the SEM image stack is hampered by the solid structure from deeper layers being visible through the pores. These so-called shine through artifacts [12] cause the typical tails visible in the planes orthogonal to the SEM imaging plane, see Figure 1(c). These artifacts featuring the same gray values as the true foreground in the current slice, thresholding methods fail to segment the solid structure properly. Several algorithms have been devised to overcome this problem [7,11,13,14,19,20]. Nevertheless, being designed for particular structures and SEM modes, they are not generally applicable. Moreover, parameterization requires expert knowledge. Machine learning methods are a popular and already widely used alternative to classical image segmentation. A random forrest is applied to FIB-SEM segmentation in [17]. Convolutional neural networks (CNN) are used with great success also for 3D image segmentation [4,15]. However, in our particular setting of FIB-SEM data of highly porous structures, the typical need of these methods for large labeled data is nearly prohibitive. FIB-SEM is rather expensive and manual labeling is difficult to impossible as even the human eye is easily mislead by the shine through artifacts.
Here, we therefore explore the option to train a CNN solely based on synthetic images for which the correct segmentation is readily available. We use a variety of stochastic geometry models [1,8,10] to create porous structures. Digitizations into 3D images of the respective realizations yield the ground truth for the training phase. The corresponding FIB-SEM stacks are generated based on an analytic representation of the structures -lists of points in space and objects like spheres or cylinders attached to them. These geometries are virtually intersected and SEM images of each planar intersection are simulated as described in [12]. The thus derived FIB-SEM stacks are then used to train the U-net 3D architecture [4,18].
Specially adapted data augmentation and weights as well as the use of versatile structures result in very good segmentations for the synthetic data. Tests on real data are promising, too, results will be reported in [5].

Network architecture and and training the model
We keep the U-net 3D architecture as specified in [4]. We also follow the original U-net setup [18] in using weighted cross-entropy for measuring similarity of image patches. [18] uses weight maps to assign higher weights to pixels in image regions where objects touch, in order to separate them. We adapt this idea by assigning a higher weight to surface pixels and their neighbors in order to force the network to learn the structure's surface particularly well.
The use of 3D patches causes them to be small (64 3 pixels) compared to the total image sizes (about 500 3 − 600 3 pixels). Simple tiling results in strong boundary artifacts. To avoid these effects, we therefore apply a sliding window approach with up to 20 pixel wide overlapping regions, depending on local structure size.
In [18], data are augmented excessively by deforming the training images elastically, to force the network to learn invariance to such formations. We combine this approach with brightening and rotating the 3D patches using any of the cube's isometries similar to [4]. To ensure that the training data represent various sizes of the local structures, the patches are chosen with a random crop and resize approach, where the scale factor is variable too.
We trained the network on a Boolean model of fibers (cylinders) [21,22] as, compared to models consisting solely of spheres, the cylinders yield a wider variety of local structures, e. g. both circular and very elongated elliptical crosssections. More precisely, we used a Boolean model of straight cylinders with circular cross-section of diameters uniformly distributed in 60 − 90 nm and with lengths uniformly distributed in 300 − 660 nm. The orientations are uniformly distributed on the upper half-sphere. That is, the model is isotropic. The solid volume fraction is 35%. See Figure 1 for the model realization used.
From the model realization, the FIB-SEM stack is simulated based on [9] as described in [12]. Throughout, the back scattered electron (BSE) signal is used. The solid component is assumed to be carbon, the primary electrons have an energy of 5 keV as in [12], and the dwelling time is 1µs (see e. g. [6] for details on SEM parameters). Both SEM image pixel size and slice distance are 3 nm. The training lasted for 100 epochs, where one epoch equals 50 steps with a batch size of 4. The initial learning rate of 0.0001 is halved after every 10 epochs.

Results
The network trained as described in the previous section is now used to segment synthetic FIB-SEM image stacks of a variety of other structures. More precisely, we segment images of Boolean models of spheres and a Cox Boolean model [8] of small spheres nested in large ones (see Figure 2(a)). These are complemented by packings of spheres by the force biased algorithm [2,3], of straight circular cylinders by random sequential adsorption (RSA), and of curved fibers by the Altendorf-Jeulin method (AJ) [1].
We measure the quality of our results by the false negative rate (FNR, the proportion of missed foreground pixels) and the Sørensen-Dice coefficient [23]. The latter is defined as where n is the total number of pixels in the volume, y represents the pixel-wise ground truth,ŷ the predicted (segmented) image, and T P , T N, F P , F N are counts of true positive, true negative, false positive and false negative pixels in the prediction, respectively. Due to the shine through effects described above and the typical coarser sampling in slicing direction, structures reconstructed from FIB-SEM image data tend to be anisotropic to an extent not explainable by the sample production or preparation [14]. All structures considered here are isotropic by design. That is, the distributions of the respective stochastic geometry models are invariant under rotations. Isotropy of the reconstructed structures is therefore a measure for their quality, too. It is by far not an easy task to test the realization of a random closed set for isotropy. Here, we just check the proportion of the mean chord lengths in x-and z-directions¯ x and¯ z as a rough indicator of artificial anisotropy. The only suspicious case is the RSA cylinder packing. However, here already the 3D ground truth image has a mean chord length ratio of¯ x /¯ z = 0.88.
All results are listed in Table 1 and visualized in Figure 2.

Conclusion
In this contribution, we show that a deep neural network -namely an adapted U-Net 3D -trained solely on synthetic FIB-SEM image stacks, is capable to reconstruct other highly porous structures from FIB-SEM images.  More details on the stochastic geometry models, wider variation of the FIB-SEM imaging parameters, and results on real data will be presented in [5].
The experiments leading to the results presented here highlighted the need for sufficient diversity of local structures in the data used for training. As a cautionary example we show in Figure 3 a result obtained for a cylinder packing using exactly the same network as described above, but trained solely on a Boolean model of spheres. Clearly, the thus "mis-trained" net tries to approximate the foreground by spheres. Open Access This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.
The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.