Reconstruction of three-dimensional anisotropic microstructures from two-dimensional micrographs imaged on orthogonal planes

Abstract

A pervasive method for reconstructing microstructures from two-dimensional microstructures imaged on orthogonal planes is presented. The algorithm reconstructs 3D images through matching of 3D slices at different voxels to the representative 2D micrographs and an optimization procedure that ensures patches from the 2D micrographs meshed together seamlessly in the 3D image. We show that the method effectively models the three-dimensional features in the microstructure using three cases (i) disperse spheres, (ii) anisotropic lamellar microstructure, and (iii) a polycrystalline microstructure. The method is validated by comparing the point probability functions of the reconstructed images to the original 2D image, as well as by comparing the elastic properties of reconstructed image to the experimental data.

Background

Three-dimensional microstructural information is essential for understanding the relationships between the material structure and its properties. Three-dimensional microstructures experimentally characterized by serial sectioning or X-ray computed tomography are expensive for routine applications due to the time and effort involved. The direct problem of measuring 2D surface images using optical or micro-diffraction methods is relatively easier. Using these 2D images, inverse models could be developed that would allow the generation of full 3D microstructural maps and speeding-up the development of microstructure databases for the purposes of microstructure selection and design.

An inverse problem of specific interest in this paper is the reconstruction of 3D microstructures from three orthogonal 2D sectional images taken along the x-, y-, and z-planes. The information contained in these three 2D micrographs is in the form of pixels containing colors corresponding to different constituent phases. The outcome of the inverse problem is a 3D microstructure containing voxels colored consistently such that any arbitrary x-, y-, or z-slice ‘looks’ similar to the corresponding input micrographs. This reconstruction problem leads to anisotropic microstructures, which is in contrast to other such works in literature that use a single reference (2D) image and make assumptions of microstructural isotropy, i.e., slices in every direction look similar to a single input image [1]. The most popular among these methods involves matching statistical features like two-point correlation functions of a single planar image to a random 3D image using optimization procedures like simulated annealing [2],[3]. Extension of these methods to achieve anisotropic microstructures has been proposed in the past using directionally dependent statistical features [4]. However, these methods are restricted to simple two-phase microstructures and are not applicable to more complex microstructures such as metallic polycrystals.

The approach proposed here involves maximizing the similarity between the solid microstructure and the 2D sectional microstructures by minimizing a neighborhood cost function. This cost function ensures that the local neighborhood on 2D slices taken along the x-, y-, or z-directions through the 3D microstructure is similar to some neighborhood in the 2D micrograph imaged along that plane. The approach is similar to those proposed in the computer graphics community [5] based on Markov random field assumption. This assumption simply states that microstructures have a stationary probability distribution or, in other words, different windows taken from a large microstructure ‘look alike’. To synthesize a voxel in the 3D image, the window in the 2D micrograph that best matches the unknown voxel’s neighborhood is chosen. The color of the voxel is decided based on the color indicated by the matching window in the 2D input image. The result is a simple method for generating 3D microstructures from 2D micrographs that generates visually striking 3D reconstructions of anisotropic microstructures, is computationally efficient, and is applicable to diverse microstructures.

Methods

Mathematical modeling of microstructures as Markov random fields

Some of the early attempts at microstructure modeling were based on Ising models [6]. In the Ising model, a N×N lattice (L) is constructed with values X_i assigned for each particle i on the lattice, i∈[1,..,N²]. In an Ising model, X_i is a binary variable equal to either +1 or −1 (e.g., magnetic moment [6]). In this work, the values X_i may contain any one of G color levels in the range {0,1,..,G−1} (following the integer range extension of the Ising model by Besag [7]). A coloring of L denoted by X maps each particle in the lattice L to a particular value in the set {0,1,..,G−1}. Ising models fall under the umbrella of undirected graph models in probability theory. In order to rewrite the Ising model as a graph, we assign neighbors to particles and link pairs of neighbors using a bond as shown in Figure 1a. The rule to assign neighbors is based on a pairwise Markov property. A particle j is said to be a neighbor of particle i only if the conditional probability of the value X_i given all other particles (except (i,j), i.e., $p\left(X_i\right|X_1,X_2,\mathrm{..},X_{i-1},X_{i+1},\mathrm{..},X_{j-1},X_{j+1},\mathrm{..},X_{N^2})$) depends on the value X_j.

Figure 1

Markov random field as an undirected graph model. Circles are pixels in the image, and bonds are used to connect neighbors. (a) Ising model with nearest neighbor interactions. (b) Microstructure modeled by including higher-order interactions in the Ising model.

Note that the above definition does not warrant the neighbor particles to be close in distance, although this is widely employed for physical reasons. For example, in the classical Ising model, each particle is bonded to the next nearest neighbor as shown in Figure 1a. In this work, we assume that a microstructure is a higher-order Ising model (Figure 1b). The particles of the microstructure correspond to pixels of the 2D image (or voxels in 3D). The neighborhood of a pixel is modeled using a square window around that pixel and bonding the center pixel to every other pixel within the window. The window size is a parameter that is chosen based on the scale of the biggest regular feature (e.g., grain size). Using this graph structure, a Markov random field can be defined as the joint probability density P(X) on the set of all possible colorings X, subject to a local Markov property. The local Markov property states that the probability of value X_i, given its neighbors, is conditionally independent of the values of all other particles. In other words, P(X_i|all particles except i)=p(X_i|neighbors of particle i). The microstructures are obtained by sampling the Markov random field P(X). In this paper, we present an algorithm to sample 3D microstructures by selecting the color of pixel X_i by sampling the conditional probability density p(X_i|neighbors of voxel i) from available 2D experimental data.

Algorithm

In the following discussion, let S^x, S^y, and S^z denote the set of orthogonal (x, y, and z, respectively) slices of the microstructure. Let V denote the solid (3D) microstructure. The color of voxel v in the 3D microstructure is denoted by V_v. In addition to the color (e.g., RGB triplet), the vector V_v may also contain other values including grain orientation and phase index. In this work, the color is represented using G color levels in the range {0,1,..,G−1}, each of which maps to an RGB triplet. The number of color levels is chosen based on the microstructure to be reconstructed, e.g., for binary images G=2.

Recall our Markovian assumption which states that the probability distribution of the color for a pixel given the colors of its spatial neighborhood is independent of the rest of the image. The vectors denoting the spatial neighborhood of voxel v in the slices orthogonal to the x, y, and z axis, respectively, are denoted as ${\mathit{V}}_v^x$, ${\mathit{V}}_v^y$, and ${\mathit{V}}_v^z$ (see Figure 2). The neighborhood is taken over a small user-assigned window around the voxel v. Let S^x,w, S^y,w, and S^z,w denote a window of the same size in the input 2D micrographs. In order to find the coloring of voxel v based on the neighbor voxels in the x-plane, one needs to compute the conditional probability density p(V_v|color of x-plane neighbors of v). Explicit construction of such a probability density is often computationally intractable. Instead, the most likely value of v is identified by first finding a window S^x,w that is most similar to ${\mathit{V}}_v^x$ in the input 2D micrograph. This window is denoted by ${\mathit{S}}_v^x$ (see Figure 2). Similarly, matching windows to the y- and z- plane neighborhoods of voxel v in the corresponding 2D sectional image (denoted as ${\mathit{S}}_v^y,{\mathit{S}}_v^z$) are found. Each of these matching windows ${\mathit{S}}_v^x$, ${\mathit{S}}_v^y$, and ${\mathit{S}}_v^z$ may have different coloring of the center pixel. Thus, we need an optimization methodology to effectively merge these disparate values and identify a unique coloring for voxel v. The optimization approach is described next.

Figure 2

The neighborhoods of v in the slices orthogonal to the x , y , and z axis, respectively, are shown. The windows in the input 2D micrograph shown in dotted lines are denoted by ${\mathit{S}}_v^i(i=x,y,z)$. These windows closely resemble the neighborhoods of v.

Let the value ${\mathit{V}}_{v,u}^x$ denote the color of voxel u in the neighborhood ${\mathit{V}}_v^x$. Similarly, the values ${\mathit{S}}_{v,u}^x$ and ${\mathit{S}}_u^{x,w}$, respectively, denote the color of pixel u in the window ${\mathit{S}}_v^x$ and S^x,w. The 3D microstructure is synthesized by posing the problem as a L² minimization of the energy [8]:

$$E\left(\mathit{V}\right)=\sum _{i\in \{x,y,z\}}\sum _v\sum _u{\omega }_{v,u}^i\parallel {\mathit{V}}_{v,u}^i-{\mathit{S}}_{v,u}^i{\parallel }^2.$$

(1)

Here, ${\omega }_{v,u}^i$ denotes a per pixel weight. In order to preserve the short-range correlations of the microstructure as much as possible, the weight for the nearby pixel is taken to be greater than those of the pixels farther away (Gaussian weighting is used).

The optimization is carried out in two steps. In the first step, the energy is minimized with respect to ${\mathit{S}}_v^i$. In this step, we assume that the most likely sample from the conditional probability distribution of the center pixel in the 3D image (e.g., p(V_v|colors of x-plane neighbors of v)) is the center pixel of a best matching window in an experimentally obtained 2D slice on the corresponding plane. The best matching neighborhood of voxel v along the x-plane is selected by solving the following problem:

$${\mathit{S}}_v^x=\text{arg}\underset{{\mathit{S}}^{x,w}}{min}\sum _u{\omega }_{v,u}^x\parallel {\mathit{V}}_{v,u}^x-{\mathit{S}}_u^{x,w}{\parallel }^2.$$

(2)

This is an exhaustive search that compares all the windows in the input 2D micrograph to the corresponding x-slice neighborhood of voxel v and identifies a window that leads to a minimum weighted squared distance. In this process, for 2D images of size 64×64 with a 16×16 neighborhood window, a matrix of size 16²×(64−16)² is built containing all possible neighborhoods of pixels that have a complete 16² window around it. The column in this matrix that has a minimum distance to the 3D slice ${\mathit{V}}_v^x$ is then found through a k-nearest neighbor algorithm [9]. Note that we are only given a limited (in this work, a single) 2D experimental sample along each cross-section, which means that the best match may not be an exact match for ${\mathit{V}}_v^x$.

Thus, for each voxel v, a set of three best matching neighborhoods are obtained, possibly with different colors corresponding to the center pixel. A unique value of v thus needs to be found by weighting colors pertaining to location v not only in the matching windows of voxel v but also in its neighbors. This is exactly done in the second step of the optimization procedure, where the optimal color of voxel v is computed by setting the derivative of the energy function with respect to V_v to zero. This leads to a simple weighted average expression for the color of voxel v:

$${\mathit{V}}_v=\left(\right.\sum _{i\in \{x,y,z\}}\sum _u{\omega }_{u,v}^i{\mathit{S}}_{u,v}^i\left)\right./\left(\right.\sum _{i\in \{x,y,z\}}\sum _u{\omega }_{u,v}^i\left)\right..$$

(3)

Note that the subscripts u and v are switched in the above expression as compared to Equation 1. This implies that the optimal color of the voxel v is the weighted average of the colors at locations corresponding to voxel v in the best matching windows (${\mathit{S}}_u^i$) of voxels (u) in the solid microstructure. Since V_v changes after this step, the set of closest input neighborhoods ${\mathit{S}}_v^i$ will also change. Hence, these two steps were repeated until convergence, i.e., until the set ${\mathit{S}}_v^i$ stops changing. As a starting condition, a random color from the input 2D images is assigned to each voxel v. The process is carried out in a multiresolution (or multigrid) fashion [10]: starting with a coarse voxel mesh and interpolating the results to a finer mesh once the coarser 3D image has converged to a local minimum. Three resolution levels (16³, 32³, and 64³) were used. Synthesizing a 64³ solid microstructure took between 10 and 15 min on a 3-GHz desktop computer, with about two-thirds of the time spend in step 1 (search) algorithm.

Results and discussion

The approach has been demonstrated for three test cases with 2D images corresponding to

• 1. Case 1. An isotropic distribution of solid circles;

• 2. Case 2. An anisotropic case with solid circles in the z-slice (similar to case 1) but an interconnected lamellar structure in the x- and y-slices;

• 3. Case 3. A polycrystalline microstructure.

In case 1, all three slices (x, y, and z) were assigned to the same 2D image depicted in Figure 3a. The resulting 3D microstructure is expected to be a random distribution of spheres. The 3D microstructure obtained by our approach is shown in Figure 3b. The internal structure of the solid microstructure is shown via slices in the x-plane at different distances from the origin. Various slices ‘look’ similar to the input image as expected from the Markov random field assumption. Case 2 builds upon this case by introducing anisotropy in the x- and y-planes. Three 2D images corresponding to x, y, and z-slices (as shown in Figure 4a) were used in the reconstruction. An interconnected lamellar structure was used in the x- and z-planes, while the z-plane image allowed merging of the solid circles to allow for a more complex microstructure. In the algorithm, we match the 2D images with all three orthogonal slices through every voxel. The resulting anisotropic 3D microstructure shown in Figure 4b is quite complex. The y-axis slices as shown in Figure 4c show the depth profile of various solid circles seen at the top surface, with intricate internal structure revealed.

Figure 3

Example of Markov random field reconstruction: case 1. (a) Input 2D microstructure showing an isotropic distribution of solid circles. (b) 3D reconstruction. (c) 3D sectional images of the reconstructed microstructure.

Figure 4

Example of Markov random field reconstruction: case 2. (a) An anisotropic case with solid circles in the z-slice (similar to case 1) but an interconnected lamellar structure in the x- and y-slices. (b) 3D reconstruction. (c) 3D sectional images of the reconstructed microstructure.

In the last example, a polycrystalline microstructure was employed to show the applicability of the algorithm to cases beyond two-phase media. The microstructure is equiaxed, and all three slices were assigned to the same 2D image shown in Figure 5a. The resulting 3D microstructure is shown in Figure 5b, and its internal structure revealed through the x-axis slices in Figure 5c. The results show that the grains built by the algorithm are also equiaxed with a variety of 3D shapes identified by the algorithm. However, some of grain boundaries do not show up well in the slices which is primarily attributed to the lower resolution of the 3D image (64³) compared to the original input image.

Figure 5

Example of Markov random field reconstruction: case 3. (a) An experimental 2D polycrystalline microstructure. (b) 3D reconstruction. (c) 3D sectional images of the reconstructed microstructure.

Validation tests

For testing the validity of the 3D reconstructions, quantitative comparisons were made between the original 2D image and the reconstructed image, through comparison of the statistical correlation functions as described in [11]. Rotationally invariant probability functions are employed as the microstructural features. Rotationally invariant N-point correlation measure ($S_{\left(N\right)}^i$) can be interpreted as the probability of finding the N vertices of a polyhedron separated by relative distances x₁,x₂,...,x_N in phase i when tossed, without regard to orientation, in the microstructure. The simplest of these probability functions is the one-point function, S₍₁₎, which is just the volume fraction (V) of phase i. The two-point correlation measure, $S_{\left(2\right)}^i\left(r\right)$, can be obtained by randomly placing line segments of length r within the microstructure and counting the fraction of times the end points fall in phase i. These statistical descriptors occur in rigorous expressions for the effective electromagnetic, mechanical, and transport properties like effective conductivity, magnetic permeability, effective elastic modulus, Poisson’s ratio, and fluid permeability of such microstructures ([4],[12]). All the required correlation measures needed for comparison and property bound calculation are obtained using a Monte Carlo sampling procedure [1]. The procedure involves initially selecting a large number of initial points in the microstructure. For every initial point, several end points at various distances are randomly sampled, and the number of successes (of all points falling in the i^th phase) are counted to obtain the required correlation measures. Statistical measures were extracted from the microstructures by sampling 15,000 initial points.

Two- and three-point correlations of the isotropic distribution of solid circles

The statistical features of the 2D distribution of solid circles from Figure 3 and its 3D reconstruction were compared. The original 2D image was a square of side 64 μ m and had a phase 1 (white phase) volume fraction of 70%. The comparison of two-point probability ($S_{\left(2\right)}^1$) and the three-point probability function $S_{\left(3\right)}^1$ is shown in Figure 6a,b, respectively. The three-point probability measure $S_{\left(3\right)}^1(r,s,t)$ is depicted in a feature vector format with the distances (r,s,t)μ m indicated for key points in Figure 6b. The first points in both graphs (Figure 6) show the volume fraction of white phase for 2D image as well as the reconstructed image. The decay in the two-point correlation function is identical for the reconstructed image up until 3μ m, showing excellent reproduction of the short-range correlation. The same aspect can also be seen from comparing the short-range correlation in the three-point probability function (Figure 6b). Although the longer range correlations match qualitatively, there is a drift seen as the distance between pixels increases. Both the excellent match in short-range correlation and the small drift in the long-range correlation can be explained based on the reconstruction algorithm, which models a stronger interaction of a center pixel to pixels in its immediate local neighborhood than pixels farther away. In effect, the algorithm gives a stronger weighting towards matching the short-range correlations in the microstructure.

Figure 6

Comparison of the features of the 2D and reconstructed image shown in Figure2. (a) Two-point probability function. (b) Three-point probability function.

Elastic properties of two-phase composite

The experimental data in [13] provides a high-resolution planar microstructure image (Figure 7a) of a silver-tungsten composite with porous tungsten matrix and molten silver (volume fraction of silver phase p=20%). The microstructure has been employed for several reconstruction studies [14],[15]. A 657×657 pixel region of the microstructure corresponding to 204- μ m square area was converted to a black-and-white image for distinguishing the two phases. This was done by selecting a threshold color below in which phases were set to white (the silver phase), and the rest of the image was set to black (the tungsten phase). The final black-and-white image is shown in the inset of Figure 7a. A 64- μ m square cell within this image was chosen to reconstruct the 3D image.

Figure 7

Markov random field reconstruction of a tungsten-silver microstructure. (a) Experimental tungsten-silver composite image (204×236 µm) from Umekawa et al. [13]. The black-and-white image corresponds to a thresholded image with white representing the silver phase and black representing tungsten. A 64-µm square cell shown in the inset was used to reconstruct the 3D image. (b) A 64-µm length cell of reconstructed 3D microstructure of the experimental image showing silver distribution. (c) The tungsten phase of the reconstructed microstructure.

An instance of the reconstructed microstructure is shown in Figure 7b,c with the distribution of each phase shown separately. The auto-correlation function for the silver phase $\gamma \left(r\right)=\frac{S_{\left(2\right)}^1\left(r\right)-p^2}{p-p^2}$ of the reconstructed 3D microstructure and the experimental image are compared in Figure 8a showing excellent match of short-range correlations with a small difference seen in longer range correlations. Short-range correlations carry the greatest weightage in determining mechanical properties such as elastic modulus (e.g., [4]), although long-range correlations have been found to be important for phenomena such as surface roughening during plastic deformation [16]. To test if the elastic properties are well captured in the reconstructed 3D microstructure, we compared against the experimental data from [13] of the elastic modulus as a function of temperature. The elastic properties of the individual components at different temperatures are available from [14] and are listed in Table 1. The data was used within a finite-element simulation to compute the elastic modulus of the reconstructed microstructure using the method described in [17]. The computed properties of the reconstructed 3D microstructure closely follow the experimentally measured Young’s modulus from [13] as shown in Figure 8b with an average error from the experimental data of about 5%.

Figure 8

Comparison of properties of 3D reconstruction of silver-tungsten composite. (a) The autocorrelation function for the silver phase. (b) Experimental Young’s modulus is shown along with the FEM results for the reconstructed 3D microstructure.

Table 1 Elastic properties of silver and tungsten phases as a function of temperature (from [[14]])

Conclusions

A method for reconstructing diverse microstructure from two-dimensional microstructures imaged on orthogonal planes is presented. The algorithm reconstructs 3D images through matching of 3D slices at different voxels to the representative 2D micrographs. This is posed as an iterative optimization problem where the first step involves searching of patches in the 2D micrographs that look alike to the 3D voxel neighborhood, followed by a second step involving the optimization of an energy function that ensures various patches from the 2D micrographs meshed together seamlessly in the 3D image. The method is particularly promising for anisotropic cases where the x-, y-, and z-slices look different. The results demonstrate that the method can effectively model three-dimensional features in the microstructure including complex interconnectivity of the features and complex shapes that are not intuitive at first sight. The approach can be useful to rapidly build a library of 3D microstructures for modeling purposes from 2D micrographs. Although, this preliminary study shows significant promise as to the feasibility of the approach, future work will focus on increasing the resolution of the reconstruction and code optimization. In addition, future work will focus on more rigorous testing of the stereological features (e.g., grain-size histograms) and other engineering properties (yield strength) of reconstructed anisotropic microstructures.

Availability of supporting data

The executables and data files for the methodology described here are available upon request.

Author’s information

VS is an Associate Professor at the University of Michigan and is a lifetime member of The Minerals, Metals and Materials Society (TMS). VS developed the methods and examples shown in this work.