Reconstructing Microstructures From Statistical Descriptors Using Neural Cellular Automata

The problem of generating microstructures of complex materials in silico has been approached from various directions including simulation, Markov, deep learning and descriptor-based approaches. This work presents a hybrid method that is inspired by all four categories and has interesting scalability properties. A neural cellular automaton is trained to evolve microstructures based on local information. Unlike most machine learning-based approaches, it does not directly require a data set of reference micrographs, but is trained from statistical microstructure descriptors that can stem from a single reference. This means that the training cost scales only with the complexity of the structure and associated descriptors. Since the size of the reconstructed structures can be set during inference, even extremely large structures can be efficiently generated. Similarly, the method is very efficient if many structures are to be reconstructed from the same descriptor for statistical evaluations. The method is formulated and discussed in detail by means of various numerical experiments, demonstrating its utility and scalability.


Introduction
The generation and analysis of random heterogeneous composite materials is a recently emerging research topic that aims at accelerating materials engineering by enabling digital workflows such as numerical simulation and inverse design [1].Specifically, microstructure characterization and reconstruction allows to (i) generate many microstructure realizations from a single example, (ii) explore hypothetic materials by interpolating between microstructures in a morphologically meaningful manner, and (iii) create 3D models from 2D observations.A multitude of approaches has been developed in the last decades that is summarized in different review articles [2,3,4].For the purpose of this work, the existing approaches can be broadly divided in four categories1 -simulation, Markov random field, deep learning and descriptor-based approaches.Naturally, some algorithms in the literature can be identified as hybrid methods that fall into two or more of these categories.After discussing the main ideas of these categories and approaches in subsection 1.1, this work presents an algorithm that bridges all four categories and exhibits some very interesting properties as described in subsection 1.2.

Existing approaches for microstructure reconstruction
Simulation-based approaches Simulating the microstructure evolution might be the most direct way.This requires to identify and to solve the physical (partial differential) equations (PDEs) that govern the process.An excellent overview is given in [2].As an example, the Cahn-Hilliard equation describing phase separation [6] has been studied extensively [7,8,9].Similarly, for granular structures, given a representative set of particles, realistic and dense packing can be achieved by simulating gravitational forces [10,11,12].As a final, more complex example, grain formation in polycristalline structures has been studied in depth.Simplified approaches reduce the description to vertices [13] or grain boundaries [14], whereas Monte Carlo methods [15] or cellular automata [16,17,18] are used to model the evolution of an entire 2D pixel field.Recently, neural cellular automata have been applied to solidification microstructure modeling [19].Approaches based on the phase field method are probably the most developed.Thereby, the evolution of a diffuse indicator function is modeled by an additional differential equation [20,21,22] that can be solved, for example, in OpenPhase [23].These approaches are often applied to simulate the complex microstructure morphologies that arise in additive manufacturing [24,25,26].This nonexhaustive list indicates that a variety of physical processes are responsible for the formation of different material classes.Even if the relevant set of physical equations is selected, it can be challenging to perform the simulations due to numerical issues or difficulties in parameterizing the underlying constitutive models [27,25].This motivates the purely image-based approaches that are presented in the following.

Markov-based reconstruction
As a first purely image-based method, this subsection discusses a class of reconstruction algorithms originally developed for computer graphics applica-tions which are herein referred to as Markov-based approaches.For this purpose, it is worth noting that a microstructure can be modeled as a stationary Markov random field if the probability of finding a certain phase at a given location does not depend directly on the location, but only on the phase distribution in the local finite-size neighborhood.
This assumption of locality and stationarity motivates reconstruction algorithms that directly rely on this conditional probability to update individual pixels based on their neighbor's values.A very simple implementation inspired by texture synthesis [28] might determine individual pixel updates by scanning the reference data for the given neighborhood in order to compute a probability [29,30].It is worth noting that this approach is akin to the multi-point statistics method that has been developed in the Geosciences literature [31] and has been applied and improved substantially by Tahmasebi [32,33,34,35].For a better scalability, improved algorithms precompute the probabilities for all neighborhoods and store them in efficient data structures for access during reconstruction [31,36].Direct sampling methods [37] as well as data structure-based alternatives are implemented in MPSLIB [38].
Despite a good local prediction quality, MRF-based approaches often fail to accurately reproduce long-range correlations.This behavior is related to the neighborhood size in the Markovian assumption: Capturing long-range correlations requires large neighborhood sizes, which are often unfeasible because of a disproportionately increased need for training data.Multigrid approaches [39,35] have been shown to alleviate this issue to a certain extent.Furthermore, to condense the information to a compact model that is also able to interpolate missing neighborhood patterns from similar examples, supervised models have been trained to predict a pixel's phase given its neighborhood.In particular, decision trees [40] and neural networks [41,42,39] have been used for 2D and 3D [43] reconstruction.This motivates the discussion of purely deep learning-based approaches in the following subsection.

Deep learning-based reconstruction
In deep learning-based methods, a generative model is fitted or trained on a sufficiently large data set of microstructures and is then used to sample new realizations of the same structure.Autoencoders [44,42,45] and generative adversarial networks (GANs) are typical examples that have been applied to MCR [46,47].For the latter, the merits of modifications like conditional GANs [48,49], Sytle-GAN [50], and gradient penalty [51] have also been discussed in the context of microstructure generation.Applications to steels [52] and earth materials [53] show high image quality.Although GANs usually operate on 2D data, 3D-to-3D reconstruction can be achieved by using 3D convolutions [54,55].For reconstructing 3D data from 2D examples, a 3D generator has been combined with a 2D discriminator [56,57].As an alternative, the third dimension can be regarded as time by combining the GAN with a recurrent neural network [58].To harness the advantage of both, autoencoders and GANs, they are sometimes combined by using the decoder simultaneously as a generator.This has proven advantageous for 2D-to-3D reconstruction [59, 60,61] and for extremely small data sets [62].
As an alternative, machine learning methods like Bayesian approaches [63] and attention-based models [64,65,66] are equally applicable.Diffusion models, which have recently replaced GANs as state-of-the-art in general-purpose image generation, have also been applied to microstructure reconstruction [67,68] and optimization [69,70].
Much research is focused on identifying suitable model types and adapting them to microstructure reconstruction by enabling 2D-to-3D reconstruction [61,57] making them applicable to small data sets [62] or ensuring that certain descriptor requirements are met [71,72].A major challenge lies in defining models with high accuracy that at the same time do not require large data sets to be trained on.These challenges motivate training-free models such as descriptorbased reconstruction, as presented in the next subsection.

Descriptor-based reconstruction
The central idea behind descriptor-based reconstruction methods is to statistically quantify the microstructure morphology by means of descriptors like volume fractions and spatial -point correlations [73].Reconstructing a microstructure from a given set of descriptors can then be formulated as an optimization problem directly in the space of possible microstructures.Here, the desired microstructure descriptors can be computed from a single microstructure example, making these methods very dataefficient.
One of the most well-known descriptor-based reconstruction methods is the Yeong-Torquato algorithm [73], which iteratively swaps individual pixels in the microstructure to solve the optimization problem.A detailed discussion is given in [74,75].This enables high flexibility, as descriptors can be replaced by new alternatives [76,77] or higher-fidelity versions of the same descriptor [78,79].However, even with computationally inexpensive descriptors, the Yeong-Torquato algorithm becomes computationally challenging at high resolutions and in 3D, where billions of iterations are sometimes required for convergence [80].A common solution is to use a multigrid scheme [81,82,83,84,85].Further ideas include different-phase neighbor sampling rules [86], efficient descriptor updates [87,80] and optimized directional weighing of correlation functions [78].More information is given in [3].
As an alternative to the pixel-based Yeong-Torquato algorithm, the optimization problem can be formulated in a much lower-dimensional space.For this purpose, the microstructure is approximated by geometric objects that can be described by a few parameters, e.g., ellipsoidal inclusions [88,89,90] or Voronoi or Laguerre cells [91,92,93].
Independently from the microstructure representation [90], differentiable descriptors allow solving the optimization problem using a gradient-based optimizer.This idea is formulated as differentiable microstructure characterization and reconstruction (DMCR) [94,95] and several approaches can be identified as special cases [71,96,97].
The Yeong-Torquato algorithm and improved versions of it, such as DMCR, have been successfully validated and applied to alloys and anisotropic metamaterials [85] sandstone [98], rock [99], chalk [100], various soils [101] and more.Some versions are publicly available in the open-source MCRpy package [102].While descriptor-based approaches are very accurate and data-efficient since no training data set is required, they are computationally intensive.More specifically, since the optimization is directly carried out in the microstructure space, the memory and computational requirements grow quickly as the microstructure size increases, especially in 3D.

Hybrid reconstruction approaches
The specific and unique advantages and disadvantages of all four categories of MCR approaches motivate hybrid methods that fall into multiple of these categories.Naturally, there is no sharp boundary between Markov-based and deep learning methods if a machine learning model like a neural network is used to predict individual pixels based on their neighborhood as in [40,41,42,39,43].Furthermore, simulation by discretized (partial) differential equations and cellular automata resemble Markov-based methods in their locality, but are derived from physical principles and sometimes incorporate various physical quantities (e.g.temperature) beyond phase indicator functions.At the boundary between machine learning and descriptor-based methods, multiple sequential approaches use Gaussian random field-based methods [103] to initialize simulated annealing2 [100,104] and diffusion models [72].Furthermore, the volume fractions [105,72,58,106,49], histograms [107] and Gram matrices [10,108] are sometimes added to the loss function of deep learning-based methods as microstructure descriptors.DRAGen [109] combines an automaton-like growth process with a nucleation point optimization based on classical descriptors and allows to use machine learning models for generating input data.At the interface between machine learning and physical simulation, autoencoders [110] and diffusion models [12] have been used as particle generators followed by a gravity simulation for aggregate structures.Besides that, the literature comprises a large number of physics-informed neural network approaches that are not discussed herein.

Objectives and contribution of this work
This work presents a hybrid approach that is inspired by all these categories.Like in a simulation-based approach, a partial differential equation models the temporal evolution of the microstructure.It is, however, not derived from physics but learned by a neural network.Similar to the Markov-based methods, this network operates based on local information and is therefore called neural cellular automaton (NCA).This constraint of locality is relaxed not by increasing the neighborhood beyond a one pixel distance, but by introducing further hidden channels to the microstructure function that the NCA can use to encode relevant information.Finally, unlike common machine learning or Markov-based approaches, the NCA is not trained directly on image data or on a set of neighborhoods, but on a statistical descriptor.This requires the NCA to be retrained whenever the statistical descriptor changes, however, it reduces the amount of required data to a bare minimum.The input image only needs to enable the computation of a statistical descriptor; hence the NCA is applicable whenever classical training-free approaches like the Yeong-Torquato algorithm and DMCR can be used.Furthermore, the size of the training data is independent of the image size during training, which is again independent from the size of the reconstructed structure.Hence, microstructures of massive resolutions or numbers can be reconstructed with very limited additional computational effort.Furthermore, due to the nature of NCA, the algorithm is inherently distributed, parallel and robust with respect to perturbations.
In summary, the central idea lies in modeling the differential equation governing the structure evolution by training neural cellular automata (NCA) on statistical descriptors.A detailed formulation is given in section 2 and validated by various numerical experiments in section 3. A conclusion is drawn in section 4.

Neural cellular automata for descriptor-based microstructure reconstruction
Based on the work of Mordvintsev et al. [111], the formulation of general neural cellular automata (NCA) is summarized in subsection 2.1.The main idea of the present work to train NCA by arbitrary descriptors is described in subsection 2.2.Finally, the implementation is discussed in subsection 2.3.

Formulation of neural cellular automata
The general idea behind a cellular automaton is to iteratively update individual pixels based on the direct neighbors.In the work of Mordvintsev et al. [111], this information source is further restricted.The neighboring pixel values are not passed directly to the cellular automaton.Instead, they are used to compute a discrete approximation to the gradient and curvature, which are then passed to the cellular automaton.Denoting  ∈ D as a position vector in the microstructure domain D ⊂ R2 and  ∈ T = { ∈ R | 0 ≤  ≤  end } as time, the evolution of the microstructure (, ) can be written as a partial differential equation where   is the cellular automaton which maps the value, gradient and curvature of the microstructure function to its temporal derivative.To be more specific, ∇  (•) and ∇ 2  (•) denote the gradient and Laplace operator, respectively.Furthermore,  takes real values within the arbitrarily chosen bounds In a neural cellular automaton specifically, a neural network is chosen as   , where  denotes the parameter vector.
In other words, the NCA defines partial differential equation (PDE)3 that needs to be discretized and solved in order to generate a microstructure.An explicit Euler scheme is chosen as a time stepping scheme where the current solution at time step   defines the update for the next time step   + 1.The dependence on  and  is dropped for the sake of brevity.The space is naturally discretized on an equidistant grid of pixel values, where ∇  (•) and ∇ 2  (•) are approximated by a Sobel and Laplace filter, respectively.Based on this discretization, the relation between the current solution, its spatial derivatives and its temporal evolution, i.e. the PDE itself, is learned by the NCA.
Given the inability of Markov-based approaches with small neighborhood sizes to accurately capture long-range correlations, it should be clear that the extremely limited local information is not sufficient to train a good NCA.For this reason, the augmented microstructure function  ′ (, ) is introduced which maps a spatial position  at time  to an dimensional vector.The first entry of the vector contains the normal microstructure function (, ) and is the only entry that affects the training.The idea behind the other entries is that the NCA can choose to allocate any quantity that is useful for passing information and increasing the image quality.As an example, for an equidistant grain microstructure, one channel might contain the distance to the next grain boundary.With this, the temporal evolution reads For reconstructing a microstructure from the trained NCA,  ′ (, 0) is initialized by zeros and the system evolves freely.

Training the model from microstructure descriptors
The function   is learned by a small neural network with two layers as shown in Figure 1: First, an initial solution  ′ (, 0) = 0 ∀  is chosen.Secondly,  ′ (, ) develops according to Equation 4 for a randomly chosen number of time steps.As a regularization and as a measure to break symmetry, asynchronous updates are chosen, whereby in every time step, a given percentage of cells is chosen at random and only those develop.The bounds given in Equation 2 are enforced by clipping.Thirdly, a loss function L is computed on the final result  end = (,  end ).The choice of L is discussed later.Note that only  end , i.e., the first component of  ′end , contributes to L. Finally, the gradient L/ of the loss function with respect to the NCA parameters is computed by conventional backpropagation and used to update .Note that this limits the number of timesteps during training for numerical reasons.
The formulation of the loss function depends on the area of application of the NCA.After initially using a pixel-wise 3To be precise, the NCA defines a PDE system, as explained later in the document.
Euclidean norm error in the RGB space for general-purpose image generation [111], Mordvintsev et al. [112] found that a Gram matrix-based style loss [113] enable NCAs to be applied to texture synthesis [112].
The novelty in the present work lies in realizing that any of the known statistical descriptors can be used, as long as they can be differentiated with respect to the microstructure field.The loss is thus formulated as a mean squared error (MSE) in the descriptor space where  denotes a statistical descriptor or a weighted concatenation of multiple descriptors that is computed on the reconstruction result, while  des denotes the desired value computed from the reference structure.Because  end results from the temporal evolution of   , it depends on the parameter vector  of the NCA.The central idea is train the NCA by gradient-based optimization of  to minimize Equation 5, whereby arbitrary descriptors can be incorporated.While the Gram matrices used in [112] can be interpreted as a statistical descriptor, the spatial two-and three-point correlations are more common in microstructure reconstruction.The idea of using high-dimensional, differentiable descriptors for direct microstructure reconstruction is given in [94], where an efficient and differentiable formulation of the three-point correlations is given.As another example, a differentiable approximation to lineal path function is presented in [102] and a descriptor based on a hierarchical wavelet transform is given in [114].All these descriptors are implemented in MCRpy [102].

Implementation
The implementation of a descriptor-based NCA for microstructure reconstruction is carried out based on the code for NCA texture synthesis [112] and the differentiable descriptors available in MCRpy [102].The former code is adapted to only a single non-hidden dimension () as opposed to three RGB channels.Then, MCRpy is used to define a loss, where different descriptors such as Gram Matrices  [71], correlations  [74], variation V [95] and volume fraction  can be combined and weighed in a single loss function in a flexible manner.More information on these descriptors is given in [102].MCRpy makes use of the automatic differentiation in TensorFlow to compute the gradient L/.Then,  is backpropagated through time to compute / and consequently L/.
Finally, a hyperparameter study is carried out on a number of structures.A 12-dimensional microstructure representation (i.e.11 hidden channels) is chosen.Hence, the NCA has 12 output and 48 input dimensions.With a single hidden layer of 120 neurons, the network amounts to a total of 7332 parameters.Further hyperparameters like the number of time steps are summarized in Table 1.
In order to visually compare the results of descriptor-based NCA with other methods from the literature, three open-source codes are selected from GitHub.To represent Markov-based methods, a patch-based texture synthesis4 algorithm based 4https://github.com/anopara/texture-synthesis-nonparametric-samplingPreprint submitted to Springer arXiv where the gradient and curvature is computed numerically.Together with the pixel value, these quantities are given to the NCA to predict a pixel update.After some time increments, the result is compared to the reference to train the NCA.This comparison is only carried out in terms of statistical descriptors.on [115,116] and a pixel-based, multi-resolution texture syn-thesis5 algorithm based on [117,118] are chosen.Furthermore, MCRpy [102] implements differentiable microstructure characterization and reconstruction (DMCR) [94,95].While MCRpy is provided by previous works of the authors, the former two methods are coded and provided by Anastasia Opara.The authors greatly acknowledge this effort and appreciate the will to share software.

Numerical experiments
The microstructure evolution and the range of applicability is investigated in subsection 3.1.These results are then compared to the literature in subsection 3.2.Finally, the scalability of descriptor-based NCA is demonstrated in subsection 3.3.All numerical experiments are carried out on a laptop with a 12 th Gen Intel(R) Core(TM) i7-12800H CPU at 2.40 GHz and an Nvidia A2000 GPU with 4 GB VRAM.

Microstructure evolution and diversity
Figure 2 shows reconstructions from different real materials taken from [71].It can be seen that descriptor-based NCA are applicable to a wide variety of fundamentally different structures, ranging from relatively noise-free examples like the grain boundary structure and the ceramics to the more noisy sandstone.Some limitations can also be seen.As a first limitation, although the grain boundary character in the alloy is captured relatively well, not all lines are connected as in the reference case.In order to use the results for downstream tasks like numerical simulations, a post-processing algorithm is first needed to close the gaps or eliminate unnecessary line segments.Alternatively, it might be worth investigating 5https://github.com/anopara/multi-resolution-texture-synthesisif a different choice of descriptor can be used in order to better quantify the connectivity information.Although this approach is arguably more elegant, its difficulty lies in the requirement that the descriptor should be differentiable with respect to the microstructure.As a second limitation, it can be seen that the fingerprint-like structure of the copolymer is not adequately represented.Although the NCA successfully creates individual sub-regions with parallel lines, these regions are not sufficiently large and the lines do not exhibit smooth curves as in the reference.It is presently unclear to the authors how this issue can be addressed.As a third limitation, it is noted that the probability distribtion of pixel values does not exactly match the original structures.Especially in the carbonate and PMMA, it can be seen that the white phase is reconstructed in bright grey color.Similar to the first limitation, the authors assume that a post-processing algorithm or a suitable descriptor should be sufficient to address this issue.
To provide a better understanding of the generation method, the temporal evolution of the microstructure as well as the first four hidden dimensions is plotted in Figure 3.All fields are initialized by zero (black) and the structure slowly emerges.Different hidden channels take different roles in representing structural features.For example, the first hidden channel (second row) might be interpreted as a local vertical coordinate in each grain.In contrast, the fourth hidden channel (last row) contains a thickened version of the gain boundaries.Interestingly, the third hidden channel (second to last row) can be interpreted in different ways.It might be used as a type of residuum, since its norm decreases as the reconstruction converges.As an alternative, it might act as a marker for specific features like triple junctions.It can be concluded that different channels take different roles, although a direct interpretation is neither possible, nor necessary.
It is demonstrated in the works of Mordvintsev et al. [111,112] that the NCA-based generation process is often robust with respect to perturbations.To test whether this trend is transferred to descriptor-based NCA for microstructure reconstruction, two numerical experiments are carried out.After the generation process has converged to a good solution, the structure is perturbed by setting all values within a circular radius to 0.5.This is applied to all channels in Figure 4 and only to the nonhidden dimension in Figure 5.It can be seen that the structure only recovers in the latter case.Besides stressing the key role of the hidden channels, this indicates that the robustness of NCA is only partially observed in descriptor-based NCA for microstructure reconstruction.

Comparison to literature
In order to compare descriptor-based NCA reconstruction results to the literature, two Markov-and one descriptor-based approach is chosen6.Figure 6 shows the results, where only three material classes are selected for the sake of brevity.At a first glance, all methods produce high-quality results.
6The authors were not able to apply deep learning-based algorithms, since only a single reference sample is available.Similarly, simulation-based approaches are omitted because they require a detailed analysis and modeling of the materials' underlying physics that exceeds the scope of this work.
Patch-based texture synthesis, however, does not produce new structural features, but copies pathes from the original structure to different locations in the target image.The patch boundaries can be distinguished upon closer inspection.As a pixel-based approach, multi-resolution texture synthesis does not suffer from this phenomenon.However, especially in the alloy, strange features like completely vertical grain boundaries can be observed and the structure coincidentally repeats itself in the top right corner.Furthermore, the highly complex fingerprintlike copolymer structure is not captured adequately.Finally, DMCR as a descriptor-based method7 produces good results for all considered materials.While the alloy and ceramics are similarly well reconstructed as with the descriptor-based NCA, DMCR produces visually superior results for the copolymer.It can be concluded that the result quality of NCA outperforms standard Markov-based techniques and almost reaches that of direct descriptor-based optimization.The advantage, with respect to the latter lies in the performance and scalability as discussed in the following.

Performance and scalability
An objective assessment of the reconstruction results in terms of microstructure descriptors is paramount to evaluating the accuracy of any reconstruction algorithm.In this context, it should be mentioned that the presented method is only partially descriptor-based since the descriptors are used during training, but not during sampling.For this reason, independent realizations of the material exhibit random deviations from the target descriptor.Naturally, these fluctuations are expected to decrease as the microstructure size increases.This is shown in Figure 7, where the error between a descriptor  ( end ) and its desired value  des from the reference structure is defined as a mean squared error (MSE).Note that the only difference between E  and the loss L defined in Equation 5 is that the former measures individual descriptors, whereas the latter is based on a weighted concatenation of multiple descriptors.For all tested descriptors, the error converges to a value which is consistently lower for the proposed loss model than for the reference NCA-based texture synthesis method by Mordvintsev et al. [112].It should be noted that the descriptor errors do not converge to zero as the resolution increases, but rather to a value that depends on the training quality.It is observed that longer training and training by larger samples reduces this value (not shown here).
An interesting aspect of NCA is that the image sizes during training and sampling are independent.This is favorable because the sampling is relatively inexpensive and scales favorably with the image size compared to other methods.To demonstrate this, Figure 8 shows a reconstruction example where the resolution is chosen such that the sampling takes 7The Yeong-Torquato algorithm can be expected to yield equally good or even better results, however, at a significantly higher computational cost.The role of the hidden channels is illustrated by perturbing the microstructure evolution at time  =  ′ .All pixel values within a given radius are set to 0.5.In the presented case, all channels are perturbed, whereas in Figure 5, the hidden channels remain intact.Only the microstructure (top) and the first hidden channel (bottom) are plotted for brevity.Unlike in Figure 5, the structure does not recover.If can be seen that regardless of the model, material and descriptor, the variance of the descriptor error over different realizations decreases as the sample size increases.The proposed model consistently outperforms the reference [112].For some structures like the ally (a), the reference model fails to converge, leading to massive discrepancies, whereas for the ceramics (c) the differences are relatively small.
as long as the training8.Three different zoom levels of the same structure are shown for visualization purposes.Without any multigrid procedures, such large reconstructions are very challenging with classical descriptor-based methods.Generally, the computational cost of sampling a microstructure scales linearly in the number of pixels, because pixel updates are computed independently by the NCA.A comparison with the 2D DMCR algorithm [94] in MCRpy [102] is given in Figure 9.Both methods scale linearly.Sampling from a trained NCA is much faster than reconstructing by DMCR, and the computational cost grows more slowly.This is because the expensive evaluation of microstructure descriptors and iterative optimization are moved to the training stage.If the training is added to the computational cost of the NCA, they are slower for the considered microstructure sizes.As a conclusion, the expensive training phase of an NCA is compensated if large or many microstructures are reconstructed.Especially the latter might speed up a potential future extension for 2D-to-3D reconstruction.Furthermore, unlike with DMCR [94, 95], the sampling can be trivially parallelized because updates are based only on local information.

Conclusion
A neural cellular automaton (NCA)-based algorithm for microstructure reconstruction is presented.The microstructure evolution is modeled as a partial differential equation which is learned by a small neural network, the NCA.Despite the purely local information in the NCA, long-range correlations are incorporated by introducing hidden dimensions to the microstructure function which can be used to communicate information.Unlike with previous approaches, this network is not trained on image data but on statistical microstructure descriptors.Thus, the method incorporates ideas from four different families of microstructure generation approaches, namely simulation, Markov, deep learning and descriptorbased methods, which are all briefly reviewed.The method is formulated, implemented and validated by a number of 2D numerical experiments.
Compared to other microstructure reconstruction approaches, descriptor-based NCAs have a unique set of advantages.The neural network in the NCA enables the evolution of highly complex morphologies in a PDE-like manner without knowledge of the governing physical equations and the material parameters.It can be controlled by statistical descriptors.However, the sampling of structures from a trained NCA is based only on local information.This self-assembling nature of the algorithm makes it an inherently distributed algorithm and therefore trivial to parallelize.The random selection of the pixels to be updated make the method robust with respect to random perturbations, as long as not all channels are affected.Finally, the method scales very favorably as arbitrarily resolved structures can be sampled.
8Because the utilized VRAM is not sufficient for the large reconstruction, it is conducted on the CPU only, whereas the training occurs on the GPU.If a similar comparison was made on identical hardware, significantly larger structures could be reconstructed.Regardless, in the authors' opinion, the presented results demonstrate the scalability sufficiently well.
In future work, the main challenge lies in enabling 3D reconstruction based on 2D or 3D reference data.

Figure 1 :
Figure 1: Training procedure for a neural cellular automaton (NCA): In every iteration , random pixel locations are chosenwhere the gradient and curvature is computed numerically.Together with the pixel value, these quantities are given to the NCA to predict a pixel update.After some time increments, the result is compared to the reference to train the NCA.This comparison is only carried out in terms of statistical descriptors.

Figure 2 :
Figure 2: Reconstructions from various real materials.The original samples are given in the top left corner and are taken from [71], where they are released under the Creative Commons license [119].

Figure 3 :Figure 4 :
Figure 3: The evolution of the alloy microstructure over time .The first channel is plotted in the top row and the first four hidden channels are given below.It can be seen that each hidden channel acts as a distinct feature map.

Figure 5 :
Figure 5:The role of the hidden channels is illustrated by perturbing the microstructure evolution at time  =  ′ .All pixel values within a given radius are set to 0.5.In the presented case, the hidden channels remain intact, whereas in Figure4, all channels are perturbed.Only the microstructure (top) and the first hidden channel (bottom) are plotted for brevity.Unlike in Figure4, the structure recovers, albeit to a different solution.

Figure 6 :
Figure 6: Comparison of three selected materials reconstructions with methods from the literature.Patch-based texture synthesis (PBTS) and multi-resolution texture synthesis are Markov-based approaches, while differentiable microstructure characterization and reconstruction (DMCR) is descriptor-based.The reconstructed structures are two times larger than the reference in each direction.More information is given in subsection 2.3.

Figure 7 :
Figure 7: Influence of the loss function on the descriptor errors.The volume fractions  (top), spatial correlations  (middle) and Gram matrices  (bottom) are compared for different materials (left to right) with 25 realizations per resolution.The resolutions are powers of two and an offset to the left (reference) and right (proposed) is applied only for visualization purposes.If can be seen that regardless of the model, material and descriptor, the variance of the descriptor error over different realizations decreases as the sample size increases.The proposed model consistently outperforms the reference[112].For some structures like the ally (a), the reference model fails to converge, leading to massive discrepancies, whereas for the ceramics (c) the differences are relatively small.

Figure 8 :
Figure 8: A reconstruction of the ceramics microstructure (Figure 2(c)) with 4096 2 pixels shows the scalability of the method.Three different zoom levels of the same structure are shown for visualization purposes.

Figure 9 :
Figure 9: Scalability comparison of descriptor-based NCA with the 2D DMCR algorithm [94] in MCRpy [102].Both scale linearly in the number of pixels, which scales quadratically in the side length.The constant offset of the NCA cost due to training is compensated by the much faster sampling, which also grows at a slower rate.

Table 1 :
Hyperparameters chosen in the present work.