3D super-resolution reconstruction of porous media based on GANs and CBAMs

Abstract

Porous media reconstruction is quite significant in the fields of environmental supervision, oil and natural gas engineering, biomedicine and material engineering. The traditional numerical reconstruction methods such as multi-point statistics (MPS) are based on the statistical characteristics of training images (TIs), but the reconstruction quality may be unsatisfactory and the process is time-consuming. Recently, with the rapid development of deep learning, its powerful ability in predicting features has been used to reconstruct porous media. Generative adversarial network (GAN) is one of the generative methods of deep learning, which is derived from the two-person zero-sum game through the confrontation between the generator and discriminator. However, the traditional GAN cannot pay special attention to the effective features in learning, and the degradation problem easily occurs with the increase of layer numbers. Besides, high-resolution (HR) and large FOV (field of view) are usually contradictory for physical imaging equipment. Therefore, in practical experiments, due to the limitations of the resolution of imaging equipment and the sample size, it is difficult to obtain large-scale HR images of porous media physically. At this point, numerical super-resolution (SR) reconstruction seems to be a cost-efficient way. In this paper, residual networks and attention mechanisms are combined with single-image GAN (SinGAN), which can learn the structural characteristics of porous media from a low-resolution (LR) 3D image, and then reconstruct 3D HR or large-scale images of porous media. Comparison to some other numerical methods has proved that our method can reconstruct high-quality HR images with practicability and effectiveness.

Appendices

Appendix A: The derivation of identity mapping in ResNets

The data flow in ResNets is designed as follows: the input x subsequently passes through the first weight layer for the first convolution computation, the first activation function Relu, the second weight layer for the second convolution computation, the second activation function Relu, and so on, until the final residual term F(x) is obtained. At the end of the residual learning unit, the input x is directly transferred to the output by a “shortcut” connection, and then the result H(x) is obtained. Through the introduction of shortcut connections, the degradation problem with the deepening of network is alleviated largely.

As mentioned previously, ResNets only need to learn F(x) = 0 to realize the identity mapping. Compared to learning H(x) = x, setting F(x) = 0 will converge much faster when updating the parameters of each layer in the learning process. The idea of constructing an identity map is introduced in this appendix.

Assume that the input and output dimensions of the nonlinear unit in the neural network are consistent, the function H(∙) to be fitted in the neural network unit can be divided into two parts, namely:

$$H\left( {a^{(l - 1)} } \right) \, = a^{(l - 1)} + F\left( {a^{(l - 1)} } \right),$$

(A.1)

where l is the layer number and a is the input; F(∙) is the residual function; a^(l−1) is the input of the (l-1)-th layer; F(a^(l−1)) is the residual term of the (l-1)-th layer; H(a^(l−1)) is the output of the (l-1)-th layer.

In ResNets, learning an identity mapping is equivalent to making the residual part close to 0, namely F(a^(l−1)) → 0. If F(x) = 0 or H(x) = x, then an identity mapping is constructed. Generally, the output of one layer in network is the input of its next layer. Consider any two layers l₂ > l₁, and recursively expand Eq. (A.1). Then the input of the l₂-th layer ${a}^{({l}_{2})}$ can be defined as follows:

$${a}^{({l}_{2})}=H\left({a}^{{l}_{2}-1}\right)={a}^{\left({l}_{2}-1\right)}+F\left({a}^{\left({l}_{2}-1\right)}\right),$$

(A.2)

$${a}^{({l}_{2})}=\left({a}^{\left({l}_{2}-2\right)}+F\left({a}^{\left({l}_{2}-2\right)}\right)\right)+F\left({a}^{\left({l}_{2}-1\right)}\right),$$

(A.3)

Finally, ${a}^{({l}_{2})}$ can be formulated as follows:

$${a}^{({l}_{2})}={a}^{({l}_{1})}+{\sum }_{i={l}_{1}}^{{l}_{2}-1}F({a}^{(i)}).$$

(A.4)

It seems that in Eq. (A.4) the input of the l₂-th layer ${a}^{({l}_{2})}$ is equal to the sum of the input of the l₁-th layer ${a}^{({l}_{1})}$ and the sum of residual terms from the l₁-th to the (l₂-1)-th layer. According to Eq. (A.4), in forward propagation, the input data can propagate directly from a lower level to a higher level. Based on the chain derivation rule, the gradient of the middle output layer will be used in the back-propagation of the gradient, that is, the gradient of the intermediate output layer will be used as the gradient of the weight. In this way, the gradient of the final loss δ to the output of a lower layer can be expressed as:

$$\frac{\partial \delta }{{\partial a^{{\left( {l_{1} } \right)}} }} = \frac{\partial \delta }{{\partial a^{{\left( {l_{2} } \right)}} }} \cdot \frac{{\partial a^{{\left( {l_{2} } \right)}} }}{{\partial a^{{\left( {l_{1} } \right)}} }}.$$

(A.5)

Then Eq. (A.4) is substituted into Eq. (A.5) to obtain Eq. (A.6):

$$\frac{\partial \delta }{{\partial a^{{\left( {l_{1} } \right)}} }} = \frac{\partial \delta }{{\partial a^{{\left( {l_{2} } \right)}} }}\underbrace {{\left( {1 + \frac{\partial }{{\partial a^{{\left( {l_{1} } \right)}} }}\mathop \sum \limits_{{i = l_{1} }}^{{l_{2} - 1}} F\left( {a^{\left( i \right)} } \right)} \right)}}_{{{\text{SG}}}},$$

(A.6)

where SG (sum of gradients plus one) stands for $\frac{\partial {a}^{({l}_{2})}}{\partial {a}^{({l}_{1})}}$. Finally, Eq. (A.6) is formulated as follows:

$$\frac{\partial \delta }{\partial {a}^{({l}_{1})}}=\frac{\partial \delta }{\partial {a}^{({l}_{2})}}+\frac{\partial \delta }{\partial {a}^{({l}_{2})}}\cdot \frac{\partial }{\partial {a}^{\left({l}_{1}\right)}}{\sum }_{i={l}_{1}}^{{l}_{2}-1}F\left({a}^{\left(i\right)}\right).$$

(A.7)

There may be several layers between the l₂-th layer and the l₁-th layer. In CNN, the gradient update is calculated according to the chain derivation rule, meaning from one layer to its next adjacent layer. As shown in Eq. (A.5), $\frac{\partial \delta }{\partial {a}^{({l}_{1})}}$ is expressed by the multiplication of two partial derivatives. Therefore, when the derivatives between adjacent layers are much larger or less than 1, $\frac{\partial \delta }{\partial {a}^{({l}_{1})}}$ will increase or decrease exponentially if there are many layers in CNN, leading to the gradient explosion or gradient disappearance. According to Eq. (A.6), SG is equal to the sum of some gradients plus one. The first term in SG is the constant (i.e. 1), and the second term is the gradient sum of residual terms (i.e. $\frac{\partial }{\partial {a}^{({l}_{1})}}{\sum }_{i={l}_{1}}^{{l}_{2}-1}F({a}^{(i)})$). It can be known that the second term cannot be all -1. In addition, even if the second term is very small, since there is a constant 1 in the first term, the overall gradient cannot disappear, i.e. SG cannot always be 0. From Eq. (A.5) to Eq. (A.7), it can be seen that the gradient is calculated by addition rather than multiplication, meaning that the gradient will increase relatively slow, thus avoiding gradient explosion. In CNN, the gradient is calculated according to Eq. (A.5), while Eq. (A.6) is used to calculate the gradient in ResNets through residual terms. To sum up, it is considered that residual connections make information move forward and backward more smoothly by addressing the degradation problem.

Appendix B: Pseudo codes of MPC calculation

As shown in the above pseudo codes, “j” is a variable representing the current number of pixels/voxels in one direction. The function indicator_function(u + j * h) returns 1 or 0: when the state value located at a specific position (i.e., u + j*h) is pore space, indicator_function(…) = 1; otherwise, indicator_function(…) = 0. The function calculate_expected_value(…) calculates the expected value of the input.