Conditioning generative adversarial networks on nonlinear data for subsurface flow model calibration and uncertainty quantification

Abstract

Conditioning complex subsurface flow models on nonlinear data is complicated by the need to preserve the expected geological connectivity patterns to maintain solution plausibility. Generative adversarial networks (GANs) have recently been proposed as a promising approach for low-dimensional representation of complex high-dimensional images. The method has also been adopted for low-rank parameterization of complex geologic models to facilitate uncertainty quantification workflows. A difficulty in adopting these methods for subsurface flow modeling is the complexity associated with nonlinear flow data conditioning. While conditional GAN (CGAN) can condition simulated images on labels, application to subsurface problems requires efficient conditioning workflows for nonlinear data, which is far more complex. We present two approaches for generating flow-conditioned models with complex spatial patterns using GAN. The first method is through conditional GAN, whereby a production response label is used as an auxiliary input during the training stage of GAN. The production label is derived from clustering of the flow responses of the prior model realizations (i.e., training data). The underlying assumption of this approach is that GAN can learn the association between the spatial features corresponding to the production responses within each cluster. An alternative method is to use a subset of samples from the training data that are within a certain distance from the observed flow responses and use them as training data within GAN to generate new model realizations. In this case, GAN is not required to learn the nonlinear relation between production responses and spatial patterns. Instead, it is tasked to learn the patterns in the selected realizations that provide a close match to the observed data. The conditional low-dimensional parameterization for complex geologic models with diverse spatial features (i.e., when multiple geologic scenarios are plausible) performed by GAN allows for exploring the spatial variability in the conditional realizations, which can be critical for decision-making. We present and discuss the important properties of GAN for data conditioning using several examples with increasing complexity.

Appendix A: Network architecture and training process

We provide a complete description of the architecture used in our study. The networks are implemented with the open-source machine learning framework Tensorflow (version 1.12). For this particular example in the appendix, each label is defined as one of the five geologic scenarios, and is assigned according to the TI used to generate the 32 × 32 realizations. CGAN (Method 1) is tasked to parameterize 500 model realizations within each geologic scenario and can be used to generate realizations from the respective geologic scenario when provided with a latent vector z from a Gaussian distribution and a geologic scenario label c (as one-hot vector encoding). Figure 18 shows the dimensions of input, output and weights (parameters) associated with each layer. A layer refers to a sequence of dense/convolution/deconvolution operation, followed by an optional batch normalization operation and finally a nonlinear operation. Note that the batch normalization operation and the nonlinear operation do not change the dimension of the input.

Fig. 18

Schematic of the architecture used in this study. Refer to the shorthand notations in Table 3 for a description of the functions used within each layer

Table 3 lists the actual Tensorflow functions and hyperparameters used within each layer. The shorthand notation for each Tensorflow function in Table 3 is consistent with the shorthand notations used in Fig. 18. For all the examples used in this paper, the weight of the gradient penalty term, λ (in Eqs. 2, and 3) is set as 10. The three loss functions (3)-(5) for training CGAN are tuned using tf.train.AdamOptimizer(α = 5 × 10^− 4, β₁ = 0.5, β₂ = 0.9) with a batch size (denoted as N_b) of 32. For the second method, the classifier $\mathcal {C}_{\phi }$ loss function (${\mathscr{L}}_{\mathcal {C}}$) is simply omitted from the training process and the input to the generator $\mathcal {G}_{\theta }$ only includes the latent vector.

Figure 18 also illustrates the flow of tensors when the components ($\mathcal {G}_{\theta }, \mathcal {D}_{\psi }, \mathcal {C}_{\phi }$) in CGAN are optimized in an alternating manner. When $\mathcal {D}_{\psi }$ and $\mathcal {C}_{\phi }$ are updated, the weights in $\mathcal {G}_{\theta }$ are fixed and the flow of tensors is represented by the red bold path for generated (fake) realizations and the red stippled path for training (real) realizations. In this update step, gradient information is backpropagated to $\mathcal {D}_{\psi }$ (calculated using ${\mathscr{L}}_{\mathcal {D}}$ via the red bold and stippled paths) to train $\mathcal {D}_{\psi }$ how to distinguish between fake and real realizations. Additionally, gradient information is backpropagated to $\mathcal {C}_{\phi }$ and $\mathcal {D}_{\psi }$ (calculated using ${\mathscr{L}}_{\mathcal {C}}$ via the red stippled path) to train $\mathcal {C}_{\phi }$ and $\mathcal {D}_{\psi }$ to learn the geologic features for each class label. When $\mathcal {G}_{\theta }$ is updated, the weights in $\mathcal {D}_{\psi }$ and $\mathcal {C}_{\phi }$ are fixed and the flow of tensors is represented by the green bold path for generated (fake) realizations. In this update step, gradient information is backpropagated to $\mathcal {G}_{\theta }$ (calculated using ${\mathscr{L}}_{\mathcal {C}}$ and ${\mathscr{L}}_{\mathcal {D}}$ via the green bold path) to teach the generator how to reproduce geologic features associated with each class label.

Figure 19(a) shows total losses of the components ($\mathcal {G}_{\theta }, \mathcal {D}_{\psi }, \mathcal {C}_{\phi }$) in CGAN when trained with model realizations labeled by the geologic scenario. The network is trained for 8000 iterations, where $\mathcal {D}_{\psi }$ is updated 5 times for every 1 iteration as recommended by [26]. A single iteration refers to loss computation on a batch - in this case since there are 2500 realizations in total, 78 iterations are needed to process the entire dataset once. Figure 19b shows samples of realizations generated by geologic scenario at selected iterations. To monitor the convergence, the input latent vector for each generated sample in Fig. 19(b) is fixed for each iteration. It is observed that $\mathcal {C}_{\phi }$ converges rather easily where generated realizations at iteration 1000 are already exhibiting the correct features (azimuth and channel geometry) for each geologic scenario. $\mathcal {G}_{\theta }$ generates continuous realizations (as shown, with satisfactory quality at iteration 8000) and can be discretized by taking a mid-point threshold in the case of binary facies. In this case, using thresholding method with a mid-point cutoff value of 0.5 (i.e., the mid-point value between 0 and 1) for the generated realizations, any pixel with value of less than 0.5 is assigned a discrete value of 0 and any pixel with value of more or equal to 0.5 is assigned a discrete value of 1. Beyond iteration 8000, the generated realizations remain consistent and only the continuous-valued pixels show minor variations in terms of location and values. The same behavior is observed in Method 2 when the network is trained without $\mathcal {C}_{\phi }$.

Fig. 19

a Total losses (normalized) of $\mathcal {C}_{\phi }$, $\mathcal {D}_{\psi }$ and $\mathcal {G}_{\theta }$ of CGAN using dataset A with geologic scenario as the label. b 5 samples of realizations per geologic scenario (row), generated by $\mathcal {G}_{\theta }$ at selected iterations