Abstract
A new low-dimensional parameterization based on principal component analysis (PCA) and convolutional neural networks (CNN) is developed to represent complex geological models. The CNN–PCA method is inspired by recent developments in computer vision using deep learning. CNN–PCA can be viewed as a generalization of an existing optimization-based PCA (O-PCA) method. Both CNN–PCA and O-PCA entail post-processing a PCA model to better honor complex geological features. In CNN–PCA, rather than use a histogram-based regularization as in O-PCA, a new regularization involving a set of metrics for multipoint statistics is introduced. The metrics are based on summary statistics of the nonlinear filter responses of geological models to a pre-trained deep CNN. In addition, in the CNN–PCA formulation presented here, a convolutional neural network is trained as an explicit transform function that can post-process PCA models quickly. CNN–PCA is shown to provide both unconditional and conditional realizations that honor the geological features present in reference SGeMS geostatistical realizations for a binary channelized system. Flow statistics obtained through simulation of random CNN–PCA models closely match results for random SGeMS models for a demanding case in which O-PCA models lead to significant discrepancies. Results for history matching are also presented. In this assessment CNN–PCA is applied with derivative-free optimization, and a subspace randomized maximum likelihood method is used to provide multiple posterior models. Data assimilation and significant uncertainty reduction are achieved for existing wells, and physically reasonable predictions are also obtained for new wells. Finally, the CNN–PCA method is extended to a more complex nonstationary bimodal deltaic fan system, and is shown to provide high-quality realizations for this challenging example.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Change history
12 June 2019
In the original paper, the units noted on the <Emphasis Type="Italic">x</Emphasis>-axes in Fig. 17a and b are incorrect.
References
Arjovsky M, Chintala S, Bottou L (2017) Wasserstein generative adversarial networks. Paper presented at the 34th international conference on machine learning, Sydney, Australia, 6–11 August, pp 214–223
Astrakova A, Oliver DS (2015) Conditioning truncated pluri-Gaussian models to facies observations in ensemble-Kalman-based data assimilation. Math Geosci 47(47):345–367
Canchumuni SA, Emerick AA, Pacheco MA (2017) Integration of ensemble data assimilation and deep learning for history matching facies models. Paper OTC-28015-MS, presented at the OTC Brasil, Rio de Janeiro, Brazil, 24–26 October
Canchumuni SA, Emerick AA, Pacheco M (2018) History matching channelized facies models using ensemble smoother with a deep learning parameterization. Paper presented at the 15th European conference on the mathematics of oil recovery, Barcelona, Spain, 3–6 September
Chan S, Elsheikh AH (2017) Parametrization and generation of geological models with generative adversarial networks. arXiv preprint arXiv:1708.01810
Chan S, Elsheikh AH (2018) Parametric generation of conditional geological realizations using generative neural networks. arXiv preprint arXiv:1807.05207
Chang H, Zhang D, Lu Z (2010) History matching of facies distribution with the EnKF and level set parameterization. J Comput Phys 229(20):8011–8030
Chen C, Gao G, Ramirez BA, Vink JC, Girardi AM (2016) Assisted history matching of channelized models by use of pluri-principal-component analysis. SPE J 21(05):1793–1812
Deng J, Dong W, Socher R, Li LJ, Li K, Li FF (2009) ImageNet: a large-scale hierarchical image database. In: 2009 IEEE conference on computer vision and pattern recognition, IEEE, pp 248–255
Dimitrakopoulos R, Mustapha H, Gloaguen E (2010) High-order statistics of spatial random fields: exploring spatial cumulants for modeling complex non-Gaussian and non-linear phenomena. Math Geosci 42(1):65–99
Dupont E, Zhang T, Tilke P, Liang L, Bailey W (2018) Generating realistic geology conditioned on physical measurements with generative adversarial networks. arXiv preprint arXiv:1802.03065
Echeverría Ciaurri D, Mukerji T, Santos ET (2009) Robust scheme for inversion of seismic and production data for reservoir facies modeling. Paper SEG-2009-2432, presented at the SEG annual meeting, Houston, Texas, 25–30 October
Emerick AA (2016) Investigation on principal component analysis parameterizations for history matching channelized facies models with ensemble-based data assimilation. Math Geosci 49(1):85–120
Gatys LA, Ecker AS, Bethge M (2015) Texture synthesis using convolutional neural networks. In: Advances in neural information processing systems, pp 262–270
Gatys LA, Ecker AS, Bethge M (2016) Image style transfer using convolutional neural networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2414–2423
Goodfellow I, Bengio Y, Courville A (2016) Deep learning. MIT Press, Cambridge
Hakim-Elahi S, Jafarpour B (2017) A distance transform for continuous parameterization of discrete geologic facies for subsurface flow model calibration. Water Resour Res 53(10):8226–8249
Insuasty E, Van den Hof PMJ, Weiland S, Jansen JD (2017) Low-dimensional tensor representations for the estimation of petrophysical reservoir parameters. Paper SPE-182707-MS presented at the SPE Reservoir Simulation Conference, Montgomery, Texas, 20–22 February
Isebor OJ, Echeverría Ciaurri D, Durlofsky LJ (2014) Generalized field-development optimization with derivative-free procedures. SPE J 19(05):891–908
Jafarpour B, Goyal VK, McLaughlin DB, Freeman WT (2010) Compressed history matching: exploiting transform-domain sparsity for regularization of nonlinear dynamic data integration problems. Math Geosci 42(1):1–27
Johnson J, Alahi A, Li FF (2016) Perceptual losses for real-time style transfer and super-resolution. In: European conference on computer vision, pp 694–711
Kadian A (2018) Pytorch implementation of an algorithm for artistic style transfer. https://github.com/abhiskk/fast-neural-style/commits/master. Accessed 1 May 2018
Kingma DP, Ba J (2014) Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980. Accessed 1 May 2018
Kitanidis PK (1986) Parameter uncertainty in estimation of spatial functions: Bayesian analysis. Water Resour Res 22(4):499–507
Laloy E, Hérault R, Lee J, Jacques D, Linde N (2017) Inversion using a new low-dimensional representation of complex binary geological media based on a deep neural network. Adv Water Resour 110:387–405
Laloy E, Hérault R, Jacques D, Linde N (2018) Training-image based geostatistical inversion using a spatial generative adversarial neural network. Water Resour Res 54(1):381–406
Liu Y (2017) Multilevel strategy for O-PCA-based history matching using mesh adaptive direct search. Master’s thesis, Stanford University
Lu P, Horne RN (2000) A multiresolution approach to reservoir parameter estimation using wavelet analysis. Paper SPE-62985-MS, presented at the SPE annual technical conference and exhibition, Dallas, TX, 1–4 October
Mosser L, Dubrule O, Blunt MJ (2017) Reconstruction of three-dimensional porous media using generative adversarial neural networks. Phys Rev E 96(4):043,309
Mosser L, Dubrule O, Blunt MJ (2018) Conditioning of three-dimensional generative adversarial networks for pore and reservoir-scale models. arXiv preprint arXiv:1802.05622
Oliver DS (1996) Multiple realizations of the permeability field from well test data. SPE J 1(2):145–154
Pardo-Igúzquiza E, Dowd PA (2003) CONNEC3D: a computer program for connectivity analysis of 3D random set models. Comput Geosci 29(6):775–785
Paszke A, Gross S, Chintala S, Chanan G, Yang E (2017) Automatic differentiation in PyTorch. NIPS 2017 workshop
Ping J, Zhang D (2013) History matching of fracture distributions by ensemble Kalman filter combined with vector based level set parameterization. J Pet Sci Eng 108:288–303
Remy N, Boucher A, Wu J (2009) Applied geostatistics with SGeMS: a user’s guide. Cambridge University Press, Cambridge
Reynolds AC, He N, Chu L, Oliver DS (1996) Reparameterization techniques for generating reservoir descriptions conditioned to variograms and well-test pressure data. SPE J 1(4):413–426
Rwechungura R, Dadashpour M, Kleppe J (2011) Application of particle swarm optimization for parameter estimation integrating production and time lapse seismic data. Paper SPE-146199-MS, presented at the SPE offshore Europe conference and exhibition, Aberdeen, UK, 6–8 September
Sarma P, Durlofsky LJ, Aziz K, Chen WH (2006) Efficient real-time reservoir management using adjoint-based optimal control and model updating. Comput Geosci 10(1):3–36
Sarma P, Durlofsky LJ, Aziz K (2008) Kernel principal component analysis for efficient, differentiable parameterization of multipoint geostatistics. Math Geosci 40(1):3–32
Simonyan K, Zisserman A (2015) Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556
Strebelle S (2002) Conditional simulation of complex geological structures using multiple-point statistics. Math Geol 34(1):1–21
Tavakoli R, Reynolds AC (2010) History matching with parametrization based on the SVD of a dimensionless sensitivity matrix. SPE J 15(02):495–508
Torquato S, Beasley JD, Chiew YC (1988) Two-point cluster function for continuum percolation. J Chem Phys 88(10):6540–6547
Tran D, Bourdev L, Fergus R, Torresani L, Paluri M (2015) Learning spatiotemporal features with 3D convolutional networks. Paper presented at the 2015 IEEE international conference on computer vision, Santiago, Chile, 13–16 December, pp 4489–4497
Vo HX, Durlofsky LJ (2014) A new differentiable parameterization based on principal component analysis for the low-dimensional representation of complex geological models. Math Geosci 46(7):775–813
Vo HX, Durlofsky LJ (2015) Data assimilation and uncertainty assessment for complex geological models using a new PCA-based parameterization. Comput Geosci 19(4):747–767
Vo HX, Durlofsky LJ (2016) Regularized kernel PCA for the efficient parameterization of complex geological models. J Comput Phys 322:859–881
Wang Y, Arns CH, Rahman SS, Arns JY (2018) Porous structure reconstruction using convolutional neural networks. Math Geosci 50(7):781–799
Zhang T, Switzer P, Journel A (2006) Filter-based classification of training image patterns for spatial simulation. Math Geol 38(1):63–80
Acknowledgements
We thank the industrial affiliates of the Stanford Smart Fields Consortium for financial support. We are grateful to Hai Vo for providing O-PCA code and geological models and for useful suggestions, and to Hang Zhang and Abhishek Kadian for their open-source PyTorch implementation of the neural style transfer and fast neural style transfer algorithms. We also acknowledge the teaching staff for the Stanford CS231N course for offering guidance and providing computing resources on Google Cloud.
Author information
Authors and Affiliations
Corresponding author
Appendix: Model Transform Net Architecture
Appendix: Model Transform Net Architecture
The architecture of the model transform net is summarized in Table 1. It is the same as that in Johnson et al. (2016) except for the first and last layers. In the table, ‘Conv’ denotes a convolutional layer immediately followed by instance normalization and a ReLU nonlinear activation. The last ‘Conv’ layer is an exception, as it only contains a convolutional layer. ‘Residual block’ contains a stack of two convolutional layers, each with 128 filters of size \(3 \times 3 \times 128\) and stride 1. Within each residual block, the first convolutional layer is followed by an instance normalization and a ReLU nonlinear activation. The second convolutional layer is followed only by an instance normalization. The final output of the residual block is the sum of the input to the first convolutional layer and the output from the second convolutional layer.
Rights and permissions
About this article
Cite this article
Liu, Y., Sun, W. & Durlofsky, L.J. A Deep-Learning-Based Geological Parameterization for History Matching Complex Models. Math Geosci 51, 725–766 (2019). https://doi.org/10.1007/s11004-019-09794-9
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11004-019-09794-9