Skip to main content
SpringerLink
Log in

Markov chain Monte Carlo with neural network surrogates: application to contaminant source identification

  • Original Paper
  • Published:
Stochastic Environmental Research and Risk Assessment Aims and scope Submit manuscript

Abstract

Subsurface remediation often involves reconstruction of contaminant release history from sparse observations of solute concentration. Markov Chain Monte Carlo (MCMC), the most accurate and general method for this task, is rarely used in practice because of its high computational cost associated with multiple solves of contaminant transport equations. We propose an adaptive MCMC method, in which a transport model is replaced with a fast and accurate surrogate model in the form of a deep convolutional neural network (CNN). The CNN-based surrogate is trained on a small number of the transport model runs based on the prior knowledge of the unknown release history. Thus reduced computational cost allows one to diminish the sampling error associated with construction of the approximate likelihood function. As all MCMC strategies for source identification, our method has an added advantage of quantifying predictive uncertainty and accounting for measurement errors. Our numerical experiments demonstrate the accuracy comparable to that of MCMC with the forward transport model, which is obtained at a fraction of the computational cost of the latter.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  • Amirabdollahian M, Datta B (2013) Identification of contaminant source characteristics and monitoring network design in groundwater aquifers: an overview. J Environ Prot 4:26–41

    Article  Google Scholar 

  • Ayvaz MT (2016) A hybrid simulation-optimization approach for solving the areal groundwater pollution source identification problems. J Hydrol 538:161–176

    Article  Google Scholar 

  • Bakker M, Post V, Langevin CD, Hughes JD, White J, Starn J, Fienen MN (2016) Scripting modflow model development using python and flopy. Groundwater 54(5):733–739

    Article  CAS  Google Scholar 

  • Barajas-Solano DA, Alexander FJ, Anghel M, Tartakovsky DM (2019) Efficient gHMC reconstruction of contaminant release history. Front Environ Sci 7:149. https://doi.org/10.3389/fenvs.2019.00149

    Article  Google Scholar 

  • Bedekar V, Morway ED, Langevin CD, Tonkin MJ (2016) MT3D-USGS version 1: A US Geological Survey release of MT3DMS updated with new and expanded transport capabilities for use with MODFLOW. Technical repprt, US Geological Survey, Reston, VA

  • Boso F, Tartakovsky DM (2020a) Data-informed method of distributions for hyperbolic conservation laws. SIAM J Sci Comput 42(1):A559–A583. https://doi.org/10.1137/19M1260773

    Article  Google Scholar 

  • Boso F, Tartakovsky DM (2020b) Learning on dynamic statistical manifolds. Proc R Soc A 476(2239):20200213. https://doi.org/10.1098/rspa.2020.0213

    Article  CAS  Google Scholar 

  • Bottou L (2010) Large-scale machine learning with stochastic gradient descent. In: Proceedings of COMPSTAT’2010. Springer, pp 177–186

  • Chaudhuri A, Hendricks-Franssen HJ, Sekhar M (2018) Iterative filter based estimation of fully 3D heterogeneous fields of permeability and Mualem-van Genuchten parameters. Adv Water Resour 122:340–354

    Article  Google Scholar 

  • Chen FC, Jahanshahi MR (2017) NB-CNN: Deep learning-based crack detection using convolutional neural network and Naïve Bayes data fusion. IEEE Trans Ind Electron 65(5):4392–4400

    Article  Google Scholar 

  • Ciriello V, Lauriola I, Tartakovsky DM (2019) Distribution-based global sensitivity analysis in hydrology. Water Resour Res 55(11):8708–8720. https://doi.org/10.1029/2019WR025844

    Article  Google Scholar 

  • Elsheikh AH, Hoteit I, Wheeler MF (2014) Efficient Bayesian inference of subsurface flow models using nested sampling and sparse polynomial chaos surrogates. Comput Methods Appl Mech Eng 269:515–537

    Article  Google Scholar 

  • Gamerman D, Lopes HF (2006) Markov chain Monte Carlo: stochastic simulation for Bayesian inference. Chapman and Hall/CRC, London

    Book  Google Scholar 

  • Geweke JF (1991) Evaluating the accuracy of sampling-based approaches to the calculation of posterior moments, vol 196. Federal Reserve Bank of Minneapolis, Research Department, Minneapolis, MN

    Google Scholar 

  • Goodfellow I, Bengio Y, Courville A (2016) Deep learning. MIT Press, Boston, MA. http://www.deeplearningbook.org

  • Green PJ, Mira A (2001) Delayed rejection in reversible jump Metropolis-Hastings. Biometrika 88(4):1035–1053

    Article  Google Scholar 

  • Haario H, Saksman E, Tamminen J (2001) An adaptive Metropolis algorithm. Bernoulli 7(2):223–242

    Article  Google Scholar 

  • Haario H, Laine M, Mira A, Saksman E (2006) DRAM: efficient adaptive MCMC. Stat Comput 16(4):339–354

    Article  Google Scholar 

  • Harbaugh AW (2005) MODFLOW-2005, the US Geological Survey modular ground-water model: the ground-water flow process. US Department of the Interior, US Geological Survey, Reston, VA

    Google Scholar 

  • Lagaris IE, Likas A, Fotiadis DI (1998) Artificial neural networks for solving ordinary and partial differential equations. IEEE Trans Neural Netw 9(5):987–1000

    Article  CAS  Google Scholar 

  • Lang A, Potthoff J (2011) Fast simulation of Gaussian random fields. Monte Carlo Methods Appl 17(3):195–214

    Article  Google Scholar 

  • Lee H, Kang IS (1990) Neural algorithm for solving differential equations. J Comput Phys 91(1):110–131

    Article  Google Scholar 

  • Leichombam S, Bhattacharjya RK (2018) New hybrid optimization methodology to identify pollution sources considering the source locations and source flux as unknown. J Hazards Toxic Radioact Waste 23(1):04018037

    Article  Google Scholar 

  • Liggett JE, Werner AD, Smerdon BD, Partington D, Simmons CT (2014) Fully integrated modeling of surface-subsurface solute transport and the effect of dispersion in tracer hydrograph separation. Water Resour Res 50(10):7750–7765

    Article  Google Scholar 

  • Liggett JE, Partington D, Frei S, Werner AD, Simmons CT, Fleckenstein JH (2015) An exploration of coupled surface-subsurface solute transport in a fully integrated catchment model. J Hydrol 529:969–979

    Article  CAS  Google Scholar 

  • Michalak AM, Kitanidis PK (2003) A method for enforcing parameter nonnegativity in Bayesian inverse problems with an application to contaminant source identification. Water Resour Res 39(2), 1033.

  • Miles P (2019) pymcmcstat: a Python package for Bayesian inference using delayed rejection adaptive Metropolis. J Open Source Soft 4(38):1417

    Article  Google Scholar 

  • Mo S, Zabaras N, Shi X, Wu J (2019a) Deep autoregressive neural networks for high-dimensional inverse problems in groundwater contaminant source identification. Water Resour Res 55(5):3856–3881

    Article  Google Scholar 

  • Mo S, Zhu Y, Zabaras NJ, Shi X, Wu J (2019b) Deep convolutional encoder-decoder networks for uncertainty quantification of dynamic multiphase flow in heterogeneous media. Water Resour Res 55(1):703–728

    Article  Google Scholar 

  • Neuman SP, Tartakovsky DM (2009) Perspective on theories of non-Fickian transport in heterogeneous media. Adv Water Resour 32(5):670–680. https://doi.org/10.1016/j.advwatres.2008.08.005

    Article  CAS  Google Scholar 

  • Paszke A, Gross S, Massa F, Lerer A, Bradbury J, Chanan G, Killeen T, Lin Z, Gimelshein N, Antiga L et al (2019) Pytorch: an imperative style, high-performance deep learning library. In: Advances in neural information processing systems, pp 8024–8035

  • Rajabi MM, Ataie-Ashtiani B, Simmons CT (2018) Model-data interaction in groundwater studies: Review of methods, applications and future directions. J Hydrol 567:457–477

    Article  Google Scholar 

  • Severino G, Tartakovsky DM, Srinivasan G, Viswanathan H (2012) Lagrangian models of reactive transport in heterogeneous porous media with uncertain properties. Proc R Soc A 468(2140):1154–1174. https://doi.org/10.1098/rspa.2011.0375

    Article  Google Scholar 

  • Sokal A (1997) Monte Carlo methods in statistical mechanics: foundations and new algorithms. In: Functional integration. Springer, pp 131–192

  • Srinivasan G, Tartakovsky DM, Dentz M, Viswanathan H, Berkowitz B, Robinson BA (2010) Random walk particle tracking simulations of non-Fickian transport in heterogeneous media. J Comput Phys 229(11):4304–4314. https://doi.org/10.1016/j.jcp.2010.02.014

    Article  CAS  Google Scholar 

  • White RE (2015) Nonlinear least squares algorithm for identification of hazards. Cogent Math 2(1):1118219

    Article  Google Scholar 

  • Xu T, Gómez-Hernández JJ (2016) Joint identification of contaminant source location, initial release time, and initial solute concentration in an aquifer via ensemble Kalman filtering. Water Resour Res 52(8):6587–6595

    Article  CAS  Google Scholar 

  • Xu T, Gómez-Hernández JJ (2018) Simultaneous identification of a contaminant source and hydraulic conductivity via the restart normal-score ensemble Kalman filter. Adv Water Resour 112:106–123

    Article  Google Scholar 

  • Ye Z, Gilman A, Peng Q, Levick K, Cosman P, Milstein L (2019) Comparison of neural network architectures for spectrum sensing. In: 2019 IEEE globecom workshops (GC Wkshps). IEEE, pp 1–6

  • Zhang J, Zeng L, Chen C, Chen D, Wu L (2015) Efficient Bayesian experimental design for contaminant source identification. Water Resour Res 51(1):576–598

    Article  Google Scholar 

  • Zhang J, Li W, Zeng L, Wu L (2016) An adaptive Gaussian process-based method for efficient Bayesian experimental design in groundwater contaminant source identification problems. Water Resour Res 52(8):5971–5984

    Article  Google Scholar 

  • Zhou H, Gómez-Hernández JJ, Li L (2014) Inverse methods in hydrogeology: evolution and recent trends. Adv Water Resour 63:22–37

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported in part by Air Force Office of Scientific Research under award FA9550-18-1-0474, National Science Foundation under award 1606192, and by a gift from TOTAL. There are no data sharing issues since all of the numerical information is provided in the figures produced by solving the equations in the paper. We used the code from Mo et al. (2019a, b) to construct and train the convolutional neural network.

Author information

Authors and Affiliations

  1. Department of Energy Resources Engineering, Stanford University, Stanford, CA, USA

    Zitong Zhou & Daniel M. Tartakovsky

Authors
  1. Zitong Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Daniel M. Tartakovsky
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Daniel M. Tartakovsky.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhou, Z., Tartakovsky, D.M. Markov chain Monte Carlo with neural network surrogates: application to contaminant source identification. Stoch Environ Res Risk Assess 35, 639–651 (2021). https://doi.org/10.1007/s00477-020-01888-9

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00477-020-01888-9

Keywords

Search

Navigation