Gaussian Process Regression Reviewed in the Context of Inverse Theory

Menke, William; Creel, Roger

doi:10.1007/s10712-021-09640-w

Published: 11 April 2021

Gaussian Process Regression Reviewed in the Context of Inverse Theory

Surveys in Geophysics volume 42, pages 473–503 (2021)Cite this article

462 Accesses
2 Citations
Metrics details

Abstract

We review Gaussian process regression (GPR) and analyze it in the context of Inverse Theory—the collection of techniques used in geophysics (among other fields) to understand the structure of data analysis problems and the quality of their solutions. By viewing GPR as a special case of generalized least squares (least squares with prior information), we derive expressions for a variety of standard Inverse Theory quantities, including the data and model resolution matrices, the importance (influence) vector, and the gradient of the solution with respect to a parameter. We study the impulse response in the one-dimensional continuum limit and provide formulas for its area and width. Finally, we demonstrate how the importance vector can be used to design an optimum GPR experiment, through a process we call importance winnowing.

This is a preview of subscription content, access via your institution.

Access options

Buy single article

Instant access to the full article PDF.

USD 39.95

Price includes VAT (USA)
Tax calculation will be finalised during checkout.

Data Availability

This paper contains no data.

References

Abers G (1994) Three-dimensional inversion of regional P and S arrival times in the East Aleutians and sources of subduction zone gravity highs. J Geophys Res 99:4395–4412. https://doi.org/10.1029/93JB03107
Article Google Scholar
Asante-Okyere S, Shen C, Yevenyo YY, Ziggah R, Zhu X (2018) Investigating the predictive performance of Gaussian process regression in evaluating reservoir porosity and permeability. Energies 11:3261. https://doi.org/10.3390/en11123261
Article Google Scholar
Aster RC, Borchers B, Thurber CH (2018) Parameter estimation and inverse problems, 3rd edn. Elsevier, Waltham, p 404
Google Scholar
Backus G, Gilbert F (1970) Uniqueness in the inversion of inaccurate gross earth data. Philos Trans R Soc Lond Ser A 266:123–192. https://doi.org/10.1098/rsta.1970.0005
Article Google Scholar
Backus G, Gilbert F (1968) The resolving power of Gross Earth Data. Geophys J Roy Astron Soc 16:169–205. https://doi.org/10.1190/1.1444834
Article Google Scholar
Bartels RH, Stewart GW (1972) Solution of the matrix equation AX + XB = C [F4]. Commun ACM 15:820–826. https://doi.org/10.1145/361573.361582
Article Google Scholar
Belsey DA, Kuh E, Welch RE (2004) Data diagnostics: identifying influential data and souces or collinearity. Wiley, New Jersey
Google Scholar
Bracewell RN (2000) The fourier transform and its applications, 3rd edn. McGraw-Hill, Boston, p 616
Google Scholar
Candes EJ, Wakin MB, Boyd SP (2008) Enhancing sparsity by reweighted ℓ1 minimization. J Fourier Anal Appl 14:877–905. https://doi.org/10.1007/s00041-008-9045-x
Article Google Scholar
Chen P, Jordan TH, Lee E-J (2010) Perturbation kernels for generalized seismological data functionals (GSDF). Geophys J Int 183:869–883. https://doi.org/10.1111/j.1365-246X.2010.04758.x]
Article Google Scholar
Gentile R, Galasso C (2020) Gaussian process regression for seismic fragility assessment of building portfolios. Struct Saf 87:101980. https://doi.org/10.1016/j.strusafe.2020.101980
Article Google Scholar
Grubbs FGE (1969) Procedures for detecting outlying observations in samples. Technometrics 11(1):1–21. https://doi.org/10.1080/00401706.1969.10490657
Article Google Scholar
De Gruttola V, Ware J, Louis T (1987) Influence analysis of generalized least squares estimators. J Am Stat Assoc 82:911–917. https://doi.org/10.2307/2288804
Article Google Scholar
Hanse TM, Journel AG, Tarantola A, Mosegaard K (2006) Linear inverse Gaussian theory and geostatistics. Geophysics. https://doi.org/10.1190/1.2345195
Article Google Scholar
Hansen PC (1992) Analysis of discrete Ill-posed problems by means of the L-curve. SIAM Rev 34:561–580. https://doi.org/10.1137/1034115
Article Google Scholar
Hines TT, Hetland EA (2018) Revealing transient strain in geodetic data with Gaussian process regression. Geophys J Int 212:2116–2130. https://doi.org/10.1093/gji/ggx525
Article Google Scholar
Kopp RE, Horton RM, Little CM, Mitrovica JX, Oppenheimer M, Rasmussen DJ, Strauss BH, Tebaldi C (2014) Probabilistic 21st and 22nd century sea-level projections at a global network of tide-gauge sites. Earth’s future 2(8):383–406. https://doi.org/10.1002/2014EF000239
Article Google Scholar
Krige DG (1951) A statistical approach to some basic mine valuation problems on the Witwatersrand. J Chem, Metall Min Soc 52:119–139
Google Scholar
Lawson C, Hanson R (1974) Solving least squares problems. Prentice-Hall, Prentice, p 337
Google Scholar
Levenberg K (1944) A method for the solution of certain non-linear problems in least-squares. Q Appl Math 2:164–168
Article Google Scholar
Li RC (2014) Matrix perturbation theory, in Hogben, Handbook of linear algebra, 2nd edn. CRC Press, London, p 1402
Google Scholar
Malinverno A, Briggs VA (2004) Expanded uncertainty quantification in inverse problems: Hierarchical Bayes and empirical Bayes. Geophysics 69:877–1103. https://doi.org/10.1190/1.1778243
Article Google Scholar
Menke W (1984) Geophysical data analysis: discrete inverse theory. Academic Press, New York
Google Scholar
Menke W (1991) Application of the POCS inversion method to interpolating topography and other geophysical fields. Geophys Res Lett 18:435–438. https://doi.org/10.1029/90GL00343
Article Google Scholar
Menke W (2014) Review of the generalized least squares method. Surv Geophys 36:1–25. https://doi.org/10.1007/s10712-014-9303-1
Article Google Scholar
Menke W (2018) Geophysical data analysis: discrete inverse theory, 4th edn. Academic Press, Elsevier, p 350
Google Scholar
Menke W, Eilon Z (2015) Relationship between data smoothing and the regularization of inverse problems. Pure Appl Geophys 172:2711–2726. https://doi.org/10.1007/s00024-015-1059-0
Article Google Scholar
Menke W, Menke J (2016) Environmental Data Analysis with MATLAB, 2nd edn. Academic Press, Elsevier, p 342
Google Scholar
Menke W (2021) Tuning of Prior Covariance in Generalized Least Squares. Appl Math 12:157–170. https://doi.org/10.4236/am.2021.123011
Neal, RM (1994) Priors of infinite networks. Technical Report CRG-TR-94-1, Department of Computer Science, University of Toronto (Toronto, Canada).
Nooria M, Hassani H, Javaheriana A, Amindavar H, Torabi S (2019) Automatic fault detection in seismic data using Gaussian process regression. J Appl Geophys 163:117–131. https://doi.org/10.1016/j.jappgeo.2019.02.018
Article Google Scholar
Parker RL (1994) Geophysical inverse theory. Princeton University Press, Princeton, p 386
Book Google Scholar
Petersen KB, Pedersen MS (2008) The Matrix Cookbook, 71p, https://thematrixcookbook.com.
Piecuch CG, Huybers P, Tingley MP (2017) Comparison of full and empirical bayes approaches for inferring sea-level changes from tide-gauge data. J Geophys Res 122:2243–2258. https://doi.org/10.1002/2016JC012506
Article Google Scholar
Press WH, Teukolsky SA, Vetterling WT, Flannery BP (2007) Numerical recipes 3rd edition: the art of scientific computing. Cambridge University Press, New York, p 1235
Google Scholar
Rasmussen CE, Williams CKI (2006) Gaussian processes for machine learning. MIT Press, Cambridge, p 272
Google Scholar
Ray A, Myer D (2019) Bayesian geophysical inversion with trans-dimensional Gaussian process machine learning. Geophys J Int 217:1706–1726. https://doi.org/10.1093/gji/ggz111
Article Google Scholar
Reid A., O'Callaghan ST, Bonilla EV, McCalman L, Rawling T, Ramos FT (2013) Bayesian joint inversions for the exploration of earth resources. In: Proceedings of the Twenty-Third International joint conference on artificial intelligence IJCAI '13, pp. 2877–2884, www.ijcai.org/Proceedings/2013.
Richardson RM, MacInnes SC (1989) The inversion of gravity data into three-dimensional polyhedral models. J Geophys Res 94:7555–7562. https://doi.org/10.1029/JB094iB06p07555
Article Google Scholar
Shure L, Parker RL, Backus GE (1982) Harmonic splines for geomagnetic modelling. Phys Earth Planet Inter 28:215–222. https://doi.org/10.1016/0031-9201(82)90003-6
Article Google Scholar
Smith WHF, Wessel P (1990) Gridding with continuous curvature splines in tension. Geophysics. https://doi.org/10.1190/11442837
Article Google Scholar
Snyman JA, Wilke DN (2018) Practical mathematical optimization - basic optimization theory and gradient-based algorithms. springer optimization and its applications, 2nd edn. Springer, New York
Google Scholar
Talagrand O, Courtier P (1987) Variational assimilation of meteorological observations with the adjoint vorticity equation. I: theory. Quart J Royal Meteorol Soc 113:1311–1328. https://doi.org/10.1002/qj.49711347812
Article Google Scholar
Tarantola A, Valette B (1982a) Generalized non-linear inverse problems solved using the least squares criterion. Rev Geophys Space Phys 20:219–232
Article Google Scholar
Tarantola A, Valette B (1982b) Inverse problems = quest for information. J Geophys 50:159–170
Google Scholar
Tarantola A (2005) Inverse problem theory and methods for model parameter estimation, First Edition, SIAM: Society for Industrial and Applied Mathematics, p. 342, ISBN 13: 9780898715729.
Tikhonov AN (1943) On the stability of inverse problems. Dokl Akad Nauk SSSR 39:195–198
Google Scholar
Trainor-Guitton WJ, Mukerji T, Knight R (2013) A methodology for quantifying the value of spatial information for dynamic Earth problems. Stoch Env Res Risk Assess 27:969–983. https://doi.org/10.1007/s00477-012-0619-4
Article Google Scholar
Tromp J, Tape C, Liu Q (2005) Seismic tomography, adjoint methods, time reversal and banana-doughnut kernels. Geophys J Int 160:195–216. https://doi.org/10.1111/j.1365-246X.2004.02453.x
Article Google Scholar
Webster R, Oliver MA (2007) Geostatistics for environmental scientists, 2nd edn. Hoboken, Wiley
Book Google Scholar
Wiggins RA (1972) The general linear inverse problem: Implications of surface waves and free oscillations for Earth structure. Rev Geophys Space Phys 10:251–285. https://doi.org/10.1029/RG010i001p00251
Article Google Scholar

Download references

Acknowledgements

This research was partially supported by the National Science Foundation under Grant EAR 20-02352.

Author information

Authors and Affiliations

Lamont-Doherty Earth Observatory of Columbia University, Palisades, NY, 10964, USA
William Menke & Roger Creel

Authors

William Menke
View author publications
You can also search for this author in PubMed Google Scholar
Roger Creel
View author publications
You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to William Menke.

Ethics declarations

Conflict of interest

The authors declare no conflicts of interest with respect to this work.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Menke, W., Creel, R. Gaussian Process Regression Reviewed in the Context of Inverse Theory. Surv Geophys 42, 473–503 (2021). https://doi.org/10.1007/s10712-021-09640-w

Download citation

Received: 29 November 2020
Accepted: 04 March 2021
Published: 11 April 2021
Issue Date: May 2021
DOI: https://doi.org/10.1007/s10712-021-09640-w

Keywords

Gaussian process regression
Geostatistics
Importance
Influence
Interpolation
Inverse Theory
Least Squares
Resolution