Skip to main content

Lévy Gradient Descent: Augmented Random Search for Geophysical Inverse Problems

Abstract

A new efficient random search algorithm is introduced for solving inversion problems in geophysical studies. The proposed algorithm is inherently a stochastic optimization method which is built on the concept of gradient descending and Lévy flights. Therefore, the algorithm is referred to as the Lévy gradient descent (L-GD). In which, the Lévy flights is a special class of random walk which consists of many short steps along with a few large steps. Such movements are observed in a varying range of fields, including animals’ foraging patterns, fluid dynamics, transport of light and so on. Meanwhile, the Lévy flights typically shows much higher speed in searching for sparsely located targets compared to the well-known Brownian walks, which make them preferable to drive random search algorithms. As shown in the paper, besides optimal solutions of the inverse problems, the L-GD algorithm could also produce estimations on the error distributions of the resultant model parameters. Following a detailed introduction of the methodology, parameter settings of the algorithm are discussed in length through statistical experiments. Subsequently, the proposed algorithm is evaluated using numeric tests and shows attracting properties of the global convergence and significant higher searching efficiency compared to commonly adopted stochastic optimization techniques in geophysical inversions. Moreover, the L-GD algorithm is applied to the inversions of gravity and seismic travel time data and has achieved the same accuracy as gradient-based optimization methods. Meanwhile, though error estimations generated by L-GD algorithm are essentially qualitative, they could still provide valuable information to help evaluating the resultant model parameters, which is of great importance for practical geophysical inversions.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8.
Fig. 9.
Fig. 10
Fig. 11

source are marked as a yellow star

Fig. 12

References

  1. Afonso JC, Fullea J, Yang Y, Connolly JAD, Jones AG (2013) 3-D multi-observable probabilistic inversion for the compositional and thermal structure of the lithosphere and upper mantle. II: General methodology and resolution analysis. J Geophys Res: Solid Earth 118(4):1650–1676. https://doi.org/10.1002/jgrb.50123

    Article  Google Scholar 

  2. Afonso JC, Rawlinson N, Yang Y, Schutt DL, Jones AG, Fullea J, Griffin WL (2016) 3-D multiobservable probabilistic inversion for the compositional and thermal structure of the lithosphere and upper mantle: III. Thermochemical tomography in the Western-Central U.S. J Geophys Res: Solid Earth 121(10):7337–7370. https://doi.org/10.1002/2016JB013049

    Article  Google Scholar 

  3. Alumbaugh DL, Newman GA (2000) Image appraisal for 2-D and 3-D electromagnetic inversion. Geophysics 65(5):1455–1467

    Article  Google Scholar 

  4. Ammon CJ, Randall GE, Zandt G (1990) On the nonuniqueness of receiver function inversions. J Geophys Res 95(B10):15303. https://doi.org/10.1029/JB095iB10p15303

    Article  Google Scholar 

  5. Ballard S, Hipp JR, Young CJ (2009) Efficient and accurate calculation of ray theory seismic travel time through variable resolution 3-D Earth models. Seismol Res Lett 80(6):989–999. https://doi.org/10.1785/gssrl.80.6.989

    Article  Google Scholar 

  6. Barmin MP, Ritzwoller MH, Levshin AL (2001) A fast and reliable method for surface wave tomography. Pure Appl Geophys 158:1351–1375

    Article  Google Scholar 

  7. Barthelemy P, Bertolotti J, Wiersma DS (2008) A Lévy flight for light. Nature 453(7194):495–498. https://doi.org/10.1038/nature06948

    Article  Google Scholar 

  8. Bartumeus F, Catalan J, Fulco UL, Lyra ML, Viswanathan GM (2002) Optimizing the encounter rate in biological interactions: lévy versus brownian strategies. Phys Rev Lett 88(9):097901. https://doi.org/10.1103/PhysRevLett.88.097901

    Article  Google Scholar 

  9. Benhamou S (2007) How many animals really do the Lévy walk? Ecology 88(8):1962–1969. https://doi.org/10.1890/06-1769.1

    Article  Google Scholar 

  10. Bianco MJ, Gerstoft P (2018) Travel time tomography with adaptive dictionaries. IEEE Trans Comput Imaging 4(4):499–511. https://doi.org/10.1109/TCI.2018.2862644

    Article  Google Scholar 

  11. Bramer, M., Ellis, R., & Petridis, M. (Eds.). (2010). Research and Development in Intelligent Systems XXVI. Springer London. https://doi.org/https://doi.org/10.1007/978-1-84882-983-1

  12. Brockmann D, Hufnagel L, Geisel T (2006) The scaling laws of human travel. Nature 439(7075):462–465. https://doi.org/10.1038/nature04292

    Article  Google Scholar 

  13. Brown EL, Petersen KD, Lesher CE (2020) Markov chain Monte Carlo inversion of mantle temperature and source composition, with application to Reykjanes Peninsula Iceland. Earth Planet Sci Lett 532:116007. https://doi.org/10.1016/j.epsl.2019.116007

    Article  Google Scholar 

  14. Butler DK (ed) (2005) Near-surface geophysics. Society of exploration geophysicists. https://doi.org/10.1190/1.9781560801719

  15. Byrd RH, Lu P, Nocedal J, Zhu C (1995) A limited memory algorithm for bound constrained optimization. SIAM J Sci Comput 16(5):1190–1208

    Article  Google Scholar 

  16. Caliciotti A, Fasano G, Roma M (2018) Preconditioned nonlinear conjugate gradient methods based on a modified secant equation. Appl Math Comput 318:196–214. https://doi.org/10.1016/j.amc.2017.08.029

    Article  Google Scholar 

  17. Chen J, Kemna A, Hubbard SS (2008) A comparison between Gauss-Newton and Markov-chain Monte Carlo–based methods for inverting spectral induced-polarization data for Cole-Cole parameters. Geophysics 73(6):F247–F259. https://doi.org/10.1190/1.2976115

    Article  Google Scholar 

  18. Corral Á (2006) Universal earthquake-occurrence jumps, correlations with time, and anomalous diffusion. Phys Rev Lett 97(17):178501. https://doi.org/10.1103/PhysRevLett.97.178501

    Article  Google Scholar 

  19. Dai YH, Yuan Y (1999) A nonlinear conjugate gradient method with a strong global convergence property. SIAM J Optim 10(1):177–182. https://doi.org/10.1137/S1052623497318992

    Article  Google Scholar 

  20. Dauphin, Y. N., Pascanu, R., Gulcehre, C., Cho, K., Ganguli, S., & Bengio, Y. (2014). Identifying and attacking the saddle point problem in high-dimensional non-convex optimization. 2933–2941.

  21. Dozat, T. (2016). Incorporating Nesterov Momentum into Adam. 1–6.

  22. Dubkov AA, Spagnolo B, Uchaikin VV (2008) Lévy flight superdiffusion: an introduction. Int J Bifurcation Chaos 18(09):2649–2672. https://doi.org/10.1142/S0218127408021877

    Article  Google Scholar 

  23. Edwards AM, Phillips RA, Watkins NW, Freeman MP, Murphy EJ, Afanasyev V, Buldyrev SV, da Luz MGE, Raposo EP, Stanley HE, Viswanathan GM (2007) Revisiting Lévy flight search patterns of wandering albatrosses, bumblebees and deer. Nature 449(7165):1044–1048. https://doi.org/10.1038/nature06199

    Article  Google Scholar 

  24. Fogedby HC (1994) Lévy flights in random environments. Phys Rev Lett 73(19):2517–2520. https://doi.org/10.1103/PhysRevLett.73.2517

    Article  Google Scholar 

  25. Foks NL, Krahenbuhl R, Li Y (2014) Adaptive sampling of potential-field data: A direct approach to compressive inversion. Geophysics. https://doi.org/10.1190/geo2013-0087.1

    Article  Google Scholar 

  26. Grandis H, Menvielle M, Roussignol M (2002) Thin-sheet electromagnetic inversion modeling using Monte Carlo Markov Chain (MCMC) algorithm. Earth, Planets and Space 54(5):511–521. https://doi.org/10.1186/BF03353042

    Article  Google Scholar 

  27. Guo P, Singh SC, Vaddineni VA, Visser G, Grevemeyer I, Saygin E (2020) Nonlinear full waveform inversion of wide-aperture OBS data for Moho structure using a trans-dimensional Bayesian method. Geophys J Int 224(2):1056–1078. https://doi.org/10.1093/gji/ggaa505

    Article  Google Scholar 

  28. M Hamoda, M Mamat, M Rivaie, and Z Salleh (2016) A conjugate gradient method with strong Wolfe Powell line search for unconstrained optimization. Applied Mathematical Sciences, https://doi.org/10.12988/ams.2016.56449

  29. Han C, Xu M, Huang Z, Wang L, Xu M, Mi N, Yu D, Gou T, Wang H, Hao S, Tian M, Bi Y (2020) Layered crustal anisotropy and deformation in the SE Tibetan plateau revealed by Markov-Chain-Monte-Carlo inversion of receiver functions. Phys Earth Planet Inter 306:106522. https://doi.org/10.1016/j.pepi.2020.106522

    Article  Google Scholar 

  30. Hoang VH, Schwab C, Stuart AM (2013) Complexity analysis of accelerated MCMC methods for Bayesian inversion. Inverse Prob 29(8):085010. https://doi.org/10.1088/0266-5611/29/8/085010

    Article  Google Scholar 

  31. Hong T, Sen MK (2009) A new MCMC algorithm for seismic waveform inversion and corresponding uncertainty analysis. Geophys J Int 177(1):14–32. https://doi.org/10.1111/j.1365-246X.2008.04052.x

    Article  Google Scholar 

  32. Houssein EH, Saad MR, Hashim FA, Shaban H, Hassaballah M (2020) Lévy flight distribution: a new metaheuristic algorithm for solving engineering optimization problems. Eng Appl Artif Intell 94:103731. https://doi.org/10.1016/j.engappai.2020.103731

    Article  Google Scholar 

  33. Humphries NE, Queiroz N, Dyer JRM, Pade NG, Musyl MK, Schaefer KM, Fuller DW, Brunnschweiler JM, Doyle TK, Houghton JDR, Hays GC, Jones CS, Noble LR, Wearmouth VJ, Southall EJ, Sims DW (2010) Environmental context explains Lévy and Brownian movement patterns of marine predators. Nature 465(7301):1066–1069. https://doi.org/10.1038/nature09116

    Article  Google Scholar 

  34. Jain P, Kar P (2017) Non-convex optimization for machine learning. Found Trends Mach Learning 10(3–4):142–336. https://doi.org/10.1561/2200000058

    Article  Google Scholar 

  35. Kaasschieter EF (1988) Preconditioned conjugate gradients for solving singular systems. J Comput Appl Math 24(1–2):265–275. https://doi.org/10.1016/0377-0427(88)90358-5

    Article  Google Scholar 

  36. Kalscheuer T, Pedersen LB (2007) A non-linear truncated SVD variance and resolution analysis of two-dimensional magnetotelluric models. Geophys J Int 169(2):435–447. https://doi.org/10.1111/j.1365-246X.2006.03320.x

    Article  Google Scholar 

  37. Kingma DP, Ba J (2017) Adam: a method for stochastic optimization. arXiv:1412.6980

  38. Leccardi, M. (2005). Comparison of Three Algorithms for Lévy Noise Generation. In Proceedings of fifth EUROMECH nonlinear dynamics conference.

  39. Lelièvre PG, Farquharson CG, Hurich CA (2011) Inversion of first-arrival seismic traveltimes without rays, implemented on unstructured grids: inversion of first-arrivals without rays. Geophys J Int 185(2):749–763. https://doi.org/10.1111/j.1365-246X.2011.04964.x

    Article  Google Scholar 

  40. Lelièvre PG, Farquharson CG, Hurich CA (2012) Joint inversion of seismic traveltimes and gravity data on unstructured grids with application to mineral exploration. Geophysics 77(1):K1–K15. https://doi.org/10.1190/geo2011-0154.1

    Article  Google Scholar 

  41. Li Y, Oldenburg DW (1996) 3-D inversion of magnetic data. Geophysics 61(2):394–408

    Article  Google Scholar 

  42. Li Y, Oldenburg DW (1998) 3-D inversion of gravity data. Geophysics 63(1):109–119

    Article  Google Scholar 

  43. Li Y, Oldenburg DW (2003) Fast inversion of large-scale magnetic data using wavelet transforms and a logarithmic barrier method. Geophys J Int 152(2):251–265. https://doi.org/10.1046/j.1365-246X.2003.01766.x

    Article  Google Scholar 

  44. Liang Q, Chen C, Li Y (2014) 3-D inversion of gravity data in spherical coordinates with application to the GRAIL data: 3-D inversion in spherical coordinates. J Geophy Res: Planets 119(6):1359–1373. https://doi.org/10.1002/2014JE004626

    Article  Google Scholar 

  45. Manassero MC, Afonso JC, Zyserman F, Zlotnik S, Fomin I (2020) A reduced order approach for probabilistic inversions of 3D magnetotelluric data I: general formulation. Geophys J Int 223(3):1837–1863

    Article  Google Scholar 

  46. Mantegna RN, Stanley HE (1994) Stochastic process with ultraslow convergence to a Gaussian: the truncated Lévy flight. Phys Rev Lett 73(22):2946–2949. https://doi.org/10.1103/PhysRevLett.73.2946

    Article  Google Scholar 

  47. Martin J, Wilcox LC, Burstedde C, Ghattas O (2012) A stochastic Newton MCMC method for large-scale statistical inverse problems with application to seismic inversion. SIAM J Sci Comput 34(3):A1460–A1487. https://doi.org/10.1137/110845598

    Article  Google Scholar 

  48. W Menke (2012). Linear inverse problems and non-Gaussian statistics. In Geophysical Data Analysis: Discrete Inverse Theory (pp. 149–161). Elsevier. https://doi.org/https://doi.org/10.1016/B978-0-12-397160-9.00008-4

  49. Metropolis N, Rosenbluth AW, Rosenbluth MN, Teller AH, Teller E (1953) Equation of state calculations by fast computing machines. J Chem Phys 21(6):1087–1092

    Article  Google Scholar 

  50. Moré JJ, Thuente DJ (1994) Line search algorithms with guaranteed sufficient decrease. ACM Trans Math Softw 20(3):286–307. https://doi.org/10.1145/192115.192132

    Article  Google Scholar 

  51. Oldenburg DW, Li Y (1999) Estimating depth of investigation in dc resistivity and IP surveys. Geophysics 64(2):403–416. https://doi.org/10.1190/1.1444545

    Article  Google Scholar 

  52. Oldenburg DW, Li Y (2005) Inversion for applied geophysics: a tutorial. Society of Exploration Geophysicists, In Near-Surface Geophysics. https://doi.org/10.1190/1.9781560801719

    Book  Google Scholar 

  53. Pallero JLG, Fernández-Martínez JL, Bonvalot S, Fudym O (2015) Gravity inversion and uncertainty assessment of basement relief via Particle Swarm Optimization. J Appl Geophys 116:180–191. https://doi.org/10.1016/j.jappgeo.2015.03.008

    Article  Google Scholar 

  54. Pallero JLG, Fernández-Martínez JL, Bonvalot S, Fudym O (2017) 3D gravity inversion and uncertainty assessment of basement relief via Particle Swarm Optimization. J Appl Geophys 139:338–350. https://doi.org/10.1016/j.jappgeo.2017.02.004

    Article  Google Scholar 

  55. Pavlyukevich I (2007) Lévy flights, non-local search and simulated annealing. J Comput Phys 226(2):1830–1844. https://doi.org/10.1016/j.jcp.2007.06.008

    Article  Google Scholar 

  56. R Pytlak (2009). Conjugate Gradient Algorithms in Nonconvex Optimization (Vol. 89). Springer, Berlin Heidelberg. https://doi.org/10.1007/978-3-540-85634-4

  57. Reynolds AM, Rhodes CJ (2009) The Lévy flight paradigm: Random search patterns and mechanisms. Ecology 90(4):877–887. https://doi.org/10.1890/08-0153.1

    Article  Google Scholar 

  58. Ritzwoller MH, Levshin AL (1998) Eurasian surface wave tomography: Group velocities. J Geophys Res: Solid Earth 103(B3):4839–4878. https://doi.org/10.1029/97JB02622

    Article  Google Scholar 

  59. Rosas-Carbajal M, Linde N, Kalscheuer T, Vrugt JA (2014) Two-dimensional probabilistic inversion of plane-wave electromagnetic data: methodology, model constraints and joint inversion with electrical resistivity data. Geophys J Int 196(3):1508–1524. https://doi.org/10.1093/gji/ggt482

    Article  Google Scholar 

  60. Ruder S (2017) An overview of gradient descent optimization algorithms. arXiv:1609.04747

  61. Sabra KG, Gerstoft P, Roux P, Kuperman WA, Fehler MC (2005) Surface wave tomography from microseisms in Southern California: surface wave tomography. Geophys Res Lett. https://doi.org/10.1029/2005GL023155

    Article  Google Scholar 

  62. MK Sen, and PL Stoffa (2013). Global Optimization Methods in Geophysical Inversion (2nd ed.). Cambridge University Press. https://doi.org/https://doi.org/10.1017/CBO9780511997570

  63. Shapiro NM, Campillo M, Stehly L, Ritzwoller MH (2005) High-resolution surface-wave tomography from ambient seismic noise. Sci, New Series 307(5715):1615–1618

    Google Scholar 

  64. Shi, Q., Wei, S., and Chen, M. (2018). An MCMC multiple point sources inversion scheme and its application to the 2016 Kumamoto Mw 6.2 earthquake. Geophysical Journal International, 215(2):737–752. https://doi.org/10.1093/gji/ggy302

  65. MF Shlesinger, GM Zaslavsky, and U Frisch (Eds.). (1995). Lévy flights and related topics in physics: Proceedings of the international workshop held at Nice, France, 27–30 June 1994. Springer-Verlag.

  66. Siripunvaraporn W, Egbert G, Lenbury Y, Uyeshima M (2005) Three-dimensional magnetotelluric inversion: data-space method. Phys Earth Planet Inter 150(1–3):3–14. https://doi.org/10.1016/j.pepi.2004.08.023

    Article  Google Scholar 

  67. Solomon TH, Weeks ER, Swinney HL (1993) Observation of anomalous diffusion and Lévy flights in a two-dimensional rotating flow. Phys Rev Lett 71(24):3975–3978. https://doi.org/10.1103/PhysRevLett.71.3975

    Article  Google Scholar 

  68. Szu H, Hartley R (1987) Fast Simulated Annealing. Phys Lett A 122(3):157–162

    Article  Google Scholar 

  69. AN Tikhonov AV Goncharsky VV Stepanov AG Yagola 1995 Numerical Methods for the Solution of Ill-Posed Problems Springer, Netherlands https://doi.org/10.1007/978-94-015-8480-7

  70. Tsallis C, Levy SVF, Souza AMC, Maynard R (1995) Statistical-mechanical foundation of the ubiquity of Lévy distributions in nature. Phys Rev Lett 75(20):3589–3593. https://doi.org/10.1103/PhysRevLett.75.3589

    Article  Google Scholar 

  71. Tyran-Kamińska M (2010) Convergence to Lévy stable processes under some weak dependence conditions. Stoch Process Appl 120(9):1629–1650. https://doi.org/10.1016/j.spa.2010.05.010

    Article  Google Scholar 

  72. Usui Y (2015) 3-D inversion of magnetotelluric data using unstructured tetrahedral elements: applicability to data affected by topography. Geophys J Int 202(2):828–849. https://doi.org/10.1093/gji/ggv186

    Article  Google Scholar 

  73. Virieux J, Operto S (2009) An overview of full-waveform inversion in exploration geophysics. Geophysics. https://doi.org/10.1190/1.3238367

    Article  Google Scholar 

  74. Visser G, Guo P, Saygin E (2019) Bayesian transdimensional seismic full-waveform inversion with a dipping layer parameterization. Geophysics 84(6):R845–R858. https://doi.org/10.1190/geo2018-0785.1

    Article  Google Scholar 

  75. Viswanathan GM, Afanasyev V, Buldyrev SV, Murphy EJ, Prince PA, Stanley HE (1996) Lévy flight search patterns of wandering albatrosses. Nature 381(6581):413–415. https://doi.org/10.1038/381413a0

    Article  Google Scholar 

  76. Viswanathan GM, Afanasyev V, Buldyrev SV, Havlin S, Stanley HE (2000) Lévy ights in random searches. Phys A 282:1–12

    Article  Google Scholar 

  77. Walker M, Curtis A (2014) Spatial Bayesian inversion with localized likelihoods: an exact sampling alternative to MCMC. J Geophys Res: Solid Earth 119(7):5741–5761. https://doi.org/10.1002/2014JB011010

    Article  Google Scholar 

  78. Wellington P, Brossier R, Virieux J (2019) Preconditioning full-waveform inversion with efficient local correlation operators. Geophysics 84(3):R321–R332. https://doi.org/10.1190/geo2018-0584.1

    Article  Google Scholar 

  79. Xu P, Roosta F, Mahoney MW (2020) Newton-type methods for non-convex optimization under inexact Hessian information. Math Program 184(1–2):35–70. https://doi.org/10.1007/s10107-019-01405-z

    Article  Google Scholar 

  80. Yang XS (2012) Cuckoo search for inverse problems and simulated-driven shape optimization. J Comput Methods Sci Eng 12(1–2):129–137. https://doi.org/10.3233/JCM-2012-0408

    Article  Google Scholar 

  81. Yang XS, Deb S (2009) Cuckoo search via Lévy flights. World Congress Nature Biol Inspir Comput (NaBIC) 2009:210–214. https://doi.org/10.1109/NABIC.2009.5393690

    Article  Google Scholar 

  82. Zaburdaev V, Denisov S, Klafter J (2015) Lévy walks. Rev Mod Phys 87(2):483–530. https://doi.org/10.1103/RevModPhys.87.483

    Article  Google Scholar 

  83. Zhang J, Wang C, Shi Y, Cai Y, Chi W, Dreger D, Cheng W, Yuan Y (2004) Three-dimensional crustal structure in central Taiwan from gravity inversion with a parallel genetic algorithm. Geophysics 69(4):917–924. https://doi.org/10.1190/1.1778235

    Article  Google Scholar 

  84. Zhang Y, Mooney WD, Chen C (2018) Forward calculation of gravitational fields with variable resolution 3D density models using spherical triangular tessellation: theory and applications. Geophys J Int 215(1):363–374. https://doi.org/10.1093/gji/ggy278

    Article  Google Scholar 

  85. Zhang Y, Mooney WD, Chen C, Du J (2019) Interface inversion of gravitational data using spherical triangular tessellation: an application for the estimation of the Moon’s crustal thickness. Geophys J Int 217(1):703–713. https://doi.org/10.1093/gji/ggz026

    Article  Google Scholar 

  86. Zhou P, Yuan X, Yan S, Feng J (2019) Faster first-order methods for stochastic non-convex optimization on Riemannian manifolds. IEEE Trans Pattern Anal Mach Intell. https://doi.org/10.1109/TPAMI.2019.2933841

    Article  Google Scholar 

  87. Zhu L, Kanamori H (2000) Moho depth variation in southern California from teleseismic receiver functions. J Geophys Res: Solid Earth 105(B2):2969–2980. https://doi.org/10.1029/1999JB900322

    Article  Google Scholar 

Download references

Acknowledgements

The authors are grateful for valuable comments from two anonymous reviewers. This study is supported by the China Postdoctoral Science Foundation (No. 2020M681824) and the Natural Science Foundation of China (41974082, 41830212).

Author information

Affiliations

Authors

Corresponding author

Correspondence to Yixian Xu.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Zhang, Y., Xu, Y. & Yang, B. Lévy Gradient Descent: Augmented Random Search for Geophysical Inverse Problems. Surv Geophys 42, 899–921 (2021). https://doi.org/10.1007/s10712-021-09644-6

Download citation

Keywords

  • Lévy flights
  • Geophysical inversions
  • Inverse problem
  • Gradient descent
  • Non-convex optimization