# A Novel Hybrid Algorithm of Particle Swarm Optimization and Evolution Strategies for Geophysical Non-linear Inverse Problems

- 111 Downloads

## Abstract

Population-based optimization algorithms are a class of stochastic global search methods, which probe the model space based on a set of nature-inspired rules in an iterative and random manner. Particle swarm optimization (PSO) is inspired by the social behavior of bird flocks and fish schools and is designed as a black-box optimization algorithm. In each iteration, a set of particles (i.e. potential solutions) simultaneously search the model space for each of which the cost function is calculated. Consequently, the computational cost of the search is directly related to the population size. Several empirical rules exist for the relationship between the model space dimension and the population size. But, since the model space dimension is problem-related, little discussion exists over the optimal population size for a successful convergence. However, compared to usual benchmark optimization problems, geophysical inversions have substantially higher dimensions, and as such large population sizes are mandatory for reaching meaningful solutions. Hence, the use of PSO becomes considerably infeasible for inverse problems in terms of the computation burden. Herein, a problem-oriented hybrid algorithm of PSO and evolution strategies, PSO/ES, is presented which integrates adaptive mutations into PSO with the goal of reducing the calculation time. As a result, instead of continuous search paths, particles follow a discrete scheme which allows them to search the model space in fewer numbers and more effectively. The algorithm is tested on a real 3D non-linear gravity inverse problem to estimate the thickness of the sedimentary cover in the South Caspian Basin. The problem is solved using both PSO and PSO/ES, where the results show that while PSO has prematurely converged due to insufficient population size, PSO/ES has been able to find a meaningful solution. The results agree well with the existing measurements in the study area.

## Keywords

Stochastic inversion particle swarm optimization evolution strategies basement topography non-linear gravity inversion## Notes

### Acknowledgements

The authors would like to acknowledge the financial support of the University of Tehran for this research under grant number 30250/01/04. Figures are prepared with the Generic Mapping Tools software (Wessel et al. 2013). The manuscript was improved by the insightful reviews by two anonymous reviewers.

## References

- Akay, B., and D. Karaboga (2009), Parameter tuning for the artificial bee colony algorithm, paper presented at International Conference on Computational Collective Intelligence, Springer.Google Scholar
- Back, T. (1996), Evolutionary algorithms in theory and practice: evolution strategies, evolutionary programming, genetic algorithms, Oxford university press.Google Scholar
- Balkaya, Ç., Ekinci, Y. L., Göktürkler, G., & Turan, S. (2017). 3D non-linear inversion of magnetic anomalies caused by prismatic bodies using differential evolution algorithm.
*Journal of Applied Geophysics,**136,*372–386.Google Scholar - Basler-Reeder, K., J. Louie, S. Pullammanappallil, and G. Kent (2016), Joint optimization of vertical component gravity and P-wave first arrivals by simulated annealing, Geophysics, 81(4), ID59-ID71.Google Scholar
- Beyer, H.-G. (2013), The theory of evolution strategies, Springer Science & Business Media.Google Scholar
- Bianchi, L., Birattari, M., Chiarandini, M., Manfrin, M., Mastrolilli, M., Paquete, L., et al. (2006). Hybrid metaheuristics for the vehicle routing problem with stochastic demands.
*Journal of Mathematical Modelling and Algorithms,**5*(1), 91–110.Google Scholar - Blum, C., Puchinger, J., Raidl, G. R., & Roli, A. (2011). Hybrid metaheuristics in combinatorial optimization: a survey.
*Applied Soft Computing,**11*(6), 4135–4151.Google Scholar - Bratton, D., and J. Kennedy (2007), Defining a standard for particle swarm optimization, paper presented at 2007 IEEE swarm intelligence symposium, IEEE.Google Scholar
- Chaimatanan, S., D. Delahaye, and M. Mongeau (2018), Hybrid metaheuristic for air traffic management with uncertainty, in Recent Developments in Metaheuristics, edited, pp. 219–251, Springer.Google Scholar
- Clerc, M., & Kennedy, J. (2002). The particle swarm-explosion, stability, and convergence in a multidimensional complex space.
*Evolutionary Computation, IEEE Transactions on,**6*(1), 58–73.Google Scholar - Dianati, M., I. Song, and M. Treiber (2002), An introduction to genetic algorithms and evolution strategies Rep., Technical report, University of Waterloo, Ontario, N2L 3G1, Canada.Google Scholar
- Diouane, Y. (2014).
*Globally convergent evolution strategies with application to Earth imaging problem in geophysics*. Doctoral dissertation. Toulouse: École Doctorale Mathématiques, Informatique et Télécommunications.Google Scholar - Dorigo, M., Maniezzo, V., & Colorni, A. (1996). Ant system: optimization by a colony of cooperating agents.
*Systems, Man, and Cybernetics, Part B: Cybernetics, IEEE Transactions on,**26*(1), 29–41.Google Scholar - Eberhart, R. C., Kennedy J. (1995). A new optimizer using particle swarm theory. In: Paper presented at Proceedings of the sixth international symposium on micro machine and human science, New York, NY.Google Scholar
- Egan, S. S., Mosar, J., Brunet, M.-F., & Kangarli, T. (2009). Subsidence and uplift mechanisms within the South Caspian Basin: insights from the onshore and offshore Azerbaijan region.
*Geological Society, London, Special Publications,**312*(1), 219–240.Google Scholar - Eiben, A. E., & Smit, S. K. (2011). Parameter tuning for configuring and analyzing evolutionary algorithms.
*Swarm and Evolutionary Computation,**1*(1), 19–31.Google Scholar - Eiben, A. E., & Smith, J. E. (2003).
*Introduction to evolutionary computing*. Berlin: Springer.Google Scholar - Farquharson, C. G., & Oldenburg, D. W. (2004). A comparison of automatic techniques for estimating the regularization parameter in non-linear inverse problems.
*Geophysical Journal International,**156*(3), 411–425.Google Scholar - Förste, C., S. Bruinsma, O. Abrikosov, F. Flechtner, J.-C. Marty, J.-M. Lemoine, C. Dahle, H. Neumayer, F. Barthelmes, and R. König (2014), EIGEN-6C4-The latest combined global gravity field model including GOCE data up to degree and order 1949 of GFZ Potsdam and GRGS Toulouse, paper presented at EGU General Assembly Conference Abstracts.Google Scholar
- Fullea, J. (2008).
*Development of numerical methods to determine the litospheric structure combining geopotential, litosthatic and heat transport equations, Application to the Gibraltar arc system, PhD*. Barcelona: Univ.Google Scholar - Fullea, J., Afonso, J. C., Connolly, J. A. D., Fernandez, M., García-Castellanos, D., & Zeyen, H. (2009). LitMod3D: An interactive 3-D software to model the thermal, compositional, density, seismological, and rheological structure of the lithosphere and sublithospheric upper mantle.
*Geochemistry, Geophysics, Geosystems, 10*(8), 1–21. https://doi.org/10.1029/2009GC002391.Google Scholar - Gallardo-Delgado, L. A., Pérez-Flores, M. A., & Gómez-Treviño, E. (2003). A versatile algorithm for joint 3D inversion of gravity and magnetic data.
*Geophysics,**68*(3), 949–959.Google Scholar - Grayver, A. V., & Kuvshinov, A. V. (2016). Exploring equivalence domain in nonlinear inverse problems using Covariance Matrix Adaption Evolution Strategy (CMAES) and random sampling.
*Geophysical Journal International,**205*(2), 971–987.Google Scholar - Guest, B., Guest, A., & Axen, G. (2007). Late Tertiary tectonic evolution of northern Iran: a case for simple crustal folding.
*Global and Planetary Change,**58*(1–4), 435–453.Google Scholar - Hansen, P. C. (1992). Analysis of discrete ill-posed problems by means of the L-curve.
*SIAM Review,**34*(4), 561–580.Google Scholar - Hansen, N., Arnold, D. V., & Auger, A. (2015).
*Evolution strategies, in Springer handbook of computational intelligence*(pp. 871–898). Berlin: Springer.Google Scholar - Hansen, N., & Ostermeier, A. (2001). Completely derandomized self-adaptation in evolution strategies.
*Evolutionary computation,**9*(2), 159–195.Google Scholar - Ingber, L. (1993). Simulated annealing: practice versus theory.
*Mathematical and Computer Modelling,**18*(11), 29–57.Google Scholar - Jackson, J., Priestley, K., Allen, M., & Berberian, M. (2002). Active tectonics of the south Caspian basin.
*Geophysical Journal International,**148*(2), 214–245.Google Scholar - Jamasb, A., Motavalli-Anbaran, S.-H., & Zeyen, H. (2017). Non-linear stochastic inversion of gravity data via quantum-behaved particle swarm optimisation: application to Eurasia-Arabia collision zone (Zagros, Iran).
*Geophysical Prospecting,**65,*274–294. https://doi.org/10.1111/1365-2478.12558.Google Scholar - Karaboga, D. (2005). An idea based on honey bee swarm for numerical optimization Rep., Technical report-tr06, Erciyes university, engineering faculty, computer engineering department.Google Scholar
- Kennedy, J., Kennedy, J. F., Eberhart, R. C., & Shi, Y. (2001).
*Swarm intelligence*. Burlington: Morgan Kaufmann.Google Scholar - Knapp, C. C., Knapp, J. H., & Connor, J. A. (2004). Crustal-scale structure of the South Caspian Basin revealed by deep seismic reflection profiling.
*Marine and Petroleum Geology,**21*(8), 1073–1081.Google Scholar - Liu, S., Hu, X., & Liu, T. (2014). A stochastic inversion method for potential field data: ant colony optimization.
*Pure and Applied Geophysics,**171*(7), 1531–1555.Google Scholar - Liu, S., Hu, X., Liu, T., Xi, Y., Cai, J., & Zhang, H. (2015). Ant colony optimisation inversion of surface and borehole magnetic data under lithological constraints.
*Journal of Applied Geophysics,**112,*115–128.Google Scholar - Martínez, J. F., & Gonzalo, E. G. (2009). The PSO family: deduction, stochastic analysis and comparison.
*Swarm Intelligence,**3*(4), 245–273.Google Scholar - Martins, C. M., Lima, W. A., Barbosa, V. C., & Silva, J. B. (2011). Total variation regularization for depth-to-basement estimate: part 1—Mathematical details and applications.
*Geophysics,**76*(1), I1–I12.Google Scholar - Mead, J., & Hammerquist, C. (2013). χ^2 Tests for the Choice of the Regularization Parameter in Nonlinear Inverse Problems.
*SIAM Journal on Matrix Analysis and Applications,**34*(3), 1213–1230.Google Scholar - Miranda, V., Fonseca N. (2002). EPSO-best-of-two-worlds meta-heuristic applied to power system problems, paper presented at Evolutionary Computation, 2002. CEC’02. In: Proceedings of the 2002 Congress on, IEEE.Google Scholar
- Mirjalili, S., & Lewis, A. (2016). The whale optimization algorithm.
*Advances in Engineering Software,**95,*51–67.Google Scholar - Motavalli-Anbaran, S., Jamasb A. (2016), Estimating the depth to the base of sedimentary layer in South Caspian Basin (Iran) by particle swarm optimization (PSO). Paper presented at 78th EAGE Conference and Exhibition 2016.Google Scholar
- Motavalli-Anbaran, S.-H., Zeyen, H., & Ardestani, V. E. (2013). 3D joint inversion modeling of the lithospheric density structure based on gravity, geoid and topography data—Application to the Alborz Mountains (Iran) and South Caspian Basin region.
*Tectonophysics,**586,*192–205.Google Scholar - Motavalli-Anbaran, S. H., Zeyen, H., Brunet, M. F., & Ardestani, V. E. (2011). Crustal and lithospheric structure of the Alborz Mountains, Iran, and surrounding areas from integrated geophysical modeling.
*Tectonics,**30*(5), 56.Google Scholar - Mühlenbein, H., Gorges-Schleuter, M., & Krämer, O. (1988). Evolution algorithms in combinatorial optimization.
*Parallel Computing,**7*(1), 65–85.Google Scholar - Nagihara, S., & Hall, S. A. (2001). Three-dimensional gravity inversion using simulated annealing: constraints on the diapiric roots of allochthonous salt structures.
*Geophysics,**66*(5), 1438–1449.Google Scholar - Nagy, D., Papp, G., & Benedek, J. (2000). The gravitational potential and its derivatives for the prism.
*Journal of Geodesy,**74*(7–8), 552–560.Google Scholar - Pallero, J., Fernández-Martínez, J. L., Bonvalot, S., & Fudym, O. (2015). Gravity inversion and uncertainty assessment of basement relief via Particle Swarm Optimization.
*Journal of Applied Geophysics,**116,*180–191.Google Scholar - Pallero, J., Fernández-Martínez, J., Bonvalot, S., & Fudym, O. (2017). 3D gravity inversion and uncertainty assessment of basement relief via Particle Swarm Optimization.
*Journal of Applied Geophysics,**139,*338–350.Google Scholar - Pallero, J. L., Fernández-Muñiz, M. Z., Cernea, A., Álvarez-Machancoses, Ó., Pedruelo-González, L. M., Bonvalot, S., et al. (2018). Particle Swarm Optimization and Uncertainty Assessment in Inverse Problems.
*Entropy,**20*(2), 96.Google Scholar - Pant, M., Thangaraj, R., & Abraham, A. (2009).
*Particle swarm optimization: performance tuning and empirical analysis, foundations of computational intelligence*(pp. 101–128). Berlin: Springer.Google Scholar - Portniaguine, O., & Zhdanov, M. S. (1999). Focusing geophysical inversion images.
*Geophysics,**64*(3), 874–887.Google Scholar - Priestley, K., Baker, C., & Jackson, J. (1994). Implications of earthquake focal mechanism data for the active tectonics of the South Caspian Basin and surrounding regions.
*Geophysical Journal International,**118*(1), 111–141.Google Scholar - Rechenberg, I. (1989).
*Evolution strategy: nature’s way of optimization, in optimization: methods and applications, possibilities and limitations*(pp. 106–126). Berlin: Springer.Google Scholar - Sarkar, S., Das S. (2010). A hybrid particle swarm with differential evolution operator approach (DEPSO) for linear array synthesis. In: Paper presented at international conference on Swarm, evolutionary, and memetic computing, Springer.Google Scholar
- Sen, M. K., & Stoffa, P. L. (2013).
*Global optimization methods in geophysical inversion*. Cambridge: Cambridge University Press.Google Scholar - Settles, M., Soule T. (2005). Breeding swarms: a GA/PSO hybrid. In: Paper presented at proceedings of the 7th annual conference on genetic and evolutionary computation, ACM.Google Scholar
- Shaw, R., & Srivastava, S. (2007). Particle swarm optimization: a new tool to invert geophysical data.
*Geophysics,**72*(2), F75–F83. https://doi.org/10.1190/1.2432481.Google Scholar - Snopek, K. (2005). Inversion of gravity data with application to density modeling of the Hellenic subduction zone, Ph. D. thesis, University of Bochum.Google Scholar
- Sun, J., Fang, W., Wu, X., Palade, V., & Xu, W. (2012). Quantum-behaved particle swarm optimization: analysis of individual particle behavior and parameter selection.
*Evolutionary Computation,**20*(3), 349–393.Google Scholar - Sun, J., Lai, C.-H., & Wu, X.-J. (2011).
*Particle swarm optimisation classical and quantum perspectives*. Boca Raton: CRC Press.Google Scholar - Trelea, I. C. (2003). The particle swarm optimization algorithm: convergence analysis and parameter selection.
*Information Processing Letters,**85*(6), 317–325.Google Scholar - Uieda, L., & Barbosa, V. C. (2017). Fast nonlinear gravity inversion in spherical coordinates with application to the South American Moho.
*Geophysical Journal International,**208*(1), 162–176.Google Scholar - Vatankhah, S., Ardestani, V. E., & Renaut, R. A. (2015). Application of the
*χ*2 principle and unbiased predictive risk estimator for determining the regularization parameter in 3-D focusing gravity inversion.*Geophysical Journal International,**200*(1), 265–277.Google Scholar - Vatankhah, S., Renaut, R. A., & Ardestani, V. E. (2014). Regularization parameter estimation for underdetermined problems by the
*χ*2 principle with application to 2D focusing gravity inversion.*Inverse Problems,**30*(8), 085002.Google Scholar - Vesterstrom, J., Thomsen R. (2004). A comparative study of differential evolution, particle swarm optimization, and evolutionary algorithms on numerical benchmark problems, paper presented at Evolutionary Computation, 2004. In: CEC2004. Congress on, IEEE.Google Scholar
- Vidal, T., Battarra, M., Subramanian, A., & Erdogan, G. (2015). Hybrid metaheuristics for the clustered vehicle routing problem.
*Computers and Operations Research,**58,*87–99.Google Scholar - Wessel, P., Smith, W. H., Scharroo, R., Luis, J., & Wobbe, F. (2013). Generic mapping tools: improved version released.
*Eos, Transactions American Geophysical Union,**94*(45), 409–410.Google Scholar - Yazdani, M., & Jolai, F. (2016). Lion optimization algorithm (LOA): a nature-inspired metaheuristic algorithm.
*Journal of Computational Design and Engineering,**3*(1), 24–36.Google Scholar - Zhang, Y., Wang, S., & Ji, G. (2015). A comprehensive survey on particle swarm optimization algorithm and its applications.
*Mathematical Problems in Engineering, 2015*, 931256. https://doi.org/10.1155/2015/931256.Google Scholar