Hybrid particle swarm optimization and pattern search algorithm

Abstract

Particle swarm optimization (PSO) is one of the most commonly used stochastic optimization algorithms for many researchers and scientists of the last two decades, and the pattern search (PS) method is one of the most important local optimization algorithms. In this paper, we test three methods of hybridizing PSO and PS to improve the global minima and robustness. All methods let PSO run first followed by PS. The first method lets PSO use a large number of particles for a limited number of iterations. The second method lets PSO run normally until tolerance is reached. The third method lets PSO run normally until the average particle distance from the global best location is within a threshold. Numerical results using non-differentiable test functions reveal that all three methods improve the global minima and robustness versus PSO. The third hybrid method was also applied to a basin network optimization problem and outperformed PSO with filter method and genetic algorithm with implicit filtering.

Appendix

Benchmark Functions

Name	Function; Bounds; Optimal Value; Optimal Location
Alpine 1	$f({\mathbf {x}})=\sum _{i=1}^D|x_i\sin (x_i)+0.1 x_i|$ ;
	$x_i\in [-10,10]$ ; 0 ; (0, ..., 0)
Bartels conn	$f({\mathbf {x}})=|x_1^2 + x_2^2 + x_1 x_2| + |\sin (x_1)| + |\cos (x_2)|$ ;
	$x_i\in [-500,500]$; 1 ; (0,0)
Bukin N. 4	$f({\mathbf {x}})=100x_2^2+0.01|x_1+10|$ ;
	$x_1\in [-15,-5]$, $x_2\in [-3,3]$ ; 0 ; (-10,0)
Bukin N. 6	$f({\mathbf {x}})=100\sqrt{|x_2-0.01x_1^2|}+0.01|x_1+10|$ ;
	$x_1\in [-15,-5]$; 0 ; (-10,1)
	$x_2\in [-3,3]$
Corana	$f({\mathbf {x}})=0.15(z_i-0.05 {z_i}^2)d_i$ if $|v+i|<A$, else $d_i x_i^2$ ;
	$x_i\in [-500,500]$ ; 0 ; (0, 0, 0, 0)
Cosine mixture	$f({\mathbf {x}})=-0.1\sum _{i=1}^n \cos (5 \pi x_i)-\sum _{i=1}^n x_i^2$;
	$x_i\in [-1,1]$ ; 0.2 if $n=2$, 0.4 if $n=4$ ; (0, 0)
Cross-in-tray	$f({\mathbf {x}})=-0.0001(|\sin (x_1)\sin (x_2)\exp (|100-\frac{\sqrt{x_1^2+x_2^2}}{\pi }|)|+1)^{0.1}$ ;
	$x_1,\,x_2\in [-10,10]$ ; -2.06261218 ; $(\pm 1.349406685353340$,
	$\pm 1.349406608602084)$
Holder-table	$f({\mathbf {x}})=-|\sin (x_1)\cos (x_2)\exp (|1-\frac{\sqrt{x_1^2+x_2^2}}{\pi }|)|$ ;
	$x_1,\,x_2\in [-10,10]$ ; -19.2085 ; $(\pm 8.05502,\pm 9.66459)$
Powell sum	$f({\mathbf {x}})=\sum _{i=1}^{n}|x_i|^{i+1}$ ;
	$x_i\in [-1,1]$ ; 0 ; (0, ..., 0)
Price 1	$f({\mathbf {x}})=(|x_1|-5)^2+(|x_2|-5)^2$ ;
	$x_i\in [-500,500]$ ; 0 ; $(\pm 5,\pm 5)$
Schwefel	$f({\mathbf {x}}) = 418.9829d -{\sum _{i=1}^{n} x_i \sin (\sqrt{|x_i|})}$ ;
	$x_i\in [-500,500]$ ; 0 ; (420.9687, ..., 420.9687)
Schwefel 2.20	$f({\mathbf {x}})=\sum _{i=1}^n |x_i|$ $x_i\in [-100,100]$ ; 0 ; (0, ..., 0)
Schwefel 2.21	$f({\mathbf {x}})=\max _{i=1,...,n}|x_i|$ ;
	$x_i\in [-100,100]$ ; 0 ; (0, ..., 0)
Schwefel 2.22	$f({\mathbf {x}})=\sum _{i=1}^{n}|x_i|+\prod _{i=1}^{n}|x_i|$ ;
	$x_i\in [-100,100]$ ; 0 ; (0, ..., 0)
Step	$f({\mathbf {x}})=\sum _{i=1}^D \lfloor |x_i| \rfloor$ ;
	$x_i\in [-100,100]$ ; 0 ; (0, ..., 0)
Step 2	$f({\mathbf {x}})=\sum _{i=1}^D (\lfloor x_i+0.5 \rfloor )^2$ ;
	$x_i\in [-100,100]$ ; 0 ; (0.5, ..., 0.5)
Step 3	$f({\mathbf {x}})=\sum _{i=1}^D \lfloor x_i^2 \rfloor$ ;
	$x_i\in [-100,100]$ ; 0 ; (0, ..., 0)
Stepint	$f({\mathbf {x}})=25+\sum _{i=1}^D \lfloor x_i \rfloor$ ;
	$x_i\in [-5.12,5.12]$ ; 0 ; (0, ..., 0)
Xin-She Yang	$f({\mathbf {x}})=\sum _{i=1}^{n}\epsilon _i|x_i|^i$ ;
	$x_i\in [-5,5]$ ; 0 ; (0, ..., 0)
Xin-She Yang N.2	$f({\mathbf {x}})=(\sum _{i=1}^{n}|x_i|)\exp (-\sum _{i=1}^{n}\sin (x_i^2))$ ;
	$x_i\in [-2\pi ,2\pi ]$ ; 0 ; (0, ..., 0)
Xin-She Yang N.4	$f({\mathbf {x}})=\left[ \sum _{i=1}^{n}\sin ^2(x_i)-\exp \left( -\sum _{i=1}^{n}x_i^2\right) \right] \exp \left( -\sum _{i=1}^{n}{\sin ^2\sqrt{|x_i|}}\right)$ ;
	$x_i\in [-10,10]$ ; -1 ; (0, ..., 0)