What Makes Particle Swarm Optimization a Very Interesting and Powerful Algorithm?
Particle swarm optimization (PSO) is an evolutionary computational technique used for optimization motivated by the social behavior of individuals in large groups in nature. Different approaches have been used to understand how this algorithm works and trying to improve its convergence properties for different kind of problems. These approaches go from heuristic to mathematical analysis, passing through numerical experimentation. Although the scientific community has been able to solve a big variety of engineering problems, the tuning of the PSO parameters still remains one of its major drawbacks.
This chapter reviews the methodology developed within our research group over the last three years, which is based in adopting a completely different approach than those followed by most of the researchers in this field. By trying to avoid heuristics we proved that PSO can be physically interpreted as a particular discretization of a stochastic damped mass-spring system. Knowledge of this analogy has been crucial in deriving the PSO continuous model and to deduce a family of PSO members with different properties with regard to their exploitation/exploration balance: the generalized PSO (GPSO), the CC-PSO (centered PSO), CP-PSO (centered-progressive PSO), PP-PSO (progressive-progressive PSO) and RR-PSO (regressive-regressive PSO). Using the theory of stochastic differential and difference equations, we fully characterize the stability behavior of these algorithms. For well posed problems, a sufficient condition to achieve convergence is to select the PSO parameters close to the upper limit of second order stability. This result is also confirmed by numerical experimentation for different benchmark functions having an increasing degree of numerical difficulties. We also address how the discrete GPSO version (stability regions and trajectories) approaches the continuous PSO model as the time step decreases to zero. Finally, in the context of inverse problems, we address the question of how to select the appropriate PSO version: CP-PSO is the most explorative version and should be selected when we want to perform sampling of the posterior distribution of the inverse model parameters. Conversely, CC-PSO and GPSO provide higher convergence rates. Based on the analysis shown in this chapter, we can affirm that the PSO optimizers are not heuristic algorithms since there exist mathematical results that can be used to explain their consistency/convergence.
Keywordsparticle swarm PSO continuous model GPSO CC-GPSO CP-GPSO stochastic stability analysis convergence
Unable to display preview. Download preview PDF.
- 1.Kennedy, J., Eberhart, R.: Particle swarm optimization. In: Proceedings of the IEEE International Conference on Neural Networks (ICNN 1995), Perth,WA, Australia, vol. 4, pp. 1942–1948 (November-December, 1995)Google Scholar
- 2.Ozcan, E., Mohan, C.K.: Analysis of a simple particle swarm optimization system. In: Intelligent Engineering Systems Through Artificial Neural Networks, vol. 8, pp. 253–258. ASME Press, St. Louis (1998)Google Scholar
- 4.Carlisle, A., Dozier, G.: 1001, An off-the-shelf PSO. In: Proceedings of The Workshop On particle Swarm Optimization, Indianapolis, USA (2001)Google Scholar
- 7.Zheng, Y.-L., Ma, L.-H., Zhang, L.-Y., Qian, J.-X.: On the convergence analysis and parameter selection in particle swarm optimisation. In: Proceedings of the 2nd International Conference on Machine Learning and Cybernetics (ICMLC 2003), Xi’an, China, vol. 3, pp. 1802–1807 (November 2003)Google Scholar
- 9.Poli, R.: Dynamics and Stability of the Sampling Distribution of Particle Swarm Optimisers via Moment Analysis. Journal of Artificial Evolution and Applications, Article ID 761459, 10 (2008), doi:10.1155/2008/761459Google Scholar
- 11.Fernández Martínez, J.L., García Gonzalo, E.: The generalized PSO: a new door for PSO evolution. Journal of Artificial Evolution and Applications Article ID 861275, 15 (2008), doi:10.1155/2008/861275Google Scholar
- 12.Fernández Martínez, J.L., García Gonzalo, E.: The PSO family: deduction, stochastic analysis and comparison. Special issue on PSO. Swarm Intelligence 3, 245–273 (2009), doi:10.1007/s11721-009-0034-8Google Scholar
- 13.Fernández Martínez, J. L., García Gonzalo, E.: Stochastic stability analysis of the linear continuous and discrete PSO models. Technical Report, Department of Mathematics, University of Oviedo, Spain (June 2009), Submitted to IEEE Transactions on Evolutionary Computation (2009)Google Scholar
- 14.García-Gonzalo, E., Fernández-Martínez, J. L.: The PP-GPSO and RR-GPSO. Technical Report. Department of Mathematics. University of Oviedo, Spain (December 2009b), Submitted to IEEE Transactions on Evolutionary Computation (2009)Google Scholar
- 17.Clerc, M.: Stagnation analysis in particle swarm optimisation or what happens when nothing happens, Tech. Rep. CSM-460, Department of Computer Science, University of Essex (August 2006)Google Scholar
- 21.Sun, J.Q.: Stochastic Dynamics and Control. In: Luo, A.C.J., Zaslavsky, G. (eds.) Monographs series on Nonlinear Science and Complexity, p. 410. Elsevier, Amsterdam (2006)Google Scholar
- 22.Lutes, L.D., Sarkani, S.: Random vibrations. In: Analysis of Structural and Mechanical Systems, p. 638. Elsevier, Amsterdam (2004)Google Scholar
- 23.García Gonzalo, E., Fernández Martínez, J.L.: Design of a simple and powerful Particle Swarm optimizer. In: Proceedings of the International Conference on Computational and Mathematical Methods in Science and Engineering, CMMSE 2009, Gijón, Spain (2009)Google Scholar
- 24.Fernández-Martínez, J.L., García-Gonzalo, E., Álvarez, J.P.F., Kuzma, H.A., Menéndez-Pérez, C.O.: PSO: A Powerful Algorithm to Solve Geophysical Inverse Problems. Application to a 1D-DC Resistivity Case. Jounal of Applied Geophysics (2010), doi:10.1016/j.jappgeo.2010.02.001Google Scholar
- 25.Fernández Martínez, J.L., García Gonzalo, E., Naudet, V.: Particle Swarm Optimization applied to the solving and appraisal of the Streaming Potential inverse problem. Geophysics (2010) special issue in Hydrogeophysics (accepted for publication)Google Scholar
- 27.Fernández Martínez, J.L., Kuzma, H., García Gonzalo, E., Fernández Díaz, J.M., Fernández Alvarez, J.P., Menéndez Pérez, C.O.: Application of global optimization algorithms to a salt water intrusion problem. In: Symposium on the Application of Geophysics to Engineering and Environmental Problems, vol. 22, pp. 252–260 (2009a)Google Scholar
- 28.Fernández Martínez, J.L., Ciaurri, D.E., Mukerji, T., Gonzalo, E.G.: Application of Particle Swarm optimization to Reservoir Modeling and Inversion. In: Fernández Martínez, J.L. (ed.) International Association of Mathematical Geology (IAMG 2009), Stanford University (August 2009b)Google Scholar
- 29.Fernández Martínez, J.L., García Gonzalo, E., Fernández Muñiz, Z., Mukerji, T.: How to design a powerful family of Particle Swarm Optimizers for inverse modeling. New Trends on Bio-inspired Computation. Transactions of the Institute of Measurement and Control (2010c)Google Scholar