Advertisement

What Makes Particle Swarm Optimization a Very Interesting and Powerful Algorithm?

  • J. L. Fernández-Martínez
  • E. García-Gonzalo
Part of the Adaptation, Learning, and Optimization book series (ALO, volume 8)

Abstract

Particle swarm optimization (PSO) is an evolutionary computational technique used for optimization motivated by the social behavior of individuals in large groups in nature. Different approaches have been used to understand how this algorithm works and trying to improve its convergence properties for different kind of problems. These approaches go from heuristic to mathematical analysis, passing through numerical experimentation. Although the scientific community has been able to solve a big variety of engineering problems, the tuning of the PSO parameters still remains one of its major drawbacks.

This chapter reviews the methodology developed within our research group over the last three years, which is based in adopting a completely different approach than those followed by most of the researchers in this field. By trying to avoid heuristics we proved that PSO can be physically interpreted as a particular discretization of a stochastic damped mass-spring system. Knowledge of this analogy has been crucial in deriving the PSO continuous model and to deduce a family of PSO members with different properties with regard to their exploitation/exploration balance: the generalized PSO (GPSO), the CC-PSO (centered PSO), CP-PSO (centered-progressive PSO), PP-PSO (progressive-progressive PSO) and RR-PSO (regressive-regressive PSO). Using the theory of stochastic differential and difference equations, we fully characterize the stability behavior of these algorithms. For well posed problems, a sufficient condition to achieve convergence is to select the PSO parameters close to the upper limit of second order stability. This result is also confirmed by numerical experimentation for different benchmark functions having an increasing degree of numerical difficulties. We also address how the discrete GPSO version (stability regions and trajectories) approaches the continuous PSO model as the time step decreases to zero. Finally, in the context of inverse problems, we address the question of how to select the appropriate PSO version: CP-PSO is the most explorative version and should be selected when we want to perform sampling of the posterior distribution of the inverse model parameters. Conversely, CC-PSO and GPSO provide higher convergence rates. Based on the analysis shown in this chapter, we can affirm that the PSO optimizers are not heuristic algorithms since there exist mathematical results that can be used to explain their consistency/convergence.

Keywords

particle swarm PSO continuous model GPSO CC-GPSO CP-GPSO stochastic stability analysis convergence 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Kennedy, J., Eberhart, R.: Particle swarm optimization. In: Proceedings of the IEEE International Conference on Neural Networks (ICNN 1995), Perth,WA, Australia, vol. 4, pp. 1942–1948 (November-December, 1995)Google Scholar
  2. 2.
    Ozcan, E., Mohan, C.K.: Analysis of a simple particle swarm optimization system. In: Intelligent Engineering Systems Through Artificial Neural Networks, vol. 8, pp. 253–258. ASME Press, St. Louis (1998)Google Scholar
  3. 3.
    Ozcan, E., Mohan, C.K.: Particle swarm optimization: surfing the waves. In: Proceedings of the IEEE Congress on Evolutionary Computation (CEC 1999), July 1999, vol. 3, pp. 1939–1944. IEEE Service Center, Washington (1999)CrossRefGoogle Scholar
  4. 4.
    Carlisle, A., Dozier, G.: 1001, An off-the-shelf PSO. In: Proceedings of The Workshop On particle Swarm Optimization, Indianapolis, USA (2001)Google Scholar
  5. 5.
    Clerc, M., Kennedy, J.: The particle swarm—explosion, stability, and convergence in a multidimensional complex space. IEEE Transactions on Evolutionary Computation 6(1), 58–73 (2002)CrossRefGoogle Scholar
  6. 6.
    Trelea, I.C.: The particle swarm optimization algorithm: convergence analysis and parameter selection. Information Processing Letters 85(6), 317–325 (2003)zbMATHCrossRefMathSciNetGoogle Scholar
  7. 7.
    Zheng, Y.-L., Ma, L.-H., Zhang, L.-Y., Qian, J.-X.: On the convergence analysis and parameter selection in particle swarm optimisation. In: Proceedings of the 2nd International Conference on Machine Learning and Cybernetics (ICMLC 2003), Xi’an, China, vol. 3, pp. 1802–1807 (November 2003)Google Scholar
  8. 8.
    van den Bergh, F., Engelbrecht, A.P.: A study of particle swarm optimization particle trajectories. Information Sciences 176(8), 937–971 (2006)zbMATHCrossRefMathSciNetGoogle Scholar
  9. 9.
    Poli, R.: Dynamics and Stability of the Sampling Distribution of Particle Swarm Optimisers via Moment Analysis. Journal of Artificial Evolution and Applications, Article ID 761459, 10 (2008), doi:10.1155/2008/761459Google Scholar
  10. 10.
    Fernández Martínez, J.L., García Gonzalo, E., Fernández Alvarez, J.P.: Theoretical analysis of particle swarm trajectories through a mechanical analogy. International Journal of Computational Intelligence Research 4(2), 93–104 (2008)CrossRefMathSciNetGoogle Scholar
  11. 11.
    Fernández Martínez, J.L., García Gonzalo, E.: The generalized PSO: a new door for PSO evolution. Journal of Artificial Evolution and Applications  Article ID 861275, 15 (2008), doi:10.1155/2008/861275Google Scholar
  12. 12.
    Fernández Martínez, J.L., García Gonzalo, E.: The PSO family: deduction, stochastic analysis and comparison. Special issue on PSO. Swarm Intelligence 3, 245–273 (2009), doi:10.1007/s11721-009-0034-8Google Scholar
  13. 13.
    Fernández Martínez, J. L., García Gonzalo, E.: Stochastic stability analysis of the linear continuous and discrete PSO models. Technical Report, Department of Mathematics, University of Oviedo, Spain (June 2009), Submitted to IEEE Transactions on Evolutionary Computation (2009)Google Scholar
  14. 14.
    García-Gonzalo, E., Fernández-Martínez, J. L.: The PP-GPSO and RR-GPSO. Technical Report. Department of Mathematics. University of Oviedo, Spain (December 2009b), Submitted to IEEE Transactions on Evolutionary Computation (2009)Google Scholar
  15. 15.
    Brandstätter, B., Baumgartner, U.: Particle swarm optimization: mass-spring system analogon. IEEE Transactions on Magnetics 38(2), 997–1000 (2002)CrossRefGoogle Scholar
  16. 16.
    Mikki, S.M., Kishk, A.A.: Physical theory for particle swarm optimisation. Progress in Electromagnetics Research 75, 171–207 (2007)CrossRefGoogle Scholar
  17. 17.
    Clerc, M.: Stagnation analysis in particle swarm optimisation or what happens when nothing happens, Tech. Rep. CSM-460, Department of Computer Science, University of Essex (August 2006)Google Scholar
  18. 18.
    Kadirkamanathan, V., Selvarajah, K., Fleming, P.J.: Stability analysis of the particle dynamics in particle swarm optimizer. IEEE Transactions on Evolutionary Computation 10(3), 245–255 (2006)CrossRefGoogle Scholar
  19. 19.
    Jiang, M., Luo, Y.P., Yang, S.Y.: Stochastic convergence analysis and parameter selection of the standard particle swarm optimization algorithm. Information Processing Letters 102, 8–16 (2007)zbMATHCrossRefMathSciNetGoogle Scholar
  20. 20.
    Poli, R., Broomhead, D.: Exact analysis of the sampling distribution for the canonical particle swarm optimiser and its convergence during stagnation. In: Proceedings of the 9th Genetic and Evolutionary Computation Conference (GECCO 2007), pp. 134–141. ACM Press, London (2007)CrossRefGoogle Scholar
  21. 21.
    Sun, J.Q.: Stochastic Dynamics and Control. In: Luo, A.C.J., Zaslavsky, G. (eds.) Monographs series on Nonlinear Science and Complexity, p. 410. Elsevier, Amsterdam (2006)Google Scholar
  22. 22.
    Lutes, L.D., Sarkani, S.: Random vibrations. In: Analysis of Structural and Mechanical Systems, p. 638. Elsevier, Amsterdam (2004)Google Scholar
  23. 23.
    García Gonzalo, E., Fernández Martínez, J.L.: Design of a simple and powerful Particle Swarm optimizer. In: Proceedings of the International Conference on Computational and Mathematical Methods in Science and Engineering, CMMSE 2009, Gijón, Spain (2009)Google Scholar
  24. 24.
    Fernández-Martínez, J.L., García-Gonzalo, E., Álvarez, J.P.F., Kuzma, H.A., Menéndez-Pérez, C.O.: PSO: A Powerful Algorithm to Solve Geophysical Inverse Problems. Application to a 1D-DC Resistivity Case. Jounal of Applied Geophysics (2010), doi:10.1016/j.jappgeo.2010.02.001Google Scholar
  25. 25.
    Fernández Martínez, J.L., García Gonzalo, E., Naudet, V.: Particle Swarm Optimization applied to the solving and appraisal of the Streaming Potential inverse problem. Geophysics (2010) special issue in Hydrogeophysics (accepted for publication)Google Scholar
  26. 26.
    Fernández Martínez, J.L., Fernández Álvarez, J.P., García Gonzalo, E., Menéndez Pérez, C.O., Kuzma, H.A.: Particle Swarm Optimization (PSO): a simple and powerful algorithm family for geophysical inversion: SEG Annual Meeting. SEG Expanded Abstracts 27, 3568 (2008)CrossRefGoogle Scholar
  27. 27.
    Fernández Martínez, J.L., Kuzma, H., García Gonzalo, E., Fernández Díaz, J.M., Fernández Alvarez, J.P., Menéndez Pérez, C.O.: Application of global optimization algorithms to a salt water intrusion problem. In: Symposium on the Application of Geophysics to Engineering and Environmental Problems, vol. 22, pp. 252–260 (2009a)Google Scholar
  28. 28.
    Fernández Martínez, J.L., Ciaurri, D.E., Mukerji, T., Gonzalo, E.G.: Application of Particle Swarm optimization to Reservoir Modeling and Inversion. In: Fernández Martínez, J.L. (ed.) International Association of Mathematical Geology (IAMG 2009), Stanford University (August 2009b)Google Scholar
  29. 29.
    Fernández Martínez, J.L., García Gonzalo, E., Fernández Muñiz, Z., Mukerji, T.: How to design a powerful family of Particle Swarm Optimizers for inverse modeling. New Trends on Bio-inspired Computation. Transactions of the Institute of Measurement and Control (2010c)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • J. L. Fernández-Martínez
    • 1
    • 2
    • 3
    • 4
  • E. García-Gonzalo
    • 4
  1. 1.Energy Resources DepartmentStanford UniversityPalo AltoUSA
  2. 2.Department of Civil and Environmental EngineeringUniversity of CaliforniaBerkeleyUSA
  3. 3.Berkeley-Lawrence Berkeley Lab.University of CaliforniaBerkeleyUSA
  4. 4.Department of MathematicsUniversity of OviedoOviedoSpain

Personalised recommendations