An Empirical Study of Parallel and Distributed Particle Swarm Optimization

  • Leonardo Vanneschi
  • Daniele Codecasa
  • Giancarlo Mauri
Part of the Studies in Computational Intelligence book series (SCI, volume 415)

Abstract

Given the implicitly parallel nature of population-based heuristics, many contributions reporting on parallel and distributed models and implementations of these heuristics have appeared so far. They range from the most natural and simple ones, i.e. fitness-level embarrassingly parallel implementations (where, for instance, each candidate solution is treated as an independent agent and evaluated on a dedicated processor), to many more sophisticated variously interacting multi-population systems. In the last few years, researchers have dedicated a growing attention to Particle Swarm Optimization (PSO), a bio-inspired population based heuristic inspired by the behavior of flocks of birds and shoals of fish, given its extremely simple implementation and its high intrinsical parallelism. Several parallel and distributed models of PSO have been recently defined, showing interesting performances both on benchmarks and real-life applications. In this chapter we report on four parallel and distributed PSO methods that have recently been proposed. They consist in a genetic algorithm whose individuals are co-evolving swarms, an “island model”- based multi-swarm system, where swarms are independent and interact by means of particle migrations at regular time steps, and their respective variants enriched by adding a repulsive component to the particles. We show that the proposed repulsive multi-swarm system has a better optimization ability than all the other presented methods on a set of hand-tailored benchmarks and complex real-life applications.

Keywords

Root Mean Square Error Particle Swarm Optimization Particle Swarm Optimization Method Standard Particle Swarm Optimization Particle Swarm Optimization Variant 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Archetti, F., Giordani, I., Vanneschi, L.: Genetic programming for anticancer therapeutic response prediction using the NCI-60 dataset. Computers and Operations Research 37(8), 1395–1405 (2010); Impact factor: 1.789MATHCrossRefGoogle Scholar
  2. 2.
    Archetti, F., Giordani, I., Vanneschi, L.: Genetic programming for QSAR investigation of docking energy. Applied Soft Computing 10(1), 170–182 (2010)CrossRefGoogle Scholar
  3. 3.
    Archetti, F., Messina, E., Lanzeni, S., Vanneschi, L.: Genetic programming for computational pharmacokinetics in drug discovery and development. Genetic Programming and Evolvable Machines 8(4), 17–26 (2007)CrossRefGoogle Scholar
  4. 4.
    Arumugam, M.S., Rao, M.: On the improved performances of the particle swarm optimization algorithms with adaptive parameters, cross-over operators and root mean square (rms) variants for computing optimal control of a class of hybrid systems. Journal of Applied Soft Computing 8, 324–336 (2008)CrossRefGoogle Scholar
  5. 5.
    Blackwell, T., Branke, J.: Multi-Swarm Optimization in Dynamic Environments. In: Raidl, G.R., Cagnoni, S., Branke, J., Corne, D.W., Drechsler, R., Jin, Y., Johnson, C.G., Machado, P., Marchiori, E., Rothlauf, F., Smith, G.D., Squillero, G. (eds.) EvoWorkshops 2004. LNCS, vol. 3005, pp. 489–500. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  6. 6.
    Bonabeau, E., Dorigo, M., Theraulaz, G.: Swarm Intelligence: From Natural to Artificial Systems. Oxford University Press, Santa Fe Institute Studies in the Sciences of Complexity, New York, NY (1999)MATHGoogle Scholar
  7. 7.
    Cagnoni, S., Vanneschi, L., Azzini, A., Tettamanzi, A.G.B.: A Critical Assessment of Some Variants of Particle Swarm Optimization. In: Giacobini, M., Brabazon, A., Cagnoni, S., Di Caro, G.A., Drechsler, R., Ekárt, A., Esparcia-Alcázar, A.I., Farooq, M., Fink, A., McCormack, J., O’Neill, M., Romero, J., Rothlauf, F., Squillero, G., Uyar, A.Ş., Yang, S. (eds.) EvoWorkshops 2008. LNCS, vol. 4974, pp. 565–574. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  8. 8.
    Clerc, M. (ed.): Particle Swarm Optimization. ISTE (2006)Google Scholar
  9. 9.
    Dioşan, L., Oltean, M.: Evolving the Structure of the Particle Swarm Optimization Algorithms. In: Gottlieb, J., Raidl, G.R. (eds.) EvoCOP 2006. LNCS, vol. 3906, pp. 25–36. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  10. 10.
    Fernández, F., Tomassini, M., Vanneschi, L.: An empirical study of multipopulation genetic programming. Genetic Programming and Evolvable Machines 4(1), 21–52 (2003)MATHCrossRefGoogle Scholar
  11. 11.
    Jiang, Y., Huang, W., Chen, L.: Applying multi-swarm accelerating particle swarm optimization to dynamic continuous functions. In: 2009 Second International Workshop on Knowledge Discovery and Data Mining, pp. 710–713 (2009)Google Scholar
  12. 12.
    Kameyama, K.: Particle swarm optimization - a survey. IEICE Transactions 92-D(7), 1354–1361 (2009)Google Scholar
  13. 13.
    Kennedy, J., Eberhart, R.: Particle swarm optimization. In: Proc. IEEE Int. conf. on Neural Networks, vol. 4, pp. 1942–1948. IEEE Computer Society (1995)Google Scholar
  14. 14.
    Kennedy, J., Mendes, R.: Population structure and particle swarm performance. In: IEEE Congress on Evolutionary Computation, CEC 2002, pp. 1671–1676. IEEE Computer Society (2002)Google Scholar
  15. 15.
    Kennedy, J., Poli, R., Blackwell, T.: Particle swarm optimisation: an overview. Swarm Intelligence 1(1), 33–57 (2007)CrossRefGoogle Scholar
  16. 16.
    Kennedy, J., Eberhart, R.C.: Swarm Intelligence. Morgan Kaufmann Publishers (2001)Google Scholar
  17. 17.
    Li, C., Yang, S.: Fast multi-swarm optimization for dynamic optimization problems. In: ICNC 2008: Proceedings of the 2008 Fourth International Conference on Natural Computation, pp. 624–628. IEEE Computer Society, Washington, DC (2008)CrossRefGoogle Scholar
  18. 18.
    Liang, J.J., Suganthan, P.N.: Dynamic multi-swarm particle swarm optimizer with local search. In: 2005 IEEE Congress on Evolutionary Computation, CEC 2005, vol. 1, pp. 522–528 (2005)Google Scholar
  19. 19.
    Niu, B., Zhu, Y., He, X., Wu, H.: MCPSO: A multi-swarm cooperative particle swarm optimizer. Applied Mathematics and Computation 2(185), 1050–1062 (2007)CrossRefGoogle Scholar
  20. 20.
    Poli, R.: Analysis of the publications on the applications of particle swarm optimisation. J. Artif. Evol. App. 2008, 3:1–3:10 (2008)Google Scholar
  21. 21.
    Poli, R.: Analysis of the publications on the applications of particle swarm optimisation. Journal of Artificial Evolution and Applications (2009) (in press)Google Scholar
  22. 22.
    N. C. M. Project. National Cancer Institute, Bethesda, MD (2008), http://genome-www.stanford.edu/nci60/
  23. 23.
    Riget, J., Vesterstrm, J.: A diversity-guided particle swarm optimizer - the arpso. Technical report, Dept. of Comput. Sci., Aarhus Univ., Denmark (2002)Google Scholar
  24. 24.
    Ross, S.M.: Introduction to Probability and Statistics for Engineers and Scientists. Academic Press, New York (2000)MATHGoogle Scholar
  25. 25.
    Ross, D.T., et al.: Systematic variation in gene expression patterns in human cancer cell lines. Nat. Genet. 24(3), 227–235 (2000)CrossRefGoogle Scholar
  26. 26.
    Sherf, U., et al.: A gene expression database for the molecular pharmacology of cancer. Nat. Genet. 24(3), 236–244 (2000)CrossRefGoogle Scholar
  27. 27.
    Shi, Y.H., Eberhart, R.: A modified particle swarm optimizer. In: Proc. IEEE Int. Conference on Evolutionary Computation, pp. 69–73. IEEE Computer Society (1998)Google Scholar
  28. 28.
    Srinivasan, D., Seow, T.H.: Particle swarm inspired evolutionary algorithm (ps-ea) for multi-objective optimization problem. In: IEEE Congress on Evolutionary Computation, CEC 2003, pp. 2292–2297. IEEE Press (2003)Google Scholar
  29. 29.
    Suganthan, P., Hansen, N., Liang, J., Deb, K., Chen, Y., Auger, A., Tiwari, S.: Problem definitions and evaluation criteria for the CEC 2005 special session on real-parameter optimization. Technical Report Number 2005005, Nanyang Technological University (2005)Google Scholar
  30. 30.
    Valle, Y.D., Venayagamoorthy, G., Mohagheghi, S., Hernandez, J., Harley, R.: Particle swarm optimization: Basic concepts, variants and applications in power systems. IEEE Transactions on Evolutionary Computation 12(2), 171–195 (2008)CrossRefGoogle Scholar
  31. 31.
    Vanneschi, L.: Theory and Practice for Efficient Genetic Programming. Ph.D. thesis, Faculty of Sciences. University of Lausanne, Switzerland (2004)Google Scholar
  32. 32.
    Vanneschi, L., Codecasa, D., Mauri, G.: An empirical comparison of parallel and distributed particle swarm optimization methods. In: Pelikan, M., Branke, J. (eds.) GECCO, pp. 15–22. ACM (2010)Google Scholar
  33. 33.
    Vanneschi, L., Codecasa, D., Mauri, G.: A study of parallel and distributed particle swarm optimization methods. In: Proceeding of the 2nd Workshop on Bio-Inspired Algorithms for Distributed Systems, BADS 2010, pp. 9–16. ACM, New York (2010)CrossRefGoogle Scholar
  34. 34.
    Vanneschi, L., Codecasa, D., Mauri, G.: A comparative study of four parallel and distributed PSO methods. New Generation Computing (2011) (to appear)Google Scholar
  35. 35.
    Wang, Y., Yang, Y.: An interactive multi-swarm pso for multiobjective optimization problems. Expert Systems with Applications (2008) (in press), http://www.sciencedirect.com (to appear)
  36. 36.
    Wu, Z., Zhou, J.: A self-adaptive particle swarm optimization algorithm with individual coefficients adjustment. In: Proc. IEEE International Conference on Computational Intelligence and Security, CIS 2007, pp. 133–136. IEEE Computer Society (2007)Google Scholar
  37. 37.
    You, X., Liu, S., Zheng, W.: Double-particle swarm optimization with induction-enhanced evolutionary strategy to solve constrained optimization problems. In: IEEE International Conference on Natural Computing, ICNC 2007, pp. 527–531. IEEE Computer Society (2007)Google Scholar
  38. 38.
    Zhigljavsky, A., Zilinskas, A.: Stochastic Global Optimization. Springer Optimization and Its Applications, vol. 9 (2008)Google Scholar
  39. 39.
    Zhiming, L., Cheng, W., Jian, L.: Solving contrained optimization via a modified genetic particle swarm optimization. In: Workshop on Knowledge Discovery and Data Mining, WKDD 2008, pp. 217–220. IEEE Computer Society (2008)Google Scholar

Copyright information

© Springer Berlin Heidelberg 2012

Authors and Affiliations

  • Leonardo Vanneschi
    • 1
  • Daniele Codecasa
    • 1
  • Giancarlo Mauri
    • 1
  1. 1.Department of Informatics, Systems and Communication (D.I.S.Co.)University of Milano-BicoccaMilanItaly

Personalised recommendations