Abstract
Particle swarm optimization (PSO) has previously been parallelized primarily by distributing the computation corresponding to particles across multiple processors. In these approaches, the only benefit of additional processors is an increased swarm size. However, in many cases this is not efficient when scaled to very large swarm sizes (on very large clusters). Current methods cannot answer well the question: “How can 1000 processors be fully utilized when 50 or 100 particles is the most efficient swarm size?” In this paper we attempt to answer that question with a speculative approach to the parallelization of PSO that we refer to as SEPSO.
In our approach, we refactor PSO such that the computation needed for iteration t+1 can be done concurrently with the computation needed for iteration t. Thus we can perform two iterations of PSO at once. Even with some amount of wasted computation, we show that this approach to parallelization in PSO often outperforms the standard parallelization of simply adding particles to the swarm. SEPSO produces results that are exactly equivalent to PSO; that is, SEPSO is a new method of parallelization and not a new PSO algorithm or variant.
However, given this new parallelization model, we can relax the requirement of exactly reproducing PSO in an attempt to produce better results. We present several such relaxations, including keeping the best speculative position evaluated instead of the one corresponding to the standard behavior of PSO, and speculating several iterations ahead instead of just one. We show that these methods dramatically improve the performance of parallel PSO in many cases, giving speed ups of up to six compared to previous parallelization techniques.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Belal, M., & El-Ghazawi, T. (2004). Parallel models for particle swarm optimizers. International Journal of Intelligent Computing and Information Sciences, 4(1), 100–111.
Beyer, H., & Schwefel, H. (2002). Evolution strategies—A comprehensive introduction. Natural Computing, 1(1), 3–52.
Bratton, D., & Kennedy, J. (2007). Defining a standard for particle swarm optimization. In Proceedings of the IEEE swarm intelligence symposium (pp. 120–127). Piscataway: IEEE Press.
Chang, J.-F., Chu, S.-C., Roddick, J. F., & Pan, J.-S. (2005). A parallel particle swarm optimization algorithm with communication strategies. Journal of Information Science and Engineering, 21, 809–818.
Chu, S.-C., & Pan, J.-S. (2006). Intelligent parallel particle swarm optimization algorithms. In Parallel evolutionary computations (p. 159–175).
Clerc, M., & Kennedy, J. (2002). The particle swarm—Explosion, stability, and convergence in a multidimensional complex space. IEEE Transactions on Evolutionary Computation, 6(1), 58–73.
Grama, A., Gupta, A., Karypis, G., & Kumar, V. (2003). Introduction to parallel computing (2nd ed.). Harlow: Addison-Wesley.
Herrera, F., Lozano, M., & Molina, D. (2010). Test suite for the special issue of soft computing on scalability of evolutionary algorithms and other metaheuristics for large scale continuous optimization problems (Tech. rep.). University of Granada.
Hsieh, S.-T., Sun, T.-Y., Liu, C.-C., & Tsai, S.-J. (2009). Efficient population utilization strategy for particle swarm optimizer. IEEE Transactions on Systems, Man, and Cybernetics. Part B. Cybernetics, 39(2), 444–456.
Jin, N., & Rahmat-Samii, Y. (2005). Parallel particle swarm optimization and finite-difference time-domain (PSO/FDTD) algorithm for multiband and wide-band patch antenna designs. IEEE Transactions on Antennas and Propagation, 53(11), 3459–3468.
Kennedy, J. (2003). Bare bones particle swarms. In Proceedings of the IEEE swarm intelligence symposium, 2003 (pp. 80–87). Piscataway: IEEE Press.
Kennedy, J., & Eberhart, R. C. (1995). Particle swarm optimization. In International conference on neural networks IV (pp. 1942–1948). Piscataway: IEEE Press.
Koh, B.-I., George, A. D., Haftka, R. T., & Fregly, B. J. (2006). Parallel asynchronous particle swarm optimization. International Journal for Numerical Methods in Engineering, 67, 578–595.
Li, J., Wang, X., He, R., & Chi, Z. (2007). An efficient fine-grained parallel genetic algorithm based on gpu-accelerated. In Network and parallel computing workshops, 2007. NPC workshops. IFIP international conference on (pp. 855–862). Piscataway: IEEE Press.
McNabb, A., Gardner, M., & Seppi, K. (2009). An exploration of topologies and communication in large particle swarms. In Proceedings of the IEEE congress on evolutionary computation (pp. 712–719). Piscataway: IEEE Press.
McNabb, A. W., Monson, C. K., & Seppi, K. D. (2007). Parallel PSO using MapReduce. In Proceedings of the IEEE congress on evolutionary computation (pp. 7–14). Piscataway: IEEE Press.
Mendes, R. (2004). Population topologies and their influence in particle swarm performance. Ph.D. thesis, Escola de Engenharia, Universidade do Minho, Guimaraes, Portugal.
Mendes, R., Kennedy, J., & Neves, J. (2004). The fully informed particle swarm: Simpler, maybe better. IEEE Transactions on Evolutionary Computation, 8(3), 204–210.
Monson, C. K., & Seppi, K. D. (2005). Exposing origin-seeking bias in PSO. In Proceedings of the genetic and evolutionary computation conference (GECCO 2005) (Vol. 1, pp. 241–248). New York: ACM Press.
Montes de Oca, M. A., Stützle, T., Van den Enden, K., & Dorigo, M. (2010). Incremental social learning in particle swarms. IEEE Transactions on Systems, Man, and Cybernetics. Part B. Cybernetics, 41(2), 368–384.
Mostaghim, S., Branke, J., & Schmeck, H. (2006). Multi-objective particle swarm optimization on computer grids (Technical Report 502). AIFB Institute, Karlsruhe, Germany.
Parsopoulos, K. E., Tasoulis, D. K., & Vrahatis, M. N. (2004). Multiobjective optimization using parallel vector evaluated particle swarm optimization. In Proceedings of the IASTED international conference on artificial intelligence and applications (pp. 823–828). Calgary: IASTED/ACTA Press.
Poli, R. (2008a). Analysis of the publications on the applications of particle swarm optimisation. Journal of Artificial Evolution and Applications, 3, 1–10.
Poli, R. (2008b). Dynamics and stability of the sampling distribution of particle swarm optimisers via moment analysis. Journal of Artificial Evolution and Applications, 8(2), 1–10.
Poli, R., Kennedy, J., & Blackwell, T. (2007). Particle swarm optimization. Swarm Intelligence, 1(1), 33–57.
Rudolph, G. (1991). Global optimization by means of distributed evolution strategies. In Parallel problem solving from nature (pp. 209–213). Berlin: Springer.
Schutte, J. F., Reinbolt, J. A., Fregly, B. J., Haftka, R. T., & George, A. D. (2004). Parallel global optimization with the particle swarm algorithm. International Journal for Numerical Methods in Engineering, 61(13), 2296–2315.
Scriven, I., Ireland, D., Lewis, A., Mostaghim, S., & Branke, J. (2008a). Asynchronous multiple objective particle swarm optimisation in unreliable distributed environments. In Proceedings of the IEEE congress on evolutionary computation (pp. 2481–2486). Piscataway: IEEE Press.
Scriven, I., Lewis, A., Ireland, D., & Lu, J. (2008b). Decentralised distributed multiple objective particle swarm optimisation using peer to peer networks. In Proceedings of the IEEE congress on evolutionary computation (pp. 2925–2928). Piscataway: IEEE Press.
Venter, G., & Sobieszczanski-Sobieski, J. (2005). A parallel particle swarm optimization algorithm accelerated by asynchronous evaluations. In Proceedings of the 6th world congresses of structural and multidisciplinary optimization. Berlin: Springer.
Witte, E., Chamberlain, R., & Franklin, M. (1991). Parallel simulated annealing using speculative computation. IEEE Transactions on Parallel and Distributed Systems, 2(4), 483–494.
Wolpert, D. H., & Macready, W. G. (1997). No free lunch theorems for optimization. IEEE Transactions on Evolutionary Computation, 1(1), 67–82.
Zhou, Y., & Tan, Y. (2009). GPU-based parallel particle swarm optimization. In Proceedings of the IEEE congress on evolutionary computation (pp. 1493–1500). Piscataway: IEEE Press.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Gardner, M., McNabb, A. & Seppi, K. A speculative approach to parallelization in particle swarm optimization. Swarm Intell 6, 77–116 (2012). https://doi.org/10.1007/s11721-011-0066-8
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11721-011-0066-8