Rigorous Runtime Analysis of Swarm Intelligence Algorithms – An Overview

  • Carsten Witt
Part of the Studies in Computational Intelligence book series (SCI, volume 242)


The theoretical runtime analysis of randomized search heuristics is an emerging research area where many results have been obtained in recent years. Most of these studies deal with evolutionary algorithms rather than swarm intelligence approaches such as ant colony optimization and particle swarm optimization. Despite the overwhelming practical success of swarm intelligence, the first runtime analyses of such approaches date only from 2006. Since then, however, significant progress has been made in analyzing the runtime of simple ACO and PSO variants. The aim of this chapter is to give an overview on existing studies in this area. Moreover, it is elaborated what direction the theory of swarm intelligence is likely to take, and the vision of a unified theory of randomized search heuristics is discussed.


Local Search Minimum Span Tree Swarm Intelligence Construction Graph Construction Procedure 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Motwani, R., Raghavan, P.: Randomized Algorithms. Cambridge University Press, Cambridge (1995)zbMATHGoogle Scholar
  2. 2.
    Mitzenmacher, M., Upfal, E.: Probability and Computing – Randomized Algorithms and Probabilistic Analysis. Cambridge University Press, Cambridge (2005)zbMATHGoogle Scholar
  3. 3.
    Mühlenbein, H.: How genetic algorithms really work: Mutation and hillclimbing. In: Proc. of Parallel Problem Solving from Nature II (PPSN 1992), pp. 15–26. Elsevier, Amsterdam (1992)Google Scholar
  4. 4.
    Droste, S., Jansen, T., Wegener, I.: On the analysis of the (1+1) evolutionary algorithm. Theoretical Computer Science 276, 51–81 (2002)zbMATHCrossRefMathSciNetGoogle Scholar
  5. 5.
    He, J., Yao, X.: A study of drift analysis for estimating computation time of evolutionary algorithms. Natural Computing 3, 21–35 (2004)zbMATHCrossRefMathSciNetGoogle Scholar
  6. 6.
    Oliveto, P.S., He, J., Yao, X.: Time complexity of evolutionary algorithms for combinatorial optimization: A decade of results. International Journal of Automation and Computing 4, 281–293 (2007)CrossRefGoogle Scholar
  7. 7.
    Dorigo, M., Stützle, T.: Ant Colony Optimization. MIT Press, Cambridge (2004)zbMATHGoogle Scholar
  8. 8.
    Gutjahr, W.J.: ACO algorithms with guaranteed convergence to the optimal solution. Information Processing Letters, 145–153 (2002)Google Scholar
  9. 9.
    Dorigo, M., Blum, C.: Ant colony optimization theory: A survey. Theoretical Computer Science 344, 243–278 (2005)zbMATHCrossRefMathSciNetGoogle Scholar
  10. 10.
    Gutjahr, W.J.: First steps to the runtime complexity analysis of Ant Colony Optimization. Computers and Operations Research 35, 2711–2727 (2008)zbMATHCrossRefGoogle Scholar
  11. 11.
    Neumann, F., Witt, C.: Runtime analysis of a simple ant colony optimization algorithm. In: Asano, T. (ed.) ISAAC 2006. LNCS, vol. 4288, pp. 618–627. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  12. 12.
    Gutjahr, W.J.: On the finite-time dynamics of ant colony optimization. Methodology and Computing in Applied Probability, 105–133 (2006)Google Scholar
  13. 13.
    Stützle, T., Hoos, H.H.: MAX-MIN ant system. Journal of Future Generations Computer Systems 16, 889–914 (2000)CrossRefGoogle Scholar
  14. 14.
    Doerr, B., Johannnsen, D.: Refined runtime analysis of a basic ant colony optimization algorithm. In: Proc. of the Congress on Evolutionary Computation (CEC 2007), pp. 501–507. IEEE Press, Los Alamitos (2007)CrossRefGoogle Scholar
  15. 15.
    Doerr, B., Neumann, F., Sudholt, D., Witt, C.: On the runtime analysis of the 1-ANT ACO algorithm. In: Proc. of the Genetic and Evolutionary Computation Conference (GECCO 2007), pp. 33–40. ACM Press, New York (2007)CrossRefGoogle Scholar
  16. 16.
    Hoeffding, W.: On the distribution of the number of successes in independent trials. Annals of Mathematical Statistics 27, 713–721 (1956)zbMATHCrossRefMathSciNetGoogle Scholar
  17. 17.
    Gleser, L.J.: On the distribution of the number of successes in independent trials. The Annals of Probability 3, 182–188 (1975)zbMATHCrossRefMathSciNetGoogle Scholar
  18. 18.
    Neumann, F., Witt, C.: Ant colony optimization and the minimum spanning tree problem. In: Maniezzo, V., Battiti, R., Watson, J.-P. (eds.) LION 2007 II. LNCS, vol. 5313, pp. 153–166. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  19. 19.
    Attiratanasunthron, N., Fakcharoenphol, J.: A running time analysis of an ant colony optimization algorithm for shortest paths in directed acyclic graphs. Information Processing Letters 105, 88–92 (2008)CrossRefMathSciNetGoogle Scholar
  20. 20.
    Neumann, F., Wegener, I.: Randomized local search, evolutionary algorithms, and the minimum spanning tree problem. Theoretical Computer Science 378, 32–40 (2007)zbMATHCrossRefMathSciNetGoogle Scholar
  21. 21.
    Broder, A.: Generating random spanning trees. In: Proc. of the 30th Annual Symposium on Foundations of Computer Science (FOCS 1989), pp. 442–447. IEEE Press, Los Alamitos (1989)Google Scholar
  22. 22.
    Gutjahr, W.J., Sebastiani, G.: Runtime analysis of ant colony optimization with best-so-far reinforcement. Methodology and Computing in Applied Probability 10, 409–433 (2008)CrossRefMathSciNetGoogle Scholar
  23. 23.
    Neumann, F., Sudholt, D., Witt, C.: Analysis of Different MMAS ACO Algorithms on Unimodal Functions and Plateaus. Swarm Intelligence 3, 35–68 (2009)CrossRefGoogle Scholar
  24. 24.
    Hoos, H.H., Stützle, T.: Stochastic Local Search: Foundations & Applications. Elsevier/Morgan Kaufmann, San Francisco (2004)Google Scholar
  25. 25.
    Balaprakash, P., Birattari, M., Stützle, T., Dorigo, M.: Incremental local search in ant colony optimization: Why it fails for the quadratic assignment problem. In: Dorigo, M., Gambardella, L.M., Birattari, M., Martinoli, A., Poli, R., Stützle, T. (eds.) ANTS 2006. LNCS, vol. 4150, pp. 156–166. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  26. 26.
    Neumann, F., Sudholt, D., Witt, C.: Rigorous analyses for the combination of ant colony optimization and local search. In: Dorigo, M., Birattari, M., Blum, C., Clerc, M., Stützle, T., Winfield, A.F.T. (eds.) ANTS 2008. LNCS, vol. 5217, pp. 132–143. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  27. 27.
    Kennedy, J., Eberhart, R.C.: Particle swarm optimization. In: Proc. of the IEEE International Conference on Neural Networks, pp. 1942–1948. IEEE Press, Los Alamitos (1995)CrossRefGoogle Scholar
  28. 28.
    Clerc, M.: Particle Swarm Optimization. ISTE (2006)Google Scholar
  29. 29.
    Kennedy, J., Eberhart, R.C., Shi, Y.: Swarm Intelligence. Morgan Kaufmann, San Francisco (2001)Google Scholar
  30. 30.
    Clerc, M., Kennedy, J.: The particle swarm - explosion, stability, and convergence in a multidimensional complex space. IEEE Transactions on Evolutionary Computation 6, 58–73 (2002)CrossRefGoogle Scholar
  31. 31.
    van den Bergh, F.: An Analysis of Particle Swarm Optimizers. PhD thesis, Department of Computer Science, University of Pretoria, South Africa (2002)Google Scholar
  32. 32.
    Trelea, I.C.: The particle swarm optimization algorithm: convergence analysis and parameter selection. Information Processing Letters 85, 317–325 (2003)zbMATHCrossRefMathSciNetGoogle Scholar
  33. 33.
    Jiang, M., Luo, Y.P., Yang, S.Y.: Stochastic convergence analysis and parameter selection of the standard particle swarm optimization algorithm. Information Processing Letters 102, 8–16 (2007)CrossRefMathSciNetGoogle Scholar
  34. 34.
    Sudholt, D., Witt, C.: Runtime analysis of binary PSO. In: Proc. of the Genetic and Evolutionary Computation Conference (GECCO 2008), pp. 135–142. ACM Press, New York (2008); extended version to appear in Theoretical Computer ScienceCrossRefGoogle Scholar
  35. 35.
    Kennedy, J., Eberhart, R.C.: A discrete binary version of the particle swarm algorithm. In: Proc. of the World Multiconference on Systemics, Cybernetics and Informatics (WMSCI), pp. 4104–4109 (1997)Google Scholar
  36. 36.
    Angeline, P.J.: Evolutionary optimization versus particle swarm optimization: Philosophy and performance differences. In: Porto, V.W., Waagen, D. (eds.) EP 1998. LNCS, vol. 1447, pp. 601–610. Springer, Heidelberg (1998)CrossRefGoogle Scholar
  37. 37.
    Kennedy, J.: Bare bones particle swarms. In: Proc. of the IEEE Swarm Intelligence Symposium, pp. 80–87 (2003)Google Scholar
  38. 38.
    Ratnaweera, A., Halgamuge, S., Watson, H.C.: Self-organizing hierarchical particle swarm optimizer with time-varying acceleration coefficients. IEEE Transactions on Evolutionary Computation 8, 240–255 (2004)CrossRefGoogle Scholar
  39. 39.
    Li, C., Liu, Y., Zhou, A., Kang, L., Wang, H.: A fast particle swarm optimization algorithm with Cauchy mutation and natural selection strategy. In: Kang, L., Liu, Y., Zeng, S. (eds.) ISICA 2007. LNCS, vol. 4683, pp. 334–343. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  40. 40.
    van den Bergh, F., Engelbrecht, A.P.: A new locally convergent particle swarm optimiser. In: Proc. of the IEEE International Conference on Systems, Man and Cybernetics (2002)Google Scholar
  41. 41.
    Witt, C.: Why standard particle swarm optimisers elude a theoretical runtime analysis. In: Proc. of Foundations of Genetic Algorithms 10, FOGA 2009 (to appear, 2009)Google Scholar
  42. 42.
    Jägersküpper, J.: Analysis of a simple evolutionary algorithm for minimization in Euclidean spaces. Theoretical Computer Science 379, 329–347 (2007)zbMATHCrossRefMathSciNetGoogle Scholar
  43. 43.
    Laumanns, M., Thiele, L., Zitzler, E.: Running time analysis of multiobjective evolutionary algorithms on pseudo-boolean functions. IEEE Transactions on Evolutionary Computation 8, 170–182 (2004)CrossRefGoogle Scholar
  44. 44.
    Brockhoff, D., Friedrich, T., Neumann, F.: Analyzing hypervolume indicator based algorithms. In: Rudolph, G., Jansen, T., Lucas, S., Poloni, C., Beume, N. (eds.) PPSN 2008. LNCS, vol. 5199, pp. 651–660. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  45. 45.
    Neumann, F., Wegener, I.: Can single-objective optimization profit from multiobjective optimization? In: Knowles, Corne, Deb (eds.) Multiobjective Problem Solving from Nature – From Concepts to Applications, pp. 115–130. Springer, Heidelberg (2007)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2009

Authors and Affiliations

  • Carsten Witt
    • 1
  1. 1.DTU Informatics, Technical University of DenmarkLyngbyDenmark

Personalised recommendations