Skip to main content
SpringerLink
Log in

Runtime Analysis for Permutation-based Evolutionary Algorithms

  • Published:
Algorithmica Aims and scope Submit manuscript

Abstract

While the theoretical analysis of evolutionary algorithms (EAs) has made significant progress for pseudo-Boolean optimization problems in the last 25 years, only sporadic theoretical results exist on how EAs solve permutation-based problems. To overcome the lack of permutation-based benchmark problems, we propose a general way to transfer the classic pseudo-Boolean benchmarks into benchmarks defined on sets of permutations. We then conduct a rigorous runtime analysis of the permutation-based \((1+1)\) EA proposed by Scharnow et al. (J Math Model Algorithm 3:349–366, 2004) on the analogues of the LeadingOnes and Jump benchmarks. The latter shows that, different from bit-strings, it is not only the Hamming distance that determines how difficult it is to mutate a permutation \(\sigma \) into another one \(\tau \), but also the precise cycle structure of \(\sigma \tau ^{-1}\). For this reason, we also regard the more symmetric scramble mutation operator. We observe that it not only leads to simpler proofs, but also reduces the runtime on jump functions with odd jump size by a factor of \(\Theta (n)\). Finally, we show that a heavy-tailed version of the scramble operator, as in the bit-string case, leads to a speed-up of order \(m^{\Theta (m)}\) on jump functions with jump size m. A short empirical analysis confirms these findings, but also reveals that small implementation details like the rate of void mutations can make an important difference.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Algorithm 1
Algorithm 2
Fig. 1
Fig. 2

Similar content being viewed by others

Notes

  1. The precise definition of a local optimum depends on a neighborhood structure. We omit the formal details since for most of this text, an informal understanding of local optima is sufficient. For jump functions with gap parameter m, we say that an \(x \in \{0,1\}^n\) with \(\Vert x\Vert _1=n-m\) (in the case of bit-string representations) and a \(\sigma \in S_n\) with exactly \(n-m\) fixed points (in the case of permutation representations) is called a local optimum.

  2. The change from the natural value k to \(k+1\) was done in [68] because for the problems regarded there, a mutation operation that returns the parent, that is, the application of \(k=0\) elementary mutations, cannot be profitable. It is easy to see, however, that all results in [68] remain valid when using k elementary mutations as mutation operator.

References

  1. Antipov, D., Buzdalov, M., Doerr, B.: First steps towards a runtime analysis when starting with a good solution. In: Parallel Problem Solving From Nature, PPSN 2020, Part II, pp. 560–573. Springer (2020)

  2. Antipov, D., Buzdalov, M., Doerr, B.: Lazy parameter tuning and control: choosing all parameters randomly from a power-law distribution. In: Genetic and Evolutionary Computation Conference, GECCO 2021, pp. 1115–1123. ACM (2021)

  3. Antipov, D., Buzdalov, M., Doerr, B.: Fast mutation in crossover-based algorithms. Algorithmica 84, 1724–1761 (2022)

    Article  MathSciNet  Google Scholar 

  4. Auger, A., Doerr, B. (eds.): Theory of Randomized Search Heuristics. World Scientific Publishing (2011)

  5. Antipov, D., Doerr, B.: Runtime analysis of a heavy-tailed \({(1+(\lambda , \lambda ))}\) genetic algorithm on jump functions. In: Parallel Problem Solving From Nature, PPSN 2020, Part II, pp. 545–559. Springer (2020)

  6. Antipov, D., Doerr, B., Karavaev, V.: A rigorous runtime analysis of the \({(1 + (\lambda,\lambda ))}\) GA on jump functions. Algorithmica 84, 1573–1602 (2022)

    Article  MathSciNet  Google Scholar 

  7. Bassin, A., Buzdalov, M.: The \((1+(\lambda ,\lambda ))\) genetic algorithm for permutations. In: Genetic and Evolutionary Computation Conference, GECCO 2020, Companion, pp. 1669–1677. ACM (2020)

  8. Benbaki, R., Benomar, Z., Doerr, B.: A rigorous runtime analysis of the 2-MMAS\(_{\rm ib}\) on jump functions: ant colony optimizers can cope well with local optima. In: Genetic and Evolutionary Computation Conference, GECCO 2021, pp. 4–13. ACM (2021)

  9. Böttcher, S., Doerr, B., Neumann, F.: Optimal fixed and adaptive mutation rates for the LeadingOnes problem. In: Parallel Problem Solving from Nature, PPSN 2010, pp. 1–10. Springer (2010)

  10. Corus, D., Dang, D.-C., Eremeev, A.V., Lehre, P.K.: Level-based analysis of genetic algorithms and other search processes. IEEE Trans. Evol. Comput. 22, 707–719 (2018)

    Article  Google Scholar 

  11. Corus, D., Lehre, P.K., Neumann, F., Pourhassan, M.: A parameterised complexity analysis of bi-level optimisation with evolutionary algorithms. Evol. Comput. 24, 183–203 (2016)

    Article  Google Scholar 

  12. Corus, D., Oliveto, P.S., Yazdani, D.: Automatic adaptation of hypermutation rates for multimodal optimisation. In: Foundations of Genetic Algorithms, FOGA 2021, pp. 4:1–4:12. ACM (2021)

  13. Do, A.V., Bossek, J., Neumann, A., Neumann, F.: Evolving diverse sets of tours for the travelling salesperson problem. In: Genetic and Evolutionary Computation Conference, GECCO 2020, pp. 681–689. ACM (2020)

  14. Doerr, B., Doerr, C., Kötzing, T.: Unbiased black-box complexities of jump functions. Evol. Comput. 23, 641–670 (2015)

    Article  Google Scholar 

  15. Dang, D.-C., Eremeev, A.V., Lehre, P.K., Qin, X.: Fast non-elitist evolutionary algorithms with power-law ranking selection. In: Genetic and Evolutionary Computation Conference, GECCO 2022, pp. 1372–1380. ACM (2022)

  16. Dang, D.-C., Friedrich, T., Kötzing, T., Krejca, M.S., Lehre, P.K., Oliveto, P.S., Sudholt, D., Sutton, A.M.: Escaping local optima with diversity mechanisms and crossover. In: Genetic and Evolutionary Computation Conference, GECCO 2016, pp. 645–652. ACM (2016)

  17. Dang, D.-C., Friedrich, T., Kötzing, T., Krejca, M.S., Lehre, P.K., Oliveto, P.S., Sudholt, D., Sutton, A.M.: Escaping local optima using crossover with emergent diversity. IEEE Trans. Evol. Comput. 22, 484–497 (2018)

    Article  Google Scholar 

  18. Doerr, B., Goldberg, L.A.: Adaptive drift analysis. Algorithmica 65, 224–250 (2013)

    Article  MathSciNet  Google Scholar 

  19. Doerr, B., Ghannane, Y., Brahim, M.I.: Towards a stronger theory for permutation-based evolutionary algorithms. In: Genetic and Evolutionary Computation Conference, GECCO 2022, pp. 1390–1398. ACM (2022)

  20. Do, A.V., Guo, M., Neumann, A., Neumann, F.: Analysis of evolutionary diversity optimisation for permutation problems. In: Genetic and Evolutionary Computation Conference, GECCO 2021, pp. 574–582. ACM (2021)

  21. Doerr, B., Happ, E.: Directed trees: A powerful representation for sorting and ordering problems. In: Congress on Evolutionary Computation, CEC 2008, pp. 3606–3613. IEEE (2008)

  22. Doerr, B., Hebbinghaus, N., Neumann, F.: Speeding up evolutionary algorithms through asymmetric mutation operators. Evol. Comput. 15, 401–410 (2007)

    Article  Google Scholar 

  23. Doerr, B., Johannsen, D.: Adjacency list matchings: an ideal genotype for cycle covers. In: Genetic and Evolutionary Computation Conference, GECCO 2007, pp. 1203–1210. ACM (2007)

  24. Droste, S., Jansen, T., Wegener, I.: On the analysis of the (1+1) evolutionary algorithm. Theoret. Comput. Sci. 276, 51–81 (2002)

    Article  MathSciNet  Google Scholar 

  25. Doerr, B., Johannsen, D., Winzen, C.: Non-existence of linear universal drift functions. Theoret. Comput. Sci. 436, 71–86 (2012)

    Article  MathSciNet  Google Scholar 

  26. Doerr, B., Kötzing, T.: Lower bounds from fitness levels made easy. In: Genetic and Evolutionary Computation Conference, GECCO 2021, pp. 1142–1150. ACM (2021)

  27. Doerr, B., Kötzing, T.: Multiplicative up-drift. Algorithmica 83, 3017–3058 (2021)

    Article  MathSciNet  Google Scholar 

  28. Doerr, B., Klein, C., Storch, T.: Faster evolutionary algorithms by superior graph representation. In: Foundations of Computational Intelligence, FOCI 2007, pp. 245–250. IEEE (2007)

  29. Dang, D.-C., Lehre, P.K.: Runtime analysis of non-elitist populations: from classical optimisation to partial information. Algorithmica 75, 428–461 (2016)

    Article  MathSciNet  Google Scholar 

  30. Doerr, B., Le, H.P., Makhmara, R., Nguyen, T.D.: Fast genetic algorithms. In: Genetic and Evolutionary Computation Conference, GECCO 2017, pp. 777–784. ACM (2017)

  31. Doerr, B., Neumann, F. (eds.): Theory of Evolutionary Computation—Recent Developments in Discrete Optimization. Springer (2020). Also available at http://www.lix.polytechnique.fr/Labo/Benjamin.Doerr/doerr_neumann_book.html

  32. Doerr, B.: The runtime of the compact genetic algorithm on Jump functions. Algorithmica 83, 3059–3107 (2021)

    Article  MathSciNet  Google Scholar 

  33. Doerr, B.: Does comma selection help to cope with local optima? Algorithmica 84, 1659–1693 (2022)

    Article  MathSciNet  Google Scholar 

  34. Doerr, B., Zhongdi, Q.: A first runtime analysis of the NSGA-II on a multimodal problem. Trans. Evolut. Comput. (2023). https://doi.org/10.1109/TEVC.2023.3250552

    Article  Google Scholar 

  35. Doerr, B., Rajabi, A.: Stagnation detection meets fast mutation. Theoret. Comput. Sci. 946, 113670 (2023)

    Article  MathSciNet  Google Scholar 

  36. Doerr, B., Zheng, W.: Theoretical analyses of multi-objective evolutionary algorithms on multi-modal objectives. In: Conference on Artificial Intelligence, AAAI 2021, pp. 12293–12301. AAAI Press (2021)

  37. Eiben, A.E., Smith, J.E.: Introduction to Evolutionary Computing, 2nd edn. Springer (2015)

  38. Friedrich, T., Göbel, A., Quinzan, F., Wagner, M.: Heavy-tailed mutation operators in single-objective combinatorial optimization. In: Parallel Problem Solving from Nature, PPSN 2018, Part I, pp. 134–145. Springer (2018)

  39. Friedrich, T., Quinzan, F., Wagner, M.: Escaping large deceptive basins of attraction with heavy-tailed mutation operators. In: Genetic and Evolutionary Computation Conference, GECCO 2018, pp. 293–300. ACM (2018)

  40. Gavenciak, T., Geissmann, B., Lengler, J.: Sorting by swaps with noisy comparisons. Algorithmica 81, 796–827 (2019)

    Article  MathSciNet  Google Scholar 

  41. Garnier, J., Kallel, L., Schoenauer, M.: Rigorous hitting times for binary mutations. Evol. Comput. 7, 173–203 (1999)

    Article  Google Scholar 

  42. Hevia Fajardo, M.A., Sudholt, D.: Self-adjusting offspring population sizes outperform fixed parameters on the cliff function. In: Foundations of Genetic Algorithms, FOGA 2021, pp. 5:1–5:15. ACM (2021)

  43. Hasenöhrl, V., Sutton, A.M.: On the runtime dynamics of the compact genetic algorithm on jump functions. In: Genetic and Evolutionary Computation Conference, GECCO 2018, pp. 967–974. ACM (2018)

  44. He, J., Yao, X.: Drift analysis and average time complexity of evolutionary algorithms. Artif. Intell. 127, 51–81 (2001)

    Article  MathSciNet  Google Scholar 

  45. Jägersküpper, J.: Combining Markov-chain analysis and drift analysis - the (1+1) evolutionary algorithm on linear functions reloaded. Algorithmica 59, 409–424 (2011)

    Article  MathSciNet  Google Scholar 

  46. Jansen, T.: Analyzing Evolutionary Algorithms–The Computer Science Perspective. Springer (2013)

  47. Jägersküpper, J., Storch, T.: When the plus strategy outperforms the comma strategy and when not. In: Foundations of Computational Intelligence, FOCI 2007, pp. 25–32. IEEE (2007)

  48. Jansen, T., Wegener, I.: The analysis of evolutionary algorithms: a proof that crossover really can help. Algorithmica 34, 47–66 (2002)

    Article  MathSciNet  Google Scholar 

  49. Jansen, T., Zarges, C.: Analysis of evolutionary algorithms: from computational complexity analysis to algorithm engineering. In: Hans-Georg B., Langdon, W.B. (eds.) Foundations of Genetic Algorithms, FOGA 2011, pp. 1–14. ACM (2011)

  50. Lehre, P.K.: Negative drift in populations. In: Parallel Problem Solving from Nature, PPSN 2010, pp. 244–253. Springer (2010)

  51. Lengler, J.: Drift analysis. In: Doerr, B., Neumann, F. (eds.) Theory of Evolutionary Computation: Recent Developments in Discrete Optimization, pp. 89–131. Springer (2020). Also available at https://arxiv.org/abs/1712.00964

  52. Martínez, C., Panholzer, A., Prodinger, H.: Generating random derangements. In: Workshop on Analytic Algorithmics and Combinatorics, ANALCO 2008, pp. 234–240. SIAM (2008)

  53. Mühlenthaler, M., Raß, A., Schmitt, M., Wanka, R.: Exact Markov chain-based runtime analysis of a discrete particle swarm optimization algorithm on sorting and OneMax. Nat. Comput. 21, 651–677 (2022)

    Article  MathSciNet  Google Scholar 

  54. Neumann, F.: Expected runtimes of evolutionary algorithms for the Eulerian cycle problem. Comput. OR 35, 2750–2759 (2008)

    Article  MathSciNet  Google Scholar 

  55. Nallaperuma, S., Neumann, F., Sudholt, D.: Expected fitness gains of randomized search heuristics for the traveling salesperson problem. Evol. Comput. 25, 673–705 (2017)

    Article  Google Scholar 

  56. Neumann, F., Witt, C.: Bioinspired Computation in Combinatorial Optimization—Algorithms and Their Computational Complexity. Springer (2010)

  57. OEIS Foundation Inc. The On-Line Encyclopedia of Integer Sequences, (2022). Published electronically at http://oeis.org

  58. Quinzan, F., Göbel, A., Wagner, M., Friedrich, T.: Evolutionary algorithms and submodular functions: benefits of heavy-tailed mutations. Nat. Comput. 20, 561–575 (2021)

    Article  MathSciNet  Google Scholar 

  59. Rowe, J.E.: Aishwaryaprajna: The benefits and limitations of voting mechanisms in evolutionary optimisation. In: Foundations of Genetic Algorithms, FOGA 2019, pp. 34–42. ACM (2019)

  60. de Montmort, P.R.: Essay d’analyse sur les jeux de hazard, 2nd edn. Quillau, Paris (1713)

  61. Rowe, J.E., Sudholt, D.: The choice of the offspring population size in the \({(1,\lambda )}\) evolutionary algorithm. Theoret. Comput. Sci. 545, 20–38 (2014)

    Article  MathSciNet  Google Scholar 

  62. Rudolph, G.: Convergence Properties of Evolutionary Algorithms. Verlag Dr, Kovǎc (1997)

  63. Rajabi, A., Witt, C.: Stagnation detection in highly multimodal fitness landscapes. In: Genetic and Evolutionary Computation Conference, GECCO 2021, pp. 1178–1186. ACM (2021)

  64. Rajabi, A., Witt, C.: Self-adjusting evolutionary algorithms for multimodal optimization. Algorithmica 84, 1694–1723 (2022)

    Article  MathSciNet  Google Scholar 

  65. Rajabi, A., Witt, C.: Stagnation detection with randomized local search. Evol. Comput. 31, 1–29 (2023)

    Article  Google Scholar 

  66. Sutton, A.M., Neumann, F.: A parameterized runtime analysis of evolutionary algorithms for the Euclidean traveling salesperson problem. In: AAAI Conference on Artificial Intelligence, AAAI 2012, pp. 1105–1111. AAAI Press (2012)

  67. Sutton, A.M., Neumann, F., Nallaperuma, S.: Parameterized runtime analyses of evolutionary algorithms for the planar Euclidean traveling salesperson problem. Evol. Comput. 22, 595–628 (2014)

    Article  Google Scholar 

  68. Scharnow, J., Tinnefeld, K., Wegener, I.: The analysis of evolutionary algorithms on sorting and shortest paths problems. J. Math. Model. Algorithms 3, 349–366 (2004)

    Article  MathSciNet  Google Scholar 

  69. Sudholt, D.: A new method for lower bounds on the running time of evolutionary algorithms. IEEE Trans. Evol. Comput. 17, 418–435 (2013)

    Article  Google Scholar 

  70. Wegener, I.: Theoretical aspects of evolutionary algorithms. In: Automata, Languages and Programming, ICALP 2001, pp. 64–78. Springer (2001)

  71. Wegener, I.: Simulated annealing beats Metropolis in combinatorial optimization. In: Automata, Languages and Programming, ICALP 2005, pp. 589–601. Springer (2005)

  72. Wu, M., Qian, C., Tang, K.: Dynamic mutation based Pareto optimization for subset selection. In; Intelligent Computing Methodologies, ICIC 2018, Part III, pp. 25–35. Springer (2018)

  73. Whitley, D. Varadarajan, S., Hirsch, R., Mukhopadhyay, A.: Exploration and exploitation without mutation: solving the jump function in \({\Theta (n)}\) time. In: Parallel Problem Solving from Nature, PPSN 2018, Part II, pp. 55–66. Springer (2018)

Download references

Acknowledgements

This work was supported by a public grant as part of the Investissements d’avenir project, reference ANR-11-LABX-0056-LMH, LabEx LMH.

Author information

Authors and Affiliations

  1. Laboratoire d’Informatique (LIX), CNRS, École Polytechnique, Institut Polytechnique de Paris, Palaiseau, France

    Benjamin Doerr, Yassine Ghannane & Marouane Ibn Brahim

Authors
  1. Benjamin Doerr
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yassine Ghannane
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Marouane Ibn Brahim
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Benjamin Doerr.

Ethics declarations

Conflicts of Interest

The authors have no conflicts of interest with regard to this work.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Extended version of a paper that appeared in the proceedings of GECCO 2022 [19]. This version contains all proofs that were omitted in [19] for reasons of space. It contains as new results the tight runtime estimates for the permutation-based LeadingOnes problem when using the classic or the heavy-tailed scramble operator. It also contains a section with experimental results, which includes an analysis of the probability that the four mutation operators discussed in this work generate an offspring equal to the parent.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Doerr, B., Ghannane, Y. & Ibn Brahim, M. Runtime Analysis for Permutation-based Evolutionary Algorithms. Algorithmica 86, 90–129 (2024). https://doi.org/10.1007/s00453-023-01146-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00453-023-01146-8

Keywords

Search

Navigation