Time complexity of evolutionary algorithms for combinatorial optimization: A decade of results

Article

Abstract

Computational time complexity analyzes of evolutionary algorithms (EAs) have been performed since the mid-nineties. The first results were related to very simple algorithms, such as the (1+1)-EA, on toy problems. These efforts produced a deeper understanding of how EAs perform on different kinds of fitness landscapes and general mathematical tools that may be extended to the analysis of more complicated EAs on more realistic problems. In fact, in recent years, it has been possible to analyze the (1+1)-EA on combinatorial optimization problems with practical applications and more realistic population-based EAs on structured toy problems. This paper presents a survey of the results obtained in the last decade along these two research lines. The most common mathematical techniques are introduced, the basic ideas behind them are discussed and their elective applications are highlighted. Solred problems that were still open are enumerated as are those still awaiting for a solution. New questions and problems arisen in the meantime are also considered.

Keywords

Evolutionary algorithms computational complexity combinatorial optimization evolutionary computation theory 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. [1]
    R. Sarker, M. Mohammadian, X. Yao, Evolutionary Optimization, Kluwer Academic Publishers, Norwell, MA, USA, 2002.MATHGoogle Scholar
  2. [2]
    D. E. Goldberg. Genetic Algorithms for Search, Optimization, and Machine Learning, Addison-Wesley, USA, 1989.Google Scholar
  3. [3]
    J. H. Holland. Adaptation in Natural and Artificial Systems, 2nd ed., MIT Press, Cambridge, MA, 1992.Google Scholar
  4. [4]
    A. E. Eiben, G. Rudolph. Theory of Evolutionary Algorithms: A Bird’s Eye View. Theoretical Computer Science, vol. 229, no.1–2, pp. 3–9, 1999.MATHCrossRefGoogle Scholar
  5. [5]
    T. Bäck, D. B. Fogel, Z. Michalewicz. Handbook of Evolutionary Computation, IOP Publishing Ltd, Bristol, UK, 1997.MATHGoogle Scholar
  6. [6]
    G. Rudolph. Convergence Analysis of Canonical Genetic Algorithms. IEEE Transactions on Neural Networks, vol. 5, no. 1, pp. 96–101, 1994.CrossRefGoogle Scholar
  7. [7]
    G. Rudolph. Finite Markov Chain Results in Evolutionary Computation: A Tour d’Horizon. Fundamenta Informaticae, vol. 35, no.1–4, pp. 67–89, 1998.MATHGoogle Scholar
  8. [8]
    H. Aytug, G. J. Koehler. Stopping Criteria for Finite Length Genetic Algorithms. ORSA Journal on Computing, vol. 8, no.2, pp. 183–191, 1996.MATHGoogle Scholar
  9. [9]
    D. Greenhalgh, S. Marshall. Convergence Criteria for Genetic Algorithms. SIAM Journal of Computing, vol. 30, no.1, pp. 269–282, 2000.MATHCrossRefGoogle Scholar
  10. [10]
    M. Safe, J. A. Carballido, I. Ponzoni, N. B. Brignole. On Stopping Criteria for Genetic Algorithms, In 17th Brazilian Symposium on Artificial Intelligence, Lecture Notes in Artificial Intelligence, Springer-Verlag, vol. 3171, pp. 405–413, 2004.Google Scholar
  11. [11]
    S. Droste, T. Jansen, I. Wegener. On the Analysis of the (1+1) Evolutionary Algorithm. Theoretical Computer Science, vol. 276, no. 1–2, pp. 51–81, 2002.MATHCrossRefGoogle Scholar
  12. [12]
    X. Yao. An Overview of Evolutionary Computation. Chinese Journal of Advanced Software Research, vol. 3, no. 1, pp. 12–29, 1996.Google Scholar
  13. [13]
    H. G. Beyer, H. G. Schwefel, I. Wegener. How to Analyse Evolutionary Algorithms. Theoretical Computer Science, vol. 287, no. 1, pp. 101–130, 2002.MATHCrossRefGoogle Scholar
  14. [14]
    I. Wegener. Methods for the Analysis of Evolutionary Algorithms on Pseudo-Boolean Functions. In Evolutionary Optimization, R. Sarker, M. Mohammadian, X. Yao (eds.), Kluwer Academic Publishers, Norwell, MA, USA, 2002.Google Scholar
  15. [15]
    S. Droste, T. Jansen, I. Wegener. A Rigorous Complexity Analysis of the (1+1) Evolutionary Algorithm for Separable Functions with Boolean Inputs. Evolutionary Computation, vol. 6, no. 2, pp. 185–196, 1998.Google Scholar
  16. [16]
    I. Wegener. On the Design and Analysis of Evolutionary Algorithms. In Proceedings of the Workshop on Algorithm Engineering as a New Paradigm, Kyoto University, Japan, pp. 37–47, 2000.Google Scholar
  17. [17]
    T. Jansen, I. Wegener. Evolutionary Algorithms: How to Cope with Plateaus of Constant Fitness and When to Reject Strings of the Same Fitness. IEEE Transactions on Evolutionary Computation, vol. 5, no. 6, pp. 589–599, 2000.CrossRefGoogle Scholar
  18. [18]
    M. Motwani, P. Raghavan. Randomized Algorithms. Cambridge University Press, UK, 1995.MATHGoogle Scholar
  19. [19]
    T. Bäck. Optimal Mutation Rates in Genetic Search. In Proceedings of the 5th International Conference on Genetic Algorithms, Morgan-Kaufman, pp. 2–8, 1993.Google Scholar
  20. [20]
    V. Cutello, G. Nicosia, P. S. Oliveto, M. Romeo. On the Convergence of Immune Algorithms. In Proceedings of the 1st IEEE Symposium on Foundations of Computational Intelligence, Honolulu, Hawaii, USA, 2007.Google Scholar
  21. [21]
    J. He, X. Yao. Towards an Analytic Framework for Analyzing the Computation Time of Evolutionary Algorithms. Artificial Intelligence, vol. 145, no. 1–2 pp. 59–97, 2003.MATHCrossRefGoogle Scholar
  22. [22]
    M. Iosifescu. Finite Markov Processes and Their Applications, John Wiley & Sons, 1980.Google Scholar
  23. [23]
    D. L. Isaacson, R. W. Madsen. Markov Chains: Theory and Applications, John Wiley & Sons, 1976.Google Scholar
  24. [24]
    J. He, X. Yao. From an Individual to a Population: An Analysis of the 1st Hitting Time of Population-based Evolutionary Algorithms. IEEE Transactions Evolutionary Computation, vol. 6, no. 5, pp. 495–511, 2002.CrossRefGoogle Scholar
  25. [25]
    W. Feller. An Introduction to Probability Theory and Its Applications, vol. 1, John Wiley & Sons, 1968.Google Scholar
  26. [26]
    I. Wegener. Theoretical Aspects of Evolutionary Algorithms. In Proceedings of the 28th International Colloquium on Automata, Languages and Programming, London, UK, pp. 64–78, 2001.Google Scholar
  27. [27]
    S. Droste, T. Jansen, I. Wegener. On the Optimization of Unimodal Functions with the (1 + 1) Evolutionary Algorithm. In Proceedings of the Parallel Problem Solving from Nature Conference, Lecture Notes in Computer Science, Springer-Verlag, vol. 1498, pp. 13–22, 1998.Google Scholar
  28. [28]
    O. Giel, I. Wegener. Evolutionary Algorithms and the Maximum Matching Problem. In Proceedings of the 20th Annual Symposium on Theoretical Aspects of Computer Science, Speringer-Verlag, pp. 415–426, 2003.Google Scholar
  29. [29]
    P. S. Oliveto, J. He, X. Yao. Evolutionary Algorithms and the Vertex Cover Problem. In Proceedings of the Congress on Evolutionary Computation, to be published.Google Scholar
  30. [30]
    C. Witt. Runtime Analysis of the (µ+1) EA on Simple pseudo-Boolean Functions. In Proceedings of the 8th Annual Conference on Genetic and Evolutionary Computation, Seattle, Washington, USA, pp. 651–658, 2006.Google Scholar
  31. [31]
    T. Jansen, I. Wegener. On the Analysis of Evolutionary Algorithms — A Proof that Crossover Really Can Help. In Proceedings of the 7th Annual European Symposium on Algorithms, Springer-Verlag, pp. 184–193, 1999.Google Scholar
  32. [32]
    J. He. Study on the Foundation of Evolutionary Computation. Technical Report, Department of Computer Science, Harbin Institute of Technology, China, 1998 (in Chinese).Google Scholar
  33. [33]
    J. He, X. Yao. Drift Analysis and Average Time Complexity of Evolutionary Algorithms. Artificial Intelligence, vol. 127, no. 1, pp. 57–85, 2001.MATHCrossRefGoogle Scholar
  34. [34]
    J. He, X. Yao. A Study of Drift Analysis for Estimating Computation Time of Evolutionary Algorithms. Natural Computing: An International Journal, vol. 3, no. 1, pp. 21–35, 2004.MATHCrossRefGoogle Scholar
  35. [35]
    T. Jansen, I. Wegener. On the Utility of Populations in Evolutionary Algorithms. In Proceedings of the 3rd Annual Conference on Genetic and Evolutionary Computation, IEEE, pp. 1034–1041, 2001.Google Scholar
  36. [36]
    T. Jansen, I. Wegener. Real Royal Road Functions: Where Crossover Provably is Essential. Discrete Applied Mathematics, vol. 149, no. 1–3, pp. 111–125, 2005.MATHCrossRefGoogle Scholar
  37. [37]
    C. Witt. Population Size vs. Runtime of a Simple EA. In Proceedings of the Congress on Evolutionary Computation, Canberra, pp. 1996–2003, 2003.Google Scholar
  38. [38]
    T. Storch. Real Royal Road Functions for Constant Population Size. Theoretical Computer Science, vol. 320, no. 1, pp. 123–134, 2004.MATHCrossRefGoogle Scholar
  39. [39]
    T. Jansen, K. A. De Jong, I. Wegener. On the Choice of the Offspring Population Size in Evolutionary Algorithms. Evolutionary Computation, vol. 13, no. 4, pp. 413–440, 2005.CrossRefGoogle Scholar
  40. [40]
    J. Jägerskupper, T. Storch. When the Plus Strategy Out-performs the Comma Strategy and When Not. In Proceedings of the 1st IEEE Symposium on Foundations of Computational Intelligence, Hawaii, USA, pp. 25–32, 2007.Google Scholar
  41. [41]
    M. R. Garey, D. S. Johnson. Computers and Intractability a Guide to the Theory of NP-Completeness, W. H. Freeman and Company, New York, 1979.MATHGoogle Scholar
  42. [42]
    J. Scharnow, K. Tinnefeld, I. Wegener. Fitness Landscapes Based on Sorting and Shortest Path Problems. In Proceedings of 7th Conference on Parallel Problem Solving from Nature Conference, pp. 54–63, 2002.Google Scholar
  43. [43]
    F. Neumann, I. Wegener. Minimum Spanning Trees Made Easier Via Multi-objective Optimization. Natural Computing, vol. 5, no. 3, pp. 305–319, 2006.MATHCrossRefGoogle Scholar
  44. [44]
    T. Friedrich, J. He, N. Hebbinghaus, F. Neumann, C. Witt. Approximating Covering Problems by Randomized Search Heuristics Using Multi-objective Models. In Proceedings of the 9th Annual Conference on Genetic and Evolutionary Computation, 2007, to be published.Google Scholar
  45. [45]
    F. Neumann. Expected Run-times of Evolutionary Algorithms for the Eulerian Cycle Problem. In Proceedings of the Congress on Evolutionary Computation, IEEE, pp. 904–910, 2004.Google Scholar
  46. [46]
    B. Doerr, N. Hebbinghaus, F. Neumann. Speeding Up Evolutionary Algorithms through Restricted Mutation Operators. In Proceedings of the 9th Parallel Problem Solving from Nature Conference, Leture Notes in Computer Science, Springer-Verlag, Reykjavik, Iceland, vol. 4193, pp. 978–987, 2006.CrossRefGoogle Scholar
  47. [47]
    B. Doerr, C. Klein, T. Storch. Faster Evolutionary Algorithms by Superior Graph Representation. In Proceedings of the 1st IEEE Symposium on Foundations of Computational Intelligence, Hawaii, USA, pp. 245–250, 2007.Google Scholar
  48. [48]
    B. Doerr, D. Johannsen. Adjacency List Matchings — An Ideal Genotype for Cycle Covers. In Proceedings of the Annual Conference on Genetic and Evolutionary Computation, 2007, to be published.Google Scholar
  49. [49]
    F. Neumann, I. Wegener. Randomized Local Search, Evolutionary Algorithms, and the Minimum Spanning Tree Problem. In Proceedings of Genetic and Evolutionary Computation Conference, Leture Notes in Computer Science, Springer-Verlag, Seattle, Washington, USA, vol. 3102, pp. 713–724, 2004.Google Scholar
  50. [50]
    C. Witt. Worst-case and Average-case Approximations by Simple Randomized Search Heuristics. In Proceedings of the 22nd Annual Symposium on Theoretical Aspects of Computer Science, Lectures Notes in Computer Science, Springer-Verlag, Stuttgart, Germany, vol. 3404, pp. 44–56, 2005.Google Scholar
  51. [51]
    D. S. Hochbaum. Approximation Algorithms for NP-Hard Problems, Wadsworth Publishing Company, USA, 1996.Google Scholar
  52. [52]
    T. Storch. How randomized search heuristics find maximum cliques in planar graphs. In Proceedings of the Annual Conference on Genetic and Evolutionary Computation, Seattle, Washington, USA, pp. 567–574, 2006.Google Scholar
  53. [53]
    T. Storch. Finding Large Cliques in Sparse Semi-random Graphs by Simple Randomised Search Heuristics, Technical Report No. CI-211/06, Fachbereich Informatik, Universitat Dortmund, 2006.Google Scholar
  54. [54]
    G. H. Sasaki, B. Hajek. The Time Complexity of Maximum Matching by Simulated Annealing. Journal of the Association of Computing Machinery, vol. 35, no. 2, pp. 387–403, 1988.Google Scholar

Copyright information

© Institute of Automation, Chinese Academy of Sciences 2007

Authors and Affiliations

  1. 1.The Center of Excellence for Research in Computational Intelligence and Applications (CERCIA), School of Computer ScienceUniversity of BirminghamEdgbaston, BirminghamUK

Personalised recommendations