Towards a Theory of Randomized Search Heuristics

  • Ingo Wegener
Part of the Lecture Notes in Computer Science book series (LNCS, volume 2747)


There is a well-developed theory about the algorithmic complexity of optimization problems. Complexity theory provides negative results which typically are based on assumptions like NP≠P or NP≠RP. Positive results are obtained by the design and analysis of clever algorithms. These algorithms are well-tuned for their specific domain. Practitioners, however, prefer simple algorithms which are easy to implement and which can be used without many changes for different types of problems. They report surprisingly good results when applying randomized search heuristics like randomized local search, tabu search, simulated annealing, and evolutionary algorithms. Here a framework for a theory of randomized search heuristics is presented. It is discussed how randomized search heuristics can be delimited from other types of algorithms. This leads to the theory of black-box optimization. Lower bounds in this scenario can be proved without any complexity-theoretical assumption. Moreover, methods how to analyze randomized search heuristics, in particular, randomized local search and evolutionary algorithms are presented.


Local Search Evolutionary Algorithm Problem Instance Search Point Randomize Search Heuristic 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Dietzfelbinger, M., Naudts, B., van Hoyweghen, C., Wegener, I.: The analysis of a recombinative hill-climber on H-IFF. Submitted for publication in IEEE Trans. on Evolutionary Computation (2002) Google Scholar
  2. 2.
    Droste, S., Jansen, T., Wegener, I.: Upper and lower bounds for randomized search heuristics in black-box optimization. Tech. Rep. Univ. Dortmund (2003) Google Scholar
  3. 3.
    Droste, S., Jansen, T., Wegener, I.: On the analysis of the (1+1) evolutionary algorithm. Theoretical Computer Science 276, 51–81 (2002)zbMATHCrossRefMathSciNetGoogle Scholar
  4. 4.
    Giel, O., Wegener, I.: Evolutionary algorithms and the maximum matching problem. In: Alt, H., Habib, M. (eds.) STACS 2003. LNCS, vol. 2607, pp. 415–426. Springer, Heidelberg (2003)CrossRefGoogle Scholar
  5. 5.
    Glover, F., Laguna, M.: Tabu search. In: Reeves, C.R. (ed.) Modern Heuristic Techniques for Combinatorial Problems, pp. 70–150. Blackwell, Oxford (1993)Google Scholar
  6. 6.
    Hajek, B.: Hitting-time and occupation-time bounds implied by drift analysis with applications. Advances in Applied Probability 14, 502–525 (1982)zbMATHCrossRefMathSciNetGoogle Scholar
  7. 7.
    He, J., Yao, X.: Drift analysis and average time complexity of evolutionary algorithms. Artificial Intelligence 127, 57–85 (2001)zbMATHCrossRefMathSciNetGoogle Scholar
  8. 8.
    Jansen, T., Wegener, I.: Real royal road functions — where crossover provably is essential. In: Proc. of 3rd Genetic and Evolutionary Computation Conf. (GECCO), pp. 375–382 (2001a)Google Scholar
  9. 9.
    Jansen, T., Wegener, I.: Evolutionary algorithms — how to cope with plateaus of constant fitness and when to reject strings of the same fitness. IEEE Trans. on Evolutionary Computation 5, 589–599 (2001b)CrossRefGoogle Scholar
  10. 10.
    Jansen, T., Wegener, I.: The analysis of evolutionary algorithms — a proof that crossover really can help. Algorithmica 34, 47–66 (2002)zbMATHCrossRefMathSciNetGoogle Scholar
  11. 11.
    Kirkpatrick, S., Gelatt, C.D., Vecchi, M.P.: Optimization by simulated annealing. Science 220, 671–680 (1983)CrossRefMathSciNetGoogle Scholar
  12. 12.
    Llewellyn, D.C., Tovey, C., Trick, M.: Local optimization on graphs. Discrete Applied Mathematics 23, 157–178 (1989)zbMATHCrossRefMathSciNetGoogle Scholar
  13. 13.
    Lovász, L., Naor, M., Newman, I., Wigderson, A.: Search problems in the decision tree model. In: Proc. of 32nd IEEE Symp. on Foundations of Computer Science (FOCS), pp. 576–585 (1991)Google Scholar
  14. 14.
    Papadimitriou, C.H., Schäffer, A.A., Yannakakis, M.: On the complexity of local search. In: Proc. of 22nd ACM Symp. on Theory of Computing (STOC), pp. 438–445 (1990)Google Scholar
  15. 15.
    Rabani, Y., Rabinovich, Y., Sinclair, A.: A computational view of population genetics. Random Structures and Algorithms 12, 314–330 (1998)CrossRefMathSciNetGoogle Scholar
  16. 16.
    Ranade, A.G.: How to emulate shared memory. Journal of Computer and System Sciences 42, 307–326 (1991)zbMATHCrossRefMathSciNetGoogle Scholar
  17. 17.
    Sasaki, G., Hajek, B.: The time complexity of maximum matching by simulated annealing. Journal of the ACM 35, 387–403 (1988)CrossRefMathSciNetGoogle Scholar
  18. 18.
    Scharnow, J., Tinnefeld, K., Wegener, I.: Fitness landscapes based on sorting and shortest paths problems. In: Guervós, J.J.M., Adamidis, P.A., Beyer, H.-G., Fernández-Villacañas, J.-L., Schwefel, H.-P. (eds.) PPSN 2002. LNCS, vol. 2439, pp. 54–63. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  19. 19.
    Wegener, I.: Theoretical aspects of evolutionary algorithms. In: Orejas, F., Spirakis, P.G., van Leeuwen, J. (eds.) ICALP 2001. LNCS, vol. 2076, pp. 64–78. Springer, Heidelberg (2001)CrossRefGoogle Scholar
  20. 20.
    Wegener, I., Witt, C.: On the optimization of monotone polynomials by simple randomized search heuristics. Combinatorics, Probability and Computing (2003) (to appear) Google Scholar
  21. 21.
    Yao, A.C.: Probabilistic computations: Towards a unified measure of complexity. In: Proc. of 17th IEEE Symp. on Foundations of Computer Science (FOCS), pp. 222–227 (1977)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2003

Authors and Affiliations

  • Ingo Wegener
    • 1
  1. 1.FB Informatik, LS 2Univ. DortmundDortmundGermany

Personalised recommendations