, Volume 59, Issue 3, pp 343–368 | Cite as

Hybridizing Evolutionary Algorithms with Variable-Depth Search to Overcome Local Optima

  • Dirk Sudholt


Hybridizing evolutionary algorithms with local search has become a popular trend in recent years. There is empirical evidence for various combinatorial problems where hybrid evolutionary algorithms perform better than plain evolutionary algorithms. Due to the rapid development of a highly active field of research, theory lags far behind and a solid theoretical foundation of hybrid metaheuristics is sorely needed.

We are aiming at a theoretical understanding of why and when hybrid evolutionary algorithms are successful in combinatorial optimization. To this end, we consider a hybrid of a simple evolutionary algorithm, the (1+1) EA, with a powerful local search operator known as variable-depth search (VDS) or Kernighan-Lin. Three combinatorial problems are investigated: Mincut, Knapsack, and Maxsat. More precisely, we focus on simply structured problem instances that contain local optima which are very hard to overcome for many common metaheuristics. The plain (1+1) EA, iterated local search, and simulated annealing need exponential time for optimization, with high probability. In sharp contrast, the hybrid algorithm using VDS finds a global optimum in expected polynomial time. These results demonstrate the usefulness of hybrid evolutionary algorithms with VDS from a rigorous theoretical perspective.


Evolutionary algorithms Hybridization Iterated local search Memetic algorithms Simulated annealing Runtime analysis Combinatorial optimization 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Aickelin, U., Burke, E.K., Li, J.: An estimation of distribution algorithm with intelligent local search for rule-based nurse rostering. J. Oper. Res. Soc. 58, 1574–1585 (2007) zbMATHCrossRefGoogle Scholar
  2. 2.
    Cormen, T.H., Leiserson, C.E., Rivest, R.L., Stein, C.: Introduction to Algorithms, 2nd edn. MIT Press, New York (2001) zbMATHGoogle Scholar
  3. 3.
    Dawkins, R.: The Selfish Gene. Oxford University Press, New York (1976) Google Scholar
  4. 4.
    Dorigo, M., Stützle, T.: Ant Colony Optimization. MIT Press, New York (2004) zbMATHCrossRefGoogle Scholar
  5. 5.
    Droste, S., Jansen, T., Wegener, I.: On the analysis of the (1+1) evolutionary algorithm. Theor. Comput. Sci. 276, 51–81 (2002) zbMATHCrossRefMathSciNetGoogle Scholar
  6. 6.
    Droste, S., Jansen, T., Wegener, I.: Optimization with randomized search heuristics—the (A)NFL theorem, realistic scenarios, and difficult functions. Theor. Comput. Sci. 287(1), 131–144 (2002) zbMATHCrossRefMathSciNetGoogle Scholar
  7. 7.
    Feller, W.: An Introduction to Probability Theory and Its Applications, 3rd edn., vol. 1. Wiley, New York (1968) zbMATHGoogle Scholar
  8. 8.
    Fischer, S.: A polynomial upper bound for a mutation-based algorithm on the two-dimensional Ising model. In: Proceedings of the Genetic and Evolutionary Computation Conference (GECCO’04), pp. 1100–1112. Springer, Berlin (2004) Google Scholar
  9. 9.
    Friedrich, T., Oliveto, P.S., Sudholt, D., Witt, C.: Theoretical analysis of diversity mechanisms for global exploration. In: Proceedings of the Genetic and Evolutionary Computation Conference (GECCO’08), pp. 945–952. ACM Press, New York (2008) CrossRefGoogle Scholar
  10. 10.
    Giel, O., Wegener, I.: Evolutionary algorithms and the maximum matching problem. In: Proceedings of the 20th Annual Symposium on Theoretical Aspects of Computer Science (STACS’03), pp. 415–426. Springer, Berlin (2003) Google Scholar
  11. 11.
    Glover, F.W., Laguna, M.: Tabu Search. Kluwer Academic, Amsterdam (1997) zbMATHGoogle Scholar
  12. 12.
    Hart, W.E.: Locally-adaptive and memetic evolutionary pattern search algorithms. Evol. Comput. 11(1), 29–51 (2003) CrossRefGoogle Scholar
  13. 13.
    Hart, W.E., Krasnogor, N., Smith, J.E. (eds.): Recent Advances in Memetic Algorithms. Studies in Fuzziness and Soft Computing, vol. 166. Springer, Berlin (2004) Google Scholar
  14. 14.
    Ishibuchi, H., Yoshida, T., Murata, T.: Balance between genetic search and local search in memetic algorithms for multiobjective permutation flowshop scheduling. IEEE Trans. Evol. Comput. 7(2), 204–223 (2003) CrossRefGoogle Scholar
  15. 15.
    Jansen, T., Wegener, I.: Real royal road functions—where crossover provably is essential. Discrete Appl. Math. 149, 111–125 (2005) zbMATHCrossRefMathSciNetGoogle Scholar
  16. 16.
    Jansen, T., Wegener, I.: A comparison of simulated annealing with a simple evolutionary algorithm on pseudo-Boolean functions of unitation. Theor. Comput. Sci. 386(1–2), 73–93 (2007) zbMATHCrossRefMathSciNetGoogle Scholar
  17. 17.
    Jerrum, M., Sorkin, G.B.: The Metropolis algorithm for graph bisection. Discrete Appl. Math. 82(1–3), 155–175 (1998) zbMATHCrossRefMathSciNetGoogle Scholar
  18. 18.
    Kernighan, B., Lin, S.: An efficient heuristic procedure for partitioning graphs. Bell Syst. Tech. J. 49(2), 291–307 (1970) Google Scholar
  19. 19.
    Krasnogor, N., Gustafson, S.: A study on the use of “self-generation” in memetic algorithms. Natural Comput. 3(1), 53–76 (2004) zbMATHCrossRefMathSciNetGoogle Scholar
  20. 20.
    Krasnogor, N., Smith, J.: A tutorial for competent memetic algorithms: model, taxonomy, and design issues. IEEE Trans. Evol. Comput. 9(5), 474–488 (2005) CrossRefGoogle Scholar
  21. 21.
    Kumar, R., Banerjee, N.: Analysis of a multiobjective evolutionary algorithm on the 0–1 knapsack problem. Theor. Comput. Sci. 358(1), 104–120 (2006) zbMATHCrossRefMathSciNetGoogle Scholar
  22. 22.
    Levine, J., Ducatelle, F.: Ant colony optimisation and local search for bin packing and cutting stock problems. J. Oper. Res. Soc. 55(7), 705–716 (2004) zbMATHCrossRefGoogle Scholar
  23. 23.
    Lin, S., Kernighan, B.W.: An effective heuristic algorithm for the traveling salesman problem. Oper. Res. 21, 498–516 (1973) zbMATHCrossRefMathSciNetGoogle Scholar
  24. 24.
    Lourenço, H.R., Martin, O., Stützle, T.: Iterated local search. In: Handbook of Metaheuristics. International Series in Operations Research & Management Science, vol. 57, pp. 321–353. Kluwer Academic, Norwell (2002) Google Scholar
  25. 25.
    Mitzenmacher, M., Upfal, E.: Probability and Computing. Cambridge University Press, Cambridge (2005) zbMATHGoogle Scholar
  26. 26.
    Mladenović, N., Hansen, P.: Variable neighborhood search. Comput. OR 24(11), 1097–1100 (1997) zbMATHCrossRefGoogle Scholar
  27. 27.
    Neri, F., Toivanen, J., Cascella, G.L., Ong, Y.S.: An adaptive multimeme algorithm for designing HIV multidrug therapies. IEEE/ACM Trans. Comput. Biol. Bioinform. 4(2), 264–278 (2007) CrossRefGoogle Scholar
  28. 28.
    Neumann, F., Reichel, J.: Approximating minimum multicuts by evolutionary multi-objective algorithms. In: Parallel Problem Solving from Nature (PPSN X). LNCS, vol. 5199, pp. 72–81. Springer, Berlin (2008) CrossRefGoogle Scholar
  29. 29.
    Neumann, F., Wegener, I.: Randomized local search, evolutionary algorithms, and the minimum spanning tree problem. Theor. Comput. Sci. 378(1), 32–40 (2007) zbMATHCrossRefMathSciNetGoogle Scholar
  30. 30.
    Neumann, F., Reichel, J., Skutella, M.: Computing minimum cuts by randomized search heuristics. In: Proceedings of the Genetic and Evolutionary Computation Conference (GECCO 2008), pp. 779–786. ACM Press, New York (2008) CrossRefGoogle Scholar
  31. 31.
    Neumann, F., Sudholt, D., Witt, C.: Rigorous analyses for the combination of ant colony optimization and local search. In: Proceedings of the Sixth International Conference on Ant Colony Optimization and Swarm Intelligence (ANTS’08). LNCS, vol. 5217, pp. 132–143. Springer, Berlin (2008) CrossRefGoogle Scholar
  32. 32.
    Oliveto, P.S., Witt, C.: Simplified drift analysis for proving lower bounds in evolutionary computation. In: Parallel Problem Solving from Nature (PPSN X). LNCS, vol. 5199, pp. 82–91. Springer, Berlin (2008) CrossRefGoogle Scholar
  33. 33.
    Oliveto, P.S., He, J., Yao, X.: Computational complexity analysis of evolutionary algorithms for combinatorial optimization: a decade of results. Int. J. Autom. Comput. 4(3), 281–293 (2007) CrossRefGoogle Scholar
  34. 34.
    Ong, Y.S., Lim, M.H., Zhu, N., Wong, K.W.: Classification of adaptive memetic algorithms: a comparative study. IEEE Trans. Syst. Man Cybern., Part B 36(1), 141–152 (2006) CrossRefGoogle Scholar
  35. 35.
    Papadimitriou, C.: Computational Complexity. Addison-Wesley, Reading (1994) zbMATHGoogle Scholar
  36. 36.
    Reichel, J., Skutella, M.: Evolutionary algorithms and matroid optimization problems. In: Proceedings of the Genetic and Evolutionary Compution Conference (GECCO’07), pp. 947–954. ACM Press, New York (2007) CrossRefGoogle Scholar
  37. 37.
    Sindhya, K., Deb, K., Miettinen, K.: A local search based evolutionary multi-objective optimization approach for fast and accurate convergence. In: Parallel Problem Solving from Nature (PPSN X). LNCS, vol. 5199, pp. 815–824. Springer, Berlin (2008) CrossRefGoogle Scholar
  38. 38.
    Smith, J.: Coevolving memetic algorithms: A review and progress report. IEEE Trans. Syst. Man Cybern., Part B 37(1), 6–17 (2007) CrossRefGoogle Scholar
  39. 39.
    Stoer, M., Wagner, F.: A simple min cut algorithm. In: Proceedings of the Second Annual European Symposium on Algorithms (ESA’94), pp. 141–147 (1994) Google Scholar
  40. 40.
    Sudholt, D.: Crossover is provably essential for the Ising model on trees. In: Proceedings of the Genetic and Evolutionary Computation Conference (GECCO’05), pp. 1161–1167. ACM Press, New York (2005) CrossRefGoogle Scholar
  41. 41.
    Sudholt, D.: Local search in evolutionary algorithms: the impact of the local search frequency. In: Proceedings of the 17th International Symposium on Algorithms and Computation (ISAAC’06). LNCS, vol. 4288, pp. 359–368. Springer, Berlin (2006) Google Scholar
  42. 42.
    Sudholt, D.: On the analysis of the (1+1) memetic algorithm. In: Proceedings of the Genetic and Evolutionary Computation Conference (GECCO’06), pp. 493–500. ACM Press, New York (2006) CrossRefGoogle Scholar
  43. 43.
    Sudholt, D.: Memetic algorithms with variable-depth search to overcome local optima. In: Proceedings of the Genetic and Evolutionary Computation Conference (GECCO’08), pp. 787–794. ACM Press, New York (2008) CrossRefGoogle Scholar
  44. 44.
    Sudholt, D.: The impact of parametrization in memetic evolutionary algorithms. Theor. Comput. Sci. 410(26), 2511–2528 (2009) zbMATHCrossRefMathSciNetGoogle Scholar
  45. 45.
    Wegener, I.: Simulated annealing beats Metropolis in combinatorial optimization. In: Proceedings of the 32nd International Colloquium on Automata, Languages and Programming (ICALP’05). LNCS, vol. 3580, pp. 589–601. Springer, Berlin (2005) CrossRefGoogle Scholar
  46. 46.
    Witt, C.: Worst-case and average-case approximations by simple randomized search heuristics. In: Proceedings of the 22nd Symposium on Theoretical Aspects of Computer Science (STACS’05). LNCS, vol. 3404, pp. 44–56. Springer, Berlin (2005) Google Scholar
  47. 47.
    Zhou, Y., He, J.: A runtime analysis of evolutionary algorithms for constrained optimization problems. IEEE Trans. Evol. Comput. 11(5), 608–619 (2007) CrossRefMathSciNetGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2010

Authors and Affiliations

  1. 1.Technische Universität DortmundDortmundGermany
  2. 2.International Computer Science InstituteBerkeleyUSA

Personalised recommendations