Skip to main content
SpringerLink
Log in

Hybridizing Evolutionary Algorithms with Variable-Depth Search to Overcome Local Optima

  • Published:
Algorithmica Aims and scope Submit manuscript

Abstract

Hybridizing evolutionary algorithms with local search has become a popular trend in recent years. There is empirical evidence for various combinatorial problems where hybrid evolutionary algorithms perform better than plain evolutionary algorithms. Due to the rapid development of a highly active field of research, theory lags far behind and a solid theoretical foundation of hybrid metaheuristics is sorely needed.

We are aiming at a theoretical understanding of why and when hybrid evolutionary algorithms are successful in combinatorial optimization. To this end, we consider a hybrid of a simple evolutionary algorithm, the (1+1) EA, with a powerful local search operator known as variable-depth search (VDS) or Kernighan-Lin. Three combinatorial problems are investigated: Mincut, Knapsack, and Maxsat. More precisely, we focus on simply structured problem instances that contain local optima which are very hard to overcome for many common metaheuristics. The plain (1+1) EA, iterated local search, and simulated annealing need exponential time for optimization, with high probability. In sharp contrast, the hybrid algorithm using VDS finds a global optimum in expected polynomial time. These results demonstrate the usefulness of hybrid evolutionary algorithms with VDS from a rigorous theoretical perspective.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Aickelin, U., Burke, E.K., Li, J.: An estimation of distribution algorithm with intelligent local search for rule-based nurse rostering. J. Oper. Res. Soc. 58, 1574–1585 (2007)

    Article  MATH  Google Scholar 

  2. Cormen, T.H., Leiserson, C.E., Rivest, R.L., Stein, C.: Introduction to Algorithms, 2nd edn. MIT Press, New York (2001)

    MATH  Google Scholar 

  3. Dawkins, R.: The Selfish Gene. Oxford University Press, New York (1976)

    Google Scholar 

  4. Dorigo, M., Stützle, T.: Ant Colony Optimization. MIT Press, New York (2004)

    Book  MATH  Google Scholar 

  5. Droste, S., Jansen, T., Wegener, I.: On the analysis of the (1+1) evolutionary algorithm. Theor. Comput. Sci. 276, 51–81 (2002)

    Article  MATH  MathSciNet  Google Scholar 

  6. Droste, S., Jansen, T., Wegener, I.: Optimization with randomized search heuristics—the (A)NFL theorem, realistic scenarios, and difficult functions. Theor. Comput. Sci. 287(1), 131–144 (2002)

    Article  MATH  MathSciNet  Google Scholar 

  7. Feller, W.: An Introduction to Probability Theory and Its Applications, 3rd edn., vol. 1. Wiley, New York (1968)

    MATH  Google Scholar 

  8. Fischer, S.: A polynomial upper bound for a mutation-based algorithm on the two-dimensional Ising model. In: Proceedings of the Genetic and Evolutionary Computation Conference (GECCO’04), pp. 1100–1112. Springer, Berlin (2004)

    Google Scholar 

  9. Friedrich, T., Oliveto, P.S., Sudholt, D., Witt, C.: Theoretical analysis of diversity mechanisms for global exploration. In: Proceedings of the Genetic and Evolutionary Computation Conference (GECCO’08), pp. 945–952. ACM Press, New York (2008)

    Chapter  Google Scholar 

  10. Giel, O., Wegener, I.: Evolutionary algorithms and the maximum matching problem. In: Proceedings of the 20th Annual Symposium on Theoretical Aspects of Computer Science (STACS’03), pp. 415–426. Springer, Berlin (2003)

    Google Scholar 

  11. Glover, F.W., Laguna, M.: Tabu Search. Kluwer Academic, Amsterdam (1997)

    MATH  Google Scholar 

  12. Hart, W.E.: Locally-adaptive and memetic evolutionary pattern search algorithms. Evol. Comput. 11(1), 29–51 (2003)

    Article  Google Scholar 

  13. Hart, W.E., Krasnogor, N., Smith, J.E. (eds.): Recent Advances in Memetic Algorithms. Studies in Fuzziness and Soft Computing, vol. 166. Springer, Berlin (2004)

    Google Scholar 

  14. Ishibuchi, H., Yoshida, T., Murata, T.: Balance between genetic search and local search in memetic algorithms for multiobjective permutation flowshop scheduling. IEEE Trans. Evol. Comput. 7(2), 204–223 (2003)

    Article  Google Scholar 

  15. Jansen, T., Wegener, I.: Real royal road functions—where crossover provably is essential. Discrete Appl. Math. 149, 111–125 (2005)

    Article  MATH  MathSciNet  Google Scholar 

  16. Jansen, T., Wegener, I.: A comparison of simulated annealing with a simple evolutionary algorithm on pseudo-Boolean functions of unitation. Theor. Comput. Sci. 386(1–2), 73–93 (2007)

    Article  MATH  MathSciNet  Google Scholar 

  17. Jerrum, M., Sorkin, G.B.: The Metropolis algorithm for graph bisection. Discrete Appl. Math. 82(1–3), 155–175 (1998)

    Article  MATH  MathSciNet  Google Scholar 

  18. Kernighan, B., Lin, S.: An efficient heuristic procedure for partitioning graphs. Bell Syst. Tech. J. 49(2), 291–307 (1970)

    Google Scholar 

  19. Krasnogor, N., Gustafson, S.: A study on the use of “self-generation” in memetic algorithms. Natural Comput. 3(1), 53–76 (2004)

    Article  MATH  MathSciNet  Google Scholar 

  20. Krasnogor, N., Smith, J.: A tutorial for competent memetic algorithms: model, taxonomy, and design issues. IEEE Trans. Evol. Comput. 9(5), 474–488 (2005)

    Article  Google Scholar 

  21. Kumar, R., Banerjee, N.: Analysis of a multiobjective evolutionary algorithm on the 0–1 knapsack problem. Theor. Comput. Sci. 358(1), 104–120 (2006)

    Article  MATH  MathSciNet  Google Scholar 

  22. Levine, J., Ducatelle, F.: Ant colony optimisation and local search for bin packing and cutting stock problems. J. Oper. Res. Soc. 55(7), 705–716 (2004)

    Article  MATH  Google Scholar 

  23. Lin, S., Kernighan, B.W.: An effective heuristic algorithm for the traveling salesman problem. Oper. Res. 21, 498–516 (1973)

    Article  MATH  MathSciNet  Google Scholar 

  24. Lourenço, H.R., Martin, O., Stützle, T.: Iterated local search. In: Handbook of Metaheuristics. International Series in Operations Research & Management Science, vol. 57, pp. 321–353. Kluwer Academic, Norwell (2002)

    Google Scholar 

  25. Mitzenmacher, M., Upfal, E.: Probability and Computing. Cambridge University Press, Cambridge (2005)

    MATH  Google Scholar 

  26. Mladenović, N., Hansen, P.: Variable neighborhood search. Comput. OR 24(11), 1097–1100 (1997)

    Article  MATH  Google Scholar 

  27. Neri, F., Toivanen, J., Cascella, G.L., Ong, Y.S.: An adaptive multimeme algorithm for designing HIV multidrug therapies. IEEE/ACM Trans. Comput. Biol. Bioinform. 4(2), 264–278 (2007)

    Article  Google Scholar 

  28. Neumann, F., Reichel, J.: Approximating minimum multicuts by evolutionary multi-objective algorithms. In: Parallel Problem Solving from Nature (PPSN X). LNCS, vol. 5199, pp. 72–81. Springer, Berlin (2008)

    Chapter  Google Scholar 

  29. Neumann, F., Wegener, I.: Randomized local search, evolutionary algorithms, and the minimum spanning tree problem. Theor. Comput. Sci. 378(1), 32–40 (2007)

    Article  MATH  MathSciNet  Google Scholar 

  30. Neumann, F., Reichel, J., Skutella, M.: Computing minimum cuts by randomized search heuristics. In: Proceedings of the Genetic and Evolutionary Computation Conference (GECCO 2008), pp. 779–786. ACM Press, New York (2008)

    Chapter  Google Scholar 

  31. Neumann, F., Sudholt, D., Witt, C.: Rigorous analyses for the combination of ant colony optimization and local search. In: Proceedings of the Sixth International Conference on Ant Colony Optimization and Swarm Intelligence (ANTS’08). LNCS, vol. 5217, pp. 132–143. Springer, Berlin (2008)

    Chapter  Google Scholar 

  32. Oliveto, P.S., Witt, C.: Simplified drift analysis for proving lower bounds in evolutionary computation. In: Parallel Problem Solving from Nature (PPSN X). LNCS, vol. 5199, pp. 82–91. Springer, Berlin (2008)

    Chapter  Google Scholar 

  33. Oliveto, P.S., He, J., Yao, X.: Computational complexity analysis of evolutionary algorithms for combinatorial optimization: a decade of results. Int. J. Autom. Comput. 4(3), 281–293 (2007)

    Article  Google Scholar 

  34. Ong, Y.S., Lim, M.H., Zhu, N., Wong, K.W.: Classification of adaptive memetic algorithms: a comparative study. IEEE Trans. Syst. Man Cybern., Part B 36(1), 141–152 (2006)

    Article  Google Scholar 

  35. Papadimitriou, C.: Computational Complexity. Addison-Wesley, Reading (1994)

    MATH  Google Scholar 

  36. Reichel, J., Skutella, M.: Evolutionary algorithms and matroid optimization problems. In: Proceedings of the Genetic and Evolutionary Compution Conference (GECCO’07), pp. 947–954. ACM Press, New York (2007)

    Chapter  Google Scholar 

  37. Sindhya, K., Deb, K., Miettinen, K.: A local search based evolutionary multi-objective optimization approach for fast and accurate convergence. In: Parallel Problem Solving from Nature (PPSN X). LNCS, vol. 5199, pp. 815–824. Springer, Berlin (2008)

    Chapter  Google Scholar 

  38. Smith, J.: Coevolving memetic algorithms: A review and progress report. IEEE Trans. Syst. Man Cybern., Part B 37(1), 6–17 (2007)

    Article  Google Scholar 

  39. Stoer, M., Wagner, F.: A simple min cut algorithm. In: Proceedings of the Second Annual European Symposium on Algorithms (ESA’94), pp. 141–147 (1994)

  40. Sudholt, D.: Crossover is provably essential for the Ising model on trees. In: Proceedings of the Genetic and Evolutionary Computation Conference (GECCO’05), pp. 1161–1167. ACM Press, New York (2005)

    Chapter  Google Scholar 

  41. Sudholt, D.: Local search in evolutionary algorithms: the impact of the local search frequency. In: Proceedings of the 17th International Symposium on Algorithms and Computation (ISAAC’06). LNCS, vol. 4288, pp. 359–368. Springer, Berlin (2006)

    Google Scholar 

  42. Sudholt, D.: On the analysis of the (1+1) memetic algorithm. In: Proceedings of the Genetic and Evolutionary Computation Conference (GECCO’06), pp. 493–500. ACM Press, New York (2006)

    Chapter  Google Scholar 

  43. Sudholt, D.: Memetic algorithms with variable-depth search to overcome local optima. In: Proceedings of the Genetic and Evolutionary Computation Conference (GECCO’08), pp. 787–794. ACM Press, New York (2008)

    Chapter  Google Scholar 

  44. Sudholt, D.: The impact of parametrization in memetic evolutionary algorithms. Theor. Comput. Sci. 410(26), 2511–2528 (2009)

    Article  MATH  MathSciNet  Google Scholar 

  45. Wegener, I.: Simulated annealing beats Metropolis in combinatorial optimization. In: Proceedings of the 32nd International Colloquium on Automata, Languages and Programming (ICALP’05). LNCS, vol. 3580, pp. 589–601. Springer, Berlin (2005)

    Chapter  Google Scholar 

  46. Witt, C.: Worst-case and average-case approximations by simple randomized search heuristics. In: Proceedings of the 22nd Symposium on Theoretical Aspects of Computer Science (STACS’05). LNCS, vol. 3404, pp. 44–56. Springer, Berlin (2005)

    Google Scholar 

  47. Zhou, Y., He, J.: A runtime analysis of evolutionary algorithms for constrained optimization problems. IEEE Trans. Evol. Comput. 11(5), 608–619 (2007)

    Article  MathSciNet  Google Scholar 

Download references

Author information

Author notes

  1. Dirk Sudholt

    Present address: International Computer Science Institute, Berkeley, CA, 94704, USA

Authors and Affiliations

  1. Technische Universität Dortmund, 44221, Dortmund, Germany

    Dirk Sudholt

Authors
  1. Dirk Sudholt
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Dirk Sudholt.

Additional information

A preliminary version of this article has been presented at GECCO’08 [43].

Rights and permissions

Reprints and permissions

About this article

Cite this article

Sudholt, D. Hybridizing Evolutionary Algorithms with Variable-Depth Search to Overcome Local Optima. Algorithmica 59, 343–368 (2011). https://doi.org/10.1007/s00453-009-9384-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00453-009-9384-2

Keywords

Search

Navigation