Algorithmica

, Volume 78, Issue 2, pp 561–586 | Cite as

Time Complexity Analysis of Evolutionary Algorithms on Random Satisfiable k-CNF Formulas

Article
  • 166 Downloads

Abstract

We contribute to the theoretical understanding of randomized search heuristics by investigating their optimization behavior on satisfiable random k-satisfiability instances both in the planted solution model and the uniform model conditional on satisfiability. Denoting the number of variables by n, our main technical result is that the simple (\(1+1\)) evolutionary algorithm with high probability finds a satisfying assignment in time \(O(n \log n)\) when the clause-variable density is at least logarithmic. For low density instances, evolutionary algorithms seem to be less effective, and all we can show is a subexponential upper bound on the runtime for densities below \(\frac{1}{k(k-1)}\). We complement these mathematical results with numerical experiments on a broader density spectrum. They indicate that, indeed, the (\(1+1\)) EA is less efficient on lower densities. Our experiments also suggest that the implicit constants hidden in our main runtime guarantee are low. Our main result extends and considerably improves the result obtained by Sutton and Neumann (Lect Notes Comput Sci 8672:942–951, 2014) in terms of runtime, minimum density, and clause length. These improvements are made possible by establishing a close fitness-distance correlation in certain parts of the search space. This approach might be of independent interest and could be useful for other average-case analyses of randomized search heuristics. While the notion of a fitness-distance correlation has been around for a long time, to the best of our knowledge, this is the first time that fitness-distance correlation is explicitly used to rigorously prove a performance statement for an evolutionary algorithm.

Keywords

Runtime analysis Satisfiability Fitness-distance correlation 

Notes

Acknowledgments

The research leading to these results has received funding from the Australian Research Council (ARC) under Grant agreement DP140103400 and from the European Union Seventh Framework Programme (FP7/2007-2013) under Grant Agreement No 618091 (SAGE).

References

  1. 1.
    Achlioptas, D.: Random satisfiability. In: Biere, A., Heule, M., van Maaren, H., Walsh, T. (eds.) Handbook of Satisfiability. Frontiers in Artificial Intelligence and Applications, vol. 185, pp. 245–270. IOS Press, Amsterdam, Netherlands (2009)Google Scholar
  2. 2.
    Achlioptas, D., Coja-Oghlan, A., Ricci-Tersenghi, F.: On the solution-space geometry of random constraint satisfaction problems. Random Struct. Algorithms 38(3), 251–268 (2011)MathSciNetCrossRefMATHGoogle Scholar
  3. 3.
    Alekhnovich, M., Ben-Sasson, E.: Linear upper bounds for random walk on small density random 3-CNFs. SIAM J. Comput. 36(5), 1248–1263 (2007)MathSciNetCrossRefMATHGoogle Scholar
  4. 4.
    Altenberg, L.: Fitness distance correlation analysis: an instructive counterexample. In: Bäck, T. (eds.) Proceedings of the Seventh International Conference on Genetic Algorithms, pp. 57–64. Morgan Kaufmann (1997)Google Scholar
  5. 5.
    Auger, A., Doerr, B. (eds.): Theory of Randomized Search Heuristics: Foundations and Recent Developments. World Scientific Publishing Co. Inc., Singapore (2011)MATHGoogle Scholar
  6. 6.
    Ben-Sasson, E., Bilu, Y., Gutfreund, D.: Finding a randomly planted assignment in a random 3-CNF (2002, unpublished manuscript)Google Scholar
  7. 7.
    Bulatov, A.A., Skvortsov, E.S.: Phase transition for local search on planted SAT. In: Italiano, G.F., Pighizzini, G., Sannella, D.T. (eds.) Mathematical Foundations of Computer Science 2015, volume 9235 of Lecture Notes in Computer Science, vol. 9235, pp.175–186. Springer Berlin, Heidelberg (2015)Google Scholar
  8. 8.
    Clark, D.A., Frank, J., Gent, I.P., MacIntyre, E., Tomov, N., Walsh, T.: Local search and the number of solutions. In: Freuder, E.C. (ed.) Proceedings of the Second International Conference on Principles and Practice of Constraint Programming. Lecture Notes in Computer Science, vol. 1118, pp. 119–133. Springer, Berlin Heidelberg (1996)Google Scholar
  9. 9.
    Coja-Oghlan, A., Frieze, A.: Analyzing walksat on random formulas. SIAM J. Comput. 43(4), 1456–1485 (2014)MathSciNetCrossRefMATHGoogle Scholar
  10. 10.
    Coja-Oghlan, A., Panagiotou, K.: The asymptotic \(k\)-SAT threshold. Adv. Math. 288, 985–1068 (2016)MathSciNetCrossRefMATHGoogle Scholar
  11. 11.
    Crawford, J.M., Auton, L.D.: Experimental results on the crossover point in random 3-SAT. Artif. Intell. 81(1–2), 31–57 (1996)MathSciNetCrossRefGoogle Scholar
  12. 12.
    Ding, J., Sly, A., Sun, N.: Proof of the satisfiability conjecture for large \(k\). In: Proceedings of the Forty-Seventh Annual ACM on Symposium on Theory of Computing, STOC 2015, pp. 59–68, ACM, New York, NY, USA (2015)Google Scholar
  13. 13.
    Doerr, B.: Analyzing randomized search heuristics: tools from probability theory. In: Auger and Doerr [5], pp. 1–20Google Scholar
  14. 14.
    Doerr, B., Ann Goldberg, L.: Adaptive drift analysis. Algorithmica 65(1), 224–250 (2013)MathSciNetCrossRefMATHGoogle Scholar
  15. 15.
    Doerr, B., Johannsen, D., Winzen, C.: Multiplicative drift analysis. Algorithmica 64(4), 673–697 (2012)MathSciNetCrossRefMATHGoogle Scholar
  16. 16.
    Doerr, B., Neumann, F., Sutton, A.M.: Improved runtime bounds for the (\(1+1\)) EA on random 3-CNF formulas based on fitness-distance correlation. In: Laredo, J.L.J., Silva, S., Esparcia-Alcázar, A.I. (eds.) Proceedings of the Genetic and Evolutionary Computation Conference (GECCO 2015), pp. 1415–1422. ACM (2015)Google Scholar
  17. 17.
    Doerr, B., Sudholt, D., Witt, C.: When do evolutionary algorithms optimize separable functions in parallel? In: Proceedings of the Twelfth ACM SIGEVO Workshop on Foundations of Genetic Algorithms (FOGA 2013), pp. 48–59. ACM (2013)Google Scholar
  18. 18.
    Droste, S., Jansen, T., Wegener, I.: On the analysis of the (\(1+1\)) evolutionary algorithm. Theor. Comput. Sci. 276(1–2), 51–81 (2002)MathSciNetCrossRefMATHGoogle Scholar
  19. 19.
    Englert, M., Röglin, H., Vöcking, B.: Worst case and probabilistic analysis of the 2-opt algorithm for the TSP. Algorithmica 68(1), 190–264 (2014)MathSciNetCrossRefMATHGoogle Scholar
  20. 20.
    Flaxman, A.D.: A spectral technique for random satisfiable 3CNF formulas. Random Struct. Algorithms 32(4), 519–534 (2008)MathSciNetCrossRefMATHGoogle Scholar
  21. 21.
    Frieze, A., Suen, S.: Analysis of two simple heuristics on a random instance of \(k\)-SAT. J. Algorithms 20(2), 312–355 (1996)MathSciNetCrossRefMATHGoogle Scholar
  22. 22.
    Jansen, T.: On classifications of fitness functions. In: Kallel, L., Naudts, B., Rogers, A. (eds.) Theoretical Aspects of Evolutionary Computing, Natural Computing Series, pp. 371–385. Springer, Berlin (2001)CrossRefGoogle Scholar
  23. 23.
    Jansen, T.: Analyzing Evolutionary Algorithms—The Computer Science Perspective. Springer, Berlin (2013). (Natural Computing Series)CrossRefMATHGoogle Scholar
  24. 24.
    Jansen, T., Zarges, C.: Performance analysis of randomised search heuristics operating with a fixed budget. Theor. Comput. Sci. 545, 39–58 (2014)MathSciNetCrossRefMATHGoogle Scholar
  25. 25.
    Jones, T., Forrest, S.: Fitness distance correlation as a measure of problem difficulty for genetic algorithms. In: Eshelman, L.J. (eds.) Proceedings of the Sixth International Conference on Genetic Algorithms, pp. 184–192. Morgan Kaufmann (1995)Google Scholar
  26. 26.
    Kirkpatrick, S., Selman, B.: Critical behavior in the satisfiability of random Boolean expressions. Science 264(5163), 1297–1301 (1994)MathSciNetCrossRefMATHGoogle Scholar
  27. 27.
    Kötzing, T., Neumann, F., Röglin, H., Witt, C.: Theoretical analysis of two ACO approaches for the traveling salesman problem. Swarm Intell 6(1), 1–21 (2012)CrossRefGoogle Scholar
  28. 28.
    Koutsoupias, E., Papadimitriou, C.H.: On the greedy algorithm for satisfiability. Inf. Process. Lett. 43(1), 53–55 (1992)MathSciNetCrossRefMATHGoogle Scholar
  29. 29.
    Krivelevich, M., Vilenchik, D.: Solving random satisfiable 3CNF formulas in expected polynomial time. In: Proceedings of the Seventeenth Symposium on Discrete Algorithms (SODA 2006), pp. 454–463 (2006)Google Scholar
  30. 30.
    Ming-Te, C., Franco, J.: Probabilistic analysis of a generalization of the unit-clause literal selection heuristics for the \(k\) satisfiability problem. Inf. Sci. 51(3), 289–314 (1990)MathSciNetCrossRefMATHGoogle Scholar
  31. 31.
    Mitchell, D., Selman, B., Levesque, H.: Hard and easy distributions of SAT problems. In: Proceedings of the Tenth National Conference on Artificial Intelligence (AAAI 1992), pp. 459–465 (1992)Google Scholar
  32. 32.
    Mitzenmacher, M.: Tight thresholds for the pure literal rule. (1997). Technical Report 1997-011, Digital SRCGoogle Scholar
  33. 33.
    Molloy, M.: Cores in random hypergraphs and Boolean formulas. Random Struct. Algorithms 27(1), 124–135 (2005)MathSciNetCrossRefMATHGoogle Scholar
  34. 34.
    Nallaperuma, S., Neumann, F., Sudholt, D.: A fixed budget analysis of randomized search heuristics for the traveling salesperson problem. In: Arnold, D.V. (eds.) Proceedings of the Genetic and Evolutionary Computation Conference (GECCO 2014), pp. 807–814. ACM (2014)Google Scholar
  35. 35.
    Neumann, F., Witt, C.: Bioinspired Computation in Combinatorial Optimization: Algorithms and Their Computational Complexity. Springer, Berlin (2010)CrossRefMATHGoogle Scholar
  36. 36.
    Papadimitriou, C.H.: On selecting a satisfying truth assignment. In: Proceedings of 32nd Annual Symposium on Foundations of Computer Science, 1991, pp. 163–169. (1991)Google Scholar
  37. 37.
    Quick, R.J., Rayward-Smith, V.J., Smith, G.D.: Fitness distance correlation and ridge functions. In Eiben, A.E., Bäck, T., Schoenauer, M., Schwefel, H-P. (eds.) Proceedings of the Fifth International Conference on Parallel Problem Solving from Nature (PPSN V), volume 1498 of Lecture Notes in Computer Science, vol. 1498, pp.77–86. Springer (1998)Google Scholar
  38. 38.
    Schmidt-Pruzan, J., Shamir, E.: Component structure in the evolution of random hypergraphs. Combinatorica 5(1), 81–94 (1985)MathSciNetCrossRefMATHGoogle Scholar
  39. 39.
    Skvortsov, E.S.: A theoretical analysis of search in GSAT. In: Kullmann, O. (ed.) Theory and Applications of Satisfiability Testing (SAT 2009), volume 5584 of Lecture Notes in Computer Science, vol. 5584, pp. 265–275. Springer Berlin, Heidelberg (2009)Google Scholar
  40. 40.
    Storch, T.: Finding large cliques in sparse semi-random graphs by simple randomized search heuristics. Theor. Comput. Sci. 386, 114–131 (2007)MathSciNetCrossRefMATHGoogle Scholar
  41. 41.
    Sutton A.M., Neumann, F.: Runtime analysis of evolutionary algorithms on randomly constructed high-density satisfiable 3-CNF formulas. In: Bartz-Beielstein, T., Branke, J., Filipic, B., Smith, J. (eds.) Proceedings of the Thirteenth International Conference on Parallel Problem Solving from Nature (PPSN XIII), volume 8672 of Lecture Notes in Computer Science, vol. 8672, pp. 942–951. Springer (2014)Google Scholar
  42. 42.
    Witt, C.: Worst-case and average-case approximations by simple randomized search heuristics. In Diekert, V., Durand, B. (eds.) STACS 2005, 22nd Annual Symposium on Theoretical Aspects of Computer Science, Stuttgart, Germany, February 24-26, 2005, Proceedings, volume 3404 of Lecture Notes in Computer Science, vol. 3404, pp. 44–56. Springer (2005)Google Scholar
  43. 43.
    Witt, C.: Fitness levels with tail bounds for the analysis of randomized search heuristics. Inf. Process. Lett. 114(1–2), 38–41 (2014)MathSciNetCrossRefMATHGoogle Scholar
  44. 44.
    Zhou, Y.R.: Exponential bounds for the random walk algorithm on random planted 3-sat. Sci. China Inf. Sci. 56(9), 1–13 (2013)MathSciNetCrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2016

Authors and Affiliations

  • Benjamin Doerr
    • 1
  • Frank Neumann
    • 2
  • Andrew M. Sutton
    • 3
  1. 1.École PolytechniqueUniversité Paris-SaclayPalaiseauFrance
  2. 2.School of Computer ScienceUniversity of AdelaideAdelaideAustralia
  3. 3.Hasso-Plattner-InstitutUniversität PotsdamPotsdamGermany

Personalised recommendations