Journal of Automated Reasoning

, Volume 24, Issue 4, pp 421–481 | Cite as

Local Search Algorithms for SAT: An Empirical Evaluation

  • Holger H. Hoos
  • Thomas Stützle


Local search algorithms are among the standard methods for solving hard combinatorial problems from various areas of artificial intelligence and operations research. For SAT, some of the most successful and powerful algorithms are based on stochastic local search, and in the past 10 years a large number of such algorithms have been proposed and investigated. In this article, we focus on two particularly well-known families of local search algorithms for SAT, the GSAT and WalkSAT architectures. We present a detailed comparative analysis of these algorithms" performance using a benchmark set that contains instances from randomized distributions as well as SAT-encoded problems from various domains. We also investigate the robustness of the observed performance characteristics as algorithm-dependent and problem-dependent parameters are changed. Our empirical analysis gives a very detailed picture of the algorithms" performance for various domains of SAT problems; it also reveals a fundamental weakness in some of the best-performing algorithms and shows how this can be overcome.

SAT stochastic search empirical evaluation run-time distributions robustness 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Alt, H., Guibas, L., Mehlhorn, K., Karp, R. and Wigderson, A.: A method for obtaining randomized algorithms with small tail probabilities, Algorithmica 16 (1996), 543–547.Google Scholar
  2. 2.
    Asahiro, Y., Iwama, K. and Miyano, E.: Random generation of test instances with controlled attributes, in [41], pp. 377–394.Google Scholar
  3. 3.
    Beringer, A., Aschemann, G., Hoos, H. H., Metzger, M. and Weiß , A.: GSAT versus simulated annealing, in A. G. Cohn (ed.), Proceedings of ECAI'94, Wiley, 1994, pp. 130–134.Google Scholar
  4. 4.
    Buro, M. and Kleine-Büning, H.: Report on a SAT competition, Technical Report 110, Dept. of Mathematics and Informatics, University of Paderborn, Germany, 1992.Google Scholar
  5. 5.
    Battiti, R. and Protasi, M.: Reactive search, a history-based heuristic for MAX-SAT, ACM J. Experimental Algorithmics 2 (1997).Google Scholar
  6. 6.
    Barlow, R. E. and Proschan, F.: Statistical Theory of Reliability and Life Testing, Holt, Reinhart and Winston, Inc., 1981 (reprint).Google Scholar
  7. 7.
    Cha, B. and Iwama, K.: Performance tests of local search algorithms using new types of random CNF formula, in Proceedings of IJCAI'95, Morgan Kaufmann, 1995, pp. 304–309.Google Scholar
  8. 8.
    Cha, B. and Iwama, K.: Adding new clauses for faster local search, in Proceedings of AAAI'96, MIT Press, 1996, pp. 332–337.Google Scholar
  9. 9.
    Cheeseman, P., Kanefsky, B. and Taylor, W. M.: Where the really hard problems are, in Proceedings of IJCAI'91, Morgan Kaufmann, 1991, pp. 331–337.Google Scholar
  10. 10.
    Crawford, J. M. and Auton, L. D.: Experimental results on the crossover point in random 3SAT, Artif. Intell. 81(1-2) (1996), 31–57.Google Scholar
  11. 11.
    Davis, M., Logemann, G. and Loveland, D.: A machine program for theorem proofing, Comm. ACM 5 (1962), 394–397.Google Scholar
  12. 12.
    de Kleer, J.: A comparison of ATMS and CSP techniques, in Proceedings of IJCAI'89, Morgan Kaufmann, 1989, pp. 290–296.Google Scholar
  13. 13.
    Dubois, O., Andre, P., Boufkhad, Y. and Carlier, J.: SAT versus UNSAT, in [41], pp. 415–436.Google Scholar
  14. 14.
    Eiben, A. E. and van der Hauw, J. K.: Solving 3-SAT with adaptive genetic algorithms, in Proceedings of the 4th IEEE Conference on Evolutionary Computation, IEEE Press, 1997, pp. 81–86.Google Scholar
  15. 15.
    Frank, J., Cheeseman, P. and Stutz, J.: When gravity fails: Local search topology, (Electronic) J. Artif. Intell. Res. 7 (1997), 249–281.Google Scholar
  16. 16.
    Frank, J.:Weighting for Godot: Learning heuristics for GSAT, in Proceedings of AAAI'96, MIT Press, 1996, pp. 338–343.Google Scholar
  17. 17.
    Frank, J.: Learning short-term clause weights for GSAT, in Proceedings of IJCAI'97, Morgan Kaufmann, 1997, pp. 384–389.Google Scholar
  18. 18.
    Frost, D., Rish, I. and Vila, L.: Summarizing CSP hardness with continuous probability distributions, in Proceedings of AAAI'97, MIT Press, 1997, pp. 327–333.Google Scholar
  19. 19.
    Gent, I. P., MacIntyre, E., Prosser, P. and Walsh, T.: The scaling of search cost, in Proceedings of AAAI'97, MIT Press, 1997, pp. 315–320.Google Scholar
  20. 20.
    Gent, I. and Walsh, T.: Unsatisfied variables in local search, in J. Hallam (ed.), Hybrid Problems, Hybrid Solutions, IOS Press, 1995, pp. 73–85.Google Scholar
  21. 21.
    Gent, I. P. and Walsh, T.: Towards an understanding of Hill-climbing procedures for SAT, in Proceedings of AAAI'93, MIT Press, 1993, pp. 28–33.Google Scholar
  22. 22.
    Gent, I. P. and Walsh, T.: An empirical analysis of search in GSAT, J. Artif. Intell. Res. 1 (1993), 47–59.Google Scholar
  23. 23.
    Glover, F.: Tabu search-Part I, ORSA J. Comput. 1(3) (1989), 190–206.Google Scholar
  24. 24.
    Gomes, C. P. and Selman, B.: Problem structure in the presence of perturbations, in Proceedings of AAAI'97, MIT Press, 1997, pp. 221–226.Google Scholar
  25. 25.
    Gomes, C. P., Selman, B. and Kautz, H.: Boosting combinatorial search trough randomization, in Proceedings of AAAI'98, MIT Press, 1998, pp. 431–437.Google Scholar
  26. 26.
    Gottlieb, J. and Voss, N.: Improving the performance of evolutionary algorithms for the satisfiability problem by refining functions, in A. E. Eiben, T. Bäck, M. Schoenauer, and H.-P. Schwefel (eds), Proceedings of PPSN V, Lecture Notes in Comput. Sci. 1498, Springer-Verlag, 1998, pp. 813–822.Google Scholar
  27. 27.
    Gu, G.: Efficient local search for very large-scale satisfiability problems, SIGART Bulletin 3 (1992), 8–12.Google Scholar
  28. 28.
    Hansen, P. and Jaumard, B.: Algorithms for the maximum satisfiability problem, Computing 44 (1990), 279–303.Google Scholar
  29. 29.
    Hogg, T.: Refining the phase transition in combinatorial search, Artif. Intell. 81 (1996), 127–154.Google Scholar
  30. 30.
    Hooker, J. N.: Needed: An empirical science of algorithms, Oper. Res. 42(2) (1994), 201–212.Google Scholar
  31. 31.
    Hooker, J. N.: Testing heuristics: We have it all wrong, J. Heuristics 1 (1996), 33–42.Google Scholar
  32. 32.
    Hoos, H. H.: Aussagenlogische SAT-Verfahren und ihre Anwendung bei der Lösung des HC-Problems in gerichteten Graphen, Master's thesis, Technische Universität Darmstadt, Darmstadt, Germany, 1996.Google Scholar
  33. 33.
    Hoos, H. H.: SAT-encodings, search space structure, and local search performance, in Proceedings of IJCAI'99, 1999.Google Scholar
  34. 34.
    Hoos, H. H.: On the run-time behavior of stochastic local search algorithms for SAT, in Proceedings of AAAI'99, MIT Press, 1999, pp. 661–666.Google Scholar
  35. 35.
    Hoos, H. H.: Stochastic Local Search-Methods, Models, Applications, PhD Thesis, Department of Computer Science, Darmstadt University of Technology, 1998.Google Scholar
  36. 36.
    Hoos, H. H. and Stützle, T.: Evaluating Las Vegas algorithms-pitfalls and remedies, in Proceedings of UAI-98, Morgan Kaufmann, 1998, pp. 238–245.Google Scholar
  37. 37.
    Hoos, H. H. and Stützle, T.: Characterizing the run-time behaviour of stochastic local search, Technical Report AIDA-98-1, FG Intellektik, TU Darmstadt, January 1998.Google Scholar
  38. 38.
    Hoos, H. H. and Stützle, T.: Systematic vs. local search for SAT, in Proceedings of the 23rd National German Conference on Artificial Intelligence (KI-99), 1999, to appear.Google Scholar
  39. 39.
    Hoos, H. H. and Stützle, T.: Towards a characterization of the behavior of stochastic local search algorithms for SAT, Artif. Intell., to appear.Google Scholar
  40. 40.
    Johnson, D. S., Aragon, C. R., McGeoch, L. A. and Schevon, C.: Optimization by simulated annealing: An experimental evaluation: Part II, graph coloring and number partitioning, Oper. Res. 39(3) (1991), 378–406.Google Scholar
  41. 41.
    Johnson, D. S. and Trick, M. A. (eds): Cliques, Coloring, and Satisfiability, DIMACS Series on Discrete Math. and Theoret. Comput. Sci. 26, Amer. Math. Soc., 1996.Google Scholar
  42. 42.
    Kamath, A. P., Karmarkar, N. K., Ramakrishnan, K. G. and Resende, M. G. C.: A continuous approach to inductive inference, Math. Programming 57 (1992), 215–238.Google Scholar
  43. 43.
    Kask, K. and Dechter, R.: Graph-based methods for improving GSAT, in Proceedings of AAAI'96, MIT Press, 1996, pp. 350–355.Google Scholar
  44. 44.
    Kautz, H., McAllester, D. and Selman, B.: Encoding plans in propositional logic, in Proceedings of KR'96, 1996, pp. 374–384.Google Scholar
  45. 45.
    Kautz, H. and Selman, B.: Pushing the envelope: Planning, propositional logic, and stochastic search, in Proceedings of AAAI'96, Vol. 2, MIT Press, 1996, pp. 1194–1201.Google Scholar
  46. 46.
    Kautz, H. and Selman, B.: BLACKBOX: A new approach to the application of theorem proving to problem solving, in Working Notes of the Workshop on Planning as Combinatorial Search, held in conjunction with AIPS-98, Pittsburgh, PA, 1998.Google Scholar
  47. 47.
    Kirkpatrick, S. and Selman, B.: Critical behavior in the satisfiability of random Boolean expressions, Science 264 (1994), 1297–1301.Google Scholar
  48. 48.
    Larrabee, T.: Test pattern generation using Boolean satisfiability, IEEE Trans. Computer-Aided Design 11(1) (1992), 6–22.Google Scholar
  49. 49.
    Luby, M., Sinclair, A. and Zuckerman, D.: Optimal speedup of Las Vegas algorithms, Inform. Process. Lett. 47 (1993), 173–180.Google Scholar
  50. 50.
    Mazure, B., Sais, L. and Grégoire, É.: Tabu search for SAT, in Proceedings of AAAI'97, 1997, pp. 281–285.Google Scholar
  51. 51.
    McAllester, D., Selman, B. and Kautz, H.: Evidence for invariants in local search, in Proceedings of AAAI'97, MIT Press, 1997, pp. 321–326.Google Scholar
  52. 52.
    Mitchell, D., Selman, B. and Levesque, H.: Hard and easy distributions of sat problems, in Proceedings of AAAI'92, MIT Press, 1992, pp. 459–465.Google Scholar
  53. 53.
    Morris, P.: The breakout method for escaping from local minima, in Proceedings of AAAI'93, MIT Press, 1993, pp. 40–45.Google Scholar
  54. 54.
    Minton, S., Johnston, M. D., Philips, A. B. and Laird, P.: Minimizing conflicts: A heuristic repair method for constraint satisfaction and scheduling problems, Artif. Intell. 52 (1992), 161–205.Google Scholar
  55. 55.
    Parkes, A. J. and Walser, J. P.: Tuning local search for satisfiability testing, in Proceedings of AAAI'96, MIT Press, 1996, pp. 356–362.Google Scholar
  56. 56.
    Resende, M. G. C. and Feo, T. A.: A GRASP for satisfiability, in [41], pp. 499–520.Google Scholar
  57. 57.
    Rohatgi, V. K.: An Introduction to Probability Theory and Mathematical Statistics, Wiley, 1976.Google Scholar
  58. 58.
    Selman, B. and Kautz, H. A.: An empirical study of greedy local search for satisfiability testing, in Proceedings of AAAI'93, MIT Press, 1993, pp. 46–51.Google Scholar
  59. 59.
    Selman, B., Kautz, H. A. and Cohen, B.: Noise strategies for improving local search, in Proceedings of AAAI'94, MIT Press, 1994, pp. 337–343.Google Scholar
  60. 60.
    Selman, B., Levesque, H. and Mitchell, D.: A new method for solving hard satisfiability problems, in Proceedings of AAAI'92, MIT Press, 1992, pp. 440–446.Google Scholar
  61. 61.
    Steinmann, O., Strohmaier, A. and Stützle, T.: Tabu search vs. random walk, in Advances in Artificial Intelligence (KI97), Lecture Notes in Artif. Intell. 1303, Springer-Verlag, 1997, pp. 337–348.Google Scholar
  62. 62.
    Stützle, T.: Local Search Algorithms for Combinatorial Problems-Analysis, Improvements, and New Applications, PhD Thesis, Department of Computer Science, Darmstadt University of Technology, 1998.Google Scholar
  63. 63.
    Spears, W. M.: Simulated annealing for hard satisfiability problems, Technical Report, Naval Research Laboratory, Washington DC, 1993.Google Scholar
  64. 64.
    Yokoo, T.: Why adding more constraints makes a problem easier for Hill-climbing algorithms: Analyzing landscapes of CSPs, in Proceedings of CP'97, Springer-Verlag, 1997, pp. 357–370.Google Scholar

Copyright information

© Kluwer Academic Publishers 2000

Authors and Affiliations

  • Holger H. Hoos
  • Thomas Stützle

There are no affiliations available

Personalised recommendations