Skip to main content
SpringerLink
Log in

Local Search Algorithms for SAT: An Empirical Evaluation

  • Published:
Journal of Automated Reasoning Aims and scope Submit manuscript

Abstract

Local search algorithms are among the standard methods for solving hard combinatorial problems from various areas of artificial intelligence and operations research. For SAT, some of the most successful and powerful algorithms are based on stochastic local search, and in the past 10 years a large number of such algorithms have been proposed and investigated. In this article, we focus on two particularly well-known families of local search algorithms for SAT, the GSAT and WalkSAT architectures. We present a detailed comparative analysis of these algorithms" performance using a benchmark set that contains instances from randomized distributions as well as SAT-encoded problems from various domains. We also investigate the robustness of the observed performance characteristics as algorithm-dependent and problem-dependent parameters are changed. Our empirical analysis gives a very detailed picture of the algorithms" performance for various domains of SAT problems; it also reveals a fundamental weakness in some of the best-performing algorithms and shows how this can be overcome.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Alt, H., Guibas, L., Mehlhorn, K., Karp, R. and Wigderson, A.: A method for obtaining randomized algorithms with small tail probabilities, Algorithmica 16 (1996), 543–547.

    Google Scholar 

  2. Asahiro, Y., Iwama, K. and Miyano, E.: Random generation of test instances with controlled attributes, in [41], pp. 377–394.

  3. Beringer, A., Aschemann, G., Hoos, H. H., Metzger, M. and Weiß , A.: GSAT versus simulated annealing, in A. G. Cohn (ed.), Proceedings of ECAI'94, Wiley, 1994, pp. 130–134.

  4. Buro, M. and Kleine-Büning, H.: Report on a SAT competition, Technical Report 110, Dept. of Mathematics and Informatics, University of Paderborn, Germany, 1992.

    Google Scholar 

  5. Battiti, R. and Protasi, M.: Reactive search, a history-based heuristic for MAX-SAT, ACM J. Experimental Algorithmics 2 (1997).

  6. Barlow, R. E. and Proschan, F.: Statistical Theory of Reliability and Life Testing, Holt, Reinhart and Winston, Inc., 1981 (reprint).

  7. Cha, B. and Iwama, K.: Performance tests of local search algorithms using new types of random CNF formula, in Proceedings of IJCAI'95, Morgan Kaufmann, 1995, pp. 304–309.

  8. Cha, B. and Iwama, K.: Adding new clauses for faster local search, in Proceedings of AAAI'96, MIT Press, 1996, pp. 332–337.

  9. Cheeseman, P., Kanefsky, B. and Taylor, W. M.: Where the really hard problems are, in Proceedings of IJCAI'91, Morgan Kaufmann, 1991, pp. 331–337.

  10. Crawford, J. M. and Auton, L. D.: Experimental results on the crossover point in random 3SAT, Artif. Intell. 81(1-2) (1996), 31–57.

    Google Scholar 

  11. Davis, M., Logemann, G. and Loveland, D.: A machine program for theorem proofing, Comm. ACM 5 (1962), 394–397.

    Google Scholar 

  12. de Kleer, J.: A comparison of ATMS and CSP techniques, in Proceedings of IJCAI'89, Morgan Kaufmann, 1989, pp. 290–296.

  13. Dubois, O., Andre, P., Boufkhad, Y. and Carlier, J.: SAT versus UNSAT, in [41], pp. 415–436.

  14. Eiben, A. E. and van der Hauw, J. K.: Solving 3-SAT with adaptive genetic algorithms, in Proceedings of the 4th IEEE Conference on Evolutionary Computation, IEEE Press, 1997, pp. 81–86.

  15. Frank, J., Cheeseman, P. and Stutz, J.: When gravity fails: Local search topology, (Electronic) J. Artif. Intell. Res. 7 (1997), 249–281.

    Google Scholar 

  16. Frank, J.:Weighting for Godot: Learning heuristics for GSAT, in Proceedings of AAAI'96, MIT Press, 1996, pp. 338–343.

  17. Frank, J.: Learning short-term clause weights for GSAT, in Proceedings of IJCAI'97, Morgan Kaufmann, 1997, pp. 384–389.

  18. Frost, D., Rish, I. and Vila, L.: Summarizing CSP hardness with continuous probability distributions, in Proceedings of AAAI'97, MIT Press, 1997, pp. 327–333.

  19. Gent, I. P., MacIntyre, E., Prosser, P. and Walsh, T.: The scaling of search cost, in Proceedings of AAAI'97, MIT Press, 1997, pp. 315–320.

  20. Gent, I. and Walsh, T.: Unsatisfied variables in local search, in J. Hallam (ed.), Hybrid Problems, Hybrid Solutions, IOS Press, 1995, pp. 73–85.

  21. Gent, I. P. and Walsh, T.: Towards an understanding of Hill-climbing procedures for SAT, in Proceedings of AAAI'93, MIT Press, 1993, pp. 28–33.

  22. Gent, I. P. and Walsh, T.: An empirical analysis of search in GSAT, J. Artif. Intell. Res. 1 (1993), 47–59.

    Google Scholar 

  23. Glover, F.: Tabu search-Part I, ORSA J. Comput. 1(3) (1989), 190–206.

    Google Scholar 

  24. Gomes, C. P. and Selman, B.: Problem structure in the presence of perturbations, in Proceedings of AAAI'97, MIT Press, 1997, pp. 221–226.

  25. Gomes, C. P., Selman, B. and Kautz, H.: Boosting combinatorial search trough randomization, in Proceedings of AAAI'98, MIT Press, 1998, pp. 431–437.

  26. Gottlieb, J. and Voss, N.: Improving the performance of evolutionary algorithms for the satisfiability problem by refining functions, in A. E. Eiben, T. Bäck, M. Schoenauer, and H.-P. Schwefel (eds), Proceedings of PPSN V, Lecture Notes in Comput. Sci. 1498, Springer-Verlag, 1998, pp. 813–822.

  27. Gu, G.: Efficient local search for very large-scale satisfiability problems, SIGART Bulletin 3 (1992), 8–12.

    Google Scholar 

  28. Hansen, P. and Jaumard, B.: Algorithms for the maximum satisfiability problem, Computing 44 (1990), 279–303.

    Google Scholar 

  29. Hogg, T.: Refining the phase transition in combinatorial search, Artif. Intell. 81 (1996), 127–154.

    Google Scholar 

  30. Hooker, J. N.: Needed: An empirical science of algorithms, Oper. Res. 42(2) (1994), 201–212.

    Google Scholar 

  31. Hooker, J. N.: Testing heuristics: We have it all wrong, J. Heuristics 1 (1996), 33–42.

    Google Scholar 

  32. Hoos, H. H.: Aussagenlogische SAT-Verfahren und ihre Anwendung bei der Lösung des HC-Problems in gerichteten Graphen, Master's thesis, Technische Universität Darmstadt, Darmstadt, Germany, 1996.

    Google Scholar 

  33. Hoos, H. H.: SAT-encodings, search space structure, and local search performance, in Proceedings of IJCAI'99, 1999.

  34. Hoos, H. H.: On the run-time behavior of stochastic local search algorithms for SAT, in Proceedings of AAAI'99, MIT Press, 1999, pp. 661–666.

  35. Hoos, H. H.: Stochastic Local Search-Methods, Models, Applications, PhD Thesis, Department of Computer Science, Darmstadt University of Technology, 1998.

  36. Hoos, H. H. and Stützle, T.: Evaluating Las Vegas algorithms-pitfalls and remedies, in Proceedings of UAI-98, Morgan Kaufmann, 1998, pp. 238–245.

  37. Hoos, H. H. and Stützle, T.: Characterizing the run-time behaviour of stochastic local search, Technical Report AIDA-98-1, FG Intellektik, TU Darmstadt, January 1998.

    Google Scholar 

  38. Hoos, H. H. and Stützle, T.: Systematic vs. local search for SAT, in Proceedings of the 23rd National German Conference on Artificial Intelligence (KI-99), 1999, to appear.

  39. Hoos, H. H. and Stützle, T.: Towards a characterization of the behavior of stochastic local search algorithms for SAT, Artif. Intell., to appear.

  40. Johnson, D. S., Aragon, C. R., McGeoch, L. A. and Schevon, C.: Optimization by simulated annealing: An experimental evaluation: Part II, graph coloring and number partitioning, Oper. Res. 39(3) (1991), 378–406.

    Google Scholar 

  41. Johnson, D. S. and Trick, M. A. (eds): Cliques, Coloring, and Satisfiability, DIMACS Series on Discrete Math. and Theoret. Comput. Sci. 26, Amer. Math. Soc., 1996.

  42. Kamath, A. P., Karmarkar, N. K., Ramakrishnan, K. G. and Resende, M. G. C.: A continuous approach to inductive inference, Math. Programming 57 (1992), 215–238.

    Google Scholar 

  43. Kask, K. and Dechter, R.: Graph-based methods for improving GSAT, in Proceedings of AAAI'96, MIT Press, 1996, pp. 350–355.

  44. Kautz, H., McAllester, D. and Selman, B.: Encoding plans in propositional logic, in Proceedings of KR'96, 1996, pp. 374–384.

  45. Kautz, H. and Selman, B.: Pushing the envelope: Planning, propositional logic, and stochastic search, in Proceedings of AAAI'96, Vol. 2, MIT Press, 1996, pp. 1194–1201.

  46. Kautz, H. and Selman, B.: BLACKBOX: A new approach to the application of theorem proving to problem solving, in Working Notes of the Workshop on Planning as Combinatorial Search, held in conjunction with AIPS-98, Pittsburgh, PA, 1998.

  47. Kirkpatrick, S. and Selman, B.: Critical behavior in the satisfiability of random Boolean expressions, Science 264 (1994), 1297–1301.

    Google Scholar 

  48. Larrabee, T.: Test pattern generation using Boolean satisfiability, IEEE Trans. Computer-Aided Design 11(1) (1992), 6–22.

    Google Scholar 

  49. Luby, M., Sinclair, A. and Zuckerman, D.: Optimal speedup of Las Vegas algorithms, Inform. Process. Lett. 47 (1993), 173–180.

    Google Scholar 

  50. Mazure, B., Sais, L. and Grégoire, É.: Tabu search for SAT, in Proceedings of AAAI'97, 1997, pp. 281–285.

  51. McAllester, D., Selman, B. and Kautz, H.: Evidence for invariants in local search, in Proceedings of AAAI'97, MIT Press, 1997, pp. 321–326.

  52. Mitchell, D., Selman, B. and Levesque, H.: Hard and easy distributions of sat problems, in Proceedings of AAAI'92, MIT Press, 1992, pp. 459–465.

  53. Morris, P.: The breakout method for escaping from local minima, in Proceedings of AAAI'93, MIT Press, 1993, pp. 40–45.

  54. Minton, S., Johnston, M. D., Philips, A. B. and Laird, P.: Minimizing conflicts: A heuristic repair method for constraint satisfaction and scheduling problems, Artif. Intell. 52 (1992), 161–205.

    Google Scholar 

  55. Parkes, A. J. and Walser, J. P.: Tuning local search for satisfiability testing, in Proceedings of AAAI'96, MIT Press, 1996, pp. 356–362.

  56. Resende, M. G. C. and Feo, T. A.: A GRASP for satisfiability, in [41], pp. 499–520.

  57. Rohatgi, V. K.: An Introduction to Probability Theory and Mathematical Statistics, Wiley, 1976.

  58. Selman, B. and Kautz, H. A.: An empirical study of greedy local search for satisfiability testing, in Proceedings of AAAI'93, MIT Press, 1993, pp. 46–51.

  59. Selman, B., Kautz, H. A. and Cohen, B.: Noise strategies for improving local search, in Proceedings of AAAI'94, MIT Press, 1994, pp. 337–343.

  60. Selman, B., Levesque, H. and Mitchell, D.: A new method for solving hard satisfiability problems, in Proceedings of AAAI'92, MIT Press, 1992, pp. 440–446.

  61. Steinmann, O., Strohmaier, A. and Stützle, T.: Tabu search vs. random walk, in Advances in Artificial Intelligence (KI97), Lecture Notes in Artif. Intell. 1303, Springer-Verlag, 1997, pp. 337–348.

  62. Stützle, T.: Local Search Algorithms for Combinatorial Problems-Analysis, Improvements, and New Applications, PhD Thesis, Department of Computer Science, Darmstadt University of Technology, 1998.

  63. Spears, W. M.: Simulated annealing for hard satisfiability problems, Technical Report, Naval Research Laboratory, Washington DC, 1993.

    Google Scholar 

  64. Yokoo, T.: Why adding more constraints makes a problem easier for Hill-climbing algorithms: Analyzing landscapes of CSPs, in Proceedings of CP'97, Springer-Verlag, 1997, pp. 357–370.

Download references

Authors
  1. Holger H. Hoos
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Thomas Stützle
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

About this article

Cite this article

Hoos, H.H., Stützle, T. Local Search Algorithms for SAT: An Empirical Evaluation. Journal of Automated Reasoning 24, 421–481 (2000). https://doi.org/10.1023/A:1006350622830

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1023/A:1006350622830

Search

Navigation