Find out how to access previewonly content
Date:
31 Dec 2004
The SAT2002 competition
 Laurent Simon,
 Daniel Le Berre,
 Edward A. Hirsch
 … show all 3 hide
Rent the article at a discount
Rent now* Final gross prices may vary according to local VAT.
Get AccessAbstract
SAT Competition 2002 held in March–May 2002 in conjunction with SAT 2002 (the Fifth International Symposium on the Theory and Applications of Satisfiability Testing). About 30 solvers and 2300 benchmarks took part in the competition, which required more than 2 CPU years to complete the evaluation. In this report, we give the results of the competition, try to interpret them, and give suggestions for future competitions.
 P.A. Abdulla, P. Bjesse and N. Eén, Symbolic reachability analysis based on SATsolvers, in: Proceedings of the 6th International Conference on Tools and Algorithms for the Construction and Analysis of Systems (TACAS’2000) (2000).
 F. Bacchus, Enhancing Davis Putnam with extended binary clause reasoning, in: Proceedings of National Conference on Artificial Intelligence (AAAI2002) (2002).
 F. Bacchus, Exploring the computational tradeoff of more reasoning and less searching, in: [49, pp. 7–16] (2002).
 L. Baptista and J.P. MarquesSilva, Using randomization and learning to solve hard realworld instances of satisfiability, in: Proceedings of the 6th International Conference on Principles and Practice of Constraint Programming (CP) (2000).
 R.J.J. Bayardo and R.C. Schrag, Using CSP lookback techniques to solve realworld SAT instances, in: Proceedings of the Fourteenth National Conference on Artificial Intelligence (AAAI’97) (AMS, Providence, RI, 1997) pp. 203–208.
 A. Biere, A. Cimatti, E.M. Clarke, M. Fujita and Y. Zhu, Symbolic model checking using SAT procedures instead of BDDs, in: Proceedings of Design Automation Conference (DAC’99) (1999).
 M. Buro and H.K. Büning, Report on a SAT competition, Bulletin of the European Association for Theoretical Computer Science 49 (1993) 143–151.
 C.M. Li, B. Jurkowiak and P.W. Purdom Jr, Integrating symmetry breaking into a DLL procedure, in: [49, pp. 149–155] (2002).
 A.E. Caldwell, A.B. Kahng and I.L. Markov, Toward CADIP reuse: The MARCO GSRC bookshelf of fundamental CAD algorithms, IEEE Design and Test (May 2002) 7281.
 P. Chatalic and L. Simon, Multiresolution on compressed sets of clauses, in: Twelth International Conference on Tools with Artificial Intelligence (ICTAI’00) (2000) pp. 2–10.
 S.A. Cook, The complexity of theoremproving procedures, in: Proceedings of the Third IEEE Symposium on the Foundations of Computer Science (1971) pp. 151–158.
 F. Copty, L. Fix, E. Giunchiglia, G. Kamhi, A. Tacchella and M. Vardi, Benefits of bounded model checking at an industrial setting, in: Proc. of CAV (2001).
 E. Dantsin, A. Goerdt, E.A. Hirsch, R. Kannan, J. Kleinberg, C. Papadimitriou, P. Raghavan and U. Schöning, Deterministic (2 − 2 (k + 1 )^{ n } algorithm for kSAT based on local search, Theoretical Computer Science 189(1) (2002) 69–83. CrossRef
 M. Davis, G. Logemann and D. Loveland, A machine program for theorem proving, Communications of the ACM 5(7) (1962) 394–397. CrossRef
 M. Davis and H. Putnam, A computing procedure for quantification theory, Journal of the ACM 7(3) (1960) 201–215. CrossRef
 O. Dubois, P. André, Y. Boufkhad and J. Carlier, SAT versus UNSAT, in: [29, pp. 415–436] (1996).
 O. Dubois and G. Dequen, A backbonesearch heuristic for efficient solving of hard 3SAT formulae, in: Proceedings of the Seventeenth International Joint Conference on Artificial Intelligence (IJCAI’01), Seattle, WA (2001).
 M.D. Ernst, T.D. Millstein and D.S. Weld, Automatic SATcompilation of planning problems, in: [28, pp. 1169–1176] (1997).
 F. Aloul, A. Ramani, I. Markov and K. Sakallah, Solving difficult SAT instances in the presence of symmetry, in: Design Automation Conference (DAC), New Orleans, LO (2002) pp. 731–736.
 J.W. Freeman, Improvements to propositional satisfiability search algorithms, Ph.D. thesis, Departement of Computer and Information Science, University of Pennsylvania, Philadelphia, PA (1995).
 E. Goldberg and Y. Novikov, BerkMin: A fast and robust SATsolver, in: Design, Automation, and Test in Europe (DATE ‘02) (2002) pp. 142–149.
 C.P. Gomes, B. Selman and H. Kautz, Boosting combinatorial search through randomization, in: Proceedings of the Fifteenth National Conference on Artificial Intelligence (AAAI’98), Madison, WI (1998) pp. 431–437.
 E.A. Hirsch, SAT local search algorithms: Worstcase study, Journal of Automated Reasoning 24(1/2) (2000) 127–143. Also reprinted in Highlights of Satisfiability Research in the Year 2000, Frontiers in Artificial Intelligence and Applications, Vol. 63 (IOS Press, 2000). CrossRef
 E.A. Hirsch, New worstcase upper bounds for SAT, Journal of Automated Reasoning 24(4) (2000) 397–420. Also reprinted in Highlights of Satisfiability Research in the Year 2000, Frontiers in Artificial Intelligence and Applications, Vol. 63 (IOS Press, 2000). CrossRef
 E.A. Hirsch and A. Kojevnikov, UnitWalk: A new SAT solver that uses local search guided by unit clause elimination, Annals of Mathematics and Artificial Intelligence 43 (2005) 91–111. CrossRef
 J.N. Hooker, Needed: An empirical science of algorithms, Operations Research 42(2) (1994) 201–212. CrossRef
 J.N. Hooker, Testing heuristics: We have it all wrong, Journal of Heuristics (1996) 32–42.
 IJCAI97, Proceedings of the 15th International Joint Conference on Artificial Intelligence (IJCAI’97), Nagoya, Japan (1997).
 D. Johnson and M. Trick (eds.), Second DIMACS Implementation Challenge: Cliques, Coloring and Satisfiability, DIMACS Series in Discrete Mathematics and Theoretical Computer Science, Vol. 26 (American Mathematical Society, 1996).
 H. Kautz and B. Selman (eds.), Proceedings of the Workshop on Theory and Applications of Satisfiability Testing (SAT2001), LICS 2001 Workshop on Theory and Applications of Satisfiability Testing (SAT 2001) (Elsevier Science, 2001).
 H.A. Kautz and B. Selman, Planning as satisfiability, in: Proceedings of the 10th European Conference on Artificial Intelligence (ECAI’92) (1992) pp. 359–363.
 H.A. Kautz and B. Selman, Pushing the envelope: Planning, propositional logic, and stochastic search, in: Proceedings of the 12th National Conference on Artificial Intelligence (AAAI’96) (1996) pp. 1194–1201.
 E. Koutsoupias and C.H. Papadimitriou, On the greedy algorithm for satisfiability, Information Processing Letters 43(1) (1992) 53–55. CrossRef
 O. Kullmann, First report on an adaptive density based branching rule for DLLlike SAT solvers, using a database for mixed random conjunctive normal forms created using the Advanced Encryption Standard (AES), Technical Report CSR 192002, University of Wales Swansea, Computer Science Report Series (2002). (Extended version of [36].)
 O. Kullmann, Investigating the behaviour of a SAT solver on random formulas, Annals of Mathematics and Artificial Intelligence (2002).
 O. Kullmann, Towards an adaptive density based branching rule for SAT solvers, using a database for mixed random conjunctive normal forms built upon the Advanced Encryption Standard (AES), in: [49] (2002).
 C.M. Li, A constrained based approach to narrow search trees for satisfiability, Information Processing Letters 71 (1999) 75–80. CrossRef
 C.M. Li, Integrating equivalency reasoning into DavisPutnam procedure, in: Proceedings of the 17th National Conference in Artificial Intelligence (AAAI’00), Austin, TX (2000) pp. 291–296.
 C.M. Li and Anbulagan, Heuristics based on unit propagation for satisfiability problems, in: [28, pp. 366–371] (1997).
 I. Lynce and J.P. Marques Silva, Efficient data structures for backtrack search SAT solvers, in: [49] (2002).
 I. Lynce, L. Baptista and J.P. Marques Silva, Stochastic systematic search algorithms for satisfiability, in: [30] (2001).
 J.P. MarquesSilva and K.A. Sakallah, GRASP — A new search algorithm for satisfiability, in: Proceedings of IEEE/ACM International Conference on ComputerAided Design (1996) pp. 220–227.
 M.W. Moskewicz, C.F. Madigan, Y. Zhao, L. Zhang and S. Malik, Chaff: Engineering an efficient SAT solver, in: Proceedings of the 38th Design Automation Conference (DAC’01) (2001) pp. 530–535.
 F. Okushi and A. Van Gelder, Persistent and quasipersistent lemmas in propositional model elimination, in: (Electronic) Proc. 6th Int’l Symposium on Artificial Intelligence and Mathematics (2000).; Annals of Mathematics and Artificial Intelligence 40(3–4) (2004) 373–402. CrossRef
 R. Ostrowski, E. Grégoire, B. Mazure and L. Sais, Recovering and exploiting structural knowledge from CNF formulas, in: Proc. of the Eighth International Conference on Principles and Practice of Constraint Programming (CP’2002), Ithaca, NY (2002).
 R. Paturi, P. Pudlàk and F. Zane, Satisfiability coding lemma, in: Proceedings of the 38th Annual IEEE Symposium on Foundations of Computer Science, FOCS’97 (1997) pp. 566–574.
 S. Prestwich, A SAT approach to query optimization in mediator systems, in: [49, pp. 252259] (2002).
 S.D. Prestwich, Randomised backtracking for linear pseudoBoolean constraint problems, in: Proceedings of Fourth International Workshop on Integration of AI and OR techniques in Constraint Programming for Combinatorial Optimisation Problems (2002).
 SAT2002, Fifth International Symposium on the Theory and Applications of Satisfiability Testing, Cincinnati, OH (2002).
 R. Schuler, U. Schöning, O. Watanabe and T. Hofmeister, A probabilistic 3SAT algorithm further improved, in: Proceedings of 19th International Symposium on Theoretical Aspects of Computer Science, STACS 2002 (2002).
 B. Selman, H.A. Kautz and B. Cohen, Noise strategies for improving local search, in: Proceedings of the 12th National Conference on Artificial Intelligence (AAAI’94), Seattle (1994) pp. 337–343.
 B. Selman, H. Levesque and D. Mitchell, A new method for solving hard satisfiability problems, in: Proceedings of the 10th National Conference on Artificial Intelligence (AAAI’92) (1992) pp. 440–446.
 Y. Shang and B.W. Wah, A discrete Lagrangianbased globalsearch method for solving satisfiability problems, Journal of Global Optimization 12(1) (1998) 61–99. CrossRef
 L. Simon and P. Chatalic, SATEx: a Webbased framework for SAT experimentation, in: [30] (2001); http://www.lri.fr/~simon/satex.
 G. Sutcliff and C. Suttner, Evaluating general purpose automated theorem proving systems, Artificial Intelligence 131 (2001) 39–54. CrossRef
 G.S. Tseitin, On the complexity of derivation in the propositional calculus, in: Structures in Constructive Mathematics and Mathematical Logic, Part II, ed. A.O. Slisenko (Consultants Bureau, New York, 1970) 115–125. Translated from Russian. CrossRef
 A. Urquhart, Hard examples for resolution, Journal of the Association for Computing Machinery 34(1) (1987) 209–219. CrossRef
 A. Van Gelder, Autarky pruning in propositional model elimination reduces failure redundancy, Journal of Automated Reasoning 23(2) (1999) 137–193. CrossRef
 A. Van Gelder, Extracting (easily) checkable proofs from a satisfiability solver that employs both preorder and postorder resolution, in: Seventh Int’l Symposium on AI and Mathematics, Fort Lauderdale, FL (2002).
 A. Van Gelder, Generalizations of watched literals for backtracking search, in: Seventh Int’l Symposium on AI and Mathematics, Fort Lauderdale, FL (2002).
 A. Van Gelder and F. Okushi, Lemma and Cut strategies for propositional model elimination, Annals of Mathematics and Artificial Intelligence 26(1–4) (1999) 113–132. CrossRef
 A. Van Gelder and Y.K. Tsuji, Satisfiability testing with more reasoning and less guessing, in: [29, pp. 559–586] (1996).
 M. Velev and R. Bryant, Effective use of Boolean satisfiability procedures in the formal verification of superscalar and VLIW microprocessors, in: Proceedings of the 38th Design Automation Conference (DAC ‘01) (2001) pp. 226–231.
 J. Warners and H. van Maaren, Solving satisfiability problems using elliptic approximations: Effective branching rules, Discrete Applied Mathematics 107 (2000) 241–259. CrossRef
 H. Zhang, SATO: An efficient propositional prover, in: Proceedings of the International Conference on Automated Deduction (CADE’97), Lecture Notes in Artificial Intelligence, Vol. 1249 (1997) pp. 272275.
 H. Zhang and M.E. Stickel, An efficient algorithm for unit propagation, in: Proceedings of the Fourth International Symposium on Artificial Intelligence and Mathematics (AIMATH’96), Fort Lauderdale, FL (1996).
 L. Zhang, C.F. Madigan, M.W. Moskewicz and S. Malik, Efficient conflict driven learning in a Boolean satisfiability solver, in: International Conference on ComputerAided Design (ICCAD’01) (2001) pp. 279–285.
 L. Zheng and P.J. Stuckey, Improving SAT using 2SAT, in: Proceedings of the TwentyFifth Australasian Computer Science Conference (ACSC2002), ed. M.J. Oudshoorn, Melbourne, Australia (2002).
 Title
 The SAT2002 competition
 Journal

Annals of Mathematics and Artificial Intelligence
Volume 43, Issue 14 , pp 307342
 Cover Date
 20050101
 DOI
 10.1007/s1047200504246
 Print ISSN
 10122443
 Online ISSN
 15737470
 Publisher
 Springer Netherlands
 Additional Links
 Topics
 Keywords

 Boolean satisfiability (SAT)
 empirical evaluation
 Industry Sectors
 Authors

 Laurent Simon ^{(1)}
 Daniel Le Berre ^{(2)}
 Edward A. Hirsch ^{(3)}
 Author Affiliations

 1. LRI, U.M.R. CNRS 8623, Université ParisSud, 91405, Orsay, Cedex, France
 2. CRIL, F.R.E. CNRS 2499, Faculté Jean Perrin, Université d’ Artois, Rue Jean Souvraz SP 18, 62300, Lens, Cedex, France
 3. Steklov Institute of Mathematics at St. Petersburg, 27 Fontanka, 191023, St. Petersburg, Russia