A parsimony tree for the SAT2002 competition
- 37 Downloads
Twenty of the programs (solvers) submitted to the SAT 2002 Contest had no disqualifying errors. These solvers were run on 2023 satisfiability problems of varying hardnesses. Each solver was judged by which problems it could solve within the allowed time limit. Twelve solvers were best on some problem — they could solve it and the others could not. Only two solvers could not beat each remaining solver on some problems (where the problems could vary depending on which solver it was trying to beat). Thus, there is evidence that 18 solvers were extremely good. It is interesting to analyze the contest results in a way that groups together solvers with similar strengths and weaknesses. This paper uses the parsimony algorithm to produce a classification of the twenty solvers. The paper also has a second classification, almost the same as the first, for the twenty solvers, updated versions of two solvers, and a fictitious state of the art solver. The contest problems came in three groups, Industrial, Hand Made, and Random. The Random group of problems was about three times as large as the other two together. The classification identifies four groups of solvers (plus a miscellaneous group): weak solvers, incomplete solvers which are very good at some satisfiable Random problems, complete solvers which are very good at most Random problems, and complete solvers which are very good at Industrial and Hand Made problems.
KeywordsBoolean satisfiability (SAT) empirical evaluation parsimony
Unable to display preview. Download preview PDF.
- S. Cocca and R. Monasson, Restart method and exponential acceleration of random 3-SAT instances resolution: A large deviation analysis of the David-Putnam-Loveland-Logemann algorithm (2002).Google Scholar
- E. Goldberg and Y. Novikov, BerkMin: A fast and robust SAT-solver, in: Design, Automation, and Test in Europe (DATE ‘02) (2002) pp. 142–149; http://eigold.tripod.com.Google Scholar
- I. Lynce and J. Marques Silva (2002).Google Scholar
- M.W. Moskewicz, C.F. Madigan, Y. Zhao, L. Zhang and S. Malik, Chaff: Engineering an efficient SAT solver, in: Proceedings of the 38th Design Automation Conference (DAC’01) (2001) pp. 530–535.Google Scholar
- L. Simon, D. Le Berre and E.A. Hirsch, The SAT2002 competition, this issue.Google Scholar
- N. Sordis and M. Nei, The neighbor-joining method: A new method for reconstructing phylogenetic trees, Mol. Biol. Evol. 4 (1987) 406–425.Google Scholar