Benchmarking SAT Solvers for Bounded Model Checking

  • Emmanuel Zarpas
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3569)


Modern SAT solvers are highly dependent on heuristics. Therefore, benchmarking is of prime importance in evaluating the performances of different solvers. However, relevant benchmarking is not necessarily straightforward. We present our experiments using the IBM CNF Benchmark on several SAT solvers. Using the results, we attempt to define guidelines for a relevant benchmarking methodology, using SAT solvers for real life BMC applications.


Unit Clause Bound Model Check Contest Result SAT04 Competition Binary Clause 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Biere, A., et al.: Symbolic Model Checking Without BDDs. In: Cleaveland, W.R. (ed.) TACAS 1999. LNCS, vol. 1579, p. 193. Springer, Heidelberg (1999)CrossRefGoogle Scholar
  2. 2.
    Goldberg, E., Novikov, Y.: BerkMin: a Fast and Robust SAT-solver. In: Proc. of the Design, Automation and Test in Europe. IEEE Computer Society, Los Alamitos (2002)Google Scholar
  3. 3.
    Mahajan, Y., Fu, Z., Malik, S.: Zchaff2004: An Efficient SAT Solver. In: Hoos, H.H., Mitchell, D.G. (eds.) SAT 2004. LNCS, vol. 3542, pp. 360–375. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  4. 4.
    Le Berre, D., Simon, L.: The essentials of the SAT 2003 competition. In: Giunchiglia, E., Tacchella, A. (eds.) SAT 2003. LNCS, vol. 2919, pp. 452–467. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  5. 5.
    Le Berre, D., Simon, L.: SAT 2003 contest results (2003),
  6. 6.
    Le Berre, D., Simon, L.: 55 Solvers in Vancouver: The SAT 2004 competition. In: Hoos, H.H., Mitchell, D.G. (eds.) SAT 2004. LNCS, vol. 3542, pp. 321–344. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  7. 7.
    Moskewicz, M., et al.: Chaff: Engineering an Efficient SAT Solver. In: 38th Design Automation Conference. ACM/IEEE (2001)Google Scholar
  8. 8.
    Ryan, L.: The siege satisfiability solver,
  9. 9.
    Shacham, O., Zarpas, E.: Tuning the VSIDS Decision Heuristic for Bounded Model Checking. In: Proc. of the 4th International Workshop on Microprocessor, Test and Verification. IEEE Computer Society, Los Alamitos (2003)Google Scholar
  10. 10.
    Van Gelder, A., Tsuji, Y.K.: Satisfiability Testing with More Reasoning and Less Guessing. In: Cliques, Coloring, and Satisfiability: Second DIMACS Implementation Challenge. American Mathematical Society, Providence (1996)Google Scholar
  11. 11.
    Zarpas, E.: Simple yet efficient improvements of SAT based Bounded Model Checking. In: Hu, A.J., Martin, A.K. (eds.) FMCAD 2004. LNCS, vol. 3312, pp. 174–185. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  12. 12.
    CNF Benchmarks from IBM Formal Verification Benchmarks Library,
  13. 13.

Copyright information

© Springer-Verlag Berlin Heidelberg 2005

Authors and Affiliations

  • Emmanuel Zarpas
    • 1
  1. 1.IBM Haifa Research Laboratory 

Personalised recommendations